The enterprise learning ecosystem in the age of advanced streaming data architectures and real-time learning analytics

When designing and implementing a scalable learning ecosystem, the most important design considerations are those of data logistics, system architecture, and data design. When these considerations are not at the forefront of the design process, the ecosystem will most likely experience unpatchable issues which hinder its potential to provide meaningful learning experiences. As in any system, the foundation on which a learning ecosystem is built determines the capabilities and the extensibility of the learning ecosystem.

The demands of scale put a significant amount of pressure on the variety of technologies that comprise the modern learning stack. Take for example a system that implements a recommendation engine to provide the learner with suggestions on how to optimize their learning experience via the presentation of relevant and impactful content. In order for the recommendation engine to be responsive and operate at near real time speeds, it must have timely access to massive amounts of data. Each consumed data point must contain sufficient but not excessive contextual information and the data must be stored in a scalable database capable of meeting the needs of the recommendation engine as well as handling the massive amount of data produced by a modern learning ecosystem. If those conditions are not met, a bottleneck ensues. This bottleneck reduces the speed at which the recommendation engine can function. It does not matter if the bottleneck is due to input data retrieval latency, prolonged computational time due to processing excessively large data points, or the input data source failing to provide any data at all. The result is the same: degradation in system performance culminating in a substandard user experience for the learner by increasing wait time for the recommendation engine to issue the suggestions which facilitate an optimal and personalized learning experience.

This is a scale problem.

Interested in solving that scale problem in your learning organization? My team at Yet Analytics just published a new white paper on the subject. Check it out and download a free copy. It covers both the challenges you may face and the types of technologies you'll need to implement. It even features an illustrated example of a streaming data architecture serving multiple data models in a modern learning ecosystem.