Four Ways That LMS Data Can Drive Student Success

Student success has several definitions.? For some, it is the quality of the experience that students have while enrolled.? For others, it is measured by the efficiency in which students progress through an educational program or system to a specific outcome, usually a degree or certificate.? While educators are invested in student success for moral and ethical reasons, it is also critically important to the overall health of the institution.? As the pool of high school graduates shrinks and employment continues to be high, limiting the incentive for adults to enter or return to higher education, every student (and her net revenue) becomes even more important to overall institutional health.

The Role of LMS Data in Student Success

We have for some time now been in a revolution around data both inside and outside higher education.? As ever-increasing zettabytes of data are generated by the Internet of Things (smartphones, tablets, computers, smart devices), we are challenged to identify those data that really matter.? How and what should we track, and how can we join data from disparate sources to get a meaningful picture of student success?? Institutions in every state and province, and of every size, type, and location are asking these critical questions.

Learning Management Systems (LMS) have grown in their usage in higher education, at the same time.? What was once used mainly as an online program delivery mechanism has now become nearly ubiquitous for courses in every modality.? Like many systems, the initial student data is created by an automated feed from the Student Information System (SIS), populating the course rosters of the LMS, and allowing students and faculty to interact there.? At that point, the data continues to grow in the LMS, creating a rich source of information about student success.

Two important concepts drive the way that we can think about the role of LMS data in student success.? First, we need to distinguish between lagging and leading indicators.? Lagging indicators are those we use for analysis of past behavior.? Leading indicators in student success are those data that show us what is happening right now and to whom. ? Both are important. Lagging indicator analyses, such as retention studies of those who left/returned help us improve systems, supports, experiences, communications, etc., for our students.? They are a shotgun approach, and hopefully hit the target for students who have similar experiences in the future.? Leading indicators allow us to identify students who need those systems, supports, etc., at that moment, and with laser-like accuracy link the student to the remedy.

Second, we know that success factors don’t act in a vacuum.? Decreased logins in the LMS may be due to an unpaid account balance, where the student needed to take on extra work to pay the outstanding balance and be eligible for re-enrollment.? A lack of social interaction may lead students to withdraw from studies, as well, and may be an indicator of other issues impacting academic performance. When we look only at one system’s data, we may see nothing that indicates attrition.? When we join the data together and look holistically at student success, we have a greater ability to head off potential problems before they worsen, and address them specifically, lessening the stress or cascading effects they may have on student wellbeing and success.

Figure 1 shows the typical/historical approach to data collection and usage in higher education.? The SIS serves as the generator of initial information for other systems but that data becomes trapped and unusable for holistic assessment of student success.?

Figure 1. Historical or traditional arrangement of systems and data

Figure 2 represents the current approach to data and systems.? This approach allows the same routines to populate initial data in systems from the SIS but captures and liberates the data produced by those systems.? They are collected in a data lake, which makes them available for CRM applications, data visualizations and analyses.

Figure 2. Modern approach to data integration across systems

While LMS data alone can help us spot times when students may not be logging into their courses, or the grade book may show patterns that warn of course failure, the greater power occurs when trapped system data is liberated to join with financial, social, and experiential learning data.? Tools such as predictive AI will be more powerful and nuanced in identifying those students whose multiple factors may show signs of distress.

Four Areas of LMS Data to Integrate

There are many areas of LMS data that could be harvested and integrated with other student data from the SIS and point solutions.? Starting with the main areas where data may have the greatest impact can form the basis of a data integration strategy.

Course and System Login Activity

Most courses have resource materials and at least some assignments in the LMS.? For online courses, this may be all assignments.? When students don’t access these on a regular cadence, this should be cause for alarm.? Is there something going on that has pulled the student away from the course, such as a family illness or job event?? Does the student need some extra flexibility or support to get back on track with assignments?

When students do not login, there may be alerts that can be set within the LMS itself; many systems have such an alert that can be turned on.? In that case, the alert itself should be sent to the CRM. The course name and the alert itself are the data to be exported to the CRM.? When an LMS alert isn’t available, some extra work will be necessary to determine the right data and routine/job that would evaluate the login data and create an alert (automated population of a data field in the CRM and corresponding automated communications).??

There is not a one size fits all cadence for student logins.? For example, in the graduate courses that I have taught, one week is far too long between logins.? It would be hard for a student to get back on track after missing the readings and assignments for that week.? Three days with no logins would trigger the alert in this situation.? For an undergraduate course, one week may be a fine time period for the alert, especially if some of the course’s material assignments are external to the LMS.? The key is to identify a process whereby faculty can set the appropriate cadence for the alert to occur.? If setting the alert is simple, some faculty or departments may be able to take care of these without assistance (run an exception report to identify courses that lack the alerts, and assist these faculty with setting them).? Otherwise, the alerts need to be set by instructional design, departmental or other staff.

Grade Book

Because SIS grade data only sometimes tracks midterm grades and usually only final grades, it is an ineffective system to identify leading indicators of student performance.? Keeping assignment grades within the LMS course is a common routine today.? This digital grade book makes it easy for faculty to perform their work from anywhere, at any time.? Rubrics for grading are best practices and have been implemented in many courses, as they support consistent grading practices.? Creating and using them in the LMS is fairly straightforward.? Usually, faculty need to enter only the final grade from their grade book into the SIS via routines managed by the registrar’s office.? Some institutions automate this, eliminating the manual faculty step.

It is not necessary to export all assignment grades to the CRM.? Only those assignments that receive a failing mark, as determined by the institution’s grading scheme, would be needed to create an alert.? Some institutions may want to include low marks that are not considered failing, especially larger institutions or those with high advisor:student ratios, as tracking individual performance may be harder.? When that occurs, an alert should be sent to the CRM, similar to the process noted earlier for logins.? Like login data, course and the alert are the needed data; the grade itself may or may not be sent, and this is a discussion for academic and registrar personnel to consider in light of institutional academic, security, and FERPA policies and practices.

Other Alerts

Not all alerts are automated or based upon metrics, such as grades, marks or logins.? Faculty, student life professionals, and others may use the LMS to trigger an alert based upon observed behaviors.? For example, a student may not fail an assignment but the instructor may observe that she needs more support for college-level writing.? A student life professional teaching a freshman experience course may have concerns about a student’s wellbeing, and use the LMS (or another database or communication tool) to trigger an alert about it.

When these alerts occur, it is important to send them to the CRM and use appropriate system safeguards to protect student privacy.? This streamlines the data flow for alerts, overall, and continues to create a “single pane of glass” where institutional professionals (advisors, faculty, staff) can view and assess the success of students.? Another strength of strong CRM platforms like Salesforce is the ability to easily generate cases from any incoming data (chat conversations, email, calls, LMS, etc.).? Having one system and one place to handle these expedites the resolution of issues and increases the likelihood of success.

Attendance

Student course attendance has become a larger issue following the pandemic.? Cited by faculty as one of the emerging issues they have seen, a decline in attendance or lack of understanding of the importance of class attendance is concerning[1].? Students experienced a much higher degree of flexibility during the pandemic, where deadlines, requirements, even grades, appeared to be fungible.? A student who misses one course session is likely not a big concern, although hybrid or graduate courses that have fewer course meetings may rise to the level of concern, even if only one session is missed.? For many courses, scanning the data to determine a pattern or cumulative level of absences will be needed.

Not every course and not every institution takes attendance in courses.? For those that do, however, this is very similar to the login data.? Attendance may also be one of the “other” alerts that faculty generate when they note a student missing for some time, even if they do not take attendance for each course meeting.? Many instructors, especially those who teach smaller or seminar courses, may realize a student’s absence over time, and note the impact these absences have on the potential to successfully complete the course.

Conclusions

The attention given to AI today as a major societal, economic, and higher education issue is hard to miss.? This monumental technological innovation coincides with strong headwinds in higher education enrollment and financial health.? The intersection of these provides a tremendous opportunity and urgency to harness institutional data that support student success, increase enrollment, and drive institutional financial health.? This is only possible when data are liberated, not trapped inside multiple systems and point solutions.

The data contained in a learning management system is rich in leading indicators.? Unless liberated, it is trapped there, preventing institutions from intervening before academic success falters and the student fails or decides to leave.? These data become more powerful in the overall context of student success – academic, financial, and social.? Integrating LMS data with these other system data allows institutions to more effectively harness new tools, including AI, and to create a single pane of glass for advisors, faculty and staff.? In a time of constrained resources, staff can be more effective with students, spend less time researching and triangulating data across screens and reports, and use that time to support student persistence and success.

[1] https://www.insidehighered.com/news/student-success/academic-life/2023/06/23/four-tips-improving-attendance-college-courses