What's new in Learning analytics?

Learning analytics itself is not new. For centuries, teachers have recorded student marks in their mark book, looked for patterns and responded to trends. What is new is the wealth of data available. From logins to the learning platform, library records, formative test results, live polling results in lectures, registration swipes and so on, a typical student carries a cloud of data points with them every week. But data is not information until it has been processed and the new promise is that the wealth of data available, combined with artificial intelligence and machine learning, might enable us to do genuinely useful things to the benefit of students and staff.

As part of the Future Teacher project, we wanted a quick snapshot of where people’s main interest and hopes for learning analytics might lie, so we created a quick survey? focusing on the four areas most likely to be of interest:

1.????? teaching and learning,

2.????? management and resources,

3.????? student support,

4.????? research.

These areas can be mapped to some of the major categories of learning analytics: descriptive, diagnostic, predictive and prescriptive.

Our findings

For each of the themes (1-4 above) we picked out three key areas of interest and asked the respondents to select their level of interest (definite interest, some interest or not interested). We also had a free text field where they could record any experiences they had in using learning analytics in that particular way.

The overall rankings

There are some caveats to the results below, including the relatively small sample size (57 respondents) as well as the self-selecting nature of the survey. A low level of interest in an option does not necessarily indicate a less important topic; it may be an extremely important topic within a very niche area. However, it is likely that those topics attracting most positive interest are good starting points for developing learning analytics applications alongside artificial intelligence (AI) applications.

This is the rank order

First place (47 "Definite interests")

The Teaching related topic "Pedagogical Insights: inform instructional design, personalised learning, and assessment".

Joint second place (37 "Definite interests")

• Student support topic - Retention Strategies: Use analytics to enhance student retention and success
• Teaching related topic - Early Intervention: identify struggling students to provide targeted support

Joint third place (36)

• Teaching related - Adaptive Learning: adapt content delivery based on student progress.
• Student support - Equity / Inclusion: Ensure analytics benefit all student populations - understanding cultural bias or misinterpretation

Fourth place (30)

Management - Institutional Goals: learning analytics supporting broader institutional objectives – e.g., widening participation, inclusion.

Fifth place (28)

Research - Methodological Advances: Explore new techniques for analysing educational data?

It is worth noting that where the results for “some interest” and “definite interest” were added together, the spread across the different themes (and individual questions) was much reduced. Both management and research were much better represented in those responses.

Current experiences

Future Teacher presentation

The April Future teacher session on Learning Analytics included three excellent case studies from the Open University, City University of London and the University of York. You can find the links to the presentations in the drop down speaker panels on the Future Teacher Learning Analytics web page. The video recording of the session will be uploaded to the same page in around a week. The three 12 minute sessions are well worth watching.

Survey responses

The following free text responses from the survey indicated current active uses (or areas of interest), sometimes at an individual interest level, other times as part of an institutional programme.

Learning analytics in teaching and learning

• I co-lead on LA at Newcastle University. We're currently piloting a system with numerous Schools within the university.
• I am a researcher at Surrey Institute of Education and researching into studying students' opinion using mixed methods methodology.
• I used data to inform personalised and cohort high school student success efforts in real time (not in HEI).
• I use analytics at the University of Oxford to understand how different types of activities in online courses are used, and inform suggestions for redesign based on UX and/ or pedagogical considerations. We don't have specialised analysis tools, so at the moment, I am using spreadsheets to combine the extracts from the VLE and different sources, and fitting this in as best as I can around my work as a learning designer because evaluation and continuous improvement is extremely important to me. My main challenge is finding 'proxy'measures between the logs in the VLE, and whether it can be equated or at least related to engagement for example.
• I recommend the use of learner analytics as part of design for online learning, however I have limited access to robust information in our current VLE (Brightspace), so I would welcome understanding correlations between learner engagement in online spaces and academic integrity and the use of GenAI.

Learning analytics in management

• I am like many colleagues trying to connect with others in my Department who are interested in Learning analytics and insights - particularly for reporting and tracking progress against our access strategy.

Learning analytics in student support

• I used it to ensure disadvantaged students progressed equitably.

Learning analytics in research

• I uses analytics in predicting the acceptance of technology in learning (University of Lancaster).

Conclusion

Learning analytics is still a field in its infancy. The main areas of interest from the Future Teacher community are related to the very human aspects of enhancing teaching and learning and supporting the student journey.

As more of the student experience is lived out online, the data generated by any individual grows exponentially. This has huge potential benefit for improving teaching, learning and support but there are also significant ethical issues around surveillance, interpretation, bias and so on that require both research and policy.

Currently, among ?our respondents, management and research were among the areas with least “definite interest” and with least experience. There is little doubt that learning analytics could contribute to better understanding of different types of resources (ebooks, journals, web sites, assistive technologies etc). Their use rates might indicate their value (or their discoverability), giving managers a better insight to returns on investment.

We strongly recommend that learning analytics is considered in relation to

• policies - how does it relate to policies about teaching and learning, well being services, student success, widening participation, privacy etc
• purpose - Who is it for? How does it help? How is this communicated?

Learning analytics has huge potential to make a positive difference, (see Open University research findings). But as a student's data cloud grows ever larger and artificial intelligence leaps ever more rapidly to untraceable conclusions, It is increasingly important that we retain our focus on humans and how the technology exists to serve the best interests of both students and staff .