3 Questions to Ask Yourself to Make Sure Your School Data is Meaningful

When it comes to understanding the needle-movers for your students’ outcomes, I’ll be the first to tell you that meaningful data is a game-changer.

In my previous life as a teacher, when I started to share data that was genuinely meaningful with key stakeholders, not only did it increase my students’ engagement and motivation, it engaged parents and my teaching team too. It helped me to see the bigger picture of student performance, which meant I could plan lessons more effectively. But with so much data you could be collecting, how do you make sure that you focus on the “meaningful” stuff?

Data is only truly meaningful if it fits these three criteria:

• It’s ACCURATE 
• It’s RELIABLE
• It’s VALID

All three are equally as important,. To break it down a bit more, here’s how to test whether your data meets the meaningful criteria. 

1. Is the measurement accurate?

Identifying which groups of pupils you need to work with and give support to, relies on accurate calculations of student performance – not rough estimates. Here are the main questions I ask myself to be sure I’m taking accurate measurements:

• Does the tracking system I’m using calculate student outcomes exactly as the exam board will do in the summer exam series?
• Does my tracking system take into consideration the weighting of each component and therefore the associated scaling factors?
• Am I using the very latest grade boundaries with the latest suggested potential percentage increases? 
• Can your tracker add a percentage challenge to the boundaries from the latest exam series, replicating the trend from the previous year?

If you can answer yes to all these questions, then your data is probably as accurate as it can be. 

2. Is the measurement reliable?

Reliability considers the process through which data is produced. Above all, reliability is about consistency. Think about it – a reliable employee consistently shows up for work, and a reliable car consistently runs without breaking down. Reliable data might be produced by using an assessment system that is reliable: for example, a tried, tested, trusted method of assessing that provides consistent accuracy.

In translation to this context, reliable data can come from  using tried, tested, trusted methods of assessing that provides consistent accuracy. An example might be using 6 mark questions from published papers to assess a student’s progress in a particular topic. This is because they have published mark schemes and examiner guidance and comments.

This is also reliable because it can be replicated, you can use this methodology to test other topics (e.g Using 2, 3, 4 or 5 mark questions from published exams). This also brings in the word consistency. There will always be published exams from which the questions can be used, so therefore a method that can be consistently used from term to term, year to year.

What is more, this is reliable because this methodology can now be used consistently across the department. The problem comes when one teacher uses assessments written by themselves, another uses some they found on social media the other uses published exams. This means that the data is now unreliable as there is no consistent standardised methodology used by all to collate and calculate progress data.

As long as you’re using consistent measurements, you shouldn’t have a problem with reliability.

3. Is the measurement valid?

A great question to help you get your head around validity is: “Am I actually measuring what I’m trying to measure?”

For example, when you calculate working at grades for your pupils, if you replicate the method of calculation used by the exam boards then your data will be valid. Because you’re using the same methodology, you’re actually measuring what you’re trying to measure.

When it comes to assessment methods, the same test applies. There’s validity in asking pupils to sit a mock exam, because it replicates the final assessment method AND the process that the pupil will need to go through. That means they’re prepared for the final exam experience, and you’ve got useful data about how they’ve performed in relation to the exam board’s mark scheme. 

When the process you’re using to assess students is as close as possible to the final process pupils will go through in their exams, you can be confident that your data is valid.

You can read a more detailed version of this article over on our website: What do we actually mean by 'Meaningful Data' in schools?

And if you want to find out more about Pupil Progress, send me a message or sign up to our 6-week free trial (with no obligations).