What does a Successful Pilot Program with a School look like?

Hi all!

As I plan Trailblaze's first school pilots, I've been diving into research about what makes educational technology actually work in schools. I wanted to share some interesting insights I've found.

The 5% Problem

Laurence Holt wrote a great article earlier this year about the history of educational technology, highlighting the "5% problem": while many modern math platforms like Khan Academy and DreamBox show impressive results, these gains typically only appear for around 5% of students - those who use the programs exactly as recommended. For example, in a massive Khan Academy study across 99 US school districts, only 4.7% of students used the platform for the recommended 30 minutes per week.

In addition, some studies suggest this 5% often skews towards higher-performing students who would likely succeed with any form of practice. You can read the full article here.

This really made me think about how to design Trailblaze's pilot programs. It's not enough to have good software - we need to figure out how to make it work for everyone, especially those who might normally disengage.

Creating Software That Doesn't Rot

The best educational tools become part of a teacher's natural workflow because they solve a real, immediate problem. For Trailblaze, I hope the trigger moment will be simple: when a teacher thinks "this student isn't doing well," their next thought should be "let's use Trailblaze to figure out why."

The key to getting there is a proper implementation plan that doesn't leave any teacher or student behind.

Three Key Elements for Success

1. Clear Structure and Timeline

Rather than trying to transform an entire school at once, the pilot focuses on:

• Two Year 4-5 classes over 6 weeks
• Defined success metrics from day one
• Regular check-in points to assess progress
• A pilot fee to ensure mutual commitment

2. Comprehensive Support System

Success depends on strong relationships and clear communication:

• Initial teacher training session
• Three in-person check-ins (weeks 1, 3, and 6)
• Student feedback sessions at each check-in
• Same-day technical support
• Data reviews at days 1, 3, 10, 20, and final

3. Meaningful Measurement

Track concrete metrics that teachers agree are relevant to the pilot

• Student progress through diagnostic assessments before and after the pilot.
• Teacher workflow integration and time savings
• Student engagement levels, especially for those who typically struggle
• Technical performance and reliability

The Road Ahead

I'm excited to start working with schools! I know there will be plenty of challenges and learning moments ahead. To prepare, I've created a detailed Pilot Program Development Plan in Notion that outlines the timeline and all the documents needed at each stage. I'm happy to share this with anyone who's interested - you can see a screenshot of it below.

I'd love to hear from any of you who have experience with school pilots - what should I be watching out for?

Best,

Hugo

P.S. If you know any primary school teachers who might be interested in exploring a pilot program with Trailblaze, please do put them in touch. We're particularly looking to work with Years 3-6, but I'm always happy to have broader conversations about how we can support teachers and students.