Testing complex tech without writing a line of code

You don't need to build large language models, algorithms, or complex systems to test if they'll work. You just need to make users think they're using technology while humans secretly do the work behind the curtain.

What's a Behind-the-Scenes Test?

It's a test where your users think they're interacting with a working product, but humans are manually performing the functions behind the scenes.

Think of it as a magic trick; users see an interface, but the "magic" is actually humans frantically working to deliver the experience.

It's particularly useful if you need to validate complex ideas with expensive development pricetags. AI means we're running a LOT of these.

?

How a logistics company saved a year of development

A big logistics provider we worked with wanted to create an AI system that would optimise delivery routes in real-time based on traffic, weather, and package priority. The estimated development investment was 14 months and ￡3.2M.

Before writing a single line of code, we ran a Behind-the-Scenes test with them. We created:

• A simple driver app that appeared to provide AI-optimised routes
• A basic dispatcher interface that showed route changes
• A team of analysts who actually calculated the routes manually

We tested this with 8 drivers over 3 weeks.

The concept worked brilliantly; drivers followed the routes and deliveries were 22% more efficient. But the drivers kept ignoring certain types of recommendations, like cutting through residential areas or using service roads.

This human behaviour would have undermined our entire model.?

Instead of spending a year building a system drivers wouldn't fully use, we redesigned the experience to incorporate driver preferences and local knowledge. The revised system launched in 7 months, cost ￡1.4M less than planned, and saw 94% driver compliance.

?

How to run a Behind-the-Scenes Test?

To run this test, you'll need:

• A simple front-end interface
• A manual process to deliver the service
• People who can act as the "intelligence" behind the scenes

1. Create a believable fa?ade

Design the simplest possible interface that makes your service feel real. It doesn't need to be pretty—it just needs to be believable.?

2. Set up your behind-the-scenes operation

This is where the real work happens. Create processes for how your team will:

• Receive user inputs
• Process them manually
• Deliver outputs that appear automated
• Handle unexpected requests

3. Launch with real users

Tell users they're testing a "beta" or "prototype" version. This sets appropriate expectations while still making the experience feel real.

4. Watch what happens

The most valuable insights come from:

• How users interact with the "system"
• What they expect it to do versus what you planned
• Where they get confused or surprised
• How they respond to the outputs you provide

5. Gradually reduce manual intervention

If the test is successful, start automating pieces of the process to scale the operation.

?

Common mistakes to avoid:

• Not having enough people behind the scenes (things get slow, breaking the illusion)
• Creating interfaces that are too complex to maintain manually
• Setting unrealistic user expectations about response times
• Missing the opportunity to learn from unexpected user requests

Try this next week

Think of a complex feature your team wants to build. Create a simple interface that simulates how users would interact with it. Have team members manually provide responses through the interface. Test with 3-5 users and observe their reactions.

You'll be shocked by what you learn about your assumptions.