Weak Benchmarks Are Making AI Selection Harder Than It Should Be

The world of artificial intelligence (AI) is advancing at breakneck speed, with businesses and individuals diving into its possibilities. But amid the excitement lies a critical issue —AI models lack proper benchmarks.

If you're considering adopting AI or purchasing related services, it’s crucial to understand what this means for you.

Benchmarks in AI are like report cards—they provide standardized tests or metrics to evaluate a model’s performance. Without them, it's impossible to objectively assess how effective or reliable a model truly is.

Today, countless models claim to be the best, but without a consistent way to measure them, how can you separate hype from substance?

This lack of evaluation standards has created a Wild West atmosphere in the AI industry. Companies often make bold claims about their products without providing solid proof. For potential buyers, this is a minefield. Choosing an AI solution without clear benchmarks could leave you with an ineffective tool—or one that’s completely unsuited to your needs.

Beyond functionality, the absence of benchmarks raises ethical concerns. Flawed AI models can perpetuate biases, produce inaccurate results, or be manipulated. Without standards to identify and address these issues, it’s harder to hold companies accountable, eroding trust and exposing businesses to significant risks.

Here’s a New Article in the MIT Technology Review:

When new AI models are released, they are often showcased as outperforming rivals on a range of benchmarks. OpenAI’s GPT-4o, for instance, debuted in May with claims of leading performance across multiple tests.

However, according to recent research, these benchmarks are poorly designed, difficult to replicate, and often rely on arbitrary metrics. This is troubling because benchmark results influence the scrutiny and regulation AI models receive.

Benchmarks, essentially tests for AI, vary widely in format. Popular ones, like the Massive Multitask Language Understanding (MMLU) benchmark, use multiple-choice questions. Others evaluate task performance or the quality of AI responses to predefined prompts.

The write-up says AI companies frequently highlight benchmark results to tout their models’ success. However, this optimization can create misleading impressions.

Governments, too, are incorporating benchmarks into AI regulations. The EU AI Act, effective August 2025, uses benchmarks to assess whether models pose systemic risks, subjecting them to stricter oversight. Similarly, the UK AI Safety Institute’s Inspect framework relies on benchmarks to evaluate model safety.

But current benchmarks may not be up to the task. “Poorly designed benchmarks give a false sense of safety, especially in high-stakes applications,” warn some experts quoted in the report.

The message is clear: building better benchmarks is essential for meaningful AI regulation and safety. Without them, we risk relying on flawed tools to govern an increasingly powerful technology.

What can you do as a consumer? Look for companies that prioritize transparency. Ask about the benchmarks they use and their testing processes. A lack of clear answers is a red flag.

In short, understanding the role of benchmarks in AI is essential. Being informed can help you avoid costly mistakes and ensure you choose AI solutions that truly meet your needs. Knowledge is your best defense in this rapidly evolving field.

Disclaimer: Just a heads-up. Remember, "Living With AI" articles are written for the curious everyday folks, not the AI expert. While we try our best to keep things accurate, sometimes, we might (over) simplify things a bit, or leave out some super technical stuff. Think of it like explaining rocket science with a baking soda volcano - fun and fizzy, but not quite the real deal! Don't worry, if you're hungry for more technical details, there's a whole universe of resources out there waiting to be explored.

Join AI For Real, a community around artificial intelligence.