“Catch 10^25”: Decoding the next chapter of the great open-source debate.

Under EU law, developers of “open-source” artificial intelligence will either struggle to compete on performance, or face the same regulatory brakes and obligations applied to high-performance closed models. The resulting impasse could create a geopolitical fault line.

First, a housekeeping note: Minerva announced a strategic partnership with Sinolytics this week, to help companies understand and respond to geopolitical disruption of complex technology supply chains; more on that here.

What’s happening? The EU’s landmark AI Act entered into force last month. As regulators in Brussels implement the Act in stages over the next two to three years, they will have to grapple with one of the big unresolved questions of AI governance: should cutting-edge models be open and freely accessible, or remain proprietary?

Advocates of open-source models — for which information about model weights, system prompts and other things, like training data, or underlying source code are widely accessible — say they will lead to fairer, more democratized innovation and help to make the market more competitive. Skeptics, including some who are primarily concerned about national security issues like cybersecurity and bioterrorism, say open models could be easier to abuse if key details about their inner workings are published for anyone to pick up and use. Here’s the catch, under the EU AI Act, open-source models that are large enough to compete with leading proprietary systems, like OpenAI’s GPT-4o, and Anthropic’s Claude 3.5 Sonnet on performance benchmarks, are also too big to qualify for legal exemptions granted to open-source models as part of the legislation.?

Why? The act imposes strict obligations on the developers of models that have had 1025 floating point computing operations-worth of training. Such models will be defined by the EU as presenting “systemic risks” if they exceed this figure; and will therefore be subject to the act’s long compliance checklists for things like model evaluations, adversarial testing, incident reporting, and cybersecurity protections.?

For context, in the US, models have to be based on ten times more compute demand to trigger lighter-touch transparency and reporting requirements set out by the Biden administration’s AI Executive Order, while industry commitments to red-teaming and other safety evaluations for these models, for the time being, are voluntary.???

So what? The EU approach, to many, seems paradoxical and confused; even though there are provisions for developers to argue that their models don’t represent such systemic risks.?

In the US, OpenAI and Anthropic, have just agreed to share detailed information about their large, closed models with the US Artificial Intelligence Safety Institute. This points to a future compliance regime that will be based on testing and scientific standards; including a plan for “monitoring” rather than “mandating restrictions” on open-source models released to the community.?

Inference: As the US begins monitoring powerful open-source models that it deems to pose potential “dual use” risks in cooperation with industry, the EU is struggling to bring open-source artificial intelligence into the remit of its sweeping legislation.

Why does it matter? Efforts to regulate open-source models could form a geopolitical fault line. Developers around the world are racing to improve model performance. The hard yards of that development are happening in lesser discussed areas like increasing the context window size with which models operate, and optimizing function calling, so that models can interact reliably with other softwares that haven’t necessarily been primed for integration, similar to the way a human assistant can navigate a website they haven’t visited before.

These incremental and important innovations need to happen in both the small-i innovation domains of start-ups and sector-specialists; as well as in the labs of larger companies.

The US has signaled that it does not currently see a need to go beyond monitoring open-source risks as this development takes place, and is focused on policing misuse of models under existing laws.?Meanwhile, the EU’s stricter approach could quash the incentive for smaller firms or other sector leaders in its jurisdiction, specifically those building applications on top of open-source models, to compete with proprietary models from bigger and extremely well-resourced companies in the US and elsewhere.