Armilla Review #85

TOP STORY

The EU AI Act: Setting the Stage for Harmonized AI Standards

The European Union’s AI Act, which came into force in August 2024, marks a significant step towards standardized AI regulations within the EU. The Act, focusing primarily on high-risk AI systems, mandates a suite of compliance requirements around risk management, data governance, transparency, cybersecurity, and human oversight. To facilitate adherence, the European Commission is working with standardization bodies like CEN and CENELEC to develop harmonized technical standards. These standards, once published in the EU's Official Journal, will grant AI developers a presumption of compliance when designing systems in alignment with them. Key challenges include ensuring standards cover risks to individual rights and safety without becoming overly restrictive. With an implementation deadline of August 2026, the standards development process is now in high gear, though it faces complexities in reaching a consensus. This initiative not only seeks to build trust in AI but also aims to level the playing field across AI providers, particularly benefiting smaller enterprises.

Source: European Commission

FEATURED

Most AI Coverage Not Fit for Purpose

A huge thank you to The Insurer TV for hosting this conversation with our?CEO, Karthik Ramakrishnan, at this year's FERMA Forum! Karthik shared some perspectives on: ?? AI Risks & Coverage Gaps ?? Liability Risks on the Rise ?? Stress Testing for Robustness

?? Watch the full interview or check out our blog to learn why the lack of clarity around AI coverage is a problem.

THE HEADLINES

The Case for Targeted AI Regulation: A Balancing Act

As AI technology progresses rapidly, so do the associated risks, prompting a need for timely, effective regulations. In response, industry experts suggest a balanced, targeted approach to regulation, emphasizing that sweeping, reactionary policies could hinder innovation. A thoughtful framework can protect against AI risks in areas like cybersecurity, biotechnology, and large-scale applications, without stifling growth in the AI sector. The call to action includes promoting transparency, incentivizing rigorous safety practices, and simplifying regulatory structures. With governments increasingly pressured to act, this approach seeks to secure public trust in AI while enabling its economic and scientific potential.

Source: Anthropic

COMPL-AI Framework: Benchmarking for EU AI Act Compliance

In a groundbreaking move towards AI regulation, the COMPL-AI framework offers the first technical interpretation of the EU AI Act, aimed specifically at large language models (LLMs). Designed as a benchmarking suite, COMPL-AI translates the Act’s broad regulatory mandates into measurable technical requirements. In an initial evaluation, the framework revealed current LLMs often lack in areas such as robustness, fairness, and safety, underscoring the need for more balanced model development. While this framework serves as an essential starting point, it highlights gaps in existing standards and benchmarks, suggesting an urgent need for enhanced evaluation tools.

Source: arXiv

Deepfake Scams on the Rise with Cheap AI Voice Cloning

Researchers have demonstrated how OpenAI's new Realtime API could be exploited to execute deepfake phone scams at scale for under a dollar per attempt. Using just over a thousand lines of code, a team at the University of Illinois Urbana-Champaign created an AI agent that could impersonate officials and scam individuals into revealing sensitive information, such as bank account details. The team’s tests showed success rates as high as 60% in specific scams, underscoring the dangers of dual-use AI technology. While OpenAI assures layers of security in its API, this research raises pressing questions about the responsibility of AI companies in preventing misuse, especially as AI-driven scams continue to evolve.

Source: Information Security Media Group

Racist Research Surfaces in AI Search Results

AI-powered search tools from Google, Microsoft, and Perplexity have been found surfacing discredited research promoting racial superiority theories. In one investigation, searches related to national IQ scores displayed figures taken directly from a widely debunked dataset by the late Richard Lynn, a controversial figure in race science. Such flawed data has long been criticized for poor sampling and cultural bias, yet remains accessible through AI-generated overviews and search results. Experts warn that featuring such content could inadvertently lend credibility to harmful ideologies, potentially radicalizing users. Google and Microsoft have responded by reassessing their search protocols to mitigate these issues.

Source: WIRED

Alexa’s Fact-Checking Failures Erode Trust

Amazon's Alexa has reportedly provided users with inaccurate information on topics ranging from MPs' expenses to scientific facts, often attributing the erroneous responses to Full Fact, a prominent UK fact-checking organization. Alexa's mistaken answers appeared to stem from misinterpreting the "claim" section of Full Fact's articles, leading to misinformation being presented as verified facts. While Amazon has addressed the issue, questions remain about the virtual assistant’s vetting process. The incident has highlighted potential pitfalls of relying on AI-driven assistants for fact-based answers, emphasizing the need for rigorous source-checking to maintain user trust.

Source: FULL FACT

Google’s Covert Campaign to Discredit Microsoft

Microsoft recently criticized Google for orchestrating covert lobbying efforts aimed at discrediting Microsoft’s cloud business in Europe. According to Microsoft, Google has organized and funded an astroturf group, featuring small European cloud providers as a front, to sway regulatory opinion against Microsoft’s practices. This group’s alleged goal is to divert regulatory scrutiny from Google’s own dominance while targeting competitors like Microsoft. Microsoft claims that Google’s tactics, including offering cash incentives and recruiting proxies, aim to skew competition policy in its favor. Google has previously faced global antitrust investigations and is reportedly intensifying efforts to undermine rivals amid ongoing regulatory challenges.

Source: Microsoft

AI Smile Alignment Increases Attraction in Dating Experiments

A recent study has shown that aligning smiles between dating partners can significantly boost romantic attraction. Using an experimental video-conference platform, researchers were able to covertly synchronize participants' facial expressions, particularly smiles, during speed-dating interactions. This alignment led to increased feelings of attraction, as well as a heightened synchronization in participants' vocal and facial expressions. By establishing a way to causally influence social interactions, the study sheds light on the powerful role of expressive alignment in romantic settings. This research not only advances understanding of attraction but also provides a novel approach for investigating social behaviors in real-time.

Source: PNAS