? Manus AI: Why Everyone Should Worry

Emerging AI Governance Challenges | Paid Subscriber Edition | #177

?? Hi, Luiza Jarovsky here. Welcome to our 177th edition, read by 55,000+ subscribers in 165+ countries. Not a subscriber yet? Join us.

?? We are a leading AI governance publication helping to shape the future of AI policy, compliance, and regulation. It's great to have you here!

? Manus AI: Why Everyone Should Worry

Three days ago, on March 6, a Chinese AI startup launched Manus AI, which many are calling the world's first general AI agent.

In this edition of the newsletter, I examine Manus AI from legal and ethical perspectives and argue that a) there are inconsistencies, b) it's a red flag for the future, and c) most people do not fully grasp the practical consequences of AI applications like this.

1?? What is Manus AI?

"Manus" is the Latin word for "hand." According to the company behind it, Manus AI is "a general AI agent that turns your thoughts into actions."

Before you keep reading, the best way to understand what it does is to watch the company's 4-minute introduction video:

There is already a lot of buzz around this AI agent, and many are calling it China's second DeepSeek moment (to understand DeepSeek's impact and legal issues, read my recent articles on the topic: DeepSeek's Legal Pitfalls and The DeepSeek Effect).

While the comparison with DeepSeek might make sense from marketing and geopolitical standpoints, it is important to remember that these are two different AI applications with different strategies and functionalities:

• DeepSeek-R1 is an open-source, general-purpose AI model designed to rival OpenAI's o1;
• Manus AI is a general AI agent currently in closed beta testing. It requires an invitation to access and is not entirely open-source;
• Manus AI hasn’t triggered a significant stock drop like the one Nvidia saw after DeepSeek.

Regarding its innovative capabilities and potential breakthroughs, Professor Wei Wei of Huazhong University of Science and Technology stated:

“while using AI agents to complete multi-tasks is not something Manus pioneered, its innovation lies in transforming multi-agent collaboration into a product. By integrating state-of-the-art AI capabilities into user experience, Manus simplifies complex tasks and pushes AI agent technology forward in a significant way.”

Some of Manus AI’s key features are highlighted in this Hugging Face community article, including:

• The types of autonomous tasks it can execute: Report writing, spreadsheet and table creation, data analysis, content generation, travel itinerary planning, file processing;
• Multi-model capabilities: text, image, code;
• Integration with external tools: web browsers, code editors, database management systems;
• Its reported state-of-the-art performance in the GAIA benchmark, outperforming AI models such as OpenAI’s GPT-4 and Microsoft’s AI systems.

Like many others in the tech industry, Clement Delangue, co-founder of Hugging Face, also posted about Manus AI yesterday:

Manus AI took over the internet in only 3 days, and I bet you’ll hear even more about it tomorrow as other news outlets start covering it.

However, beyond the hype, there are legal and ethical concerns that raise a red flag about the future of agentic AI applications.

2?? Origins

Manus AI’s privacy policy states that the company behind it is Butterfly Effect PTE. LTD, based in Singapore, and that the privacy policy is governed by Singaporean law.

Regarding the company’s location, they went as far as to add “Country: where Manus AI or the owners/founders of Manus AI are based, in this case is Singapore.” (One of this privacy policy's strange sentences that makes me suspect it was written by AI.)

From a privacy policy standpoint, the lawyers behind Manus AI have chosen to establish a legal entity in Singapore.

However, despite what the privacy policy says, recent news articles suggest that both the team and the company are actually Chinese:

• The South China Morning Post reported that “According to people close to the team, Manus was developed by Butterfly Effect, a company with a few dozen employees in Beijing and Wuhan, capital of central Hubei province.”
• Forbes wrote that it's entirely Chinese-built.
• Business Today reported that it's Chinese and developed by the startup Monica.
• Newsweek described it as “developed by a low-profile team and backed by Chinese investors and developers.”
• According to the Hugging Face article I mentioned above, “Manus AI was developed by Monica, a Chinese AI startup focused on building next-generation autonomous AI agents.”

It seems that Butterfly Effect PTE. LTD is the legal entity behind Monica. However, I couldn't find where Monica is headquartered (its location isn't mentioned in the privacy policy).

Why does Manus AI’s location matter?

If you've read my recent analyses on DeepSeek, you know that because it's based in China, it complies with Chinese laws, including censorship regulations, and there are data transfers to China.

As a result, DeepSeek has come under increased scrutiny over data protection and security. It is currently being investigated by nearly every EU data protection authority, and officials in the US, Taiwan, South Korea, and other countries have partially blocked it due to national security concerns.

It looks like Manus AI presents itself as a Chinese company (with its team based in China) while maintaining a legal entity in Singapore. From a data protection perspective, the key questions are: Where are its servers located? Is there any corporate affiliation to China? Are there data transfers to China?

I won’t go into every detail of its privacy policy (so we can focus on the more interesting aspects of AI agents), but I will point out that the document appears to be AI-generated. It includes sections titled “What is GDPR,” “What is personal data,” and “Why is GDPR important?” whose content feels out of place and more suited to an internal report on EU and US data protection law than a privacy policy, where the company should be outlining its practices.

If you are a lawyer, I invite you to check it yourself (let me know your assessment in the comments!).

3?? The Problem with Autonomous Agents Like Manus AI

According to the company's introduction video (above), Manus AI is a “truly autonomous agent that bridges the gap between conception and execution.”

Before I continue, I want to highlight a paper I recently shared in this newsletter: “Fully Autonomous AI Agents Should Not be Developed” by Margaret Mitchell, Avijit Ghosh, Alexandra Sasha Luccioni, and Giada Pistilli.

The authors—all from Hugging Face, whose CEO posted about Manus AI yesterday (see above)—suggested three critical directions:

“1. Adoption of agent levels: Widespread adoption of clear distinctions between levels of agent autonomy. This would help developers and users better understand system capabilities and associated risks.

2. Human control mechanisms: Developing robust frameworks, both technical and policy level (Cihon, 2024) that maintain meaningful human oversight while preserving beneficial semi-autonomous functionality. This includes creating reliable override systems and establishing clear boundaries for agent operation.

3. Safety verification: Creating new methods to verify that AI agents remain within intended operating parameters and cannot override human-specified constraints.”

Unfortunately, Manus AI’s positioning and goals seem to be exactly what the paper's authors advise against: full agentic autonomy.

I've written about the legal aspects of AI agents a few times in this newsletter. You can read some of my recent deep dives, such as AI Agents: RIP Autonomy and Legal Challenges of AI Agents.

Given that Manus AI is advertised as a “truly autonomous agent” and “the first general AI agent,” I'll comment on two of the use cases it showcased on its website, highlighting some of the ethical and legal concerns behind agents like this one.

1. Resume Screening for Hiring

In this case, Manus AI processes various resume files and creates a spreadsheet ranking candidates based on reinforcement learning expertise and additional information focusing on key highlights and achievements. The output produced by Manus AI is here.

There are important legal issues behind this use case: ...

?? This is a free preview. To continue reading, choose a subscription here: https://www.luizasnewsletter.com/p/manus-ai-why-everyone-should-worry

Thank you!

Best regards,

Luiza