China's AI Competition Deepens, Mindreading AI Turns Brainwaves into Images, Meta Unveils 3D GenAI ... and more

Welcome to AI Weekly Breakthroughs, a roundup of the news, technologies, and companies changing the way we work and live.

China's AI Competition Deepens

At this year's WAIC, China’s largest annual AI conference, major LLM developers SenseTime and Alibaba showcased significant advancements in their AI technologies, underscoring the intensifying competition in China’s AI market. SenseTime unveiled updated versions of its SenseNova models, including the new SenseNova 5.5, which claims a 30% performance improvement and superiority over GPT-4 in several metrics. Alibaba highlighted increased adoption of its Tongyi Qianwen models, with downloads doubling to over 20 million and corporate user growth rising significantly. The competitive landscape is further highlighted by the prediction from AI start-up MiniMax’s CEO that only a few companies will dominate the global LLM market in the future, with the potential for some of those companies to be Chinese.?

At MIT Sloan Symposium, CIOs Discuss AI Future

At the MIT Sloan CIO Symposium, discussions on generative AI mirrored the early concerns about cloud computing, emphasizing governance, security, and responsible use. Unlike the past, when CIOs often said no to new technologies, today’s leaders recognize that employees will find ways to use AI regardless. Hence, they focus on responsible implementation, employee education, and enhancing customer experience. Mathematica CIO Akira Bell advocates for data readiness and best practices, while GE Vernova’s Angelica Tritzo experiments with pilot projects to gauge AI’s potential.

Anthropic Funds Third-Party AI Evaluation Projects

Anthropic announced a new initiative to advance third-party evaluations of AI models. This program aims to fund external organizations to develop robust evaluations, focusing on AI safety levels, advanced capabilities, and necessary infrastructure. Priority areas include cybersecurity, CBRN risks, and model autonomy. The goal of Anthropic's new initiative is to elevate AI safety by providing comprehensive tools benefiting the entire ecosystem. Interested parties can submit proposals to receive tailored funding and expert guidance for their projects.

Will Apple Announce a Gemini Deal this Fall?

Apple is reportedly set to announce a partnership with Google to integrate its Gemini AI model into Apple devices, complementing the existing ChatGPT integration, according to Bloomberg’s Mark Gurman. Additionally, Apple might explore deals with other AI companies like Anthropic, while rejecting Meta’s Llama chatbot due to performance issues. Apple’s broader AI strategy includes the forthcoming Apple Intelligence, initially in beta for the iPhone 15 Pro and Pro Max, which may eventually offer subscription-based features. ?

Apple Joins OpenAI Board as Observer

Apple will join OpenAI’s board as an observer, with Phil Schiller, head of the App Store, taking the role. Effective later this year, Schiller can attend meetings without voting but gains insight into OpenAI’s decisions. This follows Apple’s June announcement of integrating OpenAI’s ChatGPT into its devices and the introduction of Apple Intelligence across its apps. OpenAI also added new directors, including CEO Sam Altman.

Stability AI Debuts Free Community License

Stability AI has updated its licensing, launching a new Community License that allows free use of its models for research, non-commercial, and commercial use under $1 million revenue. This change supports open-source principles, community engagement, and transparency. The license covers recent models, including the improved SD3 Medium, which now incorporates community feedback to address quality issues. Additionally, the weights for Stable Diffusion 3 Medium are now available on Hugging Face.

Hedra and ElevenLabs Join Forces for AI-Driven Storytelling

Hedra has partnered with ElevenLabs to enhance its Character-1 model, which turns still images into talking characters. Hedra attracted tens of thousands of users within 2 days of launching. The partnership allows Hedra to integrate ElevenLabs' realistic and emotional AI voices, available in 29 languages. This collaboration aims to democratize video creation, enabling users to easily craft stories and characters.

Meta Enhances VR Gaming with AI

Meta is integrating generative AI into VR, AR, and mixed reality games, aiming to revitalize its metaverse initiatives. A recent job listing indicates Meta’s focus on creating gameplay that evolves with each session and follows non-linear paths. This strategy includes developing tools to streamline game development, potentially expanding beyond Meta’s Horizon platform to other devices. Despite substantial sales of Quest headsets, Horizon has struggled to attract users and recover from financial losses. CEO Mark Zuckerberg’s increased focus on gaming, coupled with partnerships and licensing of Quest software features, signifies a strategic pivot, though profitability remains distant.

Grok 2 Debuts in August, Grok 3 by Year-End

Elon Musk’s xAI will release Grok 2, an advanced AI assistant inspired by AI characters like JARVIS from Iron Man, in August. Building on the success of Grok 1.5, Grok 2 aims to excel in real-time knowledge processing and reasoning. Musk emphasizes high data quality in training, addressing concerns about competitors’ datasets. Grok 2 will include real-time web search and image generation, enhancing user interaction and accuracy. Additionally, Musk has teased Grok 3, slated for release by year-end, which will use 100,000 Nvidia H100 GPUs. Supported by a collaboration with Dell Technologies to create a specialized AI factory, Grok 3 promises to push AI performance boundaries further.

Apple Debuts 4M AI Model with EPFL on Hugging Face

Apple, in collaboration with the Swiss Federal Institute of Technology Lausanne (EPFL), has launched a public demo of their 4M AI model on Hugging Face Spaces. The 4M model, capable of generating and processing content across multiple modalities, allows users to create images from text, perform object detection, and manipulate 3D scenes. This release marks a shift in Apple's approach, promoting openness and developer engagement. The demo aligns with Apple’s broader AI strategy and recent market gains, reinforcing its position as a significant player in the AI industry while maintaining a strong emphasis on user privacy.

Meta’s GenAI Transforms Text into 3D Instantly

Meta has unveiled its new 3D Gen AI model, which transforms text into high-fidelity 3D images in under a minute. This state-of-the-art pipeline also allows users to apply new textures and skins using text prompts. The process involves two models: 3D AssetGen for initial model creation, averaging 30 seconds, and 3D TextureGen for texture refinement or replacement, taking about 20 seconds. The 3D Gen model excels in representing 3D objects across view, volumetric, and UV spaces, outperforming industry baselines in text fidelity and visual quality.?

Perplexity Pro Search Now Solves Complex Queries

Perplexity has upgraded Pro Search to tackle complex research with advanced problem-solving capabilities. This new version excels in multi-step reasoning, breaking down intricate queries into manageable steps for comprehensive answers. Now featuring advanced math and programming functions via the Wolfram|Alpha engine, users can opt for Quick Search for fast, source-backed answers or Pro Search for in-depth analysis. Pro Search is available free five times every four hours, with nearly unlimited access for Perplexity Pro subscribers.

Altrove Uses AI to Accelerate Material Innovation

French startup Altrove is accelerating new material development using AI models and lab automation, with a focus on rare earth elements. By using AI to predict potential new materials and generate production recipes, Altrove addresses the traditionally slow pace of material discovery. Co-founded by Thibaud Martin and materials science expert Joonatan Laulainen, Altrove uses proprietary technology for material verification and refinement. The company plans to automate its lab for high-throughput testing, aiming to streamline the iteration process and boost innovation in material science. Altrove recently raised €3.7 million to support these efforts, inspired by AI-driven advances in biotech.

Kyutai Launches Moshi AI for Natural Conversations

Kyutai, a French AI lab with a $300 million investment, has launched Moshi, an advanced conversational AI. Unlike traditional text-based models, Moshi's real-time, multimodal capabilities enable it to understand and respond to spoken words and emotions, making interactions more natural. Embracing an open-source philosophy, Kyutai is releasing Moshi’s code, model, and research for public collaboration. Though still in development and limited to five-minute conversations, Moshi marks a significant advance in human-AI interaction, promising richer, more intuitive communication.

Mindreading AI Turns Brainwaves into Images

Researchers at Radboud University in the Netherlands have made a groundbreaking advancement in neuroscience and AI by generating highly accurate images from brain activity. Using an enhanced mind-reading AI system, they reconstructed images with unprecedented precision from direct brain signal recordings. This milestone offers potential applications in treating vision loss and improving communication for individuals with disabilities. The technology's ability to interpret brain signals and create detailed visual representations heralds promising developments in perception research and generative modeling.

Rethinking Tokens in AI's Future

Kyle Wiggers from TechCrunch highlights the impact of tokenization on modern generative AI models like OpenAI’s GPT-4. These models break down text into smaller units for processing, but this approach introduces challenges, such as varying token representation across languages and inconsistent tokenization of numeric sequences. Wiggers suggests that future advancements may come from new architectures like MambaByte, which avoid tokenization to handle text more directly, potentially overcoming current AI limitations.

The Hard Truth About AI Infrastructure Startups

John Hwang, writing for the Enterprise AI Trends Substack, highlights the challenges AI infrastructure startups face in scaling successfully, exemplified by Amazon's acquisition of Adept AI. Categorized as a "tarpit idea," these startups often struggle to differentiate themselves amid intense competition from established industry giants. Hwang suggests that many AI infrastructure companies will either be acquired or fail to sustain themselves unless they can offer unique, superior solutions as the AI landscape continues to rapidly evolve.

Foundation Capital on the Next Wave of AI Innovations

Foundation Capital partners discuss the transformative impact of LLMs on AI since their mainstream introduction with ChatGPT in 2022. Highlighting over 150 new LLMs released in 2023, they observe narrowing performance gaps and dropping costs. Despite wide-ranging potential, LLM deployment remains nascent, facing challenges like data preprocessing and scalability. The partners suggest multimodal models, multi-agent systems, and new architectures as key innovations driving future enterprise transformation. Within the enterprise, LLMs have the potential to redefine most white-collar jobs and critical business functions.

Princeton Exposes AI Agent Benchmark Shortcomings

Researchers at Princeton University have identified flaws in current AI agent benchmarks that limit real-world applicability. They found an overemphasis on accuracy, leading to complex and costly agents, and propose balancing accuracy with cost. The study also highlights the conflation of benchmarking needs between model and downstream developers and the inadequate holdout sets causing overfitting. Princeton's team recommends a framework to prevent overfitting and standardize evaluation practices, aiming for more practical and reproducible AI agents.

New Strategies for Effective Retrieval-Augmented Generation

Researchers at Fudan University in Shanghai have identified the strengths and challenges of RAG techniques in this new publication. While effective in integrating up-to-date information and improving response quality, current RAG methods suffer from complexity and slow response times. The team proposed optimal strategies to balance performance and efficiency through extensive experiments. They also showed that multimodal retrieval techniques can enhance question-answering and accelerate multimodal content generation.

Quantum Rise grabs $15M seed for its AI-driven ‘Consulting 2.0’ startup

Peter Thiel’s Founders Fund Leads $85M Seed Investment Into Open-Source AI Platform Sentient

Tembo capitalizes on the database boom and lands new cash to expand

AI4 – Las Vegas – August 12 – 14

The AI Conference 2024 – San Francisco – September 10 – 11

World Summit AI – Amsterdam – October 9 – 10

Gitex Global – Dubai – October 14 – 18

Big Data Conference Europe – Vilnius – November 19 – 22