Where Is Meta’s Generative Play?

While Microsoft and Google scramble to supercharge their businesses with text generation, Meta has yet to launch a flagship generative AI service. Reporters went looking for reasons why.

What’s new:?Staff turnover, misaligned priorities, insufficient processing power, and caution in the wake of earlier controversies have hindered Meta’s ability to take advantage of generative AI,?The Wall Street Journal?reported .

Challenges:?Reporters spoke to more than a dozen current and former Meta employees to determine why, despite extensive investments in large language models (LLMs) and vision models like DINOv2 and SAM, the company lacks a high-profile generative initiative. They pointed to several factors:

• Over the past year, Meta lost many researchers who worked on LLMs. Six of the 14 authors of the?LLaMA ?paper and eight of the 19 authors of the?OPT ?paper either were laid off or departed for other jobs.
• Researchers who worked on LLMs struggled to get processing and engineering resources because chief AI scientist Yann LeCun was unenthusiastic about the technology, according to insiders who spoke to the reporters anonymously. The company prioritized recruiting scientists over engineers and valued research over building products, further impeding progress on products based on LLMs.
• Meta’s effort to equip its data centers to run such models suffered from strategic?shifts ?and a shortage of high-end AI chips.The resources that were available often supported individual researchers’ pet projects rather than fulfilling a cohesive strategy.
• The public failures of Meta LLMs such as?Galactica ?and?BlenderBot 3 , which Meta withdrew amid controversy over their generation of false statements, left the company more cautious — especially after?years ?of ?outrage ?over negative social impacts of Facebook and Instagram.

Reorganization:?Meta has taken steps to break the logjam. Earlier this month, it?announced ?a number of generative AI products including chatbots for Messenger and WhatsApp, a photo editor for Instagram, and a productivity assistant for internal use. In February, Meta CEO Mark Zuckerburg?announced ?a new generative AI group that reports directly to chief product officer Chris Cox. The group will focus on training models to integrate with products such as Facebook, Instagram, and WhatsApp.

Why it matters:?The rapid rise of generative AI threatens to upend the tech world’s established order. Meta — like Google in response Microsoft’s aggressive launch of Bing Chat — has found itself in a defensive position.

References:

1. https://www.wsj.com/articles/mark-zuckerberg-was-early-in-ai-now-meta-is-trying-to-catch-up-94a86284
2. https://arxiv.org/abs/2302.13971
3. https://arxiv.org/abs/2205.01068
4. https://www.datacenterdynamics.com/en/news/exclusive-after-meta-cancels-odense-data-center-expansion-other-projects-are-being-rescoped/
5. https://www.deeplearning.ai/the-batch/meta-released-and-quickly-withdrew-a-demo-of-its-galactica-language-model/
6. https://ai.facebook.com/blog/blenderbot-3-a-175b-parameter-publicly-available-chatbot-that-improves-its-skills-and-safety-over-time/
7. https://www.deeplearning.ai/the-batch/how-facebook-fills-the-feed/
8. https://www.deeplearning.ai/the-batch/facebook-likes-extreme-content/
9. https://techcrunch.com/2023/06/09/meta-previews-generative-ai-tools-coming-to-whatsapp-messenger-and-instagram-plus-internal-ai-tools/
10. https://www.reuters.com/technology/meta-is-creating-new-top-level-product-group-focused-generative-ai-ceo-2023-02-27/
11. https://www.deeplearning.ai/the-batch/issue-203/