Sora-ing to New Heights in AI

ML Engineering Highlights:

• OpenAI unveils new video AI model : OpenAI has unveiled Sora, its new AI text-to-video generation model, which offers 60-second video generations and incredibly realistic and high-quality results. While it's not yet available to the masses, a limited number of creators are getting access as the company continues to test its security defenses. With the potential to simulate everything, Sora's arrival marks a milestone in the tech and media industry and raises questions about the balance of risks and rewards in AI applications.

• V-JEPA: The next step toward Yann LeCun’s vision of advanced machine intelligence (AMI) : This paper by Meta introduces V-JEPA, a collection of vision models trained solely on video feature prediction without external sources of supervision. The models, trained on 2 million videos, show versatile visual representations that perform well on both motion and appearance-based tasks without adaption of the model's parameters. For example, the largest model obtains high accuracy on Kinetics-400, Something-Something-v2, and ImageNet1K using a frozen backbone.
• Feeding the hunger for Nvidia GPU access is big business : Lambda , a GPU cloud company powered by 英伟达 GPUs, has raised $320 million at a $1.5 billion valuation to expand its AI cloud business, joining other companies in offering access to Nvidia GPUs. Nvidia's stock has tripled, and the company has surpassed 亚马逊 and 谷歌 in market capitalization, while AI startups are desperate for access to Nvidia's high-performance computing H100 GPU.

Research Highlights:

• Transformers Can Achieve Length Generalization But Not Robustly : This paper by Google DeepMind examines the challenge of length generalization for language models, even with large-scale Transformers, using the task of addition of two integers. The success of length generalization is tied to the data format and position encoding, and with the right combination, standard Transformers can extrapolate to a sequence length 2.5 times the input length. However, length generalization is still fragile and influenced by factors like random weight initialization and training data order, leading to large variances across different random seeds.

• Computing Power and the Governance of Artificial Intelligence : This paper by OpenAI , Mila - Institut québécois d'intelligence artificielle , and several other organizations discusses how governments and companies are focusing on the importance of compute in governing AI development and deployment, investing in domestic compute capacity and controlling the flow of compute to competing countries. AI-relevant compute is seen as a key intervention point due to its detectable, excludable, and quantifiable nature, and its importance for cutting-edge AI models.
• BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of data : This paper by 亚马逊 introduces a new text-to-speech (TTS) model called BASE TTS, which is the largest TTS model to-date and achieves state-of-the-art speech naturalness. Trained on 100K hours of public domain speech data, it deploys a 1-billion-parameter autoregressive Transformer and a convolution-based decoder to convert texts into waveforms in a streamable manner.

Lightning AI Studio Highlights:

• Run Ollama LLMs on a cloud GPU Lightning Studio : This Studio automatically starts a server running an optimized version of Mistral AI ’s Mixtral in the background (courtesy of Ollama ). Ollama allows us to get up and running with large language models quickly. With it we can run Mixtral, Llama 2, and Code Llama, and many other models.

• Benchmark cloud data-loading libraries : This Studio benchmarks several cloud data-loading frameworks such as?PyTorch Lightning Data ,?WebDataset, and?Mosaic ML.

• Structured LLM Output and Function Calling with Guidance : This Studio shows how to structure LLM outputs in formats like JSON and enable interactions with external tools or function calls.

Don’t Miss the Submission Deadline

• ECCV 2024: European Conference on Computer Vision 2024 Submission Deadline: Fri Mar 08 2024 06:59:00 GMT-0500
• MICCAI 2024: International Conference on Medical Image Computing and Assisted Intervention Submission Deadline: Fri Mar 08 2024 02:59:59 GMT-0500
• ECAI 2024 : European Conference on Artificial Intelligence 2024 Submission Deadline: Fri Apr 26 2024 07:59:59 GMT-0400

Want to learn more from Lightning AI? “Subscribe” to make sure you don’t miss the latest flashes of inspiration, news, tutorials, educational courses, and other AI-driven resources from around the industry. Thanks for reading!