Weekly AI Research Roundup (23 - 30 September)

This week’s roundup focuses on five research papers that highlight multimodal AI, which involves models that can process text, images, and videos together.?

Let’s explore each paper and its key contributions.

?? Invest in the Future of AI with GenAI Works!

The global AI market is booming, projected to grow at a 37.3% CAGR and reach an astounding $1.81 trillion by 2030.

We’re not just a community—we’re a full ecosystem of over 7 million followers, gaining 300,000 new members monthly, with more than 3,000 apps, courses, and events. We’re dedicated to democratizing AI by offering hands-on experiences, expert insights, and innovative tools to help individuals and businesses thrive in the AI era.

Our platform provides AI education, career opportunities, hackathons, and support for startups, all while connecting technologists and domain experts to advance AI across industries.

As we continue our rapid expansion, we’re offering an incredible chance for investors to join us. Be a part of this AI revolution and invest in GenAI Works today! Invest by October 20, 2024, and earn up to 25% in free shares! - Invest NOW!

1. Emu3: Next-Token Prediction is All You Need

The Emu3 model presents a new way to handle multiple types of data—like text, images, and videos—using one simple method. Emu3 turns everything into a series of tokens (small chunks of data) and uses a Transformer model to predict the next token, regardless of whether it’s for an image, a piece of text, or a video.?

This approach eliminates the need for different models for different tasks and has shown better performance than some of the leading models used today.

Highlights:

• Emu3 simplifies multimodal AI by using one model for all types of data.
• It outperforms other models in tasks like video generation and image understanding.

Read more: https://emu.baai.ac.cn/

2. Lotus: A Diffusion Model for Better Visual Predictions

Lotus introduces a diffusion-based AI model that excels at predicting detailed visual data, like estimating the depth in an image or understanding the surface normals (the direction a surface is facing).?

While diffusion models are usually used for generating images, Lotus adapts this method to make highly accurate predictions for visual tasks that require detailed understanding of the environment.

Highlights:

• Lotus uses diffusion models to improve visual predictions, setting new accuracy benchmarks.
• It specializes in tasks like depth estimation and surface prediction in images.

Read more: https://arxiv.org/pdf/2409.18124

3. MIO: A Foundation Model for Multimodal Tokens

The MIO model is designed to handle different types of data by turning them into a common set of tokens. This model, like Emu3, uses a simple token-based approach to work across text, images, and other modalities. MIO performs exceptionally well on tasks such as image captioning, object recognition, and answering questions about images.

Highlights:

• MIO unifies text and image data into tokens, making it easier for the model to process both types.
• It achieves top results on several multimodal benchmarks.

Read more: https://arxiv.org/pdf/2409.17692

4. Molmo and PixMo: Open-Source Models for Multimodal AI

Molmo and its dataset, PixMo, take a different approach by focusing on using open-source data. Molmo is a vision-language model, meaning it can understand and generate both text and images. Instead of relying on proprietary data, it uses a detailed open-source dataset. Despite being open-source, it outperforms several high-profile, closed models, proving that open-data AI can be just as effective as proprietary systems.

Highlights:

• Molmo uses an open-source dataset, PixMo, to achieve high performance in vision-language tasks.
• It demonstrates that open-data models can compete with and even outperform proprietary AI systems.

Read paper: https://arxiv.org/pdf/2409.17146

5. PROX: Improving Pre-Training Data Quality

PROX introduces a way to automatically clean and improve large amounts of pre-training data, a task that has traditionally required a lot of manual effort from experts. Using small AI models, PROX refines data by removing noise and fixing errors, which helps improve the quality of the data used to train larger AI models. This results in better performance on a wide range of tasks, all while reducing the need for expensive computation.

Highlights:

• PROX automatically improves pre-training data quality, leading to better model performance.
• It reduces the computational cost of training large AI models, making the process more efficient.

Read paper: https://arxiv.org/pdf/2409.17115

Conclusion

This week’s research shows how the AI community is working to make models simpler, more powerful, and more accessible.?

As these technologies improve, we’ll see AI becoming more integrated into everyday applications, allowing more people and industries to benefit from its capabilities.

Lotus shows how traditional models for image generation can be adapted to handle more complex visual tasks.?

Molmo proves that open-source models can achieve results on par with proprietary ones, and PROX automates data cleaning, making the training process more efficient.

The Goods: 4.5M+ in Followers; 2.5M+ Readers

?? Contact us if you made a great AI tool to be featured

?? Subscribe to our AI Investments Newsletter for exclusive insights on cutting-edge developments, market trends, and emerging opportunities in the AI space.

??For more AI News follow our Generative AI Daily Newsletter .

??For daily AI Content follow our official Instagram , TikTok and YouTube .

??Follow us on Medium for the latest updates in AI.

Missed prior reads … don’t fret, with GenAI nothing is old hat. Grab a beverage and slip into the archives .

Past performance is not indicative of future returns. Investing involves risk. Please read the offering circular at https://invest.genai.works/ for additional information on the company and risk factors related to the offering

In making an investment decision, investors must rely on their own examination of the issuer and the terms of the offering, including the merits and risks involved. Genai Works, Inc. has filed a Form C with the Securities and Exchange Commission in connection with its offering, a copy of which may be obtained here: bit.ly/3APlUkJ