LLMs Have Hit the Wall

OpenAI’s latest endeavours are stirring up the AI community. While some experts argue that LLMs have reached their peak, OpenAI chief Sam Altman disagrees vehemently. “There is no wall,” Altman said , challenging the notion that scaling LLMs is hitting a dead end.

Scaling the Right Things Matter?

Former OpenAI co-founder Ilya Sutskever, now leading Safe Superintelligence (SSI), echoed Altman’s optimism. “Scaling the right thing matters more now than ever,” said Sutskever , who is also working on similar lines as OpenAI but with a different approach. He’s most likely focusing on reinforcement learning and self-supervised methods to build safe superintelligence.?

OpenAI is implementing a strategy where each new model, like GPT-5 or GPT-6, is trained on high-quality synthetic data generated by its predecessor, Strawberry (o1 ). This creates a “recursive improvement cycle”, enhancing model performance incrementally with each iteration.

Andrej Karpathy , former OpenAI co-founder and founder of Eureka Labs, highlighted a critical gap: LLMs lack thought process data. Karpathy introduced the concept of “jagged intelligence ”, stressing the need for models to develop cognitive self-knowledge through sophisticated post-training methods rather than merely mimicking human labels.

‘I Told You So’

Meta’s chief AI scientist, Yann LeCun , didn’t hold back his scepticism about OpenAI’s approach. “I don’t want to say ‘I told you so’, but I told you so!” LeCun remarked, highlighting Meta’s parallel efforts in developing autonomous machine intelligence (AMI).

Under LeCun’s leadership, Meta has merged FAIR and the GenAI team to create AMI, a system designed to emulate human and animal reasoning. This year, Meta released V-JEPA (Video Joint Embedding Predictive Architecture), enhancing machine understanding through video interactions and several advanced models like SAM 2.1 and Meta Lingua , which optimise LLM performance without specialised hardware.

Meta’s Layer Skip selectively executes layers in LLMs, accelerating generation times. Also, the upcoming Llama 4 leverages self-supervised learning to enable broad data representation and flexibility across domains. Meta’s new ‘self-taught evaluator ’ employs chain-of-thought techniques to improve model accuracy in complex fields like science and mathematics.

LeCun remains steadfast in his belief that new architectures and paradigms are essential for achieving human-level AI. “Reaching human-level AI will require new architectures and new paradigms,” he replied to Gary Marcus , underscoring Meta’s commitment to advancing beyond current LLM limitations.

Enjoy the full story here .?

Salesforce and Zoho Killer?

A Bengaluru startup is building an AI-powered low-code platform that offers an affordable alternative to Salesforce and Zoho. It aims to make enterprise-level applications accessible to businesses of all sizes, combining affordability with advanced customisation and data control.

“We offer a Salesforce flexibility at a Zoho price,” said the CEO & co-founder.

Find out more about the startup here .?

AI Bytes >>?

• Anthropic has launched a chain-of-thought-based prompt improver and evaluator on its Console, leveraging advanced reasoning techniques to enhance prompt quality by 30% and enable developers to generate more accurate and refined AI outputs.
• Indonesia’s Indosat and GoTo, in collaboration with NVIDIA, have launched Sahabat-AI , an open-source suite of Bahasa Indonesia-focused LLMs designed to promote AI sovereignty, empower local industries and government, and foster community-driven generative AI innovation.
• Fran?ois Chollet , Google’s leading AI researcher and creator of the influential Keras framework, has left the company to pursue new opportunities while continuing to advise them.?
• Microsoft has unveiled specialised, industry-tailored AI models integrated into its cloud platform, leveraging sector-specific data and strategic partnerships to enhance operational efficiency and drive innovation across diverse industries.?
• Yotta Data Services recently acquired IndiQus Technologies to strengthen India’s indigenous cloud and AI infrastructure, reducing dependence on foreign providers and accelerating the nation’s digital transformation.?