2024: AI Year In Review (AI's Best Year, Yet...)

I’ll be honest: this is my second attempt at writing the ultimate 2024 AI Year in Review. The first time, I did what any ambitious AI entrepreneur or enthusiast might do: I fed over 70,000 AI news articles—nearly 30 million tokens—into a generative model, fully expecting an omnipotent, all-knowing summary that would blow everyone away. I figured “more data = better insights.”

What I got instead was an empty shell—timelines, over-hyped milestones, and boilerplate remarks. It missed the essential context that truly defined 2024: the compounding effect. Because this year wasn’t just about each breakthrough on its own; it was about how each domain’s progress fueled the next. That’s the piece we can’t overlook if we want to understand where 2025 is headed.

After working hands-on with creating hundreds of AI projects with my AI agency, consulting with our local government policy and legislature, and leading the Portland chapter of one of the largest AI organizations in the country - I've seen firsthand how AI has evolved from a tool to a force multiplier - and how that force doesn't just add it, it compounds.

Here’s the real story of 2024—why it mattered, how everything connected, and where we go from here.

From Scale to Smarts: The Optimization Revolution.

Early in 2024, the world braced for GPT-5. Instead, OpenAI doubled down on efficiency with the “o-series,” shifting the paradigm from “bigger is always better” to “thinking is better.” These new models:

• o1-mini: Outperformed GPT-4-level benchmarks in math and science, hitting 89th percentile on competitive programming tasks at a fraction of the cost.
• o1: Matched PhD-level accuracy in STEM fields, leveraging “think-before-speaking” reinforcement learning to handle complex queries more intelligently.
• o3: Smashed records with 96.7% on the AIME 2024, introducing a 3-tier adaptive reasoning mode—light, medium, and deep—tailored to the complexity of each question.

Technical Highlight

Recursive Reward Modeling: The o-series uses a novel technique where the model effectively “pauses” to reason about the next tokens. This approach drastically reduces inference overhead by only applying heavier compute to the hardest steps.

The Scaling Wall Problem & Time-Inference Compute

Traditionally, AI improvements followed the “scaling law,” where bigger models and more data led to better performance. But this approach hit diminishing returns—think of blowing more air into a balloon that’s already stretched to its limit:

• Data Quality Constraints – High-quality datasets at massive scale became scarce.
• Computational Limits – Energy usage and hardware requirements skyrocketed.
• Diminishing Returns – Extra data yielded ever-smaller gains.

Test-Time Compute (TTC) is emerging as the breakthrough approach to sidestep these barriers. Instead of front-loading all intelligence into training, models allocate extra “thinking time” during inference. This parallels how humans pause to consider tough problems, rather than memorizing every possible scenario upfront.

• OpenAI’s o1/o3: Adopts “recursive reasoning,” spending more time on complex questions in real-world usage (inference).
• DeepSeek3: Implements “dynamic depth scaling,” allocating deeper computational resources only when tasks demand it.
• Google Gemini: Uses "flash attention mechanisms" to reprocess data efficiently at inference time.

Breakthrough:

Time-inference compute shifts the paradigm from “learn everything during training” to “learn how to learn during inference.” This unlocks significant performance gains without endlessly inflating model size.

Why It Matters for Business

• Action Point: Opt for models that are cost-effective and tuned to your domain. Large, generic models might be overkill—and overpriced. Time-inference approaches let you scale on-demand, focusing compute where it matters most.
• Compounding Impact: More efficient, faster AI means more organizations can deploy it. The resulting feedback loops (data + usage) accelerate improvements in everything from agent-based systems to robotics—while inspiring new open-source initiatives.

Agents & Autonomy: Moving Beyond One-Step QA.

2024 ushered in a new era of agentic AI, where models aren’t just responding; they’re chaining tasks, deciding which external tools to call, and acting autonomously across multiple steps.

• Anthropic’s Claude (with Model Context Protocol): Standardized how AI pulls in real-time data from external sources (APIs, databases, private knowledge bases) while maintaining safety and alignment.
• OpenAI’s ChatGPT + Tools: Evolved from a chat interface into a full-blown orchestrator—capable of planning a product launch or scheduling a trip, end-to-end.

Technical Highlight

Toolformer Integration: Agents use “toolformer” architectures to call specialized APIs (e.g., calculators, Python scripts, web search). This approach reduces “hallucinations” by delegating tasks to proven tools.

Why It Matters for Business

• Action Point: Identify repetitive workflows that can be fully or partially automated (think data entry, compliance checks, scheduling).
• Compounding Impact: Agents feed massive usage data back into the ML pipeline, improving AI coding assistants (which, in turn, make agent creation easier) and enabling deeper collaboration with hardware (e.g., robots directing their own tasks).

Coding Assistants: Innovation That Accelerates Innovation.

Coding assistants in 2024 graduated from mere autocomplete tools to full-fledged AI pair programmers and low-code/no-code enablers.

• Replit & V0.dev: Replit’s AI suite helped them secure a $1B+ valuation, democratizing coding to millions of users. V0.dev automated front-end components across React, Svelte, and Vue, slashing development time for UI-heavy apps.
• Cursor (v0.43): Achieved advanced cross-file awareness, drastically simplifying large-scale refactors in codebases with tens of thousands of lines.
• Codeium’s Windsurfer: Debuted the first “agent-powered IDE” with “Cascade,” allowing partial code blocks to seamlessly chain into unit tests, integration tests, and eventual deployment.

Technical Highlight

Semantic Multi-file Context: Tools like Cursor build a vector index of your entire repository. This means your AI dev buddy can “remember” what each function does and how they interact, enabling more accurate suggestions and fewer errors.

Why It Matters for Business

• Action Point: Leverage coding assistants to shorten release cycles and free up senior engineers for higher-level architecture tasks.
• Compounding Impact: Faster dev cycles → More AI features shipped → Better user data → More training data for your next iteration of AI. This loop catapults adoption in robotics, biotech, and beyond.

Open Source Awakening: Community-Driven Momentum

While Big Tech was busy locking down proprietary models, 2024 saw an open-source renaissance in AI:

• DeepSeek-V3: A 671B-parameter behemoth that pushed the boundaries of open-source performance, outclassing many closed models on standard benchmarks.
• Alibaba’s Qwen 2.5: Offered 100+ open-source variants, excelling in coding and mathematics while surpassing 40 million downloads worldwide.
• Mistral AI's Mixtral 8x7B: Introduced a Sparse Mixture of Experts (SMoE) approach, drastically lowering inference costs by activating only the necessary “expert” sub-models.

Technical Highlight

Sparse MoE: With SMoE, different model “experts” handle specific types of queries or data. This parallelization is key to scaling models without driving up cost proportionally.

Why It Matters for Business

• Action Point: Evaluate open-source LLMs for domain-specific tasks. They can often be fine-tuned in-house at a fraction of the cost of proprietary solutions.
• Compounding Impact: Community-driven innovation fosters rapid iteration of new techniques, which feed back into coding assistants, agent frameworks, and hardware integration.

Multimodal AI: Breaking Down Digital-Physical Barriers

2024 marked the year multimodal AI moved from impressive demos to daily use, bridging voice, text, images, and video in real time. But more than that, it’s driving a cognitive compound interest effect—where each new modality doesn’t just add capabilities, it exponentially enhances learning across all modalities.

• Google Gemini's Flash: Revolutionized real-time video understanding with sub-200ms latency, enabling fluid camera-based interactions and live visual reasoning.
• OpenAI's Advanced Voice Mode(AVM): Achieved near-human conversational AI with dynamic prosody, interruption handling, and “catch your breath” natural pacing—94% user preference over standard TTS in blind tests.
• Multimodal Agents: Combined vision, voice, and text to execute commands like “Find this item in my photos and order a similar one” or “Watch this recipe video and guide me through each step.”

Deeper Perspective: Why Multimodal AI Is Transformational

Traditionally, AI has been “sensorially siloed,” akin to understanding the world with blinders on—only text, or only vision, etc. Multimodal AI removes those blinders. It’s analogous to humans developing both spoken and written language: it transforms how intelligence can develop. This deeper, cross-sensory understanding leads to:

• Knowledge gained from text (e.g., a physics principle) transfers to video understanding (e.g., analyzing physical movements), creating a powerful feedback loop of insights.
• Processing multiple streams—images, text, audio—lets AI see not just what is happening, but why it’s happening, making for more sophisticated reasoning.
• By integrating multiple “senses,” AI starts to learn in a way that’s closer to human cognition—leading to emergent insights not present in any single modality.

Implication: This “cognitive compound interest” means every new modality exponentially boosts the overall system’s ability to learn and adapt.

Technical Highlight

Streaming Multimodal Transformers: Parallel processing of audio, video, and text streams with adaptive attention mechanisms. This lowers latency while maintaining long-context understanding across modalities.

Why It Matters for Business

• Action Point: Explore multimodal interfaces for customer service, e-commerce, manufacturing QA, and hands-free operations—anywhere traditional input methods create friction or complexity.
• Compounding Impact: Better multimodal AI → Richer user interaction data → Improved agent capabilities → More natural and powerful human-AI collaboration. Over the long term, emergent understanding from integrated modalities can reshape entire industries, from entertainment and education to supply chain and R&D.

Robotics & Hardware: Bringing AI into the Physical World

Software breakthroughs alone don’t cut it if you’re solving real-world tasks that require physical action. Enter the 2024 robotics surge:

• Figure 01: Raised $675M at a $2.6B valuation, with advanced industrial manipulation capabilities for picking, packing, and inspection in warehouses.
• Tesla Optimus (Gen 3): Demonstrated advanced mobility and basic household tasks like folding laundry. Musk teased a potential price range of 30,000, with widespread consumer release on the horizon.
• Boston Dynamics Atlas (All-Electric): Moved to an all-electric architecture for improved agility, 360° joint mobility, and advanced obstacle navigation for industrial applications.

Technical Highlight

Reinforcement Learning from Real-World Data: These humanoids don’t just rely on simulation. They integrate sensor feedback, optical flow, and LIDAR data to adapt in dynamic environments. It’s an evolving synergy of vision, proprioception, and planning.

Why It Matters for Business

• Action Point: Identify labor-intensive tasks (e.g., manufacturing lines, warehouse stocking, repetitive domestic chores) where robotics can reduce costs and improve throughput.
• Compounding Impact: As AI models get better at planning (via agents), these robots become more adaptive. Meanwhile, insights from robotics (sensor data, environment feedback) push AI to become more context-aware—further fueling advancements in multimodal systems.

Infrastructure Arms Race: GPUs, Supercomputers, and Edge Kits

All these AI leaps rely on raw compute power—and 2024 saw a flurry of hardware innovations:

• NVIDIA’s H100 GPUs: Set new standards for training/inference efficiency with advanced tensor cores and memory bandwidth.
• Google’s Willow Supercomputer: Claimed record training speeds on multi-trillion-parameter models—estimates suggest 40% faster than previous TPU-based systems.
• Budget Developer Kits: Emerging “micro-supercomputers” (such as Jetson Orin -based boards) provided 20–50 TFLOPS in a desktop form factor, enabling startups to fine-tune large models on-prem without monstrous costs.

Technical Highlight

Memory-Swapping Innovations: HPC clusters in 2024 leveraged near-data processing (NDP) and advanced memory-swapping algorithms, reducing GPU idle time and accelerating training by up to 30%.

Why It Matters for Business

• Action Point: Assess your compute strategy (cloud, hybrid, on-prem) in light of new HPC offerings. In many cases, renting HPC time is cheaper than building.
• Compounding Impact: Improved hardware → Cheaper AI training → More frequent model updates → More advanced features in agents, robotics, and biotech.

Policy & Regulation: Walking the Tightrope

Rapid AI adoption raised high-stakes questions around security, ethics, and regulation:

• The White House: Formalized safety testing with Anthropic and OpenAI for pre-release models to mitigate potential misuse.
• Defense Partnerships: OpenAI, Anthropic, and Palantir redefined the boundaries of AI in intelligence analysis, raising global security and ethical concerns.

Technical Highlight

Model Auditing APIs: Early frameworks emerged for real-time auditing of generative outputs. These systems track token-level decisions to spot potential disinformation or bias, akin to “black box flight recorders” for AI.

Why It Matters for Business

• Action Point: Allocate resources for AI compliance—akin to what the finance industry does for KYC (Know Your Customer) and anti-money-laundering protocols.
• Compounding Impact: Sensible regulation fosters public trust, encouraging broader AI adoption. This synergy pushes the AI ecosystem forward—so long as it doesn’t stifle genuine innovation.

Putting It All Together: How 2024’s Innovations Compound

If there’s one overarching narrative for 2024, it’s interconnectivity and compounding growth:

• Time-Inference Compute → Models that can “think longer” during inference → Superior reasoning without ballooning training costs.
• Optimized Models → Lower inference costs → Wider adoption → More data → Agents get better.
• Better Agents → Tools for next-level coding → Faster dev cycles → Superior robotics planning.
• Robotics → Real-world sensor data → Feeds back into model improvements (multimodal AI).
• Multimodal AI → Rich, natural interactions → Boosts user data & new use cases → Improves training pipelines, accelerating the “cognitive compound interest” effect.
• Healthcare & Biotech → More advanced data pipelines → Helps refine HPC and open-source models.
• Open Source → Rapid community iteration → Unleashes new frameworks for enterprise usage.
• Hardware → Cheaper, faster compute → Accelerates every single domain above.
• Policy & Regulation → Shapes the environment in which all these developments thrive—or stall.

Every domain feeds the next, creating a flywheel of advancement that’s spinning faster than any single sector could on its own.

Looking Ahead: The 2025 Horizon

As 2024 closes, the question for entrepreneurs, engineers, and innovators is no longer “What can AI do?”—it’s “How will I harness these interconnected breakthroughs to reshape entire industries?”

Trends to Watch

• Agent Ecosystems: Autonomous workflows bridging multiple third-party tools and internal systems—think entire call centers run by multi-agent orchestration.
• Quantum-AI Experiments: Early signals suggest quantum hardware might solve extremely large optimization tasks (e.g., supply chain, protein folding) once deemed “impossible.”
• Global Accessibility & Data Sovereignty: As AI expands internationally, debates on data localization, cross-border computing, and ethical usage will intensify.

Final Thought: Don’t just adopt AI—engineer the feedback loops. The real story of 2024 was seeing how each advancement enhanced the other. If you build processes to capture and leverage that synergy, you’re not just catching up; you’re shaping the future.

Here’s to building an AI-powered 2025, together.

– AJ Green