The 9 biggest tech vibe shifts of 2024

One of the most fascinating behavioral psych aspects of the tech industry are vibes.

Vibes encapsulate our feeling about a company, product or theme

Vibes are powered by wow-ing launches, purposeful narrative, cracked talent movements, the cult of the underdog, overcoming innovator's dilemma, realizing that something is far bigger than you thought...

In honor of DeepSeek AI and their killer launch of v3 this week, lets kick of this biggest vibe shifts of 2024 with Chinese AI.

Just a Llama rip off

As we wrote in the 2024 State of AI Report , Chinese AI and tech were hit with export controls on NVIDIA GPUs, sanctions, investment controls, cratering tech markets, justifiably bad press around the mass-scale use of facial recognition services, and of course, an increasingly hawkish anti-china stance from the West.

Together, this meant that Chinese AI wasn't really in the positive vibes spotlight.

When they did try to capture open source vibes, this didn't get off to a great start:

"Chinese developers and media recently revealed that Yi-34B, a LLM from Kai-Fu Lee’s new AI startup 01.AI, closely adopts the architecture of Meta’s LLaMA 2 model, with only minor modifications like renaming two tensors. This discovery sparked wide discussions within China’s AI community about the model’s originality and adherence to open-source licensing norms."

Wow, China is cooking!

And then, the vibes started to change. First, Alibaba Qwen team open sourced Qwen-1 in Sept '23, Qwen-2 in Mar '24 and Qwen-2.5 in Aug '24. these increasingly large scale and capable language models displayed noteworthy performance on coding, math and reasoning problems. it caught the community's attention.

And then came DeepSeek with v1 in early 2024, v2 and coder which improved the model's coding capabilities, and v3 on xmas day. v3 is widely regarded as one of the best open-source models available tosay, outperforming many others such as gpt4o, sonnet 3.5 and llama 3.1 405b in coding, maths and english reasoning benchmarks.

Importantly, deepseek achieved these results on a minimal compute budget compared to their overfed peers in the west: using 2k h800 gpus that lack the same memory and interconnect as the full blown h100/h200 systems...and for a total cost of <$6m (pretraining and posttraining).

Rather fascinating to see - Chinese AI labs are lively open source contributors, pushing state of the art results despite resource constraints.