Ep. 5 | What's the latest in AI? The Autumn Update

Hi folks, it's been a minute. ??

Life happened, summer happened, and then life happened some more. Welcome back to the fifth edition of my newsletter: Perplexing Tech.

Like me, you may find that the pace of innovation in technology - and AI more specifically - is quite overwhelming. For that reason, I've written this edition to give you an overview of where we are are up to so far, and what we might expect in the final quarter of the year.

I'll start by giving you a primer on what's going on at OpenAI (because on any given day you could flip a coin and get 'major leader resigns due to safety concerns' or 'huge new development shipped to users' ... in the past couple weeks we got both, twice!), then I'll run through the major updates of September, and finally touch on some of things we might anticipate seeing before the year finishes...

In personal news, a fortnight ago I joined peers in Amsterdam for an FEC Data event, where I spoke to industry players about ethics and legislation in responsible AI, as well as major architectural research that will help us capitalise value in CDD.

And this week, I'm speaking at Delta Capita 's ESG breakfast about AI and data in ESG reporting, and then I'm joining the GenAI Risk & Governance panel at AFME (Association for Financial Markets in Europe) 's OPTIC conference in the city. Exciting!

First of all, what's going on at OpenAI?

... and who still works there? (clue: it rhymes with Ham Paltman)

? Earlier this month, OpenAI launched the o1 model series ("strawberry", to those that have been following along). I wrote about it, and my takeaways from the system card.

This is really important, so let me try and explain:

?? This is a different, smarter model structure. A reminder that OpenAI use closed source models, so we can't know everything about what is going on behind the scenes, but we do know that this is not just a bigger or better LLM architecture.

?? It introduces new components, e.g., a 'time-based component' that gives the system 'reasoning tokens' that it uses to compute a better response. We don't see this as the user, but it means more time is allocated to working through the ask, the next steps, and likely follow-ups.

??This is the first really good example of what we call "chain-of-thought" reasoning. It organises 'thinking' better, and organises recommendations better. I use, and would still use, Sonar Large 3.5 through Perplexity for text generation, text review, definitions etc., but I would absolutely now use o1 for difficult questions and - as has become my number one recommended use of AI to friends and colleagues - conversational dialogue.

?? A really good outcome from the new model series is that we don't need to give highly tailored prompts. It does a sensible structuring for you, regardless of whether you ask for it or not.

?? I've seen some really good results for code generation and review (I'm going to post a walkthrough video soon of how I used it to build a market scanning news tool in Python), and I've been using both o1-preview and o1-mini every day since they came out. I've not tried via API yet.

? In less shocking OpenAI news, you may have seen that Sam Altman told staff that the company's non-profit corporate structure will change next year. He reportedly said that the company had "outgrown" the unusual structure it currently has and will therefore change. Colour me shocked!

There's a great read in The Wall Street Journal from the weekend about how complex (and, frankly, incredibly unusual) the journey from non-profit is, and why OpenAI has an insane challenge ahead to make it happen. This journey also considers whether Sam Altman could get a 7% equity stake in the company ...

? The long-time CTO of OpenAI (and previous interim-CEO during "Samgate"), Mira Murati, resigned last week. As did the Chief of Research. As did the VP of Research. Clearly, people want to understand the real reasons behind these departures, and OpenAI's reliance on development of ChatGPT has left them in a tricky position compared to competitors who are majorly diversifying (see my section below).

Major updates in September

... and the AI status at the major tech players

? OpenAI

• As above! +
• Advanced voice mode has gone live (not in the UK, wah!). This is the part of the OpenAI tech demo earlier in the year that really caught people's attention, because they claim that the model can "sense and interpret your emotions from your tone of voice". It also does some, err, quite strange accents, as Allie K. Miller showcases in her recent post.
• Rate-limit increases to the o1 models have increased from 30 to 50 messages per week. GPT-40 has increased to 50 messages per day (thank god!)

? 谷歌

• Announced two new production-ready Gemini AI models: Gemini-1.5-Pro-002, and Gemini-10.5-Flash-002 (catchy!)
• One of the most impressive AI tools to exist so far is NotebookLM / Audio Overviews. If you've not tried this yet, you are seriously missing out. If you take ONE THING from this newsletter, please, go and try it out (and shout out to my Dad for this one, because I had totally missed the release).

The tool takes notes, PDFs, and other docs and turns them in to these insane AI-generated podcasts. The functionality got an upgrade this week, and you have to try it to believe it. You can now upload a Youtube video or audio file (e.g. lectures, long in-depth sessions) and get these really vibrant study guides

• On the other side of the office, Google DeepMind showed off AlphaChip, which is a system that uses reinforcement learning to design and layout chips. This is huge, as chips that are taking months to be built may feasibly be developed in hours. Let me say that again, this is huge! It's really interesting to think about the direction of this over the next few years.

? Meta

• Meta Connect was a hoot last week, with a ton of cool updates from Zuck and friends.
• The Orion "advanced AR glasses" were announced. "It needed to be glasses, not a headset, with no wires and weight of less than 100 grams". And, hey! they've actually achieved that. Sure, they're still a bit Stanley Tucci-esque for my liking, but it's a really impressive development in the world of AI wearables.
• Pay attention to this, because Meta aren't just trying to carve out space in the world of AI-powered glasses like we currently think of it, they're trying to make a place for dominance as the future companion device (think "bye-bye iphone").

Side note: you might have seen content from me before about 'Post Digital', and how the acronym DREAM-C covers the technology that defines the next paradigm of modern society. The 'E' is 'extended reality (AR & VR)', and developments like this prove exactly why that 'E' needs to be included. Wearable tech is absolutely going to define our future.

• Meta also dropped Llama 3.2, and Meta AI (the chatbot that will be integrated in all your Meta-owned social media like Instagram, WhatsApp) is also now multimodal. That means you can include images and image editing. Meta have been committed to open source AI, and that's commendable in this environment.
• Meta Quest 3S is here, at they low price of $299. Great news for fans of Beat Saber (are we using it for anything else at this point?)

? Stability AI

• They've just brought James Cameron on to the board of directors. Visual media is about to get some serious attention. This follows the appointment of Hanno Basse as the new CTO back in August (a veteran of the entertainment industry). Though often less visible that the Meta, OpenAI, Google, Microsoft types, Stability AI are working away to carve ground in the media sphere.

? 微软

• Microsoft's whole thing is about how AI let's "humans become more human" by passing over the boring and mundane stuff to agents. To that end, they recently announced that agents are coming to Microsoft 365 Copilot. Agents workflows are really the 'value add' application of modern AI to the corporate world, and the ability to build an agent for a particular purpose (e.g., a marketing assistant, a HR advisor) within Copilot will be hugely valuable.
• I'm a big fan of Copilot enterprise, but my main criticism of Microsoft (and of those I chat to about them) is that they roll out little, random updates, too frequently, and without an explanation of why. We got Loop, then Loop crashed, then Loop changed. Adoption of enterprise Copilot has been slow, with lots of beta, and now it's getting a v2.
• Microsoft are also working with BlackRock to raise a humble $30-100 billion to "emancipate themselves from the shackles" of insufficient power by investing in data centres and power supply chains etc.

? Anthropic & 亚马逊

• You may, or may not, have seen that in the UK our Competition and Markets Authority (CMA) was investigating Amazon to ascertain whether than four billion dollar investment in Anthropic was negatively impacting competition in the sector:

"...the CMA said that it tested Amazon’s arrangements with Anthropic against thresholds for turnover and share of supply, concluding that neither was met, and therefore a further investigation will not be pursued."

In other news ...

?? Hello to Tennibot ! Tennis, meet robot.

? At Lenovo Innovation World 2024, the tech giant went big on pioneering AI devices. They unveiled several AI-driven devices including the Yoga Slim 7i Aura Edition and the ThinkPad X1 Carbon Gen 13 Aura Edition. It's anticipated that by 2027, 60% of PCs being shipped will be AI-capable.

?? Folks got really angry that LinkedIn had been scraping their data for training before updating its terms of services, and before releasing an opt out feature.

? The US Commerce Department has proposed mandatory reporting requirements for developers of advanced AI models, through the lens of national security. Ultimately, they would require AI developers to provide detailed reports on their development activities, cybersecurity measures, and outcomes from red-teaming effort. The Bureau of Industry and Security (BIS) is pushing this on the basis that innovation is lowering the barrier to entry for non-experts to do harm with CBRN weapons. They're right - see my earlier post about the o1 models and the results in their system card.

?? The AI coding assistant Supermaven has been working away raising cash from OpenAI and Perplexity co-founders. It has a massive context window which makes code copiloting more effective / reduces hallucinations.

? In my last update, I told you about a major new industry group: the Ultra Accelerator Link (UALink) Promoter Group. Comprising of Intel Corporation, Google, Microsoft, Meta and friends, the group was announced to facilitate development of the components that link together AI accelerator chips in data centres. What's happened since?

? I came across an incredibly interesting and well-written post by Raymond Sun this month on "AI security" versus "AI Safety". It was fascinating to see a breakdown of some key points from the recently published "Artificial Intelligence Safety Governance Framework" by the China Cybersecurity Standardisation Technical Committee (catchy), but also fascinating to read about the nuance between concepts presented in English vs. Mandarin, and if there are implications from this. Well worth a read!

What can we expect in the remainder of the year?

• More changes at OpenAI as they navigate the road to becoming "for profit".
• Advanced multimodal models. In lay-terms, models that are good at integrating and processing different types of data (text, image, video, audio) more effectively. At the moment, models are generally good at one thing, e.g., a LLM for text generation. Scaling up to really advanced and effective mixture-of-expert architectures and/or multi-agent workflows is hard to do without also exponentially scaling power usage / compute requirements.
• Smarter robots, or at least one landmark announcement in the humanoid robot arena. My money is on Figure launching something before 2025, possibly a step forward in the causal reasoning space.
• Ramping up in Europe for the first obligations under the EU AI Act. You can read my overview of the timelines here. Just today, the chairs for the first General-Purpose AI Code of Practice have been announced by the European Commission.
• Some sort of mass negative story about AI use in elections (deepfakes, synthetic media), as at least 26 U.S. states have passed or are likely to pass bills with the intention of regulating the use of AI in political campaigns.

? If you're struggling to stay up to date with developments in AI and other innovative technologies, let me do the work for you! I post weekly summaries on the feed, and this newsletter is (usually) every fortnight. ?

Hope you found this interesting - let me know!