In a world of Large ACTION Models, what does the future of computing hold?

So last night I was reading a bedtime story to my daughter - we come up with a prompt together and set the task for Google Gemini. We had some great success with "Tell me a bedtime story of Warrior Princess Alysha. She meets an old lady on her way to the market. Alysha has empathy for the old lady and was rewarded with three magical beans. Alysha has an adventure in the clouds after climbing the resulting magical beanstalk."

But one time we had a typo and Gemini was prompted, "Tell me the bedtime story of Process Alysha who meets a dashing prince from another kingdom".

Our protagonist quickly gained analytical and engineering skills and, through applying her STEM skills brought progress and joy to her loyal subjects through rural electrification projects, high-efficiency railway infrastructure investments and improvements to regional governance models.

It got me thinking - in this world of increasingly sophisticated Foundational Models (FMs) - you probably know them as "Large Language Models" like Chat-GPT 4 (Turbo!) and Google's Gemini (and Anthropic's Claude!). What will the future hold for us consultants? What would an LLM-embedded in our daily operating systems look like?

Well for starters, the paradigm of mouse and keyboard will slowly evolve away to an augmented M-A-K UI : Mouse-and-Agent-and-Keyboard. After watching us interact with computers for long enough, a LAM (Large Action Model) embedded in the OS's UI would build a library of Actions (not just Languge). What do I mean by that?

"Get the latest news and summarise the headlines for me." It'll learn to open your favourite news sites, ingest the data and aggregate it.

"Book train tickets for my weekend trip" and it'll open the price comparison sites, your diary, get the right routes and book everything.

"Suggest ideas for meal prep next week, summarise suggested ingredients that meet my nutritional needs, and prepare a shopping list with my preferred suppliers" and it'll run the workflow.

Because to the outside world - my interactions with the Internet are just that : clicks, forms, emails, etc. If that were all intermediated by a Foundation Model (FM) what would the future look like?

1. The way we work will change subtly. Introductions at the beginning of work calls will become more important as well as synthesising action points and summaries will become more important. Because you never know which attendees are running with FMs - you need to help out their AI's by introducing yourself (so that the FM can build a model of what your voice sounds like) and summarising the key points (inevitably someone's going to put the call transcript through an FM asking "who was designated what tasks to do by when?"). Ever get that feeling when you're on a call but the other party has you on loudspeaker and someone else in the room at their end is listening in on the call? Get used to that feeling! With audio transcription and LLM's being invited along to work calls, expect that an FM somewhere is going to be analysing your speech and video feeds.
2. Will we be using Agents to represent us in our online interactions? With FM's being able to generate video and audio as well as text - could you imagine a time where we instruct our FM-based Agents to intermediate for us in the world? With the amount of AI-generated content being put out there - what's the future role of hand-crafted, tailored, specific content? Conversely - what about the role of Bad Actors in a world dominated by AI-generated content? With the rise of false news and extreme opinions - how hard would it be to stage a performance for the "news" outlets and social media feeds to sway popular opinion?
3. Will we be using Agents to proxy for us in learning environments? Imagine an FM that is tailored to early years education: a storybook that is constantly learning from the child's lived environment and adapts its curriculum and learnings accordingly. Kid needs to pick up a second and third language skills? Imagine the FM automatically generates Duolingo-style interactive language classes but drawing on themes and subject matter from the child's lived environment.
4. What will be the role of traditionally "people-oriented" service professions? Sports management, Public relations, media & communications, design, consulting, accountancy and legal professions? I uploaded a video of a fencing duel to Gemini and asked for it's analysis - surprisingly Gemini was able to (after a bit of prompting) produce a well-informed analysis and summary, roughly on the level of a graduate learner with some modicum of skill. I asked Gemini to break down the techniques used in the video and asked for feedback - the answers were actually something not too far off.

In a world of FMs, LLMs and LAMs - what's a professional to do? Well, now more than ever qualities like empathy, creativity, judgement, and critical thinking will become even more important. Where FM's can handle the gruntwork of doing the heavy lifting, it now falls to us to direct the FM's, and even architect workflows around FM's (organising FM's to coordinate the work of other FM's!).

Just my musings on a Sunday evening - feel free to comment on speculate further in the comments section below!