Thoughts on Generative AI

I started working on generative Artificial Intelligence (AI) a few years ago. I remember our first project: we wanted to generate destination content with Recurrent Neural Networks (RNN) around 2017. The output was... creative! It had an impressive ability to make up words and generate non-sense, even after several rounds of improvement and customization. We had lots of fun reviewing the content and trying to make the solution (at least) work. We failed.

As we all know, technology has evolved dramatically in the recent years, and even RNNs could now be a viable option in some cases. Also, even my parents are talking about Generative Pre-trained Transformers (GPT) these days. This means that we are still close to the top of the hype. The launch of chatGPT in 2022 was a tipping point for the whole world that made us think about Artificial General Intelligence (AGI) and its portrayal in movies like Her or Ready Player One.

I'd like to share three thoughts that are top of mind for me:

• Today, it is remarkably easy to start and build a prototype in a matter of hours or days thanks to e.g. Langchain, Llama-Index, or direct APIs to OpenAI and competitors like Anthropic or Mistral, among others. If you want to deploy Open Source models, at this point I'm sure everyone knows Hugging Face. Achieving the desired level of quality and scalability, though, is hard, and requires other skills besides just Machine Learning or AI, see e.g. this blog post. Integrating broad AI capabilities might be hard for small and medium-sized businesses, but success is a matter of prioritization and setting the right goals. In any case, I really believe that the only way to understand what Large Language Models (LLMs) can do for you and your organization is to start small, iterate and learn. Then, scale up the solutions that work in your particular use case and disregard everything else.
• Major tech players like Google, Microsoft, Amazon, Apple, and Meta, are heavily investing in generative AI. My advise to teams is always to critically evaluate PR materials and consult independent leaderboards like Stanford or Hugging Face. Also, nothing beats your learning by doing with the best-suited experiments for the business goals you are trying to achieve. Sometimes you can be surprised! With the rapid pace of change, try to design modular systems so that you can replace any component if needed. Working with specialized partners can also accelerate the time to market, which is more crucial than ever.
• Generative AI is just one piece of the puzzle, an interesting option you have to solve a business pain. However, everything starts with a business problem and the decision on the best-suited technology to help you do the job, not the other way around. In AI, less is more, the simpler, the better. Beauty lies in simplicity. And in a practical sense, you will have less points of failure and less components to maintain and optimize, among other benefits. Any technological change will also involve people and processes, and for AI, also a thorough data strategy. Don't forget about it, sometimes change management can be the most challenging aspect.

It's clear that I'm beyond excited about the possibilities of Generative AI in particular, and AI in general. So, what can we expect in 2024? The three areas below are key in my opinion.

• Multimodality: my focus so far has been primarily on text, although we've also seen great success with images and speech. Video is getting closer every day. Being able to work with any input and any output with high quality is evolving quickly. The possibilities are amazing, and we're moving towards the best interface: no interface at all. Our devices will always be with to us to perform tasks in the background to boost our productivity.
• Personalization: Generative AI with feedback allows for true personalization of the LLM output. With smaller models deployable on edge, we will be able to have a helpful autonomous agent with us 24x7x365 on device. Apple might lead this front with a strong enough moat, but anything can happen in this AI race. Some promising developments might still be under the radar.
• Predictability and control: chatGPT and all LLMs have helped us be more creative, but certain tasks require control and consistent output. To achieve that goal, classical Machine Learning (ML) models might be better-suited in some cases. I'm confident we'll see many more options for controlling the LLM output, not only by keeping the conditions constant, but also by fine tuning smaller models to successfully solve a task, and leveraging other new techniques. The progress is exponential and never stops.

I've touched on the past and our present, but what about the future? There are some questions we all need to address collectively:

• In business scenarios, regarding security, data privacy, and control: do we prefer a faster time-to-market and trust big corporations or should we rather sacrifice performance and deploy on-premise? Do we want to make Generative AI a core part of our offering or do we want to outsource to experts? How do we best leverage our data without losing in the long-term?
• For us as individuals, what is the impact of these generative AI solutions? How is our brain evolving? What are the consequences of being augmented at all times? Will we be ready for the challenges? Are we moving towards a dystopia or an utopia?
• And finally, for the society as a whole, will we create or destroy more jobs? Will this radical evolution increase or decrease inequality? Who will win and who will lose?

Certainly, exciting times to live! Up for the challenge?