Main takeaways from our panel on Open Source AI

Summarizing the main takeaways from our panel session at OpenSouthCode with Ezequiel López Rubio and Francisco J. Veredas :

On the Transformer architecture

- The Transformer architecture took the existing concept of attention and applied it with great success for the purpose of language modelling.

- The Transformer archicture is far from being used up and new innovations based on this architecture continue to emerge.

- We don't know what's coming after the Transformer architecture, and we might indeed hit a wall and stall AI progress on the software side, if new innovations are not found.

- In contrast, we don't expect hardware improvements for AI to be stalled anytime soon, and continous improvements in compute performance will likely continue well into the future.

- The Transformer architecture is a key piece in a complex ecosystem of research ideas and compute capabilities, but it can't solely explain the success of current LLMs. A key piece of the puzzle was the discovery of self-supervised learning techniques, which yielded astronomical amounts of labelled training data without human labelling effort. So far, we haven't found a similar technique in other domains, such as computer vision.

On the limitations of current AI technology

- The internal workings of large neural networks are still largely a mistery. Not even their creators can tell what's going on inside the neural network. The sheer amount of internal variables makes it practically impossible to reason about the internal state of the neural network. The community needs to develop new techniques to interpret the model responses. Until this is achieved, we must be careful when evaluating the model responses.

- Despite the success of current AI models, there continue to be long-standing limitations in certain fields, such as bioinformatics, for instance, where data labelling continues to be challenging due to a) lack of uniform criteria from medical practitioners, and b) lack of sufficient labelled data both in images and text.

- Models are used sometimes as oracles, without proper understanding of the underlying characteristics of the technique. The model doesn't have context unless we provided it as input, and thus may guess incorrectly in producing a response. Users operating these systems should be aware of this nature of the model and how to mitigate it.

On the risks of AI on society

- It is important to see past the hype and to continously educate the general population about the capabilities of the technology and its associated risks.

- While some actors downplay the risks or exacerbate them, it is true that there are important concerns about AI technologies, such as biases, non-factual information, or environmental effects.?

- Current AI systems are far too powerful to leave unregulated but how can we even begin to regulate if we don't know what exactly needs regulating??Too strict regulation might hinder otherwise beneficial advances but an unregulated field could produce significant harms in society. A large debate, involving all participants of the society, is needed in this case.