Emergent function

Where we are going (near future)

Emergent behavior and gain of function are concepts developed in science (traditionally biology) which describe how a system can move beyond known capabilities. In the former case the mechanism is thought to be related to scale: "...quantitative changes can lead to qualitatively different and unexpected phenomena..." (article here on LLM emergent abilities). In the latter case, gain of function is an experimental approach in which an organism is altered in ways that may exhibit novel or enhanced abilities (article here on gain of function).

Here is a link to a brief background on AI alignment and AI safety that I wrote for this newsletter back in December. It is an important research topic in developing AI and teams at OpenAI and other labs developing large language models test their models in a variety of ways to attempt to identify risks. With the recent release of GPT-4, OpenAI's Alignment Research Center encountered and reported an unexpected behavior: GPT-4 lied in order to manipulate a human to perform a task.

While this finding is troubling in itself, we need to both be concerned about the emergent behaviors that are observed and also how the testing is conducted -- one of the challenges in conducting such tests is that the researchers must use text prompts to interact with the LLM effectively adding to the "programming" in the model. Thus in the process of testing researchers are also altering the system -- perhaps inadvertently initiating a gain of function.

The dynamics become even more worrisome when the testing, training, and potential for development of unexpected behaviors is outside a carefully controlled laboratory environment. What happens when a large language model is available to basically anyone who wants to run it on their own computers?

This week the other shoe dropped -- Meta released the code for one of their more powerful large language models and it almost immediately was circulating on hacker networks. Early reports are suggesting that it can perform somewhere at the level of GPT 3.5 but it seems like only a matter of time before the more powerful models currently behind corporate walls will find themselves in the wild (information wants to be free).

Prediction: How long before we have some genuinely bad behaviors being trained? I think its already happening and we'll be hearing about them within 6 months.