Emergent function
Where we are going (near future)
Emergent behavior and gain of function are concepts developed in science (traditionally biology) which describe how a system can move beyond known capabilities. In the former case the mechanism is thought to be related to scale: "...quantitative changes can lead to qualitatively different and unexpected phenomena..." (article here on LLM emergent abilities). In the latter case, gain of function is an experimental approach in which an organism is altered in ways that may exhibit novel or enhanced abilities (article here on gain of function).
Here is a link to a brief background on AI alignment and AI safety that I wrote for this newsletter back in December. It is an important research topic in developing AI and teams at OpenAI and other labs developing large language models test their models in a variety of ways to attempt to identify risks. With the recent release of GPT-4, OpenAI's Alignment Research Center encountered and reported an unexpected behavior: GPT-4 lied in order to manipulate a human to perform a task.
While this finding is troubling in itself, we need to both be concerned about the emergent behaviors that are observed and also how the testing is conducted -- one of the challenges in conducting such tests is that the researchers must use text prompts to interact with the LLM effectively adding to the "programming" in the model. Thus in the process of testing researchers are also altering the system -- perhaps inadvertently initiating a gain of function.
领英推荐
The dynamics become even more worrisome when the testing, training, and potential for development of unexpected behaviors is outside a carefully controlled laboratory environment. What happens when a large language model is available to basically anyone who wants to run it on their own computers?
This week the other shoe dropped -- Meta released the code for one of their more powerful large language models and it almost immediately was circulating on hacker networks. Early reports are suggesting that it can perform somewhere at the level of GPT 3.5 but it seems like only a matter of time before the more powerful models currently behind corporate walls will find themselves in the wild (information wants to be free).
Prediction: How long before we have some genuinely bad behaviors being trained? I think its already happening and we'll be hearing about them within 6 months.
Global Sales Specialist - Intelligent Automation, iPaaS & Low Code
1 年Humans, by their very nature are "lazy" and will just accept generated output as "truth". With #gpt4 still?“hallucinating” facts and making reasoning errors we run the risk of dumbing down what is "truth" and instead of teaching #llm to become better it will just get worse
Go-to-Market Leader | AI Automation Strategist | Author | Driving Growth Through Intelligent Solutions
1 年It’s the plagiarism for me.