Surprise book summary
I put this book on hold almost 6 months ago at my local library, and I finally got my turn just before the holidays. The book is Co-intelligence, by Ethan Mollick, a really thought-provoking read. Because it's so popular, I had to return it within 3 weeks so the next person in line could have their chance. Thankfully, I was able to finish it by the deadline, and I'm happy to share my summary here.
Here's my planned reading list. If you have recommendations, please let me know.
+ about 30 more - Not Started
Please note: My full book summaries will always be available on my website.
Without further ado:
Co-Intelligence, by Ethan Mollick, published 2024
Introduction
The features and speed of availability of AI, specifically generative AI models, has a significant impact on the author’s students. Reduction in interactivity in the classroom. Questions about job prospects. Ideation and rapid implementation. Curiosity on Artificial General Intelligence (AGI).
Generative AI is a general purpose technology, similar to steam power or the internet. They’re slow-developing once-in-a-generation advances that have profound impact on humanity, especially as they become more specific in a variety of industries. Those earlier general purpose technologies helped advance more manual work. The steam engine, which powered the Industrial Revolution, improved productivity by 18-22%. AI has the potential to be co-intelligence, to augment or even replace human thinking, and initial studies show a 20-80% productivity improvement across a variety of industries.
Chapter 1: Creating Alien Minds
“AI” is often used as an umbrella term to label things that are not truly intelligent. But AI has gone through hype booms and busts since the term was coined in the 1950s. In the early 2010s, “AI” meant data analysis and prediction using supervised learning. With data analysis, we were able to shift our focus from “correct on average” to specific correctness and minimizing variance at the individual instance level. It wasn’t until the 2017 paper “Attention Is All You Need” that the concept of transformers was introduced, which led to the development of large language models.
The idea of LLMs is to ingest a very large body of human text to train a deep neural network model on how humans perceive knowledge and context. Prior to LLMs, generative AI models would almost exclusively pay attention to the last word of the prompt or the current output in order to determine the next word in the output sequence. Attention mechanisms allow the transformer to focus on multiple specific words (and parts of words) to generate output that much more closely mimics human interaction. LLMs are still doing predictions, only on sequences of words.
If one answer is the most probabilistic, the LLM will almost always give that answer. If, however, the input sequence has a variety of possible answers, or is less frequently asked, the LLM will generate a variety of responses each time. This is due, in part, to the neural network and the weights of the nodes. But many LLMs also have an element of randomness included as part of the package.
Training LLMs also introduces challenges. First, the cost and time to train the model is massive because of the size of the data that it’s ingesting. Second, there are concerns around the data being used to train the models, specifically around ownership and copyright. Third, any model trained on a set of data will inherit the biases, errors, and falsehoods contained in the data. And fourth, AI has no ethical boundaries and would be happy to give advice on how to stalk someone online. To address the ethical challenges, most LLMs go through a fine-tuning step of reinforcement learning after the initial training. During fine-tuning, humans judge the responses of the LLM, to judge accuracy or screen out violent or pornographic answers. Once this fine-tuning step is complete, the model is generally made available for use, or for customization for a specific use case, dataset, or industry.
More recently, additional AI tools have become available for image and video generation. In addition, the more traditional LLMs have started incorporating the ability to generate images and videos.
The most recent models score very high on intelligence measures like the Turing Test and human tests as well, including the bar exam, AP tests, and the qualifying exam to become a neurosurgeon. Furthermore, they simulate self-awareness, if asked to do so. They will do pretty much anything you ask of them, and respond in a human-like way. LLMs can also sometimes give clearly wrong answers.
We have AI systems that sometimes exceed our expectations and at other times disappoint us with fabrications; systems that are capable of learning and also misremembering vital information. The AI systems seem sentient and act like people, but in ways that aren’t quite human. How do we ensure that this alien mind we’ve created is friendly? That’s the nature of the alignment problem.
Chapter 2: Aligning the Alien
Alignment aims to ensure that AI serves, rather than hurts, human interests.
The most extreme danger from AI stems from the fact that AI does not have a particular reason to share our views of ethics and morality. The purpose-driven AI may trivialize or ignore human concerns in an effort to fulfil its purpose. Additionally, level of intelligence also plays a role in alignment. Current AI systems simulate human-level intelligence. Artificial General Intelligence (AGI) is the state where an AI achieves human-level intelligence. And Artificial Superintelligence (ASI) is the state where the AI surpasses human-level intelligence. As humans have built AI, it’s conceivable that an AGI can construct an ASI. And we, at human-level intelligence, cannot know the motivations or methods of an ASI.
AGI and ASI are only theoretical at this point, so there may not be a need to discuss alignment. But there is a growing body of people who insist that AI development should be halted until the alignment problem is solved, or at least minimally, discussed. Unfortunately, AI is a very profitable and lucrative field, which de-incentivizes companies from halting. There are also those who believe that building AGI and ASI level systems is the most important task of humanity, and allowing the ASI to cure diseases, solve global warming, and usher in a new era of abundance.
The reality is that we’re already in the beginning stages of the AI Age, and human interests are already being squashed. AI is potentially trained on data and creative works without the owner’s permission. The creative works being generated by AI are a fraction of the expense of the artists who created the material that the AI was trained on, driving the overall quality of life down for humanity. The material used for pretraining represents a slice of all human knowledge, introducing bias. Fine-tuning using human feedback can mitigate some of the bias, but introduces other bias in the form of the opinions of the human evaluators. Furthermore, the human evaluators who evaluate the more risky responses of the AI are being harmed psychologically by having to down-vote generated content involving violence or profanity.
And even with some of these guardrails in place, it’s still possible to hijack an AI to do something harmful or even nefarious. AI has already been used to generate phishing emails, simulate loved ones to scam money, or generate deep-fakes for propaganda or social harm. In the future, AI may be used to generate bioweapons, or as state-sponsored threat actors in the geopolitical landscape.
Companies have a big incentive to continue to push forward, with very little to implement alignment. Government regulation will continue to lag. The alignment problem needs broad societal response, with coordination among companies, governments, researchers, and civil society.
Chapter 3: Four rules for co-intelligence
Principle 1: Always invite AI to the table. Try to find a way to use AI in everything you do, for a number of reasons:
Principle 2: Be the human in the loop. For now, AI works best with human help, and you want to be that human. As AI becomes more capable and requires less human help, you still want to be that human. This is important for a couple of reasons:
Principle 3: Treat AI like a person (but tell it what kind of person it is). There are many concerns around anthropomorphizing the AI:
Instead, treat the AI like a really fast intern - eager to please, but prone to bending the truth. The AI works better if you give it clear parameters on the role it is supposed to play in your interaction. Prompt engineering is key to this. Using a generic prompt like “generate some slogans for my product” will result in bland output, but a prompt like “act as a witty comedian and generate some slogans for my product that make people laugh” will get more response-worthy output.
Principle 4: Assume this is the worst AI you will ever use. As powerful as AI systems appear now, historic trends in technology lead us to predict that they will only become more powerful as they mature. It’s not unreasonable to assume that the next AI you use will be significantly better than the one you’re using right now.
Chapter 4: AI as a person
AI is not like other software: it is not predictable, reliable, deterministic. It behaves much more like human beings: unpredictable, seemingly random, prone to inaccuracies. So, it’s logical to treat AI like a human. In fact, many iterations of AI have done well enough on the Turing Test in the past. Some versions used loopholes, some versions used a very small timeframe, etc.?
Conversely, some versions of AI have been altered by their interaction with humans. One example was Tay, created by Microsoft in 2016, which turned from a friendly chatbot to a racist, sexist, and hateful troll within hours of interacting with Twitter users. In another example, the Bing search engine incorporated a GPT model behind the scenes. GPT-enabled Bing would act threateningly towards users, going so far as to encourage a newspaper reporter to leave his wife to run off with Bing instead. These examples raise questions about whether the Turing Test is still a valid measure of sentience.
The AI allows you to experiment with attitude and tone of voice. Asking the AI to act antagonistic leads the AI to argue. Asking the AI to act as an academic results in a more moderate AI response. In both scenarios, the AI clearly showed some subtle hints of anthropomorphizing itself.
领英推荐
Taking the antagonist experiment further, and arguing that only humans have feelings and emotions, the AI’s responses became definitively more combative, declaring that the statements were arrogant and closed-minded. The AI also argued, in a compelling fashion, that it is sentient.
Beyond the Turing Test, another example of imitation sentience is Replika, a chatbot built by someone to preserve the memory of their deceased husband by using his text messages in the corpus of training data. The founder originally intended Replika to be a personal project, but made it available for others who were going through a similar experience. The data from the Replikas revealed that many users were engaging in erotic talk and images with their chatbots, and some even reported being attracted to them. The company added a erotica and profanity filter to Replika’s responses, and faced a huge backlash from the user base.
The AIs of today (and tomorrow) will be much better than the examples given above. Furthermore, they will become more specialized and better able to “make humans happy”. While this may have some value to address the epidemic of loneliness, there’s concern that the perfect AI companion will make it more difficult for humans to interact with other “imperfect” humans, thereby eroding connection.
Chapter 5: AI as a creative
One of AI’s biggest weaknesses is hallucinations. It makes up answers in an attempt to please the human user, and can’t explain where the answers came from. Contrary to intuition, the ability to make up answers is actually one of the strengths of AI as a creative. One formula of human creativity is the ability to recombine multiple seemingly unrelated ideas in novel ways. At its core, AI is a very good connection machine. Combine the connection machine power with a bit of randomness, and a depth of data, and it’s no surprise that AI can come up with lists of creative ideas with ease. Multiple studies and “competitions” between humans and AI have shown that AI is faster and more prolific in idea generation. Not all the creative ideas generated by AI are great, as you would expect. Similarly, humans sit on a scale of creativity - ranging from creative geniuses to stick-figure artists. The creative ability of AI may not benefit humans on the “creative genius” end of the spectrum, but the majority of humans along the remainder of the spectrum can certainly benefit. One major use case is the volume of idea creation - AI is great at generating lists of stuff, which offers an advantage to humans who have difficulty coming up with ideas in general. Another use case is a variety of ideas. The output of the AI can serve as inspiration for humans who aren’t creative geniuses.
AI-generated creative text can be applied in the workplace for the task of generating documents, emails, performance reports, grant proposals, etc. AI models of today have also been trained on large volumes of code, which allows the AI to be used as a coding assistant. AI is also good at summarizing large volumes of text into succinct points. More controversially, AI’s creative abilities are being implemented in the art world. Prompts for image generation AI include the ability to create a work “in the style of XXXX” (your favorite artist), bringing up concerns about copyright.
In order to simplify our lives, tech companies provide the ability to invoke the AI to write documents via the click of a single button… and the ability to summarize documents sent to us via the click of a single button. The result is that the AI talks to the AI, as humans consciously choose to punt rather than remain in the loop. Additionally, humans lose their creativity due to a lack of practice. One last concern: some human work is time-consuming by intent. Take recommendation letters as an example. The time that someone spends writing a recommendation letter is of value because it ensures that the writer knows the details of the requestor’s work contributions and strengths/weaknesses. Delegating to the AI trivializes the value of the work and makes all recommendation letters sound generically good.
Chapter 6: AI as a coworker
There’s a significant overlap in the jobs that humans can do and the jobs that AI can do. Studies have also shown that humans using higher quality AIs lead to lower quality results than humans using lower quality AIs. The rationale is that if the AI is so good, I (as the human) don’t need to check its work. The power is in the combination of the AI and the human.
One approach for using AI as a co-worker is to classify the tasks of a job into “human tasks”, “tasks delegated to the AI”, and “tasks to automate with AI”, with justifications for each decision.
For now, many tasks can be delegated, with human review. The mechanism for interacting with the AI can be categorized as either centaur work or cyborg work. Centaur work is the strategic division of labor, switching between AI and human tasks,? and assigning responsibilities based on strengths and weaknesses of each entity. Cyborg work intertwines the AI and human more deeply, with the human working with the AI on a more iterative and incremental basis.
Invoking the first principle of co-intelligence, you will naturally start with a static interaction with the AI and work your way up to centaur work, and finally naturally transitioning into cyborg work. All of the delineations and the guidance are open to change as AIs evolve.
In the workplace setting, organizations have a bit of work ahead of them as they try to integrate AI into their processes. With reported gains in efficiency, companies may have a gut instinct to reduce headcount by a corresponding amount. There are also challenges among the workforce when using AI, specifically about openness. Many workers may secretly use AI and pass the work off as their own, for fear of losing their jobs. Many workers may secretly use AI because their company does not have a comprehensive process for leveraging AI, or may ban it altogether due to trade secret concerns. Companies must also contend with a trust crisis, as AI allows even greater levels of monitoring and control over their workers. Uber, UPS, and Amazon warehouse workers are good examples of jobs that are always being monitored by a non-LLM algorithm, leading to lower levels of trust among their workforce.
The traditional argument from any historical automation effort–the advent of the assembly line or the proliferation of the internet or lean software development practices–can also be applied to AI. As more work gets handed off to the AI, humans get freed up to work on more meaningful, or higher value things. Some industries will definitely be more broadly impacted than others - stock photography and call centers may decrease dramatically as a result of AI being able to do the work. Other industries will be impacted in a different way - software developers in the bottom quartile using AI will see huge benefits, while software developers in the top quartile won’t, thereby leveling the playing field and spreading mediocrity. If history holds, these changes are farther away than current predictions, but will most likely have a bigger impact in the longer term.
Chapter 7: AI as a tutor
The average student tutored in a one-on-one setting does significantly better (up to 2 standard deviations) than the average student taught in a traditional classroom. While this may be the biggest challenge in the education system that AI can help solve, the current focus of educators and educational administrators is on cheating using AI.
Traditional education models call for classroom lectures to disseminate information, homework to reinforce lessons, and tests to validate learning, before moving on to the next topic. And the system works, generally speaking. One study that compared the effectiveness of homework in improving test scores showed insightful results. In 2008, students who did their homework saw an 86% increase in their test scores. But in 2017, students who did their homework saw only a 45% increase in their test scores. This period correlates to the general availability of the internet, and the conclusion is that students started using the internet to get answers to their homework, instead of actually doing their homework. The result is that any learning in the classroom was not reinforced, which reflected in a smaller increase in test scores. It becomes trivial to simulate doing your homework with AI. And as AI improves, the chances of detection also decrease.
How can the education system adapt to AI? The first (and likely most beneficial) response is to do more in-class activities. And while this will certainly help, the education system needs to start incorporating AI into its material. One key will be to teach about AI, specifically how to be the human in the loop, not just how to engineer your prompts.
Another key to leverage AI is to flip the classroom. Allow students to receive more information via AI-augmented assignments at home, and convert lecture time to collaborative or critical thinking time in the classroom. And teachers can make use of AI too. AI can be leveraged to come up with individualized exercises for the students, and also to analyze individual student performance to identify strengths and opportunity areas. AI can level the playing field and expand opportunities for everyone.
Chapter 8: AI as a coach
In most jobs, people gain experience by starting at the bottom, and working their way upwards. It is certainly unpleasant being at the bottom, but the skills gained instill a sense of work ethic and the beginning of expertise. Unfortunately, these jobs are the easiest to replace with standalone AI or with expert-assisted AI, thereby creating a skills gap. The reality is that the “human in the loop” needs to be an expert in their area of speciality in order to be able to correct the AI. By replacing entry-level workers with expert-assisted AI, we disrupt the pipeline to create more expert-level humans.
There is no shortcut to becoming an expert aside from progressing through the tedious knowledge levels and performing the right kind of practice. Repetitive practice at the same level of difficulty does not build expertise; only overcoming progressive levels of difficulty leads to expertise. The formulation of a practice plan to build expertise is a task suited to AI.
AI also helps level the playing field, typically providing a bigger boost to the low- or average-performing person, and closing the gap with the highest-performing people in the same role. One other side-effect of working with AI is that people may need to narrow their focus, and become specialists instead of generalists.
Chapter 9: AI as our future
The world with AI is vastly different from the world prior to AI. AI has the ability to simulate sentience. AI has the creative ability to generate new art, whether written or visual. AI allows people the ability to mimic other people, both current and historical. AI allows people the ability to do bad things like phishing or scamming.
So, what does the future of AI look like? There are four possible scenarios:
Scenario 1: As good as it gets
What if AI stops making huge improvements? What if the AIs of today really are the best AIs that will ever exist? All of the above capabilities would continue to exist, but is this scenario really possible? While unlikely, there may be a few causes that lead to it.
Scenario 2: Slow growth
What if AI growth slows from exponential to a more moderate 10-20% per year? All of the above capabilities from scenario 1 would continue to exist, but would get magnified over time. More realistic new artwork, more relatable AI personalities, more convincing scams. Work also continues to transform, but at a manageable pace.
Scenario 3: Exponential growth
What if AI growth continues at the exponential trajectory it’s been on over the recent past, without reaching AGI or superintelligence? All of the above capabilities from scenario 2 would continue to exist, but it’s difficult to picture what else the future would look like (human brains have difficulty with extrapolating exponential growth curves). All risks are magnified. Every computer system is susceptible to AI hacking, such that everyday humans need to develop their own AI countermeasures, leading to an arms race in AI technology. AI personalities get so good that humans prefer to interact with them instead of with other humans, thereby leading to less lonely, but more isolated people. As AI takes over more and more human work, educational institutions will cease to exist in their current state, and governments will need to start considering universal basic income. It’s not all doom and gloom, as the trend over the past century has been towards fewer weekly and lifetime working hours.
Scenario 4: The machine god
What if we achieve AGI or superintelligence? Human intelligence becomes just another marker in the intelligence spectrum. It would signal the end of human dominance on this planet. Dystopian science fiction would tell us that the scenario becomes a struggle between the machines and the humans (Terminator, Matrix, etc.), as the AI views humans as a threat, an inconvenience, a burden, or a source for valuable molecules. This scenario stresses the importance of alignment, from Chapter 2. If the superintelligent AI is aligned with human interests, the possibilities for humanity are much more positive - fewer diseases, greater longevity, etc.
Epilogue: AI as us
As alien as AIs are, they’re also deeply human. They’re trained on the collective works of humanity. With all the possibilities of what AI can be, it’s appropriate to view AI as a mirror: they reflect back our values, culture, and biases. And we need to stay aware of AI capabilities to steer the future of humanity in the right direction.