REVIEW of ChatGPT and the Future of AI: The Deep Language Revolution by T. Sejnowski – AI-Human convergences and divergences

ChatGPT and the Future of AI: The Deep Language Revolution? emerges from the computational neuroscientist Terrence Sejnowski’s interest in Large Language Models (LLMs) like ChatGPT and their potential to reshape human productivity, creativity, learning, and cognition. The subtitle?"The Deep Language Revolution"?highlights how advances in deep learning, particularly transformer-based architectures, have enabled LLMs like ChatGPT to revolutionize human-machine interaction through their mastery of language. Sejnowksi uses the term Deep language revolution to underscore the profound societal and cultural impacts of these technologies, from transforming industries and education to raising ethical questions about their role in reshaping communication and cognition.?He makes the compelling analogy between the conceptual leaps enabled by LLMs and the discovery of the DNA double helix: AI as a technology could bridge the kind of gaps in understanding intelligence that DNA has done in biology.

Why this book is important??

This book is extremely useful and insightful for educators, technologists, and policymakers navigating AI’s integration into daily life. Because the author is an expert in computation and neuroscience, he is able to demystify the capabilities of both LLMs and brains, and he has the rare skillset to explain both fields and their intersections with rare clarity and rigor.

Terrence Sejnowski is a renowned computational neuroscientist and AI researcher who has pioneered work in computational neuroscience and neural networks. I rediscovered him 2 weeks when I listened to him as a guest on the Huberman Podcast (Nov. 19, 2024) as part of his promotion efforts for this book that was released at the end of October 2024. Sejnowski, a professor at the Salk Institute and co-founder of the Neural Information Processing Systems (NeurIPS) conference, has consistently bridged the gap between biological brains and artificial intelligence. The first time I discovered him was when I read his collaboration with Barbara Oakley to write the 2021 book entitled Uncommon sense teaching—Practical insights in brain science to help students learn and develop a hugely popular course on Coursera called “Learning how to learn”.?Sejnowksi is therefore uniquely positioned to offer insights in to computation, neuroscience, learning, and education.

Some unique features of the book??

Extensive Use of ChatGPT: Sejnowski frequently integrates ChatGPT sections as a simplifier of complex technical concepts and also a chapter summarizer. This not only helps reinforce key points but also makes the book an interactive demonstration of LLM utility.?In fact, he playfully acknowledges ChatGPT as a co-author.

Interdisciplinary Analysis: The book bridges neuroscience, cognitive science, and AI computer science and in so doing, explores the technical and philosophical overlaps between human cognition and artificial intelligence.?

Focus on Prompt Engineering: The book uniquely highlights the importance of crafting detailed prompts. This is a critical skill to maximize LLM efficiency and parallels the evolution of programming in earlier technological revolutions.?

Summary of the Three Parts?

The book contains 14 chapters over 3 parts: Living with large language models,? Transformers, and Back to the future. The following lists some key insights from each part.?

Part I: Living with Large Language Models?

1. Impact on Professions: Sejnowski underscores how Large Language Models (LLMs) like ChatGPT are rapidly transforming a wide range of industries by automating tasks, enhancing creativity, and enabling data-driven decision-making. For instance, within just two months of its release, ChatGPT reached 100 million users, a record-breaking milestone far surpassing platforms like Google and Facebook. This rapid adoption is reflected in specific examples: ChatGPT has already written 7–17% of peer reviews for leading AI conferences, highlighting its ability to contribute at a high intellectual level. In healthcare, LLMs are being used to assist doctors in empathizing with patients, providing diagnostic support, and even summarizing consultations. In journalism, they help reporters craft articles under tight deadlines, while in education, they enable teachers to automate lesson plans and grading. These advances can allow professionals to shift their focus from mundane, repetitive tasks to strategic and creative problem-solving ones. LLMs can augment rather than replace human capabilities.?
2. Prompt Engineering: Sejnowski emphasizes that prompts are the key to unlocking the potential of large language models (LLMs) because they determine the quality, relevance, and accuracy of the AI’s responses. Prompting serves as both a?mirror?and a form of?priming: it reflects the user’s intent, knowledge, and biases while shaping the AI’s outputs. LLMs mirror the intelligence and specificity of their users, with the quality of the prompt directly influencing the quality of the results—what Sejnowski likens to a "reverse Turing test." Additionally,?priming?through carefully crafted prompts provides the necessary context and framing to guide the AI toward accurate, relevant, and creative responses, making prompt engineering an essential skill for effective AI interaction.?
3. Ethical Challenges: The rapid deployment of LLMs also introduces significant ethical concerns, like misinformation, bias, and job displacement. Sejnowski highlights that LLMs can "hallucinate" plausible but entirely fabricated information, as demonstrated when ChatGPT generates completely fake sources of information and links—a vivid reminder of the critical need for users to verify AI outputs. Also, concerns about bias in training data persist, with potential discriminatory effects in sensitive areas like hiring and law enforcement. On a broader societal level, LLMs are accelerating job displacement in roles dominated by repetitive or administrative tasks. However, Sejnowski suggests that AI can create opportunities for meaningful reskilling initiatives to help workers transition into roles requiring uniquely human qualities like emotional intelligence and strategic thinking.??

Part II: Transformers?

Sejnowski explains transformers as the deep learning architecture central to LLMs. They enable models to process and generate language by attending to relationships between words in a sequence, regardless of their position. This architecture, which uses self-attention mechanisms, allows LLMs to scale effectively, generalize across tasks, and exhibit emergent capabilities like reasoning and creative problem-solving (Vaswani et al., 2017).?

1. Emergent Abilities: Transformers, the architecture powering LLMs, demonstrate emergent abilities that were neither explicitly programmed nor fully anticipated during development. For instance, GPT-4, with over a trillion parameters, has exhibited capabilities like passing advanced college-level exams and generating sophisticated creative outputs, indicating that LLMs can perform multi-step reasoning tasks previously thought to require human-like cognitive depth. This is like a phase transition in physics—where small increases in model complexity yield significant leaps in functionality. This suggests that the architecture's scalability is a key driver of these surprising abilities.?
2. Scalability: The transformer architecture scales efficiently with increased data and computational power, which allows LLMs to achieve unprecedented levels of performance across diverse tasks. GPT-3's training involved 175 billion parameters and required an estimated $12 million in computational costs, a figure dwarfed by the even greater resources used for GPT-4. This scalability has profound implications, as it enables LLMs to generalize across tasks ranging from programming to medical diagnoses, but it also raises concerns about accessibility, as only companies with vast resources can afford such investments?
3. High Computational Costs: The computational infrastructure required for training and running LLMs has reached staggering levels—this makes energy efficiency and environmental impact critical concerns. Global AI-related data center costs, for example, are expected to reach $422.55 billion by 2029 and their energy consumption is projected to reach 3-4% of global electricity use by 2030.??

Part III: Back to the Future?

1. Learning from Biology: Sejnowski explores how insights from biology, particularly human and animal learning, could guide the next generation of AI systems. He emphasizes that human cognition develops through embodied interactions with the physical and social world—something LLMs currently lack. For example, integrating sensorimotor experiences, akin to how children learn through touch and movement, could help AI systems achieve more grounded and contextual understanding. This could bridge the gap between abstract reasoning and real-world applications.?
2. Artificial General Autonomy (AGA): Artificial General Autonomy (AGA) represents a potential evolution of AI beyond language tasks, where systems would not only interpret but also act on information autonomously. Achieving AGA would require incorporating principles of human autonomy, such as goal-setting, adaptive learning, and decision-making based on limited information. He speculates that coupling LLMs with physical robotics or environmental sensors might be the next step toward systems capable of functioning in dynamic, real-world environments.?
3. Simplified Models: Sejnowski advocates for the development of smaller, simplified AI models to deepen our understanding of LLM behavior (which is currently a mysterious black box) and their potential implications. He draws a parallel to breakthroughs in biology, such as the study of simple neural systems like those in sea slugs, which yielded critical insights into human neuroscience. Similarly, lightweight, interpretable AI systems could help researchers analyze how LLMs learn, generalize, and adapt, potentially accelerating innovation while mitigating risks associated with large-scale, opaque systems.?

LLMs and Human Cognition: Similarities and Differences?

Similarities

Both human brains and generative AI systems like Large Language Models (LLMs) share the ability to generalize across complex inputs and create novel outputs. For example, LLMs learn patterns from vast datasets in a manner akin to humans recognizing linguistic structures through exposure. A striking parallel lies in their reliance on high-dimensional processing: LLMs operate in vast parameter spaces, with GPT-4 utilizing over a trillion parameters, comparable to the 86 billion neurons and trillions of synaptic connections in the human brain. Furthermore, LLMs act as mirrors to reflect the knowledge, biases, and intent of their users—an analogy that aligns with how humans adapt their social cognition based on environmental cues. This mirror effect is illustrated through the "reverse Turing test," where the intelligence of LLM outputs often reflects the sophistication of the user’s prompts.?

There are compelling parallels between human language production and how LLMs generate text, which suggests shared underlying mechanisms despite vastly different substrates. Both systems rely on probabilistic models: humans subconsciously predict and select words based on linguistic context, while LLMs calculate probabilities using vast training data to determine the most likely next word. For example, LLMs like GPT-4 excel at producing coherent, contextually appropriate language by leveraging transformer architectures, which mimic human brain processes like working memory and attention to prioritize relevant information in a sequence (I went into fine detail on this here).?

Sejnowski also notes that both humans and LLMs exhibit creativity through recombination of learned elements. In humans, this ability comes in the form of metaphors, analogies, and novel expressions, while LLMs achieve similar results by blending patterns from diverse training data. A good example is ChatGPT’s ability to produce poetry and complex narratives, which reflects a capacity for structured, meaningful output that closely mirrors human linguistic creativity. However, while humans produce language grounded in embodied experience and social context, LLMs operate without true understanding, relying solely on statistical correlations?

Differences?

Despite these similarities, there are fundamental differences that underscore the limitations of LLMs compared to human cognition. Unlike humans, who learn through embodied experiences involving sensory input and emotional/social context, LLMs are "brains in a vat," trained on static text without real-world interaction. This lack of embodiment restricts their understanding of concepts tied to physical experiences. Additionally, human learning is multimodal and develops over years through trial, error, and adaptation, while LLMs rely on massive computational training upfront. For instance, training GPT-3 required approximately $12 million in computational resources, highlighting a resource-intensive process that contrasts with the efficiency of human development. Lastly, humans possess metacognition—the ability to think about their own thinking—whereas LLMs simulate understanding but lack genuine self-awareness or introspection, limiting their autonomy and adaptability.?

Takeaways for Education

1.?Personalized Learning Companions?

• AI tools like ChatGPT can serve as 24/7 personalized tutors, capable of adapting to individual students' needs in real-time. Beyond answering questions or explaining concepts, these AI companions can simulate Socratic dialogues, encouraging students to think critically and engage deeply with material. For instance, Sejnowski envisions AI-driven systems creating tailored learning paths, allowing students to explore subjects at their own pace, with interactive prompts that adapt dynamically based on their progress and interests.??

2.?Immersive Storytelling for Language and History?

• Generative AI can craft immersive narratives, blending storytelling with educational content to teach complex topics in engaging ways. For example, students learning about ancient history could interact with AI-generated "characters" like historical figures or citizens of a particular era, who explain events from a first-person perspective. This innovative use of AI storytelling can foster deeper understanding and emotional connections to learning materials, transforming dry facts into compelling experiences.?

?

3.?AI-Powered Peer Review and Feedback?

• AI can revolutionize peer learning by facilitating collaborative projects where students use LLMs for drafting, editing, and critiquing. For example, an AI might help a student writer by offering constructive feedback, highlighting areas for improvement, and suggesting revisions based on the student's style and goals. This process mimics the mentorship of skilled educators, enabling students to improve their writing, research, and argumentation skills in iterative, feedback-driven ways.?

Takeaways for the Workplace

?AI as a Real-Time Communication Coach?

• AI can act as a communication coach during meetings and provide real-time suggestions to improve clarity, tone, and conciseness. For example, LLMs can adapt language for cultural nuances, which could help multinational teams navigate linguistic and cultural barriers. This functionality fosters better understanding, reduces miscommunication, and ensures more inclusive workplace interactions.?

2.?Dynamic Idea Generation and Problem Solving?

• LLMs excel at brainstorming and offering innovative solutions by analyzing vast datasets and simulating creative thinking. Teams can use AI to generate multiple strategic options for projects and then draw on insights from various industries and historical precedents. AI-driven creativity can complement human ingenuity and enable teams to make more informed and imaginative decisions.?

3.?Enhanced Employee Onboarding and Training?

• AI tools can personalize onboarding experiences to create interactive and engaging learning modules for new employees. Imagine an AI-generated onboarding assistant that walks employees through company policies, answers common questions, and provides tailored training content. Such applications can be ways to accelerate integration and ensure that employees receive consistent, high-quality support across organizations.?

?