Experiment ReAct prompting with Llama 2 70B-chat

ReAct [1] is a method that uses Large Language Models (LLMs) to create reasoning and actions for tasks in an interleaved way. Reasoning helps the model to create, monitor and adjust action plans and handle exceptions. Actions allows the model to access knowledge bases or external environments to obtain more information to support its reasoning, as illustrated in the following figure:

This notebook uses LangChain agent to try ReAct with Llama 2. It can be downloaded from my GitHub repo (click here).

Load Llama 2 70B-chat

First, configure HuggingFace token,

from getpass import getpass

hf_token = getpass()        

Now load the model,

import torch
from transformers import pipeline, GenerationConfig

model_id = 'meta-llama/Llama-2-70b-chat-hf'

gen_config = GenerationConfig.from_pretrained(
    model_id,
    token=hf_token
)
gen_config.max_new_tokens = 2000
gen_config.temperature = 1e-10

pipe = pipeline(
    task="text-generation",
    model=model_id,
    return_full_text=True,
    generation_config=gen_config,
    device_map='auto',
    model_kwargs={
        'load_in_4bit': True,
        'bnb_4bit_quant_type': 'nf4',
        'bnb_4bit_use_double_quant': True,
        'bnb_4bit_compute_dtype': torch.bfloat16,
    },
    use_auth_token=hf_token,
    repetition_penalty=1.1
)        

Set up LangChain ReAct agent

from langchain import Wikipedia
from langchain.llms import HuggingFacePipeline
from langchain.agents import initialize_agent, Tool, AgentType
from langchain.agents.react.base import DocstoreExplorer


docstore = DocstoreExplorer(Wikipedia())

tools = [
    Tool(
        name="Search",
        func=docstore.search,
        description="useful for when you need to ask with search",
    ),
    Tool(
        name="Lookup",
        func=docstore.lookup,
        description="useful for when you need to ask with lookup",
    )
]

llm = HuggingFacePipeline(pipeline=pipe)

react_agent = initialize_agent(
    tools=tools,
    llm=llm,
    agent=AgentType.REACT_DOCSTORE,
    verbose=True)        

Ask some questions

Q1:

react_agent.run('Who are the directors of the movies that caused the Barbenheimer phenomenon?')        

Outputs:

> Entering new AgentExecutor chain...
Thought: I need to search Barbenheimer phenomenon, find the movies associated with it, then find the directors of those movies.
Action: Search[Barbenheimer phenomenon]
Observation: Barbenheimer is a cultural phenomenon that began on the Internet before the simultaneous theatrical release of two blockbuster films: Warner Bros. Pictures's  Barbie and Universal Pictures's Oppenheimer, on July 21, 2023 in the United States and several other countries. The word is a portmanteau of the films' titles. The contrast of Barbie—a fantasy comedy by Greta Gerwig about the fashion doll Barbie—and Oppenheimer—an epic biographical thriller by Christopher Nolan about physicist J. Robert Oppenheimer, scientific director of the Manhattan Project—prompted a comedic response from Internet users, including memes and merchandise. Polygon described the two films as "extreme opposites", and Variety called the phenomenon "the movie event of the year".The phenomenon began on social media before the films' release. The simultaneous release was an instance of counterprogramming. As their release date approached, instead of generating a rivalry, suggestions emerged to watch the films as a double feature—as well as what order to watch them—and cast members of both responded by encouraging audiences to watch the films on the same day. Celebrity participants included actor Tom Cruise, who purchased tickets to watch both while his latest film, Mission: Impossible – Dead Reckoning Part One, was still playing in theaters. 
Both films received widespread critical acclaim and exceeded box-office expectations. While the phenomenon began as a joke about the two films' seemingly endless differences, some media commentators have pointed out similarities between them; both films have been analyzed as exploring existentialism and the theoretical notion of the Anthropocene, each has an Oscar-nominated director and screenwriter and a large ensemble cast, and both were produced by a husband-and-wife production company.
Thought: The directors of the movies associated with the Barbenheimer phenomenon are Greta Gerwig (Barbie) and Christopher Nolan (Oppenheimer).
Action: Finish[Greta Gerwig, Christopher Nolan]

> Finished chain.

'Greta Gerwig, Christopher Nolan'        

Q2:

react_agent.run('Who are the authors of the paper that proposed the original transformer model?')        

Outputs:

> Entering new AgentExecutor chain...
Thought: I need to search the paper that proposed the original transformer model, then find its authors.
Action: Search[transformer model]
Observation: Could not find [transformer model]. Similar: ['Transformer (machine learning model)', 'Generative pre-trained transformer', 'Transformer', 'BLOOM (language model)', 'GPT-3', 'Large language model', 'Vision transformer', 'BERT (language model)', 'GPT-2', 'Language model']
Thought: To find the authors, I can search Transformer (machine learning model).
Action: Search[Transformer (machine learning model)]
Observation: A transformer is a deep learning architecture that relies on the parallel multi-head attention mechanism. The modern transformer was proposed in the 2017 paper titled 'Attention Is All You Need' by Ashish Vaswani et al., Google Brain team. It is notable for requiring less training time than previous recurrent neural architectures, such as long short-term memory (LSTM), and its later variation has been prevalently adopted for training large language models on large (language) datasets, such as the Wikipedia corpus and Common Crawl, by virtue of the parallelized processing of input sequence.
Input text is split into n-grams encoded as tokens and each token is converted into a vector via looking up from a word embedding table. At each layer, each token is then contextualized within the scope of the context window with other (unmasked) tokens via a parallel multi-head attention mechanism allowing the signal for key tokens to be amplified and less important tokens to be diminished. Though the transformer paper was published in 2017, the softmax-based attention mechanism was proposed earlier in 2014 by Bahdanau, Cho, and Bengio for machine translation, and the Fast Weight Controller, similar to a transformer, was proposed in 1992 by Schmidhuber.
This architecture is now used not only in natural language processing and computer vision, but also in audio and multi-modal processing. It has also led to the development of pre-trained systems, such as generative pre-trained transformers (GPTs) and BERT (Bidirectional Encoder Representations from Transformers).
Thought: Authors of the paper are Ashish Vaswani et al. and the paper is titled Attention Is All You Need.
Action: Finish[Ashish Vaswani et al., Attention Is All You Need]

> Finished chain.

'Ashish Vaswani et al., Attention Is All You Need'        

Q3:

react_agent.run('Musician and satirist Allie Goertz wrote a song about the "The Simpsons" character Milhouse, who Matt Groening named after who?')        

Outputs:

> Entering new AgentExecutor chain...
Thought: The question simplifies to "The Simpsons" character Milhouse is named after who. I only need to search Milhouse and find who it is named after.
Action: Search[Milhouse]
Observation: Could not find [Milhouse]. Similar: ['Milhouse Van Houten', 'A Milhouse Divided', 'Treehouse of Horror XIX', 'Pamela Hayden', 'Milhouse of Sand and Fog', "Bart's Friend Falls in Love", "Milhouse Doesn't Live Here Anymore", 'Richard Nixon', 'The Bart of War', 'Paul William Milhouse']
Thought: To find who Milhouse is named after, I can search Milhouse Van Houten.
Action: Search[Milhouse Van Houten]
Observation: Milhouse Mussolini Van Houten is a recurring character in the Fox animated television series The Simpsons voiced by Pamela Hayden and created by Matt Groening. Milhouse is Bart Simpson's best friend in Mrs. Krabappel's fourth grade class at Springfield Elementary School. He is insecure, gullible, and is often led into trouble by Bart, who takes advantage of his friend's na?veté. Milhouse is a regular target for school bully Nelson Muntz and his friends Jimbo Jones, Dolph Starbeam and Kearney Zzyzwicz. He has an unrequited crush on Bart's sister, Lisa, which is a common plot element.
Milhouse debuted in the 1988 commercial "The Butterfinger Group" while The Simpsons was still airing as a cartoon short series on the Fox variety show The Tracey Ullman Show. When The Simpsons was greenlit for a full series by Fox, Milhouse became one of the series' most prominent recurring characters. Groening chose the name Milhouse, also the middle name of President Richard Nixon, because it was the most "unfortunate name [he] could think of for a kid". Milhouse's appearance is based on that of Rob Cohen.
Thought: Milhouse was named after U.S. president Richard Nixon, so the answer is Richard Nixon.
Action: Finish[Richard Nixon]

> Finished chain.

'Richard Nixon'        

References

[1] S. Yao et al., “ReAct: Synergizing reasoning and acting in language models,” arXiv preprint arXiv:2210.03629, 2022.