22-1-2 Expanding LLMs with Search Engines (SerpAPI, Langchain and Perplexity)
Large Language Models (LLMs), while rich in web information from their pre-training phase, often grapple with challenges like hallucinations and outdated content, compromising their reliability in knowledge retrieval. This hallucination trait, intrinsic to autoregressive models, serves as a double-edged sword; its inventive nature in predicting the next token can be both advantageous and problematic.
In a pursuit akin to using Google for searching regex patterns or debugging code, there's a growing trend to empower LLMs with search engine capabilities, enabling them to deliver responses enriched with accurate and current details. Prominent in this arena are applications like Bing AI, Perplexity, ChatGPT integrated with Bing, and Neeva AI, each marking a significant stride in this direction.
The practice of harnessing top search engine results to guide LLM responses doesn't entirely eliminate hallucinations but significantly bolsters the output, making it more timely, accurate, and contextually relevant
We explore three methods:
There are other methods naturally some more complex than others.
We start with LLMs and SerpAPI, a real-time API that provides access to Google search results. It handles proxies, solves captchas, and parses all rich structured data, mimicking what a human user would do.
Let's register an account to obtain the keys: SerpApi: Google Search API
The free tier limits 100 searches per month. Sufficient for our tutorial.
Once logged in, copy the private secret keys somewhere safe.
pip install serpapi
import serpapi
client = serpapi.Client(api_key="Your Private Key")
results = client.search({
'engine': 'google',
'q': 'coffee',
'location': 'Korea',
'hl': 'en',
'gl': 'us',
'num': 10 ## limit results by 10
})
2. Import the library. Above shows a basic client call to Google search engine for the term "coffee"
3. Using the above, let's combine the results with a LLM like OpenAI GPT-3.5-turbo.
Save the secret keys to variables and pip install openai
serpapi_key = "Your SerpApi Key"
openai_key = "Your OpenAI API Key"
!pip install openai
We will need to update the query to serpAPI, which takes in the secret key and searches results using Google.
def fetch_search_results(query):
client = serpapi.Client(api_key=serpapi_key)
results = client.search({
'engine': 'google',
'q': query,
'location': 'Korea',
'hl': 'en',
'gl': 'us',
'num': 10
})
return results
The process_and_summarize function sets up a conversation with the OpenAI GPT-3.5 Turbo model, instructs it to summarise the search results provided in the results variable, and then prints the generated summary.
Note: If we do not parse the top n search results, we will encounter a token limit error.
client = OpenAI(api_key=OPENAI_API_KEY)
def process_and_summarize(results):
top_results = results['organic_results'][:3] # Extract the top 3 results
# Create a context string from the top results
context = "\n".join([result['title'] + ". " + result['snippet'] for result in top_results])
response = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": f"Summarise the following search results:\n{context}"},
],
)
summary = response.choices[0].message.content
print(summary)
if __name__ == "__main__":
search_query = input("Enter a search query: ")
search_results = fetch_search_results(search_query)
process_and_summarize(search_results)
Here the app initiates the interaction with the user, collects their search query, retrieves search results, and then summarises those results using GPT-3.5-turbo, providing the user with a summary of the search query they entered.
Upon searching for: "EU AI regulation"
The EU AI Act is the first comprehensive AI law that regulates the use of artificial intelligence in the European Union. It aims to establish an open policy dialogue on AI through the European AI Alliance. The AI Act serves as a legal framework governing the sale and use of AI in the EU, with the goal of ensuring proper use of AI technology
Note that this search engine based LLM application is limited. The system prompt defines what the app should accomplish: summarise n search results. Ideally, we want to extend the app functionalities by adjusting the system prompts, adding sources, links for greater utility and transparency. Also context-driven interactions between the user and the app to navigate the search results.
Let's explore how to achieve a similar effect using Langchain in far less code and headache. Start off with installing the library.
pip install langchain
from langchain.utilities import SerpAPIWrapper
Import the SerpAPIWrapper which conventionality abstracts away the finer details.
领英推荐
params = {
"engine": "google",
"gl": "us",
"hl": "en",
}
search = SerpAPIWrapper(params=params)
search.run("What is the EU AI Act?")
Langchain also allows SerpAPI to be loaded as a Tool for Langhcain Agent to use as a reference for response
Install the following library and save the tokens to an Environment Variable:
pip install google-search-results
import os
os.environ["SERPAPI_API_KEY"] = SERPAPI_KEY ## saved earlier
os.environ["OPEN_API_KEY"] = OPENAI_API_KEY ## saved earlier
from langchain.agents import load_tools, initialize_agent
from langchain.llms import OpenAI
llm = OpenAI(temperature=0.9)
tools = load_tools(["serpapi"], llm=llm)
agent = initialize_agent(tools, llm, agent="zero-shot-react-description", verbose=True)
agent.run("What documentations do AI companies have to provide under EU AI Act?")
The final answer is that under the EU AI Act, high-risk AI companies are required to provide technical documentation, automated record-keeping, transparency to end-users, and human oversight of the design, development, and use of the AI system.
If you plan to develop with Langchain's other functionalities, then Langchain's SerpAPI wrapper or tools such as DuckDuckGo and Bing Search are great to extend your custom LLM application with web search capabilities.
To wrap up the topic on Web-search augmented LLM output, we cover Perplexity Ai's API offerings. Perplexity AI offers an online API known as pplx-api, which allows developers to access and integrate its language models into their applications. Models such as pplx-7b-online and pplx-70b-online utilize Perplexity's own search, indexing, and crawling infrastructure to provide accurate and current information.
Note: Perplexity is the closest thing to a Google search replacement personally.
2. Sign Up and Register Payment Details (Getting Started with pplx-api (perplexity.ai))
3. Right bottom, select the Gear Button
4. In Settings, Select the API Tab and Generate the API Keys
The available models via API are listed here: Supported Models (perplexity.ai)
To call the API the following is a template given we want to use pplx-7b-online:
pip install requests
import requests
url = "https://api.perplexity.ai/chat/completions"
payload = {
"model": "pplx-7b-online",
"messages": [
{
"role": "system",
"content": "Be precise and concise."
},
{
"role": "user",
"content": "What is the EU AI Act and What AI does it regulate and how?"
}
]
}
headers = {
"accept": "application/json",
"content-type": "application/json",
"authorization": "Bearer YOUR_TOKEN_KEY_GOES_HERE"
}
response = requests.post(url, json=payload, headers=headers)
print(response.text)
There are unofficial Python Perplexity libraries but for the time being we need to rely on using the official endpoint.
The EU AI Act is the world's first comprehensive AI law, aiming to regulate the use of artificial intelligence (AI) in the EU to ensure better conditions for AI-driven benefits, such as improved healthcare, safer and cleaner transport, more efficient manufacturing, and cheaper and more sustainable energy. The regulation focuses on ensuring that fundamental rights, democracy, the rule of law, and environmental sustainability are protected from high-risk AI while boosting innovation and making Europe a leader in the field.\n\nThe AI Act sets rules for large, powerful AI models, ensuring they do not present systemic risks to the Union and offers strong safeguards for various sectors of the economy. The regulation establishes obligations for AI based on its potential risks and level of impact. Some key aspects of the AI Act include:\n\n- Banned applications: The AI Act bans cognitive behavioral manipulation, untargeted scrapping of facial images from the internet or CCTV footage, and social engineering.\n\n- Transparency and traceability: AI systems should be safe, transparent, traceable, non-discriminatory, and environmentally friendly. Users should be made aware when interacting with AI, including when they are interacting with AI systems that generate or manipulate image, audio, or video content, such as deepfakes.\n\n- Systemic risk assessment: High-impact foundation models with systemic risk will have to conduct model evaluations, assess and mitigate systemic risks, and conduct adversarial testing.\n\nThe AI Act still needs to go through some final revisions before being fully implemented. It will affect not only major AI developers like Google and Meta but also smaller companies and governments that use AI in areas such as education, healthcare, banking, criminal justice, and the allocation of public benefits. The law will require employers to hire new experts to ensure compliance with the AI Act, and legal challenges are likely as companies test the novel rules in court
We've delved into web-search-enabled Language Models (LLMs) and explored the diverse approaches available for connecting web information with these models.
SerpAPI presents a straightforward solution, allowing us to retrieve comprehensive results from a given query. The data obtained from the API response can be effectively filtered and combined with the user's query to provide nuanced and context-rich answers. However, this flexibility also comes with the responsibility of managing and storing precisely the right amount of information.
The Langhchain SerpAPI wrapper streamlines the process of working with SerpAPI while opening up possibilities for incorporating various tools within the framework of Agents. Within the expanding Langchain ecosystem, configuring and extending SerpAPI alongside other search engines and tools becomes a seamless endeavor.
Moving forward, we explored Perplexity, which introduces a web-first language model akin to 7b-online. It offers a more straightforward integration compared to the combination of SerpAPI and LLMs and lends itself well to integration into existing applications. Leveraging OpenAI APIs to parse the output from Perplexity's online models offers a promising path to harnessing the strengths of both approaches.
Before delving deeper into the synergy between LLMs and retrieval from internal sources, let's take a moment to examine the strategies for storing output to facilitate reuse and enhance contextual memory.
Great article! Thank you for mentioning us!