Web Search from OpenAI - 5 Things You May Not Know

Introduction

What better way to introduce the web search capability in ChatGPT than to use that capability to get the summary? Previously this was not possible because we are accessing information that happened after the pre-training date of GPT-4o. Now we can access that information and use it along side the information the model has been trained on.

On October 31, 2024, OpenAI announced the integration of real-time web search capabilities into ChatGPT, transforming it into a comprehensive AI-powered search engine. This feature allows ChatGPT to access up-to-date information, such as sports scores, stock quotes, news, and weather, enhancing its ability to provide timely and relevant responses. Initially available to ChatGPT Plus and Team users, the feature will be gradually rolled out to free, enterprise, and education users in the coming weeks. OpenAI has partnered with various news organizations to ensure accurate and authoritative information sources. The search functionality is built on a fine-tuned version of the GPT-4o model and aims to offer a more conversational and interactive search experience, enabling users to ask follow-up questions and receive detailed answers with source citations.

The Verge

How to Use Web Search

There is a new icon on the prompt screen. If you click on it you will enable web search. This will enable web search as a tool that GPT-4o will use to respond to a prompt. If you do not click on it, GPT-4o may still do a web search if the prompt suggests it is needed. For example asking for today's stock prices or the weather.

As you will see, the real advantage over traditional web searches is not that this is a better way to search the web. It is that web search and generative AI are working together in an integrated manner.

An Experiment

I became interested in knowing about the first time a modern computer was used to predict a presidential election in the United States. Without web search, I get the following response from using the GPT-4o model of ChatGPT.

The first time a modern computer was used to predict the outcome of a U.S. presidential election was in 1952. CBS News collaborated with Remington Rand, the company behind the UNIVAC I computer, to forecast the result between Dwight D. Eisenhower and Adlai Stevenson.

UNIVAC predicted an overwhelming victory for Eisenhower early in the evening, but due to the wide margin it calculated, CBS doubted the accuracy and chose to report a closer race based on other traditional methods. As results came in, it became clear that the UNIVAC prediction was correct: Eisenhower won by a landslide with 55% of the popular vote and a decisive Electoral College victory.

UNIVAC's prediction was within a 1% margin of the final outcome, showing the potential of computing technology for data analysis, forecasting, and political polling.

Insight 1 : The response you get will be cited with one or more links to the source of the information.

This the response I got when I added the web search capability to my prompt. The difference so far, is that the source of each piece of information added to the response is cited and for each source I can click on the link provided and it will take me to the source of this information where there is more information. In this simple example, only one source is cited in the response (EDN) at the bottom of the response.

The first use of a modern computer to predict the outcome of a presidential election occurred on November 4, 1952, during the U.S. presidential race between Dwight D. Eisenhower and Adlai Stevenson. CBS News collaborated with Remington Rand to utilize the UNIVAC I computer for this purpose. Early in the evening, with just over 3 million votes counted, UNIVAC predicted a landslide victory for Eisenhower, assigning him 438 electoral votes to Stevenson's 93, and estimating 100-to-1 odds in favor of Eisenhower. This prediction was notably accurate, as Eisenhower ultimately secured 442 electoral votes to Stevenson's 89, closely aligning with UNIVAC's forecast.

EDN

Insight 2: Not only are the primary sources of information cited see above. In a separate sidebar all of the sources that came up in the search are cited as well. The sidebar below is only a partial list of sources used for the response.

Insight 3: Unlike Google search, the sources tend to be text only, whereas Google will often lead with images and Youtube videos. Searching for images and Youtube videos is simply a matter expressing this in your prompt.

My next prompt was as follows:

"Are there any images associated with this event?".

A screenshot of the partial response is shown below. You can click on any of the images in the response and it will

Similarly when I asked for any videos of the event that can be found on YouTube. I got the following response.

Likely, I could combined all of the above in a single prompt, but I actually prefer this approach where I have a concise summary to begin with and then I can expand my understanding of this event as needed.

Insight 4: Can I invoke analysis capabilities of the GPT-4o model to derive more information about this topic? In particular lets start with the images.

There is a great deal of more information that can be gleaned from the images returned from this web search and because we are still within the context of a conversation with GPT-4o we can go further.

For example, when I asked GPT-4o to transcribe the prediction printout with the following prompt:

"From the prediction images shown can you transcribe the contents of the printout"

The result was this:

Not only did GPT-4o transcribe the text it also provided more insight into the text. Particularly the line that reads "00 to 1" odds. Also, there is a citation again at the bottom.

Insight 5: What about people in the images ? Can I prompt for more information.

For some people, the significance of the image with Walter Cronkite in it would be lost. So I prompted the model as follows:

"Describe the 3rd image from the left in as much detail as you can. Can you identify anyone in the image?"

Finally, for to complete the research. Lets ask the following:

Who was Walter Cronkite and what was the significance of him being in the picture?

Summary

These are first impressions of the web search capabilities of ChatGPT. What is clear is that we need to learn how to prompt again when we want to leverage these capabilities. There is more experimentation to be done. I have focused on a positive outcome. There are many examples where the web search capabilities completely failed. That is for a separate article.

This is another significant advancement, not in the reasoning capabilities of generative AI but in the way we work with the capabilities we have today and in the future.