Future of search: Integrating search and prompt intent

The way people request and retrieve information is evolving. Generative AI models are maturing rapidly and shifting expectations around the way our queries or prompts are answered.

Over time, as generative models continue to scale and converge with retrieval-based search models, the way we do a search query or prompt LLMs will affect the mix of generative and human-authored content we become accustomed to receiving.

Search and prompt intent

In search, people tend to search with one of four different types of intent: informational, navigational, transactional, and exploratory.

LLM prompt intents range from creating and refining content, to addressing complex exploratory questions. Prompts need to be creatively engineered to get the best results.

Search engines and LLMs are currently quite different experiences, but over the coming years (maybe not many) they may grow increasingly close together.

With this, the differentiation between searching for existing content and prompting the generation of new content may start to blur, as audiences learn how to search and prompt interchangeably depending on their need state.

Informational intent

People with informational intent seek specific answers or knowledge.

Future search engines and LLMs may be expected to evaluate whether a user's query is optimally addressed by directing them to human-authored content, or immediately generating a comprehensive response, or maybe both. This can be seen in Google's Search Generative Experience (SGE).

Navigational and transactional intent

For navigational and transactional intent, where the goal is to reach a particular website, brand experience, or make a purchase, a traditional search response and click through might continue to be preferred.

Nonetheless, LLMs could augment this by providing personalised recommendations, tailoring descriptions, suggesting alternatives, and enabling transactions through conversational and increasingly immersive sales experiences.

Exploratory and creative intent

With exploratory and creative intent, LLMs can excel in delivering a valuable and differentiated experience.

Users aiming to research a topic without a specific resource in mind will benefit from an LLM's capacity to synthesise information from multiple datasets and generate original content, offering knowledge that might not be found in any singular document. However, access and rights to use source data is essential (more to come).

Challenges for real-time intent interpretation

The AI systems we start to use as part of everyday life are likely to become highly-proficient at interpreting the subtleties of user queries, determining the type of intent being shown, and deciding the most appropriate response format.

This process requires the advanced understanding of natural language that is rapidly developing, and the ability to analyse queries in real-time relative to real-world context.

Real-time predictive analysis and personalisation

Predictive capabilities informed by an AI's memory processing may tailor the user experience, prompting generative responses or listing curated content (effectively SERPs) based on user preference and the nature of the query.

The challenge may be the ability to acquire the comprehensive present-day context required to provide the most up-to-date information that accurately answers informational or exploratory prompts.

As of writing this post, the cutoff for ChatGPT-4 training data is April 2023. The browser function built into GPT-4 does enable access to current day information, but this has recently been scaled back due to copyright issues.

Data source integration, access and rights

Diverse data sources are essential in enabling search and generative AI systems to utilise the most relevant information, whether generating responses from recent research articles or drawing from a vast repository of human knowledge.

For generative AI models this may be a challenge in the long term. Many publishers such as HBR, Guardian, and The Economist, among others, are using disallow rules to prevent GPTbot and other generative AI crawlers from training on or browsing their data. This excludes some of the best primary research, journalism, and insights from becoming a source of training data and forming a part of a generative response.

People will still initiate information requests through queries, be they searches or prompts. The future, however, is not just about finding answers but in generating them.

As the distinction between queries becomes increasingly nuanced, sometimes the best thing for the user will be a personalised generative response, and other times, the best option will be a click through to human-authored content. This optionality moves us towards more versatile, tailored, and efficient information retrieval experiences.

It will be important in this scenario for content owners to strategically plan for the ways in which search and AI robots can access content to classify and train models, while maintaining value and copyright with the source data.

This is my own opinion not that of my employer.

Please do comment if you agree / disagree / have more thoughts.