What are the benefits and drawbacks of using pseudo relevance feedback in web search?

1 How relevance feedback works

Relevance feedback works by iteratively modifying the query based on the user's feedback on the retrieved documents. The user can mark the documents as relevant or irrelevant, and the system can use this information to adjust the query weights, add or remove query terms, or use other criteria to improve the query representation. The process can be repeated until the user is satisfied with the results or gives up.

2 How pseudo relevance feedback works

Pseudo relevance feedback works by assuming that the user is satisfied with the top-ranked documents returned by the initial query and using them to expand or modify the query without the user's involvement. The system can use various techniques to extract terms or concepts from the relevant documents and add them to the query, or to reweight the query terms based on their occurrence in the relevant documents. The process can be done once or multiple times, depending on the system's design.

3 Benefits of pseudo relevance feedback

Pseudo relevance feedback offers several advantages compared to relevance feedback, such as not requiring the user's participation, overcoming the vocabulary mismatch problem, exploiting latent semantic relations between terms and documents, and handling long and complex queries better than relevance feedback. This can save time and effort for the user, reduce the cognitive load, improve query coverage and diversity, and prevent noise and overfitting.

4 Drawbacks of pseudo relevance feedback

Pseudo relevance feedback has some drawbacks when compared to relevance feedback. It assumes that the top-ranked documents are relevant, which may not be true in certain situations, such as when the initial query is vague or ambiguous. Additionally, it can introduce irrelevant or redundant terms to the query, thus decreasing its accuracy and specificity. Furthermore, it can amplify the bias or error of the initial ranking algorithm, potentially leading to lower quality results. Finally, it can reduce the user's control and transparency over the query modification process, thus affecting their trust and satisfaction.

5 When to use pseudo relevance feedback

Pseudo relevance feedback is not a one-size-fits-all solution for improving web search performance. Factors such as the user's query intent and information need, preference and behavior, the system's ranking algorithm and feedback mechanism, and its evaluation metrics and criteria can all affect the effectiveness of this technique. It is therefore important to understand the challenges of using pseudo relevance feedback and to tailor it to different scenarios and contexts.