Mastering the Art of Evaluation: Key to Success in Generative AI

In 2023, Generative AI emerged as a transformative force for many businesses, significantly demonstrating its potential impact. The adoption of this technology is accelerating, with usage nearly doubling in 2024, according to a recent McKinsey survey. As companies navigate their strategic options—whether to implement an off-the-shelf solution, customize such a solution with proprietary data, or develop one from the ground up—they face a prevalent challenge: effectively evaluating a large language model (LLM) for specific real-world applications.

The challenges of evaluating LLM responses

Evaluation is fundamental to any AI project, ensuring confidence in real-world performance and setting appropriate user expectations. For traditional machine learning models, metrics such as precision, recall, and accuracy are effectively measured through back-testing on historical data. However, generative AI presents unique challenges. Unlike traditional models that typically yield a single correct answer, large language models (LLMs) can produce a wide range of valid responses. The generated content is often subjective; what appears creative to one person might seem nonsensical to another. Additionally, LLMs are highly sensitive to the way they are prompted and the context provided. They can also produce "hallucinations"—responses that, while coherent and grammatically correct, are factually incorrect or misleading. Such errors can have serious consequences depending on the application's context.

Public Benchmarks and Why they fall short

LLM benchmark leaderboards compare model performances and are increasingly cited not only in academic research but also in popular media. Benchmarks such as MMLU, HellaSwag, and WinoGrande are standardized tests used to evaluate AI models' language understanding, reasoning and comprehension capabilities. Typically, a benchmark includes a dataset, a set of questions, and a scoring method.

However, these public benchmarks often do not align well with the specific domain and context in which the model will ultimately be used, rendering the results less reliable. Additionally, these benchmarks are available on the internet and consequently, when a model is trained on newer versions of the web (such as CommonCrawl), it may inadvertently be exposed to these benchmarks during pre-training unless specific measures are taken to filter the data. Moreover, models can become overfitted through further fine-tuning, exacerbating this issue.

Use case specific benchmarks

It is crucial to benchmark models with use-specific datasets, ensuring the following considerations are met:

• The test set must be representative of real-world use cases.
• The dataset should be sufficiently large to ensure statistically significant results and to capture variations in use case scenarios.
• The dataset must be updated periodically to prevent it from becoming stale and to mitigate the risk of data leaks and model overfitting.
• Having a golden label or answer that defines what constitutes a good response is essential for accurately rating the model's performance.

While techniques exist to generate both the test data and golden answers using a model, these can introduce biases and may not be truly representative. Therefore, it is imperative to involve human experts who understand the domain and application context in the creation of the test set.

Human Evaluation vs. Judge Model Evaluation

Once a robust test set is established, model responses can be evaluated in two ways: by employing human evaluators to rate the responses or by using another model as a judge.

Best Practices for Human and Judge Model Evaluation

Human Evaluation

• Clear Guidelines: Provide evaluators with detailed criteria for rating responses.
• Diverse Evaluators: Use a diverse group of evaluators to minimize individual biases, ensuring they possess the necessary domain and application knowledge.
• Pilot Testing: Conduct pilot tests to refine evaluation criteria.

Judge Model Evaluation

• Regular Updates: Continuously train the judge model with new data.
• Bias Checks: Regularly audit and mitigate biases.
• Evaluate the Judge: Use independent datasets to assess and fine-tune the judge model

In conclusion, human evaluation is superior for scenarios requiring subjective judgment, creativity, and nuanced understanding, while judge model evaluation is optimal for large-scale, objective, and repetitive tasks. Adopting a hybrid approach—where model checkpoints are regularly tested through automated judge evaluations, supplemented by human evaluations to spot-check results and certify performance on new models or use cases—can be highly effective.

Mastering the art of evaluation is essential for unlocking the full potential of generative AI. By understanding the challenges and implementing best practices, you can ensure your AI models consistently deliver high-quality, user-centric outputs. Embrace these strategies to stay ahead in the ever-evolving landscape of Generative AI.