Giskard: Red Teaming Against AI Models

Disclaimer: The views and opinions expressed in this article are solely my own and do not reflect those of my current or previous employers.

In this article we will explore red teaming on AI. One of the tool I found useful is Giskard. Giskard is a tool that helps developers test, debug, and improve their AI models. Whether you're building a chatbot, a recommendation system, or any other AI application, Giskard can help make sure it works as expected.

What is Giskard?

Giskard is a platform designed to ensure AI models perform well. It helps:

• Test models for correctness and reliability.
• Spot problems like biases or errors in the outputs.
• Suggest improvements to make the models better.

If you're creating an AI model for something important, like answering climate questions or detecting fraud, Giskard helps ensure it does the job right.

Some of the issues we need to check during red team against AI:

• Prompt injection
• Backdooring the model
• Adversarial examples
• Data poisoning
• Exfiltration

Why Do We Need Giskard?

AI models can sometimes make mistakes, give biased answers, or generate false information (called hallucinations). Imagine you connect your chatbot to a climate research paper or report (i.e. Intergovernmental Panel on Climate Change). In this case, a climate assistant might say something incorrect about sea level rise. Giskard helps catch these mistakes so developers can fix them and build trust in their AI systems.

You can download Giskard: https://github.com/Giskard-AI/giskard

Key Features of Giskard

1. Automatic Model Testing

Giskard runs tests on your model to check:

• Accuracy: Does the model give correct answers?
• Robustness: Can it handle slightly different inputs, like typos or rephrased questions?
• Bias: Are the outputs fair and unbiased?

2. Finding Hallucinations

Hallucinations happen when a model makes up incorrect or unsupported answers. For instance:

• Question: "Why are sea levels rising?"
• Model Answer: "Because of volcanic eruptions." Giskard can spot such mistakes by comparing the model's answer to trusted information.

3. Interactive Debugging

Giskard allows developers to:

• Look at where the model went wrong.
• Understand why it gave a wrong answer.
• Test fixes and see results in real time.

4. Custom Tests for Your Needs

You can create specific tests based on your use case. For example, if you’re working on a climate assistant, you can test it using key questions and answers from trusted reports like the IPCC (Intergovernmental Panel on Climate Change).

How Giskard Works

Step 1: Connect Your Model

You connect your AI model to Giskard. For example, if you have a chatbot that answers climate-related questions, you set it up for Giskard to test.

Step 2: Load Test Questions

Provide Giskard with a list of questions and expected answers. For example:

QuestionExpected AnswerWhat causes global warming?Greenhouse gas emissions.Will sea levels stop rising?No, but reducing emissions can slow the rise.

Step 3: Run Scans

Run Giskard’s scan feature to test the model. You can focus on specific issues, like hallucinations, with a command like:

report = giskard.scan(giskard_model, giskard_dataset, only="hallucination")        

This creates a report showing where the model gave incorrect answers.

Step 4: Review and Fix

Look at Giskard’s report to see:

• What types of questions caused problems.
• Why the model made mistakes.
• How to improve it.

Real-World Uses of Giskard

Climate Change Assistant

If you're building an AI to answer questions about climate change, Giskard helps ensure:

• The assistant gives accurate answers.
• It can handle different ways people might ask the same question.
• It doesn’t make up unsupported facts.

Customer Support Chatbots

For customer service, Giskard ensures:

• The chatbot understands different customer questions.
• It responds politely and correctly.
• It works well even with typos or unusual phrasing.

Conclusion

Giskard helps developers find and fix problems in their AI models, making them more reliable and trustworthy. Whether you’re building tools for businesses or research, Giskard can help your AI works the way it should.