What skills do you need to evaluate the impact of AI projects?

Collaborative mindset is the key. AI projects often require a multidisciplinary approach, combining expertise from data scientists, domain experts, ethicists, and other stakeholders. Being able to effectively collaborate and communicate across these diverse fields allows for a more comprehensive evaluation of AI impact. It's not just about understanding the data. Is is even more about understanding its real-world implications from various viewpoints. This means that fostering a mindset that values diverse insights and perspectives can provide a fuller picture when assessing the consequences of AI projects.

Data literacy is all about making sure that everyone in an organization knows how to work with data and put it to good use. This means that decisions are based on facts and evidence, not just on gut feelings or guesses. For instance, let's say a retail company looks at sales data to decide which products to stock. With data literacy, they can see which items are actually selling well, and that helps them stock what customers want, leading to better results. Being data-literate also helps teams make smarter decisions instead of relying on hunches. Imagine a marketing team checking customer feedback data to decide on advertising strategies and more implications for decision-making.

Using AI in real-world tasks helps you understand it better. It's good to know AI theory, but using it on real problems lets you see how it actually works. This hands-on approach helps you see the strengths and limits of AI. By trying and testing, you'll better judge the true impact of AI projects.

AI offers tangible benefits by leveraging data and analytics. For instance, it can crunch vast datasets, uncovering hidden patterns that manual analysis might miss. Think of an e-commerce company using AI to analyze customer purchase history and recommend products based on their preferences, improving customer satisfaction. AI also shines in automating repetitive tasks. For instance, in customer support, chatbots can handle routine inquiries, freeing up human agents to focus on complex issues. This automation boosts efficiency and cuts operational costs.

This means grasping the "why" behind the AI project. You should know the purpose and the end goal. With domain expertise, you should also know the project's roadmap: where it started, where it is now, and where it wants to go. This makes it easier to measure success. If, for instance, an AI tool in healthcare aims to reduce patient waiting times, understanding if it truly does this – and by how much – is vital. So, combining your deep knowledge in the field with the project's purpose is key to truly seeing the AI impact.

It's good to explain things simply. While evaluating AI projects, sometimes the details can be technical and hard to understand. So, being able to break down these complex topics into easy-to-grasp ideas is crucial. If you're sharing findings with someone who's not tech-savvy, they should still get the main point. For instance, instead of diving deep into algorithms, you might say, "The AI tool is trained to recognize patterns faster." This way, everyone stays informed and involved, no matter their background.

Thinking critically means you don't just accept things at face value. You ask questions like, "Why is this AI tool working this way?" or "Is this data we're using good quality?" When reviewing AI projects, it's essential to double-check everything and think about possible issues or mistakes. It's a bit like being a detective, looking for clues and making sure everything adds up. And just like in detective work, it's important to keep an open mind and not jump to conclusions too quickly.

Think outside-the-box. No other way to put it. Often, we build AI projects by the book. In reality, real-world situations demand innovative solutions. Theory exists as a general guide, explaining core concepts. How you apply those concepts is up to you. Being a healthy rebel goes a long way, both for you and your project.

AI systems will relentlessly optimize an objective function for a given data set. There are examples of AI systems that found it easier to exploit glitches in computer games than to learn how to play the games as intended. It is crucial to define and select data and objective carefully, but also to be mindful about unavoivable gaps and biases in data or aspects of your task that cannot be easily modeled in a mathematical function. In practice, I would recommend to run an AI audit with a diverse panel of experts to think about biases, risks and ethical implications of a system. If you didn't find any potential issue with your current real-world data science project, you probably didn't search enough.

The importance of ethics in technology, AI, and our society is growing as these systems become a bigger part of our daily lives. These technologies have the potential to change how we interact with the world and each other. But with this power, we need to make sure we use it in a good and responsible way. For example, we should set rules for keeping people's personal information safe, protect against cyberattacks, and stop AI systems from making unfair decisions. As we use more AI and tech, we should talk more about what's right and wrong in how we use them, teach people about it, and make some rules to keep things fair and safe for everyone. This way, we make sure technology benefits society instead of causing harm.

In my view, it all comes down to what alternatives you have and what additional value this AI project can bring. We don't use AI for the sake of using AI. Everything needs to come with a purpose. And, in the world of business, that is by looking at what additional business value this AI project can bring compared to the other alternative solutions.