How to make Agile work for AI Products?

Expecting 100% accuracy in AI products is like expecting your pet cat to fetch your newspaper every morning. You might have seen it in movies, but in reality, it's just not gonna happen.

AI has made some impressive strides in recent years, but let's face it, even the best AI models are not infallible. It's like they say, "to err is human, to blame it on the AI is even more human."

Why is Scrum failing for AI products?

One of the biggest reasons Scrum can go wrong with AI products is the unpredictability and complexity of AI development. Unlike more traditional software development projects, AI development involves working with data that can be messy, incomplete, and difficult to understand. This can make it challenging to estimate work accurately and plan sprints effectively, which are key components of the Scrum methodology.

Imagine estimating the effort to clean data that is coming from your local repository. You don't know the quality of the data, how many columns are defined properly? how many of them are missing? how many need to be deleted? 2 days, 2 weeks or 2 months. How would you know?

Additionally, AI development often involves working with cross-functional teams that include data scientists, software engineers, and domain experts, among others. This can lead to communication and coordination challenges, particularly when it comes to defining clear and actionable user stories and acceptance criteria.

Let's say you're working on an AI project with a team of data scientists and software developers. The data scientists are responsible for building and testing the AI models, while the software developers are responsible for integrating the models into the larger software system.

During a sprint, the data scientists identify a problem with the quality of the data they're using to train the models. They need to work with the software developers to modify the data processing pipeline to address the issue.

However, the software developers are already working on other user stories and may not have time to address the data issue immediately. This can lead to communication and coordination challenges, as the data scientists may not be able to move forward with their work until the software developers have completed their tasks.

Another challenge is that AI development often involves experimentation and iteration, which can be difficult to fit into the fixed timeboxes of Scrum sprints. In some cases, it may be more appropriate to use a more flexible development methodology that allows for more experimentation and iteration.

How can I customize my Scrum for AI?

Start with Data Quality: Spend the maximum amount of time understanding the data at hand. May be create a separate sprint just to generate an understanding of your data quality and see if the given data is good enough to meet the objectives of the product. Analyze situations where this data might fail in real world. Communicate them to your product owner.

Sizing user stories: In traditional Scrum, user stories are sized based on the amount of work required to implement them. However, in AI projects, user stories may also need to be sized based on iteration at hand. We need not go into infinite loops to ensure 100% efficiency but limit the current story to a pre-determined level. This can help ensure that user stories are realistically sized and that the team can plan sprints effectively. For example, only deal with missing values, for the current sprint. We will look at data transformations later.

Incorporating experimentation and iteration: As mentioned earlier, AI development often involves experimentation and iteration. To accommodate this, it may be helpful to incorporate shorter development cycles within Scrum sprints. For example, the team could use the first half of the sprint to experiment with different models and techniques, and then use the second half to implement and test the most promising approach.

Understanding Model Performance: Even if an AI model is 99% accurate, there's still that 1% chance it'll make a mistake. But, am I still good with the 99% or not is based on how clearly we define our acceptance criteria. So whether we are using Accuracy, Precision or Recall as a measurement metric is driven by defining clear acceptance criteria, that way the team can ensure that they are building models that meet the needs of the user and that can be tested effectively.

Continuous Integration and Deployment (CI/CD): CI/CD is a best practice in software development that involves automating the build, test, and deployment process. In AI projects, this can be particularly helpful because it allows the team to quickly test and deploy new models as they are developed. This can help reduce the time required for testing and improve the overall quality of the product.

Regular retrospectives: As with any Scrum project, regular retrospectives are essential for identifying areas for improvement and making adjustments to the development process. In AI projects, it may be particularly helpful to focus on issues related to data quality, model performance, and coordination between different team members. By addressing these issues proactively, the team can improve the quality and effectiveness of their AI product.

Data Quality as a User Story: Have regular check-ins between the data scientists and software developers to ensure that everyone is on the same page and that any issues are identified and addressed as quickly as possible. Additionally, it may be helpful to incorporate a "data quality" user story into each sprint, which can help ensure that data issues are identified and addressed early in the development process.

Who is the Ideal Product Owner for AI?

The ideal Product Owner for an AI product would be like a cat owner. Just like a cat owner needs to understand their cat's behavior, needs, and preferences to provide the best care, a Product Owner for an AI product needs to understand the needs and preferences of the user to ensure that the product meets their expectations.

Cats are known for being independent and having their own unique personalities. Similarly, AI products can be complex and have their own unique characteristics and behaviors. A good Product Owner for an AI product needs to be able to work closely with the development team to understand these characteristics and ensure that they align with the needs of the user.

Just like a cat owner needs to provide food, water, and shelter for their cat, a Product Owner for an AI product needs to provide clear and detailed requirements for the development team to ensure that the product meets the user's needs. They also need to be able to provide feedback and make adjustments as necessary to ensure that the product is delivering value to the user.

Jokes aside, A Product Owner for an AI product should have a good understanding of data, since data is a critical component of many AI applications. They should be able to work with the data scientists and engineers to ensure that the data used to train the AI models is accurate, relevant, and appropriate for the application.

In addition, the Product Owner should understand how the AI models work and how they can be applied to solve real-world problems. This will help them define clear user stories and acceptance criteria that can be used to guide the development process.

Who is the Ideal Scrum master for AI?

A guide dog is trained to provide assistance and guidance to their owner, helping them navigate the world safely and confidently. Similarly, a Scrum Master for an AI product should provide guidance and support to the development team, helping them navigate the complexities of AI development and ensuring that the team is working effectively and efficiently.

Like a guide dog, the ideal Scrum Master for an AI product should be highly trained and experienced. They should have a deep understanding of Scrum principles and practices, as well as expertise in AI development methodologies and best practices. They should be able to anticipate potential obstacles and provide guidance to the team to overcome them.

In addition, the Scrum Master should be a strong communicator and collaborator, able to build strong relationships with both the development team and stakeholders. They should be able to facilitate productive discussions and help the team make decisions that align with the overall product vision and goals.

Let's say the development team is working on an AI application that uses natural language processing (NLP) to analyze customer feedback. The team has encountered a problem where the NLP model is not accurately capturing the sentiment of the feedback, leading to inaccurate insights.

The Scrum Master, who has experience with AI development, should be able to recognize this issue in advance and suggest to the product owner that the team try using a pre-trained model that has been fine-tuned for sentiment analysis. May be include this item as a separate story.

The Scrum Master can also help the team identify potential sources of bias in the data and provides guidance on how to address them.

In conclusion, it's clear that Scrum and Agile methodologies need to be adapted to effectively build AI products. However, let's not forget the importance of a good sense of humor in dealing with the challenges of AI development.

After all, isn't AI itself a bit of a joke? We spend all this time teaching machines how to do things that we humans can do effortlessly, like recognizing images or understanding language. But hey, if it means we can spend more time napping in the sun like our feline friends, then it's all worth it!

In all seriousness though, building AI products is an exciting and challenging endeavor. By adapting Scrum and Agile methodologies and embracing a bit of humor along the way, we can work together to create innovative and valuable AI products that make a real difference in the world. So let's get to work, and remember to take time for a good laugh along the way!