How do you balance innovation and experimentation with delivery and deployment in AI projects?

Agility in AI project management is essential due to its dynamic and complex nature. Agility promotes rapid adaptation to changes, enhances collaboration among stakeholders, facilitates continuous learning and improvement, effectively manages risks, and accelerates time-to-market. As part of agile methodologies, practices like MLOps streamline machine learning workflows, enabling continuous integration, delivery, and deployment of models, thus smoothing the transition from experimental stages to production. This combination of agile and MLOps enhances efficiency and effectiveness in AI projects, balancing innovation and delivery.

I agree with following an iterative process. Rather than waiting for the project to be 100% ready and after that getting feedback from business partners and stakeholders, it's better to iterate as we go and incorporate that feedback into the design process.

To visualize the agility need in AI projects, consider a chef tweaking a recipe based on customer feedback. Similarly, agile methodologies allow teams to adjust models and algorithms on-the-fly, utilizing feedback and new data sources to enhance project outcomes. Hence, these methods not only enhance model performance and robustness but also cater to the evolving needs of the project stakeholders.

The most challenging fields for AI would be the most regulated ones, like healthcare. Unlike businesses where you can only lose a small percentage of revenue from experimenting, in those cases you have much more at stake, and failed experiment can result in shutting down completely or enormous fine. In those cases, we would need AI not to just "fail fast" but also "fail safe". We likely would have a person as a decision maker, but it poses another question: the Human-AI interaction. We all saw how careless people become after the process is automated on the self-driving example. How to ensure the increased attention of the final decision-maker which is required after the update or change is an interesting challenge of applying Agile for AI.

Adopt agile methodologies that emphasize iterative development and feedback loops. This allows for continuous improvement and flexibility to incorporate new ideas and innovations during the project lifecycle.

Innovation in AI requires balancing exploration and fast-paced learning with the need for timely delivery. It involves fostering a culture that embraces the 'fail fast' principle, maintaining focus on project objectives amid experimental diversions, and ensuring solutions can scale effectively. Essentially, successful AI projects require a balance between encouraging innovation, timely delivery, scalable impact, and sustainable practices.

It’s important to note that innovation in AI isn't always about being the first to explore new territories. We can apply it to solve simple (and boring!) problems that deliver org value. For example, instead of jumping into newer, possibly unverified datasets, why not focus on maximizing insights from the data you already have? Often, businesses collect massive amounts of data but fail to utilize them fully. By refocusing AI-driven innovation efforts on deriving novel insights from existing data, one can minimize the risks and still stay on the innovation curve. This perspective shifts our understanding of innovation from being novel or ground-breaking to being more efficient and practical.

One strategy is to adopt a portfolio approach, balancing high-risk, high-reward projects with safer bets. This could involve, for instance, allocating part of the resources to refining and improving current AI systems, and the rest to more exploratory projects. Another tactic is to encourage 'fast failing' where quickly identifying what doesn't work and learning from it becomes as valuable as finding what does. Incorporating iterative testing throughout the process allows for the validation of results and minimizes technical debt.

I agree, it would be good to have the boundaries around budget and time one wants to give to an experimentation activity. Also, when we talk about the goal, there might be an effort required to break it down to smaller chunks to allow change in direction, if needed. It’s a delicate balance that needs to be maintained between goal setting and exploration. In terms of the tactical approach, Kanban framework works well when the experimentation domain is a leaning towards research, it offers flexibility on timelines thereby giving room for exploration which is one of the outcomes we want to achieve. Lastly, even if experiments fail, those are invaluable learnings for future ones that are performed. So ultimately it’s all success :)

Investing in innovation in AI is a question of choosing what to keep steady, standard, and what you will experiment with. Assuming you’re focusing on Gen AI, you might choose a commonly used LLM, a commonly used set of tooling, but you will explore different UXs and users in your org. Or you might choose to explore a new type of infrastructure to lower the cost of compute, but keep the UX, LLM, and use cases steady. The challenge is when orgs aren’t sure what they’re exploring, and failure comes knocking, and it’s hard to know what part of the experiment failed, and what lessons were learned. Plan on the lessons you will take away from the beginning. You’ll arrive at the right place.

A common practice I use to measure the success of an AI project is to establish and track both technical AND business metrics. 1. Technical metric: This measures if the project was implemented successfully. E.g., Does the AI model meet the defined data science metrics (e.g., precision, recall)? How does the model perform against the test dataset or other benchmarks? 2. Business metric: This measures the impact of the AI project on business outcomes. This one is often tricky since many of the outcomes such as revenue, cost savings, CSAT, etc. will be improved over time. However it is important to establish a baseline and monitor progress, even if it's just with the pilot users.

In practical terms, this might involve comparing the AI solution to alternatives, such as manual processes or competing software. For instance, by demonstrating how an AI solution saves time and resources in a manufacturing process, or reduces error rates in a medical context. Constant communication and iterative refinement with the customer can not only ensure that the solution meets the current needs but also adapt to evolving demands, thereby making the delivery process more valuable and impactful.

KI kann schnelle Ergebnisse liefern, indem sie effiziente Algorithmen und leistungsstarke Rechenressourcen nutzt, um gro?e Datenmengen schnell zu verarbeiten. Durch den Einsatz von vortrainierten Modellen und Transferlernen kann KI auch schneller angepasst und implementiert werden. Zudem erm?glicht die Automatisierung von Prozessen durch KI eine beschleunigte Durchführung von Aufgaben.

A prime example of this balance is how tech giants like Google or Amazon utilize A/B testing while implementing new features. They maintain a customer-centric approach by releasing different versions to small user segments, analyzing feedback, and iterating rapidly. Such techniques, blended with the use of Agile frameworks, ensure they can pivot quickly without disrupting service delivery.

To successfully balance innovation, experimentation, delivery and deployment, you need to have in mind the following criteria/questions: - Added value: what is the added value of the AI application to business? - Business strategy fit: does the AI approach fits and complement the business strategy and vision - AI building blocks and capabilities: do we have AI resources? Do the AI pilots help us build the necessary resources over time? - Low hanging fruits vs Added value: do we start with low hanging fruits/ easy options that get us easy and quick results or focus on the most Added value projects that are difficult, take time but deliver the most value? - Competitive positioning: which AI projects will help the company to outperform?

I agree that implementing agile methodologies, fostering cross-functional collaboration, and regularly evaluating project progress can help strike a balance between the two objectives, enabling efficient management of complex, dynamic, and uncertain AI projects.

An uncontrolled pursuit of innovation might lead to "analysis paralysis" or a never-ending cycle of experiments. This massively slows down deliverables. Consider the example of a manufacturing team building an AI model for predictive maintenance. They could get engrossed in creating an innovative model that anticipates failure with extreme precision. But this quest for the "perfect" predictive model could take forever. Meanwhile, the manufacturing unit could have multiple unanticipated breakdowns leading to downtime & financial loss. One must strike a balance by innovating to deliver 'reasonable', incremental value within the constraints of time and resources. Then, rinse and repeat!

Find the right balance between exploration and delivery -this is probably one of the biggest challenges for the modern corporate entrepreneur. One promising way to combine the two? Set your roadmap. Even a prototype can be a valuable deliverable, creating interest and commitment from both top management and process owners. Then, along research, spinning-off value to core processes might also help. IMHO :-)

Agree, The rise of AI introduces unique challenges that require project managers to adapt their skills and approaches to effectively navigate the complexities of these projects. While traditional project management team primarily emphasizes executing plans and delivering projects on time and within budget, AI projects introduce a heightened level of complexity, uncertainty, and innovation to the management process. Following Skillset may be required to navigate those challenges -Adaptive Planning -Technical Proficiency -Risk and Change Management skills -Resilience and Problem-Solving skills -Team Empowerment -Clarity and Simplicity in Upwards communication -Continuous Learning

A key question that gets often neglected is the integration of AI applications in the existing business process. Without considering this aspect, it is highly likely that after a while the AI application will be put aside on the shelf. AI project management should consider on how to integrate the AI models/ applications into the business process. Key considerations include: who is going to benefit/use the outcomes of the model? Will these people receive a training on how to integrate the outcomes of the model? Is everyone clear about the added value of the AI model? who is going to run the model? Who is accountable for the sign off of the decisions made? Who is going to monitor the AI model so the performance doesn't decline over time?

One of the biggest issues is that the implementers need enough depth to really get a handle on how to extract properly from the data flows. There is a need for an understanding of business workflows so that they can comprehend the actual needs of the business. And what exactly the right questions to be asked would be. This takes a serious cognitive depth of the needs that come from these workflows. This is incredibly important. Such people need both a good understanding of functional decomposition but beyond that enough ability to cognize the depths of the workflows. They need to make sure of the questions really fit the topology of the environment