The Hidden Trap Sinking AI Projects
Is your AI project failing? You're not alone. Organizations are citing between 60-80% of funded AI projects never make it into widespread use. How can there be so much failure with so many massive technology companies and well-funded researchers? Simply put - it's not a technology problem.
It's not what you do, it's the way that you do it.
Those that implement AI and Machine Learning project learn quickly that machine learning projects are not application development projects. Much of the value of machine learning projects rest in the models, training data, and configuration information that guides how the model is applied to the specific machine learning problem. The application code is mostly a means to implement the machine learning algorithms and "operationalize" the machine learning model in a production environment.? That's not to say that application code is not necessary -- after all, the computer needs some way to operationalize the machine learning model -- but focusing a machine learning project on the application code is missing the big picture. If you want to be AI-first for your project, you need to have a data-first perspective.
Can I "Agile" My Way to AI?
Agile methodologies are extremely popular for a wide range of application development purposes, and for good reason. Prior to the widespread adoption of Agile, many organizations found themselves bogged down by traditional “waterfall” methodologies that borrowed too much from assembly line methods of production. Rather than wait months or years for a software project to wind its way through design, development, testing, and deployment, the Agile approach focused on tight, short iterations with a goal of rapidly producing a deliverable to meet immediate needs of the business owner, and then continuously iterating as requirements and needs become more refined.
However, even Agile methodologies are challenged by the requirements of AI systems. For one, what exactly is being “delivered” in an AI project? You can say that the machine learning model is a deliverable, but it’s actually just an enabler of a deliverable, not providing any functionality in and of itself. In addition, if you dig deeper into machine learning models, what exactly is in the model? The model consists of algorithmic code plus training model data (if supervised), parameter settings, hyperparameter configuration data, and additional support logic and code that together comprises the model. Indeed, you can have the same algorithm with different training data and that would generate a different model, and you can have a different algorithm with the same training data and that would also generate a different model. So is the deliverable the algorithm, the training data, the model that aggregates them, the code that uses the model for a particular application, all of the above, none of the above? The answer is yes. As such, we need to consider additional approaches to augment Agile in ways that make them more AI-relevant.
Data-Centric AI
Responding to the needs for a more iterative approach to data mining and analytics, a consortium of five vendors developed the Cross-industry standard process for data mining (CRISP-DM) focused on a continuous iteration approach to the various data intensive steps in a data mining project back in 1999. Specifically, the methodology starts with an iterative loop between business understanding and data understanding, and then a handoff to an iterative loop between data preparation and data modeling, which then gets passed to an evaluation phase, which splits its results to deployment and back to the business understanding. The whole approach is developed in a cyclic iterative loop, which leads to continuous data modeling, preparation, and evaluation.
领英推荐
However, further development of CRISP-DM seems to have stalled, with only a 1.0 version fully produced almost two decades ago, and rumors of a second version under way almost fifteen years ago. IBM and Microsoft have both iterated on the methodologies to produce their own variants that add more detail with respect to more iterative loops between data processing and modeling and more specifics around artifacts and deliverables produced during the process. However, both companies are primarily leveraging their modifications in the context of delivering their own premium service engagements or as part of product-centric implementation processes. Clearly vendor-centric, proprietary methodologies can’t be adopted by organizations that have diverse technology needs or desire to utilize vendor-agnostic approaches to technology implementation.
The primary challenge to making CRISP-DM work is in the context of existing Agile methodologies. From the perspective of Agile, the entire CRISP-DM loop is contained within the development and deployment spheres, but it also touches upon the business requirements and testing portions of the Agile loop as well. Indeed, if we bring Agile into the picture, these two independent cycles of application-focused agile development and data-focused data methodologies are intertwined in complex ways.
Building a more effective AI-centric methodology
What makes things more complex is the fact that the roles in the organization between the application-focused Agile groups and the data-focused methodologies groups are not the same. While frequently the project manager is the center of the Agile universe, connecting the sides of business and technology development, the data organization is the center of the data methodology universe, connecting the roles of data scientists, data engineer, business analyst, data analyst, and the line of business. Frequently the language of communication is not the same, with Agile sprints focused on functions and features, and data “sprints” focused on data sources, data cleansing, and data models. Clearly the two parts of the organization serve the same overall master so we need to combine these two approaches into a cohesive whole that provides organizations the power they need to deliver AI projects reliably.
The answer, of course, is a blended methodology that starts from the same root of business requirements and splits into two simultaneous iterative loops of Agile project development and Agile-enabled data methodologies. We can think of this as an Agile CRISP-DM or perhaps a CRISP-DM enhanced Agile approach. It’s quite likely that CRISP -DM is not the only data methodology we can use here, but it is certainly suitable. However, there are some parts of AI project development that are not addressed by either methodology including:
To that end, there are approaches and methodologies that fill in these gaps with an AI-centric approach. Methodologies such as Cognitive Project Management for AI (CPMAI) made specific enhancements to the methodology to meet AI-specific requirements, especially as they pertain to the above requirements, and as they can be implemented in organizations with already-running Agile teams and already-running data organizations. Introducing something new and foreign is a sure way to get resistance. So the key is to provide a blended approach that simultaneously delivers the expected results to the organization and provides a framework for continued iterative development at the lowest risk possible. Because at the end of the day successfully running and managing an AI project is everyone’s goal.
Want more insights on making AI, data, and automation projects a success for your team? We'll be sharing tips and guidance on avoiding key AI Failures and best practices moving your automation and intelligence projects forward. Share this newsletter and encourage other team leaders to subscribe!