Why has the success rate of IT projects not improved? And why has it seemingly not worked to implement agile methodologies?
Bo Starlit
IT Management | Digital Transformation | GxP | AI | Project Management | Agile
Agile is a great methodology for software development. But there are still some very interesting issues lying in the latest research from Prof. Bent Flyvbjerg . At current the evidence points towards the fact that IT projects cost overruns has only increased in later years. Even as Agile development methodologies has been massively adopted across organizations. Prof. Bent Flyvbjerg and others has documented how IT projects (both agile and traditional) are far riskier than hitherto assumed. They cite two main sources of cost overruns, namely the high degree of interdependencies in IT systems and the so called ‘self-organized criticality’ of software systems. Basically, that the technical interdependencies cannot be anticipated as changing requirements happen organically which ends up making the IT system complex to such a degree that even small changes trigger massive work & cost overruns. The interesting thing is that Agile methodologies is exactly intended to accommodate changing requirements without jeopardizing the delivery plan. One of the main arguments in favor of Agile is that it allows for faster feedback and adaptation to changing requirements and customer needs. Which is great. So how come cost overruns not improving?
To some extent Agile is often not comparable to waterfall approaches at all, since Agile is often implemented as a permanently allocated cross-functional product team. Whereas waterfall projects are temporary and supposed to end once the product is delivered. Regardless, both are delivery vehicles for organizations’ IT changes and comparable in their ability to deliver the intending outcome for the business.
Prof. Bent Flyvbjerg points to the self-organizing criticality thesis meaning, that without intentional planning and deliberate architectural decisions, systems tend towards fragile complexity. IT projects should after all, not be like sand-piles or forest fires (the usual examples of self-organizing complexity). But a result of human planning and intentionality. So, if you endlessly keep pilling new requirements and dependencies into the IT system without a plan, you will end up at a critical state. This can for sure happen in both agile and waterfall approaches. So, one result is that well-architected solution design with a modularized approach is more relevant that ever, even in agile solutions. Or at least the recognition that being open to ever changing requirements without a clear set of design principles and scope will most likely lead to fragile technical designs.
?
?
Re-reading the IT treasure trove blog “Recipes of IT” by Jim Ditmore made me think of another potential & perennial challenge in digital. And more recently with agile. Namely the ability to standardize and compartmentalize IT components to avoid the interdependencies that Flyvbjerg mentions. Jim Ditmore brilliant short essay “Flow of work” (see illustration below) is addressing the need to continually free up time for the most valuable tasks by standardizing tasks. It is essentially about the IT management core responsibility of continually moving work to where it is most efficiently executed.
But also, about why documentation, processes, procedures and ultimately automation is needed. Documentation is for instance in this perspective not to pass an audit or to remember technical choices made. But to ensure that the product can be moved from the development team to an operational team. After all, if the development team is continually filling up it’s back-log with legacy request from existing products, there will be less and less time for actual product development. Think of tasks such as patching, access management, deployment of software or testing which should be routine (semi-)automated tasks. Or in the terminology below, building new products and services is custom work, which will entail operations work later. Whereas running or changing existing products are common engineering, that can/should eventually be turned into mundane operations. The problem can turn out to be if the Product Owner in Agile, as the customer representative, is prioritizing new feature development over the Flow of Work, as would traditionally be prioritized by IT. To some extent Agile is shifting the power away from IT towards the customer/user, which can be risky if this dynamic is not realized. At least there is no novel approach in Agile on how to handle the continuous ability to free up time for development as with ITIL where the lifecycle approach to service is front and center.
Maybe I am overlooked specific research on the benefits of Agile? Or overlooked other potential problems in Agile?
领英推荐
For more see:
Global Teams That Rock | Global Site Management | Real Agile Software Teams | Program and Service Management
1 年After several years of participating in and observing agile development efforts, I do agree with many of your comments. In addition, I find that many developers, QAs, designers and even scrum masters and P.O.s are surprisingly unaware of the thoughts and principles behind agile, which means that even though intetions are good at ground level, day-to-day actions are not aligned with agile ideas. In Topdanmark, one of the ways we address this is by allocating a whole month every year to various events around agile, such as panel debates, classes, debates, presentations from outside experts etc.