Agile Practices for HSD: From Experimentation to Organizational Adaptation
For those of us who work with Human Systems Dynamics (HSD), complexity is at the heart of what we do. But what happens when complexity meets product development and technology? Agile emerged as a response to exactly that challenge.
Many years ago, when I first encountered Agile, something clicked. I saw a practical approach to working with uncertainty that mirrored what I now recognize as HSD principles. As I deepened my understanding of systems and complexity thinking, I naturally gravitated toward HSD. Now that my journey has (finally!) led me here, I’d love to offer the HSD community an understanding of Agile—what it is, why it works, and how it aligns with the principles we already know and use.
Historical Context of Software Development
In the late 1980s, as software development took off in companies, traditional project management methods were applied to the field. These methods assumed that software could be built like a bridge or a factory product - by following a predictable, step-by-step plan. The typical approach was a linear, phase-based process:
At first glance, this structured approach seemed logical, but in practice, it consistently failed. Projects ran over time, exceeded budgets, and delivered products that often didn’t meet real user needs. The issue wasn’t just poor execution - it was a mismatch between the method and the complex reality of software development.
Traditional project management assumes that a problem can be fully understood before work begins. However, software development is an inherently complex system. Much like what HSD teaches us about complex adaptive systems, software projects exist in an environment where:
At the time, many organizations responded by trying to plan better, document more, and control projects more rigidly - thinking that if they just got the first two phases right, everything else would fall into place. But others recognized that the problem wasn’t how they planned, it was the assumption that planning alone could solve uncertainty.
It was in this environment that Agile thinking started to emerge—not as a single method, but as a response to the increasing complexity of software development. However, software developers weren’t the first to face these challenges. When looking for solutions, many turned to Japan’s Toyota Production System (TPS) and the broader Lean movement.
From Manufacturing to Software: Lessons from Lean & TPS
As software developers searched for better ways to handle complexity, they found inspiration in an unexpected place - manufacturing, specifically the Toyota Production System (TPS). Developed by Toyota in the mid-20th century, TPS revolutionized how products were built by shifting away from rigid, top-down planning toward flexible, adaptive, and people-centered systems.
At its core, TPS is built on two pillars:
1. Continuous Improvement (Kaizen) – Constantly refining processes to reduce waste, improve flow, and enhance quality.
2. Respect for People – Empowering frontline workers to solve problems, make decisions, and drive improvements.
Software developers realized that many of the challenges they faced were similar to those addressed by TPS, making its principles especially relevant to their work. Much like factory production, software development had its own version of waste: building unnecessary features, waiting for approvals, and fixing preventable defects. However, one key difference emerged. In manufacturing, the goal is to refine a repeatable process, but in software, every product is unique and involves continuous learning and discovery.
This is where Agile diverges from traditional Lean thinking. While Lean focuses on efficiency and eliminating waste, Agile actively embraces experimentation, iteration, and adaptation. From an HSD perspective, Agile sees software development as a complex adaptive system - one that requires ongoing sense-making, rather than strict control.
By borrowing Lean’s focus on flow, feedback, and empowerment but adapting it for a world of uncertainty, Agile pioneers created a new approach - one that allowed teams to navigate complexity rather than fight it.
The Birth of Agile: Embracing Uncertainty
As software development teams experimented with Lean ideas, they encountered fundamental differences between manufacturing and their own work. Unlike physical production, where the goal is to eliminate variation, software development thrives on exploration and adaptation. Uncertainty isn’t just a challenge -it’s a natural part of the process.
One of the most significant shifts was the move toward iterative and incremental development. Instead of defining every detail upfront and delivering a finished product months or years later, teams started working in short cycles, producing small but functional pieces of software. These early versions could be tested, used, and refined based on real feedback - reducing the risk of building the wrong thing. In HSD terms, this reflects an adaptive action cycle:
Cross-functional collaboration also became essential. Traditional project management kept business and technical roles separate, creating delays and misunderstandings. Agile teams, by contrast, expanded the container to include both business and technology perspectives. They tightened the exchanges between these groups, ensuring that the differences that made a difference - customer needs, technical feasibility, market shifts - were surfaced and addressed in real time.
Rather than optimizing for efficiency alone, Agile methods focused on learning and responsiveness. Teams experimented, adjusted their approach based on results, and embraced change rather than resisting it. This approach mirrors the HSD idea of seeing patterns in complexity - rather than forcing rigid structures, Agile teams focused on recognizing emerging patterns and adjusting accordingly.
These principles eventually led to structured Agile frameworks such as Scrum and Kanban, offering practical ways to implement iterative work, collaboration, and continuous improvement. Then, in 2001, a group of thought leaders gathered to define the essence of Agile, resulting in the Agile Manifesto - a concise statement of values and principles that continues to guide Agile practice today:
HSD practitioners might recognize these "Agile principles" as simple rules that align with HSD’s focus on placing humans at the center and embracing adaptation.
It's important to understand that these simple rules came from software developers in companies that still mostly treated their IT as a separate department, even if they were starting to get integrated better. Agile has since expanded beyond software development, with many other fields using their frameworks and methods, like libraries, police departments, fire departments, research labs, and others. Agile has transformed from an approach to develop software to a management approach. Transforming an entire company to enable fast reaction to changes has become the goal. This includes pushing authority down the hierarchy, enabling teams to make most of their own decisions.
Agile Transformations: Customization is Key
While there are many frameworks and methods, it’s important to understand that customization is key. Taking the core ideas and adapting them to your context will lead you to success in an Agile transformation. Doing it the other way around is likely to lead to a failed transformation. Accepting and really integrating the underlying principles is difficult, as it means relinquishing much power from upper levels of a company, while simultaneously requiring those at lower levels to take on that power - and the responsibility that comes with it.
This shift isn’t just about adopting new tools or procedures; it’s about changing the very mindset of the organization. It’s about how we view leadership, collaboration, decision-making, and the process of change itself. At its heart, Agile is a philosophy that encourages collective problem-solving, continuous improvement, and the embrace of uncertainty.
However, as Agile has become mainstream, it’s not without its challenges. The "Agile is dead" movement has emerged in response to some organizations' failed attempts at scaling Agile. The issue isn’t that Agile doesn’t scale - it’s that scaling Agile requires a transformation in culture. Without embracing a shift in mindset, organizations can’t expect success, no matter how literally they apply Agile frameworks.
Agility: A Broader Organizational Philosophy
Rather than rejecting Agile, the "Agile is dead" movement is encouraging us to rethink how Agile is applied. The problem isn't the principles themselves, but how they’re operationalized within organizations. Many companies have struggled with their Agile transformations not because the practices are flawed, but because they haven’t integrated the necessary cultural changes. Agile requires a shift in the way organizations think about leadership, collaboration, and decision-making. Without this shift, Agile practices often remain isolated and ineffective.
A key factor in these failed transformations is the reliance on external consultants or Agile coaches. While these experts may understand the mechanics of Agile, they’re often not equipped to tackle the deeper cultural barriers within organizations. Successful Agile transformations require both top-down leadership buy-in and bottom-up cultural engagement. When these elements are missing, the efforts often fall short.
Organizations are now becoming increasingly frustrated with the money and time spent on Agile transformations that fail to deliver meaningful results. Too often, Agile is seen as a set of tools to apply, rather than a philosophy to embrace across the entire organization. Companies have either learned that unless they truly understand and integrate Agile’s deeper principles into their culture, simply adopting Agile frameworks and processes isn’t enough to drive long-term change, or abandoned Agile entirely.
As organizations move forward, the focus is shifting from 'Agile for IT' to 'Agility as a systemic approach' for the entire organization. This shift goes beyond applying Agile principles within product development teams and extends across all functions - HR, finance, marketing, operations, and beyond. Just as complex adaptive systems rely on all parts working together and responding to shifts in the environment, true agility requires a holistic mindset where all elements of the organization are interconnected and co-evolving.
In practice, this means that agility is no longer viewed as a set of isolated tools or processes. Rather, it becomes a way of being that shapes the culture and structure of the organization. Every part of the organization, whether it’s a function or a cross-functional team, operates within its own container. However, it’s important to regularly zoom in and out to understand how these containers are part of the larger organizational system. The entire organization needs to be able to respond to internal and external changes in real-time, learning from feedback and adapting accordingly. By shifting perspectives between the micro (individual containers) and the macro (the organization as a whole), we ensure that all parts are aligned, responsive, and interconnected within the broader adaptive system.
The future of Agile lies in how well organizations can integrate its principles into all aspects of their operations. From HR and finance to marketing and sales, agility is about embedding continuous learning, flexibility, and responsiveness into the DNA of the organization. Leaders and teams need to embrace Agile not just as a set of processes, but as a mindset that can be applied across all levels of decision-making.
This holistic approach to agility is what will allow organizations to thrive in an increasingly complex, fast-paced world. Rather than treating Agile as a buzzword or a quick fix, organizations that embrace it as an ongoing organizational capacity are positioning themselves for long-term success. It’s about understanding that agility isn’t a one-time transformation, but a continuous journey of adaptation and improvement.
Agile Practices, HSD Principles
For HSD practitioners, the evolution of Agile provides a fascinating example of how we navigate complexity. In many ways, Agile practitioners experimented their way into complexity. They didn’t start with a clear-cut, comprehensive solution to managing uncertainty. Instead, they tested different approaches, refined their practices, and learned from their experiences. Over time, this iterative process gave rise to a set of solid practices that could be applied across organizations.
This experimental, adaptive approach aligns deeply with the principles of HSD. Both Agile and HSD see complexity not as something to avoid, but as something to engage with, learn from, and adapt to - making it a natural fit for HSD practitioners.
For HSD practitioners, this means recognizing that agility isn’t something to simply adopt or impose. It’s something to be co-created through experimentation and feedback loops, gradually embedding itself into the organization's culture and systems.
In the end, the future of Agile is not about strictly following frameworks, but about embracing an adaptive, systems-oriented approach where organizations continuously experiment, learn, and evolve to stay responsive to both internal and external changes. By combining the lessons of Agile with the principles of HSD, organizations can truly thrive in an uncertain, ever-changing world.