Finding the Right Balance in Agile Engineering Between Process and Progression

Preface: The Fuel Behind This Article

This article was written with a minimum of 50 cups of tea consumed, numerous reviews of the Agile Manifesto, and an extensive deep dive into Agile videos—some of which were incredibly boring, but huzzah, here we are!

Introduction: Why Am I Writing This?

Why am I writing this? Well, simple. In our current environment—be it on one team or across the teams I interact with and train—I’ve seen firsthand that Agile frameworks aren't one-size-fits-all; success lies in thoughtful application. Whether it’s Scrum, Agile, Kanban, or even Waterfall and Waterfall 2.0, each has its place, but only when applied thoughtfully.

Too often, teams start by using what they like or what they’re familiar with. However, true agility comes when we guide people on a journey, rather than force-fitting them into rigid frameworks.

The Agile Paradox

Agile was designed to bring speed, flexibility, and collaboration to software development. Yet, many organizations find themselves trapped in a paradox: the very processes meant to enable agility often end up hindering progress.

Some teams become so bogged down in rituals—daily stand-ups, sprint planning, retrospectives, backlog grooming—that they mistake process for productivity. Others take the opposite approach, discarding structure altogether and creating chaos instead of agility.

So, how do we find the right balance? It’s a challenge I’ve wrestled with, and here’s what I’ve found.

"Too much process kills agility, too little creates chaos. How do you strike the right balance?" - Desmond Collins

When Process Becomes a Bottleneck

Agile frameworks like Scrum, Kanban, and SAFe were never meant to be rigid rulebooks. Yet, in many organizations, Agile adoption has become dogmatic rather than adaptive.

Common Signs of Too Much Process:

• Stand-ups turn into long status reports, retrospectives lose impact, and planning sessions drag on
• Grooming and prioritization feel endless, yet little actual work gets done
• Leadership prioritizes velocity over value, pushing teams toward burnout
• Teams focus on performing Agile ceremonies rather than delivering meaningful outcomes

The Result?

Agile teams start feeling like they are trapped in Waterfall 2.0, where process slows them down rather than enabling them.

The Other Extreme: When Progress Leads to Chaos

Some teams, in their pursuit of true agility, swing too far in the opposite direction. They reject structure entirely, leading to constant context-switching, unclear priorities, and burnout.

Common Pitfalls of Process-Free Engineering:

• Developers, designers, and QA struggle with ambiguity, leading to misalignment.
• Without structured planning, teams are constantly reacting instead of delivering.
• Work feels endless, quality suffers, and deadlines are always missed.
• Without structured feedback loops, teams repeat the same mistakes.

The Result?

Teams operate in chaos, where decisions are made reactively, tech debt piles up, and progress lacks direction.

Finding the Right Balance: Practical Strategies

To achieve true agility, organizations must strike a balance: enough process to maintain alignment, but enough flexibility to empower teams.

Prioritize Outcomes Over Rituals

• Agile ceremonies should serve a purpose. If a meeting doesn’t help the team deliver value, rethink it.
• Replace status-driven stand-ups with goal-focused discussions

Keep Backlogs Lean and Prioritized

• A cluttered backlog leads to decision paralysis.
• Use clear prioritization frameworks like MoSCoW or WSJF (Weighted Shortest Job First).

Adopt a Hybrid Approach to Agile

• Not all teams need pure Scrum. Consider blending elements of Scrum, Kanban, and Lean based on your needs.

Empower Scrum Masters and Agile Coaches

• True agility happens when Scrum Masters help teams navigate change, not just enforce rules.
• Agile leaders should act as servant leaders, guiding teams through adaptation, not just adherence.

Measure Success the Right Way

• Rather than fixating on velocity, measure how quickly an idea moves from concept to production (Lead Time)
• Evaluate whether we are solving real user problems—both externally and internally
• Track how efficiently teams iterate and improve using Cycle Times

Final Thoughts

The best Agile teams don’t blindly follow a framework—they adapt and evolve their processes to maximize progress.

The goal is not to check off Scrum ceremonies or track burndown charts, but to create a culture where teams deliver value continuously and sustainably.

"Agile is not about process vs. progress—it’s about balance." - Desmond Collins