The Role of Product in Enterprise IT: Why Agile Often Falls Short

In recent years, many - if not all - large enterprises have embraced agile practices. We have seen it all: Scrum, SAFe, XP, and Kanban among others - including bizarre permutations like Wagilefall with 4.5 week sprints (trust me, I've seen it). And while some organizations have seen tremendous success with Agile, others have seriously struggled. Despite the proliferation of agile methodologies and associated trainings, bootcamps, articles, blogs, webinars, and gurus, there is a critical role that is often mentioned too inconseqentially in the equation: Product. Clarifying this role is particularly important in legacy enterprises, where it is frequently undervalued, and one of the main reasons why so many articles have come across my newsfeed in the past 2-3 years declaring that "Agile Is Dead."

The Criticality of the Product Role

The product role is crucial in agile because it provides the vision and strategy that guides the team. It is responsible for ensuring that the team is aligned with the business goals, and it ensures that the product is meeting the needs of the users. There are several key skills required to be an effective product leader, including strategic thinking, measurement and evaluation, and case development.

This means when an under-skilled product manager is leading the strategic team, agile can seem to "not work." The product manager must have a clear vision for the product and be able to measure its progress toward achieving business goals. Without these skills, the team can become misaligned, and the product can fail to meet the needs of the users and stakeholders.

Similarly, when an under-skilled product owner is leading the tactical team, agile can also seem to break down. The product owner is responsible for ensuring that the team is focused on delivering outcomes, not just outputs. They must be able to manage dependencies and advocate for visibility to the users and business goals. If the product owner lacks these skills, the team can lose sight of their goals or get caught up in cross-team dependencies, and the product can be delivered late or with suboptimal results, and more often than not the team is entirely incapable of speaking to the value delivered.

The Gap in the Enterprise

One of the reasons that the product role is so often overlooked in enterprise IT is the assumption that a product owner (PO) is simply a rebranded business analyst (BA). However, there are significant differences between the two roles' responsibilities. To complete these roles, a PO has to have a distinct or expanded skill set, even if there is some overlap with that of a BA.

The BA and PO roles may seem similar, but there are several important differences between them. A BA is often focused on gathering requirements and documenting processes, and while a PO often ends up also gathering and documenting requirements, they should primarily be focused on understanding the desired outcomes, ensuring that the team is aligned with business goals, and working with the team (and stakeholders) to determine the best way for driving those goals. BAs typically primarily work closely with stakeholders to understand their needs, while POs are responsible for ensuring that the product meets the needs of the users and discussing the user value-business value tradeoffs with the business stakeholders. Additionally, POs must have strong strategic thinking skills and be able to evaluate the success of the product against business goals, whereas BAs may not have these skills to the same degree, as they are usually deployed tactically. Ultimately, both roles play important roles in delivering a successful product, but the skills required for each are a bit different and organizations don't always understand the nuanced differences.

Due to this perceived role overlap and the misunderstandings surrounding the needed skillsets, organizations often do not appropriately upskill BAs when they are redeployed as POs, leading to a gap in developing product and portfolio strategies. They do acknowledge the need for training, but that education is usually focused on tools and techniques (aka, getting a PO certification and familiarizing yourself with JIRA), rather than building analytical skillsets and cultural perspectives that a digital native company would likely screen and hire for and then look to cultivate in a PO.

Now don't get me wrong, I am not a fan of all the recent layoffs we see across the technology landscape and am not advocating for companies embarking on digital transformation to axe their BA team. However, the reluctance to appropriately re-train or reduce a large number of BAs as organizations move towards agile approaches has resulted in an over-reliance on existing personnel who may not have the necessary skills to succeed in their new roles. When these organizations don't understand the required skill nuances and leave them undeveloped, it is easy to see where problems can emerge. These sorts of companies end up struggling to evolve their thought processes and products at the pace of their digital-native competitors and fall behind with mounds of tech debt, many shiny (but non-useful) features, and generally poor ROI on their tech investments.

That leads us to the obvious next solution these companies see: as natural attrition occurs, replace the talent with Product personnel from tech companies or startups. These people should be able to "help develop" the product team and improve our products in a healthier way, right? Unfortunately, these individuals often burn and churn in the enterprise environment. It comes back to the same systemic problem: a lack of understanding of and organizational support for the role, leading to a lack of accountability to outcome measurement, enablement of experimentation, empowerment of teams, and consideration of end users over business stakeholders. These external hires face challenging structural hierarchies, minimal empowerment, and limited visibility, leading them to ultimately leave the organization. Then they take the valuable domain knowledge they built over the 6-12 months they stayed with them, repeating the cycle anew. This exacerbates the gap in enterprise product management practices, as you add churn into the mix on top of skill deficiencies.

The Way Forward

To address these challenges, enterprises must first acknowledge the existence of the problem. It is a good thing to have prioritized their people and avoid lay offs, but we need to acknowledge that a certification course does not rewire how someone approaches problems or wholesale develop a previously missing skillset. Even if the training provided touches on these topics, we need to recognize that we need to equip managers with the right expectations on what makes for a great PO. And we need to also remember that these managers likely just recently had teams of BAs, and Product is new to them too, so they likely do not totally understand and may lack the needed skillsets as well, making their mentorship and skill development challenge that much larger. As those nuanced details have often been missing in many organizations' Agile transformations, these BAs-turned-POs haven't been incentivized to strengthen the right skills and have largely struggled.

Next, organizations must develop measurement strategies. Some are already doing this, realizing the pervasive data landscape they operate in enables them to make data-driven decisions at all levels. However, when that gets passed down to poorly equipped POs, the data-driven decision-making and outcome reporting struggles to manifest. A good measurement strategy should provide clear business goal traceability from executives to teams and operationalize measurement across teams. Incentives should then be given to POs based on outcomes, not just outputs, and spaces should be created for external hires to reshape the organization's product norms. By incentivizing the desired behaviors and outcomes, organizations can begin to use those foundations as a means to reinforce and educate those in PO roles and help develop the right skills to drive things forward.

Investing in the Product role in enterprise IT is critical to the success of agile methodologies, and by addressing the gap in the enterprise and taking steps to support the product role, organizations can ensure that their agile practices deliver the results they expect. By truly learning about the Product role and investing in its development from a place of deep understanding, enterprises can drive innovation, create value for their users, and better achieve their business goals.