From promise to proof: a practical case on value tracking & delivery in SAFe?

How to define and measure actual (delivered) value in strategy execution?

Within SAFe (Scaled Agile Framework ?) delivering value is a key concept. Value is defined as “the economic worth of a capability to the business, the customer and/or employee.” Portfolio management is a process with the aim to align the strategy with the execution of value delivery. By releasing software customer and business value gets created. A practical way of defining and measuring the actual value is vital in the agile context, so companies can focus their resources on delivering the highest value in the shortest time continuously.

An advanced topic [1] in SAFe concerns how to actually operationalize, track and subsequently quantify the realized (business) value after software release. Up till now, only little information from practice is published about this topic. This article aims to fill this gap by presenting an approach on how to design and implement a form of value tracking across the strategic, tactical and operational levels within SAFe. This method is primarily rooted in Lean Six Sigma thinking and follows the quantitative, objective view to the concept as currently advocated in SAFe. The approach is developed in the business context of Essent, a Dutch energy company in The Netherlands.

This article positions the concept of value tracking in the portfolio management context of SAFe, using the flow model of Promise and Proof in strategy execution [2]. First, it reviews why value tracking is considered troublesome in practice. As an answer to this an approach is presented that helps to deal with value tracking in SAFe. This approach is illustrated with an example and corresponding tools. Finally, this method is assessed on its advantages and its challenges.

The key question answered in this article is: How to define and measure actual (delivered) value in strategy execution?

Keywords: value delivery - value tracking - hypotheses testing - benefit tree - Lean Startup Cycle - portfolio management - strategy execution

Note: This article is written for people acquainted with SAFe. More background information on SAFe can be found on: https://www.scaledagileframework.com/

1. Value tracking is not extensively operationalized in SAFe ?

Within SAFe Lean portfolio management (LPM) is considered one of the key competencies to reach business agility. In this context SAFe recommends to track the development progress against the key results of the strategic themes. Although SAFe elaborates briefly on some tools and concepts related to value translation and tracking (see for the latter the concept of leading indicators) companies are struggling with the implementation of it.

People find it fairly difficult to define how to measure the (business) value of the delivered software. Two commonly heard remarks in practice are “I can't guarantee that the software impacts the objective because something else might impact it in the meantime” and “By the time I have measured the results the company will already be working on new things”. This causes the first SAFe principle – taking an economic view – to be undermined. After all, SAFe promotes delivering the highest value most early in time. Next to the value concept, also considerations on development expense, lead time, product cost and risk determine the set of alternatives for what, how and when to build it. Although being one factor ‘only’, lacking to properly define the linkage between (business) value and the specific software built, will complicate proper execution of the before mentioned key principle severely.

This will also become visible in running through the process of the Lean Startup Cycle: if the accept/pivot decision on the Minimum Viable Product (MVP) of the epic can’t properly be made due to fuzziness on actual value delivered, proper portfolio management can’t take place. 

This article emphasizes on the need to put effort in thoroughly designing a system of aligned measurements around value tracking & delivery and introduces a practical framework to make an effective start. By doing so it contributes to more validity and communicability of value delivery throughout all organizational levels. 

2. A practical elaboration on value tracking in the SAFe ? context

2.1 The Value Tracking and Delivery Approach

In figure 1 the concept of value tracking is positioned in the process model of SAFe. It shows how value is translated from a strategic business level to the operational build level (see upper arrow - “promise”) and how - after the release of actual software - this value gets captured in practice (see lower arrow - “proof”). In the center arrow the process and tooling used for prioritization and planning of software development is shown. As can be seen, the approach covers the process from setting business targets on strategic themes till tracking the value of realized objectives at the ART level.

One can split up Figure 1 into three stages. Each stage has its own way and level of looking at value and entails corresponding tools. The three stages are interrelated and should be properly aligned for sake of value translation and tracking purposes. The concept of the benefit tree (see stage 2) and it’s positioning towards existing concepts and tools mentioned by SAFe is this article’s main contribution to practice.

2.2 Stage 1 – Strategy

In stage 1 strategic themes get formulated. Subsequently, each theme is coupled to objectives which are translated into key results. This value translation process is called OKR: from Objective to Key Result(s). Ideally, a set of key results are defined. The key results each need to be measurable. Note that the unit of measurement doesn’t have to be financial in itself (see also the example in section 3 of this article).

Per key result business ideas are defined by the Product Manager and the Epic Owner. These business ideas sketch the first contour of the software developments that might be needed to realize the (key) results. An idea might be an epic in itself or represent a set of epics that need to be broken down further in a later phase. Ideas get prioritized by executives and plotted in a time span. As such the portfolio roadmap is loaded with these ideas by the Business Owners. The upcoming year’s horizon of this roadmap is most important: this will be the input for the portfolio Kanban (see stage 2).

2.3 Stage 2 – Portfolio

In stage 2 the chosen business ideas are operationalized by epics. According to SAFe an epic is “a container for a significant solution development initiative that captures the more substantial investments that occur within a portfolio. Due to their considerable scope and impact, epics require the definition of a Minimum Viable Product (MVP) and approval by Lean Portfolio Management (LPM) before implementation.” ? Scaled Agile, Inc. [1]

The definition and intent of an epic is key since the stakeholders need to be on the same page regarding this significant enterprise investment. SAFe presents the Hypothesis statement template (? Scaled Agile, Inc). as the tool that captures, organizes and communicates critical information about the epic. Note that this template doesn’t explicate the actual hypothesis nor its specific mechanism. Epics are prioritized by Epic and Business Owners using the WSJF principle. A PI roadmap is formed.

Epics are managed at the portfolio Kanban (Figure 4) and will flow through different maturity phases in which they get accepted or rejected. These stages are:

• F â€“ Funnel
• R – Review
• A – Analyze
• B – Backlog
• I – Implementation
• D – Done

From the review phase onwards the epic gets more precisely defined. Guided by its defined objective and corresponding key result(s) it is recommended to create a benefit tree for each epic during the review phase. The benefit tree explicates the expected relation between the objective/key result(s) of the epic and the business parameters influenced by the new software developed. See figure 5.

The benefit tree is a useful tool in the process of value translation and tracking for several reasons. First, by creating a benefit tree one is forced to focus on and explicate what can be objectively and easily measured with respect to changes in business outcomes [3] due to new software releases. This overcomes the tendency to stay fuzzy on value delivery due to the ‘obvious’ lack of insight in the financials directly related to the strategic objective. The benefit tree is therefore helpful in the prioritization process of epics: it helps to give a to-the-point motivation for defining the cost of delay during WSJF prioritization. Next to this, the benefit tree stimulates profound thinking regarding the mechanism by which the strategic objective is influenced. By visually linking all drivers in a model early in the process, sound and coherent value cascading to other organizational levels is promoted. Third, by focusing on easy-to-measure outcomes, hypotheses can be validated shortly after software release. This is the key point behind the recommended use of the Lean Startup Cycle for epics in SAFe, also referred to in the topic Innovation Accounting (? Scaled Agile, Inc).

Before flowing to the implementation phase, epics require analysis (done by amongst others Epic Owners and System Architects). The result of the epic analysis phase is preferably captured by a Lean Business case (? Scaled Agile, Inc) within SAFe. In the analysis phase an epic is broken up into features (smaller pieces of functionality).

Features are central in hypothesis testing. Preferably, each feature contributes to testing one or more hypotheses set in the benefit tree. By doing so it becomes clear how the feature will hit a business parameter in the benefit tree, and what is the expected change in business outcome and the subsequent contribution to value delivery. The benefit tree can be seen as an add-on tool to the Lean Business case and the Hypothesis statement template. It more profoundly explicates and controls value delivery. Subsequently, an additional format - shown in Figure 6 - can be used to summarize the promised and proven impact of the total set of features on the objective(s) of the epic. See section 3 for an worked out example of this format.

During the WSJF session, the cost of delay can be determined based on the sum of hypotheses for all features in the epic. The prioritized epics can be found in the backlog phase at the portfolio Kanban. The epics with the highest priority are input for the PI planning event.

2.4. Stage 3 – Agile Release Train

In stage 3 the Agile teams define their PI objectives based on the prioritized features that flow into the PI. These PI objectives are scored by the Business Owners using value points (which typically rank from a scale from 1 till 10 and are not directly related or derived from the value concept talked about earlier in this article). The teams will then come up with an aligned planning: the PI plan. This PI plan for the ART is formally agreed upon if all team members of all teams give a confidence vote of at least 3 points. Then the PI starts and the software gets developed (implementation phase at the Kanban). PI objectives are tracked and eventually evaluated by the Business Owners at the end of the PI.

The Lean Startup Cycle is considered key in the implementation process of an epic. After being approved for implementation, the Epic Owner works with the Agile teams to realize the epic’s business outcomes hypotheses. SAFe considers the released (MVP) software to be ‘done’ after the related hypotheses are tested. After evaluating a hypothesis two outcomes can be reached. Either the hypothesis is proven true and additional features and capabilities will get formed and implemented. If the hypothesis is proven false there can be decided (by the LPM) to pivot by creating a new epic from this one or the epic can get dropped.

Figure 8 summarizes the total approach and shows in each stage who takes what action. Note that next to the three stages (vertically) the figure shows horizontally:

• Promise: flowing from left to right – value related
• Prioritize & Plan: flowing from left to right
• Proof: flowing from right to left – value related

3. An example from practice

In this fictive example a company has determined three main objectives (promise):

·      We increase the Customer Satisfaction to 90%

·      We decrease the Cost to Serve to €20

·      We increase the Margin by 50%

For each objective the executives have defined a strategic theme. In this example we will focus on the theme “We increase the Customer Satisfaction to 90%”. The executives have defined the following key results (promise):

·      We lower the # complaints of our customer journey by 50K

The Epic Owner and Product Manager have defined 8 ideas to accomplish this by looking at the amount of complaints. One of the ideas is to offer the customer the possibility to change the payment date, because for quite some people this moment comes at an unsuitable moment during the month.

The executives prioritize this idea amongst others and it is placed on the Portfolio Roadmap and loaded as an epic on the Portfolio Kanban.

Once the epic reaches the review stage, a benefit tree and an extended hypothesis statement (promise) are made. See Figure 9 for a fictive example.

Note that the business parameters which serve as actionable measurements are at the right end of the benefit tree. Changes in these parameters prove whether feature(s) do or do not have the hypothesized impact on the objective.

Note that each feature contributes to proving one or more hypotheses. This is also visible in Figure 10, where the Feature Benefit Hypothesis is added as an element to the Lean Business Case. For example, in feature 2 the Flexible Payment date is enabled in the apps. The epic owner expects that by doing this:

• The # of complaints will decrease by 900 a year
• The # of inbound calls will decrease by 3000 a year
• The # of cases of bad debt will decrease by 50 a year 
• The # of outbound collection calls will decrease by 50 a year

The example in Figure 10 shows how the value of the features is proven. In this example three features contribute to proving the hypothesis that the number of complaints will decrease, resulting in an expected change of the Customer Satisfaction (CSAT). Only two features influence the number of inbound calls, so only these features are taken into account in observing whether the Cost to Serve (CTS) actually decrease by a decrease in incoming calls.

Now we jump ahead in the approach and we assume that the new features are released. The Epic Owner can now prove the set hypotheses being valid by looking at the measured variables (most right in the benefit tree). Figure 11 shows the Promise and Proof for two of the objectives (Customer Satisfaction and Cost to Serve) in an easy- to-use format.

In Figure 11 one can see that the two first features have been delivered. As can be seen the second feature - enable the Flexible Payment date in the apps - proved to be more successful than promised: the expectation was that the variable number of complaints would decrease by 900, but it decreased by 1200. The Epic Owner expects that the number of complaints will decrease by 1800 once feature number 3 (Enable Flexible Payment Date via the website) is released.

4. Advantages and challenges of the approach

Advantages

• Implementing a form of value tracking contributes to insight and more consciously looking at value delivery. This promotes maximization of value delivery in itself.
• By making the promise and the proof of value transparent using the benefit tree and the corresponding tracking format, the basis for fact based learning is created.
• Value is communicated through all layers in a consistent manner.

Challenges

• The approach requires discipline and thorough thinking in a context where there is often a lot of pressure on delivery.
• The approach requires training. Practice shows that people experience difficulty in setting up a proper benefit tree without training.
• The implementation of the model relies on strongly defined objectives and key results set by executives, since these form the start of a benefit tree.

4. Conclusion

This article tried to make a contribution to existing work by making value tracking and delivery more specific and actionable in practice. This was done by 1) conceptually positioning the topic of value tracking in the operational process model of SAFe using the flow model of Promise/Proof in strategy execution; 2) introducing the benefit tree as an add-on tool/element to the Hypotheses Statement template and the Lean Business Case template; 3) linking each feature to the benefit hypotheses as set in the benefit tree; 4) introducing a summary format on quantitively tracking delivered value; 5) presenting an example based on practice.

This approach is meant as a start and will flourish by further elaboration. Two follow up questions to consider are for example:

1.    To what portfolio rituals can this approach be translated, so the steps happen in cadence and flow is maximized?

2.    The approach requires discipline from all its actors. How to make it stick?

Please share your elaborating thoughts, alternative design ideas or comments on this topic by leaving a reaction!

About Stijn Driessen

SAFe Program Consultant | Lean Six Sigma Master Black Belt | Bridging IT and business | Specialist in/enabler of change journey travels | Visually Minded | lover of Fun & Twists | Father | Amateur magician | Word joker 

My publishing about my work finds its roots in my professional motivation for making conceptual models actually succeed in the workplace. I do so by 1) creating my own (visual) thinking models to clarify and simplify issues and 2) by the development of practical tools and working methods for implementation purposes. Tuning and improving these with others sharps my mind and motivation and adds to a more successful practice.

Special thanks to:

• Toon Wijnands, for the many sparring sessions to create this approach
• Marisca Sinkiewicz for co-writing this article
• Niels Groen and Paul van der Haar, with whom I wrote the article about The flow model of Promise/Proof in strategy execution
• Celeste Scharrenberg for designing the first sketches of the visuals of the approach
• Cover picture by Nattanan Kanchanaprat via Pixabay

References:

[1] https://www.scaledagileframework.com/guidance-applied-innovation-accounting-in-safe/

[2] https://www.blinklane.com/insights/strategic-flow-integral-approach-effective-strategy-execution-and-portfolio-management/

[3] In the Innovation Accounting topic within SAFe these measurements of business outcomes are called leading indicators. They are defined as “actionable metrics focused on measuring specific early outcomes using objective data. Leading indicators are designed to harvest the results of development and deployment of a Minimal Viable Product (MVP). These indicators may include non-standard financial metrics such as active users, hours on a website, revenue per user, net promoter score, and more”.