The CTO Playbook

Accelerate pt. III - Institute a Metrics Driven Culture

In the post below, I do advocate for you to 'measure your pudding' - this might sound strange but if you read on I promise it will make more sense. Maybe.

This is part two of Accelerate, where I discuss what I regard to be one of the most important areas in which you, as CTO, should be focussing your effort.

These particular metrics drive the correct behaviours throughout your organisation, and those behaviours are necessary to build a high-performing technology capability.

In the first two Metrics articles (HERE and HERE) I covered what the metrics are and how they positively influence behaviour (the word 'behaviour' here covers everything your team does, including the automation of processes).

In this particular post I'll talk about how you can go about establishing these metrics as a core part of your technology capability.

First, some general points....

The goal here is to establish an operational discipline and cadence for discussing the Accelerate metrics. This will embed them in how the team think, and it will encourage behaviour that positively impacts the metrics.

It's also important to understand that these metrics are not particularly about achieving a particular "numeric" goal - for example, the optimal cycle time for one team or one company could be quite different than that of another. Rather, these are directional - as a rule you should be continually reducing your cycle time over a period of time. These metrics will tell you when this improvement process has stalled or reversed, as much as helping drive it in the correct direction.

You may get to a point where you feel your deployment frequency and cycle time metrics are optimal i.e. that driving them further could have negative overall impacts - and that's fine. Use your judgement for this.

Once you're there these metrics then become more about measuring and monitoring flow.

Remember, the overall system you have in place for getting value from concept through to the customer is highly complex, has many moving parts, and requires continual monitoring - so you should view these metrics as you would the temperature or oil gauges on your car dashboard. They'll tell you when something is wrong and point you in the right direction so that you (or preferably, your team!) can address it.

So here are the more specific recommendations to help you embed these metrics:

Presentations

Present these concepts to the team.

Provide examples of how they influence change in a positive direction. The key here is to help the team understand the association between performance and these metrics, and how their implementation moves the organisation in the direction of higher performance.

That's not necessarily easy - you'll have to have a good grasp of this yourself first, however good teams will be very excited to start adopting what they will consider the more modern 'best practices' of high performing teams.

I find that going back to some of the core Lean concepts helps underscore how much waste (and thus opportunity for improvement) exists in the current mode of working. A great way to illustrate this is to have the team perform a Value Chain Mapping and present the results to the team. These usually highlight dramatic discrepancies between the amount of time the team are actively creating value and how much time it takes to deliver that value.

Visibility - Dashboards

The psychology of dashboards has received a lot of attention, and for good reason, as it's a powerful way to get focus on the right area. It's simple in essence; place a dashboard where a team can see it, and then watch as the line on the graph magically goes in the right direction.

It may be a basic human compulsion that drives us to make the line on the dashboard move. There's a gamification aspect to it certainly; we're creating a feedback loop for behaviour that aligns with the organisational goals.

The corollary to this of course, is "out of sight, out of mind", which should highlight how important visibility is. If you don't have these metrics in front of people, don't be surprised when nobody thinks they're important.

So for each team, create dashboards that represent their cycle time, deployment frequency, CFR and MTTR. Often weekly averages for these is enough, but do try to make the dashboards as responsive as possible, so that the team gets as close to real-time feedback as possible.

One thing to note - if you have multiple teams, don't just present the average metrics of the overall group. Individual teams can have very different local situations, often with different environments, constraints, processes or even technology stacks.

So the metrics a given team is presented with must be scoped to that which is within their control to change. Ensure that their efforts to improve are not blocked by externalities or swamped by negative metrics from other teams.

Weekly 'Heads of' meeting (or status meeting, or whatever you happen to call it)

You should have in place a weekly status meeting where your team (by 'your team' I mean you and those reporting to you) discuss ongoing and upcoming projects, and report on events and performance of the team for the previous week.

Reporting on the team performance naturally means that on a weekly basis, they're reporting the accelerate metrics for each of their teams.

In order to do this, they've had to spend time gathering and analysing these metrics, and often with the help and involvement of various members of their team.

The first time you run one of these meetings, it will probably be a bit messy. People won't have been able to gather the right information in time, or will be missing key data, or won't have the systems or reporting in place in order for this to go smoothly.

That's ok - it will be better next week, and the week after, and so on until it runs smoothly every time.

Have each of your direct reports present the metrics for their area e.g. Operations, Engineering, QA or Product. Make it an open conversation - why did we do less deployments? What can we learn from it? What caused the dip in cycle time? How can we avoid that in the past? What direction are the numbers trending in and why?

For the purpose of these articles I'm focussed mainly on the Accelerate metrics. You'll have other areas within the software delivery process where more detailed metrics are required (QA comes to mind here), and often specific metrics that apply to operational or support areas. These should also be discussed as part of this meeting.

Job descriptions and KPIs

Establishing these metrics across the teams and continually contributing to moving them in the right direction should be a KPI for all relevant team members. I've had these written into job descriptions to underline their importance. This is an example of one I've added to the position description for a management role:

Managing the team to continually improve core performance metrics:

• Cycle Time
• Deployment Frequency
• Change Failure Rate (CFR)
• Mean Time to Restore (MTTR)

Here's an example of a KPI for an individual contributor:

• Required Objective: Contribute to improving Cycle time, deployment frequency, CFR (change fail rate) and MTTR (mean time to restore)
• Measure: Actively working to improve the processes, tooling, automation and practices that drive these metrics

There's a risk of wrong-footing it with KPIs - we've all had to deal with crap KPIs and they often have a bad reputation. It's your job to ensure you avoid having vague or un-actionable KPIs so tread carefully. On the positive side, the Accelerate metrics share most of the attributes of excellent KPIs.

Sprint Retros

You can improve visibility of the Accelerate metrics in development teams by encouraging each team to present on how they've improved various metrics on a sprint-to-sprint basis. This can be very satisfying for the teams, it allows them to celebrate their wins and will give them ownership and pride in not only the work that they do, but HOW they do it (this is a key step toward creating 'self-managed' teams).

One-on-ones

You're doing weekly one-on-ones with all of your direct reports. And naturally each of your directs are doing weekly one-on-one's with their direct reports. One-on-ones, done correctly, are the single best way to build effective professional relationship with your staff. I'll talk more about this in a future post.

In these one-on-ones, when it's your turn to talk about what's important to you (this being AFTER they've talked about what's important to them!), you should discuss the Accelerate metrics and how your staff are progressing with their implementation. Listen to the challenges and successes. Discuss possible solutions and approaches. Explain the intent and impact of the metrics if required. And check that your staff are doing the same during one-on-ones with their direct reports.

After all, you've set these as their KPIs because you want your people to achieve them. Good staff like to achieve their KPIs. They can see that you're genuinely interested in helping them meet their KPIs and they're not going to turn down assistance. Everyone's a winner.

Repeat, Repeat, Repeat

I mentioned earlier that the goal is to establish an operational cadence for discussing these metrics. Status meetings every week, no exceptions. One-on-ones every week, no exceptions. Rinse, repeat.

This will embed these metrics in how the team think and drive positive behaviour.

Repetition is how we do this.

Internal Marketing - present to the business

As I've discussed in an earlier post, one of your responsibilities as CTO is to develop productive relationships with your peers, and facilitate those same productive relationship between your teams and other teams in the organisation.

Many CTOs will miss this, but transparently demonstrating the ongoing improvement of your team's performance and capability is one of the most effective methods to garner positive support from the rest of the organisation. After all, the reason your team exists is to serve the organisation, whether that be through the creation of software products used by the end-customer, and/or by providing a technology function that enables the business to operate across practically every area.

The proof is in the pudding, and the Accelerate metrics allow you to measure the pudding - I may be straining this metaphor somewhat, but I believe I have a solid case - the pudding in this case is your team's ability to deliver value. So start measuring your pudding.

Helping the organisation understand how you measure performance, and demonstrating the improvements driven through the adoption of this approach, will constitute very effective internal marketing. It gives you - as a team - something to be proud about.

So present your efforts and your results to the rest of the business. Get your team to do the same, it's good for them!

That's all for today folks.

The next chapter: https://www.dhirubhai.net/pulse/cto-playbook-rob-hill-kc95c