Applying the DevEx framework
This is the latest issue of my newsletter. Each week I cover the latest research and perspectives on developer productivity. Subscribe here to get future issues.
Last month, Laura Tacho (CTO at DX) and I held a discussion on the DORA, SPACE, and DevEx frameworks. One question that came up after the conversation was whether we had any resources to share on how to apply the DevEx framework.?
The DevEx framework, published last year, is an approach for measuring and improving developer productivity that focuses on the developer experience. In today’s issue, I’m sharing Laura’s guide on how to apply the framework.?
Here, Laura explains who should use the framework and how to implement it. This issue will be useful if you are currently dealing with the problem of how to measure or improve productivity, or are already interested in the DevEx framework.
The DevEx framework
Developer experience (DevEx) is a developer-centric approach to improving developer productivity. Instead of focusing wholly on output or activity metrics, DevEx focuses on the lived experiences of developers by measuring the effectiveness of tools and processes through the developers’ lens, and giving them a voice to influence the factors that impact their work.??
The DevEx Framework organizes many factors of developer experience into three categories: feedback loops, cognitive load, and flow state.
Feedback Loops: When a developer makes a change, can they get feedback about that change fast enough? Feedback from tooling, like tests or a CI build, and people, like project stakeholders, are equally as important. Slow feedback loops can interrupt or delay the development process.?
Cognitive Load: How much stuff do developers need to keep track of in order to complete a task? Complex processes, as well as complex code, can lead to high cognitive load, which slows development down and increases friction.??
Flow State: Slow feedback loops and high cognitive load can make it hard to get into a flow state, as well as other factors like unplanned work and a non-optimised meeting schedule. Flow state describes the opportunity to get into a state of energized focus. This doesn’t just mean having blocks of uninterrupted time, but also systems that allow developers to become immersed in their work by reducing friction.
Who should use the DevEx framework?
Teams who are interested in using metrics to improve developer productivity and engagement will benefit most from the DevEx framework.
Similar to DORA and SPACE, the DevEx framework is used by all sizes of companies, across many industries.
How do you implement the DevEx framework?
Similar to the SPACE framework, the DevEx framework strongly advocates for including developers’ feedback and experiences in definitions and assessments of productivity.? In contrast to SPACE, the DevEx framework is more prescriptive on what to measure, introducing DevEx KPIs, along with a framework to identify potential areas of measurement.?
The perceptual measures outlined in the DevEx framework are best collected through a developer experience survey. This allows you to standardize questions and responses in order to track progress over time. A limitation of surveys is that it is often unclear to the respondent how – and if – their response data will be used to noticeably increase their own working conditions.?
A countermeasure to potential disengagement is to transparently communicate the plan for the survey data before the survey is administered, answering the questions:
For smaller teams, or for environments where survey engagement is forecasted to be very low, you may want to use interviews or feedback opportunities like team retrospectives to collect data related to these perceptual measures. Since these formats are not standardized like surveys, you will need to index and categorize the data in order to track progress over time.
Workflow measures may also be collected via surveys, or your team may opt to ingest the data directly from workflow tools like your ticketing system or source control. The benefit of using a survey to collect workflow data is that you can simplify data collection by using only one method.?
领英推荐
A well-designed survey will provide accurate data about workflows. Remember, DORA is based on survey data!
A sample of questions might look like something like this:
Note: Here’s a DevEx survey template to help you get started.
With this data, teams can identify their highest priority drivers.
After identifying which areas are causing developers friction, some teams will then want to bring in quantitative metrics based on data from their engineering systems. For example, you might learn from the survey that developers are unsatisfied with the deployment process. You can then look at Merge to Deploy, a metric calculating the time between a code change being merged and it being deployed, and see that it’s taking developers more than a day to release approved changes.?
As another example, survey results might show that developers are frustrated with the code review process. You ask developers for additional feedback on what they’re frustrated with, and learn that some PRs can take more than 24 hours to be reviewed. You then look at the median, P75, and P90 Time in Review, a measure of how long a code change is waiting on review across all of its individual review cycles. You confirm that about 20% of developers’ PRs take over 24 hours to review.?
Bringing in quantitative data alongside qualitative insights can help you learn more about a problem you’ve identified. This can also be helpful for tracking improvements at a granular level. In the example above, you can use the Time in Review metric to show progress towards a goal, alongside the developer sentiment metrics gathered in your surveys.
When using qualitative and quantitative metrics together, start with qualitative metrics to figure out where to focus. Once your developers have identified specific areas, you can then use quantitative metrics to drill in deeper.
Quantitative metrics alone lack the context needed to assess whether your developers view something as a problem or a priority, and for this reason, we advise using quantitative metrics in isolation. In the example above, you could look at the Time in Review metric, and not be sure whether what you’re seeing is good or something needs to be improved. Systems data is also not as comprehensive as data coming directly from developers, because each measurement only captures a part of the system.
The best approach is to start with qualitative data to guide your attention, then use quantitative metrics to drill in further. And remember, all measurements should have a purpose.
What’s important to consider when using the DevEx framework?
In a previous issue covering SPACE metrics, we discussed how some leaders may be hesitant to adopt new ways of thinking, based on the latest research. DevEx highlights human attitudes and opinions even more than SPACE, making a strong recommendation that measures of experience, satisfaction, and attitude are critical in order to improve developer productivity.?
To be successful with the DevEx framework, it’s crucial that your organization has an intention of using the data to drive impact, and isn’t collecting data for the singular goal of performance assessment, or just out of curiosity. Teams that have been very successful with the DevEx framework have followed this 4-step process to improve developer experience and productivity:
Final thoughts
Engineering leaders understand that increasing developer productivity and engagement leads to better business outcomes. Frameworks like DevEx help teams define productivity and measure it. This gives engineering organizations clarity into where to focus and the ability to quantify the impact of change.?
Who’s hiring right now
Here’s a roundup of open Developer Experience roles right now:
If you’re hiring, submit your job posting here.
That’s it for this week. Thanks for reading.
-Abi
Technology and Product Leader
6 个月Damn Abi Noda, this is gold. Well done.
Staff Software Engineer
7 个月I was surprised to see Flow State as both one corner of the triangle while also being defined as dependent on the other two corners of the triangle. In other words, in one place, they seem to be treated as being on equal footing, while in another not. Abstract concepts are hard to get one's hands around, and that it can be hard to find the right words, but it seems like closer examination might reveal a refactor here.
Engineering product development flow with real time data § ?????? | ???????????????????????? | ??????????????
7 个月A great post!
Advisor, Consultant | Platforms: I've made all the mistakes so you don't have to.
7 个月I really like how specific this article is. The DevEx framework focus on subjective measures is a badly needed breath of fresh air, but figuring out how to implement is still a stumbling block for many.
Product Marketing Leader | Top 100 Product Marketing Influencer | SaaS, B2B, Startups | DevEx, AI and Developer Tools
7 个月Really great content. Was happy to receive it in my inbox this morning! Thanks for sharing.