DevEx talk: flex the measure-act-learn muscle at scale in your dev teams

I had an incredible time hosting a talk on Developer Experience [DevEx] with exceptional people: Benjamin Kitt , Sr. Engineering Manager, Lob , and Jon Kern , Co-author of the Agile Manifesto from Adaptavist . Here is the webinar recording, and below are my thoughts.

Want to catch the essence of the discussion?

Watch the 5-minute recap video!

Insights from the DevEx Talk

Here are the key takeaways from our conversation during the webinar. I hope you enjoy them!

What Do DevEx Surveys Bring to the Table?

A human-centric, holistic approach to measuring progress, focused on the day-to-day operations of delivering software. A repeatable process for continuous, iterative feedback and improvement creates space for autonomy and builds confidence through an easily measurable way of making progress and demonstrating a positive impact.

One of the key things about DevEx surveys is that they ask questions incredibly relevant to the day-to-day lives of the developers you're surveying. I think this is one of the main reasons you're seeing such high engagement. - Benjamin

DevEx surveys help focus on various aspects of the experience, ensuring success that typically leads to greater confidence. - Jon

Let's break it down!??

Golden Insights You've Been Gathering for Years, Now with New Powers

Do you know how your dev team—or your 10, 100, or 1,000 dev teams—are doing across these areas? What are your strengths? Where are the roadblocks and friction points?

Team-Level Power: Insights become easier to act on—measurable, repeatable, and simpler to prioritize, implement, learn from, and celebrate

DevEx surveys clarify a process known by tech teams for decades - gathering developer feedback, identifying blockers and pain points, and pinpointing the most effective ways to accelerate value delivery.?

DevEx surveys are a tool that brings clarity to what you're already doing as a team. They formalize processes like Gemba walks, retrospectives, or lean coffee sessions—gathering feedback from ICs, identifying blockers, and accelerating value delivery. Surveys make this process measurable, repeatable, reliable, and easier at the team level. - Benjamin

With DevEx surveys, you not only have a built-in way to track "How are we doing?" and turn the team's gut feelings into tangible data; you also give the team the autonomy to drive the process.

The team knows what's going on—let them assess themselves and drive it. You get better results because they own it; it's a group initiative. - Jon

What’s the difference between what you’re already doing and DevEx surveys??

DevEx Surveys vs. Retrospectives: Surveys provide a broader, measurable perspective, while retrospectives offer deeper insights into specific topics. Surveys are excellent for identifying roadblocks, while retrospectives are ideal for diving deeper into those issues.

• Consider the Bigger Picture: Surveys capture insights from a broader scope, offering a more holistic view.
• Personal vs. Group Thinking: Surveys reduce the influence of groupthink, allowing individual voices to be heard.
• Measurable: Surveys turn gut feelings into tangible, quantifiable data.

DevEx Surveys vs. Gemba Walks / Lean Coffees: Surveys are more scalable and less likely to be influenced by the presence of leaders, unlike live sessions. Gemba walks and lean coffees offer real-time, face-to-face engagement, providing deeper context, immediate feedback, and collaborative problem-solving

• Less Likely to Change the State: Measuring with surveys doesn't alter the dynamic or how people act, as can happen in live discussions.
• Scalable: Surveys can be easily scaled across teams, providing consistent insights.

Org-Level Power: Strengths and friction points are easily identified at a scale of 10, 100, or 1,000 teams

High-level observability, built from bottom-up insights in a repeatable way, delivers real value. While a single dev team often knows the pains they face and the potential gains, DevEx surveys allow you to capture and measure those insights across many teams. Benchmarking across teams provides context for the scale of friction, while benchmarking over time enables you to measure progress.??

Measuring and benchmarking make things more visible. - Jon

DevEx surveys give you the ability to measure and group data across an organization. As an engineering manager, you often hear from your team: 'This is getting in our way,' or 'This is causing friction.' Some of these issues are within your control, while others, like infrastructure-level problems, are not. You may take the feedback to a director, VP, or CTO, but as a single data point, it can be hard to drive action. You might talk to other teams to see if they share the same issues, and DevEx metrics provide clear, straightforward data right from the start. With this data, you can see - this is a problem for 90% of teams. This is something we need to address. - Benjamin

How many engineers are involved in your organization for infrastructure and the dev sandbox—5, 50, 100? By using surveys, their work will be better aligned with real-world problems and prioritized more effectively. Their goals will be clearer, and progress will be visible faster through survey benchmarking than with passive data, which often fails to uncover certain issues.

Emergent Power: Practical, accurate insights from all engineers involved

Those on the ground know best—you can’t overestimate the insights from the incredibly smart, experienced people you work with. For example, Google has been measuring DevEx with surveys since 2018 and explored over 100 metrics to capture technical debt based on engineering log data. No single metric predicted reports of technical debt from engineers, and the models accounted for less than 1% of the variance in survey responses. What does this mean? Developers’ estimates on complex issues are often more accurate than any metrics.

Talking with your teams at all levels of leadership is how you get the best feedback on how they’re doing and what’s in their way. You’ve hired incredibly smart, experienced people to build software, and asking them what is getting in their way is the most effective way you can find the challenges that you need to to clear away first. - Benjamin

I cherish the fact that we have so many smart people. Why aren’t we using them? We could do command and control, but that’s the antithesis of how I’ve worked for decades. The critical component in DevEx is a more team-based, holistic approach. We understand our direction, and all brains can engage to make us more effective. I want everyone to focus on the 80:20 rule—20% of the effort for 80% of the value. Let’s start there. DevEx elements help us measure how well we deliver value, understand priorities, and negotiate scope. - Jon

Elevated Power: Complex issues are simplified, making them easier to understand and act on

Let’s revisit measuring technical debt at Google. It’s a complex issue to measure and address. Where do we draw the line between technical debt slowing us down and what’s an acceptable level? Where’s the highest friction, and where should we start acting? It becomes clearer when you simply ask if engineers— and to what extent— are hindered in their work by technical debt or overly complicated code in their projects.

Measuring is key. As engineering managers, we’ve been gathering feedback, observing blockers, and holding retrospectives for years. What DevEx surveys bring is a way to measure these human, creative values and turn them into actionable data. The critical point is that these surveys help prioritize and understand what’s important for your company’s success. DevEx metrics provide that data in a simple, straightforward way. - Benjamin

It’s All About Making Progress and Having a Positive Impact

A Stack Overflow report shows that 80% of developers are unhappy at work, but improvements are what make them happy. Creating a loop of measuring, acting, and measuring again—even for the smallest successes—is a powerful way to build confidence and engagement in tech teams.

You've hired amazing engineers who are most effective when they're happy in what they do—writing, shipping, and delivering code, especially value to customers. As human beings, we're tuned to succeed, deliver, and feel good about achieving something. Measuring and communicating even the smallest achievement is important, and DevEx helps capture that. - Benjamin

Teams usually know what’s going on, but being able to benchmark, have conversations about how we work, and then benchmark again creates a closed-loop feedback system. It helps us tackle an aspect we want to improve, prioritize it, and present a stack-ranked list of value versus effort to leadership. - Jon

Many things measured with DevEx surveys are things we're consistently trying to improve as teams and engineering managers. We understand that we're making progress, but sometimes you forget about the progress you've made or can't quantify how the things you've done or changed have had a positive impact. That's what DevEx brings to the table—the ability to quantify that. - Benjamin??

DevEx surveys are just one tool among many for measuring what’s in your control. To truly understand the friction, engage with your team through interviews, retrospectives, or conduct Gemba Walks and Lean Coffees. If you want to see how improvements impact the entire software development process, use DORA or FLOW metrics. The best insights come from blending data across multiple sources.

In my experience, DORA has been a great tool for measuring how we build software. FLOW focuses on what we’re building and whether it delivers value. DevEx sits between both, looking at how we can optimize the process with a more human-centered approach. They all measure different aspects, and you'll find overlaps and impacts between them. Actions based on DevEx surveys have positively impacted our DORA metrics, showing how they work together harmoniously. - Benjamin??

How to Run a DevEx Survey to Get the Most Out of It?

Tech leaders are concerned about potential hurdles with surveys, especially the fear of low response rates. However, many tech organizations are achieving remarkable response rates of 90% or more for their DevEx surveys. The key question is how to engage exceptionally smart engineers in measuring and improving DevEx. Here are some tips.

Start from a Place of Trust

Trust yourself as a leader, give yourself permission to take ownership of your team's experiences, measure them, and experiment.

One of the things as an engineering manager that took me a while to kind of get my head around is that I run this team. I have ownership of this team like I can make a difference. I can say, we're going to spend 20% of our time to improve our developer experience, and you can make that decision. You don't have to wait for someone to make that for you, but then use these opportunities to measure that and demonstrate how your team is delivering better because of that. And that's going to help you build up that trust and that empowerment. - Benjamin

If you don’t already talk to ICs regularly, do that first.?

If you're not already talking to your engineers and building trust and relationships, a survey won't help you. Before surveying, start by engaging with them and going through that process. - Benjamin??

Frame It and Tie It to Real-World Problems?

Frame the change and prototype it: set timelines, define outcomes, and outline actionable steps to get there.

Change requires effort and time, and you need to adjust it to how urgent the change is or how fast you can make it. You also need to prototype your change: do experiments, and try it. It’s not a life sentence—if you change some aspect of how you're working, you can change it again. But treat it as if you have expected outcomes. - Jon

Why are you doing these surveys? What do you hope to get out of them? What are you trying to deliver for your team(s)? What are the expected outcomes?

The framing of the survey is really important up front, like explaining to people why this relates to the work they're doing day in and day out. - Benjamin

Tie real-world problems you've observed to DevEx metrics that may relate or inform. Explain what is being measured and why. A great thing about DevEx is that metrics are more tangibly related to real on-the-job experience than your standard employee engagement survey. They provide people with confidence that metrics will drive change and prioritization.?

Tying real-world problems to the DevEx metrics you're measuring in surveys is key. For every metric, there's likely a real-world problem you can connect to it. As a leader, it's critical to explain to your engineers how this will help you advocate for them, prioritize what matters most, and make progress on those key issues. - Benjamin??

Frame the survey as a way to feed data into the decision-making and prioritization process. Get buy-in from the highest level possible. If you want to improve DevEx, you need to prioritize it with data that shows its importance and how it will deliver business value.

Flex Existing Muscles to Act: Leverage the Processes & Tools You Already Use to Build Features

If your team is regularly introspective, you likely already have the tools and frameworks in place to do this. By consistently turning retrospective conversations into action items, you can flex that same muscle.

Things we do typically with DevEx surveys in terms of process - we ask engineers: what are your top priorities? Which of these things is really dragging on you? I think it's important to understand what those priorities are. We look for the problems that we can impact immediately. - Benjamin??

Use the tools you already have to build features to solve DevEx problems. Prioritize, define expected outcomes, and apply existing tools, such as story mapping.

To solve DevEx problems, we use the tools already in place for our day-to-day work on feature development. Story mapping is a tool we use extensively when developing features for customers, but you can absolutely apply it to your own personal user journeys. What outcome are you looking for? For example, as an engineer, when I create a PR, I want it to pass the first time. I don’t want any more flaky tests. Once we have the outcome, we can break down the user journey, identify pain points, and tackle a slice, just like we would with any other user story for a customer. - Benjamin

A strength is having abstract concepts that apply recursively. Solving DevEx problems is not much different from how we approach developing features. - Jon?

It's crucial to explore metrics at various levels. Are your team's priorities in sync with the larger goals of the organization? Managers and leaders at every level should look at data from all sides. One thing to watch out for is the rise of silos, which can lead to less effective ways of working.

Craft a Story and Experiment: Outcome, Steps, and Data That Backs It Up

Use the real-life experiences your engineers share to shape the story, highlighting smaller successes that have a meaningful impact on customers. And once again, flex those existing muscles.

Share the story you've crafted during leadership-level retrospectives. These sessions provide space for senior leaders to reflect on progress, align on priorities, and discuss strategic goals. By presenting your story—clearly outlining the outcome, steps, and supporting data—you ensure leadership is aligned and engaged, helping drive focus and support for the initiatives that matter most to your team and organization.

Experimenting with new DevEx improvements can align with canary deployments, allowing you to test changes on a small scale before a full rollout. Like a canary deployment, introduce new practices or tools to a subset of your team, monitor results, and gather feedback. This approach minimizes risk, enabling you to refine and adjust improvements based on real-time data before scaling them across the organization.

Demonstrate It Worked and Stop Over-committing. That’s How You Build Confidence and Autonomy in Your Dev Team, with Internal Improvements as the Best Environment to Start

We often focus so much on solving problems that we forget to celebrate the successes. However, the loop of problem, solution, and success builds confidence, showing your team that it has the autonomy to solve problems successfully.

Demonstrate change as quickly as possible and link it directly to feedback. - Benjamin??

Stop over-committing and focus on iterating, enabling your team to make steady progress without feeling overwhelmed.

In my experience, a lack of confidence in dev teams often stems from not being able to measure success and potentially overcommitting. The way to build confidence is by tackling small problems, delivering effectively, and learning from those frequent wins. This is why we’ve as an industry moved toward a more iterative process—to achieve frequent feedback and success. Confidence grows through constant learning and achievement. It's a difficult process that requires continuous work and focus, but if you can focus on the small things and help your team consistently achieve them, their confidence will grow. - Benjamin????

When it comes to trust in a team, delivery is key—specifically value delivery. Every leader, from the CEO down, cares about success. If you can show you're delivering success, you'll gain trust and empowerment to act. Sometimes that means taking a leap of faith and doing things differently, even without being told to. - Benjamin??

Another phrase that I would often use - better to beg forgiveness than ask permission. - Jon

Here’s an example of a small change that made a difference: Documenting on-call issues as a friction point allowed the team to quickly point to documentation instead of repeatedly answering the same questions. Initially, the team saw an increase in time spent on call as they built up documentation, but a significant net decrease followed it. More importantly, the on-call experience improved, becoming less frustrating for engineers.

In my case, we saw a steep decline in time once people could simply say, 'Look here,' and that was the end of the request. This is a small change we made to address how engineering concentration time for deep work was limited. By addressing this, we saw improvements not only in the metrics but also in DORA metrics—our SDLC started running more smoothly as people weren’t constantly pulled away. Building up data and showing these results gives you the chance to tackle bigger issues. Start small, make the changes you can control, experiment, and see how they impact the numbers, then try something else. - Benjamin??

Improving developer experience internally appears to be the best environment for building confidence and autonomy in your team. Why? Because it provides the quickest cycle for the measure-act-learn loop, with developers as the users themselves. It’s also the safest place to start building confidence and autonomy—developers provide feedback, make changes, and enjoy better experiences.

Repeat. Quarterly.??

Most places I've seen have DevEx surveys quarterly, which is a reasonable cadence since we're all trying to deliver things, and carving out time for changes can be a challenge. - Benjamin??

There’s great strength in repetition. By demonstrating success quickly and often, and delivering even small wins regularly, you build confidence in your dev team, leading to the autonomy and empowerment that are highly desired and often hard to achieve.

For more DevEx tips, check out these talks:

Beyond DevEx Metrics: How to Define and Achieve High-Impact Goals? with Chas Mastin, Sr. Engineering Manager at Google - save your spot!?

How Focusing on Better DevEx Frees Developers to Do What Matters Most? featuring Ben Darfler, Director of Engineering at Honeycomb