Developer experience and productivity metrics: DevEx, DX Core 4, and what's measurable
No single number measures developer productivity. Pair team outcomes with developer experience metrics for feedback loops, cognitive load and flow state.
There is no single developer productivity metric, and the ones that try to be one usually count effort or output. The approach the research supports is to measure team outcomes alongside developer experience: how fast feedback arrives, how much mental load the work carries, and how often people get to focus, asking developers and reading system data side by side.
This chapter of the agile metrics guide covers the 2023 debate, DevEx, DX Core 4, and how to collect experience data without wasting anyone’s time.
The 2023 productivity debate
In August 2023, McKinsey published “Yes, you can measure software developer productivity”. It called software development “perennially undermeasured” next to sales or customer operations and proposed metrics at system, team and individual level. On top of DORA and SPACE, it added four measures:
- Inner/outer loop time spent: coding, building and unit testing versus integration, release and deployment. Top tech companies, it says, aim for up to 70 percent inner-loop time.
- Developer Velocity Index benchmark: a McKinsey survey of technology, working practices and organizational enablement, benchmarked against peers.
- Contribution analysis: individuals’ contributions to a team’s backlog, drawn from tools such as Jira and normalized with a proprietary algorithm.
- Talent capability score: the organization’s knowledge, skills and abilities against capability maps.
The article reported use at nearly 20 companies, and itself warned against lines of code and commit counts.
Kent Beck and Gergely Orosz replied in The Pragmatic Engineer (part 1, part 2, August 2023). They model software work as a chain: effort, output, outcome, impact. Their critique is that “nearly every one of its custom metrics which differ from DORA and SPACE metrics, measure effort or output,” and that “the act of measurement changes how developers work.” The earlier in the chain you measure, the easier it is and the more likely you are to cause unintended consequences. Their advice: measure team outcomes and impact, and use effort and output to debug problems, not as the score.
The two pieces agree on more than the headlines suggest. McKinsey warns that a single metric, or too simple a set, invites gaming, and Beck and Orosz accept the executive need: CEOs and CFOs see sales and recruitment measured against targets and ask why engineering can’t be. The real disagreement is over individual-level measurement and how much weight effort and output should carry.
Output, outcome and experience
Beck and Orosz’s chain sorts any metric, with one row added for experience:
| Layer | What it tells you | Examples | Main risk |
|---|---|---|---|
| Effort and output | How much activity happened | Commits, pull requests, story points | Easy to inflate; says nothing about value |
| Outcome | Whether delivery works | Deployment frequency, change fail rate | Shows a slowdown, not its cause |
| Impact | Whether the work paid off | Revenue, cost reduction, share of time on new capabilities | Hard to attribute to one team |
| Experience | What gets in developers’ way | Satisfaction with build times, ease of delivery, focus time | Needs careful question design |
Beck and Orosz place every DORA metric in outcome or impact; the DORA metrics chapter covers them. SPACE recommends metrics from at least three of its five dimensions, at least one of them perceptual. Experience is the row that’s easiest to skip, and SPACE suggests it may move first: “it is possible that satisfaction could serve as a leading indicator for productivity.”
DevEx: three dimensions of developer experience
The DevEx framework comes from Abi Noda, Margaret-Anne Storey, Nicole Forsgren and Michaela Greiler, “DevEx: What Actually Drives Productivity”, ACM Queue 21(2), 2023. It defines developer experience as “how developers feel about, think about, and value their work,” and distills more than 25 sociotechnical factors from the authors’ earlier research into three dimensions.
- Feedback loops: “the speed and quality of responses to actions performed.” Waiting on a build, a test run or a code reviewer all count. The levers are faster build and test tooling and smoother hand-offs.
- Cognitive load: the mental processing a task requires. Poorly documented code and systems raise it; well-organized code, clear documentation and self-service tools lower it.
- Flow state: energized, uninterrupted focus. The paper suggests clustering meetings, avoiding unplanned work and batching help requests.
The paper calls it a common misconception that developer experience is mostly about tools: “human factors such as having clear goals for projects and feeling psychologically safe on a team have a substantial impact on developers’ performance.” That’s where developer experience meets psychological safety, and why a program that only buys faster tooling can miss what slows a team down.
Example measures from the paper
Each dimension gets perceptual measures (what developers report) and workflow measures (what systems record):
| Dimension | Perceptual measures | Workflow measures |
|---|---|---|
| Feedback loops | Satisfaction with automated test speed and output; with time to validate a local change; with time to deploy a change to production | Time to generate CI results; code review turnaround time; deployment lead time |
| Cognitive load | Perceived complexity of the codebase; ease of debugging production systems; ease of understanding documentation | Time to get answers to technical questions; manual steps required to deploy a change; frequency of documentation improvements |
| Flow state | Perceived ability to focus and avoid interruptions; satisfaction with clarity of task or project goals; perceived disruptiveness of being on call | Number of blocks of time without meetings or interruptions; frequency of unplanned tasks or requests; frequency of incidents requiring team attention |
Above these sit three KPIs, the paper’s “North Star metrics”: overall perceived ease of delivering software, employee engagement or satisfaction, and perceived productivity.
Why you need both kinds of measure
The paper’s example: code review turnaround can look fast in the data and still feel disruptive if reviews keep interrupting work, while developers can feel fine about builds that the data shows are slower than they need to be. Neither source alone gives the full picture.
What the follow-up study found
“DevEx in Action” (Forsgren, Kalliamvakou, Noda, Greiler, Houck and Storey, ACM Queue 21(6), 2024) surveyed 219 developers. Those with significant time for deep work felt 50 percent more productive than those without it; those with a high understanding of their code felt 42 percent more productive; those reporting fast code review turnaround felt 20 percent more innovative. The authors state the limits: the study was cross-sectional, and participants worked at companies already engaged with DX, a developer-experience vendor. The outcomes are also self-reported. Three of the four original DevEx authors (Noda, Greiler and Storey) held roles at DX when it was published.
DX Core 4
DX Core 4 is DX’s own framework, announced in December 2024 by Abi Noda, who developed it with Laura Tacho and input from authors of the DevEx and SPACE papers. DX’s research page calls it “a unified framework for measuring developer productivity that encapsulates DORA, SPACE, and DevEx,” in four dimensions:
| Dimension | Key metric | Secondary metrics |
|---|---|---|
| Speed | Diffs per engineer (PRs or MRs), not at individual level | Lead time; deployment frequency; perceived rate of delivery |
| Effectiveness | Developer Experience Index (DXI), developed by DX | Time to 10th PR; ease of delivery; regrettable attrition (organizational level only) |
| Quality | Change failure rate | Failed deployment recovery time; perceived software quality; operational health and security metrics |
| Impact | Percentage of time spent on new capabilities | Initiative progress and ROI; revenue per engineer and R&D as percentage of revenue (organizational level only) |
The dimensions counterbalance each other, since changes to speed “may negatively affect others (e.g., quality or effectiveness).” DX says diffs per engineer “requires caution” and sets three preconditions: counterbalance it with the DXI, tie no targets or rewards to it, and communicate how it’s used. DX also recommends starting from self-reported baselines rather than waiting for full system instrumentation.
Two things to weigh. The effectiveness key metric is DX’s own index, so a team not using DX needs its own experience measure; the three DevEx KPIs above are a workable substitute. And DX says Core 4 has been used at more than 300 companies and cites a 3 to 12 percent increase in engineering efficiency; both figures are vendor-reported. Core 4’s most portable idea is its shape: one speed, one experience, one quality and one impact measure, read together.
How to collect experience data from developers
The DevEx paper calls surveys “a crucial tool,” quick to run and accurate when designed well. TeamRetro’s Developer Experience Survey (DevEx) is one built on the paper’s three dimensions: 12 statements, rated privately and discussed as a team.
Question design. The paper says questions “should be based on well-defined constructs and be rigorously tested in interviews for consistent interpretation.” In practice: one idea per question, the same wording every round so trends mean something, and questions about the respondent’s own experience. SPACE explains the last point: asking whether the team is productive “is subject to blind spots,” while asking whether the person felt productive “is a useful signal.” Add one open question about what slowed people down most; that’s where specific causes show up.
Cadence. The paper recommends quarterly or semi-annual for most organizations, plus short transactional surveys at specific touchpoints (its example: when a CLI tool fails to install). eBay and Pfizer, its two case studies, both survey quarterly.
Anonymity. Report only anonymized, aggregate results at team or group level, as SPACE advises. Tell people up front who sees what, and keep to it.
Small teams. The paper warns that company-wide averages hide problems in small groups, such as mobile developers. Small teams have the opposite problem. On a six-person team, one person moving from a 2 to a 5 shifts the average by half a point, so a single round says little. Set a minimum group size before showing any breakdown, read the spread and the comments rather than the mean, judge trends over several rounds, and don’t rank small teams against each other.
Acting on results. The paper warns about survey fatigue and names its usual cause: “a lack of follow-up action.” Share results with the team within a few weeks, let it pick one or two things to change, and show what happened by the next round. Pfizer gives teams dedicated time and resources for local improvements; eBay’s DevEx team runs regular retrospectives with feature teams. The health checks and maturity models chapter covers when perception data beats system data.
Frequently asked questions
What are developer experience metrics?
They measure the friction developers meet in everyday work. The DevEx framework groups them into feedback loops, cognitive load and flow state, each measured by asking developers and from system data, plus overall KPIs such as perceived ease of delivering software.
How do you measure developer productivity?
With several measures at team level, never one number. Pair delivery outcomes such as the DORA metrics with developer experience data, and read them together. The SPACE authors recommend metrics from at least three dimensions, at least one of them perceptual.
What is DX Core 4?
A framework the company DX announced in December 2024 that combines DORA, SPACE and DevEx into four dimensions: speed, effectiveness, quality and impact. Its key metrics are diffs per engineer (not at individual level), DX’s Developer Experience Index, change failure rate, and percentage of time spent on new capabilities.
Should you measure individual developer productivity?
No. Software is built by teams, and individual activity counts are easy to game and miss work such as reviews and pairing. The SPACE paper advises reporting only anonymized, aggregate results at team or group level.
How often should you ask developers about their experience?
Quarterly or twice a year suits most organizations, according to the DevEx paper, with short transactional surveys at specific moments in between. Ask only as often as you can act on the answers.
Related reading
- Agile and engineering metrics: the hub, with every framework side by side.
- DORA metrics: the delivery outcomes to pair with experience data.
- The SPACE framework: five dimensions for choosing productivity metrics.
- From numbers to actions: taking a metric into a retro and out as a change.
- Is velocity a useful metric?: why an output count fails as a productivity measure.