Sticky-note characters gather around a chart on a board, adding notes to explain a dip in the line

From numbers to actions: the limits of metrics, and how to use them in a retro

Metrics show what changed, not why, and they decay once they become targets. The limits of DORA metrics, Goodhart's law, and a loop that turns numbers into action.

Metrics tell you what changed. They don’t tell you why, and they stop meaning much once they become targets. Use system metrics to pick the question, ask the team how it experiences the problem, find the cause together in a retrospective, agree one or two actions, then measure again. DORA metrics are still worth tracking, and DORA itself treats them as a starting point for that conversation, not the answer.

Earlier chapters of this agile metrics guide cover what to measure: DORA, SPACE, developer experience, flow, scrum and health checks and maturity models. This one covers what goes wrong when you use them, and what to do with the numbers instead.

Where metrics run out

They show what, not why

When DORA announced its 2025 report on the Google Cloud blog, its authors wrote: “Simple software delivery metrics alone aren’t sufficient. They tell you what is happening but not why it’s happening.” That is the core limitation of any system metric. Change lead time rising from two days to five is a fact. Whether the cause is a growing review queue, a flaky test suite, or two senior engineers out on leave is something only the team knows. The number tells you where to look. The people doing the work tell you what you’re looking at.

Targets corrupt the number

Economist Charles Goodhart made the original observation, about monetary policy, in a paper for a 1975 Reserve Bank of Australia conference. The version usually quoted is anthropologist Marilyn Strathern’s, from a 1997 article in the European Review: “When a measure becomes a target, it ceases to be a good measure.”

Donald T. Campbell put the social version more sharply in 1976, in Assessing the Impact of Planned Social Change: “The more any quantitative social indicator is used for social decisionmaking, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.” Campbell’s point goes further than Goodhart’s: the target doesn’t only spoil the number, it changes the work.

DORA’s metrics guide lists this among its common pitfalls. Setting the metrics as goals, it says, “increases the likelihood that teams will try to game the metrics”, and it gives the example of a mandate that every application deploy multiple times a day. A team under that mandate can split one release into five deploys, and the dashboard improves while nothing else does. The same thing happens to velocity, which the velocity verdict covers in detail.

Individual leaderboards

Delivery metrics describe a team’s system. DORA’s guide to empowering delivery teams says: “Software delivery performance is not an individual measure.” Its example: “an individual’s commit velocity could be constrained by slow code reviews, rather than a lack of skill.” A leaderboard of commits, pull requests or story points per person ranks people on the parts of the system they don’t control, and it teaches them to optimize their own number over the team’s flow. DORA’s advice is to “apply generous levels of curiosity to find the causes of performance differences within the system” before looking at individuals. The SPACE framework shows how to keep individual, team and system levels apart.

Vanity dashboards

A dashboard can be accurate and still useless. The common pattern: many metrics, blended across teams, refreshed daily, discussed by nobody. DORA warns against several parts of it. Its metrics guide says the metrics “are best suited for measuring one application or service at a time.” Its guide to empowering delivery teams says directors should “avoid reductive aggregation across teams or comparison across different contexts”, and “The best use of the DORA metrics is to track the same application over time.”

Survey fatigue

Perception data has its own failure mode. Ask people too often and they stop answering, or answer on autopilot. A 2004 study of college students by Porter, Whitcomb and Weitzer found that “administering multiple surveys in one academic year can significantly suppress response rates in later surveys.” Team surveys aren’t student surveys, but the same risk plausibly applies. A survey that produces no visible change also gives people little reason to answer the next one. Keep surveys short, space them out, and show what changed since the last one before asking again.

The loop: from a number to a change

The fix for all five problems is the same: treat metrics as the first step of a loop that ends in a change the team makes, then check whether the change worked.

Team conversation 1 2 3 4 5 System metrics Ask the team Find the why Agree 1–2 actions Re-measure what changed health check retrospective owner and date same metrics numbers show what; people explain why
System metrics show what changed, the team explains why, one or two actions follow, and the same metrics show whether the actions worked. Skip the middle and the dashboard never changes anything.
  1. Start with system metrics. Pick the one trend that moved or matters most this period: change lead time, change fail rate, cycle time, carry-over. One, not twelve.
  2. Ask the team. A short health check or a few targeted questions show how people experience the problem. Is review slow because reviewers are overloaded, or because changes are too large to review? Perception data narrows the search.
  3. Find the why. A retrospective is where the team connects the number to what actually happened, using the techniques below.
  4. Agree one or two actions. Each with an owner and a date, and each tied to the metric you started with. More than two, and they tend to compete for attention. The Follow-Through Index covers what makes retro actions actually get done.
  5. Re-measure. Look at the same metric, with the same definition, after enough time for the change to show. If it didn’t move, that’s information too: the cause was elsewhere, or the action didn’t happen.

How to bring the numbers into the room

Numbers dropped into a retro without preparation can produce one of two reactions: defensiveness or glazed eyes. A little preparation avoids both.

  • One screen, not a dashboard tour. Three to five metrics, each with its definition, its source and the date range. If you need to scroll, you’ve brought too much.
  • Trends, not snapshots. Show six to twelve periods. A single number invites argument about whether it’s good; a trend invites the useful question of what changed.
  • Pair speed with stability. DORA advises choosing “multiple metrics, including some with a healthy amount of tension between them.” Show deployment frequency next to change fail rate, so nobody reads a speed-up without its cost.
  • Let the team annotate. Before any discussion, ask people to mark events on the timeline: the incident, the migration, the release freeze, the week half the team was out. The annotations are often the start of the explanation, and they put the team’s knowledge on the same screen as the data.
  • Frame it as the system, not the people. DORA’s culture guidance is to “Hold blameless postmortems”, because “By removing blame, you remove fear.” The same applies to a metric review, which is what the Retrospective Prime Directive asks of a retro. Our team dynamics guide covers why this matters for how a team works together.

In TeamRetro, a facilitator can paste the numbers, or a dashboard screenshot, into a retrospective’s or health check’s Context field (rich text and one image), which everyone sees on joining; the public API and MCP server can set it on an existing meeting.

Root-cause techniques that turn a metric into a cause

A trend tells you where to look. These two techniques help a team get from the symptom to something it can change.

5 whys

Ask “why?” until you reach a cause you can act on. The Institute for Healthcare Improvement’s 5 whys tool describes it as: “When a problem presents itself, ask ‘Why?’ five times until you reach the root cause,” and attributes it to Taiichi Ohno of the Toyota Production System.

A worked example, starting from a metric:

  1. Why did change fail rate rise this quarter? Most failures were in the payments service.
  2. Why there? Its deploys broke on configuration, not code.
  3. Why configuration? Config changes skip the staging environment.
  4. Why do they skip staging? Staging config drifted months ago, so people stopped trusting it.
  5. Why did it drift? Nobody owns it.

The action is now concrete: assign an owner for staging config and resync it. Note the limit IHI names: a problem can have more than one root cause, and people who see different parts of the system may answer the whys differently. That’s an argument for running it as a team, not for skipping it.

Fishbone (Ishikawa) diagram

When a metric has several plausible causes, a fishbone diagram spreads them out before the team picks one. IHI describes the cause and effect diagram as a tool “to explore and display the possible causes of a certain effect”, also known as an Ishikawa or fishbone diagram. The classic categories in IHI’s toolkit are Materials, Methods, Equipment, Environment and People. For a software team, swap in categories that fit: code, pipeline, process, environment, people, dependencies. Put the metric trend at the head, fill the bones, vote on the most likely cause, then use 5 whys on that one. The Fishbone (Ishikawa) Retrospective template sets this up as a retro format.

Frequently asked questions

Are DORA metrics still relevant?

Yes. DORA still publishes and refines them: its current guide lists five metrics in two groups, throughput and instability, and its 2025 report still analyzes both. What has changed is the framing. DORA says simple delivery metrics alone aren’t sufficient, because they tell you what is happening but not why. Use them as a starting signal for a team conversation, not as a scorecard.

What are the limitations of DORA metrics?

They describe outcomes of the delivery system, not causes, so they can’t tell you why a number moved. They suit one application or service tracked over time, not comparisons across teams with different systems. They don’t measure individuals, and DORA says delivery performance is not an individual measure. They don’t capture product value or how the work feels to the team. And once they become targets, teams game them, which DORA itself warns about.

What is Goodhart’s law in software metrics?

Goodhart’s law is usually quoted in Marilyn Strathern’s 1997 wording: when a measure becomes a target, it ceases to be a good measure. In software, it means a metric like deployment frequency or velocity gets pushed up by changing behavior around the number, such as splitting deploys or inflating estimates, without the underlying delivery getting better.

How do you use metrics in a retrospective?

Bring a one-screen summary of a few metrics as trends, not snapshots. Ask the team to mark what happened on the timeline, then pick the one trend that matters most and dig for the cause with 5 whys or a fishbone diagram. Agree one or two actions with an owner and a date, and look at the same metric again at a later retro to see whether it moved.