Decision context

Reviewing KPIs & proxy metrics

Use these evidence-reviewed lenses when a team is using scores, dashboards, targets, benchmarks, or AI evaluations to represent a broader objective.

Use this context when

Start from the situation, not a label.

A KPI is improving, but people disagree about whether the underlying service, product, learning, safety, or business outcome is actually better.
A target is tied to incentives and teams have found ways to improve the number that do not clearly improve the goal.
A dashboard or benchmark is becoming the main language of the review while the underlying objective is harder to discuss.
A benchmark or model score is being treated as the main evidence that an AI system is useful in real work.
A successful result is being used to justify the metric or target even though the quality of the measurement and decision process was never tested.
Teams mainly look for examples that support the current KPI and rarely ask what evidence would show that the measure is a poor proxy.

Evidence-reviewed lenses

4 patterns worth testing, not diagnosing.

supported in strategic performance-measure settingsEvidence

Surrogation – when a performance measure starts replacing the goal

Are we treating the measure as evidence about the objective, or have we started treating the measure as the objective itself?

Surrogation is a domain-specific management-accounting construct: a measure that was designed to represent a strategic objective can start to be treated as though it were the objective itself. It is closely related to metric fixation and proxy problems, but those broader labels should not be treated as exact synonyms without checking the setting and mechanism.

Read evidence review Use in Decision Audit →

well-supported for numerical judgments; strength depends on anchor type and contextEvidence

Anchoring Effect – when a starting number pulls later estimates toward it

Which target, benchmark, baseline, or first score is shaping later judgments before we build an independent estimate?

Anchoring is a well-supported effect in which an initial numerical value can pull a later estimate toward it. A 2026 meta-analysis covering 2,601 effect sizes found a large overall effect, but also substantial variation across studies. The effect should not be treated as a rule that every number changes every judgment: incidental anchors, anchors from a different dimension, clearly random values, incentives, and some debiasing conditions were associated with smaller or null effects.

Read evidence review Use in Decision Audit →

well established, broad constructEvidence

Confirmation Bias – when a belief shapes how evidence is searched and judged

What evidence would show that this KPI is a poor proxy, and have we actively looked for it?

Confirmation bias is an umbrella label for several ways existing beliefs or hypotheses can influence information search and interpretation. It should not be reduced to one behaviour such as reading only agreeable news, and a preference for confirming tests is not irrational in every task or environment.

Read evidence review Use in Decision Audit →

replicatedEvidence

Outcome Bias – when you judge a decision by its result, not its quality

Are we judging the quality of the metric and decision process mainly from whether the final result happened to be good or bad?

Outcome bias occurs when knowledge of a result changes how people evaluate the quality of a decision even when the information available at the time of the decision is held constant. Outcomes can still be relevant for learning, so the error is not 'never look at results'; it is using luck or hindsight as if it had been available to the original decision-maker.

Read evidence review Use in Decision Audit →

Decision workflow

Turn the context into observable questions.

1
Write the underlying objective in ordinary language before opening the dashboard. Describe what success means without using the KPI name.
2
For each important measure, list what it captures directly, what it only approximates, and at least one important dimension it misses.
3
Ask how someone could improve the number while making the real objective worse. If that path is realistic, add a guardrail, companion measure, validation check, or narrative review.
4
Identify the first target, benchmark, baseline, or score shown in the review. Rebuild important estimates from independent evidence before letting that number become the reference point.
5
Write what evidence would make you change or retire the metric before the next review. Search for that evidence, not only for examples that defend the current scorecard.
6
Judge the metric design and the eventual outcome separately. A good outcome does not prove the proxy was well chosen, and a bad outcome does not prove every part of the measurement system was wrong.
7
For AI benchmarks and automated ratings, validate against representative real tasks and human outcomes instead of assuming one convenient score is the product objective. Check the measurement process separately when data coverage, sampling, calibration, labels, or exclusions may be systematically distorted.

Open a blank Decision Audit

Practice this context

Can you match the question to the lens?

Open 4 practice exercises