Bias comparison · evidence reviewed

Surrogation vs Systematic Bias

Both can make a metric misleading, but the failure is different. Surrogation happens when people start treating a performance measure as though it were the goal or construct itself. Systematic bias is a consistent or predictable deviation in a measurement, estimate, sample, or process relative to a target or reference.

Surrogation – when a performance measure starts replacing the goalvsSystematic Bias – a measurement and statistical concept, not one cognitive bias

The shortest distinction

Ask whether the problem is that people are optimizing or interpreting the proxy as the goal, or that the measurement process itself is systematically shifted away from the target.

Question	Surrogation	Systematic Bias
Core problem	A measure replaces the underlying strategic construct in judgment or action.	A measurement or estimation process has a consistent or predictable deviation from its reference.
The metric can be accurate?	Yes. A measure can be calculated accurately and still be an incomplete proxy that people over-treat as the objective.	The issue is the deviation of the measurement or estimate itself, so accuracy relative to the reference is central.
Typical KPI example	A support team optimizes ticket-closure time and treats that score as customer service quality, even when repeat contacts rise.	A dashboard systematically undercounts a class of support contacts because the data pipeline excludes one channel.
Best check	Define the objective, list what the proxy misses, and test whether the number can improve while the objective gets worse.	Define the reference, then test calibration, sampling, collection and estimation for a directional error.

Decision diagnostic

Review the decision without letting the result rewrite the past.

1
Write the underlying objective and the metric as two separate statements.
2
Check whether the metric can improve while the objective stays flat or becomes worse. If so, Surrogation is a relevant risk when the metric starts replacing the objective.
3
Independently test whether the measurement itself is shifted relative to a reference because of calibration, sampling, collection or analysis. That is a systematic-bias problem.
4
Do not use one label to hide the other. A team can surrogate on a metric that is also systematically biased, so the behavioral and measurement failures may need separate fixes.

Same outcome, different bias

Three examples where the distinction matters.

Customer support

SurrogationAgents treat average handle time as the definition of good support and rush difficult cases to protect the score.

Systematic BiasCalls transferred through one system are missing from the dataset, so average handle time is systematically estimated from an incomplete population.

AI evaluation

SurrogationA team treats one benchmark score as the product's real-world usefulness and optimizes releases around that score.

Systematic BiasThe evaluation sample consistently overrepresents easy cases, shifting the estimate of real-world performance upward.

Education

SurrogationA program treats test scores as though they fully represent learning and narrows instruction around the tested material.

Systematic BiasA test or sampling procedure consistently disadvantages a subgroup relative to the intended construct or target population.

Retrospective review protocol

Separate forecast quality, decision quality, and learning.

Define the construct or goal in ordinary language before looking at the metric.
Document what the metric measures directly, what it only approximates, and what it leaves out.
Test the measurement process against an independent reference or validation source.
Look for actions that improve the metric without improving the construct.
Use different remedies for different failures: measurement correction for systematic error, and goal/proxy separation plus better decision processes for Surrogation.

Evidence

Reviewed sources behind this comparison.

Strategy Selection, Surrogation, and Strategic Performance Measurement Systems2013 · DOI 10.1111/j.1475-679X.2012.00465.x
Decreasing Operational Distortion and Surrogation Through Narrative Reporting2019 · DOI 10.2308/accr-52277
NIST Technical Note 1297, Appendix D1: Terminology1994

Comparison reviewed 2026-08-18. The individual bias pages contain their full evidence status and boundary conditions.