Systematic and random error are different problems
Measurement standards distinguish systematic error from random error. Random error varies unpredictably across repeated measurements. Systematic error has a consistent or predictable component relative to a reference value.
That distinction matters because repeated observations can make an average more precise while the whole process is still centred on the wrong value. Precision and accuracy are not the same thing.
The word bias does not make it a cognitive bias
Statistics, measurement science and research methods use the word bias for many systematic deviations. A sampling procedure can be biased. An estimator can be biased. An instrument can have measurement bias. None of these labels, by themselves, identify a psychological mechanism in a person.
Human judgment can also be systematically biased, but the useful scientific question is then which specific process or construct explains the pattern. Calling it only 'systematic bias' is a description of the deviation, not a mechanism.
More data can make the wrong answer look more certain
If every observation comes through the same miscalibrated instrument or the same unrepresentative sampling process, collecting more observations can reduce random uncertainty around a systematically shifted estimate. The result can look stable and precise while remaining wrong relative to the target.
That is why large datasets do not remove the need for calibration, validation, representative sampling and checks against independent sources or methods.
A practical review starts with the reference
First define what the measurement or estimate is supposed to represent. Then ask where the process could push the result consistently high, low, early, late or toward one group or condition.
Useful checks depend on the system: calibration against a known reference, holdout or validation data, alternative instruments, representative sampling, sensitivity analysis, process controls, or independent replication.
What this changes in our library
We keep Systematic Bias because people and agents encounter the term in measurement, data and research. But it is now explicitly classified as a Measurement concept rather than a cognitive bias or generic psychological phenomenon.
It remains excluded from the Decision Audit. The right response is to inspect the measurement or estimation process, not to diagnose a person with a mental bias called 'Systematic Bias'.
Sources we reviewed
- International vocabulary of metrology — systematic measurement error and measurement bias (2008, international metrology vocabulary)
- NIST Technical Note 1297, Appendix D1: Terminology (1994, measurement uncertainty guidance)
- Bias, Statistical (2023, NIST glossary definition)