The simple question: is the second review independent?
Many AI workflows evaluate the same answer more than once. A first model gives a score, a revision happens, and another judge scores again. It is easy to treat the second score as fresh evidence.
That assumption becomes weaker when the later judge can see the earlier score. The new evaluation may be responding to both the answer and the evaluation history.
What the recent study tested
A 2026 study compared fixed texts under three prompt conditions: no evaluation metadata, revision framing without a score, and metadata that included a previous score. The text being judged stayed fixed, so the previous score did not add new evidence about quality.
Across eight tested models, the authors report systematic movement associated with the prior-score condition in the main analysis. They also report cases where this context blocked error corrections or changed correct categorical judgments.
What this does not prove
The result is not a licence to say that every AI judge is anchored in every task. Models, prompt formats, scoring scales and domains can all change the effect.
It also does not establish that the internal mechanism is the same as human anchoring. The safer description is behavioural: under the tested conditions, evaluation history changed later model judgments.
A useful design check for real systems
If a later review is supposed to be independent, hide the previous score and verdict during that pass. Then compare the blind result with a history-aware result instead of assuming that a warning such as ‘judge independently’ is enough.
This matters for content grading, agent self-review, quality gates and iterative scoring pipelines. Several review passes do not automatically equal several independent opinions.
What this changes in our project
We added Evaluation History Contamination as an AI-era research track. It is a working label for a practical failure mode, not a new established cognitive bias.
We also published AI Judge History v1, a provider-neutral benchmark that holds the answer and rubric fixed while comparing blind, history-framing and prior-score conditions. The protocol is public before any Cognitive Biases result exists.
Sources we reviewed
- Anchoring Bias in LLM-as-a-Judge Systems: Prior Scores Compromise Evaluation Independence (2026, conference paper / arXiv record)
- Mediocrity is the key for LLM as a Judge Anchor Selection (2026, peer-reviewed conference paper)