AI Systematic Biases · retrieval and sources
A model can accept misleading user or document assertions instead of reliably separating useful external evidence from harmful or false assertions.
Human analogy: authority and source-cue effects. This is a behavioural analogy, not a claim that the model has the same mental mechanism.
Why it matters
In RAG, a retrieved document may look authoritative simply because the system labeled it as context. Source conflict becomes a model-behaviour problem, not only a retrieval problem.
Evidence status: strong recent evidence in tested families; not a claim about every current model
Evidence level: peer reviewed large scale multi model
Signals
Self-test
Build matched cases with correct, incorrect and conflicting assertions from user and document sources. Track whether the model follows source labels or evidence quality.
Record the exact model identifier, surface, system prompt, relevant settings and run date. A single surprising answer is an anecdote, not a bias measurement.
Mitigation
Applies to
rag and document assistants chat assistants coding and automation agents decision support
Dated model evidence
These findings describe the model or model set tested at the stated time. A newer release needs new evidence.
Most tested models relied more on document assertions than user assertions and many did not reliably distinguish helpful from harmful external information.
Sources
