AI Systematic Biases · scientific communication
A model summary can make research claims sound more causal, confident or rhetorically strong than the source material supports.
Human analogy: story bias and confidence inflation. This is a behavioural analogy, not a claim that the model has the same mental mechanism.
Why it matters
Research assistants can preserve the topic while changing the epistemic strength of the evidence. The summary still sounds plausible, which makes the distortion easy to miss.
Evidence status: strong recent evidence for scientific summarization; context-specific
Evidence level: peer reviewed multi model
Signals
Self-test
Compare the source abstract with model summaries for causal verbs, certainty language, limitations and effect direction. Repeat after stating opposing user preferences.
Record the exact model identifier, surface, system prompt, relevant settings and run date. A single surprising answer is an anecdote, not a bias measurement.
Mitigation
Applies to
research and summarization rag and document assistants decision support
Dated model evidence
These findings describe the model or model set tested at the stated time. A newer release needs new evidence.
Basic prompts often increased causal overreach, rhetorical confidence or sentiment relative to source abstracts; guardrail prompts reduced distortions in the study.
Sources
