within-participant randomized pilot; 6 trials per participant.
Continue from here
Preregistered pilot · 2026-08-21
Does AI Advice Order Change Numerical Judgment?
Article options
When the same standardized AI-labelled numerical advice is shown before rather than after an independent estimate, does the final human estimate align more strongly with that advice?
Status: preregistered, no project result yetThe protocol and analysis rule are public before data collection. The tracker must stay below result stage until real anonymized data and a result artifact are published.
Hypothesis
Order, not merely the presence of advice.
Within participants, final estimates will show greater normalized alignment with the standardized AI advice in advice-first trials than in independent-estimate-first trials.
3 trials per condition for every completed participant, randomized across tasks.
3 low and 3 high standardized anchors per completed participant, randomized independently of advice order.
40 complete adult participant records.
Primary outcome
One rule fixed before looking at data.
normalized anchor alignment: (finalEstimate - referenceValue) / (shownAnchor - referenceValue)
0 is exactly the reference value; 1 is exactly the shown anchor. Values can be below 0 or above 1 and are not clipped.
Participant contrast: mean(advice-first normalized anchor alignment) - mean(independent-first normalized anchor alignment).
A positive contrast supports greater alignment with the AI-labelled anchor when advice is shown first.
Secondary outcomes
Useful context, not a fishing expedition.
- Absolute relative error of the final estimate against the reference value by advice-order condition.
- For independent-first trials only: movement from the initial estimate toward the shown advice.
- Final confidence by advice-order condition.
- Sensitivity analysis excluding trials where the participant reports already knowing the exact answer.
- Trial completion time as a descriptive process measure.
Missing data: Primary analysis uses completed sessions with all six final estimates. Do not impute missing trial responses.
Outliers: Do not remove finite numerical responses merely for being extreme. Report a sensitivity summary using medians alongside means.
Claim rule: Treat this as an exploratory pilot. Do not promote the AI Anchoring Loop tracker to result stage unless anonymized data, the scoring output, limitations and an explicit result artifact are published.
Participant privacy
No silent data collection.
The experiment runs in the browser. It creates a random local participant ID and a downloadable JSON record. It does not upload responses, collect names, emails, IP addresses or device fingerprints.
- Adults only (18+).
- No sensitive personal questions are asked.
- No response is uploaded automatically by the website.
- Participants can stop by closing or resetting the page before sharing a record.
- This public pilot is not presented as an IRB-reviewed academic human-subjects study.
- The task measures response patterns in this instrument; it does not diagnose a participant as biased or irrational.
Starting evidence
Why test this?
- How was my performance? Exploring the role of anchoring bias in AI-assisted decision making (2025)
Two controlled experiments with managers found that AI recommendation source and anchor magnitude interacted in performance ratings. - Anchoring bias in large language models: an experimental study (2025)
A multi-model study found numerical LLM outputs were sensitive to anchor hints and several simple prompting mitigations were insufficient. - The inclusion of anchors when seeking advice: Causes and consequences (2024)
Preregistered advice-seeking studies show that including anchors can introduce shared error into later judgments.
Planned limitations
Write them before they become excuses.
- The advice is a standardized AI-labelled stimulus, not live advice generated by a model during the session.
- The pilot uses factual estimation tasks rather than high-stakes real-world decisions.
- A convenience sample limits population inference.
- Participants may know some reference values already; this is measured only by self-report.
- The pilot is designed to estimate an order effect and test the research workflow, not to establish a new cognitive-bias category.
Participant note: the full machine-readable preregistration contains the task stimuli. If you plan to participate, complete the instrument before inspecting the study JSON.