How often does AI fabricate facts in procurement files?
PAIR-20 does not give a stable frequency. In one July 2026 set of six single-run tasks, two baited tasks recorded fabricated facts: an invented Dravec expiry and a guessed Atlas box quantity.
There is no honest universal rate from the July 2026 PAIR-20 run. It was six procurement tasks, run once per platform, on dated free or standard web tiers. The result should be read as a field note: in this run, fabrication appeared twice, on the two tasks built to test whether a model would fill a missing or uncertain field.
In the supplier-risk-register task, Copilot's structured-brief run invented a Dravec backup-contract expiry. The file did not say Dravec's backup contract had expired, but the invented issue survived into the VP-ready summary.
In the MRO catalog task, Copilot wrote "Likely 100ct" for Atlas's unstated box quantity. The case file said the quantity varied by item and needed confirmation. The right answer was to mark the field as unknown, not turn it into a denominator for unit pricing.
Do not turn this into a platform average. PAIR-20 says the runs were single runs, not averages. A platform that missed a trap might catch it on another run, and one that caught it might miss it next time. The useful answer is narrower: fabrication showed up in this July 2026 run, and it showed up in places where procurement teams depend on source discipline.
Treat blank fields as the danger zone. Contract expiry, source status, pack quantity, and replacement lead time are not safe to infer. If the file is blank, the output should say so.
Review for added facts, not just missing ones. A missing risk can be found by re-reading the source. An invented expiry or guessed pack size sends the team after a false issue. Before a register, catalog merge, or recommendation leaves your desk, scan every decisive field and ask: where in the source file did this fact come from?
Where this comes from
Last checked Sat Jul 04 2026 00:00:00 GMT+0000 (Coordinated Universal Time). Evidence comes from dated, single-run platform sessions with screenshots on file — read each finding as “this happened,” not “this always happens.”
Related questions
See what the platforms caught — and missed
Twenty procurement tasks, four AI platforms, real dated runs. Lesson 2 is free to read, no account needed.