LogoLucyWorks
  • Lessons
  • Answers
  • Pricing
  • About
  • Ask Lucy
  • Book a call
LW·A49·Can AI do this?← All answers

What belongs on an AI procurement checklist?

At minimum: a structured brief, unit/date/blank-field verification, a two-platform divergence check, and a no-unverified-numbers rule. Each item traces to a recorded failure in the July 2026 PAIR-20 run.

A useful pre-flight checklist is short enough to run every time. In the July 2026 PAIR-20 run, four items would have caught the failures that appeared across platforms: a unit miss, wrong landed-cost numbers, a ranking contradiction, and stripped hidden costs. Each item below traces to one of those recorded outcomes.

Start with a structured brief, not a prompt. A structured brief names the fields that carry the decision — units, fees, contract dates, scope caps, and missing data — and asks the model to show its working. In the July 2026 run, the brief made outputs more inspectable but did not make them correct. The checklist value is auditability: a buyer can verify a normalized table against the source file.

Verify units, dates, and blank fields against the source document. In the July 2026 run, one platform carried a flat figure into a ranking despite the brief explicitly warning about a per-100-seats-per-year price. Failures appeared in unit conversion, cost layers, and the handoff from analysis to summary. Check every number that could change the recommendation by reading the model's final table against its own stated working.

Run the same task on two platforms and compare. In the July 2026 run, platforms failed in different places: one missed a unit normalization, another ranked the highest all-in total first, a third stripped hidden costs from a blended rate. A single output that looks tidy can still be wrong. A second platform exposes which parts of the answer are fragile.

Never forward a number you have not traced to source. In the July 2026 landed-cost task, all four platforms rebuilt the cost stack under a structured brief and none reached the case ground truth. The brief bought structure, not correctness. Any figure that leaves the desk should have a source cell reference the reviewer has checked — not just a plausible-looking total from the model.

Where this comes from

  • PAIR-20 July 2026 run of record — single dated runs, screenshots on file

Last checked Sun Jul 12 2026 00:00:00 GMT+0000 (Coordinated Universal Time). Evidence comes from dated, single-run platform sessions with screenshots on file — read each finding as “this happened,” not “this always happens.”

Work this yourself — from the course

Review a SaaS Master Service Agreement for Traps — Auto-Renewal, Price Escalation, Data PortabilityWalks through reviewing a SaaS master service agreement with AI to catch an auto-renewal notice trap, a price-escalation clause disguised as a cap, and a data-portability exit cost — then turning those findings into a legal question list, a VP briefing, and a vendor counter-proposal.

Related questions

  • What is a structured brief (vs. a prompt)?
  • Does a better prompt fix AI's procurement mistakes?

See what the platforms caught — and missed

Twenty procurement tasks, four AI platforms, real dated runs. Lesson 2 is free to read, no account needed.

Read the free lessonTraining for your team
LogoLucyWorks

AI training that ends with a working agent.

Product
  • The 20 Lessons
  • Answers
  • Ask Lucy
  • PAIR-20 benchmark
  • Reality Check
  • Pricing
  • Book a call
Company
  • About
Legal
  • Cookie Policy
  • Privacy Policy
  • Terms of Service
© 2026 LucyWorks. All Rights Reserved.