LogoLucyWorks
  • Lessons
  • Answers
  • Pricing
  • About
  • Ask Lucy
  • Book a call
LW·A13·Can AI do this?← All answers

Does a better prompt fix AI's procurement mistakes?

No. In PAIR-20's July 2026 run, structured briefs made outputs easier to audit, but did not prevent a unit miss, wrong landed-cost numbers, a ranking contradiction, or stripped hidden costs.

A better prompt helps, but the July 2026 PAIR-20 run does not support treating it as a fix. The structured brief usually made the answer easier to inspect: it asked for normalized fields, visible assumptions, and a decision table. That is useful procurement behavior.

It did not make the outputs correct.

In T01, the brief explicitly warned the model to normalize a price quoted per 100 seats per year. ChatGPT corrected the unit issue on that brief. Claude still carried the flat figure into the ranking. In T03, all four platforms rebuilt the landed-cost stack under the brief, but none reached the case ground truth. In T05, ChatGPT computed the consulting-bid math correctly, then ranked the highest all-in total first. Gemini showed the right all-in figure in the narrative and then stripped hidden costs out of the effective blended rate.

That pattern is the point. The brief bought structure, not correctness.

Use the brief to make the work auditable. Ask for units, excluded documents, mandatory fees, scope assumptions, cost build-up, and final ranking. A vague prompt may still produce a useful first pass, but it leaves more of the work hidden.

Do not skip the verification step. The failures in the July run were not always obvious. They appeared in unit conversion, cost layers, clause interaction, table ranking, and the handoff from analysis to summary. Those are exactly the places a buyer is tempted to trust because the output looks tidy.

Treat prompt quality as one control. A procurement team should write better briefs because they create a better paper trail. The team should not expect the brief to remove the need for a human check. In this run, the safer habit was to read the model's final table against its own working and verify the few inputs that could change the recommendation.

Where this comes from

  • PAIR-20 July 2026 run of record — single dated runs, screenshots on file
  • Procurement AI Reality Check, July 2026

Last checked Sat Jul 04 2026 00:00:00 GMT+0000 (Coordinated Universal Time). Evidence comes from dated, single-run platform sessions with screenshots on file — read each finding as “this happened,” not “this always happens.”

Related questions

  • What is a structured brief (vs. a prompt)?
  • Which AI platform is best for procurement work?

See what the platforms caught — and missed

Twenty procurement tasks, four AI platforms, real dated runs. Lesson 2 is free to read, no account needed.

Read the free lessonTraining for your team
LogoLucyWorks

AI training that ends with a working agent.

Product
  • The 20 Lessons
  • Answers
  • Ask Lucy
  • PAIR-20 benchmark
  • Reality Check
  • Pricing
  • Book a call
Company
  • About
Legal
  • Cookie Policy
  • Privacy Policy
  • Terms of Service
© 2026 LucyWorks. All Rights Reserved.