About 5% of work tasks have real AI benchmarks. Check yours before you sign.
A vendor puts up a slide: the model is right 95% of the time.
That number leaves three questions open.
What was it tested on? Mercor recently mapped 29 public AI benchmarks against the US government's catalogue of work tasks. About 5% of those tasks have meaningful coverage, most of it in software, finance and law. If you run a clinic, a warehouse or a service fleet, assume the test was someone else's work until the vendor shows you what was in it.
What happens the other 5% of the time? A wrong line in an internal summary costs someone a minute. A wrong quote sent to a customer can cost the account. The leaderboard does not know which of the two you are buying.
Who checks the output? A system that is right 95% of the time but needs every result reviewed can be worth less than one that is right 90% of the time, handles the routine cases and sends the rest to a person.
None of this means the product won't help. It means the score can't tell you, and the person presenting it is usually paid more if you believe it can.
I wrote up what to ask, and how to tell a real answer from a rehearsed one.
The AI score on the slide was probably measured on someone else's work
What a new map of AI benchmarks tells an owner about the numbers in a vendor's pitch.
Many AI sales decks have a slide with a score on it. The model "beats human experts" on some benchmark, or completes 87% of tasks, or ranks first on a leaderboard with a name you have never heard. The owner across the table has no way to tell what the number measured, and the person presenting it is rarely asked.
Mercor, a company that supplies expert reviewers to AI labs, recently tried to answer that question for the whole economy. It picked 29 public benchmarks it judged rigorous and economically relevant, then mapped each test prompt onto O*NET, the US government's catalogue of occupations and the tasks people do in them. By its count, about 5% of those tasks have meaningful coverage, and roughly 90% of working hours are poorly measured. The coverage that does exist piles up in software, finance and law.
If you run a distribution company, a clinic, a hotel or a field service business, the working assumption is simple. Until the vendor shows you what the test contained, assume the score on the slide was measured on someone else's work.
That does not mean the product won't help you. It means the number cannot tell you whether it will.
Why the gaps sit where they sit
Benchmarks get built where answers are easy to grade. Code is the cleanest example: run the tests and the code either passes or it doesn't. A contract clause can be checked against an expert's markup. Now try writing a test for whether a system calmed down a customer whose delivery arrived broken, rescheduled three technicians around a no-show without losing the afternoon's margin, or noticed that the owner was asking for the wrong report.
Those jobs need context, judgment and a result that shows up weeks later. They are expensive to grade, so they get graded less. What gets measured then gets optimized, models improve on it, and the measured work starts to look like the work AI is good at. The map may say as much about what researchers can score as about what AI can do.
Mercor's own method calls for the same caution, and Mercor says so. The 29 benchmarks are its selection, and it calls them an incomplete sample. A task counts as covered at five benchmark prompts, which may capture most of what varies in issuing a standard invoice and almost nothing of what varies in settling a billing dispute. The mapping itself was done by a language model, and expert reviewers agreed with its main match about three times in four. The 5% describes those 29 tests, read through that mapping. It says nothing about the particular test behind the number on your slide, which is why you have to ask.
What the test leaves out
Take something that looks well covered: preparing the monthly financial summary. A benchmark hands the model clean figures and asks for analysis.
In a business with $8 million in revenue, the same job starts with pulling numbers from the accounting system, the payment processor and a spreadsheet the operations manager keeps. Two of those disagree. Someone has to know which one is right this month, how the company counts revenue on deposits, why there is a $14,000 credit nobody remembers, who signs off, and who should not receive the file.
The analysis may be the smaller part of that job. Finding and trusting the inputs can take longer, and most benchmarks charge the model nothing for it, because they supply the inputs. The better recent ones do more. Mercor's own accounting benchmark makes the model reconcile conflicting files inside a simulated company. But the simulated company is not yours: its records are not your operations manager's spreadsheet, and its revenue rule is not the one your accountant applies to deposits. If the system cannot reach your data, cannot tell which number is authoritative, or gets it wrong one month in five, the score on the slide stops mattering.
How it fails matters more than how often it succeeds
A benchmark reports how often the model got it right. In your business, the expensive number is what happens the rest of the time.
An 8% error rate means one thing on an internal summary someone reads before it goes anywhere. It means something else on a quote sent to a customer, a refund issued to the wrong account, or a delivery window promised to a restaurant that needs the stock by Friday. The same rate is a convenience in the first case and a liability in the others, and the cost arrives as refunds, rework and the account that leaves without telling you why.
Three things decide whether the errors are tolerable: how often they happen, how much each one costs, and who catches them before a customer does. None of the three is on the leaderboard.
Catching them costs time. Picture a job that takes an employee an hour. If the system offers suggestions, maybe 45 minutes are left. If it writes a draft someone corrects, perhaps 20. If it does the work and a person approves it, five. A system that is right 95% of the time but needs every output checked can be worth less than one that is right 90% of the time, handles the routine cases by itself and sends the uncertain ones to a person. Which one fits depends on what an error that slips through would cost you, and that is a judgment about your business that no benchmark makes.
What you are actually buying
Say a tool saves your dispatcher six hours a week. Those hours turn into value in one of four ways. They become capacity if work is waiting: more jobs scheduled with the same team. They become service if customers are waiting: quotes out the same day instead of on Thursday. They become reliability if they go into the follow-ups and checks that currently slip. They become savings only if you skip a hire or reorganize roles, and that is your decision and your work, not the software's. If none of the four applies, the hours disappear into the week.
A vendor's ROI slide usually counts all four at once. Pick the one you are buying, and decide now how you will know in ninety days whether you got it.
Two side effects are worth expecting. When one step speeds up, the next one becomes the bottleneck: faster drafts mean more drafts waiting for someone to approve them. And when software takes the easy cases, the work left for your people is harder on average, which may change who you need in those seats.
Read the map the way you read a proposal
One more question applies to the study itself. Mercor sells expert-built evaluations of professional work to the companies that train AI models. A report concluding that most of the economy lacks good expert-built evaluations supports the market Mercor serves. That does not make the findings wrong. It is the question you would ask of any proposal on your desk, because the person who measured the value is often the person paid to deliver it.
The vendor pitching you sits in the same position, with more riding on your answer.
Judging the answers
Asking good questions is the easier half. Most vendors have rehearsed answers to the obvious ones, and a rehearsed answer can sound exactly like a real one. The difference is usually specificity, and it tends to show in the first sentence.
Which benchmark produced that number? A real answer names it, says what the tasks were and admits where they differ from your business. "Industry-leading" and "trained on millions of examples" are not answers. If the vendor will not name the test, treat the number as decoration.
Can we test it on fifty of our own cases, including the messy ones, before we commit? A real answer comes with a date, a success measure agreed in advance and a price for the pilot. Watch for a pilot that measures "engagement" or "user satisfaction", or one that runs on data your team cleans up for it first. A pilot that cannot fail cannot tell you anything.
Where does the system get the information it needs? A real answer names your systems, says who connects them, what breaks when one of them changes and who pays to fix it. "It learns your business over time" describes a hope. The cost of building that access often shows up later, in a change order.
On a normal day, how many outputs does a person still check, and what happens when one is wrong? A real answer is a percentage, a rule for which cases go to a person, and a name for who owns the error. "Accuracy is 95%" answers a different question.
The answers should move the decision, and they move it in specific directions. If a person still checks every output, review time is the main cost and the subscription is the smaller one, so negotiate the pilot before the license. If connecting your data is left to your team, the proposal is missing its largest line item. If the vendor cannot say what you keep when you stop (the data, the setup, the process changes), price the exit before you price the entry.
Reading those answers is a judgment call, and it gets harder when everyone in the room benefits from a yes: the vendor, the agency that would set it up, sometimes the employee who championed it. When the commitment is large, or the answers sound fine and you still cannot tell, it is worth asking someone independent to look at it with you: someone who understands the technology, has nothing to sell you and is not paid more if you sign.
Where to start instead
Before anyone shows you a model, pick one job your business does every week and time it end to end: where the information comes from, who touches it, where it waits, where it goes wrong. Look hardest at the handoffs, the re-keying between systems and the exceptions, because demos rarely show them. Some of what you find is a job for AI. Some is a setting nobody turned on in software you already pay for, or a process change that costs nothing.
That list, measured in your own hours, errors and missed work, is a better guide than any leaderboard, and it is the one thing no vendor can bring to the meeting for you.