Executive summary
AI evals test what an answer says and how it says it. They rarely test whether the number inside it is right, and that number is what the business acts on.
- A wrong number passes every standard eval. Relevance, sourcing, fluency, and human review all grade the words. In one BIRD-Interact question (preliminary results), a join error gave the wrong medication-adherence rates for 42 of 157 doctors, and every one of those checks would have passed it.
- Better prompts don’t fix the math. More context changes what the model reads. Only about one in four enterprises checks the computed result against a known-correct answer.
- Testing the math is repeatable. Compute every number from one definition, written once by people who know the business. Check it against a known-correct answer every time, and keep every miss as a permanent test. That’s the method behind AtScale ACE, which computes every number, and AtScale Canary, which tests each one and preserves the fix.
Who checks the math in an AI answer?
Language models learn to predict the next word. Arithmetic comes along as an unreliable side effect. Researchers at the University of Chicago found that standard-trained models got under 1% of four-digit multiplication problems right. At KTH Royal Institute of Technology, GPT-4o solved 14 of 50 high-school math problems correctly.
The answer itself won’t tell you any of this. A wrong total comes back in the same clean format as the right one, so the person reading it can’t tell them apart.
That was manageable when a person read every answer before anyone acted on it. Now agents act on numbers with no person in between. In a survey of 157 enterprises reported by The Machine Learning Times, 66% said they permit or are building toward zero-human deployment. In about three in four of those enterprises, nobody checks live answers at all. Anyone accountable for AI answers should be asking who reviews the number before the agent acts on it.
A wrong number passes every check
Take a question a BI steward’s agent might field: “What share of each doctor’s patients take their medication as prescribed?” The agent returns a clean table with one rate per doctor. Run it through a typical eval checklist:
- Relevant? Yes. It answers the question asked.
- Sourced? Yes. It cites the visits and treatments tables.
- Fluent? Yes. The summary reads well.
- Reviewed? Yes. A reviewer scanned it, the rates looked plausible, and they signed off.
The check missing from that list is whether the math is right. Here it wasn’t. When we ran this question in a benchmark called BIRD-Interact, the agent gave the wrong rate for 42 of the 157 doctors (preliminary results).
The KTH researchers saw the same pattern. Of 400 answers they graded, 21 reached the right result through flawed reasoning, and they warned that “erroneous solutions can look good at a cursory inspection.”
The Machine Learning Times survey also found that half of enterprises have shipped an agent that passed its evals and then failed a customer. When unchecked AI output reached a government client last year, Deloitte issued a partial refund. That error was in the citations. Errors in the arithmetic are harder to spot.
The math breaks in the SQL
The error happens in the SQL the model writes, before anyone sees a result.
The medication-adherence rate needs two tables: visits, and the treatments given at each visit. When you join them, a visit with three treatments shows up three times and gets counted three times. In this database, 123 of 351 visits carry more than one treatment. That’s how 42 doctors ended up with the wrong rate.
That’s a grain problem, and grain problems are only one kind of error.
The model has to work out how to join the tables, and at what grain, from scratch on every question. On one preliminary BIRD-Interact task about insider-trading surveillance, Kimi K2.5 wrote 21 lines of SQL with four subqueries, five tables, and six joins. The same metric, defined once in an AtScale semantic layer, takes three.
Better prompts change what the model reads
The usual fix for AI accuracy is more context: richer prompts, better metadata, documented definitions, a smarter model. All of that helps. A model that knows what “adherence” means is better off than one that doesn’t.
Each of these fixes works on the model’s input. Checking its output takes a separate step. A model can read a perfect definition of adherence and still join two tables at the wrong grain.
Guessing also costs money. When a model explores the schema, tests queries, and retries, it burns compute. A Tier 1 bank ran five everyday analyst questions through its Google BigQuery warehouse two ways. The rediscovery path cost $17.93 and scanned 3.15 terabytes. With a semantic layer in front, the same five queries cost less than a tenth of a cent and cut warehouse compute by a factor of 21,903.
Even the answer key needed checking
If a public benchmark’s own answer key had untested math, your dashboards probably do too.
BIRD-Interact is one of the most rigorous tests of AI on business data. It has 600 tasks across 22 messy PostgreSQL databases, with a simulated user who pushes back like a colleague would. When we ran it, the models weren’t the only source of math that needed checking. Preliminary findings from our October 2026 runs, not yet verified by the BIRD team:
- The grader needed six fixes to score answers fairly: dates, row order (only when the question asks for it), letter case, column order, ties, and numeric tolerance.
- 93 of the 410 business questions had a broken answer key. In some, the key computed a term differently than the benchmark’s own documentation defined it. In others, it read a field the database doesn’t contain, or returned rows in a random order, so the same right answer passed on one run and failed the next.
That’s no knock on the BIRD team. Building a benchmark is hard, and this one’s valuable because it’s hard. It shows how easily unchecked arithmetic hides, even in a system built to check arithmetic. See the full BIRD-Interact benchmark results for AtScale here.
Testing the math, every time
You can test the number in an AI answer the same way you’d test code, on every question:
- Compute every number from one definition. People who know the business write the joins and the formula once, and every question reuses them. The model asks for the metric by name instead of rebuilding it.
- Check every number against a known-correct answer. That means a test with an expected value, run every time.
- Keep every miss as a permanent test. When a number comes back wrong, the fix goes into the definition and the case goes into the test suite, so the same error can’t come back unnoticed.
Anthropic reported that its internal analytics agent went from 21% to more than 95% accuracy once business context and skills were added (self-reported). Anthropic could report those figures because they measured answers against known-correct results.
Step 1 is the job of AtScale ACE, the AI Computation Engine. ACE computes every number from the definitions in the AtScale semantic layer, the same way every time. Steps 2 and 3 belong to AtScale Canary. Canary checks each number against its expected value and keeps every miss as a test, so the steward gets a list of every number that failed, with the expected value next to the one the agent returned. Each row is a real error to trace and fix.
See what happened when we tested every number
We ran seven AI models through all 600 BIRD-Interact tasks and checked every number they returned. On Oct 28, we’ll walk through what we found and what it takes to test the math in your own AI answers. Register for the webinar.