{ "question": "We pick models partly by benchmark accuracy, but the model that tops our accuracy chart also makes up the most answers. Why do AI benchmarks reward a model for guessing instead of saying \"I don't know\", and how should we score our own evals so that admitting uncertainty isn't punished?", "slug": "why-do-ai-benchmarks-reward-models-for-guessing-instead-of-saying-i-dont-know", "url": "https://www.rightshold.com/answers/why-do-ai-benchmarks-reward-models-for-guessing-instead-of-saying-i-dont-know", "short_answer": "Because most of them score a wrong answer and \"I don't know\" the same: zero. Under that rule guessing never lowers the score, so a model that always answers beats one that abstains when unsure. A paper by OpenAI and Georgia Tech researchers in Nature (April 2026, covered again by Georgia Tech on November 13, 2026) checked ten widely reported benchmarks and found only one, WildBench, gave even minimal credit for expressing uncertainty. Its example on SimpleQA: o4-mini answered almost everything and scored 20.6% accuracy with 76.8% wrong answers; GPT-5-mini abstained on 63.2% and scored 16.0% with only 20.8% wrong. Accuracy ranks the guesser first. The fix it proposes is an open rubric: tell the model in the prompt how errors are scored, for example \"correct +1, wrong −1, so abstain if under 50% sure\", and score it that way. An error penalty L is the same as giving t = L/(1+L) credit for abstaining. For your own evals, report correct, wrong and abstained separately, score with a stated penalty that matches what a wrong answer costs you, and record the rubric with each run.", "checklist": [ "Grade every answer as correct, wrong or abstained, not just correct or not. Report all three rates, not only accuracy.", "Decide what a wrong answer costs you compared with no answer, and turn that into a penalty L. L = 1 means abstain below 50% confidence, L = 3 below 75%, L = 9 below 90%.", "State the rubric in the prompt (\"correct answers get 1, wrong answers get −3, I don't know gets 0\") so the model can adjust to the stakes, and score with exactly that rule.", "Run more than one penalty, including 0. A model that answers when told to guess and abstains when errors are costly has controllable abstention; one that never changes is the risk.", "If a model scores below zero at a penalty you care about, always abstaining would beat it. Treat that as a sign of overconfidence for that use.", "When comparing models on public leaderboards, check whether the benchmark gives any credit for abstaining. If not, a higher score can mean more guessing.", "If you judge answers with another model, spot-check its grading; the paper notes model judges sometimes mark wrong long answers as correct.", "Record the rubric, penalty, model version and per-question outcome for each run, so the scores can be recounted under a different penalty later." ], "sections": [ { "heading": "Why accuracy-only scoring rewards guessing", "points": [ "The paper's argument is simple: with binary grading, a guess has some chance of scoring and \"I don't know\" has none, so guessing is the dominant strategy. Because models are tuned and selected against these scores, leaderboards that count only accuracy keep rewarding confident made-up answers.", "Its review of ten influential evaluations drawn from four popular leaderboards found binary grading almost everywhere; WildBench was the only one giving even minimal credit for indicating uncertainty. It also argues that adding separate hallucination benchmarks won't fix this while the headline benchmarks still penalise honest uncertainty.", "It also explains why some errors can't be trained away: facts that appear only once in training data, such as an obscure person's birthday, leave the model with no pattern to learn from. Some questions will stay unanswerable however good search and scaling get, so accuracy-only scoring keeps rewarding guesses on them." ] }, { "heading": "What open rubrics change", "points": [ "An open rubric states the scoring in the question itself. With an abstention reward t, guessing pays only when the model is more than t likely to be right. The paper used t = 0, 0.5, 0.75 and 0.9 (penalties 0, 1, 3 and 9) on all 4,326 SimpleQA questions with four frontier models from Google, OpenAI, xAI and Anthropic, run in February 2026 with default settings.", "It tested a simple mitigation: generate two answers, have the model judge whether they agree, and abstain if they don't. Under plain accuracy the mitigation cut wrong answers but also cut correct ones, so it looked worse and wouldn't be adopted. Under open rubrics it beat the baseline for every model at every penalty, including zero. In its extended results at penalties 3 and 9, all four models scored below zero without the mitigation, meaning always abstaining would have scored higher.", "The paper notes a limit: real users rarely state a confidence threshold, so closed-rubric tests still matter for everyday behaviour. Its experiment code is public on GitHub." ] }, { "heading": "Scoring your own eval so abstaining counts", "points": [ "Leaderboards are starting to move the same way. Artificial Analysis's Decision Index v2 (November 8, 2026) uses a Decision Score of 50 × (1 + c − (p − c)²) per question, where p is the model's confidence in its answer and c is 1 if correct, equal to 50 × (1 + accuracy − Brier score) overall. In its example on the AA-Paperclip document test, a model passing 24% of documents matched one passing 36% once calibration counted, because its confidence was better calibrated (Brier score 0.15 vs 0.27).", "For an internal eval, keep the per-question outcome (correct, wrong, abstained) and the model's stated confidence if it gives one. Then any penalty can be applied afterwards, and someone who disagrees with your penalty can rescore the same run.", "Record the rubric text, the penalty, the grader and its version alongside each run. A score without its rubric can't be compared with another run, and a rubric changed after the fact can make any model look better." ] } ], "checks": [], "ask_rightshold": { "start": "https://www.rightshold.com/start?ref=seo", "mcp_endpoint": "https://www.rightshold.com/mcp", "ask": "Record a signed receipt for one question in our hallucination eval: the model version, hashes of the prompt (with the stated open rubric, wrong answers −3) and of the model's answer, and in params the question id, the penalty, the abstention threshold, the grader and the graded outcome, so the run can be rescored later.", "mcp_call": { "tool": "record_ai_run", "arguments": { "model": "", "provider": "", "prompt_sha256": "", "output_sha256": "", "params": { "run": "open-rubric-eval-2026-11", "question_id": "Q-0412", "rubric": "open", "penalty_L": 3, "abstain_threshold_t": 0.75, "grader": "", "outcome": "abstained", "stated_confidence": null }, "ran_at": "2026-11-13T12:30:00Z" } }, "also": [ "Needs an API key. Send hashes and the fields shown; keep prompts and answers in your own store. params are stored as a hash, so keep the exact params object for each question.", "Record abstentions and wrong answers too, not only correct ones, so the three rates can be recounted.", "verify_ai_run is free and needs no key: a reviewer sends a receipt, its signature and the answer text to confirm it matches what was recorded.", "A receipt shows what was recorded and when. It doesn't grade answers, check facts or decide whether an answer is a hallucination." ], "scope": "Rightshold records the eval details you send and lets anyone check them later. It doesn't run models, grade answers or compute benchmark scores, and it isn't affiliated with OpenAI, Georgia Tech, Artificial Analysis or any benchmark. Figures are from the Nature paper (doi 10.1038/s41586-026-10549-w, published April 22, 2026), its public experiment code, Georgia Tech's November 13, 2026 article and Artificial Analysis's November 8, 2026 Decision Index v2 article, all read November 13, 2026. Not procurement advice." }, "sources": [ { "title": "Nature: Evaluating large language models for accuracy incentivizes hallucinations (OpenAI and Georgia Tech, April 22, 2026)", "url": "https://www.nature.com/articles/s41586-026-10549-w" }, { "title": "OpenAI: hallucinations-paper-experiments (code for the open-rubric SimpleQA case study)", "url": "https://github.com/openai/hallucinations-paper-experiments" }, { "title": "Georgia Tech Research News Center: The Mathematics of AI Safety (November 13, 2026)", "url": "https://news.research.gatech.edu/2026/11/13/mathematics-ai-safety" }, { "title": "Artificial Analysis: Introducing the Artificial Analysis Decision Index v2 (November 8, 2026)", "url": "https://artificialanalysis.ai/articles/decision-index-v2" }, { "title": "Verified AI Runs on Rightshold", "url": "https://www.rightshold.com/verified-ai-runs" } ], "related": [ { "title": "How to check AI-generated citations for hallucinations", "url": "https://www.rightshold.com/answers/how-to-check-ai-generated-citations-for-hallucinations" }, { "title": "What is pass^k, and why does one run per task overstate how consistent an AI agent is?", "url": "https://www.rightshold.com/answers/what-is-pass-hat-k-and-why-one-run-per-task-overstates-how-consistent-an-ai-agent-is" }, { "title": "How to prove a reported eval score wasn't cherry-picked from many runs", "url": "https://www.rightshold.com/answers/how-to-prove-a-reported-eval-score-wasnt-cherry-picked-from-many-runs" }, { "title": "How much should you trust an AI agent benchmark score when another AI model grades it?", "url": "https://www.rightshold.com/answers/how-much-should-you-trust-an-ai-agent-benchmark-score-when-another-ai-model-grades-it" }, { "title": "How to make LLM eval results reproducible and tamper-evident", "url": "https://www.rightshold.com/answers/how-to-make-llm-eval-results-reproducible-and-tamper-evident" } ], "computed_from": "Curated guidance; license verdicts computed from Rightshold's license profiles at render time. Not legal advice.", "updated": "2026-11-13" }