Ask HN: What is "response-level error rate" and how is it measured?

2 myyke 0 8/8/2025, 7:22:06 AM

There's this chart around gpt-5's hallu and error rates:

https://api.wandb.ai/files/byyoung3/images/projects/37269171/0da61431.png

from:

https://wandb.ai/byyoung3/ml-news/reports/GPT-5-Benchmark-Scores---VmlldzoxMzkwMTYyMg

I'm wondering what "response-level error rate" is exactly and it is measured?

gpt 4.1 says it's sampled production prompts, rated by humans. Is that it?

Open models by OpenAI (openai.com)

GPT-5 (openai.com)

Genie 3: A new frontier for world models (deepmind.google)

Perplexity is using stealth, undeclared crawlers to evade no-crawl directives (blog.cloudflare.com)

uBlock Origin Lite now available for Safari (apps.apple.com)

Show HN: I spent 6 years building a ridiculous wooden pixel display (benholmen.com)

Slow (michaelnotebook.com)

If you're remote, ramble (stephango.com)

Show HN: Kitten TTS – 25MB CPU-Only, Open-Source TTS Model (github.com)

Things that helped me get out of the AI 10x engineer imposter syndrome (colton.dev)

Emailing a one-time code is worse than passwords (blog.danielh.cc)

Modern Node.js Patterns (kashw1n.com)

Claude Opus 4.1 (anthropic.com)

I gave the AI arms and legs then it rejected me (grell.dev)

Vibechart (vibechart.net)

Claude Code IDE integration for Emacs (github.com)

Telo MT1 (telotrucks.com)

Corporation for Public Broadcasting ceasing operations (cpb.org)

Job-seekers are dodging AI interviewers (fortune.com)

Mastercard deflects blame for NSFW games being taken down (pcgamer.com)

Monitor your security cameras with locally processed AI (frigate.video)

6 weeks of Claude Code (blog.puzzmo.com)

Writing a good design document (grantslatton.com)

GPT-5: Key characteristics, pricing and system card (simonwillison.net)

Qwen-Image: Crafting with native text rendering (qwenlm.github.io)

Live coding interviews measure stress, not coding skills (hadid.dev)

The anti-abundance critique on housing is wrong (derekthompson.org)

MacBook Pro Insomnia (manuel.bernhardt.io)

Tesla withheld data, lied, misdirected police to avoid blame in Autopilot crash (electrek.co)

We may not like what we become if A.I. solves loneliness (newyorker.com)

Objects should shut up (dustri.org)

At 17, Hannah Cairo solved a major math mystery (quantamagazine.org)

Gemini 2.5 Deep Think (blog.google)

US reportedly forcing TSMC to buy 49% stake in Intel to secure tariff relief (notebookcheck.net)

How we made JSON.stringify more than twice as fast (v8.dev)

Cerebras Code (cerebras.ai)

This Month in Ladybird (ladybird.org)

PHP 8.5 adds pipe operator (thephp.foundation)

Japan: Apple Must Lift Browser Engine Ban by December (open-web-advocacy.org)

Lina Khan points to Figma IPO as vindication of M&A scrutiny (techcrunch.com)

Ollama Turbo (ollama.com)

Scientific fraud has become an 'industry,' analysis finds (science.org)

Ozempic shows anti-aging effects in trial (trial.medpath.com)

GPT-5 for Developers (openai.com)

Persona vectors: Monitoring and controlling character traits in language models (anthropic.com)

So you want to parse a PDF? (eliot-jones.com)

PixiEditor 2.0 – A FOSS universal 2D graphics editor (pixieditor.net)

Historical Tech Tree (historicaltechtree.com)

Ubiquiti launches UniFi OS Server for self-hosting (lazyadmin.nl)

DrawAFish.com Postmortem (aldenhallak.com)

Ask HN: What is "response-level error rate" and how is it measured?

Comments (0)