Ask HN: How do you personally evaluate new LLM models?

2 _samjarman 0 8/3/2025, 11:29:05 PM

Hey folks, how do you personally evaluate new HN models? Vibes? Or do you have some tests you like to run? Or do you just use them in your IDE/text iterface for a bit and see how it feels? I know we could probably trust some more public benchmarks but I'm curious on personal evaluation techniques. Thanks!

Fast (catherinejue.com)

Study mode (openai.com)

Copyparty – Turn almost any device into a file server (github.com)

Helsinki records zero traffic deaths for full year (helsinkitimes.fi)

EU age verification app to ban any Android system not licensed by Google (reddit.com)

Dumb Pipe (dumbpipe.dev)

‘I witnessed war crimes’ in Gaza – former worker at GHF aid site [video] (bbc.com)

Enough AI copilots, we need AI HUDs (geoffreylitt.com)

Slow (michaelnotebook.com)

Performance and telemetry analysis of Trae IDE, ByteDance's VSCode fork (github.com)

Show HN: Draw a fish and watch it swim with the others (drawafish.com)

M8.7 earthquake in Western Pacific, tsunami warning issued (earthquake.usgs.gov)

Show HN: Use Their ID – Use your local UK MP’s ID for the Online Safety Act (use-their-id.com)

Vibe code is legacy code (blog.val.town)

Our $100M Series B (oxide.computer)

Face it: you're a crazy person (experimental-history.com)

‘No Other Land’ consultant Awdah Hathaleen killed by Israeli settler (latimes.com)

VPN use surges in UK as new online safety rules kick in (ft.com)

If you're remote, ramble (stephango.com)

Tom Lehrer has died (nytimes.com)

Sleep all comes down to the mitochondria (science.org)

Visa and Mastercard are getting overwhelmed by gamer fury over censorship (polygon.com)

Corporation for Public Broadcasting ceasing operations (cpb.org)

Claude Code weekly rate limits

Telo MT1 (telotrucks.com)

My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air) (simonwillison.net)

Sign in with Google in Chrome (underpassapp.com)

Ollama's new app (ollama.com)

6 weeks of Claude Code (blog.puzzmo.com)

Tao on “blue team” vs. “red team” LLMs (mathstodon.xyz)

How was the Universal Pictures 1936 opening logo created? (movies.stackexchange.com)

4k NASA employees opt to leave agency through deferred resignation program (kcrw.com)

iPhone 16 cameras vs. traditional digital cameras (candid9.com)

The anti-abundance critique on housing is wrong (derekthompson.org)

Live coding interviews measure stress, not coding skills (hadid.dev)

MacBook Pro Insomnia (manuel.bernhardt.io)

Irrelevant facts about cats added to math problems increase LLM errors by 300% (science.org)

We may not like what we become if A.I. solves loneliness (newyorker.com)

Many countries that said no to ChatControl in 2024 are now undecided (digitalcourage.social)

When we get Komooted (bikepacking.com)

At 17, Hannah Cairo solved a major math mystery (quantamagazine.org)

Gemini 2.5 Deep Think (blog.google)

LLM Embeddings Explained: A Visual and Intuitive Guide (huggingface.co)

Cerebras Code (cerebras.ai)

I designed my own fast game streaming video codec – PyroWave (themaister.net)

This Month in Ladybird (ladybird.org)

RIP Shunsaku Tamiya, the man who made plastic model kits a global obsession (japanesenostalgiccar.com)

Try the Mosquito Bucket of Death (energyvanguard.com)

Debian switches to 64-bit time for everything (theregister.com)

The Math Is Haunted (overreacted.io)

Ask HN: How do you personally evaluate new LLM models?

Comments (0)