Ask HN: How do you measure "AI slop"?
4 crakhamster01 1 7/31/2025, 8:48:29 AM
Recently, my employer has been pushing hard for LLM adoption across eng, with an expectation of increased productivity. Eng has followed suit, and as a result I've been getting a lot more PRs that are clearly AI generated. 100 line diffs that could have been 10, missed error cases, breaking convention. It's not just from junior engineers, but often from other senior engineers now.
With our incentive structures, it doesn't seem like there's a great way to prevent this decline in quality. It's been hard for me to quantify _why_ "slop" is bad, but my gut feelings are that:
1. The codebase becomes unreadable to human engineers.
2. Having more bad examples in the codebase creates a negative feedback loop for future LLM changes. And maybe this is the new norm, but ->
3. Once enough slop gets in, future incidents/SEVs become increasingly more difficult to resolve.
I’ve seen other posts lamenting the ills of vibe coding, but is there a concrete way to justify code quality in the era of LLMs? My thoughts are that it might be useful to track some code quality metric like cyclomatic complexity, and see if there’s any correlation with regressions over time, but that feels kind of thin (and retroactive).
You can tell it’s AI for a surprisingly high amount of LLM outputs. If you feel it’s regurgitating what you or your team already know, it’s slop.
This gets tricky of course, but it’s a tricky question. Though, I don’t think objective metrics work (in your case Cyclomatic Complexity), because information is relative by nature. What’s slop to someone is high-quality code or new information to someone else.