AI Coding Tools Underperform in Field Study with Experienced Developers

22 maxloh 4 7/21/2025, 7:59:24 PM infoq.com ↗

Comments (4)

gnabgib · 7h ago
Discussion (771 points, 11 days ago, 487 comments) https://news.ycombinator.com/item?id=44522772

Or (37 points, 8 days ago, 39 comments) https://news.ycombinator.com/item?id=44549436

g42gregory · 5h ago
I can not evaluate the merits of this study without first understanding who funds the organization METR (Model Evaluation & Threat Research), performing the research.

It's clearly an NGO/Policy think tank/Lobbying group. I need to understand for whom they lobby first.

suddenlybananas · 5h ago
https://metr.org/about

>

METR is funded by donations. Our largest funding to date was through The Audacious Project, a funding initiative housed at TED. METR has not accepted funding from AI companies, though we make use of free compute credits, as noted above. Being independently funded has been crucial for our ability to pursue the most promising research directions and set standards for evidence-based understanding of risks from AI. It is also part of how we ensure that our research is as accurate as possible.

g42gregory · 3h ago
You are not suggesting taking (any) organization at its word, when looking for the source of their funding, are you?