Are LLMs better suited for PR reviews than full codebases?
I’ve been thinking about this problem and wanted to share a perspective.
When evaluating LLMs for static analysis, I see four main dimensions: accuracy, coverage, context size, and cost.
On accuracy and coverage, today’s LLMs feel nowhere close to replacing dedicated SAST tools on real-world codebases. They do better on isolated snippets or smaller repos, but once you introduce deep dependency chains, results drop off quickly.
Context size is another bottleneck. Feeding an LLM a repo with millions of lines creates huge problems for reasoning across files, and the runtime gets impractical.
That leads to cost. Running an LLM across a massive codebase can be significantly more expensive than traditional scanners, without obvious ROI.
Where they do shine is at smaller scales — reviewing PRs, surfacing potential issues in context, or even suggesting precise fixes when the input is well-scoped. That seems like the most practical application right now. Whether providers will invest in solving the big scaling problems is still an open question.
Curious how others here think about the trade-offs between LLM-based approaches and existing SAST tools.