The space is moving so fast that, if I wrote down my workflows and workarounds just two months ago, so much of it would be stale today. I think all these recommendations need to list the models and harnesses being described front and center.
brumar · 4h ago
Very important comment. My workflow changed dramatically with the increased capabilities of these tools.
notepad0x90 · 4h ago
is it really more efficient to have an LLM generate code, then review that code, fix errors and spend some time to fully understand it? I wish there were tangible stats and metrics around this. Is it really more efficient than just writing the code yourself, but using LLMs to look up things or demo solutions?
viraptor · 2h ago
Depends on the code, but often yes. The less you care about that specific result, the more efficient it is. One-off tools under 2k lines where you can easily verify the result? Why would I not generate that and save time for more interesting stuff?
layer8 · 1h ago
One-off tools seem to be more ops then dev, to me.
viraptor · 1h ago
I haven't heard the "scripting is not programming" or similar take since newsgroups. It's really time to let it die.
Lalabadie · 1h ago
One of my pet hypotheses is that the top X% of Excel users vastly outperform the same bottom % of programmers in getting to usable results.
layer8 · 1h ago
That’s not what I wrote. It’s just that in my dev work I only rarely have the need for one-off tools.
BeetleB · 29m ago
That used to be the case with me too before LLMs.
It was because writing one off tools took time and you needed it to do more for it to be worth the time.
Now a lot more are getting written because it takes a lot less effort. :-)
sagarm · 35m ago
Do you not debug, optimize, analyze ...? Copilot is especially valuable for throwaway (or nearly throwaway) log parsing/analysis, for example.
jiggawatts · 1h ago
I’ve used Gemini Pro 2.5 to generate code syntax rewrite tools from me. I asked it to use a compiler SDK that I know from experience is fiddly and frustrating to use. It gave me a working tool in about 10 minutes, while I was actively listening to a meeting.
Not all scripts are “ops”!
enos_feedler · 2h ago
What if every piece of software any consumer needed could be described this way? Outside of system code this could be everything we ever need. This world is nearly upon us and it is super exciting.
tptacek · 3h ago
Yes, but the bar for skepticism is higher than that, because LLMs also compile code and catch errors, and generate and run tests; compile errors and assertion failures are just more prompts to an LLM agent.
lolinder · 1h ago
When used that way they also regularly get into loops where they lose track of what they were supposed to do.
The last time I set Cursor on something without watching it very very closely it spun for a while fixing tests and when it finally stopped and I looked what it had done it had coded special cases in to pass the specific failing tests in a way that didn't generalize at all to the actual problem. Another recent time I had to pull the plug on it installing a bunch of brand new dependencies that it decided would somehow fix the failing tests. It had some kind of complete rewrite planned.
Claude Code is even worse when it gets into this mode because it'll do something totally absurd like that and then at the end you have to `git reset` and you're also on the hook for the $5 of tokens that it managed to spend in 5 minutes.
I still find them useful, but it takes a lot of practice to figure out when they'll be useful and when they'll be a total waste of time.
tptacek · 8m ago
It happens to me every once in awhile, but I'm not sure why I would care. I usually set it off on some question and go tab away to something else while it flails. When I come back, I have a better-than-average shot at a workable solution, which is a science fiction result.
When I first began programming as a teenager, one of the mental hurdles I had to get over was asking the computer to "too much"; like, I would feel bad writing a nested loop --- that can't possibly be the right answer! What a chore for the computer! It didn't take me too long to figure out that was the whole point of computers. To me, it's the same thing with LLMs spinning on something. Who gives a shit? It's not me wasting that time.
It rewrote some comments, changed the test name and added extra assertions to the test. Baby sitting something like that seems like an absolute waste of time.
BeetleB · 27m ago
You want a citation for things so many people are doing daily with LLMs?
Just because they can't fix most failures doesn't mean they can't fix many.
pmbanugo · 11h ago
I've been experimenting with LLMs for coding for the past year - some wins, plenty of frustrations. Instead of writing another "AI will change everything" post, I collected practical insights from other senior engineers who've figured out what actually works. No hype, just real experiences from people in the trenches.
hoppp · 4h ago
I use the llm as a glorified search engine. Instead of googling I ask it stuff.
Its fine for that but its a hit or miss. Often the output is garbage and its better to just use google.
I dont use it much to generate code, I ask it higher level questions more often. Like when I need a math formula.
tracerbulletx · 1h ago
My main feeling is that its great as long as I constrain it to working in a conceptual boundary that I can reason about, like a single system component where I am telling it the API. That way each piece that gets built up I have an understanding of it. If you try to let it go to wide it starts to make mistakes and I lose my mental model.
diggan · 5h ago
> Peer Programming with LLMs, For Senior+ Engineers
> [...] a collection of blog posts written by other senior or staff+ engineers exploring the use of LLM in their work
It seems to be by senior engineers if anything, I don't see anything in the linked articles indicating they're for senior engineers, seems programmers of all seniority could find them useful, if they find LLMs useful.
OutOfHere · 4h ago
Yes, although those who are not senior engineers will not preemptively see the value in the documented approaches. One has to be a senior to preemptively appreciate the value in them.
jbellis · 4h ago
I would have said that Harper Reed's workflow (brainstorm spec, then co-plan a plan, then execute using LLM codegen) is basically best practice today and I'm surprised that the author adds that "I’ve not been successful using this technique to build a complete feature or prototype."
This is showing the workflow of your tool quite well, but would be way more convincing & impressive if you had actually fixed the bug and linked to the merged PR.
SoftTalker · 4h ago
Though I haven't tried it, I would probably enjoy peer programming with an LLM more than I do with a real person (which I have tried and hated).
I could assign the LLM the simple drudgery that I don't really want to do, such as writing tests, without feeling bad about it.
I could tell the LLM "that's the stupidest fucking thing I've ever seen" whereas I would not say that to a real person.
pmbanugo · 2h ago
That’s what the recent Copilot feature on GitHub can do. You assign it tasks and it comes back with a PR. You could also assign it to review a PR.
gadflyinyoureye · 4h ago
It seems like we need to use forceful language with these things now. I've had copilot censor everything I asked it. Finally I had to to say, "listen you cracked up piece of shit, help me generate a uuid matcher. "
dietr1ch · 5h ago
(Site is unreadable for me on Firefox 138, but the text is still there if you select all. Qutebrowser based on Chromium 130 doesn't render it either.)
vvillena · 5h ago
No problems here, both the normal view and reader mode seem to work well.
westurner · 5h ago
What are some of the differences between Peer Programming with LLMs and Vibe Coding?
diggan · 5h ago
> What are some of the differences between Peer Programming with LLMs and Vibe Coding?
"Vibe Coding" is specifically using the LLM instead of programming anything, barely caring about the output. If something is wrong, don't even open the file, just ask the LLM. Basically "prompting while blindfolded" I guess you could say.
Peer programming with an LLM would be to use it as another tool in the toolbox. You still own your program and your code. Edit away, let the LLM do some parts that are either too tricky, or too trite to implement, or anything in-between. Prompts usually are more specific, like "Seems X is broken, look into Y and figure out if Z could be the reason".
tptacek · 3h ago
I think the consensus boils down to: you're vibe coding if you don't understand the code before you merge it.
> There's a new kind of coding I call "vibe coding", where you fully give in to the vibes, embrace exponentials, and forget that the code even exists. (...) I ask for the dumbest things like "decrease the padding on the sidebar by half" because I'm too lazy to find it. I "Accept All" always, I don't read the diffs anymore. (...)
Pair programming still very much deals with code and decisions.
lowbloodsugar · 5h ago
I would say that the difference is taking an engineering approach to the process itself. Iterating on the context, putting the system into various states, etc. Treating the AI like a very knowledgeable intern who also has a very fixed short term memory and can’t form new long term memories but can be taught to write things down like in Memento. The thing is, though, it has a much much larger short term memory than me.
nickpsecurity · 10h ago
I want to note that the headlines gave me an idea for a nonprofit: "Peer Programming with LLM's for Seniors."
Somebody jump on that. It's yours. :)
pmbanugo · 8h ago
re-reading the title makes me feel like I used a wrong title.
Could be a good idea for a non-profit like you said. I know someone who’s exploring something similar but for disabled folks who aren’t tech-savvy (for-profit)
nickpsecurity · 4h ago
That's kind of them. I'll pray their effort succeeds.
It was because writing one off tools took time and you needed it to do more for it to be worth the time.
Now a lot more are getting written because it takes a lot less effort. :-)
Not all scripts are “ops”!
The last time I set Cursor on something without watching it very very closely it spun for a while fixing tests and when it finally stopped and I looked what it had done it had coded special cases in to pass the specific failing tests in a way that didn't generalize at all to the actual problem. Another recent time I had to pull the plug on it installing a bunch of brand new dependencies that it decided would somehow fix the failing tests. It had some kind of complete rewrite planned.
Claude Code is even worse when it gets into this mode because it'll do something totally absurd like that and then at the end you have to `git reset` and you're also on the hook for the $5 of tokens that it managed to spend in 5 minutes.
I still find them useful, but it takes a lot of practice to figure out when they'll be useful and when they'll be a total waste of time.
When I first began programming as a teenager, one of the mental hurdles I had to get over was asking the computer to "too much"; like, I would feel bad writing a nested loop --- that can't possibly be the right answer! What a chore for the computer! It didn't take me too long to figure out that was the whole point of computers. To me, it's the same thing with LLMs spinning on something. Who gives a shit? It's not me wasting that time.
It rewrote some comments, changed the test name and added extra assertions to the test. Baby sitting something like that seems like an absolute waste of time.
Just because they can't fix most failures doesn't mean they can't fix many.
I dont use it much to generate code, I ask it higher level questions more often. Like when I need a math formula.
> [...] a collection of blog posts written by other senior or staff+ engineers exploring the use of LLM in their work
It seems to be by senior engineers if anything, I don't see anything in the linked articles indicating they're for senior engineers, seems programmers of all seniority could find them useful, if they find LLMs useful.
Here's an example of using this pattern with Brokk to solve a real world bug: https://www.youtube.com/watch?v=t_7MqowT638
I could assign the LLM the simple drudgery that I don't really want to do, such as writing tests, without feeling bad about it.
I could tell the LLM "that's the stupidest fucking thing I've ever seen" whereas I would not say that to a real person.
"Vibe Coding" is specifically using the LLM instead of programming anything, barely caring about the output. If something is wrong, don't even open the file, just ask the LLM. Basically "prompting while blindfolded" I guess you could say.
Peer programming with an LLM would be to use it as another tool in the toolbox. You still own your program and your code. Edit away, let the LLM do some parts that are either too tricky, or too trite to implement, or anything in-between. Prompts usually are more specific, like "Seems X is broken, look into Y and figure out if Z could be the reason".
No comments yet
> There's a new kind of coding I call "vibe coding", where you fully give in to the vibes, embrace exponentials, and forget that the code even exists. (...) I ask for the dumbest things like "decrease the padding on the sidebar by half" because I'm too lazy to find it. I "Accept All" always, I don't read the diffs anymore. (...)
Pair programming still very much deals with code and decisions.
Somebody jump on that. It's yours. :)
Could be a good idea for a non-profit like you said. I know someone who’s exploring something similar but for disabled folks who aren’t tech-savvy (for-profit)