… because programming languages are the right level of precision for specifying a program you want. Natural language isn’t it. Of course you need to review and edit what it generates. Of course it’s often easier to make the change yourself instead of describing how to make the change.
I wonder if the independent studies that show Copilot increasing the rate of errors in software have anything to do with this less bold attitude. Most people selling AI are predicting the obsolescence of human authors.
soulofmischief · 1h ago
Transformers can be used to automate testing, create deeper and broader specification, accelerate greenfield projects, rapidly and precisely expand a developer's knowledge as needed, navigate unfamiliar APIs without relying on reference, build out initial features, do code review and so much more.
Even if code is the right medium for specifying a program, transformers act as an automated interface between that medium and natural language. Modern high-end transformers have no problem producing code, while benefiting from a wealth of knowledge that far surpasses any individual.
> Most people selling AI are predicting the obsolescence of human authors.
It's entirely possible that we do become obsolete for a wide variety of programming domains. That's simply a reality, just as weavers saw massive layoffs in the wake of the automated loom, or scribes lost work after the printing press, or human calculators became pointless after high-precision calculators became commonplace.
This replacement might not happen tomorrow, or next year, or even in the next decade, but it's clear that we are able to build capable models. What remains to be done is R&D around things like hallucinations, accuracy, affordability, etc. as well as tooling and infrastructure built around this new paradigm. But the cat's out of the bag, and we are not returning to a paradigm that doesn't involve intelligent automation in our daily work; programming is literally about automating things and transformers are a massive forward step.
That doesn't really mean anything, though; You can still be as involved in your programming work as you'd like. Whether you can find paid, professional work depends on your domain, skill level and compensation preferences. But you can always program for fun or personal projects, and decide how much or how little automation you use. But I will recommend that you take these tools seriously, and that you aren't too dismissive, or you could find yourself left behind in a rapidly evolving landscape, similarly to the advent of personal computing and the internet.
interstice · 35m ago
> Modern high-end transformers have no problem producing code, while benefiting from a wealth of knowledge that far surpasses any individual.
It will also still happily turn your whole codebase into garbage rather than undo the first thing it tried to try something else. I've yet to see one that can back itself out of a logical corner.
recursive · 8m ago
> It will also still happily turn your whole codebase into garbage rather than undo the first thing it tried to try something else.
That's not true at all.
...
It's only pretending to be happy.
johnnyjeans · 13m ago
> That's simply a reality, just as weavers saw massive layoffs in the wake of the automated loom, or scribes lost work after the printing press, or human calculators became pointless after high-precision calculators became commonplace.
See, this is the kind of conception of a programmer I find completely befuddling. Programming isn't like those jobs at all. There's a reason people who are overly attached to code and see their job as "writing code" are pejoratively called "code monkeys." Did CAD kill the engineer? No. It didn't. The idea is ridiculous.
soulofmischief · 23s ago
[delayed]
JoeOfTexas · 1h ago
Doesn't AI have diminishing returns on it's pseudo creativity? Throw all the training output of LLM into a circle. If all input comes from other LLM output, the circle never grows. Humans constantly step outside the circle.
Perhaps LLM can be modified to step outside the circle, but as of today, it would be akin to monkeys typing.
msgodel · 55m ago
In my experience the best use of AI is to stay in the flow state when you get blocked by an API you don't understand or a feature you don't want to implement for whatever reason.
JamesBarney · 1h ago
Right level for exactly specifying program behavior in a global domain without context.
But once you add repo context, domain knowledge etc... programming languages are far too verbose.
sysmax · 2h ago
AI can very efficiently apply common patterns to vast amounts of code, but it has no inherent "idea" of what it's doing.
Here's a fresh example that I stumbled upon just a few hours ago. I needed to refactor some code that first computes the size of a popup, and then separately, the top left corner.
For brevity, one part used an "if", while the other one had a "switch":
I wanted the LLM to refactor it to store the position rather than applying it immediately. Turns out, it just could not handle different things (if vs. switch) doing a similar thing. I tried several variations of prompts, but it very strongly leaning to either have two ifs, or two switches, despite rather explicit instructions not to do so.
It sort of makes sense: once the model has "completed" an if, and then encounters the need for a similar thing, it will pick an "if" again, because, well, it is completing the previous tokens.
Harmless here, but in many slightly less trivial examples, it would just steamroll over nuance and produce code that appears good, but fails in weird ways.
That said, splitting tasks into smaller parts devoid of such ambiguities works really well. Way easier to say "store size in m_StateStorage and apply on render" than manually editing 5 different points in the code. Especially with stuff like Cerebras, that can chew through complex code at several kilobytes per second, expanding simple thoughts faster than you could physically type them.
gametorch · 2h ago
Yeah that's one model that you happen to be using in June 2025.
Give it to o3 and it could definitely handle that today.
Sweeping generalizations about how LLMs will never be able to do X, Y, or Z coding task will all be proven wrong with time, imo.
npinsker · 2h ago
Sweeping generalizations about how LLMs will always (someday) be able to do arbitrary X, Y, and Z don't really capture me either
gametorch · 2h ago
In response to your sweeping generalization, I posit a sweeping generalization of my own, said the bard:
Whatever can be statistically predicted
by the human brain
Will one day also be
statistically predicted by melted sand
agentultra · 2h ago
Until the day that thermodynamics kicks in.
Or the current strategies to scale across boards instead of chips gets too expensive in terms of cost, capital, and externalities.
gametorch · 2h ago
I mean fair enough, I probably don't know as much about hardware and physics as you
agentultra · 1h ago
Just pointing out that there are limits and there’s no reason to believe that models will improve indefinitely at the rates we’ve seen these last couple of years.
soulofmischief · 1h ago
There is reason to believe that humans will keep trying to push the limitations of computation and computer science, and that recent advancements will greatly accelerate our ability to research and develop new paradigms.
Look at how well Deepseek performed with the limited, outdated hardware available to its researchers. And look at what demoscene practitioners have accomplished on much older hardware. Even if physical breakthroughs ceased or slowed down considerably, there is still a ton left on the table in terms of software optimization and theory advancement.
And remember just how young computer science is as a field, compared to other human practices that have been around for hundreds of thousands of years. We have so much to figure out, and as knowledge begets more knowledge, we will continue to figure out more things at an increasing pace, even if it requires increasingly large amounts of energy and human capital to make a discovery.
I am confident that if it is at all possible to reach human-level intelligence at least in specific categories of tasks, we're gonna figure it out. The only real question is whether access to energy and resources becomes a bigger problem in the future, given humanity's currently extraordinarily unsustainable path and the risk of nuclear conflict or sustained supply chain disruption.
sysmax · 1h ago
I am working on a GUI for delegating coding tasks to LLMs, so I routinely experiment with a bunch of models doing all kinds of things. In this case, Claude Sonnet 3.7 handled it just fine, while Llama-3.3-70B just couldn't get it. But that is literally the simplest example that illustrates the problem.
When I tried giving top-notch LLMs harder tasks (scan an abstract syntax tree coming from a parser in a particular way, and generate nodes for particular things) they completely blew it. Didn't even compile, let alone logical errors and missed points. But once I broke down the problem to making lists of relevant parsing contexts, and generating one wrapper class at a time, it saved me a whole ton of work. It took me a day to accomplish what would normally take a week.
Maybe they will figure it out eventually, maybe not. The point is, right now the technology has fundamental limitations, and you are better off knowing how to work around them, rather than blindly trusting the black box.
gametorch · 1h ago
Yeah exactly.
I think it's a combination of
1) wrong level of granularity in prompting
2) lack of engineering experience
3) autistic rigidity regarding a single hallucination throwing the whole experience off
4) subconscious anxiety over the threat to their jerbs
5) unnecessary guilt over going against the tide; anything pro AI gets heavily downvoted on Reddit and is, at best, controversial as hell here
I, for one, have shipped like literally a product per day for the last month and it's amazing. Literally 2,000,000+ impressions, paying users, almost 100 sign ups across the various products. I am fucking flying. Hit the front page of Reddit and HN countless times in the last month.
Idk if I break down the prompts better or what. But this is production grade shit and I don't even remember the last time I wrote more than two consecutive lines of code.
sysmax · 1h ago
If you are launching one product per day, you are using LLMs to convert unrefined ideas into proof-of-concept prototypes. That works really well, that's the kind of work that nobody should be doing by hand anymore.
Except, not all work is like that. Fast-forward to product version 2.34 where a particular customer needs a change that could break 5000 other customers because of non-trivial dependencies between different parts of the design, and you will be rewriting the entire thing by humans or having it collapse under its own weight.
But out of 100 products launched on the market, only 1 or 2 will ever reach that stage, and having 100 LLM prototypes followed by 2 thoughtful redesigns is way better than seeing 98 human-made products die.
guappa · 2h ago
If you need a model per task, we're very far from AGI.
DataDaoDe · 2h ago
The interesting questions happen when you define X, Y and Z and time. For example, will llms be able to solve the P=NP problem in two weeks, 6 months, 5 years, a century? And then exploring why or why not
soulofmischief · 1h ago
> AI can very efficiently apply common patterns to vast amounts of code, but it has no inherent "idea" of what it's doing.
AI stands for Artificial Intelligence. There are no inherent limits around what AI can and can't do or comprehend. What you are specifically critiquing is the capability of today's popular models, specifically transformer models, and accompanying tooling. This is a rapidly evolving landscape, and your assertions might no longer be relevant in a month, much less a year or five years. In fact, your criticism might not even be relevant between current models. It's one thing to speak about idiosyncrasies between models, but any broad conclusions drawn outside of a comprehensive multi-model review with strict procedure and controls is to be taken with a massive grain of salt, and one should be careful to avoid authoritative language about capabilities.
It would be useful to be precise in what you are critiquing, so that the critique actually has merit and applicability. Even saying "LLM" is a misnomer, as modern transformer models are multi-modal and trained on much more than just textual language.
mattbee · 1h ago
What a ridiculous response, to scold the GP for criticising today's AI because tomorrow's might be better. Sure, it might! But it ain't here yet buddy.
Lots of us are interested in technology that's actually available, and we can all read date stamps on comments.
soulofmischief · 1h ago
You're projecting that I am scolding OP, but I'm not. My language was neutral and precise. I presented no judgment, but gave OP the tools to better clarify their argument and express valid, actionable criticism instead of wholesale criticizing "AI" in a manner so imprecise as to reduce the relevance and effectiveness of their argument.
> But it ain't here yet buddy . . . we can all read date stamps on comments.
That has no bearing on the general trajectory that we are currently on in computer science and informatics. Additionally, your language is patronizing and dismissive, trading substance for insult. This is generally frowned upon in this community.
You failed to actually address my comment, both by failing to recognize that it was mainly about using the correct terminology instead of criticizing an entire branch of research that extends far beyond transformers or LLMs, and by failing to establish why a rapidly evolving landscape does not mean that certain generalizations cannot yet be made, unless they are presented with several constraints and caveats, which includes not making temporally-invariant claims about capabilities.
I would ask that you reconsider your approach to discourse here, so that we can avoid this thread degenerating into an emotional argument.
mattbee · 32m ago
The GP was very precise in the experience they shared, and I thought it was interesting.
They were obviously not trying to make a sweeping comment about the entire future of the field.
Are you using ChatGPT to write your loquacious replies?
soulofmischief · 18m ago
> They were obviously not trying to make a sweeping comment about the entire future of the field
OP said “AI can very efficiently apply common patterns to vast amounts of code, but it has no inherent "idea" of what it's doing.”
I'm not going to patronize you by explaining why this is not "very precise", or why its lack of temporal caveats is an issue, as I've already done so in an earlier comment. If you're still confused, you should read the sentence a few times until you understand. OP did not even mention which specific model they tested, and did not provide any specific prompt example.
> Are you using ChatGPT to write your loquacious replies?
If you can't handle a few short paragraphs as a reply, or find it unworthy of your time, you are free to stop arguing. The Hacker News guidelines actually encourage substantive responses.
I also assume that in the future, accusing a user of using ChatGPT will be against site guidelines, so you may as well start phasing that out of your repertoire now.
Here are some highlights from the Hacker News guidelines regarding comments:
- Don't be snarky
- Comments should get more thoughtful and substantive, not less, as a topic gets more divisive.
- Assume good faith
- Please don't post insinuations about astroturfing, shilling, brigading, foreign agents, and the like. It degrades discussion and is usually mistaken.
Personally, I define the job of a software engineer as transform requirements into software.
Software is not only code. Requirements are not only natural language.
At the moment I can't manage to be faster with the AI than manually. Unless its a simple task or software.
In my experience AI's are atm junior or mid-level developers. And in the last two years, they didn't get significant better.
nicbou · 1h ago
Most of the time, the requirements are not spelled out. Nobody even knows what the business logic is supposed to be. A lot of it has to be decided by the software engineer based on available information. It sometimes involve walking around the office asking people things.
It also requires a fair bit of wisdom to know where the software is expected to grow, and how to architect for that eventuality.
I can't picture an LLM doing a fraction of that work.
exiguus · 1h ago
I think that's my problem with AI. Let's say I have all the requirements, down to the smallest detail. Then I make my decisions at a micro level. Formulate an architecture. Take all the non-functionals into account. I would write a book as a prompt that is not able to express my thoughts as accurately as if I were writing code right away. Apart from the fact that the prompt is generally a superfluous intermediate step in which I struggle to create an exact programming language with an imprecise natural language with a result that is not reproduce-able.
taysix · 2h ago
I had a fun result the other day from Claude. I opened a script in Zed and asked it to "fix the error on line 71".
Claude happily went and fixed the error on line 91....
1. There was no error on line 91, it did some inconsequential formatting on that line
2. More importantly, it just ignored the very specific line I told it to go to. It's like I was playing telephone with the LLM which felt so strange with text-based communication.
This was me trying to get better at using the LLM while coding and seeing if I could "one-shot" some very simple things. Of course me doing this _very_ tiny fix myself would have been faster. Just felt weird and reinforces this idea that the LLM isn't actually thinking at all.
klysm · 1h ago
LLMs probably have bad awareness of line numbers
mcintyre1994 · 1h ago
I suspect if OP highlighted line 71 and added it to chat and said fix the error, they’d get a much better response. I assume Cursor could create a tool to help it interpret line numbers, but that’s not how they expect you to use it really.
recursive · 1h ago
How is this better from just using a formal language again?
senko · 1h ago
> This was me trying to get better at using the LLM while coding
And now you've learned that LLMs can't count lines. Next time, try asking it to "fix the error in function XYZ" or copy/paste the line in question, and see if you get better results.
> reinforces this idea that the LLM isn't actually thinking at all.
Of course it's not thinking, how could it? It's just a (rather big) equation.
54 def dicts_to_table_string(
55 headings: List[str], dicts: List[Dict[str, str]]
56 ) -> List[str]:
57 max_lengths = [len(h) for h in headings]
58
59 # Compute maximum length for each column
60 for d in dicts:
emp17344 · 1h ago
That’s not what he’s saying there. There’s a separate tool that adds line numbers before feeding the prompt into the LLM. It’s not the LLM doing it itself.
toephu2 · 1h ago
Sounds like operator error to me.
You need to give LLMs context. Line number isn't good context.
meepmorp · 48m ago
> Line number isn't good context.
a line number is plenty of context - it's directly translatable into a range of bytes/characters in the file
layer8 · 2h ago
One of the most useful properties of computers is that they enable reliable, eminently reproducible automation. Formal languages (like programming languages) not only allow to unambiguously specify the desired automation to the upmost level of precision, they also allow humans to reason about the automation with precision and confidence. Natural language is a poor substitute for that. The ground truth of programs will always be the code, and if humans want to precisely control what a program does, they’ll be best served by understanding, manipulating, and reasoning about the code.
CoffeeOnWrite · 3h ago
“Manual” has a negative connotation. If I understand the article correctly, they mean “human coding remains key”. It’s not clear to me the GitHub CEO actually used the word “manual”, that would surprise me. Is there another source on this that’s either more neutral or better at choosing accurate words? The last thing we need is to put down human coding as “manual”; human coders have a large toolbox of non-AI tools to automate their coding.
(Wow I sound triggered! sigh)
upghost · 3h ago
It's almost as bad as "manual" thinking!
anamexis · 2h ago
What is the distinction between manual coding and human coding?
dalyons · 1h ago
Acoustic coding
GuinansEyebrows · 2h ago
> Wow I sound triggered! sigh
this is okay! it's a sign of your humanity :)
layer8 · 2h ago
How about “organic coding”? ;)
vram22 · 2h ago
>Manual” has a negative connotation. If I understand the article correctly, they mean “human coding remains key”.
I think so many devs fail to realise that to your product manager / team lead, the interface between you and the LLM is basically the same. They write a ticket/prompt and they get back a bunch of code that undoubtedly has bugs and misinterpretations in it, will probably go through a few rounds of revisions of back and forth until its good enough to ship (ie they tested it black-box style and it worked once) then they can move on to the next thing until whatever this ticket was about rears its ugly head again at some point in the future. If you arent used to writing user stories / planning, youre really gonna be obsolete soon.
h4kunamata · 34m ago
Too late, I am seeing developer after developer doing copy/paste from AI tools and when asked, they have no idea how the code works coz "it just works"
Google itself said 30% of their code is AI generated, and yet they had a recent outage worldwide, coincidence??
You tell me.
boshalfoshal · 53m ago
Imo this is a misunderstanding of what AI companies want AI tools to be and where the industry is heading in the near future. The endgame for many companies is SWE automation, not augmentation.
To expand -
1. Models "reason" and can increasingly generate code given natural language. Its not just fancy autocomplete, its like having an intern - mid level engineer at your beck and call to implement some feature. Natural language is generally sufficient enough when I interact with other engineers, why is it not sufficient for an AI, which (in the limit), approaches an actual human engineer?
2. Business wise, companies will not settle for augmentation. Software companies pay tons of money in headcount, its probably most mid-sized companies top or second line item. The endgame for leadership at these companies is to do more with less. This necessitates automation (in addition to augmenting the remaining roles).
People need to stop thinking of LLMs as "autocomplete on steroids" and actually start thinking of them as a "24/7 junior SWE who doesn't need to eat or sleep and can do small tasks at 90% accuracy with some reasonable spec." Yeah you'll need to edit their code once in a while but they also get better and cost less than an actual person.
__loam · 47m ago
Folks who believe this are going to lose a lot of money fixing broken software and shipping products that piss off their users.
jstummbillig · 3h ago
Going by the content of the linked post, this is very much a misleading headline. There is nothing in the quotes that I would read as an endorsement of "manual coding", at least not in the sense that we have used the term "coding" for the past decades.
hnthrow90348765 · 2h ago
My guess is they will settle for 2x the productivity as a before-AI developer as their skill floor, but then not take a look at how long meetings and other processes take.
Why not look at Bob who takes like 2 weeks to write tickets on what they actually want in a feature? Or Alice who's really slow getting Figma designs done and validated? How nice would having a "someone's bothered a developer" metric be and having the business seek to get that to zero and talk very loudly about it as they have about developers?
strict9 · 3h ago
It's interesting to see a CEO express thoughts on AI and coding go in a slightly different direction.
Usually the CEO or investor says 30% (or some other made up number) of all code is written by AI and the number will only increase, implying that developers will soon be obsolete.
It's implied that 30% of all code submitted and shipped to production is from AI agents with zero human interaction. But of course this is not the case, it's the same developers as before using tools to more rapidly write code.
And writing code is only one part of a developer's job in building software.
madeofpalk · 3h ago
He’s probably more right than not. But he also has a vested interest in this (just like the other CEOs who say the opposite), being in the business of human-mediated code.
yodon · 2h ago
Presumably you're aware that the full name of Microsoft's Copilot AI code authoring tool is "GitHub Copilot", that GitHub developed it, and that he runs GitHub.
Imustaskforhelp · 2h ago
Yea, which is why I was surprised too when he said this.
madeofpalk · 2h ago
Copilot. Not Pilot.
heisenbit · 3h ago
Well, I suspect GitHub's income is a function of the number of developers using it so it is not surprising that he takes this position.
mewc · 1h ago
More complaining & pessimism means better signal for teams building the AI coding tools! Keep it up!
The ceiling for AI is not even close to being met. We have to be practical with whats reasonable, but getting 90% complete in a few prompts is magic.
mycocola · 2h ago
I think most programmers would agree that thinking represents the majority of our time. Writing code is no different than writing down your thoughts, and that process in itself can be immensely productive -- it can spark new ideas, grant epiphanies, or take you in an entirely new direction altogether. Writing is thinking.
I think an over-reliance, or perhaps any reliance, on AI tools will turn good programmers into slop factories, as they consistently skip over a vital part of creating high-quality software.
You could argue that the prompt == code, but then you are adding an intermediary step between you and the code, and something will always be lost in translation.
I'd say just write the code.
sothatsit · 1h ago
I think this misses the point. You're right that programmers still need to think. But you're wrong thinking that AI does not help with that.
With AI, instead of starting with zero and building up, you can start with a result and iterate on it straight away. This process really shines when you have a good idea of what you want to do, and how you want it implemented. In these cases, it is really easy to review the code, because you knew what you wanted it to look like. And so, it lets me implement some basic features in 15 minutes instead of an hour. This is awesome.
For more complex ideas, AI can also be a great idea sparring partner. Claude Code can take a paragraph or two from me, and then generate a 200-800 line planning document fleshing out all the details. That document: 1) helps me to quickly spot roadblocks using my own knowledge, and 2) helps me iterate quickly in the design space. This lets me spend more time thinking about the design of the system. And Claude 4 Opus is near-perfect at taking one of these big planning specifications and implementing it, because the feature is so well specified.
So, the reality is that AI opens up new possible workflows. They aren't always appropriate. Sometimes the process of writing the code yourself and iterating on it is important to helping you build your mental model of a piece of functionality. But a lot of the time, there's no mystery in what I want to write. And in these cases, AI is brilliant at speeding up design and implementation.
mycocola · 1h ago
Based on your workflow, I think there is considerable risk of you being wooed by AI into believing what you are doing is worthwhile. The plan AI offers is coherent, specific, it sounds good. It's validation. Sugar.
sothatsit · 15m ago
That is a very weak excuse to avoid these tools.
I know the tools and environments I am working in. I verify the implementations I make by testing them. I review everything I am generating.
The idea that AI is going to trick me is absurd. I'm a professional, not some vibe coding script kiddie. I can recognise when the AI makes mistakes.
Have the humility to see that not everyone using AI is someone who doesn't know what they are doing and just clicks accept on every idea from the AI. That's not how this works.
swyx · 2h ago
> In an appearance on “The MAD Podcast with Matt Turck,” Dohmke said that
> Source: The Times of India
what in the recycled content is this trash?
bamboozled · 13m ago
We've generated a lot of code with claude code recently...then we've had to go back and rationalize it... :) fun times...you absolutely must have a well defined architecture established before using these tools.
jasonthorsness · 2h ago
"He warned that depending solely on automated agents could lead to inefficiencies. For instance, spending too much time explaining simple changes in natural language instead of editing the code directly."
Lots of changes where describing them in English takes longer than just performing the change. I think the most effective workflow with AI agents will be a sort of active back-and-forth.
sodality2 · 2h ago
Yeah, I can’t count the number of times I’ve thought about a change, explained it in natural language, pressed enter, then realized I’ve already arrived at the exact change I need to apply just by thinking it through. Oftentimes I even beat the agent at editing it, if it’s a context-heavy change.
dgfitz · 1h ago
Rubber duck. I’ve kept one on my desk for over a decade. It was also like a dollar, which is more than I’ve spent on LLMs. :)
How active are you ok with/want? I've just joined an agent tooling startup (yesh...I wrote that huh...) - and it's something we talk a lot about internally, we're thinking it's fine to do back and forth, tell it frankly it's not doing it right, etc, but some stuff might be annoying? Do you have a sense of how this might work to your mind? Thanks! :)
klysm · 1h ago
CEOs are possibly the last person you should listen to on any given subject.
dgfitz · 4m ago
So do we ignore the Github CEO or all the other ones?
beej71 · 1h ago
I'm just vibe-CEOing my new company. It's amazing how productive it is!
randomNumber7 · 2h ago
Code monkeys that doesn't understand the limits of LLMs and can't solve problems where the LLM fails are not needed in the world of tomorrow.
Why wouldn't your boss ask ChatGPT directly?
another_twist · 2h ago
I think these are coordinated posts by Microsoft execs. First their director of product, now this. Its like they're trying to calm the auto coding hype until they catchup and thus keep OpenAI from running away.
exabrial · 2h ago
Amazingly, so does air and water. What AI salesman could have predicted this?
lawgimenez · 2h ago
Not gonna lie, first time I've heard of manual coding.
OJFord · 2h ago
This seems to be an AI summary of a (not linked) podcast.
treefarmer · 3h ago
I get a 403 forbidden error when trying to view the page. Anyone else get that?
voidhorse · 45m ago
Hopefully a CEO finally tempering some expectations and the recent Apple paper bring some sanity back into the discourse around these tools.[^1]
Are they cool and useful? Yes.
Do they reason? No. (Before you complain, please first define reason).
Are they end all be all of all problem solving and the dawn of AGI? Also no.
Once we actually bring some rationality back into the radius of discourse maybe we'll actually begin to start figuring out how these things actually fit into an engineering workflow and stop throwing around ridiculous terms like vibe coding.
If you are an engineer, you are signing up to build rigorously verified and validated system, preferably with some amount of certainty about their behavior under boundary conditions. All the current hype-addled discussion around LLM seems to have had everything but correctness as it's focus.
[^1]: It shouldn't take a CEO but many people, even technologists, who should be more rational about whose opinions they deem worthy of consideration, m seem to overvalue the opinions of the csuite for some bizarre, inexplicable reason.
lunarboy · 2h ago
It was only 2 years ago we were still taking about GPTs making up completely nonsense, and now hallucinations are almost gone from the discussions. I assume it will get even better, but I also think there is an inherent plateau. Just like how machines solved mass manufacturing work, but we still have factory workers and overseers. Also, "manually" hand crafted pieces like fashion and watches continue to be the most expensive luxury goods. So I don't believe good design architects and consulting will ever be fully replaced.
jashmatthews · 2h ago
Hallucinations are now plausibly wrong which is in some ways harder to deal with. GPT4.1 still generates Rust with imaginary crates and says “your tests passed, we can now move on” to a completely failed test run.
bmitc · 1h ago
It would be nice if he would back that up by increasing focus on quality of life issues on GitHub. GitHub's feature set seems to get very little attention unless it overlaps with Copilot.
dboreham · 2h ago
That's him out the Illuminati then.
FirmwareBurner · 3h ago
I wonder how much coding he does and how does he know which code is human written and which by machine.
GiorgioG · 1h ago
No fucking shit Sherlock.
guluarte · 2h ago
AI is good for boilerplate, suggestions, nothing more.
johnisgood · 1h ago
For you, perhaps.
beej71 · 1h ago
For me, too. On the poorly-documented stuff that I get into, where I really need help, it flails.
I wanted some jq script and it made an incomprehensible thing that usually worked. And every fix it made broke something else. It just kept digging a deeper hole.
After a while of that, I bit the bullet and learned jq script. I wrote a solution that was more capable, easier to read, and, importantly, always worked.
The thing is, jq is completely documented. Everything I needed was online. But there just aren't many examples to for LLMs, so they choke.
Start asking it questions about complexity classes and you can get it to contradict itself in no time.
Most of the code in the world right now is very formulaic, and it's great at that. Sorry, WordPress devs.
I wonder if the independent studies that show Copilot increasing the rate of errors in software have anything to do with this less bold attitude. Most people selling AI are predicting the obsolescence of human authors.
Even if code is the right medium for specifying a program, transformers act as an automated interface between that medium and natural language. Modern high-end transformers have no problem producing code, while benefiting from a wealth of knowledge that far surpasses any individual.
> Most people selling AI are predicting the obsolescence of human authors.
It's entirely possible that we do become obsolete for a wide variety of programming domains. That's simply a reality, just as weavers saw massive layoffs in the wake of the automated loom, or scribes lost work after the printing press, or human calculators became pointless after high-precision calculators became commonplace.
This replacement might not happen tomorrow, or next year, or even in the next decade, but it's clear that we are able to build capable models. What remains to be done is R&D around things like hallucinations, accuracy, affordability, etc. as well as tooling and infrastructure built around this new paradigm. But the cat's out of the bag, and we are not returning to a paradigm that doesn't involve intelligent automation in our daily work; programming is literally about automating things and transformers are a massive forward step.
That doesn't really mean anything, though; You can still be as involved in your programming work as you'd like. Whether you can find paid, professional work depends on your domain, skill level and compensation preferences. But you can always program for fun or personal projects, and decide how much or how little automation you use. But I will recommend that you take these tools seriously, and that you aren't too dismissive, or you could find yourself left behind in a rapidly evolving landscape, similarly to the advent of personal computing and the internet.
It will also still happily turn your whole codebase into garbage rather than undo the first thing it tried to try something else. I've yet to see one that can back itself out of a logical corner.
That's not true at all.
...
It's only pretending to be happy.
See, this is the kind of conception of a programmer I find completely befuddling. Programming isn't like those jobs at all. There's a reason people who are overly attached to code and see their job as "writing code" are pejoratively called "code monkeys." Did CAD kill the engineer? No. It didn't. The idea is ridiculous.
Perhaps LLM can be modified to step outside the circle, but as of today, it would be akin to monkeys typing.
But once you add repo context, domain knowledge etc... programming languages are far too verbose.
Here's a fresh example that I stumbled upon just a few hours ago. I needed to refactor some code that first computes the size of a popup, and then separately, the top left corner.
For brevity, one part used an "if", while the other one had a "switch":
I wanted the LLM to refactor it to store the position rather than applying it immediately. Turns out, it just could not handle different things (if vs. switch) doing a similar thing. I tried several variations of prompts, but it very strongly leaning to either have two ifs, or two switches, despite rather explicit instructions not to do so.It sort of makes sense: once the model has "completed" an if, and then encounters the need for a similar thing, it will pick an "if" again, because, well, it is completing the previous tokens.
Harmless here, but in many slightly less trivial examples, it would just steamroll over nuance and produce code that appears good, but fails in weird ways.
That said, splitting tasks into smaller parts devoid of such ambiguities works really well. Way easier to say "store size in m_StateStorage and apply on render" than manually editing 5 different points in the code. Especially with stuff like Cerebras, that can chew through complex code at several kilobytes per second, expanding simple thoughts faster than you could physically type them.
Give it to o3 and it could definitely handle that today.
Sweeping generalizations about how LLMs will never be able to do X, Y, or Z coding task will all be proven wrong with time, imo.
Whatever can be statistically predicted
by the human brain
Will one day also be
statistically predicted by melted sand
Or the current strategies to scale across boards instead of chips gets too expensive in terms of cost, capital, and externalities.
Look at how well Deepseek performed with the limited, outdated hardware available to its researchers. And look at what demoscene practitioners have accomplished on much older hardware. Even if physical breakthroughs ceased or slowed down considerably, there is still a ton left on the table in terms of software optimization and theory advancement.
And remember just how young computer science is as a field, compared to other human practices that have been around for hundreds of thousands of years. We have so much to figure out, and as knowledge begets more knowledge, we will continue to figure out more things at an increasing pace, even if it requires increasingly large amounts of energy and human capital to make a discovery.
I am confident that if it is at all possible to reach human-level intelligence at least in specific categories of tasks, we're gonna figure it out. The only real question is whether access to energy and resources becomes a bigger problem in the future, given humanity's currently extraordinarily unsustainable path and the risk of nuclear conflict or sustained supply chain disruption.
When I tried giving top-notch LLMs harder tasks (scan an abstract syntax tree coming from a parser in a particular way, and generate nodes for particular things) they completely blew it. Didn't even compile, let alone logical errors and missed points. But once I broke down the problem to making lists of relevant parsing contexts, and generating one wrapper class at a time, it saved me a whole ton of work. It took me a day to accomplish what would normally take a week.
Maybe they will figure it out eventually, maybe not. The point is, right now the technology has fundamental limitations, and you are better off knowing how to work around them, rather than blindly trusting the black box.
I think it's a combination of
1) wrong level of granularity in prompting
2) lack of engineering experience
3) autistic rigidity regarding a single hallucination throwing the whole experience off
4) subconscious anxiety over the threat to their jerbs
5) unnecessary guilt over going against the tide; anything pro AI gets heavily downvoted on Reddit and is, at best, controversial as hell here
I, for one, have shipped like literally a product per day for the last month and it's amazing. Literally 2,000,000+ impressions, paying users, almost 100 sign ups across the various products. I am fucking flying. Hit the front page of Reddit and HN countless times in the last month.
Idk if I break down the prompts better or what. But this is production grade shit and I don't even remember the last time I wrote more than two consecutive lines of code.
Except, not all work is like that. Fast-forward to product version 2.34 where a particular customer needs a change that could break 5000 other customers because of non-trivial dependencies between different parts of the design, and you will be rewriting the entire thing by humans or having it collapse under its own weight.
But out of 100 products launched on the market, only 1 or 2 will ever reach that stage, and having 100 LLM prototypes followed by 2 thoughtful redesigns is way better than seeing 98 human-made products die.
AI stands for Artificial Intelligence. There are no inherent limits around what AI can and can't do or comprehend. What you are specifically critiquing is the capability of today's popular models, specifically transformer models, and accompanying tooling. This is a rapidly evolving landscape, and your assertions might no longer be relevant in a month, much less a year or five years. In fact, your criticism might not even be relevant between current models. It's one thing to speak about idiosyncrasies between models, but any broad conclusions drawn outside of a comprehensive multi-model review with strict procedure and controls is to be taken with a massive grain of salt, and one should be careful to avoid authoritative language about capabilities.
It would be useful to be precise in what you are critiquing, so that the critique actually has merit and applicability. Even saying "LLM" is a misnomer, as modern transformer models are multi-modal and trained on much more than just textual language.
Lots of us are interested in technology that's actually available, and we can all read date stamps on comments.
> But it ain't here yet buddy . . . we can all read date stamps on comments.
That has no bearing on the general trajectory that we are currently on in computer science and informatics. Additionally, your language is patronizing and dismissive, trading substance for insult. This is generally frowned upon in this community.
You failed to actually address my comment, both by failing to recognize that it was mainly about using the correct terminology instead of criticizing an entire branch of research that extends far beyond transformers or LLMs, and by failing to establish why a rapidly evolving landscape does not mean that certain generalizations cannot yet be made, unless they are presented with several constraints and caveats, which includes not making temporally-invariant claims about capabilities.
I would ask that you reconsider your approach to discourse here, so that we can avoid this thread degenerating into an emotional argument.
They were obviously not trying to make a sweeping comment about the entire future of the field.
Are you using ChatGPT to write your loquacious replies?
OP said “AI can very efficiently apply common patterns to vast amounts of code, but it has no inherent "idea" of what it's doing.”
I'm not going to patronize you by explaining why this is not "very precise", or why its lack of temporal caveats is an issue, as I've already done so in an earlier comment. If you're still confused, you should read the sentence a few times until you understand. OP did not even mention which specific model they tested, and did not provide any specific prompt example.
> Are you using ChatGPT to write your loquacious replies?
If you can't handle a few short paragraphs as a reply, or find it unworthy of your time, you are free to stop arguing. The Hacker News guidelines actually encourage substantive responses.
I also assume that in the future, accusing a user of using ChatGPT will be against site guidelines, so you may as well start phasing that out of your repertoire now.
Here are some highlights from the Hacker News guidelines regarding comments:
- Don't be snarky
- Comments should get more thoughtful and substantive, not less, as a topic gets more divisive.
- Assume good faith
- Please don't post insinuations about astroturfing, shilling, brigading, foreign agents, and the like. It degrades discussion and is usually mistaken.
https://news.ycombinator.com/newsguidelines.html
It also requires a fair bit of wisdom to know where the software is expected to grow, and how to architect for that eventuality.
I can't picture an LLM doing a fraction of that work.
1. There was no error on line 91, it did some inconsequential formatting on that line 2. More importantly, it just ignored the very specific line I told it to go to. It's like I was playing telephone with the LLM which felt so strange with text-based communication.
This was me trying to get better at using the LLM while coding and seeing if I could "one-shot" some very simple things. Of course me doing this _very_ tiny fix myself would have been faster. Just felt weird and reinforces this idea that the LLM isn't actually thinking at all.
And now you've learned that LLMs can't count lines. Next time, try asking it to "fix the error in function XYZ" or copy/paste the line in question, and see if you get better results.
> reinforces this idea that the LLM isn't actually thinking at all.
Of course it's not thinking, how could it? It's just a (rather big) equation.
You need to give LLMs context. Line number isn't good context.
a line number is plenty of context - it's directly translatable into a range of bytes/characters in the file
(Wow I sound triggered! sigh)
this is okay! it's a sign of your humanity :)
A man is a human.
“Manual” comes from Latin manus, meaning “hand”: https://en.wiktionary.org/wiki/manus. It literally means “by hand”.
Google itself said 30% of their code is AI generated, and yet they had a recent outage worldwide, coincidence??
You tell me.
To expand -
1. Models "reason" and can increasingly generate code given natural language. Its not just fancy autocomplete, its like having an intern - mid level engineer at your beck and call to implement some feature. Natural language is generally sufficient enough when I interact with other engineers, why is it not sufficient for an AI, which (in the limit), approaches an actual human engineer?
2. Business wise, companies will not settle for augmentation. Software companies pay tons of money in headcount, its probably most mid-sized companies top or second line item. The endgame for leadership at these companies is to do more with less. This necessitates automation (in addition to augmenting the remaining roles).
People need to stop thinking of LLMs as "autocomplete on steroids" and actually start thinking of them as a "24/7 junior SWE who doesn't need to eat or sleep and can do small tasks at 90% accuracy with some reasonable spec." Yeah you'll need to edit their code once in a while but they also get better and cost less than an actual person.
Why not look at Bob who takes like 2 weeks to write tickets on what they actually want in a feature? Or Alice who's really slow getting Figma designs done and validated? How nice would having a "someone's bothered a developer" metric be and having the business seek to get that to zero and talk very loudly about it as they have about developers?
Usually the CEO or investor says 30% (or some other made up number) of all code is written by AI and the number will only increase, implying that developers will soon be obsolete.
It's implied that 30% of all code submitted and shipped to production is from AI agents with zero human interaction. But of course this is not the case, it's the same developers as before using tools to more rapidly write code.
And writing code is only one part of a developer's job in building software.
I think an over-reliance, or perhaps any reliance, on AI tools will turn good programmers into slop factories, as they consistently skip over a vital part of creating high-quality software.
You could argue that the prompt == code, but then you are adding an intermediary step between you and the code, and something will always be lost in translation.
I'd say just write the code.
With AI, instead of starting with zero and building up, you can start with a result and iterate on it straight away. This process really shines when you have a good idea of what you want to do, and how you want it implemented. In these cases, it is really easy to review the code, because you knew what you wanted it to look like. And so, it lets me implement some basic features in 15 minutes instead of an hour. This is awesome.
For more complex ideas, AI can also be a great idea sparring partner. Claude Code can take a paragraph or two from me, and then generate a 200-800 line planning document fleshing out all the details. That document: 1) helps me to quickly spot roadblocks using my own knowledge, and 2) helps me iterate quickly in the design space. This lets me spend more time thinking about the design of the system. And Claude 4 Opus is near-perfect at taking one of these big planning specifications and implementing it, because the feature is so well specified.
So, the reality is that AI opens up new possible workflows. They aren't always appropriate. Sometimes the process of writing the code yourself and iterating on it is important to helping you build your mental model of a piece of functionality. But a lot of the time, there's no mystery in what I want to write. And in these cases, AI is brilliant at speeding up design and implementation.
I know the tools and environments I am working in. I verify the implementations I make by testing them. I review everything I am generating.
The idea that AI is going to trick me is absurd. I'm a professional, not some vibe coding script kiddie. I can recognise when the AI makes mistakes.
Have the humility to see that not everyone using AI is someone who doesn't know what they are doing and just clicks accept on every idea from the AI. That's not how this works.
> Source: The Times of India
what in the recycled content is this trash?
Lots of changes where describing them in English takes longer than just performing the change. I think the most effective workflow with AI agents will be a sort of active back-and-forth.
https://en.m.wikipedia.org/wiki/Rubber_duck_debugging
Why wouldn't your boss ask ChatGPT directly?
Are they cool and useful? Yes.
Do they reason? No. (Before you complain, please first define reason).
Are they end all be all of all problem solving and the dawn of AGI? Also no.
Once we actually bring some rationality back into the radius of discourse maybe we'll actually begin to start figuring out how these things actually fit into an engineering workflow and stop throwing around ridiculous terms like vibe coding.
If you are an engineer, you are signing up to build rigorously verified and validated system, preferably with some amount of certainty about their behavior under boundary conditions. All the current hype-addled discussion around LLM seems to have had everything but correctness as it's focus.
[^1]: It shouldn't take a CEO but many people, even technologists, who should be more rational about whose opinions they deem worthy of consideration, m seem to overvalue the opinions of the csuite for some bizarre, inexplicable reason.
I wanted some jq script and it made an incomprehensible thing that usually worked. And every fix it made broke something else. It just kept digging a deeper hole.
After a while of that, I bit the bullet and learned jq script. I wrote a solution that was more capable, easier to read, and, importantly, always worked.
The thing is, jq is completely documented. Everything I needed was online. But there just aren't many examples to for LLMs, so they choke.
Start asking it questions about complexity classes and you can get it to contradict itself in no time.
Most of the code in the world right now is very formulaic, and it's great at that. Sorry, WordPress devs.
It's a powerful tool, but it's not that powerful.