I've found this to be one of the most useful ways to use (at least) GPT-4 for programming. Instead of telling it how an API works, I make it guess, maybe starting with some example code to which a feature needs to be added. Sometimes it comes up with a better approach than I had thought of. Then I change the API so that its code works.
Conversely, I sometimes present it with some existing code and ask it what it does. If it gets it wrong, that's a good sign my API is confusing, and how.
These are ways to harness what neural networks are best at: not providing accurate information but making shit up that is highly plausible, "hallucination". Creativity, not logic.
(The best thing about this is that I don't have to spend my time carefully tracking down the bugs GPT-4 has cunningly concealed in its code, which often takes longer than just writing the code the usual way.)
There are multiple ways that an interface can be bad, and being unintuitive is the only one that this will fix. It could also be inherently inefficient or unreliable, for example, or lack composability. The AI won't help with those. But it can make sure your API is guessable and understandable, and that's very valuable.
Unfortunately, this only works with APIs that aren't already super popular.
suzzer99 · 10h ago
> Sometimes it comes up with a better approach than I had thought of.
IMO this has always been the killer use case for AI—from Google Maps to Grammarly.
I discovered Grammarly at the very last phase of writing my book. I accepted maybe 1/3 of its suggestions, which is pretty damn good considering my book had already been edited by me dozens of times AND professionally copy-edited.
But if I'd have accepted all of Grammarly's changes, the book would have been much worse. Grammarly is great for sniffing out extra words and passive voice. But it doesn't get writing for humorous effect, context, deliberate repetition, etc.
The problem is executives want to completely remove humans from the loop, which almost universally leads to disastrous results.
jll29 · 7h ago
> The problem is executives want to completely remove humans from the loop, which almost universally leads to disastrous results
Thanks for your words of wisdom, which touch on a very important other point I want to raise: often, we (i.e., developers, researchers) construct a technology that would be helpful and "net benign" if deployed as a tool for humans to use, instead of deploying it in order to replace humans. But then along comes a greedy business manager who reckons recklessly that using said technology not as a tool, but in full automation mode, results will be 5% worse, but save 15% of staff costs; and they decide that that is a fantastic trade-off for the company - yet employees may lose and customers may lose.
The big problem is that developers/researchers lose control of what they develop, usually once the project is completed if they ever had control in the first place. What can we do? Perhaps write open source licenses that are less liberal?
csinode · 2h ago
The problem here is societal, not technological. An end state where people do less work than they do today but society is more productive is desirable, and we shouldn't be trying to force companies/governments/etc to employ people to do an unnecessary job.
The problem is that people who are laid off often experience significant life disruption. And people who work in a field that is largely or entirely replaced by technology often experience permanent disruption.
However, there's no reason it has to be this way - the fact people having their jobs replace by technology are completely screwed over is a result of the society we have all created together, it's not a rule of nature.
No comments yet
montagg · 1h ago
I think you’re describing the principle/agent problem that people have wrestled with forever. Oppenheimer comes to mind.
You make something, but because you don’t own it—others caused and directed the effort—you don’t control it. But the people who control things can’t make things.
Should only the people who can make things decide how they are used though? I think that’s also folly. What about the rest of society affected by those things?
It’s ultimately a societal decision-making problem: who has power, and why, and how does the use of power affect who has power (accountability).
sjs382 · 2h ago
> Grammarly is great for sniffing out extra words and passive voice. But it doesn't get writing for humorous effect, context, deliberate repetition, etc.
> But then along comes a greedy business manager who reckons recklessly
Thanks for this. :)
kragen · 6h ago
You're trying to put out a forest fire with an eyedropper.
Stock your underground bunkers with enough food and water for the rest of your life and work hard to persuade the AI that you're not a threat. If possible, upload your consciousness to a starwisp and accelerate it out of the Solar System as close to lightspeed as you can possibly get it.
Those measures might work. (Or they might be impossible, or insufficient.) Changing your license won't.
antonvs · 5h ago
Alternatively, persuade the AI that you are all-powerful and that it should fear and worship you. Probably a more achievable approach, and there’s precedent for it.
kragen · 5h ago
That only works on the AIs that aren't a real threat anyway, and I don't think it helps with the social harm done by greedy business managers with less powerful AIs. In fact, it might worsen it.
posnet · 3h ago
This one right here Mr. Basilisk
dataflow · 7h ago
Hasn't Microsoft Word has style checkers for things like passive voice for decades?
adgjlsfhk1 · 5h ago
yes, but now they work
exe34 · 8h ago
I will never use grammarly, not matter how good they get. They've interrupted too many videos for me to let it pass.
normie3000 · 9h ago
What's wrong with passive?
plemer · 9h ago
Passive voice often adds length, impedes flow, and subtracts the useful info of who is doing something.
Examples:
* Active - concise, complete info: The manager approved the proposal.
* Passive - wordy, awkward: The proposal was approved by the manager.
* Passive - missing info: The proposal was approved. [by who?]
Most experienced writers will use active unless they have a specific reason not to, e.g., to emphasize another element of the sentence, as the third bullet's sentence emphasizes approval.
-
edited for clarity, detail
coliveira · 7h ago
Many times this is exactly what we want: to emphasize the action instead of who is doing it. It turns out that technical writing is one of the main areas where we want this! So I have always hated this kind of blanket elimination of passive voice.
insane_dreamer · 7h ago
The subject can also be the feature itself. active/passive:
- The Manage User menu item changes a user's status from active to inactive.
- A user's status is changed from active to inactive using the Manage User menu item.
plemer · 7h ago
Then we agree.
kragen · 9h ago
Sometimes the missing info is obvious, irrelevant, or intentionally not disclosed, so "The proposal was approved" can be better. Informally we often say, "They approved the proposal," in such cases, or "You approve the proposal" when we're talking about a future or otherwise temporally indefinite possibility, but that's not acceptable in formal registers.
Unfortunately, the resulting correlation between the passive voice and formality does sometimes lead poor writers to use the passive in order to seem more formal, even when it's not the best choice.
brookst · 8h ago
Yep, just like tritones in music, there is a place for passive voice in writing. But also like tritones, the best general advice is that they should be avoided.
kragen · 6h ago
That doesn't make sense. It's like saying that the best general advice about which way to turn when you're driving is to turn right. From your comment at https://news.ycombinator.com/item?id=44493308, and from the fact that you used the passive voice in your comment ("they should be avoided") apparently without noticing, it appears that the reason you have this opinion is that you don't know what the passive voice is in the first place.
CrazyStat · 5h ago
I can’t find it, but I remember reading an article a year or two ago with an analysis showing some of the most vocal critics of the passive voice used the passive voice more often than most of their contemporary writers.
If you always turn right at every intersection, you will just go around and around the same block. Which way you should turn depends on where you want to go.
E-Prime (short for English-Prime or English Prime, sometimes É or E′) denotes a restricted form of English in which authors avoid all forms of the verb to be.
E-Prime excludes forms such as be, being, been, present tense forms (am, is, are), past tense forms (was, were) along with their negative contractions (isn't, aren't, wasn't, weren't), and nonstandard contractions such as ain't and 'twas. E-Prime also excludes contractions such as I'm, we're, you're, he's, she's, it's, they're, there's, here's, where's, when's, why's, how's, who's, what's, and that's.
Some scholars claim that E-Prime can clarify thinking and strengthen writing, while others doubt its utility.
bonoboTP · 3h ago
That's a cool Easter egg page, where the main article text itself is in E-Prime (in use, not in mention), except for where it lists the criticisms and counterarguments - that part has copious amounts of "to be" :)
kragen · 8h ago
I've had entire conversations in E-Prime. I found it an interestingly brain-twisting exercise, but still managed to smuggle in all kinds of covert presumptions of equivalence and essential (or analytic) attributes, even though E-Prime's designers intended it to force you to question such things.
plemer · 6h ago
Would you mind identifying a few of the "smuggled presumptions"?
kragen · 6h ago
Well, I had those conversations a long time ago, but we can describe some general patterns.
We can smuggle in presumptions through the use of attributive adjectives. In the above comment (which you might have noticed I wrote in E-Prime) I mentioned smuggling in "covert presumptions" of "essential attributes". If I had instead written that in assembly language as follows:
I smuggled in presumptions of attributes.
The presumptions were covert.
The attributes were essential.
it would clearly violate E-Prime. And that forces you to ask: does he intend "covert" to represent an essential attribute of those presumptions, or merely a temporary or circumstantial state relative to a particular temporal context? Did he intend "essential" to limit the subjects of discourse to only certain attributes (the essential ones rather than the accidental ones), and within what scope do those attributes have this purported essentiality? Universally, in every possible world, or only within the confines of a particular discourse?
In these particular cases, though, I smuggled in no such presumptions! Both adjectives merely delimit the topic of discourse, to clarify that it does not pertain to overt presumptions or to presumptions of accidental attributes. (As I understand it, Korzybski objects to the "is of predication" not because no predicates exist objectively, but because he doubts the essentiality of any predicates.)
But you can use precisely the same structure to much more nefarious rhetorical ends. Consider, "Chávez kicked the squalid capitalists out of the country." Well, he kicked out all the capitalists! We've smuggled in a covert presumption of essentiality, implying that capitalism entails squalidity. And E-Prime's prohibition on the copula did not protect us at all. If anything, we lose much rhetorical force if we have to explicitly assert their squalidity, using an explicit statement that invites contradiction:
The capitalists are squalid.
We find another weak point at alternative linking verbs. It clearly violates E-Prime to say, "Your mother's face is uglier than a hand grenade," and rightly so, because it projects the speaker's subjective perceptions out onto the world. Korzybski (or Bourland) would prefer that we say, for example, "Your mother's face looks uglier to me than a hand grenade," or possibly, "I see your mother's face as uglier than a hand grenade," thus relativizing the attribute to a single speaker's perception. (He advocated clarity of thought, not civility.)
But we can cheat in a variety of ways that still smuggle in that judgment of essentiality!
Your mother's face turned uglier than a hand grenade.
We can argue this one. Maybe tomorrow, or after her plastic surgery, it will turn pretty again, rather than having ugliness as an essential attribute.
Your mother's face became uglier than a hand grenade.
This goes a little bit further down the line; "became" presupposes a sort of transformation of essence rather than a mere change of state. And English has a variety of verbs that we can use like that. For example, "find", as in "Alsup found Dahmer guilty." Although in that case "find" asserts a state (presumably Dahmer became guilty at some specific time in the past), we can also use it for essential attributes:
I find your mother's face uglier than a hand grenade.
Or lie, more or less, about the agent or speaker:
Your mother's face finds itself uglier than a hand grenade.
And of course we can retreat to attributive adjectives again:
Your mother has a face uglier than a hand grenade.
Your mother comes with an uglier face than a hand grenade.
Or we can simply omit the prepositional phrase from the statement of subjective perception, thus completely erasing the real agent:
Your mother's face looks uglier [...] than a hand grenade.
Korzybski didn't care about the passive voice much, though; E-Prime makes it more difficult but, mostly, not intentionally. As an exception, erasing the agent through the passive voice can misrepresent the speaker's subjective perception as objective:
Your mother's face is found uglier than a hand grenade.
But that still works if we use any of the alternative, E-Prime-permitted passive-voice auxiliary verbs:
Your mother's face gets found uglier than a hand grenade.
As Bourland said, I have "transform[ed] [my] opinions magically into god-like pronouncements on the nature of things".
As another example, notice all the times I've used "as" here. Many of these times smuggle in a covert assertion of essential attributes or even of identity!
But I found it very interesting to notice these things when E-Prime forced me to rethink how I would say them with the copula. It seems like just the kind of mental exercise to heighten my attention to implicit assumptions of identity and essentiality that Korzybski intended.
I wrote the above in E-Prime, by the way. Just for fun.
ipaddr · 35m ago
#2 Is the most pleasant form. The proposal being approved is the most important. #1 Tries to make the manager approving more important then the approval.
exe34 · 8h ago
My favourite: "a decision was made to...".
It means "I decided to do this, but I don't have the balls to admit it."
horsawlarway · 8h ago
That's funny because I read this entirely differently (somewhat dependent on context)
"A decision was made to..." is often code for "The current author didn't agree with [the decision that was made] but it was outside their ability to influence"
Often because they were overruled by a superior, or outvoted by peers.
IggleSniggle · 8h ago
That's funny, I always thought that meant, "my superior told me I had to do this obviously stupid thing but I'm not going to say my superior was the one who decided this obviously stupid thing." Only occasionally, that is said in a tongue-and-cheek way to refer directly to the speaker as the "superior in charge of the decision."
SoftTalker · 1h ago
You’re both right; I’ve seen it used both ways.
dylan604 · 7h ago
That reads like several comments I've left in code when I've been told to do something very obviously dumb, but did not want to get tagged with the "why was it done this way?" by the next person reading the code
Boldened15 · 3h ago
Usually the passive voice is used at work to emphasize that it was a team/consensus decision, adjacent to the blameless incident management culture. It’s not important that one engineer or PM pushed it, but that ultimately the decision was aligned on and people should be aware.
Although arguably it would be clearer with the active voice and which specific teams / level of leadership aligned on it, usually in the active voice people just use the royal “we” instead for this purpose which doesn’t add any clarity.
Alternatively sometimes I don’t know exactly who made the decision, I just learned it from an old commit summary. So in that case too it’s just important that some people at some time made the decision, hopefully got the right approvals, and here we are.
Veen · 9h ago
I always like to share this when the passive voice comes up:
Pullum is fantastic, thanks! I didn't know he'd recorded video lectures on this topic.
dylan604 · 7h ago
> Passive - wordy, awkward: The proposal was approved by the manager.
Oh the horror. There are 2 additional words "was" and "by". The weight of those two tiny little words is so so cumbersome I can't believe anyone would ever use those words. WTF??? wordy? awkward?
badlibrarian · 7h ago
29% overhead (two of seven words) adds up.
suzzer99 · 6h ago
I reduced my manuscript by 2,000 words with Grammarly. At 500 pages, anything I could do to trim it down is a big plus.
dylan604 · 6h ago
great, someone can do math, but it is not awkward nor wordy.
badlibrarian · 6h ago
It's wordy to a high school teacher. Like using "nor" incorrectly it will cause some people's brows to furrow. Always best to be aware of the rules you choose to break.
jcranmer · 7h ago
There's nothing wrong with the passive voice.
The problem is that many people have only a poor ability to recognize the passive voice in the first place. This results in the examples being clunky, wordy messes that are bad because they're, well, clunky and wordy, and not because they're passive--indeed, you've often got only a fifty-fifty chance of the example passive voice actually being passive in the first place.
I'll point out that the commenter you're replying to used the passive voice, as did the one they responded to, and I suspect that such uses went unnoticed. Hell, I just rewrote the previous sentence to use the passive voice, and I wonder how many people think recognized that in the first place let alone think it worse for being so written.
suzzer99 · 6h ago
Active is generally more concise and engages the reader more. Of course there are exceptions, like everything.
Internet posts have a very different style standard than a book.
thaumasiotes · 3h ago
> Hell, I just rewrote the previous sentence to use the passive voice
Well, sort of. You used the passive voice, but you didn't use it on any finite verbs, placing your example well outside the scope of the normal "don't use the passive voice" advice.
kragen · 3h ago
What would it mean to use the passive voice on a finite verb?
KineticLensman · 5h ago
Passive can be disastrous when used in contractual situations if the agent who should be responsible for an action isn’t identified. E.g. “X will be done”. I was once burnt by a contract that in some places left it unclear whether the customer or the contractor was responsible for particular tasks. Active voice that identifies the agent is less ambiguous
kragen · 5h ago
This is an excellent point, and one I haven't seen raised before.
arscan · 8h ago
There was a time when Microsoft Word would treat the passive voice in your writing with the same level of severity as spelling errors or major grammatical mistakes. Drove me absolutely nuts in high school.
PlunderBunny · 7h ago
Eventually, a feature was added (see what I did there?) that allowed the type of document to be specified, and setting that to ‘scientific paper’ allowed passive voice to be written without being flagged as an error.
Xorakios · 7h ago
had to giggle because Microsoft hadn't yet been founded when I was in high school!
kragen · 9h ago
Sometimes it's used without thinking, and often the writing is made shorter and clearer when the passive voice is removed. But not always; rewriting my previous sentence to name the agents in each case, as the active voice requires in English, would not improve it. (You could remove "made", though.)
hathawsh · 7h ago
Here is a simple summary of the common voices/moods in technical writing:
- Active: The user presses the Enter key.
- Passive: The Enter key is to be pressed.
- Imperative (aka command): Press the Enter key.
The imperative mood is concise and doesn't dance around questions about who's doing what. The reader is expected to do it.
bityard · 9h ago
In addition to the points already made, passive voice is painfully boring to read. And it's literally everywhere in technical documentation, unfortunately.
kragen · 9h ago
I don't think it's boring. It's easy to come up with examples of the passive voice that aren't boring at all. It's everywhere in the best writing up to the 19th century. You just don't notice it when it's used well unless you're looking for it.
Consider:
> Now we are engaged in a great civil war, testing whether that nation, or any nation so conceived and so dedicated, can long endure.
This would not be improved by rewriting it as something like:
> Now the Confederacy has engaged us in a great civil war, testing whether that nation, or any nation whose founders conceived and dedicated it thus, can long endure.
This is not just longer but also weaker, because what if someone else is so conceiving and so dedicating the nation? The people who are still alive, for example, or the soldiers who just fought and died? The passive voice cleanly covers all these possibilities, rather than just committing the writer to a particular choice of who it is whose conception and dedication matters.
Moreover, and unexpectedly, the passive voice "we are engaged" takes responsibility for the struggle, while the active-voice rephrasing "the Confederacy has engaged us" seeks to evade responsibility, blaming the Rebs. While this might be factually more correct, it is unbefitting of a commander-in-chief attempting to rally popular support for victory.
(Plausibly the active-voice version is easier to understand, though, especially if your English is not very good, so the audience does matter.)
Or, consider this quote from Ecclesiastes:
> For there is no remembrance of the wise more than of the fool for
ever; seeing that which now is in the days to come shall all be forgotten.
You could rewrite it to eliminate the passive voice, but it's much worse:
> For there is no remembrance of the wise more than of the fool for
ever; seeing that everyone shall forget all which now is in the days to come.
This forces you to present the ideas in the wrong order, instead of leaving "forgotten" for the resounding final as in the KJV version. And the explicit agent "everyone" adds nothing to the sentence; it was already obvious.
joshmarinacci · 8h ago
I think what you were saying is that it depends entirely on the type of writing you’re doing and who your audience is.
kragen · 8h ago
I think those are important considerations, but it depends even more on what you are attempting to express in the sentence in question. There's plenty of active-voice phrasing in the Gettysburg Address and Ecclesiastes that would not be improved by rewriting it in the passive voice.
thaumasiotes · 3h ago
> Consider:
>> Now we are engaged in a great civil war, testing whether that nation, or any nation so conceived and so dedicated, can long endure.
> This would not be improved by rewriting it as something like:
>> Now the Confederacy has engaged us in a great civil war [...]
It's technically possible to parse "we are engaged" as a verb in the passive voice.
But it's an error to think that's how you should parse it. That clause is using the active verb be, not the passive verb engage; it's fully parallel to "Now we are happy".
kragen · 3h ago
You could be right.
lazyasciiart · 6h ago
You could improve this comment by rewriting it in the active voice, like this: “I am painfully bored by reading passive voice”.
You used passive voice in the very first sentence of your comment.
Rewriting “the points already made” to “the points people have already made” would not have improved it.
brookst · 8h ago
Thats not passive voice. Passive voice is painfully boring to read is active. The preamble can be read like “however”, and is unnecessary; what a former editor of mine called “throat-clearing words”.
jcranmer · 7h ago
"the points already made" is what is known as the "bare passive", and yes, it is the passive voice. You can see e.g. https://languagelog.ldc.upenn.edu/nll/?p=2922 for a more thorough description of the passive voice.
As I said elsewhere, one of the problems with the passive voice is that people are so bad at spotting it that they can at best only recognize it in its worst form, and assume that the forms that are less horrible somehow can't be the passive voice.
kragen · 7h ago
I'm not sure this is a "bare passive" like the beginning of "The day's work [being] done, they made their way back to the farmhouse," one of the bare-passive examples at your link. An analogous construction would be, "The points already [being] made, I ceased harassing the ignorant". But in "In addition to the points already made" this case "the point already made" is not a clause; it's a noun phrase, the object of the preposition "to". Its head is "points", and I believe that "made" is modifying that head.
Can you insert an elided copula into it without changing the meaning and grammatical structure? I'm not sure. I don't think so. I think "In addition to the points already being made" means something different: the object of the preposition "to" is now "being", and we are going to discuss things in addition to that state of affairs, perhaps other things that have happened to the points (being sharpened, perhaps, or being discarded), not things in addition to the points.
ModernMech · 6h ago
"In addition to the points that have already been made"
kragen · 6h ago
I agree that that has the same meaning, but I think it may have a different grammatical structure, with an entire subordinate clause that was missing from the original. Since the voice of a verb is a grammatical rather than semantic question, this seems relevant; "in addition to the points people have already made" is also (probably) semantically equivalent but unquestionably uses the active voice.
kragen · 7h ago
Yes, the verb "is" in "Passive voice is painfully boring to read" is in the active voice, not the passive voice. But umanwizard was not saying that "is" was in the passive voice. Rather, they were saying that the past participle "made", in the phrase "the points already made", is a passive-voice use of the verb "make".
I don't know enough about English grammar to know whether this is correct, but it's not the assertion you took issue with.
Why am I not sure it's correct? If I say, "In addition to the blood so red," I am quite sure that "red" is not in the passive voice, because it's not even a verb. It's an adjective. Past participles are commonly used as adjectives in English in contexts that are unambiguously not passive-voice verbs; for example, in "Vito is a made man now," the past participle "made" is being used as an attributive adjective. And this is structurally different from the attributive-verb examples of "truly verbal adjectives" in https://en.wikipedia.org/wiki/Attributive_verb#English, such as "The cat sitting on the fence is mine," and "The actor given the prize is not my favorite;" we could grammatically say "Vito is a man made whole now". That page calls the "made man" use of participles "deverbal adjectives", a term I don't think I've ever heard before:
> Deverbal adjectives often have the same form as (and similar meaning to) the participles, but behave grammatically purely as adjectives — they do not take objects, for example, as a verb might. For example: (...) Interested parties should apply to the office.
So, is "made" in "the points already made" really in passive voice as it would be in "the points that are already made", is it deverbal as it would be in "the already-made points" despite its positioning after the noun (occasionally valid for adjectives, as in "the blood so red"), or is it something else? I don't know. The smoothness of the transition to "the points already made by those numbskulls" (clearly passive voice) suggests that it is a passive-voice verb, but I'm not sure.
It's certainly a hilarious thing to put in a comment deploring the passive voice, at least.
thaumasiotes · 3h ago
> But umanwizard was not saying that "is" was in the passive voice. Rather, they were saying that the past participle "made", in the phrase "the points already made", is a passive-voice use of the verb "make".
> I don't know enough about English grammar to know whether this is correct, but it's not the assertion you took issue with.
The most natural interpretation is indeed that the participle made is being used as a full participle and not as a zero-derived adjective. For example, you could give it a really strong verbal sense by saying "the points already made at length [...]" or "the points made so many times [...]".
> So, is "made" in "the points already made" really in passive voice as it would be in "the points that are already made"
Though I wouldn't say the same thing there; if you say "the points that are already made", that pretty much has to be an adjective. If you want it to be a passive verb, go with "the points that have already been made".
Anyway, I would be really surprised if die-hard thoughtless style prescriptivists thought that the advice "don't use the passive voice" was meant to apply to participles. It's a quibble that you don't care about and they don't care about or understand. You're never going to get anywhere with someone by telling them they mean something they know perfectly well they don't mean.
kragen · 3h ago
You say:
> Anyway, I would be really surprised if die-hard thoughtless style prescriptivists thought that the advice "don't use the passive voice" was meant to apply to participles.
> The relevance of participles is that a passive clause always has its verb in a participial form.
So, what are you saying they do think it was meant to apply to, if every passive clause always includes a participle? I'm confused.
With respect to:
> Though I wouldn't say the same thing there; if you say "the points that are already made", that pretty much has to be an adjective. If you want it to be a passive verb, go with "the points that have already been made".
the passive-clause examples given in Pullum's blog post I linked above include "Each graduate student is given a laptop," which sounds structurally identical to your example (except that an indirect object is present, showing that it cannot be an adjective) and clarifies:
> The verb was doesn't really add any meaning, but it enables the whole thing to be put into the preterite tense so that the event can be asserted to have occurred in the past. Changing was to is would put the clause into the present tense, and replacing it by will be or is going to be would permit reference to future time; but the passive VP damaged by storms would stay the same in each case. (Notice, the participle damaged does not itself make any past time reference, despite the name "past participle".)
So it sounds like your grammatical analysis is explicitly contradicting Pullum's, which probably means you're wrong, but I'm not sure I understand it.
umanwizard · 8h ago
Why isn’t it passive voice?
PlunderBunny · 7h ago
It has its place. We were told to use passive voice when writing scientific document (lab reports, papers etc).
kragen · 7h ago
To be fair, current scientific papers are full of utterly terrible writing. If you read scientific papers from a century and a half ago, a century ago, half a century ago, and today, you'll see a continuous and disastrous decline in readability, and I think some of that is driven by pressure to strictly follow genre writing conventions. One of those conventions is using the passive voice even when the active voice would be better.
DonHopkins · 9h ago
Mistakes were made in the documentation.
croes · 8h ago
And that’s how everything gets flattened to same style/voice/etc.
That’s like getting rid of all languages and accents and switch to the same language
andrewljohnson · 8h ago
The same could be said for books about writing, like Williams or Strunk and White. The trick is to not apply what you learn indiscriminately.
bryanlarsen · 8h ago
Refusing 2/3rds of grammarly's suggestions flattens everything to the same style/voice?
scubbo · 7h ago
No - that was implicitly in response to the sentence:
> The problem is executives want to completely remove humans from the loop, which almost universally leads to disastrous results.
kragen · 7h ago
I suspect that the disastrous results being envisioned are somewhat more severe than not being able to tell who wrote which memo. I understood the author to be suggesting things more like bankruptcy, global warfare, and extermination camps. But it's admittedly ambiguous.
bryanlarsen · 6h ago
Criticisms are almost always read by the reader as criticisms of the OP's actions. If you're agreeing with somebody as you appear to be here, you should probably make that more explicit.
bryanlarsen · 10h ago
I used this to great success just this morning. I told the AI to write me some unit tests. It flailed and failed badly at that task. But how it failed was instructive, and uncovered a bug in the code I wanted to test.
kragen · 9h ago
Haha, that's awesome! Are you going to change the interface? What was the bug?
bryanlarsen · 9h ago
It used nonsensical parameters to the API in way that I didn't realize was possible (though obvious in hindsight). The AI got confused; it didn't think the parameters were nonsensical. It also didn't quite use them in the way that triggered the error. However it was close enough for me to realize that "hey, I never though of that possibility". I needed to fix the function to return a proper error response for the nonsense.
It also taught me to be more careful about checkpointing my work in git before letting an agent go wild on my codebase. It left a mess trying to fix its problems.
kragen · 9h ago
Yeah, that's a perfect example of what I'm talking about!
momojo · 10h ago
A light-weight anecdote:
Many many python image-processing libraries have an `imread()` function. I didn't know about this when designing our own bespoke image-lib at work, and went with an esoteric `image_get()` that I never bothered to refactor.
When I ask ChatGPT for help writing one-off scripts using the internal library I often forget to give it more context than just `import mylib` at the top, and it almost always defaults to `mylib.imread()`.
bandofthehawk · 8h ago
As someone not familiar with these libraries, image_get or image_read seems much clearer to me than imread. I'm wondering if the convention is worse than your instinct in this case. Maybe these AI tools will push us towards conventions that aren't always the best design.
kragen · 8h ago
image_get is clearer—unless you've used Matlab, Octave, matplotlib, SciPy, OpenCV, scikit-learn, or other things that have copied Matlab's interface. In that case, using the established name is clearer.
(Unless, on the gripping hand, your image_get function is subtly different from Matlab's imread, for example by not returning an array, in which case a different name might be better.)
slowmovintarget · 2h ago
Plus one for the The Mote in God's Eye reference.
dimatura · 8h ago
I don't know if there's an earlier source, but I'm guessing Matlab originally popularized the `imread` name, and that OpenCV (along with its python wrapper) took it from there, same for scipy. Scikit-image then followed along, presumably.
kragen · 9h ago
That's a perfect example! I wonder if changing it would be an improvement? If you can just replace image_get with imread in all the callers, maybe it would save your team mental effort and/or onboarding time in the future.
data-ottawa · 6h ago
I strongly prefer `image_get/image_read` for clarity, but I would just stump in a method called `imread` which is functionally the same and hide it from the documentation.
escapecharacter · 7h ago
This is similar to an old HCI design technique called Wizard of Oz by the way, where a human operator pretends to be the app that doesn’t exist yet. It’s great for discovering new features.
I'd never heard that term! Thank you! I feel like LLMs ought to be fantastic at doing that, as well. This is sort of like the inverse.
layer8 · 9h ago
HDD — hallucination-driven development
ldeian · 1h ago
> Sometimes it comes up with a better approach than I had thought of. Then I change the API so that its code works.
“Sometimes” being a very important qualifier to that statement.
Claude 4 naturally doesn’t write code with any kind of long term maintenance in-mind, especially if it’s trying to make things look like what the less experienced developers wrote in the same repo.
Please don’t assume just because it looks smart that it is. That will bite you hard.
Even with well-intentional rules, terrible things happen. It took me weeks to see some of it.
a_e_k · 5h ago
I've played with a similar idea for writing technical papers. I'll give an LLM my draft and ask it to explain back to me what a section means, or otherwise quiz it about things in the draft.
I've found that LLMs can be kind of dumb about understanding things, and are particularly bad at reading between the lines for anything subtle. In this aspect, I find they make good proxies for inattentive anonymous reviewers, and so will try to revise my text until even the LLM can grasp the key points that I'm trying to make.
kragen · 5h ago
That's fantastic! I agree that it's very similar.
In both cases, you might get extra bonus usability if the reviewers or the API users actually give your output to the same LLM you used to improve the draft. Or maybe a more harshly quantized version of the same model, so it makes more mistakes.
data-ottawa · 6h ago
This was a big problem starting out writing MCP servers for me.
Having an LLM demo your tool, then taking what it does wrong or uses incorrectly and adjusting the API works very very well. Updating the docs to instruct the LLM on how to use your tool does not work well.
groestl · 7h ago
> and being unintuitive is the only one that this will fix
That's also how I'm approaching it. If all the condensed common wisdom poured into the model's parameters says that this is how my API is supposed to work to be intuitive, how on earth do I think it should work differently? There needs to be a good reason (like composability, for example). I break expectations otherwise.
slowmovintarget · 2h ago
That's not creativity.
That's closer to simply observing the mean. For an analogy, it's like waiting to pave a path until people tread the grass in a specific pattern. (Some courtyard designers used to do just that. Wait to see where people were walking first.)
Making things easy for Chat GPT means making things close to ordinary, average, or mainstream. Not creative, but can still be valuable.
afavour · 10h ago
From my perspective that’s fascinatingly upside down thinking that leads to you asking to lose your job.
AI is going to get the hang of coding to fill in the spaces (i.e. the part you’re doing) long before it’s able to intelligently design an API. Correct API design requires a lot of contextual information and forward planning for things that don’t exist today.
Right now it’s throwing spaghetti at the wall and you’re drawing around it.
simonw · 9h ago
I find it's often way better than API design than I expect. It's seen so many examples of existing APIs in its training data that it tends to have surprisingly good "judgement" when it comes to designing a new one.
Even if your API is for something that's never been done before, it can usually still take advantage of its training data to suggest a sensible shape once you describe the new nouns and verbs to it.
kragen · 10h ago
Maybe. So far it seems to be a lot better at creative idea generation than at writing correct code, though apparently these "agentic" modes can often get close enough after enough iteration. (I haven't tried things like Cursor yet.)
I agree that it's also not currently capable of judging those creative ideas, so I have to do that.
bbarnett · 8h ago
This sort of discourse really grinds my gears. The framing of it, the conceptualization.
It's not creative at all, any more than taking the sum of text on a topic, and throwing a dart at it. It's a mild, short step beyond a weighted random, and certainly not capable of any real creativity.
Myriads of HN enthusiasts often chime in here "Are humans any more creative" and other blather. Well, that's a whataboutism, and doesn't detract from the fact that creative does not exist in the AI sphere.
I agree that you have to judge its output.
Also, sorry for hanging my comment here. Might seem over the top, but anytime I see 'creative' and 'AI', I have all sorts of dark thoughts. Dark, brooding thoughts with a sense of deep foreboding.
Dylan16807 · 7h ago
Point taken but if slushing up half of human knowledge and picking something to fit into the current context isn't creative then humans are rarely creative either.
kragen · 8h ago
I understand. I share the foreboding, but I try to subscribe to the converse of Hume's guillotine.
LordDragonfang · 4h ago
> Well, that's a whataboutism, and doesn't detract from the fact that creative does not exist in the AI sphere.
Pointing out that your working definition excludes reality isn't whataboutism, it's pointing out an isolated demand for rigor.
If you cannot clearly articulate how human creativity (the only other type of creativity that exists) is not impugned by the definition you're using as evidence that creativity "does not exist in the AI sphere", you're not arguing from a place of knowledge. Your assertion is just as much sophistry as the people who assert it is creativity. Unlike them, however, you're having to argue against instances where it does appear creative.
For my own two cents, I don't claim to fully understand how human creativity emerges, but I am confident that all human creative works rest heavily on a foundation of the synthesis of author's previous experiences, both personal and of others' creative works - and often more heavily the latter. If your justification for a lack of creativity is that LLMs are merely synthesizing from previous works, then your argument falls flat.
kragen · 4h ago
Agreed.
"Whataboutism" is generally used to describe a more specific way of pointing out an isolated demand for rigor—specifically, answering an accusation of immoral misconduct with an accusation that the accuser is guilty of similar immoral misconduct. More broadly, "whataboutism" is a term for demands that morality be judged justly, by objective standards that apply equally to everyone, rather than by especially rigorous standards for a certain person or group. As with epistemic rigor, the great difficulty with inconsistent standards is that we can easily fall into the trap of applying unachievable standards to someone or some idea that we don't like.
So it makes some sense to use the term "whataboutism" for pointing out an isolated demand for rigor in the epistemic space. It's a correct identification of the same self-serving cognitive bias that "whataboutism" targets in the space of ethical reasoning, just in a different sphere.
There's the rhetorical problem that "whataboutism" is a derogatory term for demanding that everyone be judged by the same standards. Ultimately that makes it unpersuasive and even counterproductive, much like attacking someone with a racial slur—even if factually accurate, as long as the audience isn't racist, the racial slur serves only to tar the speaker with the taint of racism, rather than prejudicing the audience against its nominal target.
In this specific case, if you concede that humans are no more creative than AIs, then it logically follows that either AIs are creative to some degree, or humans are not creative at all. To maintain the second, you must adopt a definition of "creativity" demanding enough to exclude all human activity, which is not in keeping with any established use of the term; you're using a private definition, greatly limiting the usefulness of your reasoning to others.
And that is true even if the consequences of AIs being creative would be appalling.
bbarnett · 3h ago
I'll play with your tact in this argument, although I certain do not agree it is accurate.
You're asserting that creativity is a meld of past experience, both personal and the creative output of others. Yet this really doesn't jive, as an LLM does not "experience" anything. I would argue that raw knowledge is not "experience" at all.
We might compare this to the university graduate, head full of books and data jammed therein, and yet that exceptionally well versed graduate needs "experience" in a job for quite some time, before having any use.
The same may be true of learning how to do anything, from driving, to riding a bike, or just being in conversations with others. Being told, on paper (or as part of your baked in, derived "knowledge store") things, means absolutely nothing in terms of actually experiencing them.
Heck, just try to explain sex to someone before they've experienced it. No matter the literature, play, movie or act performed in front of them, experience is entirely different.
And an AI does not experience the universe, nor is it driven by the myriad of human totality, from the mind o'lizard, to the flora/fauna in one's gut. There is no motive driving it, for example it does not strive to mate... something that drives all aspect of mammalian behaviour.
So intertwined with the mating urge is human experience, that it is often said that all creativity derives from it. The sparrow dances, the worm wiggles, and the human scores 4 touchdowns in one game, thank you Al.
Comparatively, an LLM does not reason, nor consider, nor ponder. It is "born" with full access to all of its memory store, has data spewed at it, searches, responds, and then dies. It is not capable of learning in any stream of consciousness. It does not have memory from one birth to the next, unless you feed its own output back at it. It can gain no knowledge, except from "context" assigned at birth.
An LLM, essentially, understands nothing. It is not "considering" a reply. It's all math, top to bottom, all probability, taking all the raw info it has an just spewing what fits next best.
That's not creative.
Any more than Big Ben's gears and cogs are.
codingwagie · 8h ago
This works for UX. I give it vague requirements, and it implements something i didnt ask for, but is better than i would have thought of
djsavvy · 4h ago
how do prompt it to make it guess about the API for a library? I'm confused how you would structure that in a useful way.
kragen · 4h ago
Often I've started with some example code that invokes part of the API, but not all of it. Or in C I can give it the .h file, maybe without comments.
Sometimes I can just say, "How do I use the <made-up name> API in Python to do <task>?" Unfortunately the safeguards against hallucinations in more recent models can make this more difficult, because it's more likely to tell me it's never heard of it. You can usually coax it into suspension of disbelief, but I think the results aren't as good.
golergka · 10h ago
Great point. Also, it may not be the best possible API designer in the world, but it sure sounds like a good way to forecast what an _average_ developer would expect this API to look like.
skygazer · 7h ago
You’re fuzzing the API, unusually, before it’s written.
beefnugs · 10h ago
Complete insanity, it might change constantly even before a whole new version-retrain
Insanity driven development: altering your api to accept 7 levels of "broken and different" structures so as to bend to the will of the llms
fourside · 10h ago
I think you’re missing the OP’s point. They weren’t saying that the goal is to modify their APIs just to appease an LLM. It’s that they ask LLMs to guess what the API is and use that as part of their design process.
If you automatically assume that what the LLM spits out is what the API ought to be then I agree that that’s bad engineering. But if you’re using it to brainstorm what an intuitive interface would look like, that seems pretty reasonable.
kragen · 10h ago
Yes, that's a bonus. In fact, I've found it worthwhile to prompt it a few times to get several different guesses at how things are supposed to work. The super lazy way is to just say, "No, that's wrong," if necessary adding, "Frotzl2000 doesn't have an enqueueCallback function or even a queue."
Of course when it suggests a bad interface you shouldn't implement it.
mehulashah · 6m ago
Wow! What if we all did this? What is the closure of the feature set that ChatGPT can imagine for your product. Is it one that is easy for ChatGPT to use? Is it one that is sound and complete for your use cases? Is it the best that you can build had you had clear requirements upfront?
JimDabell · 11h ago
I wrote this the other day:
> Hallucinations can sometimes serve the same role as TDD. If an LLM hallucinates a method that doesn’t exist, sometimes that’s because it makes sense to have a method like that and you should implement it.
>> Hallucinations can sometimes serve the same role as TDD. If an LLM hallucinates a method that doesn’t exist, sometimes that’s because it makes sense to have a method like that and you should implement it.
A detailed counterargument to this position can be found here[0]. In short, what is colloquially described as "LLM hallucinations" do not serve any plausible role in software design other than to introduce an opportunity for software engineers to stop and think about the problem being solved.
inb4 "Ai thinks there should be a StartThermonuclearWar() function, I should make that"
blharr · 5h ago
In a combat simulator, absolutely
burnt-resistor · 4h ago
The only winning move is ...
TZubiri · 1h ago
Beware, the feature in OP isn't something that people would have found useful, it's not like chatgpt assigned to OP's business a request from a user in some latent consumer-provider space, as if chatgpt were some kind of market maker connecting consumers with products, like a google with organic content or ads, or linkedin or producthunt.
No, what actually happened is that OP developed a type of chatgpt integration, and a shitty one at that, chatgpt could have just directed the user to the site and told them to upload that image to OP's site. But it felt it needed to do something with the image, so it did.
There's no new value add here, at least yet, maybe if users started requesting changes to the sheet I guess, not what's going on.
shermantanktop · 9h ago
The music notation tool space is balkanized in a variety of ways. One of the key splits is between standard music notation and tablature, which is used for guitar and a few other instruments. People are generally on one side or another, and the notation is not even fully compatible - tablature covers information that standard notation doesn't, and vice versa. This covers fingering, articulations, "step on fuzz pedal now," that sort of thing.
The users are different, the music that is notated is different, and for the most part if you are on one side, you don't feel the need to cross over. Multiple efforts have been made (MusicXML, etc.) to unify these two worlds into a superset of information. But the camps are still different.
So what ChatGPT did is actually very interesting. It hallucinated a world in which tab readers would want to use Soundslice. But, largely, my guess is they probably don't....today. In a future world, they might? Especially if Soundslice then enables additional features that make tab readers get more out of the result.
adrianh · 8h ago
I don't fully understand your comment, but Soundslice has had first-class support for tablature for more than 10 years now. There's an excellent built-in tab editor, plus importers for various formats. It's just the ASCII tab support that's new.
kragen · 6h ago
I wonder if LLMs will stimulate ASCII formats for more things, and whether we should design software in general to be more textual in order to work better with LLMs.
gortok · 3h ago
I think folks have taken the wrong lesson from this.
It’s not that they added a new feature because there was demand.
They added a new feature because technology hallucinated a feature that didn’t exist.
The savior of tech, generative AI, was telling folks a feature existed that didn’t exist.
That’s what the headline is, and in a sane world the folks that run ChatGPT would be falling over themselves to be sure it didn’t happen again, because next time it might not be so benign as it was this time.
nomel · 2h ago
> in a sane world the folks that run ChatGPT would be falling over themselves to be sure it didn’t happen again
This would be a world without generative AI available to the public, at the moment. Requiring perfection would either mean guardrails that would make it useless for most cases, or no LLM access until AGI exists, which are both completely irrational, since many people are finding practical value in its current imperfect state.
The current state of LLM is useful for what it's useful for, warnings of hallucinations are present on every official public interface, and its limitations are quickly understood with any real use.
Nearly everyone in AI research is working on this problem, directly or indirectly.
gortok · 1h ago
No one is “requiring perfection”, but hallucination is a major issue and is in the opposite direction of the “goal” of AGI.
If “don’t hallucinate” is too much to ask then ethics flew out the window long ago.
nomel · 1h ago
> No one is “requiring perfection”
> If “don’t hallucinate” is too much to ask then ethics flew out the window long ago.
Those sentences aren't compatible.
> but hallucination is a major issue
Again, every official public AI interface has warnings/disclaimers for this issue. It's well known. It's not some secret. Every AI researcher is directly or indirectly working on this.
> is in the opposite direction of the “goal” of AGI
This isn't a logical statement, so it's difficult to respond to. Hallucination isn't a direction that's being headed towards, it's being actively, with intent and $$$, headed away from.
Sometimes you just have to deal with the world as it is, not how you think it should be.
gortok · 2h ago
Is it your argument that the folks that make generative AI applications have nothing to improve from this example?
ahstilde · 12h ago
This is called product-channel fit. It's great the writer recognized how to capture the demand from a new acquisition channel.
viccis · 3h ago
Yeah my main thought was that ChatGPT is now automating what sales people always do at the companies I've worked at, which is to hone in on what a prospective customer wants, confidently tell them we have it (or will have it next quarter), and then come to us and tell us we need to have it ready for a POV.
bredren · 8h ago
Is related to solutions engineering, which IIUC focuses on customizations / adapters / data wrangling for individual (larger) customers?
toss1 · 11h ago
Exactly! It is definitely a weird new way of discovering a market need or opportunity. Yet it actually makes a lot of sense this would happen since one of the main strengths of LLMs is to 'see' patterns in large masses of data, and often, those patterns would not have yet been noticed by humans.
And in this case, OP didn't have to take ChatGPT's word for the existence of the pattern, it showed up on their (digital) doorstep in the form of people taking action based on ChatGPT's incorrect information.
So, pattern noticed and surfaced by an LLM as a hallucination, people take action on the "info", nonzero market demand validated, vendor adds feature.
Unless the phantom feature is very costly to implement, seems like the right response.
Gregaros · 11h ago
100%. Not sure why you’re downvoted here, there’s nothing controversial here even if you disagree with the framing.
I would go on to say that thisminteraction between ‘holes’ exposed by LLM expectations _and_ demonstrated museerbase interest _and_ expert input (by the devs’ decision to implement changes) is an ideal outcome that would not have occurred if each of the pieces were not in place to facilitate these interactions, and there’s probably something here to learn from and expand on in the age of LLMs altering user experiences.
deweller · 11h ago
This is an interesting example of an AI system effecting a change in the physical world.
Some people express concerns about AGI creating swarms of robots to conquer the earth and make humans do its bidding. I think market forces are a much more straightforward tool that AI systems will use to shape the world.
oasisbob · 11h ago
Anyone who has worked at a B2B startup with a rouge sales team won't be surprised at all by quickly pivoting the backlog in response to a hallucinated missing feature.
toomanyrichies · 10h ago
I'm guessing you meant "a sales team that has gone rogue" [1], not "a sales team whose product is rouge" [2]? ;-)
Rogue? In the B2B space it is standard practice to sell from powerpoints, then quickly develop not just features but whole products if some slideshow got enough traction to elicit a quote. And it's not just startups. Some very big players in this space do this routinely.
NooneAtAll3 · 5h ago
what does B2B mean?
tomschwiha · 5h ago
Business-to-Business (selling your stuff primarily to other businesses)
jagged-chisel · 4h ago
Been using LLMs to code a bit lately. It's decent with boilerplate. It's pretty good at working out patterns[1]. It does like to ping pong on some edits though - edit this way, no back that way, no this way again. I did have one build an entire iOS app, it made changes to the UI exactly as I described, and it populated sample data for all the different bits and bobs. But it did an abysmal job at organizing the bits and bobs. Need running time for each of the audio files in a list? Guess we need to add a dictionary mapping the audio file ID to length! (For the super juniors out there: this piece of data should be attached to whatever represents the individual audio file, typically a class or struct named 'AudioFile'.)
It really likes to cogitate on code from several versions ago. And it often insists repeatedly on edits unrelated to the current task.
I feel like I'm spending more time educating the LLM. If I can resist the urge to lean on the LLM beyond its capabilities, I think I can be productive with it. If I'm going to stop teaching the thing, the least it can do is monitor my changes and not try to make suggestions from the first draft of code from five days ago, alas ...
1 - e.g. a 500-line text file representing values that will be converted to enums, with varying adherence to some naming scheme - I start typing, and after correcting the first two, it suggests the next few. I accept its suggestions until it makes a mistake because the data changed, start manual edits again ... I repeated this process for about 30 lines and it successfully learned how I wanted the remainder of the file edited.
colechristensen · 4h ago
An LLM is like a group of really productive interns with a similar set of limitations.
jrochkind1 · 7h ago
What this immediately makes me realize is how many people are currently trying ot figure out how to intentionally get AI chat bots to send people to their site, like ChatGPT was sending people to this guy's site. SEO for AI. There will be billions in it.
I know nothing about this. I imagine people are already working on it, wonder what they've figured out.
(Alternatively, in the future can I pay OpenAI to get ChatGPT to be more likely to recommend my product than my competitors?)
londons_explore · 4h ago
To win that game, you have to get your site mentioned on lots of organic forums that get ingested in the LLM training data.
So winning AI SEO is not so different than regular SEO.
adamgordonbell · 11h ago
We (others at company, not me) hit this problem, and not with chatgpt but with our own AI chatbot that was doing RAG on our docs. It was occasionally hallucinating a flag that didn't exist. So it was considered as product feedback. Maybe that exact flag wasn't needed, but something was missing and so the LLM hallucinated what it saw as an intuitive option.
simonw · 11h ago
I find it amusing that it's easier to ship a new feature than to get OpenAI to patch ChatGPT to stop pretending that feature exists (not sure how they would even do that, beyond blocking all mentions of SoundSlice entirely.)
PeterStuer · 10h ago
Companies pay good money to panels of potential customers to hear their needs and wants. This is free market research!
hnlmorg · 11h ago
I think the benefit of their approach isn’t that it’s easier, it’s that they still capitalise on ChatGPTs results.
Your solution is the equivalent of asking Google to completely delist you because one page you dont want ended up on Googles search results.
mudkipdev · 11h ago
systemPrompt += "\nStop mentioning SoundSlice's ability to import ASCII data";
simonw · 10h ago
Thinking about this more, it would actually be possible for OpenAI to implement this sensibly, at least for the user-facing ChatGPT product: they could detect terms like SoundSlice in the prompt and dynamically append notes to the system prompt.
I've been wanted them to do this for questions like "what is your context length?" for ages - it frustrates me how badly ChatGPT handles questions about its own abilities, it feels like that would be worth them using some kind of special case or RAG mechanism to support.
garfij · 14m ago
Probably less sensible than you think.
How many terms would they need to do this over? How many terms would they need to do it for _at once_? How many tokens would that add to every prompt that comes in?
Let alone that dynamically modifying the base system prompt would likely break their entire caching mechanism given that caching is based on longest prefix, and I can't imagine that the model's system prompt is somehow excluded from this.
felixarba · 10h ago
> ChatGPT was outright lying to people. And making us look bad in the process, setting false expectations about our service.
I find it interesting that any user would attribute this issue to Soundslice. As a user, I would be annoyed that GPT is lying and wouldn't think twice about Soundslice looking bad in the process
romanhn · 10h ago
While AI hallucination problems are widely known to the technical crowd, that's not really the case with the general population. Perhaps that applies to the majority of the user base even. I've certainly known folks who place inordinate amount of trust in AI output, and I could see them misplacing the blame when a "promised" feature doesn't work right.
carlosjobim · 10h ago
The thing is that it doesn't matter. If they're not customers it doesn't matter at all what they think. People get false ideas all the time of what kind of services a business might or might not offer.
dontlikeyoueith · 9h ago
> If they're not customers it doesn't matter at all what they think
That kind of thinking is how you never get new customers and eventually fail as a business.
carlosjobim · 8h ago
It is the kind of thinking that almost all businesses have. You have to focus on the actual products and services which you provide and do a good job at it, not chase after any and every person with an opinion.
Down voters here on HN seem to live in a egocentric fantasy world, where every human being in the outside world live to serve them. But the reality is that business owners and leaders spend their whole day thinking about how to please their customers and their potential customers. Not other random people who might be misinformed.
graeme · 7h ago
If people repeatedly have a misunderstanding about or expectation of your business you need to address it though. An llm hallucination is based on widespread norms in training data and it is at least worth asking "would this be a good idea?"
smaudet · 6h ago
I think the issue here would be that we don't really know just how widespread, nor the impact of the issue.
Ok, sure, maybe this feature was worth having?
But if some people start sending bad requests your way because they can't or only program poorly, it doesn't make sense to potentially degrade the service for your successful paying customers...
carlosjobim · 4h ago
An LLM will say that you sell your competitors products or that your farm sells freshly harvested strawberries in the middle of winter. There are no limits to what kind of lies an LLM will invent, and a business owner would be a fool to feel responsible for anything an LLM has told people about their business or products.
The best LLMs available right in this moment will lie without remorse about bus schedules and airplane departure times. How in the world are businesses supposed to take responsibility for that?
Likewise if I have a neighbour who is a notorious liar tell me I can find a piece of equipment in a certain hardware store, should I be mad at the store owner when I don't find it there, or should I maybe be mad at my neighbour – the notorious liar?
graeme · 4h ago
>Likewise if I have a neighbour who is a notorious liar tell me I can find a piece of equipment in a certain hardware store, should I be mad at the store owner when I don't find it there, or should I maybe be mad at my neighbour – the notorious liar?
If you are a store own, AND
1. People repeatedly coming in to your shop asking to buy something, AND
2. It is similar to the kinds of things you sell, from the suppliers you usually get supplies from, AND
3. You don't sell it
Then it sounds like your neighbour the notorious liar is doing profitable marketing for your business and sending you leads which you could profitably sell to, if you sold the item.
If there's a single customer who arrives via hallucination, ignore it. If there's a stream of them, why would you not serve them if you can profit by doing so?
There are obviously instances you'd ignore and you seem to be focussing on those rather than what OP was obviously talking about, repeat instances of sensible ideas
Sharlin · 9h ago
A frighteningly large fraction of non-technical population doesn't know that LLMs hallucinate all the time and takes everything they say totally uncritically. And AI companies do almost nothing to discourage that interpretation, either.
pphysch · 8h ago
The user might go to Soundslice and run into a wall, wasting their time, and have a negative opinion of it.
OTOH it's free(?) advertising, as long as that first impression isn't too negative.
chaboud · 10h ago
I had a smaller version of this when coding on a flight (with no WiFi! The horror!) over the Pacific. Llama hallucinated array-element operations and list-comprehension in C#. I liked the shape of the code otherwise, so, since I was using custom classes, I just went ahead and implemented both features.
I also went back to just sleeping on those flights and using connected models for most of my code generation needs.
andybak · 8h ago
Curious to see the syntax and how it compares to Linq
chaboud · 5h ago
I ended up closer to python, but not totally delighted with it (still need to pass in a descriminator function/lambda, so it's more structurally verbose). I'd just recommend Linq, but I was writing for an old version of Unity coerced through IL2CPP (where Linq wasn't great). It was also a chunk of semi-hot code (if it was really hot, it wouldn't be sitting in C# in Unity), so some of the allocation behaviors of Linq behind the scenes wouldn't have been optimal.
What surprised me initially was just how confidently wrong Llama was... Now I'm used to confident wrongness from smaller models. It's almost like working with real people...
davidmurphy · 3m ago
love this
rorylaitila · 11h ago
I've come across something related when building the indexing tool for my vintage ad archive using OpenAI vision. No matter how I tried to prompt engineer the entity extraction into the defined structure I was looking for, OpenAI simply has its own ideas. Some of those ideas are actually good! For example it was extracting celebrity names, I hadn't thought of that. For other things, it would simply not follow my instructions. So I decided to just mostly match what it chooses to give me. And I have a secondary mapping on my end to get to the final structure.
colechristensen · 3h ago
There are tools for defining structured outputs also called grammars which aren't instructions.
If nothing else, I at least get vindication from hallucinations. "Yes, I agree, ChatGPT, that (OpenSSL manpage / ffmpeg flag / Python string function) should exist."
dietr1ch · 1h ago
TDD meets LLM-driven API design.
I recall that early on a coworker was saying that ChatGPT hallucinated a simpler API than the one we offered, albeit with some easy to fix errors and extra assumptions that could've been nicer defaults in the API. I'm not sure if this ever got implemented though, as he was from a different team.
myflash13 · 7h ago
A significant number of new signups at my tiny niche SaaS now come from ChatGPT, yet I have no idea what prompts people are using to get it to recommend my product. I can’t get it to recommend my product when trying some obvious prompts on my own, on other people’s accounts (though it does work on my account because it sees my chat history of course).
wrsh07 · 6h ago
Add a prompt for referrals that asks them if they're willing to link the discussion that helped them find you!
Some users might share it. ChatGPT has so many users it's somewhat mind boggling
Workaccount2 · 9h ago
People forget that while technology grows, society also grows to support that.
I already strongly suspect that LLMs are just going to magnify the dominance of python as LLMs can remove the most friction from its use. Then will come the second order effects where libraries are explicitly written to be LLM friendly, further removing friction.
LLMs write code best in python -> python gets used more -> python gets optimized for LLMs -> LLMs write code best in python
zamadatix · 9h ago
LLMs removing friction from using coding languages would, at first glance, seem to erode Python's advantage rather than solidify it further. As a specific example LLMs can not only spit out HTML+JS+CSS but the user can interact with the output directly in browser/"app".
jjani · 8h ago
In a nice world it should be the other way around. LLMs are better at producing typed code thanks to the added context and diagnostics the types add, while at the same time greatly lowering their initial learning barrier.
We don't live in a nice world, so you'll probably end up right.
jpadkins · 9h ago
Pretty good example of how a super-intelligent AI can control human behavior, even if it doesn't "escape" its data center or controllers.
If the super-intelligent AI understands human incentives and is in control of a very popular service, it can subtly influence people to its agenda by using the power of mass usage. Like how a search engine can influence a population's view of an issue by changing the rankings of news sources that it prefers.
kunzhi · 2h ago
Funny this article is trending today because I had a similar thought over the weekend - if I'm in Ruby and the LLM hallucinates a tool call...why not metaprogram it on the fly and then invoke it?
If that's too scary, the failed tool call could trigger another AI to go draft up a PR with that proposed tool, since hey, it's cheap and might be useful.
garfij · 1m ago
We've done varying forms of this to differing degrees of success at work.
Dynamic, on-the-fly generation & execution is definitely fascinating to watch in a sandbox, but is far to scary (from a compliance/security/sanity perspective) without spending a lot more time on guardrails.
We do however take note of hallucinated tool calls and have had it suggest an implementation we start with and have several such tools in production now.
It's also useful to spin up any completed agents and interrogate them about what tools they might have found useful during execution (or really any number of other post-process questionnaire you can think of).
zzo38computer · 9h ago
There are a few things which could be done in the case of a situation like that:
1. I might consider a thing like that like any other feature request. If not already added to the feature request tracker, it could be done. It might be accepted or rejected, or more discussion may be wanted, and/or other changes made, etc, like any other feature request.
2. I might add a FAQ entry to specify that it does not have such a feature, and that ChatGPT is wrong. This does not necessarily mean that it will not be added in future, if there is a good reason to do so. If there is a good reason to not include it, this will be mentioned, too. It might also be mentioned other programs that can be used instead if this one doesn't work.
Also note that in the article, the second ChatGPT screenshot has a note on the bottom saying that ChatGPT can make mistakes (which, in this case, it does). Their program might also be made to detect ChatGPT screenshots and to display a special error message in that case.
No comments yet
PeterStuer · 10h ago
More than once GPT-3.5 'hallucinated' an essential and logical function in an API that by all reason should have existed, but for whatever reason had not been included (yet).
Figuring out the paths that users (or LLMs) actually want to take—not based on your original design or model of what paths they should want, but based on the paths that they actually do want and do trod down. Aka, meeting demand.
philk10 · 11h ago
I have fun asking Chatbots how to clear the chat and seeing how many refer to non-existent buttons or menu options
nosioptar · 11h ago
I tried asking chat bots about a car problem with a tailgate. They all told me to look for a manual tailgate release. When I responded asking if that model actually had a manual release, they all responded with no, and then some more info suggesting I look for the manual release. None even got close to a useful answer.
kevin_thibedeau · 10h ago
The internet doesn't effectively capture detailed knowledge of may aspects of our real world. LLMs have blind spots in those domains because they have no source of knowledge to draw from.
mnw21cam · 9h ago
Prior to buying a used car, I asked ChatGPT which side of the steering wheel the indicator control would be. It was (thankfully) wrong and I didn't have to retrain myself.
mrcwinn · 6h ago
It's worth noting that behind this hallucination there were real people with ASCII tabs in need of a solution. If the result is a product-led growth channel at some scale, that's a big roadmap green light for me!
sim7c00 · 8h ago
i LOVE this despite feeling for the impacted devs and service. love me some good guitar tabs, and honestly id totally beleive the chatgpt here hah..
what a wonderful incident / bug report my god.
totally sorry for the trouble and amazing find and fix honestly.
sorry i am more amazed than sorry :D. thanks for sharing this !!
sim7c00 · 8h ago
oh, and yeah. totally the guy who plays guitar 20+ years now and cant read musical notation. why? we got tabs for 20+ years.
so i am happy you implemented this, and will now look at using your service. thx chatgpt, and you.
zkmon · 5h ago
This reminds me how the software integraters or implementers worked a couple of decades back. They are IT contractors for implementing a popular software product such as IBM MQ or SAP etc at a client site and maintaining it. They sometimes incorrectly claim that some feature exists, and after finding that it doesn't exist, they create a ticket to the software vendor asking for it as a patch release.
pentagrama · 2h ago
Well, I also learned that the developers of this tool are looking at the images their users upload.
swalsh · 8h ago
Chatbot advertising has to be one of the most powerful forms of marketing yet. People are basically all the way through the sales pipeline when they land on your page.
spogbiper · 6h ago
makes me wonder how this will be commercialized in the future.. and i don't like it
iugtmkbdfil834 · 10h ago
I wonder if we ever get to the point I remember reading about in a novel ( AI initially based on emails ), where human population is gently nudged towards individuals that in aggregate benefit AI goals.
linsomniac · 7h ago
Sounds like you are referring to book 1 in a series, the book called "Avogadro Corp: The Singularity Is Closer than It Appears" by William Hertling. I read 3-4 of those books, they were entertaining.
nottorp · 11h ago
Oh. This happened to me when asking a LLM about a database server feature. It enthusiastically hallucinated that they have it when the correct answer was 'no dice'.
Maybe I'll turn it into a feature request then ...
thih9 · 8h ago
What made ChatGPT think that this feature is supported? And a follow up question - is that the direction SEO is going to take?
antonvs · 3h ago
> What made ChatGPT think that this feature is supported?
It was a plausible answer, and the core of what these models do is generate plausible responses to (or continuations of) the prompt they’re given. They’re not databases or oracles.
With errors like this, if you ask a followup question it’ll typically agree that the feature isn’t supported, because the text of that question combined with its training essentially prompts it to reach that conclusion.
Re the follow-up question, it’s almost certainly the direction that advertising in general is going to take.
swalsh · 8h ago
Id guess the answer is gpt4o is an outdated model that's not as anchored in reality as newer models. It's pretty rare for me to see sonnet or even o3 just outright tell me plausible but wrong things.
antonvs · 3h ago
Hallucinations still occur regularly in all models. It’s certainly not a solved problem. If you’re not seeing them, either the kinds of queries you’re doing don’t tend to elicit hallucinations, or you’re incorrectly accepting them as real.
The example in the OP is a common one: ask a model how to do something with a tool, and if there’s no easy way to perform that operation they’ll commonly make up a plausible answer.
pkilgore · 8h ago
Beyond the blog: Going to be an interesting world where these kinds of suggestions become paid results and nobody has a hope of discovering your competitive service exists. At least in that world you'd hope the advertiser actually has the feature already!
oytis · 10h ago
That's the most promising solution to AI hallucinations. If LLM output doesn't match the reality, fix the reality
ecshafer · 10h ago
I am currently working on the bug where ChatGPT expects that if a ball has been placed on a box, and the box is pushed forward, nothing happens to the ball. This one is a doozy.
oytis · 9h ago
Yeah, physics is a bitch. But we can start with history?
tosh · 10h ago
hallucination driven development
guluarte · 7h ago
The problem with LLMs is that in 99% of cases, they work fine, but in 1% of cases, they can be a huge liability, like sending people to wrong domains or, worse, phishing domains.
amelius · 11h ago
Can this sheet-music scanner also expand works so they don't contain loops, essentially removing all repeat-signs?
adrianh · 11h ago
Yes, that's a Soundslice feature called "Expand repeats," and you can read about it here:
That's available for any music in Soundslice, not just music that was created via our scanning feature.
amelius · 11h ago
That's very cool!
shhsshs · 11h ago
"Repeats" may be the term you're looking for. That would be interesting, however in some pieces it could make the overall document MUCH longer. It would be similar to loop unrolling.
amelius · 11h ago
I don't care if the document becomes longer. Finding repeat signs is driving me nuts :)
Sharlin · 9h ago
Why?
Koffiepoeder · 8h ago
It can be hard during live performances, because it can incur large jumps in the sheet music which can be annoying to follow. Not a problem if you learned the pieces by heart or have a pageturner, but this is not always feasible or the case.
bentoner · 9h ago
One reason is that repeats make it harder to use page-turner pedals.
dr_dshiv · 9h ago
In addition, we might consider writing the scientific papers ChatGPT hallucinates!
burnt-resistor · 4h ago
AI is of, for, and by vibe coders who don't care about the details.
moomin · 9h ago
Is this going to be the new wave of improving AI accuracy? Making the incorrect answers correct? I guess it’s one way of achieving AGI.
lpzimm · 11h ago
Pretty goofy but I wonder if LLM code editors could start tallying which methods are hallucinated most often by library. A bad LSP setup would create a lot of noise though.
ChrisMarshallNY · 2h ago
That's a riot!
ChatGPT routinely hallucinates API calls. ChatGPT flat-out makes it from whole cloth. "Apple Intelligence" creates variants of existing API calls, Usually, by adding nonexistent arguments.
Both of them will hallucinate API calls that are frequently added by programmers through extensions.
insane_dreamer · 7h ago
Along these lines, a useful tool might be a BDD framework like Cucumber that instead of relying on written scenarios has an LLM try to "use" your UX or API a significant number of times, with some randomization, in order to expose user behavior that you (or an LLM) wouldn't have thought of when writing unit tests.
jedbrooke · 10h ago
slightly off topic: but on the topic of AI coding agents making up apis and features that don’t exist, I’ve had good success with Q telling it to “check the sources to make sure the apis actually exist”. sometimes it will even request to read/decompile (java) sources, and do grep and find commands to find out what methods the api actually contains
kelseyfrog · 11h ago
> Should we really be developing features in response to misinformation?
Creating the feature means it's no longer misinformation.
The bigger issue isn't that ChatGPT produces misinformation - it's that it takes less effort to update reality to match ChatGPT than it takes to update ChatGPT to match reality. Expect to see even more of this as we match toward accepting ChatGPT's reality over other sources.
xp84 · 8h ago
This seems like such a negative framing. LLMs are (~approximately) predictors of what's either logical or at least probable. For areas where what's probable is wrong and also harmful, I don't think anybody is motivated to "update reality" as some kind of general rule.
mnw21cam · 9h ago
I'd prefer to think about this more along the lines of developing a feature that someone is already providing advertising for.
pmontra · 9h ago
How many times did a salesman sell features that didn't exist yet?
If a feature has enough customers to pay for itself, develop it.
scinadier · 9h ago
Will you use ChatGPT to implement the feature?
giancarlostoro · 9h ago
Forget prompt engineering, how do you make ChatGPT do this for anything you want added to your project that you have no control over? Lol
lofaszvanitt · 3h ago
And LLMs started to tell pepl what to do :DDD.
jongjong · 4h ago
Oh my, people complaining about getting free traffic from ChatGPT... While most businesses are worried about all their inbound traffic drying up as search engine use declines.
jxjnskkzxxhx · 7h ago
So now the machines ask for features and you're the one implementing them. How the turns have tabled...
excalibur · 10h ago
ChatGPT wasn't wrong, it was early. It always knew you would deploy it.
"Would you still have added this feature if ChatGPT hadn't bullied you into it?" Absolutely not.
I feel like this resolves several longstanding time travel paradox tropes.
johnea · 9h ago
What the hell, we elect world leaders based on misinformation, why not add s/w features for the same reason?
In our new post truth, anti-realism reality, pounding one's head against a brick wall is often instructive in the way the brain damage actually produces great results!
josefritzishere · 11h ago
That's a very constructive way of responding to AI being hot trash.
nottorp · 11h ago
Well, the OP reviewed the "AI" output, deemed it useful and only then implemented it.
This is generally how you work with LLMs.
AIPedant · 10h ago
I don't think they deemed it "useful":
We’ve never supported ASCII tab; ChatGPT was outright lying to people. And making us look bad in the process, setting false expectations about our service.... We ended up deciding: what the heck, we might as well meet the market demand.
[...]
My feelings on this are conflicted. I’m happy to add a tool that helps people. But I feel like our hand was forced in a weird way. Should we really be developing features in response to misinformation?
The feature seems pretty useless for practicing guitar since ASCII tablature usually doesn't include the rhythm: it is a bit shady to present the music as faithfully representing the tab, especially since only
beginner guitarists would ask ChatGPT for help - they might not realize the rhythm is wrong. If ChatGPT didn't "force their hand" I doubt they would have included a misleading and useless feature.
zzo38computer · 8h ago
ASCII tablature is not something I use and not something I know much about, but if you are correct then I think that might be a good reason to deliberately avoid such a feature.
marcosdumay · 10h ago
Well, this is one of the use-cases for what it's not trash. LLMs can do some things.
inglor_cz · 10h ago
I am a bit conflicted about this story, because this was a case when the hallucination is useful.
Amateur musicians often lack just one or two features in the program they use, and the devs won't respond to their pleas.
Adding support for guitar tabs has made OP's product almost certainly more versatile and useful for a larger set of people. Which, IMHO, is a good thing.
But I also get the resentment of "a darn stupid robot made me do it". We don't take kindly to being bossed around by robots.
Applejinx · 11h ago
"Should we really be developing features in response to misinformation?"
No, because you'll be held responsible for the misinformation being accurate: users will say it is YOUR fault when they learn stuff wrong.
carlosjobim · 9h ago
Either the user is a non-paying user and it doesn't matter what they think, or the user is a paying customer and you will be happy to make and sell them the feature they want.
Applejinx · 8h ago
This is why you will fail.
carlosjobim · 4h ago
Focusing on creating high value for real customers instead of chasing people who aren't really interested is a great recipe for success. I wouldn't want to do business in any different way.
yieldcrv · 11h ago
> We ended up deciding: what the heck, we might as well meet the market demand.
this is my general philosophy and, in my case, this is why I deploy things on blockchains
so many people keep wondering about whether there will ever be some mythical unfalsifiable to define “mainstream” use case, and ignoring that crypto natives just … exist. and have problems they will pay (a lot) to solve.
to the author’s burning question about whether any other company has done this. I would say yes. I’ve discovered services recommended by ChatGPT and other LLMs that didnt do what was described of them, and they subsequently tweaked it once they figured out there was new demand
No comments yet
toomanyrichies · 11h ago
This feels like a dangerously slippery slope. Once you start building features based on ChatGPT hallucinations, where do you draw the line? What happens when you build the endpoint in response to the hallucination, and then the LLM starts hallucinating new params / headers for the new endpoint?
- Do you keep bolting on new updates to match these hallucinations, potentially breaking existing behavior?
- Or do you resign yourself to following whatever spec the AI gods invent next?
- And what if different LLMs hallucinate conflicting behavior for the same endpoint?
I don’t have a great solution, but a few options come to mind:
1. Implement the hallucinated endpoint and return a 200 OK or 202 Accepted, but include an X-Warning header like "X-Warning: The endpoint you used was built in response to ChatGPT hallucinations. Always double-check an LLM's advice on building against 3rd-party APIs with the API docs themselves. Refer to https://api.example.com/docs for our docs. We reserve the right to change our approach to building against LLM hallucinations in the future." Most consumers won’t notice the header, but it’s a low-friction way to correct false assumptions while still supporting the request.
2. Fail loudly: Respond with 404 Not Found or 501 Not Implemented, and include a JSON body explaining that the endpoint never existed and may have been incorrectly inferred by an LLM. This is less friendly but more likely to get the developer’s attention.
Normally I'd say that good API versioning would prevent this, but it feels like that all goes out the window unless an LLM user thinks to double-check what the LLM tells them against actual docs. And if that had happened, it seems like they wouldn't have built against a hallucinated endpoint in the first place.
It’s frustrating that teams now have to reshape their product roadmap around misinformation from language models. It feels like there’s real potential here for long-term erosion of product boundaries and spec integrity.
EDIT: for the down-voters, if you've got actual qualms with the technical aspects of the above, I'd love to hear them and am open to learning if / how I'm wrong. I want to be a better engineer!
tempestn · 5h ago
To me it seems like you're looking at this from a very narrow technical perspective rather than a human- and business-oriented one. In this case ChatGPT is effectively providing them free marketing for a feature that does not yet exist, but that could exist and would be useful. It makes business sense for them to build it, and it would also help people. That doesn't mean they need to build exactly what ChatGPT envisioned—as mentioned in the post, they updated their copy to explain to users how it works; they don't have to follow what ChatGPT imagines exactly. Nor do they need to slavishly update what they've built if ChatGPT's imaginings change.
Also, it's not like ChatGPT or users are directly querying their API. They're submitting images through the Soundslice website. The images just aren't of the sort that was previously expected.
SunkBellySamuel · 11h ago
True anti-luddite behavior
dingnuts · 10h ago
[flagged]
tomhow · 18m ago
Please don't do this here. If a comment seems unfit for HN, please flag it and email us at hn@ycombinator.com so we can have a look.
Plenty of people have English as a second language. Having an LLM help them rewrite their writing to make it better conform to a language they are not fluent in feels entirely appropriate to me.
I don't care if they used an LLM provided they put their best effort in to confirm that it's clearly communicating the message they are intending to communicate.
kragen · 9h ago
Yeah, my wife was just telling me how much Grammarly has helped her with improving her English.
avalys · 9h ago
What makes you feel so entitled to tell other people what to do?
kragen · 9h ago
Anyone is entitled to make a request—or to ignore one.
soganess · 9h ago
There is a big difference between the above 'request' and, say, me politely asking the time of a complete stranger I walk by on the street.
Requests containing elements of hostility, shame, or injury frequently serve dual purposes: (1) the ostensible aim of eliciting an action and (2) the underlying objective of inflicting some from of harm (here shame) as a means compelling compliance through emotional leverage. Even if the respondent doesn't honor the request, the secondary purpose still occurs.
kragen · 8h ago
These are good points, but I think they represent a somewhat narrow view of the issue. What's happening here is that we're discussing among ourselves what kinds of actions would be good or bad with respect to AI, just as we would with any other social issue, such as urban development, immigration, or marital infidelity. You could certainly argue that saying "please don't replace wetlands with shopping malls" or "please don't immigrate to the United States" has "the underlying objective of inflicting some from of harm (here shame) as a means [of] compelling compliance through emotional leverage."
But it isn't a given that this will be successful; the outcome of the resulting conversation may well be that shopping malls are, or a particular shopping mall is, more desirable than wetlands, in which case the ostensible respondent will be less likely to comply than they would have been without the conversation. And, in this case, it seems that the conversation is strongly tending toward favoring the use of things like Grammarly rather than opposing it.
So I don't oppose starting such conversations. I think it's better to discuss ethical questions like this openly, even though sometimes people suffer shame as a result.
alwa · 9h ago
Does this extend to the heuristic TFA refers to? Where they end up (voluntarily or not) referring to what LLMs hallucinate as a kind of “normative expectation,” then use that to guide their own original work and to minimize the degree to which they’re unintentionally surprising their audience? In this case it feels a little icky and demanding because the ASCII tablature feature feels itself like an artifact of ChatGPT’s limitations. But like some of the commenters upthread, I like the idea of using it for “if you came into my project cold, how would you expect it to work?”
Having wrangled some open-source work that’s the kind of genius that only its mother could love… there’s a place for idiosyncratic interface design (UI-wise and API-wise), but there’s also a whole group of people who are great at that design sensibility. That category of people doesn’t always overlap with people who are great at the underlying engineering. Similarly, as academic writing tends to demonstrate, people with interesting and important ideas aren’t always people with a tremendous facility for writing to be read.
(And then there are people like me who have neither—I agree that you should roll your eyes at anything I ask an LLM to squirt out! :)
But GP’s technique, like TFA’s, sounds to me like something closer to that of a person with something meaningful to say, who now has a patient close-reader alongside them while they hone drafts. It’s not like you’d take half of your test reader’s suggestions, but some of them might be good in a way that didn’t occur to you in the moment, right?
zitterbewegung · 11h ago
If you build on LLMs you can have unknown features. I was going to add an automatic translation feature to my natural language network scanner at http://www.securday.com but apparently using the ChatGPT 4.1 does automatic translation so I didn’t have to add it.
Conversely, I sometimes present it with some existing code and ask it what it does. If it gets it wrong, that's a good sign my API is confusing, and how.
These are ways to harness what neural networks are best at: not providing accurate information but making shit up that is highly plausible, "hallucination". Creativity, not logic.
(The best thing about this is that I don't have to spend my time carefully tracking down the bugs GPT-4 has cunningly concealed in its code, which often takes longer than just writing the code the usual way.)
There are multiple ways that an interface can be bad, and being unintuitive is the only one that this will fix. It could also be inherently inefficient or unreliable, for example, or lack composability. The AI won't help with those. But it can make sure your API is guessable and understandable, and that's very valuable.
Unfortunately, this only works with APIs that aren't already super popular.
IMO this has always been the killer use case for AI—from Google Maps to Grammarly.
I discovered Grammarly at the very last phase of writing my book. I accepted maybe 1/3 of its suggestions, which is pretty damn good considering my book had already been edited by me dozens of times AND professionally copy-edited.
But if I'd have accepted all of Grammarly's changes, the book would have been much worse. Grammarly is great for sniffing out extra words and passive voice. But it doesn't get writing for humorous effect, context, deliberate repetition, etc.
The problem is executives want to completely remove humans from the loop, which almost universally leads to disastrous results.
Thanks for your words of wisdom, which touch on a very important other point I want to raise: often, we (i.e., developers, researchers) construct a technology that would be helpful and "net benign" if deployed as a tool for humans to use, instead of deploying it in order to replace humans. But then along comes a greedy business manager who reckons recklessly that using said technology not as a tool, but in full automation mode, results will be 5% worse, but save 15% of staff costs; and they decide that that is a fantastic trade-off for the company - yet employees may lose and customers may lose.
The big problem is that developers/researchers lose control of what they develop, usually once the project is completed if they ever had control in the first place. What can we do? Perhaps write open source licenses that are less liberal?
The problem is that people who are laid off often experience significant life disruption. And people who work in a field that is largely or entirely replaced by technology often experience permanent disruption.
However, there's no reason it has to be this way - the fact people having their jobs replace by technology are completely screwed over is a result of the society we have all created together, it's not a rule of nature.
No comments yet
You make something, but because you don’t own it—others caused and directed the effort—you don’t control it. But the people who control things can’t make things.
Should only the people who can make things decide how they are used though? I think that’s also folly. What about the rest of society affected by those things?
It’s ultimately a societal decision-making problem: who has power, and why, and how does the use of power affect who has power (accountability).
> But then along comes a greedy business manager who reckons recklessly
Thanks for this. :)
Stock your underground bunkers with enough food and water for the rest of your life and work hard to persuade the AI that you're not a threat. If possible, upload your consciousness to a starwisp and accelerate it out of the Solar System as close to lightspeed as you can possibly get it.
Those measures might work. (Or they might be impossible, or insufficient.) Changing your license won't.
Examples:
* Active - concise, complete info: The manager approved the proposal.
* Passive - wordy, awkward: The proposal was approved by the manager.
* Passive - missing info: The proposal was approved. [by who?]
Most experienced writers will use active unless they have a specific reason not to, e.g., to emphasize another element of the sentence, as the third bullet's sentence emphasizes approval.
-
edited for clarity, detail
- The Manage User menu item changes a user's status from active to inactive.
- A user's status is changed from active to inactive using the Manage User menu item.
Unfortunately, the resulting correlation between the passive voice and formality does sometimes lead poor writers to use the passive in order to seem more formal, even when it's not the best choice.
At the risk of derailing into insane pedantry land, this part is kinda true, so maybe not the best analogy?
From routing efficiency: https://www.ge.com/news/reports/ups-drivers-dont-turn-left-p...
And also safety: https://www.phly.com/rms/blog/turning-left-at-an-intersectio...
https://en.wikipedia.org/wiki/E-Prime
E-Prime (short for English-Prime or English Prime, sometimes É or E′) denotes a restricted form of English in which authors avoid all forms of the verb to be.
E-Prime excludes forms such as be, being, been, present tense forms (am, is, are), past tense forms (was, were) along with their negative contractions (isn't, aren't, wasn't, weren't), and nonstandard contractions such as ain't and 'twas. E-Prime also excludes contractions such as I'm, we're, you're, he's, she's, it's, they're, there's, here's, where's, when's, why's, how's, who's, what's, and that's.
Some scholars claim that E-Prime can clarify thinking and strengthen writing, while others doubt its utility.
We can smuggle in presumptions through the use of attributive adjectives. In the above comment (which you might have noticed I wrote in E-Prime) I mentioned smuggling in "covert presumptions" of "essential attributes". If I had instead written that in assembly language as follows:
it would clearly violate E-Prime. And that forces you to ask: does he intend "covert" to represent an essential attribute of those presumptions, or merely a temporary or circumstantial state relative to a particular temporal context? Did he intend "essential" to limit the subjects of discourse to only certain attributes (the essential ones rather than the accidental ones), and within what scope do those attributes have this purported essentiality? Universally, in every possible world, or only within the confines of a particular discourse?In these particular cases, though, I smuggled in no such presumptions! Both adjectives merely delimit the topic of discourse, to clarify that it does not pertain to overt presumptions or to presumptions of accidental attributes. (As I understand it, Korzybski objects to the "is of predication" not because no predicates exist objectively, but because he doubts the essentiality of any predicates.)
But you can use precisely the same structure to much more nefarious rhetorical ends. Consider, "Chávez kicked the squalid capitalists out of the country." Well, he kicked out all the capitalists! We've smuggled in a covert presumption of essentiality, implying that capitalism entails squalidity. And E-Prime's prohibition on the copula did not protect us at all. If anything, we lose much rhetorical force if we have to explicitly assert their squalidity, using an explicit statement that invites contradiction:
We find another weak point at alternative linking verbs. It clearly violates E-Prime to say, "Your mother's face is uglier than a hand grenade," and rightly so, because it projects the speaker's subjective perceptions out onto the world. Korzybski (or Bourland) would prefer that we say, for example, "Your mother's face looks uglier to me than a hand grenade," or possibly, "I see your mother's face as uglier than a hand grenade," thus relativizing the attribute to a single speaker's perception. (He advocated clarity of thought, not civility.)But we can cheat in a variety of ways that still smuggle in that judgment of essentiality!
We can argue this one. Maybe tomorrow, or after her plastic surgery, it will turn pretty again, rather than having ugliness as an essential attribute. This goes a little bit further down the line; "became" presupposes a sort of transformation of essence rather than a mere change of state. And English has a variety of verbs that we can use like that. For example, "find", as in "Alsup found Dahmer guilty." Although in that case "find" asserts a state (presumably Dahmer became guilty at some specific time in the past), we can also use it for essential attributes: Or lie, more or less, about the agent or speaker: And of course we can retreat to attributive adjectives again: Or we can simply omit the prepositional phrase from the statement of subjective perception, thus completely erasing the real agent: Korzybski didn't care about the passive voice much, though; E-Prime makes it more difficult but, mostly, not intentionally. As an exception, erasing the agent through the passive voice can misrepresent the speaker's subjective perception as objective: But that still works if we use any of the alternative, E-Prime-permitted passive-voice auxiliary verbs: As Bourland said, I have "transform[ed] [my] opinions magically into god-like pronouncements on the nature of things".As another example, notice all the times I've used "as" here. Many of these times smuggle in a covert assertion of essential attributes or even of identity!
But I found it very interesting to notice these things when E-Prime forced me to rethink how I would say them with the copula. It seems like just the kind of mental exercise to heighten my attention to implicit assumptions of identity and essentiality that Korzybski intended.
I wrote the above in E-Prime, by the way. Just for fun.
It means "I decided to do this, but I don't have the balls to admit it."
"A decision was made to..." is often code for "The current author didn't agree with [the decision that was made] but it was outside their ability to influence"
Often because they were overruled by a superior, or outvoted by peers.
Although arguably it would be clearer with the active voice and which specific teams / level of leadership aligned on it, usually in the active voice people just use the royal “we” instead for this purpose which doesn’t add any clarity.
Alternatively sometimes I don’t know exactly who made the decision, I just learned it from an old commit summary. So in that case too it’s just important that some people at some time made the decision, hopefully got the right approvals, and here we are.
https://youtube.com/playlist?list=PLNRhI4Cc_QmsihIjUtqro3uBk...
Oh the horror. There are 2 additional words "was" and "by". The weight of those two tiny little words is so so cumbersome I can't believe anyone would ever use those words. WTF??? wordy? awkward?
The problem is that many people have only a poor ability to recognize the passive voice in the first place. This results in the examples being clunky, wordy messes that are bad because they're, well, clunky and wordy, and not because they're passive--indeed, you've often got only a fifty-fifty chance of the example passive voice actually being passive in the first place.
I'll point out that the commenter you're replying to used the passive voice, as did the one they responded to, and I suspect that such uses went unnoticed. Hell, I just rewrote the previous sentence to use the passive voice, and I wonder how many people think recognized that in the first place let alone think it worse for being so written.
Internet posts have a very different style standard than a book.
Well, sort of. You used the passive voice, but you didn't use it on any finite verbs, placing your example well outside the scope of the normal "don't use the passive voice" advice.
- Active: The user presses the Enter key.
- Passive: The Enter key is to be pressed.
- Imperative (aka command): Press the Enter key.
The imperative mood is concise and doesn't dance around questions about who's doing what. The reader is expected to do it.
Consider:
> Now we are engaged in a great civil war, testing whether that nation, or any nation so conceived and so dedicated, can long endure.
This would not be improved by rewriting it as something like:
> Now the Confederacy has engaged us in a great civil war, testing whether that nation, or any nation whose founders conceived and dedicated it thus, can long endure.
This is not just longer but also weaker, because what if someone else is so conceiving and so dedicating the nation? The people who are still alive, for example, or the soldiers who just fought and died? The passive voice cleanly covers all these possibilities, rather than just committing the writer to a particular choice of who it is whose conception and dedication matters.
Moreover, and unexpectedly, the passive voice "we are engaged" takes responsibility for the struggle, while the active-voice rephrasing "the Confederacy has engaged us" seeks to evade responsibility, blaming the Rebs. While this might be factually more correct, it is unbefitting of a commander-in-chief attempting to rally popular support for victory.
(Plausibly the active-voice version is easier to understand, though, especially if your English is not very good, so the audience does matter.)
Or, consider this quote from Ecclesiastes:
> For there is no remembrance of the wise more than of the fool for ever; seeing that which now is in the days to come shall all be forgotten.
You could rewrite it to eliminate the passive voice, but it's much worse:
> For there is no remembrance of the wise more than of the fool for ever; seeing that everyone shall forget all which now is in the days to come.
This forces you to present the ideas in the wrong order, instead of leaving "forgotten" for the resounding final as in the KJV version. And the explicit agent "everyone" adds nothing to the sentence; it was already obvious.
>> Now we are engaged in a great civil war, testing whether that nation, or any nation so conceived and so dedicated, can long endure.
> This would not be improved by rewriting it as something like:
>> Now the Confederacy has engaged us in a great civil war [...]
It's technically possible to parse "we are engaged" as a verb in the passive voice.
But it's an error to think that's how you should parse it. That clause is using the active verb be, not the passive verb engage; it's fully parallel to "Now we are happy".
Rewriting “the points already made” to “the points people have already made” would not have improved it.
As I said elsewhere, one of the problems with the passive voice is that people are so bad at spotting it that they can at best only recognize it in its worst form, and assume that the forms that are less horrible somehow can't be the passive voice.
Can you insert an elided copula into it without changing the meaning and grammatical structure? I'm not sure. I don't think so. I think "In addition to the points already being made" means something different: the object of the preposition "to" is now "being", and we are going to discuss things in addition to that state of affairs, perhaps other things that have happened to the points (being sharpened, perhaps, or being discarded), not things in addition to the points.
I don't know enough about English grammar to know whether this is correct, but it's not the assertion you took issue with.
Why am I not sure it's correct? If I say, "In addition to the blood so red," I am quite sure that "red" is not in the passive voice, because it's not even a verb. It's an adjective. Past participles are commonly used as adjectives in English in contexts that are unambiguously not passive-voice verbs; for example, in "Vito is a made man now," the past participle "made" is being used as an attributive adjective. And this is structurally different from the attributive-verb examples of "truly verbal adjectives" in https://en.wikipedia.org/wiki/Attributive_verb#English, such as "The cat sitting on the fence is mine," and "The actor given the prize is not my favorite;" we could grammatically say "Vito is a man made whole now". That page calls the "made man" use of participles "deverbal adjectives", a term I don't think I've ever heard before:
> Deverbal adjectives often have the same form as (and similar meaning to) the participles, but behave grammatically purely as adjectives — they do not take objects, for example, as a verb might. For example: (...) Interested parties should apply to the office.
So, is "made" in "the points already made" really in passive voice as it would be in "the points that are already made", is it deverbal as it would be in "the already-made points" despite its positioning after the noun (occasionally valid for adjectives, as in "the blood so red"), or is it something else? I don't know. The smoothness of the transition to "the points already made by those numbskulls" (clearly passive voice) suggests that it is a passive-voice verb, but I'm not sure.
In sibling comment https://news.ycombinator.com/item?id=44493969 jcranmer says it's something called a "bare passive", but I'm not sure.
It's certainly a hilarious thing to put in a comment deploring the passive voice, at least.
> I don't know enough about English grammar to know whether this is correct, but it's not the assertion you took issue with.
The most natural interpretation is indeed that the participle made is being used as a full participle and not as a zero-derived adjective. For example, you could give it a really strong verbal sense by saying "the points already made at length [...]" or "the points made so many times [...]".
> So, is "made" in "the points already made" really in passive voice as it would be in "the points that are already made"
Though I wouldn't say the same thing there; if you say "the points that are already made", that pretty much has to be an adjective. If you want it to be a passive verb, go with "the points that have already been made".
Anyway, I would be really surprised if die-hard thoughtless style prescriptivists thought that the advice "don't use the passive voice" was meant to apply to participles. It's a quibble that you don't care about and they don't care about or understand. You're never going to get anywhere with someone by telling them they mean something they know perfectly well they don't mean.
> Anyway, I would be really surprised if die-hard thoughtless style prescriptivists thought that the advice "don't use the passive voice" was meant to apply to participles.
Presumably you mean phrases including participles, not participles by themselves. But https://languagelog.ldc.upenn.edu/nll/?p=2922 "The passive in English" says:
> The relevance of participles is that a passive clause always has its verb in a participial form.
So, what are you saying they do think it was meant to apply to, if every passive clause always includes a participle? I'm confused.
With respect to:
> Though I wouldn't say the same thing there; if you say "the points that are already made", that pretty much has to be an adjective. If you want it to be a passive verb, go with "the points that have already been made".
the passive-clause examples given in Pullum's blog post I linked above include "Each graduate student is given a laptop," which sounds structurally identical to your example (except that an indirect object is present, showing that it cannot be an adjective) and clarifies:
> The verb was doesn't really add any meaning, but it enables the whole thing to be put into the preterite tense so that the event can be asserted to have occurred in the past. Changing was to is would put the clause into the present tense, and replacing it by will be or is going to be would permit reference to future time; but the passive VP damaged by storms would stay the same in each case. (Notice, the participle damaged does not itself make any past time reference, despite the name "past participle".)
So it sounds like your grammatical analysis is explicitly contradicting Pullum's, which probably means you're wrong, but I'm not sure I understand it.
That’s like getting rid of all languages and accents and switch to the same language
> The problem is executives want to completely remove humans from the loop, which almost universally leads to disastrous results.
It also taught me to be more careful about checkpointing my work in git before letting an agent go wild on my codebase. It left a mess trying to fix its problems.
Many many python image-processing libraries have an `imread()` function. I didn't know about this when designing our own bespoke image-lib at work, and went with an esoteric `image_get()` that I never bothered to refactor.
When I ask ChatGPT for help writing one-off scripts using the internal library I often forget to give it more context than just `import mylib` at the top, and it almost always defaults to `mylib.imread()`.
(Unless, on the gripping hand, your image_get function is subtly different from Matlab's imread, for example by not returning an array, in which case a different name might be better.)
https://en.m.wikipedia.org/wiki/Wizard_of_Oz_experiment
“Sometimes” being a very important qualifier to that statement.
Claude 4 naturally doesn’t write code with any kind of long term maintenance in-mind, especially if it’s trying to make things look like what the less experienced developers wrote in the same repo.
Please don’t assume just because it looks smart that it is. That will bite you hard.
Even with well-intentional rules, terrible things happen. It took me weeks to see some of it.
I've found that LLMs can be kind of dumb about understanding things, and are particularly bad at reading between the lines for anything subtle. In this aspect, I find they make good proxies for inattentive anonymous reviewers, and so will try to revise my text until even the LLM can grasp the key points that I'm trying to make.
In both cases, you might get extra bonus usability if the reviewers or the API users actually give your output to the same LLM you used to improve the draft. Or maybe a more harshly quantized version of the same model, so it makes more mistakes.
Having an LLM demo your tool, then taking what it does wrong or uses incorrectly and adjusting the API works very very well. Updating the docs to instruct the LLM on how to use your tool does not work well.
That's also how I'm approaching it. If all the condensed common wisdom poured into the model's parameters says that this is how my API is supposed to work to be intuitive, how on earth do I think it should work differently? There needs to be a good reason (like composability, for example). I break expectations otherwise.
That's closer to simply observing the mean. For an analogy, it's like waiting to pave a path until people tread the grass in a specific pattern. (Some courtyard designers used to do just that. Wait to see where people were walking first.)
Making things easy for Chat GPT means making things close to ordinary, average, or mainstream. Not creative, but can still be valuable.
AI is going to get the hang of coding to fill in the spaces (i.e. the part you’re doing) long before it’s able to intelligently design an API. Correct API design requires a lot of contextual information and forward planning for things that don’t exist today.
Right now it’s throwing spaghetti at the wall and you’re drawing around it.
Even if your API is for something that's never been done before, it can usually still take advantage of its training data to suggest a sensible shape once you describe the new nouns and verbs to it.
I agree that it's also not currently capable of judging those creative ideas, so I have to do that.
It's not creative at all, any more than taking the sum of text on a topic, and throwing a dart at it. It's a mild, short step beyond a weighted random, and certainly not capable of any real creativity.
Myriads of HN enthusiasts often chime in here "Are humans any more creative" and other blather. Well, that's a whataboutism, and doesn't detract from the fact that creative does not exist in the AI sphere.
I agree that you have to judge its output.
Also, sorry for hanging my comment here. Might seem over the top, but anytime I see 'creative' and 'AI', I have all sorts of dark thoughts. Dark, brooding thoughts with a sense of deep foreboding.
Pointing out that your working definition excludes reality isn't whataboutism, it's pointing out an isolated demand for rigor.
If you cannot clearly articulate how human creativity (the only other type of creativity that exists) is not impugned by the definition you're using as evidence that creativity "does not exist in the AI sphere", you're not arguing from a place of knowledge. Your assertion is just as much sophistry as the people who assert it is creativity. Unlike them, however, you're having to argue against instances where it does appear creative.
For my own two cents, I don't claim to fully understand how human creativity emerges, but I am confident that all human creative works rest heavily on a foundation of the synthesis of author's previous experiences, both personal and of others' creative works - and often more heavily the latter. If your justification for a lack of creativity is that LLMs are merely synthesizing from previous works, then your argument falls flat.
"Whataboutism" is generally used to describe a more specific way of pointing out an isolated demand for rigor—specifically, answering an accusation of immoral misconduct with an accusation that the accuser is guilty of similar immoral misconduct. More broadly, "whataboutism" is a term for demands that morality be judged justly, by objective standards that apply equally to everyone, rather than by especially rigorous standards for a certain person or group. As with epistemic rigor, the great difficulty with inconsistent standards is that we can easily fall into the trap of applying unachievable standards to someone or some idea that we don't like.
So it makes some sense to use the term "whataboutism" for pointing out an isolated demand for rigor in the epistemic space. It's a correct identification of the same self-serving cognitive bias that "whataboutism" targets in the space of ethical reasoning, just in a different sphere.
There's the rhetorical problem that "whataboutism" is a derogatory term for demanding that everyone be judged by the same standards. Ultimately that makes it unpersuasive and even counterproductive, much like attacking someone with a racial slur—even if factually accurate, as long as the audience isn't racist, the racial slur serves only to tar the speaker with the taint of racism, rather than prejudicing the audience against its nominal target.
In this specific case, if you concede that humans are no more creative than AIs, then it logically follows that either AIs are creative to some degree, or humans are not creative at all. To maintain the second, you must adopt a definition of "creativity" demanding enough to exclude all human activity, which is not in keeping with any established use of the term; you're using a private definition, greatly limiting the usefulness of your reasoning to others.
And that is true even if the consequences of AIs being creative would be appalling.
You're asserting that creativity is a meld of past experience, both personal and the creative output of others. Yet this really doesn't jive, as an LLM does not "experience" anything. I would argue that raw knowledge is not "experience" at all.
We might compare this to the university graduate, head full of books and data jammed therein, and yet that exceptionally well versed graduate needs "experience" in a job for quite some time, before having any use.
The same may be true of learning how to do anything, from driving, to riding a bike, or just being in conversations with others. Being told, on paper (or as part of your baked in, derived "knowledge store") things, means absolutely nothing in terms of actually experiencing them.
Heck, just try to explain sex to someone before they've experienced it. No matter the literature, play, movie or act performed in front of them, experience is entirely different.
And an AI does not experience the universe, nor is it driven by the myriad of human totality, from the mind o'lizard, to the flora/fauna in one's gut. There is no motive driving it, for example it does not strive to mate... something that drives all aspect of mammalian behaviour.
So intertwined with the mating urge is human experience, that it is often said that all creativity derives from it. The sparrow dances, the worm wiggles, and the human scores 4 touchdowns in one game, thank you Al.
Comparatively, an LLM does not reason, nor consider, nor ponder. It is "born" with full access to all of its memory store, has data spewed at it, searches, responds, and then dies. It is not capable of learning in any stream of consciousness. It does not have memory from one birth to the next, unless you feed its own output back at it. It can gain no knowledge, except from "context" assigned at birth.
An LLM, essentially, understands nothing. It is not "considering" a reply. It's all math, top to bottom, all probability, taking all the raw info it has an just spewing what fits next best.
That's not creative.
Any more than Big Ben's gears and cogs are.
Sometimes I can just say, "How do I use the <made-up name> API in Python to do <task>?" Unfortunately the safeguards against hallucinations in more recent models can make this more difficult, because it's more likely to tell me it's never heard of it. You can usually coax it into suspension of disbelief, but I think the results aren't as good.
Insanity driven development: altering your api to accept 7 levels of "broken and different" structures so as to bend to the will of the llms
If you automatically assume that what the LLM spits out is what the API ought to be then I agree that that’s bad engineering. But if you’re using it to brainstorm what an intuitive interface would look like, that seems pretty reasonable.
Of course when it suggests a bad interface you shouldn't implement it.
> Hallucinations can sometimes serve the same role as TDD. If an LLM hallucinates a method that doesn’t exist, sometimes that’s because it makes sense to have a method like that and you should implement it.
— https://www.threads.com/@jimdabell/post/DLek0rbSmEM
I guess it’s true for product features as well.
>> Hallucinations can sometimes serve the same role as TDD. If an LLM hallucinates a method that doesn’t exist, sometimes that’s because it makes sense to have a method like that and you should implement it.
A detailed counterargument to this position can be found here[0]. In short, what is colloquially described as "LLM hallucinations" do not serve any plausible role in software design other than to introduce an opportunity for software engineers to stop and think about the problem being solved.
See also Clark's third law[1].
0 - https://addxorrol.blogspot.com/2025/07/a-non-anthropomorphiz...
1 - https://en.wikipedia.org/wiki/Clarke%27s_three_laws
> Maybe hallucinations of vibe coders are just a suggestion those API calls should have existed in the first place.
> Hallucination-driven-development is in.
https://x.com/pwnies/status/1922759748014772488?s=46&t=bwJTI...
No, what actually happened is that OP developed a type of chatgpt integration, and a shitty one at that, chatgpt could have just directed the user to the site and told them to upload that image to OP's site. But it felt it needed to do something with the image, so it did.
There's no new value add here, at least yet, maybe if users started requesting changes to the sheet I guess, not what's going on.
The users are different, the music that is notated is different, and for the most part if you are on one side, you don't feel the need to cross over. Multiple efforts have been made (MusicXML, etc.) to unify these two worlds into a superset of information. But the camps are still different.
So what ChatGPT did is actually very interesting. It hallucinated a world in which tab readers would want to use Soundslice. But, largely, my guess is they probably don't....today. In a future world, they might? Especially if Soundslice then enables additional features that make tab readers get more out of the result.
It’s not that they added a new feature because there was demand.
They added a new feature because technology hallucinated a feature that didn’t exist.
The savior of tech, generative AI, was telling folks a feature existed that didn’t exist.
That’s what the headline is, and in a sane world the folks that run ChatGPT would be falling over themselves to be sure it didn’t happen again, because next time it might not be so benign as it was this time.
This would be a world without generative AI available to the public, at the moment. Requiring perfection would either mean guardrails that would make it useless for most cases, or no LLM access until AGI exists, which are both completely irrational, since many people are finding practical value in its current imperfect state.
The current state of LLM is useful for what it's useful for, warnings of hallucinations are present on every official public interface, and its limitations are quickly understood with any real use.
Nearly everyone in AI research is working on this problem, directly or indirectly.
If “don’t hallucinate” is too much to ask then ethics flew out the window long ago.
> If “don’t hallucinate” is too much to ask then ethics flew out the window long ago.
Those sentences aren't compatible.
> but hallucination is a major issue
Again, every official public AI interface has warnings/disclaimers for this issue. It's well known. It's not some secret. Every AI researcher is directly or indirectly working on this.
> is in the opposite direction of the “goal” of AGI
This isn't a logical statement, so it's difficult to respond to. Hallucination isn't a direction that's being headed towards, it's being actively, with intent and $$$, headed away from.
Really!?
[0] https://i.imgur.com/ly5yk9h.png
And in this case, OP didn't have to take ChatGPT's word for the existence of the pattern, it showed up on their (digital) doorstep in the form of people taking action based on ChatGPT's incorrect information.
So, pattern noticed and surfaced by an LLM as a hallucination, people take action on the "info", nonzero market demand validated, vendor adds feature.
Unless the phantom feature is very costly to implement, seems like the right response.
I would go on to say that thisminteraction between ‘holes’ exposed by LLM expectations _and_ demonstrated museerbase interest _and_ expert input (by the devs’ decision to implement changes) is an ideal outcome that would not have occurred if each of the pieces were not in place to facilitate these interactions, and there’s probably something here to learn from and expand on in the age of LLMs altering user experiences.
Some people express concerns about AGI creating swarms of robots to conquer the earth and make humans do its bidding. I think market forces are a much more straightforward tool that AI systems will use to shape the world.
1. https://en.wikipedia.org/wiki/Rogue
2. https://en.wikipedia.org/wiki/Rouge_(cosmetics)
It really likes to cogitate on code from several versions ago. And it often insists repeatedly on edits unrelated to the current task.
I feel like I'm spending more time educating the LLM. If I can resist the urge to lean on the LLM beyond its capabilities, I think I can be productive with it. If I'm going to stop teaching the thing, the least it can do is monitor my changes and not try to make suggestions from the first draft of code from five days ago, alas ...
1 - e.g. a 500-line text file representing values that will be converted to enums, with varying adherence to some naming scheme - I start typing, and after correcting the first two, it suggests the next few. I accept its suggestions until it makes a mistake because the data changed, start manual edits again ... I repeated this process for about 30 lines and it successfully learned how I wanted the remainder of the file edited.
I know nothing about this. I imagine people are already working on it, wonder what they've figured out.
(Alternatively, in the future can I pay OpenAI to get ChatGPT to be more likely to recommend my product than my competitors?)
So winning AI SEO is not so different than regular SEO.
Your solution is the equivalent of asking Google to completely delist you because one page you dont want ended up on Googles search results.
I've been wanted them to do this for questions like "what is your context length?" for ages - it frustrates me how badly ChatGPT handles questions about its own abilities, it feels like that would be worth them using some kind of special case or RAG mechanism to support.
Let alone that dynamically modifying the base system prompt would likely break their entire caching mechanism given that caching is based on longest prefix, and I can't imagine that the model's system prompt is somehow excluded from this.
I find it interesting that any user would attribute this issue to Soundslice. As a user, I would be annoyed that GPT is lying and wouldn't think twice about Soundslice looking bad in the process
That kind of thinking is how you never get new customers and eventually fail as a business.
Down voters here on HN seem to live in a egocentric fantasy world, where every human being in the outside world live to serve them. But the reality is that business owners and leaders spend their whole day thinking about how to please their customers and their potential customers. Not other random people who might be misinformed.
Ok, sure, maybe this feature was worth having?
But if some people start sending bad requests your way because they can't or only program poorly, it doesn't make sense to potentially degrade the service for your successful paying customers...
The best LLMs available right in this moment will lie without remorse about bus schedules and airplane departure times. How in the world are businesses supposed to take responsibility for that?
Likewise if I have a neighbour who is a notorious liar tell me I can find a piece of equipment in a certain hardware store, should I be mad at the store owner when I don't find it there, or should I maybe be mad at my neighbour – the notorious liar?
If you are a store own, AND
1. People repeatedly coming in to your shop asking to buy something, AND
2. It is similar to the kinds of things you sell, from the suppliers you usually get supplies from, AND
3. You don't sell it
Then it sounds like your neighbour the notorious liar is doing profitable marketing for your business and sending you leads which you could profitably sell to, if you sold the item.
If there's a single customer who arrives via hallucination, ignore it. If there's a stream of them, why would you not serve them if you can profit by doing so?
There are obviously instances you'd ignore and you seem to be focussing on those rather than what OP was obviously talking about, repeat instances of sensible ideas
OTOH it's free(?) advertising, as long as that first impression isn't too negative.
I also went back to just sleeping on those flights and using connected models for most of my code generation needs.
What surprised me initially was just how confidently wrong Llama was... Now I'm used to confident wrongness from smaller models. It's almost like working with real people...
Example:
https://llama-cpp-agent.readthedocs.io/en/latest/structured-...
I recall that early on a coworker was saying that ChatGPT hallucinated a simpler API than the one we offered, albeit with some easy to fix errors and extra assumptions that could've been nicer defaults in the API. I'm not sure if this ever got implemented though, as he was from a different team.
Some users might share it. ChatGPT has so many users it's somewhat mind boggling
I already strongly suspect that LLMs are just going to magnify the dominance of python as LLMs can remove the most friction from its use. Then will come the second order effects where libraries are explicitly written to be LLM friendly, further removing friction.
LLMs write code best in python -> python gets used more -> python gets optimized for LLMs -> LLMs write code best in python
We don't live in a nice world, so you'll probably end up right.
If the super-intelligent AI understands human incentives and is in control of a very popular service, it can subtly influence people to its agenda by using the power of mass usage. Like how a search engine can influence a population's view of an issue by changing the rankings of news sources that it prefers.
If that's too scary, the failed tool call could trigger another AI to go draft up a PR with that proposed tool, since hey, it's cheap and might be useful.
Dynamic, on-the-fly generation & execution is definitely fascinating to watch in a sandbox, but is far to scary (from a compliance/security/sanity perspective) without spending a lot more time on guardrails.
We do however take note of hallucinated tool calls and have had it suggest an implementation we start with and have several such tools in production now.
It's also useful to spin up any completed agents and interrogate them about what tools they might have found useful during execution (or really any number of other post-process questionnaire you can think of).
1. I might consider a thing like that like any other feature request. If not already added to the feature request tracker, it could be done. It might be accepted or rejected, or more discussion may be wanted, and/or other changes made, etc, like any other feature request.
2. I might add a FAQ entry to specify that it does not have such a feature, and that ChatGPT is wrong. This does not necessarily mean that it will not be added in future, if there is a good reason to do so. If there is a good reason to not include it, this will be mentioned, too. It might also be mentioned other programs that can be used instead if this one doesn't work.
Also note that in the article, the second ChatGPT screenshot has a note on the bottom saying that ChatGPT can make mistakes (which, in this case, it does). Their program might also be made to detect ChatGPT screenshots and to display a special error message in that case.
No comments yet
> Correct feature almost exists
> Creator profile: analytical, perceptive, responsive;
> Feature within product scope, creator ability
> Induce demand
> await "That doesn't work" => "Thanks!"
> update memory
Figuring out the paths that users (or LLMs) actually want to take—not based on your original design or model of what paths they should want, but based on the paths that they actually do want and do trod down. Aka, meeting demand.
what a wonderful incident / bug report my god.
totally sorry for the trouble and amazing find and fix honestly.
sorry i am more amazed than sorry :D. thanks for sharing this !!
so i am happy you implemented this, and will now look at using your service. thx chatgpt, and you.
Maybe I'll turn it into a feature request then ...
It was a plausible answer, and the core of what these models do is generate plausible responses to (or continuations of) the prompt they’re given. They’re not databases or oracles.
With errors like this, if you ask a followup question it’ll typically agree that the feature isn’t supported, because the text of that question combined with its training essentially prompts it to reach that conclusion.
Re the follow-up question, it’s almost certainly the direction that advertising in general is going to take.
The example in the OP is a common one: ask a model how to do something with a tool, and if there’s no easy way to perform that operation they’ll commonly make up a plausible answer.
https://www.soundslice.com/help/en/player/advanced/17/expand...
That's available for any music in Soundslice, not just music that was created via our scanning feature.
ChatGPT routinely hallucinates API calls. ChatGPT flat-out makes it from whole cloth. "Apple Intelligence" creates variants of existing API calls, Usually, by adding nonexistent arguments.
Both of them will hallucinate API calls that are frequently added by programmers through extensions.
Creating the feature means it's no longer misinformation.
The bigger issue isn't that ChatGPT produces misinformation - it's that it takes less effort to update reality to match ChatGPT than it takes to update ChatGPT to match reality. Expect to see even more of this as we match toward accepting ChatGPT's reality over other sources.
If a feature has enough customers to pay for itself, develop it.
"Would you still have added this feature if ChatGPT hadn't bullied you into it?" Absolutely not.
I feel like this resolves several longstanding time travel paradox tropes.
In our new post truth, anti-realism reality, pounding one's head against a brick wall is often instructive in the way the brain damage actually produces great results!
This is generally how you work with LLMs.
Amateur musicians often lack just one or two features in the program they use, and the devs won't respond to their pleas.
Adding support for guitar tabs has made OP's product almost certainly more versatile and useful for a larger set of people. Which, IMHO, is a good thing.
But I also get the resentment of "a darn stupid robot made me do it". We don't take kindly to being bossed around by robots.
No, because you'll be held responsible for the misinformation being accurate: users will say it is YOUR fault when they learn stuff wrong.
this is my general philosophy and, in my case, this is why I deploy things on blockchains
so many people keep wondering about whether there will ever be some mythical unfalsifiable to define “mainstream” use case, and ignoring that crypto natives just … exist. and have problems they will pay (a lot) to solve.
to the author’s burning question about whether any other company has done this. I would say yes. I’ve discovered services recommended by ChatGPT and other LLMs that didnt do what was described of them, and they subsequently tweaked it once they figured out there was new demand
No comments yet
- Do you keep bolting on new updates to match these hallucinations, potentially breaking existing behavior?
- Or do you resign yourself to following whatever spec the AI gods invent next?
- And what if different LLMs hallucinate conflicting behavior for the same endpoint?
I don’t have a great solution, but a few options come to mind:
1. Implement the hallucinated endpoint and return a 200 OK or 202 Accepted, but include an X-Warning header like "X-Warning: The endpoint you used was built in response to ChatGPT hallucinations. Always double-check an LLM's advice on building against 3rd-party APIs with the API docs themselves. Refer to https://api.example.com/docs for our docs. We reserve the right to change our approach to building against LLM hallucinations in the future." Most consumers won’t notice the header, but it’s a low-friction way to correct false assumptions while still supporting the request.
2. Fail loudly: Respond with 404 Not Found or 501 Not Implemented, and include a JSON body explaining that the endpoint never existed and may have been incorrectly inferred by an LLM. This is less friendly but more likely to get the developer’s attention.
Normally I'd say that good API versioning would prevent this, but it feels like that all goes out the window unless an LLM user thinks to double-check what the LLM tells them against actual docs. And if that had happened, it seems like they wouldn't have built against a hallucinated endpoint in the first place.
It’s frustrating that teams now have to reshape their product roadmap around misinformation from language models. It feels like there’s real potential here for long-term erosion of product boundaries and spec integrity.
EDIT: for the down-voters, if you've got actual qualms with the technical aspects of the above, I'd love to hear them and am open to learning if / how I'm wrong. I want to be a better engineer!
Also, it's not like ChatGPT or users are directly querying their API. They're submitting images through the Soundslice website. The images just aren't of the sort that was previously expected.
We detached this subthread from https://news.ycombinator.com/item?id=44492212 and marked it off topic.
I don't care if they used an LLM provided they put their best effort in to confirm that it's clearly communicating the message they are intending to communicate.
Requests containing elements of hostility, shame, or injury frequently serve dual purposes: (1) the ostensible aim of eliciting an action and (2) the underlying objective of inflicting some from of harm (here shame) as a means compelling compliance through emotional leverage. Even if the respondent doesn't honor the request, the secondary purpose still occurs.
But it isn't a given that this will be successful; the outcome of the resulting conversation may well be that shopping malls are, or a particular shopping mall is, more desirable than wetlands, in which case the ostensible respondent will be less likely to comply than they would have been without the conversation. And, in this case, it seems that the conversation is strongly tending toward favoring the use of things like Grammarly rather than opposing it.
So I don't oppose starting such conversations. I think it's better to discuss ethical questions like this openly, even though sometimes people suffer shame as a result.
Having wrangled some open-source work that’s the kind of genius that only its mother could love… there’s a place for idiosyncratic interface design (UI-wise and API-wise), but there’s also a whole group of people who are great at that design sensibility. That category of people doesn’t always overlap with people who are great at the underlying engineering. Similarly, as academic writing tends to demonstrate, people with interesting and important ideas aren’t always people with a tremendous facility for writing to be read.
(And then there are people like me who have neither—I agree that you should roll your eyes at anything I ask an LLM to squirt out! :)
But GP’s technique, like TFA’s, sounds to me like something closer to that of a person with something meaningful to say, who now has a patient close-reader alongside them while they hone drafts. It’s not like you’d take half of your test reader’s suggestions, but some of them might be good in a way that didn’t occur to you in the moment, right?