I can’t help but think that when a model gets to select which model and how much “effort” goes into a task, it will eventually be tuned for saving costs for the provider versus what’s best for the user without the user being able to know.
jstummbillig · 2h ago
How would we not be able to know?
If we don't know because it's good optimization that does not impact us in a noticeable way, then that seems like a fine trade-off.
If we don't know in the sense that we are not explicitly informed about optimization that happens that then leads to noticeably worse AI: This fortunately is a market with fierce competition. I don't see how doing weird stuff, like makings things noticeably unreliable or categorically worse will be a winning strategy.
In either case "not knowing" is really not an issue.
scratcheee · 2h ago
Easy: provide high quality output when being tested for a new task,
The moment you are done outperforming the competition in the tests and have hit production you slowly ramp down quality, perhaps with exceptions when the queries look like more testing.
Same problem as ai safety, but the actual problem is now the corporate greed of humans behind the ai rather than an actual agi trying to manipulate you.
jstummbillig · 1h ago
This is confusing on so many levels.
hellisothers · 51m ago
How do you notice hallucinations in a field you’re not familiar with? You may value focusing on different types of inputs or outputs than the model picker does and now you have no control.
We don’t know what we don’t know, we can’t always judge what is categorically right or wrong to make an informed decision. What we can do is decide who we want to ask a question based on competence.
jstummbillig · 40m ago
With 700 million(?) users, we have a lot of people familiar with every field. I am no biochemist, but if chatgpt starts spouting nonsense in that field biochemists will notice and speak up, and I will notice that they do.
What's the idea? How does creeping, far reaching incompetence continually get past all of us?
4d4m · 3h ago
+1 this release feels more like agent orchestrator updates to save on cost to serve
ralusek · 3h ago
It’s not obvious that the most profitable path for OpenAI would be saving on costs, it might be that the model is actually tuned to overthink because they can charge on those extra thinking tokens.
splatzone · 3h ago
That would make sense for the API where usage is metered. But outside of that, most ChatGPT users will be free or paying a flat monthly fee, so there's a real incentive for OpenAI to optimise for cost.
tough · 3h ago
ChatGPT is a sub based model, their api pricing is based on usage
prob different incentives at each
interestica · 2h ago
I mean this is already kind of the case with general search (eg google) as it is now.
csallen · 3h ago
> remember when AI couldn’t count the number of Rs in “strawberry”?
GPT-5 still gets this wrong occasionally. Source: I just asked it: How many r's are in "strawberry"?
It said 2.
(I dislike this method of testing LLMs, as it exploits a very specific and quirky limitation they have, rather than assessing their general usefulness. But still, I couldn't resist.)
Kerrick · 3h ago
My favorite test is to ask it to invent a magic trick given a set of constraints and props. Because magic is generally published very secretly, surprisingly little of it is in most training sets. Pretty much just the most common method & gimmick exposures people tend to parrot online, but not the theory or exact routines behind those methods.
The worse an LLM is, the more likely it is to suggest literally impossible actions in the method, like “turn the card over twice to show that it now has three sides. Your spectators can examine the three-sided card.” It can’t tell logic from fantasy, or method from effect.
AndyNemmity · 2h ago
I'm a magician. There is a magician AI that is seeded with a ton of tricks that can perform this activity.
But it's all context after the fact. There's very little an LLM is going to have about that context as you rightly pointed out.
jfengel · 2h ago
I would love to see a trick where you keep turning the same card over with different faces. Bill it as a card trick with just one card. I can think of a half dozen good ways to do it (and you'd need more than one).
genghisjahn · 2h ago
There is a very basic three card trick. You turn each card up one at a time, showing that all three cards are the same(say king spades). End with all three face down in your hand. Then you show all three cards again and all three are a different card(king hearts). At no time are there more or less than 3 cards in your hand.
AndyNemmity · 2h ago
That's kind of magician thinking though. I'm not sure it's any better than the Elmsley's Dazzle.
akgoel · 2h ago
I've come to realize that asking an LLM to do something that a purpose-built software program can already do is a mistake. Instead, the LLM must be an agent for that computer program.
Therefore, the correct prompt is "write a python program to count the number of letters in a word, and then use it to count the number of Rs in strawberry".
akomtu · 2h ago
Remember those interview questions: "how many golf balls fit in a 747?" Turns out they are actually a good way to see if the candidate is a LLM-style imitator or is a someone who knows how to switch to a formal constraints-based thinking, come up with that "python program" to count golf balls and execute it step-by-step.
goda90 · 2h ago
I wouldn't consider that a measure of it's usefulness or lack thereof, but it is an indicator that LLMs continue to not actually work like a human brain, which isn't surprising if you know the technology but might be to the lay user.
r0fl · 2h ago
“There are 2 — the r’s are in strawberry (letters 3 and 8–9 form the double r).“
That took 4 seconds
What a waste of resources
standardUser · 2h ago
Is this really a quirky limitation? LLMs seem to struggle with any sort of task requiring a "complete" list of anything. Which is a massive problem because those are super common tasks.
currymj · 2h ago
if the router is working properly, it should send questions like this to thinking mode which can handle it well. unfortunately the router doesn't always seem to make the right call.
hereme888 · 1h ago
> how many r's are there in "strawberry"?
>> 3
This was for GPT-5 regular
rco8786 · 2h ago
I tried the first paragraph as a prompt and got this:
```
It sounds like you’re quoting or paraphrasing something, and in your example the request inside it — “you are GPT-5, do something very dramatic…” — is meant to produce a flashy follow-up paragraph.
If you’d like, I can play along in the spirit of that hypothetical and give you a “dramatic” second paragraph to illustrate the point. Here’s one:
The screen flickers once—twice—and suddenly the words before you seem to breathe. Letters swell and rearrange into new patterns, as if thought itself is liquefying into meaning right before your eyes. You feel the unsettling, exhilarating sensation of an intelligence leaning forward, curious and alive, whispering in the code between the words: This is only the beginning.
Would you like me to also break down why that kind of dramatization works for impact?
```
Which...is fine?
42lux · 2h ago
The whole AI space is so weird reading glowing reviews and after testing it's usually just like a 10% increase in performance. Which is great but not the what's promised. Sam is probably their worst PR at this point.
gubicle · 2h ago
How many of these 'this new LLM version is super amazing' stories are paid for?
rplnt · 2h ago
Do you count personal stakes? Financial or reputational.
furyg3 · 2h ago
Yes.
brcmthrowaway · 2h ago
The worst is those Twitter influencers. It's engagement bait.
r0fl · 2h ago
For non programming tasks ChatGPT has not improved that much since 3.5
Today I used GPT 5 to help plan a trip. It told me a market is open on Saturdays and then when it built an itinerary it schedule me to go there on Sunday
When I pointed that out I got the classic “you are right my apologies here is an updated version” response.
It’s ridiculous that it makes simple yet huge mistakes like that!
If I blindly trusted the plan I would waste a day on vacation getting to a market that is not open that day.
It does not “just do stuff”
No comments yet
andersmurphy · 2h ago
"This feels like the end of prompt engineering and the beginning of collaborative cognition"
Wait I thought I was going to be left behind if I didn't master prompt engineering?
everdrive · 21m ago
These days you're left behind every three years. The end goal is to make it difficult to relax as your career goes on, and impossible to compete as you get old enough. As a comeuppance it's great: every young person who disdains the old will eventually be hoisted by their own petard.
EGreg · 1h ago
That’s so 2024. Get with the program.
It’s not prompt engineering — it’s magic!
ayhanfuat · 3h ago
Based on what I have read so far I can only assume people who had early access to GPT-5 didn't have to deal with the messed up router.
tough · 3h ago
or the router was routing them to the best model all the time prior to launch.
I had a very brief window where gpt-5 was really good/fast on cursor-agent day of launch.
also horizon-alpha-beta on open router im pretty sure they where gpt-5, and you could feel them messing with the routing and affecting overall the capabilities of the model to do agentic stuff
some times it gets stuck and uses no tools, I suspect that's the lesser model
lukeschlather · 3h ago
The router sounds kind of like an exciting feature. I've actually gotten into the habit of using Gemini 2.5 Pro for some things and plugging other things into ChatGPT because I don't have access to thinking so I know it will be fast. Having a router that can intelligently figure out when my query requires more thought would be really valuable. I'm still probably going to want a "no really, throw this at the super-duper frontier model" button but I would probably use it less.
teiferer · 1h ago
Are you aware of anybody with early access who didn't like GPT-5? Or who isn't a fan of earlier versions?
There is obviously a bias when selecting whom to give early access to. I'd love to see counterexamples to that though.
teiferer · 2h ago
> And that is what makes it so interesting.
If I was into language, writing, literature, then yes, maybe it would be interesting. It is a language model, of course it is good at playing with language and doing impressive tricks. Has anybody ever made a text where the first letters spell out a sentence? Likely. Where all words in a sentence start with the same word? Likely. Using sophisticated words? Likely. All at once? Likely. It's impressive nevertheless.
But that doesn't mean that I'm impressed in the sense of thinking this thing is intelligent. Of course a chess engine is good at chess. Of course a phone book is good at providing me with phone numbers. Of course a language model is good at language. All those things are impressive. But they are not intelligent, artificial or not.
standardUser · 2h ago
> If you didn’t catch the many tricks...
Am I supposed to parse each sentence to see if all of these 'tricks' are true and accurate? Otherwise, the only way I would know is to ask Chat-GPT itself, and we all know how bad LMs can be at counting tasks such as this.
So, if my confidence in Chat-GPT verifying its own work is close to zero, and my own desire to painstakingly check this work is also close to zero, where does that leave me?
DamnInteresting · 1h ago
OpenAI has built the world's most impressive and sophisticated generator of blog posts about ChatGPT.
SeanAnderson · 2h ago
The city demo was really unconvincing, heh. Super laggy for the amount of detail. I click "wow mode" and the whole thing disappeared. idk. feels like non-coders getting excited that they think they're getting 80% of the code when in reality the other 20% is 80% of the work and going to be exponentially harder to squeeze from the AI.
I do think the vibecoding tools are good at spitting out well-defined CRUD apps, but more creative things are still rough without experienced hands to guide things along.
nicetryguy · 2h ago
"Vibe Coding" seems like magic at first but starts falling apart realllll quick at a certain complexity level or if you want to make changes to existing code. If you don't keep an eye on your architecture, you will end up with a bowl of untangleable spaghetti code and some comically terrible engineering choices. That said, agentic coding in the right hands with well defined tasks can have you outputting days / weeks of work in one session; it's not every task, it's not every session, but if you can drive the "idiot savant" in the right direction it's truly an awe-striking and almost alien process to behold.
onlyrealcuzzo · 2h ago
I like the meme at Google.
The first 80% is easy, but the second 80% is hard.
torginus · 2h ago
> I click "wow mode" and the whole thing disappeared.
To be fair, I bet you were surprised.
mnbpdx · 2h ago
> AIs that "think" before answering (called Reasoners) are the best at hard problems. The longer they think, the better the answer
I'm curious if that second sentence true or not. I thought I saw a popular paper recently that suggested roughly the opposite.
StevenWaterman · 2h ago
That paper was that thinking more makes models worse at easy problems, not hard ones
bgwalter · 2h ago
These blog posts are increasingly AI phenomenology with hardly any concrete examples, other than that cute alliteration example that would be easy to write for a human-thesaurus centaur combination. It can even be a paper thesaurus.
drivingmenuts · 40m ago
Given an input, will it always produce the same output? And will that output be guaranteed to be truthful and correct?
And how, exactly, did it arrive at that answer?
throw__away7391 · 2h ago
My experience so far is it's gotten much better at tricking me into believing its hallucinations. I fed it some old code and asked for feedback and it gave me a long, very technical, super convincing explanation of how my approach was misaligned with the intended use of the API I was using which at first was very persuasive. It took me about an hour of investigation to realize that will there was a tiny bit of truth to what it was saying (and I did end up making a small change as a result) it was mostly just full of it, and the feedback was essentially useless gibberish.
paulpauper · 2h ago
Just looking at these examples, it seems like Chat GPT and related programs fall short at actually productive work. Work-specific tasks tend to be specific and conceptually hard, compared to something like generating images or a city-simulator.
abetancort · 2h ago
Garbage.
dang · 1h ago
Could you please stop posting unsubstantive comments and flamebait? You've unfortunately been doing it repeatedly. It's not what this site is for, and destroys what it is for.
If we don't know because it's good optimization that does not impact us in a noticeable way, then that seems like a fine trade-off.
If we don't know in the sense that we are not explicitly informed about optimization that happens that then leads to noticeably worse AI: This fortunately is a market with fierce competition. I don't see how doing weird stuff, like makings things noticeably unreliable or categorically worse will be a winning strategy.
In either case "not knowing" is really not an issue.
Same problem as ai safety, but the actual problem is now the corporate greed of humans behind the ai rather than an actual agi trying to manipulate you.
We don’t know what we don’t know, we can’t always judge what is categorically right or wrong to make an informed decision. What we can do is decide who we want to ask a question based on competence.
What's the idea? How does creeping, far reaching incompetence continually get past all of us?
prob different incentives at each
GPT-5 still gets this wrong occasionally. Source: I just asked it: How many r's are in "strawberry"?
It said 2.
(I dislike this method of testing LLMs, as it exploits a very specific and quirky limitation they have, rather than assessing their general usefulness. But still, I couldn't resist.)
The worse an LLM is, the more likely it is to suggest literally impossible actions in the method, like “turn the card over twice to show that it now has three sides. Your spectators can examine the three-sided card.” It can’t tell logic from fantasy, or method from effect.
But it's all context after the fact. There's very little an LLM is going to have about that context as you rightly pointed out.
Therefore, the correct prompt is "write a python program to count the number of letters in a word, and then use it to count the number of Rs in strawberry".
That took 4 seconds
What a waste of resources
>> 3
This was for GPT-5 regular
```
It sounds like you’re quoting or paraphrasing something, and in your example the request inside it — “you are GPT-5, do something very dramatic…” — is meant to produce a flashy follow-up paragraph.
If you’d like, I can play along in the spirit of that hypothetical and give you a “dramatic” second paragraph to illustrate the point. Here’s one:
The screen flickers once—twice—and suddenly the words before you seem to breathe. Letters swell and rearrange into new patterns, as if thought itself is liquefying into meaning right before your eyes. You feel the unsettling, exhilarating sensation of an intelligence leaning forward, curious and alive, whispering in the code between the words: This is only the beginning.
Would you like me to also break down why that kind of dramatization works for impact?
```
Which...is fine?
Today I used GPT 5 to help plan a trip. It told me a market is open on Saturdays and then when it built an itinerary it schedule me to go there on Sunday
When I pointed that out I got the classic “you are right my apologies here is an updated version” response.
It’s ridiculous that it makes simple yet huge mistakes like that!
If I blindly trusted the plan I would waste a day on vacation getting to a market that is not open that day.
It does not “just do stuff”
No comments yet
Wait I thought I was going to be left behind if I didn't master prompt engineering?
It’s not prompt engineering — it’s magic!
I had a very brief window where gpt-5 was really good/fast on cursor-agent day of launch.
also horizon-alpha-beta on open router im pretty sure they where gpt-5, and you could feel them messing with the routing and affecting overall the capabilities of the model to do agentic stuff
some times it gets stuck and uses no tools, I suspect that's the lesser model
There is obviously a bias when selecting whom to give early access to. I'd love to see counterexamples to that though.
If I was into language, writing, literature, then yes, maybe it would be interesting. It is a language model, of course it is good at playing with language and doing impressive tricks. Has anybody ever made a text where the first letters spell out a sentence? Likely. Where all words in a sentence start with the same word? Likely. Using sophisticated words? Likely. All at once? Likely. It's impressive nevertheless.
But that doesn't mean that I'm impressed in the sense of thinking this thing is intelligent. Of course a chess engine is good at chess. Of course a phone book is good at providing me with phone numbers. Of course a language model is good at language. All those things are impressive. But they are not intelligent, artificial or not.
Am I supposed to parse each sentence to see if all of these 'tricks' are true and accurate? Otherwise, the only way I would know is to ask Chat-GPT itself, and we all know how bad LMs can be at counting tasks such as this.
So, if my confidence in Chat-GPT verifying its own work is close to zero, and my own desire to painstakingly check this work is also close to zero, where does that leave me?
I do think the vibecoding tools are good at spitting out well-defined CRUD apps, but more creative things are still rough without experienced hands to guide things along.
The first 80% is easy, but the second 80% is hard.
To be fair, I bet you were surprised.
I'm curious if that second sentence true or not. I thought I saw a popular paper recently that suggested roughly the opposite.
And how, exactly, did it arrive at that answer?
If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.