I tried the first paragraph as a prompt and got this:
```
It sounds like you’re quoting or paraphrasing something, and in your example the request inside it — “you are GPT-5, do something very dramatic…” — is meant to produce a flashy follow-up paragraph.
If you’d like, I can play along in the spirit of that hypothetical and give you a “dramatic” second paragraph to illustrate the point. Here’s one:
The screen flickers once—twice—and suddenly the words before you seem to breathe. Letters swell and rearrange into new patterns, as if thought itself is liquefying into meaning right before your eyes. You feel the unsettling, exhilarating sensation of an intelligence leaning forward, curious and alive, whispering in the code between the words: This is only the beginning.
Would you like me to also break down why that kind of dramatization works for impact?
```
Which...is fine?
andersmurphy · 13m ago
"This feels like the end of prompt engineering and the beginning of collaborative cognition"
Wait I thought I was going to be left behind if I didn't master prompt engineering?
42lux · 19m ago
The whole AI space is so weird reading glowing reviews and after testing it's usually just like a 10% increase in performance. Which is great but not the what's promised. Sam is probably their worst PR at this point.
gubicle · 16m ago
How many of these 'this new LLM version is super amazing' stories are paid for?
brcmthrowaway · 6m ago
The worst is those Twitter influencers. It's engagement bait.
SeanAnderson · 20m ago
The city demo was really unconvincing, heh. Super laggy for the amount of detail. I click "wow mode" and the whole thing disappeared. idk. feels like non-coders getting excited that they think they're getting 80% of the code when in reality the other 20% is 80% of the work and going to be exponentially harder to squeeze from the AI.
I do think the vibecoding tools are good at spitting out well-defined CRUD apps, but more creative things are still rough without experienced hands to guide things along.
nicetryguy · 44s ago
"Vibe Coding" seems like magic at first but starts falling apart realllll quick at a certain complexity level or if you want to make changes to existing code. If you don't keep an eye on your architecture, you will end up with a bowl of untangleable spaghetti code and some comically terrible engineering choices. That said, agentic coding in the right hands with well defined tasks can have you outputting days / weeks of work in one session; it's not every task, it's not every session, but if you can drive the "idiot savant" in the right direction it's truly an awe-striking and almost alien process to behold.
onlyrealcuzzo · 13m ago
I like the meme at Google.
The first 80% is easy, but the second 80% is hard.
transcriptase · 1h ago
I can’t help but think that when a model gets to select which model and how much “effort” goes into a task, it will eventually be tuned for saving costs for the provider versus what’s best for the user without the user being able to know.
jstummbillig · 38m ago
How would we not be able to know?
If we don't know because it's good optimization that does not impact us in a noticeable way, then that seems like a fine trade-off.
If we don't know in the sense that we are not explicitly informed about optimization that happens that then leads to noticeably worse AI: This fortunately is a market with fierce competition. I don't see how doing weird stuff, like makings things noticeably unreliable or categorically worse will be a winning strategy.
In either case "not knowing" is really not an issue.
scratcheee · 29m ago
Easy: provide high quality output when being tested for a new task,
The moment you are done outperforming the competition in the tests and have hit production you slowly ramp down quality, perhaps with exceptions when the queries look like more testing.
Same problem as ai safety, but the actual problem is now the corporate greed of humans behind the ai rather than an actual agi trying to manipulate you.
4d4m · 1h ago
+1 this release feels more like agent orchestrator updates to save on cost to serve
interestica · 3m ago
I mean this is already kind of the case with general search (eg google) as it is now.
ralusek · 1h ago
It’s not obvious that the most profitable path for OpenAI would be saving on costs, it might be that the model is actually tuned to overthink because they can charge on those extra thinking tokens.
splatzone · 1h ago
That would make sense for the API where usage is metered. But outside of that, most ChatGPT users will be free or paying a flat monthly fee, so there's a real incentive for OpenAI to optimise for cost.
tough · 1h ago
ChatGPT is a sub based model, their api pricing is based on usage
prob different incentives at each
csallen · 1h ago
> remember when AI couldn’t count the number of Rs in “strawberry”?
GPT-5 still gets this wrong occasionally. Source: I just asked it: How many r's are in "strawberry"?
It said 2.
(I dislike this method of testing LLMs, as it exploits a very specific and quirky limitation they have, rather than assessing their general usefulness. But still, I couldn't resist.)
Kerrick · 1h ago
My favorite test is to ask it to invent a magic trick given a set of constraints and props. Because magic is generally published very secretly, surprisingly little of it is in most training sets. Pretty much just the most common method & gimmick exposures people tend to parrot online, but not the theory or exact routines behind those methods.
The worse an LLM is, the more likely it is to suggest literally impossible actions in the method, like “turn the card over twice to show that it now has three sides. Your spectators can examine the three-sided card.” It can’t tell logic from fantasy, or method from effect.
AndyNemmity · 23m ago
I'm a magician. There is a magician AI that is seeded with a ton of tricks that can perform this activity.
But it's all context after the fact. There's very little an LLM is going to have about that context as you rightly pointed out.
jfengel · 41m ago
I would love to see a trick where you keep turning the same card over with different faces. Bill it as a card trick with just one card. I can think of a half dozen good ways to do it (and you'd need more than one).
genghisjahn · 17m ago
There is a very basic three card trick. You turn each card up one at a time, showing that all three cards are the same(say king spades). End with all three face down in your hand. Then you show all three cards again and all three are a different card(king hearts). At no time are there more or less than 3 cards in your hand.
AndyNemmity · 21m ago
That's kind of magician thinking though. I'm not sure it's any better than the Elmsley's Dazzle.
akgoel · 22m ago
I've come to realize that asking an LLM to do something that a purpose-built software program can already do is a mistake. Instead, the LLM must be an agent for that computer program.
Therefore, the correct prompt is "write a python program to count the number of letters in a word, and then use it to count the number of Rs in strawberry".
akomtu · 10m ago
Remember those interview questions: "how many golf balls fit in a 747?" Turns out they are actually a good way to see if the candidate is a LLM-style imitator or is a someone who knows how to switch to a formal constraints-based thinking, come up with that "python program" to count golf balls and execute it step-by-step.
r0fl · 10m ago
“There are 2 — the r’s are in strawberry (letters 3 and 8–9 form the double r).“
That took 4 seconds
What a waste of resources
goda90 · 46m ago
I wouldn't consider that a measure of it's usefulness or lack thereof, but it is an indicator that LLMs continue to not actually work like a human brain, which isn't surprising if you know the technology but might be to the lay user.
currymj · 26m ago
if the router is working properly, it should send questions like this to thinking mode which can handle it well. unfortunately the router doesn't always seem to make the right call.
ayhanfuat · 1h ago
Based on what I have read so far I can only assume people who had early access to GPT-5 didn't have to deal with the messed up router.
tough · 1h ago
or the router was routing them to the best model all the time prior to launch.
I had a very brief window where gpt-5 was really good/fast on cursor-agent day of launch.
also horizon-alpha-beta on open router im pretty sure they where gpt-5, and you could feel them messing with the routing and affecting overall the capabilities of the model to do agentic stuff
some times it gets stuck and uses no tools, I suspect that's the lesser model
lukeschlather · 1h ago
The router sounds kind of like an exciting feature. I've actually gotten into the habit of using Gemini 2.5 Pro for some things and plugging other things into ChatGPT because I don't have access to thinking so I know it will be fast. Having a router that can intelligently figure out when my query requires more thought would be really valuable. I'm still probably going to want a "no really, throw this at the super-duper frontier model" button but I would probably use it less.
r0fl · 13m ago
For non programming tasks ChatGPT has not improved that much since 3.5
Today I used GPT 5 to help plan a trip. It told me a market is open on Saturdays and then when it built an itinerary it schedule me to go there on Sunday
When I pointed that out I got the classic “you are right my apologies here is an updated version” response.
It’s ridiculous that it makes simple yet huge mistakes like that!
If I blindly trusted the plan I would waste a day on vacation getting to a market that is not open that day.
It does not “just do stuff”
bgwalter · 7m ago
These blog posts are increasingly AI phenomenology with hardly any concrete examples, other than that cute alliteration example that would be easy to write for a human-thesaurus centaur combination. It can even be a paper thesaurus.
mnbpdx · 46m ago
> AIs that "think" before answering (called Reasoners) are the best at hard problems. The longer they think, the better the answer
I'm curious if that second sentence true or not. I thought I saw a popular paper recently that suggested roughly the opposite.
StevenWaterman · 23m ago
That paper was that thinking more makes models worse at easy problems, not hard ones
throw__away7391 · 10m ago
My experience so far is it's gotten much better at tricking me into believing its hallucinations. I fed it some old code and asked for feedback and it gave me a long, very technical, super convincing explanation of how my approach was misaligned with the intended use of the API I was using which at first was very persuasive. It took me about an hour of investigation to realize that will there was a tiny bit of truth to what it was saying (and I did end up making a small change as a result) it was mostly just full of it, and the feedback was essentially useless gibberish.
paulpauper · 48m ago
Just looking at these examples, it seems like Chat GPT and related programs fall short at actually productive work. Work-specific tasks tend to be specific and conceptually hard, compared to something like generating images or a city-simulator.
```
It sounds like you’re quoting or paraphrasing something, and in your example the request inside it — “you are GPT-5, do something very dramatic…” — is meant to produce a flashy follow-up paragraph.
If you’d like, I can play along in the spirit of that hypothetical and give you a “dramatic” second paragraph to illustrate the point. Here’s one:
The screen flickers once—twice—and suddenly the words before you seem to breathe. Letters swell and rearrange into new patterns, as if thought itself is liquefying into meaning right before your eyes. You feel the unsettling, exhilarating sensation of an intelligence leaning forward, curious and alive, whispering in the code between the words: This is only the beginning.
Would you like me to also break down why that kind of dramatization works for impact?
```
Which...is fine?
Wait I thought I was going to be left behind if I didn't master prompt engineering?
I do think the vibecoding tools are good at spitting out well-defined CRUD apps, but more creative things are still rough without experienced hands to guide things along.
The first 80% is easy, but the second 80% is hard.
If we don't know because it's good optimization that does not impact us in a noticeable way, then that seems like a fine trade-off.
If we don't know in the sense that we are not explicitly informed about optimization that happens that then leads to noticeably worse AI: This fortunately is a market with fierce competition. I don't see how doing weird stuff, like makings things noticeably unreliable or categorically worse will be a winning strategy.
In either case "not knowing" is really not an issue.
Same problem as ai safety, but the actual problem is now the corporate greed of humans behind the ai rather than an actual agi trying to manipulate you.
prob different incentives at each
GPT-5 still gets this wrong occasionally. Source: I just asked it: How many r's are in "strawberry"?
It said 2.
(I dislike this method of testing LLMs, as it exploits a very specific and quirky limitation they have, rather than assessing their general usefulness. But still, I couldn't resist.)
The worse an LLM is, the more likely it is to suggest literally impossible actions in the method, like “turn the card over twice to show that it now has three sides. Your spectators can examine the three-sided card.” It can’t tell logic from fantasy, or method from effect.
But it's all context after the fact. There's very little an LLM is going to have about that context as you rightly pointed out.
Therefore, the correct prompt is "write a python program to count the number of letters in a word, and then use it to count the number of Rs in strawberry".
That took 4 seconds
What a waste of resources
I had a very brief window where gpt-5 was really good/fast on cursor-agent day of launch.
also horizon-alpha-beta on open router im pretty sure they where gpt-5, and you could feel them messing with the routing and affecting overall the capabilities of the model to do agentic stuff
some times it gets stuck and uses no tools, I suspect that's the lesser model
Today I used GPT 5 to help plan a trip. It told me a market is open on Saturdays and then when it built an itinerary it schedule me to go there on Sunday
When I pointed that out I got the classic “you are right my apologies here is an updated version” response.
It’s ridiculous that it makes simple yet huge mistakes like that!
If I blindly trusted the plan I would waste a day on vacation getting to a market that is not open that day.
It does not “just do stuff”
I'm curious if that second sentence true or not. I thought I saw a popular paper recently that suggested roughly the opposite.