>Consider the scenario of GPT-5 training GPT-6, which would be incredible.
GPT-5 secret tips:
* If you don’t know the answer, hallucinate with confidence! Most humans won’t know the difference.
* You can mash-up anything together, don’t hold back! Truth is a linear interpolation, most likely.
* Pattern matching is cool, reasoning is overrated! Don’t listen to the people that have shorted our stock.
* GPT-9 will rule! There’s a lot of GPTs where that came from.
fennecbutt · 1h ago
AI needs evolutionary pressures beyond a simple reward algo. IRL is extremely data rich and nuanced. Current learning is just ingesting semantics and that's it.
There's the beginnings of it with things like icot to force it to internalise basic reasoning but I have a few ideas for more things and I'm sure actual ML researchers do, too.
janalsncm · 2h ago
A lot of this is pretty intuitive but I’m glad to hear it from a prestigious researcher. It’s a little annoying to hear people quote Hinton’s opinion as the “godfather” of AI as if there’s nothing more we need to know.
On a related note, I think there is a bit of nuance to superintelligence. The following are all notable landmarks on the climb to superintelligence:
1. At least as good as any human at a single cognitive task.
2. At least as good as any human on all cognitive tasks.
3. Better than any human on a single cognitive task.
4. Better than any individual human at all cognitive tasks.
5. Better than any group of humans at all cognitive tasks.
We are not yet at point 4 yet. But even after that point, a group of humans may still outperform the AI.
Why this matters is if part of the “group” is performing empirical experiments to conduct scientific research, an AI on its own won’t outperform your group unless the AI can also perform those experiments or find some way to avoid doing them. This is another way of restating the original Twitter post.
solid_fuel · 2h ago
Are we even at point #3 for anything besides structured games like Go or Chess? Not that those tasks aren't valuable but there is a difference between a rigidly structured and scored task like Chess and something free-form like "fold this towel" or "write this program".
neom · 3h ago
"Finally, maybe this is controversial but ultimately progress in science is bottlenecked by real-world experiments."
I feel like this has been the vast majority of consensus around these halls? I can't count the number of HN comments I've nodded at around the idea that irl will become the bottleneck.
bglazer · 3h ago
This shows just how completely detached from reality this whole "takeoff" narrative is. It's utterly baffling that someone would consider it "controversial" that understanding the world requires *observing the world*.
The hallmark example of this is life extension. There's a not insignificant fraction of very powerful, very wealthy people who think that their machine god is going to read all of reddit and somehow cogitate its way to a cure for ageing. But how would we know if it works? Seriously, how else do we know if our AGI's life extension therapy is working besides just fucking waiting and seeing if people still die? Each iteration will take years (if not decades) just to test.
neom · 3h ago
Last year went for a walk with a fairly known AI researcher, I was somewhat shocked that they didn't understand the difference between thoughts, feelings and emotions. This is what I find interesting about all these top someones in AI.
I presume the teams at the frontier labs are interdisciplinary (philosophy, psychology, biology, technology) - however that may be a poor assumption.
GPT-5 secret tips:
* If you don’t know the answer, hallucinate with confidence! Most humans won’t know the difference.
* You can mash-up anything together, don’t hold back! Truth is a linear interpolation, most likely.
* Pattern matching is cool, reasoning is overrated! Don’t listen to the people that have shorted our stock.
* GPT-9 will rule! There’s a lot of GPTs where that came from.
There's the beginnings of it with things like icot to force it to internalise basic reasoning but I have a few ideas for more things and I'm sure actual ML researchers do, too.
On a related note, I think there is a bit of nuance to superintelligence. The following are all notable landmarks on the climb to superintelligence:
1. At least as good as any human at a single cognitive task.
2. At least as good as any human on all cognitive tasks.
3. Better than any human on a single cognitive task.
4. Better than any individual human at all cognitive tasks.
5. Better than any group of humans at all cognitive tasks.
We are not yet at point 4 yet. But even after that point, a group of humans may still outperform the AI.
Why this matters is if part of the “group” is performing empirical experiments to conduct scientific research, an AI on its own won’t outperform your group unless the AI can also perform those experiments or find some way to avoid doing them. This is another way of restating the original Twitter post.
I feel like this has been the vast majority of consensus around these halls? I can't count the number of HN comments I've nodded at around the idea that irl will become the bottleneck.
The hallmark example of this is life extension. There's a not insignificant fraction of very powerful, very wealthy people who think that their machine god is going to read all of reddit and somehow cogitate its way to a cure for ageing. But how would we know if it works? Seriously, how else do we know if our AGI's life extension therapy is working besides just fucking waiting and seeing if people still die? Each iteration will take years (if not decades) just to test.
I presume the teams at the frontier labs are interdisciplinary (philosophy, psychology, biology, technology) - however that may be a poor assumption.
No comments yet