This is Geoffrey Hinton, https://en.wikipedia.org/wiki/Geoffrey_Hinton, issuing his estimate of p(doom). He has worked in AI for a long time, and played an important role in foundational concepts such as the rise of the idea of backpropagation. So he knows enough about the subject that he should be taken seriously.
See https://pauseai.info/pdoom for a list of other knowledgeable people with their values of p(doom). Geoffrey Hinton would seem to be within the typical range, but on the optimistic side of our odds.
I believe that this should be taken seriously. Not in a Chicken Little, the sky is falling, kind of way. But we should still take the possibility of real risk seriously.
quantified · 9h ago
That's not what the quotes speaker says. He says "wipe us out". Sure won't displace us in agriculture.
blueflow · 9h ago
What data is that probability based on?
quantified · 9h ago
To be fair, once-in-a-species events don't have experimental evidence to go on. It either happens or it doesn't. End of the day, it's the humans who develop and deploy it that do the deed.
An Outside Context Problem was the sort of thing most civilizations encountered just once, and which they tended to encounter rather in the same way a sentence encountered a full stop. The usual example given to illustrate an Outside Context Problem was imagining you were a tribe on a largish, fertile island; you'd tamed the land, invented the wheel or writing or whatever, the neighbors were cooperative or enslaved but at any rate peaceful and you were busy raising temples to yourself with all the excess productive capacity you had, you were in a position of near-absolute power and control which your hallowed ancestors could hardly have dreamed of and the whole situation was just running along nicely like a canoe on wet grass... when suddenly this bristling lump of iron appears sailless and trailing steam in the bay and these guys carrying long funny-looking sticks come ashore and announce you've just been discovered, you're all subjects of the Emperor now, he's keen on presents called tax and these bright-eyed holy men would like a word with your priests.
iab · 9h ago
If it happens or it doesn’t then it must be 50/50 odds
techpineapple · 8h ago
I'm not a statistician, but I'm pretty sure this is not how odds work. "Either I win the lottery tomorrow or I don't!"
See https://pauseai.info/pdoom for a list of other knowledgeable people with their values of p(doom). Geoffrey Hinton would seem to be within the typical range, but on the optimistic side of our odds.
I believe that this should be taken seriously. Not in a Chicken Little, the sky is falling, kind of way. But we should still take the possibility of real risk seriously.
An Outside Context Problem was the sort of thing most civilizations encountered just once, and which they tended to encounter rather in the same way a sentence encountered a full stop. The usual example given to illustrate an Outside Context Problem was imagining you were a tribe on a largish, fertile island; you'd tamed the land, invented the wheel or writing or whatever, the neighbors were cooperative or enslaved but at any rate peaceful and you were busy raising temples to yourself with all the excess productive capacity you had, you were in a position of near-absolute power and control which your hallowed ancestors could hardly have dreamed of and the whole situation was just running along nicely like a canoe on wet grass... when suddenly this bristling lump of iron appears sailless and trailing steam in the bay and these guys carrying long funny-looking sticks come ashore and announce you've just been discovered, you're all subjects of the Emperor now, he's keen on presents called tax and these bright-eyed holy men would like a word with your priests.