Some psychologists finetuned a Llama-family LLM on some psychology data - “a data set called Psych-101, which contained data from 160 previously published psychology experiments, covering more than 60,000 participants who made more than 10 million choices in total”, so I guess maybe 10 million tokens?
That part seems to make sense, but I cannot rightly comprehend the confusion that follows.
Some psychology researchers are claiming it has become a model of human cognition? (Because it can imitate the way a psychology study participant answers psychology study questions?)
Other psychology researchers are disputing this by testing its reaction time and digit span memory? (Are they administering an iq test? A cranial nerves exam?)
lewdwig · 2h ago
I don’t think healthy scepticism is (or should be) controversial. But I find it interesting how willing certain people are to confidently claim that a model does or does not accurately model human cognition when we clearly still _barely understand human cognition_.
Where do people derive their certainty, which seems to me largely misplaced?
jedimastert · 2h ago
The tech industry has a very long and proud history of gaining a very surface level understanding of a different industry then immediately claiming to "disrupt" it.
The unearned confidence of tech bros should be studied
add-sub-mul-div · 48m ago
I don't understand the "does or does not" framing, as if there's symmetry to the debate. One side is making the claim that we're on the verge of creating that which you say we barely understand. The other side should remain at certainty that no such thing is happening until there's proof that it is. And what would that proof look like when, without said understanding, all we can do is point to output from a program that looks or sounds like its training data?
monkaiju · 39m ago
The burden of proof is on the one making the (extraordinary) claim.
senectus1 · 2h ago
I have the same objections to the term "IQ".
allears · 2h ago
Neuroscience is still struggling to understand the basic operations of the brain, let alone the "mind." There's no agreed-upon definition of "intelligence." Can you define cognition (literally, "knowing") without defining intelligence?
These fields are making remarkable strides, but they're still in their infancy. Whoever writes these breathless press releases, they probably have a degree in marketing.
That part seems to make sense, but I cannot rightly comprehend the confusion that follows.
Some psychology researchers are claiming it has become a model of human cognition? (Because it can imitate the way a psychology study participant answers psychology study questions?)
Other psychology researchers are disputing this by testing its reaction time and digit span memory? (Are they administering an iq test? A cranial nerves exam?)
Where do people derive their certainty, which seems to me largely misplaced?
The unearned confidence of tech bros should be studied
These fields are making remarkable strides, but they're still in their infancy. Whoever writes these breathless press releases, they probably have a degree in marketing.