Collapse might not be the right word because it conjures up an image of releases that are broken and unusable.
Instead we may see (and may already be seeing) an increasing slowdown of progress as good training material becomes more difficult to source. That means longer time between releases and less obvious improvement with each release.
behnamoh · 41m ago
I've heard that OpenAI and many AI labs put watermarks [0] in their LLM outputs to detect AI-generated content and filter it out.
[0] Like statistics of words, etc.
jsheard · 7m ago
Maybe they do, but there's enough players all working on this stuff independently of each other that filtering out their own noise would only get them so far.
I noticed the Llama 4 system prompt has a whole paragraph devoted to suppressing various GPT-isms, implying they weren't able to scrub GPT output from the training set.
IAmGraydon · 39s ago
Interesting. That could certainly come in handy if it’s something they can’t avoid. We, too, might be able to better detect and filter their output.
dustingetz · 6m ago
do they also watermark the code?
Rabbit_Brave · 39m ago
These companies are sitting on a never-ending stream of human created data. What do you think happens to your conversations or other interactions with AI? Quality might be a bit sus though.
phillipcarter · 8m ago
Most of the human-created data is also very low quality. But it's also limited in other ways, such as how a lot of so-called high-quality data online is typically the finished answer to a question, with no serialization of the thought process that lead to that answer.
PessimalDecimal · 3m ago
How will they tell if data is human-created or not?
AstroBen · 19m ago
I'd imagine it's really low quality data. Most or all of my conversations with an LLM are questions or telling it to do something, with varying levels of specificity
I'm not sure what they'd get from training on that
insin · 14m ago
I sometimes wonder if they're vulnerable to a coordinated effort of deliberately upvoting shit assistant turns and praising in the next user turn - how much does that actually contribute to future training, if at all?
I had a very basic React question about useState while porting some vanilla code last week which all models of all stripes I've tried it on have been confidently and completely incorrect about, up to stating the code absolutely will not work, even when I take a turn to assert that I ran it and it does, so there's plenty of shit in there already.
blooddragon · 33m ago
Time for GANs to make a resurgence?
abc-1 · 13m ago
They’ve been spouting this for years. I have yet to see an actual practitioner who’s job it is to collect data for these LLMs say it’s actually an issue.
They’ve been spouting it for years because it’s an entertaining thought and scratches an itch to many different reader bases. That doesn’t make it true.
sidibe · 10m ago
Yeah I think it's because people want it to be true that LLMs will stop improving and regress. If nothing else they can always just access data from the before times if it was an actual issue
So much burying the head in the sand from people in this industry and wishful thinking that AI will stop before whatever they're good at. A little reminder a couple years ago most people hadn't even heard of LLMs.
_se · 1m ago
Most of us wish we could stop hearing about LLMs. A little reminder that every year since 2022 has been the last year that anyone will ever do anything by hand according to you prophets.
I'll give it 3-4 years before it's gone the way of crypto: still exists, still used (and still useful in the right cases), not something most people are talking about.
Instead we may see (and may already be seeing) an increasing slowdown of progress as good training material becomes more difficult to source. That means longer time between releases and less obvious improvement with each release.
[0] Like statistics of words, etc.
I noticed the Llama 4 system prompt has a whole paragraph devoted to suppressing various GPT-isms, implying they weren't able to scrub GPT output from the training set.
I'm not sure what they'd get from training on that
I had a very basic React question about useState while porting some vanilla code last week which all models of all stripes I've tried it on have been confidently and completely incorrect about, up to stating the code absolutely will not work, even when I take a turn to assert that I ran it and it does, so there's plenty of shit in there already.
They’ve been spouting it for years because it’s an entertaining thought and scratches an itch to many different reader bases. That doesn’t make it true.
So much burying the head in the sand from people in this industry and wishful thinking that AI will stop before whatever they're good at. A little reminder a couple years ago most people hadn't even heard of LLMs.
I'll give it 3-4 years before it's gone the way of crypto: still exists, still used (and still useful in the right cases), not something most people are talking about.
It's going to be really nice.