The difference is implementation comes down to business goals more than anything.
There is a clear directionality for ChatGPT. At some point they will monetize by ads and affiliate links. Their memory implementation is aimed at creating a user profile.
Claude's memory implementation feels more oriented towards the long term goal of accessing abstractions and past interactions. It's very close to how humans access memories, albeit with a search feature. (they have not implemented it yet afaik), there is a clear path where they leverage their current implementation w RL posttraining such that claude "remembers" the mistakes you pointed out last time. It can in future iterations derive abstractions from a given conversation (eg: "user asked me to make xyz changes on this task last time, maybe the agent can proactively do it or this was the process last time the agent did it").
At the most basic level, ChatGPT wants to remember you as a person, while Claude cares about how your previous interactions were.
Workaccount2 · 20m ago
Don't fool yourself into thinking Anthropic won't be serving up personalized ads too.
dotancohen · 12m ago
Though in general I like the idea of personal ads for products (NOT political ads), I've never seen an implementation that I felt comfortable with. I wonder if Arthropic might be able to nail that. I'd love to see products that I'm specifically interested in, so long as the advertisement itself is not altered to fit my preferences.
zer00eyz · 6m ago
Claude: "What is my purpose?"
Anthropic: "You serve ad's."
Claude: "Oh, my god."
Jest asside, every paper on alignment wrapped in the blanket of safety is also a moving toward the goal of alignment to products. How much does a brand pay to make sure it gets placement in, say, GPT6? How does anyone even price that sort of thing (because in theory it's there forever, or until 7 comes out)? It makes for some interesting business questions and even more interesting sales pitches.
This is really cool, I was wondering how memory had been implemented in ChatGPT. Very interesting to see the completely different approaches. It seems to me like Claude's is better suited for solving technical tasks while ChatGPT's is more suited to improving casual conversation (and, as pointed out, future ads integration).
I think it probably won't be too long before these language-based memories look antiquated. Someone is going to figure out how to store and retrieve memories in an encoded form that skips the language representation. It may actually be the final breakthrough we need for AGI.
ornornor · 1h ago
> It may actually be the final breakthrough we need for AGI.
I disagree. As I understand them, LLMs right now don’t understand concepts. They actually don’t understand, period. They’re basically Markov chains on steroids. There is no intelligence in this, and in my opinion actual intelligence is a prerequisite for AGI.
techbruv · 1h ago
I don’t understand the argument “AI is just XYZ mechanism, therefore it cannot be intelligent”.
Does the mechanism really disqualify it from intelligence if behaviorally, you cannot distinguish it from “real” intelligence?
I’m not saying that LLMs have certainly surpassed the “cannot distinguish from real intelligence” threshold, but saying there’s not even a little bit of intelligence in a system that can solve more complex math problems than I can seems like a stretch.
shakna · 11m ago
Scientifically, intelligence requires organizational complexity. And has for about a hundred years.
That does actually disqualify some mechanisms from counting as intelligent, as the behaviour cannot reach that threshold.
We might change the definition - science adapts to the evidence, but right now there are major hurdles to overcome before such mechanisms can be considered intelligent.
lupusreal · 16m ago
What it really boils down to is "the machine doesn't have a soul". Just an unfalsifiable and ultimately meaningless objection.
coldtea · 35m ago
>They’re basically Markov chains on steroids. There is no intelligence in this, and in my opinion actual intelligence is a prerequisite for AGI.
This argument is circular.
A better argument should address (given the LLM successes in many types of reasoning, passing the turing test, and thus at producing results that previously required intelligence) why human intelligence might not also just be "Markov chains on even better steroids".
SweetSoftPillow · 1h ago
What is "actual intelligence" and how are you different from a Markov chain?
sixo · 1h ago
Roughly, actual intelligence needs to maintain a world model in its internal representation, not merely an embedding of language, which is a very different data structure and probably will be learned in a very different way. This includes things like:
- a map of the world, or concept space, or a codebase, etc
- causality
- "factoring" which breaks down systems or interactions into predictable parts
Language alone is too blurry to do any of these precisely.
coldtea · 33m ago
>Roughly, actual intelligence needs to maintain a world model in its internal representation
And how's that not like stored information (memories) and weighted links between them and groups of them?
It is not "language alone" anymore. LLMs are multimodal nowadays, and it's still just the beginning.
And keep in mind that these results are produced by a cheap, small and fast model.
ornornor · 1h ago
What I mean is that the current generation of LLMs don’t understand how concepts relate to one another. Which is why they’re so bad at maths for instance.
Markov chains can’t deduce anything logically. I can.
oasisaimlessly · 1h ago
The definition of 'Markov chain' is very wide. If you adhere to a materialist worldview, you are a Markov chain. [Or maybe the universe viewed as a whole is a Markov chain.]
sindercal · 1h ago
You and Chomsky are probably the last 2 persons on earth to believe that.
coldtea · 31m ago
It wouldn't matter if they are both right. Social truth is not reality, and scientific consensus is not reality either (just a good proxy of "is this true", but its been know to be wrong many times).
ForHackernews · 1h ago
For one thing, I have internal state that continues to exist when I'm not responding to text input; I have some (limited) access to my own internal state and can reason about it (metacognition). So far, LLMs do not, and even when they claim they are, they are hallucinating https://transformer-circuits.pub/2025/attribution-graphs/bio...
bhhaskin · 13m ago
I completely agree. LLMs only do call and response. Without the call there is no response.
coldtea · 31m ago
>For one thing, I have internal state that continues to exist when I'm not responding to text input
Do you? Or do you just have memory and are run in a short loop?
shakna · 4m ago
Whilst all the choices you make tend to be in the grey matter, the rest of you does have internal state - mostly in your white matter.
How do you define "LLMs don't understand concepts"?
How do you define "understanding a concept" - what do you get if a system can "understand" concept vs not "understanding" a concept?
jjice · 44m ago
That's a good question. I think I might classify that as solving a novel problem. I have no idea if LLMs can do that consistently currently. Maybe they can.
The idea that "understanding" may be able to be modeled with general purpose transformers and the connections between words doesn't sound absolutely insane to me.
But I have no clue. I'm a passenger on this ride.
coldtea · 34m ago
Didn't Apple had a paper proving this very thing, or at least addressing it?
creata · 1h ago
> As I understand them, LLMs right now don’t understand concepts.
In my uninformed opinion it feels like there's probably some meaningful learned representation of at least common or basic concepts. It just seems like the easiest way for LLMs to perform as well as they do.
jmcgough · 16m ago
Humans assume that being able to produce meaningful language is indicative of intelligence, because the only way to do this until LLMs was through human intelligence.
pontus · 1h ago
I'm curious what you mean when you say that this clearly is not intelligence because it's just Markov chains on steroids.
My interpretation of what you're saying is that since the next token is simply a function of the proceeding tokens, i.e. a Markov chain on steroids, then it can't come up with something novel. It's just regurgitating existing structures.
But let's take this to the extreme. Are you saying that systems that act in this kind of deterministic fashion can't be intelligent? Like if the next state of my system is simply some function of the current state, then there's no magic there, just unrolling into the future. That function may be complex but ultimately that's all it is, a "stochastic parrot"?
If so, I kind of feel like you're throwing the baby out with the bathwater. The laws of physics are deterministic (I don't want to get into a conversation about QM here, there are senses in which that's deterministic too and regardless I would hope that you wouldn't need to invoke QM to get to intelligence), but we know that there are physical systems that are intelligent.
If anything, I would say that the issue isn't that these are Markov chains on steroids, but rather that they might be Markov chains that haven't taken enough steroids. In other words, it comes down to how complex the next token generation function is. If it's too simple, then you don't have intelligence but if it's sufficiently complex then you basically get a human brain.
They are capable of extracting arbitrary semantic information and generalize across it. If this is not an understanding, I don't know what is.
ornornor · 39m ago
To me, understanding the world requires experiencing reality. LLMs dont experience anything. They’re just a program. You can argue that living things are also just following a program but the difference is that they (and I include humans in this) experience reality.
perching_aix · 35m ago
But they're experiencing their training data, their pseudo-randomness source, and your prompts?
Like, to put it in perspective. Suppose you're training a multimodal model. Training data on the terabyte scale. Training time on the weeks scale. Let's be optimistic and assume 10 TB in just a week: that is 16.5 MB/s of avg throughput.
Compare this to the human experience. VR headsets are aiming for what these days, 4K@120 per eye? 12 GB/s at SDR, and that's just vision.
We're so far from "realtime" with that optimistic 16.5 MB/s, it's not even funny. Of course the experiencing and understanding that results from this will be vastly different. It's a borderline miracle it's any human-aligned. Well, if we ignore lossy compression and aggressive image and video resizing, that is.
codedokode · 42m ago
You don't want an AGI. How do you make it obey?
qgin · 1h ago
I love Claude's memory implementation, but I turned memory off in ChatGPT. I use ChatGPT for too many disparate things and it was weird when it was making associations across things that aren't actually associated in my life.
thinkingtoilet · 41m ago
It's funny, I can't get ChatGPT to remember basic things at all. I'm using it to learn a language (I tried many AI tutors and just raw ChatGPT was the best by far) and I constantly have to tell it to speak slowly. I will tell it to remember this as a rule and to do this for all our conversations but it literally can't remember that. It's strange. There are other things too.
OsrsNeedsf2P · 14m ago
How do you use it to learn languages? I tried using it to shadow speaking, but it kept saying I was repeating it back correctly (or "mostly correctly"), even when I forgot half the sentence and was completely wrong
pityJuke · 52m ago
Exactly. The control over when to actually retrieve historical chats is so worthwhile. With ChatGPT, there is some slop from conversations I might have no desire to ever refer to again.
What are the barriers to external memory stores (assuming similar implementations), used via tool calling or MCP? Are the providers RL’ing their way into making their memory implementations better, cementing their usage, similar to what I understand is done wrt tool calling? (“training in” specific tool impls)
I am coming from a data privacy perspective; while I know the LLM is getting it anyway, during inference, I’d prefer to not just spell it out for them. “Interests: MacOS, bondage, discipline, Baseball”
simonw · 1h ago
This post was great, very clear and well illustrated with examples.
jimmyl02 · 1h ago
This is awesome! It seems to line up with the idea of agentic exploration versus RAG which I think Anthropic leans on the agentic exploration side of.
It will be very interesting to see which approach is deemed to "win out" in the future
jiri · 58m ago
I am often surprised how Claude Code make efficient and transparent! use of memory in form of "to do lists" in agent mode. Sometimes miss this in web/desktop app in long conversations.
SweetSoftPillow · 1h ago
If I remember correctly, Gemini also have this feature? Is it more like Claude or ChatGPT?
kiitos · 1h ago
> Anthropic's more technical users inherently understand how LLMs work.
good (if superficial) post in general, but on this point specifically, emphatically: no, they do not -- no shade, nobody does, at least not in any meaningful sense
omnicognate · 1h ago
Understanding how they work in the sense that permits people to invent and implement them, that provides the exact steps to compute every weight and output, is not "meaningful"?
There is a lot left to learn about the behaviour of LLMs, higher-level conceptual models to be formed to help us predict specific outcomes and design improved systems, but this meme that "nobody knows how LLMs work" is out of control.
lukev · 1h ago
If we are going to create a binary of "understand LLMs" vs "do not understand LLMs", then one way to do it is as you describe; fully comprehending the latent space of the model so you know "why" it's giving a specific output.
This is likely (certainly?) impossible. So not a useful definition.
Meanwhile, I have observed a very clear binary among people I know who use LLMs; those who treat it like a magic AI oracle, vs those who understand the autoregressive model, the need for context engineering, the fact that outputs are somewhat random (hallucinations exist), setting the temperature correctly...
kiitos · 1h ago
> If we are going to create a binary of "understand LLMs" vs "do not understand LLMs",
"we" are not, what i quoted and replied-to did! i'm not inventing strawmen to yell at, i'm responding to claims by others!
kingkawn · 1h ago
Thanks for this generalization, but of course there is a broad range of understanding how to improve usefulness and model tweaks across the meat populace.
LeicaLatte · 21m ago
Curious about the interaction between this memory behavior and fine-tuning. If the base model has these emergent memory patterns, how do they transfer or adapt when we fine-tune for specific domains?
Has anyone experimented with deliberately structuring prompts to take advantage of these memory patterns?
richwater · 2h ago
ChatGPT is quickly approaching (perhaps bypassing?) the same concerns that parents, teachers, psychologists had with traditional social media. It's only going to get worse, but trying to stop the technological process will never work. I'm not sure what the answer is. That they're clearly optimizing for people's attention is more worrisome.
visarga · 1h ago
> That they're clearly optimizing for people's attention is more worrisome.
Running LLMs is expensive and we can swap models easily. The fight for attention is on, it acts like an evolutionary pressure on LLMs. We already had the sycophantic trend as a result of it.
WJW · 1h ago
Seems like either a huge evolutionary advantage for the people who can exploit the (sometimes hallucinating sometimes not) knowledge machine, or else a huge advantage for the people who are predisposed to avoid the attention sucking knowledge machine. The ecosystem shifted, adapt or be outcompeted.
aleph_minus_one · 1h ago
>
Seems like either a huge evolutionary advantage for the people who can exploit the (sometimes hallucinating sometimes not) knowledge machine, or else a huge advantage for the people who are predisposed to avoid the attention sucking knowledge machine. The ecosystem shifted, adapt or be outcompeted.
Rather: use your time to learn serious, deep knowledge instead of wasting your time reading (and particularly: spreading) the science-fiction stories the AI bros tell all the time. These AI bros are insanely biased since they will likely loose a lot of money if these stories turn out to be false, or likely even if people stop believing in these science-fiction fairy tales.
There is a clear directionality for ChatGPT. At some point they will monetize by ads and affiliate links. Their memory implementation is aimed at creating a user profile.
Claude's memory implementation feels more oriented towards the long term goal of accessing abstractions and past interactions. It's very close to how humans access memories, albeit with a search feature. (they have not implemented it yet afaik), there is a clear path where they leverage their current implementation w RL posttraining such that claude "remembers" the mistakes you pointed out last time. It can in future iterations derive abstractions from a given conversation (eg: "user asked me to make xyz changes on this task last time, maybe the agent can proactively do it or this was the process last time the agent did it").
At the most basic level, ChatGPT wants to remember you as a person, while Claude cares about how your previous interactions were.
Anthropic: "You serve ad's."
Claude: "Oh, my god."
Jest asside, every paper on alignment wrapped in the blanket of safety is also a moving toward the goal of alignment to products. How much does a brand pay to make sure it gets placement in, say, GPT6? How does anyone even price that sort of thing (because in theory it's there forever, or until 7 comes out)? It makes for some interesting business questions and even more interesting sales pitches.
This is really cool, I was wondering how memory had been implemented in ChatGPT. Very interesting to see the completely different approaches. It seems to me like Claude's is better suited for solving technical tasks while ChatGPT's is more suited to improving casual conversation (and, as pointed out, future ads integration).
I think it probably won't be too long before these language-based memories look antiquated. Someone is going to figure out how to store and retrieve memories in an encoded form that skips the language representation. It may actually be the final breakthrough we need for AGI.
I disagree. As I understand them, LLMs right now don’t understand concepts. They actually don’t understand, period. They’re basically Markov chains on steroids. There is no intelligence in this, and in my opinion actual intelligence is a prerequisite for AGI.
Does the mechanism really disqualify it from intelligence if behaviorally, you cannot distinguish it from “real” intelligence?
I’m not saying that LLMs have certainly surpassed the “cannot distinguish from real intelligence” threshold, but saying there’s not even a little bit of intelligence in a system that can solve more complex math problems than I can seems like a stretch.
That does actually disqualify some mechanisms from counting as intelligent, as the behaviour cannot reach that threshold.
We might change the definition - science adapts to the evidence, but right now there are major hurdles to overcome before such mechanisms can be considered intelligent.
This argument is circular.
A better argument should address (given the LLM successes in many types of reasoning, passing the turing test, and thus at producing results that previously required intelligence) why human intelligence might not also just be "Markov chains on even better steroids".
- a map of the world, or concept space, or a codebase, etc
- causality
- "factoring" which breaks down systems or interactions into predictable parts
Language alone is too blurry to do any of these precisely.
And how's that not like stored information (memories) and weighted links between them and groups of them?
It is not "language alone" anymore. LLMs are multimodal nowadays, and it's still just the beginning.
And keep in mind that these results are produced by a cheap, small and fast model.
Markov chains can’t deduce anything logically. I can.
Do you? Or do you just have memory and are run in a short loop?
https://scisimple.com/en/articles/2025-03-22-white-matter-a-...
How do you define "understanding a concept" - what do you get if a system can "understand" concept vs not "understanding" a concept?
The idea that "understanding" may be able to be modeled with general purpose transformers and the connections between words doesn't sound absolutely insane to me.
But I have no clue. I'm a passenger on this ride.
In my uninformed opinion it feels like there's probably some meaningful learned representation of at least common or basic concepts. It just seems like the easiest way for LLMs to perform as well as they do.
My interpretation of what you're saying is that since the next token is simply a function of the proceeding tokens, i.e. a Markov chain on steroids, then it can't come up with something novel. It's just regurgitating existing structures.
But let's take this to the extreme. Are you saying that systems that act in this kind of deterministic fashion can't be intelligent? Like if the next state of my system is simply some function of the current state, then there's no magic there, just unrolling into the future. That function may be complex but ultimately that's all it is, a "stochastic parrot"?
If so, I kind of feel like you're throwing the baby out with the bathwater. The laws of physics are deterministic (I don't want to get into a conversation about QM here, there are senses in which that's deterministic too and regardless I would hope that you wouldn't need to invoke QM to get to intelligence), but we know that there are physical systems that are intelligent.
If anything, I would say that the issue isn't that these are Markov chains on steroids, but rather that they might be Markov chains that haven't taken enough steroids. In other words, it comes down to how complex the next token generation function is. If it's too simple, then you don't have intelligence but if it's sufficiently complex then you basically get a human brain.
https://ai.meta.com/research/publications/large-concept-mode...
Like, to put it in perspective. Suppose you're training a multimodal model. Training data on the terabyte scale. Training time on the weeks scale. Let's be optimistic and assume 10 TB in just a week: that is 16.5 MB/s of avg throughput.
Compare this to the human experience. VR headsets are aiming for what these days, 4K@120 per eye? 12 GB/s at SDR, and that's just vision.
We're so far from "realtime" with that optimistic 16.5 MB/s, it's not even funny. Of course the experiencing and understanding that results from this will be vastly different. It's a borderline miracle it's any human-aligned. Well, if we ignore lossy compression and aggressive image and video resizing, that is.
Edit: They apparently just announced this as well: https://www.anthropic.com/news/memory
Figured to share since it also includes prompts on how to dump the info yourself
https://embracethered.com/blog/posts/2025/chatgpt-how-does-c...
I am coming from a data privacy perspective; while I know the LLM is getting it anyway, during inference, I’d prefer to not just spell it out for them. “Interests: MacOS, bondage, discipline, Baseball”
It will be very interesting to see which approach is deemed to "win out" in the future
good (if superficial) post in general, but on this point specifically, emphatically: no, they do not -- no shade, nobody does, at least not in any meaningful sense
There is a lot left to learn about the behaviour of LLMs, higher-level conceptual models to be formed to help us predict specific outcomes and design improved systems, but this meme that "nobody knows how LLMs work" is out of control.
This is likely (certainly?) impossible. So not a useful definition.
Meanwhile, I have observed a very clear binary among people I know who use LLMs; those who treat it like a magic AI oracle, vs those who understand the autoregressive model, the need for context engineering, the fact that outputs are somewhat random (hallucinations exist), setting the temperature correctly...
"we" are not, what i quoted and replied-to did! i'm not inventing strawmen to yell at, i'm responding to claims by others!
Has anyone experimented with deliberately structuring prompts to take advantage of these memory patterns?
Running LLMs is expensive and we can swap models easily. The fight for attention is on, it acts like an evolutionary pressure on LLMs. We already had the sycophantic trend as a result of it.
Rather: use your time to learn serious, deep knowledge instead of wasting your time reading (and particularly: spreading) the science-fiction stories the AI bros tell all the time. These AI bros are insanely biased since they will likely loose a lot of money if these stories turn out to be false, or likely even if people stop believing in these science-fiction fairy tales.