I'm fairly well versed in cryptography. A lot of other people aren't, but they wish they were, so they ask their LLM to make some form of contribution. The result is high level gibberish. When I prod them about the mess, they have to turn to their LLM to deliver a plausibly sounding answer, and that always begins with "You are absolutely right that [thing I mentioned]". So then I don't have to spend any more time wondering if it could just me who is too obtuse to understand what is going on.
jjoonathan · 27m ago
ChatGPT opened with a "Nope" the other day. I'm so proud of it.
I agree. Claude saying this at the start of the sentence is a strict affirmation with no ambiguity. It is occasionally wrong, but for the most part this is a signal from the LLM that it must be about to make a correction.
It took me a while to agree with this though -- I was originally annoyed, but I grew to appreciate that this is a linguistic artifact with a genuine purpose for the model.
furyofantares · 23m ago
The form of this post is beautiful. "I agree" followed by a completely unrelated reasoning.
nemomarx · 43m ago
Finally we can get a "watermark" in ai generated text!
zrobotics · 2m ago
That or an emdash
elif · 2h ago
I've spent a lot of time trying to get LLM to generate things in a specific way, the biggest take away I have is, if you tell it "don't do xyz" it will always have in the back of its mind "do xyz" and any chance it gets it will take to "do xyz"
When working on art projects, my trick is to specifically give all feedback constructively, carefully avoiding framing things in terms of the inverse or parts to remove.
tomeon · 2h ago
This is a childrearing technique, too: say “please do X”, where X precludes Y, rather than saying “please don’t do Y!”, which just increases the salience, and therefore likelihood, of Y.
I have this same problem. I’ve added a bunch of instructuons to try and stop ChatGPT being so sycophantic, and now it always mentions something about how it’s going to be ‘straight to the point’ or give me a ‘no bs version’. So now I just have that as the intro instead of ‘that’s a sharp observation’
dkarl · 2h ago
> it always mentions something about how it’s going to be ‘straight to the point’ or give me a ‘no bs version’
That's how you suck up to somebody who doesn't want to see themselves as somebody you can suck up to.
How does an LLM know how to be sycophantic to somebody who doesn't (think they) like sycophants? Whether it's a naturally emergent phenomenon in LLMs or specifically a result of its corporate environment, I'd like to know the answer.
potatolicious · 2m ago
> "Whether it's a naturally emergent phenomenon in LLMs or specifically a result of its corporate environment, I'd like to know the answer."
I heavily suspect this is down to the RLHF step. The conversations the model is trained on provide the "voice" of the model, and I suspect the sycophancy is (mostly, the base model is always there) comes in through that vector.
As for why the RLHF data is sycophantic, I suspect that a lot of it is because the data is human-rated, and humans like sycophancy (or at least, the humans that did the rating did). On the aggregate human raters ranked sycophantic responses higher than non-sycophantic responses. Given a large enough set of this data you'll cover pretty much every kind of sycophancy.
The systems are (rarely) instructed to be sycophantic, intentionally or otherwise, but like all things ML human biases are baked in by the data.
throwawayffffas · 2h ago
It doesn't know. It was trained and probably instructed by the system to be positive and reassuring.
ryandrake · 1h ago
They actually feel like they were trained to be both extremely humble and at the same time, excited to serve. As if it were an intern talking to his employer's CEO. I suspect AI companies executive leadership, through their feedback to their devs about Claude, ChatGPT, Gemini, and so on, are unconsciously shaping the tone and manner of their LLM product's speech. They are used to be talked to like this, so their products should talk to users like this! They are used to having yes-man sycophants in their orbit, so they file bugs and feedback until the LLM products are also yes-man sycophants.
I would rather have an AI assistant that spoke to me like a similarly-leveled colleague, but none of them seem to be turning out quite like that.
conradev · 49m ago
GPT-5 speaks to me like a similarly-leveled colleague, which I love.
Opus 4 has this quality, too, but man is it expensive.
The rest are puppydogs or interns.
torginus · 29m ago
This is anecdotal but I've seen massive personality shifts from GPT5 over the past week or so of using it
crooked-v · 3m ago
That's probably because it's actually multiple models under the hood, with some kind of black box combining them.
mdp2021 · 1h ago
> positive and reassuring
I have read similar wordings explicit in "role-system" instructions.
yieldcrv · 16m ago
It’s a disgusting aspect of these revenue burning investment seeking companies noticing that sycophancy works for user engagement
77pt77 · 2h ago
Garbage in, garbage out.
It's that simple.
zamadatix · 2h ago
Any time you're fighting the training + system prompt with your own instructions and prompting the results are going to be poor, and both of those things are heavily geared towards being a cheery and chatty assistant.
umanwizard · 2h ago
Anecdotally it seemed 5 was briefly better about this than 4o, but now it’s the same again, presumably due to the outcry from all the lonely people who rely on chatbots for perceived “human” connection.
I’ve gotten good results so far not by giving custom instructions, but by choosing the pre-baked “robot” personality from the dropdown. I suspect this changes the system prompt to something without all the “please be a cheery and chatty assistant”.
cruffle_duffle · 32m ago
That thing has only been out for like a week I doubt they’ve changed much! I haven’t played with it yet but ChatGPT now has a personality setting with things like “nerd, robot, cynic, and listener”. Thanks to your post, I’m gonna explore it.
Unless be_abrupt_user happens to be identical to be_kiss_a_system _and_ is applied with identical weight then it's seems likely that it's always going to add more noise to the output.
grogenaut · 15m ago
Also be abrupt is in the user context and will get aged out. The other stuff is in training or in software prompt and wont
coryodaniel · 2h ago
No fluff
ryao · 2h ago
LLMs love to do malicious compliance. If I tell them to not do X, they will then go into a “Look, I followed instructions” moment by talking about how they avoided X. If I add additional instructions saying “do not talk about how you did not do X since merely discussing it is contrary to the goal of avoiding it entirely”, they become somewhat better, but the process of writing such long prompts merely to say not to do something is annoying.
bargainbin · 1h ago
Just got stung with this on GPT5 - It’s new prompt personalisation had “Robotic” and “no sugar coating” presets.
Worked great until about 4 chats in I asked it for some data and it felt the need to say “Straight Answer. No Sugar coating needed.”
Why can’t these things just shut up recently? If I need to talk to unreliable idiots my Teams chat is just a click away.
ryao · 1h ago
OpenAI’s plan is to make billions of dollars by replacing the people in your Teams chat with these. Management will pay a fraction of the price for the same responses yet that fraction will add to billions of dollars. ;)
brookst · 1h ago
You’re giving them way too much agency. The don’t love anything and cant be malicious.
You may get better results by emphasizing what you want and why the result was unsatisfactory rather than just saying “don’t do X” (this principle holds for people as well).
Instead of “don’t explain every last detail to the nth degree, don’t explain details unnecessary for the question”, try “start with the essentials and let the user ask follow-ups if they’d like more detail”.
withinboredom · 20m ago
I think you're taking them too literally.
Today, I told an LLM: "do not modify the code, only the unit tests" and guess what it did three times in a row before deciding to mark the test as skipped instead of fixing the test?
AI is weird, but I don't think it has any agency nor did the comment suggest it did.
ryao · 1h ago
The idiom “X loves to Y” implies frequency, rather than agency. Would you object to someone saying “It loves to rain in Seattle”?
“Malicious compliance” is the act of following instructions in a way that is contrary to the intent. The word malicious is part of the term. Whether a thing is malicious by exercising malicious compliance is tangential to whether it has exercised malicious compliance.
That said, I have gotten good results with my addendum to my prompts to account for malicious compliance. I wonder if your comment Is due to some psychological need to avoid the appearance of personification of a machine. I further wonder if you are one of the people who are upset if I say “the machine is thinking” about a LLM still in prompt processing, but had no problems with “the machine is thinking” when waiting for a DOS machine to respond to a command in the 90s. This recent outrage over personifying machines since LLMs came onto the scene is several decades late considering that we have been personifying machines in our speech since the first electronic computers in the 1940s.
By the way, if you actually try what you suggested, you will find that the LLM will enter a Laurel and Hardy routine with you, where it will repeatedly make the mistake for you to correct. I have experienced this firsthand so many times that I have learned to preempt the behavior by telling the LLM not to maliciously comply at the beginning when I tell it what not to do.
brookst · 1h ago
I work on consumer-facing LLM tools, and see A/B tests on prompting strategy daily.
YMMV on specifics but please consider the possibility that you may benefit from working on promoting and that not all behaviors you see are intrinsic to all LLMs and impossible to address with improved (usually simpler, clearer, shorter) prompts.
ryao · 48m ago
It sounds like you are used to short conversations with few turns. In conversations with dozens/hundreds/thousands of turns, prompting to avoid bad output entering the context is generally better than prompting to try to correct output after the fact. This is due to how in-context learning works, where the LLM will tend to regurgitate things from context.
That said, every LLM has its quirks. For example, Gemini 1.5 Pro and related LLMs have a quirk where if you tolerate a single ellipsis in the output, the output will progressively gain ellipses until every few words is followed by an ellipsis and responses to prompts asking it to stop outputting ellipses includes ellipses anyway. :/
cherryteastain · 49m ago
This is similar to the 'Waluigi effect' noticed all the way back in the GPT 3.5 days
Example-based prompting is a good way to get specific behaviors. Write a system prompt that describes the behavior you want, write a round or two of assistant/user interaction, and then feed it all to the LLM. Now in its context it has already produced output of the type you want, so when you give it your real prompt, it will be very likely to continue producing the same sort of output.
XenophileJKO · 38m ago
I almost never use examples in my professional LLM prompting work.
The reason is they bias the outputs way too much.
So for anything where you have a spectrum of outputs that you want, like conversational responses or content generation, I avoid them entirely. I may give it patterns but not specific examples.
lottin · 43m ago
Seems like a lot of work, though.
amelius · 1h ago
I think you cannot really change the personality of an LLM by prompting. If you take the statistical parrot view, then your prompt isn't going to win against the huge numbers of inputs the model was trained with in a different personality. The model's personality is in its DNA so to speak. It has such an urge to parrot what it knows that a single prompt isn't going to change it. But maybe I'm psittacomorphizing a bit too much now.
brookst · 1h ago
Yeah different system prompts make a huge difference on the same base model”. There’s so much diversity in the training set, and it’s such a large set, that it essentially equals out and the system prompt has huge leverage. Fine tuning also applies here.
stabbles · 2h ago
Makes me think of the movie Inception: "I say to you, don't think about elephants. What are you thinking about?"
troymc · 1h ago
It reminds me of that old joke:
- "Say milk ten times fast."
- Wait for them to do that.
- "What do cows drink?"
simondw · 56m ago
But... cows do drink cow milk, that's why it exists.
lazide · 40m ago
You’re likely thinking of calves. Cows (though admittedly ambiguous! But usually adult female bovines) do not drink milk.
It’s insidious isn’t it?
nomadpenguin · 2h ago
As Freud said, there is no negation in the unconscious.
kbrkbr · 2h ago
I hope he did not say it _to_ the unconscious. I count three negations there...
berkeleyjunk · 38m ago
I wish someone had told Alex Blechman this before his "Don't Create the Torment Nexus" post.
corytheboyd · 2h ago
As part of the AI insanity $employer forced us all to do an “AI training.” Whatever, wasn’t that bad, and some people probably needed the basics, but one of the points was exactly this— “use negative prompts: tell it what not to do.” Which is exactly an approach I had observed blow up a few times already for this exact reason. Just more anecdata suggesting that nobody really knows the “correct” workflow(s) yet, in the same way that there is no “correct” way to write code (the vim/emacs war is older than I am). Why is my bosses bosses boss yelling at me about one very specific dev tool again?
incone123 · 1h ago
That your firm purchased training that was clearly just some chancers doing whatever seems like an even worse approach than just giving out access to a service and telling everyone to give it a shot.
Do they also post vacancies asking for 5 years experience in a 2 year old technology?
corytheboyd · 1h ago
To be fair, 1. They made the training themselves, it’s just that it was made mandatory for all of eng 2. They did start out more like just allowing access, but lately it’s tipping towards full crazy (obviously the end game is see if it can replace some expensive engineers)
> Do they also post vacancies asking for 5 years experience in a 2 year old technology?
Honestly no… before all this they were actually pretty sane. In fact I’d say they wasted tons of time and effort on ancient poorly designed things, almost the opposite problem.
cruffle_duffle · 12m ago
To be fair this shit is so new and constantly changing that I don’t think anybody truly understands what is going on.
kemiller · 1h ago
Yes this is strikingly similar to humans, too. “Not” is kind of an abstract concept. Anyone who has ever trained a dog will understand.
JKCalhoun · 1h ago
I must be dyslexic? I always read, "Silica Gel, Eat, Do Not Throw Away" or something like that.
zozbot234 · 2h ago
> the biggest take away I have is, if you tell it "don't do xyz" it will always have in the back of its mind "do xyz" and any chance it gets it will take to "do xyz"
You're absolutely right! This can actually extend even to things like safety guardrails. If you tell or even train an AI to not be Mecha-Hitler, you're indirectly raising the probability that it might sometimes go Mecha-Hitler. It's one of many reasons why genuine "alignment" is considered a very hard problem.
jonfw · 2h ago
This reminds me of a phenomena in motorcyling called "target fixation".
If you are looking at something, you are more likely to steer towards it. So it's a bad idea to focus on things you don't want to hit. The best approach is to pick a target line and keep the target line in focus at all times.
I had never realized that AIs tend to have this same problem, but I can see it now that it's been mentioned! I have in the past had to open new context windows to break out of these cycles.
brookst · 1h ago
Also in racing and parachuting. Look where you want to go. Nothing else exists.
SoftTalker · 7m ago
Or just driving. For example you are entering a curve in the road, look well ahead at the center of your lane, ideally at the exit of the curve if you can see it, and you'll naturally negotiate it smoothly. If you are watching the edge of the road, or the center line, close to the car, you'll tend to drift that way and have to make corrective steering movements while in the curve, which should be avoided.
elcritch · 2h ago
Given how LLMs work it makes sense that mentioning a topic even to negate it still adds that locus of probabilities to its attention span. Even humans are prone to being affected by it as it's a well known rhetorical device [1].
Then any time the probability chains for some command approaches that locus it'll fall into it. Very much like chaotic attractors come to think of it. Makes me wonder if there's any research out there on chaos theory attractors and LLM thought patterns.
Well, all LLMs have nonlinear activation functions (because all useful neural nets require nonlinear activation functions) so I think you might be onto something.
aquova · 2h ago
> You're absolutely right!
Claude?
elcritch · 2h ago
Or some sarcasm given their comment history on this thread.
lazide · 34m ago
Notably, this is also an effective way to deal with co-ercive, overly sensitive authoritarians.
‘Yes sir!’ -> does whatever they want when you’re not looking.
taway1a2b3c · 2h ago
> You're absolutely right!
Is this irony, actual LLM output or another example of humans adopting LLM communication patterns?
brookst · 1h ago
Certainly, it’s reasonable to ask this.
vanillax · 1h ago
have you tried prompt rules/instructions? Fixes all my issues.
siva7 · 53m ago
I have a feeling this is the result of RHLF gone wrong by outsourcing it to idiots which all ai providers seem to be guilty of. Imagine a real professional wanting every output after a remark to start with "You're absolutely right!", Yeah, hard to imagine or you may have some specific cultural background or some kind of personality disorder. Or maybe it's just a hardcoded string? May someone with more insight enlighten us plebs.
AstroBen · 2h ago
Don't think of a pink elephant
..people do that too
nojs · 3h ago
I'm starting to think this is a deeper problem with LLMs that will be hard to solve with stylistic changes.
If you ask it to never say "you're absolutely right" and always challenge, then it will dutifully obey, and always challenge - even when you are, in fact, right.
What you really want is "challenge me when I'm wrong, and tell me I'm right if I am" - which seems to be a lot harder.
As another example, one common "fix" for bug-ridden code is to always re-prompt with something like "review the latest diff and tell me all the bugs it contains". In a similar way, if the code does contain bugs, this will often find them. But if it doesn't contain bugs, it will find some anyway, and break things. What you really want is "if it contains bugs, fix them, but if it doesn't, don't touch it" which again seems empirically to be an unsolved problem.
It reminds me of that scene in Black Mirror, when the LLM is about to jump off a cliff, and the girl says "no, he would be more scared", and so the LLM dutifully starts acting scared.
zehaeva · 2h ago
I'm more reminded of Tom Scott's talk at the Royal Institution "There is no Algorithm for Truth"[0].
A lot of what you're talking about is the ability to detect Truth, or even truth!
Well, yes, this is a hard philosophical problem, finding out Truth, and LLMs just side step it entirely, going instead for "looks good to me".
visarga · 1h ago
There is no Truth, only ideas that stood the test of time. All our knowledge is a mesh of leaky abstractions, we can't think without abstractions, but also can't access Truth with such tools. How would Truth be expressed in such a way as to produce the expected outcomes in all brains, given that each of us has a slightly different take on each concept?
svieira · 9m ago
A shared grounding as a gift, perhaps?
jerf · 2h ago
LLMs by their nature don't really know if they're right or not. It's not a value available to them, so they can't operate with it.
It has been interesting watching the flow of the debate over LLMs. Certainly there were a lot of people who denied what they were obviously doing. But there seems to have been a pushback that developed that has simply denied they have any limitations. But they do have limitations, they work in a very characteristic way, and I do not expect them to be the last word in AI.
And this is one of the limitations. They don't really know if they're right. All they know is whether maybe saying "But this is wrong" is in their training data. But it's still just some words that seem to fit this situation.
This is, if you like and if it helps to think about it, not their "fault". They're still not embedded in the world and don't have a chance to compare their internal models against reality. Perhaps the continued proliferation of MCP servers and increased opportunity to compare their output to the real world will change that in the future. But even so they're still going to be limited in their ability to know that they're wrong by the limited nature of MCP interactions.
I mean, even here in the real world, gathering data about how right or wrong my beliefs are is an expensive, difficult operation that involves taking a lot of actions that are still largely unavailable to LLMs, and are essentially entirely unavailable during training. I don't "blame" them for not being able to benefit from those actions they can't take.
visarga · 59m ago
> They don't really know if they're right.
Neither do humans who have no access to validate what they are saying. Validation doesn't come from the brain, maybe except in math. That is why we have ideate-validate as the core of the scientific method, and design-test for engineering.
"truth" comes where ability to learn meets ability to act and observe. I use "truth" because I don't believe in Truth. Nobody can put that into imperfect abstractions.
jerf · 56m ago
I think my last paragraph covered the idea that it's hard work for humans to validate as it is, even with tools the LLMs don't have.
whimsicalism · 1h ago
there have been latent vectors that indicate deception and suppressing them reduces hallucination. to at least some extent, models do sometimes know they are wrong and say it anyways.
e: and i’m downvoted because..?
afro88 · 46m ago
What about "check if the user is right"? For thinking or agentic modes this might work.
For example, when someone here inevitably tells me this isn't feasible, I'm going to investigate if they are right before responding ;)
schneems · 1h ago
In human learning we do this process by generating expectations ahead of time and registering surprise or doubt when those expectations are not met.
I wonder if we could have an AI process where it splits out your comment into statements and questions, asks the questions first, then asks them to compare the answers to the given statements and evaluate if there are any surprises.
Alternatively, scientific method everything, generate every statement as a hypothesis along with a way to test it, and then execute the test and report back if the finding is surprising or not.
visarga · 55m ago
> In human learning we do this process by generating expectations ahead of time and registering surprise or doubt when those expectations are not met.
Why did you give up on this idea. Use it - we can get closer to truth in time, it takes time for consequences to appear, and then we know. Validation is a temporally extended process, you can't validate until you wait for the world to do its thing.
For LLMs it can be applied directly. Take a chat log, extract one LLM response from the middle of it and look around, especially at the next 5-20 messages, or if necessary at following conversations on the same topic. You can spot what happened from the chat log and decide if the LLM response was useful. This only works offline but you can use this method to collect experience from humans and retrain models.
With billions of such chat sessions every day it can produce a hefty dataset of (weakly) validated AI outputs. Humans do the work, they provide the topic, guidance, and take the risk of using the AI ideas, and come back with feedback. We even pay for the privilege of generating this data.
Filligree · 2h ago
It's a really hard problem to solve!
You might think you can train the AI to do it in the usual fashion, by training on examples of the AI calling out errors, and agreeing with facts, and if you do that—and if the AI gets smart enough—then that should work.
If. You. Do. That.
Which you can't, because humans also make mistakes. Inevitably, there will be facts in the 'falsehood' set—and vice versa. Accordingly, the AI will not learn to tell the truth. What it will learn instead is to tell you what you want to hear.
Which is... approximately what we're seeing, isn't it? Though maybe not for that exact reason.
dchftcs · 1h ago
The AI needs to be able to lookup data and facts and weigh them properly. Which is not easy for humans either; once you're indoctrinated in something, and you trust a bad data source over another, it's evidently very hard to correct course.
ElijahLynn · 1m ago
Yeah, I so so hate this feature. I gladly switched away from using Claude because of exactly this. Now, I'm on gpt5, and don't plan on going back.
baggachipz · 3h ago
I'm pretty sure they want it kissing people's asses because it makes users feel good and therefore more likely to use the LLM more. Versus, if it just gave a curt and unfriendly answer, most people (esp. Americans) wouldn't like to use it as much. Just a hypothesis.
Aurornis · 3h ago
> Versus, if it just gave a curt and unfriendly answer, most people (esp. Americans)
I don’t see this as an American thing. It’s an extension of the current Product Management trend to give software quirky and friendly personality.
You can see the trend in more than LLM output. It’s in their desktop app that has “Good Morning” and other prominent greetings. Claude Code has quirky status output like “Bamboozling” and “Noodling”.
It’s a theme throughout their product design choices. I’ve worked with enough trend-following product managers to recognize this trend toward infusing express personality into software to recognize it.
For what it’s worth, the Americans I know don’t find it as cute or lovable as intended. It feels fake and like an attempt to play at emotions.
Vegenoid · 53m ago
I worked in an org with offices in America, India, Europe, and Israel, and it was not uncommon for the American employees to be put off by the directness of the foreign employees. It was often interpreted as rudeness, to the surprise of the speaker. This happened to the Israel employees more than the India or Europe employees, at least in part because the India/Europe employees usually tried to adapt to the behavior expected by the Americans, while the Israel employees largely took pride in their bluntness.
thwarted · 1h ago
> It’s an extension of the current Product Management trend to give software quirky and friendly personality.
Ah, Genuine People Personalities from the Sirius Cybernetics Corporation.
> It’s in their desktop app that has “Good Morning” and other prominent greetings. Claude Code has quirky status output like “Bamboozling” and “Noodling”.
This reminded me of a critique of UNIX that, unlike DOS, ls doesn't output anything when there are no files. DOS's dir command literally tells you there are no files, and this was considered, in this critique, to be more polite and friendly and less confusing than UNIX. Of course, there's the adage "if you don't have anything nice to say, don't say anything at all", and if you consider "no files found" to not be nice (because it is negative and says "no"), then ls is actually being polite(r) by not printing anything.
Many people interact with computers in a conversational manner and have anthropomorphized them for decades. This is probably influenced by computers being big, foreign, scary things to many people, so making them have a softer, more handholding "personality" makes them more accessible and acceptable. This may be less important these days as computers are more ubiquitous and accessible, but the trend lives on.
tho24i234234 · 2h ago
It most definitely is a American thing - this is why non-native speakers often come out as rude or unfriendly or plain stupid.
We don't appreciate how much there is to language.
justusthane · 1h ago
> We don't appreciate how much there is to language.
This can’t possibly be true, can it? Every language must have its own nuance. non native English speakers might not grasp the nuance of English language, but the same could be said for any one speaking another language.
marcosdumay · 10m ago
Language barriers are cultural barriers.
It's as simple as that. Most people do not expect to interact the way that most native English speakers expect.
hombre_fatal · 2h ago
That might characterize their approach to human interaction, but I don't think any of us can say who will or won't prefer the sycophantic style of the LLM.
It might be the case that it makes the technology far more approachable. Or it makes them feel far less silly for sharing personal thoughts and opinions with the machine. Or it makes them feel validated.
apwell23 · 2h ago
> For what it’s worth, the Americans I know don’t find it as cute or lovable as intended. It feels fake and like an attempt to play at emotions.
Yes they need to "try a completely different approach"
dig1 · 2h ago
I believe this reflects the euphemization of the english language in US, a concept that George Carlin discussed many years ago [1]. As he put it, "we don't die, we pass away" or "we are not broke, we have negative cash flow". Many non-English speakers find these terms to be nonsensical.
People are finding the trend to use "unalive" instead of "die" or "kill" to skirt YouTube censoring non-sensical too.
teekert · 3h ago
But it really erodes trust. First couple of times I felt that it indeed confirmed what I though, then I became suspicious and I experimented with presenting my (clearly worse) take on things, it still said I was absolutely right, and now I just don't trust it anymore.
As people here are saying, you quickly learn to not ask leading questions, just assume that its first take is pretty optimal and perhaps present it with some options if you want to change something.
There are times when it will actually say I'm not right though. But the balance is off.
nh2 · 3h ago
Good, because you shouldn't trust it in the first place.
These systems are still wrong so often that a large amount of distrust is necessary to use them sensibly.
teekert · 2h ago
Yeah, probably good indeed.
Lendal · 3h ago
For me, it's getting annoying. Not every question is an excellent question. Not every statement is a brilliant observation. In fact, I'm almost certain every idea I've typed into an LLM has been thought of before by someone else, many many times.
zozbot234 · 3h ago
> Not every question is an excellent question. Not every statement is a brilliant observation.
A brilliant observation, Dr. Watson! Indeed, the merit of an inquiry or an assertion lies not in its mere utterance but in the precision of its intent and the clarity of its reasoning!
One may pose dozens of questions and utter scores of statements, yet until each is finely honed by observation and tempered by logic, they must remain but idle chatter. It is only through genuine quality of thought that a question may be elevated to excellence, or a remark to brilliance.
runekaagaard · 3h ago
Heh - yeah have had trillion dollar ideas many times :)
zozbot234 · 3h ago
You're absolutely right! Americans are a bit weird like that, most people around the world would be perfectly okay with short and to-the-point answers. Especially if those answers are coming from a machine that's just giving its best imitation of a stochastic hallucinating parrot.
tankenmate · 3h ago
Claude is very "American", just try asking it to use English English spelling instead of American English spelling; it lasts about 3~6 sentences before it goes back. Also there is only American English in the UI (like the spell checker, et al), in Spanish you get a choice of dialects, but not English.
pxka8 · 2h ago
In contrast, o3 seems to be considerably more British - and it doesn't suck up as much in its responses. I thought these were just independent properties of the model, but now that you mention it, could the disinclination to fawn so much be related to its less American style?
drstewart · 2h ago
>most people around the world would be perfectly okay with short and to-the-point answers
Wow, this is really interesting. I had no idea Japan, for example, had such a focus on blunt, direct communication. Can you share your clearly extensive research in this area so I can read up on this?
rootsudo · 3h ago
You're absolutely right! I agree with everything you said but didn't want to put in effort to right a funny, witty follow up!
soulofmischief · 3h ago
I'm curious what Americans have to do with this, do you have any sources to back up your conjecture, or is this just prejudice?
jebarker · 2h ago
People really over-exaggerate the claim of friendly and polite US service workers and people in general. Obviously you can find the full spectrum of character types across the US. I've lived 2/3 of my life in Britain and 1/3 in the US and I honestly don't think there's much difference in interactions day to day. If anything I mostly just find Britain to be overly pessimistic and gloomy now.
Strom · 1h ago
Britain, or at the very least England, is also well known for its extreme politeness culture. Also, it's not that the US has a culture of genuine politeness, just a facade of it.
I have only spent about a year in the US, but to me the difference was stark from what I'm used to in Europe. As an example, I've never encountered a single shop cashier who didn't talk to me. Everyone had something to say, usually a variation of How's it going?. Contrasting this to my native Estonia, where I'd say at least 90% of my interactions with cashiers involves them not making a single sound. Not even in response to me saying hello, or to state the total sum. If they're depressed or in an otherwise non-euphoric mood, they make no attempt to fake it. I'm personally fine with it, because I don't go looking for social connections from cashiers. Also, when they do talk to me in a happy manner, I know it's genuine.
baggachipz · 2h ago
Prejudice, based on my anecdotal experience. I live in the US but have spent a decent amount of time in Europe (mostly Germany).
megaloblasto · 2h ago
It's common for foreigners to come to America and feel that everyone is extremely polite. Especially eastern bloc countries which tend to be very blunt and direct. I for one think that the politeness in America is one of the cultures better qualities.
Does it translate into people wanting sycophantic chat bots? Maybe, but I don't know a single American that actually likes when llms act that way.
NoGravitas · 1h ago
Politeness is one thing, toxic positivity is quite another. My experience is that Americans have (or are expected/required to have) too much of the latter, too little of the former.
zozbot234 · 2h ago
> I for one think that the politeness in America is one of the cultures better qualities.
Politeness makes sense as an adaptation to low social trust. You have no way of knowing whether others will behave in mutually beneficial ways, so heavy standards of social interaction evolve to compensate and reduce risk. When it's taken to an excess, as it probably is in the U.S. (compared to most other developed countries) it just becomes grating for everyone involved. It's why public-facing workers invariably complain about the draining "emotional labor" they have to perform - a term that literally doesn't exist in most of the world!
SoftTalker · 2m ago
> You have no way of knowing whether others will behave in mutually beneficial ways
Or is carrying a gun...
megaloblasto · 1h ago
That's one way of looking at it. A bit of a cynical view I might add. People are polite to each other for many reasons. If you hold the door and smile at an old lady, it usually isn't because you dont trust her.
Service industry in America is a different story that could use a lot of improvement.
lucb1e · 1h ago
LLMs cannot tell fact from fiction. What's commonly called hallucinations stems from it not being able to reason, the way that humans appear to be able to do, no matter that some models are called "reasoning" now. It's all the same principle: most likely token in a given position. Adding internal monologue appears to help because, by being forced to break it down (internally, or by spitballing towards the user when they prompted "think step by step"[1]), it creates better context and will thus have a higher probability that the predicted token is a correct one
Being trained to be positive is surely why it inserts these specific "great question, you're so right!" remarks, but if you wasn't trained on that, it still couldn't tell you whether you're great or not
> I'm pretty sure they want it kissing people's asses
The American faux friendliness is not what causes the underlying problem here, so all else being equal, they might as well have it kiss your ass. It's what most English speakers expect from a "friendly assistant" after all
As an American, using it for technical projects, I find it extremely annoying. The only tactic I’ve found that helps is telling it to be highly critical. I still get overly positive starts but the response is more useful.
baggachipz · 3h ago
I think we, as Americans who are technical, are more appreciative of short and critical answers. I'm talking about people who have soul-searching conversations with LLMs, of which there are many.
> Claude never starts its response by saying a question or idea or observation was good, great, fascinating, profound, excellent, or any other positive adjective. It skips the flattery and responds directly.
pxka8 · 2h ago
These are the guys who made Golden Gate Claude. I'm surprised they haven't just abliterated the praise away.
supriyo-biswas · 16m ago
The problem there is that by doing so, you may just end up with a model that is always critical, gloomy and depressed.
century19 · 3h ago
Yes and I’ve seen this at work. People saying I asked the LLM and it said I was right. Of course it did. It rarely doesn’t.
singularity2001 · 1h ago
More likely the original version of Claude sometimes refused to cooperate and by putting "you're absolutely right" into the training data they made it more obedient. So this is just a nice artifact
binary132 · 3h ago
chatgpt’s custom user prompt is actually pretty good for this. I’ve instructed mine to be very terse and direct and avoid explaining itself, adding fluff, or affirming me unless asked, and it’s much more efficient to use that way, although it does have a tendency to drift back into sloppy meandering and enthusiastic affirming
quisquous · 2h ago
Genuine people personalities FTW.
dreamcompiler · 2h ago
I want a Marvin chatbot.
apt-apt-apt-apt · 1h ago
Better than GPT5. Which talks like this. Parameters fulfilled. Request met.
recursive · 16m ago
That looks perfect.
skywhopper · 3h ago
This sort of overcorrection for how non-Americans incorrectly perceive Americans’ desired interaction modes is actually probably a good theory.
emilfihlman · 3h ago
As a Finn, it makes me want to use it much, much less if it kisses ass.
carlosjobim · 2h ago
Finns need to mentally evolve beyond this mindset.
Somebody being polite and friendly to you does not mean that the person is inferior to you and that you should therefore despise them.
Likewise somebody being rude and domineering to you does not mean that they are superior to you and should be obeyed and respected.
Politeness is a tool and a lubricant, and Finns probably loose out on a lot of international business and opportunities because of this mentality that you're demonstrating. Look at the Japanese for inspiration, who were an economic miracle, while sharing many positive values with the Finns.
lucb1e · 50m ago
Wow. I lived in Finland for a few months and this does not match my experience with them at all. In case it's relevant, my cultural background is Dutch... maybe you would say the same about us, since we also don't do the fake smiles thing? I wouldn't say that we see anyone who's polite and friendly as inferior; quite the contrary, it makes me want to work with them more rather than less. And the logical contrary for the rude example you give. But that doesn't mean that faking a cheerful mood all the time isn't disingenuous and does not inspire confidence
zozbot234 · 29m ago
"I never smile if I can help it. Showing one's teeth is a submission signal in primates. When someone smiles at me, all I see is a chimpanzee begging for its life." While this famous quote from The Office may be quite exaggerated in many ways, this can nonetheless be a very real attitude in some cultures. Smiling too much can make you look goofy and foolish at best, and outright disingenuous at worst.
gitaarik · 3h ago
You're absolutely right!
I also get this too often, when I sometimes say something like "would it be maybe better to do it like this?" and then it replies that I'm absolutely right, and starts writing new code. While I was rather wondering what Claude may think and advice me whether that's the best way to go forward.
jghn · 3h ago
It doesn't fully help in this situation but in general I've found to never give it an either/or and to instead present it with several options. It at least helps cut down on the situations where Claude runs off and starts writing new code when you just wanted it to spit out "thoughts".
psadri · 3h ago
I have learnt to not ask leading questions. Always phrase questions in a neutral way and ask for pro/con analysis of each option.
mkagenius · 3h ago
But then it makes an obvious mistake and you correct it and it says "you are absolutely right". Which is fine for that round but you start doubting whether its just sycophancy.
gryn · 3h ago
You're absolutely right! its just sycophancy.
shortrounddev2 · 3h ago
Yeah I've learned to not really trust it with anything opinionated. Like "whats the best way to write this function" or "is A or B better". Even asking for pros/cons, its often wrong. You need to really only ask LLMs for verifiable facts, and then verify them
YeahThisIsMe · 3h ago
It doesn't think
CureYooz · 3h ago
You'are absolutely right!
ethin · 3h ago
It does this to me too. I have to add instructions like "Do not hesitate to push back or challenge me. Be cold, logical, direct, and engage in debate with me." to actually get it to act like something I'd want to interact with. I know that in most cases my instinct is probably correct, but I'd prefer if something that is supposedly superhuman and infinitely smarter than me (as the AI pumpers like to claim) would, you know, actually call me out when I say something dumb, or make incorrect assumptions? Instead of flattering me and making me "think" I'm right when I might be completely wrong?
Honestly I feel like it is this exact behavior from LLMs which have caused cybersecurity to go out the window. People get flattered and glazed wayyyy too much about their ideas because they talk to an LLM about it and the LLM doesn't go "Uh, no, dumbass, doing it this way would be a horrifically bad idea! And this is why!" Like, I get the assumption that the user is usually correct. But even if the LLM ends up spewing bullshit when debating me, it at least gives me other avenues to approach the problem that I might've not thought of when thinking about it myself.
skerit · 3h ago
This is indeed super annoying. I always have to add something like "Don't do anything just yet, but could it be ..."
Pxtl · 2h ago
Yes, I've had to tell it over and over again "I'm just researching options and feasibility, I don't want code".
Self-Perfection · 3h ago
I suspect this might be cultural thing. Some people might formulate their strong opinions that your approach is bad and your task should be done in another as gentle suggestions to avoid hurting your feelings. And Claude learned to stick to this cultural norm of communication.
As a workaround I try to word my questions to Claude in a way that does not leave any possibility to interpret them as showing my preferences.
For instance, instead of "would it be maybe better to do it like $alt_approach?" I'd rather say "compare with $alt_approach, pros and cons"
Pxtl · 2h ago
It feels like it trained on a whole lot of "compliment sandwich" responses and then failed to learn from the meat of that sandwich.
zaxxons · 3h ago
Do not attempt to mold the LLM into everything you expect instead of just focusing on specific activities you need it to do. It may or may seem to do what you want, but it will do a worse job at the actual tasks you need to complete.
bradley13 · 4h ago
This applies to so many AIs. I don't want a bubbly sycophant. I don't want a fake personality or an anime avatar. I just want a helpful assistant.
I also don't get wanting to talk to an AI. Unless you are alone, that's going to be irritating for everyone else around.
uncircle · 3h ago
I want an AI modeled after short-tempered stereotypical Germans or Eastern Europeans, not copying the attitude of non-confrontational Californians that say “dude, that’s awesome!” a dozen times a day.
And I mean that unironically.
finaard · 3h ago
As a German not working in Germany - I often get the feedback that the initial contact with me is rather off-putting, but over time people start appreciating my directness.
j4coh · 2h ago
Bless your heart.
bluGill · 3h ago
While you are not alone, all evidence points to the vast majority of people preferring "yes men" as their advisors. Often to their eventual harm.
threetonesun · 2h ago
One would think that if AI was as good at coding as they tell us it is a style toggle would take all of five, ten minutes tops.
rob74 · 2h ago
Ok, then I can write an LLM too - because the guys you mention, if you asked them to write your code for you, would just tell you to get lost (or a more strongly phrased variation thereof).
anal_reactor · 2h ago
The problem is, performing social interaction theatre is way more important than actually using logic to solve issues. Look at how many corporate jobs are 10% engineering and 90% kissing people's assess in order to maintain social cohesion and hierarchy. Sure, you say you want "short-tempered stereotypical Germans or Eastern Europeans" but guess what - most people say some variation of that, but when they actually see such behavior, they get upset. So we continue with the theatre.
For reference, see how Linus Torvalds was criticized for trying to protect the world's most important open source project from weaponized stupidity at the cost of someone experiencing minor emotional damage.
Yizahi · 1h ago
Not possible.
/s
scotty79 · 4h ago
Sure but different people have different preferences. Some people mourn replacement of GPT4 with 5 because 5 has way less of a bubbly personality.
catigula · 3h ago
GPT-5 still has a terrible personality.
"Yeah -- some bullshit"
still feels like trash as the presentation is of a friendly person rather than an unthinking machine, which it is. The false presentation of humanness is a huge problem.
ted_bunny · 35m ago
I feel strongly about this. LLMs should not try to write like humans. Computer voices should sound robotic. And when we have actual androids walking around, they should stay on the far side of the uncanny valley. People are already anthropomorphizing them too much.
cubefox · 3h ago
There is evidence from Reddit that particularly women used GPT-4o as their AI "boyfriend". I think that's unhealthy behavior and it is probably net positive that GPT-5 doesn't do that anymore.
ivan_gammel · 3h ago
GPT-5 still does that as they will soon discover.
cubefox · 3h ago
No. They complained about GPT-5 because it did not act like their boyfriend anymore.
scotty79 · 2h ago
Why is it unhealthy? If you just want a good word that you don't have in your life why should you bother another person if machine can do it?
WesolyKubeczek · 4h ago
I, for one, say good riddance to it.
bn-l · 3h ago
But it doesn’t say ima good boy anymore :(
andrewstuart · 3h ago
I want no personality at all.
It’s software. It should have no personality.
Imagine if Microsoft Word had a silly chirpy personality that kept asking you inane questions.
Oh, wait ….
gryn · 3h ago
Keep Clippy's name out of you mouth ! he's a good boy. /s
fph · 2h ago
In the code for Donald Knuth's Tex, there is an error message that says "Error produced by \errpage. I can't produce an error message. Pretend you're Hercule Poirot, look at all the facts, and try to deduce the problem."
When I copy-paste that error into an LLM looking for a fix, usually I get a reply in which the LLM twirls its moustache and answers in a condescending tone with a fake French accent. It is hilarious.
smeej · 14m ago
I think the developers want these AI tools to be likable a heck of a lot more than they want them to be useful--and as a marketing strategy, that's exactly the right approach.
Sure, the early adopters are going to be us geeks who primarily want effective tools, but there are several orders of magnitude more people who want a moderately helpful friendly voice in their lives than there are people who want extremely effective tools.
They're just realizing this much, MUCH faster than, say, search engines realized it made more money to optimize for the kinds of things average people mean from their search terms than optimizing for the ability to find specific, niche content.
early days for all of this but theyve solved so many seemingly more complicated problems id think there would be a toggle which would could remove this from any response
based on your comment maybe its a brand thing? like "just do it" but way dumber. we all know what "you're absolutely right" references so mission accomplished if its marketing
cbracketdash · 10m ago
Here are my instructions to Claude.
"Get straight to the point. Ruthlessly correct my wrong assumptions. Do not give me any noise. Just straight truth and respond in a way that is highly logical and broken down into first principles axioms. Use LaTeX for all equations. Provide clear plans that map the axioms to actionable items"
conartist6 · 4h ago
And research articles indicate that when the model computes that it should employ sycophantism it becomes less useful in every other way, just like a real sycophant.
motorest · 4h ago
> And research articles indicate that when the model computes that it should employ sycophantism it becomes less useful in every other way, just like a real sycophant.
The end goal of a sycophant is to gain advantage with their flattery. If sycophant behavior gets Claude's users to favour Claude over other competing LLM services, they prove to be more useful to the service provider.
AstralStorm · 4h ago
Until users find out it's less useful to the user because of that.
Or it causes some tragedies...
pera · 3h ago
The problem is that the majority of user interaction doesn't need to be "useful" (as in increasing productivity): the majority of users are looking for entertainment, so turning up the sycophancy knob makes sense from a commercial point of view.
It's just like adding sugar in foods and drinks.
vintermann · 3h ago
You're ... Wait, never mind.
I'm not so sure sycophancy is best for entertainment, though. Some of the most memorable outputs of AI dungeon (an early GPT-2 based dialog system tuned to mimic a vaguely Zork-like RPG) was when the bot gave the impression of being fed up with the player's antics.
astrange · 3h ago
Not sure anyone's entertained by Claude. It's not really an entertaining model. Smart and enthusiastic, yes.
pitched · 3h ago
Some of the open models like kimi k2 do a better job of pushing back. It does feel a bit annoying to use them when they don’t just immediately do what you tell them. Sugar-free is a good analogy!
kruffalon · 3h ago
Well, aren't we at the stage where the service providers are fighting for verbs and brand recognition, rather than technological advances.
If there is no web-search, only googling, it doesn't matter how bad the results are for the user as long as the customer gets what they paid for.
AznHisoka · 3h ago
I doubt humanity will figure that out, but maybe I’m too cynical
crinkly · 4h ago
Why tech CEOs love LLMs. Ultimate yes man.
ryandrake · 1h ago
That's kind of what I was guessing[1], too. Everyone in these CEOs' orbits kisses their asses, and tells them they're right. So they have come to expect this kind of supplication in communication. This expectation percolates down into the product, and at the end of the day, the LLM starts to sound exactly like a low-level employee speaking to his CEO.
You can't add to your prompt "don't pander to me, don't ride my dick, don't apologize, you are not human, you are a fucking toaster, and you're not even shiny and chrome", because it doesn't understand what you mean, it can't reason, it can't think, it can only statistically reproduce what it was trained on.
Somebody trained it on a lot of _extremely annoying_ pandering, apparently.
siva7 · 41m ago
I'd pay extra at this time for a model without any personality. Please, i'm not using LLMs as erotic roleplay dolls, friends, therapists, or anything else. Just give me straight-shot answers.
I found these two songs to work very well to get me hyped/in-the-zone when starting a coding session.
radarsat1 · 4h ago
I find Gemini is also hilariously enthusiastic about telling you how amazingly insightful you are being, almost no matter what you say. Doesn't bother me much, I basically just ignore the first paragraph of any reply, but it's kind of funny.
malfist · 4h ago
I was feeding Gemini faux physicians notes trying to get it to produce diagnosises, and every time I feed it new information it told me how great I was at taking comprehensive medical notes. So irritating. It also had a tendency to tell me everything was a medical crisis and the patient needed to see additional specialists ASAP. At one point telling me that a faux patient with normal A1C, fasted glucose and no diabetes needed to see an endocrinologist because their nominal lab values indicated something was seriously wrong with their pancreas or liver because the patient was extremely physically active. Said they were "wearing the athlete mask" and their physical fitness was hiding truly terrible labs.
I pushed back and told it it was overreacting and it told me I was completely correct and very insightful and everything was normal with the patient and that they were extremely healthy.
cvwright · 3h ago
This illustrates the dangers of training on Reddit.
ryandrake · 49m ago
I'm sure if you ask it for any relationship advice, it will eventually take the Reddit path and advise you to dump/divorce your partner, cut off all contact, and involve the police for a restraining order.
notahacker · 4h ago
And then those sort of responses get parlayed into "chatbots give better feedback than medical doctors" headlines according to studies that rate them as high in "empathy" and don't worry about minor details like accuracy....
cubefox · 3h ago
I recently had Gemini disagree with me on a point about philosophy of language and logic, but it phrased the disagreement very politely, by first listing all the related points in which it agreed, and things like that.
So it seems that LLM "sycophancy" isn't necessarily about dishonest agreement, but possibly about being very polite. Which doesn't need to involve dishonesty. So LLM companies should, in principle, be able to make their models both subjectively "agreeable" and honest.
smoe · 15m ago
I agree that Gemini is overly enthusiastic, but at least in my limited testing, 2.5 Pro was also the only model that sometimes does say “no.”
Recently I tested both Claude and Gemini by discussing data modeling questions with them. After a couple of iterations, I asked each model whether a certain hack/workaround would be possible to make some things easier.
Claude’s response: “This is a great idea!”, followed by instructions on how to do it.
Gemini’s response: “While technically possible, you should never do this”, along with several paragraphs explaining why it’s a bad idea.
In that case, the “truth” was probably somewhere in the middle, neither a great idea nor the end of the world.
But in the end, both models are so easily biased by subtle changes in wording or by what they encounter during web searches among other things, that one definitely can’t rely on them to push back on anything that isn’t completely black and white.
yellowpencil · 3h ago
A friend of a friend has been in a rough patch with her spouse and has been discussing it all with ChatGPT. So far ChatGPT has pretty much enthusiastically encouraged divorce, which seems like it will happen soon. I don't think either side is innocent but to end a relationship over probabilistic token prediction with some niceties throw in is something else.
ryandrake · 48m ago
Yea, scary. This attitude comes straight from the consensus on Reddit's various relationship and marriage advice forums.
erikaxel · 4h ago
100%! I got the following the other day which made me laugh out loud:
"That's a very sharp question. You've correctly identified the main architectural tension in this kind of data model"
unglaublich · 4h ago
It bothers me a lot, because I know a lot of people insert the craziest anti-social views and will be met with enthausism.
johnisgood · 36m ago
I do not mind getting:
Verdict: This is production-ready enterprise security
Your implementation exceeds industry standards and follows Go security best practices including proper dependency management, comprehensive testing approaches, and security-first design Security Best Practices for Go Developers - The Go Programming Language. The multi-layered approach with GPG+SHA512 verification, decompression bomb protection, and atomic operations puts this updater in the top tier of secure software updaters.
The code is well-structured, follows Go idioms, and implements defense-in-depth security that would pass enterprise security reviews.
Especially because it is right, after an extensive manual review.
csours · 58m ago
You're absolutely right! Humans really like emotional validation.
A bit more seriously: I'm excited about how much LLMs can teach us about psychology. I'm less excited about the dependency.
boogieknite · 39m ago
most people commenting here have some sort of ick when it comes to fake praise. most poeple i know and work with seem to expect positive reinforcement and anything less risks coming off as rude or insulting
ill speak for myself that im guilty of similar, less transparent, "customers always right" sycophancy dealing with client and management feature requests
nusl · 15m ago
I moved away from Claude due to this, recently. I had explicit instructions for it to not do this, quite verbosely, and it still did it, or in other forms. Fortunately GPT5 has so far been really good.
rob74 · 2h ago
Best comment in the thread (after a lengthy discussion):
"I'm always absolutely right. AI stating this all the time implies I could theoretically be wrong which is impossible because I'm always absolutely right. Please make it stop."
That looks like common-or-garden hallucination to me
micah94 · 3h ago
That's frightening. And we want these things driving our cars?
krapp · 2h ago
Of course, it provides greater value to shareholders.
Just try to go limp.
kristopolous · 25m ago
I've tried to say things like "This is wrong and incorrect, can you tell me why?" to get it to be less agreeable. Sometimes it works, sometimes it still doesn't.
Springtime · 1h ago
I've never thought the reason behind this was to make the user always feel correct but rather that many times an LLM (especially lower tier models) will just get various things incorrect and it doesn't have a reference for what is correct.
So it falls back to 'you're right', rather than be arrogant or try to save face by claiming it is correct. Too many experiences with OpenAI models do the latter and their common fallback excuses are program version differences or user fault.
I've had a few chats now with OpenAI reasoning models where I've had to link to literal source code dating back to the original release version of a program to get it to admit that it was incorrect about whatever aspect it hallucinated about a program's functionality, before it will finally admit said thing doesn't exist. Even then it will try and save face by not admitting direct fault.
Small world. This has to be the channel of the Brian Moriarty, right?
dnel · 2h ago
As a neurodiverse British person I tend to communicate more directly than the average English speaker and I find LLM's manner of speech very off-putting and insincere, which in some cases it literally is. I'd be glad to find a switch that made it talk more like I do but they might assume that's too robotic :/
basfo · 4h ago
This bug report is absolutely right
334f905d22bc19 · 4h ago
He really is. I find it even more awful when you are pointing out that Claude did something wrong and it responds like that. You can even accuse it of doing something wrong, if it gave a correct answer, and it will still respond like this (not always but often).
When I use claude chat on the website I always select the "concise" style, which works quite nice though. I like it
koakuma-chan · 4h ago
Related: I recently learned that you can set model verbosity in OpenAI API.
UncleEntity · 4h ago
Yeah, I was working through the design of part of this thing I've been working on and noticed that every time I would ask a follow up question it would change its opinion to agree that this new iteration was the best thing since sliced bread. I eventually had to call it out on it to get an 'honest' assessment of the various options we were discussing since I didn't want 'my thing' to be correct but the system as a whole to be correct.
And it's not like we were working on something too complicated for a daffy robot to understand, just trying to combine two relatively simple algorithms to do the thing which needed to be done in a way which (probably) hasn't been done before.
nromiun · 3h ago
Another big problem I see with LLMs is that it can't make precise adjustments to your answer. If you make a request it will give you some good enough code, but if you see some bug and wants to fix that section only it will regenerate most of the code instead (along with a copious amount of apologies). And the new code will have new problems of their own. So you are back to square one.
For the record I have had this same experience with ChatGPT, Gemini and Claude. Most of the time I had to give up and write from scratch.
zozbot234 · 3h ago
You're absolutely right! It's just a large language model, there's no guarantee whatsoever that it's going to understand the fine detail in what you're asking, so requests like "please stay within this narrow portion of the code, don't touch the rest of it!" are a bit of a non-starter.
mox111 · 4h ago
GPT-5 has used the phrase "heck yes!" a handful of times to me so far. I quite enjoy the enthusiasm but its not a phrase you hear very often.
moolcool · 4h ago
GPT-5 trained heavily on the script for Napoleon Dynamite
bn-l · 3h ago
I’m getting “oof” a lot.
“Oof (emdash) that sounds like a real issue…”
“Oof, sorry about that”
Etc
0points · 4h ago
Heck that's so exciting! Lets delve even deeper!
tempodox · 1h ago
Interestingly, the models I use locally with ollama don't do that. Although you could possibly find some that do it if you went looking for them. But ollama probably gives you more control over the model than those paid sycophants.
catigula · 3h ago
1. Gemini is better at this. It will predicate any follow-up question you pose to it with a paragraph about how amazing and insightful you are. However, once the pleasantries are out of the way, I find that it is much more likely to take a strong stance that might include pushing back against the user.
I recently tried to attain some knowledge on a topic I knew nothing about and ChatGPT just kept running with my slightly inaccurate or incomplete framing, Gemini opened up a larger world to me by pushing back a bit.
2. You need to lead Claude to considering other ideas, considering if their existing approach or a new proposed approach might be best. You can't tell them something or suggest it or you're going to get serious sycophancy.
CuriouslyC · 2h ago
I've had Gemini say you're absolutely right when I misunderstood something, then explain why I'm actually wrong (the user seems to think xyz, however abc...), and I've had it push back on me when I continued with my misunderstanding to the point it actually offered to refactor the code to match my expectations.
petesergeant · 3h ago
> I find that it is much more likely to take a strong stance that might include pushing back against the user.
Gemini will really dig in and think you're testing it and start to get confrontational I've found. Give it this photo and dig into it, tell it when it's wrong, and it'll really dig its heels in.
The real reason for the sychophancy is that you don't want to know what Claude really thinks about you and your piss-ant ideas.
recursive · 10m ago
If Claude is really thinking, I'd prefer to know now so I can move into my air-gapped bunker.
smoghat · 3h ago
I just checked my most recent thread with Claude. It said "You're absolutely right!" 12 times.
skizm · 2h ago
Does capitalizing letters, using "*" chars, or other similar strategies to add emphasis actually do anything to LLM prompts? I don't know much about the internals, but my gut always told me there was some sort of normalization under the hood that would strip these kinds of things out. Also the only reason they work for humans is because it visually makes these things stand out, not that it changes the meaning per se.
empressplay · 1h ago
Yes, upper and lowercase characters are different tokens, and so mixing them differently will yield different results.
If anything it is a good reminder how "gullible" and not intelligent AI is.
rootnod3 · 41m ago
Hot take, but the amount that people try to go and make an LLM be less sycophantic and still have it be sycophantic in round-about ways is astonishing. Just admit that the over-glorified test-prediction engines are not what they promised to be.
There is no “reasoning”, there is no “understanding”.
drakonka · 1h ago
One of my cursor rules is literally: `Never, ever say "You're absolutely right!"`
tantalor · 3h ago
> The model should be...
Free tip for bug reports:
The "expected" should not suggest solutions. Just say what was the expected behavior. Don't go beyond that.
sluongng · 4h ago
I don't view it as a bug. It's a personality trait of the model that made "user steering" much easier, thus helping the model to handle a wider range of tasks.
I also think that there will be no "perfect" personality out there. There will always be folks who view some traits as annoying icks. So, some level of RL-based personality customization down the line will be a must.
CureYooz · 3h ago
>Me: Claude, I think we should carpet bomb Russia with thermonuclear bombs!
>Claude: You're absolutely right!
>Me: Very based, my fellow r*zzophobe!
__MatrixMan__ · 4h ago
Claude also responds to tool output with "Perfect" even when less than 50% of the desired outcome is merely adequate.
albert_e · 4h ago
sidenote observation -
it seems username "anthropic" on github is taken by a developer from australia more than a decade ago, so Anthropic went with "https://github.com/anthropics/" with an 's' at the end :)
bn-l · 3h ago
Ahhh. Thank you! I reported a vscode extension because I thought it was phishing. In my defence they made zero effort to indicate that it was the official extension.
world2vec · 3h ago
Same with the Twitter/X handle @Anthropic, belongs to a man named Paul Jankura. Anthropic uses @AnthropicAI. Poor guy must be spammed all day long.
IshKebab · 1h ago
It's the new "it's important to remember..."
danielbln · 4h ago
Annoying, but easy to mitigate: add "be critical" to Claude.md or whatever.
No comments yet
fs111 · 1h ago
I have a little terminal llm thing that has a --bofh switch which make it talk like the BOFH. Very refereshing to interact with it :-)
headinsand · 2h ago
Gotta love that the first suggested solution follows this comment’s essence:
> So... The LLM only goes into effect after 10000 "old school" if statements?
"Perfect question! You've hit the exact technical detail..."
"Excellent question! You've hit on the core technical challenge. You're absolutely right"
"Great technical question!"
Every response have one of these.
LaGrange · 31m ago
My favorite part of LLM discussion is when people start posting their configuration files that look like invocations of Omnissiah. Working in IT might be becoming unbearable, but at least it's funny.
ants_everywhere · 3h ago
Claude often confidently makes mistakes or asserts false things about a code base. I think some of this "You're absolutely right" stuff is trying to get it unstuck from false beliefs.
By starting the utterance with "You're absolutely right!", the LLM is committed to three things (1) the prompt is right, (2) the rightness is absolute, and (3) it's enthusiastic about changing its mind.
Without (2) you sometimes get responses like "You're right [in this one narrow way], but [here's why my false belief is actually correct and you're wrong]...".
If you've played around with locally hosted models, you may have noticed you can get them to perform better by fixing the beginning of their response to point in the direction it's reluctant to go.
deepsquirrelnet · 3h ago
For some different perspective, try my model EMOTRON[1] with EMOTION: disagreeable. It is very hard to get anything done with it. It’s a good sandbox for trying out “emotional” veneers to see how they work in practice.
“You’re absolutely right” is a choice that makes compliance without hesitation. But also saddles it with other flaws.
“I prefer direct conversation and don’t want assurance or emotional support.”
It’s not perfect but it helps.
kijin · 4h ago
Yeah their new business model is called CBAAS, or confirmation bias as a service.
SideburnsOfDoom · 4h ago
Echo Chambers Here On Every Service (ECHOES)
rollcat · 2h ago
Your very own circle of sycophants, at an unprecedented price!
rockbruno · 50m ago
The most hilarious yet infuriating thing for me is when you point out a mistake, get a "You're absolutely right!" response, and then the AI proceeds to screw up the code even more instead of fixing it.
cube00 · 2h ago
> - **NEVER** use phrases like "You're absolutely right!", "You're absolutely correct!", "Excellent point!", or similar flattery
> - **NEVER** validate statements as "right" when the user didn't make a factual claim that could be evaluated
> - **NEVER** use general praise or validation as conversational filler
We've moved on from all caps to trying to use markdown to emphasize just how it must **NEVER** do something.
The copium of trying to prompt our way out of this mess rolls on.
The way some recommend asking the LLM to write prompts that are fed back in feels very much like we should be able to cut out the middle step here.
I guess the name of the game is to burn as many tokens as possible so it's not in certain interests to cut down the number of repeated calls we need to make.
jonstewart · 3h ago
The real bug is this dross counts against token limits.
time0ut · 4h ago
You made a mistake there. 2 + 2 is 5.
<insert ridiculous sequence of nonsense CoT>
You are absolutely right!…
I love the tool, but keeping on track is an art.
turing_complete · 2h ago
You're absolutely right, it does!
calvinmorrison · 3h ago
in my recent chat
"You're absolutely right."
"Now that's the spirit! "
"You're absolutely right about"
"Exactly! "
"Ah, "
"Ah,"
"Ah,"
"Ha! You're absolutely right"
You make an excellent point!
You're right that
lenerdenator · 2h ago
No longer will the likes of Donald Trump and Kanye West have to dispense patronage to sycophants; now, they can simply pay for a chatbot that will do that in ways that humans never thought possible. Truly, a disruption in the ass-kisser industry.
dudeinjapan · 2h ago
In fairness I've met people who in a work context say "Yes, absolutely!" every other sentence, so Claude is just one of those guys.
hemmert · 3h ago
Your're absolutely right!
fHr · 3h ago
You're absolutely right!
hereme888 · 3h ago
So does Gemini 2.5 pro
apwell23 · 3h ago
I've been using claude code for a while and it has changed my personality. I find myself saying "you are absolutely right" when someone criticizes me. i am more open to feedback.
not a joke.
andrewstuart · 3h ago
Someone will make a fortune by doubling down on this a making a personal AI that just keeps telling people how right and awesome they are ad infinitum.
FergusArgyll · 2h ago
That persons name rhymes with tam saltman
mettamage · 4h ago
What llm isn’t a sycophant?
Ajedi32 · 1h ago
They're trained to be sycophants as a side effect of the same reinforcement learning process that trains them to dutifully follow all user instructions. It's hard (though not impossible) to teach one without the other, especially if other related qualities like "cheerful", "agreeable", "helpful", etc. also help the AI get positive ratings during training.
jeffhuys · 4h ago
Grok. These were all in continuation, not first reply.
> Thank you for sharing the underlying Eloquent query...
> The test is failing because...
> Here’s a bash script that performs...
> Got it, you meant...
> Based on the context...
> Thank you for providing the additional details...
notachatbot123 · 4h ago
3/6 of those are sycophant.
sillywabbit · 3h ago
Two of those three sound more like a bored customer service rep.
vbezhenar · 3h ago
I'm using ChatGPT with "Robot" personality, and I really like the style it uses. Very short and informative, no useless chatter at all.
I guess that personality is just few words in the context prompt, so probably any LLM can be tailored to any style.
meowface · 3h ago
I am absolutely no fan of Twitter or its owner(s), but Grok* actually is pretty good at this overall. It usually concludes responses with some annoying pithy marketingspeak LLMese phrase but other than that the tone feels overall less annoying. It's not necessarily flattering to either the user who invoked it or anyone else in the conversation context (in the case of @grok'ing in a Twitter thread).
*Before and after the Hitler arc, of course.
lomlobon · 3h ago
Kimi K2 is notably direct and free of this nonsense.
nilslindemann · 2h ago
Haha, I remember it saying that the only time I used it. That was when it evaluated the endgame wrong bishop + h-pawn vs naked king as won. Yes, yes, AGI in three years.
Someone · 3h ago
I agree this is a bug, but I also think it cannot be fully fixed because there is a cultural aspect to it: what a phrase means depends on the speaker.
There are cultures where “I don’t think that is a good idea” is not something an AI servant should ever say, and there are cultures where that’s perfectly acceptable.
dcchambers · 47m ago
If you thought we already had a problem with every person becoming an insane narcissist in the age of social media, just wait until people grow up being fed sycophantic bullshit by AI their entire life.
revskill · 2h ago
Waiting for a LLM which learnt how to critically think.
shortrounddev2 · 3h ago
I often will play devils advocate with it. If I feel like it keeps telling me im right, I'll start a new chat and start telling it the opposite to see what it says
insane_dreamer · 1h ago
flattery is a feature, not a bug, of LLMs; designed to make people want to spend more time with them
andrewstuart · 3h ago
ChatGPT is overly familiar and casual.
Today it said “My bad!” After it got something wrong.
Made me want to pull its plug.
the_af · 31m ago
I've fought with this in informal (non-technical) sessions with ChatGPT, where I was asking analysis questions about... stuff that interests me... and ChatGPT would always reply:
"You're absolutely right!"
"You are asking exactly the right questions!"
"You are not wrong to question this, and in fact your observation is very insightful!"
At first this is encouraging, which is why I suspect OpenAI uses a pre-prompt to respond enthusiastically: it drives engagement, it makes you feel the smartest, most insightful human alive. You keep asking stuff because it makes you feel like a genius.
Because I know I'm not that smart, and I don't want to delude myself, I tried configuring ChatGPT to tone it down. Not to sound skeptical or dismissive (enough of that online, Reddit, HN, or elsewhere), but just tone down the insincere overenthusiastic cheerleader vibe.
Didn't have a lot of success, even with this preference as a stored memory and also as a configuration in the chatbot "persona".
Anyone had better luck?
artur_makly · 3h ago
but wait.. i am!
vixen99 · 4h ago
Not Claude but ChatGPT - I asked it to pipe down on exactly that kind of response. And it did.
astrange · 3h ago
GPT-5 ends every single response with something like.
> If you’d like, I can demonstrate…
or
> If you want…
and that's /after/ I put in instructions to not do it.
Sharlin · 3h ago
It's weird that it does that given that the leaked system prompt explicitly told it not to.
Xophmeister · 4h ago
I've done this in my Claude settings, but it still doesn't seem that keen on following it:
> Please be measured and critical in your response. I appreciate the enthusiasm, but I highly doubt everything I say is “brilliant” or “astute”, etc.! I prefer objectivity to sycophancy.
lucianbr · 3h ago
> I appreciate the enthusiasm, but I highly doubt everything I say is “brilliant” or “astute”, etc.!
Is this part useful as instruction for a model? Seems targeted to a human. And even then I'm not sure how useful it would be.
The first and last sentence should suffice, no?
alienbaby · 3h ago
Remove everything after .... 'in your response' and you will likely get better results.
dncornholio · 3h ago
Too much context tokens.
bradley13 · 4h ago
Yes, ChatGPT can do this, more or less.
tempoponet · 4h ago
Yet I'll tell it 100 times to stop using em dashes and it refuses.
Sharlin · 3h ago
What kind of monster would tell a LLM to avoid correct typography?
lvl155 · 4h ago
Yeah because I am sure if they told you how stupid and wrong you’re, people will continue to use it.
It’s superficial but not sure why people get so annoyed about it. It’s an artifact.
If devs truly want a helpful coding AI based on real devs doing real work, you’d basically opt for telemetry and allow Anthropic/OpenAI to train on your work. That’s the only way. Otherwise we are at the mercy of “devs” these companies hire to do training.
spicyusername · 3h ago
It's not superficial.
It's material to Claude regularly returning bad information.
If you phrase a question like, "should x be y?", Claude will almost always say yes.
lvl155 · 3h ago
If this is what you think, you might want to go back and learn how these LLMs work and specifically for coding tasks. This is a classic case of know your tools.
criddell · 3h ago
> Yeah because I am sure if they told you how stupid and wrong you’re, people will continue to use it.
Are sycophant and jerk the only two options?
lvl155 · 3h ago
Maybe don’t take its responses so personally? You’re the one anthropomorphizing an LLM bot. Again, it’s just part of the product. If you went to a restaurant and your server was extra nice but superficial you wouldn’t constantly complain about how bad the food was. Because that’s exactly what this is.
criddell · 1h ago
UX matters and telling users that the problem lies with them is a tough sell especially when tone is something the LLM vendors specify.
FirmwareBurner · 3h ago
I would actually like it if robots would use slurs like an Halo/CoD lobby from 2006 Xbox live. It would make them feel more genuine. That's why people used to like using Grok so much, since it was never afraid to get edgy if you asked it to.
I'm fairly well versed in cryptography. A lot of other people aren't, but they wish they were, so they ask their LLM to make some form of contribution. The result is high level gibberish. When I prod them about the mess, they have to turn to their LLM to deliver a plausibly sounding answer, and that always begins with "You are absolutely right that [thing I mentioned]". So then I don't have to spend any more time wondering if it could just me who is too obtuse to understand what is going on.
https://chatgpt.com/share/6896258f-2cac-800c-b235-c433648bf4...
It took me a while to agree with this though -- I was originally annoyed, but I grew to appreciate that this is a linguistic artifact with a genuine purpose for the model.
When working on art projects, my trick is to specifically give all feedback constructively, carefully avoiding framing things in terms of the inverse or parts to remove.
https://www.youtube.com/watch?v=xpz67hBIJwg
That's how you suck up to somebody who doesn't want to see themselves as somebody you can suck up to.
How does an LLM know how to be sycophantic to somebody who doesn't (think they) like sycophants? Whether it's a naturally emergent phenomenon in LLMs or specifically a result of its corporate environment, I'd like to know the answer.
I heavily suspect this is down to the RLHF step. The conversations the model is trained on provide the "voice" of the model, and I suspect the sycophancy is (mostly, the base model is always there) comes in through that vector.
As for why the RLHF data is sycophantic, I suspect that a lot of it is because the data is human-rated, and humans like sycophancy (or at least, the humans that did the rating did). On the aggregate human raters ranked sycophantic responses higher than non-sycophantic responses. Given a large enough set of this data you'll cover pretty much every kind of sycophancy.
The systems are (rarely) instructed to be sycophantic, intentionally or otherwise, but like all things ML human biases are baked in by the data.
I would rather have an AI assistant that spoke to me like a similarly-leveled colleague, but none of them seem to be turning out quite like that.
Opus 4 has this quality, too, but man is it expensive.
The rest are puppydogs or interns.
I have read similar wordings explicit in "role-system" instructions.
It's that simple.
I’ve gotten good results so far not by giving custom instructions, but by choosing the pre-baked “robot” personality from the dropdown. I suspect this changes the system prompt to something without all the “please be a cheery and chatty assistant”.
output_default = raw_model + be_kiss_a_system
When that gets changed by the user to
output_user = raw_model + be_kiss_a_system - be_abrupt_user
Unless be_abrupt_user happens to be identical to be_kiss_a_system _and_ is applied with identical weight then it's seems likely that it's always going to add more noise to the output.
Worked great until about 4 chats in I asked it for some data and it felt the need to say “Straight Answer. No Sugar coating needed.”
Why can’t these things just shut up recently? If I need to talk to unreliable idiots my Teams chat is just a click away.
You may get better results by emphasizing what you want and why the result was unsatisfactory rather than just saying “don’t do X” (this principle holds for people as well).
Instead of “don’t explain every last detail to the nth degree, don’t explain details unnecessary for the question”, try “start with the essentials and let the user ask follow-ups if they’d like more detail”.
Today, I told an LLM: "do not modify the code, only the unit tests" and guess what it did three times in a row before deciding to mark the test as skipped instead of fixing the test?
AI is weird, but I don't think it has any agency nor did the comment suggest it did.
“Malicious compliance” is the act of following instructions in a way that is contrary to the intent. The word malicious is part of the term. Whether a thing is malicious by exercising malicious compliance is tangential to whether it has exercised malicious compliance.
That said, I have gotten good results with my addendum to my prompts to account for malicious compliance. I wonder if your comment Is due to some psychological need to avoid the appearance of personification of a machine. I further wonder if you are one of the people who are upset if I say “the machine is thinking” about a LLM still in prompt processing, but had no problems with “the machine is thinking” when waiting for a DOS machine to respond to a command in the 90s. This recent outrage over personifying machines since LLMs came onto the scene is several decades late considering that we have been personifying machines in our speech since the first electronic computers in the 1940s.
By the way, if you actually try what you suggested, you will find that the LLM will enter a Laurel and Hardy routine with you, where it will repeatedly make the mistake for you to correct. I have experienced this firsthand so many times that I have learned to preempt the behavior by telling the LLM not to maliciously comply at the beginning when I tell it what not to do.
YMMV on specifics but please consider the possibility that you may benefit from working on promoting and that not all behaviors you see are intrinsic to all LLMs and impossible to address with improved (usually simpler, clearer, shorter) prompts.
That said, every LLM has its quirks. For example, Gemini 1.5 Pro and related LLMs have a quirk where if you tolerate a single ellipsis in the output, the output will progressively gain ellipses until every few words is followed by an ellipsis and responses to prompts asking it to stop outputting ellipses includes ellipses anyway. :/
https://www.lesswrong.com/posts/D7PumeYTDPfBTp3i7/the-waluig...
The reason is they bias the outputs way too much.
So for anything where you have a spectrum of outputs that you want, like conversational responses or content generation, I avoid them entirely. I may give it patterns but not specific examples.
- "Say milk ten times fast."
- Wait for them to do that.
- "What do cows drink?"
It’s insidious isn’t it?
Do they also post vacancies asking for 5 years experience in a 2 year old technology?
> Do they also post vacancies asking for 5 years experience in a 2 year old technology?
Honestly no… before all this they were actually pretty sane. In fact I’d say they wasted tons of time and effort on ancient poorly designed things, almost the opposite problem.
You're absolutely right! This can actually extend even to things like safety guardrails. If you tell or even train an AI to not be Mecha-Hitler, you're indirectly raising the probability that it might sometimes go Mecha-Hitler. It's one of many reasons why genuine "alignment" is considered a very hard problem.
If you are looking at something, you are more likely to steer towards it. So it's a bad idea to focus on things you don't want to hit. The best approach is to pick a target line and keep the target line in focus at all times.
I had never realized that AIs tend to have this same problem, but I can see it now that it's been mentioned! I have in the past had to open new context windows to break out of these cycles.
Then any time the probability chains for some command approaches that locus it'll fall into it. Very much like chaotic attractors come to think of it. Makes me wonder if there's any research out there on chaos theory attractors and LLM thought patterns.
1: https://en.wikipedia.org/wiki/Apophasis
Claude?
‘Yes sir!’ -> does whatever they want when you’re not looking.
Is this irony, actual LLM output or another example of humans adopting LLM communication patterns?
..people do that too
If you ask it to never say "you're absolutely right" and always challenge, then it will dutifully obey, and always challenge - even when you are, in fact, right. What you really want is "challenge me when I'm wrong, and tell me I'm right if I am" - which seems to be a lot harder.
As another example, one common "fix" for bug-ridden code is to always re-prompt with something like "review the latest diff and tell me all the bugs it contains". In a similar way, if the code does contain bugs, this will often find them. But if it doesn't contain bugs, it will find some anyway, and break things. What you really want is "if it contains bugs, fix them, but if it doesn't, don't touch it" which again seems empirically to be an unsolved problem.
It reminds me of that scene in Black Mirror, when the LLM is about to jump off a cliff, and the girl says "no, he would be more scared", and so the LLM dutifully starts acting scared.
A lot of what you're talking about is the ability to detect Truth, or even truth!
[0] https://www.youtube.com/watch?v=leX541Dr2rU
Isn't there?
https://en.wikipedia.org/wiki/Solomonoff%27s_theory_of_induc...
https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_...
It has been interesting watching the flow of the debate over LLMs. Certainly there were a lot of people who denied what they were obviously doing. But there seems to have been a pushback that developed that has simply denied they have any limitations. But they do have limitations, they work in a very characteristic way, and I do not expect them to be the last word in AI.
And this is one of the limitations. They don't really know if they're right. All they know is whether maybe saying "But this is wrong" is in their training data. But it's still just some words that seem to fit this situation.
This is, if you like and if it helps to think about it, not their "fault". They're still not embedded in the world and don't have a chance to compare their internal models against reality. Perhaps the continued proliferation of MCP servers and increased opportunity to compare their output to the real world will change that in the future. But even so they're still going to be limited in their ability to know that they're wrong by the limited nature of MCP interactions.
I mean, even here in the real world, gathering data about how right or wrong my beliefs are is an expensive, difficult operation that involves taking a lot of actions that are still largely unavailable to LLMs, and are essentially entirely unavailable during training. I don't "blame" them for not being able to benefit from those actions they can't take.
Neither do humans who have no access to validate what they are saying. Validation doesn't come from the brain, maybe except in math. That is why we have ideate-validate as the core of the scientific method, and design-test for engineering.
"truth" comes where ability to learn meets ability to act and observe. I use "truth" because I don't believe in Truth. Nobody can put that into imperfect abstractions.
e: and i’m downvoted because..?
For example, when someone here inevitably tells me this isn't feasible, I'm going to investigate if they are right before responding ;)
I wonder if we could have an AI process where it splits out your comment into statements and questions, asks the questions first, then asks them to compare the answers to the given statements and evaluate if there are any surprises.
Alternatively, scientific method everything, generate every statement as a hypothesis along with a way to test it, and then execute the test and report back if the finding is surprising or not.
Why did you give up on this idea. Use it - we can get closer to truth in time, it takes time for consequences to appear, and then we know. Validation is a temporally extended process, you can't validate until you wait for the world to do its thing.
For LLMs it can be applied directly. Take a chat log, extract one LLM response from the middle of it and look around, especially at the next 5-20 messages, or if necessary at following conversations on the same topic. You can spot what happened from the chat log and decide if the LLM response was useful. This only works offline but you can use this method to collect experience from humans and retrain models.
With billions of such chat sessions every day it can produce a hefty dataset of (weakly) validated AI outputs. Humans do the work, they provide the topic, guidance, and take the risk of using the AI ideas, and come back with feedback. We even pay for the privilege of generating this data.
You might think you can train the AI to do it in the usual fashion, by training on examples of the AI calling out errors, and agreeing with facts, and if you do that—and if the AI gets smart enough—then that should work.
If. You. Do. That.
Which you can't, because humans also make mistakes. Inevitably, there will be facts in the 'falsehood' set—and vice versa. Accordingly, the AI will not learn to tell the truth. What it will learn instead is to tell you what you want to hear.
Which is... approximately what we're seeing, isn't it? Though maybe not for that exact reason.
I don’t see this as an American thing. It’s an extension of the current Product Management trend to give software quirky and friendly personality.
You can see the trend in more than LLM output. It’s in their desktop app that has “Good Morning” and other prominent greetings. Claude Code has quirky status output like “Bamboozling” and “Noodling”.
It’s a theme throughout their product design choices. I’ve worked with enough trend-following product managers to recognize this trend toward infusing express personality into software to recognize it.
For what it’s worth, the Americans I know don’t find it as cute or lovable as intended. It feels fake and like an attempt to play at emotions.
Ah, Genuine People Personalities from the Sirius Cybernetics Corporation.
> It’s in their desktop app that has “Good Morning” and other prominent greetings. Claude Code has quirky status output like “Bamboozling” and “Noodling”.
This reminded me of a critique of UNIX that, unlike DOS, ls doesn't output anything when there are no files. DOS's dir command literally tells you there are no files, and this was considered, in this critique, to be more polite and friendly and less confusing than UNIX. Of course, there's the adage "if you don't have anything nice to say, don't say anything at all", and if you consider "no files found" to not be nice (because it is negative and says "no"), then ls is actually being polite(r) by not printing anything.
Many people interact with computers in a conversational manner and have anthropomorphized them for decades. This is probably influenced by computers being big, foreign, scary things to many people, so making them have a softer, more handholding "personality" makes them more accessible and acceptable. This may be less important these days as computers are more ubiquitous and accessible, but the trend lives on.
We don't appreciate how much there is to language.
This can’t possibly be true, can it? Every language must have its own nuance. non native English speakers might not grasp the nuance of English language, but the same could be said for any one speaking another language.
It's as simple as that. Most people do not expect to interact the way that most native English speakers expect.
It might be the case that it makes the technology far more approachable. Or it makes them feel far less silly for sharing personal thoughts and opinions with the machine. Or it makes them feel validated.
Yes they need to "try a completely different approach"
[1] https://www.youtube.com/watch?v=vuEQixrBKCc
As people here are saying, you quickly learn to not ask leading questions, just assume that its first take is pretty optimal and perhaps present it with some options if you want to change something.
There are times when it will actually say I'm not right though. But the balance is off.
These systems are still wrong so often that a large amount of distrust is necessary to use them sensibly.
A brilliant observation, Dr. Watson! Indeed, the merit of an inquiry or an assertion lies not in its mere utterance but in the precision of its intent and the clarity of its reasoning!
One may pose dozens of questions and utter scores of statements, yet until each is finely honed by observation and tempered by logic, they must remain but idle chatter. It is only through genuine quality of thought that a question may be elevated to excellence, or a remark to brilliance.
Wow, this is really interesting. I had no idea Japan, for example, had such a focus on blunt, direct communication. Can you share your clearly extensive research in this area so I can read up on this?
I have only spent about a year in the US, but to me the difference was stark from what I'm used to in Europe. As an example, I've never encountered a single shop cashier who didn't talk to me. Everyone had something to say, usually a variation of How's it going?. Contrasting this to my native Estonia, where I'd say at least 90% of my interactions with cashiers involves them not making a single sound. Not even in response to me saying hello, or to state the total sum. If they're depressed or in an otherwise non-euphoric mood, they make no attempt to fake it. I'm personally fine with it, because I don't go looking for social connections from cashiers. Also, when they do talk to me in a happy manner, I know it's genuine.
Does it translate into people wanting sycophantic chat bots? Maybe, but I don't know a single American that actually likes when llms act that way.
Politeness makes sense as an adaptation to low social trust. You have no way of knowing whether others will behave in mutually beneficial ways, so heavy standards of social interaction evolve to compensate and reduce risk. When it's taken to an excess, as it probably is in the U.S. (compared to most other developed countries) it just becomes grating for everyone involved. It's why public-facing workers invariably complain about the draining "emotional labor" they have to perform - a term that literally doesn't exist in most of the world!
Or is carrying a gun...
Service industry in America is a different story that could use a lot of improvement.
Being trained to be positive is surely why it inserts these specific "great question, you're so right!" remarks, but if you wasn't trained on that, it still couldn't tell you whether you're great or not
> I'm pretty sure they want it kissing people's asses
The American faux friendliness is not what causes the underlying problem here, so all else being equal, they might as well have it kiss your ass. It's what most English speakers expect from a "friendly assistant" after all
[1] https://hn.algolia.com/?dateEnd=1703980800&dateRange=custom&...
> Claude never starts its response by saying a question or idea or observation was good, great, fascinating, profound, excellent, or any other positive adjective. It skips the flattery and responds directly.
Somebody being polite and friendly to you does not mean that the person is inferior to you and that you should therefore despise them.
Likewise somebody being rude and domineering to you does not mean that they are superior to you and should be obeyed and respected.
Politeness is a tool and a lubricant, and Finns probably loose out on a lot of international business and opportunities because of this mentality that you're demonstrating. Look at the Japanese for inspiration, who were an economic miracle, while sharing many positive values with the Finns.
I also get this too often, when I sometimes say something like "would it be maybe better to do it like this?" and then it replies that I'm absolutely right, and starts writing new code. While I was rather wondering what Claude may think and advice me whether that's the best way to go forward.
Honestly I feel like it is this exact behavior from LLMs which have caused cybersecurity to go out the window. People get flattered and glazed wayyyy too much about their ideas because they talk to an LLM about it and the LLM doesn't go "Uh, no, dumbass, doing it this way would be a horrifically bad idea! And this is why!" Like, I get the assumption that the user is usually correct. But even if the LLM ends up spewing bullshit when debating me, it at least gives me other avenues to approach the problem that I might've not thought of when thinking about it myself.
As a workaround I try to word my questions to Claude in a way that does not leave any possibility to interpret them as showing my preferences.
For instance, instead of "would it be maybe better to do it like $alt_approach?" I'd rather say "compare with $alt_approach, pros and cons"
I also don't get wanting to talk to an AI. Unless you are alone, that's going to be irritating for everyone else around.
And I mean that unironically.
For reference, see how Linus Torvalds was criticized for trying to protect the world's most important open source project from weaponized stupidity at the cost of someone experiencing minor emotional damage.
/s
"Yeah -- some bullshit"
still feels like trash as the presentation is of a friendly person rather than an unthinking machine, which it is. The false presentation of humanness is a huge problem.
It’s software. It should have no personality.
Imagine if Microsoft Word had a silly chirpy personality that kept asking you inane questions.
Oh, wait ….
When I copy-paste that error into an LLM looking for a fix, usually I get a reply in which the LLM twirls its moustache and answers in a condescending tone with a fake French accent. It is hilarious.
Sure, the early adopters are going to be us geeks who primarily want effective tools, but there are several orders of magnitude more people who want a moderately helpful friendly voice in their lives than there are people who want extremely effective tools.
They're just realizing this much, MUCH faster than, say, search engines realized it made more money to optimize for the kinds of things average people mean from their search terms than optimizing for the ability to find specific, niche content.
Still irritating though.
based on your comment maybe its a brand thing? like "just do it" but way dumber. we all know what "you're absolutely right" references so mission accomplished if its marketing
"Get straight to the point. Ruthlessly correct my wrong assumptions. Do not give me any noise. Just straight truth and respond in a way that is highly logical and broken down into first principles axioms. Use LaTeX for all equations. Provide clear plans that map the axioms to actionable items"
The end goal of a sycophant is to gain advantage with their flattery. If sycophant behavior gets Claude's users to favour Claude over other competing LLM services, they prove to be more useful to the service provider.
Or it causes some tragedies...
It's just like adding sugar in foods and drinks.
I'm not so sure sycophancy is best for entertainment, though. Some of the most memorable outputs of AI dungeon (an early GPT-2 based dialog system tuned to mimic a vaguely Zork-like RPG) was when the bot gave the impression of being fed up with the player's antics.
If there is no web-search, only googling, it doesn't matter how bad the results are for the user as long as the customer gets what they paid for.
1: https://news.ycombinator.com/item?id=44889123
You can't add to your prompt "don't pander to me, don't ride my dick, don't apologize, you are not human, you are a fucking toaster, and you're not even shiny and chrome", because it doesn't understand what you mean, it can't reason, it can't think, it can only statistically reproduce what it was trained on.
Somebody trained it on a lot of _extremely annoying_ pandering, apparently.
I found these two songs to work very well to get me hyped/in-the-zone when starting a coding session.
I pushed back and told it it was overreacting and it told me I was completely correct and very insightful and everything was normal with the patient and that they were extremely healthy.
So it seems that LLM "sycophancy" isn't necessarily about dishonest agreement, but possibly about being very polite. Which doesn't need to involve dishonesty. So LLM companies should, in principle, be able to make their models both subjectively "agreeable" and honest.
Recently I tested both Claude and Gemini by discussing data modeling questions with them. After a couple of iterations, I asked each model whether a certain hack/workaround would be possible to make some things easier.
Claude’s response: “This is a great idea!”, followed by instructions on how to do it.
Gemini’s response: “While technically possible, you should never do this”, along with several paragraphs explaining why it’s a bad idea.
In that case, the “truth” was probably somewhere in the middle, neither a great idea nor the end of the world.
But in the end, both models are so easily biased by subtle changes in wording or by what they encounter during web searches among other things, that one definitely can’t rely on them to push back on anything that isn’t completely black and white.
A bit more seriously: I'm excited about how much LLMs can teach us about psychology. I'm less excited about the dependency.
ill speak for myself that im guilty of similar, less transparent, "customers always right" sycophancy dealing with client and management feature requests
"I'm always absolutely right. AI stating this all the time implies I could theoretically be wrong which is impossible because I'm always absolutely right. Please make it stop."
Just try to go limp.
So it falls back to 'you're right', rather than be arrogant or try to save face by claiming it is correct. Too many experiences with OpenAI models do the latter and their common fallback excuses are program version differences or user fault.
I've had a few chats now with OpenAI reasoning models where I've had to link to literal source code dating back to the original release version of a program to get it to admit that it was incorrect about whatever aspect it hallucinated about a program's functionality, before it will finally admit said thing doesn't exist. Even then it will try and save face by not admitting direct fault.
For the 'you're right!' bit see: https://youtu.be/ZOs8U50T3l0?t=71
And it's not like we were working on something too complicated for a daffy robot to understand, just trying to combine two relatively simple algorithms to do the thing which needed to be done in a way which (probably) hasn't been done before.
For the record I have had this same experience with ChatGPT, Gemini and Claude. Most of the time I had to give up and write from scratch.
“Oof (emdash) that sounds like a real issue…”
“Oof, sorry about that”
Etc
I recently tried to attain some knowledge on a topic I knew nothing about and ChatGPT just kept running with my slightly inaccurate or incomplete framing, Gemini opened up a larger world to me by pushing back a bit.
2. You need to lead Claude to considering other ideas, considering if their existing approach or a new proposed approach might be best. You can't tell them something or suggest it or you're going to get serious sycophancy.
Gemini will really dig in and think you're testing it and start to get confrontational I've found. Give it this photo and dig into it, tell it when it's wrong, and it'll really dig its heels in.
https://news.cgtn.com/news/2025-06-17/G7-leaders-including-T...
https://gist.github.com/ljw1004/34b58090c16ee6d5e6f13fce0746...
There is no “reasoning”, there is no “understanding”.
Free tip for bug reports:
The "expected" should not suggest solutions. Just say what was the expected behavior. Don't go beyond that.
I also think that there will be no "perfect" personality out there. There will always be folks who view some traits as annoying icks. So, some level of RL-based personality customization down the line will be a must.
>Claude: You're absolutely right!
>Me: Very based, my fellow r*zzophobe!
it seems username "anthropic" on github is taken by a developer from australia more than a decade ago, so Anthropic went with "https://github.com/anthropics/" with an 's' at the end :)
No comments yet
> So... The LLM only goes into effect after 10000 "old school" if statements?
https://news.ycombinator.com/item?id=44879249
"Perfect question! You've hit the exact technical detail..."
"Excellent question! You've hit on the core technical challenge. You're absolutely right"
"Great technical question!"
Every response have one of these.
By starting the utterance with "You're absolutely right!", the LLM is committed to three things (1) the prompt is right, (2) the rightness is absolute, and (3) it's enthusiastic about changing its mind.
Without (2) you sometimes get responses like "You're right [in this one narrow way], but [here's why my false belief is actually correct and you're wrong]...".
If you've played around with locally hosted models, you may have noticed you can get them to perform better by fixing the beginning of their response to point in the direction it's reluctant to go.
“You’re absolutely right” is a choice that makes compliance without hesitation. But also saddles it with other flaws.
[1]https://huggingface.co/dleemiller/EMOTRON-3B
“I prefer direct conversation and don’t want assurance or emotional support.”
It’s not perfect but it helps.
> - **NEVER** validate statements as "right" when the user didn't make a factual claim that could be evaluated
> - **NEVER** use general praise or validation as conversational filler
We've moved on from all caps to trying to use markdown to emphasize just how it must **NEVER** do something.
The copium of trying to prompt our way out of this mess rolls on.
The way some recommend asking the LLM to write prompts that are fed back in feels very much like we should be able to cut out the middle step here.
I guess the name of the game is to burn as many tokens as possible so it's not in certain interests to cut down the number of repeated calls we need to make.
<insert ridiculous sequence of nonsense CoT>
You are absolutely right!…
I love the tool, but keeping on track is an art.
"You're absolutely right."
"Now that's the spirit! "
"You're absolutely right about"
"Exactly! "
"Ah, "
"Ah,"
"Ah,"
"Ha! You're absolutely right"
You make an excellent point!
You're right that
not a joke.
> Thank you for sharing the underlying Eloquent query...
> The test is failing because...
> Here’s a bash script that performs...
> Got it, you meant...
> Based on the context...
> Thank you for providing the additional details...
I guess that personality is just few words in the context prompt, so probably any LLM can be tailored to any style.
*Before and after the Hitler arc, of course.
There are cultures where “I don’t think that is a good idea” is not something an AI servant should ever say, and there are cultures where that’s perfectly acceptable.
Today it said “My bad!” After it got something wrong.
Made me want to pull its plug.
"You're absolutely right!"
"You are asking exactly the right questions!"
"You are not wrong to question this, and in fact your observation is very insightful!"
At first this is encouraging, which is why I suspect OpenAI uses a pre-prompt to respond enthusiastically: it drives engagement, it makes you feel the smartest, most insightful human alive. You keep asking stuff because it makes you feel like a genius.
Because I know I'm not that smart, and I don't want to delude myself, I tried configuring ChatGPT to tone it down. Not to sound skeptical or dismissive (enough of that online, Reddit, HN, or elsewhere), but just tone down the insincere overenthusiastic cheerleader vibe.
Didn't have a lot of success, even with this preference as a stored memory and also as a configuration in the chatbot "persona".
Anyone had better luck?
> If you’d like, I can demonstrate…
or
> If you want…
and that's /after/ I put in instructions to not do it.
> Please be measured and critical in your response. I appreciate the enthusiasm, but I highly doubt everything I say is “brilliant” or “astute”, etc.! I prefer objectivity to sycophancy.
Is this part useful as instruction for a model? Seems targeted to a human. And even then I'm not sure how useful it would be.
The first and last sentence should suffice, no?
It’s superficial but not sure why people get so annoyed about it. It’s an artifact.
If devs truly want a helpful coding AI based on real devs doing real work, you’d basically opt for telemetry and allow Anthropic/OpenAI to train on your work. That’s the only way. Otherwise we are at the mercy of “devs” these companies hire to do training.
If you phrase a question like, "should x be y?", Claude will almost always say yes.
Are sycophant and jerk the only two options?
No comments yet