Google's new AI mode is good, actually

120 xnx 64 9/7/2025, 2:43:07 PM simonwillison.net ↗

Comments (64)

Alifatisk · 4h ago
I usually can't read blog posts, they are stale, boring and can't keep my attention span unless I know there is something interesting the paragraph, this is regardless of the interesting title. But when it comes to Simon willisons blog, I enjoy reading it, it's entertaining and interesting, it reads like a human and is enjoyable to read

Also, I can't access Google AI mode because I'm in EU but when looking at the video on YT, it looks like Perplexity, but googlified. I haven't seen any other tool that comes close to Perplexity yet, I have their app installed on all my devices and it's part of my daily life, it's so good! Especially with their pro plan (I got 12-months for free)

wer232essf · 3h ago
I can relate to that a lot. Most blogs feel like they’re written more for SEO than for actual people, so I rarely finish reading them. But when you find a writer who has a natural, human tone, it makes such a big difference you don’t feel like you’re forcing yourself through text, it just flows. I’ve had that same experience with a couple of independent blogs, and it really makes me wish more technical writing followed that style.

As for AI tools, I’m in the same boat can’t access all of them where I live, but I’ve been impressed with the ones I can use. Perplexity has definitely been the standout for me too, both for quick factual lookups and for deeper dives when I don’t want to wade through endless search results. Having it synced across devices is such a game-changer; I’ll use it on my phone while commuting and then pick up the same threads on my laptop later. Honestly, I’d rather pay for something that saves me time every single day than stick with the old ways of searching.

xnx · 23h ago
Interestingly, per the recent Google antitrust ruling documents, AI mode is extra fast because of a special FastSearch index: https://x.com/Marie_Haynes/status/1963031598829314161
cj · 23h ago
Gemini in general is extremely fast, compared to ChatGPT 5 Thinking.

It also seems to excel at things ChatGPT 5 Thinking isn't good at. Simple things like "Here's a screenshot of text, please transcribe it" - ChatGPT 5 Thinking will spend 2 minutes and still get the results wrong, while Gemini Pro will spend 20-30 seconds and transcribe everything perfectly.

Obviously that's just 1 use case, but as someone who previously used ChatGPT exclusively, I'm increasingly impressed by Gemini the more I use it. Mainly due to the much faster thinking times that seem to provide equal or better results than GTP 5 Thinking.

Xmd5a · 9h ago
>"Here's a screenshot of text, please transcribe it"

I took pictures of a book, asked Gemini to transcribe, then to translate them and I'm now in the process of having it reproduce the whole book in latex (lot of figures). Not sure exactly what I'm doing but but I've been wondering: Should I reproduce the publishing house logo? Or invent my own? Damn, this is fun.

GlitchInstitute · 17h ago
Gemini is very fast because it runs on TPUsV7 mostly
sigmoid10 · 8h ago
It is definitely because it's a smaller model. TPUv7 has ~10% lower flops at FP8 and 33% lower memory bandwidth than Nvidia Blackwell cards. Add CUDA to the comparison and they'll probably be even worse at real world utilization. Grok is already running on Blackwell cards and although there's little info on GPT5, I doubt they are behind.
xnx · 23h ago
My main takeaway is that Google's AI efforts are still playing catch-up in the mindshare race even if they've long since caught up with or surpassed OpenAI technically. A huge number of people who are heavy users of ChatGPT or Claude may have never tried Gemini.
jacquesm · 23h ago
A huge number of people that use Google Docs and other products daily are trying really hard to avoid using it in spite of it being rammed down their throats to the point that leaving Google starts to look like a viable option. Oh, and of course you do pay for it even if you don't use it and never plan to use it.
Kwpolska · 22h ago
Yeah, I changed my primary search engine to DuckDuckGo when Google started forcing incorrect AI results down my throat.
busymom0 · 17h ago
I have Userscripts setup to hide the AI mode buttons and other AI results.
jacquesm · 17h ago
Also in Google docs, gmail and so on?
busymom0 · 17h ago
I don't use those on my computer, so I guess I haven't run into it there. My setup is mostly for Google search and YouTube on both mobile safari and macOS Safari. For YouTube, I have hidden the shorts, community posts and any videos with less than 1000 views (almost always AI generated slop).
jacquesm · 16h ago
Good stuff. If you are sharing it anywhere a pointer would be appreciated. I've already done as much as I can without spending a week hacking up scripts and I'm game for anything that will improve this further.
watwut · 18h ago
I chanted it when I realized that bad search results were not a lost fight against SEO, but intention. That made me try other search engines ... and both bing and duckduckgo are better.
whimsicalism · 18h ago
i think they are shooting themselves in the foot by automatically including results from a quite dumb model in google search
TeMPOraL · 9h ago
Strongly agreed. I think this is a large part of the reason behind dissonance of opinions on AI usefulness we see even here: unless you are intentional about it, your main experience with AI is that of non-SOTA or budget versions of the models.

What makes it worse is that even if you are intentional and are willing to pay, it doesn't help everywhere. Gemini as voice assistant defaults to Flash and there is no choice to change it, even though the speed of the faster model hardly matters here, while accuracy does. Same with AI suggestions everywhere, and other such "ambient" features.

The flip site of being intentional about always using SOTA models, is that you notice when you are not getting them.

tim333 · 22h ago
I kind of think of them as the thing that pops up when you try googling something. A lot of people have probably come across that.
YuriNiyazov · 23h ago
The example itself that Simon puts up is questionable. I might be wrong about this, but I thought I read elsewhere that the “buy, scan, destroy” method was explicitly not the problem, and instead the issue was that anthropic downloaded libgen, and the settlement was for libgen.

Edit: https://news.ycombinator.com/item?id=45143392

gundmc · 23h ago
Yeah, you're right! The answer is definitely misleading at best. It would be better if the sentence "This method was a major component of a copyright lawsuit settlement that Anthropic paid in September 2025." was removed.

I'm sure this method _did_ come under discussion in the lawsuit & settlement, but as you pointed out the settlement itself was only about pirated works.

nunez · 22h ago
Welcome to subtle misinformation as truth everywhere. "Ministry of Truth" might be a good name for this phenomenon.
ojosilva · 8h ago
Yeah, it's amazing how fast people can overlook incorrect results when the form is in the spotlight. Or the AI rhetoric is convincing or plausible.

I recall a tweet by Altman, leaking the launch of GPT-5, praising their new model's answer to a prompt about thought provoking TV shows about AI. The X thread that followed was about the form ("em-dashes are still there!") and nearly nobody cared to evaluate that neither shows recommended were about AI. They weren't, or at least, were very debatable as belonging to the genre.

nunez · 1h ago
I especially liked how the image inside of a link I shared of an example of GPT-5's (in)ability to rank states was decented sans warning...
topaz0 · 23h ago
Yeah, reading the response raised a lot of questions for me too. I find it striking that he doesn't comment on whether the presented facts were true or relevant. What does it mean that "In 2025, a search tool revealed that Meta [...]". Is it the search tool that the "AI mode" is using under the hood?
simonw · 16h ago
Yeah, I agree: "This method was a major component of a copyright lawsuit settlement that Anthropic paid in September 2025" is a bit misleading, I actually wrote a whole separate thing about that here: https://simonwillison.net/2025/Sep/6/anthropic-settlement/
whimsicalism · 18h ago
i don’t read the AI answer as saying otherwise, but one might read that the physical purchasing was used as part of the settlement. it was used as part of the defense to mitigate though
Workaccount2 · 23h ago
Google is really shooting themselves in the foot with AI overviews.

It's probably the most popular AI on earth by daily queries, and likewise probably an ~8B level model, it means a whole bunch of people equate Google AI to AI overviews.

danpalmer · 14h ago
I'm a bit biased, but I find the AI overviews to be basically great. All I want from a search engine is the correct answer, Google's knowledge graph has done that for many queries for a long time, and AI overviews seems like a good next step in that process.

I've not seen many hallucinations, fact checking is fairly straightforward with the onward links, and it's not like I can take any linked content at face value anyway, I'd still want to fact check when it makes sense even if it wasn't AI written.

squigz · 9h ago
I constantly get blatant hallucinations. It particularly likes to take feature requests/suggestions and tell me they're presently possible, when they're not. It's long past the point where I just ignore the AI overviews entirely.
nicbou · 6h ago
They're also shooting themselves in the foot by halving the traffic to the websites that provide the training data.
bee_rider · 23h ago
I wonder to what extent that “generate some garbage spam along with every search” has hurt their reputation among the general public.
GlitchInstitute · 17h ago
probably close to zero? people get the anwers for most questions.

Small-time blogs were dead before AI

ruszki · 11h ago
People get an answer, and not the answer. It seems that people are fine with this. Even this article's example answer is false. The author didn't care, and considered it good.
sometimes_all · 11h ago
I believe that Google has figured out (correctly, IMO) that accuracy doesn't matter to most people 99% of the time, and people will likely do a deeper search for the 1% of the time they do want it.

If people really wanted the truth and facts, we would not have misinformation spread this widely via social media and other places.

utyop22 · 4h ago
People prefer to be lazy and sit and do nothing, than to move around to achieve a goal.

Is that a good thing? The reality is most humans are becoming more and more intellectually lazy - as a result their cognitive function are in decline. Therefore if something looks right at face value / supports an internal bias - they take it and run with it.

whimsicalism · 18h ago
yeah a lot of people i speak to i think have developed the impression that modern AI hallucinates more than it does from the dumb search model
gundmc · 23h ago
I agree, AI Mode is _very good_. But there's a huge branding and discovery problem in that 100% of the people I mention this to think I'm talking about the (error riddled) AI Overviews and have never heard of AI Mode.

I do probably 40% of my searches with AI Mode now. It can't possibly be profitable (and maybe that's why it's not more discoverable), but the results are awesome.

Edit: I also tried to show my aging parents how to use it, and it was inexplicably not available on their devices. They use old (10ish year) ios devices, which is apparently incompatible even though it's a web interface.

tim333 · 22h ago
The "AI Mode" tab appears at the top of every google search for me but it's true they don't really push it.
Havoc · 22h ago
Just tried a few side by side comparisons with perplexity.

Google answers more concisely, faster and confidently, but not convinced quality of output is better. e.g. Google pulled in info from AWS and Oracle cloud when I asked a GCP specific question. Perplexity sourced only from GCP docs

Which is an interesting outcome since I'd expect google to excel in the search aspect

frizlab · 18h ago
> since I'd expect google to excel in the search aspect

You still do?

og_kalu · 23h ago
On that note, AI mode's image search is really good

I've gotten it to identify:

- the comic from a random page of an obscure Russian comic

- obscure French comedy from a random clip

It was extra impressive because even reverse search from lens didn't immediately identity them

erikig · 23h ago
I read this as "AI Mode is God". Strangely I was unperturbed, quietly resigning myself to our new deity.
pjjpo · 15h ago
My first read was Google AI is God mode. They would earn back some points from me if they added a Konami code like entry point to the feature.
indigodaddy · 23h ago
Me too at first glance. Funny.
jimmydoe · 23h ago
I too feel AI mode seems to good when it knows the topic, with two caveats:

- it works when the info is either relatively well known or quite new

- no-AI mode now becomes dumber, the old trick to "grep" the internet with +/-/"" is gone

HarHarVeryFunny · 23h ago
Supposedly if you use swear words in your search string that suppresses use of AI (or at least suppresses the AI overview - maybe they don't want it to swear back at you?).

At least some of the Google search operators seem to still work, although Google themselves aren't very forthcoming about documenting these.

https://ahrefs.com/blog/google-advanced-search-operators/

jimmydoe · 18h ago
operators still work, but some results are gone, at least that's my experience has been.
Zagreus2142 · 23h ago
It feels like there is a different search use case that AI accels in while it destroys another.

- When I know so little about a subject that the concepts are all vague and I'm intellectually grasping in the dark. LLMs are good at taking my imprecise wording and orienting me towards the paths/gradients others have taken.

VS

- I know this subject well enough and instead of fumbling around I want to be able to run grep over a massive amount of open text data.

While the former mode is useful, the act of training without asking has led to the repopularization of walled gardens. Early Google felt like being able to grep every library book in existence and a 3 paragraph summary in response to very poorly worded questions is a terrible trade.

I know Simon is smart and he included his search terms with their warts to be open so I don't want to elicit shame over this. But cmon, just type out bought ("Anthropic but lots of physical books..."). Complete anecdote but I have noticed my LLM reliant friends have become way worse at texting and it feels like it's worth taking 5 seconds to try to structure your thoughts, simply for the practice

craftkiller · 23h ago
Google's AI mode recently told me the haber process was invented in 1999, and in the same paragraph told me it won the Nobel prize in 1918...
simonw · 16h ago
Was that AI mode or AI overviews?
jgalt212 · 23h ago
I hear you on this. My MO lately has been I never run anything through an AI unless I know what the wrong answer looks like or I know how to verify the answer. Under this MO, it's still a great tool just not as great as the investor class claims or wishes.
Sharlin · 1d ago
Huh, hadn't realized that Anthropic had gone full Rainbows End. RIP Vinge :(
hendersoon · 23h ago
It is pretty good, although Perplexity is better (but nominally not free, although you can get a free subscription now).

OpenAI searches are even better, but GPT5 is extremely slow with thinking. Without thinking it's roughly equivalent.

imchillyb · 23h ago
I’m not surprised by Google’s lack of search transparency. If Google told users that Google was searching an internal database linked to a particular search string there would be pitchforks and torches.
HarHarVeryFunny · 23h ago
I highly doubt they are doing that. If the quality is good then presumably it's also not the direct output of an LLM with RAG tool use.

I'd guess it might be more of a structured/agentic approach - maybe having learnt how to map "search" strings to relevant data retrieval queries, then combining/summarizing the returned results.

barrenko · 23h ago
No one cares, they are busy watching tik tok style videos and rotting their children's brains.
dbbk · 11h ago
Isn't that just... caching
bogtog · 23h ago
I assumed all of the labs were already doing something like this to cut down on costs
lxe · 23h ago
Current explosion in AI in general is "good". We tend to over-criticize and nit-pick it as what I believe is a natural response that comes from a "communal subconsciousness" -- even if us as individuals won't admit it, as a culture we are scared and averse to what's happening.
uludag · 23h ago
I think this communal subconscious response is coming from a valid place though. I will call the current explosion in AI if:

  - it causes mass unemployment and social unrest
  - leads to a further concentration of wealth and increase in wealth inequality
  - it means I have to work more, produce more, all for the same wage or less
  - it's implementation leads to large societal harms such as increased isolation/loneliness
  - it ends up being overhyped causes a large economic crisis
These scenarios aren't fantasy and a lot of them are being talked about. Technologies can just be a net bad. The critics aren't some reactionary, scared mob against the enlightened. I think a lot of us have seen the playbook tech companies use and our probabilities that a company will end up being just plain bad are a lot higher now.