AI overviews cause massive drop in search clicks

At some point, an article about how Google was showing this crap made it to the top of the rankings and they started taking the overview from it rather than the original Quora answer it used before. Somehow it still got it wrong, and just lifted the absurd answer from the article rather than the part where the article says it’s very wrong.

Amusingly, they now refuse to show an AI answer for that particular search.

1. Here's the answer (but it's misinformation) 2. Here are some websites that look like they might have the answer

?

But SEO slop machines have made it so hard to find the good websites without putting in more legwork than makes sense a lot of the time. Funnily enough, this makes AI look like a good option to cut through all the noise despite its hallucinations. That's obviously not acceptable when it comes to food safety concerns though.

First result under the overview is the National Pork Board, shows the answer above the fold, and includes visual references: https://pork.org/pork-cooking-temperature/

Most of the time if there isn't a straightforward primary source in the top results, Google's AI overview won't get it right either.

Given the enormous scale and latency constraints they're dealing with, they're not using SOTA models, and they're probably not feeding the model 5000 words worth of context from every result on the page.

Maybe they could just show the links that match your query and skip the overview. Sounds like a billion-dollar startup idea, wonder why nobody’s done it.

When our grandmothers and grandfathers were growing up, there was a real threat to their health that we don’t face anymore. No, I’m not talking about the lack of antibiotics, nor the scarcity of nutritious food. It was trichinosis, a parasitic disease that used to be caught from undercooked pork.

The legitimate worry of trichinosis led their mothers to cook their pork until it was very well done. They learned to cook it that way and passed that cooking knowledge down to their offspring, and so on down to us. The result? We’ve all eaten a lot of too-dry, overcooked pork.

But hark! The danger is, for the most part, past, and we can all enjoy our pork as the succulent meat it was always intended to be. With proper temperature control, we can have better pork than our ancestors ever dreamed of. Here, we’ll look at a more nuanced way of thinking about pork temperatures than you’ve likely encountered before."

Sorry, what temperature was it again?

Luckily there's the National Pork Board which has bought its way to the top, just below the AI overview. So this time around I won't die from undercooked pork at least.

In your case, I think that it is just the interrogation point in itself at the end that somehow has an impact on the results you see.

You might think that's the correct way to do it, but there is likely much more to it than it seems.

If it wasn't tricky at all you'd bet they would've done it already to maximize revenue.

It never will. By disincentivizing publishers they're stripping away most of the motivation for the legitimate source content to exist.

AI search results are a sort of self-cannibalism. Eventually AI search engines will only have what they cached before the web became walled gardens (old data), and public gardens that have been heavily vandalized with AI slop (bad data).

Google search, as others have mentioned in this thread, increasingly fails to give me high-quality material anyway. Mostly it's just pages of SEO spam. I prefer that the LLM eat that instead of me (just spit back up the relevant stuff, thankyouverymuch).

Honestly though, increasingly the internet for me is 1) a distraction from doing real work 2) YouTube (see 1) and 3) a wonderful library called archive.org (which, if I could grab a local snapshot would make leaving the internet altogether much, much easier).

- Hobbyist site

- Forum or UGC

- Academic/gov

- Quality news which is often paywalled

Most of that stuff doesn't depend on ad clicks. The things that do depend on ad clicks are usually infuriating slop. I refuse to scroll through three pages of BS to get to the information I want.

If you do a Google (or other engine) search, you have to invest time pawing through the utter pile of shit that Google ads created on the web. Info that's hidden under reams of unnecessary text, potentially out of date, potentially not true; you'll need to evaluate a list of links and, probably, open multiple of them.

If you do an AI "search", you ask one question and get one answer. But the answer might a hallucination or based on incorrect info.

However, a lot of the time, you might be searching for something you already have an idea of, whether it's how to structure a script or what temperature pork is safe at; you can use your existing knowledge to assess the AI's answer. In that case the AI search is fast.

The rest of the time, you can at least tell the AI to include links to its references, and check those. Or its answer may help you construct a better Google search.

Ultimately search is a trash heap of Google's making, and I have absolute confidence in them also turning AI into a trash heap, but for now it is indeed faster for many purposes.

First, in the pre training stage humans curate and filter the data thats actually used for training.

Then in the fine tuning stage people write ideal examples to teach task performance

Then there is reinforcement learning from human feedback RLHF where people rank multiple variations of the answer an AI gives, and thats part of the reinforcement loop

So there is really quite a bit of human effort and direction that goes into preventing the garbage-in garbage-out type situation you're referring to

But they're not often confidently wrong like AI summaries are.