Ask HN: Why hasn't x86 caught up with Apple M series?

I spent a lot of time trying to find a good solution to this problem and failed, so what I ended up doing was to give up and remove my sites from the public web entirely.

I'm eager for a good solution that will allow me to put them back, but I'm doubtful that's going to happen. In any case, I'm extremely interested in other people's replies here. Maybe there's a solution that I haven't been able to find!

johng · 1h ago

Some of our sites have been getting absolutely hammered by the AI bots -- so much so they are taking down the sites. Even with cloudflare protection and caching. The only thing We've been able to do so far is tell Cloudflare to block all AI bots, modify the robots.txt and even then we've had to manually identify IP addresses and bots that ignore all of the above and block them specifically or at the ASN level.

Cloudflare makes doing this kind of stuff easy but I would hate to have to do this manually on a webserver. And I don't like the idea of how much of the internet already relies on Cloudflare.

bediger4000 · 2h ago

I have a lot of them in robots.txt as disallow /, of course. I have several getting 404 on any request whatsoever, Meta's AI crawler, Bytespider mainly, via Apache httpd mod_rewrite.

Ask HN: GitHub Copilot down?

Ask HN: Why hasn't x86 caught up with Apple M series?

Ask HN: What measures are you taking to stop AI crawlers?

Ask HN: Did modern AI's coding abilities make you lose interest in programming?

Ask HN: How can I recover and run my old mobile game from the 2010s?

iOS 26 Launches Sept 15 – Even GPT-5 Doesn't Know It Exists

Ask HN: Does sentience put stress on the brain?

Ask HN: What to Do with Old iPads?

Units of Economics of LLMs. Reply to Ed Zitron's "AI Is a Money Trap"

Ask HN: Is there a temp phone number like temp email?

Out of curiosity: what kind of people use this "forum" (I mean Hacker News)?

Ask HN: What should I use to run React Native tests on a device?

Ask HN: Why are so many services rejecting Google Voice numbers for signups?

Ask HN: Why can't ChatGPT count to a million?

Ask HN: Any experienced devs who use AI extensively in their work?

Ask HN: Best codebases to study to learn software design?

Stop squashing your commits. You're squashing your AI too

Ask HN: How are you attributing your AI usage when developing software?

Patient Lisp Hacker Seeks Same for Long Walks Through IPL-V Code

Ask HN: Has anyone else used online communities that are archetypically "savvy"?

High rate of LLM (GPT5) hallucinations in dense stats domains (cricket)

Ask HN: Are AI filters becoming stricter than society itself?

Ask HN: What are you working on (August 2025)?

429 Too Many Requests from registry.npmjs.org

Ask HN: I just abandoned my PyCharm subscription, what should I use now?

How can a mutex in Wine be faster than a native one on Linux

Ask HN: What is wrong with modern software development

Ask HN: Recommandation for an Ergonomic Keyboard?

Ask HN: Someone has committed 20K+ LoC to a PR, exhausting my CI & AI workflows

Ask HN: How is Nvidia streaming 4k games while remote control sw is low q lag

Ask HN: How do you find early stage startups to join

Ask HN: Best Marketplaces for Used Servers?

Ask HN: How can I trace what user queries make AI bots crawl my site?

Ask HN: Why do people hate on Sabine Hossenfelder so much?

Ask HN: What is the biggest problem LLMs solved in your life/work?

Problem with Payment Gateways

ASK HN: AI in high school. Will teachers and schools have to compensate?

Ask HN: What is your source for answers?

Ask HN: Devices to allow children to listen to podcasts on my local network?

DSPy GEPA Example: Listwise Reranker

Ask HN: Does using public transportation make you more creative than driving?

Ask HN: No easy way for tvOS to display long documents (e.g., terms of service)?

Ask HN: Why aren't Android manufacturers interested in GrapheneOS?

Ask HN: What measures are you taking to stop AI crawlers?

Comments (3)