> Neither Cloudflare, nor any other service, will ever be able to block all scrapers. They can make their operations more expensive,
Cloudflare presents like single platform for crawlers. The get the same amount of data as platforms to bock crawlers they don't want. Other big platforms can prevent scrapers effectively when they don't want them Google, Facebook. etc. Nifty new scraper might crawl few million url's before it's detected.
mmarian · 22m ago
Hey! Sorry, didn't quite catch what you meant.
Is it that Cloudlare can always spot crawlers because of the amount of data they collect? Or is it there's always a nifty new scraper that will get away with it?
Cloudflare presents like single platform for crawlers. The get the same amount of data as platforms to bock crawlers they don't want. Other big platforms can prevent scrapers effectively when they don't want them Google, Facebook. etc. Nifty new scraper might crawl few million url's before it's detected.
Is it that Cloudlare can always spot crawlers because of the amount of data they collect? Or is it there's always a nifty new scraper that will get away with it?