Keeping the Web Up Under the Weight of AI Crawlers