Ask HN: Has anybody built search on top of Anna's Archive?
287 points by neonate 4d ago 146 comments
Ask HN: Is synthetic data generation practical outside academia?
4 points by cpard 1d ago 9 comments
The /llms.txt file, helping language models use your website
4 mooreds 1 6/7/2025, 8:45:33 PM github.com ↗
Comments (1)
TimorousBestie · 4h ago
LLMs broadly speaking disregard robots.txt, so I don’t see why they would bother regarding this.