Show HN: Doctor – tool to crawl and index websites and MCP server for LLM agents

Comments (2)

kixpanganiban · 9h ago

Hi! I wrote Doctor because I keep struggling with grounding on docs when working with agentic code editing (ex Roo, Claude Code).

Doctor uses crawl4ai to crawl websites, and then chunks and embeds them with langchain + litellm + openai, and finally stores all the vectors in duckdb. This allows your LLM to query the docs using semantic search over MCP, giving it grounded and up-to-date information for the things you're working on.

It requires an OpenAI key for the embedding process, but I'm working on giving users options in the future (different providers, local embedding using something like DPR or other transformer libs, etc.)

downrightmike · 9h ago

And now we have a new bot for cloudflare to send to the infinite AI crawler labyrinth

Firefox moves to GitHub (github.com)

A programming language made for me (zylinski.se)

Ask HN: How are you acquiring your first hundred users?

FastVLM: Efficient vision encoding for vision language models (github.com)

Open Hardware Ethernet Switch project, part 1 (serd.es)

TransMLA: Multi-head latent attention is all you need (arxiv.org)

Anti-Personnel Computing (2023) (erratique.ch)

The Microsoft unit working to thwart hackers (bloomberg.com)

The Barbican (arslan.io)

Air Traffic Control (computer.rip)

15 Years of Shader Minification (ctrl-alt-test.fr)

Alephic Writing Style Guide (alephic.com)

Trial by Fire: The crash of Aeroflot flight 1492 (admiralcloudberg.medium.com)

Revisiting Image Maps (css-tricks.com)

Can you trust that permission pop-up on macOS? (wts.dev)

A conversation about AI for science with Jason Pruet (lanl.gov)

RIP Usenix ATC (bcantrill.dtrace.org)

Understanding LucasArts' iMUSE System (github.com)

Show HN: Why hanging out on Hacker News is a good use of your time? (parliant.ai)

A community-led fork of Organic Maps (comaps.app)

How to avoid P hacking (nature.com)

HealthBench – An evaluation for AI systems and human health (openai.com)

University of Texas-led team solves a big problem for fusion energy (news.utexas.edu)

Reviving a modular cargo bike design from the 1930s (core77.com)

Launch HN: ParaQuery (YC X25) – GPU Accelerated Spark/SQL

Writing N-body gravity simulations code in Python (alvinng4.github.io)

Nextcloud cries foul over Google Play Store app rejection (theregister.com)

Ruby 3.5 Feature: Namespace on read (bugs.ruby-lang.org)

NASA study reveals Venus crust surprise (science.nasa.gov)

Offline vs. online ML pipelines (decodingml.substack.com)

The Beam (erlang-solutions.com)

Low-intensity transcranial focused ultrasound amygdala neuromodulation (nature.com)

Continuous glucose monitors reveal variable glucose responses to the same meals (examine.com)

Wtfis: Passive hostname, domain and IP lookup tool for non-robots (github.com)

FedRAMP 20x – One Month in and Moving Fast (fedramp.gov)

Universe expected to decay in 10⁷⁸ years, much sooner than previously thought (phys.org)

Legion Health (YC S21) is hiring engineers to help fix mental health with AI (workatastartup.com)

Policy of Transience (chiark.greenend.org.uk)

Demonstrably Secure Software Supply Chains with Nix (nixcademy.com)

Build your own Siri locally and on-device (thehyperplane.substack.com)

Show HN: Lumoar – Free SOC 2 tool for SaaS startups (lumoar.com)

Show HN: Airweave – Let agents search any app (github.com)

Terence Tao started a YouTube channel (youtube.com)

A Typical Workday at a Japanese Hardware Tool Store [video] (youtube.com)

New Bill Would Make All Pornography a Federal Crime in the U.S. (404media.co)

Spade Hardware Description Language (spade-lang.org)

Plain Vanilla Web (plainvanillaweb.com)

Continuous Thought Machines (pub.sakana.ai)

The Acid King (2001) (rollingstone.com)

Why GADTs matter for performance (2015) (blog.janestreet.com)

Show HN: Doctor – tool to crawl and index websites and MCP server for LLM agents

Comments (2)