A simple search engine from scratch

116 bertman 8 5/20/2025, 9:58:56 AM bernsteinbear.com ↗

Comments (8)

franczesko · 2h ago
On the topic of search engines, I really liked classes by David Evans. The task was also building a simple search engine from scratch. It's really for beginners, as the emphasis is on coding in general, but I've found it to be very approachable.

https://www.cs.virginia.edu/~evans/courses/

ktallett · 44m ago
I always wonder if the days of search engines for specific topics could return. With LLM's providing less than accurate results in some areas, and Google, bing, etc being taken over by adverts or well organised SEO, there feels like a place for accurate, specialised search.
datadrivenangel · 40m ago
The curation of an index of resources is what's needed for niche search
dcist · 34m ago
WestLaw and Lexis Nexis provide this for legal search, but quite frankly, these services are subpar. It's amazing that these two companies rake in hundreds of millions but they are both slower than Google, Bing, Yandex, or any LLM service (ChatGPT, Claude, Gemini, etc.) while scouring a universe of text that is orders of magnitude smaller. The user experience is also terrible (you have to login and specify a client each and every time you attempt to use the service and both services log you out after a short -- in my opinion -- period of inactivity, creating friction and needless annoyance to the user). There's an opportunity there.
ktallett · 28m ago
I haven't personally used the mentioned services as they aren't in my field, however what is the accuracy of their results? Are they double checked? I don't find LLMs particularly accurate in my field (that's being kind), if anything I find they make up sources that simply don't exist.

I mean poor UX has no excuse but slow speed can be reasoned if it makes the quality of the service better.

ordersofmag · 12m ago
Here’s a place to start if you want to go down the rabbit hole of how search at places like this is approached. https://haystackconf.com/us2022/talk-12/

https://www.youtube.com/watch?v=9vCMFIJRiKk

fanwood · 15m ago
I already directly search on Wikipedia for most topics (with a search shortcut on URL bar)
ktallett · 13m ago
Wikipedia is useful up to a point for sure. I feel whether it could be a expansion of Wikipedia in it's current use case, but for emerging research and niche topics it can sometimes be less useful.