Would love to see a system that blends cheap lexical (Fulltext Search) or semantic/vector search using SQLite and chooses the best approach given the input.
bob1029 · 4h ago
If you want the best possible solution vertical for most business, I'd be looking at using Lucene for FTS duty.
Having the FTS engine provide a google-style snippet of the most relevant document chunk is the holy grail for RAG applications. Lucene does this kind of thing better than anyone else:
This is for LLMs. In general RAG takes a user prompt and uses it to find potentially relevant documents in the database. It then enriches the original prompt with those documents so that the LLM has context that wasn't in its training dataset.
Octplane · 8h ago
RAG -> Vector search -> means that your documents are not indexed as full text but as Vectorized objects which mean that then you can search using concepts instead of exacts strings you would use with a regular "Fulltext search".
This makes the search less precise and more powerful at the same time (ie it could look clever to some extent).
rcarmo · 5h ago
This and SQLite-vec (or whatever extension is trendy these days) can do a lot in a very limited amount of compute.
almosthere · 8h ago
Sqlite has an embedding search? Or is that being provided by this tool?
ethan_smith · 3h ago
SQLite itself doesn't have native embedding search, but extensions like sqlite-vss and sqlite-vectorize add vector similarity search capabilities to SQLite.
Having the FTS engine provide a google-style snippet of the most relevant document chunk is the holy grail for RAG applications. Lucene does this kind of thing better than anyone else:
https://lucene.apache.org/core/8_0_0/highlighter/org/apache/...
It is also very easy to customize this engine and align the document tokenization & indexing concerns with your specific retrieval scenarios.
Anyone want to help out?
This makes the search less precise and more powerful at the same time (ie it could look clever to some extent).