Not trying to be snarky, just curious -- How is this different from TurboPuffer and other serverless, object storage backed vector DBs?
ge96 · 1h ago
M is minutes
HarHarVeryFunny · 40m ago
I was starting to think this was impressive, if not impossible. 1B vectors in 48 MB of storage => < 1 bit per vector.
Maybe not impossible using shared/lossy storage if they were sparsely scattered over a large space ?
But anyways - minutes. Thanks.
Edit: Gemini suggested that this sort of (lossy) storage size could be achieved using "Product Quantization" (sub vectors, clustering, cluster indices), giving an example of 256 dimensional vectors being stored at an average of 6 bits per vector, with ANN being one application that might use this.
stevemk14ebr · 23m ago
Thank you, title needs edited.
ikanade · 58m ago
Legend
l5870uoo9y · 36m ago
Thankfully not months.
softwaredoug · 35m ago
Oh the horrors of search indexing Ive seen... including weeks / months to rebuild an index.
ashvardanian · 29m ago
Very curious about the hardware setup used for this benchmark!
OutOfHere · 50m ago
Proprietary closed-source lock-in. Nothing to see here.
CuriouslyC · 24m ago
Seriously. The amount of lift a SaaS product needs to give me is insane for me to even bother evaluating it, and there's a near zero percent chance I'll use it in my core.
stronglikedan · 21m ago
Nothing for you to see here. Surely you just aren't their target customer.
OutOfHere · 2m ago
Who is? Who really needs to index 1 billion new vectors every 48 minutes, or perhaps equivalently 1 million new vectors every 3 seconds? And why is this problem not embarrassingly parallel anyway? (It is, by sharding it.)
Maybe not impossible using shared/lossy storage if they were sparsely scattered over a large space ?
But anyways - minutes. Thanks.
Edit: Gemini suggested that this sort of (lossy) storage size could be achieved using "Product Quantization" (sub vectors, clustering, cluster indices), giving an example of 256 dimensional vectors being stored at an average of 6 bits per vector, with ANN being one application that might use this.