Sounds interesting, though that durability tradeoff is not one that I’d think most people/applications want to make. When you save something to the DB, you generally want that to mean it’s been durably stored.
Are there specific applications you’re targeting where latency matters more than durability?
mehrant · 1h ago
Thanks for the comment. :)
The target use cases include:
1- Session stores (can be reconstructed from auth service)
2- leaderboards/counters (recent scores/counters can be recalculated)
3- Real-time analytics/metrics (losing ~100ms of metrics is acceptable)
4- Caching layers with upstream persistence
5- High-frequency systems where latency > everything
I generally think that for KV stores, there are more use cases that can accept this _slightly_ relaxed durability model than not. of course this isn't the case for a main DB. KV stores often handle derived data, caches, or state that can be rebuilt.
That said, for cases needing stronger durability, you can call flush_all() after critical operations - gives you fsync-level guarantees. Also considering adding a "sync" or "Full ACID" mode that auto-flushes on every write for users who want strong durability.
The philosophy is: make the fast path really fast for those who need it, but provide escape hatches for stronger guarantees when needed.
_davide_ · 5h ago
Nice! On paper looks really promising!
There is actual need for embedded databases as SQLite is painful for highly concurrent programs (I actually hit this issue in rust)
mehrant · 5h ago
Thanks! yeah, SQLite's write lock is painful for concurrent apps.
I'm comfortable with kernel development, so I brought some of those patterns here - RCU-style lock-free reads, per-CPU inspired sharded buffers, and io_uring for kernel-bypass I/O.
would love to hear your thoughts if you had the chance to give it a spin :)
Are there specific applications you’re targeting where latency matters more than durability?
The target use cases include:
1- Session stores (can be reconstructed from auth service) 2- leaderboards/counters (recent scores/counters can be recalculated) 3- Real-time analytics/metrics (losing ~100ms of metrics is acceptable) 4- Caching layers with upstream persistence 5- High-frequency systems where latency > everything
I generally think that for KV stores, there are more use cases that can accept this _slightly_ relaxed durability model than not. of course this isn't the case for a main DB. KV stores often handle derived data, caches, or state that can be rebuilt.
That said, for cases needing stronger durability, you can call flush_all() after critical operations - gives you fsync-level guarantees. Also considering adding a "sync" or "Full ACID" mode that auto-flushes on every write for users who want strong durability.
The philosophy is: make the fast path really fast for those who need it, but provide escape hatches for stronger guarantees when needed.