Cannot wait for OrioldeDB to reach General Availability. Postgres needs options for open-source separation of storage and compute.
dkhenry · 2d ago
OrioleDB isn't a separation of storage and compute, its a more efficient storage engine for Postgres to replace the existing HEAP engine. This is like how in MySQL we could swap MyISAM for InnoDB and eventually RocksDB.
I did some benchmarks on it previously to show how much of an improvement it gives over the stock HEAP engine
EDIT: correct link to the public dashboard below, thanks for the heads up @kiwicopple
Is the need for Oriole negated by using a system that separates storage from compute like Neon, Xata?
nikita · 2d ago
(Neon CEO)
Not really. OrioleDB solve the vacuum problem with the introduction of the undo log. Neon gives you scale out storage which is in a way orthogonal to OrielDB. With some work you can run OrioleDB AND neon storage and get benefits of both.
akorotkov · 2d ago
> OrioleDB solve the vacuum problem with the introduction of the undo log.
Way more than just this!
> With some work you can run OrioleDB AND neon storage and get benefits of both.
This would require significant design work, given that significant OrioleDB benefits are derived from row-level WAL.
tudorg · 2d ago
Answering on behalf of Xata, it is orthogonal. I'm curious to try out Oriole on our platform when I get some time.
kiwicopple · 2d ago
fwiw I couldn't access your airtable link, but I found this one online:
Neon is indeed unrelated to OreoleDB, but Neon does also provide the separation of storage and compute in Postgres which GP asked about ("Postgres needs options for open-source separation of storage and compute"). A mention of Neon (which is Apache 2 licensed) therefore isn't totally unwarranted.
rubenvanwyk · 3h ago
I understand Neon is open source and I think it’s an awesome product, but apart from the risks associated with longevity of open source once a company gets acquired - although the storage engine is open source, the control plane isn’t and is non-trivial to implement oneself. Orioldedb is positioned as a Postgres extension, which is must easier to setup (even on an existing operating database) than migrating to a completely different architecture that Neon provides.
apavlo · 2d ago
"Bridged Indexes" is a non-standard term. These are just secondary indexes using logical pointers with a mapping index. IIRC, Oracle, Hana, and HyPer do the same thing.
> secondary indexes using logical pointers with a mapping index.
rolls off the tongue.
mattashii · 1d ago
It's more than just one mapping index, though.
The CMU paper indicates the logical keys are either TupleID or Primary Key, while the bridged index is actually a TupleID that resolves to a Primary Key, which resolves to the actual tuple - one more level than indicated by 6.1's explanation of logical pointers.
I did some benchmarks on it previously to show how much of an improvement it gives over the stock HEAP engine
EDIT: correct link to the public dashboard below, thanks for the heads up @kiwicopple
https://airtable.com/app7jp5t0dEHyDpa8/shr00etqywoDW2N6N
https://www.orioledb.com/docs/usage/decoupled-storage
Not really. OrioleDB solve the vacuum problem with the introduction of the undo log. Neon gives you scale out storage which is in a way orthogonal to OrielDB. With some work you can run OrioleDB AND neon storage and get benefits of both.
Way more than just this!
> With some work you can run OrioleDB AND neon storage and get benefits of both.
This would require significant design work, given that significant OrioleDB benefits are derived from row-level WAL.
https://airtable.com/app7jp5t0dEHyDpa8/shr00etqywoDW2N6N
thanks for running the benchmarks, it helps to have external parties verifying the progress
Althoug with them being recently acquired by Databricks it remains to be seen how the open source version will fare.
This is unrelated to NeonDB. OrioleDB has been acquired by Supabase. https://supabase.com/blog/supabase-acquires-oriole
Source: https://db.cs.cmu.edu/papers/2017/p781-wu.pdf (see Table 1 + Section 6.1)
rolls off the tongue.
The CMU paper indicates the logical keys are either TupleID or Primary Key, while the bridged index is actually a TupleID that resolves to a Primary Key, which resolves to the actual tuple - one more level than indicated by 6.1's explanation of logical pointers.