We recently migrated from RDS to Iceberg and the savings have been mind-blowing. That said, it's like a deconstructed database and everything you took for granted is now on the outside. Sometimes it's cool, like instead of a Cron inside the database you have a cloud Cron on the outside which is more natively monitor able. Some of it sucks like write contention on inserts so you need a write coordinator lock, and external vacuums.
However, the savings are extreme. Better cost per byte stored. Better compression. Less table bloat.
The universal compatibility has not materialised yet. We are still waiting for duckdb and click house predicate pushdown but if that appears we will be able to have specialized query engines over a generic storage layer which would be amazing. Tools like Nimtable underline what compatability unlocks.
whyho · 10h ago
Thought this was about the programming language nim.
graemep · 8h ago
Nim is a bad name for a language TBF. Its too short and a common sound and letter sequence (e.g. in words like "nimble", "numbus", "denim"). It is going to be used elsewhere.
I far preferred the original name.
esafak · 10h ago
Same. Why not 'Icetable'?
Imustaskforhelp · 10h ago
Me too!!
bradhe · 10h ago
Ah this looks great. Whence been doing a lot of work with Iceberg lately and a lot of the basics are still missing in the ecosystem. Like, for instance, a really easy way to explore your tables.
It’ll be interesting to see how table maintenance works out. Everyone is implementing their own. Turns out storage engines have been doing a lot of work for us for the past 50 years…
Anyway, seems like pairing this with Lakekeeper would be great.
nwsm · 1h ago
Well, what query engine do you use?
We use normal RDB tool like DBeaver against Trino JDBC and it works fine.
konsalexee · 7h ago
Amazing work! An interactive demo of how it work, before investing time to set things up would be really cool
The universal compatibility has not materialised yet. We are still waiting for duckdb and click house predicate pushdown but if that appears we will be able to have specialized query engines over a generic storage layer which would be amazing. Tools like Nimtable underline what compatability unlocks.
I far preferred the original name.
It’ll be interesting to see how table maintenance works out. Everyone is implementing their own. Turns out storage engines have been doing a lot of work for us for the past 50 years…
Anyway, seems like pairing this with Lakekeeper would be great.
We use normal RDB tool like DBeaver against Trino JDBC and it works fine.