Minor quibble but: probably get another 5-15% by upgrading from oldoldstable Debian 11/Bullseye with kernel 5.10 to something modern like Debian 13/Trixie.
Edit: oh these are containers, right; the difference will be far less since the kernel is determined by the host. Still, I find it sad to see old software versions like this used.
_peregrine_ · 1h ago
nice one
jandrewrogers · 2h ago
I don't know about ClickHouse specifically but these days it isn't that hard to ingest >10M rows/sec per server, including parsing, processing, indexing, and storage. You can just throw hardware at it.
_peregrine_ · 1h ago
yeah I mean that's basically what Javi talks about in the post... if you can throw hardware at it you can scale it (ingestion scales linearly with shards)
but the post has some interesting thoughts on how you do the high-scale ingestion while also handling background merge processes, reads, etc.
Edit: oh these are containers, right; the difference will be far less since the kernel is determined by the host. Still, I find it sad to see old software versions like this used.
but the post has some interesting thoughts on how you do the high-scale ingestion while also handling background merge processes, reads, etc.