GizmoSQL completed the 1T row challenge

1 philbe77 0 7/3/2025, 1:42:26 PM

GizmoSQL completed the One-Trillion Row Challenge! We powered through 1 TRILLION rows using GizmoSQL ( DuckDB + Apache Arrow Flight SQL) with AWS Graviton4– on a single r8gd.metal-48xl instance:

Instance: r8gd.metal-48xl (Graviton4) 192 vCPUs, 1536 GiB RAM 11.4TB RAID0 NVMe on AWS $14.11 / hr on-demand $2.82 / hr spot

Dataset: 2.3TB Parquet files (100,000 files with 10,000,000 rows each!) Copy from S3: 11m 24s Challenge Query (cold start): 2m 22s Challenge Query (warm start): 2m 09s Total cost: ~$0.11 (spot) per query And yes...

SELECT COUNT(*) on 1 trillion rows? Just 21.8 seconds.

Why it matters: GizmoSQL lets you query massive Parquet datasets interactively, directly, and efficiently – using the DuckDB SQL engine, and the Arrow Flight SQL protocol for blazing-fast client connections.

No complex Spark or distributed compute cluster setup. No hidden egress fees. Just pure, fast SQL on massive data.

Check out open-source GizmoSQL here: https://github.com/gizmodata/gizmosql and https://gizmosql.com

GizmoSQL is free for self-hosting!

Full 1TRC details, scripts, and results: https://github.com/coiled/1trc/issues/7

Converge (YC S23) well-capitalized New York startup seeks product developers (runconverge.com)

Kyber (YC W23) Is Hiring Enterprise BDRs (ycombinator.com)

MindsDB (YC W20) is hiring an AI solutions engineer (job-boards.greenhouse.io)

Recurse Center (YC S10) Is Hiring a Career Facilitator (recurse.notion.site)

Cua (YC X25) is hiring an engineer (ycombinator.com)

Noloco (YC S21) is hiring a founder's associate in Barcelona (ycombinator.com)

14.ai (YC W24) hiring founding engineers in SF to build a Zendesk alternative (14.ai)

Lago (Open-Source Usage Based Billing) is hiring for ten roles (ycombinator.com)

Spark AI (YC W24) is hiring a full-stack engineer in SF (founding team) (ycombinator.com)

Bitmovin (YC S15) Is Hiring a Junior Solutions Engineer in Denver (bitmovin.com)

SigNoz (YC W21, Open Source Datadog) Is Hiring DevRel Engineers (Remote)(US) (ycombinator.com)

AccessOwl (YC S22) is hiring an Elixir Engineer to connect 100s of SaaS (ycombinator.com)

FurtherAI (YC W24) Is Hiring for Software and AI Roles (ycombinator.com)

Yarn (YC W24) is hiring engineers in NYC (ycombinator.com)

Expand.ai (YC S24) is hiring a founding engineer

Optifye.ai (YC W25) is hiring a back end engineer

Kastle (S24) is hiring an engineer (ycombinator.com)

Weave (YC W25) is hiring a founding AI engineer (ycombinator.com)

Qfex (YC X25) – Back End Engineer for a 24/7 Stock Exchange (ycombinator.com)

Attimet (YC F24) – Quant Trading Research Lab – Is Hiring Founding Engineer (ycombinator.com)

Jiga (YC W21) Is Hiring Software Engs to Make Life of Mech Engs Easier (workatastartup.com)

Foundry (YC F24) Hiring Early Engineer to Build Web Agent Infrastructure (ycombinator.com)

Blaze (YC S24) Is Hiring (ycombinator.com)

Infracost (YC W21) is hiring software engineers (GMT+2 to GMT-6) (infracost.io)

Solidroad (YC W25) Is Hiring (solidroad.com)

Kyber (YC W23) Is Hiring a Technical Account Manager (ycombinator.com)

Roundtable (YC S23) Is Hiring a President / CRO (ycombinator.com)

Roame (YC S23) Is Hiring (ycombinator.com)

GauntletAI (YC S17): All expenses paid AI training and guaranteed $200k+ job (gauntletai.com)

SchemeFlow (YC S24) Is Hiring an Engineer (London) to Speed Up Construction (ycombinator.com)

GizmoSQL completed the 1T row challenge

Comments (0)