Launch HN: ParaQuery (YC X25) – GPU Accelerated Spark/SQL
Here's a short demo video demonstrating ParaQuery (vs. BigQuery) on a simple ETL job: https://www.youtube.com/watch?v=uu379YnccGU
It's well known that GPUs are very good for many SQL and dataframe tasks, at least by researchers and GPU companies like NVIDIA. So much so that, in 2018, NVIDIA launched the RAPIDS program and the Spark-RAPIDS plugin (https://github.com/NVIDIA/spark-rapids). I actually found out because, at the time, I was trying to craft a CUDA-based lambda calculus interpreter…one of several ideas I didn't manage to implement, haha.
There seems to be a perception among at least some engineers that GPUs are only good for AI, graphics, and maybe image processing (maybe! someone actually told me they thought GPUs are bad for image processing!) Traditional data processing doesn’t come to mind. But actually GPUs are good for this as well!
At a high level, big data processing is a high-throughput, massively parallel workload. GPUs are a type of hardware specialized for this, are highly programmable, and (now) happen to be highly available on the cloud! Even better, GPU memory is tuned for bandwidth over raw latency, which only improves their throughput capabilities compared to a CPU. And by just playing with cloud cost calculators for a couple of minutes, it's clear that GPUs are cost-effective even on the major clouds.
To be honest, I thought using GPUs for SQL processing would have taken off by now, but it hasn't. So, just over a year ago, I started working on actually deploying a cloud-based data platform powered by GPUs (i.e. Spark-RAPIDS), spurred by a friend-of-a-friend(-of-a-friend) who happened to have BigQuery cost concerns at his startup. After getting a proof of concept done and a letter of intent... well, nothing happened! Even after over half a year. But then, something magical did happen: their cloud credits ran out!
And now, they're saving over 60% off of their BigQuery bill by using ParaQuery, while also being 2x faster -- with zero data migration needed (courtesy of Spark's GCS connector). By the way, I'm not sure about other people's experiences but... we're pretty far from being IO-bound (to the surprise of many engineers I've spoken to).
I think that the future of high-throughput compute is computing on high-throughput hardware. If you think so too, or you have scaling data challenges, you can sign up here: https://paraquery.com/waitlist. Sorry for the waitlist, but we're not ready for a self-serve experience just yet—it would front-load significant engineering and hardware cost. But we’ll get there, so stay tuned!
Thanks for reading! What have your experiences been with huge ETL / processing loads? Was cost or performance an issue? And what do you think about GPU acceleration (GPGPU)? Did you think GPUs were simply expensive? Would love to just talk about tech here!
https://x.com/HotAisle/status/1921983426972025023
The software stack for AMD is still a bit too nascent for ParaQuery's Spark engine, but certain realtime/online workloads can definitely be programmed pretty fast. They also happen to benefit greatly from the staggering levels of HBM on AMD chips. Hopefully I can take a mini-vacation later in the summer to hack on your GPUs :)
how big is their data?
A lot of BigQuery users would be surprised to find they don't need BigQuery.
This[0] post (written by founding engineer of BigQuery) has a bit of hyperbole, but this part is inline with my experience:
> A couple of years ago I did an analysis of BigQuery queries, looking at customers spending more than $1000 / year. 90% of queries processed less than 100 MB of data. I sliced this a number of different ways to make sure it wasn’t just a couple of customers who ran a ton of queries skewing the results. I also cut out metadata-only queries, which are a small subset of queries in BigQuery that don’t need to read any data at all. You have to go pretty high on the percentile range until you get into the gigabytes, and there are very few queries that run in the terabyte range.
We're[1] built on duckdb and I couldn't be happier about it. Insanely easy to get started with, runs locally and client-side in WASM, great language features.
0 - https://motherduck.com/blog/big-data-is-dead/
1 - https://www.definite.app/
It's very true that most users don't need something like BigQuery or Snowflake. That's why some startups have come up to save Snowflake cost by "simply" putting a postgres instance in front of it!
In fact, I just advised someone recently to simply use Postgres instead of BigQuery since they had <1TB and their queries weren't super intensive.
Im curious about what kinds of workloads you see GPU-accelerated compute have a significant impact, and what kinds still pose challenges. You mentioned that I/O is not the bottleneck, is that still true for queries that require large scale shuffles?
It's difficult to say what "workloads" are significant, and easier to talk about what doesn't really work AFAIK. Large-scale shuffles might see 4x efficiency, assuming you can somehow offload the hash shuffle memory, have scalable fast storage, etc... which we do. Note this is even on GCP, where there isn't any "great" networking infra available.
Things that don't get accelerated include multi-column UDFs and some incompatible operations. These aren't physical/logical limitations, it's just where the software is right now: https://github.com/NVIDIA/spark-rapids/issues
Multi-column UDF support would likely require some compiler-esque work in Scala (which I happen to have experience in).
A few things I expect to be "very" good: joins, string aggregations (empirically), sorting (clustering). Operations which stress memory bandwidth will likely be "surprisingly" good (surprising to most people).
Otherwise, Nvidia has published a bunch of really-good-looking public data, along with some other public companies.
Outside of Spark, I think many people underestimate how "low-latency" GPUs can be. 100 microseconds and above is highly likely to be a good fit for GPU acceleration in general, though that could be as low as 10 microseconds (today).
That being said, ParaQuery mainly uses T4 and L4 GPUs with "just" ~300 GB/s bandwidth. I believe (correct me if I'm wrong) that should be around a 64-core VM, though obviously dependent on the actual VM family.
disclaimer: my team is working on this very problem as well, as I was a speaker at VeloxCon.
FPGAs... I somehow highly doubt their efficiency in terms of being the "core" (heh) processor. However, "compute storage" with FPGAs right next to the flash is really interesting.
How would you contrast it against HeavyDB?
https://github.com/heavyai/heavydb
- We're fully compatible with Spark SQL (and Spark). Meaning little to no migration overhead.
- Our focus is on distributed compute first.
- That means ParaQuery isn't a database, just an MPP engine (for now). Also means no data ingestion/migration needed.
http://heterodb.github.io/pg-strom/
Btw, interesting thing they said here: "By utilization of GPU (Graphic Processor Unit) device which has thousands cores per chip"
It's more like "hundreds", since the number of "real" cores is like (CUDA cores / 32). Though I think we're about to see 1k cores (SMSPs).
That being said, I do believe CUDA cores have more interesting capabilities than a typical vector lane, i.e. for memory operations (thank the compiler). Would love to be corrected!
SQL on GPUs is definitely a research classic, dating back to 2004 at least: https://gamma.cs.unc.edu/DB/
Set Theory is the classical foundation of SQL:
https://www.sqlshack.com/mathematics-sql-server-fast-introdu...
It's analogous to how functional programming expressed through languages like lisp is the classical foundation of spreadsheets.
I believe that skipping first principles (sort of like premature optimization) is the root of all evil. Some other examples:
- If TCP had been a layer above UDP instead of its own protocol beside it, we would have had real peer to peer networking this whole time instead of needing WebRTC.
- If we had a common serial communication standard analogous to TCP for sockets, then we wouldn't need different serial ports like USB, Thunderbolt and HDMI.
- If we hid the web browser's progress bar and used server-side rendering with forms, we could implement the rich interfaces of single-page applications with vastly reduced complexity by keeping the state, logic and validation in one place with no perceptible change for the average user.
- If there was a common scripting language bundled into all operating systems, then we could publish native apps as scripts with substantially less code and not have to choose between web and mobile for example.
- If we had highly multicore CPUs with hundreds or thousands of cores, then multiprocessing, 3D graphics and AI frameworks could be written as libraries running on them instead of requiring separate GPUs.
And it's not just tech. The automative industry lacks standard chassis types and even OEM parts. We can't buy Stirling engines or Tesla turbines off the shelf. CIGS solar panels, E-ink displays, standardized removable batteries, thermal printers for ordinary paper, heck even "close enough" contact lenses, where are these products?
We make a lot of excuses for why the economy is bad, but just look at how much time and effort we waste by having to use cookie cutter solutions instead of having access to the underlying parts and resources we need. I don't think that everyone is suddenly becoming neurodivergent from vaccines or some other scapegoat, I think it's just become so obvious that the whole world is broken and rigged to work us all to the grave to make some guy rich that it's giving all of us ADHD symptoms from having to cope with it.
It makes sense to have two specialized systems: a low-latency system, and a high-throughput system, as it's a real tradeoff. Most people/apps need low-latency.
As for throughput and efficiency... turns out that shaving off lots of circuitry allows you to power less circuitry! GPUs have a lot of sharing going on and not a lot of "smarts". That doesn't even touch on their integrated throughput optimized DRAM (VRAM/HBM). So... not quite. We'd still be gaming on GPUs :)
I'm not very familiar with Gluten, but I'll still comment on the CPU side though, assuming that one of Gluten's goals is to use the full vector processing (SIMD) potential of the CPU. In that case, we'd still be memory(-bandwidth)-bound, not to mention the significantly lower FLOPs of the CPU itself. If we vectorize Spark (or any MPP) for efficient compute, perhaps we should run it on hardware optimized for vectorized, super-parallel, high-throughput compute.
Also, there's nothing which says we can't use Gluten to have even more CPU+GPU utilization!
How is it priced? I couldn't see anything on the site.
However, for deployments to the customer's cloud, it would be a stereotypical enterprise license + support.
Can't wait to actually add an FAQ to the site, hopefully based off the questions asked here. Pricing is one of the things preventing me from just allowing self-serve, since it has to be stable, sustainable, and cheap!
Also, with the GPU clouds, pricing would have to be different per cloud, though I guess I can worry about that later. Would be crazy cheap(er) to process on them.
As far as I know, GPUs are definitely still being used in crypto/web3... and AI for that matter :P
8x faster!