Thank you for writing this. This comes up constantly, and it'll be great to have another reference to cite.
Another interesting thing about TPC-C is how the cross-warehouse contention was designed. About 10% of new order transactions need to do a cross-warehouse transaction. If you can keep up with the workload, then the rate of contention is relatively low; most of the workload isn't pushing on concurrency control. If, however, you fall behind, and transactions start to take too long, then the contention will pile up.
When you run without the keying time, it turns out that concurrency control begins to dominate. For distributed databases, concurrency control and deadlock detection is fundamentally more expensive than it can be for single-node databases -- so it makes sense that a classically single-node database would absolutely trounce distributed databases. I like to think of tpcc "nowait" as really a benchmark of concurrency control because, due (I believe) to Amdahl's law the majority of its execution time ends up just in the contended portion of the workload.
Also very interesting that, as Justin points out, the workload sets up the warehouses so there is never cross-node contention. That's wild! I'm glad they didn't go and benchmark against even more distributed databases (like YugabyteDB, Spanner, or CockroachDB) and call it a fair fight.
Another interesting thing about TPC-C is how the cross-warehouse contention was designed. About 10% of new order transactions need to do a cross-warehouse transaction. If you can keep up with the workload, then the rate of contention is relatively low; most of the workload isn't pushing on concurrency control. If, however, you fall behind, and transactions start to take too long, then the contention will pile up.
When you run without the keying time, it turns out that concurrency control begins to dominate. For distributed databases, concurrency control and deadlock detection is fundamentally more expensive than it can be for single-node databases -- so it makes sense that a classically single-node database would absolutely trounce distributed databases. I like to think of tpcc "nowait" as really a benchmark of concurrency control because, due (I believe) to Amdahl's law the majority of its execution time ends up just in the contended portion of the workload.
Also very interesting that, as Justin points out, the workload sets up the warehouses so there is never cross-node contention. That's wild! I'm glad they didn't go and benchmark against even more distributed databases (like YugabyteDB, Spanner, or CockroachDB) and call it a fair fight.