How not to benchmark Cassandra: a case study

The benchmark by Thumbtack Technology for Aerospike has been analyzed with regard to its methodology and results. While the test was performed on bare metal machines with SSD disks, there were issues with benchmark hygiene such as not properly isolating read runs or accounting for ongoing compactions in Cassandra. The benchmark also measures different things in each tested system, particularly in the insert and read-heavy workloads. Further investigation revealed that Aerospike and Couchbase's faster write times compared to Cassandra could be attributed to their asynchronous buffering of writes, while Cassandra synchronously writes to its commitlog by default. The author suggests using a standardized methodology for benchmarking databases to ensure accurate results.