How We Use ClickHouse to Analyze Trends Across Millions of Builds

Buildbuddy uses ClickHouse to analyze trends across millions of builds. The Trends page displays information about Bazel invocations such as build duration and cache hit rate. Initially, MySQL was used for data aggregation but performance issues arose with large numbers of builds. ClickHouse, a column-based database, improved query performance significantly compared to MySQL. Data migration from MySQL to ClickHouse involved ensuring accuracy and consistency before enabling the Trends page to read from ClickHouse. Future plans include providing more analytical insights into builds, targets, tests, and remote execution.