Company
Date Published
Author
Wenquan Liu, Longbridge Technology
Word count
1929
Language
English
Hacker News points
None

Summary

Longbridge Technology, a next-generation social-driven online brokerage firm based in Singapore, has re-architected its market data services to leverage ClickHouse for improved performance, scalability, and cost-effectiveness. The company initially used a complex architecture with PostgreSQL, Redis, DynamoDB, and Aliyun's OSS object storage, but faced challenges such as high storage costs, poor write performance, and complexity in ETL operations. After evaluating several time-series databases, including ClickHouse, TimescaleDB, and TDengine, Longbridge chose ClickHouse for its compatibility with PostgreSQL, outstanding performance advantages, and cloud-native operational benefits. The company conducted performance testing on ClickHouse's bulk data insertion capabilities using 13 years of A-share market data and found significant improvements in write latency. After migrating historical data from DynamoDB, PostgreSQL, and Redis to ClickHouse, Longbridge unified its data storage solution across all markets, reducing storage costs by 4x and improving write performance by 10x. The new architecture also simplified the technology stack, reduced development and operational pressure, and brought about substantial performance improvements, making it an ideal choice for the company's market data business.