/plushcap/analysis/cloudflare/cloudflare-migrating-billions-of-records-moving-our-active-dns-database-while-in-use

Migrating billions of records: moving our active DNS database while it’s in use

What's this blog post about?

Cloudflare, an authoritative DNS provider used by 14.5% of all websites as of October 2024, has successfully migrated its DNS records data to a new database cluster. The migration was driven by the need to optimize the company's main Postgres database cluster (cfdb), which was becoming increasingly strained under the pressures of several services, many unrelated to DNS. The move to a new database allowed Cloudflare to create a more tightly controlled interface to the DNS data and improve the systems that interact with it. The migration process involved building a Change Data Capture and Transfer Service (CDCTS) to ensure no data loss and minimal downtime during the transfer of millions of records daily.

Company
Cloudflare

Date published
Oct. 29, 2024

Author(s)
Alex Fattouche, Corey Horton

Word count
3762

Hacker News points
None found.

Language
English


By Matt Makai. 2021-2024.