/plushcap/analysis/neon/neon-stream-data-from-neon-to-external-data-sources-via-logical-replication

Stream data from Neon to external data sources via logical replication

What's this blog post about?

Logical replication allows you to replicate data from your Neon database to various external systems, enabling Change Data Capture (CDC) and real-time analytics. It's a publish-subscribe model where all changes are tracked via Postgres' Write-Ahead Log (WAL), ensuring that the data in both sources remains synchronized. Logical replication is useful for selective data sync, real-time data replication, production migrations without downtime, and safer Postgres upgrades in large databases. It differs from pg_dump/restore, which creates static snapshots of the database or selected tables, but not ongoing changes. When to use logical replication depends on your needs, as it requires both data sources to be completely in sync in some cases. To set up logical replication in Neon, navigate to project settings, select Logical Replication, and click Enable, verifying that the right roles have the required privilege. Publications define which tables and changes are replicated from the publisher to subscribers, while replication slots ensure data consistency and prevent potential data loss.

Company
Neon

Date published
Aug. 9, 2024

Author(s)
Carlota Soto

Word count
1565

Language
English

Hacker News points
None found.


By Matt Makai. 2021-2024.