Introducing Airbyte Destinations V2 - Typing & Deduping
Airbyte has introduced Destinations V2, which brings improvements to data syncing and handling in destination tables. Key features include one-to-one table mapping, enhanced per-row error handling with _airbyte_meta column, internal Airbyte tables in the airbyte_internal schema, and incremental delivery for large syncs. These updates are now available in Snowflake and BigQuery destinations, with plans to roll out to more connectors soon. Destinations V2 separates data-moving problems from data-content problems, allowing users to handle content errors on a case-by-case basis. Additionally, the new tooling helps manage connector updates with breaking changes, providing flexibility in upgrading and testing new updates.
Company
Airbyte
Date published
Aug. 29, 2023
Author(s)
Alex Cuoci
Word count
629
Language
English
Hacker News points
None found.