Why a Monolithic Data Warehouse Is the Right Choice for Most Organizations
The enterprise data warehouse (EDW) has been criticized due to problems associated with extract-transform-load (ETL) processes, such as latency, inability to handle real-time use cases, and difficulty adapting to changing business needs. However, these issues are not inherent to the EDW concept but rather stem from ETL. Alternatives like Query Federation, Data Virtualization, and Hybrid Transactional/Analytical Databases (HTAP) have been proposed, but each has its limitations. A successful data warehouse requires proper separation of concerns, with replication of all business data into the EDW followed by transformation into a user-friendly schema. This approach eliminates ETL problems like latency and data gravity issues while supporting non-relational data sources and allowing for flexible schema evolution.
Company
Fivetran
Date published
Aug. 7, 2019
Author(s)
George Fraser
Word count
829
Hacker News points
4
Language
English