How we model our data warehouse
The text discusses the design principles behind the data warehouse of a company. It follows dbt labs' high-level approach for data modeling, dividing the data warehouse into staging (stg_[source]__[model name]), intermediate (int_[source]__[model name]), and marts layers (dim/fct for internal facing data models, and insights for customer-facing data models). The company also applies these design principles to achieve flexible but consistent data modeling in their BI tool. They use pre-joined, modeled marts tables to answer most queries and allow power users the ability to go beyond that. They avoid surfacing intermediate tables in their BI tool and do not "over model" their marts models. Additionally, they do not allow staging tables to become part of these pre-joined datasets in their BI tool or save custom columns there. This approach helps them maintain consistency while providing flexibility.
Company
Incident.io
Date published
Nov. 8, 2024
Author(s)
Jack Colsey
Word count
2028
Hacker News points
None found.
Language
English