Updating our data stack
In this update on their data stack, incident.io has made several tweaks and improvements over the past two months. They have migrated Fivetran to a separate BigQuery project to deal with concurrent query limits, set up CircleCI to run the dbt pipeline sequentially, moved from using Google Sheets for GBP:USD FX rates to an API feed, materialized all of their dbt models as tables, introduced intermediate tables into their dbt pipeline, and used SQLFluff VSCode extension for automatic formatting and error detection. They have also updated their folder structure in Metabase to differentiate one-off analyses from those used by stakeholders across the company.
Company
Incident.io
Date published
Aug. 24, 2022
Author(s)
Jack Colsey
Word count
2451
Language
English
Hacker News points
None found.