Under the hood: How to build custom dbt alerting infrastructure
The dbt alerting tool was built to provide more granular and context-rich alerts for dbt users, allowing them to configure custom alert rules without modifying their dbt projects. The tool operates under the hood using two key artifacts: `manifest.json` and `run_result.json`, which contain metadata about dbt resources and execution results, respectively. The system leverages dbt exposures to store custom configuration data and defines alert rule structures with filters and destinations, allowing users to specify conditions for matching nodes and destinations for notifications. When a dbt job runs, the system extracts alert rules from `manifest.json`, applies them to `run_result.json`, merges relevant data, generates detailed alerts, and distributes them to specified destinations. The tool also handles model-test relationships by sending alerts for both tests and models when necessary. By providing more context-rich alerts, the tool aims to enable users to diagnose issues directly from the alert itself, making it a powerful and free resource for all dbt users.
Company
Metaplane
Date published
Dec. 10, 2024
Author(s)
Todd Pollak
Word count
1108
Language
English
Hacker News points
None found.