Company
Date Published
Author
Aman Gupta
Word count
762
Language
English
Hacker News points
None

Summary

The Write. Audit. Publish (WAP) framework is a software engineering discipline applied to data engineering, aiming to ensure the quality and trustworthiness of data before it reaches production. This approach involves writing data in isolation, auditing for correctness, quality, and compliance, and publishing only validated data. A hypothetical scenario using the dlt dataset tool demonstrates how WAP can be implemented to catch errors upstream, preventing breakages and ensuring trustworthy data. By following this pattern, data teams can prevent bad data from reaching production, save time on last-minute fixes, and gain confidence in their data pipeline.