Data Validation at Scale – Detecting and Responding to Data Misbehavior

In today's data-driven world, ensuring the accuracy and consistency of large amounts of data is crucial for businesses. Data validation can be challenging as the volume of data grows. This tutorial introduces the concept of data logging and demonstrates how to validate data at scale using the whylogs open-source package. The case study focuses on validating Airbnb listing activity and metrics from Rio de Janeiro, Brazil. By leveraging data logging, businesses can generate statistical summaries of their data for monitoring, visualization, drift detection, and data validation purposes. Metric Constraints are a powerful feature built on top of whylogs profiles that enable users to quickly and easily validate the quality of their data.