Zillow Talk: How Companies Leverage AI & How to Keep It In Check

Recently, Zillow experienced significant losses due to their machine learning models' failure to accurately predict housing prices. This incident highlights the importance of monitoring and maintaining ML models in production. Businesses should be aware that a model performing well during training may not perform as expected when deployed, due to constant changes in the environment and data. Ongoing monitoring is essential to detect drift, bias, and integrity issues early on and ensure models are working as intended. Despite many companies utilizing AI in their business, comprehensive monitoring solutions or dedicated resources for building ML monitoring solutions remain limited.