At Fivetran, the company's core business is building and maintaining data connectors at scale to minimize user intervention. The engineering team consists of over 300 developers divided into teams that own specific connectors, posing challenges such as ensuring consistent behavior across disparate features and products. Poor error messages have clear costs, including hurting the user experience, posing a brand risk, and consuming support resources. To address this issue, Fivetran's Analytics team partnered with the Product team to develop a lightweight solution using natural language processing (NLP) techniques, identifying semantically equivalent error messages and clustering them based on actual failure states. This effort resulted in 122 weeks of saved engineering time and a 30-50% increase in error messaging accuracy across pilot programs, demonstrating the value of subtle machine learning use cases such as pattern recognition and predictive problems.