Reducing our pager load
Incident.io, a small team that provides a great product to its customers, has been using an app that raises errors via Sentry and alerts them through pager. However, they found this system too noisy as it lacks the concept of working hours and is sensitive to all exceptions. To reduce their pager load without compromising on error detection, they implemented several strategies: 1) introducing expected errors with helpers that return a warning or an error; 2) labeling best effort paths by setting default urgency in context; 3) only paging for transient errors after three failed attempts; and 4) not paging for issues related to demo accounts. These changes significantly reduced their pager load, improving the team's work-life balance without compromising on error detection and fixing.
Company
Incident.io
Date published
April 22, 2022
Author(s)
Lisa Karlin Curtis
Word count
1201
Language
English
Hacker News points
None found.