At incident.io, they prioritize excellent observability to ensure high availability for their customers. They achieve this by creating a user-focused lens on their internal setup, treating it like a product with great UX. Their system follows the principle of "alerts in, notifications out" and is structured around four key areas: alerts, alert routing, escalations, and notifications. Each area has an overview dashboard that provides a high-level view of the system's health, followed by more specific dashboards for each subsystem. They also use event logs to provide a single, consistently formatted log line for each task, and tracing to visualize what a request spent its time doing. The goal is to make their observability setup feel great, with clear structure and hierarchy, and to achieve this through deliberate investment and buy-in from the team.