Monitoring and Alerting Configuration Template

Start with 5-10 alerts for a typical service. If you have more than 20, most of them are probably noise. Every alert should pass the "would I wake someone up for this?" test for paging alerts, or the "would I investigate this today?" test for Slack alerts. Remove or downgrade alerts that consistently fire without requiring action. Track your [key metrics](/glossary/prioritization) to identify which alerts correlate with real user impact.

Three approaches: raise thresholds (alert at 2% error rate instead of 0.5%), increase evaluation windows (alert after 10 minutes sustained instead of 1 minute), and consolidate related alerts (one "service degraded" alert instead of separate CPU, memory, and latency alerts). Review firing frequency monthly and tune or remove alerts that fire more than 3 times per week.

Yes. PMs do not need access to alerting configurations, but they should be able to view dashboards that show feature adoption, error rates, and performance metrics. Create a PM-friendly dashboard with business metrics (conversion rate, feature usage) alongside the technical metrics that explain them.

Monitoring tells you when something is broken (via predefined metrics and alerts). Observability tells you why it is broken (via logs, traces, and ad-hoc queries). This template covers monitoring. Observability requires additional investment in structured logging, distributed tracing, and query tools. Start with monitoring. Add observability when "the alert fired but we could not figure out why" becomes a recurring problem.

Deploy the service with monitoring but no alerts for the first 1-2 weeks. Observe the baseline behavior: normal latency range, typical error rate, peak traffic patterns. Set thresholds at 2-3x the baseline for warning alerts and 5x for critical alerts. Refine after the first month of production data. The [Technical PM Handbook](/technical-pm-guide) covers how to use these baselines in product planning. ---