Anomaly Detection System Specification Template

Start with statistical methods (Z-score, STL decomposition). They are interpretable, require no training data, and work well for univariate time-series monitoring. Use ML-based methods (Isolation Forest, autoencoders) when you need to detect anomalies in multivariate data or when the normal behavior pattern is too complex for statistical rules. The [AI PM Handbook](/ai-guide) covers algorithm selection for production ML systems.

Target less than 10% of alerts being false positives. Above 10%, on-call engineers start ignoring alerts (alert fatigue), which means true anomalies get missed. Below 5% usually means your thresholds are too loose and you are missing real anomalies. Track the rate by having responders label each alert as true or false after investigation.

Use seasonal decomposition methods (STL, Prophet) that separate the signal into trend, seasonal, and residual components. Apply anomaly thresholds to the residual component only. Document all known seasonal patterns (hourly, daily, weekly, holiday) in the Normal Behavior Definition section. Different thresholds for weekends versus weekdays are common and valid.

Use both. Absolute thresholds catch catastrophic failures (revenue drops to zero). Relative thresholds (standard deviations, percentage changes) catch gradual degradation. Absolute floors are typically critical-severity alerts. Relative changes are warning-severity alerts. Document both types in the Threshold Configuration section.

Three tactics: (1) Set duration requirements so transient spikes do not trigger alerts. (2) Use cooldown periods to prevent repeated alerts for the same ongoing anomaly. (3) Implement correlation rules that suppress downstream alerts when a root cause alert already fired. Review false positive rates weekly during the first month and adjust thresholds based on responder feedback. The [model drift](/glossary/model-drift) concept applies to anomaly detection baselines, which need periodic recalibration. ---