Most data teams set up threshold alerts early and outgrow them fast. Revenue drops below $X per day: alert. Churn rate exceeds Y%: alert. Page load time crosses Z seconds: alert. The first few alerts catch real problems. Then the alert volume increases, some of the alerts are spurious (weekends, seasonality, known campaigns), the team stops looking at them carefully, and eventually the alerts become background noise that everyone learns to tune out.
This is not a problem with alerting as a concept. It is a problem with threshold alerts specifically. A threshold alert answers one question: "did this number cross a line?" It does not answer the question that matters for taking action: "why did this number cross the line, and should I care right now?"
Why Threshold Alerts Generate Noise
Threshold alerts have two failure modes that are structural, not configurable away.
The first is false positives from seasonality. A SaaS product with strong weekday usage patterns will fire a low-DAU alert every Saturday unless someone explicitly adjusts the threshold for Saturday. But Saturday alerts that fire every week without exception are noise. The threshold that is appropriate for Monday is not appropriate for Saturday. You can work around this with day-of-week-specific thresholds, but that is configuration complexity that multiplies for every metric you want to monitor.
The second is false positives from growth trends. A threshold of "alert when revenue drops below $50,000/day" was appropriate when the business was at $55,000/day average. A year later when the average is $90,000/day, that same threshold is so far below the normal range that the only time it fires is during a genuine catastrophic outage. The alert is still technically correct but useless as an early warning system because it would not fire until the problem was already obvious from other signals.
Anomaly detection addresses both of these by defining "anomalous" relative to expected, not relative to a fixed line. The expected value is computed from historical data at the appropriate granularity (same day of week, same period of the year, accounting for growth trend). A deviation is flagged when it is statistically unlikely given that expected distribution. This means the anomaly detector calibrates automatically as the business grows and as seasonality patterns change.
What Statistical Anomaly Detection Actually Involves
The basic machinery of anomaly detection for a business metric is a model of expected values and a test of whether the observed value is unlikely under that model. The simplest version uses a rolling mean and standard deviation: flag anything that deviates by more than 2 or 3 standard deviations. This handles growth trend poorly (the mean is a lagging estimate) and handles seasonality poorly (weekend vs. weekday variance is pooled).
Better approaches decompose the time series into trend, seasonality, and residual components. STL decomposition (Seasonal and Trend decomposition using Loess) is a well-established approach. Facebook Prophet is a more accessible model that handles multiple seasonality periods (weekly and yearly) and trend changepoints. For a metrics monitoring use case, the key requirement is that the model captures the periodicity of your specific metric so that expected values are computed at the right level of granularity.
The output of the anomaly detection step is a flag: this period's value is outside the expected range. At that point, the anomaly detection step has done its job. But the output is still only "something changed." The next question is always "what changed and why," and that is where attribution enters.
Combining Detection with Attribution
An anomaly detection notification that includes attribution results changes the downstream workflow substantially. Instead of:
- Alert fires.
- Analyst picks up alert.
- Analyst runs root-cause investigation (45 to 90 minutes).
- Analyst determines: "EMEA mobile conversion dropped; it is a rate effect in the new user cohort from last week's campaign."
- Analyst notifies stakeholders.
The workflow becomes:
- Alert fires with attribution: "Conversion rate down 12% vs. expected. 71% of variance explained by EMEA mobile new user cohort; rate effect (not mix). Likely linked to campaign traffic from prior 7 days."
- Analyst reviews attribution, validates with spot-check query, notifies stakeholders with context already assembled.
The investigation step does not disappear. An analyst should still review the attribution result before sending it to stakeholders, because attribution results can be artifacts of data quality issues rather than real business signals. But the investigation is now a validation exercise rather than a discovery exercise. That difference is meaningful in terms of time and cognitive load.
Alert Fatigue and the Minimum Attribution Threshold
One of the underappreciated benefits of attribution-backed anomaly detection is that it enables smarter alert routing. If the anomaly detection fires and the attribution shows that 85% of the variance is explained by a single segment that the team already knows about (a known data pipeline delay, a planned marketing pause), that information is immediately actionable: this is likely a known cause, low urgency, route to data engineering.
If the attribution shows that the anomaly is broadly distributed with no single segment explaining more than 20% of the variance, that is a different pattern: something systemic may have changed, higher urgency, escalate to product and ops.
The signal-to-noise improvement from including attribution at alert time comes from this routing capability. Not every alert needs the same level of urgency. Attribution provides the context to triage correctly at the time of the alert rather than after a manual investigation.
What Does Not Work Out of the Box
Combining anomaly detection and attribution requires that both systems operate on the same metric definitions and dimensional schema. If your anomaly detection runs against a raw table and your attribution runs against a transformed model, the decomposition results may not match the anomaly's actual composition. We encountered this while building the alert pipeline at Golden Analytics: attribution results that were computed against a slightly different aggregation of the same underlying data than the anomaly detection used produced confusing discrepancies where the attribution total did not match the anomaly magnitude.
The resolution is to define the metric once, at the semantic layer, and have both the anomaly detector and the attribution engine query the same definition. This sounds simple but requires discipline: the anomaly detection system has to be connected to the same metric store as the attribution system, rather than each having its own ad-hoc query that computes "roughly the same thing."
There is also a cold-start problem for anomaly detection on new metrics. A metric that was just added to your monitoring system has no historical distribution to compute expected values from. Depending on the seasonality period you care about, you may need 4 to 12 weeks of history before the model's expected value estimates are reliable enough to produce a low false-positive rate. Monitoring new metrics through threshold alerts first, then migrating to anomaly detection once history accumulates, is a practical pattern. It is not permanent, but it avoids a period of high-noise anomaly alerts that train the team to ignore the system.
Choosing the Right Approach by Metric Type
Not every metric warrants the full anomaly detection and attribution pipeline. Operational metrics with binary behavior (is the pipeline running or not, is the endpoint returning 200 or 5xx) are better served by threshold alerts because the expected behavior is either "working" or "not working," and there is no useful statistical distribution to model. The binary case where a threshold alert is exactly right: "alert if daily_row_count is zero for more than 2 hours" does not need anomaly detection.
Business KPIs with continuous distributions, clear seasonality, and growth trends are the anomaly detection use case. Conversion rate, DAU, revenue, activation rate, churn rate. These metrics have enough historical variance and enough structured patterns that a model of expected values adds real signal over a static threshold.
Attribution is most useful when the metric has meaningful dimensional structure: multiple segments that plausibly contribute different amounts to movements. A metric that is only ever broken down by one dimension with two values does not need sophisticated attribution; the two-cell comparison is trivial. The attribution value scales with the number of dimensions and the complexity of the metric's composition. Metrics with three to eight meaningful dimensions are where attribution-backed alerts deliver the clearest benefit.