Attribution systems depend on a stable relationship between the column names in your warehouse and the dimension definitions in your metric layer. When that relationship breaks, attribution results shift in ways that look real but are not. A segment that appeared to drive 18% of last week's metric movement might have driven 5% in reality, with the remainder being an artifact of a column that was renamed or a dimension that started returning null for a subset of rows.
This problem is well-known in data quality circles, but it has a specific character when it hits an attribution pipeline. The failure mode is not that queries throw an error. Often queries succeed, return plausible-looking numbers, and the incorrect attribution travels downstream to decisions before anyone notices something is wrong. This post covers where schema drift enters the attribution pipeline, how it manifests in output, and how to catch it early.
How Schema Drift Enters the Attribution Pipeline
Attribution pipelines touch the warehouse at two points: when ingesting raw event data into the fact layer, and when querying that fact layer to compute dimension contribution scores. Schema drift can enter at either point.
At the ingestion layer, upstream changes are the most common source. An engineering team renames user_country to geo_country_code without notifying the data team. The ingestion pipeline was referencing the old column name. Depending on how the warehouse handles column references, this either fails (visible, catchable) or silently returns nulls for the new column while data continues to flow (invisible, dangerous).
At the query layer, drift happens when a dimension definition references a column whose semantic meaning has changed even though the column name remains the same. A column called acquisition_channel that previously contained values like "organic_search", "paid_search", and "direct" now contains a different taxonomy after the marketing team changed their tagging convention. The column exists, the query succeeds, but the contribution scores for "organic_search" are now computed on a different population than they were previously.
Symptoms in Attribution Output
Schema drift in an attribution pipeline produces several recognizable patterns in output, though none of them are obvious without a baseline to compare against.
The first pattern is a sudden shift in the relative contribution of a dimension to zero or near-zero without a corresponding shift in the raw metric. If geo_country_code was previously contributing 12% of a metric's variance and is now contributing 0.2%, and the metric itself has not changed significantly, the most likely explanation is that the dimension is now returning mostly nulls. The attribution algorithm is assigning nearly all variance to "null" as a dimension value.
The second pattern is contribution instability across successive runs on the same time window. An attribution run from Monday morning and an attribution run from Tuesday evening on the same event window should return stable contribution scores. If scores for a dimension shift more than a few percentage points across runs on identical input data, something in the dimension data or the metric definition has changed between runs.
The third pattern is a contribution score that is mathematically plausible but semantically inconsistent. If attribution has historically shown "mobile" accounting for about 35% of activation-related metric movements and this week it is 8%, without a corresponding change in mobile acquisition volume or product behavior, that is a signal to check the dimension's data completeness before acting on the result.
Detection Strategies Before Stakeholders See Wrong Numbers
The most practical early detection mechanism is a dimension health check that runs before each attribution pass. For each dimension in the attribution schema, check three things:
- Null rate: what fraction of rows have a null value for this dimension in the current time window, compared against the historical baseline null rate for that dimension.
- Cardinality: how many distinct values does this dimension have in the current window, compared against the historical baseline. A sudden increase (new values appeared) or decrease (expected values disappeared) both indicate a change.
- Distribution shift: for high-cardinality dimensions, whether the top N values by row count have changed significantly in composition. A Chi-squared test or a Jensen-Shannon divergence on the value distribution catches this without requiring a full manual review.
Any dimension that fails these checks before an attribution run should be flagged and its contribution scores marked as potentially unreliable in the output. An attribution result that shows "geo_country_code: 0.3% contribution" alongside a flag "null rate 81%, baseline 2.1%" gives the analyst context to investigate rather than a confident-looking number that gets acted on uncritically.
dbt Tests in the Attribution Context
dbt's built-in schema tests (not_null, accepted_values, unique) are the standard first layer of defense, but they have limitations in the attribution context.
A not_null test catches a column that transitions from having no nulls to having some nulls, but only if you have been running it on every model refresh. Many teams run dbt tests on a schedule that does not align with attribution runs, which creates a gap where a dimension can be degraded for hours before the test catches it.
More critically, accepted_values tests require an explicit list of values, which breaks when the taxonomy evolves. The acquisition_channel example above would pass an accepted_values test if the new values happen to be in the accepted list, even though the semantic meaning has shifted entirely.
What works better in practice is a combination of dbt tests for structural integrity (nulls, uniqueness, referential integrity) and a separate, attribution-specific distribution monitoring step that computes and stores dimension profiles on each run. This profile layer acts as the baseline for the null rate and cardinality checks described above. Tools like dbt's audit_helper package or custom dimensional profiling macros can implement this at the model level.
-- dbt macro: dimension profile snapshot
{% macro profile_dimension(model, dimension_col, date_col, window_days=30) %}
SELECT
'{{ dimension_col }}' AS dimension_name,
{{ date_col }}::date AS snapshot_date,
COUNT(*) AS total_rows,
SUM(CASE WHEN {{ dimension_col }} IS NULL THEN 1 ELSE 0 END) AS null_count,
COUNT(DISTINCT {{ dimension_col }}) AS cardinality,
current_timestamp AS profiled_at
FROM {{ model }}
WHERE {{ date_col }} >= CURRENT_DATE - INTERVAL '{{ window_days }} days'
GROUP BY {{ date_col }}::date
{% endmacro %}Running this macro on each attribution dimension after every dbt run creates a history you can query to detect drift before the next attribution pass.
Lineage and Schema Contracts
The organizational side of schema drift is often more tractable than the technical side. The engineering teams making source schema changes typically do not know that the analytics pipeline depends on those columns. Adding attribution-layer column dependencies to a shared schema contract, whether in dbt's YAML model documentation or in a shared data catalog, gives engineering teams visibility into downstream impact before they make changes.
This is not a technical solution; it is a coordination solution. A column tagged with attribution_critical: true in your dbt model YAML is a signal to whoever is refactoring the upstream source that they need to notify the data team first. It does not prevent schema changes, but it converts silent breakage into a deliberate coordination point. That is the right level of defense for a small team without a full data observability platform.
Recovery Patterns When Attribution Has Already Broken
When attribution has already produced incorrect results and stakeholders have seen them, the priority is to understand which results are affected and to what degree before correcting them.
The key question is when the schema change occurred relative to the attribution run timestamps. If the change happened at 3pm and the attribution run that produced incorrect results started at 3:45pm, the affected outputs are the ones timestamped after the change. Runs before the change can be used as a baseline to assess the magnitude of the corruption.
We are not saying you should withhold incorrect results from stakeholders while you investigate. We are saying that the correction communication is more credible when you can say "the dimension contributions from the 4pm run on August 7 were affected, here is what they should have been" rather than "some of the attribution numbers we sent this week may be wrong." Specificity about which outputs were affected and which were not limits the damage to trust in the system.
The technical fix is usually quick once the drift is diagnosed. The data quality issue is rerunning the affected attribution windows against corrected dimension data and updating the outputs. The harder part is making sure the detection mechanism that should have caught it earlier is actually in place before the next run.