Is your media actually driving growth or just piggybacking on it?
A recent MMM project left both us and the client quite baffled. Despite millions in media spend, the model kept assigning very low contribution to media across multiple iterations.
Naturally, our first instinct was: “Are we getting the model wrong?”
But then we decided to go back to the EDA charts. The EDA charts pointed to a very good correlation between media spends vs sales in the latest years.
At face value, it seemed that the media is working.
But we quickly reminded ourselves what we teach in our causality course – correlation is a low bar.
Remember for anything to have a good correlation, things just have to move in the same direction as the other variable.
High correlation between Nicholas cage movie release and drownings in pool don’t sound that surprising anymore. Isn’t it 😅?
📌 What we uncovered
Media bursts were consistently aligned with a major recurring event – one that historically drives sales even without media.
In other words, Media wasn’t driving the spike. It was showing up at the same time as the spike !! Classic piggybacking.
📌 The Test
We ran the model two ways:
– With event captured properly (RBF)
– Without the event control
We observed that when the RBF dummy was present, the Media contribution was around 0.8 and 0.5 for Meta and TikTok respectively but after removing the dummy it went up to 1.2-1.3%.
📌 What this informed us:
If your model isn’t controlling for event-driven demand properly, media will happily take credit for it. And correlation will back it up.
But a well specified MMM will expose the truth.
The media in this case wasn’t lifting its weight. It was just riding the wave.
📌 The Business Impact
The previous vendor had suggested the client to continue spending millions on media during and around this event.
We figured that the client was paying for sales that would happen anyway.
This insight alone saved the client $2–3M in avoidable ad spend.
📌 Bottomline:
High correlation is easy. True incrementality is not.
If you are not explicitly modeling:
– Events (use RBF)
– Seasonality properly (use RBF)
– Demand spikes (use RBF)
Then your media is probably getting over credited.
And you end probably overspending on media.
