Good MMMs are not built to predict the future, they are built to change the future!!
Every good MMM is destined to be proven wrong. This may sound counterintuitive, but a successful Marketing Mix Model (MMM) is actually designed to be proven wrong.
Let me elaborate.
📌 Old Policy Regime vs New Policy Regime
An MMM produces a prediction of the future based on historical relationships between marketing inputs and business outcomes.
But the moment you use the MMM to make a decision – change budgets, shift channels, alter media flighting patterns etc, you are intervening in the very system that generated the historical data.
And when you intervene, the system changes.
So if the MMM says:
“If everything continues as before (BAU), sales next month should be X.”
And you decide to double Meta spends, reduce Google spends, and increase promotions based on the model’s insights, then the original prediction should fail.
Why?
Because the prediction assumed the old policy regime.
The MMM is describing a world where things were as per the old policy regime. The moment you act on the model’s recommendation, you create a new world.
In other words, Good MMMs are not built to predict the future.
They are built to change the future.
If your MMM keeps predicting perfectly month after month, it usually means one of two things:
▪️No meaningful decisions were taken from the model
▪️The business kept operating exactly the same way (which is ok, but then it defeats the purpose and goal of marketing measurement – to improve)
Both in my opinion are signs that the model isn’t actually being used.
Ironically, the best validation of an MMM is when the business changes strategy because of it and the original prediction no longer holds.
This is the real success of MMM.
The job of MMM is not to forecast reality. Its job is to inform better decisions that reshape reality.
At Aryma Labs, we are happy to state that 70-80% of our MMM suggestions gets implemented on the ground and by virtue of that our model predicted KPI does go wrong in the future period. But this just goes to show the acceptance of our models. 😎
