Dynamic Base MMM Models May Not Be Worth the Effort

So what is Dynamic Base?

In MMM, the Base is nothing but the intercept and one interprets it as the ‘organic sales (or any KPI) that one could get if there were no marketing / media efforts or extraneous factors’.

Marketing theory says that the Base (Brand Equity) of a brand is very sticky and hardly moves.

Even in our experience of more than a decade, we have seen a established CPG brands base go up or down only by 2 -3 percentage points.

Even for E-commerce brands, Pharma, Banking, sports clubs, Streaming companies we have not seen Base vary too much.

However some think that the base varies dynamically and hence it should be modeled that way.

In the past we too have tried dynamic base models like unobserved components model (UCM). See link in comments.

But the gains from such methodology have not been worth the effort.

I believe the same holds true for any fancy Bayesian time varying parameter model. Too much mathematical sophistication but very little practical utility.

Let me highlight the major issues of Dynamic Base Modeling

📌 The problem of Identifiability

The primary task of MMM is accurate attribution – identify which variables resulted in sales and by how much.

If the base itself is allowed to move freely, the model can start reallocating variation between base and media almost arbitrarily.

Clear identifiability takes a back seat.

📌 Blunts decision clarity

As far as we have seen, C-level have these primary questions

• What portion of sales is organic?
• What portion is marketing driven?
• What is the incremental effect of spend?

Dynamic base models often blur this separation.

📌 Overfitting

One thing we repeatedly noticed when we built Dynamic base models is that – A time-varying baseline can easily absorb noise.

Now the noise here is actually not some random noise but actually factors like seasonality. You see seasonality effects are temporary and time bound.

But to construe the effect of seasonality to be more spread out within the base across months or years is wrong.

📌 Contradiction with Business Reality

Brand equity do not jump around wildly every month. Those who have done manual brand equity surveys can ascribe to it. The reason this exercise is conducted quarterly and not monthly is for the sheer fact that this metric moves slow.

📌 Identify variance with stability

This may sound ironic but in accurate attribution tasks, if you want to know ‘which change in spends led to what and why’ you need some stable assumptions. Stable Base is that assumption.

If there are too many moving pieces in a model, you can’t be sure about anything.

Statistically also, if you have many extra parameters to estimate, you will need more data and more degree of freedom.

For a small data problem like MMM, even one additional parameter to estimate (Base) could be one too many.

Even in statistically complex MMM, the principle of KISS applies – ‘Keep it Simple Stupid’.