MMM – A Collection of Tiny Experiments

A lot of people see MMM and Experiments (Geo Lift / Incrementality experiments) as totally distinct things.

But if you pay attention and think deeply, you will notice that MMM is not that different from Experiments.

Let me explain:

📌 MMM is a Collection of Tiny Experiments.

In case of an experiment: one makes a change to the variable, hold other variables constant (or at least assume it) and then observes the impact on outcome.

However in real world, especially marketing we don’t get this luxury.

Everything moves at the same time because multiple media / marketing spends are implemented at the same time or overlapping times.

Example : Meta spends changes, pricing changes, Branded paid search, Distribution fluctuation, Seasonality etc.

Now given that so many forces are at play, we need a mechanism to include the effect of all these variables and yet answer the question about any one single variables’ effect (the marginal effect) at any point in time.

Enter MMM.
MMM tries to aggregate all these micro experiments or micro variations.

📌 Sharing of Effect

In an experiment, due to the uni-variable nature, the effect attributed to the variable is often inflated. It does not consider that other variables are at play too and they could have contributed to the change in KPI.

However what MMM does is – it makes one to account for the multi variable reality. When you include many variables into the equation, you get an conditional effect. Every effect of each variable is conditioned on other variables. This tussle between the variables is important and one gets to know the true effect of each variable when other variables are at play.

So instead of asking “Did Meta spends work in 2 weeks?”

MMM helps rephrase this to “How well did Meta work in the past 2 years given that other channels like Google, TikTok, TV, Search etc, were at play”.

📌 MMM – A Decision History Book Keeping

Yesterday Ridhima Kumar wrote an excellent post – Decision History Scoreboard.

MMM is not a monolithic model. In a way it is a memory system of your marketing decisions.

Every budget increase, Every campaign spend increase / cut, Every seasonal spike is a tiny experiment.

MMM just stitches them together into a coherent story.

In marketing, we seldom care about the isolated effect of one single variable (unless you are a new company that just got started with marketing in 1-2 channels or a particular need arose for you to just focus on one single channel).

We predominantly look to answer the collective question and this is answered well by MMM.