Why Experimentation is not a substitute for Marketing Mix Modeling (MMM)

Experimentation is not a substitute for Marketing Mix Modeling (MMM)
Experimentation is not a substitute for Marketing Mix Modeling (MMM)

So a lot of myths have been propagated by digital marketers of late, such as

MMMs should be adopted only if :

– Companies have revenue of $50M+ yearly.
– they spend a lot on non click media (tv, radio etc.)
– you have 5 or more channels.

I have been studying and practicing statistics for nearly 17 years now, take it from me that MMMs have no statistical limitations for any of the above.

So what gives? where do these numbers and limitations come from ?

Answer – The reasons are not statistical but commercial.

๐Ÿ“Œ The myth of $50M + revenue

Digital marketers came up with this number because they don’t want companies < $50m yearly to adopt MMM and instead want them to do experimentation.

But is Experimentation the right replacement for MMM?
No, not really.

๐Ÿ“Œ MMM and Experimentation answer different questions.

MMMs provide holistic read of marketing effectiveness while experimentation do not.

What do I mean by holistic read?

Marketing Mix Modeling (yes it is not just media mix), involves understanding how marketing and non marketing variables together affect your KPI.

Modeling is all about abstraction of reality. Do you think your sales is only affected by one variable at a time and not other variables?

I am sure it is a No.

Replacing MMM with Experimentation is kind of saying that “MY KPI (e.g. sales) is only affected by one media variable and by nothing else”.

Sure you can say you could do these experimentation with other variables one by one.

But you see you are missing out on the interaction effects of these variables on the KPI.

Marketing Effectiveness is a multivariable problem and it simply can’t be answered by experimentation, which in most cases involves considering univariable treatment.

Experiments can help validate certain aspects of MMMs like incremental lift. But Experiments themselves don’t provide attribution coefficients that is conditioned upon other factors.

Anybody proposing Experimentation in place of MMM, is doing a great disservice to client.

๐Ÿ“Œ Why the push for Experimentation

Relatively speaking, Experimentation are quick cadence and generate quicker recurring revenue. Therefore it is beneficial for agencies/vendors doing it.

But Experimentation don’t give you a holistic read of marketing effectiveness.

What is beneficial for agencies/vendors financially may not be beneficial for you the clients.

As a client, you want to address marketing effectiveness. But experimentation does not do that.

๐Ÿ“ŒMMMs catching up to Experimentation

MMMs have become faster, accurate, bespoke and cost effective. If the goal is ascertaining marketing effectiveness and somebody offers me 2 options:

Experimentation at $10k
vs
MMM at $10k.

I would choose MMM in a blink of an eye.

P.S : Link to related Interesting reads in resources.

Resources:

Ridhima Kumar’s post : Why you do need MMM and Why experimentation alone is not enough.
https://www.linkedin.com/posts/ridhima-kumar7_marketingmixmodeling-marketingattribution-activity-7154085847909175296-qY1M?utm_source=share&utm_medium=member_desktop

Dale W. Harrison excellent post on Marketing Effectiveness vs Efficiency
https://www.linkedin.com/posts/dalewharrison_%3F%3F%3F%3F%3F%3F%3F%3F%3F-%3F%3F%3F%3F%3F%3F%3F%3F%3F%3F%3F%3F%3F-activity-7148782037108178944-IEUL?utm_source=share&utm_medium=member_desktop

Facebook
Twitter
LinkedIn

Recommended Posts

Chebyshev’s Inequality for Marketing Mix Model Diagnostics

Chebyshev’s Inequality for Marketing…

At Aryma Labs, we constantly endeavor to add as much science as possible…

How to use Robyn’s…

In my last post (ICYMI link in resources), I talked about the similarities…

Similarities between Decomp RSSD and Bayesian Priors in Marketing Mix Modeling (MMM)

Similarities between Decomp RSSD…

Open source Marketing Mix Modeling (MMM) tools are great for democratizing MMM. But…

Scroll to Top