In MMM, everybody talks about incrementality. How to discern the incremental sales (or any KPI).
But what one should also question is – how much incremental insights one is getting out of MMM?
Has the MMM exercise informed you something extra that you did not know already?
We recently had a conversation with one client and here is a paraphrased excerpt of it
Client : “Why is that MMM sometimes gives us answers that we already know?”
Me: “Let me guess, your earlier vendor used Bayesian MMM ?”
Client : “yes, we were using Bayesian MMM and we actively helped the vendor in setting priors”
Me: “ah I know what the problem is”
The client later on lamented on how they have paid lots of money and practically got to know ‘what they knew already’.
They knew the marketing effectiveness of some channels but they didn’t get incremental information on other channels. Some of the channels didn’t even feature in the model !!
So what is the issue here?
𝐓𝐡𝐞 𝐩𝐫𝐢𝐨𝐫𝐬
The biggest Achilles heel of Bayesian MMM are priors. Because of which one always starts off on the wrong foot (pun intended 🙂 )
MMM is relatively a small data problem. This means that one never actually has enough data to overpower the biased priors one is setting.
The result – Your priors overpower the data, and in the output, you get what you knew already !!
So it shouldn’t be surprising that the model is stating the obvious.
I would highly urge readers to read Nuno Reis’ posts of Bayesian Thinking (Link in resources).
In his latest post he says “While Bayesian Thinking refines our beliefs with ‘tangible evidence’, it CONFINES us to ‘what is possible’ — within what is already understood and quantified.”
In a MMM setting, the prior defining exercise is more or less about ‘what the marketing effectiveness of a marketing variable should be’. It in a way confines us to ‘what is possible’.
𝐁𝐮𝐭 𝐰𝐡𝐚𝐭 𝐚𝐛𝐨𝐮𝐭 𝐅𝐫𝐞𝐪𝐮𝐞𝐧𝐭𝐢𝐬𝐭 𝐌𝐌𝐌 , 𝐝𝐨𝐞𝐬 𝐢𝐭 𝐧𝐨𝐭 𝐡𝐚𝐯𝐞 𝐭𝐡𝐞 𝐩𝐫𝐨𝐛𝐥𝐞𝐦 𝐨𝐟 𝐬𝐭𝐚𝐭𝐢𝐧𝐠 𝐭𝐡𝐞 𝐨𝐛𝐯𝐢𝐨𝐮𝐬?
In short No. Frequentist MMM allows data to talk. There is no stifling of the voice of the data through priors. See my post on Truth whisperers in MMM (link in resources).
Of course any skilled statistician can also manipulate frequentist MMM to state the obvious or what client already knew. But the tools to do so are limited.
Bayesian MMM on the other hand provides a plethora of tools and opportunities to manipulate MMM. See my post on the same in resources.
Of late we are seeing a lot of Bayesian MMM adopters switch to us because of the above stated problem and more (See link in resources).
𝐈𝐧 𝐬𝐮𝐦𝐦𝐚𝐫𝐲 :
Don’t fall into the pitfall of knowing the obvious. Bayesian MMM will inevitably lead you to that. It is more prudent to adopt Frequentist MMM.
Resources:
Nuno Reis post : https://www.linkedin.com/posts/nuno-reis_nunopost-activity-7170681948971986944-xtuR?utm_source=share&utm_medium=member_desktop
Bayesian MMM vs Frequentist MMM – Key Comparisons
https://www.linkedin.com/posts/venkat-raman-analytics_frequentist-mmm-vs-bayesian-mmm-comparison-activity-7161591828285210624-qyUm?utm_source=share&utm_medium=member_desktop
Which technique provides for great manipulation in MMM – Bayesian or Frequentist?
https://www.linkedin.com/posts/venkat-raman-analytics_marketingmixmodeling-statistics-marketingattribution-activity-7156533790130003968-qVTr?utm_source=share&utm_medium=member_desktop
Want performance guarantees ? choose Frequentist MMM.
https://www.linkedin.com/posts/venkat-raman-analytics_marketingmixmodeling-statistics-marketingattribution-activity-7151460386980945920-H-4U?utm_source=share&utm_medium=member_desktop
Adopting MMM for the first time ? Use Frequentist MMM.
https://www.linkedin.com/posts/venkat-raman-analytics_marketingmixmodeling-marketingattribution-activity-7148928932862472192-ozrW?utm_source=share&utm_medium=member_desktop
