I have written in my previous posts on why one can only validate (partially) MMMs and not calibrate it through experiments. Let’s quickly recap what is calibration and validation. Calibration is a process where you try to improve the model fit by tweaking various knobs and levers. There are metrics that tell you how well […]
One of the fundamental question that you as a client should be asking an MMM vendor is – “Which technique do you employ to build MMM ? Frequentist or Bayesian”. Many vendors (predominantly inclined to Bayesian methods) would often try to dissuade you from going in that direction. Their usual ploy would be to say
Bayesian MMM vs Frequentist MMM – Key Comparisons Read More »
Technically the right answer is – It depends. But there are also a lot of heuristics and rule of thumb on how much data is adequate. Statistically speaking we need to have enough degrees of freedom to estimate the parameters. When one has more predictors than data, we call it the P>N problem. In this
How much data you need for MMM? Read More »
ICYMI (link in resources), in my previous posts I had highlighted why one can’t RCT MMM. I will just recap two points from that post that will provide a perfect segue for the topic that we are going to cover today. ◾ Marketing Reality is complex and RCT can’t control for all variables When we
RCT in MMM violates Principle of Marginality Read More »
As a statistician, it pains me to see marketers do the following: ▪ Make million dollar marketing decision on just correlation ▪ Specify Marketing Mix Models (MMM) without any statistical rigor ▪ Think that MMM can be specified without controlling for all factors ▪ Think confidence interval from disparate models and experiments can be compared
What Marketing Mix Modeling domain can learn from Biostatistics Read More »
Experimentation has become a buzz word in MMM. Rightly so. Experimentation like Difference in Difference (DID) can help one to holistically prove the efficacy of your MMM model. But, please save yourself the time and money and don’t do RCTs on MMM (check the link in resources to know why). Coming to today’s topic,
Marketing Mix Modeling (MMM) Calibration Experiments Read More »
The MMM model speaks but the problem is we often don’t listen. Much like a poorly tuned guitar produces noise rather than good notes, your model too shows tell tale signs of poor fit. So who are the truth whisperers? I consider Maximum Likelihood Estimation and Bootstrapping as truth whisperers. 📌 Maximum Likelihood Estimation Simply
I keep stumbling upon articles and posts where people talk about using Randomized control trial tests (RCTs) to calibrate MMM (absolutely wrong way to go about things – see post link in resources) or to validate the MMM models. In this post we will focus on why one can’t use RCTs to validate the MMM
You can’t RCT Marketing Mix Models (MMMs) Read More »
We are currently in talks with a company to replace their existing MMM vendor. The company realized that the estimates given by this vendor was consistently inaccurate. We dug deeper and the usual suspects turned up. 1) The vendor was using Bayesian MMM (big big red flag). Ok, since I have already talked so much
Why you shouldn’t use media ROI benchmarks to set the priors Read More »
At Aryma Labs we constantly try to push the frontier of what’s possible in Marketing Measurement and Attribution. In this regard, we have been working on a very interesting problem for a long time. What is this problem? Well, basically your MMM model should not be just good at prediction but should also have good
Innovating with Robyn Read More »
Aryma Labs is your global partner to drive marketing efficiency in a privacy-first era. Aryma Labs leverages cutting edge Data Science solutions to solve marketing measurement and attribution problems. We are on a mission to help enterprises of all sizes adopt Marketing ROI Solutions for a privacy first era.
