How Do you know your adstock values are accurate?
During our recently concluded Marketing Measurement Marathon Course, a participant posed some excellent questions.
“How do we know we have the right adstock parameter values?
There are some thumb rules that carry over TV should be (0.3-0.7), Digital (0-0.3), Print /Radio/OOH (0.1-0.4).
Why TV has a higher carry over than say Digital?”
Let me try to answer the above
📌 How do we know we have the right adstock parameter values?
Depending on the adstock type – Geometric or Weibull, you will have one or two parameters to estimate.
We know we have the right adstock parameter values when the MMM model has good goodness of fit metrics (calibration metrics). We use MMMDiagnose to see how well the MMM model is fit.
The Aryma Score given by MMMDiagnose is a synthesis of nearly 12 different goodness of fit metrics like NMRSE, PIT, KL Divergence, Chebyshev Inequality, Adj R squared and In sample MAPE.
Achieving good calibration metrics assures that what you have done through adstock parameter tunning is accurate.
Analysts can also adjust the parameters to better align with the observed marketing reality (ground truth data).
When building MMM models, fine-tuning these parameters is important to obtain the best series for use in the model.
This involves tuning hyperparameters like the carryover impact theta (in geometric) or shape and scale parameters (in Weibull).
For example, if theta is 0.80, it means 80% of the past advertisement’s effect carries forward to the present.
Different combinations of parameters are tested to find the best one. When I say tested, I mean that these parameter values are checked for correlation with your KPI.
Correlation is a faulty metric to base the best selection of adstock but still many vendors use this. We at Aryma Labs use Information theoretic measures like transfer entropy and granger causality.
📌 “There are some thumb rules that carry over TV should be (0.3-0.7), Digital (0-0.3), Print /Radio/OOH (0.1-0.4). Why TV has a higher carry over than say Digital?”
There is both theoretical and empirical reason as to why these ranges were arrived at.
TV and now CTV are considered to be a brand building medium. When people see TV advertisements, they tend to remember them more, and the impact lingers in their minds for a longer time.
This is why brands are willing to pay a premium for ad slots during events like the Super Bowl.
TV marketing spends generally have a long carryover impact. The carryover impact for TV advertisements could start from 30% and go up to 80%. Hence 0.3-0.8.
Digital channels may have a high impact in the short term, but they are generally not believed to be effective for brand building. Hence the range of 0-0.3.
Radio and OOH are often underestimated. At Aryma Labs, we believe their carryover can rival TV (0.3-0.8) due to stronger recall than commonly assumed.
P.S: Related link in comments
