Why we report both confidence Interval and Prediction Interval in our MMM models.
MMM is a type of linear regression but with lot more bells and whistles (check the link under resources for a primer on MMM).
If you must have noticed in Linear Regression, the confidence interval are always narrower than the prediction interval.
You see in Linear Regression, we don’t model the raw response, rather we model the conditional mean (expected value) -> E(Y|X).
This conditional mean is the parameter which we try to estimate.
Through CI, we are trying to answer the question – “If we were to do this Linear Regression again and again (in a way redrawing samples from the population),
then does the CI constructed each time contain the true parameter or not”? If in 95% of the cases, we do end up having the true parameter, then we are set to have 95% CI.
Now you see, the only error that is accounted for while calculating CI is the inherent sampling error.
Where as in the case of prediction Interval, the sampling error as well as the variance around that prediction is accounted for.
This is why prediction intervals are always wider than CI.
Now in the context of MMM, we report both the CI and PI because CI and PI gives a true picture of the MMM model.
In a way the PI is the delta of error that is only because of ‘variance around the expected mean’. This in a way could further point to issues with multicollinearity.
Again multicollinearity is a huge issue in MMM. But there are methods to address them such as regularization or residualization.
So overall, reporting PI and CI leads you to better diagnose and specify a MMM model.
Resources:
MMM 101: https://arymalabs.com/market-mix-modeling/
Clearing Misconceptions around Confidence Interval. โ https://bit.ly/3HgOOIn
Multicollinearity issues in MMM :https://www.linkedin.com/posts/ridhima-kumar7_marketingmixmodeling-marketingattribution-activity-7111329767273988096-ZhG4?utm_source=share&utm_medium=member_desktop
Image courtesy: https://online.stat.psu.edu/stat501/lesson/3/3.3
