Calibration vs Validation
A lot of people use calibration and Validation interchangeably. The two are not the same.
โช Calibration
In a regression setting, calibration of a model is about understanding the model fit.
Goodness of fit measures like R squared values, P value, Standard Error, Cross validation and within sample MAPE/MAE/RMSE inform you how well you have fit the model.
Based on these metrics, one could tune and calibrate the model better.
โช Validation
Validation of a model is in a way a test of your model fit. One can normally validate their model through the following ways:
– Incrementality testing
– Geo-lift
– Causal experiments
– Out of sample MAPE/MAE/RMSE
Without a strong focus on Calibration, Validation would be a futile exercise.
ICYMI, I wrote about the importance of goodness of fit in Marketing Mix Modeling (MMM) yesterday (link in resources).
One should not compromise on calibration.
The path to validating any MMM model (or any regression model) starts with Goodness of Fit.
Resources:
Validating MMM Models the right way – https://www.linkedin.com/posts/venkat-raman-analytics_marketingmixmodeling-statistics-linearregression-activity-7132601614355374082-lWLK?utm_source=share&utm_medium=member_desktop
