Standardization before Regularization in MMM
In my last post (link under resources), I covered the topics of Multicollinearity and Endogeneity. And how solving for Multicollinearity can lead to Endogeneity.
To solve for Multicollinearity, many adopt regularization like Lasso or Ridge. But here are some key points to keep in mind.
๐ L1 and L2 are the most common regularization techniques. However, one common mistake while using this approach is not applying standardization to the data.
๐ Many libraries do not explicitly tell the user to ‘standardize’ their data before applying regularization by default!
๐ L1 and L2 techniques penalize large coefficient values more while applying shrinkage. If the data are not standardized before applying regularization, it can penalize features with larger coefficients and can give inaccurate results.
Let’s take an example in Marketing Mix Modeling (MMM)
Here we will take the advertisement spends data.
If the TV spends are in dollars, then the coefficient will be 100 times larger than the coefficient which we get when the TV spends are in cents. E.g. (USD 1000 vs 100, 000 cents).
Although the amount is same but the magnitude of these two figures is different thus changing the scale of the coefficient. The coefficient of USD would be larger and thus will be penalized more.
Similarly, if Digital spends are in cents and TV spends in dollars, the TV coefficient will be penalized more in regularization.
๐ So, remember the features should be standardized before using any regularization technique.
In case you want to know more about Lasso and Ridge, a compilation of stackexchange questions and answers on the topic is under resources.
Resources:
Multicollinearity and Endogeneity – https://www.linkedin.com/posts/ridhima-kumar7_marketingmixmodeling-marketingeffectiveness-activity-7023899326837858304-RtO3?utm_source=share&utm_medium=member_desktop
Everything you wanted to know about Lasso and Ridge – https://www.linkedin.com/posts/venkat-raman-analytics_lasso-ridge-regression-everything-you-activity-6959735196116037632-gItp?utm_source=share&utm_medium=member_desktop
