Marketers are sometimes given bad advice that they should not go for advanced methods for marketing impact measurements.
Instead they are suggested to adopt simple analysis like correlation.
Ill advised suggestions like “All you need is only correlation” will do more harm than good.
So I will detail out the advice that I came across in one such post and explain why it is wrong to simply rely only on correlation.
The advice goes something like this:
Calculate correlation coefficient between CAC and advertising spends.
If you observe a correlation coefficient closer to 1, then your ad efforts were a waste.
Now why is the above advice flawed?
For this we will have to delve a little bit into statistics.
Correlation has limitations:
1. Correlation assumes linear relationship.
The first question to ask even before calculating Correlation coefficient is that: is the relationship between my variables linear?
Does CAC and ad spends have a linear relationship? In most cases you will find that this is not the case.
So application correlation will simply give you erroneous results.
2. Correlation is prone to outliers.
Now what is an outlier is subjective. But lets assume a brand is spending on $10k every month on ad spends. Now if this brand suddenly spent $50k in 1 month, then that would be considered an outlier. Does such a scenario happen? yes, such pattern is observable during BFCM.
Now imagine you have this huge bump in ad spends for 1 month. Then you calculate correlation between CAC and ad spends.
The corr coefficient will be either severely affected. Now if this outlier inflated your correlation coefficient, you would assume that your ad spends are not working !! You see, just one data point can really make you take bad decisions.
3. Correlation != causation
In the above advice, one major drawback is that we are assuming the relationship to be linear and also we believe there is a causal relationship between CAC and ad spends.
Correlation is an insufficient condition for causality.
CAC (in most cases) can get impacted by variables other than ad spends. But our correlation coefficient only accounts for ad spends.
If you are making million dollar marketing decisions, then correlation coefficient simply won’t cut it.
So now the next question is where do we go from here?
We need a solution to measure marketing impact that can also factor in non linear relationships, is robust to outliers and can also factor in causality.
The answer – marketing mix modeling (MMM).
Plain vanilla regression will have most of the drawbacks as that of correlation.
But with a lot more bells and whistles (adstocks, robust regression,
controlling for other variables, incrementality and causal experiments), MMM becomes a potent and efficient solution.
So is MMM a simple method? No. But it is a solution that most marketers need.
As always, pursuit of simple solutions is great but pls see to it that simple solution adequately solves the problem.
