The Chief Scrutinizing Officer (CSO)
Every Marketing Mix Modeling (MMM) project or for that matter any Data Science project needs a Chief Scrutinizing Officer (CSO).
Here is how the skill of scrutinizing comes in handy:
1. Scrutinizing the data:
The first step in any Data Science is to examine the data carefully. In our recent projects, we found that the media investments for a particular medium were in a range of 100k, but then one data point showed value in millions. We requested the client to recheck the data and it turned out there was indeed a mistake in capturing the data!!
2. Scrutinizing the models:
This phase involves asking the data science team questions on the models built. Starting from the choice of model to the accuracy metrics of the model.
Not to mention, one must scrutinize whether the model makes domain sense. For example, one must understand how contributions (effect share) stand vis-a-vis marketing investments (spend share). A medium/campaign with miniscule investments cannot have a very high contribution.
3. Scrutinize the data visualization:
Recently I came across a funny visualization of average adult male height across countries where the depiction reminded me of Gulliver and Lilliput. ๐
Similar funny (and even dangerous) things happen when one gets the axis wrong. Narrowing or widening of the interval in the axis can lead to wrong interpretations. (Check the posts link under resources highlighting this issue in detail)
4. Scrutinize ROI Numbers:
In marketing science, every ROI number must be scrutinized.
For e.g. performance marketing campaigns showing high ROIs could indicate that TV is not as effective. But studies show that TV builds long term brand equity.
5. Scrutinizing budget allocation:
In marketing, there is no one best marketing strategy but many. Optimization could provide a mathematically feasible and optimal solution but might not provide the solution which is best for the brand in reality. Hence scrutinizing the output from the optimizer becomes paramount. Often adding domain driven constraints on the optimizer results in an accurate and realistic solution.
6. Scrutinizing the insights:
Before the final insights are unraveled to the client, it is important to play devil’s advocate with your team internally. Every client question needs to be anticipated and prepared for.
At the end of the day, to ensure the success of a data science project, scrutiny is necessary.
Resources:
Get the Y axis right -> https://www.linkedin.com/posts/ridhima-kumar7_datavisualization-dataanalysis-datascience-activity-6969204896197734400-R-w5?utm_source=share&utm_medium=member_desktop
Image credit: https://pin.it/4XM5270
