Decision History Scoreboard – MMM vs Marketers
At the beginning of every year, we at Aryma Labs chalk out interesting product features to build.
One idea that we discussed early this year was ‘Decision History Scoreboard’.
Why this idea? Well let me ask you the reader the following interesting questions:
▪️How many times has a seasoned marketer vetoed an MMM recommendation and was proven right?
or
▪️How many times did MMM go against conventional wisdom, its suggestions get implemented and was proven right?
or
▪️How many times both marketers and MMM were on the same page, and were proven correct?
or
▪️How many times both marketers and MMM were on the same page, and were proven wrong?
Most organizations don’t track these. But they should.
The score keeping is not to create a rift but to observe and learn from past decisions.
We have seen both cases where a marketer vetoed a MMM suggestion or questioned its numbers and was proven right vs where the MMM suggestions / numbers though appeared contrary to conventional
wisdom was proven right.
But I do have a little soft corner for marketers.
📌 The quiet wins of marketer intuition
Every experienced CMO / brand manager has done this at some point:
“This doesn’t look right. Don’t cut on this channel that aggressively.”
“Digital may look saturated, but we are in a competitive burst phase.”
“The model is missing something here.”
And sometimes they are absolutely right with the above.
But this doesn’t mean the model is useless.
Models operate on observed data and marketers operate on lived context.
Venkat Raman in one of his posts talked about context windows in LLMs. And he rightly opined that for MMM, the context window is only 2-3 years or whatever data period you feed the model.
But a seasoned marketer’s context window is their entire career !
Yes LLMs are good at pattern matching and has decent ‘reasoning’. But they will be accurate in so far as they have seen the patterns.
When MMM is right, it is hailed as data driven decision making When a marketer is right, It is often dismissed as gut feel.
That’s a bit unfortunate misattribution because good marketers are not guessing. They are compressing years of experience into a decision.
📌 How Decision History augments MMM and Peripheral Agentic MMM
Imagine if every marketing decision and its outcomes was recorded. With added details of context – why those decisions were made and under what circumstances.
This data could richly enhance the explainability or reasoning of the AI models.
📌 Decision History in Aryma AI Products
This quarter we will be rolling out some really cool products that leverage decision history. AI needs human experts and human experts need AI.
Despite much of wisdom of marketers, we all know human minds are fallible and are forgetful. A genius move or idea may not come repeatedly into recollection.
But an AI that has captured this ‘idea’ can pattern match and apply it in the right situation.
