Blogs

Are you modeling carryover and lagged effects for promotions and seasonality effects? You Should. In Marketing Mix Modeling (MMM), almost all vendors adstock transform their media variables. This is done because the ad displayed today has a lingering impact into the future and more airing of ad sometimes stops having a incremental effect (the saturation). However when it comes to assessing the impact of a promotional effect or holiday, no carryover or lagged effect is taken into consideration. In most MMMs, holidays and events are treated like switches. Example: Black Friday = 1 All other days = 0 This form of encoding is also known as ‘Dummy encoding’. But this

The USS Scorpion Story: A Bayesian Myth That Refuses to Die I have seen a lot of debates on Frequentist vs Bayesian. Bayesians often bring up the USS Scorpion as a success story of Bayesian method application in real world. For context: USS Scorpion- a US nuclear submarine, imploded in the Atlantic in 1968, triggering an urgent search for its wreckage. 📌 The Exaggeration of Bayesian Success The success of Bayesian methods is often exaggerated and mythologized in the case of the USS Scorpion. Firstly, the search area was already significantly narrowed down using hydroacoustic signals. This happened before any meaningful application of Bayesian methods. In a way, the hardest

Peripheral Agentic MMM Lot of MMM vendors have started to use the word ‘Agentic MMM’. As a person who started with TF-IDF embeddings and have seen the evolution of NLP to AI, I somehow can’t come to terms with the word ‘Agentic MMM’ yet. I believe at the current stage, it is a bit of misnomer. In my opinion, for something to be truly Agentic MMM, the core nucleus of MMM should be agent-driven. That nucleus is the model itself: – Variable selection – Functional form decisions – Adstock specification – Saturation curves – Bias diagnostics – Causal validity checks AI still can’t do this reliably. Not at the level where

Bayesian MMM’s Token and Context Window Tax Everyone debates Bayesian vs Frequentist MMM on statistical grounds. Now this debate has moved to a new turf – AI. Last week I wrote about Claude code and Jevons Paradox in MMM. I think regardless of all the warnings, people will use Claude or similar LLMs to write MMM code. This brings us to the question- Bayesian MMM or Frequentist MMM? At least from a statistical point of view it has been clearly proven that Frequentist MMM is the superior one. Every Bayesian MMM zeroes in on a point estimate eventually anyways (see link to post in comments). 📌 What are Tokens and

Don’t apply Machine Learning Yardsticks to MMM I had a interesting question from a client last week “Why doesn’t Aryma Labs have a hold out sample for MMM?” The client was referring to the popular “Train/Test” paradigm. At Aryma Labs, we have a different philosophy or perhaps the same philosophy that yesteryear statisticians had “Don’t waste data”. Ideally if your goal is inference, you don’t need to train/test split your data. In case of prediction, train/test split is justified as the model making such predictions is often black box-ish. The only way you know your model is working is by testing the predictive accuracy of the model on an ‘unseen

Claude Code and Jevons Paradox in MMM There’s a lot of excitement around Claude. Naturally, the question is: Will this make building Marketing Mix Models (MMM) easier? Short answer: Yes. But what happens when MMM becomes easy? There is a concept of AI Slop. Because it has become easier to generate content, anybody with an access to LLM can write a surface level well sounding article on any subject. Similarly we will see MMM Slop. MMM that looks good on the surface but misses crucial pieces. 📌 Enter Jevons Paradox Jevons Paradox states: When efficiency increases, consumption doesn’t go down. It goes up. If Claude makes MMM – Faster to

The Myth of “Triangulation = Truth” “We used DiD, Synthetic Control and BSTS and triangulated the results.” “We used MMM, MTA and Experimentation and triangulated the results.” Both these statements sound very scientific and sophisticated, but they are not. I have already talked about the second one in my post before (link in comments). So let me focus on the first one. To be really honest, I really felt very disappointed that some marketing measurement vendors and ‘causal experts’ celebrated a paper that claims to triangulate between DiD, SCM and BSTS. Which then helped them reallocate $25 Mn !! Anybody who has a decent knowledge of causality will know, how

MMM’s Scenario Planning vs Budget Optimization – How They are Different In Marketing Mix Modeling (MMM), two terms often get used interchangeably: ▪️Scenario Planning ▪️Budget Optimization They are related. They use the same underlying model. But they operate at very different levels of decision making. I view Scenario Planning to be more of a “Tactical Adjustment” where as Budget Optimization is more of “Strategy Change”. 📌 Scenario Planning = Tactics The simplest heuristic to remember scenario planning is, It asks the question – “What happens if I do X ?” Now this ‘X’ could be any change in spends to your marketing / media variables. I am saying Marketing also

MMM is about hedging against bad bets. Some vendors tell clients that – make many bets, some will fail. But on average, you will win. I disagree, the above advice or logic comes from gambling. The very purpose of measurement is to ‘Not to gamble’ with your marketing spends. I take inspiration from Nassim Taleb’s writing and I think all marketers should do too. Taleb opines in many of his books that the goal is to avoid total ruin. Taking many irrational bets inflates tail risks and risks ruin. He also opines that apart from Financial traders, lawyers are the ones who get this concept. They always try to hedge

The Pecking Order of Meta’s Engaged-Through Metrics Last week, I wrote about Meta’s click based attribution recategorization. In it, I had opined that Meta’s ‘Engaged Through’ Attribution could help marketers glean signal about their brand performance (i.e. awareness, affinity, consideration etc.) The ‘Engaged-Through Attribution’ consists of Likes, Comment, Save, Share and Engaged-View. This raises a important question – “Are all Engagement signals equal?” I don’t think so. If anything, they have a very clear pecking order. I define this pecking order based on : – Intent (Did the user mean to engage?) – Effort (How much friction was involved?) – Downstream Effect (Does it lead to action?) 📌 The Pecking