Should we forget the older data in Marketing Mix Modeling (MMM)?

One problem in MMM that very few discuss deeply is:

How much should the past influence the present?

The industry thumb rule is to use at least 2 years of monthly data or 1 year of weekly data, mainly to capture seasonality effects reliably.

But people also recommend “More the better” for data.

But what if the oldest year behaved very differently from the current?

In year 1, lets say the brand experimented on a digital channel for the first time and experience good ROAS, low saturation effects and very high contribution. But in later years, the effect wanned a lot.

The old behaviour can quietly influence the following even in the latest year.

• Adstock decay
• Contribution shares
• Channel efficiency

📌 The Statistical Take

Statistically, in OLS regression, an old data point has no inherent decay in influence based on its temporality. All data points are weighted equally. However, where the data point starts having influence on downstream data points is based on two things:

1) Leverage
2) Residual size

Leverage:

Imagine a lever for convenience. Few years back I also wrote a post on Moments in Statistics (link in comments).

If you a have lever, you would notice that it is far easier to push down or pull up the lever from the edges, assuming the fulcrum is right in the middle.

Now imagine a linear regression. For simplicity, a regression straight line (though in reality it can easily be any squiggly shape, the linearity in regression is wrt to only parameters).

Now if you have data points that are outliers in the very beginning, it can have a huge impact on the slope of the regression line. (see interactive demo).

Residual Size

This is basically how far the regression prediction is from ground truth, in MMM we often notice that the first data point is never accurately predicted. This can lead to inflated MAPE for the whole time period.

Now that we have understood Leverage and Residual size, we get to the useful metric – Influence. Influence is defined as Leverage * Residual size.

📌 The Solutions

Rolling window
Drop old data points, keep only recent n observations. Now this could mean if you built a model using 3 years of data (2023, 24, 25, 26), one can drop the year 2023.

Cook’s distance – We at Aryma Labs compute something called the cook’s distance. It basically tells you how much the model changes when omitting a data point. So you get to know the influence of that data point.

Recursive least squares – This is a bit advanced algorithm. This algorithm basically allows you to selectively ‘discount’ the influence of old data points.

Exponential smoothing – This Weights recent observations more heavily with exponentially decreasing weights for older data.

But this may not be good application for MMM as it impacts the adstock effect.

Overall, I just wanted to spark a discussion on a technical MMM topic that rarely gets discussed. Your thoughts are welcome