Humans Don’t Think in Probabilities

So Bayesians all over the world are celebrating that Elon Musk himself has endorsed “Bayes Theorem or Bayesian Thinking.”

But sorry Bayesians, as great an entrepreneur Musk is, he like all humans can’t be right about everything. And lately he has been wrong about a lot of things. I won’t get into politics here and will restrict myself to what I know best – statistics.

Humans don’t think in probabilities. We think we do because it makes us look smarter than we actually are. Thinking in probabilities means thinking in ranges, not point estimates. There was a recent example where Uber experimented with a price range vs a specific price. No prize for guessing which one won (link in comments).

📌 Not a problem with Bayesian thinking but with the tool.

Philosophically, there is nothing wrong in believing that we should update our beliefs based on prior knowledge or new data. But the mechanism used to accomplish this is often deeply unscientific, subjective and unfalsifiable. One simply can’t give a convincing answer for choosing a particular prior.

Most often priors are chosen for convenience – either to help convergence or to ensure results come back favorably (no negative signs on spends in MMM).

Any questions about the veracity of Bayesian techniques are usually met with following four ways.

▪️You are labelled dumb for not understanding an “advanced” technique.
▪️Credentialism waving “I have a PhD, do you think I am wrong?”
▪️Appeal to popularity “So many people use Bayes, can they all be wrong?”
▪️Or you get blocked for causing cognitive dissonance 😅.

📌 The skepticism of marketers towards statistics

Marketers who truly know their craft are naturally skeptical of statistical techniques and rightfully so. Statistics can never perfectly abstract ground reality. Domain knowledge is the best guardrail. Yet many MMM vendors (especially Bayesian ones) talk over clients rather than talk to them.

The same pattern follows:

▪️See these scientific looking charts.
▪️See these complex equations.
▪️See the statisticians who built them.

Can we really be wrong? Just trust us. Trust the Bayesian Methods.

Marketers sometimes go along with this, but some eventually develop disenchantment.

Just because marketers cannot translate experience into equations or scientific jargons does not make them less intelligent.

I am glad that before my statistics degree, I took up many small sales and marketing jobs. It gives you an added lens of reality.

Every domain would benefit if Bayesian techniques were treated not as unquestionable science, but as tools that deserve scrutiny. Yes, they may have uses but they must be examined critically.

Link to the tweet in comments (do read the replies).