marketingscience.dev

writing

Notes from the measurement room.

Short pieces on what is worth modeling, what is not, and how the workflows in /skills got their shape.

Bayesian vs. Frequentist MMM Is a False Dichotomy

The Bayesian-vs-frequentist MMM debate has gotten religious, and it mostly misses the point. Both methods solve the same structural equation, both regularize, and neither can identify effects from observational data alone. Here's what actually changes — and the three questions to ask any MMM vendor.

The Three Cardinal Sins of Experimentation: Peeking, Early Stopping, and Data Dredging

Power is the unappreciated cousin of significance, and ignoring it is how good analysts get burned. Here's the iPhone-vs-telescope intuition for power, why peeking inflates your false-positive rate toward 100%, and how data dredging lets you 'drain the river and find something' that was never there.

Context Engineering for Long Analytics Sessions (Escaping the Dumb Zone)

Long analytics sessions rot. Here's how I keep Claude Code and Codex sharp across a two-hour MMM build — six patterns, one war story, and why a bigger context window is just a bigger haystack.

We're Statisticians, Not Magicians: Why Some Channels Can't Be Modeled

No variability, no model. Affiliates aren't a media channel. CRM isn't media. You don't choose theta — the data does. A field-tested rant on the limits of MMM, and the two places teams cheat.

The Measurement Trinity: MMM, Attribution & Experiments

Three methods, one question — how much is my channel worth? When each is trustworthy, how each is biased, and why writing down your assumptions in a DAG matters more than the code you'll generate to run them.

7 MMM Vendor Red Flags (and How to Rescue a Distrusted Model)

How to tell whether your MMM vendor built you something useful or something sophisticated-looking — and a field-tested playbook for saving a model the org has already stopped trusting, without burning it down and starting from scratch.

Why this exists

A short note on what marketingscience.dev is and what's going to live here.