Experiment
Designing experiments that end arguments
An experiment is only useful if both sides agree in advance what result would change their mind. A short protocol we use before running anything.
On this page
The pre-commitment
Write down the decision the experiment will inform, the result that would change it, and the smallest sample that would make the result credible. If any of the three cannot be written, the test is activity, not evidence.
Cost discipline
Budget the test against the value of the decision, not the size of the channel. A question worth a small decision does not deserve a large experiment.
Was this useful?
If this describes something happening in your business, the fastest way to find out whether it's worth a conversation is to see whether we're a fit.
Related blogs
Spend & efficiency
Your acquisition cost isn't always the problem
Rising CAC is usually treated as a media issue. More often it is an offer, margin or conversion issue that media is being asked to absorb.
Priorities & strategy
More marketing activity won't fix an unclear growth strategy
Activity is easy to add and hard to evaluate. When direction is unclear, adding more of it mostly increases the cost of finding out.
Related insights
Framework
The cost of answering the wrong question
Most underperforming growth programmes are executed well. They are simply answering a question the business did not need answered. A working method for finding the real one.
Analysis
Acquisition cost is a symptom, not a diagnosis
Rising CAC is usually treated as a media problem. Broken down properly, it is often an offer, margin or conversion problem wearing a media costume.
Related capability
Measurement & Experimentation
Used when the numbers cannot be trusted.
Explore Capabilities