How Leaders Should Decide Which Experiments to Pursue
Choose experiments that answer consequential questions, generate trustworthy evidence, control risk, and force a scale, iterate, continue, or stop decision.
Five Principles for Choosing Experiments
- Start with a consequential business question.
- Require a measurable hypothesis and decision path.
- Score impact, confidence, learning, effort, and risk.
- Verify instrumentation, capacity, safeguards, and dependencies.
- Scale, iterate, continue, or stop with evidence.
The Experiment Decision Process
| Step | What to do | Output | Owner | Timeframe |
|---|---|---|---|---|
| 1 | Define the business question, hypothesis, audience, and decision | Standard experiment brief | Experiment owner | Before scoring |
| 2 | Score fit, impact, confidence, learning, effort, risk, and readiness | Ranked experiment backlog | Growth or portfolio lead | Monthly |
| 3 | Check data, sample, dependencies, safeguards, and capacity | Approved test design | Analytics and functional owners | Before launch |
| 4 | Run the test with primary, guardrail, and stopping metrics | Valid evidence and documented limits | Experiment owner | Test duration |
| 5 | Scale, iterate, continue, or stop and update playbooks | Decision record and next action | Decision owner | Within one week |
Build an Experiment Decision Portfolio
Experiment selection should begin with the decision the organization needs to make. Require every proposal to state the business question, target audience, hypothesis, expected behavior change, primary metric, guardrail metrics, minimum evidence, owner, cost, duration, and the decision that follows a positive, negative, or inconclusive result. Ideas without a measurable decision path should not enter the backlog.
Score qualified experiments using strategic fit, expected customer or revenue impact, confidence, learning value, effort, risk, and measurement readiness. TPG recommends ICE- or RICE-style scoring, then adjusting for strategic fit, risk, instrumentation, and dependencies. Protect some capacity for fast incremental tests, some for foundational measurement or process questions, and a smaller portion for higher-uncertainty bets. Do not let low-risk optimization crowd out experiments that could change the growth model.
Before launch, confirm sample feasibility, data quality, exposure limits, compliance, and a stopping rule. After the test, choose one action: scale, iterate, continue collecting evidence, or stop. Record the result and decision so teams do not repeat failed tests or lose validated learning. Review the backlog monthly and refresh it when strategy, customer evidence, or capacity changes.
Source: pedowitzgroup.com, 2026; mckinsey.com, 2015-2024; gartner.com, 2024-2026
TPG Point of View
Use an Experiment Decision Portfolio - question, hypothesis, value, confidence, learning, risk, evidence, and decision.
Why TPG? TPG connects governed experiments to pipeline, velocity, win rate, retention, and commercial scale decisions.
Balance the Experiment Portfolio
| Option | Best for | Pros | Cons | TPG POV |
|---|---|---|---|---|
| Quick optimization | Known funnel friction with reliable measurement | Fast learning; low exposure; easy adoption | Often creates incremental lift only | Use to fund and improve the system |
| Foundational test | Uncertain audience, offer, process, or measurement assumption | Removes downstream uncertainty | May not create immediate revenue | Prioritize when many initiatives depend on it |
| Strategic bet | New motion, market, offer, channel, or operating model | Can create disproportionate growth | Higher uncertainty and coordination cost | Limit exposure and define stage gates |
| Instrumentation test | Weak data, attribution, routing, or event capture | Improves future decision quality | Delays visible optimization | Fix measurement before scaling tests |
Frequently Asked Questions
Use strategic fit, expected impact, confidence, learning value, effort, risk, dependencies, audience reach, and measurement readiness. Weight the criteria to reflect the current strategy.
Set a work-in-progress limit based on available traffic, analysis capacity, operational risk, and the ability to act on results. More concurrent tests are not better when they create interference or weak follow-through.
Maintain a portfolio with near-term optimization, foundational learning, and a smaller number of higher-uncertainty bets. Protect bold tests from being displaced by easy but low-value ideas.
Stop when guardrail metrics breach agreed limits, the test creates material customer or compliance risk, instrumentation fails, the hypothesis becomes irrelevant, or the required sample is no longer feasible.
Validate the analysis, document the result and limitations, and make an explicit scale, iterate, continue, or stop decision. Update playbooks, templates, budgets, and the experiment backlog.