Experiment Design

Design robust experiments to test product hypotheses.

When to Use

When you have a hypothesis and need to design an experiment to validate it.

The Process

1. Factor Breakdown

Analyze your goal and system:

• What factors could influence the outcome?
• Which are controllable vs. environmental?
• What are the key variables?

2. Experiment Structure

For each factor, define:

• Hypothesis: What you expect to happen
• Independent variable: What you're changing
• Dependent variable: What you're measuring
• Control group: Baseline comparison
• Experimental group: Who gets the change
• Measurement method: How you'll collect data
• Confounding variables: What else could affect results

3. Sample Design

• Who participates?
• How many needed for statistical significance?
• How will you recruit/select?

4. Timeline

• How long to run?
• When to check results?
• What's the minimum detectable effect?

5. Stop/Scale Rules

• What results mean "stop"?
• What results mean "scale"?
• What's inconclusive?

Output Format

For each experiment:

• Hypothesis statement
• Variables (independent, dependent)
• Groups (control, experimental)
• Sample size and selection
• Measurement approach
• Success criteria
• Stop/scale rules

When Not to Use

Do not use this skill when the request is unrelated, low-stakes, or better handled by a simpler direct response.