Framework for designing structured product feature tests
PromptBusiness & MoneySuperhuman · 3h ago

Framework for designing structured product feature tests

Helps product managers structure controlled experiments to validate new software features with actionable decision guidelines.

Copy this prompt

Act as a senior product manager and experimentation strategist. Help me design a rigorous product experiment for [FEATURE OR PROPOSED CHANGE] that produces a clear decision, not just interesting data.

Context:
- Product: [PRODUCT NAME]
- Target users: [USER SEGMENT]
- Problem: [PROBLEM STATEMENT]
- Proposed change: [FEATURE VARIATION]
- Hypothesis: [EXPECTED OUTCOME]
- Primary business goal: [ACTIVATION / RETENTION / CONVERSION / ENGAGEMENT / REVENUE]
- Current baseline: [METRIC AND BASELINE VALUE]
- Sample size available: [AVAILABLE USER VOLUME]
- Experiment duration: [TIMEFRAME]
- Constraints: [ENGINEERING OR LEGAL CONSTRAINTS]
- Known risks: [POTENTIAL RISKS]

Design the experiment with:
1. A precise hypothesis
2. Control and treatment definitions
3. Primary success metric
4. Secondary metrics
5. Guardrail metrics
6. Target segment and exclusions
7. Minimum meaningful effect to detect
8. Recommended duration approach
9. Key confounders or failure modes
10. Clear launch, stop, and rollback criteria

Separate what would prove the hypothesis from what would merely look positive. Identify any potential misleading factors like novelty effects or selection bias.

Finish with an experiment brief, pre-launch checklist, results framework, and clear Ship / Iterate / Reject decision rules.
Read the original