wowprompt

A/B test result readout with statistical significance

Calculates lift, confidence and significance for an A/B test from raw counts and writes a decision-ready readout. Use it when a test ends and you need to say clearly whether to ship, stop or keep running.

A results table with conversion rates, lift and confidence interval for each variant, a significance verdict and a one-paragraph recommendation
The output this prompt produced, unedited.

The prompt

Fill in these first

  • What was tested: [e.g. new checkout button text]
  • Primary metric: [e.g. purchase conversion rate]
  • Variant A, control: visitors [number], conversions [number]
  • Variant B, and any others: visitors [number], conversions [number]
  • Test dates and duration: [start date, end date]
  • Significance level agreed before the test: [e.g. 95 percent]
  • Minimum lift worth shipping: [e.g. 3 percent relative]
  • Secondary metrics, optional: [e.g. average order value, refund rate]

Task

Analyse the test and write a readout a product manager can act on in five minutes.

Output format

  1. Results table:

| Variant | Visitors | Conversions | Conversion rate | Relative lift vs A | 95 percent confidence interval of lift | p-value |

  1. Method: one line naming the test used, such as a two-proportion z-test, and why it fits.
  2. Verdict: one of significant winner, significant loser, or not significant.
  3. Practical check: whether the lift clears the minimum worth shipping.
  4. Risks to the result: sample ratio mismatch, short duration, novelty effect, peeking.
  5. Recommendation: ship, stop or extend, with the extra sample size needed if extending.

Quality rules

  • Show the formulas used and round rates to two decimal places.
  • Check that traffic split matches the intended split; flag a gap above 1 percentage point.
  • Never call a result significant because it looks large; use the agreed level only.
  • Separate statistical significance from business value in plain words.
  • If the test ran under one full week, say that weekday effects may not be covered.

Example

| B | 10,240 | 532 | 5.20% | +8.3% | +1.1% to +15.9% | 0.021 |

ab test analysisstatistical significance calculatorconversion rate liftsplit test resultsexperiment readout

How to run this

Copy the prompt and paste it into Gemini, then set the aspect ratio to 3:4. No reference photo is needed for this one.

The prompt is written in Markdown with each part under its own heading, which is what the copy action hands over. Change one section at a time, keeping the rest intact, so you can tell which change did the work.

Related prompts

All productivity