Email A/B test plan with one variable and a fair sample size
Plans an email split test properly: one clear hypothesis, one variable changed, a sample size large enough to trust, the metric that decides the winner and the date to call it. Then it reads the results and says whether the difference is real. Use it when a team tests emails but keeps picking winners from tiny lists or mixed changes.
The output this prompt produced, unedited.
The prompt
Constraints first
Change one thing between version A and version B; everything else is identical.
Pick the deciding metric before sending: clicks for content tests, opens only for subject line tests, orders for offer tests.
Never call a winner from fewer than the sample size worked out below.
Opens are unreliable on many mail apps; say so when a test relies on them.
Inputs
List size and usual results: [e.g. 18,000 subscribers, 2.4% click rate]
What to test: [e.g. a product grid against a single hero product]
Why: [e.g. we think a single product makes the choice easier]
Send window: [e.g. Tuesday 11 am]
Tool used: [e.g. any email platform with a split test feature]
Plan
Write the hypothesis: If we [change], then [metric] will [rise or fall], because [reason].
Describe version A and version B in one line each.
Work out the sample per version needed to detect the expected lift, and say if the list is too small.
Set the test length and the rule for sending the winner to the rest.
Output format
Test card: hypothesis, versions, metric, sample per version, end date.
Results table to fill: | Version | Sent | Clicks | Rate | Orders |
Once results are pasted: whether the gap is likely real or noise, in plain words.
One follow-up test the result suggests.
email ab testsplit testing emailsemail experiment planemail sample sizeemail test resultsemail optimisation
How to run this
Copy the prompt and paste it into Gemini, then set the aspect ratio to 3:4. No reference photo is needed for this one.
The prompt is written in Markdown with each part under its own heading, which is what the copy action hands over. Change one section at a time, keeping the rest intact, so you can tell which change did the work.