A/B TestingRunning an experiment that produces a trustworthy answer.Business & ManagementThe discipline is in the design, not the analysis.1State one hypothesisWhat change, on what metric, by how much.2Choose the primary metricOne. Guardrail metrics may be watched, not optimised.3Calculate the sample sizeFrom the baseline rate and the effect you care about detecting.4Randomise properlyBy user, not by session. Assign once and keep it.5Run the full durationAt least one full weekly cycle. Do not peek and stop.6Analyse onceAgainst the pre-declared metric.COMMON ERRORSStopping when it looks significantTesting many metrics and reporting the winnerToo small a sample to detect the effectIgnoring novelty effectsPeeking and stopping when p < 0.05 raises the real false-positive ratefar above 5%. Fix the duration in advance.A/B Testinglearnposters.com
A/B Testing — printable business wall chart from LearnPosters. Free vector PDF, US Letter and A4.

A/B Testing, step by step

The discipline is in the design, not the analysis.

  1. State one hypothesisWhat change, on what metric, by how much.
  2. Choose the primary metricOne. Guardrail metrics may be watched, not optimised.
  3. Calculate the sample sizeFrom the baseline rate and the effect you care about detecting.
  4. Randomise properlyBy user, not by session. Assign once and keep it.
  5. Run the full durationAt least one full weekly cycle. Do not peek and stop.
  6. Analyse onceAgainst the pre-declared metric.

Common errors

  • Stopping when it looks significant
  • Testing many metrics and reporting the winner
  • Too small a sample to detect the effect
  • Ignoring novelty effects

Peeking and stopping when p < 0.05 raises the real false-positive rate far above 5%. Fix the duration in advance.

Questions about the A/B Testing poster

What’s on the A/B Testing poster?
6 numbered steps, with a full worked run-through. The discipline is in the design, not the analysis. State one hypothesis — What change, on what metric, by how much.; Choose the primary metric — One. Guardrail metrics may be watched, not optimised.; Calculate the sample size — From the baseline rate and the effect you care about dete…; Randomise properly — By user, not by session. Assign once and keep it.; Run the full duration — At least one full weekly cycle. Do not peek and stop.; Analyse once — Against the pre-declared metric.. Peeking and stopping when p < 0.05 raises the real false-positive rate far above 5%. Fix the duration in advance.
Who is the A/B Testing poster for?
A/B Testing belongs to the Business & Management section rather than to a school year, because business is not something one grade owns. Anyone learning analytics can pin it up — a beginner, a student mid-course, or someone revising years later.
When should you use the A/B Testing poster?
Fix the sample size and the duration before starting. Stopping early inflates false positives badly. A wall chart earns its place by being glanceable from where the work is happening, so A/B Testing belongs on the wall where that business work actually happens, within glancing distance, rather than filed away.
What other posters go with A/B Testing?
Cohort Analysis, Analysis Traps and Reading a Relationship sit alongside A/B Testing in the Business & Management section. Printed together they make a wall rather than a single sheet, which is how a reference set actually gets used.Cohort AnalysisAnalysis TrapsReading a Relationship
Is the A/B Testing poster free to download and print?
Yes. A/B Testing downloads as a free PDF with no account, no email and no watermark, like everything else in the Business & Management section. Print as many copies as you like for a home, a classroom, a library or a tutoring group; reselling the file is the only thing the licence rules out.Read the licence
What size does the A/B Testing poster print at?
A/B Testing is a vector PDF laid out for US Letter, and prints on A4 with Fit to page — the same file, no separate download. Because every mark on it is drawn rather than photographed, it stays sharp enlarged to A3, A2 or A1 at a copy shop. Colour carries emphasis only, so a greyscale print of A/B Testing loses nothing.Printing guide

Related posters

Charts that sit alongside A/B Testing on the same wall.

Browse every Business & Management poster, or start from the full catalogue.