Split Test Segment Size Calculator

Calculates when to act on a split test, based on desired confidence level

The split test sample size calculator will show you how much traffic you need to send to each test segment, based on how confident you want to be in the results.

1. Choose your desired confidence level.
2. Put in your average conversion ratio.
3. The split test sample size calculator will show you the total amount of traffic required.

User Defined Fields

• Conversion Ratio - If you have less than 5 segments in your test use a conversion ratio of 1 for each null segment.
• Confidence Level - We suggest 99% or even 99.9%. The more confident you want to be in the test the more traffic you will need.
• Confidence Interval = ratio between the size of conversion range, and the conversion percentage.

Additional Fields (look but don't touch, unless you want to get deep into the math)

• Standard Error = How much a new observation is likely to be different from historical results. Inversely proportional to Sample Size.
• Normal Percentile = The number of standard deviations in each direction that define our conversion range.
• Half Width = The Standard Error x Normal Percentile.

The Split Test Sample Size Calculator is from the Metric & Analytical Marketing workshop presented at Internext by Jeff Random & Katie Smith from (THROB) 3ob.com .

The workshop also included the Marketing Metrics & Website Analysis Worksheets that allow you to calculate the effectiveness of different strategies & segments to efficiently increase the revenues from your site.

The spreadsheets are provided in .ods format, which is the Open Office spreadsheet format. Open Office is a multiplatform and multilingual office suite and an open-source project. Compatible with all other major office suites, it's free to download, use, and distribute.