Two-Sample Sample Size Calculation with DataStatPro: Zero to Hero Tutorial

This tutorial takes you from independent-group study planning through effect sizes, allocation ratios, mean differences, proportion differences, calculator use, interpretation, reporting, and common mistakes. It is designed for trials, A/B tests, cohort comparisons, intervention studies, and independent group experiments.

Table of Contents

1. Prerequisites and Background Concepts
2. What Is Two-Sample Sample Size Planning?
3. When to Use Two-Sample Calculations
4. The Mathematics Behind Two-Sample Power
5. Allocation Ratios and Attrition
6. Using the DataStatPro Calculator
7. Worked Examples
8. Common Mistakes and How to Avoid Them
9. Troubleshooting
10. Quick Reference Cheat Sheet

1. Prerequisites and Background Concepts

1.1 Independent Groups

Two-sample planning applies when observations in one group are independent of observations in the other group. Examples include treatment vs control, exposed vs unexposed, or design A vs design B.

1.2 Primary Outcome

Choose one primary outcome for the main sample size justification. Secondary outcomes may be underpowered unless planned separately.

1.3 Minimum Meaningful Difference

The key input is the smallest difference worth detecting, not the largest plausible difference.

2. What Is Two-Sample Sample Size Planning?

Two-sample sample size planning answers:

How many participants or observations are needed in each independent group to detect a meaningful difference with adequate power?

It can be used for two means, two proportions, and related independent-group tests.

3. When to Use Two-Sample Calculations

Use two-sample calculations for:

• Randomized two-arm trials.
• Independent treatment and control groups.
• Exposed vs unexposed cohort comparisons.
• A/B tests.
• Two independent schools, clinics, production lines, or regions.

Do not use this method for pre-post designs, matched pairs, crossover trials, or cluster randomized designs without adjustment.

4. The Mathematics Behind Two-Sample Power

4.1 Two Means

For equal group sizes and common standard deviation:

$d = \frac{\mu_1 - \mu_2}{\sigma}$

Approximate sample size per group:

$n_{per\ group} \approx 2\left(\frac{z_{1-\alpha/2} + z_{1-\beta}}{d}\right)^2$

4.2 Two Proportions

For proportions $p_1$ and $p_2$:

$\Delta = p_1 - p_2$

An approximate equal-group sample size uses pooled and unpooled variance terms:

$n \approx \frac{\left[z_{1-\alpha/2}\sqrt{2\bar{p}(1-\bar{p})} + z_{1-\beta}\sqrt{p_1(1-p_1)+p_2(1-p_2)}\right]^2}{(p_1-p_2)^2}$

Where:

$\bar{p} = \frac{p_1+p_2}{2}$

4.3 Absolute vs Standardized Effects

Use absolute differences for planning decisions, then standardized effects for communication:

$d = \frac{Difference}{SD}$

5. Allocation Ratios and Attrition

5.1 Equal Allocation

A 1:1 allocation is usually most statistically efficient.

5.2 Unequal Allocation

Unequal allocation may be justified when:

• One group is cheaper to recruit.
• Treatment slots are scarce.
• Ethical or operational constraints limit one arm.
• Historical controls are available.

Large imbalance reduces efficiency.

5.3 Attrition Inflation

Apply dropout inflation by group if dropout differs:

$n_{recruit} = \frac{n_{required}}{1 - dropout}$

6. Using the DataStatPro Calculator

Step-by-Step Guide

Step 1: Select two-sample independent analysis.

Choose means or proportions.

Step 2: Enter expected group values.

Use clinically or practically meaningful values.

Step 3: Enter variability for means.

Use the common SD, pooled SD, or conservative estimate.

Step 4: Set alpha, power, and tail direction.

Use two-tailed tests unless the directional hypothesis is pre-specified.

Step 5: Set allocation ratio.

Start with 1:1 unless practical constraints require otherwise.

Step 6: Review per-group and total sample size.

Round up and add attrition allowance.

7. Worked Examples

Example 1: Blood Pressure Trial

Expected difference = 10 mmHg, SD = 15 mmHg.

$d = 10/15 = 0.67$

With 90% power and $\alpha=0.05$, two-tailed, the calculator estimates the required sample per group. After calculating, inflate for expected dropout.

If required $n=48$ per group and dropout is 15%:

$n_{recruit} = 48/0.85 = 57$

Recruit 57 per group.

Example 2: A/B Conversion Test

Current conversion = 5%; target conversion = 7%.

Absolute difference:

$0.07 - 0.05 = 0.02$

Because the effect is small, the required sample may be large. The plan should state traffic assumptions and the minimum runtime needed to avoid weekday or seasonal bias.

8. Common Mistakes and How to Avoid Them

Mistake 1: Planning for an Unrealistically Large Effect

Use the smallest worthwhile difference.

Mistake 2: Ignoring Unequal Attrition

Different dropout by arm can reduce final power.

Mistake 3: Treating Clustered Groups as Independent

Cluster randomized studies require design-effect adjustment.

Mistake 4: Switching to One-Tailed Testing to Reduce n

This should be justified in the protocol, not chosen for convenience.

Mistake 5: Forgetting Multiple Primary Endpoints

Multiple primary outcomes may require alpha adjustment or a larger sample.

9. Troubleshooting

10. Quick Reference Cheat Sheet

Reporting Template

A two-sample power analysis for [means/proportions] used $\alpha=[value]$, power = [value], expected group values [value] and [value], [SD if applicable], and an allocation ratio of [ratio]. The required sample was [n] per group, inflated to [n] per group for expected attrition.