Paired t-Test: Zero to Hero Tutorial

This comprehensive tutorial takes you from the foundational concepts of dependent-samples inference all the way through the mathematics, assumptions, variants, effect sizes, interpretation, reporting, and practical usage of the Paired t-Test within the DataStatPro application. Whether you are encountering the paired t-test for the first time or seeking a rigorous understanding of within-subjects comparison, this guide builds your knowledge systematically from the ground up.

Table of Contents

1. Prerequisites and Background Concepts
2. What is a Paired t-Test?
3. The Mathematics Behind the Paired t-Test
4. Assumptions of the Paired t-Test
5. Variants of the Paired t-Test
6. Using the Paired t-Test Calculator Component
7. Full Step-by-Step Procedure
8. Effect Sizes for the Paired t-Test
9. Confidence Intervals
10. Power Analysis and Sample Size Planning
11. Advanced Topics
12. Worked Examples
13. Common Mistakes and How to Avoid Them
14. Troubleshooting
15. Quick Reference Cheat Sheet

1. Prerequisites and Background Concepts

Before diving into the paired t-test, it is essential to be comfortable with the following foundational statistical concepts. Each is briefly reviewed below.

1.1 Populations, Parameters, and Paired Designs

A population is the complete collection of individuals or measurements of interest. A sample is a subset drawn from that population. In a paired design, each participant (or experimental unit) contributes exactly two measurements — one under each of two conditions. The two measurements within a pair are inherently linked.

The paired t-test is an inferential procedure — it uses sample statistics computed from difference scores to draw conclusions about an unknown population parameter, namely the mean of the population difference scores $\mu_d$.

The fundamental question: "Is the mean difference between the two paired conditions large enough to conclude that a true population-level difference exists?"

1.2 Why Pairing Matters: Removing Between-Person Variability

In most research involving repeated measurements, individuals vary considerably from one another — some participants score high on both measurements, others score low on both. This between-person variability is a source of noise that has nothing to do with the treatment or condition effect.

By computing difference scores $d_i = x_{1i} - x_{2i}$ for each participant, the paired design removes between-person variability from the error term entirely:

$s_d^2 = s_1^2 + s_2^2 - 2r_{12}s_1s_2$

When $r_{12} > 0$ (which is typical when measuring the same people twice), $s_d < s_{pooled}$, meaning the paired test has a smaller denominator and greater statistical power than the independent samples t-test for the same data.

1.3 The Sampling Distribution of the Mean Difference

If we repeatedly drew samples of $n$ pairs from a population where the true mean difference is $\mu_d$, the sampling distribution of $\bar{d}$ (the mean of the difference scores) would, by the Central Limit Theorem, be approximately normal:

$\bar{d} \sim \mathcal{N}\!\left(\mu_d,\; \frac{\sigma_d^2}{n}\right)$

Because the population standard deviation of differences $\sigma_d$ is unknown, we estimate it with the sample standard deviation $s_d$, giving the estimated standard error of the mean difference:

$\widehat{SE}_{\bar{d}} = \frac{s_d}{\sqrt{n}}$

This substitution of $s_d$ for $\sigma_d$ is exactly what introduces the t-distribution rather than the standard normal distribution into the inference.

1.4 The t-Distribution and Degrees of Freedom

The Student's t-distribution arises when estimating a normally distributed population mean from a small sample with unknown variance. It is characterised by degrees of freedom $\nu$. For the paired t-test:

$\nu = n - 1$

where $n$ is the number of pairs (not the total number of observations, which would be $2n$). As $\nu \to \infty$, the t-distribution converges to the standard normal $Z$.

The t-distribution has heavier tails than the standard normal, reflecting greater uncertainty from estimating $\sigma_d$ from the data rather than knowing it exactly.

1.5 The Null and Alternative Hypotheses

The paired t-test operates within the Neyman-Pearson hypothesis testing framework:

$H_0: \mu_d = 0$ (the population mean difference is zero)

$H_1: \mu_d \neq 0$ (two-tailed, default)

or directional alternatives:

$H_1: \mu_d > 0$ (upper one-tailed — Condition 1 > Condition 2)

$H_1: \mu_d < 0$ (lower one-tailed — Condition 1 < Condition 2)

The null hypothesis can also be generalised to test against a non-zero value $\delta_0$:

$H_0: \mu_d = \delta_0$

which is useful for non-inferiority, superiority, or equivalence testing.

1.6 Statistical Significance vs. Practical Significance

A paired t-test answers: "Is the mean difference statistically distinguishable from zero, given sampling variability?" It does not answer: "Is the difference large enough to matter in practice?"

With a large number of pairs, even a trivially small mean difference can be statistically significant. Always report:

1. The t-statistic, degrees of freedom, and p-value (statistical significance).
2. An effect size (e.g., Cohen's $d_z$) and its 95% CI (practical significance).
3. The 95% CI for the mean difference (in original units).

1.7 Confidence Intervals and Their Relationship to the Test

A 95% confidence interval for $\mu_d$ is directly related to the two-tailed t-test at $\alpha = .05$: the null hypothesis $H_0: \mu_d = 0$ is rejected at $\alpha = .05$ if and only if $0$ lies outside the 95% CI. The CI provides strictly more information than the p-value because it communicates both the precision and magnitude of the estimated difference in original units.

1.8 Type I and Type II Errors

• Type I error: Concluding a difference exists when none does (false positive). Rate controlled by $\alpha$.
• Type II error: Missing a true difference (false negative). Rate $\beta$; power $= 1-\beta$.
• Power: Probability of correctly detecting a true effect of a given size.

2. What is a Paired t-Test?

2.1 The Core Idea

The paired t-test (also called: dependent samples t-test, matched pairs t-test, repeated measures t-test, or within-subjects t-test) is a parametric inferential procedure for testing whether the mean of a set of difference scores is significantly different from zero (or another specified value).

Rather than comparing two separate group means directly, the paired t-test:

1. Computes a difference score $d_i$ for each pair of observations.
2. Reduces the problem to a one-sample t-test on those difference scores.
3. Tests whether the mean difference $\bar{d}$ is significantly different from zero.

This reduction is elegant: the paired t-test is mathematically identical to a one-sample t-test applied to the difference scores.

2.2 When to Use a Paired t-Test

A paired t-test is appropriate when:

• The dependent variable is continuous (interval or ratio scale).
• You are comparing exactly two related conditions or two time points.
• Each observation in Condition 1 is meaningfully linked to exactly one observation in Condition 2.
• The difference scores are approximately normally distributed (or $n$ is large enough for the CLT to apply).

2.3 What Makes Observations "Paired"?

Observations are paired when there is a natural, meaningful, one-to-one correspondence between observations in the two conditions:

The key criterion is that the pairing must be established before data collection, not post-hoc. The correlation between the paired measurements must be positive (or at least non-negative) for pairing to confer a power advantage.

2.4 The Paired t-Test vs. Related Procedures

2.5 The Power Advantage of Pairing

The paired t-test is more powerful than the independent samples t-test when:

1. The within-pair correlation $r_{12}$ is positive (which is almost always true for repeated measures on the same participant).
2. Between-person variability is large relative to within-person change.

The power gain is quantified by the relationship between paired and independent standard errors:

$SE_{paired} = SE_{independent} \times \sqrt{2(1-r_{12})}$

When $r_{12} = 0.50$: $SE_{paired} = SE_{independent} \times \sqrt{1.0} = SE_{independent}$ (no gain).

When $r_{12} = 0.80$: $SE_{paired} = SE_{independent} \times \sqrt{0.4} = 0.632 \times SE_{independent}$ (37% reduction in SE — substantial power gain).

When $r_{12} = 0.30$: $SE_{paired} = SE_{independent} \times \sqrt{1.4} = 1.183 \times SE_{independent}$ — pairing actually hurts power when correlation is low and one degree of freedom is lost for pairing.

💡 Pairing is most advantageous when the within-pair correlation is high ($r_{12} > 0.50$). When participants differ greatly from each other but respond consistently to conditions, the paired design dramatically reduces error and increases power.

3. The Mathematics Behind the Paired t-Test

3.1 The Difference Score Reduction

Let $(x_{1i}, x_{2i})$ denote the pair of observations for participant $i$, where $i = 1, 2, \ldots, n$. Define the difference score:

$d_i = x_{1i} - x_{2i}$

The sign convention matters: consistently subtracting Condition 2 from Condition 1 means a positive $\bar{d}$ indicates that Condition 1 has higher values.

The mean and standard deviation of the difference scores are:

$\bar{d} = \frac{1}{n}\sum_{i=1}^n d_i$

$s_d = \sqrt{\frac{1}{n-1}\sum_{i=1}^n (d_i - \bar{d})^2}$

3.2 The t-Statistic

The paired t-statistic is:

$t = \frac{\bar{d} - \delta_0}{s_d / \sqrt{n}}$

Where:

• $\bar{d}$ = mean of the difference scores.
• $\delta_0$ = null hypothesis value (typically $0$).
• $s_d$ = standard deviation of the difference scores.
• $n$ = number of pairs.

Under $H_0: \mu_d = \delta_0$, this statistic follows a t-distribution with $\nu = n - 1$ degrees of freedom.

3.3 Standard Error of the Mean Difference

The standard error of the mean difference measures the precision of $\bar{d}$ as an estimate of $\mu_d$:

$SE_{\bar{d}} = \frac{s_d}{\sqrt{n}}$

This is the only standard error needed for the paired t-test. Note that it is computed entirely from the difference scores — the original scores $x_{1i}$ and $x_{2i}$ are used only to compute $d_i$.

3.4 The p-value

Two-tailed p-value:

$p = 2 \times P(T_{n-1} \geq |t_{obs}|)= 2 \times [1 - F_{t,\;n-1}(|t_{obs}|)]$

One-tailed p-value (upper): $H_1: \mu_d > \delta_0$

$p = P(T_{n-1} \geq t_{obs}) = 1 - F_{t,\;n-1}(t_{obs})$

One-tailed p-value (lower): $H_1: \mu_d < \delta_0$

$p = P(T_{n-1} \leq t_{obs}) = F_{t,\;n-1}(t_{obs})$

Where $F_{t,\;n-1}$ is the CDF of the t-distribution with $n-1$ degrees of freedom.

3.5 Relationship Between $s_d$ and the Raw Score Statistics

The standard deviation of differences $s_d$ is algebraically related to the standard deviations of the original scores and their correlation:

$s_d^2 = s_1^2 + s_2^2 - 2r_{12}s_1 s_2$

Where:

• $s_1, s_2$ = standard deviations of Condition 1 and Condition 2 scores respectively.
• $r_{12}$ = Pearson correlation between the paired measurements.

This relationship has several important implications:

When $r_{12} = 0$: $s_d^2 = s_1^2 + s_2^2$ — the paired test is equivalent to the independent test (no benefit from pairing).

When $r_{12} > 0$: $s_d^2 < s_1^2 + s_2^2$ — pairing reduces error variance and increases power.

When $r_{12} < 0$: $s_d^2 > s_1^2 + s_2^2$ — pairing increases error variance and reduces power. This is rare in practice but can occur with counterbalanced designs where learning effects operate.

3.6 The Mean Difference and Its Relationship to Raw Means

The mean difference score always equals the difference of the condition means:

$\bar{d} = \bar{x}_1 - \bar{x}_2$

This means the paired and independent tests produce identical estimates of the mean difference — the only difference is in the standard error used to evaluate that difference.

3.7 Computing the t-Statistic from Summary Statistics

If raw data are unavailable, the paired t-statistic can be computed from summary statistics in several ways:

From $\bar{d}$ and $s_d$:

$t = \frac{\bar{d}}{s_d/\sqrt{n}}$

From the correlation $r_{12}$ and group SDs:

$s_d = \sqrt{s_1^2 + s_2^2 - 2r_{12}s_1 s_2}$

$t = \frac{\bar{x}_1 - \bar{x}_2}{\sqrt{(s_1^2 + s_2^2 - 2r_{12}s_1 s_2)/n}}$

From the t-statistic, recovering effect size:

$d_z = \frac{t}{\sqrt{n}}$

3.8 Non-Central t-Distribution and Exact CIs for Effect Sizes

Under $H_1$ (when a true effect exists), the t-statistic follows a non-central t-distribution with non-centrality parameter:

$\lambda = d_z \sqrt{n} = \frac{\mu_d}{\sigma_d}\sqrt{n}$

The exact 95% CI for Cohen's $d_z$ inverts this relationship numerically:

$P(T_{n-1}(\lambda_L) \geq t_{obs}) = 0.025$ and $P(T_{n-1}(\lambda_U) \leq t_{obs}) = 0.025$

$d_{z,L} = \frac{\lambda_L}{\sqrt{n}}, \qquad d_{z,U} = \frac{\lambda_U}{\sqrt{n}}$

No closed form exists for these bounds — DataStatPro computes them automatically using numerical iteration of the non-central t-distribution CDF.

3.9 Statistical Power of the Paired t-Test

Power is the probability that the paired t-test correctly rejects $H_0$ when a true effect of size $d_z$ exists:

$\text{Power} = P\!\left(T_{n-1}(\lambda) > t_{\alpha/2,\;n-1}\right)$

Where $\lambda = d_z\sqrt{n}$ is the non-centrality parameter.

The relationship between power, effect size, and sample size:

All values assume two-tailed $\alpha = .05$.

4. Assumptions of the Paired t-Test

4.1 Normality of Difference Scores

The paired t-test assumes that the difference scores $d_i = x_{1i} - x_{2i}$ are drawn from a normally distributed population. Note that:

• This is not the same as assuming the individual conditions are normally distributed.
• The differences can be normal even if the raw scores are not, as long as the two conditions' non-normalities cancel out.
• This is a weaker assumption than requiring both raw distributions to be normal.

How to check:

• Shapiro-Wilk test on the difference scores ($n < 50$): $H_0$: differences are normally distributed. A significant result suggests departure from normality.
• Q-Q plot of difference scores: points should fall approximately on the diagonal reference line.
• Histogram of difference scores: should be approximately bell-shaped.
• Skewness ($|z_{skew}| < 2$) and kurtosis ($|z_{kurt}| < 7$).

Robustness: The paired t-test is robust to mild violations of normality, especially when:

• $n \geq 30$ pairs (Central Limit Theorem ensures the sampling distribution of $\bar{d}$ is approximately normal even if $d_i$ are not).
• The distribution of differences is symmetric, even if not perfectly normal.
• The violation consists of light tails rather than heavy tails or extreme skewness.

When violated: Use the Wilcoxon Signed-Rank test as a non-parametric alternative, or consider data transformations (log, square root) if the differences are right-skewed.

4.2 Independence of Pairs

All pairs must be independent of each other. That is, knowing the difference score for pair $i$ gives no information about the difference score for pair $j$. Within a pair, the two measurements are of course correlated — that is the whole point of the design. It is the independence across pairs that is required.

Common violations:

• Multiple measurements from the same participant treated as separate pairs.
• Family members or social contacts in the same study.
• Clustered data (e.g., pairs sampled from the same school or ward).
• Time series where successive differences are autocorrelated.

How to check: Independence is a property of the study design, not of the data. Inspect the sampling procedure. Check for patterns in residuals over time (Durbin-Watson test) if measurements were sequential.

When violated: For clustered pairs, use multilevel models. For time series, use time-series methods (ARIMA, mixed models with autocorrelation structure).

4.3 Correct Pairing

The pairing must be meaningful and pre-specified. Each observation in Condition 1 must correspond to the correct partner observation in Condition 2. Incorrect or arbitrary pairing does not create a valid paired test — it creates noise.

How to check: Verify the data file structure — each row should represent one pair (one participant or one matched pair), with Condition 1 and Condition 2 values in separate columns.

⚠️ A common data-entry error is accidentally shifting one column so that rows no longer correspond to the same participant across conditions. Always verify that participant IDs match across the two columns before running a paired t-test.

4.4 Interval Scale of Measurement

The dependent variable must be measured on at least an interval scale (equal-spaced intervals between values). Difference scores must be meaningful — they require that the distance between score values is consistent throughout the scale.

When violated: If the DV is ordinal (e.g., a single Likert item, rank data), use the Wilcoxon Signed-Rank test instead.

4.5 Absence of Influential Outliers in Difference Scores

The paired t-test is sensitive to extreme outliers in the difference scores because they distort both $\bar{d}$ and $s_d$.

How to check:

• Boxplot of difference scores: values beyond $1.5 \times IQR$ from the quartiles.
• Standardised difference scores: flag $|z_i| > 3$.
• Grubbs' test for formal outlier detection in the differences.

When outliers are present: Investigate whether the outlier represents a data entry error, measurement error, or a genuine extreme response. Report analyses with and without the outlier(s). Consider using the Wilcoxon Signed-Rank test (which is rank-based and thus robust to outliers in the differences).

4.6 Assumption Summary Table

5. Variants of the Paired t-Test

5.1 Overview of Effect Size Variants

Multiple variants of the paired t-test exist primarily because of different choices of effect size standardiser — the denominator of the standardised mean difference. Choosing the wrong variant leads to incomparable effect sizes across studies.

5.2 Cohen's $d_z$ — The Standardised Mean Difference of Differences

Cohen's $d_z$ is the most straightforward effect size for the paired t-test. It expresses the mean difference in units of the standard deviation of the difference scores:

$d_z = \frac{\bar{d}}{s_d}$

It is directly recoverable from the t-statistic: $d_z = t/\sqrt{n}$.

When to use $d_z$:

• Comparing effect sizes across studies that all use paired designs.
• Within-study power analysis for the same paired design.
• When the research question is about the magnitude of within-person change relative to individual variability in change.

Limitation of $d_z$: It is not directly comparable to Cohen's $d$ from an independent samples design because $s_d$ reflects within-person variability in change, which is typically much smaller than between-person variability. $d_z$ is therefore typically larger than $d_{av}$ for the same mean difference.

5.3 Cohen's $d_{av}$ — Average Standard Deviation Standardiser

Cohen's $d_{av}$ (Lakens, 2013) standardises the mean difference by the average of the two condition standard deviations:

$s_{av} = \frac{s_1 + s_2}{2}$

$d_{av} = \frac{\bar{d}}{s_{av}} = \frac{\bar{x}_1 - \bar{x}_2}{(s_1 + s_2)/2}$

When to use $d_{av}$:

• When comparing a paired design effect to a between-subjects design effect using the same measurement scale.
• When the research question concerns the magnitude of mean change relative to the variability of the original measurements.
• Meta-analyses combining within-subjects and between-subjects designs.

5.4 Cohen's $d_{rm}$ — Repeated Measures Corrected

Cohen's $d_{rm}$ (Morris & DeShon, 2002) explicitly accounts for the within-subjects correlation $r_{12}$ to produce an effect size that is directly comparable to a between- subjects Cohen's $d$:

$d_{rm} = d_{av} \times \sqrt{2(1-r_{12})}$

Or equivalently:

$d_{rm} = \frac{\bar{d}}{s_{av}} \times \sqrt{2(1-r_{12})}$

Properties:

• When $r_{12} = 0.5$: $d_{rm} = d_{av} \times 1.0 = d_{av}$.
• When $r_{12} > 0.5$: $d_{rm} < d_{av}$ (correlation inflated $d_{av}$ — correction deflates it toward a between-subjects comparable value).
• When $r_{12} < 0.5$: $d_{rm} > d_{av}$.

$d_{rm}$ is the most theoretically appropriate effect size for comparing paired designs to independent samples designs.

5.5 Glass's $\Delta$ for Pre-Post Designs

Glass's $\Delta$ standardises by the pre-test (Condition 1) standard deviation only. This is most appropriate for treatment-control or pre-post designs where the pre-test represents the baseline, unaffected by the treatment:

$\Delta = \frac{\bar{d}}{s_{pre}} = \frac{\bar{x}_1 - \bar{x}_2}{s_1}$

It answers: "How many standard deviations (in the original, pre-intervention metric) does the average participant change?"

When to use $\Delta$: Pre-post designs where the treatment may change the variability of the outcome (e.g., an intervention that reduces both mean and variance of depression scores). Standardising by the pre-test SD anchors the effect in the pre-intervention distribution.

5.6 Relationship Between $d_z$ and $d_{av}$

$d_z$ and $d_{av}$ are related through the within-pair correlation $r_{12}$:

$d_z = d_{av} \times \sqrt{\frac{2}{1 + r_{12}}} \times \frac{1}{\sqrt{2(1-r_{12})}} \times \sqrt{2(1-r_{12})}$

More directly:

$d_z = \frac{d_{av}}{\sqrt{1 - r_{12}}} \times \frac{1}{\sqrt{2}}$

Wait — the exact relationship (Lakens, 2013):

$d_z = \frac{\bar{d}}{s_d} = \frac{\bar{d}}{s_{av}\sqrt{2(1-r_{12})}} = \frac{d_{av}}{\sqrt{2(1-r_{12})}}$

Therefore:

$d_z = \frac{d_{av}}{\sqrt{2(1-r_{12})}}$ and $d_{av} = d_z \times \sqrt{2(1-r_{12})}$

This means $d_z > d_{av}$ when $r_{12} > 0$ (which is almost always the case for repeated measures), explaining why paired designs appear to produce larger effect sizes than between-subjects designs when the same $d_z$ metric is uncritically applied to both.

Numerical example with $r_{12} = 0.70$:

$d_z = d_{av}/\sqrt{2(1-0.70)} = d_{av}/\sqrt{0.60} = d_{av}/0.775 = 1.291 \times d_{av}$

So if $d_{av} = 0.60$, then $d_z = 0.775$ — nearly 30% larger.

6. Using the Paired t-Test Calculator Component

The Paired t-Test Calculator in DataStatPro provides a comprehensive tool for running, diagnosing, visualising, and reporting paired t-tests and their alternatives.

Step-by-Step Guide

Step 1 — Select "Paired Samples t-Test"

From the "Test Type" dropdown, select:

• Paired Samples t-Test — for parametric analysis of normally distributed differences.
• Wilcoxon Signed-Rank Test — the non-parametric alternative (automatically suggested if DataStatPro's normality check flags a violation).

Step 2 — Input Method

Choose how to provide the data:

• Raw data (paired columns): Upload or paste two columns — Condition 1 and Condition 2 — with one row per participant. DataStatPro automatically computes difference scores, runs assumption checks, and generates all statistics.
• Raw data (difference scores): If you already have difference scores, upload a single column and DataStatPro treats it as a one-sample t-test on the differences.
• Summary statistics: Enter $n$, $\bar{d}$, $s_d$ directly. All test statistics and effect sizes are computed but full assumption checks are unavailable.
• Summary statistics with correlation: Enter $n$, $\bar{x}_1$, $\bar{x}_2$, $s_1$, $s_2$, $r_{12}$ to compute all effect size variants.
• t-statistic and df: Enter $t$ and $n-1$ to compute p-values and effect sizes from a published result.

💡 When using paired columns, DataStatPro verifies that column lengths are equal, flags any missing data, and alerts you if participant IDs are provided and do not match across columns.

Step 3 — Specify the Null Hypothesis Value $\delta_0$

Default: $\delta_0 = 0$ (testing whether the mean difference is zero). To test a non-zero null (e.g., for non-inferiority testing with a margin of $\delta_0 = -2$ points), enter the appropriate value.

Step 4 — Select the Alternative Hypothesis

• Two-tailed (default): $H_1: \mu_d \neq 0$ — most appropriate for most research.
• Upper one-tailed: $H_1: \mu_d > 0$ — use only with strong a priori directional prediction, pre-registered before data collection.
• Lower one-tailed: $H_1: \mu_d < 0$ — use only with pre-registered directional prediction.

Step 5 — Choose the Significance Level

Select $\alpha$ (default: $.05$). DataStatPro simultaneously shows results for $\alpha = .05$, $\alpha = .01$, and $\alpha = .001$ for reference.

Step 6 — Select Effect Size Variants

Choose which effect sizes to compute and report:

• ✅ Cohen's $d_z$ (default) — standardised by $s_d$.
• ✅ Cohen's $d_{av}$ — standardised by average of condition SDs.
• ✅ Cohen's $d_{rm}$ — RM-corrected (requires $r_{12}$; from raw data or entered manually).
• ✅ Glass's $\Delta$ — standardised by Condition 1 (pre-test) SD.
• ✅ Hedges' $g_z$ — bias-corrected $d_z$.
• ✅ Common Language Effect Size (CL).

Step 7 — Select Display Options

• ✅ t-statistic, df, p-value, and decision.
• ✅ Descriptive statistics: $n$, $\bar{x}_1$, $\bar{x}_2$, $\bar{d}$, $s_1$, $s_2$, $s_d$, $r_{12}$.
• ✅ 95% CI for mean difference (in original units).
• ✅ All selected effect sizes with exact 95% CIs.
• ✅ Common Language Effect Size (CL%).
• ✅ Assumption test panel: Shapiro-Wilk on differences, Q-Q plot, histogram of differences, boxplot of differences.
• ✅ Visualisation: overlapping density plots for Conditions 1 and 2; distribution of difference scores with reference line at zero.
• ✅ Individual trajectory plot: each participant's scores connected across conditions.
• ✅ Cohen's $d$ diagram ($U_1$, $U_2$, $U_3$ overlap statistics).
• ✅ Power curve: power vs. $n$ for the observed effect size.
• ✅ Equivalence test (TOST) output.
• ✅ Bayesian paired t-test (Bayes Factor $BF_{10}$).
• ✅ APA 7th edition-compliant results paragraph (auto-generated).

Step 8 — Run the Analysis

Click "Run Paired t-Test". DataStatPro will:

1. Compute difference scores and all descriptive statistics.
2. Run Shapiro-Wilk normality test on the differences.
3. Compute the t-statistic, df, and p-value.
4. Construct exact 95% CIs for the mean difference and all effect sizes.
5. Generate all selected visualisations.
6. Auto-generate the APA results paragraph.

7. Full Step-by-Step Procedure

7.1 Complete Computational Procedure

This section walks through every computational step for the paired t-test, from raw data to a full APA-style conclusion.

Given: $n$ pairs of observations $(x_{1i}, x_{2i})$ for $i = 1, 2, \ldots, n$.

Step 1 — Verify and Arrange the Data

Arrange data in a table with one row per pair:

Establish the sign convention: a positive $d_i$ means the participant scored higher in Condition 1 than Condition 2. State this convention explicitly before analysis.

Step 2 — Compute the Mean Difference

$\bar{d} = \frac{1}{n}\sum_{i=1}^n d_i = \frac{\sum_{i=1}^n d_i}{n}$

Equivalently: $\bar{d} = \bar{x}_1 - \bar{x}_2$

Step 3 — Compute the Standard Deviation of Differences

$s_d = \sqrt{\frac{\sum_{i=1}^n (d_i - \bar{d})^2}{n-1}} = \sqrt{\frac{\sum_{i=1}^n d_i^2 - n\bar{d}^2}{n-1}}$

Step 4 — Compute the Standard Error

$SE_{\bar{d}} = \frac{s_d}{\sqrt{n}}$

Step 5 — Check the Normality Assumption

Run the Shapiro-Wilk test on the difference scores $d_i$:

• If $p_{SW} > .05$: normality is not contradicted; proceed with paired t-test.
• If $p_{SW} \leq .05$ and $n < 30$: consider the Wilcoxon Signed-Rank test.
• If $p_{SW} \leq .05$ and $n \geq 30$: proceed with caution (CLT generally provides protection); inspect Q-Q plot for severe violations.

Step 6 — Compute the t-Statistic

$t = \frac{\bar{d} - \delta_0}{SE_{\bar{d}}} = \frac{\bar{d} - \delta_0}{s_d/\sqrt{n}}$

For the default null ($\delta_0 = 0$): $t = \bar{d} \cdot \sqrt{n} / s_d$

Step 7 — Determine Degrees of Freedom

$\nu = n - 1$

Step 8 — Compute the p-value

Using the t-distribution with $\nu = n-1$ df:

Two-tailed: $p = 2 \times P(T_{n-1} \geq |t|)$

Compare $p$ to $\alpha$. Reject $H_0$ if $p \leq \alpha$.

Step 9 — Compute the 95% Confidence Interval for $\mu_d$

$\bar{d} \pm t_{\alpha/2,\; n-1} \times SE_{\bar{d}} = \bar{d} \pm t_{\alpha/2,\; n-1} \times \frac{s_d}{\sqrt{n}}$

The CI directly answers: "What are plausible values for the true population mean difference, given this sample?"

Step 10 — Compute Effect Sizes

Cohen's $d_z$:

$d_z = \frac{\bar{d}}{s_d} = \frac{t}{\sqrt{n}}$

Hedges' $g_z$ (bias-corrected $d_z$):

$g_z = d_z \times \left(1 - \frac{3}{4(n-1) - 1}\right) = d_z \times J$

Where $J = 1 - 3/(4n-5)$ is the bias correction factor.

Cohen's $d_{av}$ (requires $s_1$ and $s_2$):

$s_{av} = \frac{s_1 + s_2}{2}, \qquad d_{av} = \frac{\bar{d}}{s_{av}}$

Cohen's $d_{rm}$ (requires $r_{12}$):

$d_{rm} = d_{av} \times \sqrt{2(1-r_{12})}$

Common Language Effect Size (CL):

$CL = \Phi\!\left(\frac{d_z}{\sqrt{2}}\right)$

$CL$ is the probability that a randomly selected participant scores higher in Condition 1 than in Condition 2 (for positive $d_z$).

Step 11 — Compute the 95% CI for Cohen's $d_z$

Exact CI (via non-central t-distribution — computed by DataStatPro):

Find $\lambda_L$ and $\lambda_U$ such that:

$P(T_{n-1}(\lambda_L) \geq t_{obs}) = 0.025$ and $P(T_{n-1}(\lambda_U) \leq t_{obs}) = 0.025$

$d_{z,L} = \lambda_L/\sqrt{n}, \qquad d_{z,U} = \lambda_U/\sqrt{n}$

Approximate CI (adequate for $n > 30$):

$SE_{d_z} \approx \sqrt{\frac{1}{n} + \frac{d_z^2}{2(n-1)}}$

$d_z \pm 1.96 \times SE_{d_z}$

Step 12 — Interpret and Report

Combine all results into a complete, APA-compliant report:

1. Report $t(\nu) =$ [value], $p =$ [value] (or $p < .001$).
2. Report $\bar{d}$ and $s_d$ with units.
3. Report the 95% CI for the mean difference.
4. Report Cohen's $d_z$ (and/or $d_{av}$) with 95% CI.
5. Classify the effect size using benchmarks.
6. State the practical conclusion.

8. Effect Sizes for the Paired t-Test

8.1 Cohen's $d_z$ — Step-by-Step

$d_z = \frac{\bar{d}}{s_d}$

Interpretation: $d_z = 0.50$ means the mean difference is half a standard deviation of the difference scores. This is not directly comparable to Cohen's $d$ from an independent samples design without knowing $r_{12}$.

8.2 Hedges' $g_z$ — Bias Correction

Cohen's $d_z$ is slightly positively biased in small samples — it overestimates the true population effect. Hedges' $g_z$ applies the bias correction:

$g_z = d_z \times J, \qquad J = 1 - \frac{3}{4(n-1)-1}$

More precise gamma function form:

$J = \frac{\Gamma((n-1)/2)}{\sqrt{(n-1)/2} \cdot \Gamma((n-2)/2)}$

The bias is negligible for $n > 20$ (less than 5%) but can be substantial for very small samples ($n < 10$):

8.3 Cohen's Benchmark Classification

Cohen (1988) proposed the following conventions for $d_z$ (and equivalently for $d_{av}$ and $d_{rm}$):

⚠️ Cohen's benchmarks were intended as rough conventions of last resort — to be used only when no domain-specific information is available. Always contextualise within your research domain. In clinical psychology, $d_z = 0.50$ may be a meaningful effect; in some neuroimaging contexts, $d_z = 0.20$ may be large relative to typical findings.

Extended benchmarks (Sawilowsky, 2009):

8.4 The Common Language Effect Size

The Common Language Effect Size (McGraw & Wong, 1992) translates $d_z$ into a probability that is intuitive for non-statistical audiences:

$CL = \Phi\!\left(\frac{d_z}{\sqrt{2}}\right)$

$CL = 0.70$ means: "In 70% of repeated measurements of the same individual, their score in Condition 1 exceeds their score in Condition 2."

8.5 Which Effect Size to Report: A Decision Guide

💡 Always specify which effect size variant was computed. Writing "Cohen's $d = 0.78$" without specifying whether it is $d_z$, $d_{av}$, or $d_{rm}$ is ambiguous and prevents accurate meta-analytic synthesis.

9. Confidence Intervals

9.1 CI for the Mean Difference (Original Units)

The 95% CI for the population mean difference $\mu_d$ provides the most practically interpretable interval — it is expressed in the original measurement units and directly answers: "How large might the true effect be?"

$\bar{d} \pm t_{\alpha/2,\; n-1} \times \frac{s_d}{\sqrt{n}}$

Interpreting the CI:

9.2 CI for Cohen's $d_z$ (Standardised)

The exact 95% CI for $d_z$ uses the non-central t-distribution (computed automatically by DataStatPro). The approximate CI (adequate for $n > 30$) is:

$SE_{d_z} \approx \sqrt{\frac{1}{n} + \frac{d_z^2}{2(n-1)}}$

$d_z \pm 1.96 \times SE_{d_z}$ (approximate, two-tailed $\alpha = .05$)

Width of the 95% CI for $d_z$ as a function of $n$ (for true $d_z = 0.50$):

⚠️ With $n = 10$ pairs, the 95% CI for $d_z = 0.50$ spans approximately [−0.16, 1.16] — from "negligible" to "very large." A point estimate of $d_z = 0.50$ from a study of only 10 pairs is essentially uninterpretable without the CI. Always report the CI.

9.3 CI for Other Effect Size Variants

95% CI for $d_{av}$: Convert using $d_{av} = d_z \times \sqrt{2(1-r_{12})}$ and apply the same conversion to both CI bounds.

$d_{av,L} = d_{z,L} \times \sqrt{2(1-r_{12})}, \qquad d_{av,U} = d_{z,U} \times \sqrt{2(1-r_{12})}$

95% CI for $d_{rm}$: DataStatPro computes this by bootstrapping when raw data are available, or via the delta method for summary statistics.

10. Power Analysis and Sample Size Planning

10.1 A Priori Power Analysis

A priori power analysis determines the required number of pairs before data collection to achieve desired power $1-\beta$ at significance level $\alpha$ for a hypothesised effect of size $d_z$.

Required $n$ for two-tailed test:

The exact calculation uses the non-central t-distribution (numerical). An excellent approximation:

$n \approx \frac{(z_{1-\alpha/2} + z_{1-\beta})^2}{d_z^2} + \frac{z_{1-\alpha/2}^2}{2}$

For $\alpha = .05$ (two-tailed, $z_{.975} = 1.96$) and power $= 0.80$ ($z_{.80} = 0.842$):

$n \approx \frac{(1.96 + 0.842)^2}{d_z^2} = \frac{7.849}{d_z^2}$

All values assume two-tailed $\alpha = .05$. Add 1–2 pairs to account for rounding.

10.2 Sensitivity Analysis (Post-Hoc Power)

Sensitivity analysis determines the minimum effect size that could have been detected with the study's sample size at a specified power level. It answers: "What was the smallest effect this study was designed to detect?"

$d_{z,\;min} = \sqrt{\frac{(z_{1-\alpha/2} + z_{1-\beta})^2}{n}}$

For $n = 20$ pairs, $\alpha = .05$, power $= 0.80$:

$d_{z,\;min} = \sqrt{7.849/20} = \sqrt{0.392} = 0.626$

This study could reliably detect only effects of $d_z \geq 0.63$ — near Cohen's "large" threshold. Smaller effects may exist but would frequently be missed.

⚠️ Post-hoc power computed from the observed effect size (sometimes called "observed power") is circular, redundant with the p-value, and should NOT be reported as a justification for a non-significant result. Sensitivity analysis using the minimum detectable effect is the appropriate post-hoc power tool.

10.3 Planning Based on $d_{av}$ Instead of $d_z$

When planning based on an expected $d_{av}$ (e.g., from a published between-subjects study), first convert to $d_z$ using the anticipated within-pair correlation $r_{12}$:

$d_z = \frac{d_{av}}{\sqrt{2(1-r_{12})}}$

Then apply the standard $n$ formula. If $r_{12}$ is unknown, use a conservative estimate of $r_{12} = 0.50$:

$d_z = \frac{d_{av}}{\sqrt{2(1-0.50)}} = \frac{d_{av}}{\sqrt{1.0}} = d_{av}$

With $r_{12} = 0.50$, $d_z = d_{av}$, so the sample size formula is the same.

10.4 The Effect of Pre-Post Correlation on Required Sample Size

The required sample size for a paired design decreases as $r_{12}$ increases — reflecting the power advantage of pairing. Compared to an independent samples design with the same $d_{av}$:

$n_{paired}/n_{independent}$ is the ratio of total observations needed (paired has $n$ pairs = $n$ total; independent has $2n$ total for same power on $d_{av}$).

💡 For $r_{12} = 0.80$, the paired design requires only 40% as many participants as the independent design to achieve the same power. When within-pair correlations are high, pairing provides a dramatic efficiency gain.

11. Advanced Topics

11.1 The Paired t-Test as a One-Sample t-Test

The paired t-test is mathematically identical to a one-sample t-test applied to the difference scores. This has several practical implications:

1. Software implementation: Many software packages implement the paired t-test by computing difference scores and running a one-sample test.
2. Missing data: If some participants have data for only one condition, those pairs cannot contribute difference scores and are excluded entirely from the analysis.
3. Non-zero null: Testing $H_0: \mu_d = 5$ (e.g., "does the mean improvement exceed a clinically significant threshold of 5 points?") is as straightforward as testing $H_0: \mu_d = 0$.

11.2 The Relationship Between Paired and Independent t-Tests

For the same dataset, the paired and independent t-statistics are related through the within-pair correlation $r_{12}$:

$t_{paired} = \frac{\bar{d}}{s_d/\sqrt{n}} = \frac{\bar{x}_1-\bar{x}_2}{\sqrt{(s_1^2+s_2^2-2r_{12}s_1s_2)/n}}$

$t_{independent} = \frac{\bar{x}_1-\bar{x}_2}{s_{pooled}\sqrt{2/n}} = \frac{\bar{x}_1-\bar{x}_2}{\sqrt{(s_1^2+s_2^2)/n}}$

The ratio of the t-statistics:

$\frac{t_{paired}}{t_{independent}} = \frac{\sqrt{s_1^2+s_2^2}}{\sqrt{s_1^2+s_2^2-2r_{12}s_1s_2}} = \frac{1}{\sqrt{1-\frac{2r_{12}s_1s_2}{s_1^2+s_2^2}}}$

For equal SDs ($s_1 = s_2 = s$):

$\frac{t_{paired}}{t_{independent}} = \frac{1}{\sqrt{1-r_{12}}}$

When $r_{12} = 0.75$: $t_{paired}/t_{independent} = 1/\sqrt{0.25} = 2.0$ — the paired t-statistic is twice as large, corresponding to vastly higher power.

Also note the degrees of freedom differ: $\nu_{paired} = n-1$ vs. $\nu_{independent} = 2n-2$. The paired test loses $n-1$ degrees of freedom by pairing, but gains far more through the reduced error term when $r_{12}$ is high.

11.3 Equivalence Testing with TOST

Standard paired t-testing can reject $H_0: \mu_d = 0$ but cannot establish that the mean difference is negligibly small. The Two One-Sided Tests (TOST) procedure tests whether the mean difference falls within a pre-specified equivalence interval $[-\Delta_L, \Delta_U]$:

$H_{01}: \mu_d \leq -\Delta_L$ (the difference is meaningfully negative) $H_{02}: \mu_d \geq \Delta_U$ (the difference is meaningfully positive)

Equivalence is concluded when both one-sided tests reject their respective nulls at level $\alpha$ — equivalently, when the 90% CI (for $\alpha = .05$) for $\mu_d$ falls entirely within $(-\Delta_L, \Delta_U)$.

The TOST t-statistics:

$t_1 = \frac{\bar{d} - (-\Delta_L)}{s_d/\sqrt{n}}, \qquad t_2 = \frac{\bar{d} - \Delta_U}{s_d/\sqrt{n}}$