Repeated Measures ANOVA: Zero to Hero Tutorial

This comprehensive tutorial takes you from the foundational concepts of within-subjects experimental designs all the way through advanced interpretation, reporting, assumption checking, and practical usage within the DataStatPro application. Whether you are encountering repeated measures ANOVA for the first time or deepening your understanding of analysing data where the same participants contribute observations across multiple conditions or time points, this guide builds your knowledge systematically from the ground up.

Table of Contents

1. Prerequisites and Background Concepts
2. What is Repeated Measures ANOVA?
3. The Mathematics Behind Repeated Measures ANOVA
4. Assumptions of Repeated Measures ANOVA
5. Variants of Repeated Measures ANOVA
6. Using the Repeated Measures ANOVA Calculator Component
7. Step-by-Step Procedure
8. Interpreting the Output
9. Effect Sizes for Repeated Measures ANOVA
10. Confidence Intervals
11. Advanced Topics
12. Worked Examples
13. Common Mistakes and How to Avoid Them
14. Troubleshooting
15. Quick Reference Cheat Sheet

1. Prerequisites and Background Concepts

Before diving into repeated measures ANOVA, it is essential to be comfortable with the following foundational statistical concepts. Each is briefly reviewed below.

1.1 Within-Subjects vs. Between-Subjects Designs

A fundamental distinction in experimental design concerns how participants contribute data across conditions:

• Between-subjects design: Different participants are assigned to different conditions. Each participant contributes one observation. Variability between participants is indistinguishable from variability between conditions — it becomes part of the error term.

• Within-subjects design: The same participants are measured under all conditions (or at all time points). Each participant contributes multiple observations. Because we can model each participant's general tendency to score high or low, this individual variability is removed from the error term, substantially increasing statistical power.

Repeated measures ANOVA is the inferential framework for within-subjects designs with three or more conditions or time points.

1.2 The Logic of Variance Partitioning

Analysis of Variance (ANOVA) tests hypotheses by partitioning the total variability in the data ($SS_{Total}$) into meaningful components:

$SS_{Total} = SS_{Effect} + SS_{Error}$

The key insight is that what constitutes "error" differs between designs:

• In a between-subjects one-way ANOVA: $SS_{Total} = SS_{Between} + SS_{Within}$, where $SS_{Within}$ includes both random measurement error and individual differences between participants.

• In a within-subjects (repeated measures) ANOVA: $SS_{Total} = SS_{Within\text{-}subjects}$ (since all variation is within the same individuals), and:

  $SS_{Within\text{-}subjects} = SS_{Condition} + SS_{Subjects} + SS_{Error}$

By extracting $SS_{Subjects}$ (the variability attributable to stable individual differences), the residual error term $SS_{Error}$ is much smaller than in a between-subjects design, producing larger F-ratios and greater power.

1.3 The F-Ratio

The F-ratio is the core test statistic of ANOVA:

$F = \frac{\text{Variance explained by the effect (Mean Square Effect)}}{\text{Unexplained variance (Mean Square Error)}}$

$F = \frac{MS_{Effect}}{MS_{Error}}$

Under $H_0$ (all condition means are equal), $F \approx 1$. Under $H_1$ (at least one condition mean differs), $F > 1$. The larger the $F$, the stronger the evidence against $H_0$.

1.4 The F-Distribution

The F-distribution is parameterised by two degrees of freedom: $df_1$ (numerator, associated with the effect) and $df_2$ (denominator, associated with the error). It is:

• Right-skewed, defined only for non-negative values.
• Indexed by $df_1$ and $df_2$; approaches a normal distribution for very large $df_2$.
• The p-value is always computed from the right tail: $p = P(F_{df_1, df_2} \geq F_{obs})$.

1.5 The Null and Alternative Hypotheses in ANOVA

For a within-subjects factor with $k$ levels (conditions or time points):

• $H_0$: All population condition means are equal: $\mu_1 = \mu_2 = \cdots = \mu_k$

• $H_1$: At least one population condition mean differs from at least one other: $\mu_i \neq \mu_j$ for some $i \neq j$

$H_1$ is omnibus — it does not specify which means differ or in what direction. A significant F-test must therefore be followed by post-hoc tests or planned contrasts to identify the specific pattern of differences.

1.6 The p-Value and Significance Level

As in all hypothesis tests, the p-value is the probability of observing an F-ratio as large or larger than obtained, assuming $H_0$ is true. The significance level $\alpha$ (conventionally $.05$) is the threshold below which we reject $H_0$.

⚠️ A significant omnibus F-test tells you only that the condition means are not all equal. It does not tell you which conditions differ, how large the differences are, or whether the differences are practically meaningful. Always follow up with effect sizes, confidence intervals, and post-hoc comparisons.

1.7 Carryover Effects and Counterbalancing

A unique concern in within-subjects designs is that participating in one condition may influence performance in a subsequent condition:

• Practice effects: Performance improves with experience across conditions.
• Fatigue effects: Performance deteriorates across conditions due to tiredness.
• Contrast effects: The subjective experience of one condition is influenced by the preceding condition.

Counterbalancing — systematically varying the order in which participants complete conditions — distributes these carryover effects evenly across conditions, preventing them from confounding the main effect of interest.

1.8 Mauchly's Sphericity and Why It Matters

The repeated measures ANOVA relies on an assumption called sphericity: the variances of all pairwise difference scores between condition levels must be equal. This is analogous to the homogeneity of variance assumption in between-subjects ANOVA, but specific to within-subjects designs. Violating sphericity inflates the Type I error rate. Mauchly's test and epsilon ($\varepsilon$) corrections (Greenhouse-Geisser, Huynh-Feldt) are the standard diagnostic and remedial tools — covered fully in Section 4.

2. What is Repeated Measures ANOVA?

2.1 The Core Question

Repeated measures ANOVA (also called within-subjects ANOVA) is a parametric inferential test that determines whether the means of a continuous dependent variable differ significantly across three or more levels of a within-subjects factor — conditions, time points, or stimuli to which all participants are exposed.

Unlike the paired t-test (which compares two conditions), or between-subjects ANOVA (which compares independent groups), repeated measures ANOVA is the appropriate framework when:

• The same participants complete all conditions, OR
• Participants are measured at multiple time points (longitudinal panel data), OR
• The same participants are exposed to multiple stimuli or multiple tasks.

2.2 The General Logic

Repeated measures ANOVA exploits the within-person correlation across conditions. By modelling each participant's general response level (their row mean in the data matrix), the test removes stable individual differences from the error term:

$SS_{Error_{RM}} = SS_{Error_{BS}} - SS_{Subjects}$

This reduction in error variance means that for the same true effect size, repeated measures ANOVA has substantially greater statistical power than a comparable between-subjects design — particularly when individual differences are large (i.e., when participants consistently differ from one another regardless of condition).

2.3 When to Use Repeated Measures ANOVA

2.4 Real-World Applications

2.5 Distinguishing from Related Tests

3. The Mathematics Behind Repeated Measures ANOVA

3.1 Data Structure

Consider $n$ participants each measured under $k$ conditions. The data form an $n \times k$ matrix of scores $X_{ij}$, where $i = 1, \ldots, n$ indexes participants and $j = 1, \ldots, k$ indexes conditions:

3.2 Partitioning the Total Sum of Squares

The total sum of squares across all $N = n \times k$ observations is:

$SS_{Total} = \sum_{i=1}^{n}\sum_{j=1}^{k}(X_{ij} - \bar{X}_{\cdot\cdot})^2$

In a repeated measures design, this partitions as:

$SS_{Total} = SS_{Between\text{-}Subjects} + SS_{Within\text{-}Subjects}$

The between-subjects component reflects how participants differ from each other (averaged across conditions):

$SS_{Between\text{-}Subjects} = k\sum_{i=1}^{n}(\bar{X}_{i\cdot} - \bar{X}_{\cdot\cdot})^2$

The within-subjects component captures how each participant's scores vary across conditions. This is further partitioned into the condition effect and error:

$SS_{Within\text{-}Subjects} = SS_{Condition} + SS_{Error}$

Condition sum of squares (systematic variability between condition means):

$SS_{Condition} = n\sum_{j=1}^{k}(\bar{X}_{\cdot j} - \bar{X}_{\cdot\cdot})^2$

Error sum of squares (residual variability after removing both condition and participant effects):

$SS_{Error} = SS_{Total} - SS_{Between\text{-}Subjects} - SS_{Condition}$

Or equivalently:

$SS_{Error} = \sum_{i=1}^{n}\sum_{j=1}^{k}\left[(X_{ij} - \bar{X}_{i\cdot} - \bar{X}_{\cdot j} + \bar{X}_{\cdot\cdot})^2\right]$

3.3 Degrees of Freedom

Verification: $(n-1) + (k-1) + (k-1)(n-1) = n-1 + k-1 + nk - n - k + 1 = nk - 1$ ✓

3.4 Mean Squares and the F-Ratio

Mean squares are obtained by dividing each sum of squares by its degrees of freedom:

$MS_{Condition} = \frac{SS_{Condition}}{k-1}$

$MS_{Error} = \frac{SS_{Error}}{(k-1)(n-1)}$

The F-ratio for the within-subjects condition effect:

$F = \frac{MS_{Condition}}{MS_{Error}}$

Under $H_0$: $\mu_1 = \mu_2 = \cdots = \mu_k$, this F-ratio follows an F-distribution with $df_1 = k - 1$ and $df_2 = (k-1)(n-1)$ degrees of freedom.

3.5 The ANOVA Source Table

The complete one-way repeated measures ANOVA source table:

⚠️ The between-subjects row ($SS_{BS}$, $df = n-1$) is typically not tested with an F-ratio — it represents stable individual differences that are partialled out, not an experimental factor of interest. Some software omits this row entirely.

3.6 Epsilon ($\varepsilon$) Corrections for Sphericity Violations

When the sphericity assumption is violated (see Section 4), the actual sampling distribution of $F$ has heavier tails than the nominal $F_{k-1,\,(k-1)(n-1)}$ distribution — producing inflated Type I error. Two corrections adjust the degrees of freedom to match the true distribution.

Greenhouse-Geisser (GG) epsilon:

$\hat{\varepsilon}_{GG} = \frac{k^2(\bar{s}_{jj'} - \bar{s}_{..})^2}{(k-1)\left[\sum_{j}\sum_{j'} s^2_{jj'} - 2k\sum_j \bar{s}^2_{j.} + k^2\bar{s}^2_{..}\right]}$

Where $s_{jj'}$ are elements of the covariance matrix of condition scores, $\bar{s}_{jj'}$ are column means, and $\bar{s}_{..}$ is the grand mean of the covariance matrix.

Practically, $\hat{\varepsilon}_{GG}$ ranges from $1/(k-1)$ (maximum violation of sphericity) to $1.0$ (perfect sphericity). GG is known to be conservative — it sometimes overcorrects, especially with larger $k$ and $n$.

Huynh-Feldt (HF) epsilon:

$\tilde{\varepsilon}_{HF} = \frac{n(k-1)\hat{\varepsilon}_{GG} - 2}{(k-1)[n - 1 - (k-1)\hat{\varepsilon}_{GG}]}$

HF epsilon is less conservative than GG and is recommended when $\hat{\varepsilon}_{GG} > .75$. If $\tilde{\varepsilon}_{HF} > 1$, it is set to $1.0$.

Corrected degrees of freedom:

$df_1^* = (k-1)\hat{\varepsilon}, \qquad df_2^* = (k-1)(n-1)\hat{\varepsilon}$

The F-statistic itself is unchanged; only the reference distribution is adjusted.

Decision rule for epsilon corrections:

3.7 The Multivariate Approach (MANOVA)

An alternative to the univariate F-test with epsilon corrections is the fully multivariate approach, which makes no sphericity assumption whatsoever. The $k$ repeated conditions are recast as $k-1$ contrast variables and tested with multivariate test statistics:

• Pillai's Trace: $V = \text{tr}[\mathbf{H}(\mathbf{H}+\mathbf{E})^{-1}]$
• Wilks' Lambda: $\Lambda = |\mathbf{E}|/|\mathbf{H}+\mathbf{E}|$
• Hotelling-Lawley Trace: $T = \text{tr}[\mathbf{H}\mathbf{E}^{-1}]$
• Roy's Largest Root: $\theta = \lambda_{max}(\mathbf{H}\mathbf{E}^{-1})$

Where $\mathbf{H}$ is the hypothesis matrix and $\mathbf{E}$ is the error matrix.

The multivariate approach is always valid regardless of sphericity, but requires $n > k$ and loses power relative to the corrected univariate test when sphericity holds approximately. For small $n$ relative to $k$, the univariate approach with epsilon corrections is preferred.

3.8 Effect Size — Eta-Squared ($\eta^2$)

The most straightforward effect size for repeated measures ANOVA is eta-squared:

$\eta^2 = \frac{SS_{Condition}}{SS_{Total}}$

$\eta^2$ is the proportion of total variance explained by the condition effect. However, in repeated measures designs, the between-subjects variance ($SS_{BS}$) is irreducible and not of interest. This makes $\eta^2$ artificially small compared to a between-subjects design with the same true effect — it is not directly comparable across designs.

3.9 Effect Size — Partial Eta-Squared ($\eta^2_p$)

Partial eta-squared removes the between-subjects variance from the denominator:

$\eta^2_p = \frac{SS_{Condition}}{SS_{Condition} + SS_{Error}}$

$\eta^2_p$ represents the proportion of variance explained by the condition effect after removing individual differences. It is the standard effect size reported by most software (SPSS, SAS, R's ez package) and is comparable across between- and within-subjects designs.

Relationship to $F$:

$\eta^2_p = \frac{F \cdot df_1}{F \cdot df_1 + df_2}$

3.10 Effect Size — Generalised Eta-Squared ($\eta^2_G$)

Generalised eta-squared (Olejnik & Algina, 2003) is designed for comparability across studies with different designs. For a pure within-subjects design:

$\eta^2_G = \frac{SS_{Condition}}{SS_{Condition} + SS_{Between\text{-}Subjects} + SS_{Error}}$

$\eta^2_G$ is the recommended effect size for meta-analysis involving repeated measures designs because it is invariant to the number of conditions measured, unlike $\eta^2_p$.

3.11 Effect Size — Omega-Squared ($\omega^2$) and Partial Omega-Squared ($\omega^2_p$)

Both $\eta^2$ and $\eta^2_p$ are positively biased (they overestimate the population effect, especially in small samples). Omega-squared applies a bias correction:

$\omega^2 = \frac{SS_{Condition} - (k-1)MS_{Error}}{SS_{Total} + MS_{Between\text{-}Subjects}}$

Partial omega-squared (preferred for repeated measures):

$\omega^2_p = \frac{SS_{Condition} - (k-1)MS_{Error}}{SS_{Condition} + (n - k + 1) \cdot MS_{Error}}$

For large samples, $\omega^2_p \approx \eta^2_p$. For small samples ($n < 30$), $\omega^2_p$ is the recommended effect size to report alongside $\eta^2_p$.

3.12 Cohen's $f$ and Statistical Power

Cohen's $f$ is the standardised effect size for ANOVA, defined as:

$f = \sqrt{\frac{\eta^2_p}{1 - \eta^2_p}}$

Required sample size for desired power $1-\beta$ at two-sided $\alpha$ (approximate):

$n \approx \frac{\lambda}{f^2 \cdot k}$

Where $\lambda$ is the non-centrality parameter satisfying the power equation.

Required $n$ per condition combination, one-way within-subjects ($k = 4$, $\alpha = .05$, $\rho = .50$ average intercorrelation):

⚠️ Power for repeated measures ANOVA depends critically on the average correlation among conditions ($\rho$). Higher $\rho$ → greater power (more individual variability removed). Always specify $\rho$ in power analyses for within-subjects designs.

4. Assumptions of Repeated Measures ANOVA

4.1 Normality of Residuals (or Condition Scores)

The repeated measures ANOVA assumes that the residual scores (or equivalently, the scores within each condition) are approximately normally distributed in the population.

How to check:

Robustness: The F-test is moderately robust to non-normality when $n \geq 30$ (via the Central Limit Theorem) and when the violation is symmetric. Severe skewness with small $n$ is the primary concern.

When violated:

• Use the Friedman test (non-parametric alternative) for small samples with non-normal data.
• Consider log or square-root transformation for right-skewed outcome variables.
• Use a linear mixed-effects model with robust standard errors.

4.2 Sphericity

Sphericity is the assumption that the variances of all pairwise difference scores between conditions are equal. For $k$ conditions, there are $k(k-1)/2$ pairwise differences, each of which must have the same variance.

Formally, if $d_{jj'} = X_{ij} - X_{ij'}$ is the difference score between conditions $j$ and $j'$ for participant $i$, then sphericity requires:

$\text{Var}(d_{jj'}) = \text{Var}(d_{jj''}) \quad \text{for all } j, j', j''$

Compound symmetry (equal variances and equal covariances across all conditions) is a sufficient but not necessary condition for sphericity. Sphericity is a weaker requirement than compound symmetry and is the actual assumption of the F-test.

Mauchly's Test of Sphericity:

$W = \frac{\prod_{l=1}^{k-1} \lambda_l}{\left(\frac{\sum_{l=1}^{k-1}\lambda_l}{k-1}\right)^{k-1}}$

Where $\lambda_l$ are the eigenvalues of the transformed covariance matrix (using orthonormal contrasts). The test statistic:

$\chi^2 = -\left(n - 1 - \frac{2(k-1)^2 + k + 1}{6(k-1)}\right)\ln(W)$

With $df = (k-1)(k+2)/2 - 1 = k(k-1)/2 - 1$.

$H_0$: Sphericity holds ($W = 1$). $p \leq .05$ → reject sphericity; apply corrections.

⚠️ Mauchly's test is sensitive to non-normality and can give misleading results in small samples (underpowered) and large samples (overpowered — detecting trivial violations). Always report $\hat{\varepsilon}$ alongside Mauchly's test result. $\hat{\varepsilon}_{GG} < 0.75$ indicates a practically meaningful violation regardless of the Mauchly p-value.

Epsilon values and their implications:

Note: The sphericity assumption is irrelevant when $k = 2$ (only two conditions form exactly one difference score, whose variance always equals itself). This is why the paired t-test needs no sphericity correction.

4.3 Independence of Observations Between Participants

While the design deliberately induces correlation within participants (across conditions), the observations between participants must be independent. Each participant's data must not influence another participant's data.

Common violations:

• Participants in the same lab session who can observe or influence each other.
• Family members or partners in the same study.
• Hierarchically nested data (e.g., students within classrooms all treated as independent) — use linear mixed-effects models instead.

4.4 Interval or Ratio Scale of Measurement

The dependent variable must be continuous and measured on at least an interval scale — equal numerical differences must represent equal psychological or physical differences across the entire scale range.

When violated: If the dependent variable is ordinal (e.g., ranks or Likert ratings treated as ordinal), use the Friedman test instead.

4.5 No Extreme Multivariate Outliers

Outliers in any condition can distort condition means and inflate $SS_{Error}$, potentially masking real effects or creating spurious ones.

How to check:

• Boxplots for each condition.
• Standardised scores $|z_i| > 3.29$ within each condition.
• Mahalanobis distance across conditions: flags participants who are outliers in the multivariate sense (unusual profile across all conditions simultaneously).

When outliers present: Investigate the cause. Report analyses with and without outliers. Consider the Friedman test or trimmed-mean ANOVA as robust alternatives.

4.6 Assumption Summary

5. Variants of Repeated Measures ANOVA

5.1 One-Way Repeated Measures ANOVA

The standard form described throughout this tutorial: one within-subjects factor with $k \geq 3$ levels. Tests whether condition means differ significantly.

5.2 Factorial Repeated Measures ANOVA (Two or More Within Factors)

When each participant is measured across all combinations of two or more within-subjects factors, a factorial repeated measures ANOVA is used.

For factors $A$ (with $a$ levels) and $B$ (with $b$ levels), the partition is:

$SS_{Within} = SS_A + SS_B + SS_{A \times B} + SS_{A \times S} + SS_{B \times S} + SS_{A \times B \times S}$

Where $S$ denotes subjects. Each main effect and interaction has its own error term (the corresponding subjects-by-factor interaction). This allows each effect to be tested against a different, tailored error term.

5.3 Mixed ANOVA (Split-Plot Design)

The mixed ANOVA (also called split-plot ANOVA) combines:

• One or more between-subjects factors (different participants per group).
• One or more within-subjects factors (same participants across conditions).

Example: Comparing three treatment groups (between) measured at pre, mid, and post (within). The interaction between group and time (Group × Time) is typically the focal test — it assesses whether the trajectory of change over time differs across groups.

Variance partitioning:

$SS_{Total} = SS_{Between\text{-}Subjects} + SS_{Within\text{-}Subjects}$

$SS_{Between\text{-}Subjects} = SS_{Group} + SS_{Subjects(Group)}$

$SS_{Within\text{-}Subjects} = SS_{Time} + SS_{Group \times Time} + SS_{Time \times Subjects(Group)}$

Each between-subjects effect uses $MS_{Subjects(Group)}$ as its error term; each within-subjects effect uses $MS_{Time \times Subjects(Group)}$ as its error term.

5.4 Friedman Test (Non-Parametric Alternative)

When the normality assumption is severely violated or the data are ordinal, the Friedman test is the non-parametric equivalent of one-way repeated measures ANOVA.

Procedure:

1. Rank each participant's scores across the $k$ conditions (1 = lowest, $k$ = highest).
2. Compute the mean rank for each condition: $\bar{R}_j = \frac{1}{n}\sum_i R_{ij}$.
3. Compute the Friedman statistic:

$\chi^2_F = \frac{12n}{k(k+1)}\sum_{j=1}^k \left(\bar{R}_j - \frac{k+1}{2}\right)^2$

Under $H_0$, $\chi^2_F \approx \chi^2_{k-1}$ for large $n$.

Effect size: Kendall's $W$ (coefficient of concordance):

$W = \frac{\chi^2_F}{n(k-1)}$

$W$ ranges from 0 (no agreement in rankings across participants) to 1 (perfect agreement). Conversion to $r$: $r = 2W - 1$ (for $k = 2$).

5.5 Trend Analysis (Polynomial Contrasts)

When the within-subjects factor is quantitative and equally spaced (e.g., time points at regular intervals, dosage levels at equal increments), trend analysis decomposes the condition effect into orthogonal polynomial components:

• Linear trend: Does the mean increase or decrease monotonically across levels?
• Quadratic trend: Is there a U-shaped or inverted-U-shaped pattern?
• Cubic trend: Is there an S-shaped or more complex pattern?

Each trend component has $df = 1$ and is tested separately against the error mean square. Trend analysis is more powerful and more informative than the omnibus F-test when a specific trajectory is hypothesised.

5.6 Linear Mixed-Effects Models (LMM)

Linear mixed-effects models (also called multilevel models or hierarchical linear models) subsume repeated measures ANOVA as a special case while offering several important generalisations:

• Handle missing data without excluding participants (ANOVA requires complete data or imputation).
• Model unequal time intervals between measurements.
• Allow time-varying covariates as predictors.
• Specify flexible covariance structures (not restricted to compound symmetry or sphericity).
• Accommodate both balanced and unbalanced designs.

For complex longitudinal designs, LMMs are generally preferred over repeated measures ANOVA. DataStatPro's repeated measures ANOVA module automatically suggests LMM when missing data are detected.

5.7 Bayesian Repeated Measures ANOVA

The Bayesian approach computes Bayes Factors comparing models with and without the condition effect. Under default priors (Rouder et al., 2012):

$BF_{10} = \frac{P(\text{data} \mid H_1: \text{condition effect exists})}{P(\text{data} \mid H_0: \text{no condition effect})}$

Interpreting $BF_{10}$:

6. Using the Repeated Measures ANOVA Calculator Component

The Repeated Measures ANOVA Calculator in DataStatPro provides a comprehensive tool for running, diagnosing, and reporting within-subjects analyses.

Step-by-Step Guide

Step 1 — Select the Test

Navigate to Statistical Tests → ANOVA → Repeated Measures ANOVA.

Step 2 — Input Method

Choose how to provide data:

• Raw data (wide format): Each row is one participant; each column is one condition. DataStatPro automatically identifies the within-subjects structure.
• Raw data (long format): Three columns required: participant ID, condition label, and dependent variable value. DataStatPro reshapes to wide format internally.
• Summary statistics: Enter $n$, condition means ($\bar{X}_{\cdot j}$), standard deviations ($s_j$), and the correlation matrix across conditions. DataStatPro reconstructs the ANOVA source table.

Step 3 — Define the Within-Subjects Factor

• Specify the factor name (e.g., "Time", "Condition", "Dosage").
• Label each level (e.g., "Pre", "Mid", "Post").
• For factorial designs, define additional within-subjects factors and their levels.
• For mixed designs, specify the between-subjects grouping variable.

Step 4 — Select Post-Hoc Tests and Contrasts

• Post-hoc tests (exploratory): Bonferroni, Holm, Tukey's HSD (adapted for within-subjects), Sidák.
• Planned contrasts: Simple (each level vs. first), Helmert (each level vs. mean of preceding), polynomial (linear, quadratic, cubic trend), custom.

Step 5 — Set Significance Level and Confidence Level

Default: $\alpha = .05$, 95% CI. Results at $\alpha = .01$ and $\alpha = .001$ are simultaneously displayed.

Step 6 — Select Display Options

• ✅ Full ANOVA source table with $SS$, $df$, $MS$, $F$, exact $p$.
• ✅ Mauchly's test of sphericity and $\hat{\varepsilon}_{GG}$, $\tilde{\varepsilon}_{HF}$.
• ✅ Greenhouse-Geisser and Huynh-Feldt corrected results (automatically displayed when sphericity is violated).
• ✅ Multivariate test statistics (Pillai, Wilks, Hotelling, Roy) as an alternative.
• ✅ Partial eta-squared ($\eta^2_p$) and omega-squared ($\omega^2_p$) with 95% CI.
• ✅ Generalised eta-squared ($\eta^2_G$).
• ✅ Cohen's $f$ for power analysis.
• ✅ Condition means, standard deviations, and 95% CIs with error bar plots.
• ✅ Post-hoc comparison table (pairwise $t$-tests with corrections).
• ✅ Profile plot (means across conditions with individual participant trajectories).
• ✅ Interaction plot (for factorial and mixed designs).
• ✅ Residual Q-Q plots and normality test per condition.
• ✅ Mahalanobis distance outlier detection.
• ✅ Power analysis: current post-hoc power and required $n$ for 80%, 90%, 95% power.
• ✅ Bayesian ANOVA (Bayes Factor $BF_{10}$).
• ✅ APA 7th edition results paragraph (auto-generated).

Step 7 — Run the Analysis

Click "Run Repeated Measures ANOVA". DataStatPro will:

1. Compute all $SS$, $df$, $MS$, $F$, and exact p-values.
2. Conduct Mauchly's test; apply GG and HF corrections as appropriate.
3. Compute multivariate test statistics.
4. Compute $\eta^2_p$, $\omega^2_p$, $\eta^2_G$, and Cohen's $f$ with 95% CIs.
5. Run selected post-hoc comparisons or planned contrasts.
6. Conduct normality and outlier diagnostics.
7. Estimate post-hoc power.
8. Output an APA-compliant results paragraph.

7. Step-by-Step Procedure

7.1 Full Manual Procedure

Step 1 — State the Hypotheses

$H_0: \mu_1 = \mu_2 = \cdots = \mu_k$ (all condition population means are equal)

$H_1:$ At least one $\mu_j$ differs from at least one other $\mu_{j'}$

Specify the within-subjects factor and the number of levels $k$ based on the research design, before examining the data.

Step 2 — Organise the Data Matrix

Arrange data in an $n \times k$ matrix with one row per participant and one column per condition. Verify that there are no missing values (or plan imputation/LMM approach).

Step 3 — Check Assumptions

• Inspect distributions within each condition (histograms, Q-Q plots, Shapiro-Wilk).
• Investigate multivariate outliers (Mahalanobis distance).
• Confirm independence of observations between participants (design review).
• Proceed to compute expected frequencies — irrelevant here; proceed to compute $E_{ij}$ — irrelevant; proceed to test sphericity after computing $SS$ components.

Step 4 — Compute Grand Mean and Condition Means

$\bar{X}_{\cdot\cdot} = \frac{1}{nk}\sum_{i=1}^n\sum_{j=1}^k X_{ij}$

$\bar{X}_{\cdot j} = \frac{1}{n}\sum_{i=1}^n X_{ij} \quad \text{for each condition } j$

$\bar{X}_{i\cdot} = \frac{1}{k}\sum_{j=1}^k X_{ij} \quad \text{for each participant } i$

Step 5 — Compute Sums of Squares

$SS_{Total} = \sum_{i=1}^n\sum_{j=1}^k (X_{ij} - \bar{X}_{\cdot\cdot})^2$

$SS_{Between\text{-}Subjects} = k\sum_{i=1}^n (\bar{X}_{i\cdot} - \bar{X}_{\cdot\cdot})^2$

$SS_{Condition} = n\sum_{j=1}^k (\bar{X}_{\cdot j} - \bar{X}_{\cdot\cdot})^2$

$SS_{Error} = SS_{Total} - SS_{Between\text{-}Subjects} - SS_{Condition}$

Step 6 — Compute Degrees of Freedom

$df_{Condition} = k - 1$

$df_{Error} = (k-1)(n-1)$

Step 7 — Conduct Mauchly's Test of Sphericity

Compute the covariance matrix of condition scores $\boldsymbol{\Sigma}$ and obtain $\hat{\varepsilon}_{GG}$ and $\tilde{\varepsilon}_{HF}$. If Mauchly's $p < .05$ or $\hat{\varepsilon}_{GG} < 0.75$, apply the appropriate correction.

Step 8 — Compute Mean Squares and the F-Ratio

$MS_{Condition} = \frac{SS_{Condition}}{df^*_{Condition}}$ (use corrected $df^*$ if applicable)

$MS_{Error} = \frac{SS_{Error}}{df^*_{Error}}$

$F = \frac{MS_{Condition}}{MS_{Error}}$

Step 9 — Compute the p-Value

$p = P(F_{df^*_1, df^*_2} \geq F_{obs})$

Reject $H_0$ if $p \leq \alpha$.

Step 10 — Compute Effect Sizes

$\eta^2_p = \frac{SS_{Condition}}{SS_{Condition} + SS_{Error}}$

$\omega^2_p = \frac{SS_{Condition} - (k-1)MS_{Error}}{SS_{Condition} + (n-k+1) \cdot MS_{Error}}$

$f = \sqrt{\frac{\eta^2_p}{1 - \eta^2_p}}$

Step 11 — Conduct Post-Hoc Comparisons (if $H_0$ rejected)

For each pair of conditions $(j, j')$, compute the pairwise paired t-test:

$t_{jj'} = \frac{\bar{X}_{\cdot j} - \bar{X}_{\cdot j'}}{s_{d_{jj'}}/\sqrt{n}}$

Where $s_{d_{jj'}}$ is the standard deviation of the difference scores $d_i = X_{ij} - X_{ij'}$. Apply Bonferroni, Holm, or Tukey correction for the $k(k-1)/2$ pairwise comparisons.

Step 12 — Interpret and Report

Use the APA reporting template in Section 15. Always report $F$, both $df$ values (with corrections if applicable), $p$, $\eta^2_p$, $\omega^2_p$, and 95% CI for the effect size. Report Mauchly's test result and the epsilon correction applied.

8. Interpreting the Output

8.1 The F-Statistic

8.2 The p-Value

⚠️ A significant F-test is omnibus: it indicates only that at least one pair of condition means differs. It does not identify which conditions differ or how large those differences are. Always follow a significant omnibus test with post-hoc comparisons or planned contrasts, and always report effect sizes.

8.3 Mauchly's Test and Epsilon

⚠️ Always report $\hat{\varepsilon}_{GG}$ regardless of Mauchly's test outcome, since Mauchly's test may lack power in small samples. A reader can use $\hat{\varepsilon}_{GG}$ to judge the severity of any sphericity violation.

8.4 The ANOVA Source Table

When reading the output table, focus on:

1. $SS_{Condition}$: Total systematic variability across conditions — larger values indicate greater spread of condition means.
2. $SS_{Error}$: Residual within-subjects variability after removing condition and participant effects — smaller values (more tightly correlated conditions) indicate a more sensitive design.
3. $MS_{Condition}/MS_{Error}$ ratio: The F-ratio — the key test statistic.
4. $df$ values: Verify these match $(k-1)$ and $(k-1)(n-1)$ (uncorrected) or their epsilon-adjusted equivalents.

8.5 Partial Eta-Squared ($\eta^2_p$) — Magnitude Interpretation

Cohen's (1988) benchmarks for $\eta^2_p$ (and $f$):

Extended benchmarks (Lakens, 2013, contextualised for within-subjects designs):

⚠️ These benchmarks were developed for between-subjects designs. In within-subjects designs, $\eta^2_p$ values tend to be larger because individual differences are removed from the error term. Do not mechanically apply Cohen's benchmarks without considering the typical effect sizes in your specific research domain.

8.6 Post-Hoc Comparisons

After a significant omnibus F-test, post-hoc pairwise comparisons identify which specific pairs of conditions differ. For $k$ conditions there are $k(k-1)/2$ pairs:

For each comparison, report the mean difference $(\bar{X}_{\cdot j} - \bar{X}_{\cdot j'})$, the $t$-statistic, the adjusted p-value, and Cohen's $d_{rm}$ for the paired comparison:

$d_{rm} = \frac{\bar{X}_{\cdot j} - \bar{X}_{\cdot j'}}{s_{d_{jj'}}}$

8.7 Profile Plots

A profile plot (line chart with conditions on the x-axis and mean outcome on the y-axis) is the primary visualisation for repeated measures ANOVA. Key features to examine:

• Parallel lines (in mixed designs): Suggests no interaction between between- and within-subjects factors.
• Crossing or diverging lines: Suggests a Group × Time interaction — the critical test in most mixed designs.
• Error bars: Should represent 95% within-subjects confidence intervals (not the standard between-subjects SEM), computed using the Cousineau-Morey correction, which removes between-subjects variance from the error.

9. Effect Sizes for Repeated Measures ANOVA

9.1 Partial Eta-Squared ($\eta^2_p$)

$\eta^2_p = \frac{SS_{Condition}}{SS_{Condition} + SS_{Error}}$

The proportion of within-subjects variance (after removing participant-level variance) explained by the condition effect. This is the standard effect size reported by SPSS, SAS, and most ANOVA software. Upwardly biased in small samples.

9.2 Generalised Eta-Squared ($\eta^2_G$)

$\eta^2_G = \frac{SS_{Condition}}{SS_{Condition} + SS_{Between\text{-}Subjects} + SS_{Error}}$

Designed for comparability across studies that differ in the number of conditions and design type (between vs. within). Recommended for meta-analyses. Note that $\eta^2_G \leq \eta^2_p$ always, since the denominator of $\eta^2_G$ is larger.

9.3 Partial Omega-Squared ($\omega^2_p$) — Bias-Corrected

$\omega^2_p = \frac{SS_{Condition} - (k-1)MS_{Error}}{SS_{Condition} + (n - k + 1) \cdot MS_{Error}}$

The unbiased (or less biased) estimate of the true population partial effect size. Preferred over $\eta^2_p$ for small samples ($n < 30$). Note that $\omega^2_p$ can be negative for very small effects — negative values should be reported as $\omega^2_p \approx 0$ (indicating no effect).

9.4 Cohen's $f$

$f = \sqrt{\frac{\eta^2_p}{1 - \eta^2_p}} = \frac{\sigma_m}{\sigma_\varepsilon}$

Where $\sigma_m$ is the standard deviation of the $k$ true condition means around the grand mean, and $\sigma_\varepsilon$ is the within-condition population standard deviation (error). Cohen's $f$ is used primarily in power analysis. Benchmarks: $f = 0.10$ (small), $f = 0.25$ (medium), $f = 0.40$ (large).

9.5 Pairwise Cohen's $d$ for Post-Hoc Comparisons

For each pairwise comparison $(j, j')$ after a significant omnibus test:

$d_{rm} = \frac{\bar{X}_{\cdot j} - \bar{X}_{\cdot j'}}{s_{d_{jj'}}}$

Where $s_{d_{jj'}}$ is the standard deviation of the difference scores between conditions $j$ and $j'$. This is the paired Cohen's $d$ and is directly interpretable as the magnitude of the difference between two specific conditions.

9.6 Effect Size Summary Table