Meta-Analysis: Zero to Hero Tutorial

This comprehensive tutorial takes you from the foundational concepts of meta-analysis all the way through advanced effect size computation, heterogeneity analysis, publication bias assessment, and practical usage within the DataStatPro application. Whether you are a complete beginner or an experienced researcher, this guide is structured to build your understanding step by step.

Table of Contents

1. Prerequisites and Background Concepts
2. What is Meta-Analysis?
3. The Mathematics Behind Meta-Analysis
4. Effect Size Measures
5. Meta-Analysis Based on Original Measures
6. Fixed-Effect vs. Random-Effects Models
7. Assumptions of Meta-Analysis
8. Heterogeneity
9. Forest Plots
10. Publication Bias
11. Moderator Analysis: Subgroup Analysis and Meta-Regression
12. Sensitivity Analysis
13. Using the Meta-Analysis Component
14. Computational and Formula Details
15. Worked Examples
16. Common Mistakes and How to Avoid Them
17. Troubleshooting
18. Quick Reference Cheat Sheet

1. Prerequisites and Background Concepts

Before diving into meta-analysis, it is helpful to be familiar with the following foundational concepts. Do not worry if you are not — each concept is briefly explained here.

1.1 The Concept of a Study Effect

In any individual research study, the effect is the quantitative result of interest — for example, the difference in mean blood pressure between a treatment group and a control group, or the correlation between study hours and exam scores. Meta-analysis combines these effects across multiple studies.

1.2 Variance and Standard Error

Variance ($\sigma^2$) measures the spread of a distribution. The standard error (SE) is the standard deviation of a sampling distribution (i.e., how much an estimate, such as a mean, is expected to vary across samples):

$SE = \frac{\sigma}{\sqrt{n}}$

In meta-analysis, each study has its own effect estimate with its own standard error. Studies with larger sample sizes produce smaller standard errors (more precise estimates).

1.3 Confidence Intervals

A 95% confidence interval (CI) provides a range of plausible values for the true population parameter. For a normally distributed estimator:

$\text{CI} = \hat{\theta} \pm z_{\alpha/2} \times SE(\hat{\theta})$

Where $z_{0.025} = 1.96$ for a 95% CI. In meta-analysis, every study's effect estimate is paired with a confidence interval.

1.4 Weighted Averages

A weighted average gives more influence to certain values based on their importance (weight):

$\bar{X}_w = \frac{\sum_{i=1}^k w_i X_i}{\sum_{i=1}^k w_i}$

In meta-analysis, studies with greater precision (smaller variance) receive larger weights, so that more reliable studies contribute more to the pooled estimate.

1.5 The Normal (Z) and Chi-Squared Distributions

The standard normal distribution $\mathcal{N}(0,1)$ underpins significance testing of pooled effects. The chi-squared distribution $\chi^2$ is used for testing heterogeneity across studies. Both are used extensively in meta-analytic calculations.

2. What is Meta-Analysis?

Meta-analysis is a quantitative statistical technique that combines the results of multiple independent studies addressing the same research question to produce a single, more precise and reliable estimate of the true effect.

2.1 The Systematic Review Context

Meta-analysis is almost always conducted as part of a systematic review — a rigorous, structured process of:

1. Formulating a precise research question.
2. Systematically searching the literature.
3. Screening studies against inclusion/exclusion criteria.
4. Extracting data from eligible studies.
5. Assessing study quality and risk of bias.
6. Synthesising results statistically (the meta-analysis).

Meta-analysis is the statistical step within a systematic review. Not every systematic review includes a meta-analysis (if studies are too heterogeneous, a narrative synthesis may be preferred), but every meta-analysis should be embedded within a systematic review.

2.2 Why Conduct a Meta-Analysis?

Individual studies are often limited by:

• Small sample sizes (low statistical power).
• Conflicting results across studies.
• Publication bias (positive results are more likely to be published).
• Imprecise effect estimates (wide confidence intervals).

Meta-analysis addresses these issues by:

• Increasing statistical power through the combination of samples.
• Improving precision of the pooled effect estimate.
• Resolving apparent conflicts between studies.
• Quantifying and exploring heterogeneity in effects across studies.
• Detecting publication bias formally.

2.3 Real-World Applications

Meta-analysis is one of the most influential tools in evidence-based practice. Common applications include:

• Medicine & Clinical Research: Does Drug A reduce mortality compared to placebo? Do statins lower the risk of cardiovascular events?
• Psychology: What is the true effect size of cognitive behavioural therapy for depression?
• Education: Does class size reduction improve student achievement?
• Management & Organisational Science: What is the relationship between employee engagement and performance?
• Public Health: What is the effect of smoking on lung cancer risk across populations?
• Ecology: How does habitat fragmentation affect species diversity?

2.4 The Two Broad Approaches

Meta-analysis can be conducted using two broad data inputs:

Both approaches are covered in depth in this tutorial.

3. The Mathematics Behind Meta-Analysis

3.1 The Basic Meta-Analytic Model

Let $k$ be the number of studies included in the meta-analysis. Each study $i$ ($i = 1, 2, \dots, k$) provides:

• An observed effect estimate $\hat{\theta}_i$ (e.g., mean difference, log odds ratio, correlation).
• A within-study variance $v_i = SE_i^2$ (the squared standard error of the effect estimate).

The fundamental assumption is that each observed effect estimate is drawn from a distribution centred on a true effect:

$\hat{\theta}_i = \theta_i + \epsilon_i, \quad \epsilon_i \sim \mathcal{N}(0, v_i)$

Where $\epsilon_i$ is sampling error for study $i$. What $\theta_i$ represents depends on the model chosen (fixed-effect or random-effects — see Section 6).

3.2 The Pooled Effect Estimate

The pooled (combined) effect estimate is a weighted average of the individual study effects:

$\hat{\theta}_{\text{pooled}} = \frac{\sum_{i=1}^k w_i \hat{\theta}_i}{\sum_{i=1}^k w_i}$

Where $w_i$ is the weight assigned to study $i$. The exact form of $w_i$ differs between the fixed-effect and random-effects models (see Section 6).

3.3 Variance and Standard Error of the Pooled Estimate

The variance of the pooled estimate is:

$\text{Var}(\hat{\theta}_{\text{pooled}}) = \frac{1}{\sum_{i=1}^k w_i}$

The standard error of the pooled estimate is:

$SE(\hat{\theta}_{\text{pooled}}) = \sqrt{\frac{1}{\sum_{i=1}^k w_i}}$

3.4 Confidence Interval for the Pooled Effect

A $(1-\alpha) \times 100\%$ confidence interval for the pooled effect is:

$\left[\hat{\theta}_{\text{pooled}} - z_{\alpha/2} \times SE(\hat{\theta}_{\text{pooled}}), \quad \hat{\theta}_{\text{pooled}} + z_{\alpha/2} \times SE(\hat{\theta}_{\text{pooled}})\right]$

For a 95% CI, $z_{0.025} = 1.96$.

3.5 Z-Test for the Pooled Effect

To test the null hypothesis $H_0: \theta = 0$ (no overall effect), the test statistic is:

$Z = \frac{\hat{\theta}_{\text{pooled}}}{SE(\hat{\theta}_{\text{pooled}})}$

Under $H_0$, $Z$ follows a standard normal distribution $\mathcal{N}(0,1)$. The two-sided p-value is:

$p\text{-value} = 2 \times (1 - \Phi(|Z|))$

4. Effect Size Measures

An effect size is a standardised, scale-free numerical index of the magnitude of a phenomenon. Meta-analysis based on effect sizes is the most common approach because it allows combining studies that may use different scales to measure the same construct.

4.1 Effect Sizes for Comparing Two Groups (Continuous Outcomes)

4.1.1 Raw (Unstandardised) Mean Difference (MD)

The simplest effect size for two-group comparisons on a continuous outcome measured on the same scale:

$MD = \bar{X}_T - \bar{X}_C$

Where:

• $\bar{X}_T$ = mean of the treatment/experimental group.
• $\bar{X}_C$ = mean of the control/comparison group.

Variance of MD:

$v_{MD} = \frac{SD_T^2}{n_T} + \frac{SD_C^2}{n_C}$

Where $SD_T$, $SD_C$ are the standard deviations and $n_T$, $n_C$ are the sample sizes of the treatment and control groups.

💡 Use MD when all studies measure the outcome on the same scale (e.g., all report blood pressure in mmHg). If studies use different scales, use a standardised effect size instead.

4.1.2 Cohen's d (Standardised Mean Difference)

Cohen's d standardises the mean difference by dividing by a pooled standard deviation, making it scale-free and comparable across studies using different measurement instruments:

$d = \frac{\bar{X}_T - \bar{X}_C}{SD_{\text{pooled}}}$

Where the pooled standard deviation is:

$SD_{\text{pooled}} = \sqrt{\frac{(n_T - 1)SD_T^2 + (n_C - 1)SD_C^2}{n_T + n_C - 2}}$

Variance of Cohen's d:

$v_d = \frac{n_T + n_C}{n_T \times n_C} + \frac{d^2}{2(n_T + n_C - 2)}$

Interpretation of Cohen's d:

⚠️ Cohen's conventional benchmarks are rough guidelines. "Small," "medium," and "large" are context-dependent — always interpret effect sizes in the context of the domain.

4.1.3 Hedges' g (Bias-Corrected Standardised Mean Difference)

Cohen's $d$ is slightly positively biased (overestimates the true effect) in small samples. Hedges' g applies a correction factor $J$ to remove this bias:

$g = J \times d$

Where the correction factor is:

$J = 1 - \frac{3}{4(n_T + n_C - 2) - 1}$

For large samples, $J \approx 1$ and $g \approx d$. For small samples, the correction matters.

Variance of Hedges' g:

$v_g = J^2 \times v_d$

💡 Hedges' g is generally preferred over Cohen's d in meta-analysis due to its reduced bias in small samples.

4.1.4 Glass's Delta ($\Delta$)

Glass's Delta standardises the mean difference using only the control group standard deviation, rather than the pooled SD:

$\Delta = \frac{\bar{X}_T - \bar{X}_C}{SD_C}$

💡 Use Glass's Delta when the treatment may have affected the variability of scores (i.e., the SD of the treatment group may be inflated or deflated by the intervention). Using the control group SD provides a cleaner baseline.

4.2 Effect Sizes for Binary Outcomes

When outcomes are binary (event/non-event), effect sizes are based on $2 \times 2$ contingency tables:

4.2.1 Odds Ratio (OR)

The odds ratio compares the odds of the event in the treatment group to the odds in the control group:

$OR = \frac{a/b}{c/d} = \frac{ad}{bc}$

Because the OR is positively skewed, meta-analysis is performed on the natural logarithm of the OR:

$\ln(OR) = \ln\left(\frac{ad}{bc}\right) = \ln(a) + \ln(d) - \ln(b) - \ln(c)$

Variance of $\ln(OR)$:

$v_{\ln(OR)} = \frac{1}{a} + \frac{1}{b} + \frac{1}{c} + \frac{1}{d}$

The pooled $\ln(OR)$ is back-transformed to the OR scale for reporting: $OR_{\text{pooled}} = e^{\ln(OR)_{\text{pooled}}}$.

Interpretation:

⚠️ The OR can be a misleading measure when the event is common (prevalence > 10%), because it exaggerates the relative risk. In such cases, the Risk Ratio may be preferred.

4.2.2 Risk Ratio (Relative Risk, RR)

The risk ratio compares the probability (risk) of the event in the treatment group to the control group:

$RR = \frac{a/(a+b)}{c/(c+d)} = \frac{a \cdot (c+d)}{c \cdot (a+b)}$

Meta-analysis is performed on the log RR:

$\ln(RR) = \ln\left(\frac{a/(a+b)}{c/(c+d)}\right)$

Variance of $\ln(RR)$:

$v_{\ln(RR)} = \frac{1}{a} - \frac{1}{a+b} + \frac{1}{c} - \frac{1}{c+d}$

Interpretation:

4.2.3 Risk Difference (RD)

The risk difference (also called the absolute risk reduction or attributable risk) is the difference in event probabilities between groups:

$RD = \frac{a}{a+b} - \frac{c}{c+d}$

Variance of RD:

$v_{RD} = \frac{a \cdot b}{(a+b)^3} + \frac{c \cdot d}{(c+d)^3}$

💡 The RD is on an absolute scale (e.g., 0.05 = 5 percentage points), making it clinically interpretable. The number needed to treat (NNT) = 1/|RD|.

4.3 Effect Sizes for Correlation

4.3.1 Pearson's r

When studies report the correlation coefficient $r$ between two continuous variables, it can be used directly as an effect size.

Interpretation of $|r|$:

4.3.2 Fisher's Z Transformation

Because $r$ has a bounded range $[-1, 1]$ and a non-normal sampling distribution (especially when $r$ is far from zero), meta-analysis is performed on the Fisher's Z-transformed correlation:

$z_r = \frac{1}{2} \ln\left(\frac{1+r}{1-r}\right) = \text{arctanh}(r)$

Variance of $z_r$ (remarkably simple):

$v_{z_r} = \frac{1}{n - 3}$

Where $n$ is the sample size of the study. The pooled $z_r$ is back-transformed to $r$ for reporting:

$r_{\text{pooled}} = \frac{e^{2\hat{z}_r} - 1}{e^{2\hat{z}_r} + 1} = \tanh(\hat{z}_r)$

4.4 Effect Sizes for Single Proportions

When studies report a single event proportion $p = a/n$ (e.g., prevalence of a disease):

$p = \frac{a}{n}$

Variance of $p$:

$v_p = \frac{p(1-p)}{n}$

Because proportions are bounded $[0,1]$ and can have skewed distributions, transformations are commonly applied:

Logit Transformation:

$\text{logit}(p) = \ln\left(\frac{p}{1-p}\right), \quad v_{\text{logit}(p)} = \frac{1}{np(1-p)}$

Double Arcsine (Freeman-Tukey) Transformation:

$\phi = \arcsin\left(\sqrt{\frac{a}{n+1}}\right) + \arcsin\left(\sqrt{\frac{a+1}{n+1}}\right)$

$v_\phi = \frac{1}{n + 0.5}$

💡 The double arcsine transformation is particularly useful when proportions are close to 0 or 1 (rare or very common events), as it stabilises the variance effectively.

4.5 Comparison of Effect Size Measures

5. Meta-Analysis Based on Original Measures

When all studies in a meta-analysis measure the same outcome on the same original scale, it is valid — and often preferable — to pool the studies using the original (unstandardised) measures directly, without standardisation.

5.1 Meta-Analysis of Means (Single Group)

When studies report a single-group mean (e.g., mean age, mean biomarker level in a specific population), the effect of interest is the mean $\mu$ itself.

Input data per study $i$:

• Sample mean: $\bar{X}_i$
• Standard deviation: $SD_i$
• Sample size: $n_i$

Within-study variance:

$v_i = \frac{SD_i^2}{n_i}$

The pooled mean is estimated as the weighted average described in Section 3.2, using weights $w_i = 1/v_i$.

5.2 Meta-Analysis of Raw Mean Differences (Two Groups)

When studies compare a treatment group to a control group using the same outcome measure, the raw mean difference is directly pooled.

Input data per study $i$:

• Treatment mean: $\bar{X}_{T,i}$, $SD_{T,i}$, $n_{T,i}$
• Control mean: $\bar{X}_{C,i}$, $SD_{C,i}$, $n_{C,i}$

Effect estimate:

$MD_i = \bar{X}_{T,i} - \bar{X}_{C,i}$

Within-study variance:

$v_i = \frac{SD_{T,i}^2}{n_{T,i}} + \frac{SD_{C,i}^2}{n_{C,i}}$

5.3 Meta-Analysis of Proportions

When studies report a single proportion (e.g., the prevalence of hypertension in a population), the goal is to pool the proportions across studies.

Input data per study $i$:

• Number of events: $a_i$
• Total sample size: $n_i$

Observed proportion:

$p_i = \frac{a_i}{n_i}$

The pooling is typically performed on the logit or double arcsine transformed proportions (see Section 4.4) to improve normality, and the result is back-transformed to the proportion scale.

5.4 Meta-Analysis of Incidence Rates

When studies report event counts and person-time (incidence rate data), the effect measure is the incidence rate:

$\lambda_i = \frac{a_i}{T_i}$

Where $a_i$ is the number of events and $T_i$ is the total person-time. Meta-analysis is typically performed on the log incidence rate:

$\ln(\lambda_i) = \ln(a_i) - \ln(T_i)$

Variance of $\ln(\lambda_i)$:

$v_{\ln(\lambda_i)} = \frac{1}{a_i}$

5.5 Meta-Analysis of 2×2 Table Data (Raw Counts)

When studies report raw event counts from a $2 \times 2$ contingency table (events and non-events in two groups), several effect measures can be computed:

From these, the OR, RR, or RD are computed (see Section 4.2) and pooled. This is the most common input format for meta-analyses of clinical trials with binary outcomes.

💡 When cell counts are zero (e.g., no events in one group), a small continuity correction (typically adding 0.5 to all cells) is applied to allow computation of log-scale measures. This is known as the Haldane-Anscombe correction.

6. Fixed-Effect vs. Random-Effects Models

The choice between the fixed-effect model and the random-effects model is one of the most important decisions in meta-analysis. They differ fundamentally in their assumptions about the nature of the true effect across studies.

6.1 The Fixed-Effect Model

Core Assumption: All studies in the meta-analysis share the same single true effect size $\theta$. Differences between observed study results are due only to sampling error (within-study variability).

$\hat{\theta}_i = \theta + \epsilon_i, \quad \epsilon_i \sim \mathcal{N}(0, v_i)$

Fixed-Effect Weight:

$w_i^{FE} = \frac{1}{v_i}$

Each study is weighted by the inverse of its within-study variance. Larger, more precise studies get more weight.

Pooled Effect (Fixed-Effect):

$\hat{\theta}_{FE} = \frac{\sum_{i=1}^k w_i^{FE} \hat{\theta}_i}{\sum_{i=1}^k w_i^{FE}}$

When to use:

• When all studies are virtually identical in design, population, intervention, and outcome (a very rare situation).
• When the goal is to estimate the effect for the specific set of studies included, not to generalise to other settings.

⚠️ The fixed-effect model is theoretically appropriate only when all studies are functionally identical. In most real-world meta-analyses, this assumption is untenable. The random-effects model is almost always more appropriate.

6.2 The Random-Effects Model

Core Assumption: The studies in the meta-analysis are a random sample from a population of studies with varying true effects. Each study has its own true effect $\theta_i$, which is drawn from a distribution of true effects:

$\theta_i \sim \mathcal{N}(\mu, \tau^2)$

Where:

• $\mu$ is the mean of the distribution of true effects (the overall pooled effect we estimate).
• $\tau^2$ (tau-squared) is the between-study variance — the variance of the true effects across studies.

The observed effect in study $i$ is then:

$\hat{\theta}_i = \mu + u_i + \epsilon_i$

Where:

• $u_i \sim \mathcal{N}(0, \tau^2)$ is the study-specific deviation from the grand mean.
• $\epsilon_i \sim \mathcal{N}(0, v_i)$ is the within-study sampling error.

Random-Effects Weight (DerSimonian-Laird):

$w_i^{RE} = \frac{1}{v_i + \hat{\tau}^2}$

The weights now include both the within-study variance $v_i$ and the estimated between-study variance $\hat{\tau}^2$. This shrinks the weight differences between large and small studies compared to the fixed-effect model.

Pooled Effect (Random-Effects):

$\hat{\mu}_{RE} = \frac{\sum_{i=1}^k w_i^{RE} \hat{\theta}_i}{\sum_{i=1}^k w_i^{RE}}$

When to use:

• When studies differ in any way (population, intervention version, outcome measurement, study design) — which is almost always.
• When the goal is to generalise the findings beyond the specific set of studies.
• When there is evidence of heterogeneity ($\tau^2 > 0$).

6.3 Methods for Estimating $\tau^2$

Several estimators for the between-study variance $\tau^2$ exist:

💡 The DataStatPro application implements the DerSimonian-Laird and REML estimators. REML is generally recommended, especially when the number of studies $k \geq 5$.

6.4 The Prediction Interval

In the random-effects model, a prediction interval captures the expected range of the true effect in a new (future) study, accounting for between-study heterogeneity:

$\hat{\mu}_{RE} \pm t_{\alpha/2, k-2} \times \sqrt{\hat{\tau}^2 + \text{Var}(\hat{\mu}_{RE})}$

Where $t_{\alpha/2, k-2}$ is the critical value from the $t$-distribution with $k-2$ degrees of freedom.

The prediction interval is wider than the confidence interval and reflects the true variability of effects across settings. It is arguably more informative than the confidence interval for clinical or practical decision-making.

💡 If the 95% prediction interval crosses zero (for a mean difference) or 1 (for an OR/RR), this indicates that in some settings, the true effect may be negligible or even reversed — even if the pooled effect is statistically significant.

6.5 Comparison: Fixed-Effect vs. Random-Effects

7. Assumptions of Meta-Analysis

7.1 Independence of Studies

Each study in the meta-analysis must represent an independent sample. Including multiple effect sizes from the same participants (without accounting for dependence) inflates precision.

⚠️ If a study reports multiple relevant outcomes or time points, use multilevel meta-analysis or select one primary outcome to avoid violating independence.

7.2 Consistent Effect Size Metric

All included studies must report (or allow computation of) the same effect size metric (e.g., all report Hedges' $g$, not a mix of $g$ and $r$ without conversion).

7.3 Unbiased Study Results (No Selective Reporting)

Meta-analysis assumes that the available study results are representative of all studies conducted. Systematic publication bias or outcome reporting bias violates this assumption and can lead to inflated pooled effects (see Section 10).

7.4 Accurate Extraction of Study Data

Effect sizes and their variances must be correctly extracted or computed from primary studies. Errors in data extraction propagate directly into the pooled estimate.

7.5 Sufficient Overlap in Study Characteristics (Conceptual Homogeneity)

While statistical heterogeneity is expected and modelled in the random-effects model, studies should be conceptually similar enough that combining them is meaningful. Pooling studies of entirely different populations or interventions under one estimate is referred to as the "apples and oranges" problem.

7.6 Normality of Effect Size Distribution

The statistical methods assume that the distribution of effect sizes (after any required transformation) is approximately normal. This is generally satisfied when within-study sample sizes are reasonable. For very small within-study samples, transformations (log, logit, Fisher's Z) are used to improve normality.

8. Heterogeneity

Heterogeneity refers to variability in the true effects across studies, beyond what would be expected from sampling error alone. Assessing and understanding heterogeneity is central to any meta-analysis.

8.1 Cochran's Q Test

Cochran's Q tests the null hypothesis $H_0: \theta_1 = \theta_2 = \dots = \theta_k$ (all true effects are identical):

$Q = \sum_{i=1}^k w_i^{FE}(\hat{\theta}_i - \hat{\theta}_{FE})^2$

Under $H_0$, $Q$ follows a chi-squared distribution with $k-1$ degrees of freedom:

$Q \sim \chi^2_{k-1}$

A statistically significant $Q$ ($p < 0.05$) indicates the presence of heterogeneity. However, the Q test has low statistical power (especially with few studies) and high power (detecting trivial heterogeneity) with many studies, so it should not be used in isolation.

8.2 $I^2$ Statistic

$I^2$ quantifies the proportion of total variability in effect sizes that is due to between-study heterogeneity (as opposed to sampling error):

$I^2 = \max\left(0, \frac{Q - (k-1)}{Q}\right) \times 100\%$

$I^2$ ranges from 0% to 100%:

⚠️ $I^2$ is a relative measure and should be interpreted alongside $\tau^2$ and the prediction interval. A large $I^2$ with a small $\tau^2$ in absolute terms may still imply a practically negligible spread of true effects.

8.3 $\tau^2$ (Between-Study Variance)

$\tau^2$ is the estimated variance of the distribution of true effects in the random-effects model. Unlike $I^2$, it is on the same scale as the squared effect size and thus conveys the absolute magnitude of heterogeneity.

The DerSimonian-Laird estimator of $\tau^2$ is:

$\hat{\tau}^2_{DL} = \max\left(0, \frac{Q - (k-1)}{C}\right)$

Where:

$C = \sum_{i=1}^k w_i^{FE} - \frac{\sum_{i=1}^k (w_i^{FE})^2}{\sum_{i=1}^k w_i^{FE}}$

$\tau$ (tau, the square root of $\tau^2$) is the standard deviation of the distribution of true effects and is reported in the same units as the effect size.

8.4 $H^2$ Statistic

$H^2 = \frac{Q}{k-1}$

$H^2 = 1$ means no heterogeneity (all variability is sampling error). $H^2 > 1$ indicates excess heterogeneity. It is related to $I^2$ by: $I^2 = (H^2 - 1)/H^2$.

8.5 Confidence Intervals for $\tau^2$ and $I^2$

$\tau^2$ and $I^2$ are estimates with their own uncertainty. Confidence intervals for these quantities (e.g., using the Q-profile method or the Biggerstaff-Jackson method) should always be reported alongside the point estimates, particularly with few studies.

8.6 Interpreting Heterogeneity: A Framework

When heterogeneity is detected, the analyst should:

1. Report the Q statistic, p-value, $I^2$, $\tau^2$, and its 95% CI.
2. Report the prediction interval to show the plausible range of true effects.
3. Explore sources of heterogeneity via subgroup analysis or meta-regression (Section 11).
4. Consider whether pooling is still appropriate or whether separate analyses by subgroup are needed.

9. Forest Plots

The forest plot is the canonical visual summary of a meta-analysis. It displays the effect size and confidence interval from each individual study, along with the pooled estimate, in a single graphic.

9.1 Anatomy of a Forest Plot

A standard forest plot contains the following elements:

9.2 Reading a Forest Plot

• Studies where the CI does not cross the null line have a statistically significant individual effect.
• Studies where the CI crosses the null line do not individually show significance.
• A narrow CI indicates a precise (usually large-sample) study.
• A wide CI indicates an imprecise (usually small-sample) study.
• If study squares are spread widely around the pooled diamond, heterogeneity is high.
• If the pooled diamond does not cross the null line, the overall effect is statistically significant.

9.3 Separate Forest Plots for Fixed-Effect and Random-Effects

The DataStatPro application generates two forest plots — one for the fixed-effect model and one for the random-effects model. Comparing them is instructive:

• If results are similar, heterogeneity is low.
• If the random-effects estimate is notably different (and the CI wider), substantial between-study heterogeneity exists.

10. Publication Bias

Publication bias is the tendency for studies with statistically significant or large effects to be published more readily than studies with non-significant or small effects. If the meta-analyst only has access to published studies, the pooled effect will be overestimated.

10.1 The Funnel Plot

The funnel plot is the primary graphical tool for assessing publication bias. It plots each study's effect size (x-axis) against a measure of its precision (y-axis, typically the standard error — note: inverted so that more precise studies appear at the top).

Expected appearance under no publication bias:

• Studies scatter symmetrically around the pooled effect estimate in a funnel (inverted-V) shape.
• Large, precise studies (top of plot) cluster tightly near the pooled estimate.
• Small, imprecise studies (bottom) scatter more widely.

Signs of publication bias:

• Asymmetry in the funnel plot: a gap at the bottom-left (missing small studies with small or negative effects).
• The funnel is "skewed" rather than symmetric.

⚠️ Funnel plot asymmetry can result from publication bias but also from other causes: small-study effects (small studies genuinely showing larger effects), heterogeneity, artefacts of particular effect size measures, or chance. Interpret with caution and use formal tests.

10.2 Egger's Test

Egger's test is a formal statistical test for funnel plot asymmetry based on a weighted linear regression of the standardised effect ($\hat{\theta}_i / SE_i$) on precision ($1/SE_i$):

$\frac{\hat{\theta}_i}{SE_i} = a + b \times \frac{1}{SE_i} + \epsilon_i$

• The intercept $a$ captures asymmetry: if $a \neq 0$, the funnel plot is asymmetric.
• $H_0: a = 0$ (no asymmetry).
• A significant Egger's test ($p < 0.05$, often $p < 0.10$ given low power) suggests asymmetry.

10.3 Begg's Test (Rank Correlation Test)

Begg's test examines whether there is a rank correlation between the standardised effect sizes and their variances. It uses Kendall's $\tau$ to test for correlation:

$H_0: \text{rank}(\hat{\theta}_i) \text{ is uncorrelated with } \text{rank}(v_i)$

A significant $\tau$ suggests asymmetry. Begg's test generally has lower power than Egger's test.

10.4 Trim and Fill Method

The trim and fill method (Duval & Tweedie) is a non-parametric procedure that:

1. Trims the asymmetric studies (assumed to be the more extreme studies on one side).
2. Re-estimates the centre of the funnel.
3. Fills in the missing (unpublished) mirror-image studies.
4. Produces an adjusted pooled estimate that accounts for publication bias.

The adjusted estimate represents what the pooled effect would be if the funnel plot were symmetric.

⚠️ The trim and fill method assumes that asymmetry is caused solely by publication bias. If heterogeneity or other factors cause asymmetry, the adjusted estimate may be misleading. It should be treated as a sensitivity analysis, not the primary result.

10.5 Fail-Safe N (Rosenthal's Method)

Fail-safe N ($N_{fs}$) estimates the number of unpublished null-result studies that would be needed to reduce the pooled effect to non-significance:

$N_{fs} = \left(\frac{\sum z_i}{z_{\alpha}}\right)^2 - k$

Where $\sum z_i$ is the sum of the $z$-statistics from all included studies, $z_{\alpha} = 1.645$ (one-tailed at 5%), and $k$ is the number of included studies.

A large $N_{fs}$ relative to the number of included studies suggests the results are robust to publication bias.

⚠️ Fail-safe N has been criticised as it does not consider the quality or magnitude of missing studies — only their number. It should be supplemented with other methods.

11. Moderator Analysis: Subgroup Analysis and Meta-Regression

When heterogeneity is detected, the natural next step is to explain it by identifying study-level variables (moderators) that systematically relate to the effect size.

11.1 Subgroup Analysis

Subgroup analysis divides studies into groups based on a categorical moderator (e.g., type of intervention, country income level, participant age group) and estimates a separate pooled effect for each subgroup.

Procedure:

1. Define subgroups based on a theoretically motivated moderator.
2. Run separate meta-analyses within each subgroup.
3. Test for between-subgroup heterogeneity using a Q-test for subgroup differences:

$Q_B = \sum_{g=1}^G w_g^* \left(\hat{\theta}_g - \hat{\theta}_{\text{all}}\right)^2$

Where $\hat{\theta}_g$ is the pooled effect for subgroup $g$ and $w_g^* = 1/\text{Var}(\hat{\theta}_g)$.

Under $H_0$ (no subgroup differences), $Q_B \sim \chi^2_{G-1}$ where $G$ is the number of subgroups.

⚠️ Subgroup analyses are subject to multiple testing inflation and should be pre-specified (not data-driven post hoc). Treat unexpected subgroup findings as exploratory and hypothesis-generating, not confirmatory.

11.2 Meta-Regression

Meta-regression models the effect size as a function of one or more continuous or categorical study-level covariates (moderators):

$\hat{\theta}_i = \beta_0 + \beta_1 Z_{1i} + \beta_2 Z_{2i} + \dots + \beta_p Z_{pi} + u_i + \epsilon_i$

Where:

• $Z_{ji}$ are study-level moderator variables (e.g., mean age, publication year, dose, methodological quality score).
• $\beta_j$ is the regression coefficient for moderator $j$: the expected change in the true effect per one-unit increase in $Z_j$.
• $u_i \sim \mathcal{N}(0, \tau^2_{\text{residual}})$ is the residual between-study heterogeneity after accounting for moderators.
• $\epsilon_i \sim \mathcal{N}(0, v_i)$ is the within-study sampling error.

Weighted Least Squares Estimation: Meta-regression is estimated using weighted least squares with weights $w_i = 1/(v_i + \hat{\tau}^2_{\text{residual}})$.

Testing a Moderator: The significance of moderator $Z_j$ is tested using the Wald statistic:

$z_j = \frac{\hat{\beta}_j}{SE(\hat{\beta}_j)}$

A significant $z_j$ indicates that $Z_j$ explains a portion of the heterogeneity.

$R^2$ Analogue (Proportion of Heterogeneity Explained):

$R^2_{\text{meta}} = \frac{\hat{\tau}^2_{\text{null}} - \hat{\tau}^2_{\text{model}}}{\hat{\tau}^2_{\text{null}}}$

This estimates the proportion of between-study variance ($\tau^2$) explained by the moderator(s).

⚠️ Meta-regression requires a sufficient number of studies (a common guideline is at least 10 studies per moderator). With few studies, the regression will be underpowered and potentially spurious.

12. Sensitivity Analysis

Sensitivity analysis examines the robustness of the pooled results to the methodological choices made in the meta-analysis.

12.1 Leave-One-Out Analysis

The leave-one-out (also called "one-study-removed") analysis re-runs the meta-analysis $k$ times, each time excluding one study. If removing any single study dramatically changes the pooled estimate, that study is influential and warrants investigation.

Interpretation:

• If the pooled estimate is stable across all leave-one-out analyses → results are robust.
• If removing one study markedly shifts the pooled estimate → that study may be an outlier or have excessive influence.

12.2 Influence Statistics

For each study $i$, the following influence diagnostics can be computed:

💡 Influential studies should not be automatically excluded — they should be examined for data quality, unique population characteristics, or methodological anomalies. Exclusion decisions should be pre-specified or transparently justified.

12.3 Sensitivity to Model Choice

It is good practice to compare the pooled effect and heterogeneity estimates under:

• Fixed-effect vs. random-effects models.
• Different $\tau^2$ estimators (e.g., DL vs. REML).
• With and without continuity corrections (for binary data).
• With and without influential studies.

Consistent results across these analyses strengthen confidence in the conclusions.

13. Using the Meta-Analysis Component

The Meta-Analysis component in the DataStatPro application provides a full end-to-end workflow for performing meta-analysis on your datasets.

Step-by-Step Guide

Step 1 — Select Dataset Choose the dataset you want to analyse from the "Dataset" dropdown. The dataset should contain one row per study, with columns for the relevant study-level statistics.

Step 2 — Select Analysis Type Choose the type of meta-analysis:

• Continuous Outcomes, Two Groups (MD, Cohen's $d$, Hedges' $g$, Glass's $\Delta$)
• Binary Outcomes, Two Groups (OR, RR, RD)
• Correlations (Pearson's $r$ / Fisher's $z_r$)
• Single Proportions
• Single Means
• Incidence Rates
• Pre-Computed Effect Sizes (if effect sizes and their variances are already available in the dataset)

Step 3 — Select Input Variables Depending on the analysis type, map the relevant dataset columns:

Step 4 — Select Effect Size Measure For continuous two-group outcomes, select the desired effect size:

• Raw Mean Difference (MD)
• Cohen's $d$
• Hedges' $g$ (recommended)
• Glass's $\Delta$

For binary outcomes, select OR, RR, or RD.

Step 5 — Select Statistical Model Choose between:

• Fixed-Effect Model
• Random-Effects Model (recommended in most cases)

If Random-Effects is selected, choose the $\tau^2$ estimation method (DerSimonian-Laird, REML, etc.).

Step 6 — Select Confidence Level Choose the confidence level for confidence intervals (default: 95%).

Step 7 — Select Display Options Choose which outputs to display:

• ✅ Forest Plot
• ✅ Funnel Plot
• ✅ Heterogeneity Statistics ($Q$, $I^2$, $\tau^2$)
• ✅ Pooled Effect and CI
• ✅ Prediction Interval (random-effects only)
• ✅ Publication Bias Tests (Egger's, Begg's, Trim and Fill)
• ✅ Study-Level Effect Size Table
• ✅ Leave-One-Out Sensitivity Analysis

Step 8 — Configure Moderators (Optional) If performing subgroup analysis, specify the column containing the categorical moderator variable. If performing meta-regression, specify the continuous or categorical covariate columns.

Step 9 — Run the Analysis Click "Run Meta-Analysis". The application will:

1. Compute per-study effect sizes and variances (or use pre-computed values).
2. Apply any required transformations (log, logit, Fisher's Z).
3. Estimate the fixed-effect and/or random-effects pooled estimate.
4. Estimate $\tau^2$, $I^2$, $Q$, and the prediction interval.
5. Generate the forest plot and funnel plot.
6. Run publication bias tests.
7. Run leave-one-out sensitivity analysis.
8. Run subgroup analysis or meta-regression if specified.

14. Computational and Formula Details

14.1 Full Step-by-Step Calculation Workflow

For any meta-analysis, the computation proceeds as follows:

Step A: Compute per-study effect sizes $\hat{\theta}_i$ and variances $v_i$ Using the appropriate formula from Section 4 or 5.

Step B: Apply variance-stabilising transformations if needed (e.g., $\ln(OR)$, Fisher's $z_r$, logit($p$))

Step C: Compute fixed-effect weights

$w_i^{FE} = \frac{1}{v_i}$

Step D: Compute the fixed-effect pooled estimate

$\hat{\theta}_{FE} = \frac{\sum w_i^{FE} \hat{\theta}_i}{\sum w_i^{FE}}, \quad SE(\hat{\theta}_{FE}) = \frac{1}{\sqrt{\sum w_i^{FE}}}$

Step E: Compute Cochran's Q and estimate $\tau^2$

$Q = \sum w_i^{FE}(\hat{\theta}_i - \hat{\theta}_{FE})^2$

$\hat{\tau}^2_{DL} = \max\left(0, \frac{Q-(k-1)}{C}\right), \quad C = \sum w_i^{FE} - \frac{\sum(w_i^{FE})^2}{\sum w_i^{FE}}$

Step F: Compute random-effects weights

$w_i^{RE} = \frac{1}{v_i + \hat{\tau}^2}$

Step G: Compute the random-effects pooled estimate

$\hat{\mu}_{RE} = \frac{\sum w_i^{RE} \hat{\theta}_i}{\sum w_i^{RE}}, \quad SE(\hat{\mu}_{RE}) = \frac{1}{\sqrt{\sum w_i^{RE}}}$

Step H: Compute CIs, Z-test, and prediction interval

$\text{CI}_{95\%}: \hat{\mu}_{RE} \pm 1.96 \times SE(\hat{\mu}_{RE})$

$Z = \frac{\hat{\mu}_{RE}}{SE(\hat{\mu}_{RE})}, \quad p = 2(1-\Phi(|Z|))$

$\text{PI}_{95\%}: \hat{\mu}_{RE} \pm t_{0.025, k-2} \sqrt{\hat{\tau}^2 + SE^2(\hat{\mu}_{RE})}$

Step I: Back-transform to original scale if needed (e.g., $e^{\ln(OR)_{\text{pooled}}}$ for OR; $\tanh(z_{r,\text{pooled}})$ for $r$)

Step J: Compute $I^2$ and $H^2$

$I^2 = \max\left(0, \frac{Q-(k-1)}{Q}\right) \times 100\%, \quad H^2 = \frac{Q}{k-1}$

14.2 Handling Zero Events (Continuity Correction)

When $a = 0$, $b = 0$, $c = 0$, or $d = 0$ in a $2 \times 2$ table, log-scale measures (OR, RR) are undefined. The Haldane-Anscombe correction adds 0.5 to all four cells:

$a' = a + 0.5, \quad b' = b + 0.5, \quad c' = c + 0.5, \quad d' = d + 0.5$

This is applied only to studies with zero cells. Double-zero studies (where both $a = 0$ and $c = 0$, meaning no events in either group) are typically excluded from OR/RR meta-analyses as they carry no information about the relative effect.

14.3 Computing Effect Sizes from Alternative Inputs

Not all studies report means and SDs directly. The following conversions are commonly needed:

From SE to SD: $SD = SE \times \sqrt{n}$

From 95% CI to SE: $SE = \frac{\text{Upper CI} - \text{Lower CI}}{2 \times 1.96}$

From median and IQR (Wan et al. method, for non-normal distributions): $\hat{\bar{X}} \approx \frac{Q_1 + \text{Median} + Q_3}{3}$ $\hat{SD} \approx \frac{Q_3 - Q_1}{1.35}$

From t-statistic (two groups): $d = t \sqrt{\frac{n_T + n_C}{n_T \times n_C}}$

15. Worked Examples

Example 1: Meta-Analysis of Standardised Mean Differences (Hedges' g)

Research Question: Does mindfulness-based stress reduction (MBSR) reduce anxiety (standardised mean difference) compared to a control condition?

Included Studies (hypothetical):

Step 1: Compute $SD_{\text{pooled}}$, $d$, and $J$ (for Hedges' $g$) for each study.

For Adams (2018):

$SD_{\text{pooled}} = \sqrt{\frac{(45-1)(4.2)^2 + (44-1)(4.8)^2}{45+44-2}} = \sqrt{\frac{44(17.64) + 43(23.04)}{87}} = \sqrt{\frac{776.16 + 990.72}{87}} = \sqrt{\frac{1766.88}{87}} \approx \sqrt{20.31} \approx 4.507$

$d = \frac{12.3 - 15.7}{4.507} = \frac{-3.4}{4.507} \approx -0.755$

$J = 1 - \frac{3}{4(45+44-2)-1} = 1 - \frac{3}{347} \approx 1 - 0.00865 \approx 0.991$

$g = 0.991 \times (-0.755) \approx -0.748$

$v_g = J^2 \times \left[\frac{n_T+n_C}{n_T \times n_C} + \frac{d^2}{2(n_T+n_C-2)}\right] = 0.991^2 \times \left[\frac{89}{45 \times 44} + \frac{0.570}{174}\right] = 0.982 \times [0.04499 + 0.00328] = 0.982 \times 0.04827 \approx 0.04740$

(Calculations for Brown, Chen, Davis, and Evans proceed identically.)

Summary of computed effect sizes (illustrative):

Step 2: Fixed-Effect Pooled Estimate

$\hat{\theta}_{FE} = \frac{21.10(-0.748) + 13.85(-0.876) + 56.18(-0.604) + 9.92(-0.913) + 35.34(-0.681)}{136.39} = \frac{-15.78 - 12.13 - 33.93 - 9.05 - 24.07}{136.39} = \frac{-94.96}{136.39} \approx -0.696$

$SE_{FE} = \frac{1}{\sqrt{136.39}} \approx 0.0856, \quad 95\%\text{ CI: } [-0.864, -0.528]$

Step 3: Cochran's Q and $\tau^2$

$Q = 21.10(-0.748-(-0.696))^2 + 13.85(-0.876-(-0.696))^2 + 56.18(-0.604-(-0.696))^2 + 9.92(-0.913-(-0.696))^2 + 35.34(-0.681-(-0.696))^2$

$= 21.10(0.002704) + 13.85(0.032400) + 56.18(0.008464) + 9.92(0.047089) + 35.34(0.000225)$

$\approx 0.057 + 0.449 + 0.476 + 0.467 + 0.008 = 1.457$

$df = k - 1 = 4$, $p \approx 0.835$ → No significant heterogeneity ($p > 0.05$).

$I^2 = \max(0, (1.457-4)/1.457) \times 100\% = \max(0, -1.744) = 0\%$

$\hat{\tau}^2_{DL} = \max(0, (1.457-4)/C) = 0 \quad \text{(since } Q < k-1\text{)}$

Step 4: Random-Effects Pooled Estimate

Since $\hat{\tau}^2 = 0$, the random-effects model is identical to the fixed-effect model here:

$\hat{\mu}_{RE} \approx -0.696, \quad 95\%\text{ CI: } [-0.864, -0.528]$

$Z = \frac{-0.696}{0.0856} \approx -8.13, \quad p < 0.001$

Conclusion: The pooled Hedges' $g \approx -0.696$ (95% CI: $[-0.864, -0.528]$), indicating a medium-to-large reduction in anxiety following MBSR compared to control. The effect is highly statistically significant ($p < 0.001$). No significant heterogeneity was detected ($I^2 = 0\%$, $Q(4) = 1.46$, $p = 0.83$).

Example 2: Meta-Analysis of Odds Ratios (Binary Outcome)

Research Question: Does low-dose aspirin reduce the risk of myocardial infarction (MI)?

Input Data (2×2 tables, hypothetical):

Step 1: Compute $\ln(OR)$ and $v_{\ln(OR)}$ for each study.

Study 1:

$\ln(OR) = \ln\left(\frac{28 \times 955}{972 \times 45}\right) = \ln\left(\frac{26740}{43740}\right) = \ln(0.6113) \approx -0.4912$

$v_{\ln(OR)} = \frac{1}{28} + \frac{1}{972} + \frac{1}{45} + \frac{1}{955} = 0.03571 + 0.00103 + 0.02222 + 0.00105 = 0.06001$

$w_1^{FE} = 1/0.06001 \approx 16.66$

(Calculations for Studies 2–4 proceed identically.)

Summary (illustrative):

Step 2: Pooled $\ln(OR)$

$\widehat{\ln(OR)}_{FE} = \frac{16.66(-0.491)+8.67(-0.612)+33.11(-0.425)+5.03(-0.634)}{63.47} = \frac{-8.180-5.306-14.072-3.189}{63.47} = \frac{-30.747}{63.47} \approx -0.4845$

$OR_{\text{pooled}} = e^{-0.4845} \approx 0.616$

$SE = 1/\sqrt{63.47} \approx 0.1255, \quad 95\%\text{ CI for }\ln(OR): [-0.731, -0.238]$

$\Rightarrow 95\%\text{ CI for OR: } \left[e^{-0.731}, e^{-0.238}\right] = [0.481, 0.788]$

Step 3: Test and Heterogeneity

$Z = \frac{-0.4845}{0.1255} = -3.86, \quad p < 0.001$

$Q$ (not calculated in detail here) is non-significant → low heterogeneity.

Conclusion: The pooled OR $\approx 0.616$ (95% CI: $[0.481, 0.788]$), indicating that aspirin reduces the odds of MI by approximately 38% compared to control. The effect is highly statistically significant ($p < 0.001$).

Example 3: Meta-Analysis of Proportions (Single Group)

Research Question: What is the pooled prevalence of depression in university students?

Input Data:

Step 1: Logit transform each proportion

Study A:

$\text{logit}(0.170) = \ln(0.170/0.830) = \ln(0.2048) \approx -1.585$

$v_{\text{logit}} = \frac{1}{500 \times 0.170 \times 0.830} = \frac{1}{70.55} \approx 0.01418, \quad w = 70.55$

(Steps for B–E proceed identically.)

Step 2: Pool on logit scale → back-transform

$\widehat{\text{logit}(p)} = \frac{\sum w_i \cdot \text{logit}(p_i)}{\sum w_i} \approx -1.626$

$\hat{p} = \frac{e^{-1.626}}{1 + e^{-1.626}} \approx \frac{0.1968}{1.1968} \approx 0.164$

Conclusion: The pooled prevalence of depression is approximately 16.4% (95% CI to be computed from the SE of the logit-scale pooled estimate, back-transformed).

16. Common Mistakes and How to Avoid Them

Mistake 1: Combining Apples and Oranges

Problem: Pooling studies that are conceptually too different (different populations, interventions, or outcomes) into a single meta-analysis.
Solution: Apply strict inclusion/exclusion criteria. Consider whether the research question is specific enough. Use subgroup analysis for conceptually different study types rather than pooling indiscriminately.

Mistake 2: Ignoring Heterogeneity

Problem: Reporting only the pooled effect and ignoring significant heterogeneity ($I^2 > 50\%$), implying a single universal effect when the true effects vary widely.
Solution: Always report $Q$, $I^2$, $\tau^2$, and the prediction interval. Explore heterogeneity via subgroup analysis and meta-regression. Be transparent about what the pooled estimate represents when heterogeneity is high.

Mistake 3: Using Fixed-Effect When Random-Effects Is Appropriate

Problem: Applying the fixed-effect model (which assumes all studies share a single true effect) to heterogeneous studies, producing an overconfident (too-narrow) confidence interval.
Solution: Use the random-effects model as the default unless there is a compelling theoretical reason all studies share exactly the same true effect. Compare the two models as a sensitivity check.

Mistake 4: Misinterpreting the Confidence Interval vs. Prediction Interval

Problem: Interpreting the narrow 95% confidence interval of the pooled effect as the range of effects across all possible settings, ignoring that the true effect varies across contexts.
Solution: Report and interpret the prediction interval. Communicate that the CI describes uncertainty about the average effect, while the PI describes the range of true effects across settings.

Mistake 5: Data Extraction Errors

Problem: Incorrectly recording means, SDs, or cell counts from primary studies — a common and serious source of error in meta-analyses.
Solution: Use dual independent extraction by two reviewers, with discrepancies resolved by consensus. Double-check all calculated effect sizes against reported statistics where possible.

Mistake 6: Misinterpreting the Odds Ratio as a Risk Ratio

Problem: Describing an OR of 2.0 as "the treatment doubles the probability of the event" when in fact it doubles the odds, which only approximates doubling the probability when events are rare.
Solution: Always specify clearly whether the effect measure is an OR or RR. If the OR is used with common outcomes (>10%), note its potential to overestimate the RR and consider reporting both.

Mistake 7: Over-Reliance on Statistical Significance of the Pooled Effect

Problem: Concluding that an intervention is "effective" solely because $p < 0.05$ for the pooled estimate, without considering the magnitude and clinical relevance of the effect.
Solution: Interpret the pooled effect size in context. A statistically significant but tiny effect (e.g., Hedges' $g = 0.05$) may have no practical importance. Always report effect sizes with confidence intervals and discuss clinical/practical significance.

Mistake 8: Ignoring Publication Bias

Problem: Assuming that all relevant studies are captured and that the literature is a representative sample of all research conducted.
Solution: Conduct a comprehensive search (including grey literature, trial registries, and non-English publications). Assess publication bias formally using funnel plot inspection, Egger's test, and Trim and Fill. Report the adjusted estimate from Trim and Fill as a sensitivity analysis.

Mistake 9: Post Hoc Subgroup Analysis Without Correction

Problem: Conducting many unplanned subgroup analyses and reporting only those that are statistically significant, leading to false positives.
Solution: Pre-specify all planned subgroup and moderator analyses before running the meta-analysis. Treat unplanned analyses as exploratory. Apply appropriate corrections for multiple testing if many comparisons are made.

Mistake 10: Confusing Within-Study and Between-Study Variance

Problem: Using $v_i$ (within-study variance) as the measure of heterogeneity, or conflating $SE(\hat{\mu})$ with $\tau$.
Solution: Clearly distinguish $v_i$ (uncertainty within each study, reduced with larger $n$) from $\tau^2$ (variance of true effects across studies, a property of the study population). The SE of the pooled estimate reflects both.

17. Troubleshooting