What if the same patients are measured again and again?

Start here

Why are measurements on the same patient not independent?

A patient with high pain at week 0 usually still has higher pain than most other patients at week 8. Each patient carries their own level through every visit, so their values are correlated.

Two errors follow from treating repeated values as separate patients. When groups of patients are compared, the analysis believes it has more independent information than it does, so its standard errors are too small and its p-values too small. When the same patients are compared across time, it leaves the variation between patients in the error term, so it can miss a real change. The figure shows the second error, and the interactive element below shows the first.

The amount of duplication is measured by the intraclass correlation (ICC): the share of the total variance that lies between patients. With m visits per patient, the variance of a group mean is inflated by the design effect, the same formula used for cluster samples.

design effect = 1 + (m − 1) × ICC · effective number of patients = number of values ÷ design effect

t-tests at each visit do not solve the problem. Matthews, Altman, Campbell and Royston (BMJ, 1990) described the common method that compares groups at a series of time points, possibly with t-tests, as flawed on two counts: it may not answer the clinical question, and it may be statistically invalid. Each extra test is another chance of a false positive, and the tests are correlated with each other because they use the same patients.

Interactive

Change how much patients differ and watch the standard error of the treatment difference

The unpaired t-test on all 64 values assumes 64 independent patients. The mixed model knows which values come from the same patient.

The method

How does repeated measures ANOVA use each patient as their own control?

It splits the variation into three parts: differences between patients, differences between visits, and what is left over. The differences between patients are removed before the visits are compared.

F = mean square for time ÷ mean square for error · df = (k − 1) and (n − 1)(k − 1)

k

number of visits (4)

n

number of patients (10)

mean square

sum of squares ÷ its degrees of freedom

In the teaching data, F = 19.40 with 3 and 27 df (p < 0.001). The critical F at 5% is 2.96. Mean pain was 63.7 at week 0, 52.3 at week 2, 46.4 at week 4 and 45.2 at week 8.

With two or more groups of patients, for example physiotherapy against exercise at home, the design becomes a mixed (split-plot) ANOVA: group is a between-patient factor and time is a within-patient factor. The question that matters in a trial is usually the group × time interaction: does the change over time differ between the groups?

Assumptions: a measurement outcome, roughly normal residuals, every patient measured at every visit, and sphericity. A patient with one missing visit is dropped from a repeated measures ANOVA unless the value is imputed.

Assumption

What is sphericity, and what do Mauchly, Greenhouse-Geisser and Huynh-Feldt do?

Take every patient's change between two visits. Sphericity means that the variance of these changes is the same for every pair of visits. Then the F test gives correct p-values.

1. Mauchly's test (1940) tests the null hypothesis that sphericity holds. Teaching data: W = 0.154, χ²(5) = 14.42, p = 0.01, so sphericity is rejected. IBM's documentation for SPSS notes that the test has little power in small samples and may be significant in large samples when the departure has little effect.
2. Epsilon (ε) measures how far the data are from sphericity. It is 1 when sphericity holds, and its lowest possible value is 1 ÷ (k − 1), here 0.33. Both degrees of freedom of F are multiplied by ε, and F is compared with the new degrees of freedom.
3. Greenhouse-Geisser (1959) described a degrees-of-freedom adjustment that gives conservative F tests. Teaching data: ε = 0.52, F(1.56, 14.04) = 19.40, p = 0.0002.
4. Huynh-Feldt (1976) proposed a second estimate of ε because the first is seriously biased when ε is not low. Teaching data: ε = 0.61, F(1.83, 16.45) = 19.40, p = 0.00006.
5. Lower bound: ε = 1 ÷ (k − 1) = 0.33, F(1, 9), p = 0.0017. It is the most conservative choice.

Sphericity is automatically true when there are only two visits, because there is only one pair of visits. A linear mixed model avoids the assumption by modelling the correlation between visits directly.

Interactive

Change how the visits are correlated and watch epsilon and the corrected p-value

Sphericity is a statement about the covariance matrix of the visits. Equal variances with equal correlations always satisfy it. Unequal variances, or correlations that fall with distance, usually do not.

Non-parametric test

What if the scores are skewed or ordinal? The Friedman test

The Friedman test ranks each patient's own values from lowest to highest, then asks whether some visits collect higher ranks than others. It is the rank version of a one-way repeated measures ANOVA.

1. Within each patient, rank the 4 visits (1 = lowest pain). Tied values share the average rank. Patient 5 scored 72 at weeks 2 and 4, so both get rank 1.5.
2. Add the ranks for each visit over the 10 patients: 39, 25.5, 16.5 and 19. If the visits did not differ, each sum would be close to 10 × 2.5 = 25.
3. Calculate the statistic (without the correction for ties):

χ²_F = [12 ÷ (n k (k + 1))] × ΣR² − 3n(k + 1) = 0.06 × 2804.5 − 150 = 18.27

4. The one tied pair gives a correction factor of 0.99, and software reports 18.27 ÷ 0.99 = 18.45.
5. Compare with χ² on k − 1 = 3 df (critical value 7.81): p = 0.0004. Kendall's W = 18.45 ÷ (10 × 3) = 0.62.

The Friedman test needs every patient to have every visit, and it handles only one within-patient factor. If it is significant, compare pairs of visits with Wilcoxon signed-rank tests and a correction for multiple testing.

Crossover trials

How does a crossover trial make each patient their own control?

Each patient receives both treatments, one in each period, in a randomly allocated order. The comparison is within patients, so the variation between patients does not enter the treatment comparison.

ICH E9 says the design reduces the number of subjects needed for a given power, sometimes markedly. It also sets the conditions: the disease should be chronic and stable, the effects of the medication should develop fully within the treatment period, and the washout should be long enough for the drug effect to reverse completely. Crossover does not suit curable conditions, outcomes such as death, or trials in which many patients are expected to drop out.

Worked example: a 2 × 2 crossover

Made-up teaching data. Systolic BP (mmHg) at the end of each 4-week period in 12 patients with hypertension.

1. In sequence AB, d = (A − B) + (period 1 − period 2). In sequence BA, d = (B − A) + (period 1 − period 2). Subtracting the two cancels the period effect.
2. Mean d: AB −4.83 (SD 3.66), BA 9.17 (SD 4.67). Pooled variance 17.57.
3. Treatment effect A − B = (−4.83 − 9.17) ÷ 2 = −7.0 mmHg. SE = √[17.57 × (1/6 + 1/6)] ÷ 2 = 1.21.
4. t = −7.0 ÷ 1.21 = −5.79 with 10 df, p = 0.0002. 95% CI −9.7 to −4.3 mmHg (critical t 2.228).
5. Period effect (period 2 − period 1) = −(−4.83 + 9.17) ÷ 2 = −2.17 mmHg, p = 0.10.
6. Carryover: compare each patient's total. AB mean 294.5, BA mean 290.2, difference 4.3 mmHg (95% CI −26.8 to 35.5), p = 0.76.

A − B = (mean d_AB − mean d_BA) ÷ 2 = −7.0 mmHg (95% CI −9.7 to −4.3)

A linear mixed model with treatment and period as fixed effects and a random intercept for each patient gives the same estimate and SE (−7.0, SE 1.21). The same 12 patients analysed with a paired t-test that ignores period also give −7.0 here, because both sequences have 6 patients, but the SE comes from 11 df and the period effect stays in the error. Analysing period 1 alone, as a parallel trial of 6 against 6 patients, gives −4.8 mmHg (95% CI −19.3 to 9.7), p = 0.48.

Why the carryover test cannot be relied on. Its CI above runs from −26.8 to 35.5 mmHg, because patient totals carry all the variation between patients. ICH E9 notes that in the 2 × 2 design carryover cannot be distinguished from the treatment × period interaction, and that the test for either lacks power. Freeman (1989) showed that the two-stage procedure (test for carryover, then use period 1 only if it is significant) gives actual significance levels higher than the nominal level, and concluded that it is too potentially misleading to be of practical use. Carryover is prevented by the design, with an adequate washout, not by a test.

Interactive

Add carryover and watch the treatment estimate and the carryover test

Carryover of drug A into period 2 biases the within-patient estimate by half the carryover. The test for carryover compares patient totals between the two sequences, so it has little power.

Linear mixed models

What do random intercepts and random slopes mean?

A linear mixed model has fixed effects, which are the questions you ask (treatment, visit, their interaction), and random effects, which describe how patients differ from each other.

Why a mixed model handles missing visits. It is fitted to every value that was recorded, one row per patient per visit. A patient seen at weeks 0 and 2 only still contributes those two values, and the random intercept links them to the rest of the patient's profile. The estimates are valid if the data are missing at random: whether a value is missing may depend on values already recorded (for example earlier pain), but not on the missing value itself after allowing for those. A repeated measures ANOVA keeps only patients with complete data, which is valid only if completers are a random subset of all patients.

The correlation between visits is chosen, not assumed away. A random intercept alone implies the same correlation between any two visits (compound symmetry). Other structures offered by software include first-order autoregressive, AR(1), where the correlation falls as visits get further apart, and unstructured, where every pair of visits has its own correlation.

Other advantages: visit times can differ between patients, time can be a continuous variable, baseline and other covariates can be added, and the output gives an estimate with a 95% CI for each effect. Laird and Ware (1982) described random-effects models that include both growth models and repeated-measures models as special cases, fitted to unbalanced data.

Interactive

Make patients with more pain leave early and compare completers with the mixed model

When dropout depends on earlier recorded values, the completers are no longer a random subset. The mixed model uses the earlier values of the patients who left.

GEE

Population-average or subject-specific: what is the difference between GEE and a mixed model?

Generalized estimating equations (GEE) model the average outcome in the population and correct the standard errors for correlation within patients. A mixed model describes the outcome for an individual patient with a given underlying level.

Zeger, Liang and Albert (1988) described the two approaches as subject-specific models, in which heterogeneity between subjects is modelled explicitly, and population-averaged models, in which the aggregate response for the population is the focus. For a continuous outcome analysed on its own scale, the two give the same fixed-effect estimates. For yes/no outcomes and counts they differ, as the figure shows.

• Working correlation: the analyst chooses a structure. SPSS offers independent, AR(1), exchangeable, M-dependent and unstructured.
• Standard errors: IBM's documentation states that the robust estimator provides a consistent estimate of the covariance even when the working correlation matrix is misspecified.
• Use GEE when the question is about the population, for example how much a water intervention changes the proportion of children with diarrhoea each month.
• Use a mixed model when the question is about an individual patient, or when patient-level variation is itself of interest.

Summary measures

Can one number per patient replace all the visits?

Often yes. Choose the summary that answers the clinical question, calculate it for each patient, then compare the summaries between groups with a simple test.

Matthews and colleagues (1990) proposed this two-stage method: first calculate a suitable summary for each subject, such as a rate of change or an area under a curve, then analyse those summaries with simple statistical techniques as though they were raw data. ICH E9 lists a summary measure such as area under the curve as a way to avoid multiplicity with repeated measures.

• Total or average exposure: AUC, or AUC ÷ duration (time-averaged value).
• Rate of change: each patient's regression slope over time (fall in eGFR per year).
• Peak or time to peak: highest drug concentration, time to lowest pain.
• Final value adjusted for baseline: see the ANCOVA chapter.

Choose the summary in the protocol, before seeing the data. A summary chosen after looking at the curves is exploratory.

Decide

Which method fits your repeated measurements?

Answer the questions in order.

Comparison

Three pairs residents confuse

Repeated measures ANOVA or linear mixed model?

Parallel group trial or crossover trial?

GEE or mixed model?

The formula

Worked example: repeated measures ANOVA step by step

Made-up teaching data: pain score in 10 patients at weeks 0, 2, 4 and 8. Computed with Python (pingouin 0.6.1, SciPy 1.15.3).

1. Grand mean = 51.9. Total SS = Σ(value − 51.9)² = 9775.6.
2. SS between patients = 4 × Σ(patient mean − 51.9)² = 6635.1, df 9.
3. SS time = 10 × Σ(visit mean − 51.9)² = 2145.4, df 3, mean square 715.1.
4. SS error = 9775.6 − 6635.1 − 2145.4 = 995.1, df 9 × 3 = 27, mean square 36.86.
5. F = 715.1 ÷ 36.86 = 19.40 on (3, 27) df, p < 0.001.
6. Mauchly W = 0.154, p = 0.01. Greenhouse-Geisser ε = 0.52: F(1.56, 14.04), p = 0.0002.

The same scores in a one-way ANOVA that ignores patients: within-groups SS = 6635.1 + 995.1 = 7630.2 on 36 df, mean square 212.0, F = 3.37, p = 0.029. The Friedman test on the same data: χ²_F = 18.45, 3 df, p = 0.0004.

In a real paper

Repeated measures in two published trials

Both articles: no retraction, correction or expression of concern found in PubMed, Europe PMC or scite (checked 15 September 2026). Europe PMC lists a commentary on ALCAT (J Neurol 2022).

Software

How to run it

Enter repeated measures in long format for mixed models and GEE (one row per patient per visit) and in wide format for SPSS GLM Repeated Measures (one row per patient, one column per visit).

SPSS

• Repeated measures ANOVA: Analyze > General Linear Model > Repeated Measures. Type a within-subject factor name (week) and the number of levels (4), click Add, then Define and select the four visit columns. Mauchly's test is displayed automatically, with epsilon adjustments for the F test.
• Friedman test: Analyze > Nonparametric Tests > Legacy Dialogs > K Related Samples, test type Friedman (Kendall's W and Cochran's Q are also offered).
• Linear mixed model: Analyze > Mixed Models > Linear. Put the patient ID in Subjects and, for a residual structure, the visit in Repeated with a covariance type (for example AR(1), compound symmetry or unstructured). Then choose the dependent variable, factors, and the Fixed and Random effects.
• GEE: Analyze > Generalized Linear Models > Generalized Estimating Equations. Choose the subject and within-subject variables and a working correlation matrix (independent, AR(1), exchangeable, M-dependent or unstructured).

R

# d: long format with columns id, week, pain
library(afex)
aov_ez("id", "pain", d, within = "week")   # Greenhouse-Geisser correction by default
friedman.test(pain ~ week | id, data = d)  # stats package

library(lme4); library(lmerTest)           # lmerTest adds Satterthwaite p-values
m <- lmer(pain ~ factor(week) + (1 | id), data = d)
summary(m)

library(geepack)                           # rows for each patient must be contiguous
g <- geeglm(pain ~ factor(week), id = id, data = d, corstr = "exchangeable")
summary(g)

# 2 x 2 crossover: x has one row per patient per period
lm(sbp ~ drug + period + factor(id), data = x)

jamovi

• Repeated Measures ANOVA (jmv function anovaRM): tick the sphericity tests, and choose the Greenhouse-Geisser or Huynh-Feldt sphericity correction.
• Repeated Measures ANOVA (non-parametric) (anovaRMNP) runs the Friedman test.
• Mixed models need the GAMLj module: put the patient ID in the cluster variable and choose the random coefficients.

Reporting

How to report it

State the design, the number of patients and of measurements, the model, how the correlation within patients was handled, how missing data were handled, and the estimates with 95% CIs.

Repeated measures ANOVA

"Pain was measured in 10 patients at weeks 0, 2, 4 and 8 and analysed with one-way repeated measures ANOVA. Mauchly's test indicated that sphericity was violated (W = 0.15, χ²(5) = 14.42, p = 0.01), so the Greenhouse-Geisser correction was applied (ε = 0.52). Mean pain fell from 63.7 at week 0 to 45.2 at week 8, F(1.56, 14.04) = 19.40, p < 0.001."

2 × 2 crossover trial

"In a 2 × 2 crossover trial, 12 patients were randomised to drug A then B (n = 6) or B then A (n = 6). The treatment effect was estimated within patients with a term for period. Systolic BP was 7.0 mmHg lower on drug A than on drug B (95% CI 4.3 to 9.7; p < 0.001). The period effect was −2.2 mmHg (p = 0.10)."

Linear mixed model

"HbA1c at 3, 6, 9 and 12 months was analysed with a linear mixed model with treatment, visit and treatment × visit as fixed effects and a random intercept for each patient, fitted by restricted maximum likelihood. All recorded values were included, assuming data missing at random."

By specialty

Repeated measures problems by specialty

Common myths

Common mistaken beliefs

• ✗Myth: 40 measurements from 10 patients give a sample of 40.

  ✓They give 10 patients. Values from one patient are correlated, and the effective sample size lies between 10 and 40, depending on the intraclass correlation.

• ✗Myth: A t-test at each visit shows when the groups start to differ.

  ✓Matthews and colleagues (1990) called this approach flawed: it can fail to answer the clinical question and can be statistically invalid. Use one model with a group × time interaction, or a summary measure.

• ✗Myth: If Mauchly's test is not significant, sphericity holds.

  ✓The test has low power in small samples. Report epsilon and the corrected p-value, whatever the test shows.

• ✗Myth: Test for carryover first, then choose the crossover analysis.

  ✓Freeman (1989) showed that this two-stage analysis inflates the false-positive rate. The EMA bioequivalence guideline states that a test for carryover is not considered relevant. Prevent carryover with an adequate washout.

• ✗Myth: A mixed model makes missing data harmless.

  ✓It gives valid estimates when data are missing at random. If patients drop out because of values that were never recorded, a mixed model is biased too. Plan sensitivity analyses.

• ✗Myth: GEE and a logistic mixed model estimate the same odds ratio.

  ✓GEE estimates a population-average odds ratio, which is closer to 1. The mixed model estimates a subject-specific odds ratio. In the illustration above: 2.49 against 4.48.

Why this matters to you

Five consequences of analysing repeated measures wrongly

1. A significant result that is not real

   Treating 64 values from 16 patients as 64 patients made p = 0.002 out of a difference the mixed model gives p = 0.13 for.

2. A real change that is missed

   Ignoring the patients left 6635.1 of the 7630.2 error sum of squares in the one-way ANOVA, and F fell from 19.40 to 3.37.

3. Patients lost from the analysis

   Repeated measures ANOVA drops everyone with a missed visit. In a thesis with dropouts, the completers may be the patients who did well.

4. Questions at the viva

   "How did you handle sphericity?", "What was the washout?" and "Why did you not use a mixed model?" need answers written into the protocol.

5. Misreading papers

   A crossover trial analysed as two independent groups, or a population-average odds ratio read as the effect in one patient, leads to wrong clinical conclusions.

History

History of repeated measures methods

1. 1937

   Milton Friedman

   Published "The use of ranks to avoid the assumption of normality implicit in the analysis of variance" in the Journal of the American Statistical Association.

2. 1940

   John W. Mauchly

   Published a significance test for sphericity of a normal n-variate distribution in the Annals of Mathematical Statistics.

3. 1959

   Samuel W. Greenhouse and Seymour Geisser

   In "On methods in the analysis of profile data" (Psychometrika), presented approximate analysis of variance procedures, including an adjustment to the degrees of freedom resulting in conservative F tests.

4. 1965

   J. E. Grizzle

   Published "The two-period change-over design and its use in clinical trials" (Biometrics), which first proposed a two-stage procedure for analysing such trials.

5. 1976

   Huynh Huynh and Leonard S. Feldt

   Showed that the sample estimate of the Box correction factor ε is seriously biased when ε is not low, and proposed an alternative estimate that is less biased for moderately large ε.

6. 1982

   N. M. Laird and J. H. Ware

   Described random-effects models for longitudinal data (Biometrics) that include growth models and repeated-measures models as special cases.

7. 1986

   Kung-Yee Liang and Scott L. Zeger

   Published "Longitudinal data analysis using generalized linear models" in Biometrika.

8. 1988

   S. L. Zeger, K. Y. Liang and P. S. Albert

   Distinguished subject-specific and population-averaged models for longitudinal data and fitted both with a generalized estimating equation approach.

9. 1989

   P. R. Freeman

   Examined the two-stage analysis of two-treatment, two-period crossover trials and concluded that it is too potentially misleading to be of practical use.

10. 1990

    J. N. S. Matthews, D. G. Altman, M. J. Campbell and P. Royston

    In the BMJ, recommended analysing serial measurements with summary measures, such as a rate of change or an area under a curve, calculated for each subject.

11. 2019

    Kerry Dwan, Tianjing Li, Douglas G. Altman and Diana Elbourne

    Published the CONSORT extension for randomised crossover trials in the BMJ.

Guidelines

Repeated measures and crossover trials in guidelines

Reading a paper

Five checks on repeated measures in any paper

• Do the authors state how many patients there were, and not only how many measurements?
• For repeated measures ANOVA, is sphericity addressed and the corrected degrees of freedom or epsilon reported?
• How were missing visits handled: complete cases, imputation, or a mixed model assuming missing at random?
• In a crossover trial: is the washout justified, is the analysis within patients with a period term, and was the analysis chosen without a carryover pre-test?
• For a yes/no outcome, is the odds ratio population-average (GEE) or subject-specific (mixed model), and is it interpreted accordingly?

Why is it called that?

Word surgery

Exam answers

Model answer outlines

Past papers

TNMGRMU questions on repeated measures and mixed models

Questions from MD, MS, DM, MCh and Ph.D. papers that this chapter helps you answer. Tap a sitting to open the university's own question paper.

1. ×2Cross over TrialsMD Community Medicine · Apr 2015MD Community Medicine · Apr 2015
2. ×1Cross-over study designDM Virology · Feb 2025
3. ×1Explain the differences between ‘parallel trials’ and ‘cross-over trials’MD Community Medicine · Jul 2021
4. ×1Cross over design drug trialsMD Pharmacology · Apr 2015
5. ×1Value of crossover design trials in drug trialsMD Pharmacology · May 2011