What do you do about missing values?

Start here

Why does it matter which patients are missing?

Missing values cause two problems. There are fewer patients, so the estimate is less precise. And if the missing patients differ from the others, the estimate is biased, however many patients remain.

Three types of missing data

Why are the values missing?

The reason decides which analysis gives an unbiased answer. The three standard terms come from Rubin's framework for missing data.

• Missing completely at random (MCAR): the chance of a value being missing is unrelated to anything about the patient, recorded or not. A freezer failure destroys a batch of samples. The patients left are a random subset, so a complete-case analysis loses precision but is not biased.
• Missing at random (MAR): the chance depends on things you have recorded, and not further on the missing value. Older patients miss the 1-year visit more often, age is recorded, and among patients of the same age attendance is unrelated to the result. Methods that use age can correct the bias.
• Missing not at random (MNAR): the chance still depends on the missing value after allowing for everything recorded. Patients whose mood worsens stop answering the questionnaire. No recorded variable explains it.

The names mislead. “Missing at random” does not mean the missing patients are like the others: in the MAR example they are older. As Sterne and colleagues put it, the phrases convey little about their technical meaning. More important for practice: it is not possible to distinguish MAR from MNAR using the observed data, so the assumption has to be argued from how the data were collected and then tested with sensitivity analyses.

Interactive

Remove BP values by each mechanism and watch the complete-case mean, the imputed SD and the true mean

The same 200 made-up patients each time. Only the reason for the missing values changes.

Interactive

Read each scenario and name the mechanism: MCAR, MAR or MNAR

Look for the one fact that decides it: is missingness linked to nothing, to something recorded, or to the missing value itself?

Comparison

Missing at random or missing not at random: what is the difference?

The default

What does your software do with a missing value?

Most procedures silently drop every patient with a missing value in any variable the analysis uses. This is complete-case (listwise) analysis. In R, lm() does this by default.

When a complete-case analysis is acceptable

• Data are MCAR. The estimate is not biased, only less precise.
• Only the outcome is missing, measured once, and all variables associated with the outcome being missing are included as covariates (under MAR). Sterne and colleagues note that such analyses will not be biased.
• Only predictors are missing, for reasons unrelated to the outcome. There is no bias, but multiple imputation can recover lost precision.
• Very little is missing. Jakobsen and colleagues give a rule of thumb: complete-case analysis may be the primary analysis if under about 5% is missing and particular patient groups (the very sick or the very well) are not lost from one of the compared groups.

Otherwise complete-case analysis carries a risk of overestimating benefit and underestimating harm. It is still the most common approach: in 77 trials published in 2013 in the BMJ, JAMA, Lancet and NEJM, 73 had missing outcome data and 33 of those used complete cases for the primary analysis (Bell et al., 2014).

Single imputation

Why does filling in the mean or the last value mislead?

Both give every missing patient one invented value and then analyse it as if it had been measured.

• Mean imputation puts every missing value at the centre, which reduces the variability of the data.
• LOCF and baseline observation carried forward give unbiased estimates only under restrictive assumptions. Depending on the disease and who drops out, LOCF can make a treatment look better or worse, and it is not always conservative (EMA, 2010).
• Every single-imputation method treats invented values as real, so standard errors are too small (Sterne et al., 2009). The NRC panel recommended that LOCF and baseline observation carried forward should not be used as the primary approach unless their assumptions are scientifically justified.

Multiple imputation

How does multiple imputation fill the gaps without pretending to know the values?

It fills each gap several times with different plausible values, analyses every completed dataset, and combines the results so that the uncertainty about the missing values is included in the standard error.

1. Choose the imputation model: predict each variable with missing values from the others. Include the outcome, every variable in the planned analysis, and variables that predict the missing values or the reason they are missing.
2. Create m completed datasets. Each missing value is a prediction plus random variation, and the model's coefficients are themselves drawn with their uncertainty. The mice package in R uses m = 5 by default. The trial in the paper below used 50.
3. Run the planned analysis, unchanged, in each dataset.
4. Combine the m estimates with Rubin's rules (formula below).

Multiple imputation by chained equations (MICE) imputes one variable at a time, each from its own regression model, cycling through the variables. It assumes MAR. If the data are MNAR, its bias can be as large as that of complete cases. Leaving the outcome out of the imputation model is a known error: in the QRISK example described by Sterne and colleagues, it contributed to an unexpected null association between cholesterol and cardiovascular risk.

Interactive

Change who drops out and watch complete cases, LOCF and multiple imputation against the true difference

A made-up trial in which the whole truth is known, analysed the four ways used in real papers.

Missing data in trials

If outcomes are missing, is your analysis still intention to treat?

Intention to treat means analysing every randomised patient in the group they were allocated to. A patient without an outcome can be included only by making an assumption about that missing outcome.

• Prevention comes first. The NRC panel concluded that there is no foolproof way to analyse substantial amounts of missing data, and that the key is to design and conduct the trial to limit the problem.
• Stopping treatment is not leaving the study. Too many investigators stop recording outcomes when a patient stops the drug. Continuing to measure these patients keeps an intention-to-treat analysis possible.
• ICH E9(R1) makes the same distinction. Discontinuing randomised treatment is an intercurrent event, handled in the definition of the question (the estimand). Withdrawal from the study gives missing data, handled in the analysis.
• Recruiting extra patients does not fix it. Inflating the sample size for expected dropouts restores precision but does nothing about bias from dropouts who differ from completers.
• Report it. CONSORT 2025 asks for losses and exclusions after randomisation in each group with reasons (item 22b), who was included in each analysis (21b) and how missing data were handled (21c).

Sensitivity analysis

How do you show that the result does not depend on the MAR assumption?

Repeat the analysis under different, plausible assumptions about the missing values and see whether the conclusion changes.

ICH E9(R1) defines a sensitivity analysis as a series of analyses that explore whether the inferences from the main estimator still hold when its underlying assumptions are not met, and gives tipping point analysis as an example. Common approaches:

• Best-worst and worst-best case: give all missing patients in one group a good outcome and in the other a bad one, then reverse. This shows the widest range of uncertainty for a yes/no outcome (Jakobsen et al., 2017).
• Delta adjustment (offset): assume that patients who dropped out have a mean outcome that differs by δ from similar patients who stayed, impute with that shift, and vary δ. The NRC panel describes this pattern-mixture approach.
• Tipping point: increase δ until the conclusion changes, then judge whether a difference that large is clinically plausible.
• Complete cases alongside multiple imputation: report both. Where they differ, Sterne and colleagues advise explaining why.

Interactive

Make the missing drug-arm patients worse and find the tipping point

Each step re-imputes the missing scores with a shift added. The tipping point is the smallest shift at which the 95% CI includes zero.

Decide

Which approach fits your missing data?

Answer two or three questions. Each answer links to the section that explains it.

Comparison

Complete-case analysis or multiple imputation: which should you report?

Usually both, with one named as the primary analysis in the protocol.

The formula

Rubin's rules: combining the results of m imputed datasets

Average the estimates, and add the variation between imputations to the average variance within them.

Q̄ = (1/m) Σ Qᵢ  ·  W̄ = (1/m) Σ SEᵢ²  ·  B = Σ(Qᵢ − Q̄)² ÷ (m − 1)

T = W̄ + (1 + 1/m) × B  ·  SE = √T

Qᵢ, SEᵢ

estimate and standard error from imputed dataset i

W̄

within-imputation variance: the uncertainty there would be with complete data

B

between-imputation variance: the extra uncertainty because values were missing

df

Rubin (1987): (m − 1) × (1 + W̄ ÷ [(1 + 1/m)B])². The mice package in R uses the Barnard-Rubin small-sample version

Worked example: five imputations

Made-up teaching numbers. Difference in week-12 depression score (drug − placebo) from five imputed datasets:

1. Q̄ = (−3.1 − 2.6 − 3.4 − 2.9 − 2.5) ÷ 5 = −2.90.
2. W̄ = (1.10² + 1.05² + 1.12² + 1.08² + 1.06²) ÷ 5 = 1.171.
3. B = [(−0.2)² + 0.3² + (−0.5)² + 0² + 0.4²] ÷ 4 = 0.54 ÷ 4 = 0.135.
4. T = 1.171 + 1.2 × 0.135 = 1.333, so SE = 1.155. Using the average SE alone (√1.171 = 1.082) would understate the uncertainty.
5. df = 4 × (1 + 1.171 ÷ 0.162)² = 271, critical t = 1.969.

−2.90 ± 1.969 × 1.155 → 95% CI −5.17 to −0.63, p = 0.013

The share of the total variance due to missing data, (1 + 1/m)B ÷ T, is 0.162 ÷ 1.333 = 12% here. The mice package reports it as lambda.

In a real paper

Multiple imputation and a tipping-point analysis in a randomised trial

How to run it

How to run multiple imputation

Every step below was checked against the official documentation. Steps that could not be checked are left out.

R (mice package)

library(mice)
md.pattern(d)                          # display the missing-data patterns
imp <- mice(d, m = 20, seed = 2029)    # default m = 5; default method for numeric data is "pmm"
fit <- with(imp, lm(hamd12 ~ arm + hamd0 + hamd6))   # the analysis, run in every imputed dataset
summary(pool(fit))                     # Rubin's rules; df with the Barnard-Rubin adjustment

naniar::mcar_test(d)                   # Little's test of missing completely at random
lm(hamd12 ~ arm, data = d)             # na.action default na.omit: complete cases only

SPSS Statistics

These procedures need SPSS Statistics Professional Edition or the Missing Values option. The documentation pages retrieved give the dialog names but not the menu path, so the steps below name the dialogs.

1. Little's MCAR test: in the Missing Value Analysis dialog box, click EM, then OK. The test appears in footnotes to each EM estimate table. Its null hypothesis is that the data are missing completely at random.
2. Analyze Patterns (a Multiple Imputation procedure): select at least two analysis variables. The output can include a summary of missing values, the patterns of missing values, and the variables with the highest percentage missing.
3. Impute Missing Data Values (Multiple Imputation): select at least two variables for the imputation model, set the number of imputations (default 5), and name a dataset or file for the imputed data. The new dataset contains the variable Imputation_ (0 for the original data), which SPSS defines as a split variable.
4. Run the planned analysis on the imputed dataset. Procedures that support pooling, such as the Independent-Samples T Test, automatically give results for each imputation, the original data and pooled results.

DATASET DECLARE imputedData.
MULTIPLE IMPUTATION hamd0 hamd6 hamd12
 /IMPUTE METHOD=AUTO NIMPUTATIONS=20
 /OUTFILE IMPUTATIONS = imputedData.

jamovi

The jamovi documentation checked for this chapter gives no steps for multiple imputation. The BSS module in the jamovi library lists “missing value analysis and data imputation using the missRanger and MICE” among its features, but its documentation gives no step-by-step instructions, so none are given here.

How to report it

What to write about missing data

Numbers missing in each group and why, the assumption, the method with enough detail to repeat it, and a sensitivity analysis.

Methods:

“Week-12 scores were assumed to be missing at random given baseline and week-6 scores. Missing scores were imputed 20 times from a linear regression on baseline and week-6 scores within each arm, and estimates were combined with Rubin's rules. In a sensitivity analysis, imputed scores in the drug arm were made progressively worse to find the shift at which the conclusion changed.”

Results (the trial in the interactive elements):

“Week-12 scores were missing for 25 of 100 patients on placebo and 15 of 100 on the drug. After multiple imputation the difference was −2.50 points (95% CI −4.28 to −0.72). The complete-case estimate was −1.36 (−3.00 to 0.29). The 95% CI included zero if missing drug-arm patients were assumed to score 4.2 points worse than predicted.”

Sterne and colleagues (2009) also ask authors to report the number of imputed datasets and the variables in the imputation model, to compare patients with and without missing data, and, where possible, to give the complete-case results for comparison.

By specialty

Missing data problems by specialty

Common myths

Common mistaken beliefs

• ✗Myth: Missing at random means the missing patients are just like the others.

  ✓MAR means missingness depends only on recorded variables. The missing patients may be very different, for example older.

• ✗Myth: Under 5% missing never matters.

  ✓The rule of thumb (Jakobsen et al., 2017) also requires that particular patient groups are not lost from one of the compared groups.

• ✗Myth: Filling gaps with the mean is harmless.

  ✓It lowers the SD: in the teaching example from 19.9 to 10.3 mmHg. Single imputation often underestimates variability.

• ✗Myth: LOCF is always conservative.

  ✓The EMA guideline states that in some situations LOCF does not produce conservative estimates, and in a worsening disease it is likely to be optimistic.

• ✗Myth: Multiple imputation fixes any missing data.

  ✓It assumes MAR. When data are MNAR its bias can be as big as or bigger than that of complete cases (Sterne et al., 2009).

• ✗Myth: Recruiting 20% extra patients deals with dropout.

  ✓It restores the sample size, not the comparability of the groups. Bias from dropouts who differ is unchanged (NRC panel).

• ✗Myth: A test can show which mechanism you have.

  ✓Little's test can detect departures from MCAR, but MAR and MNAR cannot be distinguished using the observed data.

Why this matters to you

Five consequences of mishandling missing data

1. Protocol sent back

   A protocol needs the expected losses, how they will be prevented, the primary method for missing data and the sensitivity analysis.

2. A thesis result on a biased subset

   Software drops incomplete patients without warning. The analysed patients may no longer be the randomised groups.

3. Questions you cannot answer at the viva

   "How many were lost from each group, why, and what did you assume about them?" needs numbers and a reason.

4. Misreading papers

   Of 73 trials in top journals with missing outcomes, 45% used complete cases and 8% multiple imputation for the primary analysis (Bell et al., 2014). Many results rest on untested assumptions.

5. Harm to patients

   When data are not MCAR, complete-case analysis risks overestimating benefit and underestimating harm (Jakobsen et al., 2017).

History

History of methods for missing data

1. 1976

   Donald Rubin

   Published "Inference and missing data" in Biometrika, the framework of missing-data mechanisms behind the terms used today.

2. 1987

   Donald Rubin

   Published Multiple Imputation for Nonresponse in Surveys, the source of the combining rules now called Rubin's rules.

3. 2007

   STROBE initiative

   The STROBE statement for observational studies asked authors to explain how missing data were addressed and to report the number of participants with missing data for each variable.

4. 2009

   Jonathan Sterne and colleagues

   Found 59 articles using multiple imputation in four general medical journals from 2002 to 2007. Only seven fully reported both imputed and complete-case results.

5. 2010

   European Medicines Agency

   Adopted the Guideline on Missing Data in Confirmatory Clinical Trials (in force from 1 January 2011), with cautions on LOCF.

6. 2012

   Roderick Little and colleagues

   Summarised in the NEJM the National Research Council panel report requested by the FDA in 2008: prevent missing data, avoid LOCF as the primary method, and run sensitivity analyses.

7. 2019

   ICH

   Adopted the E9(R1) addendum on estimands and sensitivity analysis (20 November 2019), separating intercurrent events from missing data.

8. 2025

   CONSORT 2025 group

   Added item 21c to the trial reporting checklist: how missing data were handled in the analysis.

Guidelines

Missing data in guidelines

Reading a paper

Five checks on how a paper handled missing data

• Are the numbers with missing outcome data given for each group, with reasons?
• Is the assumed mechanism (MCAR, MAR, MNAR) stated, and is it plausible for how the data were collected?
• Which method was used? Single imputation such as LOCF or mean imputation needs a stated justification.
• For multiple imputation: are the number of imputations and the variables in the imputation model reported?
• Is there a sensitivity analysis under other assumptions, and does the conclusion survive it?

Why is it called that?

Word surgery

Exam answers

Short-note outlines