Do two factors act together?

Start here

What are factors, levels and cells?

A factor is a variable that divides patients into groups. Its categories are its levels. When two factors are crossed, every combination of their levels is a cell.

Two-way ANOVA analyses a measurement outcome with two factors at once. It asks three questions, each with its own F-test:

1. Does the outcome differ between the levels of factor A, averaged over factor B? (main effect of A)
2. Does it differ between the levels of factor B, averaged over factor A? (main effect of B)
3. Does the effect of A depend on the level of B? (the interaction, A × B)

One-way ANOVA and post hoc tests are covered in the multiple comparisons chapter. The logic of F, variation between groups divided by variation within groups, is explained in the choosing a test chapter.

Two types of effect

Main effect or interaction: what is the difference?

A main effect is the average effect of one factor. An interaction means that this effect is not the same at every level of the other factor.

In the four means on the resident's board, iron raised haemoglobin by 1.18 − 0.36 = 0.82 g/dL in children who were not dewormed, and by 2.04 − 0.40 = 1.64 g/dL in children who were dewormed. The effect of iron depends on deworming. The difference between the two effects, 1.64 − 0.82 = 0.82 g/dL, is the interaction. Seen the other way, deworming added 0.04 g/dL without iron and 0.86 g/dL with iron.

When there is an interaction, a main effect is an average of two different effects. In the worked example the main effect of deworming, 0.45 g/dL, describes neither group of children: those without iron gained 0.04 g/dL from deworming and those with iron gained 0.86 g/dL. When the lines cross (a qualitative or crossover interaction), Montgomery, Peters and Little warn that the main effects “will almost certainly be misleading”.

Interactive

Set the four group means and watch the main effects and the interaction change

After the F-tests

What do you do when the interaction is significant?

Report the effect of each factor separately at each level of the other factor. These are called simple effects.

Each test uses the error mean square from the two-way ANOVA (0.112, 16 df), so the standard error of each difference is √(0.112 × 2 ÷ 5) = 0.212. Four simple effects are four tests, so adjust the p-values for the number of tests. With the Holm method the conclusions here do not change (adjusted p = 0.003, < 0.001, 0.85 and 0.003).

How to adjust p-values for several tests is explained in the multiple comparisons chapter.

Design

Why test two treatments in one factorial trial?

In a factorial design every level of one factor is combined with every level of the other, so each patient contributes to the comparison of both treatments.

The interaction needs more patients

A main-effect comparison uses the difference between two averages of 2n patients each, so its standard error is σ ÷ √n (n = patients per cell). The interaction contrast, (iron and deworming − deworming alone) − (iron alone − neither), combines four cell means and has a standard error of 2σ ÷ √n, twice as large. To detect an interaction contrast as large as a main-effect difference with the same power, a trial needs about four times as many patients. In the worked example the two standard errors are 0.150 and 0.299 g/dL.

SE(main effect) = σ ÷ √n

SE(interaction contrast) = 2σ ÷ √n

ICH E9 states that the test of interaction “may lack power to detect an interaction if the sample size was calculated based on the test for main effects”. A non-significant interaction in a trial sized for main effects does not show that the treatments are additive.

Design

What does blocking do?

A block is a group of patients who are similar in something that affects the outcome, such as age band, centre or severity. Randomising within each block and analysing the block as a factor removes the differences between blocks from the error term.

In a completely randomised design patients are allocated to doses without regard to age. The one-way ANOVA error term then contains both the differences between patients of the same age and the differences between age bands. In a randomised block design the block is entered into a two-way analysis (treatment + block). The treatment sum of squares is the same, but the error sum of squares becomes smaller, so F for treatment becomes larger. The cost is 3 degrees of freedom for 4 blocks.

ICH E9 (section 5.7) states that if one or more factors are used to stratify the design, it is appropriate to account for those factors in the analysis. In a multicentre trial with randomisation stratified by centre, centre is a block.

Interactive

Change how much the age bands differ and watch blocking change the p-value for dose

The model

How is two-way ANOVA the same as a regression model?

Code each factor as 0 or 1 and add their product. A linear regression with these three terms reproduces the four cell means exactly.

Hb rise = b0 + b1 × iron + b2 × deworming + b3 × (iron × deworming)

b0 = 0.36

mean in children given neither treatment

b1 = 0.82

effect of iron in children not dewormed

b2 = 0.04

effect of deworming in children not given iron

b3 = 0.82

interaction: the extra effect when both are given (95% CI 0.19 to 1.45, p = 0.015)

With 0/1 (dummy) coding and the product term in the model, b1 and b2 are simple effects at the reference level of the other factor, not main effects. To get main-effect tests from a regression, software codes factors as −1 and +1 (sum-to-zero coding). This is why the documentation of the R package car warns that type III tests only make sense with contrasts such as contr.sum, and not with the default contr.treatment. Reading regression output is covered in the correlation and regression chapter.

A caveat

What changes when the groups are not the same size?

With equal numbers in every cell, the effects of the two factors can be separated exactly. With unequal numbers they overlap, and the sum of squares for a factor depends on what else is already in the model.

• Type I (sequential): each term is tested after the terms entered before it. The order matters: iron then deworming gives a different answer from deworming then iron.
• Type II: each main effect is tested after the other main effect, ignoring the interaction.
• Type III: each term is tested after all the others, including the interaction.

All three types agree when the design is balanced, and the interaction sum of squares is the same in all three because it is always tested last. SPSS GLM Univariate uses Type III by default, as does jamovi. R's aov() documentation states that it is designed for balanced designs and that results can be hard to interpret without balance. Plan equal groups where you can, and state in the methods which type of sums of squares you used.

Interactive

Change the number of children in each group and watch the three types of sums of squares separate

Effect size

How large is each effect? Partial eta squared

A p-value tells you whether an effect could be chance. Eta squared and partial eta squared describe how much of the variation in the outcome an effect accounts for.

η² = SS effect ÷ SS total

partial η² = SS effect ÷ (SS effect + SS error)

Eta squared values from one analysis add up to no more than 1, because each is a share of the same total. Partial eta squared leaves the other effects out of the denominator, so the values can add up to more than 1: here 0.81 + 0.36 + 0.32 = 1.49. SPSS labels its “Estimates of effect size” output as partial eta squared. Lakens (2013) notes that researchers predominantly report partial eta squared because SPSS provides it. State which one you report.

Decide

Which analysis fits a study with two factors?

Answer the questions in order.

Comparison

Pairs of ideas that residents confuse

Two-way ANOVA or two separate t-tests?

Main effect or interaction?

Completely randomised or randomised block design?

Eta squared or partial eta squared?

The formula

Worked example: two-way ANOVA by hand

Made-up teaching data. Rise in haemoglobin (g/dL) at 12 weeks, 5 children in each of four groups.

1. Grand mean = 19.9 ÷ 20 = 0.995. Row means: no iron 0.38, iron 1.61. Column means: no deworming 0.77, deworming 1.22.
2. SS iron = 10 × [(0.38 − 0.995)² + (1.61 − 0.995)²] = 7.565, with 1 df.
3. SS deworming = 10 × [(0.77 − 0.995)² + (1.22 − 0.995)²] = 1.013, with 1 df.
4. SS interaction = 5 × Σ (cell mean − row mean − column mean + grand mean)² = 5 × 4 × 0.205² = 0.841, with 1 df.
5. SS error = Σ (value − its cell mean)² = 1.792, with 4 × (5 − 1) = 16 df. MS error = 1.792 ÷ 16 = 0.112.
6. F = MS effect ÷ MS error. The critical F for (1, 16) df at 5% is 4.49.

SS

sum of squares

MS

mean square = SS ÷ df

n

children per cell (5); each row or column mean is based on 2n = 10 children

F(interaction) = 0.841 ÷ 0.112 = 7.50, df 1 and 16 → p = 0.015

Read the interaction first. It is significant, so the main effect of deworming (p = 0.008) is not a useful summary on its own. Report the simple effects instead: iron raised haemoglobin by 0.82 g/dL without deworming and 1.64 g/dL with it. Deworming added 0.04 g/dL without iron (p = 0.85) and 0.86 g/dL with iron (p < 0.001).

The resident's separate t-tests on the same 20 values gave t(18) = 6.11, p < 0.001 for iron and t(18) = 1.34, p = 0.20 for deworming. The deworming t-test mixed children with and without iron, so the effect of deworming in children given iron was lost.

All results were computed in Python (statsmodels 0.14.5). The design is balanced, so Type I, II and III sums of squares are identical.

In a real paper

How a 2 × 2 factorial trial reports the interaction

Software

How to run a two-way ANOVA

The steps below were checked against the IBM SPSS Statistics documentation, the CRAN reference manuals and the jamovi documentation in September 2026.

SPSS

1. Analyze > General Linear Model > Univariate. Put Hb rise in Dependent Variable and iron and deworming in Fixed Factor(s).
2. Model: Full factorial contains all factor main effects and all factor-by-factor interactions. Type III is the default sum of squares.
3. Plots: deworming on the horizontal axis and iron as separate lines, then Add. SPSS calls these profile plots (interaction plots).
4. EM Means: move iron*deworming across, tick Compare simple main effects, and choose a Bonferroni or Sidak confidence interval adjustment.
5. Options: tick Estimates of effect size (partial eta-squared) and Homogeneity tests (Levene).

R

d$iron   <- factor(d$iron)
d$deworm <- factor(d$deworm)

fit <- aov(hb ~ iron * deworm, data = d)   # aov() is designed for balanced designs
summary(fit)
interaction.plot(d$deworm, d$iron, d$hb)

# Type III tests need sum-to-zero contrasts (car documentation)
library(car)
fit3 <- lm(hb ~ iron * deworm, data = d,
           contrasts = list(iron = contr.sum, deworm = contr.sum))
Anova(fit3, type = "III")                  # Anova() uses type "II" unless told otherwise

library(effectsize)
eta_squared(fit, partial = TRUE)           # uses anova(fit): fine for a balanced design

library(emmeans)
emmeans(fit, pairwise ~ iron | deworm)     # simple effects of iron within each deworming level

jamovi

1. Use the ANOVA analysis, with Hb rise as the dependent variable and iron and deworming as fixed factors.
2. Sum of squares: Type 3 is the default. Types 1 and 2 are also available.
3. Effect size: choose η², partial η² or ω².
4. Estimated marginal means: add the iron × deworming term. Plots of the estimated marginal means are drawn by default.

# the same analysis from R with the jmv package that runs jamovi analyses
jmv::ANOVA(formula = hb ~ iron + deworm + iron:deworm, data = d,
           ss = "3", effectSize = "partEta", emMeans = ~ iron:deworm)

Reporting

How to report a two-way ANOVA

Report the design, the cell means and SDs, the interaction first, then the main effects or the simple effects, each with an effect size and a confidence interval.

1. Name both factors, their levels, and whether they were randomised or observed.
2. Give the number, mean and SD for every cell, not only the row and column means.
3. State that you checked the assumptions (residuals, Levene test) and how you handled unequal cell sizes (type of sums of squares).
4. Report F, both df and the exact p for the interaction, then for the main effects, with partial η² and what it means.
5. If the interaction matters, report simple effects with 95% CIs and say how you adjusted for multiple tests.
6. Name the software and version.

A sentence for your thesis, using this chapter's teaching numbers:

Haemoglobin rise at 12 weeks was analysed by two-way ANOVA with iron, deworming and their interaction as fixed factors. The iron × deworming interaction was significant (F(1, 16) = 7.50, p = 0.015, partial η² = 0.32). Iron raised haemoglobin by 0.82 g/dL (95% CI 0.37 to 1.27) in children who were not dewormed and by 1.64 g/dL (95% CI 1.19 to 2.09) in children who were dewormed. Deworming without iron had little effect (0.04 g/dL, 95% CI −0.41 to 0.49).

The SAMPL guidelines (Lang and Altman) ask authors reporting ANOVA to “specify whether the explanatory variables were tested for interaction, and if so how these interactions were treated”. For factorial trials, the CONSORT 2010 extension (Kahan et al., 2023) asks for the estimated interaction effect and its precision for each primary outcome.

By specialty

Two-factor questions by specialty

Common myths

Common mistaken beliefs

• ✗Myth: A non-significant interaction proves that the two factors do not interact.

  ✓The interaction test has less power than the main-effect tests. ICH E9 warns that it may lack power if the sample size was based on main effects. Look at the confidence interval.

• ✗Myth: Two separate t-tests give the same answer as a two-way ANOVA.

  ✓In the worked example the t-test for deworming gave p = 0.20, while the two-way analysis showed that deworming added 0.86 g/dL in children given iron.

• ✗Myth: Test the interaction first, then decide which analysis to report.

  ✓The CONSORT extension for factorial trials states that using a test of interaction to guide the choice of analysis can introduce bias and is not recommended. Decide in the protocol.

• ✗Myth: Parallel-looking lines on the plot prove there is no interaction.

  ✓Plotted means vary by chance. Whether lines differ from parallel is judged by the interaction test and its confidence interval. An interaction can also appear on one scale and not another, for example on the risk difference scale but not the risk ratio scale.

• ✗Myth: Partial eta squared values add up to 100%.

  ✓Only eta squared values share one denominator. In the worked example the partial eta squared values add up to 1.49.

• ✗Myth: Main effects can always be interpreted.

  ✓When lines cross, main effects average opposite effects and can be misleading. Report simple effects.

Why this matters to you

Five consequences of ignoring the interaction

1. Protocol sent back

   A factorial design with a sample size based only on main effects cannot answer a question about the combination. The committee may ask whether the interaction is the aim.

2. A wrong conclusion in the thesis

   Separate t-tests can report "no effect" for a treatment that works only together with the other treatment.

3. Questions you cannot answer at the viva

   "What is an interaction?", "Why did you not report simple effects?" and "Which type of sums of squares did you use?" need direct answers.

4. Misreading factorial trials

   Main effects from a factorial trial are valid summaries only if the two treatments do not interact. Check whether the trial could detect an interaction.

5. Harm to patients

   A combination may be adopted, or rejected, on the basis of main effects when the benefit depends on giving both treatments together.

History

History of factorial experiments and two-way analysis

1. 1923

   Ronald Fisher and W. A. Mackenzie

   Published "The manurial response of different potato varieties", the first paper on analysis of variance. Its title names two factors studied together: variety and manure.

2. 1934

   Frank Yates

   Published "The analysis of multiple classifications with unequal numbers in the different classes". Herr (1986) traces the history of analysing unbalanced factorial designs back to Yates's papers of 1933 and 1934.

3. 1988

   ISIS-2 Collaborative Group

   Reported a randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17,187 patients with suspected myocardial infarction: a 2 × 2 factorial trial.

4. 1998

   ICH E9

   Described factorial designs in its guideline on statistical principles for clinical trials, and noted that the test of interaction may lack power.

5. 2003

   Finlay McAlister, Sharon Straus, David Sackett and Douglas Altman

   Reviewed 44 factorial trials with binary outcomes: 26 (59%) reported testing for interactions, and only 2 of 31 comparisons showed a statistically significant interaction.

6. 2023

   Brennan Kahan and colleagues

   Published the CONSORT 2010 extension for reporting factorial randomised trials.

Guidelines

Factorial designs and interactions in guidelines

Reading a paper

Five checks on any two-way ANOVA or factorial trial

• Are the mean, SD and number given for every cell, not only for each factor?
• Was the interaction tested, with its estimate and confidence interval reported?
• If there was an interaction, did the authors report simple effects instead of relying on main effects?
• Was the trial large enough to detect an interaction, or was it sized only for main effects?
• If the cells were unequal, did the authors state which type of sums of squares they used?

Why is it called that?

Word surgery

Exam answers

Short-note outlines