ANOVA vs ANCOVA: When to Use Each (With Examples)

You run a one-way ANOVA to compare customer satisfaction scores across three onboarding flows. The result is not significant (p = 0.14). You conclude the flows perform equally. But six months later, a colleague re-analyzes the same data with ANCOVA, controlling for customer tenure, and finds a highly significant difference (p = 0.003). The effect was there all along -- tenure-related noise was hiding it.

This is not a rare edge case. ANCOVA frequently detects effects that ANOVA misses, because it removes variance attributable to covariates before testing group differences. The question is not which method is "better" -- it is which method matches your data and design. This guide covers the mechanics of both, when each is appropriate, and the assumptions you need to check before trusting your results.

Worked Example (Simulated): ANOVA and ANCOVA on Three Training Groups

Question: Do the mean post scores differ across three training groups after we account for the base score?

Inputs: Simulated data, seed 1847. There are 90 people in three groups, 30 in each. Each person has a base score and a post score. The script builds in group effects of 0, 5 and 2 points for groups A, B and C, and adds random noise with a standard deviation of 8.

Method: The script first compares post scores with one-way ANOVA, a test of group means. It then uses ANCOVA to compare group means while taking the base score into account. It fits the model by least squares, with group and base score (the mean taken off the base score), and tests the group terms against a model with base score alone. The adjusted means are model means at the mean base score.

import numpy as np
from scipy import stats

# Simulated data, seed 1847
rng = np.random.default_rng(1847)
n_per_group = 30
labels = np.repeat(np.array(['A', 'B', 'C']), n_per_group)
baseline = rng.normal(50, 10, size=labels.size)
# Built-in group effects (points): A 0, B 5, C 2
group_effect = np.select(
    [labels == 'A', labels == 'B', labels == 'C'], [0.0, 5.0, 2.0]
)
noise = rng.normal(0, 8, size=labels.size)
post = 20.0 + 0.7 * baseline + group_effect + noise

# One-way ANOVA from group means and sums of squares
levels = ['A', 'B', 'C']
arrays = [post[labels == level] for level in levels]
means = np.array([a.mean() for a in arrays])
grand_mean = post.mean()
ss_between = sum(len(a) * (a.mean() - grand_mean) ** 2 for a in arrays)
ss_within = sum(((a - a.mean()) ** 2).sum() for a in arrays)
df_between = len(levels) - 1
df_within = len(post) - len(levels)
f_anova = (ss_between / df_between) / (ss_within / df_within)
p_anova = stats.f.sf(f_anova, df_between, df_within)

# ANCOVA: full model (intercept, group indicators, centered baseline)
# compared with reduced model (intercept, centered baseline only).
x = baseline - baseline.mean()
g_b = (labels == 'B').astype(float)
g_c = (labels == 'C').astype(float)
X_full = np.column_stack([np.ones(len(post)), g_b, g_c, x])
X_reduced = np.column_stack([np.ones(len(post)), x])
b_full, _, _, _ = np.linalg.lstsq(X_full, post, rcond=None)
b_reduced, _, _, _ = np.linalg.lstsq(X_reduced, post, rcond=None)
resid_full = post - X_full @ b_full
resid_reduced = post - X_reduced @ b_reduced
sse_full = resid_full @ resid_full
sse_reduced = resid_reduced @ resid_reduced
df_full = len(post) - X_full.shape[1]
f_ancova = ((sse_reduced - sse_full) / df_between) / (sse_full / df_full)
p_ancova = stats.f.sf(f_ancova, df_between, df_full)
adjusted = np.array([b_full[0], b_full[0] + b_full[1], b_full[0] + b_full[2]])
sst = (post - post.mean()) @ (post - post.mean())

print('Simulated data, seed 1847')
print(f'Inputs: {len(post)} people, 3 groups, {n_per_group} per group; '
      'baseline and post scores')
print('Built-in group effects (points): '
      + ', '.join(f'{lv}={group_effect[labels == lv][0]:.0f}' for lv in levels)
      + '; noise SD 8')
print('Method: one-way ANOVA; ANCOVA by least squares with group and '
      'centered baseline,')
print('        group test against a baseline-only model')
print('Raw post means: '
      + ', '.join(f'{lv}={m:.2f}' for lv, m in zip(levels, means)))
print(f'ANOVA: F({df_between}, {df_within})={f_anova:.3f}, p={p_anova:.4f}')
print(f'ANCOVA: F({df_between}, {df_full})={f_ancova:.3f}, p={p_ancova:.4f}')
print('Adjusted means at mean baseline: '
      + ', '.join(f'{lv}={m:.2f}' for lv, m in zip(levels, adjusted)))
print(f'Baseline slope={b_full[3]:.3f}; '
      f'R-squared for baseline alone={1 - sse_reduced / sst:.3f}')

Printed output:

Simulated data, seed 1847
Inputs: 90 people, 3 groups, 30 per group; baseline and post scores
Built-in group effects (points): A=0, B=5, C=2; noise SD 8
Method: one-way ANOVA; ANCOVA by least squares with group and centered baseline,
        group test against a baseline-only model
Raw post means: A=54.75, B=59.50, C=59.91
ANOVA: F(2, 87)=2.349, p=0.1015
ANCOVA: F(2, 86)=2.522, p=0.0862
Adjusted means at mean baseline: A=55.41, B=59.66, C=59.09
Baseline slope=0.648; R-squared for baseline alone=0.404

Result: The post means are A = 54.75, B = 59.50, and C = 59.91. ANOVA gives F(2, 87) = 2.349 and p = 0.1015. ANCOVA gives F(2, 86) = 2.522 and p = 0.0862. Its adjusted means are A = 55.41, B = 59.66, and C = 59.09. The script builds in group effects of 0, 5 and 2 points. With 30 people per group, neither test flagged them. A result that is not clear is not proof of no difference.

Limit: These scores are made up. They show how to run the two tests. They do not show what may happen in a real training study.

Sources

• Welch B (1951). On the Comparison of Several Mean Values: An Alternative Approach. Biometrika. doi:10.2307/2332579. Covers: Welch's ANOVA, an alternative when group spreads differ.

What Each Method Does

ANOVA: Comparing Group Means Directly

Analysis of Variance (ANOVA) tests whether the means of three or more groups differ significantly on a continuous dependent variable. It partitions total variance into between-group variance (differences due to group membership) and within-group variance (individual variation within each group). The F-statistic is the ratio of these two quantities.

To compare groups after adjusting for a baseline, run your own file through the free ANCOVA tool.

A one-way ANOVA has one factor (e.g., treatment group). A two-way ANOVA has two factors (e.g., treatment group and gender). The logic is the same: if between-group variance is large relative to within-group variance, the groups likely differ.

ANCOVA: Adjusting for Covariates First

Analysis of Covariance (ANCOVA) extends ANOVA by including one or more continuous covariates. Before comparing group means, ANCOVA removes the portion of variance in the dependent variable that is explained by the covariate. This does two things: it reduces error variance (increasing power) and it adjusts group means to account for covariate differences (reducing bias).

Think of ANCOVA as asking: "After accounting for differences in X, do the groups still differ on Y?" For example: "After adjusting for baseline blood pressure, do the three drug treatments produce different outcomes?"

Side-by-Side Comparison

When ANOVA Is Sufficient

ANOVA is the right choice when your design already handles the concerns that ANCOVA addresses. Specifically, use ANOVA when:

• Groups are randomly assigned and balanced. Randomization ensures that covariates are distributed equally across groups on average, so there is no systematic bias to adjust for.
• No strong covariate exists. If no continuous variable has a meaningful linear relationship with your dependent variable, ANCOVA gains you nothing.
• You need simplicity. ANOVA results are easier to explain to non-statistical audiences. "Group A scored higher than Group B" is clearer than "After adjusting for baseline scores, Group A's adjusted mean was higher."
• Sample sizes are large and equal. With large samples, ANOVA already has good power, and the gain from ANCOVA is marginal.

When ANCOVA Is Better

ANCOVA becomes the stronger choice in several common scenarios:

• A strong covariate exists. If a baseline measure (pre-test score, prior revenue, initial weight) correlates with the outcome, ANCOVA will have substantially more power.
• Groups differ on a baseline measure. Even with randomization, small samples often produce imbalanced groups. ANCOVA adjusts for this. Without adjustment, a group that happened to start with higher baseline scores may appear to perform better, even if the treatment had no effect.
• Pre-post designs. When you have a pre-test and post-test, ANCOVA with a pre-test score can work better than a test of change scores. The best choice depends on the study plan, the rules for the test, and the question.
• Observational studies. When random assignment is impossible (comparing departments, regions, or naturally occurring groups), ANCOVA can adjust for measured confounders. It does not eliminate confounding entirely, but it reduces bias from known sources.
• Small sample sizes. When you have limited data, reducing error variance through covariate adjustment can mean the difference between detecting an effect and missing it.

Assumptions: What to Check

Shared Assumptions (Both Methods)

• Independence: Observations are independent of each other. Violated by repeated measures, clustered data, or time-series data.
• Normality: Residuals are approximately normally distributed. Both methods are robust to moderate violations with n > 30 per group.
• Homogeneity of variance: Variances are roughly equal across groups. If the group spreads look different, Welch's ANOVA is an alternative (Welch 1951).

Additional ANCOVA Assumptions

• Homogeneity of regression slopes: The relationship between the covariate and the dependent variable must be the same across all groups. Test this by including a group-by-covariate interaction term. If significant, standard ANCOVA is inappropriate.
• Linear relationship: The covariate must have a linear relationship with the dependent variable. Check with a scatterplot. If the relationship is curvilinear, consider polynomial terms or nonparametric alternatives.
• Covariate measured without error: Measurement error in the covariate attenuates the adjustment, leaving residual confounding. Reliable measures (high internal consistency) are preferred.
• Covariate not affected by treatment: If the treatment influences the covariate, adjusting for it can remove part of the treatment effect, biasing your results toward zero.

Hypothetical Illustration: Training Program Effectiveness

This is a hypothetical illustration: the company and every figure in it are invented. A company tests three sales training programs (A, B, C) with 25 salespeople per group. The dependent variable is post-training sales revenue. The covariate is pre-training sales revenue (past 6 months).

ANOVA Result

F(2, 72) = 2.41, p = 0.097
Group means: A = $142K, B = $158K, C = $151K
Conclusion: No significant difference (p > 0.05)

ANCOVA Result (Controlling for Pre-Training Revenue)

F(2, 71) = 8.73, p < 0.001
Adjusted means: A = $148K, B = $156K, C = $147K
Covariate (pre-training revenue): F(1, 71) = 34.2, p < 0.001
Conclusion: Significant difference after adjustment

Why the different conclusions? Pre-training revenue explained 32% of the variance in post-training revenue. Adjustment moved Group A up (from $142K to $148K) and Group C down (from $151K to $147K), which means Group A started with lower pre-training revenue than average and Group C with higher, so the raw means were partly measuring who was in each group. With baseline held constant, and with the variance the covariate explains removed from the error term, Group B stands out. ANOVA missed the effect because baseline differences added noise that obscured the treatment effect.

Decision Guide

Use ANOVA when:

• Groups are randomly assigned with balanced sample sizes
• No strong continuous covariate is available or theoretically relevant
• You need simple, easy-to-communicate results
• Sample sizes are large enough that power is not a concern

Use ANCOVA when:

• A pre-treatment baseline measure of the outcome exists
• A continuous variable is known to correlate with the outcome
• Groups may differ at baseline (small samples or non-random assignment)
• You need more statistical power with a limited sample
• You are analyzing a pre-post design

Use neither when:

• Your outcome variable is categorical (use chi-square or logistic regression)
• Data is severely non-normal with small samples (use Kruskal-Wallis)
• You need causal inference from observational data with many confounders (use propensity score methods)

You can use ANCOVA with two groups. A t-test is an option if you do not need to adjust for another score.

Common Mistakes to Avoid

1. Using ANCOVA to "fix" a bad design. ANCOVA adjusts for measured covariates, not unmeasured ones. If your groups differ systematically on factors you did not measure, ANCOVA cannot save you.
2. Adding too many covariates. Each covariate costs a degree of freedom. With small samples, adding three or four weak covariates can actually reduce power. Include only covariates with a meaningful relationship to the outcome.
3. Ignoring the homogeneity of regression slopes assumption. If the covariate-outcome relationship differs across groups, ANCOVA-adjusted means are meaningless. Always test the interaction first.
4. Confusing ANCOVA with "controlling for" in regression. ANCOVA is a special case of the general linear model, but it is designed for comparing group means. If your primary interest is the relationship between continuous predictors, use regression.
5. Reporting raw means instead of adjusted means. After running ANCOVA, the adjusted (estimated marginal) means are the relevant comparison. Reporting raw means defeats the purpose of the covariate adjustment.

To run ANCOVA on your own data and get adjusted group means, see the ANCOVA analysis.