Solved Exercises on Calculating ANOVA: Between-Group and Within-Group Variation

Master ANOVA calculations: between-group and within-group variation, F-statistic, and ANOVA table construction through these 5 detailed exercises.

Solution: Exercises 1 to 3
1 Basic ANOVA Calculation
Exercise 1
Calculate the ANOVA for three groups with equal sample sizes: Group A: 10, 12, 14, 16; Group B: 15, 17, 19, 21; Group C: 20, 22, 24, 26. Find between-group and within-group variation.
Definition:

Between-group variation: Measures the variation between the group means and the overall mean.

ANOVA Calculation Methodology:
  1. Calculate group means and overall mean
  2. Calculate sum of squares between groups (SSB)
  3. Calculate sum of squares within groups (SSW)
  4. Calculate degrees of freedom
  5. Calculate mean squares
  6. Calculate F-statistic
  7. Construct ANOVA table
Step 1: Calculate group means

Group A: (10+12+14+16)/4 = 13

Group B: (15+17+19+21)/4 = 18

Group C: (20+22+24+26)/4 = 23

Step 2: Calculate overall mean

Overall mean: (13+18+23)/3 = 18

Step 3: Calculate SSB (Sum of Squares Between)

SSB = Σn_i(ȳ_i - ȳ)²

SSB = 4(13-18)² + 4(18-18)² + 4(23-18)²

SSB = 4(25) + 4(0) + 4(25) = 100 + 0 + 100 = 200

Step 4: Calculate SSW (Sum of Squares Within)

Group A: (10-13)² + (12-13)² + (14-13)² + (16-13)² = 9+1+1+9 = 20

Group B: (15-18)² + (17-18)² + (19-18)² + (21-18)² = 9+1+1+9 = 20

Group C: (20-23)² + (22-23)² + (24-23)² + (26-23)² = 9+1+1+9 = 20

SSW = 20 + 20 + 20 = 60

Step 5: Calculate degrees of freedom

df_between = k - 1 = 3 - 1 = 2

df_within = N - k = 12 - 3 = 9

df_total = N - 1 = 12 - 1 = 11

Step 6: Calculate mean squares

MSB = SSB/df_between = 200/2 = 100

MSW = SSW/df_within = 60/9 = 6.67

Step 7: Calculate F-statistic

F = MSB/MSW = 100/6.67 = 15

SSB = 200, SSW = 60, F = 15
Final answer:

Between-group variation (SSB) = 200, Within-group variation (SSW) = 60, F-statistic = 15. The large F-value suggests significant differences between groups.

Applied rules:

• SSB formula: SSB = Σn_i(ȳ_i - ȳ)²

• SSW formula: SSW = ΣΣ(y_ij - ȳ_i)²

• F-statistic: F = MSB/MSW

ANOVA Summary:

• SSB = 200, df = 2, MSB = 100

• SSW = 60, df = 9, MSW = 6.67

• F = 15, df = (2, 9)

2 Unequal Sample Sizes
Exercise 2
Calculate ANOVA for three groups with different sample sizes: Group 1: 5, 7, 9 (n₁=3); Group 2: 10, 12, 14, 16, 18 (n₂=5); Group 3: 20, 22 (n₃=2). Find all ANOVA components.
Definition:

Unequal sample sizes: When groups have different numbers of observations, calculations adjust for sample size weights.

Step 1: Calculate group means

Group 1: (5+7+9)/3 = 7

Group 2: (10+12+14+16+18)/5 = 14

Group 3: (20+22)/2 = 21

Step 2: Calculate overall mean

Total observations: N = 3+5+2 = 10

Overall mean: (3×7 + 5×14 + 2×21)/10 = (21+70+42)/10 = 13.3

Step 3: Calculate SSB

SSB = Σn_i(ȳ_i - ȳ)²

SSB = 3(7-13.3)² + 5(14-13.3)² + 2(21-13.3)²

SSB = 3(39.69) + 5(0.49) + 2(59.29) = 119.07 + 2.45 + 118.58 = 240.1

Step 4: Calculate SSW

Group 1: (5-7)² + (7-7)² + (9-7)² = 4 + 0 + 4 = 8

Group 2: (10-14)² + (12-14)² + (14-14)² + (16-14)² + (18-14)² = 16+4+0+4+16 = 40

Group 3: (20-21)² + (22-21)² = 1 + 1 = 2

SSW = 8 + 40 + 2 = 50

Step 5: Calculate degrees of freedom

df_between = k - 1 = 3 - 1 = 2

df_within = N - k = 10 - 3 = 7

Step 6: Calculate mean squares

MSB = 240.1/2 = 120.05

MSW = 50/7 = 7.14

Step 7: Calculate F-statistic

F = 120.05/7.14 = 16.82

SSB = 240.1, SSW = 50, F = 16.82
Final answer:

Between-group variation (SSB) = 240.1, Within-group variation (SSW) = 50, F-statistic = 16.82. Sample sizes are properly weighted in calculations.

Applied rules:

• Weighted means: Larger groups contribute more to overall mean

• Unequal n: Formula adjusts for different sample sizes

• DF calculation: df = n_i - 1 for each group

ANOVA Summary:

• SSB = 240.1, df = 2, MSB = 120.05

• SSW = 50, df = 7, MSW = 7.14

• F = 16.82, df = (2, 7)

3 ANOVA Table Construction
Exercise 3
For the data: Group A: 8, 10, 12; Group B: 14, 16, 18; Group C: 20, 22, 24; Group D: 26, 28, 30. Calculate all components and construct the complete ANOVA table.
Definition:

ANOVA table: Organized summary of ANOVA calculations showing sources of variation, degrees of freedom, sum of squares, mean squares, and F-statistic.

Step 1: Calculate group means

Group A: (8+10+12)/3 = 10

Group B: (14+16+18)/3 = 16

Group C: (20+22+24)/3 = 22

Group D: (26+28+30)/3 = 28

Step 2: Calculate overall mean

Overall mean: (10+16+22+28)/4 = 19

Step 3: Calculate SSB

SSB = 3(10-19)² + 3(16-19)² + 3(22-19)² + 3(28-19)²

SSB = 3(81) + 3(9) + 3(9) + 3(81) = 243 + 27 + 27 + 243 = 540

Step 4: Calculate SSW

Group A: (8-10)² + (10-10)² + (12-10)² = 4 + 0 + 4 = 8

Group B: (14-16)² + (16-16)² + (18-16)² = 4 + 0 + 4 = 8

Group C: (20-22)² + (22-22)² + (24-22)² = 4 + 0 + 4 = 8

Group D: (26-28)² + (28-28)² + (30-28)² = 4 + 0 + 4 = 8

SSW = 8 + 8 + 8 + 8 = 32

Step 5: Calculate total sum of squares

SST = SSB + SSW = 540 + 32 = 572

Step 6: Calculate degrees of freedom

df_between = 4 - 1 = 3

df_within = 12 - 4 = 8

df_total = 12 - 1 = 11

Step 7: Calculate mean squares

MSB = 540/3 = 180

MSW = 32/8 = 4

Step 8: Calculate F-statistic

F = 180/4 = 45

Step 9: Construct ANOVA table
Source df SS MS F
Between Groups 3 540 180 45
Within Groups 8 32 4
Total 11 572
F = 45, ANOVA table completed
Final answer:

The complete ANOVA table shows between-group variation of 540 and within-group variation of 32, with F-statistic = 45 indicating significant differences between groups.

Applied rules:

• Table organization: Clear source, df, SS, MS, F columns

• Relationships: df_total = df_between + df_within

• SS relationships: SST = SSB + SSW

ANOVA Summary:

• SSB = 540, df = 3, MSB = 180

• SSW = 32, df = 8, MSW = 4

• F = 45, df = (3, 8)

ANOVA Formulas and Calculation Methods
\( SSB = \sum_{i=1}^{k} n_i(\bar{y}_i - \bar{y})^2, \quad SSW = \sum_{i=1}^{k} \sum_{j=1}^{n_i} (y_{ij} - \bar{y}_i)^2 \)
Between-Group and Within-Group Sums of Squares
Total Sum of Squares
\( SST = \sum \sum (y_{ij} - \bar{y})^2 \)
SST = SSB + SSW
Degrees of Freedom
\( df_{total} = df_{between} + df_{within} \)
N-1 = (k-1) + (N-k)
Mean Squares
\( MSB = \frac{SSB}{df_{between}}, \quad MSW = \frac{SSW}{df_{within}} \)
Variance estimates
F-Statistic
\( F = \frac{MSB}{MSW} \)
Ratio of variances
Key definitions:

Sum of squares between (SSB): The variation between group means and the overall mean, weighted by sample sizes.

Sum of squares within (SSW): The variation within each group around its own mean.

Total sum of squares (SST): The total variation in the data around the overall mean.

Degrees of freedom: The number of independent pieces of information in a calculation.

Mean squares: Sum of squares divided by degrees of freedom, representing variance estimates.

F-statistic: The ratio of between-group variance to within-group variance.

Complete ANOVA Calculation Methodology:
  1. Calculate group statistics: Means and sample sizes for each group
  2. Calculate overall mean: Weighted average of all observations
  3. Calculate SSB: Sum of squares between groups using group means
  4. Calculate SSW: Sum of squares within groups using individual deviations
  5. Verify relationship: Check that SST = SSB + SSW
  6. Calculate degrees of freedom: For between, within, and total
  7. Calculate mean squares: Divide SS by respective df
  8. Calculate F-statistic: Ratio of mean squares
  9. Construct table: Organize all components in ANOVA table
Tip 1: Always verify that SST = SSB + SSW as a calculation check.
Tip 2: Larger SSB relative to SSW indicates greater differences between groups.
Tip 3: The F-statistic is always non-negative and typically ≥ 1 for meaningful differences.
Tip 4: Equal sample sizes simplify calculations but aren't required.
Tip 5: Check that df_total = df_between + df_within.

Common applications: Clinical trials, agricultural experiments, industrial quality control, educational research.
Limitations: Requires assumptions, sensitive to outliers, doesn't identify specific differences.
Key calculation rules:

• SSB calculation: SSB = Σn_i(ȳ_i - ȳ)²

• SSW calculation: SSW = ΣΣ(y_ij - ȳ_i)²

• DF relationships: df_total = N-1, df_between = k-1, df_within = N-k

• MS calculation: MS = SS/df

• F-statistic: F = MSB/MSW

Important relationships:

• SST = SSB + SSW (always true)

• df_total = df_between + df_within (always true)

• F = MSB/MSW (always non-negative)

• If groups are identical, F = 1

Solution: Exercises 4 to 5
4 Verification and Relationships
Exercise 4
Verify all ANOVA relationships using the data: Group 1: 2, 4, 6; Group 2: 8, 10, 12; Group 3: 14, 16, 18. Show that SST = SSB + SSW and df_total = df_between + df_within.
Definition:

ANOVA relationships: Fundamental mathematical relationships that must hold in any ANOVA calculation.

Step 1: Calculate group and overall means

Group 1: Mean = 4, Group 2: Mean = 10, Group 3: Mean = 16

Overall mean: (4+10+16)/3 = 10

Step 2: Calculate SSB

SSB = 3(4-10)² + 3(10-10)² + 3(16-10)²

SSB = 3(36) + 3(0) + 3(36) = 108 + 0 + 108 = 216

Step 3: Calculate SSW

Group 1: (2-4)² + (4-4)² + (6-4)² = 4 + 0 + 4 = 8

Group 2: (8-10)² + (10-10)² + (12-10)² = 4 + 0 + 4 = 8

Group 3: (14-16)² + (16-16)² + (18-16)² = 4 + 0 + 4 = 8

SSW = 8 + 8 + 8 = 24

Step 4: Calculate SST directly

SST = (2-10)² + (4-10)² + (6-10)² + (8-10)² + (10-10)² + (12-10)² + (14-10)² + (16-10)² + (18-10)²

SST = 64 + 36 + 16 + 4 + 0 + 4 + 16 + 36 + 64 = 240

Step 5: Verify relationship

SSB + SSW = 216 + 24 = 240 = SST ✓

Step 6: Verify degrees of freedom

df_between = 3 - 1 = 2

df_within = 9 - 3 = 6

df_total = 9 - 1 = 8

df_between + df_within = 2 + 6 = 8 = df_total ✓

Step 7: Complete ANOVA calculations

MSB = 216/2 = 108

MSW = 24/6 = 4

F = 108/4 = 27

SST = 240 = SSB + SSW = 216 + 24 ✓
Final answer:

All ANOVA relationships are verified: SST = SSB + SSW and df_total = df_between + df_within. The F-statistic is 27, indicating significant differences between groups.

Applied rules:

• Verification: Always check fundamental relationships

• Mathematical identity: SST = SSB + SSW is always true

• DF identity: df relationships must hold

5 Practical Interpretation
Exercise 5
A study compares test scores from four different teaching methods: Method A: 75, 78, 72; Method B: 82, 85, 80; Method C: 88, 90, 86; Method D: 70, 74, 71. Calculate ANOVA components and interpret the results.
Definition:

Practical interpretation: Understanding what the ANOVA results mean in the context of the research question.

Step 1: Calculate group means

Method A: (75+78+72)/3 = 75

Method B: (82+85+80)/3 = 82.33

Method C: (88+90+86)/3 = 88

Method D: (70+74+71)/3 = 71.67

Step 2: Calculate overall mean

Overall mean: (75+82.33+88+71.67)/4 = 79.25

Step 3: Calculate SSB

SSB = 3(75-79.25)² + 3(82.33-79.25)² + 3(88-79.25)² + 3(71.67-79.25)²

SSB = 3(18.06) + 3(9.49) + 3(76.56) + 3(57.76) = 54.18 + 28.47 + 229.68 + 173.28 = 485.61

Step 4: Calculate SSW

Method A: (75-75)² + (78-75)² + (72-75)² = 0 + 9 + 9 = 18

Method B: (82-82.33)² + (85-82.33)² + (80-82.33)² = 0.11 + 7.11 + 5.44 = 12.66

Method C: (88-88)² + (90-88)² + (86-88)² = 0 + 4 + 4 = 8

Method D: (70-71.67)² + (74-71.67)² + (71-71.67)² = 2.79 + 5.44 + 0.44 = 8.67

SSW = 18 + 12.66 + 8 + 8.67 = 47.33

Step 5: Calculate degrees of freedom

df_between = 4 - 1 = 3

df_within = 12 - 4 = 8

Step 6: Calculate mean squares

MSB = 485.61/3 = 161.87

MSW = 47.33/8 = 5.92

Step 7: Calculate F-statistic

F = 161.87/5.92 = 27.34

Step 8: Interpret results

The large F-statistic (27.34) suggests significant differences between teaching methods

Between-group variation (485.61) is much larger than within-group variation (47.33)

The differences in mean scores (75, 82.33, 88, 71.67) appear statistically significant

F = 27.34, SSB = 485.61, SSW = 47.33
Final answer:

The F-statistic of 27.34 indicates highly significant differences between teaching methods. Method C (mean = 88) appears most effective, while Method D (mean = 71.67) appears least effective.

Applied rules:

• Relative magnitude: Compare SSB to SSW to assess differences

• Practical significance: Consider real-world implications

• Statistical significance: Large F suggests differences exist

Detailed Concepts and Applications
\( F = \frac{\text{Variance between groups}}{\text{Variance within groups}} = \frac{MSB}{MSW} \)
F-Statistic Conceptual Formula
Key definitions:

Analysis of Variance: A statistical method that partitions total variation into components attributable to different sources.

Between-group variation: The portion of total variation that can be attributed to differences between treatment groups.

Within-group variation: The portion of total variation that represents random error or individual differences within groups.

Mean squares: Sum of squares divided by degrees of freedom, providing unbiased variance estimates.

Statistical inference: Using sample statistics to make inferences about population parameters.

Experimental design: The plan for collecting data that allows for valid statistical inference.

Advanced ANOVA Calculation Methodology:
  1. Design consideration: Plan the experiment to allow for valid ANOVA
  2. Data organization: Structure data in appropriate format for analysis
  3. Descriptive analysis: Calculate preliminary statistics and visualize data
  4. Component calculation: Compute all ANOVA components systematically
  5. Relationship verification: Check that mathematical identities hold
  6. Assumption checking: Verify ANOVA assumptions are met
  7. Statistical inference: Conduct hypothesis test using F-statistic
  8. Practical interpretation: Connect statistical results to research question
Tip 1: Calculate means first to reduce arithmetic errors in subsequent steps.
Tip 2: Use calculator or spreadsheet for complex arithmetic to avoid errors.
Tip 3: Check that SSB is positive and typically larger than SSW when groups differ.
Tip 4: Remember that F-statistic is always ≥ 0 and usually ≥ 1 for meaningful differences.

Tip 5: Interpret results in the context of the original research question.

Common applications: Psychology, medicine, agriculture, manufacturing, market research, quality control.
Advanced considerations: Two-way ANOVA, repeated measures, factorial designs, mixed models, non-parametric alternatives.
Key principles and guidelines:

• Variance partitioning: ANOVA decomposes total variance into components

• Ratio comparison: F-statistic compares systematic to random variation

• Weighted averages: Sample sizes affect the contribution of each group

• Mathematical consistency: All relationships must hold algebraically

• Statistical inference: Connect sample statistics to population parameters

Additional considerations:

• ANOVA assumes independence, normality, and equal variances

• Large sample sizes help meet normality assumption

• Outliers can significantly affect results

• Consider effect size measures along with significance tests

Questions & Answers

Question: Why is the F-statistic calculated as MSB/MSW rather than SSB/SSW?

Answer: The F-statistic uses mean squares (MS) rather than sum of squares (SS) for several important reasons:

  • Adjustment for degrees of freedom: Mean squares account for the different amounts of information in each component
  • Unbiased estimation: MSB and MSW provide unbiased estimates of population variances
  • Size adjustment: Dividing by df adjusts for the different sample sizes and complexity

SSB and SSW are affected by sample size and number of groups, making direct comparison inappropriate. By dividing by degrees of freedom, we standardize the measures to comparable units of variance.

Under the null hypothesis (no differences between groups), both MSB and MSW estimate the same population variance, so F ≈ 1.

When group means differ, MSB tends to be larger than MSW, resulting in F > 1.

The F-distribution is specifically derived for the ratio of mean squares, not sum of squares.

Question: What does it mean when SSB is much larger than SSW?

Answer: When SSB (between-group variation) is much larger than SSW (within-group variation), it indicates:

  • Large group differences: The group means are substantially different from each other
  • Small within-group variability: Observations within each group are relatively homogeneous
  • Strong treatment effects: The factor being studied has a pronounced effect

This situation results in a large F-statistic, providing strong evidence against the null hypothesis of equal means.

In practical terms, it suggests that the grouping variable (factor) explains a large proportion of the total variation in the response variable.

The ratio SSB/SSW (multiplied by the appropriate degrees of freedom) gives the F-statistic. A large ratio indicates that the systematic variation between groups dominates the random variation within groups.

This is the ideal scenario for detecting significant differences between groups.

Conversely, when SSW is much larger than SSB, it suggests minimal differences between groups relative to the random variation within groups.

Question: How do I interpret the practical significance of ANOVA results?

Answer: Statistical significance (low p-value) doesn't automatically mean practical significance. Consider:

  • Effect size: Calculate eta-squared (η² = SSB/SST) to measure proportion of variance explained
  • Magnitude of differences: Are the mean differences large enough to matter in practice?
  • Context: What do the results mean in your specific field or application?

Eta-squared interpretation:

  • η² = 0.01: Small effect
  • η² = 0.06: Medium effect
  • η² = 0.14: Large effect

For example, in a medical study, a statistically significant difference of 0.1 mmHg in blood pressure may not be clinically meaningful.

Always consider the cost-benefit ratio, practical feasibility, and real-world impact of implementing changes based on ANOVA results.

Look at confidence intervals for mean differences to understand the precision and practical range of effects.

Statistical significance indicates evidence of differences; practical significance considers whether those differences matter.