F-statistic: The ratio of between-group variance to within-group variance, testing whether group means differ significantly.
- Compare p-value with significance level α
- Interpret the F-statistic magnitude
- State statistical decision
- Provide practical interpretation
- Consider effect size
p-value = 0.0008
α = 0.05
Since 0.0008 < 0.05, we reject H₀
F = 8.45 is considerably larger than 1
This indicates strong evidence against the null hypothesis
We reject the null hypothesis of equal means
There are significant differences in effectiveness among the four teaching methods
The large F-statistic suggests substantial differences between methods
We reject the null hypothesis. There is strong evidence (p = 0.0008) that at least one teaching method differs significantly in effectiveness from the others at α = 0.05.
• Decision rule: Reject H₀ if p-value ≤ α
• F-statistic interpretation: F > 1 suggests differences exist
• Statistical significance: p < 0.001 indicates strong evidence
• The probability of observing F = 8.45 or larger, if means were truly equal, is only 0.0008
• This is extremely unlikely, providing strong evidence against H₀
• At least one pair of teaching methods differs significantly
Non-significant result: When p-value > α, we fail to reject the null hypothesis, indicating insufficient evidence of differences.
p-value = 0.307
α = 0.05
Since 0.307 > 0.05, we fail to reject H₀
F = 1.23 is close to 1
This suggests the between-group variation is similar to within-group variation
We fail to reject the null hypothesis of equal means
There is insufficient evidence to conclude that the three drug treatments differ in effectiveness
Failing to reject H₀ does NOT prove the treatments are equally effective
We fail to reject the null hypothesis. There is insufficient evidence (p = 0.307) to conclude that the three drug treatments differ significantly in effectiveness at α = 0.05.
• Decision rule: Fail to reject H₀ if p-value > α
• Non-proof: Failure to reject ≠ proof of equality
• Insufficient evidence: May be due to small sample size or small differences
• The probability of observing F = 1.23 or larger, if means were truly equal, is 0.307
• This is not unusual enough to reject the null hypothesis
• No significant differences detected between treatments
Significance level (α): The threshold for determining statistical significance, representing the maximum acceptable probability of Type I error.
p-value = 0.039
α = 0.01
Since 0.039 > 0.01, we fail to reject H₀
p-value = 0.039
α = 0.05
Since 0.039 < 0.05, we reject H₀
p-value = 0.039
α = 0.10
Since 0.039 < 0.10, we reject H₀
Significant at α = 0.05 and α = 0.10
Not significant at α = 0.01
The result is significant at conventional levels (0.05) but not at stricter levels (0.01)
There is moderate evidence of differences, but not overwhelming evidence
At α = 0.01: Fail to reject H₀. At α = 0.05 and α = 0.10: Reject H₀. The result is significant at moderate significance levels but not at the most stringent level.
• Level dependency: Same result can be significant or not depending on α
• Stricter α: Lower α makes rejection harder
• Context matters: Choose α based on consequences of errors
• The evidence is strong enough for moderate significance levels
• But not strong enough for very strict significance level
• The differences may be meaningful but not extremely pronounced
Statistical significance: When the p-value is less than or equal to the significance level, indicating evidence against the null hypothesis.
Practical significance: Whether the observed effect is meaningful in the real world, regardless of statistical significance.
Type I error: Rejecting the null hypothesis when it is actually true (false positive).
Type II error: Failing to reject the null hypothesis when it is actually false (false negative).
Power: The probability of correctly rejecting a false null hypothesis.
Effect size: A measure of the magnitude of the observed effect, independent of sample size.
- Examine the p-value: Compare with the chosen significance level
- Make statistical decision: Reject or fail to reject the null hypothesis
- State conclusion: Express in terms of the original research question
- Consider practical significance: Evaluate the real-world importance
- Assess effect size: Determine the strength of the relationship
- Discuss limitations: Consider sample size and potential biases
- Draw final interpretation: Combine statistical and practical considerations
• Decision rule: Reject H₀ if p-value ≤ α
• Fail to reject: Does NOT prove H₀ is true
• Statistical vs practical: Distinguish between the two
• Effect size: Report strength of association
• Context: Always interpret in research context
• P-value = probability that H₀ is true (incorrect!)
• P-value = probability that results occurred by chance (misleading)
• Non-significant = proves no effect (incorrect!)
• Significant = important in practice (not necessarily)
Sample size effect: Larger samples can detect smaller effects as statistically significant.
F = 4.2, p-value = 0.025
At α = 0.05: p-value < α, reject H₀
Statistically significant result
F = 4.2, p-value = 0.001
At α = 0.05: p-value < α, reject H₀
Highly statistically significant result
Same F-statistic implies same effect size
Both studies detect the same magnitude of difference
Study B has greater power to detect effects
Smaller p-value in Study B due to larger sample size
Both studies suggest the same practical importance
Study B provides more confidence in the result
Both studies find significant differences (p < 0.05), but Study B (larger sample) provides stronger evidence (p = 0.001 vs p = 0.025). The practical significance is the same in both studies.
• Sample size impact: Larger n increases power to detect effects
• Effect size: Same F-statistic implies same practical importance
• Statistical vs practical: Distinguish between significance and importance
Effect size (η²): A measure of the proportion of total variance explained by the group differences.
η² = SSB/SST = SSB/(SSB + SSW)
η² = 384/(384 + 468) = 384/852 = 0.451
η² = 0.451 means 45.1% of the total variance is explained by group differences
This is considered a large effect size (Cohen's guidelines)
p-value = 0.0001 < α = 0.05
We reject H₀: There is a statistically significant difference
F = 12.8 is substantially greater than 1
Strong evidence of group differences
Statistically significant result with large practical effect
The group differences are both significant and meaningful
The independent variable explains nearly half of the variance in the dependent variable
There is a statistically significant difference between groups (p < 0.001) with a large effect size (η² = 0.451), indicating that group membership explains 45.1% of the variance in the outcome.
• Effect size calculation: η² = SSB/SST
• Effect size interpretation: η² ≈ 0.01 (small), 0.06 (medium), 0.14 (large)
• Statistical vs practical: Both aspects matter in interpretation
P-value: The probability of obtaining results as extreme or more extreme than those observed, given that the null hypothesis is true.
Statistical significance: The result is unlikely to have occurred by chance alone, according to the chosen significance level.
Practical significance: Whether the observed effect is meaningful or important in real-world applications.
Effect size: A quantitative measure of the magnitude of a phenomenon, independent of sample size.
Power: The probability of correctly rejecting a false null hypothesis.
F-distribution: The probability distribution of the F-statistic under the null hypothesis.
- Examine test statistics: Review F value, degrees of freedom, and sample size
- Analyze p-value: Compare with significance level α
- Make statistical decision: Reject or fail to reject null hypothesis
- Calculate effect size: Use η² or partial η² for strength of association
- Interpret effect size: Determine practical significance
- Contextualize findings: Relate to research question and field of study
- Assess limitations: Consider sample characteristics and study design
- Draw comprehensive conclusion: Integrate statistical and practical interpretations
• Decision rule: Reject H₀ if p-value ≤ α
• Effect size: η² = SSB/SST
• Effect size interpretation: Small ≈ 0.01, Medium ≈ 0.06, Large ≈ 0.14
• P-value interpretation: Probability under null hypothesis
• Practical significance: Consider real-world impact
1. Statistical significance does not imply practical importance
2. Large samples can make trivial differences significant
3. Small samples may miss important effects
4. Association does not imply causation
5. Always consider the research context
6. Report both statistical and practical findings