P-value: Probability of obtaining results at least as extreme as observed data, assuming null hypothesis is true
Statistical Significance: Result is unlikely to occur by chance alone, based on predetermined threshold
Alpha Level (α): Threshold for determining significance (typically 0.05, 0.01, or 0.10)
Decision Rule: If p-value ≤ α, reject H₀; if p-value > α, fail to reject H₀
Small p-value (≤ α): Strong evidence against H₀, suggests significant result
Large p-value (> α): Weak evidence against H₀, suggests non-significant result
Smaller p-value: Stronger evidence against null hypothesis
Given: z = 2.1, two-tailed test, α = 0.05
For two-tailed test: p-value = 2 × P(Z ≥ |observed value|)
P(Z ≥ 2.1) = 1 - P(Z ≤ 2.1) = 1 - 0.9821 = 0.0179
(Using standard normal table)
p-value = 2 × 0.0179 = 0.0358
Since p-value (0.0358) < α (0.05), reject H₀
The result is statistically significant at the 0.05 level
P-value = 0.0358
Decision: Reject H₀ (statistically significant)
• Two-tailed Formula: Multiply one-tail probability by 2
• Decision Rule: Reject H₀ if p-value ≤ α
• Significance Interpretation: p-value < 0.05 indicates significance
Given: t = 1.85, df = 24, right-tailed test, α = 0.05
This is a right-tailed test, so p-value = P(T ≥ 1.85)
With df = 24, P(T ≥ 1.85) ≈ 0.038
(Using t-table or calculator)
Since p-value (0.038) < α (0.05), reject H₀
There is sufficient evidence to support the alternative hypothesis
P-value ≈ 0.038
Decision: Reject H₀ (statistically significant)
• Right-tailed Formula: p-value = P(T ≥ observed value)
• Degrees of Freedom: Required for t-distribution
• Decision Rule: Compare p-value to α threshold
Given: z = -1.64, left-tailed test, α = 0.01
This is a left-tailed test, so p-value = P(Z ≤ -1.64)
P(Z ≤ -1.64) = 0.0505
(Using standard normal table)
Since p-value (0.0505) > α (0.01), fail to reject H₀
There is insufficient evidence to support the alternative hypothesis at α = 0.01
P-value = 0.0505
Decision: Fail to reject H₀ (not statistically significant at α = 0.01)
• Left-tailed Formula: p-value = P(Z ≤ observed value)
• Negative Z-value: Still follows same probability calculation
• Strict Alpha: α = 0.01 requires stronger evidence
Given: p-value = 0.035, multiple α levels to evaluate
Since 0.035 > 0.01, fail to reject H₀
Conclusion: Not significant at 1% level
Since 0.035 ≤ 0.05, reject H₀
Conclusion: Significant at 5% level
Since 0.035 ≤ 0.10, reject H₀
Conclusion: Significant at 10% level
α = 0.01: Most conservative, requires strongest evidence
α = 0.05: Standard threshold, balance between Type I and II errors
α = 0.10: More liberal, willing to accept higher risk of Type I error
α = 0.01: Fail to reject H₀ (not significant)
α = 0.05: Reject H₀ (significant)
α = 0.10: Reject H₀ (significant)
• Alpha Sensitivity: Same p-value leads to different decisions
• Conservative vs Liberal: Lower α requires stronger evidence
• Context Matters: Choose α based on consequences of errors
Study 1: n = 5000, difference = 0.001, p = 0.02
Study 2: n = 25, difference = 1.5, p = 0.08
Study 1: p = 0.02 < 0.05 → Statistically significant
Study 2: p = 0.08 > 0.05 → Not statistically significant
Study 1: 0.001 units may be too small to matter practically
Study 2: 1.5 units is likely practically important
Larger samples can detect tiny differences as significant
Smaller samples may miss practically important differences
Statistical significance ≠ Practical significance
Always consider effect size and context
Study 1: Statistically significant but likely not practically significant
Study 2: Not statistically significant but practically significant
Both statistical and practical significance matter for interpretation
• Sample Size Impact: Large samples increase statistical power
• Effect Size Importance: Consider magnitude, not just significance
• Contextual Interpretation: Statistical results must make practical sense