Solved Exercises on Confidence Intervals for Regression Slope

Master confidence intervals for regression slope: construction, interpretation, and applications through these 5 detailed exercises.

Solution: Exercises 1 to 3
1 Basic Confidence Interval Construction
Exercise 1
A researcher studied the relationship between study hours and exam scores for 25 students. The regression equation is ŷ = 52.3 + 2.8x, with standard error of slope SE_b = 0.65. Construct a 95% confidence interval for the population slope.
Definition:

Confidence interval for slope: An interval estimate that captures the true population slope with a specified level of confidence.

CI Construction Methodology:
  1. Identify sample slope (b) and standard error (SE_b)
  2. Determine confidence level and calculate α
  3. Calculate degrees of freedom: df = n - 2
  4. Find critical t-value for the desired confidence level
  5. Apply formula: CI = b ± t*(SE_b)
  6. Interpret the resulting interval
Step 1: Identify parameters

Sample slope (b) = 2.8

Standard error of slope (SE_b) = 0.65

Sample size (n) = 25

Confidence level = 95%

Step 2: Calculate degrees of freedom

df = n - 2 = 25 - 2 = 23

Step 3: Find critical t-value

For 95% confidence level, α = 0.05, α/2 = 0.025

t₀.₀₂₅,₂₃ = 2.069

Step 4: Calculate margin of error

Margin of error = t*(SE_b) = 2.069 × 0.65 = 1.345

Step 5: Construct confidence interval

Lower bound = b - margin of error = 2.8 - 1.345 = 1.455

Upper bound = b + margin of error = 2.8 + 1.345 = 4.145

CI = (1.455, 4.145)

Step 6: Interpret the interval

We are 95% confident that the true population slope falls between 1.455 and 4.145.

95% CI for slope: (1.455, 4.145)
Final answer:

The 95% confidence interval for the population slope is (1.455, 4.145). We are 95% confident that for every additional hour of study, exam scores increase by between 1.455 and 4.145 points.

Applied rules:

• CI formula: b ± t*(SE_b)

• Degrees of freedom: df = n - 2

• Interpretation: Confidence level indicates reliability

Confidence interval interpretation:

• If we repeatedly sampled and constructed CIs, 95% would contain the true slope

• The interval does not contain 0, suggesting a significant relationship

• The range indicates precision of our estimate

2 Higher Confidence Level
Exercise 2
Using the same data as Exercise 1 (n=25, b=2.8, SE_b=0.65), construct a 99% confidence interval for the population slope and compare it to the 95% CI.
Definition:

Higher confidence level: Increases the certainty that the interval contains the true parameter, but results in a wider interval.

Step 1: Identify parameters

Sample slope (b) = 2.8

Standard error of slope (SE_b) = 0.65

Sample size (n) = 25

New confidence level = 99%

Step 2: Calculate degrees of freedom

df = n - 2 = 25 - 2 = 23

Step 3: Find critical t-value

For 99% confidence level, α = 0.01, α/2 = 0.005

t₀.₀₀₅,₂₃ = 2.807

Step 4: Calculate margin of error

Margin of error = t*(SE_b) = 2.807 × 0.65 = 1.825

Step 5: Construct confidence interval

Lower bound = 2.8 - 1.825 = 0.975

Upper bound = 2.8 + 1.825 = 4.625

CI = (0.975, 4.625)

Step 6: Compare with 95% CI

95% CI: (1.455, 4.145)

99% CI: (0.975, 4.625)

99% CI is wider than 95% CI

99% CI: (0.975, 4.625) vs 95% CI: (1.455, 4.145)
Final answer:

The 99% confidence interval is (0.975, 4.625), which is wider than the 95% CI (1.455, 4.145). The higher confidence level increases certainty but decreases precision.

Applied rules:

• Confidence level trade-off: Higher confidence = wider interval

• Increased certainty: More likely to contain true parameter

• Decreased precision: Less specific estimate

Trade-offs in confidence intervals:

• 95% CI: More precise but less certain

• 99% CI: More certain but less precise

• Balance depends on application requirements

3 Effect of Sample Size
Exercise 3
A researcher found that for n=40, b=1.5, SE_b=0.4. For comparison, with n=15, b=1.5, SE_b=0.7. Construct 95% CIs for both and discuss the effect of sample size.
Definition:

Sample size effect: Larger samples generally produce narrower confidence intervals, increasing precision of estimates.

Step 1: Calculate CI for n=40

df = 40 - 2 = 38

t₀.₀₂₅,₃₈ ≈ 2.024

Margin of error = 2.024 × 0.4 = 0.810

CI = 1.5 ± 0.810 = (0.690, 2.310)

Step 2: Calculate CI for n=15

df = 15 - 2 = 13

t₀.₀₂₅,₁₃ = 2.160

Margin of error = 2.160 × 0.7 = 1.512

CI = 1.5 ± 1.512 = (-0.012, 3.012)

Step 3: Compare intervals

n=40: (0.690, 2.310) - Width = 1.620

n=15: (-0.012, 3.012) - Width = 3.024

Larger sample produces narrower CI

Step 4: Interpretation

The larger sample (n=40) provides a more precise estimate of the slope

The smaller sample CI includes 0, suggesting possible non-significance

Step 5: Practical implications

Smaller samples require more caution in interpretation

Increasing sample size improves precision significantly

n=40: (0.690, 2.310) vs n=15: (-0.012, 3.012)
Final answer:

The 95% CI for n=40 is (0.690, 2.310), while for n=15 it's (-0.012, 3.012). Larger sample size produces a narrower, more precise confidence interval.

Applied rules:

• Sample size effect: Larger n → smaller SE → narrower CI

• Precision: Narrower CI = more precise estimate

• Significance: CI including 0 suggests non-significant relationship

Sample size impact:

• Precision increases with larger samples

• Confidence intervals become narrower

• Statistical power increases

Key Formulas and Concepts
\( CI = b \pm t^* \cdot SE_b \)
Confidence Interval for Slope
Standard Error
\( SE_b = \frac{s}{s_x\sqrt{n-1}} \)
Standard error of slope
Degrees of Freedom
\( df = n - 2 \)
For simple linear regression
Critical Value
\( t_{\alpha/2, df} \)
T-multiplier for confidence level
Key definitions:

Confidence interval: An interval estimate that provides a range of plausible values for an unknown population parameter with a specified level of confidence.

Population slope (β): The true slope parameter in the population regression line that we aim to estimate.

Sample slope (b): The estimated slope from the sample data, used as a point estimate for the population slope.

Standard error of slope (SE_b): Measures the variability of the slope estimate across different samples.

Confidence level: The percentage of confidence intervals that would contain the true parameter if we repeated the sampling process many times.

Complete CI Construction Methodology:
  1. Verify conditions: Check linearity, independence, normality, and equal variance assumptions
  2. Extract parameters: Identify sample slope (b) and its standard error (SE_b)
  3. Determine confidence level: Choose desired confidence level (e.g., 90%, 95%, 99%)
  4. Calculate degrees of freedom: df = n - 2 for simple linear regression
  5. Find critical t-value: Look up t-value for α/2 and df
  6. Calculate margin of error: ME = t*(SE_b)
  7. Construct interval: Lower bound = b - ME, Upper bound = b + ME
  8. Interpret results: State the interval in context with confidence level
Tip 1: The confidence interval for slope provides a range of plausible values for the true relationship strength.
Tip 2: If the CI contains 0, the slope is not significantly different from 0 at that confidence level.
Tip 3: Larger sample sizes produce narrower confidence intervals, improving precision.
Tip 4: Higher confidence levels result in wider intervals, trading precision for certainty.
Common errors: Forgetting to check regression assumptions, misinterpreting confidence level, confusing confidence interval with prediction interval.
Exam preparation: Know the CI formula, understand df = n-2, practice with different confidence levels.
Key interpretation rules:

• Interval interpretation: We are X% confident the true slope falls within this range

• Significance: If CI excludes 0, the relationship is statistically significant

• Practical significance: Consider whether the range represents meaningful relationships

• Precision: Narrower intervals indicate more precise estimates

Important considerations:

• Confidence intervals do not prove causation

• Extrapolation beyond data range is risky

• Assumptions must be met for valid inference

• Outliers can affect both slope and CI

Solution: Exercises 4 to 5
4 Comparing Confidence Intervals
Exercise 4
Two studies examined the relationship between exercise and health scores: Study A (n=30): ŷ = 65.2 + 0.78x, SE_b = 0.15; Study B (n=45): ŷ = 63.8 + 0.82x, SE_b = 0.12. Construct 95% CIs and compare the results.
Definition:

Comparing CIs: Evaluating whether confidence intervals from different studies overlap, indicating similar or different relationships.

Step 1: Calculate CI for Study A

df = 30 - 2 = 28

t₀.₀₂₅,₂₈ = 2.048

ME = 2.048 × 0.15 = 0.307

CI_A = 0.78 ± 0.307 = (0.473, 1.087)

Step 2: Calculate CI for Study B

df = 45 - 2 = 43

t₀.₀₂₅,₄₃ ≈ 2.017

ME = 2.017 × 0.12 = 0.242

CI_B = 0.82 ± 0.242 = (0.578, 1.062)

Step 3: Compare intervals

CI_A: (0.473, 1.087)

CI_B: (0.578, 1.062)

Intervals overlap significantly

Step 4: Analyze precision

Study B has narrower CI due to larger sample size

Both studies suggest positive relationships

Step 5: Draw conclusions

Both studies indicate similar positive relationships

Study B provides more precise estimate

Study A: (0.473, 1.087), Study B: (0.578, 1.062)
Final answer:

Both confidence intervals suggest positive relationships between exercise and health scores. Study B provides a more precise estimate with a narrower interval due to the larger sample size.

Applied rules:

• Comparison: Overlapping CIs suggest similar relationships

• Sample size: Larger n leads to narrower CI

• Consistency: Both studies support similar conclusions

5 Practical Significance Assessment
Exercise 5
A study found: ŷ = 45.1 + 0.023x, n=100, SE_b = 0.008. Construct a 95% CI and assess both statistical and practical significance.
Definition:

Practical significance: Whether the magnitude of the relationship is meaningful in real-world applications, beyond statistical significance.

Step 1: Calculate degrees of freedom

df = 100 - 2 = 98

For large df, t₀.₀₂₅ ≈ 1.984

Step 2: Calculate margin of error

ME = 1.984 × 0.008 = 0.016

Step 3: Construct confidence interval

CI = 0.023 ± 0.016 = (0.007, 0.039)

Step 4: Assess statistical significance

CI does not contain 0, so relationship is statistically significant

Step 5: Assess practical significance

Slope suggests 0.023 unit increase per unit of x

Whether this is meaningful depends on context

For some applications, this might be too small to be useful

Step 6: Comprehensive conclusion

Statistically significant but may lack practical importance

95% CI: (0.007, 0.039) - Statistically significant but practically questionable
Final answer:

The 95% confidence interval is (0.007, 0.039), indicating a statistically significant relationship. However, the small slope magnitude may limit practical significance.

Applied rules:

• Statistical vs practical: Distinguish between the two

• Significance assessment: CI not containing 0 = statistical significance

• Context matters: Practical importance depends on application

Detailed Concepts and Applications
\( SE_b = \frac{\sqrt{\frac{\sum(y-\hat{y})^2}{n-2}}}{\sqrt{\sum(x-\bar{x})^2}} = \frac{s}{s_x\sqrt{n-1}} \)
Standard Error of Slope Formula
Key definitions:

Simple linear regression: A statistical method that models the relationship between one independent variable and one dependent variable using a straight line.

Point estimate: A single value (the sample slope b) used to estimate the population parameter (β).

Interval estimate: A range of values (confidence interval) that likely contains the population parameter.

Sampling distribution: The distribution of all possible values of the sample slope across different samples.

Standard error: The standard deviation of the sampling distribution of the slope estimator.

Advanced CI Methodology:
  1. Model verification: Confirm that linear regression assumptions are met
  2. Parameter extraction: Obtain slope estimate and its standard error from regression output
  3. Confidence level selection: Choose appropriate level based on application needs
  4. Degree of freedom calculation: df = n - 2 for simple regression
  5. Critical value determination: Use t-distribution with appropriate df
  6. Interval calculation: Apply CI formula with correct multiplier
  7. Assumption checking: Verify that assumptions hold for validity
  8. Interpretation in context: State results in terms of the original research question
  9. Practical assessment: Evaluate whether results have real-world significance
Tip 1: Always verify regression assumptions before constructing confidence intervals.
Tip 2: The width of the CI reflects the precision of the slope estimate.
Tip 3: A CI containing zero indicates the slope is not significantly different from zero.
Tip 4: Larger sample sizes reduce the standard error and narrow the confidence interval.
Tip 5: Consider both statistical and practical significance when interpreting results.
Common applications: Economics, psychology, medicine, engineering, social sciences, business analytics.
Limitations: Assumes linear relationship, sensitive to outliers, cannot establish causation, requires valid assumptions.
Key formulas and rules:

• Confidence interval: CI = b ± t*(SE_b)

• Degrees of freedom: df = n - 2

• Standard error: SE_b = s/[s_x√(n-1)]

• Critical value: t_{α/2, df}

• Significance: CI excluding 0 = significant relationship

Additional considerations:

• Confidence intervals are symmetric around the point estimate

• The t-distribution approaches normal distribution as df increases

• Multiple regression extends these concepts to multiple predictors

• Prediction intervals are different from confidence intervals

Questions & Answers

Question: What does it mean when a confidence interval for the slope contains zero?

Answer: When a confidence interval for the slope contains zero, it means:

  • The slope is not significantly different from zero at the given confidence level
  • There is insufficient evidence to conclude that a linear relationship exists between the variables
  • The p-value for testing H₀: β = 0 would be greater than α (where α = 1 - confidence level)
  • We cannot reject the null hypothesis of no linear relationship

For example, if the 95% CI for slope is (-0.2, 0.3), since it contains zero, we cannot conclude with 95% confidence that there is a significant linear relationship. The relationship might be positive, negative, or nonexistent.

However, failing to reject the null hypothesis doesn't prove that there is no relationship—it just means we don't have enough evidence to conclude there is one.

This is equivalent to saying the correlation coefficient is not significantly different from zero.

Question: How does the width of a confidence interval relate to the precision of the slope estimate?

Answer: The width of a confidence interval is inversely related to the precision of the slope estimate:

  • Narrower CI: More precise estimate, smaller standard error, greater confidence in the estimate
  • Wider CI: Less precise estimate, larger standard error, less confidence in the estimate

Factors affecting CI width:

  • Sample size (n): Larger n → smaller SE → narrower CI → more precise
  • Standard error (SE_b): Smaller SE → narrower CI → more precise
  • Confidence level: Higher confidence → wider CI → less precise
  • Variability in x values: More spread in x → smaller SE → narrower CI

A narrow CI indicates that we have a good idea of the true slope value, while a wide CI suggests considerable uncertainty about the true value.

Precision refers to how closely repeated samples would cluster around the true parameter value.

Accuracy refers to how close the estimate is to the true parameter value.

Question: What's the difference between a confidence interval for the slope and a prediction interval for a new observation?

Answer: These are fundamentally different types of intervals:

  • Confidence interval for slope: Estimates the range of plausible values for the true population slope parameter
  • Prediction interval: Estimates the range of plausible values for a new individual observation

Key differences:

  • Purpose: CI estimates parameter; PI estimates future observation
  • Uncertainty sources: CI only accounts for uncertainty in slope estimate; PI accounts for both parameter uncertainty and residual variation
  • Width: Prediction intervals are always wider than confidence intervals
  • Application: CI for understanding relationship; PI for making predictions

For example, a 95% CI for slope might be (0.5, 1.5), telling us the true slope likely falls in this range. A 95% PI for a new observation might be (40, 80), giving the likely range for an individual response.

The prediction interval incorporates both the uncertainty about the slope and the natural variation in individual observations around the regression line.

Confidence intervals for parameters vs. prediction intervals for observations serve different inferential purposes.