Solved Exercises on Inference for Slope in Linear Regression

Master inference for slope in linear regression: t-tests, confidence intervals, hypothesis testing, and interpretation through these 5 detailed exercises.

Solution: Exercises 1 to 3
1 Basic T-Test for Slope
Exercise 1
A researcher studied the relationship between hours studied and exam scores for 20 students. The regression equation is ŷ = 55.2 + 3.4x, with standard error of slope SE_b = 0.8. Test at α = 0.05 whether there is a significant linear relationship between hours studied and exam scores.
Definition:

Regression slope: The change in the dependent variable (y) for each unit increase in the independent variable (x).

Inference methodology:
  1. State null and alternative hypotheses
  2. Calculate test statistic: t = (b - β₀) / SE_b
  3. Determine degrees of freedom: df = n - 2
  4. Find critical value or p-value
  5. Make decision
  6. Draw conclusion
Step 1: State hypotheses

H₀: β = 0 (no linear relationship)

H₁: β ≠ 0 (linear relationship exists)

Step 2: Calculate test statistic

t = (b - β₀) / SE_b = (3.4 - 0) / 0.8 = 4.25

Step 3: Determine degrees of freedom

df = n - 2 = 20 - 2 = 18

Step 4: Find critical value

For α = 0.05, two-tailed test, df = 18

t_critical = ±2.101

Step 5: Make decision

|t| = 4.25 > 2.101, so we reject H₀

t = 4.25, df = 18, reject H₀ at α = 0.05
Final answer:

We reject the null hypothesis. There is sufficient evidence to conclude that there is a significant linear relationship between hours studied and exam scores at α = 0.05.

Applied rules:

• Test statistic: t = (sample slope - hypothesized slope) / standard error

• Degrees of freedom: df = n - 2 for simple linear regression

• Decision rule: Reject H₀ if |t| > critical value

2 Confidence Interval for Slope
Exercise 2
A study of 25 individuals found a regression equation: ŷ = 12.5 + 0.75x, with SE_b = 0.15. Construct a 95% confidence interval for the population slope.
Definition:

Confidence interval: An interval estimate that captures the true population parameter with a specified level of confidence.

Step 1: Identify parameters

Sample slope (b) = 0.75

Standard error (SE_b) = 0.15

n = 25

Confidence level = 95%

Step 2: Calculate degrees of freedom

df = n - 2 = 25 - 2 = 23

Step 3: Find critical t-value

For 95% confidence, α = 0.05, α/2 = 0.025

t₀.₀₂₅,₂₃ = 2.069

Step 4: Calculate confidence interval

CI = b ± t*(SE_b)

CI = 0.75 ± 2.069(0.15)

CI = 0.75 ± 0.310

CI = (0.440, 1.060)

Step 5: Interpret the interval

We are 95% confident that the true population slope is between 0.440 and 1.060.

95% CI for slope: (0.440, 1.060)
Final answer:

The 95% confidence interval for the population slope is (0.440, 1.060). We are 95% confident that the true slope falls within this range.

Applied rules:

• CI formula: b ± t*(SE_b)

• Degrees of freedom: df = n - 2

• Interpretation: Confidence level indicates reliability

3 One-Tailed Test
Exercise 3
A nutritionist claims that daily calorie intake has a positive effect on weight gain. With n = 18, the regression showed: ŷ = 65.2 + 0.08x, SE_b = 0.03. Test at α = 0.01 whether calorie intake positively affects weight gain.
Definition:

One-tailed test: A hypothesis test where the alternative hypothesis specifies a direction (positive or negative).

Step 1: State hypotheses

H₀: β ≤ 0 (calorie intake has no positive effect or negative effect)

H₁: β > 0 (calorie intake has positive effect)

Step 2: Calculate test statistic

t = (b - β₀) / SE_b = (0.08 - 0) / 0.03 = 2.67

Step 3: Determine degrees of freedom

df = n - 2 = 18 - 2 = 16

Step 4: Find critical value

For α = 0.01, one-tailed test, df = 16

t_critical = 2.583

Step 5: Make decision

t = 2.67 > 2.583, so we reject H₀

Step 6: Draw conclusion

There is sufficient evidence to support the claim that calorie intake positively affects weight gain.

t = 2.67 > 2.583, reject H₀
Final answer:

We reject the null hypothesis. There is sufficient evidence at α = 0.01 to conclude that daily calorie intake has a positive effect on weight gain.

Applied rules:

• One-tailed test: Critical value is only on one side

• Directional hypothesis: Specifies positive or negative relationship

• Decision rule: Compare test statistic with one-sided critical value

Key Formulas and Concepts
\( t = \frac{b - \beta_0}{SE_b} \)
T-Test for Slope
Confidence Interval
\( b \pm t^* \cdot SE_b \)
For slope estimation
Standard Error
\( SE_b = \frac{s}{s_x\sqrt{n-1}} \)
Standard error of slope
Degrees of Freedom
\( df = n - 2 \)
For simple linear regression
Key definitions:

Population slope (β): The true slope parameter in the population regression line.

Sample slope (b): The estimated slope from the sample data.

Standard error of slope (SE_b): Measures the variability of the slope estimate.

Simple linear regression: A model that relates one independent variable to one dependent variable using a straight line.

Complete methodology:
  1. Verify conditions: Linearity, independence, normality, equal variance
  2. State hypotheses: H₀: β = β₀ vs H₁: β ≠ β₀ (or directional)
  3. Calculate test statistic: t = (b - β₀) / SE_b
  4. Determine degrees of freedom: df = n - 2
  5. Find critical value or p-value
  6. Make decision: Compare t-statistic with critical value
  7. Draw conclusion: Interpret in context of the problem
Tip 1: Always check regression assumptions before inference.
Tip 2: The t-distribution is used because the population standard deviation is unknown.
Tip 3: A significant slope means there's a linear relationship, not necessarily causation.
Tip 4: The confidence interval for slope provides a range of plausible values.
Common errors: Forgetting to check assumptions, misinterpreting p-values, confusing correlation with causation.
Exam preparation: Know the formula for t-statistic, understand df = n-2, practice interpretation.
Assumptions to verify:

• Linearity: Relationship between x and y is linear

• Independence: Observations are independent

• Normality: Residuals are normally distributed

• Equal variance: Constant variance of residuals (homoscedasticity)

Solution: Exercises 4 to 5
4 P-Value Approach
Exercise 4
A study of 30 students found: ŷ = 42.1 + 1.8x, SE_b = 0.65. Test at α = 0.05 whether the slope differs from 1.0 using the p-value approach.
Definition:

P-value approach: Compare the p-value with the significance level α to make a decision.

Step 1: State hypotheses

H₀: β = 1.0

H₁: β ≠ 1.0

Step 2: Calculate test statistic

t = (b - β₀) / SE_b = (1.8 - 1.0) / 0.65 = 1.23

Step 3: Determine degrees of freedom

df = n - 2 = 30 - 2 = 28

Step 4: Find p-value

This is a two-tailed test

p-value = 2 × P(T > |1.23|) = 2 × P(T > 1.23)

Using t-table with df = 28: p-value ≈ 0.229

Step 5: Make decision

p-value = 0.229 > α = 0.05

We fail to reject H₀

Step 6: Draw conclusion

There is insufficient evidence to conclude that the slope differs from 1.0.

p-value = 0.229 > 0.05, fail to reject H₀
Final answer:

We fail to reject the null hypothesis. There is insufficient evidence at α = 0.05 to conclude that the slope differs from 1.0.

Applied rules:

• P-value method: Compare p-value with α

• Decision rule: Reject H₀ if p-value ≤ α

• Two-tailed test: Multiply probability by 2

5 Comparison of Slopes
Exercise 5
Two groups were studied: Group A (n=22): ŷ = 15.3 + 0.92x, SE_b = 0.18; Group B (n=25): ŷ = 18.7 + 1.05x, SE_b = 0.22. Test at α = 0.05 whether the slopes are significantly different.
Definition:

Comparing slopes: Testing whether two regression lines have significantly different slopes.

Step 1: State hypotheses

H₀: β_A = β_B (slopes are equal)

H₁: β_A ≠ β_B (slopes are different)

Step 2: Calculate test statistic

SE_diff = √(SE_bA² + SE_bB²) = √(0.18² + 0.22²) = √(0.0324 + 0.0484) = √0.0808 = 0.284

t = (b_A - b_B) / SE_diff = (0.92 - 1.05) / 0.284 = -0.13 / 0.284 = -0.458

Step 3: Determine degrees of freedom

df = n_A + n_B - 4 = 22 + 25 - 4 = 43

Step 4: Find critical value

For α = 0.05, two-tailed test, df = 43

t_critical ≈ ±2.017

Step 5: Make decision

|t| = 0.458 < 2.017

We fail to reject H₀

|t| = 0.458 < 2.017, fail to reject H₀
Final answer:

We fail to reject the null hypothesis. There is insufficient evidence at α = 0.05 to conclude that the slopes for the two groups are significantly different.

Applied rules:

• Comparing slopes: SE_diff = √(SE_b1² + SE_b2²)

• Degrees of freedom: df = n₁ + n₂ - 4

• Decision rule: Compare |t| with critical value

Detailed Concepts and Applications
\( SE_b = \frac{s}{s_x\sqrt{n-1}} = \frac{\sqrt{\frac{\sum(y-\hat{y})^2}{n-2}}}{\sqrt{\sum(x-\bar{x})^2}} \)
Standard Error of Slope
Key definitions:

Linear regression: A statistical method that models the relationship between a dependent variable and one or more independent variables using a linear equation.

Population regression line: The true linear relationship in the population: μ_y = α + βx.

Sample regression line: The estimated relationship from sample data: ŷ = a + bx.

Residuals: The differences between observed and predicted values: e_i = y_i - ŷ_i.

R-squared: The proportion of variance in the dependent variable explained by the independent variable.

Step-by-step methodology:
  1. Check conditions: Verify linearity, independence, normality, and equal variance
  2. Fit regression line: Calculate sample slope (b) and intercept (a)
  3. Calculate standard error: Determine SE_b for the slope
  4. Formulate hypotheses: Define H₀ and H₁ about the population slope
  5. Calculate test statistic: Use t = (b - β₀) / SE_b
  6. Determine critical value: Find t-value based on α and df
  7. Make decision: Compare test statistic with critical value
  8. Construct confidence interval: Calculate b ± t*(SE_b)
  9. Interpret results: State conclusion in context
Tip 1: The standard error of the slope decreases as sample size increases.
Tip 2: A narrow confidence interval indicates precise estimation of the slope.
Tip 3: Outliers can greatly influence the regression line and inference.
Tip 4: Always plot the data to visually assess the linear relationship.
Common applications: Economics, psychology, medicine, engineering, social sciences.
Limitations: Assumes linear relationship, sensitive to outliers, cannot prove causation.
Key formulas and rules:

• Test statistic: t = (b - β₀) / SE_b

• Degrees of freedom: df = n - 2

• Confidence interval: b ± t*(SE_b)

• Standard error: SE_b = s/[s_x√(n-1)]

• Decision rule: Reject H₀ if |t| > t_critical

Questions & Answers

Question: Why do we use the t-distribution instead of the z-distribution when doing inference for the slope in linear regression?

Answer: We use the t-distribution for slope inference for several key reasons:

  • Unknown population standard deviation: In regression, we estimate the standard deviation of the residuals (σ) using the sample standard deviation (s), introducing additional uncertainty
  • Estimation of standard error: The standard error of the slope (SE_b) is estimated from the data, not known exactly
  • Small sample sizes: The t-distribution accounts for the extra variability when the sample size is small

The t-distribution has heavier tails than the z-distribution, which provides more conservative estimates when dealing with estimated standard errors. As the sample size increases, the t-distribution approaches the z-distribution.

In regression inference, we're estimating both the mean response and the variability around that mean, which requires using the t-distribution with df = n - 2 degrees of freedom.

This is similar to why we use t-tests for means when the population standard deviation is unknown, rather than z-tests.

Question: What's the difference between testing if the slope equals zero versus testing if it equals some other value?

Answer: Testing whether the slope equals zero versus another value serves different purposes:

  • Testing β = 0: Tests whether there is any linear relationship between x and y (most common test)
  • Testing β = β₀ (where β₀ ≠ 0): Tests whether the slope equals a specific theoretical or historical value

The test statistic formula remains the same: t = (b - β₀) / SE_b

For example, if theory predicts that each unit increase in x should increase y by 2 units, you might test H₀: β = 2 vs H₁: β ≠ 2.

Testing β = 0 is equivalent to testing whether the correlation coefficient ρ = 0, which determines if there's a significant linear relationship.

Testing against a non-zero value is useful when you want to compare your observed slope to a benchmark or theoretical expectation.

The interpretation changes: rejecting β = 0 means there's a relationship; rejecting β = β₀ means the slope differs from the specific value β₀.

Question: How do I interpret the confidence interval for the slope, and what does it tell me that the hypothesis test doesn't?

Answer: The confidence interval for the slope provides valuable information beyond what the hypothesis test tells us:

  • Range of plausible values: The CI gives all values for the population slope that are consistent with the data at the specified confidence level
  • Magnitude assessment: Shows how large or small the true slope might be
  • Practical significance: Helps determine if the relationship is meaningful in practice

For example, if the 95% CI for slope is (0.1, 0.3), we're 95% confident that the true slope falls between 0.1 and 0.3. This tells us not just that there's a significant relationship (hypothesis test), but also the likely range of the relationship strength.

The CI also reveals practical significance: if the interval includes values close to zero, the relationship might be statistically significant but practically trivial.

Additionally, if the CI doesn't contain zero, it confirms the hypothesis test result (significant relationship), but with more detail about the magnitude.

The CI provides more information about precision of the estimate and helps with decision-making about the practical implications of the relationship.