Least-Squares Regression Line: The line that minimizes the sum of squared residuals (vertical distances from points to line)
Regression Equation: y = a + bx where b is slope and a is y-intercept
Slope (b): b = r(s_y/s_x) or b = [Σ(x-x̄)(y-ȳ)]/[Σ(x-x̄)²]
Y-intercept (a): a = ȳ - bx̄
- Calculate means (x̄, ȳ)
- Calculate standard deviations (s_x, s_y)
- Calculate correlation coefficient (r)
- Calculate slope: b = r(s_y/s_x)
- Calculate y-intercept: a = ȳ - bx̄
- Write equation: y = a + bx
x̄ = (2 + 3 + 4 + 5 + 6) ÷ 5 = 20 ÷ 5 = 4
ȳ = (65 + 70 + 75 + 80 + 85) ÷ 5 = 375 ÷ 5 = 75
s_x = √[Σ(x-x̄)²/(n-1)] = √[(4+1+0+1+4)/4] = √2.5 ≈ 1.58
s_y = √[Σ(y-ȳ)²/(n-1)] = √[(100+25+0+25+100)/4] = √62.5 ≈ 7.91
r = Σ[(x-x̄)(y-ȳ)]/[√Σ(x-x̄)² × √Σ(y-ȳ)²]
r = [(-2)(-10) + (-1)(-5) + (0)(0) + (1)(5) + (2)(10)]/[√10 × √250]
r = [20 + 5 + 0 + 5 + 20]/√2500 = 50/50 = 1
b = r(s_y/s_x) = 1 × (7.91/1.58) ≈ 5
a = ȳ - bx̄ = 75 - 5(4) = 75 - 20 = 55
Y-intercept (a) = 55
Regression equation: y = 55 + 5x
The equation of the least-squares regression line is y = 55 + 5x. This means that for each additional hour of studying, the predicted test score increases by 5 points. The line passes through the point (4, 75), which is the centroid of the data.
• Least squares: Minimizes sum of squared residuals
• Regression line: Always passes through (x̄, ȳ)
• Perfect correlation: r = 1 means all points on the line
Slope (b): Change in y for each unit increase in x
Y-intercept (a): Predicted y-value when x = 0
Interpretation: Making practical sense of regression parameters
- Identify the variables and their units
- Interpret slope as change in y per unit change in x
- Interpret y-intercept in context (consider if x = 0 is meaningful)
- Express interpretations in practical terms
Explanatory variable: height (cm)
Response variable: weight (kg)
Slope = 1.2 kg/cm
For each additional cm in height, weight increases by 1.2 kg
Y-intercept = -100 kg
When height = 0 cm, predicted weight = -100 kg
This is not practically meaningful since height cannot be 0
Y-intercept: -100 kg
Meaningful only for practical x-range
The slope of 1.2 kg/cm means that for each additional centimeter of height, weight is predicted to increase by 1.2 kilograms. The y-intercept of -100 kg represents the predicted weight when height is 0 cm, which is not practically meaningful since humans don't have 0 cm height. The intercept is just a mathematical artifact of the regression line.
• Slope interpretation: Change in y per unit change in x
• Y-intercept: May not be practically meaningful
• Context: Always interpret in practical terms
Interpolation: Predicting within the range of observed x-values
Extrapolation: Predicting outside the range of observed x-values
Residual: Difference between observed and predicted y-values- Identify the regression equation
- Substitute the x-value into the equation
- Determine if prediction is interpolation or extrapolation
- Consider the reliability of the prediction
y = 55 + 5x
y = 55 + 5(4.5) = 55 + 22.5 = 77.5
Observed x-range: 2 to 6 hours
4.5 hours is within this range → interpolation
Predicted score = 77.5 points
Type: Interpolation
X-value: 4.5 hours (within range 2-6)
The predicted test score for a student who studies 4.5 hours is 77.5 points. This is interpolation since 4.5 hours falls within the observed range of 2 to 6 hours. Interpolation is generally more reliable than extrapolation because it's based on the observed data pattern.
• Interpolation: More reliable than extrapolation
• Substitution: Replace x with desired value in equation
• Range consideration: Check if prediction is within observed data range
Residual: Observed y - Predicted y (actual - predicted)
Positive Residual: Actual value is above the regression line
Negative Residual: Actual value is below the regression line
Residual Analysis: Checking how well the line fits the data
- Identify the observed x and y values
- Use the regression equation to predict y
- Calculate residual = observed y - predicted y
- Interpret the sign and magnitude of the residual
Observed x = 3 hours, Observed y = 72 points
Predicted y = 55 + 5(3) = 55 + 15 = 70 points
Residual = Observed y - Predicted y = 72 - 70 = 2 points
Positive residual of 2 points means the student scored 2 points higher than predicted
Observed: 72 points
Residual: +2 points
The residual is +2 points. This means the student scored 2 points higher than what the regression line predicted for 3 hours of studying. A positive residual indicates that the actual value is above the regression line, suggesting this student performed better than expected for the amount of time spent studying.
• Residual formula: Residual = Observed - Predicted
• Positive residual: Actual value above regression line
• Negative residual: Actual value below regression line
Coefficient of Determination (r²): Proportion of variation in y explained by the regression line
Formula: r² = (correlation coefficient)²
Interpretation: Percentage of variation in y explained by x
- Identify the correlation coefficient (r)
- Calculate r² by squaring r
- Convert to percentage
- Interpret as percentage of explained variation
r = 0.8
r² = (0.8)² = 0.64
0.64 = 64%
64% of the variation in y is explained by the regression line
Percentage explained: 64%
Unexplained: 36%
The coefficient of determination is r² = 0.64, or 64%. This means that 64% of the variation in the response variable (y) can be explained by the linear relationship with the explanatory variable (x). The remaining 36% of the variation is due to other factors not included in the model or random variation.
• r² calculation: Square the correlation coefficient
• Interpretation: Proportion of explained variation
• Range: 0 ≤ r² ≤ 1 (or 0% to 100%)
Regression line: y = a + bx
Slope: b = r(s_y/s_x)
Y-intercept: a = ȳ - bx̄
Residual: e = y - ŷ
Coefficient of determination: r²
Least-Squares Regression Line: The unique line that minimizes the sum of squared residuals. It is the "best fit" line that makes the vertical distances from points to the line as small as possible when squared and summed.
Regression Assumptions: Linear relationship, independence of residuals, constant variance (homoscedasticity), and normality of residuals.
Extrapolation Risk: Making predictions outside the range of observed x-values is risky because the linear relationship may not continue.
- Line Calculation: Calculate slope and intercept using appropriate formulas
- Parameter Interpretation: Understand practical meaning of slope and intercept
- Prediction: Substitute x-values into the equation
- Residual Analysis: Calculate and interpret residuals
- Goodness of Fit: Evaluate using r² and residual plots
• Least squares: Minimizes sum of squared residuals
• Regression line: Passes through (x̄, ȳ)
• Residual calculation: e = y - ŷ
• r² interpretation: Proportion of variation explained