Solved Exercises on Residuals and Residual Plots

Master residuals and residual plots: calculation, analysis, interpretation, and statistical assessment through these 5 detailed exercises.

Solution: Exercises 1 to 3
1 Basic Residual Calculation
Exercise 1
Given the regression equation y = 2.5x + 10, calculate the residuals for the following data points:

Data Points: (2, 15), (4, 20), (6, 27), (8, 30), (10, 38)

Then create a residual plot and interpret the results.
Definition:

Residual: The difference between the actual observed value (y_actual) and the predicted value (y_predicted) from the regression line. Residual = y_actual - y_predicted.

Residual Plot: A graph showing residuals on the y-axis versus the explanatory variable (x) on the x-axis. It helps assess the appropriateness of the linear regression model.

Good Linear Model: Residuals should be randomly scattered around zero with no systematic pattern.

Residual Calculation Method:
  1. Calculate predicted y-value using the regression equation for each x-value
  2. Subtract predicted y-value from actual y-value
  3. Residual = y_actual - y_predicted
  4. Plot residuals vs x-values
  5. Interpret the pattern of residuals
Step 1: Calculate Predicted Values

Using y = 2.5x + 10:

  • For x = 2: y_pred = 2.5(2) + 10 = 15
  • For x = 4: y_pred = 2.5(4) + 10 = 20
  • For x = 6: y_pred = 2.5(6) + 10 = 25
  • For x = 8: y_pred = 2.5(8) + 10 = 30
  • For x = 10: y_pred = 2.5(10) + 10 = 35
Step 2: Calculate Residuals

Residual = y_actual - y_predicted:

  • Point (2, 15): 15 - 15 = 0
  • Point (4, 20): 20 - 20 = 0
  • Point (6, 27): 27 - 25 = 2
  • Point (8, 30): 30 - 30 = 0
  • Point (10, 38): 38 - 35 = 3
Step 3: Create Residual Plot

Plot (x, residual): (2, 0), (4, 0), (6, 2), (8, 0), (10, 3)

Step 4: Analyze Residual Pattern

The residuals show a slight positive trend, with larger residuals at higher x-values. Most residuals are close to zero, but there's a pattern suggesting the model may be less accurate at higher values.

Step 5: Assess Model Quality

While most residuals are small, the pattern suggests the linear model may not perfectly capture the relationship, especially at higher x-values.

y = 2.5x + 10
Residual Plot
Residuals: [0, 0, 2, 0, 3]
Slight positive trend at higher x-values
Final answer:

The residuals are 0, 0, 2, 0, and 3. The residual plot shows most residuals close to zero, but with a slight positive trend at higher x-values, suggesting the model may be less accurate for larger values. Overall, the linear model provides a reasonable fit.

Applied rules:

• Residual Formula: Residual = y_actual - y_predicted

• Good Fit: Residuals randomly scattered around zero

• Pattern Detection: Look for systematic patterns in residuals

• Model Assessment: Random residuals indicate appropriate model

2 Residual Analysis for Model Assessment
Exercise 2
A company's regression model for predicting sales based on advertising expenditure is y = 1.8x + 25. The following data was collected:

Advertising (x): [5, 10, 15, 20, 25, 30, 35, 40]
Actual Sales (y): [34, 42, 52, 60, 68, 75, 82, 88]

Calculate residuals, create a residual plot, and assess the model's appropriateness.
Definition:

Model Appropriateness: Whether the chosen model (linear, in this case) adequately describes the relationship between variables.

Random Residuals: Residuals that show no systematic pattern, indicating a good model fit.

Patterned Residuals: Residuals that show systematic patterns (curved, increasing, etc.), indicating model inadequacy.

\text{Residual} = y_{\text{actual}} - y_{\text{predicted}}
Residual Formula
Step 1: Calculate Predicted Values

Using y = 1.8x + 25:

  • x = 5: y_pred = 1.8(5) + 25 = 34
  • x = 10: y_pred = 1.8(10) + 25 = 43
  • x = 15: y_pred = 1.8(15) + 25 = 52
  • x = 20: y_pred = 1.8(20) + 25 = 61
  • x = 25: y_pred = 1.8(25) + 25 = 70
  • x = 30: y_pred = 1.8(30) + 25 = 79
  • x = 35: y_pred = 1.8(35) + 25 = 88
  • x = 40: y_pred = 1.8(40) + 25 = 97
Step 2: Calculate Residuals

Residual = y_actual - y_predicted:

  • (5, 34): 34 - 34 = 0
  • (10, 42): 42 - 43 = -1
  • (15, 52): 52 - 52 = 0
  • (20, 60): 60 - 61 = -1
  • (25, 68): 68 - 70 = -2
  • (30, 75): 75 - 79 = -4
  • (35, 82): 82 - 88 = -6
  • (40, 88): 88 - 97 = -9
Step 3: Create Residual Plot

Plot (x, residual): (5, 0), (10, -1), (15, 0), (20, -1), (25, -2), (30, -4), (35, -6), (40, -9)

Step 4: Analyze Residual Pattern

The residuals show a clear negative trend - they become increasingly negative as x increases. This systematic pattern indicates that the linear model systematically underpredicts sales at higher advertising levels.

Step 5: Assess Model Quality

The systematic pattern in residuals indicates that the linear model is not appropriate for this data. A non-linear model might be more suitable.

y = 1.8x + 25
Residual Plot
Residuals show negative trend: [0, -1, 0, -1, -2, -4, -6, -9]
Linear model inadequate - consider non-linear alternative
Final answer:

The residuals are [0, -1, 0, -1, -2, -4, -6, -9]. The residual plot shows a clear negative trend, indicating that the linear model systematically underpredicts sales at higher advertising levels. The model is not appropriate for this data, suggesting a non-linear relationship may be more suitable.

Applied rules:

• Model Assessment: Random residuals indicate good model fit

• Pattern Recognition: Systematic patterns indicate model inadequacy

• Non-linear Indicators: Curved residual patterns suggest non-linear models

• Improvement Strategy: Consider different model types when residuals show patterns

3 Detecting Outliers with Residuals
Exercise 3
A dataset of temperature and ice cream sales produced the regression equation y = 0.8x - 5. The data points are:

Temperature (x): [70, 72, 75, 78, 80, 82, 85, 88, 90, 92, 95, 98]
Sales (y): [51, 53, 58, 62, 65, 67, 70, 74, 76, 78, 80, 105]

Calculate residuals and identify any outliers using the residual plot.
Definition:

Outlier: A data point that deviates significantly from the overall pattern of the data. In regression, outliers often have large residuals.

Residual-Based Outlier Detection: Identifying outliers by finding data points with residuals that are unusually large in absolute value.

Typical Threshold: Residuals with absolute values greater than 2 standard deviations from the mean (which is 0 for residuals).

Step 1: Calculate Predicted Values

Using y = 0.8x - 5:

  • x = 70: y_pred = 0.8(70) - 5 = 51
  • x = 72: y_pred = 0.8(72) - 5 = 52.6
  • x = 75: y_pred = 0.8(75) - 5 = 55
  • x = 78: y_pred = 0.8(78) - 5 = 57.4
  • x = 80: y_pred = 0.8(80) - 5 = 59
  • x = 82: y_pred = 0.8(82) - 5 = 60.6
  • x = 85: y_pred = 0.8(85) - 5 = 63
  • x = 88: y_pred = 0.8(88) - 5 = 65.4
  • x = 90: y_pred = 0.8(90) - 5 = 67
  • x = 92: y_pred = 0.8(92) - 5 = 68.6
  • x = 95: y_pred = 0.8(95) - 5 = 71
  • x = 98: y_pred = 0.8(98) - 5 = 73.4
Step 2: Calculate Residuals

Residual = y_actual - y_predicted:

  • (70, 51): 51 - 51 = 0
  • (72, 53): 53 - 52.6 = 0.4
  • (75, 58): 58 - 55 = 3
  • (78, 62): 62 - 57.4 = 4.6
  • (80, 65): 65 - 59 = 6
  • (82, 67): 67 - 60.6 = 6.4
  • (85, 70): 70 - 63 = 7
  • (88, 74): 74 - 65.4 = 8.6
  • (90, 76): 76 - 67 = 9
  • (92, 78): 78 - 68.6 = 9.4
  • (95, 80): 80 - 71 = 9
  • (98, 105): 105 - 73.4 = 31.6
Step 3: Identify Potential Outliers

The residual for the point (98, 105) is 31.6, which is significantly larger than all other residuals. This suggests this point is an outlier.

Step 4: Create Residual Plot

Plot (x, residual): Most residuals are between -1 and 10, except one at (98, 31.6)

Step 5: Investigate the Outlier

The point (98, 105) has a residual of 31.6, which is much larger than the next largest residual (9.4). This suggests either an error in data collection or an unusual circumstance at this temperature.

Step 6: Assess Impact

This outlier significantly affects the regression line. It may indicate an unusual event (like a major festival) or a data entry error.

y = 0.8x - 5
Residual Plot
Outlier detected: Point (98, 105) with residual = 31.6
Residuals for other points range from 0 to 9.4
Final answer:

The point (98, 105) is an outlier with a residual of 31.6, which is significantly larger than other residuals (range: 0 to 9.4). This suggests either an unusual event occurred at 98°F or there was an error in data collection. The outlier should be investigated and possibly excluded from analysis.

Applied rules:

• Outlier Detection: Large residuals indicate potential outliers

• Threshold: Residuals > 2-3 standard deviations suggest outliers

• Investigation: Always investigate outliers before removing them

• Impact Assessment: Evaluate how outliers affect regression results

Key Concepts and Statistical Measures for Residuals and Residual Plots
\text{Residual} = y_{\text{actual}} - y_{\text{predicted}}
Residual Formula
Sum of Residuals
\sum \text{residuals} = 0
For least squares regression
Standard Error
\text{SE} = \sqrt{\frac{\sum \text{residuals}^2}{n-2}}
Estimate of prediction error
Residual Mean
\bar{e} = 0
Mean of residuals
Key definitions:

Residual: The difference between the actual observed value and the value predicted by the regression line. Residual = y_actual - y_predicted. It represents the error in prediction for each data point.

Residual Plot: A scatterplot of residuals versus the explanatory variable (x-values). It helps assess the adequacy of the regression model and identify patterns in the errors.

Good Linear Model: A model where residuals are randomly scattered around zero with no systematic pattern, constant variance, and no outliers.

Systematic Patterns: Non-random structures in residual plots that indicate model inadequacy, such as curves, increasing/decreasing trends, or funnel shapes.

Heteroscedasticity: When the spread of residuals changes across the range of x-values, violating the assumption of constant variance.

Homoscedasticity: When the spread of residuals remains constant across the range of x-values, indicating constant variance.

Outlier: A data point with a residual that is unusually large in absolute value, potentially affecting the regression model.

Residual Analysis Methodology:
  1. Calculate Predicted Values: Use regression equation to find y_predicted for each x_value
  2. Calculate Residuals: Subtract predicted from actual values
  3. Create Residual Plot: Plot residuals vs x-values
  4. Assess Randomness: Check for random scatter around zero
  5. Check for Patterns: Look for systematic trends, curves, or shapes
  6. Identify Outliers: Find residuals with unusually large absolute values
  7. Assess Variance: Check if residual spread is constant
  8. Model Appropriateness: Determine if linear model is suitable
Tip 1: The sum of all residuals in least squares regression equals zero.
Tip 2: Look for patterns in residual plots - randomness indicates good fit.
Tip 3: Always investigate outliers before deciding to remove them.
Tip 4: Constant spread of residuals indicates homoscedasticity.
Tip 5: Curved patterns in residuals suggest non-linear relationships.
Common errors: Misinterpreting random scatter as a pattern, failing to check for heteroscedasticity, not investigating outliers, confusing correlation with causation, misreading residual plots, calculation errors in residuals, not considering non-linear alternatives when residuals show patterns.
Exam preparation: Practice calculating residuals, learn to interpret residual plots, understand what patterns indicate, know when to question model appropriateness, recognize signs of outliers, understand the difference between homoscedasticity and heteroscedasticity.
Essential Formulas and Rules:

• Residual Formula: Residual = y_actual - y_predicted

• Sum of Residuals: Σ(residuals) = 0 for least squares regression

• Mean of Residuals: Mean = 0 for least squares regression

• Standard Error: SE = √[Σ(residuals)² / (n-2)]

• Good Fit Criteria: Random scatter around zero, constant variance, no outliers

• Bad Fit Indicators: Systematic patterns, increasing/decreasing trends, funnel shapes

Solution: Exercises 4 to 5
4 Advanced Residual Pattern Recognition
Exercise 4
A study on plant growth produced the following data with the regression equation y = 0.5x + 10:

Time (weeks) (x): [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]
Height (cm) (y): [10.2, 11.8, 13.5, 15.8, 18.2, 20.5, 22.8, 24.5, 25.8, 26.2, 26.8, 27.1]

Calculate residuals and identify the pattern. What does this suggest about the model?
Definition:

Non-linear Pattern: A systematic curved pattern in residuals that suggests the relationship between variables is not linear.

Growth Curve: A pattern where growth starts rapidly but slows over time, often following a logarithmic or sigmoidal curve.

Step 1: Calculate Predicted Values

Using y = 0.5x + 10:

  • x = 1: y_pred = 0.5(1) + 10 = 10.5
  • x = 2: y_pred = 0.5(2) + 10 = 11
  • x = 3: y_pred = 0.5(3) + 10 = 11.5
  • x = 4: y_pred = 0.5(4) + 10 = 12
  • x = 5: y_pred = 0.5(5) + 10 = 12.5
  • x = 6: y_pred = 0.5(6) + 10 = 13
  • x = 7: y_pred = 0.5(7) + 10 = 13.5
  • x = 8: y_pred = 0.5(8) + 10 = 14
  • x = 9: y_pred = 0.5(9) + 10 = 14.5
  • x = 10: y_pred = 0.5(10) + 10 = 15
  • x = 11: y_pred = 0.5(11) + 10 = 15.5
  • x = 12: y_pred = 0.5(12) + 10 = 16
Step 2: Calculate Residuals

Residual = y_actual - y_predicted:

  • (1, 10.2): 10.2 - 10.5 = -0.3
  • (2, 11.8): 11.8 - 11 = 0.8
  • (3, 13.5): 13.5 - 11.5 = 2.0
  • (4, 15.8): 15.8 - 12 = 3.8
  • (5, 18.2): 18.2 - 12.5 = 5.7
  • (6, 20.5): 20.5 - 13 = 7.5
  • (7, 22.8): 22.8 - 13.5 = 9.3
  • (8, 24.5): 24.5 - 14 = 10.5
  • (9, 25.8): 25.8 - 14.5 = 11.3
  • (10, 26.2): 26.2 - 15 = 11.2
  • (11, 26.8): 26.8 - 15.5 = 11.3
  • (12, 27.1): 27.1 - 16 = 11.1
Step 3: Analyze Residual Pattern

The residuals start negative, become increasingly positive, reach a peak around x = 9, then slightly decrease. This creates a curved pattern, suggesting the relationship is not linear.

Step 4: Interpret the Pattern

The curved residual pattern indicates that a linear model is not appropriate. The plant growth follows a curve - rapid growth initially, then slowing down, which is typical of biological growth processes.

Step 5: Suggest Alternative Model

A logarithmic or sigmoidal model would likely provide a better fit for this type of growth data.

y = 0.5x + 10
Residual Plot
Residuals show curved pattern: initially increasing then decreasing
Linear model inadequate for biological growth data
Final answer:

The residuals show a curved pattern, initially increasing then decreasing, indicating that the linear model is not appropriate for this biological growth data. The plant growth follows a curve with rapid initial growth that slows over time. A non-linear model (logarithmic or sigmoidal) would provide a better fit.

Applied rules:

• Pattern Recognition: Curved residuals indicate non-linear relationship

• Biological Processes: Growth often follows non-linear patterns

• Model Selection: Choose model type based on residual analysis

• Domain Knowledge: Consider the nature of the phenomenon being modeled

5 Comprehensive Residual Analysis
Exercise 5
A manufacturing company collected data on production volume and cost per unit. The regression equation is y = -0.05x + 25:

Volume (1000s) (x): [10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65]
Cost/unit ($) (y): [20.2, 19.8, 18.5, 17.2, 16.8, 16.5, 16.2, 16.8, 17.5, 18.2, 19.0, 19.8]

Perform a complete residual analysis and assess model appropriateness.
Definition:

Comprehensive Residual Analysis: A thorough examination of residuals including pattern recognition, outlier detection, variance assessment, and model appropriateness evaluation.

Economies of Scale: The concept that cost per unit decreases with increased production volume, which may not continue indefinitely.

Step 1: Calculate Predicted Values

Using y = -0.05x + 25:

  • x = 10: y_pred = -0.05(10) + 25 = 24.5
  • x = 15: y_pred = -0.05(15) + 25 = 24.25
  • x = 20: y_pred = -0.05(20) + 25 = 24
  • x = 25: y_pred = -0.05(25) + 25 = 23.75
  • x = 30: y_pred = -0.05(30) + 25 = 23.5
  • x = 35: y_pred = -0.05(35) + 25 = 23.25
  • x = 40: y_pred = -0.05(40) + 25 = 23
  • x = 45: y_pred = -0.05(45) + 25 = 22.75
  • x = 50: y_pred = -0.05(50) + 25 = 22.5
  • x = 55: y_pred = -0.05(55) + 25 = 22.25
  • x = 60: y_pred = -0.05(60) + 25 = 22
  • x = 65: y_pred = -0.05(65) + 25 = 21.75
Step 2: Calculate Residuals

Residual = y_actual - y_predicted:

  • (10, 20.2): 20.2 - 24.5 = -4.3
  • (15, 19.8): 19.8 - 24.25 = -4.45
  • (20, 18.5): 18.5 - 24 = -5.5
  • (25, 17.2): 17.2 - 23.75 = -6.55
  • (30, 16.8): 16.8 - 23.5 = -6.7
  • (35, 16.5): 16.5 - 23.25 = -6.75
  • (40, 16.2): 16.2 - 23 = -6.8
  • (45, 16.8): 16.8 - 22.75 = -5.95
  • (50, 17.5): 17.5 - 22.5 = -5
  • (55, 18.2): 18.2 - 22.25 = -4.05
  • (60, 19.0): 19.0 - 22 = -3
  • (65, 19.8): 19.8 - 21.75 = -1.95
Step 3: Create Residual Plot

Plot residuals vs x-values showing a U-shaped pattern

Step 4: Analyze Residual Pattern

The residuals form a U-shape: negative at low volumes, most negative around middle volumes, then becoming less negative at higher volumes. This suggests a quadratic relationship rather than linear.

Step 5: Assess Model Appropriateness

The systematic U-shaped pattern in residuals indicates that the linear model is not appropriate. The data suggests economies of scale initially, but diseconomies of scale at very high volumes.

Step 6: Suggest Improvements

A quadratic model (y = ax² + bx + c) would better capture the relationship between production volume and cost per unit.

y = -0.05x + 25
Residual Plot
Residuals show U-shaped pattern
Quadratic model more appropriate than linear
Final answer:

The residuals show a clear U-shaped pattern, indicating that the linear model is not appropriate. The data suggests that cost per unit initially decreases with volume (economies of scale), reaches a minimum, then begins to increase (diseconomies of scale at very high volumes). A quadratic model would better capture this relationship.

Applied rules:

• Pattern Recognition: U-shaped residuals suggest quadratic relationship

• Business Application: Understand economic principles in modeling

• Model Improvement: Suggest appropriate alternatives based on patterns

• Comprehensive Analysis: Consider all aspects of residual behavior

Detailed Summary: Residuals and Residual Plots
\text{Residual} = y_{\text{actual}} - y_{\text{predicted}}
Core Residual Formula
Key definitions:

Residual: The difference between the actual observed value and the value predicted by the regression line. It represents the error in prediction for each data point. Residual = y_actual - y_predicted. This measures how far each data point deviates from the regression line.

Residual Plot: A scatterplot that displays residuals on the vertical axis versus the explanatory variable (x-values) on the horizontal axis. It provides a visual tool for assessing the appropriateness of the regression model.

Least Squares Regression: A method of fitting a regression line that minimizes the sum of the squares of the residuals. This ensures the best possible linear fit to the data.

Good Linear Model: A model where residuals exhibit random scatter around zero with no systematic patterns, constant variance (homoscedasticity), and no extreme outliers.

Homoscedasticity: The condition where the variance of residuals remains constant across all values of the explanatory variable. This is an important assumption in regression analysis.

Heteroscedasticity: The condition where the variance of residuals changes across the range of the explanatory variable, violating a key regression assumption.

Systematic Patterns: Non-random structures in residual plots that indicate the regression model is not appropriate for the data, such as curves, trends, or funnel shapes.

Outlier: A data point with a residual that is unusually large in absolute value, potentially having a disproportionate influence on the regression model.

Standard Error of the Estimate: A measure of the accuracy of predictions made by the regression model, calculated as the standard deviation of the residuals.

Complete Residual Analysis Process:
  1. Data Preparation: Ensure regression equation is calculated correctly
  2. Predicted Value Calculation: Use regression equation to find y_predicted for each x_value
  3. Residual Calculation: Compute residual = y_actual - y_predicted for each point
  4. Residual Plot Creation: Plot residuals vs x-values on a scatterplot
  5. Randomness Assessment: Check if residuals are randomly scattered around zero
  6. Pattern Detection: Look for systematic trends, curves, or shapes in residuals
  7. Variance Assessment: Check if residual spread remains constant across x-values
  8. Outlier Identification: Find residuals with unusually large absolute values
  9. Model Appropriateness: Determine if linear model is suitable based on residual analysis
  10. Alternative Model Consideration: Suggest different models if linear is inadequate
  11. Documentation: Record findings and recommendations
  12. Validation: Consider the practical context of the analysis
Tip 1: The sum of all residuals in least squares regression always equals zero.
Tip 2: Look for randomness in residual plots - systematic patterns indicate model problems.
Tip 3: Always investigate outliers before deciding to remove them from analysis.
Tip 4: Constant spread of residuals indicates homoscedasticity, which is desirable.
Tip 5: Curved patterns in residuals suggest non-linear relationships may be more appropriate.
Tip 6: Consider the context and domain knowledge when interpreting residual patterns.
Common errors: Misinterpreting random scatter as a pattern, failing to check for heteroscedasticity, not investigating outliers, confusing correlation with causation, misreading residual plots, calculation errors in residuals, not considering non-linear alternatives when residuals show patterns, ignoring the context of the data, failing to validate model assumptions.
Exam preparation: Practice calculating residuals, learn to interpret residual plots, understand what patterns indicate, know when to question model appropriateness, recognize signs of outliers, understand the difference between homoscedasticity and heteroscedasticity, distinguish between various types of systematic patterns, know when to suggest alternative models.
Essential Formulas and Rules:

• Residual Formula: Residual = y_actual - y_predicted

• Sum of Residuals: Σ(residuals) = 0 for least squares regression

• Mean of Residuals: Mean = 0 for least squares regression

• Standard Error: SE = √[Σ(residuals)² / (n-2)]

• Good Fit Criteria: Random scatter around zero, constant variance, no outliers

• Bad Fit Indicators: Systematic patterns, increasing/decreasing trends, funnel shapes, curvature

• Homoscedasticity: Constant residual variance across x-values

• Heteroscedasticity: Changing residual variance across x-values

Residual Analysis Framework
📊
Step 1: Calculate
Compute predicted values and residuals
Residual = Actual - Predicted
Step 2: Plot
Create residual vs x-value plot
Residuals on Y-axis, X on X-axis
Step 3: Examine
Look for patterns in residuals
Random = Good, Pattern = Problem
Step 4: Identify
Find outliers and unusual points
Large |residual| = Potential outlier
Step 5: Assess
Evaluate model appropriateness
Random scatter = Adequate model
Step 6: Improve
Suggest model improvements
Patterns → Non-linear model
Analysis Process
1
Calculate
2
Plot
3
Examine
4
Identify
5
Assess
6
Improve
Pattern Recognition
Random Scatter: Good Model
Curved Pattern: Non-linear
Funnel Shape: Heteroscedastic
Trend: Missing Variable
Always validate with residuals!

Questions & Answers

Question: How do I interpret a residual plot to determine if my linear model is appropriate?

Answer: Look for these patterns in your residual plot:

  • Good Model: Residuals randomly scattered around zero with constant spread
  • Non-linear Relationship: Curved pattern in residuals
  • Heteroscedasticity: Funnel shape or changing spread
  • Missing Variable: Systematic increasing or decreasing trend
  • Outliers: Individual points far from the zero line

Random scatter around zero with constant variance indicates a good linear model fit.

Question: What is the difference between homoscedasticity and heteroscedasticity?

Answer: These terms describe the variance of residuals:

  • Homoscedasticity: The spread of residuals remains constant across all values of the explanatory variable. This is ideal for regression analysis.
  • Heteroscedasticity: The spread of residuals changes across the range of the explanatory variable. This violates regression assumptions.

In a residual plot, homoscedasticity looks like a consistent band of points around zero, while heteroscedasticity shows varying spread (often funnel-shaped).

Question: Why do we say the sum of residuals equals zero in least squares regression?

Answer: This is a mathematical property of least squares regression:

  • The least squares method mathematically forces the sum of residuals to equal zero
  • This occurs because the regression line passes through the point (x̄, ȳ)
  • It's a constraint built into the calculation of the regression coefficients
  • The positive and negative residuals balance each other out

This property helps ensure that the regression line is centered properly in the data cloud.

Question: How do I determine if a residual indicates an outlier?

Answer: There are several approaches to identify outlier residuals:

  • Rule of Thumb: Residuals with absolute values greater than 2 standard deviations
  • Visual Inspection: Points that stand out significantly from other residuals
  • Standardized Residuals: Residuals divided by their standard deviation (values > 2 or < -2 suggest outliers)
  • Context: Consider the practical significance of large residuals

Always investigate outliers for potential data errors before removing them from analysis.

Question: What should I do if my residual plot shows a clear pattern?

Answer: A pattern in residuals indicates your model needs improvement:

  • Curved Pattern: Consider polynomial regression or other non-linear models
  • Heteroscedasticity: Consider data transformation or weighted regression
  • Systematic Trend: Consider adding other variables to your model
  • Funnel Shape: Indicates changing variance, may need transformation

The pattern in residuals provides clues about how to improve your model specification.