Residual: The difference between the observed value and the predicted value from the regression line. Residual = Observed - Predicted.
- Find the observed value (y-coordinate of data point)
- Substitute x-value into regression equation to find predicted value
- Calculate residual: Residual = Observed - Predicted
- Interpret the sign and magnitude of the residual
From the data point (4, 12), the observed value is y = 12.
Substitute x = 4 into the regression equation y = 2x + 3
ŷ = 2(4) + 3 = 8 + 3 = 11
Residual = Observed - Predicted = 12 - 11 = 1
The positive residual of 1 means the actual value (12) is 1 unit above the predicted value (11). The model underestimated this point.
A positive residual indicates the regression line lies below the actual data point. The point is above the regression line.
The residual is 1. This means the actual y-value (12) is 1 unit higher than the predicted y-value (11) from the regression line. The positive residual indicates the model slightly underestimated this particular data point.
• Residual Formula: Residual = Observed - Predicted
• Positive Residual: Actual value > Predicted value
• Negative Residual: Actual value < Predicted value
Residual Plot: A scatter plot with x-values on the horizontal axis and residuals on the vertical axis. Used to assess the appropriateness of a regression model.
For x = 2: ŷ = 0.5(2) + 2 = 3
For x = 4: ŷ = 0.5(4) + 2 = 4
For x = 6: ŷ = 0.5(6) + 2 = 5
For x = 8: ŷ = 0.5(8) + 2 = 6
For x = 10: ŷ = 0.5(10) + 2 = 7
Point (2,3): Residual = 3 - 3 = 0
Point (4,4): Residual = 4 - 4 = 0
Point (6,6): Residual = 6 - 5 = 1
Point (8,7): Residual = 7 - 6 = 1
Point (10,8): Residual = 8 - 7 = 1
Plot points: (2,0), (4,0), (6,1), (8,1), (10,1)
The residuals are scattered randomly around zero with no clear pattern, indicating a good linear fit.
Since residuals show no systematic pattern and are randomly distributed, the linear model appears appropriate for this data.
The residuals are 0, 0, 1, 1, 1. The residual plot shows a random scatter around zero with no systematic pattern, indicating that the linear model is appropriate for this data set.
• Residual Plot Construction: x-values vs residuals
• Good Model Indication: Random scatter around zero
• Bad Model Indication: Systematic patterns or curvature
Pattern Recognition: Systematic patterns in residual plots indicate that the linear model may not be appropriate for the data.
A U-shaped pattern in residuals indicates that the residuals are negative in the middle and positive at both ends (or vice versa).
This pattern suggests that the actual data follows a curved pattern that the linear model fails to capture. The model systematically underpredicts at the extremes and overpredicts in the middle (or vice versa).
The linear model is inadequate for this data. A non-linear model (such as quadratic) would better represent the relationship between variables.
Consider transforming the variables or using a non-linear regression model that can capture the curved relationship.
Other problematic patterns include funnel shapes (heteroscedasticity), increasing/decreasing trends, or cyclical patterns.
A U-shaped pattern in residuals indicates that the relationship between variables is non-linear. The linear model is inappropriate for this data. A non-linear model (such as quadratic) should be considered to better capture the curved relationship.
• Random Pattern: Good linear model fit
• Systematic Pattern: Linear model inadequate
• U-Shaped/Curved: Non-linear relationship exists
Residual: The difference between an observed value and the value predicted by the regression line (Residual = Observed - Predicted).
Residual Plot: A scatter plot with x-values on the horizontal axis and residuals on the vertical axis, used to assess model adequacy.
Positive Residual: When the observed value is greater than the predicted value (point lies above the regression line).
Negative Residual: When the observed value is less than the predicted value (point lies below the regression line).
Random Pattern: A desirable pattern in residual plots where residuals are randomly scattered around zero.
Systematic Pattern: An undesirable pattern indicating the model may be inappropriate for the data.
Model Adequacy: The extent to which a regression model appropriately represents the relationship in the data.
- Calculate Residuals: For each data point, compute residual = observed - predicted
- Create Residual Plot: Plot x-values vs residuals
- Identify Patterns: Look for systematic patterns or randomness
- Assess Model: Determine if linear model is appropriate
- Take Action: Consider alternative models if necessary
• Residual Formula: eᵢ = yᵢ - ŷᵢ (Observed minus Predicted)
• Sign Interpretation: Positive = Above line, Negative = Below line
• Sum Property: Σ(residuals) = 0 for least-squares regression line
• Good Model: Random scatter around zero with no pattern
• Bad Model: Systematic patterns indicate model inadequacy
Influential Point: A data point that significantly affects the position of the regression line. Outlier: A point that deviates substantially from the overall pattern.
For x = 5: ŷ = 2(5) + 1 = 11
Observed value is 15, predicted value is 11, so residual = 15 - 11 = 4
A residual of 4 is quite large compared to other residuals near zero, making this point potentially influential.
This point pulls the regression line upward, causing it to have a steeper slope than it would without this point.
Check if the point is a data entry error, investigate the circumstances of this observation, and consider running regression with and without this point.
The point (5, 15) is influential because it has a large residual of 4. This point pulls the regression line upward, affecting both slope and intercept. The point should be investigated for data accuracy and its impact on the model should be assessed by comparing regressions with and without it.
• Large Residuals: May indicate outliers or influential points
• Influential Points: Significantly affect regression line parameters
• Investigation: Check data accuracy and model assumptions
Business Application: Residuals in real-world contexts provide actionable insights about model performance and business outcomes.
Advertising = $50k = 50 thousand, so x = 50
Predicted sales: ŷ = 1.5(50) + 100 = 75 + 100 = 175 thousand = $175k
Actual sales = $180k, Predicted sales = $175k
Residual = 180 - 175 = 5 thousand = $5k
The positive residual of $5k means actual sales were $5,000 higher than predicted. The model underestimated this particular outcome.
This suggests the advertising campaign was more effective than the model predicted. Management should investigate why sales exceeded expectations.
A positive residual indicates good performance relative to expectations, but management should monitor if this pattern continues.
The residual is +$5,000, meaning actual sales were $5,000 higher than predicted. This indicates the advertising campaign was more successful than expected. Management should investigate factors contributing to this superior performance for future campaigns.
• Unit Consistency: Ensure all values use same units as model
• Business Interpretation: Connect residuals to actionable insights
• Performance Assessment: Positive residuals indicate better than expected outcomes
Residual: The difference between an observed value and the value predicted by the regression line. Formula: Residual = Observed - Predicted.
Residual Plot: A scatter plot with x-values on the horizontal axis and residuals on the vertical axis, used to assess the adequacy of a regression model.
Positive Residual: Occurs when the observed value is greater than the predicted value (the data point lies above the regression line).
Negative Residual: Occurs when the observed value is less than the predicted value (the data point lies below the regression line).
Random Pattern: A desirable characteristic in residual plots where residuals are randomly scattered around zero with no systematic pattern.
Systematic Pattern: An undesirable characteristic indicating that the regression model may not be appropriate for the data.
Model Adequacy: The degree to which a regression model appropriately represents the relationship between variables in the data.
Outlier: A data point that deviates significantly from the overall pattern of the data.
Influential Point: A data point that significantly affects the position of the regression line.
- Calculate Predicted Values: Use regression equation to find ŷ for each x-value
- Compute Residuals: Calculate eᵢ = yᵢ - ŷᵢ for each data point
- Create Residual Plot: Plot x-values vs residuals
- Identify Patterns: Look for systematic patterns or randomness
- Assess Model Fit: Determine if linear model is appropriate
- Check for Outliers: Identify points with unusually large residuals
- Interpret Results: Connect findings to the context of the problem
Residual Properties:
• Formula: eᵢ = yᵢ - ŷᵢ (always observed minus predicted)
• Sign Interpretation: Positive = above line, Negative = below line
• Sum Property: Σ(residuals) = 0 for least-squares line
Pattern Recognition:
• Good Model: Random scatter around zero with no systematic pattern
• Non-linear Relationship: Curved or U-shaped pattern
• Heteroscedasticity: Funnel-shaped pattern (variance changes)
• Trend: Increasing or decreasing pattern
Calculate Predicted: Use regression equation for each x-value
Find Residuals: Subtract predicted from observed for each point
Create Plot: x-values vs residuals
Check Randomness: Look for random scatter around zero
Identify Patterns: Note any systematic arrangements
Assess Fit: Determine if model is appropriate
Investigate Issues: Examine points with large residuals
Draw Conclusions: Interpret results in context
Standardized Residuals: Residuals divided by their standard deviation, allowing comparison across different scales.
Studentized Residuals: A type of standardized residual that accounts for the fact that residuals have different variances.
Homoscedasticity: The assumption that residuals have constant variance across all values of the predictor variable.
- Basic Residual Calculation: Compute eᵢ = yᵢ - ŷᵢ for all points
- Pattern Analysis: Examine residual plot for systematic patterns
- Scale Assessment: Check for constant variance (homoscedasticity)
- Outlier Detection: Identify points with large residuals
- Influence Assessment: Evaluate impact of extreme points
- Model Validation: Confirm assumptions are met
• Visual Inspection: Always examine residual plots visually to identify patterns
• Context Consideration: Interpret residuals in the context of the original problem
• Pattern Recognition: Distinguish between random scatter and systematic patterns
• Follow-up Action: Investigate points with large residuals or systematic patterns
• Model Validation: Use residuals to confirm regression assumptions are met
• Decision Making: Use residual analysis to guide model selection and improvements
Questions & Answers
Question: Why do we use residuals instead of just looking at the original scatter plot to assess model fit?
Answer: Residuals provide several advantages over examining the original scatter plot:
- Focus on errors: Residuals isolate the prediction errors, making it easier to see how well the model performs
- Remove trend: Original scatter plots show the overall relationship; residual plots focus only on deviations from the model
- Pattern detection: Subtle patterns in model inadequacy are more visible in residuals
- Constant scale: Residuals allow for consistent evaluation across different x-values
The original scatter plot shows the relationship between variables, while the residual plot shows how well the model captures that relationship. Both views are important for complete analysis.
Question: What does it mean when residuals form a curved pattern in the residual plot?
Answer: A curved pattern in residuals indicates that the relationship between variables is not linear. This means:
- Linear model inadequate: The straight line fails to capture the true relationship
- Non-linear relationship: The actual relationship may be quadratic, exponential, or another curved form
- Systematic error: The model consistently overpredicts in some regions and underpredicts in others
When you see a curved pattern, you should consider using a non-linear model or transforming the variables to better capture the relationship. This is one of the most important signals that your linear model is inappropriate.
Question: Is it bad if some residuals are positive and some are negative? Shouldn't they all be close to zero?
Answer: Having both positive and negative residuals is actually normal and expected! Here's why:
- Random variation: Some points naturally fall above the line (positive residuals) and some below (negative residuals)
- Model characteristic: For least-squares regression, residuals should average zero (some positive, some negative)
- Good sign: Mixed positive and negative residuals suggest the line is positioned appropriately
What matters is that residuals are randomly scattered around zero with no systematic pattern. It's normal for residuals to range from negative to positive values. The key is that they should be randomly distributed, not that they should all be close to zero.
If all residuals were positive or all negative, that would indicate a serious problem with the model!