Residual: The difference between the observed y-value and the predicted y-value from the regression line
Formula: Residual = Observed y - Predicted y = y - ŷ
Positive Residual: Observed value is above the regression line
Negative Residual: Observed value is below the regression line
- Identify the observed x and y values
- Use the regression equation to predict y
- Calculate residual = observed y - predicted y
- Interpret the sign and magnitude of the residual
Observed x = 10, Observed y = 73
ŷ = 50 + 2(10) = 50 + 20 = 70
Residual = Observed y - Predicted y = 73 - 70 = 3
Positive residual of 3 means the actual value is 3 units above the predicted value
Observed: 73
Residual: +3
The residual is +3. This means that for x = 10, the observed y-value (73) is 3 units above what the regression line predicted (70). The positive residual indicates that the actual value is above the regression line, suggesting this data point has a higher y-value than expected based on the linear relationship.
• Residual formula: e = y - ŷ
• Positive residual: Actual value above regression line
• Negative residual: Actual value below regression line
Residual Plot: A scatterplot of residuals vs. predicted values (or x-values)
Random Pattern: Indicative of a good linear model fit
Systematic Pattern: Indicates poor model fit
- Examine the overall pattern of residuals
- Look for randomness or systematic patterns
- Identify any trends or curvatures
- Assess the appropriateness of the linear model
Residuals form a clear curved pattern
Curved pattern indicates systematic departure from randomness
Systematic pattern suggests linear model is inappropriate
Non-linear model might be more appropriate
Model: Linear is inappropriate
Alternative: Non-linear model needed
A curved pattern in the residual plot indicates that the linear regression model is not appropriate for the data. The systematic curvature suggests that the relationship between the variables is not linear, and a non-linear model (such as quadratic or exponential) might better capture the relationship. The linear model fails to account for the curvature in the data.
• Good model: Residuals randomly scattered around zero
• Poor model: Systematic patterns in residuals
• Curved pattern: Non-linear relationship
Random Scattering: Residuals show no systematic pattern
Homoscedasticity: Constant variance of residuals across all x-values
Linearity Assumption: Linear model is appropriate
- Look for overall randomness in residual placement
- Check for constant spread (homoscedasticity)
- Assess if residuals are centered around zero
- Determine if the linear model is appropriate
Residuals are randomly scattered with no pattern
Residuals maintain consistent spread across x-values
Residuals are centered around zero line
Random pattern indicates good linear model fit
Model: Appropriate
Assumptions: Met
A random scattering of residuals around the zero line with no clear pattern indicates that the linear regression model is appropriate for the data. This suggests that the linear relationship adequately captures the relationship between the variables, and the assumptions of linear regression (linearity, constant variance, independence) are satisfied.
• Random pattern: Linear model is appropriate
• Constant spread: Homoscedasticity assumption met
• Centered residuals: No systematic bias
Outlier: A data point with a large residual (unusual y-value given its x-value)
Influential Point: A data point that significantly affects the regression line
High Leverage Point: A data point with an unusual x-value
- Identify the point with large residual
- Investigate the cause of the large residual
- Determine if it's an error or genuine observation
- Assess the impact on the regression model
- Decide whether to keep, modify, or remove the point
Point with very large residual stands out in residual plot
Determine if it's a data entry error or genuine observation
Calculate regression with and without the point
Keep if genuine, remove if error, consider robust methods
Action: Investigate cause
Impact: Assess on model
A point with a very large residual is an outlier that may indicate an unusual observation. It should be investigated to determine if it's a data error or a genuine observation. If it's an error, it should be corrected or removed. If it's genuine, consider whether it represents a special case or if the model needs to be reconsidered. Always assess the impact of such points on the regression results.
• Investigate outliers: Don't automatically remove
• Assess impact: Compare models with and without outlier
• Document decisions: Record rationale for handling outliers
Heteroscedasticity: Non-constant variance of residuals across x-values
Homoscedasticity: Constant variance of residuals across x-values
Violated Assumption: Equal variance assumption of linear regression
- Identify the pattern of increasing/decreasing spread
- Confirm the violation of equal variance assumption
- Assess the severity of the violation
- Consider appropriate remedial actions
Residuals fan out as x increases (conical pattern)
Equal variance assumption is violated (heteroscedasticity)
Standard errors and confidence intervals may be inaccurate
Transform variables, use weighted regression, or robust methods
Issue: Heteroscedasticity
Remedies: Transform or weighted regression
The fanning out pattern indicates heteroscedasticity (non-constant variance), violating the equal variance assumption of linear regression. This means the variability of residuals increases with higher x-values. The implications include potentially biased standard errors, unreliable confidence intervals, and invalid hypothesis tests. Remedies include transforming variables, using weighted least squares regression, or employing robust regression techniques.
• Homoscedasticity: Equal variance assumption required
• Consequences: Invalid inferences when violated
• Remedies: Variable transformation or alternative methods
Random: Good linear model fit
Curved: Non-linear relationship
Fan out: Heteroscedasticity
Funnel: Unequal variance
Clusters: Subgroups in data
Residual Analysis: A fundamental diagnostic tool for evaluating the adequacy of a regression model. Residuals should be randomly distributed with constant variance around zero.
Model Assumptions: Linear regression assumes linearity, independence, homoscedasticity (constant variance), and normality of residuals.
Diagnostic Plots: Residual plots, normal probability plots, and scatterplots help assess model assumptions.
- Residual Calculation: Compute eᵢ = yᵢ - ŷᵢ for each data point
- Residual Plot: Plot residuals vs. predicted values or x-values
- Pattern Recognition: Identify systematic patterns in residuals
- Assumption Testing: Verify regression assumptions
- Model Improvement: Consider alternatives if assumptions violated
• Residual formula: e = y - ŷ
• Good model: Random residuals around zero
• Constant variance: Homoscedasticity required
• No correlation: Residuals vs x should show no pattern