Residual: The difference between the observed value (actual y) and the predicted value (from the model). Residual = Observed - Predicted.
- Find the predicted value using the model equation
- Calculate the residual using the formula
- Interpret the sign and magnitude of the residual
- Evaluate the model's accuracy
Given model: y = 2x + 3
At x = 4: Predicted y = 2(4) + 3 = 8 + 3 = 11
Observed y = 10 (from the data point)
Predicted y = 11 (calculated above)
Residual = Observed - Predicted = 10 - 11 = -1
A residual of -1 means the model overpredicted by 1 unit.
The actual value (10) is 1 unit below the predicted value (11).
Since |residual| = 1 is small, the model fits this point reasonably well.
The predicted value is 11, and the residual is -1. The model overpredicts by 1 unit.
• Residual formula: Residual = Observed - Predicted
• Positive residual: Model underpredicts (actual > predicted)
• Negative residual: Model overpredicts (actual < predicted)
Model fit: How well a mathematical model represents the actual data, evaluated by examining residuals across all data points.
Model: y = 0.5x + 2
Point (2, 3): Predicted = 0.5(2) + 2 = 3, Residual = 3 - 3 = 0
Point (4, 4): Predicted = 0.5(4) + 2 = 4, Residual = 4 - 4 = 0
Point (6, 6): Predicted = 0.5(6) + 2 = 5, Residual = 6 - 5 = 1
Residuals: 0, 0, 1
Most residuals are close to zero, indicating good fit.
Small residuals indicate the model fits the data well.
The pattern suggests the linear model is appropriate.
The model fits the data reasonably well, with residuals close to zero.
The residuals are 0, 0, and 1. The model has a good fit with small residuals.
• Good fit: Residuals are small and randomly scattered
• Poor fit: Large residuals or systematic patterns
• Pattern analysis: Look for trends in residuals
Residual analysis: Systematic examination of residuals to evaluate model appropriateness and identify potential issues.
Model: y = 1.8x + 1.2
At x=1: Predicted = 1.8(1) + 1.2 = 3.0
At x=2: Predicted = 1.8(2) + 1.2 = 4.8
At x=3: Predicted = 1.8(3) + 1.2 = 6.6
At x=4: Predicted = 1.8(4) + 1.2 = 8.4
At x=5: Predicted = 1.8(5) + 1.2 = 10.2
Point (1, 3): Residual = 3 - 3.0 = 0
Point (2, 5): Residual = 5 - 4.8 = 0.2
Point (3, 7): Residual = 7 - 6.6 = 0.4
Point (4, 8): Residual = 8 - 8.4 = -0.4
Point (5, 10): Residual = 10 - 10.2 = -0.2
Residuals: 0, 0.2, 0.4, -0.4, -0.2
Values are small and oscillate around zero.
Small residuals indicate good fit.
Random pattern suggests the linear model is appropriate.
The residuals are small and randomly scattered around zero, indicating the linear model is appropriate.
• Random residuals: Indicate good model fit
• Systematic patterns: Suggest model inadequacy
• Small magnitude: Better model fit
Residual: The vertical distance between an observed data point and the value predicted by the model.
Model fit: How well a mathematical model represents the actual data.
Linear model: A straight-line model of the form y = mx + b used to predict relationships between variables.
Scatter plot: A graph showing the relationship between two variables using plotted points.
Trend line: A line drawn through data points showing the general direction of the relationship.
- Identify the model: Obtain the equation of the trend line
- Find predicted values: Substitute x-values into the model
- Calculate residuals: Subtract predicted from observed values
- Analyze residuals: Examine size and pattern
- Evaluate fit: Determine if model is appropriate
• Random residuals: Indicate good model fit
• Systematic patterns: Suggest model inadequacy
• Small magnitude: Better model fit
• Zero mean: Good models have residuals centered at zero
Individual residuals: Examining each residual to understand specific predictions.
Overall fit: Looking at the pattern of all residuals together.
Quantitative assessment: Using statistical measures to evaluate model quality.
- Residual plots: Graph residuals against x-values to identify patterns
- Sum of squared residuals: Calculate SSR for quantitative comparison
- Standard deviation of residuals: Measure typical prediction error
- Outlier detection: Identify points with unusually large residuals
- Pattern analysis: Look for curvature or trends in residuals
• Random scatter: Linear model is appropriate
• Curved pattern: Non-linear model needed
• Funnel shape: Variability increases with x-value
• Systematic trend: Model misses important features
Questions & Answers
Question: What does it mean if all residuals are positive or all are negative?
Answer: If all residuals have the same sign, it indicates a systematic issue with the model:
- All positive residuals: The model consistently underpredicts the actual values
- All negative residuals: The model consistently overpredicts the actual values
This suggests the model is biased in one direction. Possible causes include:
- Incorrect slope in the linear model
- Incorrect y-intercept
- Non-linear relationship that requires a different model
A good model should have residuals randomly scattered above and below zero.
Question: How do I know if the residuals are "small enough" for a good fit?
Answer: Determining if residuals are "small enough" involves several considerations:
- Relative to data scale: Compare residuals to the range of your y-values
- Context-dependent: What's acceptable varies by application
- Pattern over magnitude: Random small residuals are better than systematic large ones
As a general rule:
- If residuals are less than 10% of the range of y-values, the fit is often considered good
- If residuals show random scatter around zero, the model is likely appropriate
Always consider the context and purpose of your model when evaluating fit.
Question: Can residuals help me choose between different models?
Answer: Yes, residuals are excellent tools for comparing models:
- Compare sum of squared residuals: Lower SSR indicates better fit
- Analyze residual patterns: Choose model with more random residuals
- Look at residual magnitudes: Smaller residuals indicate better predictions
- Check for systematic errors: Avoid models with consistent bias
For example, if a linear model has residuals: 0.1, -0.2, 0.3, 0.1 and a quadratic model has residuals: 0.05, -0.05, 0.02, -0.03, the quadratic model provides a better fit.
Residual analysis is one of the primary methods for model selection.