Linear Regression Basics (Lines of Best Fit)

Master linear regression basics including scatter plots, correlation, and lines of best fit with exercises, visual infographics, and detailed solutions.

Practice Exercises
1 Study Time vs Test Scores
Exercise 1
The table shows study time (hours) and test scores (%) for 6 students. Draw a scatter plot, estimate a line of best fit, and interpret its meaning.
Definition:

Scatter Plot: A graph showing the relationship between two variables as points

Line of Best Fit: A straight line that best represents the data on a scatter plot

Solution method:
  1. Plot the data points on a coordinate plane
  2. Visually estimate the trend line
  3. Draw the line of best fit
  4. Calculate the slope and y-intercept
  5. Interpret the meaning in context
Data
Hours vs %
Trend
Positive
Equation
y ≈ 8x + 65
Step 1: Plot the data points

(1, 72), (2, 78), (3, 85), (4, 92), (5, 95), (6, 98)

Step 2: Estimate the trend

Points generally move upward from left to right

Step 3: Draw the line of best fit

Line passes close to all points, balancing above/below

Approximate equation: y = 8x + 65
Interpretation:

Slope (8): Test score increases by approximately 8% for each additional hour of study

y-intercept (65): Predicted score with 0 hours of study

Applied rules:

• Line of best fit: Minimizes distance to all data points

• Positive correlation: Both variables increase together

• Interpretation: Connect mathematical meaning to context

2 Temperature vs Ice Cream Sales
Exercise 2
The data shows daily temperature (°F) and ice cream sales ($). Create a scatter plot, determine correlation, and predict sales at 85°F.
Definition:

Correlation: The degree to which two variables are related

Prediction: Using the line of best fit to estimate unknown values

Data
Temp vs Sales
Correlation
Positive
Prediction
$120
Step 1: Analyze the pattern

Higher temperatures correspond to higher sales

Step 2: Draw the line of best fit

Approximately y = 2x - 30

Step 3: Make prediction

At 85°F: y = 2(85) - 30 = 170 - 30 = $140

Prediction: $140 in sales at 85°F
Interpretation:

Slope (2): Sales increase by $2 for each degree increase in temperature

Prediction: At 85°F, expect approximately $140 in sales

Applied rules:

• Positive correlation: Variables move in same direction

• Prediction accuracy: More reliable within data range

• Extrapolation risk: Be cautious beyond data range

3 Training Hours vs Race Time
Exercise 3
Data shows training hours per week and race completion time (minutes). Determine if there's correlation and find the line of best fit.
Definition:

Negative correlation: As one variable increases, the other decreases

Line of best fit: Models the relationship between variables

Data
Training vs Time
Correlation
Negative
Equation
y ≈ -2x + 40
Step 1: Identify the relationship

More training leads to faster (lower) race times

Step 2: Draw the line of best fit

Downward sloping line showing inverse relationship

Step 3: Interpret the results

Negative slope indicates inverse relationship

Equation: y = -2x + 40
Interpretation:

Slope (-2): Race time decreases by 2 minutes for each additional hour of training

Y-intercept (40): Predicted time with no training

Applied rules:

• Negative correlation: Variables move in opposite directions

• Causation vs correlation: Correlation doesn't imply causation

• Model limitations: Consider practical constraints

Key Concepts & Interpretations
Linear Regression Components
📊
Scatter Plot: Graphical representation of paired data points
📈
Line of Best Fit: Linear model minimizing distance to data points
🔄
Correlation: Strength and direction of relationship between variables
y = mx + b

Where m = slope and b = y-intercept of the line of best fit

Strong Positive
r ≈ 1
Points closely follow upward trend
No Correlation
r ≈ 0
Points scattered randomly
Strong Negative
r ≈ -1
Points closely follow downward trend
Correlation Strength
✅
Strong correlation: |r| > 0.7 (points closely follow line)
⚠️
Moderate correlation: 0.3 < |r| ≤ 0.7
❌
Weak correlation: |r| ≤ 0.3 (points widely scattered)
Methods, Tips & Common Mistakes
Key definitions:

Linear Regression: Statistical method to model relationship between variables

Line of Best Fit: Straight line that minimizes sum of squared distances to data points

Correlation Coefficient (r): Measures strength and direction of linear relationship

Drawing line of best fit:
  1. Plot all data points accurately on coordinate plane
  2. Visualize the trend - imagine a line through the middle
  3. Balance points - roughly equal points above and below
  4. Minimize distances - line should be closest to most points
Tip 1: The line doesn't need to pass through any actual data points.
Tip 2: Outliers can significantly affect the line of best fit.
Tip 3: Always consider if correlation implies causation in context.
Tip 4: Predictions are most accurate within the range of your data.
Common errors: Forcing line through origin, ignoring outliers, confusing correlation with causation.
Exam preparation: Practice drawing lines, interpreting slopes, making predictions.
Key principles to remember:

• Line of best fit: y = mx + b

• Slope interpretation: Rate of change between variables

• Correlation range: -1 ≤ r ≤ 1

• Prediction caution: Extrapolation beyond data range risky

Comprehensive Summary: Linear Regression Basics

Definitions and Key Concepts

Linear Regression: A statistical method used to model the relationship between two variables by fitting a linear equation to observed data. It helps us understand how one variable (dependent) changes as another variable (independent) changes.

Scatter Plot: A graph where individual data points are plotted to show the relationship between two variables. The horizontal axis (x-axis) typically represents the independent variable, while the vertical axis (y-axis) represents the dependent variable.

Line of Best Fit: Also known as the trend line or regression line, this is a straight line that best represents the data on a scatter plot. It minimizes the sum of the squared distances between the actual data points and the line.

Core Rules and Principles

Correlation vs Causation: Correlation indicates a relationship between variables, but it doesn't prove that one causes the other. Just because two variables are correlated doesn't mean that one causes the other.

Line of Best Fit Properties: The line should pass through the center of the data cluster, with roughly equal numbers of points above and below the line. The line minimizes the sum of squared residuals (vertical distances from points to the line).

Correlation Strength: Measured by the correlation coefficient (r), which ranges from -1 to 1. Values close to -1 or 1 indicate strong correlation, while values near 0 indicate weak correlation.

Step-by-Step Method for Finding Line of Best Fit
  1. Organize Data: Create a table with your paired data points
  2. Create Scatter Plot: Plot all points on a coordinate plane
  3. Visualize Trend: Look for the general direction of the data points
  4. Draw Line: Sketch a line that appears to go through the center of the data, balancing points above and below
  5. Select Two Points: Choose two points on your line (not necessarily data points)
  6. Calculate Slope: Use m = (y₂ - y₁)/(x₂ - x₁)
  7. Find Equation: Use point-slope form to write the equation y = mx + b
Types of Correlation

Positive Correlation: As one variable increases, the other also increases. The line of best fit has a positive slope. Example: Hours studied vs test scores.

Negative Correlation: As one variable increases, the other decreases. The line of best fit has a negative slope. Example: Hours of exercise vs resting heart rate.

No Correlation: The variables have no apparent relationship. Points appear randomly scattered. Example: Shoe size vs SAT scores.

Concrete Examples

Simple Example: Height vs arm span - taller people tend to have longer arms (positive correlation, r ≈ 0.8).

Intermediate Example: Temperature vs heating bills - as temperature rises, heating costs decrease (negative correlation, r ≈ -0.7).

Advanced Example: Advertising spend vs sales revenue with diminishing returns - initially strong correlation that weakens at higher spending levels.

Tips and Common Pitfalls

Tips: Always label axes clearly, use consistent scales, and consider outliers when drawing the line. The line doesn't need to pass through any actual data points.

Common Mistakes: Assuming causation from correlation, extrapolating too far beyond the data range, forcing the line through the origin when it shouldn't, and ignoring influential outliers.

Key Takeaways for Memorization

Memory Aid: "Line of Best Fit" = "Least Squares Line" - it minimizes the sum of squared distances.

Correlation Range: Remember -1 ≤ r ≤ 1, where -1 is perfect negative correlation, 0 is no correlation, and 1 is perfect positive correlation.

Interpretation Rule: Always interpret slope and y-intercept in the context of the problem, including appropriate units.

Questions & Answers

Question: How do I draw a line of best fit when the points don't seem to follow a perfectly straight pattern? Should I try to make the line go through as many points as possible?

Answer: No, the line of best fit should not try to go through as many points as possible! The goal is to minimize the overall distance between the line and all points, not to pass through many points. Here's the correct approach:

  • Visual balance: The line should have roughly equal numbers of points above and below it
  • Central tendency: The line should pass through the center of the data cluster
  • Minimize distances: The sum of squared vertical distances from points to the line should be as small as possible

Think of the line as representing the "trend" of the data rather than connecting individual points. Even if points don't lie exactly on the line, the line still models the overall relationship between the variables.

Question: What's the difference between correlation and causation? Why is this important in linear regression?

Answer: This is a crucial distinction in statistics:

  • Correlation: A statistical measure showing the extent to which two variables change together. It tells us if variables move in the same direction (positive), opposite directions (negative), or not at all (no correlation).
  • Causation: A cause-and-effect relationship where one variable directly influences the other.

For example, ice cream sales and drowning incidents may be positively correlated (both increase in summer), but eating ice cream doesn't cause drowning! The common factor is warmer weather. In linear regression, we can identify relationships but cannot conclude causation without further investigation.

Question: How do I know if my line of best fit is accurate enough to make predictions? When should I be concerned about the reliability of my predictions?

Answer: Here are key factors to assess the reliability of your line of best fit and predictions:

  • Strength of correlation: Look at how closely the points cluster around your line. Stronger correlations (points closer to the line) yield more reliable predictions.
  • Data range: Predictions within the range of your original data (interpolation) are more reliable than predictions outside that range (extrapolation).
  • Sample size: Larger datasets generally provide more reliable trend lines.
  • Outliers: Be cautious if your dataset contains outliers that might skew your line.

As a rule of thumb, be very cautious about extrapolating beyond your data range, especially if the correlation isn't very strong. The further you move from your data, the less reliable your predictions become.