Linear regression: Statistical method to find the line that best fits a set of data points, minimizing the sum of squared residuals
- Plot the data points to visualize the relationship
- Calculate means of x and y values
- Compute the slope using the least squares formula
- Find the y-intercept using the point-slope form
- Calculate the correlation coefficient
- Interpret the strength and direction of the relationship
Mean of x: \(\bar{x} = \frac{1+2+3+4+5}{5} = \frac{15}{5} = 3\)
Mean of y: \(\bar{y} = \frac{3+5+7+9+11}{5} = \frac{35}{5} = 7\)
Using the least squares formula: \(m = \frac{\sum(x_i - \bar{x})(y_i - \bar{y})}{\sum(x_i - \bar{x})^2}\)
For each point:
(1,3): \((1-3)(3-7) = (-2)(-4) = 8\), \((1-3)^2 = 4\)
(2,5): \((2-3)(5-7) = (-1)(-2) = 2\), \((2-3)^2 = 1\)
(3,7): \((3-3)(7-7) = (0)(0) = 0\), \((3-3)^2 = 0\)
(4,9): \((4-3)(9-7) = (1)(2) = 2\), \((4-3)^2 = 1\)
(5,11): \((5-3)(11-7) = (2)(4) = 8\), \((5-3)^2 = 4\)
\(\sum(x_i - \bar{x})(y_i - \bar{y}) = 8 + 2 + 0 + 2 + 8 = 20\)
\(\sum(x_i - \bar{x})^2 = 4 + 1 + 0 + 1 + 4 = 10\)
\(m = \frac{20}{10} = 2\)
Using point-slope form: \(y - \bar{y} = m(x - \bar{x})\)
\(y - 7 = 2(x - 3)\)
\(y - 7 = 2x - 6\)
\(y = 2x + 1\)
\(r = \frac{\sum(x_i - \bar{x})(y_i - \bar{y})}{\sqrt{\sum(x_i - \bar{x})^2 \sum(y_i - \bar{y})^2}}\)
\(\sum(y_i - \bar{y})^2 = (-4)^2 + (-2)^2 + 0^2 + 2^2 + 4^2 = 16 + 4 + 0 + 4 + 16 = 40\)
\(r = \frac{20}{\sqrt{10 \times 40}} = \frac{20}{\sqrt{400}} = \frac{20}{20} = 1\)
The line of best fit is \(y = 2x + 1\) with a perfect positive correlation coefficient \(r = 1\).
• Least squares method: Minimizes sum of squared residuals
• Correlation coefficient: \(r\) ranges from -1 to 1, indicating strength and direction
• Perfect correlation: \(r = ±1\) means all points lie exactly on the regression line
Quadratic regression: Finds the parabola that best fits data points using the form \(y = ax^2 + bx + c\)
Looking at y-values: 1, 2, 5, 10, 17
First differences: 1, 3, 5, 7 (not constant)
Second differences: 2, 2, 2 (constant) → suggests quadratic relationship
General form: \(y = ax^2 + bx + c\)
Using the point (0, 1): \(1 = a(0)^2 + b(0) + c\), so \(c = 1\)
Using (1, 2): \(2 = a(1)^2 + b(1) + 1\), so \(a + b = 1\)
Using (2, 5): \(5 = a(2)^2 + b(2) + 1\), so \(4a + 2b = 4\)
From: \(a + b = 1\) and \(4a + 2b = 4\)
Multiply first equation by 2: \(2a + 2b = 2\)
Subtract from second: \(2a = 2\), so \(a = 1\)
Substitute: \(1 + b = 1\), so \(b = 0\)
\(y = x^2 + 0x + 1 = x^2 + 1\)
(0,1): \(0^2 + 1 = 1\) ✓
(1,2): \(1^2 + 1 = 2\) ✓
(2,5): \(2^2 + 1 = 5\) ✓
(3,10): \(3^2 + 1 = 10\) ✓
(4,17): \(4^2 + 1 = 17\) ✓
Linear model: \(y = 4x - 1\) (from least squares)
Linear R² ≈ 0.94, Quadratic R² = 1.00
Quadratic model is superior with perfect fit
The quadratic model \(y = x^2 + 1\) fits the data perfectly with \(R^2 = 1.00\), making it a better fit than any linear model.
• Quadratic pattern: Constant second differences indicate quadratic relationship
• System of equations: Use known points to solve for unknown coefficients
• Model comparison: Higher R² indicates better fit
Exponential regression: Finds the exponential function that best fits data using the form \(y = a \cdot b^x\)
Looking at y-values: 3, 6, 12, 24, 48
Ratio between consecutive terms: 6/3 = 2, 12/6 = 2, 24/12 = 2, 48/24 = 2
Constant ratio of 2 suggests exponential growth
When x = 0, y = 3, so the initial value a = 3
Since values double each time, the growth factor b = 2
\(y = 3 \cdot 2^x\)
(0,3): \(3 \cdot 2^0 = 3 \cdot 1 = 3\) ✓
(1,6): \(3 \cdot 2^1 = 3 \cdot 2 = 6\) ✓
(2,12): \(3 \cdot 2^2 = 3 \cdot 4 = 12\) ✓
(3,24): \(3 \cdot 2^3 = 3 \cdot 8 = 24\) ✓
(4,48): \(3 \cdot 2^4 = 3 \cdot 16 = 48\) ✓
Linear model: Would not capture the exponential growth pattern
Quadratic model: Would not show constant multiplicative growth
Exponential model captures the doubling behavior perfectly
a = 3: Initial value when x = 0
b = 2: Growth factor (doubling each unit increase in x)
The exponential model \(y = 3 \cdot 2^x\) fits the data perfectly with \(R^2 = 1.00\), confirming it's the appropriate model for this data.
• Exponential pattern: Constant ratios between consecutive terms
• Initial value: Value when x = 0 gives coefficient a
• Growth factor: Ratio of consecutive y-values gives base b
Regression analysis: Statistical method for determining the relationship between variables by finding the best-fitting curve through data points
Line of best fit: The line that minimizes the sum of squared residuals (vertical distances from points to line)
Correlation coefficient (r): Measure of linear relationship strength ranging from -1 to 1, where |r| = 1 indicates perfect correlation
Coefficient of determination (R²): Proportion of variance in dependent variable explained by independent variable, ranges from 0 to 1
Residual: The difference between observed value and predicted value from the regression model
Linear regression: Fits a straight line to data using least squares method, appropriate for constant rate of change
Quadratic regression: Fits a parabola to data, appropriate for curved relationships with constant second differences
Exponential regression: Fits an exponential curve to data, appropriate for constant multiplicative growth/decay
- Visual inspection: Plot data points to identify potential patterns
- Calculate differences: First differences for linear, second differences for quadratic
- Calculate ratios: For exponential patterns, check if consecutive terms have constant ratio
- Fit multiple models: Try linear, quadratic, and exponential regressions
- Compare goodness of fit: Use R² values or residual analysis
- Select best model: Choose model with highest R² and most appropriate pattern
- Validate model: Check if model makes sense in real-world context
• Linear: r ≈ ±1, R² ≈ 1.0, constant first differences
• Quadratic: Curved pattern, constant second differences, R² > linear model
• Exponential: Constant ratios, exponential growth/decay pattern, R² > other models
• Model comparison: Choose model with highest R² and appropriate pattern
Model comparison: Systematic process of evaluating multiple regression models to determine which best represents the data
y-values: 2, 5, 10, 17, 26
First differences: 5-2=3, 10-5=5, 17-10=7, 26-17=9
Differences: 3, 5, 7, 9 (not constant) → Not linear
First differences: 3, 5, 7, 9
Second differences: 5-3=2, 7-5=2, 9-7=2
Second differences: 2, 2, 2 (constant) → Quadratic pattern
Ratios: 5/2=2.5, 10/5=2, 17/10=1.7, 26/17≈1.53
Ratios: 2.5, 2, 1.7, 1.53 (not constant) → Not exponential
General form: \(y = ax^2 + bx + c\)
Using points (1,2), (2,5), (3,10):
1: \(a + b + c = 2\)
2: \(4a + 2b + c = 5\)
3: \(9a + 3b + c = 10\)
From equations 2-1: \(3a + b = 3\)
From equations 3-2: \(5a + b = 5\)
Subtracting: \(2a = 2\), so \(a = 1\)
Substituting: \(3(1) + b = 3\), so \(b = 0\)
Substituting: \(1 + 0 + c = 2\), so \(c = 1\)
Quadratic model: \(y = x^2 + 1\)
(1,2): \(1^2 + 1 = 2\) ✓
(2,5): \(2^2 + 1 = 5\) ✓
(3,10): \(3^2 + 1 = 10\) ✓
(4,17): \(4^2 + 1 = 17\) ✓
(5,26): \(5^2 + 1 = 26\) ✓
Quadratic model: R² = 1.00 (perfect fit)
Linear model: R² ≈ 0.94
Quadratic model is superior
The quadratic model \(y = x^2 + 1\) best fits the data with perfect correlation (R² = 1.00).
• Pattern recognition: Constant second differences indicate quadratic relationship
• System of equations: Use known points to solve for quadratic coefficients
• Model comparison: Higher R² indicates better fit
Business modeling: Using regression analysis to identify growth patterns and make predictions about future performance
Sales: $100k, $120k, $144k, $173k, $207k
Check for constant ratios: 120/100 = 1.2, 144/120 = 1.2, 173/144 ≈ 1.20, 207/173 ≈ 1.20
Approximately constant ratio of 1.2 → Exponential growth pattern
Ratio ≈ 1.2 means 20% annual growth rate
Initial value: $100k (Year 1)
Since Year 1 corresponds to x = 1, we have:
\(y = 100 \cdot (1.2)^x\) (where y is sales in thousands of dollars)
Year 1: \(100 \cdot (1.2)^1 = 120\) (close to 100) - Actually this doesn't match!
Let's adjust: If y = a·b^x and (1,100), then 100 = a·b^1, so a·b = 100
Using ratio b = 1.2: a·1.2 = 100, so a = 83.33
Model: \(y = 83.33 \cdot (1.2)^x\)
Year 1: \(83.33 \cdot (1.2)^1 = 100\) ✓
Year 2: \(83.33 \cdot (1.2)^2 = 83.33 \cdot 1.44 = 120\) ✓
Year 3: \(83.33 \cdot (1.2)^3 = 83.33 \cdot 1.728 = 144\) ✓
Year 4: \(83.33 \cdot (1.2)^4 = 83.33 \cdot 2.074 = 173\) ✓
Year 5: \(83.33 \cdot (1.2)^5 = 83.33 \cdot 2.488 = 207\) ✓
Year 6 corresponds to x = 6:
\(y = 83.33 \cdot (1.2)^6\)
\(y = 83.33 \cdot 2.986 = 249\)
The model predicts sales of approximately $249,000 in Year 6
This represents continued 20% annual growth
The exponential model is \(y = 83.33 \cdot (1.2)^x\) and predicted sales for Year 6 are approximately $249,000.
• Business growth: Exponential models often represent compound growth
• Ratio identification: Constant ratios indicate exponential relationship
• Model validation: Verify model fits all known data points
Regression analysis: A statistical method used to examine the relationship between one dependent variable and one or more independent variables by fitting a model to the data.
Least squares method: A technique that minimizes the sum of the squares of the residuals (differences between observed and predicted values) to find the best-fitting line.
Correlation coefficient (r): A measure of the strength and direction of a linear relationship between two variables, ranging from -1 to 1.
Coefficient of determination (R²): The proportion of the variance in the dependent variable that is predictable from the independent variable(s), ranging from 0 to 1.
Residual: The difference between an observed value and the value predicted by the regression model: residual = observed - predicted.
Linear regression: A regression model that assumes a linear relationship between variables, taking the form \(y = mx + b\).
Quadratic regression: A regression model that fits a parabolic curve to data, taking the form \(y = ax^2 + bx + c\).
Exponential regression: A regression model that fits an exponential curve to data, taking the form \(y = a \cdot b^x\).
Model selection: The process of choosing the most appropriate mathematical model to represent a given dataset based on pattern recognition and goodness-of-fit measures.
- Data visualization: Plot the data points to observe potential patterns
- Pattern recognition: Identify whether data appears linear, quadratic, exponential, or other
- Model selection: Choose appropriate regression model based on pattern
- Parameter estimation: Calculate model coefficients using appropriate method
- Goodness-of-fit assessment: Calculate R², examine residuals, and validate assumptions
- Model validation: Test model against additional data if available
- Prediction: Use validated model for forecasting or interpolation
• Linear model: \(y = mx + b\), constant first differences, R² for linearity assessment
• Quadratic model: \(y = ax^2 + bx + c\), constant second differences, parabolic shape
• Exponential model: \(y = a \cdot b^x\), constant ratios between consecutive terms
• Power model: \(y = ax^b\), useful for relationships with variable exponents
• Logarithmic model: \(y = a + b \ln(x)\), useful for data that increases rapidly then levels off
• Model selection criterion: Choose model with highest R² that makes contextual sense
• Residual analysis: Plot residuals to check for patterns indicating model inadequacy
(1, 2), (2, 5), (3, 10), (4, 17), (5, 26)
\(f_1(x) = 6x - 5\) (Linear)
\(f_2(x) = x^2 + 1\) (Quadratic)
\(f_3(x) = 2 \cdot 1.8^x\) (Exponential)
Analysis: The chart compares how different models fit the same data set.
- \(f_1(x) = 6x - 5\): Linear model with R² ≈ 0.94
- \(f_2(x) = x^2 + 1\): Quadratic model with R² = 1.00 (perfect fit)
- \(f_3(x) = 2 \cdot 1.8^x\): Exponential model (less appropriate)