Forward difference method: Estimates the derivative using the formula \(f'(x) \approx \frac{f(x+h) - f(x)}{h}\), where \(h\) is a small positive increment.
- Choose a small value of \(h > 0\)
- Calculate \(f(x)\) at the point of interest
- Calculate \(f(x+h)\) at the shifted point
- Apply the forward difference formula
- Smaller values of \(h\) generally give better approximations
\(f(2) = 2^2 + 3(2) = 4 + 6 = 10\)
\(f(2.1) = (2.1)^2 + 3(2.1) = 4.41 + 6.3 = 10.71\)
\(f'(2) \approx \frac{f(2.1) - f(2)}{0.1} = \frac{10.71 - 10}{0.1} = \frac{0.71}{0.1} = 7.1\)
Using the derivative rule: \(f'(x) = 2x + 3\)
Exact value: \(f'(2) = 2(2) + 3 = 7\)
Error: \(|7.1 - 7| = 0.1\)
Using the forward difference method with \(h = 0.1\), the estimated value of \(f'(2)\) is 7.1, compared to the exact value of 7.
• Forward difference formula: \(f'(x) \approx \frac{f(x+h) - f(x)}{h}\)
• Secant line approximation: Forward difference estimates tangent slope
• Error reduction: Smaller \(h\) reduces truncation error
Backward difference method: Estimates the derivative using the formula \(f'(x) \approx \frac{f(x) - f(x-h)}{h}\), where \(h\) is a small positive increment.
\(f(3) = 3^3 - 2(3) = 27 - 6 = 21\)
\(f(2.9) = (2.9)^3 - 2(2.9) = 24.389 - 5.8 = 18.589\)
\(f'(3) \approx \frac{f(3) - f(2.9)}{0.1} = \frac{21 - 18.589}{0.1} = \frac{2.411}{0.1} = 24.11\)
Using the derivative rule: \(f'(x) = 3x^2 - 2\)
Exact value: \(f'(3) = 3(3^2) - 2 = 27 - 2 = 25\)
Error: \(|24.11 - 25| = 0.89\)
Using the backward difference method with \(h = 0.1\), the estimated value of \(f'(3)\) is 24.11, compared to the exact value of 25.
• Backward difference formula: \(f'(x) \approx \frac{f(x) - f(x-h)}{h}\)
• Secant line approximation: Backward difference estimates tangent slope
• Directional bias: Uses points to the left of the target point
Central difference method: Estimates the derivative using the formula \(f'(x) \approx \frac{f(x+h) - f(x-h)}{2h}\), providing a more accurate approximation than forward or backward differences.
\(f(1.01) = e^{1.01} \approx 2.74560104\)
\(f(0.99) = e^{0.99} \approx 2.69153480\)
\(f'(1) \approx \frac{f(1.01) - f(0.99)}{2(0.01)} = \frac{2.74560104 - 2.69153480}{0.02}\)
\(f'(1) \approx \frac{0.05406624}{0.02} = 2.703312\)
Using the derivative rule: \(f'(x) = e^x\)
Exact value: \(f'(1) = e^1 \approx 2.71828183\)
Error: \(|2.703312 - 2.71828183| \approx 0.015\)
Forward difference with \(h = 0.01\): \(\frac{e^{1.01} - e^1}{0.01} \approx 2.731922\)
Central difference error: ~0.015
Forward difference error: ~0.014
Using the central difference method with \(h = 0.01\), the estimated value of \(f'(1)\) is approximately 2.703312, compared to the exact value of \(e \approx 2.718282\).
• Central difference formula: \(f'(x) \approx \frac{f(x+h) - f(x-h)}{2h}\)
• Higher accuracy: Central difference is typically more accurate than forward/backward
• Symmetric approach: Uses points on both sides of the target point
Numerical differentiation: Techniques to estimate derivatives when analytical differentiation is difficult or impossible.
Forward difference: Uses points ahead of the target point to estimate the derivative.
Backward difference: Uses points behind the target point to estimate the derivative.
Central difference: Uses points on both sides of the target point to estimate the derivative.
- Select method: Choose forward, backward, or central difference based on available data
- Choose increment: Select an appropriately small value of \(h\)
- Calculate function values: Evaluate the function at required points
- Apply formula: Substitute values into the chosen difference formula
- Assess accuracy: Consider the trade-off between truncation and roundoff errors
• Forward difference: \(f'(x) \approx \frac{f(x+h) - f(x)}{h}\)
• Backward difference: \(f'(x) \approx \frac{f(x) - f(x-h)}{h}\)
• Central difference: \(f'(x) \approx \frac{f(x+h) - f(x-h)}{2h}\)
• Higher-order central difference: \(f'(x) \approx \frac{-f(x+2h) + 8f(x+h) - 8f(x-h) + f(x-2h)}{12h}\)
| x | g(x) |
|---|---|
| 1.8 | 3.24 |
| 1.9 | 3.61 |
| 2.0 | 4.00 |
| 2.1 | 4.41 |
| 2.2 | 4.84 |
Tabular data estimation: When function values are given in a table, select the appropriate difference method based on available neighboring points.
We have: \(g(1.8) = 3.24\), \(g(1.9) = 3.61\), \(g(2.0) = 4.00\), \(g(2.1) = 4.41\), \(g(2.2) = 4.84\)
Since we have points both before and after \(x = 2.0\), we can use the central difference method
With \(h = 0.1\): \(g'(2.0) \approx \frac{g(2.1) - g(1.9)}{2(0.1)}\)
\(g'(2.0) \approx \frac{g(2.1) - g(1.9)}{2(0.1)} = \frac{4.41 - 3.61}{0.2} = \frac{0.80}{0.2} = 4.0\)
Forward difference: \(\frac{g(2.1) - g(2.0)}{0.1} = \frac{4.41 - 4.00}{0.1} = 4.1\)
Backward difference: \(\frac{g(2.0) - g(1.9)}{0.1} = \frac{4.00 - 3.61}{0.1} = 3.9\)
The central difference value of 4.0 is between these values, as expected
Looking at the data, it appears that \(g(x) = x^2\) since \(g(2.0) = 4.00 = 2^2\), \(g(2.1) = 4.41 = 2.1^2\), etc.
If \(g(x) = x^2\), then \(g'(x) = 2x\) and \(g'(2) = 4\)
Our estimate of 4.0 is very close to the actual value of 4!
Using the central difference method with the tabular data, the estimated value of \(g'(2)\) is 4.0.
• Data availability: Choose method based on available neighboring points
• Central difference: Best choice when points are available on both sides
• Accuracy comparison: Central difference typically provides better estimates
Error analysis: Examining how the accuracy of numerical estimates improves as the increment \(h\) decreases, balancing truncation and roundoff errors.
For \(f(x) = \sin(x)\), we know that \(f'(x) = \cos(x)\)
Exact value: \(f'(\pi/4) = \cos(\pi/4) = \frac{\sqrt{2}}{2} \approx 0.70710678\)
\(f(\pi/4) = \sin(\pi/4) \approx 0.70710678\)
\(f(\pi/4 + 0.1) = \sin(\pi/4 + 0.1) \approx \sin(0.885398) \approx 0.777146\)
Estimate: \(\frac{0.777146 - 0.707107}{0.1} = \frac{0.070039}{0.1} = 0.70039\)
Error: \(|0.70039 - 0.707107| = 0.006717\)
\(f(\pi/4 + 0.01) = \sin(\pi/4 + 0.01) \approx \sin(0.800785) \approx 0.714141\)
Estimate: \(\frac{0.714141 - 0.707107}{0.01} = \frac{0.007034}{0.01} = 0.7034\)
Error: \(|0.7034 - 0.707107| = 0.003707\)
\(f(\pi/4 + 0.001) = \sin(\pi/4 + 0.001) \approx \sin(0.786175) \approx 0.707810\)
Estimate: \(\frac{0.707810 - 0.707107}{0.001} = \frac{0.000703}{0.001} = 0.703\)
Error: \(|0.703 - 0.707107| = 0.004107\)
As \(h\) decreases from 0.1 to 0.01, error decreases from 0.006717 to 0.003707
However, as \(h\) decreases further to 0.001, error increases to 0.004107
This indicates that roundoff error begins to dominate as \(h\) becomes very small
The forward difference estimates for \(f'(\pi/4)\) with decreasing \(h\) values show initial improvement but eventual degradation due to roundoff error, demonstrating the trade-off in numerical differentiation.
• Truncation error: Decreases as \(h\) decreases for forward difference
• Roundoff error: Increases as \(h\) decreases due to division by small numbers
• Optimal h: Balance between truncation and roundoff errors
Numerical differentiation: The process of approximating the derivative of a function using discrete values of the function. It's essential when analytical derivatives are unavailable or when working with empirical data.
Truncation error: The error introduced by approximating the derivative using a finite difference instead of the limit definition. For forward/backward differences, it's proportional to \(h\); for central differences, it's proportional to \(h^2\).
Roundoff error: The error introduced by finite precision arithmetic when computing the difference quotient. It becomes significant when \(h\) is very small.
Condition number: Measures the sensitivity of the numerical derivative to small changes in function values, indicating potential numerical instability.
- Problem identification: Determine which method is most appropriate based on available data
- Increment selection: Choose \(h\) to balance truncation and roundoff errors
- Function evaluation: Calculate required function values accurately
- Formula application: Apply the appropriate difference formula
- Error assessment: Evaluate the quality of the approximation
• Error bounds: Forward/backward: \(O(h)\), Central: \(O(h^2)\), Higher-order: \(O(h^4)\) and beyond
• Stability condition: \(\frac{f(x+h) - f(x-h)}{2h}\) is stable for well-behaved functions
• Optimal increment: For double precision, \(h \approx \sqrt{\epsilon}\) where \(\epsilon\) is machine epsilon
• Consistency: As \(h \to 0\), numerical derivative approaches true derivative (if it exists)
- Forward: \(\frac{f(1+h) - f(1)}{h}\)
- Backward: \(\frac{f(1) - f(1-h)}{h}\)
- Central: \(\frac{f(1+h) - f(1-h)}{2h}\)
- Exact: \(f'(1) = 2\)
Analysis: The chart shows how different methods exhibit different convergence behaviors:
- Central difference shows quadratic convergence (\(O(h^2)\)) for a longer range
- Forward and backward differences show linear convergence (\(O(h)\))
- All methods eventually suffer from roundoff error as \(h\) becomes very small