Solved Exercises on Estimating Derivatives in Calculus 1

Master derivative estimation: numerical methods, secant lines, difference quotients, and applications through these 5 detailed exercises.

Solution: Exercises 1 to 3
1 Forward Difference Method
Exercise 1
Estimate \(f'(2)\) for \(f(x) = x^2 + 3x\) using the forward difference method with \(h = 0.1\)
Definition:

Forward difference method: Estimates the derivative using the formula \(f'(x) \approx \frac{f(x+h) - f(x)}{h}\), where \(h\) is a small positive increment.

Forward difference method:
  1. Choose a small value of \(h > 0\)
  2. Calculate \(f(x)\) at the point of interest
  3. Calculate \(f(x+h)\) at the shifted point
  4. Apply the forward difference formula
  5. Smaller values of \(h\) generally give better approximations
Function
\(f(x) = x^2 + 3x\)
Point
\(x = 2\)
Increment
\(h = 0.1\)
Step 1: Calculate \(f(2)\)

\(f(2) = 2^2 + 3(2) = 4 + 6 = 10\)

Step 2: Calculate \(f(2 + h) = f(2.1)\)

\(f(2.1) = (2.1)^2 + 3(2.1) = 4.41 + 6.3 = 10.71\)

Step 3: Apply forward difference formula

\(f'(2) \approx \frac{f(2.1) - f(2)}{0.1} = \frac{10.71 - 10}{0.1} = \frac{0.71}{0.1} = 7.1\)

Step 4: Compare with exact value

Using the derivative rule: \(f'(x) = 2x + 3\)

Exact value: \(f'(2) = 2(2) + 3 = 7\)

Error: \(|7.1 - 7| = 0.1\)

\(f'(2) \approx 7.1\)
Final answer:

Using the forward difference method with \(h = 0.1\), the estimated value of \(f'(2)\) is 7.1, compared to the exact value of 7.

Applied rules:

• Forward difference formula: \(f'(x) \approx \frac{f(x+h) - f(x)}{h}\)

• Secant line approximation: Forward difference estimates tangent slope

• Error reduction: Smaller \(h\) reduces truncation error

2 Backward Difference Method
Exercise 2
Estimate \(f'(3)\) for \(f(x) = x^3 - 2x\) using the backward difference method with \(h = 0.1\)
Definition:

Backward difference method: Estimates the derivative using the formula \(f'(x) \approx \frac{f(x) - f(x-h)}{h}\), where \(h\) is a small positive increment.

Function
\(f(x) = x^3 - 2x\)
Point
\(x = 3\)
Increment
\(h = 0.1\)
Step 1: Calculate \(f(3)\)

\(f(3) = 3^3 - 2(3) = 27 - 6 = 21\)

Step 2: Calculate \(f(3 - h) = f(2.9)\)

\(f(2.9) = (2.9)^3 - 2(2.9) = 24.389 - 5.8 = 18.589\)

Step 3: Apply backward difference formula

\(f'(3) \approx \frac{f(3) - f(2.9)}{0.1} = \frac{21 - 18.589}{0.1} = \frac{2.411}{0.1} = 24.11\)

Step 4: Compare with exact value

Using the derivative rule: \(f'(x) = 3x^2 - 2\)

Exact value: \(f'(3) = 3(3^2) - 2 = 27 - 2 = 25\)

Error: \(|24.11 - 25| = 0.89\)

\(f'(3) \approx 24.11\)
Final answer:

Using the backward difference method with \(h = 0.1\), the estimated value of \(f'(3)\) is 24.11, compared to the exact value of 25.

Applied rules:

• Backward difference formula: \(f'(x) \approx \frac{f(x) - f(x-h)}{h}\)

• Secant line approximation: Backward difference estimates tangent slope

• Directional bias: Uses points to the left of the target point

3 Central Difference Method
Exercise 3
Estimate \(f'(1)\) for \(f(x) = e^x\) using the central difference method with \(h = 0.01\)
Definition:

Central difference method: Estimates the derivative using the formula \(f'(x) \approx \frac{f(x+h) - f(x-h)}{2h}\), providing a more accurate approximation than forward or backward differences.

Function
\(f(x) = e^x\)
Point
\(x = 1\)
Increment
\(h = 0.01\)
Step 1: Calculate \(f(1+h) = f(1.01)\)

\(f(1.01) = e^{1.01} \approx 2.74560104\)

Step 2: Calculate \(f(1-h) = f(0.99)\)

\(f(0.99) = e^{0.99} \approx 2.69153480\)

Step 3: Apply central difference formula

\(f'(1) \approx \frac{f(1.01) - f(0.99)}{2(0.01)} = \frac{2.74560104 - 2.69153480}{0.02}\)

\(f'(1) \approx \frac{0.05406624}{0.02} = 2.703312\)

Step 4: Compare with exact value

Using the derivative rule: \(f'(x) = e^x\)

Exact value: \(f'(1) = e^1 \approx 2.71828183\)

Error: \(|2.703312 - 2.71828183| \approx 0.015\)

Step 5: Compare with forward difference

Forward difference with \(h = 0.01\): \(\frac{e^{1.01} - e^1}{0.01} \approx 2.731922\)

Central difference error: ~0.015

Forward difference error: ~0.014

\(f'(1) \approx 2.703312\)
Final answer:

Using the central difference method with \(h = 0.01\), the estimated value of \(f'(1)\) is approximately 2.703312, compared to the exact value of \(e \approx 2.718282\).

Applied rules:

• Central difference formula: \(f'(x) \approx \frac{f(x+h) - f(x-h)}{2h}\)

• Higher accuracy: Central difference is typically more accurate than forward/backward

• Symmetric approach: Uses points on both sides of the target point

Rules and methods, laws,...
\(\text{Forward: } f'(x) \approx \frac{f(x+h) - f(x)}{h}\)
Numerical Differentiation
Method
Forward Difference
\(O(h)\) accuracy
Method
Backward Difference
\(O(h)\) accuracy
Method
Central Difference
\(O(h^2)\) accuracy
Key definitions:

Numerical differentiation: Techniques to estimate derivatives when analytical differentiation is difficult or impossible.

Forward difference: Uses points ahead of the target point to estimate the derivative.

Backward difference: Uses points behind the target point to estimate the derivative.

Central difference: Uses points on both sides of the target point to estimate the derivative.

Complete methodology:
  1. Select method: Choose forward, backward, or central difference based on available data
  2. Choose increment: Select an appropriately small value of \(h\)
  3. Calculate function values: Evaluate the function at required points
  4. Apply formula: Substitute values into the chosen difference formula
  5. Assess accuracy: Consider the trade-off between truncation and roundoff errors
Tip 1: Central difference is usually the most accurate method for smooth functions.
Tip 2: Choose \(h\) small enough for accuracy but large enough to avoid roundoff errors.
Tip 3: Forward difference is useful when function values are only available ahead of the point.
Tip 4: Backward difference is useful when function values are only available behind the point.
Common errors: Choosing \(h\) too large (truncation error) or too small (roundoff error), incorrect formula application, misunderstanding the direction of differences.
Exam preparation: Practice all three methods, understand error analysis, recognize when each method is most appropriate.
Formulas to know by heart:

• Forward difference: \(f'(x) \approx \frac{f(x+h) - f(x)}{h}\)

• Backward difference: \(f'(x) \approx \frac{f(x) - f(x-h)}{h}\)

• Central difference: \(f'(x) \approx \frac{f(x+h) - f(x-h)}{2h}\)

• Higher-order central difference: \(f'(x) \approx \frac{-f(x+2h) + 8f(x+h) - 8f(x-h) + f(x-2h)}{12h}\)

Solution: Exercises 4 to 5
4 Estimating from Tabular Data
Exercise 4
Given the table of values for function \(g(x)\), estimate \(g'(2)\) using the best available method:
xg(x)
1.83.24
1.93.61
2.04.00
2.14.41
2.24.84
Definition:

Tabular data estimation: When function values are given in a table, select the appropriate difference method based on available neighboring points.

Available data
x = 1.8, 1.9, 2.0, 2.1, 2.2
Target point
x = 2.0
Best method
Central difference
Step 1: Identify available data points

We have: \(g(1.8) = 3.24\), \(g(1.9) = 3.61\), \(g(2.0) = 4.00\), \(g(2.1) = 4.41\), \(g(2.2) = 4.84\)

Step 2: Choose the optimal method

Since we have points both before and after \(x = 2.0\), we can use the central difference method

With \(h = 0.1\): \(g'(2.0) \approx \frac{g(2.1) - g(1.9)}{2(0.1)}\)

Step 3: Apply central difference formula

\(g'(2.0) \approx \frac{g(2.1) - g(1.9)}{2(0.1)} = \frac{4.41 - 3.61}{0.2} = \frac{0.80}{0.2} = 4.0\)

Step 4: Verify with alternative methods

Forward difference: \(\frac{g(2.1) - g(2.0)}{0.1} = \frac{4.41 - 4.00}{0.1} = 4.1\)

Backward difference: \(\frac{g(2.0) - g(1.9)}{0.1} = \frac{4.00 - 3.61}{0.1} = 3.9\)

The central difference value of 4.0 is between these values, as expected

Step 5: Note the pattern

Looking at the data, it appears that \(g(x) = x^2\) since \(g(2.0) = 4.00 = 2^2\), \(g(2.1) = 4.41 = 2.1^2\), etc.

If \(g(x) = x^2\), then \(g'(x) = 2x\) and \(g'(2) = 4\)

Our estimate of 4.0 is very close to the actual value of 4!

\(g'(2) \approx 4.0\)
Final answer:

Using the central difference method with the tabular data, the estimated value of \(g'(2)\) is 4.0.

Applied rules:

• Data availability: Choose method based on available neighboring points

• Central difference: Best choice when points are available on both sides

• Accuracy comparison: Central difference typically provides better estimates

5 Error Analysis and Convergence
Exercise 5
For \(f(x) = \sin(x)\), estimate \(f'(\pi/4)\) using forward difference with \(h = 0.1, 0.01, 0.001\) and analyze the convergence toward the exact value.
Definition:

Error analysis: Examining how the accuracy of numerical estimates improves as the increment \(h\) decreases, balancing truncation and roundoff errors.

Function
\(f(x) = \sin(x)\)
Point
\(x = \pi/4\)
Exact value
\(f'(\pi/4) = \cos(\pi/4) = \frac{\sqrt{2}}{2}\)
Step 1: Calculate the exact value

For \(f(x) = \sin(x)\), we know that \(f'(x) = \cos(x)\)

Exact value: \(f'(\pi/4) = \cos(\pi/4) = \frac{\sqrt{2}}{2} \approx 0.70710678\)

Step 2: Apply forward difference with \(h = 0.1\)

\(f(\pi/4) = \sin(\pi/4) \approx 0.70710678\)

\(f(\pi/4 + 0.1) = \sin(\pi/4 + 0.1) \approx \sin(0.885398) \approx 0.777146\)

Estimate: \(\frac{0.777146 - 0.707107}{0.1} = \frac{0.070039}{0.1} = 0.70039\)

Error: \(|0.70039 - 0.707107| = 0.006717\)

Step 3: Apply forward difference with \(h = 0.01\)

\(f(\pi/4 + 0.01) = \sin(\pi/4 + 0.01) \approx \sin(0.800785) \approx 0.714141\)

Estimate: \(\frac{0.714141 - 0.707107}{0.01} = \frac{0.007034}{0.01} = 0.7034\)

Error: \(|0.7034 - 0.707107| = 0.003707\)

Step 4: Apply forward difference with \(h = 0.001\)

\(f(\pi/4 + 0.001) = \sin(\pi/4 + 0.001) \approx \sin(0.786175) \approx 0.707810\)

Estimate: \(\frac{0.707810 - 0.707107}{0.001} = \frac{0.000703}{0.001} = 0.703\)

Error: \(|0.703 - 0.707107| = 0.004107\)

Step 5: Analyze convergence pattern

As \(h\) decreases from 0.1 to 0.01, error decreases from 0.006717 to 0.003707

However, as \(h\) decreases further to 0.001, error increases to 0.004107

This indicates that roundoff error begins to dominate as \(h\) becomes very small

\(h=0.1: f' \approx 0.7004, h=0.01: f' \approx 0.7034, h=0.001: f' \approx 0.703\)
Final answer:

The forward difference estimates for \(f'(\pi/4)\) with decreasing \(h\) values show initial improvement but eventual degradation due to roundoff error, demonstrating the trade-off in numerical differentiation.

Applied rules:

• Truncation error: Decreases as \(h\) decreases for forward difference

• Roundoff error: Increases as \(h\) decreases due to division by small numbers

• Optimal h: Balance between truncation and roundoff errors

Key Estimating Derivatives Concepts, Theorems, and Applications
\(f'(x) \approx \frac{f(x+h) - f(x-h)}{2h} \text{ (central difference)}\)
Most Accurate Formula
Key definitions:

Numerical differentiation: The process of approximating the derivative of a function using discrete values of the function. It's essential when analytical derivatives are unavailable or when working with empirical data.

Truncation error: The error introduced by approximating the derivative using a finite difference instead of the limit definition. For forward/backward differences, it's proportional to \(h\); for central differences, it's proportional to \(h^2\).

Roundoff error: The error introduced by finite precision arithmetic when computing the difference quotient. It becomes significant when \(h\) is very small.

Condition number: Measures the sensitivity of the numerical derivative to small changes in function values, indicating potential numerical instability.

Complete methodology:
  1. Problem identification: Determine which method is most appropriate based on available data
  2. Increment selection: Choose \(h\) to balance truncation and roundoff errors
  3. Function evaluation: Calculate required function values accurately
  4. Formula application: Apply the appropriate difference formula
  5. Error assessment: Evaluate the quality of the approximation
Tip 1: Central difference is generally preferred when points are available on both sides of the target.
Tip 2: For smooth functions, use smaller \(h\); for noisy data, use larger \(h\).
Tip 3: Consider Richardson extrapolation to improve accuracy.
Tip 4: Verify results by comparing with multiple methods or known analytical results.
Tip 5: Be aware of the limitations when working with experimental or noisy data.
Common error patterns: Choosing inappropriate \(h\) values, not considering data quality, ignoring boundary effects, failing to account for noise amplification, misunderstanding the trade-offs between different methods.
Advanced applications: Numerical differentiation is fundamental to solving differential equations, optimization algorithms, and scientific computing applications where analytical derivatives are unavailable.
Essential theorems and properties:

• Error bounds: Forward/backward: \(O(h)\), Central: \(O(h^2)\), Higher-order: \(O(h^4)\) and beyond

• Stability condition: \(\frac{f(x+h) - f(x-h)}{2h}\) is stable for well-behaved functions

• Optimal increment: For double precision, \(h \approx \sqrt{\epsilon}\) where \(\epsilon\) is machine epsilon

• Consistency: As \(h \to 0\), numerical derivative approaches true derivative (if it exists)

Exercise with Visualization: Convergence of Numerical Derivatives
Exercise 6: Graphing Accuracy vs. Increment Size
For \(f(x) = x^2\) at \(x = 1\), plot the absolute error of forward, backward, and central difference methods as \(h\) varies from \(10^{-6}\) to \(10^{-1}\):
- Forward: \(\frac{f(1+h) - f(1)}{h}\)
- Backward: \(\frac{f(1) - f(1-h)}{h}\)
- Central: \(\frac{f(1+h) - f(1-h)}{2h}\)
- Exact: \(f'(1) = 2\)

Analysis: The chart shows how different methods exhibit different convergence behaviors:

  • Central difference shows quadratic convergence (\(O(h^2)\)) for a longer range
  • Forward and backward differences show linear convergence (\(O(h)\))
  • All methods eventually suffer from roundoff error as \(h\) becomes very small

Questions & Answers

Question: When should I use forward, backward, or central difference methods for estimating derivatives?

Answer: The choice depends on your situation:

  • Central difference: Use when possible, as it provides the highest accuracy (second-order) with the formula \(\frac{f(x+h) - f(x-h)}{2h}\)
  • Forward difference: Use when you only have access to future values or at the left boundary of your data set
  • Backward difference: Use when you only have access to past values or at the right boundary of your data set

In practice, if you have function values at \(x-h\), \(x\), and \(x+h\), always use the central difference method because it has superior accuracy properties compared to the first-order forward and backward methods.

Example: For a time series where you want to estimate the rate of change at time \(t\) and only have past data, use backward difference. If you have future data available, use forward difference. If you have both, use central difference.

Question: How do I choose the right value of \(h\) for numerical differentiation? How small is too small?

Answer: Choosing \(h\) involves balancing two competing errors:

Truncation error: Decreases as \(h\) gets smaller. For forward difference, this is approximately proportional to \(h\).

Roundoff error: Increases as \(h\) gets smaller because you're dividing by a very small number, which amplifies the inherent floating-point errors.

Guidelines for choosing \(h\):

  • For double precision (most common): \(h \approx \sqrt{\epsilon} \approx 10^{-8}\) where \(\epsilon \approx 10^{-16}\) is machine epsilon
  • Start with \(h = 0.001\) or \(h = 0.01\) for initial estimates
  • Test multiple values of \(h\) to see convergence behavior
  • Don't make \(h\) smaller than \(10^{-15}\) for double precision

The optimal \(h\) minimizes the total error, which is the sum of truncation and roundoff errors.

Question: How accurate are numerical derivative estimates compared to analytical derivatives, and when are they necessary?

Answer: The accuracy depends on the method and the choice of \(h\):

Accuracy comparison:

  • Forward/backward difference: Accurate to \(O(h)\) - error decreases linearly with \(h\)
  • Central difference: Accurate to \(O(h^2)\) - error decreases quadratically with \(h\)
  • Higher-order methods: Can achieve \(O(h^4)\), \(O(h^6)\), etc.
  • Analytical: Exact (within computational precision of constants)

When numerical methods are necessary:

1. When the function is only known at discrete points (experimental data)

2. When the analytical derivative is too complex to compute

3. When working with black-box functions where the analytical form is unknown

4. In computer implementations where symbolic differentiation isn't available

For smooth functions with known formulas, analytical derivatives are always preferred, but numerical methods serve as excellent checks and are essential in practical applications.

Question: How are numerical derivatives used in economics and finance for marginal analysis?

Answer: Numerical derivatives are crucial in economics and finance:

Marginal concepts:

  • Marginal cost: \(\frac{dC}{dq} \approx \frac{C(q+h) - C(q)}{h}\)
  • Marginal revenue: \(\frac{dR}{dq} \approx \frac{R(q+h) - R(q)}{h}\)
  • Marginal utility: \(\frac{dU}{dx} \approx \frac{U(x+h) - U(x)}{h}\)

Real-world applications:

1. Empirical cost functions: When cost data is collected empirically, numerical methods estimate marginal costs

2. Option pricing: Greeks like delta (\(\partial V/\partial S\)) are often computed numerically

3. Policy analysis: Estimating the marginal effect of policy changes using observed data

4. Production functions: Computing marginal products of inputs from discrete data points

In practice, economists often work with discrete data sets where analytical derivatives aren't directly available, making numerical methods essential for marginal analysis.

Question: What are higher-order numerical differentiation formulas, and when should they be used?

Answer: Higher-order formulas provide greater accuracy:

Third-order forward difference:

\(f'(x) \approx \frac{-f(x+2h) + 4f(x+h) - 3f(x)}{2h}\) with \(O(h^2)\) accuracy

Fourth-order central difference:

\(f'(x) \approx \frac{-f(x+2h) + 8f(x+h) - 8f(x-h) + f(x-2h)}{12h}\) with \(O(h^4)\) accuracy

When to use higher-order methods:

  • When extremely high accuracy is required
  • When computational cost is not a primary concern
  • When working with smooth functions (high differentiability)
  • For research applications requiring precise derivative estimates

Trade-offs:

Higher-order methods require more function evaluations and more distant points, which can increase roundoff error and reduce stability. They're also more sensitive to noise in the data. The fourth-order central difference is often a good compromise between accuracy and stability for smooth functions.