Solved Exercises on Newton's Method Introduction

Master Newton's method: iterative approximation of roots, convergence analysis, and practical applications through these 5 detailed exercises.

Newton's Method: Fundamentals & Exercises 1-3
\(x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)}\)
Newton's Method Formula
Definition:

Newton's Method: An iterative algorithm for finding successively better approximations to the roots (or zeros) of a real-valued function.

Newton's Method Algorithm:
  1. Start: Choose an initial guess \(x_0\)
  2. Compute: \(f(x_n)\) and \(f'(x_n)\)
  3. Update: Apply formula \(x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)}\)
  4. Repeat: Until convergence or maximum iterations
Tip 1: Choose initial guess close to actual root for faster convergence.
Tip 2: Ensure \(f'(x_n) \neq 0\) at each iteration.
1 Basic Square Root Approximation
Exercise 1
Use Newton's method to approximate \(\sqrt{2}\) starting with \(x_0 = 1\). Find the first 3 iterations.
Problem Setup:

To find \(\sqrt{2}\), solve \(f(x) = x^2 - 2 = 0\)

Derivative: \(f'(x) = 2x\)

Newton's Method Application:

Formula: \(x_{n+1} = x_n - \frac{x_n^2 - 2}{2x_n} = \frac{x_n^2 + 2}{2x_n}\)

Initial
\(x_0 = 1\)
Iteration 1
\(x_1 = 1.5\)
Iteration 2
\(x_2 = 1.4167\)
Step 1: Set up the equation

For \(\sqrt{2}\), let \(f(x) = x^2 - 2\), so we want \(f(x) = 0\)

Derivative: \(f'(x) = 2x\)

Step 2: Apply Newton's formula

\(x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)} = x_n - \frac{x_n^2 - 2}{2x_n}\)

\(x_{n+1} = \frac{2x_n^2 - x_n^2 + 2}{2x_n} = \frac{x_n^2 + 2}{2x_n}\)

Step 3: First iteration

\(x_0 = 1\)

\(x_1 = \frac{1^2 + 2}{2(1)} = \frac{3}{2} = 1.5\)

Step 4: Second iteration

\(x_1 = 1.5\)

\(x_2 = \frac{(1.5)^2 + 2}{2(1.5)} = \frac{2.25 + 2}{3} = \frac{4.25}{3} = 1.4167\)

Step 5: Third iteration

\(x_2 = 1.4167\)

\(x_3 = \frac{(1.4167)^2 + 2}{2(1.4167)} = \frac{2.0069 + 2}{2.8334} = \frac{4.0069}{2.8334} = 1.4142\)

\(\sqrt{2} \approx 1.4142\) (after 3 iterations)
Results:

\(x_0 = 1.0000\), \(x_1 = 1.5000\), \(x_2 = 1.4167\), \(x_3 = 1.4142\)

True value: \(\sqrt{2} = 1.41421356...\)

Applied Rules:

• Newton's Formula: \(x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)}\)

• Convergence: Quadratic convergence near the root

• Derivative Calculation: Proper differentiation is crucial

2 Cube Root Approximation
Exercise 2
Approximate \(\sqrt[3]{5}\) using Newton's method with \(x_0 = 2\). Perform 2 iterations.
Problem Setup:

To find \(\sqrt[3]{5}\), solve \(f(x) = x^3 - 5 = 0\)

Derivative: \(f'(x) = 3x^2\)

Initial
\(x_0 = 2\)
Iteration 1
\(x_1 = 1.75\)
Iteration 2
\(x_2 = 1.710\)
Step 1: Set up the equation

For \(\sqrt[3]{5}\), let \(f(x) = x^3 - 5\), so we want \(f(x) = 0\)

Derivative: \(f'(x) = 3x^2\)

Step 2: Apply Newton's formula

\(x_{n+1} = x_n - \frac{x_n^3 - 5}{3x_n^2} = \frac{3x_n^3 - x_n^3 + 5}{3x_n^2} = \frac{2x_n^3 + 5}{3x_n^2}\)

Step 3: First iteration

\(x_0 = 2\)

\(x_1 = \frac{2(2)^3 + 5}{3(2)^2} = \frac{16 + 5}{12} = \frac{21}{12} = 1.75\)

Step 4: Second iteration

\(x_1 = 1.75\)

\(x_2 = \frac{2(1.75)^3 + 5}{3(1.75)^2} = \frac{2(5.3594) + 5}{3(3.0625)} = \frac{10.7188 + 5}{9.1875} = \frac{15.7188}{9.1875} = 1.710\)

\(\sqrt[3]{5} \approx 1.710\) (after 2 iterations)
Results:

\(x_0 = 2.000\), \(x_1 = 1.750\), \(x_2 = 1.710\)

True value: \(\sqrt[3]{5} = 1.70997594...\)

Applied Rules:

• Power Rule: \(\frac{d}{dx}(x^n) = nx^{n-1}\)

• Cube Root Equation: Set \(x^3 - a = 0\) for \(\sqrt[3]{a}\)

• Algebraic Simplification: Combine terms carefully

3 Trigonometric Root Finding
Exercise 3
Find the positive root of \(f(x) = \cos(x) - x\) near \(x_0 = 0.7\). Perform 2 iterations.
Problem Setup:

Solve \(f(x) = \cos(x) - x = 0\)

Derivative: \(f'(x) = -\sin(x) - 1\)

Initial
\(x_0 = 0.7\)
Iteration 1
\(x_1 = 0.739\)
Iteration 2
\(x_2 = 0.739\)
Step 1: Set up the equation

We want to solve \(\cos(x) = x\), so \(f(x) = \cos(x) - x\)

Derivative: \(f'(x) = -\sin(x) - 1\)

Step 2: Apply Newton's formula

\(x_{n+1} = x_n - \frac{\cos(x_n) - x_n}{-\sin(x_n) - 1} = x_n + \frac{\cos(x_n) - x_n}{\sin(x_n) + 1}\)

Step 3: First iteration

\(x_0 = 0.7\)

\(f(0.7) = \cos(0.7) - 0.7 = 0.7648 - 0.7 = 0.0648\)

\(f'(0.7) = -\sin(0.7) - 1 = -0.6442 - 1 = -1.6442\)

\(x_1 = 0.7 - \frac{0.0648}{-1.6442} = 0.7 + 0.0394 = 0.7394\)

Step 4: Second iteration

\(x_1 = 0.7394\)

\(f(0.7394) = \cos(0.7394) - 0.7394 = 0.7391 - 0.7394 = -0.0003\)

\(f'(0.7394) = -\sin(0.7394) - 1 = -0.6736 - 1 = -1.6736\)

\(x_2 = 0.7394 - \frac{-0.0003}{-1.6736} = 0.7394 - 0.0002 = 0.7392\)

Root ≈ 0.739 (Dottie number)
Results:

\(x_0 = 0.7000\), \(x_1 = 0.7394\), \(x_2 = 0.7392\)

This converges to the Dottie number ≈ 0.739085

Applied Rules:

• Trigonometric Derivatives: \(\frac{d}{dx}[\cos(x)] = -\sin(x)\)

• Numerical Precision: Track significant digits carefully

• Convergence Check: Monitor differences between iterations

Newton's Method: Theory & Properties
\(x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)}\)
Newton's Method Formula
Key Definitions:

Root/Zeros: Values where \(f(x) = 0\)

Tangent Line: Line touching curve at one point with slope \(f'(x)\)

Convergence: Sequence approaches exact solution

Convergence Analysis:

Quadratic Convergence: Error squared at each step when \(f'(root) \neq 0\)

Linear Convergence: When derivative is zero at root

Linearization
\(L(x) = f(x_0) + f'(x_0)(x - x_0)\)
Approximation near \(x_0\)
Error Bound
\(|x_{n+1} - r| \leq C|x_n - r|^2\)
Quadratic convergence
Geometric
\(x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)}\)
Intersection with x-axis
Advantages: Fast convergence, simple implementation, works well for smooth functions.
Limitations: Requires derivative, may diverge, needs good initial guess.
Tip 1: Plot the function first to visualize roots and choose initial guess.
Tip 2: Check that \(f'(x_n) \neq 0\) to avoid division by zero.
Tip 3: Stop when consecutive iterations differ by less than tolerance.
Tip 4: Newton's method finds only one root per initial guess.
Conditions for Convergence:

• Continuity: \(f\) and \(f'\) must be continuous near root

• Non-zero derivative: \(f'(r) \neq 0\) at the root

• Initial proximity: Initial guess must be sufficiently close

• Function behavior: Function should be well-behaved

Advanced Applications: Exercises 4-5
4 Logarithmic Equation
Exercise 4
Solve \(x \ln(x) = 1\) using Newton's method with \(x_0 = 2\). Find 2 iterations.
Problem Setup:

Rewrite as \(f(x) = x \ln(x) - 1 = 0\)

Using product rule: \(f'(x) = \ln(x) + 1\)

Initial
\(x_0 = 2\)
Iteration 1
\(x_1 = 1.772\)
Iteration 2
\(x_2 = 1.763\)
Step 1: Rewrite the equation

\(x \ln(x) = 1\) becomes \(f(x) = x \ln(x) - 1 = 0\)

Derivative: \(f'(x) = \ln(x) + x \cdot \frac{1}{x} = \ln(x) + 1\)

Step 2: Apply Newton's formula

\(x_{n+1} = x_n - \frac{x_n \ln(x_n) - 1}{\ln(x_n) + 1}\)

Step 3: First iteration

\(x_0 = 2\)

\(f(2) = 2 \ln(2) - 1 = 2(0.6931) - 1 = 1.3863 - 1 = 0.3863\)

\(f'(2) = \ln(2) + 1 = 0.6931 + 1 = 1.6931\)

\(x_1 = 2 - \frac{0.3863}{1.6931} = 2 - 0.2282 = 1.7718\)

Step 4: Second iteration

\(x_1 = 1.7718\)

\(f(1.7718) = 1.7718 \ln(1.7718) - 1 = 1.7718(0.5722) - 1 = 1.0139 - 1 = 0.0139\)

\(f'(1.7718) = \ln(1.7718) + 1 = 0.5722 + 1 = 1.5722\)

\(x_2 = 1.7718 - \frac{0.0139}{1.5722} = 1.7718 - 0.0088 = 1.7630\)

Solution: \(x \approx 1.763\) where \(x \ln(x) = 1\)
Results:

\(x_0 = 2.000\), \(x_1 = 1.772\), \(x_2 = 1.763\)

The exact solution is approximately 1.7632228343518967

Applied Rules:

• Product Rule: \(\frac{d}{dx}[uv] = u'v + uv'\)

• Logarithmic Derivative: \(\frac{d}{dx}[\ln(x)] = \frac{1}{x}\)

• Algebraic Manipulation: Simplify complex expressions

5 Failure Case Analysis
Exercise 5
Apply Newton's method to \(f(x) = x^{1/3}\) with \(x_0 = 1\). Analyze why it fails.
Problem Setup:

\(f(x) = x^{1/3}\), so we want \(f(x) = 0\) which means \(x = 0\)

Derivative: \(f'(x) = \frac{1}{3}x^{-2/3} = \frac{1}{3x^{2/3}}\)

Initial
\(x_0 = 1\)
Iteration 1
\(x_1 = -2\)
Iteration 2
\(x_2 = 4\)
Step 1: Set up the function

\(f(x) = x^{1/3}\) has root at \(x = 0\)

Derivative: \(f'(x) = \frac{1}{3}x^{-2/3} = \frac{1}{3x^{2/3}}\)

Step 2: Apply Newton's formula

\(x_{n+1} = x_n - \frac{x_n^{1/3}}{\frac{1}{3x_n^{2/3}}} = x_n - 3x_n^{1/3} \cdot x_n^{2/3} = x_n - 3x_n = -2x_n\)

Step 3: First iteration

\(x_0 = 1\)

\(x_1 = -2(1) = -2\)

Step 4: Second iteration

\(x_1 = -2\)

\(x_2 = -2(-2) = 4\)

Step 5: Pattern identification

Each iteration: \(x_{n+1} = -2x_n\), so sequence is: \(1, -2, 4, -8, 16, ...\)

The method diverges exponentially!

Newton's method diverges: \(|x_n| \to \infty\)
Analysis:

The method fails because \(f'(0) = \infty\) (not defined), violating convergence conditions.

The function \(x^{1/3}\) is not differentiable at its root.

Failure Causes:

• Zero derivative: \(f'(r) = 0\) at root

• Undefined derivative: Function not differentiable at root

• Poor initial guess: Too far from actual root

• Oscillatory behavior: Function changes concavity rapidly

Newton's Method: Comprehensive Guide
\(x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)}\)
Newton's Method Formula
Key definitions:

Newton's Method: Iterative algorithm for finding roots of \(f(x) = 0\)

Root: Value \(r\) where \(f(r) = 0\)

Tangent Line: Linear approximation of function at current point

Complete methodology:
  1. Problem setup: Write equation in form \(f(x) = 0\)
  2. Derivative calculation: Find \(f'(x)\) analytically
  3. Initial guess: Choose \(x_0\) reasonably close to expected root
  4. Iteration loop: Apply Newton's formula repeatedly
  5. Convergence check: Stop when desired accuracy reached
Tip 1: Plot the function to visualize roots and select good initial guesses.
Tip 2: Always verify results by substituting back into original equation.
Tip 3: Be aware of multiple roots - try different initial guesses.
Tip 4: Set maximum iteration count to prevent infinite loops.
Common errors: Incorrect derivative, poor initial guess, division by zero, not checking convergence.
Exam preparation: Practice derivative calculations, understand convergence conditions, recognize failure cases.
Formulas to know by heart:

• Newton's formula: \(x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)}\)

• Linearization: \(L(x) = f(a) + f'(a)(x-a)\)

• Error relation: \(|x_{n+1} - r| \approx C|x_n - r|^2\)

• Convergence condition: \(|f'(r)| > 0\) and \(x_0\) close enough

Visualization: Newton's Method Geometric Interpretation
Exercise 6: Tangent Line Approximations
Visualize Newton's method geometrically for \(f(x) = x^2 - 2\):
  • First tangent line at \(x_0 = 2\)
  • Second tangent line at \(x_1 = 1.5\)
  • Third tangent line at \(x_2 = 1.4167\)

Geometric interpretation: Each iteration follows the tangent line to the x-axis.

  • The tangent line at \((x_n, f(x_n))\) has slope \(f'(x_n)\)
  • It intersects the x-axis at \(x_{n+1}\)
  • This provides increasingly accurate approximations to the root

Questions & Answers

Question: I'm confused about why Newton's method has quadratic convergence. Can you explain the mathematical reasoning behind this rapid convergence rate?

Answer: Great question! The quadratic convergence of Newton's method comes from Taylor series analysis:

If we denote the error at step n as \(e_n = x_n - r\) (where r is the root), then by Taylor expansion:

\(f(x_n) = f(r + e_n) = f(r) + f'(r)e_n + \frac{f''(r)}{2}e_n^2 + O(e_n^3)\)

Since \(f(r) = 0\): \(f(x_n) = f'(r)e_n + \frac{f''(r)}{2}e_n^2 + O(e_n^3)\)

And: \(f'(x_n) = f'(r) + f''(r)e_n + O(e_n^2)\)

Substituting into Newton's formula and simplifying (assuming \(f'(r) \neq 0\)):

\(e_{n+1} = x_{n+1} - r = \frac{f''(r)}{2f'(r)}e_n^2 + O(e_n^3)\)

This shows that the error at the next step is proportional to the square of the current error, which is quadratic convergence: \(|e_{n+1}| \approx C|e_n|^2\).

This explains why Newton's method doubles the number of correct digits with each iteration when close to the root.

Question: In Exercise 5, Newton's method failed for \(f(x) = x^{1/3}\). Are there modified versions of Newton's method that can handle such cases?

Answer: Yes, there are several modifications that can handle cases where the standard Newton's method fails:

  • Modified Newton's Method: Uses a constant derivative value \(f'(x_0)\) instead of recomputing at each step: \(x_{n+1} = x_n - \frac{f(x_n)}{f'(x_0)}\)
  • Secant Method: Approximates the derivative using two previous points, avoiding analytical derivative computation
  • Halley's Method: Uses second derivatives for cubic convergence: \(x_{n+1} = x_n - \frac{2f(x_n)f'(x_n)}{2[f'(x_n)]^2 - f(x_n)f''(x_n)}\)
  • Robust variants: Combine Newton's method with bisection to guarantee convergence

For the specific case of \(f(x) = x^{1/3}\), even the modified methods may struggle due to the vertical tangent at the root. The key insight is that the function doesn't satisfy the smoothness requirements for Newton's method to work effectively.

Question: How do I choose a good initial guess \(x_0\) for Newton's method? Sometimes I get unexpected results.

Answer: Choosing a good initial guess is crucial for Newton's method success. Here are effective strategies:

  • Graphical approach: Plot the function to visually identify where it crosses the x-axis
  • Physical intuition: Use domain knowledge to estimate reasonable values
  • Bracketing: Find values where \(f(x)\) changes sign to locate intervals containing roots
  • Simple approximation: Use linear approximations or known special cases

Additionally, consider these guidelines:

  • Avoid initial guesses where \(f'(x_0) \approx 0\) as this causes large jumps
  • Stay away from points of inflection where concavity changes dramatically
  • For multiple roots, try several different initial guesses to find all solutions

If you get unexpected results, plot the function and trace the first few iterations geometrically to understand the behavior.

Question: How does Newton's method compare to other root-finding algorithms like bisection or secant methods?

Answer: Each method has distinct advantages and trade-offs:

Method Convergence Requirements Guarantees
Bisection Linear Continuous function, bracketing interval Always converges
Secant Superlinear (~1.6) Two initial guesses Often converges
Newton Quadratic Function and derivative Local convergence

Newton's method is fastest when it works, but requires more stringent conditions. Bisection is most reliable but slowest. Secant method offers a compromise between speed and reliability.

Question: Can Newton's method be used for optimization problems? I've heard it relates to finding minimums and maximums.

Answer: Yes! Newton's method is commonly used for optimization. To find extrema of a function \(g(x)\), you apply Newton's method to find where the derivative is zero (critical points):

To minimize/maximize \(g(x)\), solve \(g'(x) = 0\) using Newton's method:

\(x_{n+1} = x_n - \frac{g'(x_n)}{g''(x_n)}\)

This is called Newton's method for optimization. The second derivative \(g''(x)\) helps determine whether the critical point is a minimum (if \(g''(x) > 0\)) or maximum (if \(g''(x) < 0\)).

This approach is very effective for optimization problems, especially when the objective function is smooth and well-behaved. It's widely used in machine learning for training models and in engineering for design optimization.