Approximating with Newton’s Method

Learning Outcomes

Describe the steps of Newton’s method
Explain what an iterative process means
Recognize when Newton’s method does not work

Describing Newton’s Method

Consider the task of finding the solutions of $f(x)=0$ . If $f$ is the first-degree polynomial $f(x)=ax+b$ , then the solution of $f(x)=0$ is given by the formula $x=-\frac{b}{a}$ . If $f$ is the second-degree polynomial $f(x)=ax^2+bx+c$ , the solutions of $f(x)=0$ can be found by using the quadratic formula. However, for polynomials of degree 3 or more, finding roots of $f$ becomes more complicated. Although formulas exist for third- and fourth-degree polynomials, they are quite complicated. Also, if $f$ is a polynomial of degree 5 or greater, it is known that no such formulas exist. For example, consider the function

f (x) = x^{5} + 8 x^{4} + 4 x^{3} - 2 x - 7

$f(x)=x^5+8x^4+4x^3-2x-7$

No formula exists that allows us to find the solutions of $f(x)=0$ . Similar difficulties exist for nonpolynomial functions. For example, consider the task of finding solutions of $\tan (x)-x=0$ . No simple formula exists for the solutions of this equation. In cases such as these, we can use Newton’s method to approximate the roots.

Newton’s method makes use of the following idea to approximate the solutions of $f(x)=0$ . By sketching a graph of $f$ , we can estimate a root of $f(x)=0$ . Let’s call this estimate $x_0$ . We then draw the tangent line to $f$ at $x_0$ . If $f^{\prime}(x_0)\ne 0$ , this tangent line intersects the $x$ -axis at some point $(x_1,0)$ . Now let $x_1$ be the next approximation to the actual root. Typically, $x_1$ is closer than $x_0$ to an actual root. Next we draw the tangent line to $f$ at $x_1$ . If $f^{\prime}(x_1)\ne 0$ , this tangent line also intersects the $x$ -axis, producing another approximation, $x_2$ . We continue in this way, deriving a list of approximations: $x_0, x_1, x_2, \cdots$ . Typically, the numbers $x_0,x_1,x_2, \cdots$ quickly approach an actual root $x*$ , as shown in the following figure.

This function f(x) is drawn with points (x0, f(x0)), (x1, f(x1)), and (x2, f(x2)) marked on the function. From (x0, f(x0)), a tangent line is drawn, and it strikes the x axis at x1. From (x0, f(x0)), a tangent line is drawn, and it strikes the x axis at x2. If a tangent line were drawn from (x2, f(x2)), it appears that it would come very close to x*, which is the actual root. Each tangent line drawn in this order appears to get closer and closer to x*.

Figure 1. The approximations $x_0,x_1,x_2, \cdots$ approach the actual root $x*$ . The approximations are derived by looking at tangent lines to the graph of $f$ .

Now let’s look at how to calculate the approximations $x_0,x_1,x_2, \cdots$ . If $x_0$ is our first approximation, the approximation $x_1$ is defined by letting $(x_1,0)$ be the $x$ -intercept of the tangent line to $f$ at $x_0$ . The equation of this tangent line is given by

y = f (x_{0}) + f^{'} (x_{0}) (x - x_{0})

$y=f(x_0)+f^{\prime}(x_0)(x-x_0)$

Therefore,

x_{1}

$x_1$ must satisfy

f (x_{0}) + f^{'} (x_{0}) (x_{1} - x_{0}) = 0

$f(x_0)+f^{\prime}(x_0)(x_1-x_0)=0$

Solving this equation for $x_1$ , we conclude that

x_{1} = x_{0} - \frac{f (x_{0})}{f^{'} (x_{0})}

$x_1=x_0-\dfrac{f(x_0)}{f^{\prime}(x_0)}$

Similarly, the point $(x_2,0)$ is the $x$ -intercept of the tangent line to $f$ at $x_1$ . Therefore, $x_2$ satisfies the equation

x_{2} = x_{1} - \frac{f (x_{1})}{f^{'} (x_{1})}

$x_2=x_1-\dfrac{f(x_1)}{f^{\prime}(x_1)}$

In general, for $n>0, \, x_n$ satisfies

x_{n} = x_{n - 1} - \frac{f (x_{n - 1})}{f^{'} (x_{n - 1})}

$x_n=x_{n-1}-\dfrac{f(x_{n-1})}{f^{\prime}(x_{n-1})}$

Next we see how to make use of this technique to approximate the root of the polynomial $f(x)=x^3-3x+1$ .

Example: Finding a Root of a Polynomial

Use Newton’s method to approximate a root of $f(x)=x^3-3x+1$ in the interval $[1,2]$ . Let $x_0=2$ and find $x_1,x_2,x_3,x_4$ , and $x_5$ .

Show Solution

From Figure 2, we see that $f$ has one root over the interval $(1,2)$ . Therefore $x_0=2$ seems like a reasonable first approximation. To find the next approximation, we use the equation we found for $x_n$ . Since $f(x)=x^3-3x+1$ , the derivative is $f^{\prime}(x)=3x^2-3$ . Using the equation for $x_n$ with $n=1$ (and a calculator that displays 10 digits), we obtain

$x_1=x_0-\dfrac{f(x_0)}{f^{\prime}(x_0)}=2-\dfrac{f(2)}{f^{\prime}(2)}=2-\dfrac{3}{9} \approx 1.666666667$

To find the next approximation, $x_2$ , we use (Figure) with $n=2$ and the value of $x_1$ stored on the calculator. We find that

$x_2=x_1=\dfrac{f(x_1)}{f^{\prime}(x_1)} \approx 1.548611111$

Continuing in this way, we obtain the following results:

$\begin{array}{l} x_1 \approx 1.666666667 \\ x_2 \approx 1.548611111 \\ x_3 \approx 1.532390162 \\ x_4 \approx 1.532088989 \\ x_5 \approx 1.532088886 \\ x_6 \approx 1.532088886 \end{array}$

We note that we obtained the same value for $x_5$ and $x_6$ . Therefore, any subsequent application of Newton’s method will most likely give the same value for $x_n$ .

The function f(x) = x3 – 3x + 1 is drawn. It has roots between −2 and −1, 0 and 1, and 1 and 2.

Figure 2. The function $f(x)=x^3-3x+1$ has one root over the interval $[1,2]$ .

Watch the following video to see the worked solution to Example: Finding a Root of a Polynomial.

Closed Captioning and Transcript Information for Video

Try It

Letting $x_0=0$ , use Newton’s method to approximate the root of $f(x)=x^3-3x+1$ over the interval $[0,1]$ by calculating $x_1$ and $x_2$ .

Hint

Use the equation for $x_n$ .

Show Solution

$x_1 \approx 0.33333333, \, x_2 \approx 0.347222222$

Newton’s method can also be used to approximate square roots. Here we show how to approximate $\sqrt{2}$ . This method can be modified to approximate the square root of any positive number.

example: Finding a Square Root

Use Newton’s method to approximate $\sqrt{2}$ (Figure 3). Let $f(x)=x^2-2$ , let $x_0=2$ , and calculate $x_1,x_2,x_3,x_4,x_5$ . (We note that since $f(x)=x^2-2$ has a zero at $\sqrt{2}$ , the initial value $x_0=2$ is a reasonable choice to approximate $\sqrt{2}$ .)

Show Solution

For $f(x)=x^2-2, \, f^{\prime}(x)=2x$ . From the equation for $x_n$ , we know that

\begin{matrix} x_{n} & = x_{n - 1} - \frac{f (x_{n - 1})}{f^{'} (x_{n - 1})} = x_{n - 1} - \frac{(x_{n - 1})^{2} - 2}{2 x_{n - 1}} = \frac{1}{2} x_{n - 1} + \frac{1}{x_{n - 1}} = \frac{1}{2} (x_{n - 1} + \frac{2}{x_{n - 1}}) \end{matrix}

$\begin{array}{ll} x_n & = x_{n-1}-\frac{f(x_{n-1})}{f^{\prime}(x_{n-1})} \\ & = x_{n-1}-\frac{(x_{n-1})^2-2}{2x_{n-1}} \\ & = \frac{1}{2}x_{n-1}+\frac{1}{x_{n-1}} \\ & = \frac{1}{2}(x_{n-1}+\frac{2}{x_{n-1}}) \end{array}$

Therefore,

\begin{matrix} x_{1} = \frac{1}{2} (x_{0} + \frac{2}{x_{0}}) = \frac{1}{2} (2 + \frac{2}{2}) = 1.5 x_{2} = \frac{1}{2} (x_{1} + \frac{2}{x_{1}}) = \frac{1}{2} (1.5 + \frac{2}{1.5}) \approx 1.416666667 \end{matrix}

$\begin{array}{l} x_1 = \frac{1}{2}(x_0+\frac{2}{x_0})=\frac{1}{2}(2+\frac{2}{2})=1.5 \\ x_2 = \frac{1}{2}(x_1+\frac{2}{x_1})=\frac{1}{2}(1.5+\frac{2}{1.5})\approx 1.416666667 \end{array}$

Continuing in this way, we find that

\begin{matrix} x_{1} = 1.5 x_{2} \approx 1.416666667 x_{3} \approx 1.414215686 x_{4} \approx 1.414213562 x_{5} \approx 1.414213562 \end{matrix}

$\begin{array}{l} x_1=1.5 \\ x_2 \approx 1.416666667 \\ x_3 \approx 1.414215686 \\ x_4 \approx 1.414213562 \\ x_5 \approx 1.414213562 \end{array}$

Since we obtained the same value for $x_4$ and $x_5$ , it is unlikely that the value $x_n$ will change on any subsequent application of Newton’s method. We conclude that $\sqrt{2} \approx 1.414213562$ .

The function y = x2 – 2 is drawn. A dashed line comes up from x0 = 2, and a tangent line is drawn down from there. It touches x1 = 1.5, which is near x* = the square root of 2.

Figure 3. We can use Newton’s method to find $\sqrt{2}$ .

Try It

Use Newton’s method to approximate $\sqrt{3}$ by letting $f(x)=x^2-3$ and $x_0=3$ . Find $x_1$ and $x_2$ .

Hint

For $f(x)=x^2-3$ , the equation for $x_n$ reduces to $x_n=\frac{x_{n-1}}{2}+\frac{3}{2x_{n-1}}$ .

Show Solution

$x_1=2, \, x_2=1.75$

Try It

When using Newton’s method, each approximation after the initial guess is defined in terms of the previous approximation by using the same formula. In particular, by defining the function $F(x)=x-\left[\frac{f(x)}{f^{\prime}(x)}\right]$ , we can rewrite the equation for $x_n$ as $x_n=F(x_{n-1})$ . This type of process, where each $x_n$ is defined in terms of $x_{n-1}$ by repeating the same function, is an example of an iterative process. Shortly, we examine other iterative processes. First, let’s look at the reasons why Newton’s method could fail to find a root.

Failures of Newton’s Method

Typically, Newton’s method is used to find roots fairly quickly. However, things can go wrong. Some reasons why Newton’s method might fail include the following:

At one of the approximations $x_n$ , the derivative $f^{\prime}$ is zero at $x_n$ , but $f(x_n) \ne 0$ . As a result, the tangent line of $f$ at $x_n$ does not intersect the $x$ -axis. Therefore, we cannot continue the iterative process.
The approximations $x_0,x_1,x_2, \cdots$ may approach a different root. If the function $f$ has more than one root, it is possible that our approximations do not approach the one for which we are looking, but approach a different root (see Figure 4). This event most often occurs when we do not choose the approximation $x_0$ close enough to the desired root.
The approximations may fail to approach a root entirely. In the example below, we provide an example of a function and an initial guess $x_0$ such that the successive approximations never approach a root because the successive approximations continue to alternate back and forth between two values.

A function is drawn with two roots, labeled root sought and root found. A point x0 is chosen such that when the tangent of x0 is taken, even though it is nearer to the root sought, the tangent points to the root found.

Figure 4. If the initial guess $x_0$ is too far from the root sought, it may lead to approximations that approach a different root.

example: When Newton’s Method Fails

Consider the function $f(x)=x^3-2x+2$ . Let $x_0=0$ . Show that the sequence $x_1,x_2, \cdots$ fails to approach a root of $f$ .

Show Solution

For $f(x)=x^3-2x+2$ , the derivative is $f^{\prime}(x)=3x^2-2$ . Therefore,

$x_1=x_0-\frac{f(x_0)}{f^{\prime}(x_0)}=0-\frac{f(0)}{f^{\prime}(0)}=-\frac{2}{-2}=1$ .

In the next step,

$x_2=x_1-\frac{f(x_1)}{f^{\prime}(x_1)}=1-\frac{f(1)}{f^{\prime}(1)}=1-\frac{1}{1}=0$ .

Consequently, the numbers $x_0,x_1,x_2, \cdots$ continue to bounce back and forth between 0 and 1 and never get closer to the root of $f$ which is over the interval $[-2,-1]$ (see Figure 5). Fortunately, if we choose an initial approximation $x_0$ closer to the actual root, we can avoid this situation.

The function f(x) = x3 – 2x + 2 is drawn, which has a root between −2 and −1. The tangent from x = 0 goes to x = 1, and the tangent from x = 1 goes to x = 0.

Figure 5. The approximations continue to alternate between 0 and 1 and never approach the root of $f$ .

Watch the following video to see the worked solution to Example: When Newton’s Method Fails.

Closed Captioning and Transcript Information for Video

Try It

For $f(x)=x^3-2x+2$ , let $x_0=-1.5$ and find $x_1$ and $x_2$ .

Hint

Use the equation for $x_n$ .

Show Solution

$x_1 \approx -1.842105263, \, x_2 \approx -1.772826920$

From the example above, we see that Newton’s method does not always work. However, when it does work, the sequence of approximations approaches the root very quickly. Discussions of how quickly the sequence of approximations approach a root found using Newton’s method are included in texts on numerical analysis.

Module 4: Applications of Derivatives