Abstract

The inexact perturbed Newton methods recently introduced by us are variant of Newton method, which assume that at each step the linear systems are perturbed, and then they are only approximately solved. The q-convergence orders of the iterates were characterized using the results of Dembo, Eisenstat and Steihaug on inexact Newton methods.

In this note we deduce, in the same manner, the characterization of the r-convergence orders of these iterates.

Authors

Emil Cătinaş
(Tiberiu Popoviciu Institute of Numerical Analysis, Romanian Academy)

Keywords

inexact perturbed Newton methods; nonlinear systems in Rn; convergence orders; r-convergence order; forcing terms.

References

PDF

Cite this paper as:

E. Cătinaş, On the r-convergence orders of the inexact perturbed Newton methods, Bul. ştiinţ. Univ. Baia Mare, Ser. B, Mat. Inf., 15 (1999) no. 1-2, pp. 75-78.

About this paper

Publisher Name

Universitatea Baia Mare

Print ISSN

Not available yet.

Online ISSN

Not available yet.

Google Scholar citations

Google Scholar citations

Note

??

Paper (preprint) in HTML form

carpathian_1999_15_075_078

Dedicated to Professor lon PAVĂLOIU on his 60'h anniversary

On the r r rrr-convergence orders of the inexact perturbed Newton methods

Emil. Cátinas

Abstract

A hstract The inexact perturbed Newton methods rocently introduced by us are variant of Newton method, which assume that at exuh steep the linear systems are perturbed, aud then they are only approximntoly solved.

The q-convergence orders of the iterates were characterizod using the results of Dembo, Eisenstat and Streihang on inexact Newton methods.

In this note we deduce, in the same manner, the characterization of the r r rrr-convergence orders of these iterates.

1 Introduction.

Given an open set D R n D R n D subeR^(n)D \subseteq \mathbb{R}^{n}DRn and F : D R n F : D R n F:D rarrR^(n)F: D \rightarrow \mathbb{R}^{n}F:DRn, the Newton method
x k 1 = x k F ( x k ) 1 F ( x k ) , k = 0 , 1 , , x 0 D , x k 1 = x k F x k 1 F x k , k = 0 , 1 , , x 0 D , x_(k-1)=x_(k)-F^(')(x_(k))^(-1)F(x_(k)),quad k=0,1,dots,quadx_(0)in D,x_{k-1}=x_{k}-F^{\prime}\left(x_{k}\right)^{-1} F\left(x_{k}\right), \quad k=0,1, \ldots, \quad x_{0} \in D,xk1=xkF(xk)1F(xk),k=0,1,,x0D,
is a classical way which allows in certain circumstances the approximation of a solution x x x^(**)x^{*}x of the nonlinear system F ( x ) = 0 F ( x ) = 0 F(x)=0F(x)=0F(x)=0.
Its local convergence is usually studied in the following hypotheacs:
(C1) there exists x R n x R n x^(**)inR^(n)x^{*} \in \mathbb{R}^{n}xRn such that F ( x ) = 0 F x = 0 F(x^(**))=0F\left(x^{*}\right)=0F(x)=0;
(C2) the mapping F F FFF is differentiable on a neighborhood of x x x^(**)x^{*}x and F F F^(**)F^{*}F is conttinuous at x x x^(')x^{\prime}x;
(C3) the Jacobian F ( x ) F x F^(')(x^(**))F^{\prime}\left(x^{*}\right)F(x) is nonsingular;
(C4) the derivative ξ v ξ v xi^(v)\xi^{v}ξv is Holder continuous with exponent p ( 0 , 1 p 0 , 1 p in(0,1∣:}p \in\left(0,1 \mid\right.p(0,1 at x x x^(**)x^{*}x, i.c. for an arbitrary fixexd norm ||-||\|-\| on R n R n R^(n)\mathbb{R}^{n}Rn there exist L , ε > 0 L , ε > 0 L,epsi > 0L, \varepsilon>0L,ε>0 such that
F ( x ) F ( x ) < x x p , when x x < ε . F ( x ) F x < x x p ,  when  x x < ε . ||F^(')(x)-F^(')(x^)|| < ||x-x^(**)||^(p),quad" when "||x-x^(**)|| < epsi.\left\|F^{\prime}(x)-F^{\prime}\left(x^{\wedge}\right)\right\|<\left\|x-x^{*}\right\|^{p}, \quad \text { when }\left\|x-x^{*}\right\|<\varepsilon .F(x)F(x)<xxp, when xx<ε.
Before enouncing the known convergence results concerning the Newton iterates, we briefly remind the definitions conourning the convergence orders of the sexulences. An arbitrary soxucnoc ( y k ) k > 0 R y k k > 0 R (y_(k))_(k > 0)subR^('')\left(y_{k}\right)_{k>0} \subset \mathbb{R}^{\prime \prime}(yk)k>0R is said that converges q q q^(-)q^{-}q superlinearly to its limit y ¯ R n y ¯ R n bar(y)inR^(n)\bar{y} \in \mathbb{R}^{n}y¯Rn if
lim k y k + 1 y ¯ y k y ¯ = 0 , when y k y ¯ , k k 0 , lim k y k + 1 y ¯ y k y ¯ = 0 ,  when  y k y ¯ , k k 0 , lim_(k rarr oo)(||y_(k+1)-( bar(y))||)/(||y_(k)-( bar(y))||)=0,quad" when "y_(k)!= bar(y),k >= k_(0),\lim _{k \rightarrow \infty} \frac{\left\|y_{k+1}-\bar{y}\right\|}{\left\|y_{k}-\bar{y}\right\|}=0, \quad \text { when } y_{k} \neq \bar{y}, k \geq k_{0},limkyk+1y¯yky¯=0, when yky¯,kk0,

76

and with q q qqq-convergence order α > 1 α > 1 alpha > 1\alpha>1α>1 if
lim sup k y k + 1 y ¯ y k y ¯ ω < + , when y k y ¯ 1 k k 0 lim sup k y k + 1 y ¯ y k y ¯ ω < + ,  when  y k y ¯ 1 k k 0 l i m   s u p_(k rarr oo)(||y_(k+1)-( bar(y))||)/(||y_(k)-( bar(y))||^(omega)) < +oo,quad" when "y_(k)!= bar(y)_(1)k >= k_(0)\limsup _{k \rightarrow \infty} \frac{\left\|y_{k+1}-\bar{y}\right\|}{\left\|y_{k}-\bar{y}\right\|^{\omega}}<+\infty, \quad \text { when } y_{k} \neq \bar{y}_{1} k \geq k_{0}lim supkyk+1y¯yky¯ω<+, when yky¯1kk0
The sexuence ( y k ) k 0 R n y k k 0 R n (y_(k))_(k >= 0)subR^(n)\left(y_{k}\right)_{k \geq 0} \subset \mathbb{R}^{n}(yk)k0Rn is said that converges r r rrr-supcrlincarly to its limit y ¯ R n y ¯ R n bar(y)inR^(n)\bar{y} \in \mathbf{R}^{n}y¯Rn if
lim k y k y 1 / k = 0 lim k y k y 1 / k = 0 lim_(k rarr oo)||y_(k)-y||^(1//k)=0\lim _{k \rightarrow \infty}\left\|y_{k}-y\right\|^{1 / k}=0limkyky1/k=0
and with r r rrr-convergence order α > 1 α > 1 alpha > 1\alpha>1α>1 if
lim sup k y k y ¯ 1 / α 2 < 1 , when y k y 1 k k 0 . lim sup k y k y ¯ 1 / α 2 < 1 ,  when  y k y 1 ¯ k k 0 . l i m   s u p_(k rarr oo)||y_(k)-( bar(y))||^(1//alpha^(2)) < 1,quad" when "y_(k)!= bar(y_(1))k >= k_(0).\limsup _{k \rightarrow \infty}\left\|y_{k}-\bar{y}\right\|^{1 / \alpha^{2}}<1, \quad \text { when } y_{k} \neq \overline{y_{1}} k \geq k_{0} .lim supkyky¯1/α2<1, when yky1kk0.
These definitions are more rigorously treated in the classical book of Ortega and Rheinboldt [12, ch.9] (see also [15], [14]).
Conditions (C1)-(C3) assure an attraction theorem for the Newton iterates: there exists ε > 0 ε > 0 epsi > 0\varepsilon>0ε>0 such that for any initial approximation x 0 D x 0 D x_(0)in Dx_{0} \in Dx0D with x 0 x < ε x 0 x < ε ||x_(0)-x^|| < epsi\left\|x_{0}-x^{\wedge}\right\|<\varepsilonx0x<ε, the Newton iterates ( x k ) k 0 x k k 0 (x_(k))_(k >= 0)\left(x_{k}\right)_{k \geq 0}(xk)k0 are well delined, remain in the ball of center x x x^(**)x^{*}x and radius c c ccc, and converge y y yyy-superlincarly to x x x^(**)x^{*}x (sce 12 12 ∣12\mid 1212, Th. 10.2 .2 ]). Under the additional condition (C4), the convergence is with q q qqq-order 1 + p 1 + p 1+p1+p1+p.
The y y yyy-convergence orders are more general than the q q qqq-convergence orders, in the sense that a sequence converging q q qqq-superlinearly converges also r r rrr-superlincarly, and when it converges with q q qqq-order α > 1 α > 1 alpha > 1\alpha>1α>1, it also converges with r r rrr-order α α alpha\alphaα (the converse being not true as a general affirmation). Therefore, in the conditions of the above attraction theorem, we automatically obtain r r rrr-superlinear convergence, resp, convergence with r r rrr-orders 1 + p 1 + p 1+p1+p1+p of the Newton iterates.

2 The r r rrr-convergence orders of the inexact perturbed Newton methods.

When used in practice, the Newton method results in the following algorithm:
Choose an initial approximation x 0 D x 0 D x_(0)in Dx_{0} \in Dx0D
For k = 0 , 1 , k = 0 , 1 , k=0,1,dotsk=0,1, \ldotsk=0,1, until "convergence" do
Solve F ( x k ) s k = F ( x k ) F x k s k = F x k F^(')(x_(k))s_(k)=-F(x_(k))F^{\prime}\left(x_{k}\right) s_{k}=-F\left(x_{k}\right)F(xk)sk=F(xk)
Set x k 1 = x k + s k x k 1 = x k + s k x_(k-1)=x_(k)+s_(k)x_{k-1}=x_{k}+s_{k}xk1=xk+sk.
Each step requires the solving of a linear system, which is usually a difficult task. There exist two main approaches in order to overcome this difficulty. The first one considers some linear systems easier to solve, by perturbing the matrices; the resulting itcrations are called quasi-Newton methods. The second approach considers that the linear systems are not solved exactly;
Choose an initial approximation x 0 D x 0 D x_(0)sub Dx_{0} \subset Dx0D
For k = 0 , 1 , k = 0 , 1 , k=0,1,dotsk=0,1, \ldotsk=0,1, until "convergence" do
Find s k s k s_(k)s_{k}sk such that F ( x k ) s k = F ( x k ) + r k F x k s k = F x k + r k F^(')(x_(k))s_(k)=-F(x_(k))+r_(k)F^{\prime}\left(x_{k}\right) s_{k}=-F\left(x_{k}\right)+r_{k}F(xk)sk=F(xk)+rk
Set x k + 1 = x k + y k x k + 1 = x k + y k x_(k+1)=x_(k)+y_(k)x_{k+1}=x_{k}+y_{k}xk+1=xk+yk.
The error terms (the residuals) r k r k r_(k)r_{k}rk represent the amounts by which the solutions s 8 s 8 s_(8)s_{8}s8 fail to satisfy the exact linear systems. Dembo, Eisenstat and Stcibaug characterized the convergence orders of the incxact Newton (IN) method above. We remind the result dealing with the r r rrr-convergence orders.
Theorem 1 [7] Assume that conditions (C1)-(C4) hold and the IN ilerales ( x k ) k 0 x k k 0 (x_(k))_(k >= 0)\left(x_{k}\right)_{k \geq 0}(xk)k0 consuerge to x x x^x^{\wedge}x. Then the comoungence is wath r-oruler at least 1 + p 1 + p 1+p1+p1+p if and ondy if r k 0 r k 0 r_(k)rarr0\mathrm{r}_{k} \rightarrow 0rk0 with r r rrr-comoveryence order 1 + p 1 + p 1+p1+p1+p as k k k rarr ook \rightarrow \inftyk;
We have recently proposed in [4] a new model for the Newton methods, which reflexts the different situations that usually arise in praction:
( F ( x k ) + Δ k ) s k = ( F ( x k ) + δ k ) + r ˙ k x k + 1 = x k + s k , k = 0 , 1 , , x 0 D . F x k + Δ k s k = F x k + δ k + r ˙ k x k + 1 = x k + s k , k = 0 , 1 , , x 0 D . {:[(F^(')(x_(k))+Delta_(k))s_(k)=(-F(x_(k))+delta_(k))+r^(˙)_(k)],[x_(k+1)=x_(k)+s_(k)","quad k=0","1","dots","quadx_(0)in D.]:}\begin{aligned} & \left(F^{\prime}\left(x_{k}\right)+\Delta_{k}\right) s_{k}=\left(-F\left(x_{k}\right)+\delta_{k}\right)+\dot{r}_{k} \\ & x_{k+1}=x_{k}+s_{k}, \quad k=0,1, \ldots, \quad x_{0} \in D . \end{aligned}(F(xk)+Δk)sk=(F(xk)+δk)+r˙kxk+1=xk+sk,k=0,1,,x0D.
The matrices ( Δ k ) k 0 R n × n Δ k k 0 R n × n (Delta_(k))_(k >= 0)subR^(n xx n)\left(\Delta_{k}\right)_{k \geq 0} \subset \mathbb{R}^{n \times n}(Δk)k0Rn×n represent perturbations to the Jacobians, the vectors ( δ k ) k > 0 R n δ k k > 0 R n (delta_(k))_(k > 0)subR^(n)\left(\delta_{k}\right)_{k>0} \subset \mathbf{R}^{n}(δk)k>0Rn perturbations to the function evaluations for F ( x k ) F x k -F(x_(k))-F\left(\boldsymbol{x}_{k}\right)F(xk), while r ~ k r ~ k tilde(r)_(k)\tilde{r}_{k}r~k are the residuals of the approximate solutions s k s k s_(k)s_{k}sk of the perturbed linear systems ( F ( x k ) + Δ k ) s = F ( x k ) + δ k F x k + Δ k s = F x k + δ k (F^(')(x_(k))+Delta_(k))s=-F(x_(k))+delta_(k)\left(F^{\prime}\left(x_{k}\right)+\Delta_{k}\right) s=-F\left(x_{k}\right)+\delta_{k}(F(xk)+Δk)s=F(xk)+δk. These iterations were called inexact perturbed Newton (IPN) iterations, and in the mentioned paper we have characterized their q q qqq-convergence orders.
The r-convergence orders may also be characterized in the same manner:
Theorem 2 Assume that conditons (C1)-(C4) hold, the matrices Δ k Δ k Delta_(k)\Delta_{k}Δk are chosen such that the perturbed Jacobians F ( x k ) + Δ k F x k + Δ k F^(')(x_(k))+Delta_(k)F^{\prime}\left(x_{k}\right)+\Delta_{k}F(xk)+Δk are invertible for k = 0 , 1 , k = 0 , 1 , k=0,1,dotsk= 0,1, \ldotsk=0,1, and that the JPN iterates converge to x + x + x^(+)x^{+}x+. Then the convergence is with r-onder at least 1 + p 1 + p 1+p1+p1+p if and only if
Δ k ( F ( x k ) + Δ k ) 1 F ( x k ) + ( I Δ k ( F ( x k ) + Δ k ) 1 ) ( δ k + r ˙ k ) 0 Δ k F x k + Δ k 1 F x k + I Δ k F x k + Δ k 1 δ k + r ˙ k 0 ||Delta_(k)(F^(')(x_(k))+Delta_(k))^(-1)F(x_(k))+(I-Delta_(k)(F^(')(x_(k))+Delta_(k))^(-1))(delta_(k)+r^(˙)_(k))||rarr0\left\|\Delta_{k}\left(F^{\prime}\left(x_{k}\right)+\Delta_{k}\right)^{-1} F\left(x_{k}\right)+\left(I-\Delta_{k}\left(F^{\prime}\left(x_{k}\right)+\Delta_{k}\right)^{-1}\right)\left(\delta_{k}+\dot{r}_{k}\right)\right\| \rightarrow 0Δk(F(xk)+Δk)1F(xk)+(IΔk(F(xk)+Δk)1)(δk+r˙k)0
with r r rrr-order 1 + p 1 + p 1+p1+p1+p.
Proof. As for the q q qqq-convergence orders discussed in [4], the IPN can be viewed as an IN method:
s k = ( F ( x k ) + Δ k ) 1 F ( x k ) + ( F ( x k ) + Δ k ) 1 ( δ k + r ^ k ) F ( x k ) s k = Δ k ε k F ( x k ) + δ k + r ˙ k = F ( x k ) + Δ k ( F ( x k ) + Δ k ) 1 F ( x k ) Δ k ( F ( x k ) + Δ k ) 1 ( δ k + r ˙ k ) + δ k + r ˙ k = F ( x k ) + Δ k ( F ( x k ) + Δ k ) 1 F ( x k ) + ( I Δ k ( F ( x k ) + Δ k ) 1 ) ( δ k + r ¯ k ) s k = F x k + Δ k 1 F x k + F x k + Δ k 1 δ k + r ^ k F x k s k = Δ k ε k F x k + δ k + r ˙ k = F x k + Δ k F x k + Δ k 1 F x k Δ k F x k + Δ k 1 δ k + r ˙ k + δ k + r ˙ k = F x k + Δ k F x k + Δ k 1 F x k + I Δ k F x k + Δ k 1 δ k + r ¯ k {:[s_(k)=-(F^(')(x_(k))+Delta_(k))^(-1)F(x_(k))+(F^(')(x_(k))+Delta_(k))^(-1)(delta_(k)+ hat(r)_(k))],[F^(')(x_(k))s_(k)=-Delta_(k)epsi_(k)-F(x_(k))+delta_(k)+r^(˙)_(k)],[=-F(x_(k))+Delta_(k)(F^(')(x_(k))+Delta_(k))^(-1)F(x_(k))],[Delta_(k)(F^(')(x_(k))+Delta_(k))^(-1)(delta_(k)+r^(˙)_(k))+delta_(k)+r^(˙)_(k)],[=F(x_(k))+Delta_(k)(F^(')(x_(k))+Delta_(k))^(-1)F(x_(k))+],[(I-Delta_(k)(F^(')(x_(k))+Delta_(k))^(-1))(delta_(k)+ bar(r)_(k))]:}\begin{aligned} s_{k}= & -\left(F^{\prime}\left(x_{k}\right)+\Delta_{k}\right)^{-1} F\left(x_{k}\right)+\left(F^{\prime}\left(x_{k}\right)+\Delta_{k}\right)^{-1}\left(\delta_{k}+\hat{r}_{k}\right) \\ F^{\prime}\left(x_{k}\right) s_{k}= & -\Delta_{k} \varepsilon_{k}-F\left(x_{k}\right)+\delta_{k}+\dot{r}_{k} \\ = & -F\left(x_{k}\right)+\Delta_{k}\left(F^{\prime}\left(x_{k}\right)+\Delta_{k}\right)^{-1} F\left(x_{k}\right) \\ & \Delta_{k}\left(F^{\prime}\left(x_{k}\right)+\Delta_{k}\right)^{-1}\left(\delta_{k}+\dot{r}_{k}\right)+\delta_{k}+\dot{r}_{k} \\ = & F\left(x_{k}\right)+\Delta_{k}\left(F^{\prime}\left(x_{k}\right)+\Delta_{k}\right)^{-1} F\left(x_{k}\right)+ \\ & \left(I-\Delta_{k}\left(F^{\prime}\left(x_{k}\right)+\Delta_{k}\right)^{-1}\right)\left(\delta_{k}+\bar{r}_{k}\right) \end{aligned}sk=(F(xk)+Δk)1F(xk)+(F(xk)+Δk)1(δk+r^k)F(xk)sk=ΔkεkF(xk)+δk+r˙k=F(xk)+Δk(F(xk)+Δk)1F(xk)Δk(F(xk)+Δk)1(δk+r˙k)+δk+r˙k=F(xk)+Δk(F(xk)+Δk)1F(xk)+(IΔk(F(xk)+Δk)1)(δk+r¯k)
Denoting
r k = Δ k ( F ( x k ) + Δ k ) 1 F ( x k ) + ( I Δ k ( F ( x k ) + Δ k ) 1 ) ( δ k + r ¯ k ) , r k = Δ k F x k + Δ k 1 F x k + I Δ k F x k + Δ k 1 δ k + r ¯ k , r_(k)=Delta_(k)(F^(')(x_(k))+Delta_(k))^(-1)F(x_(k))+(I-Delta_(k)(F^(')(x_(k))+Delta_(k))^(-1))(delta_(k)+ bar(r)_(k)),r_{k}=\Delta_{k}\left(F^{\prime}\left(x_{k}\right)+\Delta_{k}\right)^{-1} F\left(x_{k}\right)+\left(I-\Delta_{k}\left(F^{\prime}\left(x_{k}\right)+\Delta_{k}\right)^{-1}\right)\left(\delta_{k}+\bar{r}_{k}\right),rk=Δk(F(xk)+Δk)1F(xk)+(IΔk(F(xk)+Δk)1)(δk+r¯k),
the assertion follows from Theorem 1.
It is interesting to note that there excist iterations with convergence orders to be analyzed and characterized, as we shall see in a forthcoming paper.

References

[1] P. N. Brown, A Local convergence theory for combined ineract-Newton/finite-difference projection methods, SIAM J. Numer. Anal., 24 (1987), pp. 407-434.
[2] P. N. Brown and Y. Saad, Convergence theory of nonlinear NewtonKrylov abjerithens, SIAM J. Optimization, 4 (1994), pp. 297-330.
33 E. CAtmas, A note on incract secont methods, Rev. Anal. Numér. Théor. Approx., 25 (1996), pp. 33-41.
[4] E. CAtinAs, Inexact perturbed Newton methods, and some applications for a class of Krylov solvers, J. Optim. Theory Appl., to appear.
5. E. CATINAs, On the high convergence orders of the Newton-GMBACK methodes, Rev. Anal. Numér. Théor. Approx., 28 (1999), to appear.
[6] E. CAtenAs, Newton and Newton-Krylov methods for solving nonlinear systems in R n R n R^(n)\mathbb{R}^{n}Rn, Ph.D. thesis, "Babeq-Bolyai" University of Cluj-Napoca, Romania, 1999.
7 R. S. Dembo, S. C. Eisenstat and T. Steiiiaug, Incract Newton methods, SIAM J. Numer. Anal., 19 (1982), pp. 400-408.
[8] J. E. DENNIS, JR. AND J. J. MORE., A characterization of superlinear converyence and its application to quasi-Neuton methods, Math. Comput., 28 (1974), pp. 549-560.
9 J.E. Dennis, Jr. And J.J. Mork, Quasi-Newton methods, motivation and theory, SLAM Review, 19 (1977), pp. 46-89.
[10] J. E. Dennis, Jr. And R. B. Scinabel, Numerical Methods for Unconstruined Optimization and Nonlinear Equations, Prentice-Hall Series in Computational Mathematics, Englewood Cliffs, NJ, 1983.
[11] Şt. MAruster, Numerical Methods for Solving Nonlinear Equations, Ed. Tehnical, Bucharest, 1981 (in Romanian).
[12] J. M. Orthisa and W. O. Rheinboldt, Aterative Solution of Nonlinear Equations in Several Varrables, Academic Press, New York, 1970.
[13] I. Pivaloiu, Introduction to Theory of Approximating the Solutions of Equations, Ed. Dacia, Cluj-Napora, 1976 (in Romanian).
14] F. A. Potra, On Q-order and R-order of convergence, J. Optim. Thoory Appl., 63 (1989), pp. 415-431.
[15] W. C. Rheinboldt, Methods for Solving Systems of Nonlinear Equations, Second ed., SIAM, Philadelphia, 1998.
Received: 1.09.1999

Emil Cátinus"T. Popoviciu" Institute of Numerical AnalysisP.O. Box 68-13400 Cluj-NapocaRomania


  1. 0 0 ^(0){ }^{0}0 AMS Subject Classification. 65H10
1999

Related Posts