The inexact perturbed Newton methods recently introduced by us are variant of Newton method, which assume that at each step the linear systems are perturbed, and then they are only approximately solved. The q-convergence orders of the iterates were characterized using the results of Dembo, Eisenstat and Steihaug on inexact Newton methods.
In this note we deduce, in the same manner, the characterization of the r-convergence orders of these iterates.
Authors
Emil Cătinaş
(Tiberiu Popoviciu Institute of Numerical Analysis, Romanian Academy)
Keywords
inexact perturbed Newton methods; nonlinear systems in Rn; convergence orders; r-convergence order; forcing terms.
E. Cătinaş, On the r-convergence orders of the inexact perturbed Newton methods, Bul. ştiinţ. Univ. Baia Mare, Ser. B, Mat. Inf., 15 (1999) no. 1-2, pp. 75-78.
Dedicated to Professor lon PAVĂLOIU on his 60'h anniversary
On the rr-convergence orders of the inexact perturbed Newton methods
Emil. Cátinas
Abstract
A hstract The inexact perturbed Newton methods rocently introduced by us are variant of Newton method, which assume that at exuh steep the linear systems are perturbed, aud then they are only approximntoly solved.
The q-convergence orders of the iterates were characterizod using the results of Dembo, Eisenstat and Streihang on inexact Newton methods.
In this note we deduce, in the same manner, the characterization of the rr-convergence orders of these iterates.
1 Introduction.
Given an open set D subeR^(n)D \subseteq \mathbb{R}^{n} and F:D rarrR^(n)F: D \rightarrow \mathbb{R}^{n}, the Newton method
is a classical way which allows in certain circumstances the approximation of a solution x^(**)x^{*} of the nonlinear system F(x)=0F(x)=0.
Its local convergence is usually studied in the following hypotheacs:
(C1) there exists x^(**)inR^(n)x^{*} \in \mathbb{R}^{n} such that F(x^(**))=0F\left(x^{*}\right)=0;
(C2) the mapping FF is differentiable on a neighborhood of x^(**)x^{*} and F^(**)F^{*} is conttinuous at x^(')x^{\prime};
(C3) the Jacobian F^(')(x^(**))F^{\prime}\left(x^{*}\right) is nonsingular;
(C4) the derivative xi^(v)\xi^{v} is Holder continuous with exponent p in(0,1∣:}p \in\left(0,1 \mid\right. at x^(**)x^{*}, i.c. for an arbitrary fixexd norm ||-||\|-\| on R^(n)\mathbb{R}^{n} there exist L,epsi > 0L, \varepsilon>0 such that
||F^(')(x)-F^(')(x^)|| < ||x-x^(**)||^(p),quad" when "||x-x^(**)|| < epsi.\left\|F^{\prime}(x)-F^{\prime}\left(x^{\wedge}\right)\right\|<\left\|x-x^{*}\right\|^{p}, \quad \text { when }\left\|x-x^{*}\right\|<\varepsilon .
Before enouncing the known convergence results concerning the Newton iterates, we briefly remind the definitions conourning the convergence orders of the sexulences. An arbitrary soxucnoc (y_(k))_(k > 0)subR^('')\left(y_{k}\right)_{k>0} \subset \mathbb{R}^{\prime \prime} is said that converges q^(-)q^{-} superlinearly to its limit bar(y)inR^(n)\bar{y} \in \mathbb{R}^{n} if
lim_(k rarr oo)(||y_(k+1)-( bar(y))||)/(||y_(k)-( bar(y))||)=0,quad" when "y_(k)!= bar(y),k >= k_(0),\lim _{k \rightarrow \infty} \frac{\left\|y_{k+1}-\bar{y}\right\|}{\left\|y_{k}-\bar{y}\right\|}=0, \quad \text { when } y_{k} \neq \bar{y}, k \geq k_{0},
76
and with qq-convergence order alpha > 1\alpha>1 if
l i m s u p_(k rarr oo)(||y_(k+1)-( bar(y))||)/(||y_(k)-( bar(y))||^(omega)) < +oo,quad" when "y_(k)!= bar(y)_(1)k >= k_(0)\limsup _{k \rightarrow \infty} \frac{\left\|y_{k+1}-\bar{y}\right\|}{\left\|y_{k}-\bar{y}\right\|^{\omega}}<+\infty, \quad \text { when } y_{k} \neq \bar{y}_{1} k \geq k_{0}
The sexuence (y_(k))_(k >= 0)subR^(n)\left(y_{k}\right)_{k \geq 0} \subset \mathbb{R}^{n} is said that converges rr-supcrlincarly to its limit bar(y)inR^(n)\bar{y} \in \mathbf{R}^{n} if
and with rr-convergence order alpha > 1\alpha>1 if
l i m s u p_(k rarr oo)||y_(k)-( bar(y))||^(1//alpha^(2)) < 1,quad" when "y_(k)!= bar(y_(1))k >= k_(0).\limsup _{k \rightarrow \infty}\left\|y_{k}-\bar{y}\right\|^{1 / \alpha^{2}}<1, \quad \text { when } y_{k} \neq \overline{y_{1}} k \geq k_{0} .
These definitions are more rigorously treated in the classical book of Ortega and Rheinboldt [12, ch.9] (see also [15], [14]).
Conditions (C1)-(C3) assure an attraction theorem for the Newton iterates: there exists epsi > 0\varepsilon>0 such that for any initial approximation x_(0)in Dx_{0} \in D with ||x_(0)-x^|| < epsi\left\|x_{0}-x^{\wedge}\right\|<\varepsilon, the Newton iterates (x_(k))_(k >= 0)\left(x_{k}\right)_{k \geq 0} are well delined, remain in the ball of center x^(**)x^{*} and radius cc, and converge yy-superlincarly to x^(**)x^{*} (sce ∣12\mid 12, Th. 10.2 .2 ]). Under the additional condition (C4), the convergence is with qq-order 1+p1+p.
The yy-convergence orders are more general than the qq-convergence orders, in the sense that a sequence converging qq-superlinearly converges also rr-superlincarly, and when it converges with qq-order alpha > 1\alpha>1, it also converges with rr-order alpha\alpha (the converse being not true as a general affirmation). Therefore, in the conditions of the above attraction theorem, we automatically obtain rr-superlinear convergence, resp, convergence with rr-orders 1+p1+p of the Newton iterates.
2 The rr-convergence orders of the inexact perturbed Newton methods.
When used in practice, the Newton method results in the following algorithm:
Choose an initial approximation x_(0)in Dx_{0} \in D
For k=0,1,dotsk=0,1, \ldots until "convergence" do
Solve F^(')(x_(k))s_(k)=-F(x_(k))F^{\prime}\left(x_{k}\right) s_{k}=-F\left(x_{k}\right)
Set x_(k-1)=x_(k)+s_(k)x_{k-1}=x_{k}+s_{k}.
Each step requires the solving of a linear system, which is usually a difficult task. There exist two main approaches in order to overcome this difficulty. The first one considers some linear systems easier to solve, by perturbing the matrices; the resulting itcrations are called quasi-Newton methods. The second approach considers that the linear systems are not solved exactly;
Choose an initial approximation x_(0)sub Dx_{0} \subset D
For k=0,1,dotsk=0,1, \ldots until "convergence" do
Find s_(k)s_{k} such that F^(')(x_(k))s_(k)=-F(x_(k))+r_(k)F^{\prime}\left(x_{k}\right) s_{k}=-F\left(x_{k}\right)+r_{k}
Set x_(k+1)=x_(k)+y_(k)x_{k+1}=x_{k}+y_{k}.
The error terms (the residuals) r_(k)r_{k} represent the amounts by which the solutions s_(8)s_{8} fail to satisfy the exact linear systems. Dembo, Eisenstat and Stcibaug characterized the convergence orders of the incxact Newton (IN) method above. We remind the result dealing with the rr-convergence orders.
Theorem 1 [7] Assume that conditions (C1)-(C4) hold and the IN ilerales (x_(k))_(k >= 0)\left(x_{k}\right)_{k \geq 0} consuerge to x^x^{\wedge}. Then the comoungence is wath r-oruler at least 1+p1+p if and ondy if r_(k)rarr0\mathrm{r}_{k} \rightarrow 0 with rr-comoveryence order 1+p1+p as k rarr ook \rightarrow \infty;
We have recently proposed in [4] a new model for the Newton methods, which reflexts the different situations that usually arise in praction:
The matrices (Delta_(k))_(k >= 0)subR^(n xx n)\left(\Delta_{k}\right)_{k \geq 0} \subset \mathbb{R}^{n \times n} represent perturbations to the Jacobians, the vectors (delta_(k))_(k > 0)subR^(n)\left(\delta_{k}\right)_{k>0} \subset \mathbf{R}^{n} perturbations to the function evaluations for -F(x_(k))-F\left(\boldsymbol{x}_{k}\right), while tilde(r)_(k)\tilde{r}_{k} are the residuals of the approximate solutions s_(k)s_{k} of the perturbed linear systems (F^(')(x_(k))+Delta_(k))s=-F(x_(k))+delta_(k)\left(F^{\prime}\left(x_{k}\right)+\Delta_{k}\right) s=-F\left(x_{k}\right)+\delta_{k}. These iterations were called inexact perturbed Newton (IPN) iterations, and in the mentioned paper we have characterized their qq-convergence orders.
The r-convergence orders may also be characterized in the same manner:
Theorem 2 Assume that conditons (C1)-(C4) hold, the matrices Delta_(k)\Delta_{k} are chosen such that the perturbed Jacobians F^(')(x_(k))+Delta_(k)F^{\prime}\left(x_{k}\right)+\Delta_{k} are invertible for k=0,1,dotsk= 0,1, \ldots and that the JPN iterates converge to x^(+)x^{+}. Then the convergence is with r-onder at least 1+p1+p if and only if
the assertion follows from Theorem 1.
It is interesting to note that there excist iterations with convergence orders to be analyzed and characterized, as we shall see in a forthcoming paper.
References
[1] P. N. Brown, A Local convergence theory for combined ineract-Newton/finite-difference projection methods, SIAM J. Numer. Anal., 24 (1987), pp. 407-434.
[2] P. N. Brown and Y. Saad, Convergence theory of nonlinear NewtonKrylov abjerithens, SIAM J. Optimization, 4 (1994), pp. 297-330.
33 E. CAtmas, A note on incract secont methods, Rev. Anal. Numér. Théor. Approx., 25 (1996), pp. 33-41.
[4] E. CAtinAs, Inexact perturbed Newton methods, and some applications for a class of Krylov solvers, J. Optim. Theory Appl., to appear.
5. E. CATINAs, On the high convergence orders of the Newton-GMBACK methodes, Rev. Anal. Numér. Théor. Approx., 28 (1999), to appear.
[6] E. CAtenAs, Newton and Newton-Krylov methods for solving nonlinear systems in R^(n)\mathbb{R}^{n}, Ph.D. thesis, "Babeq-Bolyai" University of Cluj-Napoca, Romania, 1999.
7 R. S. Dembo, S. C. Eisenstat and T. Steiiiaug, Incract Newton methods, SIAM J. Numer. Anal., 19 (1982), pp. 400-408.
[8] J. E. DENNIS, JR. AND J. J. MORE., A characterization of superlinear converyence and its application to quasi-Neuton methods, Math. Comput., 28 (1974), pp. 549-560.
9 J.E. Dennis, Jr. And J.J. Mork, Quasi-Newton methods, motivation and theory, SLAM Review, 19 (1977), pp. 46-89.
[10] J. E. Dennis, Jr. And R. B. Scinabel, Numerical Methods for Unconstruined Optimization and Nonlinear Equations, Prentice-Hall Series in Computational Mathematics, Englewood Cliffs, NJ, 1983.
[11] Şt. MAruster, Numerical Methods for Solving Nonlinear Equations, Ed. Tehnical, Bucharest, 1981 (in Romanian).
[12] J. M. Orthisa and W. O. Rheinboldt, Aterative Solution of Nonlinear Equations in Several Varrables, Academic Press, New York, 1970.
[13] I. Pivaloiu, Introduction to Theory of Approximating the Solutions of Equations, Ed. Dacia, Cluj-Napora, 1976 (in Romanian).
14] F. A. Potra, On Q-order and R-order of convergence, J. Optim. Thoory Appl., 63 (1989), pp. 415-431.
[15] W. C. Rheinboldt, Methods for Solving Systems of Nonlinear Equations, Second ed., SIAM, Philadelphia, 1998.
Received: 1.09.1999
Emil Cátinus"T. Popoviciu" Institute of Numerical AnalysisP.O. Box 68-13400 Cluj-NapocaRomania