Methods of Newton and Newton-Krylov type

Introduction

Notations

Ch. 1 Krylov methods for large linear systems

1.1 Motivation
1.2 Backward error minimization Krylov solvers
1.2.1 The GMRES method
1.2.2 The GMBACK method
1.2.3 The MINPERT method

Ch. 2 Local convergence of the Newton methods

2.1 Historical notes
2.2 The local convergence of the Newton method
2.3 Local convergence of the quasi-Newton methods
2.4 Inexact Newton methods
2.4.1 The local convergence of the IN methods
2.4.2 Choosing the forcing terms in the IN methods
2.4.3 Global convergence of the IN methods
2.5 Inexact(ly solved) Perturbed Newton methods
2.6 Newton-Krylov methods
2.6.1 The Newton-GMRES method
2.6.2 The Newton-GMBACK method
2.6.3 The Newton-MINPERT method
2.6.4 Numerical examples
2.7 Finite difference Newton-Krylov methods
2.8 The high convergence orders of the successive approximations

Ch. 3 Semilocal convergence results

3.1 The eigenpair problem
3.2 The Chebyshev method
3.3 A Chebyshev-type method
3.4 A Newton-type method
3.5 Numerical examples

Ch. 4 The chord method

4.1 Definitions and examples
4.2 The local convergence of the chord method
4.3 The local convergence of the inexact chord method
4.4 The chord method for second degree polynomial operators
4.5 Hybrid methods

Appendix. Rates of convergence

A.1 The quotient convergence order
A.2 The root convergence order
A.3 Relations between Q and R-factors

[1] G. Albeanu, On the Convergence of Quasi-Gauss-Newton Methods for Solving Singular Nonlinear Systems, Int. J. Comput. Math., 66 (1998) no.1-2, pp. 93-99
https://doi.org/10.1080/00207169808804627

[2] G., Alefeld, A. Gienger and F. Potra, Efficient numerical validation of solutions of nonlinear systems, SIAM J. Numer. Anal. 31 (1994), no.1, pp. 252-260, https://doi.org/10.1137/0731013

[3] G.E. Alefeld and F.A. Potra, Some efficient methods for enclosing simple zeros of nonlinear equations, BIT 32 (1992) no. 2, pp. 334-344, https://doi.org/10.1007/bf01994885

[4] G. Alefeld, F.A. Potra and Yixun Shi, On enclosing simple roots of nonlinear equations, Math. Comp., 61 (1993) no.204, pp. 733-744.

[5] E.L. Allgower, K. Böhmer, F.A. Potra and W.C. Rheinboldt, A Mesh-Independence Principle for Operator Equations and their Discretizations, SIAM J. Numer. Anal., 23 (1986) no.1, pp. 160-169.

[6] M.P. Anselone and B.L. Rall, The Solution of Characteristic Value-Vector Problems by Newton Method, Numer. Math. 11 (1968), pp. 38-45.

[7] I. Argyros, On the Convergence of Inexact Newton Like Methods, Publ. Math. Debrecen, 43 (1993) no.1-2, pp. 79-85.

[8] I. Argyros, On the Secant Method and the Pták Error Estimates, Rev. Anal. Numér. Théor. Approx., 24 (1995) no.1-2, pp. 3-14.

[9] I. Argyros, Concerning the Convergence of Inexact Newton Methods, J. Comput. Appl. Math., 79 (1997), no.2, pp. 235-247.

[10] I. Argyros, A Local Convergence Theorem for the Inexact Newton Method at Singular Points, Southwest Journal of Pure and Applied Mathematics, (1997) no.1, pp. 30-36.

[11] I. Argyros, A Mesh-Independence Principle for Operators Equations and the Steffensen Method, Korean J. Comp. Appl. Math., 4 (1997) no.2, pp.263-280.

[12] I. Argyros, Comparing the Radii of Some Balls Appearing in Connection to Three Local Convergence Theorems for Newton’s Method, Southwest Journal of Pure and Applied Mathematics, (1998) 1, pp. 24-28.

[13] I. Argyros, E. Cătinaş and I. Păvăloiu, Improving the Rate of Convergence of Some Newton-Like Methods for the Solution of Nonlinear Equations Containing a Nondifferentiable Term, Rev. Anal. Numér. Théor. Approx., 27 (1998) no.2, pp.191-202.

[14] I. Argyros, E. Cătinaş and I. Păvăloiu, On the Convergence of Steffensen-Aitken-like Methods Using Divided Differences Obtained Recursively, Rev. Anal. Numér. Théor. Approx., 28 (1999) no.2, to appear.

[15] I. Argyros and F. Szidarovsky, The Theory and Applications of Iteration Methods, C.R.C. Press Inc. Boca Raton Florida, USA, 1993.

[16] I. Argyros, E. Cătinaş and I. Păvăloiu, Local and global convergence results for a class of Steffensen-Aitken-type methods, Adv. Nonlinear Var. Inequal., 2 (1999) no. 2, pp. 117-126.

[17] I. Argyros, E. Cătinaş and I. Păvăloiu, Perturbed-Steffensen-Aitken projection methods for solving equations with nondifferentiable operators, Punjab Univ. J. Math. (Lahore), 33 (2000), pp. 105-113.

[18] I. Argyros, E. Cătinaş and I. Păvăloiu, On the convergence of Steffensen-Aitken-like methods using divided differences obtained recursively, Adv. Nonlinear Var. Inequal., 3 (2000) no. 1, pp. 7-13.

[19] I. Argyros, E. Cătinaş and I. Păvăloiu, On the approximate solutions of implicit functions using the Steffensen method, Proyecciones, 19 (2000) no. 3, pp. 291-303, Universidad Catolica del Norte, Antofagasta, Chile.

[20] I. Argyros, E. Cătinaş and I. Păvăloiu, On some general iterative methods for solving nonlinear operator equations containing a nondifferentiable term, Adv. Nonlinear Var. Inequal., 3 (2000) no. 1, pp. 15-21.

[21] I. Argyros, E. Cătinaş and I. Păvăloiu, Conditions for the convergence of perturbed Steffensen methods on a Banach space with a convergence structure, Adv. Nonlinear Var. Inequal., 3 (2000) no. 1, pp. 23-35.

[22] M. Arioli and C. Fassino, Roundoff Error Analysis of Algorithms Based on Krylov Subspace Methods, BIT, 36 (1996) no.2, pp. 189-206.

[23] M. Arioli, I. Duff and D. Ruiz, Stopping Criteria for Iterative Solvers, SIAM J. Matrix Anal. Appl., 13 (1992) no.1, pp. 138-144.

[24] M. Arioli, V. Pták and Z. Strakoš, Krylov Sequences of Maximal Length and Convergence of GMRES, BIT, 38 (1998) no.4, pp. 636-643.

[25] W.E. Arnoldi, The Principle of Minimized Iteration in the Solution of the Matrix Eigenvalue Problem, Quart. Appl. Math., 9 (1951), pp. 17-29.

[26] O. Axelsson, On Global Convergence of Iterative Methods, in Iterative Solution of Nonlinear Systems of Equations, Lect. Notes in Math. (R. Ansorge et. al., eds.), Springer Verlag, 1981, pp. 1-15.

[27] O. Axelsson, Iterative Solution Methods, Cambridge University Press, New York, 1996.

[28] O. Axelsson and A.T. Chronopoulos, On Nonlinear Generalized Conjugate Gradient Methods, Numer. Math., 69 (1994), pp.1-15.

[29] O. Axelsson and W. Layton, A Two-Level Method for the Discretization of Nonlinear Boundary Value Problems, SIAM J. Numer. Anal., 33 (1996) no.6, pp. 2359-2374.

[30] N. Bakhvalov, Méthodes Numériques, Mir, Moscou, 1976.

[31] M. Balázs, On the Approximate Solution of Certain Operatorial Equations, Rev. Anal. Numér. Théor. Approx., 18 (1989) no.2, pp. 105-110.

[32] M. Balázs, A Bilateral Approximating Method for Finding the Real Roots of Real Equations, Rev. Anal. Numér. Théor. Approx., 21 (1992) no.2, pp. 111-117.

[33] M. Balázs and G. Goldner, Remarks Concerning the Divided Differences and Chord Method, Rev. Anal. Numér. Théor. Approx., 3 (1974) no.1, pp. 19-30.

[34] M. Balázs and I. Muntean, A Unification of Newton’s Method for Solving Equations, Mathematica, 21 (44) (1979) no.2, pp. 117-122.

[35] R.E. Bank and D.J. Rose, Global Approximate Newton Methods, Numer. Math., 37 (1981), pp. 279-295.

[36] R. Barret, M. Berry, T.F. Chan, J. Demmel, J. Donato, J. Dongarra, V. Eijkhout, R. Pozo, C. Romine and H.A. van der Vorst, Templates for the Solution of Linear Systems: Building Blocks for Iteration Methods, SIAM, Philadelphia, PA, 1995.

[37] V. Berinde, On Some Exit Criteria for the Newton Method, Novi Sad J. Math., 27 (1997) no.1, pp. 19-26.

[38] V. Berinde, Remarks on the Convergence of the Newton-Raphson Method, Rev. Anal. Numér. Théor. Approx., 24 (1995) no.1-2, pp. 15-21.

[39] P.N. Brown, A Local Convergence Theory for Combined Inexact-Newton/Finite-Difference Projection Methods, SIAM J. Numer. Anal., 24 (1987) no.2, pp. 407-434.

[40] P.N. Brown, A Theoretical Comparison of the Arnoldi and GMRES Algorithms, SIAM J. Sci. Stat. Comput., 12 (1991) no.1, pp. 58-78.

[41] P.N. Brown and A.C. Hindmarsh, Reduced Storage Methods in Stiff ODE Systems, J. Appl. Math. Comput., 31 (1989), pp. 40-91.

[42] P.N. Brown and A.C. Hindmarsh, Matrix-Free Methods for Stiff Systems of ODE’s, SIAM J. Numer. Anal., 23 (1986) no.3, pp.610-638.

[43] P.N. Brown, A.C. Hindmarsh and L.R. Petzold, Using Krylov Methods in the Solution of Large-Scale Differential-Algebraic Systems, SIAM J. Sci. Comput., 15 (1994) no.6, pp.1467-1488.

[44] P.N. Brown, A.C. Hindmarsh and H.F. Walker, Experiments with quasi-Newton Methods in Solving Stiff ODE Systems, SIAM J. Sci. Stat. Comput., 6 (1985) no.2, pp.297-313.

[45] P.N. Brown and Y. Saad, Hybrid Krylov Methods for Nonlinear Systems of Equations, SIAM J. Sci. Stat. Comput., 11 (1990) no.3, pp. 450-481.

[46] P.N. Brown and Y. Saad, Convergence Theory of Nonlinear Newton-Krylov Algorithms, SIAM J. Optimization, 4 (1994) no.2, pp. 297-330.

[47] P.N. Brown and H.F. Walker, GMRES on (Nearly) Singular Systems, SIAM J. Matrix Anal. Appl., 18 (1997) no.1, pp. 37-51.

[48] S.L. Campbell, I.C.F. Ipsen, C.T. Kelley and C.D. Meyer, GMRES and the Minimal Polynomial, BIT, 36 (1996) no.4, pp. 664-675.

[49] E. Cătinaş, On some Iterative Methods for Solving Nonlinear Equations, Rev. Anal. Numér. Théor. Approx., 23 (1994) no. 1, pp. 47-53.

[50] E. Cătinaş, On some Steffensen-Type Iterative Methods for a Class of Nonlinear Equations, Rev. Anal. Numér. Théor. Approx., 24 (1995) no. 1-2, pp. 37-43.

[51] E. Cătinaş, A Note on Inexact Secant Methods, Rev. Anal. Numér. Théor. Approx., 25 (1996) no. 1-2, pp. 33-41.

[52] E. Cătinaş and I. Păvăloiu, On the Chebyshev Method for Approximating the Eigenvalues of Linear Operators, Rev. Anal. Numér. Théor. Approx., 25 (1996) no. 1-2, pp. 43-56.

[53] E. Cătinaş and I. Păvăloiu, On a Chebyshev-Type Method for Approximating the Solutions of Polynomial Operators Equations of Degree 2, Approximation and Optimization. Proceedings of the International Conference on Approximation and ptimization, Romania (ICAOR), Cluj-Napoca, July 29 – August 1, 1996, vol. I, pp. 219-226.

[54] E. Cătinaş and I. Păvăloiu, On Approximating the Eigenvalues and Eigenvectors of Linear Continuous Operators, Rev. Anal. Numér. Théor. Approx., 26 (1997) no. 1-2, pp. 19-27.

[55] E. Cătinaş and I. Păvăloiu, On Some Interpolatory Iterative Methods for the Second Degree Polynomial Operators (I), Rev. Anal. Numér. Théor. Approx., 27 (1998) no.1, pp. 35–45.

[56] E. Cătinaş and I. Păvăloiu, On Some Interpolatory Iterative Methods for the Second Degree Polynomial Operators (II), Rev. Anal. Numér. Théor. Approx., 28 (1999) no. 2, pp. 133-143.

[57] E. Cătinaş and I. Păvăloiu, Some Numerical Aspects in Approximating the Eigenpairs of Matrices by Newton Methods, Acta Technica Napocensis, Series: Applied Mathematics and Mechanics, 42, (1999), pp. 41-44.

[58] E. Cătinaş, On the High Convergence Orders of Newton-GMBACK Methods, Rev. Anal. Numér. Théor. Approx., 28 (1999) no.2, pp. 125-132.

[59] E. Cătinaş, On the r-convergence orders of the inexact perturbed Newton methods, Bul. stiint. Univ. Baia Mare, Ser. B, Mat. Inf., 15 (1999) no. 1-2, pp. 75-78.

[60] E. Cătinaş, A note on the quadratic convergence of the inexact Newton methods, Rev. Anal. Numér. Théor. Approx., 29 (2000) no. 2, pp. 129-134.

[61] E. Cătinaş, On accelerating the convergence of the successive approximations method, Rev. Anal. Numér. Théor. Approx., 30 (2001) no. 1, pp. 3-8.

[62] E. Cătinaş and I. Păvăloiu, On a third order iterative method for solving polynomial operator equations, Rev. Anal. Numér. Théor. Approx., 31 (2002) no. 1, pp. 21-28.

[63] E. Cătinaş and I. Păvăloiu, Solving polynomial operator equations of degree 2 by Steffensen-type iterations with approximate inverses, Proceedings of the International Symposium on Numerical Analysis and Approximation Theory, May 9-11, Cluj-Napoca, 2002, R. Trimbitas (ed.), pp. 101-113.

[64] E. Cătinaş and I. Păvăloiu, On the Chebyshev method with numerical applications to the eigenpair problem, Symbolic and Numeric Algorithms for Scientific Computations (SYNASC03), Proceedings of the 5th International Workshop, Timisoara, Romania, October 1-4, D. Petcu, V. Negru, D. Zaharie and T. Jebelean (eds.), pp. 204-210, 2003.

[65] E. Cătinaş, Affine invariant conditions for the inexact perturbed Newton method, Rev. Anal. Numér. Théor. Approx., 31 (2002) no. 1, pp. 17-20.

[66] E. Cătinaş, Inexact perturbed Newton methods and some applications for a class of Krylov solvers, J. Optim. Theory Appl., 108 (2001) no. 3, pp. 543-570.

[67] E. Cătinaş, On the superlinear convergence of the successive approximations method, J. Optim. Theory Appl., 113 (2002) no. 3, pp. 473-485.

[68] E. Cătinaş, The inexact, inexact perturbed and quasi-Newton methods are equivalent models, Math. Comp., 74 (2005) no. 249, pp. 291-301.

[69] E. Cătinaş, The relationship between the models of perturbed Newton iterations, with applications, PAMM, 5 (2005) no. 1, pp. 785-786.

[70] E. Cătinaş, Sufficient convergence conditions for certain accelerated successive approximations, Trends and Applications in Constructive Approximation (Eds.) M.G. de Bruin, D.H. Mache and J. Szabados, International Series of Numerical Mathematics Vol. 1, 2005.

[71] F. Chatelin, Valeurs propres de matrices, Mason, Paris, 1988.

[72] X. Chen and T. Yamamoto, Convergence Domains of Certain Iterative Methods for Solving Nonlinear Equations, Numer. Funct. Anal. and Optimiz., 10 (1989) no.1-2, pp. 37-48.

[73] P.G. Ciarlet, Introduction à l’Analyse Numérique Matricielle et à l’Optimisation, Masson, Paris, 1990.

[74] L. Collatz, Functionalanalysis und Numerische Mathematik, Springer-Verlag, Berlin, 1964.

[75] Gh. Coman, Some Practical Approximation Methods for Nonlinear Equations, Rev. Anal. Numér. Théor. Approx., 11 (1982) no.1-2, pp.41-48.

[76] Gh. Coman, Some Efficient Methods for Nonlinear Equations, Itinerant Seminar on Functional Equations, Approximation and Convexity, Cluj-Napoca, 1985, pp.53-58.

[77] Gh. Coman, Optimal Algorithms for the Solution of Nonlinear Equation with Regard to the Efficiency, Studia Univ. “Babeş-Bolyai”, Mathematica, XXXII (1987) no.3, pp. 46-52.

[78] Gh. Coman, Numerical Analysis, Ed. Libris, Cluj-Napoca, 1995 (in Romanian).

[79] Gh. Coman, G. Pavel, I. Rus şi I.A. Rus, Introduction to the Theory of Operator Equations, Ed. Dacia, Cluj-Napoca, 1976 (in Romanian).

[80] D. Cores and R.A. Tapia, Perturbation Lemma for the Newton Method with Application to the SQP Newton Method, JOTA, 97 (1998) no.2, pp.271-280.

[81]. Cullum and A. Greenbaum, Relations Between Galerkin and Norm-Minimizing Iterative Methods for Solving Linear Systems, SIAM J. Matrix. Anal. Appl., 17 (1996) no.2, pp. 223-247.

[82] D.W. Decker, H.B. Keller and C.T. Kelley, Convergence Rates for Newton’s Method at Singular Points, SIAM J. Numer. Anal., 20 (1983), pp. 296-314.

[83] D.W. Decker and C.T. Kelley, Newton’s Method at Singular Points I, SIAM J. Numer. Anal., 17 (1980), pp. 66-70.

[84] D.W. Decker and C.T. Kelley, Newton’s Method at Singular Points II, SIAM J. Numer. Anal., 17 (1980), pp. 465-471.

[85] D.W. Decker and C.T. Kelley, Convergence Acceleration for Newton’s Method at Singular Points, SIAM J. Numer. Anal., 19 (1982), pp. 219-229.

[86] D.W. Decker and C.T. Kelley, Expanded Convergence Domains for Newton’s Method at Nearly Singular Roots, SIAM J. Sci. Stat. Comput., 6 (1985), pp. 951-966.

[87] R.S. Dembo, S.C. Eisenstat and T. Steihaug, Inexact Newton Methods, SIAM J. Numer. Anal., 19 (1982) no.2, pp. 400-408.

[88] B. Démidovitch and I. Maron, Éléments de Calcul Numérique, Editions Mir, Moscou, 1987.

[89] J.M. Demmel, Applied Numerical Linear Algebra, SIAM, Philadelphia, 1997.

[90] J.E. Dennis, Jr. and J.J. Moré, A Characterization of Superlinear Convergence and Its Application to Quasi-Newton Methods, Math. Comput., 28 (1974), no.126, pp. 549-560.

[91] J.E. Dennis, Jr. and J.J. Moré, Quasi-Newton Methods, Motivation and Theory, SIAM Review, 19 (1977), no.1, pp. 46-89.

[92] J.E. Dennis, Jr. and R.B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice-Hall Series in Computational Mathematics, Englewood Cliffs, NJ, 1983.

[93] J.E. Dennis, Jr. and H.F. Walker, Inaccuracy in Quasi-Newton Methods: Local Improvement Theorems, Math. Prog. Stud., 22 (1984), pp. 70-85.

[94] G. Deslauriers and S. Dubuc, Le Calcul de la Racine Cubique selon Heron, El. Math., 51 (1996) no.1, pp.28-34.

[95] P. Deuflhard, Global Inexact Newton Methods for Very Large Scale Nonlinear Problems, Impact. Comput. Sci. Engrg., 3 (1991), pp. 366-393.

[96] P. Deuflhard and G. Heindl, Affine Invariant Convergence Theorems for Newton’s Method and Extensions to Related Methods, SIAM J. Numer. Anal., 16 (1979) no.1, pp. 1-10

[97] P. Deuflhard and F.A. Potra, Asymptotic Mesh Independence of Newton-Galerkin Methods via a Refined Mysovskii Theorem, SIAM J. Numer. Anal., 29 (1992) no.5, pp. 1395-1412.

[98] P. Deuflhard and M. Weiser, Local Inexact Newton Multilevel FEM for Nonlinear Elliptic Problems, Konrad-Zuse-Zentrum für Informationstechnik Berlin, Preprint SC 96-29, 1996.

[99] P. Deuflhard and M. Weiser, Global Inexact Newton Multilevel FEM for Nonlinear Elliptic Problems, Konrad-Zuse-Zentrum für Informationstechnik Berlin, Preprint SC 96-32, 1996.

[100] A. Diaconu, Sur quelques Méthods Itératives Combinées, Mathematica, 22 (45) (1989) no. 2, pp. 247-261.

[101] A. Diaconu, Sur la manière d’obtenir et sur la convergence de certaines méthodes itératives, “Babeş-Bolyai” University, Faculty of Mathematics and Physics, Research Seminars, Seminar on Functional Analysis and Numerical Methods, Preprint nr.1 (1987), pp.27-74.

[102] A. Diaconu, On the Convergence of an Iterative Proceeding of Chebyshev Type, Rev. Anal. Numér. Théor. Approx., 24 (1995) no. 1-2, pp. 91-102.

[103] A. Diaconu and I. Păvăloiu, Sur quelque méthodes itératives pour la résolution des équations opérationelles, Rev. Anal. Numér. Théor. Approx., 1 (1972) no. 1, pp. 45-61.

[104] J.J. Dongarra, C.B. Moler and J.H. Wilkinson, Improving the Accuracy of the Computed Eigenvalues and Eigenvectors, SIAM J. Numer. Anal., 20 (1983) no. 1, pp. 23-45.

[105] J. Drkošová, A. Greenbaum, M. Rozložník and Z. Strakoš, Numerical Stability of GMRES, BIT, 35 (1995), pp. 309-330.

[106] S.C. Eisenstat and H.F. Walker, Globally Convergent Inexact Newton Methods, SIAM J. Optimization, 4 (1994) no.2, pp. 393-422.

[107] S.C. Eisenstat and H.F. Walker, Choosing the Forcing Terms in an Inexact Newton Method, SIAM J. Sci. Comput., 17 (1996) no.1, pp. 16-32.

[108] L.V. Foster, Gaussian Elimination with Partial Pivoting Can Fail in Practice, SIAM J. Matrix Anal. Appl., 15 (1994) no. 4, pp. 1354-1362.

[109] R.W. Freund, A Transpose-free Quasi-minimal Residual Algorithm for non-Hermitian Linear Systems, SIAM J. Sci. Comput., 14 (1993), pp. 470-482.

[110] R.W. Freund and N.M. Nachtigal, QMR: a Quasi-Minimal Residual Method for non-Hermitian Linear Systems, Numer. Math., 60 (1991), pp. 315-339.

[111] M. Gasparo and B. Morini, Inexact methods: forcing terms and conditioning, J. Optim. Theory Appl. 107 (2000), 573-589.

[112] C.W. Gear and Y. Saad, Iterative Solution of Linear Equations in ODE Codes, SIAM J. Sci. Stat. Comput., 4 (1983) no.4, pp.583-601.

[113] J. Gerlach, Accelerated Convergence in Newton’s Method, SIAM Review, 36 (1994) no.2, pp. 272-276.

[114] Glazman, I. and Liubitch, Y., Analyse linéaire dans les espaces de dimensions finies, manuel en problèmes, Éditions Mir, Moscou, Russie, 1972.

[115] R. Glowinsky, H.B. Keller and L. Reinhart, Continuation-Conjugate Gradient Methods for the Least-Squares Solution of Nonlinear Boundary Value Problems, SIAM J. Sci. Statist. Comput., 6 (1985), pp. 793-832.

[116] G. Goldner, Divided Differences, Fréchet Derivative, (0)-convexity, Approximation and Optimization. Proceedings of the International Conference on Approximation and Optimization, Romania (ICAOR), Cluj-Napoca, July 29 – August 1, 1996, vol. I, pp. 259-264.

[117]. Goldner, M. Balázs, On the Chord Method and on a Modification of It for Solving Nonlinear Operator Equations, Stud. Cerc. Mat., 20 (1968) 7, pp. 981-990.

[118] G.H. Golub and D.P. O’Leary, Some History of the Conjugate Gradient and Lanczos Algorithms:1948-1976, SIAM Review, 31 (1989) no.1, pp. 50-102.

[119] G.H. Golub and C.F. Van Loan, An Analysis of the Total Least Squares Problem, SIAM J. Numer. Anal., 17 (1980), pp. 883-893.

[120] G.H. Golub and C.F. Van Loan, Matrix Computations, Third Edition, The Johns Hopkins University Press, Baltimore and London, 1996.

[121] A. Greenbaum, Iterative Methods for Solving Linear Systems, SIAM, Philadelphia, 1997.

[122] A. Greenbaum, V. Pták and Z. Strakoš, Any Nonincreasing Convergence Curve is Possible for GMRES, SIAM J. Matrix Anal. Appl., 17 (1996) no.3, pp. 465-469.

[123] A. Greenbaum, M. Rozložník and Z. Strakoš, Numerical Behavior of the Modified Gram-Schmidt GMRES Implementation, BIT 37 (1997) no.3, pp. 706-719.

[124] A. Greenbaum and L.N. Trefethen, GMRES/CR and Arnoldi/Lanczos as Matrix Approximation Problems, SIAM J. Sci. Comput., 15 (1994), pp. 359-368.

[125] A. Griewank, Starlike Domains of Convergence for Newton’s Method at Singularities, Numer. Math., 35 (1980), pp. 95-111.

[126] A. Griewank, On Solving Nonlinear Equations with Simple Singularities or Nearly Singular Solutions, SIAM Review, 27 (1985), pp. 537-572.

[127] A. Griewank and M.R. Osborne, Newton’s Method for Singular Problems When the Dimension of the Null Space is >1, SIAM J. Numer. Anal., 18 (1981), pp. 179-189.

[128] A. Griewank and M.R. Osborne, Analysis of Newton’s Method at Irregular Singular Points, SIAM J. Numer. Anal., 20 (1983), pp. 747-773.

[129] S. Groze şi B. Janko, On the Generalized Divided Differences, Anal. St. Univ. “Al. I. Cuza” Iaşi, Ser. Mathematica, XVIII (1971) no.2, pp. 375-379.

[130] S.M. Grzegórski, On the Scaled Newton Method for the Symmetric Eigenvalue Problem, Computing, 45 (1990), pp. 277-282.

[131] S.M. Grzegórski, Rescaled Multilevel Least-Change Almost Secant Methods, Roczniki Polskiego Towarzystwa Matematycznego, Seria III: Matematyka Stosowana, XXXIII (1991), pp. 37-56.

[132] S.M. Grzegórski, Remarks on the Convergence of Newton-like Methods at Regular Singularities, Annales Universitatis Mariae Curie-Sklodowska, Lublin-Polonia, Sectio A, vol. XLV (1996) no.6, pp. 35-44.

[133] W. Hackbusch, Iterative Solution of Large Sparse Systems of Equations, Springer-Verlag, Berlin, 1994.

[134] M. Hanke, History of Krylov-type methods, Electronic Posting to N.A. Digest, C. Moler ed., v98n35, 1998.

[135] C. den Heijer and W.C. Rheinboldt, On Steplength Algorithms for a Class of Continuation Methods, SIAM J. Numer. Anal., 18 (1981) no.5, pp. 925-948.

[136] M.R. Hestenes and E. Stiefel, Methods of Conjugate Gradients for Solving Linear Systems, J. Res. Nat. Bur. Stand., 49 (1952), pp. 409-436.

[137] N.J. Higham, Accuracy and Stability of Numerical Algorithms, SIAM, Philadelphia, 1996.

[138] A.S. Householder, The Theory of Matrices in Numerical Analysis, Dover, New York, New York, 1974.

[139] I. Ichim and I. Molnar, A Bairstow’s Type Method for Trigonometric Polynomials, Numer. Math., 67 (1994) no.2, pp.251-259.

[140] D.V. Ionescu, Divided Differences , Ed. Academiei RSR, Bucureşti, 1978 (in Romanian).

[141] I.C.F. Ipsen and C.D. Meyer, The Idea Behind Krylov Methods, Amer. Math. Monthly, 105 (1998) no.10, pp. 889-899.

[142] V.I. Istrăţescu, Introdction to the Fixed Points Theory, Ed. Acad. RSR, Bucureşti, 1973 (in Romanian).

[143] B. Jankó, The Solving of Nonlinear Operator Equations in Banach Spaces, Ed. Acad. RSR, Bucureşti, 1969 (in Romanian).

[144] L.V. Kantorovici şi G.P. Akilov, Functional Analysis, Ed. Şt. Encicl., Bucureşti, 1986 (in Romanian).

[145] V.S. Kartîşov and F.L. Iuhno, O nekotorîh Modifikaţiah Metoda Niutona dlea Resenia Nelineinoi Spektralnoi Zadaci, J. Vîcisl. matem. i matem. fiz. 33 (1973) no. 9, pp. 1403-1409 (in Russian).

[146] E.M. Kasenally, GMBACK: A Generalised Minimum Backward Error Algorithm for Nonsymmetric Linear Systems, SIAM J. Sci. Comput., 16 (1995) no.3, pp. 698-719.

[147] E.M. Kasenally and V. Simoncini, Analysis of a Minimum Perturbation Algorithm for Nonsymmetric Linear Systems, SIAM J. Numer. Anal, 34 (1997) no.1, pp. 48-66.

[148] C.T. Kelley, Iterative Methods for Linear and Nonlinear Equations, SIAM, Philadelphia, 1995.

[149] C.T. Kelley and Z.Q Xue, Inexact Newton Methods for Singular Problems, Optim. Maths. Software, 2 (1993), pp. 249-267.

[150] C.T. Kelley and R. Suresh, A New Acceleration Method for Newton’s method at Singular Points, SIAM J. Numer. Anal., 20 (1983), pp. 1001-1009.

[151] M.A. Krasnoselskii, Priblijennae reşenie operatornîh uravnenii, Izdatelstvo Nauka Glavnaia redactia fiziko-matematiceskoi literaturî, Moskva, 1969 (in Russian).

[152] M. Królikowska, Nonlinear Mappings with an Almost Sparse Jacobian Matrix, Annales Univ. Mariae Curie-Skłodowska Lublin-Polonia, vol. XLIX, 11, sectio A, 1995, pp. 145-157.

[153] A.N. Krylov, On the Numerical Solution of Equations which in Technical Questions are Determined by the Frequency of Small Vibrations of Material Systems, Izv. Akad. Nauk. SSSR Otd Mat. Estest., 1 (1931), pp. 491-539.

[154] C. Lanczos, An Iteration Method for the Solution of the Eigenvalue Problem of Linear Differential and Integral Operators, J. Res. Nat. Bur. Stand., 45 (1950), pp. 255-282.

[155] I. Lazăr, On a Newton Type Method, Rev. Anal. Numér. Théor. Approx., 23 (1994) no.2, pp. 167-174.

[156] D. Luca and I. Păvăloiu, On the Heron’s Method for Approximating the Cubic Root of a Real Number, Rev. Anal. Numér. Théor. Approx., 26 (1996) no. 1-2, pp. 103-108.

[157] H.J. Martinez, Z. Parada and R.A. Tapia, On the Characterization of Q-superlinear Convergence of Quasi-Newton Interior-Point Methods for Nonlinear Programming, Bol. Soc. Mat. Mexicana, 1 (1995) no. 3, pp. 137-148.

[158] Şt. Măruşter, Quasi-nonexpansivity and two classical methods for solving nonlinear equations, Proc. Amer. Math. Soc., 62 (1976) no.1, pp. 119-123.

[159] Şt. Măruşter, The solution by iteration of nonlinear equations in Hilbert spaces, Proc. Amer. Math. Soc., 63 (1977) no.1, pp. 69-73.

[160] Şt. Măruşter, Numerical Methods for Solving Nonlinear Equations, Ed. Tehnică, Bucureşti, 1981 (in Romanian).

[161] Şt. Măruşter, On the two-step gradient method for nonlinear equations, Proceedings of the colloquium on approximation and optimization (Cluj-Napoca, 1985), pp. 88–104, Univ. Cluj-Napoca, Cluj-Napoca, 1985.

[162] Şt. Măruşter, On the two-step gradient method for nonlinear equations, An. Univ. Timişoara, Ser. Ştiinţ. Mat. 25 (1987) no.3, pp. 69-84.

[163] Şt. Măruşter and P. Popovici, Generalized gradient’s method, An. Univ. Timişoara, Ser. Ştiinţ. Mat., 21 (1983) no.1-2, pp.85-94.

[164] J.J. Moré, A Collection of Nonlinear Model Problems, in Computational Solutions of Nonlinear Systems of Equations, E.L. Allgower and K. Georg, eds., Lectures in Applied Mathematics, vol. 26, American Mathematical Society, Providence, RI, 1990, pp. 723-762.

[165] B. Morini, Convergence behaviour of inexact Newton methods, Math. Comp. 68 (1999), 1605-1613.

[166] I. Muntean, Functional Analysis, Universitatea “Babeş-Bolyai”, Cluj-Napoca, 1993 (in Romanian).

[167] N.M. Nachtigal, S.C. Reddy and L.N. Trefethen, How Fast Are Nonsymmetric Matrix Iterations?, SIAM J. Matrix Anal. Appl., 13 (1992) no.3, pp. 778-795.

[168] S. Nash, Who was -Raphson? (an answer), Electronic Posting to N.A. Digest, C. Moler ed., v92n23, 1992.

[169] I. Newton, Philosophiae Naturalis Principia Mathematica, Ed. Acad. R.P.R., 1956, (in Romanian; translated and notations by V. Marian, revised by V. Vâlcovici).

[170].M. Ortega and W.C. Rheinboldt, Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, New York, 1970.

[171] A.M. Ostrowski, Solution of Equations and Systems of Equations, Second Edition, Academic Press, New York, 1966.

[172] I. Păvăloiu, Sur la méthode de Steffensen pour la résolution des équations operationnelles nonlinéaires, Revue Roumaine des Mathématiques Pures et Appliquées, 13 (1968) no.1, pp.857-861.

[173] I. Păvăloiu, Interpolation dans des espaces linéaires normés et applications, Mathematica (Cluj) 12 (35) (1970) no. 1, pp. 149-158.

[174] I. Păvăloiu, Sur les procédés itératifs à un ordre élevé de convergence, Mathematica (Cluj) 12 (35) (1970) no. 2, pp. 309-324.

[175] I. Păvăloiu, Contributions to the Study of the Iterative Methods with High Convergence Orders, and Applications to Solving Nonlinear Operators, PhD Thesis, Institute of Numerical Analysis, Cluj-Napoca, 1971 (in Romanian).

[176] I. Păvăloiu, The Error Evaluation in the Numerical Solving of Operator Equations, Stud. Cerc. Mat., 23 (1971) no.9, pp. 1459-1464.

[177] I. Păvăloiu, Introduction to Approximating the Solution of Equations, Ed. Dacia, Cluj-Napoca, 1976 (in Romanian).

[178] I. Păvăloiu, Solving Equations by Interpolation, Ed. Dacia, Cluj-Napoca, 1981 (in Romanian).

[179] I. Păvăloiu, Optimal Problems Concerning Interpolation Methods of Solution of Equations, Publications de L’Institut Mathématique (Nouvelle série) Beograd, 52 (1992) (66), pp.113-126.

[180] I. Păvăloiu, Sur une généralisation de la méthode de Steffensen, Rev. Anal. Numér. Théor. Approx., 21 (1992) no.1, pp. 59-67.

[181] I. Păvăloiu, On the Monotonicity of the Sequences of Approximations Obtained by Steffensen’s Method, Mathematica, 35 (58) (1993) no.1, pp. 71-76.

[182] I. Păvăloiu, Observations concerning some Approximation Methods for the Solutions of Operator Equations, Rev. Anal. Numér. Théor. Approx., 23 (1994) no. 2, pp. 185-196.

[183] I. Păvăloiu, Bilateral Approximations for the Solutions of Scalar Equations, Rev. Anal. Numér. Théor. Approx., 23 (1994) no.1, pp. 95-100.

[184] I. Păvăloiu, On Computational Complexity in Solving Equations by Interpolation Methods, Rev. Anal. Numér. Théor. Approx., 24 (1995) no.1, pp. 201-214.

[185] I. Păvăloiu, On Computational Complexity in Solving Equations by Steffensen Type Methods, Rev. Anal. Numér. Théor. Approx., 24 (1995) no.2, pp. 215-220.

[186] I. Păvăloiu, Approximation of the Roots of Equations by Aitken-Steffensen-type Monotonic Sequences, Calcolo, 32 (1995) no.1-2, pp. 69-82.

[187] I. Păvăloiu, Optimal Efficiency Indexes for Iterative Methods of Interpolatory-type, Computer Science Journal of Moldova, 5 (1997) no. 1(13), pp. 20–43.

[188] I. Păvăloiu and E. Cătinaş, Remarks on Some Newton and Chebyshev-Type Methods for Approximating the Eigenvalues and Eigenvectors of Matrices, Computer Science Journal of Moldova, 7 (1999) no. 1, pp. 3–15.

[189] G. Peters and J.H. Wilkinson, Inverse Iteration, Ill-Conditioned Equations and Newton’s Method, SIAM Review, 21 (1979) no. 3, pp. 339-360.

[190] T. Popoviciu, Sur le reste dans certaines formules linéaires d’approximation de l’analyse, Mathematica, 1 (24) (1959) no.1, pp. 95-142.

[191] T. Popoviciu, Sur la délimitation de l’erreur dans l’approximation des racines d’une équation par interpolation linéaires ou quadratique, Revue Roumaine de Mathematiques Pures et Appliquées, XIII (1968) 1, pp. 75-78.

[192] F. A. Potra, On the numerical stability of a class of iterative processes, Itinerant Seminar of Functional Equations: Approximation and Convexity, Timişoara, Romania, 1983, 51-54.

[193] F. A. Potra, On a class of iterative procedures for solving nonlinear equations in Banach spaces, Computational Mathematics, Banach Center Publications, PWN-Polish Scientific Publishers, Warsaw, Poland, vol. 13, 607-621, 1984.

[194] F.A. Potra, On the A Posteriori Error Estimates for Newton’s Method, Beitraege Numer. Math., 12 (1984), pp. 125-138.

[195] F.A. Potra, On an iterative algorithm of order 1.839… for solving nonlinear operator equations, Numer. Funct. Anal. Optim., 7 (1984/85) no. 1, pp. 75-106.

[196] F.A. Potra, Sharp error bounds for a class of Newton-like methods, Libertas Math. 5 (1985), pp. 71-84.

[197] F.A. Potra, On Q-order and R-order of Convergence, J. Optim. Theory Appl., 63 (1989) no.3, pp. 415-431.

[198] F.A. Potra, Liqun and Defeng, Secant Methods for Semismooth Equations, Numer. Math., 80 (1998) no.2, pp. 305-324.

[199] F.A. Potra and V. Pták, Sharp Error Bounds for Newton’s Process, Numer. Math., 34 (1980), pp. 63-72.

[200] F.A. Potra and V. Pták, Nondiscrete Induction and Iterative Processes, Pitman, London, 1984.

[201] F.A. Potra and W.C. Rheinboldt, On the monotone convergence of Newton’s method, Computing 36 (1986) no. 1-2, pp. 81-90.

[202] D. Petcu, On the Error Estimate of Newton’s Method for a Hölder Continuous F-Derivate, An. Univ. Timişoara Ser. Ştiinţ. Mat., 28 (1990), no.2-3, pp. 149-163.

[203] D. Petcu, On the Hypothesis for the Error Estimate of Newton’s Method, Proceedings of the Fourth Symposium of Mathematics and its Applications (Timişoara, 1991), pp. 130–140, Mirton, Timişoara, 1992.

[204] G.W. Reddien, On Newton’s Method for Singular Problems, SIAM J. Numer. Anal., 15 (1978), pp. 993-986.

[205] G.W. Reddien, Newton’s Method and High Order Singularities, Comput. Math. Appl., 5 (1980), pp. 79-86.

[206] W.C. Rheinboldt, An Adaptive Continuation Process for Solving Systems of Nonlinear Equations, Polish Academy of Science, Banach Ctr. Publ., 3 (1977), pp. 129-142.

[207] W.C. Rheinboldt, Methods for Solving Systems of Nonlinear Equations, 2nd ed., SIAM, Philadelphia, 1998.

[208] J.L. Rigal and J. Gaches, On the Compatibility of a Given Solution with the Data of a Linear System, J. Assoc. Comput. Mach., 14 (1967), pp. 543-548.

[209] I.A. Rus, Principles and Applications of the Fixed Point Theory, Ed. Dacia, Cluj-Napoca, 1979 (in Romanian).

[210] Y. Saad, Krylov Subspace Methods for Solving Large Unsymmetric Linear Systems, Math. Comp., 37 (1981) no.155, pp. 105-126.

[211] Y. Saad and M.H. Schultz, GMRES: A Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems, SIAM J. Sci. Stat. Comput., 7 (1986) no.3, pp. 856-869.

[212] I.Gh. Şabac, P. Cocârlan, O. Stănăşilă şi A. Topală, A Course on Special Topics in Mathematics, E.D.P., Bucureşti, 1983 (in Romanian).

[213] M.C. Santos, A Note on the Newton Iteration for the Algebraic Eigenvalue Problem, SIAM J. Matrix Anal. Appl., 9 (1988) no. 4, pp. 561-569.

[214] A.I. Şchiop, Approximating Methods in the Nonlinear Analysis, Editura Academiei RSR, Bucureşti, 1972 (in Romanian).

[215] A.H. Sherman, On Newton-Iterative Methods for the Solution of Systems of Nonlinear Equations, SIAM J. Numer. Anal., 15 (1978) no.4, pp. 755-771.

[216] D.D. Stancu, Some Considerents Regarding the Multivariate Interpolation, Bul. Univ. “Babeş-Bolyai”, Ser. Şt. Nat., 1 (1957) no.1-2, pp. 43-82 (in Romanian).

[217] D.D. Stancu, Generalization of Some Interpolating Polynomials to the Multivariate Case, Bul. Inst. Polit. Iaşi, tom. III (VII) (1957) fs. 1-2, pp. 31-38 (in Romanian).

[218] D.D. Stancu, On Hermite’s Osculatory Interpolation Formula and on Some Generalizations of It, Mathematica, 8 (31) (1966) no.2, pp. 373-391.

[219] D.D. Stancu, Numerical Analysis, vol. I, Lit. Univ. “Babeş-Bolyai”, Cluj-Napoca, 1977 (in Romanian).

[220] D.D. Stancu, Application of Divided Differences to the Study of Monotonicity of the Derivatives of the Sequences of Bernstein Polynomials, Calcolo, XVI (1979) no. 4, pp. 431-445.

[221] G.W. Stewart and J.-G. Sun, Matrix Perturbation Theory, Academic Press, New York, 1990.

[222] R.A. Tapia and L.D. Whitley, The Projected Newton Method has Order 1+√2 for the Symmetric Eigenvalue Problem, SIAM J. Numer. Anal., 25 (1988) no. 6, pp. 1376-1382.

[223] K.-C. Toh, GMRES vs. Ideal GMRES, SIAM J. Matrix Anal. Appl., 18 (1997) no.1, pp. 30-36.

[224] F. J. Traub, Iterative Methods for the Solution of Equations, Prentice-Hall Inc., Englewood Cliffs, N. J., 1964.

[225] R. Trîmbiţaş, Computer-Aided Analysis and Synthesis of Numerical Methods, Approximation and Optimization. Proceedings of the International Conference on Approximation and Optimization, Romania (ICAOR), Cluj-Napoca, July 29 – August 1, 1996, vol. II, pp. 233-238.

[226] L.N. Trefethen and D. Bau, III, Numerical Linear Algebra, SIAM, Philadelphia, 1997.

[227] V. Trénoguine, Analyse Fonctionelle, Editions Mir, Moscou, 1985.

[228] K. Turner and H.F. Walker, Efficient High Accuracy Solutions with GMRES(m), SIAM J. Sci. Stat. Comput., 13 (1992) no. 3, pp. 815-825.

[229] S. Ul’m, On the Iterative Method with Simultaneous Approximation of the Inverse of the Operator, Izv. Acad. Nauk. Estonskoi S.S.R. (16) 4 (1967), pp. 403-411.

[230] M. Urabe, Error Estimation in Numerical Solution of Equations by Iterations Process, J. Sci. Hiroshima Univ., 19 (1956), pp. 479-489.

[231] R.S. Varga, Matrix Iterative Analysis, Prentice-Hall, New Jersey, 1962.

[232] H.A. van der Vorst, Bi-CGSTAB: A Fast and Smoothly Converging Variant of Bi-CG for the Solution of Nonsymmetric Linear Systems, SIAM J. Sci. Stat. Comput., 13 (1992) no.2, pp. 631-644.

[233] H.F. Walker, Implementation of the GMRES Method using Householder Transformations, SIAM J. Sci. Stat. Comput., 9 (1988) no.1, pp.152-163.

[234] H.F. Walker, Implementations of the GMRES Method, Computer Physics Communications 53, North Holland, Amsterdam, 1989, pp. 311-320.

[235] H.F. Walker, An Approach to Continuation Using Krylov Subspace Methods, Research Report 1/97/89, Dept. of Math., Utah State University, submitted to Computational Science in the 21st Century.

[236] H.F. Walker and L. Zhou, A Simpler GMRES, J. Numer. Lin. Alg. Appl., 1 (1994) no.6, pp. 571-581.

[237] R. Weiss, Error-minimizing Krylov Subspace Methods, SIAM J. Sci. Comput., 15 (1994) no.3, pp. 511-527.

[238] J. H. Wilkinson, The Algebraic Eigenvalue Problem, Clarendon Press, Oxford, 1965.

[239] H. Woźniakowski, Numerical Stability for Solving Nonlinear Equations, Numer. Math., 27 (1977), pp. 373-390.

[240] T. Yamamoto, Error Bounds for Computed Eigenvalues and Eigenvectors, Numer. Math. 34 (1980), pp. 189-199.

[241] T. Yamamoto, A Convergence Theorem for Newton’s Method in Banach Spaces, Japan Journal of Applied Mathematics, 3 (1986) no.1, pp. 37-52.

[242] T. Yamamoto, Error Bounds for Newton-like Method Under Kantorovich Type Assumptions, Japan Journal of Applied Mathematics, 3 (1986) no.2, pp. 295-313.

[243] T. Yamamoto, A Note on a Posteriori Error Bound of Zabrejko and Nguen for Zincenko’s Iteration, Numer. Funct. Anal. and Optimiz., 9 (1987) no.9-10, pp. 987-994.

[244] T. Yamamoto and X. Chen, Ball-Convergence Theorems and Error Estimates for Certain Iterative Methods for Nonlinear Equations, Japan Journal of Applied Mathematics, 7 (1990) no.1, pp. 131-143.

[245] T.J. Ypma, Local Convergence of Inexact Newton Methods, SIAM J. Numer. Anal., 21 (1983) no. 3, pp. 583-590.

[246] T.J. Ypma, The Effect of Rounding Errors on Newton-Like Methods, IMA J. Numer. Anal., 3 (1983), pp. 109-118.

[247] T.J. Ypma, Finding a Multiple Zero by Transformations and Newton-Like Methods, SIAM Review, 25 (1983) no.3, pp.365-378.

[248] T.J. Ypma, Historical Development of the Newton-Raphson Method, SIAM Review, 37 (1995) no.4, pp. 531-551.

[249] E. Zeidler, Nonlinear Functional Analysis and Its Applications, I. Fixed-Point Theorems, Springer-Verlag, New York, 1986.