Return to Article Details General multivariate arctangent function activated neural network approximations

General multivariate arctangent function activated neural network approximations

George A. Anastassiou $^{*}$

received: March 22, 2022; accepted: May 28, 2022; published online: August 25, 2022.

Here we expose multivariate quantitative approximations of Banach space valued continuous multivariate functions on a box or $R^{N},$ $N \in N$ , by the multivariate normalized, quasi-interpolation, Kantorovich type and quadrature type neural network operators. We treat also the case of approximation by iterated operators of the last four types. These approximations are derived by establishing multidimensional Jackson type inequalities involving the multivariate modulus of continuity of the engaged function or its high order Fréchet derivatives. Our multivariate operators are defined by using a multidimensional density function induced by the arctangent function. The approximations are pointwise and uniform. The related feed-forward neural network is with one hidden layer.

MSC. 41A17, 41A25, 41A30, 41A36.

Keywords. arctangent function, multivariate neural network approximation, quasi-interpolation operator, Kantorovich type operator, quadrature type operator, multivariate modulus of continuity, abstract approximation, iterated approximation.

$^{*}$ Department of Mathematical Sciences, University of Memphis, Memphis, TN 38152, U.S.A., e-mail: ganastss@memphis.edu.

1 Introduction

The author in [ 2 ] and [ 3 ] , see chapters 2–5, was the first to establish neural network approximations to continuous functions with rates by very specifically defined neural network operators of Cardaliagnet-Euvrard and “Squashing” types, by employing the modulus of continuity of the engaged function or its high order derivative, and producing very tight Jackson type inequalities. He treats there both the univariate and multivariate cases. The defining these operators “bell-shaped” and “squashing” functions are assumed to be of compact support. Also in [ 3 ] he gives the $N$ th order asymptotic expansion for the error of weak approximation of these two operators to a special natural class of smooth functions, see chapters 4–5 there.

For this article the author is motivated by the article [ 13 ] of Z. Chen and F. Cao, also by [ 4 ] , [ 5 ] , [ 6 ] , [ 7 ] , [ 8 ] , [ 9 ] , [ 10 ] , [ 11 ] , [ 14 ] , [ 15 ] .

The author here performs multivariate arctangent function based neural network approximations to continuous functions over boxes or over the whole $R^{N}$ , $N \in N$ . Also he does iterated approximation. All convergences here are with rates expressed via the multivariate modulus of continuity of the involved function or its high order Fréchet derivative and given by very tight multidimensional Jackson type inequalities.

The author here comes up with the “right” precisely defined multivariate normalized, quasi-interpolation neural network operators related to boxes or $R^{N}$ , as well as Kantorovich type and quadrature type related operators on $R^{N}$ . Our boxes are not necessarily symmetric to the origin. In preparation to prove our results we establish important properties of the basic multivariate density function induced by arctangent function and defining our operators.

Feed-forward neural networks (FNNs) with one hidden layer, the only type of networks we deal with in this chapter, are mathematically expressed as

N_{n} (x) = \sum_{j = 0}^{n} c_{j} σ (⟨ a_{j} \cdot x ⟩ + b_{j}), \   \   \ x \in R^{s}, \   \ s \in N,

where for $0 \leq j \leq n$ , $b_{j} \in R$ are the thresholds, $a_{j} \in R^{s}$ are the connection weights, $c_{j} \in R$ are the coefficients, $⟨ a_{j} \cdot x ⟩$ is the inner product of $a_{j}$ and $x$ , and $σ$ is the activation function of the network. In many fundamental network models, the activation function is the arctangent function. About neural networks read [ 16 ] , [ 17 ] , [ 18 ] .

2 Auxiliary Notions

We consider the

\arctan x = \int_{0}^{x} \frac{d z}{1 + z^{2}}, \ x \in R .

We will be using

h (x) := \frac{2}{π} \arctan (\frac{π}{2} x) = \frac{2}{π} \int_{0}^{\frac{π x}{2}} \frac{d z}{1 + z^{2}}, \ x \in R,

which is a sigmoid type function and it is strictly increasing. We have that

h (0) = 0, \ h (- x) = - h (x), \ h (+ \infty) = 1, \ h (- \infty) = - 1,

and

h^{'} (x) = \frac{4}{4 + π^{2} x^{2}} > 0, \  all x \in R .

We consider the activation function

ψ (x) := \frac{1}{4} (h (x + 1) - h (x - 1)), \ x \in R,

and we notice that

ψ (- x) = ψ (x),

it is an even function.

Since $x + 1 > x - 1$ , then $h (x + 1) > h (x - 1)$ , and $ψ (x) > 0$ , all $x \in R$ .

We see that

ψ (0) = \frac{1}{π} \arctan \frac{π}{2} ≅ 0.319 .

Let $x > 0$ , we have that

ψ^{'} (x) = \frac{1}{4} (h^{'} (x + 1) - h^{'} (x - 1)) = \frac{- 4 π^{2} x}{(4 + π^{2} {(x + 1)}^{2}) (4 + π^{2} {(x - 1)}^{2})} < 0.

That is

ψ^{'} (x) < 0, for x > 0.

That is $ψ$ is strictly decreasing on $[0, \infty)$ and clearly is strictly increasing on $(- \infty, 0]$ , and $ψ^{'} (0) = 0.$

Observe that

\begin{array}{l} lim_{x \to + \infty} ψ (x) = \frac{1}{4} (h (+ \infty) - h (+ \infty)) = 0, \\ and \\ lim_{x \to - \infty} ψ (x) = \frac{1}{4} (h (- \infty) - h (- \infty)) = 0. \end{array}

That is the $x$ -axis is the horizontal asymptote on $ψ$ .

All in all, $ψ$ is a bell symmetric function with maximum $ψ (0) ≅ 0.319 .$

We need

Theorem 1 [ 11 , p. 286 ]

We have that

\sum_{i = - \infty}^{\infty} ψ (x - i) = 1, \ \forall x \in R .

Theorem 2 [ 11 , p. 287 ]

It holds

\int_{- \infty}^{\infty} ψ (x) d x = 1.

So that $ψ (x)$ is a density function on $R .$

We mention

Theorem 3 [ 11 , p. 288 ]

Let $0 < α < 1$ , and $n \in N$ with $n^{1 - α} > 2$ . It holds

\sum_{\begin{matrix} k = - \infty \\ | n x - k | \geq n^{1 - α} \end{matrix}}^{\infty} ψ (n x - k) < \frac{2}{π^{2} (n^{1 - α} - 2)} .

Denote by $⌊ \cdot ⌋$ the integral part of the number and by $⌈ \cdot ⌉$ the ceiling of the number.

We need

Theorem 4 [ 11 , p. 289 ]

Let $x \in [a, b] \subset R$ and $n \in N$ so that $⌈ n a ⌉ \leq ⌊ n b ⌋$ . It holds

\frac{1}{\sum_{k = ⌈ n a ⌉}^{⌊ n b ⌋} ψ (n x - k)} < \frac{1}{ψ (1)} ≅ 4.9737, \ \forall x \in [a, b] .

Note 1 [ 11 , pp. 290–291 ]

i) We have that

lim_{n \to \infty} \sum_{k = ⌈ n a ⌉}^{⌊ n b ⌋} ψ (n x - k) \neq 1,

for at least some $x \in [a, b] .$

ii) For large enough $n \in N$ we always obtain $⌈ n a ⌉ \leq ⌊ n b ⌋$ . Also $a \leq \frac{k}{n} \leq b$ , iff $⌈ n a ⌉ \leq k \leq ⌊ n b ⌋$ .

In general, by theorem 1, it holds

\sum_{k = ⌈ n a ⌉}^{⌊ n b ⌋} ψ (n x - k) \leq 1.

We introduce

Z (x_{1}, . . ., x_{N}) := Z (x) := \prod_{i = 1}^{N} ψ (x_{i}), \   \ x = (x_{1}, . . ., x_{N}) \in R^{N}, N \in N .

It has the properties:

(i) $Z (x) > 0$ , $\forall$ $x \in R^{N},$

(ii)

\sum_{k = - \infty}^{\infty} Z (x - k) := \sum_{k_{1} = - \infty}^{\infty} \sum_{k_{2} = - \infty}^{\infty} . . . \sum_{k_{N} = - \infty}^{\infty} Z (x_{1} - k_{1}, . . ., x_{N} - k_{N}) = 1,

where $k := (k_{1}, . . ., k_{n}) \in Z^{N}$ , $\forall$ $x \in R^{N},$

hence

(iii)

\sum_{k = - \infty}^{\infty} Z (n x - k) = 1,

$\forall$ $x \in R^{N};$ $n \in N$ ,

and

(iv)

\int_{R^{N}} Z (x) d x = 1,

that is $Z$ is a multivariate density function.

Here denote ${‖ x ‖}_{\infty} := max {| x_{1} |, . . ., | x_{N} |}$ , $x \in R^{N}$ , also set $\infty := (\infty, . . ., \infty)$ , $- \infty := (- \infty, . . ., - \infty)$ upon the multivariate context, and

\begin{matrix} ⌈ n a ⌉ := (⌈ n a_{1} ⌉, . . ., ⌈ n a_{N} ⌉), \\ ⌊ n b ⌋ := (⌊ n b_{1} ⌋, . . ., ⌊ n b_{N} ⌋), \end{matrix}

where $a := (a_{1}, . . ., a_{N})$ , $b := (b_{1}, . . ., b_{N}) .$

We obviously see that

\begin{aligned} \sum_{k = ⌈ n a ⌉}^{⌊ n b ⌋} Z (n x - k) & = \sum_{k = ⌈ n a ⌉}^{⌊ n b ⌋} (\prod_{i = 1}^{N} ψ (n x_{i} - k_{i})) \\ = \sum_{k_{1} = ⌈ n a_{1} ⌉}^{⌊ n b_{1} ⌋} . . . \sum_{k_{N} = ⌈ n a_{N} ⌉}^{⌊ n b_{N} ⌋} (\prod_{i = 1}^{N} ψ (n x_{i} - k_{i})) \\ = \prod_{i = 1}^{N} (\sum_{k_{i} = ⌈ n a_{i} ⌉}^{⌊ n b_{i} ⌋} ψ (n x_{i} - k_{i})) . \end{aligned}

For $0 < β < 1$ and $n \in N$ , a fixed $x \in R^{N}$ , we have that

\sum_{k = ⌈ n a ⌉}^{⌊ n b ⌋} ψ (n x - k) = \sum_{\begin{matrix} k = ⌈ n a ⌉ \\ {‖ \frac{k}{n} - x ‖}_{\infty} \leq \frac{1}{n^{β}} \end{matrix}}^{⌊ n b ⌋} ψ (n x - k) + \sum_{\begin{matrix} k = ⌈ n a ⌉ \\ {‖ \frac{k}{n} - x ‖}_{\infty} > \frac{1}{n^{β}} \end{matrix}}^{⌊ n b ⌋} ψ (n x - k) .

In the last two sums the counting is over disjoint vector sets of $k$ ’s, because the condition ${‖ \frac{k}{n} - x ‖}_{\infty} > \frac{1}{n^{β}}$ implies that there exists at least one $| \frac{k_{r}}{n} - x_{r} | > \frac{1}{n^{β}}$ , where $r \in {1, . . ., N} .$

(v) As in [ 10 , pp. 379–380 ] , we derive that

\sum_{\begin{matrix} k = ⌈ n a ⌉ \\ {‖ \frac{k}{n} - x ‖}_{\infty} > \frac{1}{n^{β}} \end{matrix}}^{⌊ n b ⌋} Z (n x - k) \overset{(???)}{<} \frac{2}{π^{2} (n^{1 - β} - 2)}, \ 0 < β < 1,

with $n \in N : n^{1 - β} > 2$ , $x \in \prod_{i = 1}^{N} [a_{i}, b_{i}] .$

(vi) By theorem 4 we get that

0 < \frac{1}{\sum_{k = ⌈ n a ⌉}^{⌊ n b ⌋} Z (n x - k)} < \frac{1}{{(ψ (1))}^{N}} ≅ {(4.9737)}^{N},

$\forall$ $x \in (\prod_{i = 1}^{N} [a_{i}, b_{i}])$ , $n \in N$ .

It is also clear that

(vii)

\sum_{\begin{matrix} k = - \infty \\ ‖ \frac{k}{n} - x ‖_{\infty} > \frac{1}{n^{β}} \end{matrix}}^{\infty} Z (n x - k) < \frac{2}{π^{2} (n^{1 - β} - 2)},

$0 < β < 1$ , $n \in N : n^{1 - β} > 2$ , $x \in R^{N} .$

Furthermore it holds

lim_{n \to \infty} \sum_{k = ⌈ n a ⌉}^{⌊ n b ⌋} Z (n x - k) \neq 1,

for at least some $x \in (\prod_{i = 1}^{N} [a_{i}, b_{i}]) .$

Here $(X, {‖ \cdot ‖}_{γ})$ is a Banach space.

Let $f \in C (\prod_{i = 1}^{N} [a_{i}, b_{i}], X),$ $x = (x_{1}, . . ., x_{N}) \in \prod_{i = 1}^{N} [a_{i}, b_{i}],$ $n \in N$ such that $⌈ n a_{i} ⌉ \leq ⌊ n b_{i} ⌋$ , $i = 1, . . ., N .$

We introduce and define the following multivariate linear normalized neural network operator ( $x := (x_{1}, . . ., x_{N}) \in (\prod_{i = 1}^{N} [a_{i}, b_{i}])$ ):

\begin{aligned} A_{n} (f, x_{1}, . . ., x_{N}) := \\ := A_{n} (f, x) \\ := \frac{\sum_{k = ⌈ n a ⌉}^{⌊ n b ⌋} f (\frac{k}{n}) Z (n x - k)}{\sum_{k = ⌈ n a ⌉}^{⌊ n b ⌋} Z (n x - k)} \\ = \frac{\sum_{k_{1} = ⌈ n a_{1} ⌉}^{⌊ n b_{1} ⌋} \sum_{k_{2} = ⌈ n a_{2} ⌉}^{⌊ n b_{2} ⌋} . . . \sum_{k_{N} = ⌈ n a_{N} ⌉}^{⌊ n b_{N} ⌋} f (\frac{k_{1}}{n}, . . ., \frac{k_{N}}{n}) (\prod_{i = 1}^{N} ψ (n x_{i} - k_{i}))}{\prod_{i = 1}^{N} (\sum_{k_{i} = ⌈ n a_{i} ⌉}^{⌊ n b_{i} ⌋} ψ (n x_{i} - k_{i}))} . \end{aligned}

For large enough $n \in N$ we always obtain $⌈ n a_{i} ⌉ \leq ⌊ n b_{i} ⌋$ , $i = 1, . . ., N$ . Also $a_{i} \leq \frac{k_{i}}{n} \leq b_{i}$ , iff $⌈ n a_{i} ⌉ \leq k_{i} \leq ⌊ n b_{i} ⌋$ , $i = 1, . . ., N$ .

When $g \in C (\prod_{i = 1}^{N} [a_{i}, b_{i}])$ we define the companion operator

{\tilde{A}}_{n} (g, x) := \frac{\sum_{k = ⌈ n a ⌉}^{⌊ n b ⌋} g (\frac{k}{n}) Z (n x - k)}{\sum_{k = ⌈ n a ⌉}^{⌊ n b ⌋} Z (n x - k)} .

Clearly ${\tilde{A}}_{n}$ is a positive linear operator. We have that

{\tilde{A}}_{n} (1, x) = 1, \ \forall x \in (\prod_{i = 1}^{N} [a_{i}, b_{i}]) .

Notice that $A_{n} (f) \in C (\prod_{i = 1}^{N} [a_{i}, b_{i}], X)$ and ${\tilde{A}}_{n} (g) \in C (\prod_{i = 1}^{N} [a_{i}, b_{i}]) .$

Furthermore it holds

{‖ A_{n} (f, x) ‖}_{γ} \leq \frac{\sum_{k = ⌈ n a ⌉}^{⌊ n b ⌋} {‖ f (\frac{k}{n}) ‖}_{γ} Z (n x - k)}{\sum_{k = ⌈ n a ⌉}^{⌊ n b ⌋} Z (n x - k)} = {\tilde{A}}_{n} ({‖ f ‖}_{γ}, x),

$\forall$ $x \in \prod_{i = 1}^{N} [a_{i}, b_{i}] .$

Clearly ${‖ f ‖}_{γ} \in C (\prod_{i = 1}^{N} [a_{i}, b_{i}]) .$

So, we have that

{‖ A_{n} (f, x) ‖}_{γ} \leq {\tilde{A}}_{n} ({‖ f ‖}_{γ}, x),

$\forall$ $x \in \prod_{i = 1}^{N} [a_{i}, b_{i}]$ , $\forall$ $n \in N$ , $\forall$ $f \in C (\prod_{i = 1}^{N} [a_{i}, b_{i}], X) .$

Let $c \in X$ and $g \in C (\prod_{i = 1}^{N} [a_{i}, b_{i}])$ , then $c g \in C (\prod_{i = 1}^{N} [a_{i}, b_{i}], X) .$

Furthermore it holds

A_{n} (c g, x) = c {\tilde{A}}_{n} (g, x), \ \forall x \in \prod_{i = 1}^{N} [a_{i}, b_{i}] .

Since ${\tilde{A}}_{n} (1) = 1$ , we get that

A_{n} (c) = c, \ \forall c \in X .

We call ${\tilde{A}}_{n}$ the companion operator of $A_{n}$ .

For convinience we call

\begin{aligned} A_{n}^{*} (f, x) := \sum_{k = ⌈ n a ⌉}^{⌊ n b ⌋} f (\frac{k}{n}) Z (n x - k) = \\ = \sum_{k_{1} = ⌈ n a_{1} ⌉}^{⌊ n b_{1} ⌋} \sum_{k_{2} = ⌈ n a_{2} ⌉}^{⌊ n b_{2} ⌋} . . . \sum_{k_{N} = ⌈ n a_{N} ⌉}^{⌊ n b_{N} ⌋} f (\frac{k_{1}}{n}, . . ., \frac{k_{N}}{n}) (\prod_{i = 1}^{N} ψ (n x_{i} - k_{i})), \end{aligned}

$\forall$ $x \in (\prod_{i = 1}^{N} [a_{i}, b_{i}]) .$

That is

A_{n} (f, x) := \frac{A_{n}^{*} (f, x)}{\sum_{k = ⌈ n a ⌉}^{⌊ n b ⌋} Z (n x - k)},

$\forall$ $x \in (\prod_{i = 1}^{N} [a_{i}, b_{i}])$ , $n \in N .$

Hence

A_{n} (f, x) - f (x) = \frac{A_{n}^{*} (f, x) - f (x) (\sum_{k = ⌈ n a ⌉}^{⌊ n b ⌋} Z (n x - k))}{\sum_{k = ⌈ n a ⌉}^{⌊ n b ⌋} Z (n x - k)} .

Consequently we derive

{‖ A_{n} (f, x) - f (x) ‖}_{γ} \overset{(???)}{\leq} {(4.9737)}^{N} {‖ A_{n}^{*} (f, x) - f (x) \sum_{k = ⌈ n a ⌉}^{⌊ n b ⌋} Z (n x - k) ‖}_{γ},

$\forall$ $x \in (\prod_{i = 1}^{N} [a_{i}, b_{i}]) .$

We will estimate the right hand side of (36).

For the last and others we need

Definition 1 [ 11 , p. 274 ]

Let $M$ be a convex and compact subset of $(R^{N}, {‖ \cdot ‖}_{p})$ , $p \in [1, \infty]$ , and $(X, {‖ \cdot ‖}_{γ})$ be a Banach space. Let $f \in C (M, X) .$ We define the first modulus of continuity of $f$ as

ω_{1} (f, δ) := sup_{\begin{matrix} x, y \in M \\ {‖ x - y ‖}_{p} \leq δ \end{matrix}} {‖ f (x) - f (y) ‖}_{γ}, \ 0 < δ \leq d i a m (M) .

If $δ > d i a m (M)$ , then

ω_{1} (f, δ) = ω_{1} (f, d i a m (M)) .

Notice $ω_{1} (f, δ)$ is increasing in $δ > 0$ . For $f \in C_{B} (M, X)$ (continuous and bounded functions) $ω_{1} (f, δ)$ is defined similarly.

Lemma 1 [ 11 , p. 274 ]

We have $ω_{1} (f, δ) \to 0$ as $δ ↓ 0$ , iff $f \in C (M, X)$ , where $M$ is a convex compact subset of $(R^{N}, {‖ \cdot ‖}_{p})$ , $p \in [1, \infty] .$

Clearly we have also: $f \in C_{U} (R^{N}, X)$ (uniformly continuous functions), iff $ω_{1} (f, δ) \to 0$ as $δ ↓ 0$ , where $ω_{1}$ is defined similarly to (37). The space $C_{B} (R^{N}, X)$ denotes the continuous and bounded functions on $R^{N} .$

When $f \in C_{B} (R^{N}, X)$ we define,

\begin{aligned} B_{n} (f, x) & := B_{n} (f, x_{1}, . . ., x_{N}) \\ := \sum_{k = - \infty}^{\infty} f (\frac{k}{n}) Z (n x - k) \\ := \sum_{k_{1} = - \infty}^{\infty} \sum_{k_{2} = - \infty}^{\infty} . . . \sum_{k_{N} = - \infty}^{\infty} f (\frac{k_{1}}{n}, \frac{k_{2}}{n}, . . ., \frac{k_{N}}{n}) (\prod_{i = 1}^{N} ψ (n x_{i} - k_{i})), \end{aligned}

$n \in N$ , $\forall$ $x \in R^{N},$ $N \in N$ , the multivariate quasi-interpolation neural network operator.

Also for $f \in C_{B} (R^{N}, X)$ we define the multivariate Kantorovich type neural network operator

\begin{aligned} C_{n} (f, x) := C_{n} (f, x_{1}, . . ., x_{N}) \\ := \sum_{k = - \infty}^{\infty} (n^{N} \int_{\frac{k}{n}}^{\frac{k + 1}{n}} f (t) d t) Z (n x - k) \\ = \sum_{k_{1} = - \infty}^{\infty} \sum_{k_{2} = - \infty}^{\infty} . . . \sum_{k_{N} = - \infty}^{\infty} (n^{N} \int_{\frac{k_{1}}{n}}^{\frac{k_{1} + 1}{n}} \int_{\frac{k_{2}}{n}}^{\frac{k_{2} + 1}{n}} . . . \int_{\frac{k_{N}}{n}}^{\frac{k_{N} + 1}{n}} f (t_{1}, . . ., t_{N}) d t_{1} . . . d t_{N}) \\ \cdot (\prod_{i = 1}^{N} ψ (n x_{i} - k_{i})), \end{aligned}

$n \in N, \forall$ $x \in R^{N} .$

Again for $f \in C_{B} (R^{N}, X),$ $N \in N,$ we define the multivariate neural network operator of quadrature type $D_{n} (f, x)$ , $n \in N,$ as follows.

Let $θ = (θ_{1}, . . ., θ_{N}) \in N^{N},$ $r = (r_{1}, . . ., r_{N}) \in Z_{+}^{N}$ , $w_{r} = w_{r_{1}, r_{2}, . . . r_{N}} \geq 0$ , such that $\sum_{r = 0}^{θ} w_{r} = \sum_{r_{1} = 0}^{θ_{1}} \sum_{r_{2} = 0}^{θ_{2}} . . . \sum_{r_{N} = 0}^{θ_{N}} w_{r_{1}, r_{2}, . . . r_{N}} = 1;$ $k \in Z^{N}$ and

\begin{aligned} δ_{n k} (f) & := δ_{n, k_{1}, k_{2}, . . ., k_{N}} (f) := \sum_{r = 0}^{θ} w_{r} f (\frac{k}{n} + \frac{r}{n θ}) \\ = \sum_{r_{1} = 0}^{θ_{1}} \sum_{r_{2} = 0}^{θ_{2}} . . . \sum_{r_{N} = 0}^{θ_{N}} w_{r_{1}, r_{2}, . . . r_{N}} f (\frac{k_{1}}{n} + \frac{r_{1}}{n θ_{1}}, \frac{k_{2}}{n} + \frac{r_{2}}{n θ_{2}}, . . ., \frac{k_{N}}{n} + \frac{r_{N}}{n θ_{N}}), \end{aligned}

where $\frac{r}{θ} := (\frac{r_{1}}{θ_{1}}, \frac{r_{2}}{θ_{2}}, . . ., \frac{r_{N}}{θ_{N}}) .$

We set

\begin{aligned} D_{n} (f, x) & := D_{n} (f, x_{1}, . . ., x_{N}) := \sum_{k = - \infty}^{\infty} δ_{n k} (f) Z (n x - k) \\ = \sum_{k_{1} = - \infty}^{\infty} \sum_{k_{2} = - \infty}^{\infty} . . . \sum_{k_{N} = - \infty}^{\infty} δ_{n, k_{1}, k_{2}, . . ., k_{N}} (f) (\prod_{i = 1}^{N} ψ (n x_{i} - k_{i})), \end{aligned}

$\forall$ $x \in R^{N} .$

In this article we study the approximation properties of $A_{n}, B_{n}, C_{n},$ $D_{n}$ neural network operators and as well of their iterates. That is, the quantitative pointwise and uniform convergence of these operators to the unit operator $I$ .

3 Multivariate general Neural Network Approximations

Here we present several vectorial neural network approximations to Banach space valued functions given with rates.

We give

Theorem 5

Let $f \in C (\prod_{i = 1}^{N} [a_{i}, b_{i}], X),$ $0 < β < 1$ , $x \in (\prod_{i = 1}^{N} [a_{i}, b_{i}]),$ $N, n \in N$ with $n^{1 - β} > 2$ . Then

{‖ A_{n} (f, x) - f (x) ‖}_{γ} \leq {(4.9737)}^{N} [ω_{1} (f, \frac{1}{n^{β}}) + \frac{4 {‖ {‖ f ‖}_{γ} ‖}_{\infty}}{π^{2} (n^{1 - β} - 2)}] =: λ_{1} (n),

and

{‖ {‖ A_{n} (f) - f ‖}_{γ} ‖}_{\infty} \leq λ_{1} (n) .

We notice that $lim_{n \to \infty} A_{n} (f) \overset{{‖ \cdot ‖}_{γ}}{=} f$ , pointwise and uniformly.

Above $ω_{1}$ is with respect to $p = \infty .$

Proof â–¼

We observe that

\begin{aligned} Δ (x) & := A_{n}^{*} (f, x) - f (x) \sum_{k = ⌈ n a ⌉}^{⌊ n b ⌋} Z (n x - k) \\ = \sum_{k = ⌈ n a ⌉}^{⌊ n b ⌋} f (\frac{k}{n}) Z (n x - k) - \sum_{k = ⌈ n a ⌉}^{⌊ n b ⌋} f (x) Z (n x - k) \\ = \sum_{k = ⌈ n a ⌉}^{⌊ n b ⌋} (f (\frac{k}{n}) - f (x)) Z (n x - k) . \end{aligned}

Thus

Unknown environment 'bgroup'

So that

{‖ Δ (x) ‖}_{γ} \leq ω_{1} (f, \frac{1}{n^{β}}) + \frac{4 {‖ {‖ f ‖}_{γ} ‖}_{\infty}}{π^{2} (n^{1 - β} - 2)} .

Now using (36) we finish the proof.

Proof â–¼

We make

Remark 1 [ 11 , pp. 263–266 ]

Let $(R^{N}, {‖ \cdot ‖}_{p})$ , $N \in N$ ; where ${‖ \cdot ‖}_{p}$ is the $L_{p}$ -norm, $1 \leq p \leq \infty$ . $R^{N}$ is a Banach space, and ${(R^{N})}^{j}$ denotes the $j$ -fold product space $R^{N} \times . . . \times R^{N}$ endowed with the max-norm ${‖ x ‖}_{{(R^{N})}^{j}} := max_{1 \leq λ \leq j} {‖ x_{λ} ‖}_{p}$ , where $x := (x_{1}, . . ., x_{j}) \in {(R^{N})}^{j} .$

Let $(X, {‖ \cdot ‖}_{γ})$ be a general Banach space. Then the space $L_{j} := L_{j} ((R^{N})^{j}; X)$ of all $j$ -multilinear continuous maps $g : (R^{N})^{j} \to X$ , $j = 1, . . ., m$ , is a Banach space with norm

‖ g ‖ := {‖ g ‖}_{L_{j}} := sup_{({‖ x ‖}_{(R^{N})^{j}} = 1)} {‖ g (x) ‖}_{γ} = sup \frac{{‖ g (x) ‖}_{γ}}{{‖ x_{1} ‖}_{p} . . . {‖ x_{j} ‖}_{p}} .

Let $M$ be a non-empty convex and compact subset of $R^{k}$ and $x_{0} \in M$ is fixed.

Let $O$ be an open subset of $R^{N} : M \subset O$ . Let $f : O \to X$ be a continuous function, whose Fréchet derivatives (see [ 19 ] ) $f^{(j)} : O \to L_{j} = L_{j} ({(R^{N})}^{j}; X)$ exist and are continuous for $1 \leq j \leq m$ , $m \in N$ .

Call ${(x - x_{0})}^{j} := (x - x_{0}, . . ., x - x_{0}) \in {(R^{N})}^{j}$ , $x \in M$ .

We will work with $f |_{M} .$

Then, by Taylor’s formula [ 12 ] , [ 19 , p. 124 ] , we get

f (x) = \sum_{j = 0}^{m} \frac{1}{j!} f^{(j)} (x_{0}) {(x - x_{0})}^{j} + R_{m} (x, x_{0}),  all x \in M,

where the remainder is the Riemann integral

R_{m} (x, x_{0}) := \int_{0}^{1} \frac{{(1 - u)}^{m - 1}}{(m - 1)!} (f^{(m)} (x_{0} + u (x - x_{0})) - f^{(m)} (x_{0})) {(x - x_{0})}^{m} d u,

here we set $f^{(0)} (x_{0}) {(x - x_{0})}^{0} = f (x_{0}) .$

We consider

w := ω_{1} (f^{(m)}, h) := \underset{{‖ x - y ‖}_{p} \leq h}{sup_{x, y \in M :}} ‖ f^{(m)} (x) - f^{(m)} (y) ‖,

$h > 0.$

We obtain

\begin{aligned} {‖ (f^{(m)} (x_{0} + u (x - x_{0})) - f^{(m)} (x_{0})) {(x - x_{0})}^{m} ‖}_{γ} \leq \\ \leq ‖ f^{(m)} (x_{0} + u (x - x_{0})) - f^{(m)} (x_{0}) ‖ \cdot {‖ x - x_{0} ‖}_{p}^{m} \\ \leq w {‖ x - x_{0} ‖}_{p}^{m} ⌈ \frac{u {‖ x - x_{0} ‖}_{p}}{h} ⌉, \end{aligned}

by Lemma 7.1.1, [ 1 , p. 208 ] , where $⌈ \cdot ⌉$ is the ceiling.

Therefore for all $x \in M$ (see [ 1 , pp. 121–122 ] ):

\begin{aligned} {‖ R_{m} (x, x_{0}) ‖}_{γ} & \leq w {‖ x - x_{0} ‖}_{p}^{m} \int_{0}^{1} ⌈ \frac{u {‖ x - x_{0} ‖}_{p}}{h} ⌉ \frac{{(1 - u)}^{m - 1}}{(m - 1)!} d u \\ = w Φ_{m} ({‖ x - x_{0} ‖}_{p}) \end{aligned}

by a change of variable, where

Φ_{m} (t) := \int_{0}^{| t |} ⌈ \frac{s}{h} ⌉ \frac{{(| t | - s)}^{m - 1}}{(m - 1)!} d s = \frac{1}{m!} (\sum_{j = 0}^{\infty} {(| t | - j h)}_{+}^{m}), \ \forall t \in R,

is a (polynomial) spline function, see [ 1 , p. 210–211 ] .

Also from there we get

Φ_{m} (t) \leq (\frac{{| t |}^{m + 1}}{(m + 1)! h} + \frac{{| t |}^{m}}{2 m!} + \frac{h {| t |}^{m - 1}}{8 (m - 1)!}), \ \forall t \in R,

with equality true only at $t = 0$ .

Therefore it holds

{‖ R_{m} (x, x_{0}) ‖}_{γ} \leq w (\frac{{‖ x - x_{0} ‖}_{p}^{m + 1}}{(m + 1)! h} + \frac{{‖ x - x_{0} ‖}_{p}^{m}}{2 m!} + \frac{h {‖ x - x_{0} ‖}_{p}^{m - 1}}{8 (m - 1)!}), \ \forall x \in M .

We have found that

\begin{aligned} {‖ f (x) - \sum_{j = 0}^{m} \frac{1}{j!} f^{(j)} (x_{0}) {(x - x_{0})}^{j} ‖}_{γ} \leq \\ \leq ω_{1} (f^{(m)}, h) (\frac{{‖ x - x_{0} ‖}_{p}^{m + 1}}{(m + 1)! h} + \frac{{‖ x - x_{0} ‖}_{p}^{m}}{2 m!} + \frac{h {‖ x - x_{0} ‖}_{p}^{m - 1}}{8 (m - 1)!}) < \infty, \end{aligned}

$\forall$ $x, x_{0} \in M .$

Here $0 < ω_{1} (f^{(m)}, h) < \infty$ , by $M$ being compact and $f^{(m)}$ being continuous on $M$ .

One can rewrite (60) as follows:

\begin{aligned} {‖ f (\cdot) - \sum_{j = 0}^{m} \frac{f^{(j)} (x_{0}) {(\cdot - x_{0})}^{j}}{j!} ‖}_{γ} \leq \\ \leq ω_{1} (f^{(m)}, h) (\frac{{‖ \cdot - x_{0} ‖}_{p}^{m + 1}}{(m + 1)! h} + \frac{{‖ \cdot - x_{0} ‖}_{p}^{m}}{2 m!} + \frac{h {‖ \cdot - x_{0} ‖}_{p}^{m - 1}}{8 (m - 1)!}), \forall x_{0} \in M, \end{aligned}

a pointwise functional inequality on $M$ .

Here ${(\cdot - x_{0})}^{j}$ maps $M$ into ${(R^{N})}^{j}$ and it is continuous, also $f^{(j)} (x_{0})$ maps ${(R^{N})}^{j}$ into $X$ and it is continuous. Hence their composition $f^{(j)} (x_{0}) {(\cdot - x_{0})}^{j}$ is continuous from $M$ into $X$ .

Clearly $f (\cdot) - \sum_{j = 0}^{m} \frac{f^{(j)} (x_{0}) {(\cdot - x_{0})}^{j}}{j!} \in C (M, X)$ , hence ${‖ f (\cdot) - \sum_{j = 0}^{m} \frac{f^{(j)} (x_{0}) {(\cdot - x_{0})}^{j}}{j!} ‖}_{γ} \in C (M)$ .

Let ${{\tilde{L}}_{N}}_{N \in N}$ be a sequence of positive linear operators mapping $C (M)$ into $C (M) .$

Therefore we obtain

\begin{aligned} ({\tilde{L}}_{N} ({‖ f (\cdot) - \sum_{j = 0}^{m} \frac{f^{(j)} (x_{0}) {(\cdot - x_{0})}^{j}}{j!} ‖}_{γ})) (x_{0}) \leq \\ \leq ω_{1} (f^{(m)}, h) [\frac{({\tilde{L}}_{N} ({‖ \cdot - x_{0} ‖}_{p}^{m + 1})) (x_{0})}{(m + 1)! h} + \frac{({\tilde{L}}_{N} ({‖ \cdot - x_{0} ‖}_{p}^{m})) (x_{0})}{2 m!} \\ + \frac{h ({\tilde{L}}_{N} ({‖ \cdot - x_{0} ‖}_{p}^{m - 1})) (x_{0})}{8 (m - 1)!}], \end{aligned}

$\forall$ $N \in N$ , $\forall$ $x_{0} \in M$ . â–¡

Clearly (62) is valid when $M = \prod_{i = 1}^{N} [a_{i}, b_{i}]$ and ${\tilde{L}}_{n} = {\tilde{A}}_{n}$ , see (28).

All the above is preparation for the following theorem, where we assume Fréchet differentiability of functions.

This will be a direct application of Theorem 10.2, [ 11 , pp. 268–270 ] . The operators $A_{n},$ ${\tilde{A}}_{n}$ fulfill its assumptions, see (27), (28), (30), (31) and (32).

We present the following high order approximation results.

Theorem 6

Let $O$ open subset of $(R^{N}, {‖ \cdot ‖}_{p})$ , $p \in [1, \infty]$ , such that $\prod_{i = 1}^{N} [a_{i}, b_{i}] \subset O \subseteq R^{N}$ , and let $(X, {‖ \cdot ‖}_{γ})$ be a general Banach space. Let $m \in N$ and $f \in C^{m} (O, X)$ , the space of $m$ -times continuously Fréchet differentiable functions from $O$ into $X$ . We study the approximation of $f |_{\prod_{i = 1}^{N} [a_{i}, b_{i}]} .$ Let $x_{0} \in (\prod_{i = 1}^{N} [a_{i}, b_{i}])$ and $r > 0$ . Then

\begin{aligned} {‖ (A_{n} (f)) (x_{0}) - \sum_{j = 0}^{m} \frac{1}{j!} (A_{n} (f^{(j)} (x_{0}) {(\cdot - x_{0})}^{j})) (x_{0}) ‖}_{γ} \leq \\ \leq \frac{1}{r m!} ω_{1} (f^{(m)}, r {(({\tilde{A}}_{n} ({‖ \cdot - x_{0} ‖}_{p}^{m + 1})) (x_{0}))}^{\frac{1}{m + 1}}) \\ \cdot {(({\tilde{A}}_{n} ({‖ \cdot - x_{0} ‖}_{p}^{m + 1})) (x_{0}))}^{(\frac{m}{m + 1})} [\frac{1}{(m + 1)} + \frac{r}{2} + \frac{m r^{2}}{8}], \end{aligned}

2) additionally if $f^{(j)} (x_{0}) = 0$ , $j = 1, . . ., m$ , we have

\begin{aligned} {‖ (A_{n} (f)) (x_{0}) - f (x_{0}) ‖}_{γ} \leq \\ \leq \frac{1}{r m!} ω_{1} (f^{(m)}, r {(({\tilde{A}}_{n} ({‖ \cdot - x_{0} ‖}_{p}^{m + 1})) (x_{0}))}^{\frac{1}{m + 1}}) \\ \cdot {(({\tilde{A}}_{n} ({‖ \cdot - x_{0} ‖}_{p}^{m + 1})) (x_{0}))}^{(\frac{m}{m + 1})} [\frac{1}{(m + 1)} + \frac{r}{2} + \frac{m r^{2}}{8}], \end{aligned}

\begin{aligned} {‖ (A_{n} (f)) (x_{0}) - f (x_{0}) ‖}_{γ} \leq \\ \leq \sum_{j = 1}^{m} \frac{1}{j!} {‖ (A_{n} (f^{(j)} (x_{0}) {(\cdot - x_{0})}^{j})) (x_{0}) ‖}_{γ} \\ + \frac{1}{r m!} ω_{1} (f^{(m)}, r {(({\tilde{A}}_{n} ({‖ \cdot - x_{0} ‖}_{p}^{m + 1})) (x_{0}))}^{\frac{1}{m + 1}}) \\ \cdot {(({\tilde{A}}_{n} ({‖ \cdot - x_{0} ‖}_{p}^{m + 1})) (x_{0}))}^{(\frac{m}{m + 1})} [\frac{1}{(m + 1)} + \frac{r}{2} + \frac{m r^{2}}{8}], \end{aligned}

and

\begin{aligned} {‖ {‖ A_{n} (f) - f ‖}_{γ} ‖}_{\infty, \prod_{i = 1}^{N} [a_{i}, b_{i}]} \leq \\ \leq \sum_{j = 1}^{m} \frac{1}{j!} {‖ {‖ (A_{n} (f^{(j)} (x_{0}) {(\cdot - x_{0})}^{j})) (x_{0}) ‖}_{γ} ‖}_{\infty, x_{0} \in \prod_{i = 1}^{N} [a_{i}, b_{i}]} \\ + \frac{1}{r m!} ω_{1} (f^{(m)}, r {‖ ({\tilde{A}}_{n} ({‖ \cdot - x_{0} ‖}_{p}^{m + 1})) (x_{0}) ‖}_{\infty, x_{0} \in \prod_{i = 1}^{N} [a_{i}, b_{i}]}^{\frac{1}{m + 1}}) \\ \cdot {‖ ({\tilde{A}}_{n} ({‖ \cdot - x_{0} ‖}_{p}^{m + 1})) (x_{0}) ‖}_{\infty, x_{0} \in \prod_{i = 1}^{N} [a_{i}, b_{i}]}^{(\frac{m}{m + 1})} [\frac{1}{(m + 1)} + \frac{r}{2} + \frac{m r^{2}}{8}] . \end{aligned}

We need

Lemma 2

The function $({\tilde{A}}_{n} ({‖ \cdot - x_{0} ‖}_{p}^{m})) (x_{0})$ is continuous in $x_{0} \in (\prod_{i = 1}^{N} [a_{i}, b_{i}])$ , $m \in N$ .

Proof â–¼

By Lemma 10.3, [ 11 , p. 272 ] .

Proof â–¼

We make

Remark 2

By Remark 10.4, [ 11 , p. 273 ] , we get that

\begin{aligned} {‖ ({\tilde{A}}_{n} ({‖ \cdot - x_{0} ‖}_{p}^{k})) (x_{0}) ‖}_{\infty, x_{0} \in \prod_{i = 1}^{N} [a_{i}, b_{i}]} \leq \\ \leq {‖ ({\tilde{A}}_{n} ({‖ \cdot - x_{0} ‖}_{p}^{m + 1})) (x_{0}) ‖}_{\infty, x_{0} \in \prod_{i = 1}^{N} [a_{i}, b_{i}]}^{(\frac{k}{m + 1})}, \end{aligned}

for all $k = 1, . . ., m .$ â–¡

We give

Corollary 1

(to theorem 6, case of $m = 1$ ) Then

\begin{aligned} {‖ (A_{n} (f)) (x_{0}) - f (x_{0}) ‖}_{γ} \leq \\ \leq {‖ (A_{n} (f^{(1)} (x_{0}) (\cdot - x_{0}))) (x_{0}) ‖}_{γ} \\ + \frac{1}{2 r} ω_{1} (f^{(1)}, r {(({\tilde{A}}_{n} ({‖ \cdot - x_{0} ‖}_{p}^{2})) (x_{0}))}^{\frac{1}{2}}) \\ \cdot {(({\tilde{A}}_{n} ({‖ \cdot - x_{0} ‖}_{p}^{2})) (x_{0}))}^{\frac{1}{2}} [1 + r + \frac{r^{2}}{4}], \end{aligned}

and

\begin{aligned} {‖ {‖ (A_{n} (f)) - f ‖}_{γ} ‖}_{\infty, \prod_{i = 1}^{N} [a_{i}, b_{i}]} \leq \\ \leq {‖ {‖ (A_{n} (f^{(1)} (x_{0}) (\cdot - x_{0}))) (x_{0}) ‖}_{γ} ‖}_{\infty, x_{0} \in \prod_{i = 1}^{N} [a_{i}, b_{i}]} \\ + \frac{1}{2 r} ω_{1} (f^{(1)}, r {‖ ({\tilde{A}}_{n} ({‖ \cdot - x_{0} ‖}_{p}^{2})) (x_{0}) ‖}_{\infty, x_{0} \in \prod_{i = 1}^{N} [a_{i}, b_{i}]}^{\frac{1}{2}}) \\ \cdot {‖ ({\tilde{A}}_{n} ({‖ \cdot - x_{0} ‖}_{p}^{2})) (x_{0}) ‖}_{\infty, x_{0} \in \prod_{i = 1}^{N} [a_{i}, b_{i}]}^{\frac{1}{2}} [1 + r + \frac{r^{2}}{4}], \end{aligned}

$r > 0.$

We make

Remark 3

We estimate $0 < α < 1$ , $m, n \in N : n^{1 - α} > 2$ ,

Unknown environment 'bgroup'

(where $b - a = (b_{1} - a_{1}, . . ., b_{N} - a_{N})$ ).

We have proved that ( $\forall$ $x_{0} \in \prod_{i = 1}^{N} [a_{i}, b_{i}]$ )

{\tilde{A}}_{n} ({‖ \cdot - x_{0} ‖}_{\infty}^{m + 1}) (x_{0}) < {(4.9737)}^{N} {\frac{1}{n^{α (m + 1)}} + \frac{2 {‖ b - a ‖}_{\infty}^{m + 1}}{π^{2} (n^{1 - α} - 2)}} =: φ_{1} (n)

( $0 < α < 1$ , $m, n \in N : n^{1 - α} > 2$ ).

And, consequently it holds

\begin{aligned} {‖ {\tilde{A}}_{n} ({‖ \cdot - x_{0} ‖}_{\infty}^{m + 1}) (x_{0}) ‖}_{\infty, x_{0} \in \prod_{i = 1}^{N} [a_{i}, b_{i}]} < \\ < {(4.9737)}^{N} {\frac{1}{n^{α (m + 1)}} + \frac{2 {‖ b - a ‖}_{\infty}^{m + 1}}{π^{2} (n^{1 - α} - 2)}} = φ_{1} (n) \to 0, \  as n \to + \infty . \end{aligned}

So, we have that $φ_{1} (n) \to 0$ , as $n \to + \infty$ . Thus, when $p \in [1, \infty]$ , from theorem 6 we have the convergence to zero in the right hand sides of parts (1), (2).

Next we estimate ${‖ ({\tilde{A}}_{n} (f^{(j)} (x_{0}) {(\cdot - x_{0})}^{j})) (x_{0}) ‖}_{γ} .$

We have that

({\tilde{A}}_{n} (f^{(j)} (x_{0}) {(\cdot - x_{0})}^{j})) (x_{0}) = \frac{\sum_{k = ⌈ n a ⌉}^{⌊ n b ⌋} f^{(j)} (x_{0}) {(\frac{k}{n} - x_{0})}^{j} Z (n x_{0} - k)}{\sum_{k = ⌈ n a ⌉}^{⌊ n b ⌋} Z (n x_{0} - k)} .

When $p = \infty$ , $j = 1, . . ., m,$ we obtain

{‖ f^{(j)} (x_{0}) {(\frac{k}{n} - x_{0})}^{j} ‖}_{γ} \leq ‖ f^{(j)} (x_{0}) ‖ {‖ \frac{k}{n} - x_{0} ‖}_{\infty}^{j} .

We further have that

Unknown environment 'bgroup'

That is

{‖ ({\tilde{A}}_{n} (f^{(j)} (x_{0}) {(\cdot - x_{0})}^{j})) (x_{0}) ‖}_{γ} \to 0, as n \to \infty .

Therefore when $p = \infty$ , for $j = 1, . . ., m$ , we have proved:

\begin{aligned} {‖ ({\tilde{A}}_{n} (f^{(j)} (x_{0}) {(\cdot - x_{0})}^{j})) (x_{0}) ‖}_{γ} < \\ < {(4.9737)}^{N} ‖ f^{(j)} (x_{0}) ‖ {\frac{1}{n^{α j}} + \frac{2 {‖ b - a ‖}_{\infty}^{j}}{π^{2} (n^{1 - α} - 2)}} \\ \leq {(4.9737)}^{N} {‖ f^{(j)} ‖}_{\infty} {\frac{1}{n^{α j}} + \frac{2 {‖ b - a ‖}_{\infty}^{j}}{π^{2} (n^{1 - α} - 2)}} =: φ_{2 j} (n) < \infty, \end{aligned}

and converges to zero, as $n \to \infty .$ â–¡

We conclude:

In theorem 6, the right hand sides of (65) and (66) converge to zero as $n \to \infty$ , for any $p \in [1, \infty]$ .

Also in Corollary 1, the right hand sides of (68) and (69) converge to zero as $n \to \infty$ , for any $p \in [1, \infty] .$

Conclusion 1

We have proved that the left hand sides of 63, 64, 65, 66 and 68, 69 converge to zero as $n \to \infty$ , for $p \in [1, \infty]$ . Consequently $A_{n} \to I$ (unit operator) pointwise and uniformly, as $n \to \infty$ , where $p \in [1, \infty]$ . In the presence of initial conditions we achieve a higher speed of convergence, see 64. Higher speed of convergence happens also to the left hand side of 63.

We give

Corollary 2

(to 6) Let $O$ open subset of $(R^{N}, {‖ \cdot ‖}_{\infty})$ , such that
$\prod_{i = 1}^{N} [a_{i}, b_{i}] \subset O \subseteq R^{N}$ , and let $(X, {‖ \cdot ‖}_{γ})$ be a general Banach space. Let $m \in N$ and $f \in C^{m} (O, X)$ , the space of $m$ -times continuously Fréchet differentiable functions from $O$ into $X$ . We study the approximation of $f |_{\prod_{i = 1}^{N} [a_{i}, b_{i}]} .$ Let $x_{0} \in (\prod_{i = 1}^{N} [a_{i}, b_{i}])$ and $r > 0$ . Here $φ_{1} (n)$ as in 74 and $φ_{2 j} (n)$ as in 82, where $n \in N : n^{1 - α} > 2$ , $0 < α < 1$ , $j = 1, . . ., m .$ Then

\begin{aligned} {‖ (A_{n} (f)) (x_{0}) - \sum_{j = 0}^{m} \frac{1}{j!} (A_{n} (f^{(j)} (x_{0}) {(\cdot - x_{0})}^{j})) (x_{0}) ‖}_{γ} \leq \\ \leq \frac{1}{r m!} ω_{1} (f^{(m)}, r {(φ_{1} (n))}^{\frac{1}{m + 1}}) {(φ_{1} (n))}^{(\frac{m}{m + 1})} [\frac{1}{(m + 1)} + \frac{r}{2} + \frac{m r^{2}}{8}], \end{aligned}

2) additionally, if $f^{(j)} (x_{0}) = 0$ , $j = 1, . . ., m$ , we have

\begin{aligned} {‖ (A_{n} (f)) (x_{0}) - f (x_{0}) ‖}_{γ} \leq \\ \leq \frac{1}{r m!} ω_{1} (f^{(m)}, r {(φ_{1} (n))}^{\frac{1}{m + 1}}) {(φ_{1} (n))}^{(\frac{m}{m + 1})} [\frac{1}{(m + 1)} + \frac{r}{2} + \frac{m r^{2}}{8}], \end{aligned}

\begin{aligned} {‖ {‖ A_{n} (f) - f ‖}_{γ} ‖}_{\infty, \prod_{i = 1}^{N} [a_{i}, b_{i}]} \leq \\ \leq \sum_{j = 1}^{m} \frac{φ_{2 j} (n)}{j!} + \frac{1}{r m!} ω_{1} (f^{(m)}, r {(φ_{1} (n))}^{\frac{1}{m + 1}}) {(φ_{1} (n))}^{(\frac{m}{m + 1})} [\frac{1}{(m + 1)} + \frac{r}{2} + \frac{m r^{2}}{8}] \\ =: φ_{3} (n) \to 0, as n \to \infty . \end{aligned}

We continue with

Theorem 7

Let $f \in C_{B} (R^{N}, X),$ $0 < β < 1$ , $x \in R^{N},$ $N, n \in N$ with $n^{1 - β} > 2$ , $ω_{1}$ is for $p = \infty$ . Then

{‖ B_{n} (f, x) - f (x) ‖}_{γ} \leq ω_{1} (f, \frac{1}{n^{β}}) + \frac{4 {‖ {‖ f ‖}_{γ} ‖}_{\infty}}{π^{2} (n^{1 - β} - 2)} =: λ_{2} (n),

{‖ {‖ B_{n} (f) - f ‖}_{γ} ‖}_{\infty} \leq λ_{2} (n) .

Given that $f \in (C_{U} (R^{N}, X) \cap C_{B} (R^{N}, X))$ , we obtain $lim_{n \to \infty} B_{n} (f) = f$ , uniformly.

Proof â–¼

We have that

\begin{aligned} B_{n} (f, x) - f (x) & \overset{(???)}{=} \sum_{k = - \infty}^{\infty} f (\frac{k}{n}) Z (n x - k) - f (x) \sum_{k = - \infty}^{\infty} Z (n x - k) \\ = \sum_{k = - \infty}^{\infty} (f (\frac{k}{n}) - f (x)) Z (n x - k) . \end{aligned}

Hence

Unknown environment 'bgroup'

proving the claim.

Proof â–¼

We give

Theorem 8

Let $f \in C_{B} (R^{N}, X),$ $0 < β < 1$ , $x \in R^{N},$ $N, n \in N$ with $n^{1 - β} > 2$ , $ω_{1}$ is for $p = \infty$ . Then

{‖ C_{n} (f, x) - f (x) ‖}_{γ} \leq ω_{1} (f, \frac{1}{n} + \frac{1}{n^{β}}) + \frac{4 {‖ {‖ f ‖}_{γ} ‖}_{\infty}}{π^{2} (n^{1 - β} - 2)} =: λ_{3} (n),

{‖ {‖ C_{n} (f) - f ‖}_{γ} ‖}_{\infty} \leq λ_{3} (n) .

Given that $f \in (C_{U} (R^{N}, X) \cap C_{B} (R^{N}, X)),$ we obtain $lim_{n \to \infty} C_{n} (f) = f$ , uniformly.

Proof â–¼

We notice that

\int_{\frac{k}{n}}^{\frac{k + 1}{n}} f (t) d t = \int_{\frac{k_{1}}{n}}^{\frac{k_{1} + 1}{n}} \int_{\frac{k_{2}}{n}}^{\frac{k_{2} + 1}{n}} . . . \int_{\frac{k_{N}}{n}}^{\frac{k_{N} + 1}{n}} f (t_{1}, t_{2}, . . ., t_{N}) d t_{1} d t_{2} . . . d t_{N} =

\int_{0}^{\frac{1}{n}} \int_{0}^{\frac{1}{n}} . . . \int_{0}^{\frac{1}{n}} f (t_{1} + \frac{k_{1}}{n}, t_{2} + \frac{k_{2}}{n}, . . ., t_{N} + \frac{k_{N}}{n}) d t_{1} . . . d t_{N} = \int_{0}^{\frac{1}{n}} f (t + \frac{k}{n}) d t .

Thus it holds (by (40))

C_{n} (f, x) = \sum_{k = - \infty}^{\infty} (n^{N} \int_{0}^{\frac{1}{n}} f (t + \frac{k}{n}) d t) Z (n x - k) .

We observe that

Unknown environment 'bgroup'

proving the claim.

Proof â–¼

We also present

Theorem 9

Let $f \in C_{B} (R^{N}, X),$ $0 < β < 1$ , $x \in R^{N},$ $N, n \in N$ with $n^{1 - β} > 2$ , $ω_{1}$ is for $p = \infty .$ Then

{‖ D_{n} (f, x) - f (x) ‖}_{γ} \leq ω_{1} (f, \frac{1}{n} + \frac{1}{n^{β}}) + \frac{4 {‖ {‖ f ‖}_{γ} ‖}_{\infty}}{π^{2} (n^{1 - β} - 2)} = λ_{4} (n),

103

{‖ {‖ D_{n} (f) - f ‖}_{γ} ‖}_{\infty} \leq λ_{4} (n) .

104

Given that $f \in (C_{U} (R^{N}, X) \cap C_{B} (R^{N}, X)),$ we obtain $lim_{n \to \infty} D_{n} (f) = f$ , uniformly.

Proof â–¼

We have that (by (42))

Unknown environment 'bgroup'

proving the claim.

Proof â–¼

We make

Definition 2

Let $f \in C_{B} (R^{N}, X)$ , $N \in N$ , where $(X, {‖ \cdot ‖}_{γ})$ is a Banach space. We define the general neural network operator

F_{n} (f, x) := \sum_{k = - \infty}^{\infty} l_{n k} (f) Z (n x - k) = {\begin{cases} B_{n} (f, x), \  if l_{n k} (f) = f (\frac{k}{n}), \\ C_{n} (f, x), \  if l_{n k} (f) = n^{N} \int_{\frac{k}{n}}^{\frac{k + 1}{n}} f (t) d t, \\ D_{n} (f, x), \  if l_{n k} (f) = δ_{n k} (f) . \end{cases}

111

Clearly $l_{n k} (f)$ is an $X$ -valued bounded linear functional such that ${‖ l_{n k} (f) ‖}_{γ} \leq {‖ {‖ f ‖}_{γ} ‖}_{\infty} .$

Hence $F_{n} (f)$ is a bounded linear operator with ${‖ {‖ F_{n} (f) ‖}_{γ} ‖}_{\infty} \leq {‖ {‖ f ‖}_{γ} ‖}_{\infty}$ .

We need

Theorem 10

Let $f \in C_{B} (R^{N}, X)$ , $N \geq 1$ . Then $F_{n} (f) \in C_{B} (R^{N}, X) .$

Proof â–¼

Clearly

F_{n} (f)

is a bounded function.

Next we prove the continuity of $F_{n} (f)$ . Notice for $N = 1$ , $Z = ψ$ by (16).

We will use the generalized Weierstrass $M$ test: If a sequence of positive constants $M_{1}, M_{2}, M_{3}, . . .,$ can be found such that in some interval

(a) ${‖ u_{n} (x) ‖}_{γ} \leq M_{n}$ , $n = 1, 2, 3, . . .$

(b) $\sum M_{n}$ converges,

then $\sum u_{n} (x)$ is uniformly and absolutely convergent in the interval.

Also we will use:

If ${u_{n} (x)}$ , $n = 1, 2, 3, . . .$ are continuous in $[a, b]$ and if $\sum u_{n} (x)$ converges uniformly to the sum $S (x)$ in $[a, b]$ , then $S (x)$ is continuous in $[a, b]$ . I.e. a uniformly convergent series of continuous functions is a continuous function. First we prove claim for $N = 1$ .

We will prove that $\sum_{k = - \infty}^{\infty} l_{n k} (f) ψ (n x - k)$ is continuous in $x \in R$ .

There always exists $λ \in N$ such that $n x \in [- λ, λ] .$

Since $n x \leq λ$ , then $- n x \geq - λ$ and $k - n x \geq k - λ \geq 0$ , when $k \geq λ$ . Therefore

\sum_{k = λ}^{\infty} ψ (n x - k) = \sum_{k = λ}^{\infty} ψ (k - n x) \leq \sum_{k = λ}^{\infty} ψ (k - λ) = \sum_{k^{'} = 0}^{\infty} ψ (k^{'}) \leq 1.

112

So for $k \geq λ$ we get

{‖ l_{n k} (f) ‖}_{γ} ψ (n x - k) \leq {‖ {‖ f ‖}_{γ} ‖}_{\infty} ψ (k - λ),

and

{‖ {‖ f ‖}_{γ} ‖}_{\infty} \sum_{k = λ}^{\infty} ψ (k - λ) \leq {‖ {‖ f ‖}_{γ} ‖}_{\infty} .

Hence by the generalized Weierstrass $M$ test we obtain that $\sum_{k = λ}^{\infty} l_{n k} (f) ψ (n x - k)$ is uniformly and absolutely convergent on $[- \frac{λ}{n}, \frac{λ}{n}] .$

Since $l_{n k} (f) ψ (n x - k)$ is continuous in $x$ , then $\sum_{k = λ}^{\infty} l_{n k} (f) ψ (n x - k)$ is continuous on $[- \frac{λ}{n}, \frac{λ}{n}] .$

Because $n x \geq - λ$ , then $- n x \leq λ$ , and $k - n x \leq k + λ \leq 0$ , when $k \leq - λ$ . Therefore

\sum_{k = - \infty}^{- λ} ψ (n x - k) = \sum_{k = - \infty}^{- λ} ψ (k - n x) \leq \sum_{k = - \infty}^{- λ} ψ (k + λ) = \sum_{k^{'} = - \infty}^{0} ψ (k^{'}) \leq 1.

So for $k \leq - λ$ we get

{‖ l_{n k} (f) ‖}_{γ} ψ (n x - k) \leq {‖ {‖ f ‖}_{γ} ‖}_{\infty} ψ (k + λ),

113

and

{‖ {‖ f ‖}_{γ} ‖}_{\infty} \sum_{k = - \infty}^{- λ} ψ (k + λ) \leq {‖ {‖ f ‖}_{γ} ‖}_{\infty} .

Hence by Weierstrass $M$ test we obtain that $\sum_{k = - \infty}^{- λ} l_{n k} (f) ψ (n x - k)$ is uniformly and absolutely convergent on $[- \frac{λ}{n}, \frac{λ}{n}] .$

Since $l_{n k} (f) ψ (n x - k)$ is continuous in $x$ , then $\sum_{k = - \infty}^{- λ} l_{n k} (f) ψ (n x - k)$ is continuous on $[- \frac{λ}{n}, \frac{λ}{n}] .$

So we proved that $\sum_{k = λ}^{\infty} l_{n k} (f) ψ (n x - k)$ and $\sum_{k = - \infty}^{- λ} l_{n k} (f) ψ (n x - k)$ are continuous on $R$ . Since $\sum_{k = - λ + 1}^{λ - 1} l_{n k} (f) ψ (n x - k)$ is a finite sum of continuous functions on $R$ , it is also a continuous function on $R$ .

Writing

\begin{aligned} \sum_{k = - \infty}^{\infty} l_{n k} (f) ψ (n x - k) & = \sum_{k = - \infty}^{- λ} l_{n k} (f) ψ (n x - k) \\ + \sum_{k = - λ + 1}^{λ - 1} l_{n k} (f) ψ (n x - k) + \sum_{k = λ}^{\infty} l_{n k} (f) ψ (n x - k) \end{aligned}

we have it as a continuous function on $R$ . Therefore $F_{n} (f)$ , when $N = 1$ , is a continuous function on $R$ .

When $N = 2$ we have

\begin{aligned} F_{n} (f, x_{1}, x_{2}) & = \sum_{k_{1} = - \infty}^{\infty} \sum_{k_{2} = - \infty}^{\infty} l_{n k} (f) ψ (n x_{1} - k_{1}) ψ (n x_{2} - k_{2}) \\ = \sum_{k_{1} = - \infty}^{\infty} ψ (n x_{1} - k_{1}) (\sum_{k_{2} = - \infty}^{\infty} l_{n k} (f) ψ (n x_{2} - k_{2})) \end{aligned}

(there always exist $λ_{1}, λ_{2} \in N$ such that $n x_{1} \in [- λ_{1}, λ_{1}]$ and $n x_{2} \in [- λ_{2}, λ_{2}]$ )

\begin{aligned} = \sum_{k_{1} = - \infty}^{\infty} ψ (n x_{1} - k_{1}) [\sum_{k_{2} = - \infty}^{- λ_{2}} l_{n k} (f) ψ (n x_{2} - k_{2}) \\ + \sum_{k_{2} = - λ_{2} + 1}^{λ_{2} - 1} l_{n k} (f) ψ (n x_{2} - k_{2}) + \sum_{k_{2} = λ_{2}}^{\infty} l_{n k} (f) ψ (n x_{2} - k_{2})] \\ = \sum_{k_{1} = - \infty}^{\infty} \sum_{k_{2} = - \infty}^{- λ_{2}} l_{n k} (f) ψ (n x_{1} - k_{1}) ψ (n x_{2} - k_{2}) \\ + \sum_{k_{1} = - \infty}^{\infty} \sum_{k_{2} = - λ_{2} + 1}^{λ_{2} - 1} l_{n k} (f) ψ (n x_{1} - k_{1}) ψ (n x_{2} - k_{2}) \\ + \sum_{k_{1} = - \infty}^{\infty} \sum_{k_{2} = λ_{2}}^{\infty} l_{n k} (f) ψ (n x_{1} - k_{1}) ψ (n x_{2} - k_{2}) =: (*) . \end{aligned}

(For convenience call

F (k_{1}, k_{2}, x_{1}, x_{2}) := l_{n k} (f) ψ (n x_{1} - k_{1}) ψ (n x_{2} - k_{2}) .)

Thus

\begin{aligned} (*) & = \sum_{k_{1} = - \infty}^{- λ_{1}} \sum_{k_{2} = - \infty}^{- λ_{2}} F (k_{1}, k_{2}, x_{1}, x_{2}) + \sum_{k_{1} = - λ_{1} + 1}^{λ_{1} - 1} \sum_{k_{2} = - \infty}^{- λ_{2}} F (k_{1}, k_{2}, x_{1}, x_{2}) \\ + \sum_{k_{1} = λ_{1}}^{\infty} \sum_{k_{2} = - \infty}^{- λ_{2}} F (k_{1}, k_{2}, x_{1}, x_{2}) + \sum_{k_{1} = - \infty}^{- λ_{1}} \sum_{k_{2} = - λ_{2} + 1}^{λ_{2} - 1} F (k_{1}, k_{2}, x_{1}, x_{2}) \\ + \sum_{k_{1} = - λ_{1} + 1}^{λ_{1} - 1} \sum_{k_{2} = - λ_{2} + 1}^{λ_{2} - 1} F (k_{1}, k_{2}, x_{1}, x_{2}) + \sum_{k_{1} = λ_{1}}^{\infty} \sum_{k_{2} = - λ_{2} + 1}^{λ_{2} - 1} F (k_{1}, k_{2}, x_{1}, x_{2}) \\ + \sum_{k_{1} = - \infty}^{- λ_{1}} \sum_{k_{2} = λ_{2}}^{\infty} F (k_{1}, k_{2}, x_{1}, x_{2}) + \sum_{k_{1} = - λ_{1} + 1}^{λ_{1} - 1} \sum_{k_{2} = λ_{2}}^{\infty} F (k_{1}, k_{2}, x_{1}, x_{2}) \\ + \sum_{k_{1} = λ_{1}}^{\infty} \sum_{k_{2} = λ_{2}}^{\infty} F (k_{1}, k_{2}, x_{1}, x_{2}) . \end{aligned}

Notice that the finite sum of continuous functions $F (k_{1}, k_{2}, x_{1}, x_{2})$ ,

$\sum_{k_{1} = - λ_{1} + 1}^{λ_{1} - 1} \sum_{k_{2} = - λ_{2} + 1}^{λ_{2} - 1} F (k_{1}, k_{2}, x_{1}, x_{2})$ is a continuous function.

The rest of the summands of $F_{n} (f, x_{1}, x_{2})$ are treated all the same way and similarly to the case of $N = 1$ . The method is demonstrated as follows.

We will prove that $\sum_{k_{1} = λ_{1}}^{\infty} \sum_{k_{2} = - \infty}^{- λ_{2}} l_{n k} (f) ψ (n x_{1} - k_{1}) ψ (n x_{2} - k_{2})$ is continuous in $(x_{1}, x_{2}) \in R^{2}$ .

The continuous function

{‖ l_{n k} (f) ‖}_{γ} ψ (n x_{1} - k_{1}) ψ (n x_{2} - k_{2}) \leq {‖ {‖ f ‖}_{γ} ‖}_{\infty} ψ (k_{1} - λ_{1}) ψ (k_{2} + λ_{2}),

and

\begin{aligned} {‖ {‖ f ‖}_{γ} ‖}_{\infty} \sum_{k_{1} = λ_{1}}^{\infty} \sum_{k_{2} = - \infty}^{- λ_{2}} ψ (k_{1} - λ_{1}) ψ (k_{2} + λ_{2}) = \\ = {‖ {‖ f ‖}_{γ} ‖}_{\infty} (\sum_{k_{1} = λ_{1}}^{\infty} ψ (k_{1} - λ_{1})) (\sum_{k_{2} = - \infty}^{- λ_{2}} ψ (k_{2} + λ_{2})) \\ \leq {‖ {‖ f ‖}_{γ} ‖}_{\infty} (\sum_{k_{1}^{'} = 0}^{\infty} ψ (k_{1}^{'})) (\sum_{k_{2}^{'} = - \infty}^{0} ψ (k_{2}^{'})) \leq {‖ {‖ f ‖}_{γ} ‖}_{\infty} . \end{aligned}

So by the Weierstrass $M$ test we get that

$\sum_{k_{1} = λ_{1}}^{\infty} \sum_{k_{2} = - \infty}^{- λ_{2}} l_{n k} (f) ψ (n x_{1} - k_{1}) ψ (n x_{2} - k_{2})$ is uniformly and absolutely convergent. Therefore it is continuous on $R^{2} .$

Next we prove continuity on $R^{2}$ of

$\sum_{k_{1} = - λ_{1} + 1}^{λ_{1} - 1} \sum_{k_{2} = - \infty}^{- λ_{2}} l_{n k} (f) ψ (n x_{1} - k_{1}) ψ (n x_{2} - k_{2})$ .

Notice here that

\begin{aligned} {‖ l_{n k} (f) ‖}_{γ} ψ (n x_{1} - k_{1}) ψ (n x_{2} - k_{2}) \leq \\ \leq {‖ {‖ f ‖}_{γ} ‖}_{\infty} ψ (n x_{1} - k_{1}) ψ (k_{2} + λ_{2}) \\ \leq {‖ {‖ f ‖}_{γ} ‖}_{\infty} ψ (0) ψ (k_{2} + λ_{2}) = 0.319 \cdot {‖ {‖ f ‖}_{γ} ‖}_{\infty} ψ (k_{2} + λ_{2}), \end{aligned}

and

\begin{aligned} 0.319 \cdot {‖ {‖ f ‖}_{γ} ‖}_{\infty} (\sum_{k_{1} = - λ_{1} + 1}^{λ_{1} - 1} 1) (\sum_{k_{2} = - \infty}^{- λ_{2}} ψ (k_{2} + λ_{2})) = \\ = 0.319 \cdot {‖ {‖ f ‖}_{γ} ‖}_{\infty} (2 λ_{1} - 1) (\sum_{k_{2}^{'} = - \infty}^{0} ψ (k_{2}^{'})) \\ \leq 0.319 \cdot (2 λ_{1} - 1) {‖ {‖ f ‖}_{γ} ‖}_{\infty} . \end{aligned}

So the double series under consideration is uniformly convergent and continuous. Clearly $F_{n} (f, x_{1}, x_{2})$ is proved to be continuous on $R^{2} .$

Similarly reasoning one can prove easily now, but with more tedious work, that $F_{n} (f, x_{1}, . . ., x_{N})$ is continuous on $R^{N}$ , for any $N \geq 1$ . We choose to omit this similar extra work.

Proof â–¼

Remark 4

By (27) it is obvious that ${‖ {‖ A_{n} (f) ‖}_{γ} ‖}_{\infty} \leq {‖ {‖ f ‖}_{γ} ‖}_{\infty} < \infty$ , and $A_{n} (f) \in C (\prod_{i = 1}^{N} [a_{i}, b_{i}], X)$ , given that $f \in C (\prod_{i = 1}^{N} [a_{i}, b_{i}], X) .$

Call $L_{n}$ any of the operators $A_{n}, B_{n}, C_{n}, D_{n} .$

Clearly then

{‖ {‖ L_{n}^{2} (f) ‖}_{γ} ‖}_{\infty} = {‖ {‖ L_{n} (L_{n} (f)) ‖}_{γ} ‖}_{\infty} \leq {‖ {‖ L_{n} (f) ‖}_{γ} ‖}_{\infty} \leq {‖ {‖ f ‖}_{γ} ‖}_{\infty},

117

etc.

Therefore we get

{‖ {‖ L_{n}^{k} (f) ‖}_{γ} ‖}_{\infty} \leq {‖ {‖ f ‖}_{γ} ‖}_{\infty}, \ \forall k \in N,

118

the contraction property.

Also we see that

{‖ {‖ L_{n}^{k} (f) ‖}_{γ} ‖}_{\infty} \leq {‖ {‖ L_{n}^{k - 1} (f) ‖}_{γ} ‖}_{\infty} \leq . . . \leq {‖ {‖ L_{n} (f) ‖}_{γ} ‖}_{\infty} \leq {‖ {‖ f ‖}_{γ} ‖}_{\infty} .

119

Here $L_{n}^{k}$ are bounded linear operators. â–¡

Notation 1

Here $N \in N$ , $0 < β < 1.$ Denote by

\begin{aligned} c_{N} & := {\begin{matrix} {(4.9737)}^{N}, \  if L_{n} = A_{n}, \\ 1, \  if L_{n} = B_{n}, C_{n}, D_{n}, \end{matrix} \\ φ (n) & := {\begin{matrix} \frac{1}{n^{β}}, \  if L_{n} = A_{n}, B_{n}, \\ \frac{1}{n} + \frac{1}{n^{β}}, \  if L_{n} = C_{n}, D_{n}, \end{matrix} \\ Ω & := {\begin{matrix} C (\prod_{i = 1}^{N} [a_{i}, b_{i}], X),  if L_{n} = A_{n}, \\ C_{B} (R^{N}, X), \  if L_{n} = B_{n}, C_{n}, D_{n}, \end{matrix} and \\ Y & := {\begin{matrix} \prod_{i = 1}^{N} [a_{i}, b_{i}], \  if L_{n} = A_{n}, \\ R^{N}, \  if L_{n} = B_{n}, C_{n}, D_{n} . \end{matrix} \end{aligned}

We give the condensed

Theorem 11

Let $f \in Ω$ , $0 < β < 1$ , $x \in Y;$ $n,$ $N \in N$ with $n^{1 - β} > 2$ . Then

(i)

{‖ L_{n} (f, x) - f (x) ‖}_{γ} \leq c_{N} [ω_{1} (f, φ (n)) + \frac{4 {‖ {‖ f ‖}_{γ} ‖}_{\infty}}{π^{2} (n^{1 - β} - 2)}] =: τ (n),

124

where $ω_{1}$ is for $p = \infty,$

and

(ii)

{‖ {‖ L_{n} (f) - f ‖}_{γ} ‖}_{\infty} \leq τ (n) \to 0, as n \to \infty .

125

For $f$ uniformly continuous and in $Ω$ we obtain

lim_{n \to \infty} L_{n} (f) = f,

pointwise and uniformly.

Proof â–¼

By ??.

Proof â–¼

Next we do iterated neural network approximation (see also [ 9 ] ).

We make

Remark 5

Let $r \in N$ and $L_{n}$ as above. We observe that

\begin{aligned} L_{n}^{r} f - f & = (L_{n}^{r} f - L_{n}^{r - 1} f) + (L_{n}^{r - 1} f - L_{n}^{r - 2} f) \\ + (L_{n}^{r - 2} f - L_{n}^{r - 3} f) + . . . + (L_{n}^{2} f - L_{n} f) + (L_{n} f - f) . \end{aligned}

Then

\begin{aligned} {‖ {‖ L_{n}^{r} f - f ‖}_{γ} ‖}_{\infty} \leq \\ \leq {‖ {‖ L_{n}^{r} f - L_{n}^{r - 1} f ‖}_{γ} ‖}_{\infty} + {‖ {‖ L_{n}^{r - 1} f - L_{n}^{r - 2} f ‖}_{γ} ‖}_{\infty} \\ + {‖ {‖ L_{n}^{r - 2} f - L_{n}^{r - 3} f ‖}_{γ} ‖}_{\infty} + . . . + {‖ {‖ L_{n}^{2} f - L_{n} f ‖}_{γ} ‖}_{\infty} + {‖ {‖ L_{n} f - f ‖}_{γ} ‖}_{\infty} \\ = {‖ {‖ L_{n}^{r - 1} (L_{n} f - f) ‖}_{γ} ‖}_{\infty} + {‖ {‖ L_{n}^{r - 2} (L_{n} f - f) ‖}_{γ} ‖}_{\infty} + {‖ {‖ L_{n}^{r - 3} (L_{n} f - f) ‖}_{γ} ‖}_{\infty} \\ + . . . + {‖ {‖ L_{n} (L_{n} f - f) ‖}_{γ} ‖}_{\infty} + {‖ {‖ L_{n} f - f ‖}_{γ} ‖}_{\infty} \leq r {‖ {‖ L_{n} f - f ‖}_{γ} ‖}_{\infty} . \end{aligned}

That is

{‖ {‖ L_{n}^{r} f - f ‖}_{γ} ‖}_{\infty} \leq r {‖ {‖ L_{n} f - f ‖}_{γ} ‖}_{\infty} .

127

We give

Theorem 12

All here as in theorem 11 and $r \in N$ , $τ (n)$ as in (124). Then

{‖ {‖ L_{n}^{r} f - f ‖}_{γ} ‖}_{\infty} \leq r τ (n) .

128

So that the speed of convergence to the unit operator of $L_{n}^{r}$ is not worse than of $L_{n} .$

Proof â–¼

By (127) and (125).

Proof â–¼

We make

Remark 6

Let $m_{1}, . . ., m_{r} \in N : m_{1} \leq m_{2} \leq . . . \leq m_{r}$ , $0 < β < 1$ , $f \in Ω$ . Then $φ (m_{1}) \geq φ (m_{2}) \geq . . . \geq φ (m_{r})$ , $φ$ as in (121).

Therefore

ω_{1} (f, φ (m_{1})) \geq ω_{1} (f, φ (m_{2})) \geq . . . \geq ω_{1} (f, φ (m_{r})) .

129

Assume further that $m_{i}^{1 - β} > 2$ , $i = 1, . . ., r$ . Then

\frac{2}{π^{2} (m_{1}^{1 - β} - 2)} \geq \frac{2}{π^{2} (m_{2}^{1 - β} - 2)} \geq . . . \geq \frac{2}{π^{2} (m_{r}^{1 - β} - 2)} .

130

Let $L_{m_{i}}$ as above, $i = 1, . . ., r,$ all of the same kind.

We write

\begin{aligned} L_{m_{r}} (L_{m_{r - 1}} (. . . L_{m_{2}} (L_{m_{1}} f))) - f = \\ = L_{m_{r}} (L_{m_{r - 1}} (. . . L_{m_{2}} (L_{m_{1}} f))) - L_{m_{r}} (L_{m_{r - 1}} (. . . L_{m_{2}} f)) \\ + L_{m_{r}} (L_{m_{r - 1}} (. . . L_{m_{2}} f)) - L_{m_{r}} (L_{m_{r - 1}} (. . . L_{m_{3}} f)) \\ + L_{m_{r}} (L_{m_{r - 1}} (. . . L_{m_{3}} f)) - L_{m_{r}} (L_{m_{r - 1}} (. . . L_{m_{4}} f)) + . . . \\ + L_{m_{r}} (L_{m_{r - 1}} f) - L_{m_{r}} f + L_{m_{r}} f - f \\ = L_{m_{r}} (L_{m_{r - 1}} (. . . L_{m_{2}})) (L_{m_{1}} f - f) + L_{m_{r}} (L_{m_{r - 1}} (. . . L_{m_{3}})) (L_{m_{2}} f - f) \\ + L_{m_{r}} (L_{m_{r - 1}} (. . . L_{m_{4}})) (L_{m_{3}} f - f) + . . . + L_{m_{r}} (L_{m_{r - 1}} f - f) + L_{m_{r}} f - f . \end{aligned}

Hence by the triangle inequality property of ${‖ {‖ \cdot ‖}_{γ} ‖}_{\infty}$ we get

\begin{aligned} {‖ {‖ L_{m_{r}} (L_{m_{r - 1}} (. . . L_{m_{2}} (L_{m_{1}} f))) - f ‖}_{γ} ‖}_{\infty} \leq \\ \leq {‖ {‖ L_{m_{r}} (L_{m_{r - 1}} (. . . L_{m_{2}})) (L_{m_{1}} f - f) ‖}_{γ} ‖}_{\infty} \\ + {‖ {‖ L_{m_{r}} (L_{m_{r - 1}} (. . . L_{m_{3}})) (L_{m_{2}} f - f) ‖}_{γ} ‖}_{\infty} \\ + {‖ {‖ L_{m_{r}} (L_{m_{r - 1}} (. . . L_{m_{4}})) (L_{m_{3}} f - f) ‖}_{γ} ‖}_{\infty} + . . . \\ + {‖ {‖ L_{m_{r}} (L_{m_{r - 1}} f - f) ‖}_{γ} ‖}_{\infty} + {‖ {‖ L_{m_{r}} f - f ‖}_{γ} ‖}_{\infty} \end{aligned}

(repeatedly applying (117))

\begin{aligned} \leq {‖ {‖ L_{m_{1}} f - f ‖}_{γ} ‖}_{\infty} + {‖ {‖ L_{m_{2}} f - f ‖}_{γ} ‖}_{\infty} + {‖ {‖ L_{m_{3}} f - f ‖}_{γ} ‖}_{\infty} + . . . \\ + {‖ {‖ L_{m_{r - 1}} f - f ‖}_{γ} ‖}_{\infty} + {‖ {‖ L_{m_{r}} f - f ‖}_{γ} ‖}_{\infty} \\ = \sum_{i = 1}^{r} {‖ {‖ L_{m_{i}} f - f ‖}_{γ} ‖}_{\infty} . \end{aligned}

That is, we proved

{‖ {‖ L_{m_{r}} (L_{m_{r - 1}} (. . . L_{m_{2}} (L_{m_{1}} f))) - f ‖}_{γ} ‖}_{\infty} \leq \sum_{i = 1}^{r} {‖ {‖ L_{m_{i}} f - f ‖}_{γ} ‖}_{\infty} .

134

We give

Theorem 13

Let $f \in Ω$ ; $N,$ $m_{1}, m_{2}, . . ., m_{r} \in N : m_{1} \leq m_{2} \leq . . . \leq m_{r},$ $0 < β < 1;$ $m_{i}^{1 - β} > 2$ , $i = 1, . . ., r,$ $x \in Y,$ and let $(L_{m_{1}}, . . ., L_{m_{r}})$ as $(A_{m_{1}}, . . ., A_{m_{r}})$ or $(B_{m_{1}}, . . ., B_{m_{r}})$ or $(C_{m_{1}}, . . ., C_{m_{r}})$ or $(D_{m_{1}}, . . ., D_{m_{r}})$ , $p = \infty .$ Then