General multivariate arctangent function activated neural network approximations
George A. Anastassiou\(^\ast \)
received: March 22, 2022; accepted: May 28, 2022; published online: August 25, 2022.
Here we expose multivariate quantitative approximations of Banach space valued continuous multivariate functions on a box or \(\mathbb {R}^{N},\) \(N\in \mathbb {N}\), by the multivariate normalized, quasi-interpolation, Kantorovich type and quadrature type neural network operators. We treat also the case of approximation by iterated operators of the last four types. These approximations are derived by establishing multidimensional Jackson type inequalities involving the multivariate modulus of continuity of the engaged function or its high order Fréchet derivatives. Our multivariate operators are defined by using a multidimensional density function induced by the arctangent function. The approximations are pointwise and uniform. The related feed-forward neural network is with one hidden layer.
MSC. 41A17, 41A25, 41A30, 41A36.
Keywords. arctangent function, multivariate neural network approximation, quasi-interpolation operator, Kantorovich type operator, quadrature type operator, multivariate modulus of continuity, abstract approximation, iterated approximation.
\(^\ast \)Department of Mathematical Sciences, University of Memphis, Memphis, TN 38152, U.S.A., e-mail: ganastss@memphis.edu.
1 Introduction
The author in
[
2
]
and
[
3
]
, see chapters 2–5, was the first to establish neural network approximations to continuous functions with rates by very specifically defined neural network operators of Cardaliagnet-Euvrard and “Squashing” types, by employing the modulus of continuity of the engaged function or its high order derivative, and producing very tight Jackson type inequalities. He treats there both the univariate and multivariate cases. The defining these operators “bell-shaped” and “squashing” functions are assumed to be of compact support. Also in
[
3
]
he gives the \(N\)th order asymptotic expansion for the error of weak approximation of these two operators to a special natural class of smooth functions, see chapters 4–5 there.
For this article the author is motivated by the article
[
13
]
of Z. Chen and F. Cao, also by
[
4
]
,
[
5
]
,
[
6
]
,
[
7
]
,
[
8
]
,
[
9
]
,
[
10
]
,
[
11
]
,
[
14
]
,
[
15
]
.
The author here performs multivariate arctangent function based neural network approximations to continuous functions over boxes or over the whole \(\mathbb {R}^{N}\), \(N\in \mathbb {N}\). Also he does iterated approximation. All convergences here are with rates expressed via the multivariate modulus of continuity of the involved function or its high order Fréchet derivative and given by very tight multidimensional Jackson type inequalities.
The author here comes up with the “right” precisely defined multivariate normalized, quasi-interpolation neural network operators related to boxes or \(\mathbb {R}^{N}\), as well as Kantorovich type and quadrature type related operators on \(\mathbb {R}^{N}\). Our boxes are not necessarily symmetric to the origin. In preparation to prove our results we establish important properties of the basic multivariate density function induced by arctangent function and defining our operators.
Feed-forward neural networks (FNNs) with one hidden layer, the only type of networks we deal with in this chapter, are mathematically expressed as
\begin{equation*} N_{n}\left( x\right) =\sum _{j=0}^{n}c_{j}\sigma \left( \left\langle a_{j}\cdot x\right\rangle +b_{j}\right) ,\text{ \ \ \ }x\in \mathbb {R}^{s}\text{, \ \ }s\in \mathbb {N}\text{,} \end{equation*}
where for \(0\leq j\leq n\), \(b_{j}\in \mathbb {R}\) are the thresholds, \(a_{j}\in \mathbb {R}^{s}\) are the connection weights, \(c_{j}\in \mathbb {R}\) are the coefficients, \(\left\langle a_{j}\cdot x\right\rangle \) is the inner product of \(a_{j}\) and \(x\), and \(\sigma \) is the activation function of the network. In many fundamental network models, the activation function is the arctangent function. About neural networks read
[
16
]
,
[
17
]
,
[
18
]
.
2 Auxiliary Notions
We consider the
\begin{equation} \arctan x=\int _{0}^{x}\tfrac {dz}{1+z^{2}},\text{ \ }x\in \mathbb {R}. \label{1} \end{equation}
1
We will be using
\begin{equation} h\left( x\right) :=\tfrac {2}{\pi }\arctan \left( \tfrac {\pi }{2}x\right) =\tfrac {2}{\pi }\int _{0}^{\frac{\pi x}{2}}\tfrac {dz}{1+z^{2}}\text{, \ }x\in \mathbb {R}, \label{2} \end{equation}
2
which is a sigmoid type function and it is strictly increasing. We have that
\begin{equation*} h\left( 0\right) =0\text{, \ }h\left( -x\right) =-h\left( x\right) \text{, \ }h\left( +\infty \right) =1\text{, \ }h\left( -\infty \right) =-1, \end{equation*}
and
\begin{equation} h^{\prime }\left( x\right) =\tfrac {4}{4+\pi ^{2}x^{2}}>0\text{, \ all }x\in \mathbb {R}. \label{3.} \end{equation}
3
We consider the activation function
\begin{equation} \psi \left( x\right) :=\tfrac {1}{4}\left( h\left( x+1\right) -h\left( x-1\right) \right) \text{, \ }x\in \mathbb {R}, \label{4} \end{equation}
4
and we notice that
\begin{equation} \psi \left( -x\right) =\psi \left( x\right) , \label{5} \end{equation}
5
it is an even function.
Since \(x+1{\gt}x-1\), then \(h\left( x+1\right) {\gt}h\left( x-1\right) \), and \(\psi \left( x\right) {\gt}0\), all \(x\in \mathbb {R}\).
We see that
\begin{equation} \psi \left( 0\right) =\tfrac {1}{\pi }\arctan \tfrac {\pi }{2}\cong 0.319. \label{6} \end{equation}
6
Let \(x{\gt}0\), we have that
\begin{equation} \psi ^{\prime }\left( x\right) =\tfrac {1}{4}\left( h^{\prime }\left( x+1\right) -h^{\prime }\left( x-1\right) \right) =\tfrac {-4\pi ^{2}x}{\left( 4+\pi ^{2}\left( x+1\right) ^{2}\right) \left( 4+\pi ^{2}\left( x-1\right) ^{2}\right) }<0. \label{7} \end{equation}
7
That is
\begin{equation} \psi ^{\prime }\left( x\right) <0\text{, for }x>0. \label{8} \end{equation}
8
That is \(\psi \) is strictly decreasing on \([0,\infty )\) and clearly is strictly increasing on \((-\infty ,0]\), and \(\psi ^{\prime }\left( 0\right) =0.\)
Observe that
\begin{equation} \begin{array}{l} \underset {x\rightarrow +\infty }{\lim }\psi \left( x\right) =\frac{1}{4}\left( h\left( +\infty \right) -h\left( +\infty \right) \right) =0, \\ \text{and} \\ \underset {x\rightarrow -\infty }{\lim }\psi \left( x\right) =\frac{1}{4}\left( h\left( -\infty \right) -h\left( -\infty \right) \right) =0.\end{array} \label{9} \end{equation}
9
That is the \(x\)-axis is the horizontal asymptote on \(\psi \).
All in all, \(\psi \) is a bell symmetric function with maximum \(\psi \left( 0\right) \cong 0.319.\)
We need
Theorem
1
[
11
,
p.
286
]
We have that
\begin{equation} \sum _{i=-\infty }^{\infty }\psi \left( x-i\right) =1\text{, \ }\forall \text{ }x\in \mathbb {R}. \label{10} \end{equation}
10
Theorem
2
[
11
,
p.
287
]
It holds
\begin{equation} \int _{-\infty }^{\infty }\psi \left( x\right) dx=1. \label{11} \end{equation}
11
So that \(\psi \left( x\right) \) is a density function on \(\mathbb {R}.\)
We mention
Theorem
3
[
11
,
p.
288
]
Let \(0{\lt}\alpha {\lt}1\), and \(n\in \mathbb {N}\) with \(n^{1-\alpha }{\gt}2\). It holds
\begin{equation} \sum _{\substack { k=-\infty \\ \left\vert nx-k\right\vert \geq n^{1-\alpha } }}^{\infty }\psi \left( nx-k\right) <\tfrac {2}{\pi ^{2}\left( n^{1-\alpha }-2\right) }. \label{12} \end{equation}
12
Denote by \(\left\lfloor \cdot \right\rfloor \) the integral part of the number and by \(\left\lceil \cdot \right\rceil \) the ceiling of the number.
We need
Theorem
4
[
11
,
p.
289
]
Let \(x\in \left[ a,b\right] \subset \mathbb {R}\) and \(n\in \mathbb {N}\) so that \(\left\lceil na\right\rceil \leq \left\lfloor nb\right\rfloor \). It holds
\begin{equation} \tfrac {1}{\sum _{k=\left\lceil na\right\rceil }^{\left\lfloor nb\right\rfloor }\psi \left( nx-k\right) }<\tfrac {1}{\psi \left( 1\right) }\cong 4.9737,\text{ \ }\forall \text{ }x\in \left[ a,b\right] . \label{13} \end{equation}
13
Note
1
[
11
,
pp.
290–291
]
i) We have that
\begin{equation} \underset {n\rightarrow \infty }{\lim }\sum _{k=\left\lceil na\right\rceil }^{\left\lfloor nb\right\rfloor }\psi \left( nx-k\right) \neq 1, \label{14} \end{equation}
14
for at least some \(x\in \left[ a,b\right] .\)
ii) For large enough \(n\in \mathbb {N}\) we always obtain \(\left\lceil na\right\rceil \leq \left\lfloor nb\right\rfloor \). Also \(a\leq \frac{k}{n}\leq b\), iff \(\left\lceil na\right\rceil \leq k\leq \left\lfloor nb\right\rfloor \).
In general, by theorem 1, it holds
\begin{equation} \sum _{k=\left\lceil na\right\rceil }^{\left\lfloor nb\right\rfloor }\psi \left( nx-k\right) \leq 1. \label{15} \end{equation}
15
We introduce
\begin{equation} Z\left( x_{1},...,x_{N}\right) :=Z\left( x\right) :=\prod _{i=1}^{N}\psi \left( x_{i}\right) \text{, \ \ }x=\left( x_{1},...,x_{N}\right) \in \mathbb {R}^{N},\text{ }N\in \mathbb {N}. \label{16} \end{equation}
16
It has the properties:
(i) \(Z\left( x\right) {\gt}0\), \(\forall \) \(x\in \mathbb {R}^{N},\)
(ii)
\begin{equation} \sum _{k=-\infty }^{\infty }Z\left( x-k\right) :=\sum _{k_{1}=-\infty }^{\infty }\sum _{k_{2}=-\infty }^{\infty }...\sum _{k_{N}=-\infty }^{\infty }Z\left( x_{1}-k_{1},...,x_{N}-k_{N}\right) =1,\text{ } \label{17} \end{equation}
17
where \(k:=\left( k_{1},...,k_{n}\right) \in \mathbb {Z}^{N}\), \(\forall \) \(x\in \mathbb {R}^{N},\)
hence
(iii)
\begin{equation} \sum _{k=-\infty }^{\infty }Z\left( nx-k\right) =1, \label{18} \end{equation}
18
\(\forall \) \(x\in \mathbb {R}^{N};\) \(n\in \mathbb {N}\),
and
(iv)
\begin{equation} \int _{\mathbb {R}^{N}}Z\left( x\right) dx=1, \label{19} \end{equation}
19
that is \(Z\) is a multivariate density function.
Here denote \(\left\Vert x\right\Vert _{\infty }:=\max \left\{ \left\vert x_{1}\right\vert ,...,\left\vert x_{N}\right\vert \right\} \), \(x\in \mathbb {R}^{N}\), also set \(\infty :=\left( \infty ,...,\infty \right) \), \(-\infty :=\left( -\infty ,...,-\infty \right) \) upon the multivariate context, and
\begin{equation} \begin{array}{c} \left\lceil na\right\rceil :=\left( \left\lceil na_{1}\right\rceil ,...,\left\lceil na_{N}\right\rceil \right) , \\ \left\lfloor nb\right\rfloor :=\left( \left\lfloor nb_{1}\right\rfloor ,...,\left\lfloor nb_{N}\right\rfloor \right) ,\end{array} \label{20.} \end{equation}
20
where \(a:=\left( a_{1},...,a_{N}\right) \), \(b:=\left( b_{1},...,b_{N}\right) .\)
We obviously see that
\begin{align} \sum _{k=\left\lceil na\right\rceil }^{\left\lfloor nb\right\rfloor }Z\left( nx-k\right) & =\sum _{k=\left\lceil na\right\rceil }^{\left\lfloor nb\right\rfloor }\left( \prod _{i=1}^{N}\psi \left( nx_{i}-k_{i}\right) \right) \label{21} \\ & =\sum _{k_{1}=\left\lceil na_{1}\right\rceil }^{\left\lfloor nb_{1}\right\rfloor }...\sum _{k_{N}=\left\lceil na_{N}\right\rceil }^{\left\lfloor nb_{N}\right\rfloor }\left( \prod _{i=1}^{N}\psi \left( nx_{i}-k_{i}\right) \right) \notag \\ & =\prod _{i=1}^{N}\left( \sum _{k_{i}=\left\lceil na_{i}\right\rceil }^{\left\lfloor nb_{i}\right\rfloor }\psi \left( nx_{i}-k_{i}\right) \right) . \notag \end{align}
For \(0{\lt}\beta {\lt}1\) and \(n\in \mathbb {N}\), a fixed \(x\in \mathbb {R}^{N}\), we have that
\begin{equation} \sum _{k=\left\lceil na\right\rceil }^{\left\lfloor nb\right\rfloor }\psi \left( nx-k\right) =\sum _{\substack { k=\left\lceil na\right\rceil \\ \left\Vert \frac{k}{n}-x\right\Vert _{\infty }\leq \frac{1}{n^{\beta }}}}^{\left\lfloor nb\right\rfloor }\psi \left( nx-k\right) +\sum _{\substack { k=\left\lceil na\right\rceil \\ \left\Vert \frac{k}{n}-x\right\Vert _{\infty }>\frac{1}{n^{\beta }}}}^{\left\lfloor nb\right\rfloor }\psi \left( nx-k\right) . \label{22} \end{equation}
22
In the last two sums the counting is over disjoint vector sets of \(k\)’s, because the condition \(\left\Vert \frac{k}{n}-x\right\Vert _{\infty }{\gt}\frac{1}{n^{\beta }}\) implies that there exists at least one \(\left\vert \frac{k_{r}}{n}-x_{r}\right\vert {\gt}\frac{1}{n^{\beta }}\), where \(r\in \left\{ 1,...,N\right\} .\)
(v) As in
[
10
,
pp.
379–380
]
, we derive that
\begin{equation} \sum _{\substack { k=\left\lceil na\right\rceil \\ \left\Vert \frac{k}{n}-x\right\Vert _{\infty }>\frac{1}{n^{\beta }}}}^{\left\lfloor nb\right\rfloor }Z\left( nx-k\right) \overset {\text{(\ref{12})}}{<}\tfrac {2}{\pi ^{2}\left( n^{1-\beta }-2\right) }\text{, \ }0<\beta <1, \label{23} \end{equation}
23
with \(n\in \mathbb {N}:n^{1-\beta }{\gt}2\), \(x\in \prod _{i=1}^{N}\left[ a_{i},b_{i}\right] .\)
(vi) By theorem 4 we get that
\begin{equation} 0<\tfrac {1}{\sum _{k=\left\lceil na\right\rceil }^{\left\lfloor nb\right\rfloor }Z\left( nx-k\right) }<\tfrac {1}{\left( \psi \left( 1\right) \right) ^{N}}\cong \left( 4.9737\right) ^{N}, \label{24} \end{equation}
24
\(\forall \) \(x\in \left( \prod _{i=1}^{N}\left[ a_{i},b_{i}\right] \right) \), \(n\in \mathbb {N}\).
It is also clear that
(vii)
\begin{equation} \sum _{\substack { k=-\infty \\ \Vert \tfrac {k}{n}-x \Vert _{\infty }>\frac{1}{n^{\beta }}}}^{\infty }Z\left( nx-k\right) <\tfrac {2}{\pi ^{2}\left( n^{1-\beta }-2\right) }, \label{25} \end{equation}
25
\(0{\lt}\beta {\lt}1\), \(n\in \mathbb {N}:n^{1-\beta }{\gt}2\), \(x\in \mathbb {R}^{N}.\)
Furthermore it holds
\begin{equation} \underset {n\rightarrow \infty }{\lim }\sum _{k=\left\lceil na\right\rceil }^{\left\lfloor nb\right\rfloor }Z\left( nx-k\right) \neq 1, \label{26} \end{equation}
26
for at least some \(x\in \left( \prod _{i=1}^{N}\left[ a_{i},b_{i}\right] \right) .\)
Here \(\left( X,\left\Vert \cdot \right\Vert _{\gamma }\right) \) is a Banach space.
Let \(f\in C\left( \prod _{i=1}^{N}\left[ a_{i},b_{i}\right] ,X\right) ,\) \(x=\left( x_{1},...,x_{N}\right) \in \prod _{i=1}^{N}\left[ a_{i},b_{i}\right] ,\) \(n\in \mathbb {N}\) such that \(\left\lceil na_{i}\right\rceil \leq \left\lfloor nb_{i}\right\rfloor \), \(i=1,...,N.\)
We introduce and define the following multivariate linear normalized neural network operator (\(x:=\left( x_{1},...,x_{N}\right) \in \left( \prod _{i=1}^{N}\left[ a_{i},b_{i}\right] \right) \)):
\begin{align} & A_{n}\left( f,x_{1},...,x_{N}\right) := \label{27} \\ & :=A_{n}\left( f,x\right) \notag \\ & :=\tfrac {\sum \limits _{k=\left\lceil na\right\rceil }^{\left\lfloor nb\right\rfloor }f\left( \tfrac {k}{n}\right) Z\left( nx-k\right) }{\sum \limits _{k=\left\lceil na\right\rceil }^{\left\lfloor nb\right\rfloor }Z\left( nx-k\right) } \notag \\ & =\tfrac {\sum \limits _{k_{1}=\left\lceil na_{1}\right\rceil }^{\left\lfloor nb_{1}\right\rfloor }\sum \limits _{k_{2}=\left\lceil na_{2}\right\rceil }^{\left\lfloor nb_{2}\right\rfloor }...\sum \limits _{k_{N}=\left\lceil na_{N}\right\rceil }^{\left\lfloor nb_{N}\right\rfloor }f\left( \tfrac {k_{1}}{n},...,\tfrac {k_{N}}{n}\right) \left( \prod \limits _{i=1}^{N}\psi \left( nx_{i}-k_{i}\right) \right) }{\prod \limits _{i=1}^{N}\left( \sum \limits _{k_{i}=\left\lceil na_{i}\right\rceil }^{\left\lfloor nb_{i}\right\rfloor }\psi \left( nx_{i}-k_{i}\right) \right) }. \notag \end{align}
For large enough \(n\in \mathbb {N}\) we always obtain \(\left\lceil na_{i}\right\rceil \leq \left\lfloor nb_{i}\right\rfloor \), \(i=1,...,N\). Also \(a_{i}\leq \frac{k_{i}}{n}\leq b_{i}\), iff \(\left\lceil na_{i}\right\rceil \leq k_{i}\leq \left\lfloor nb_{i}\right\rfloor \), \(i=1,...,N\).
When \(g\in C\left( \prod _{i=1}^{N}\left[ a_{i},b_{i}\right] \right) \) we define the companion operator
\begin{equation} \widetilde{A}_{n}\left( g,x\right) :=\tfrac {\sum \limits _{k=\left\lceil na\right\rceil }^{\left\lfloor nb\right\rfloor }g\left( \frac{k}{n}\right) Z\left( nx-k\right) }{\sum \limits _{k=\left\lceil na\right\rceil }^{\left\lfloor nb\right\rfloor }Z\left( nx-k\right) }. \label{28} \end{equation}
28
Clearly \(\widetilde{A}_{n}\) is a positive linear operator. We have that
\begin{equation*} \widetilde{A}_{n}\left( 1,x\right) =1\text{, \ }\forall \text{ }x\in \left( \prod _{i=1}^{N}\left[ a_{i},b_{i}\right] \right) . \end{equation*}
Notice that \(A_{n}\left( f\right) \in C\left( \prod _{i=1}^{N}\left[ a_{i},b_{i}\right] ,X\right) \) and \(\widetilde{A}_{n}\left( g\right) \in C\left( \prod _{i=1}^{N}\left[ a_{i},b_{i}\right] \right) .\)
Furthermore it holds
\begin{equation} \left\Vert A_{n}\left( f,x\right) \right\Vert _{\gamma }\leq \tfrac {\sum \limits _{k=\left\lceil na\right\rceil }^{\left\lfloor nb\right\rfloor }\left\Vert f\left( \frac{k}{n}\right) \right\Vert _{\gamma }Z\left( nx-k\right) }{\sum \limits _{k=\left\lceil na\right\rceil }^{\left\lfloor nb\right\rfloor }Z\left( nx-k\right) }=\widetilde{A}_{n}\left( \left\Vert f\right\Vert _{\gamma },x\right) , \label{29} \end{equation}
29
\(\forall \) \(x\in \prod _{i=1}^{N}\left[ a_{i},b_{i}\right] .\)
Clearly \(\left\Vert f\right\Vert _{\gamma }\in C\left( \prod _{i=1}^{N}\left[ a_{i},b_{i}\right] \right) .\)
So, we have that
\begin{equation} \left\Vert A_{n}\left( f,x\right) \right\Vert _{\gamma }\leq \widetilde{A}_{n}\left( \left\Vert f\right\Vert _{\gamma },x\right) , \label{30} \end{equation}
30
\(\forall \) \(x\in \prod _{i=1}^{N}\left[ a_{i},b_{i}\right] \), \(\forall \) \(n\in \mathbb {N}\), \(\forall \) \(f\in C\left( \prod _{i=1}^{N}\left[ a_{i},b_{i}\right] ,X\right) .\)
Let \(c\in X\) and \(g\in C\left( \prod _{i=1}^{N}\left[ a_{i},b_{i}\right] \right) \), then \(cg\in C\left( \prod _{i=1}^{N}\left[ a_{i},b_{i}\right] ,X\right) .\)
Furthermore it holds
\begin{equation} A_{n}\left( cg,x\right) =c\widetilde{A}_{n}\left( g,x\right) \text{, \ }\forall \text{ }x\in \prod _{i=1}^{N}\left[ a_{i},b_{i}\right] . \label{31} \end{equation}
31
Since \(\widetilde{A}_{n}\left( 1\right) =1\), we get that
\begin{equation} A_{n}\left( c\right) =c\text{, \ }\forall \text{ }c\in X\text{.} \label{32} \end{equation}
32
We call \(\widetilde{A}_{n}\) the companion operator of \(A_{n}\).
For convinience we call
\begin{align} & A_{n}^{\ast }\left( f,x\right) :=\sum _{k=\left\lceil na\right\rceil }^{\left\lfloor nb\right\rfloor }f\left( \tfrac {k}{n}\right) Z\left( nx-k\right) = \label{33} \\ & =\sum _{k_{1}=\left\lceil na_{1}\right\rceil }^{\left\lfloor nb_{1}\right\rfloor }\sum _{k_{2}=\left\lceil na_{2}\right\rceil }^{\left\lfloor nb_{2}\right\rfloor }...\sum _{k_{N}=\left\lceil na_{N}\right\rceil }^{\left\lfloor nb_{N}\right\rfloor }f\left( \tfrac {k_{1}}{n},...,\tfrac {k_{N}}{n}\right) \left( \prod _{i=1}^{N}\psi \left( nx_{i}-k_{i}\right) \right) , \notag \end{align}
\(\forall \) \(x\in \left( \prod _{i=1}^{N}\left[ a_{i},b_{i}\right] \right) .\)
That is
\begin{equation} A_{n}\left( f,x\right) :=\tfrac {A_{n}^{\ast }\left( f,x\right) }{\sum \limits _{k=\left\lceil na\right\rceil }^{\left\lfloor nb\right\rfloor }Z\left( nx-k\right) }, \label{34} \end{equation}
34
\(\forall \) \(x\in \left( \prod _{i=1}^{N}\left[ a_{i},b_{i}\right] \right) \), \(n\in \mathbb {N}.\)
Hence
\begin{equation} A_{n}\left( f,x\right) -f\left( x\right) =\tfrac {A_{n}^{\ast }\left( f,x\right) -f\left( x\right) \left( \sum \limits _{k=\left\lceil na\right\rceil }^{\left\lfloor nb\right\rfloor }Z\left( nx-k\right) \right) }{\sum \limits _{k=\left\lceil na\right\rceil }^{\left\lfloor nb\right\rfloor }Z\left( nx-k\right) }. \label{35} \end{equation}
35
Consequently we derive
\begin{equation} \left\Vert A_{n}\left( f,x\right) -f\left( x\right) \right\Vert _{\gamma }\overset {\text{(\ref{24})}}{\leq }\left( 4.9737\right) ^{N}\left\Vert A_{n}^{\ast }\left( f,x\right) -f\left( x\right) \sum _{k=\left\lceil na\right\rceil }^{\left\lfloor nb\right\rfloor }Z\left( nx-k\right) \right\Vert _{\gamma }, \label{36} \end{equation}
36
\(\forall \) \(x\in \left( \prod _{i=1}^{N}\left[ a_{i},b_{i}\right] \right) .\)
We will estimate the right hand side of (36).
For the last and others we need
Definition
1
[
11
,
p.
274
]
Let \(M\) be a convex and compact subset of \(\left( \mathbb {R}^{N},\left\Vert \cdot \right\Vert _{p}\right) \), \(p\in \left[ 1,\infty \right] \), and \(\left( X,\left\Vert \cdot \right\Vert _{\gamma }\right) \) be a Banach space. Let \(f\in C\left( M,X\right) .\) We define the first modulus of continuity of \(f\) as
\begin{equation} \omega _{1}\left( f,\delta \right) :=\underset {\begin{array}{c} x,y\in M \\ \left\Vert x-y\right\Vert _{p}\leq \delta \end{array}}{\sup }\left\Vert f\left( x\right) -f\left( y\right) \right\Vert _{\gamma }\text{, \ }0<\delta \leq diam\left( M\right) . \label{37.} \end{equation}
37
If \(\delta {\gt}diam\left( M\right) \), then
\begin{equation} \omega _{1}\left( f,\delta \right) =\omega _{1}\left( f,diam\left( M\right) \right) . \label{38.} \end{equation}
38
Notice \(\omega _{1}\left( f,\delta \right) \) is increasing in \(\delta {\gt}0\). For \(f\in C_{B}\left( M,X\right) \) (continuous and bounded functions) \(\omega _{1}\left( f,\delta \right) \) is defined similarly.
We have \(\omega _{1}\left( f,\delta \right) \rightarrow 0 \) as \(\delta \downarrow 0\), iff \(f\in C\left( M,X\right) \), where \(M\) is a convex compact subset of \(\left( \mathbb {R}^{N},\left\Vert \cdot \right\Vert _{p}\right) \), \(p\in \left[ 1,\infty \right] .\)
Clearly we have also: \(f\in C_{U}\left( \mathbb {R}^{N},X\right) \) (uniformly continuous functions), iff \(\omega _{1}\left( f,\delta \right) \rightarrow 0\) as \(\delta \downarrow 0\), where \(\omega _{1}\) is defined similarly to (37). The space \(C_{B}\left( \mathbb {R}^{N},X\right) \) denotes the continuous and bounded functions on \(\mathbb {R}^{N}.\)
When \(f\in C_{B}\left( \mathbb {R}^{N},X\right) \) we define,
\begin{align} B_{n}\left( f,x\right) & :=B_{n}\left( f,x_{1},...,x_{N}\right) \label{39.} \\ & :=\sum _{k=-\infty }^{\infty }f\left( \frac{k}{n}\right) Z\left( nx-k\right) \notag \\ & :=\sum _{k_{1}=-\infty }^{\infty }\sum _{k_{2}=-\infty }^{\infty }...\sum _{k_{N}=-\infty }^{\infty }f\left( \tfrac {k_{1}}{n},\tfrac {k_{2}}{n},...,\tfrac {k_{N}}{n}\right) \left( \prod _{i=1}^{N}\psi \left( nx_{i}-k_{i}\right) \right) , \notag \end{align}
\(n\in \mathbb {N}\), \(\forall \) \(x\in \mathbb {R}^{N},\) \(N\in \mathbb {N}\), the multivariate quasi-interpolation neural network operator.
Also for \(f\in C_{B}\left( \mathbb {R}^{N},X\right) \) we define the multivariate Kantorovich type neural network operator
\begin{align} & C_{n}\left( f,x\right) :=C_{n}\left( f,x_{1},...,x_{N}\right) \label{40.} \\ & :=\sum _{k=-\infty }^{\infty }\left( n^{N}\int _{\frac{k}{n}}^{\frac{k+1}{n}}f\left( t\right) dt\right) Z\left( nx-k\right) \notag \\ & =\sum _{k_{1}=-\infty }^{\infty }\sum _{k_{2}=-\infty }^{\infty }...\sum _{k_{N}=-\infty }^{\infty }\left( n^{N}\int _{\frac{k_{1}}{n}}^{\frac{k_{1}+1}{n}}\int _{\frac{k_{2}}{n}}^{\frac{k_{2}+1}{n}}...\int _{\frac{k_{N}}{n}}^{\frac{k_{N}+1}{n}}f\left( t_{1},...,t_{N}\right) dt_{1}...dt_{N}\right) \notag \\ & \cdot \left( \prod _{i=1}^{N}\psi \left( nx_{i}-k_{i}\right) \right) , \notag \end{align}
\(n\in \mathbb {N},\ \forall \) \(x\in \mathbb {R}^{N}.\)
Again for \(f\in C_{B}\left( \mathbb {R}^{N},X\right) ,\) \(N\in \mathbb {N},\) we define the multivariate neural network operator of quadrature type \(D_{n}\left( f,x\right) \), \(n\in \mathbb {N},\) as follows.
Let \(\theta =\left( \theta _{1},...,\theta _{N}\right) \in \mathbb {N}^{N},\) \(r=\left( r_{1},...,r_{N}\right) \in \mathbb {Z}_{+}^{N}\), \(w_{r}=w_{r_{1},r_{2},...r_{N}}\geq 0\), such that \(\sum \limits _{r=0}^{\theta }w_{r}=\sum \limits _{r_{1}=0}^{\theta _{1}}\sum \limits _{r_{2}=0}^{\theta _{2}}...\sum \limits _{r_{N}=0}^{\theta _{N}}w_{r_{1},r_{2},...r_{N}}=1;\) \(k\in \mathbb {Z}^{N}\) and
\begin{align} \delta _{nk}\left( f\right) & :=\delta _{n,k_{1},k_{2},...,k_{N}}\left( f\right) :=\sum \limits _{r=0}^{\theta }w_{r}f\left( \tfrac {k}{n}+\tfrac {r}{n\theta }\right) \label{41.} \\ & =\sum \limits _{r_{1}=0}^{\theta _{1}}\sum \limits _{r_{2}=0}^{\theta _{2}}...\sum \limits _{r_{N}=0}^{\theta _{N}}w_{r_{1},r_{2},...r_{N}}f\left( \tfrac {k_{1}}{n}+\tfrac {r_{1}}{n\theta _{1}},\tfrac {k_{2}}{n}+\tfrac {r_{2}}{n\theta _{2}},...,\tfrac {k_{N}}{n}+\tfrac {r_{N}}{n\theta _{N}}\right) , \notag \end{align}
where \(\tfrac {r}{\theta }:=\left( \tfrac {r_{1}}{\theta _{1}},\frac{r_{2}}{\theta _{2}},...,\frac{r_{N}}{\theta _{N}}\right) .\)
We set
\begin{align} D_{n}\left( f,x\right) & :=D_{n}\left( f,x_{1},...,x_{N}\right) :=\sum _{k=-\infty }^{\infty }\delta _{nk}\left( f\right) Z\left( nx-k\right) \label{42.} \\ & =\sum _{k_{1}=-\infty }^{\infty }\sum _{k_{2}=-\infty }^{\infty }...\sum _{k_{N}=-\infty }^{\infty }\delta _{n,k_{1},k_{2},...,k_{N}}\left( f\right) \left( \prod _{i=1}^{N}\psi \left( nx_{i}-k_{i}\right) \right) , \notag \end{align}
\(\forall \) \(x\in \mathbb {R}^{N}.\)
In this article we study the approximation properties of \(A_{n},B_{n},C_{n},\) \(D_{n}\) neural network operators and as well of their iterates. That is, the quantitative pointwise and uniform convergence of these operators to the unit operator \(I\).
3 Multivariate general Neural Network Approximations
Here we present several vectorial neural network approximations to Banach space valued functions given with rates.
We give
Theorem
5
Let \(f\in C\left( \prod _{i=1}^{N}\left[ a_{i},b_{i}\right] ,X\right) ,\) \(0{\lt}\beta {\lt}1\), \(x\in \left( \prod _{i=1}^{N}\left[ a_{i},b_{i}\right] \right) ,\) \(N,n\in \mathbb {N}\) with \(n^{1-\beta }{\gt}2\). Then
1)
\begin{equation} \left\Vert A_{n}\left( f,x\right) -f\left( x\right) \right\Vert _{\gamma }\leq \left( 4.9737\right) ^{N}\left[ \omega _{1}\left( f,\tfrac {1}{n^{\beta }}\right) +\tfrac {4\left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }}{\pi ^{2}\left( n^{1-\beta }-2\right) }\right] =:\lambda _{1}\left( n\right) , \label{43} \end{equation}
43
and
2)
\begin{equation} \left\Vert \left\Vert A_{n}\left( f\right) -f\right\Vert _{\gamma }\right\Vert _{\infty }\leq \lambda _{1}\left( n\right) . \label{44} \end{equation}
44
We notice that \(\underset {n\rightarrow \infty }{\lim }A_{n}\left( f\right) \overset {\left\Vert \cdot \right\Vert _{\gamma }}{=}f\), pointwise and uniformly.
Above \(\omega _{1}\) is with respect to \(p=\infty .\)
We observe that
\begin{align} \Delta \left( x\right) & :=A_{n}^{\ast }\left( f,x\right) -f\left( x\right) \sum _{k=\left\lceil na\right\rceil }^{\left\lfloor nb\right\rfloor }Z\left( nx-k\right) \label{45} \\ & =\sum _{k=\left\lceil na\right\rceil }^{\left\lfloor nb\right\rfloor }f\left( \tfrac {k}{n}\right) Z\left( nx-k\right) -\sum _{k=\left\lceil na\right\rceil }^{\left\lfloor nb\right\rfloor }f\left( x\right) Z\left( nx-k\right) \notag \\ & =\sum _{k=\left\lceil na\right\rceil }^{\left\lfloor nb\right\rfloor }\left( f\left( \tfrac {k}{n}\right) -f\left( x\right) \right) Z\left( nx-k\right) . \notag \end{align}
Thus
\begin{align} \left\Vert \Delta \left( x\right) \right\Vert _{\gamma } & \leq \sum _{k=\left\lceil na\right\rceil }^{\left\lfloor nb\right\rfloor }\left\Vert f\left( \tfrac {k}{n}\right) -f\left( x\right) \right\Vert _{\gamma }Z\left( nx-k\right) \label{46} \\ & =\sum _{\substack { k=\left\lceil na\right\rceil \\ \begin{bgroup} \left\Vert \frac{k}{n}-x\right\Vert _{\infty }\leq \frac{1}{n^{\beta }}
\end{bgroup}}}^{\left\lfloor nb\right\rfloor }\left\Vert f\left( \tfrac {k}{n}\right) -f\left( x\right) \right\Vert _{\gamma }Z\left( nx-k\right) \notag \\ & \quad +\sum _{\substack { k=\left\lceil na\right\rceil \\ \begin{bgroup} \left\Vert \frac{k}{n}-x\right\Vert _{\infty }{\gt}\frac{1}{n^{\beta }}
\end{bgroup}}}^{\left\lfloor nb\right\rfloor }\left\Vert f\left( \tfrac {k}{n}\right) -f\left( x\right) \right\Vert _{\gamma }Z\left( nx-k\right) \notag \\ & \overset {\text{(\ref{18})}}{\leq }\omega _{1}\left( f,\tfrac {1}{n^{\beta }}\right) +2\left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }\sum _{\substack { k=\left\lceil na\right\rceil \\ \begin{bgroup} \left\Vert \frac{k}{n}-x\right\Vert _{\infty }{\gt}\frac{1}{n^{\beta }}
\end{bgroup}}}^{\left\lfloor nb\right\rfloor }Z\left( nx-k\right) \notag \\ & \overset {\text{(\ref{23})}}{\leq }\omega _{1}\left( f,\tfrac {1}{n^{\beta }}\right) +\tfrac {4\left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }}{\pi ^{2}\left( n^{1-\beta }-2\right) }. \notag \end{align}
So that
\begin{equation} \left\Vert \Delta \left( x\right) \right\Vert _{\gamma }\leq \omega _{1}\left( f,\tfrac {1}{n^{\beta }}\right) +\tfrac {4\left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }}{\pi ^{2}\left( n^{1-\beta }-2\right) }. \label{47} \end{equation}
50
Now using (36) we finish the proof.
We make
Clearly (62) is valid when \(M=\prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] \) and \(\widetilde{L}_{n}=\widetilde{A}_{n}\), see (28).
All the above is preparation for the following theorem, where we assume Fréchet differentiability of functions.
This will be a direct application of Theorem 10.2,
[
11
,
pp.
268–270
]
. The operators \(A_{n},\) \(\widetilde{A}_{n}\) fulfill its assumptions, see (27), (28), (30), (31) and (32).
We present the following high order approximation results.
Theorem
6
Let \(O\) open subset of \(\left( \mathbb {R}^{N},\left\Vert \cdot \right\Vert _{p}\right) \), \(p\in \left[ 1,\infty \right] \), such that \(\prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] \subset O\subseteq \mathbb {R}^{N}\), and let \(\left( X,\left\Vert \cdot \right\Vert _{\gamma }\right) \) be a general Banach space. Let \(m\in \mathbb {N}\) and \(f\in C^{m}\left( O,X\right) \), the space of \(m\)-times continuously Fréchet differentiable functions from \(O\) into \(X\). We study the approximation of \(f|_{\prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] }.\) Let \(x_{0}\in \left( \prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] \right) \) and \(r{\gt}0\). Then
1)
\begin{align} & \left\Vert \left( A_{n}\left( f\right) \right) \left( x_{0}\right) -\sum _{j=0}^{m}\tfrac {1}{j!}\left( A_{n}\left( f^{\left( j\right) }\left( x_{0}\right) \left( \cdot -x_{0}\right) ^{j}\right) \right) \left( x_{0}\right) \right\Vert _{\gamma }\leq \label{60} \\ & \leq \tfrac {1}{rm!}\omega _{1}\left( f^{\left( m\right) },r\left( \left( \widetilde{A}_{n}\left( \left\Vert \cdot -x_{0}\right\Vert _{p}^{m+1}\right) \right) \left( x_{0}\right) \right) ^{\frac{1}{m+1}}\right) \notag \\ & \quad \cdot \left( \left( \widetilde{A}_{n}\left( \left\Vert \cdot -x_{0}\right\Vert _{p}^{m+1}\right) \right) \left( x_{0}\right) \right) ^{\left( \frac{m}{m+1}\right) }\left[ \tfrac {1}{\left( m+1\right) }+\tfrac {r}{2}+\tfrac {mr^{2}}{8}\right] , \notag \end{align}
2) additionally if \(f^{\left( j\right) }\left( x_{0}\right) =0\), \(j=1,...,m\), we have
\begin{align} & \left\Vert \left( A_{n}\left( f\right) \right) \left( x_{0}\right) -f\left( x_{0}\right) \right\Vert _{\gamma }\leq \label{61} \\ & \leq \tfrac {1}{rm!}\omega _{1}\left( f^{\left( m\right) },r\left( \left( \widetilde{A}_{n}\left( \left\Vert \cdot -x_{0}\right\Vert _{p}^{m+1}\right) \right) \left( x_{0}\right) \right) ^{\frac{1}{m+1}}\right) \notag \\ & \quad \cdot \left( \left( \widetilde{A}_{n}\left( \left\Vert \cdot -x_{0}\right\Vert _{p}^{m+1}\right) \right) \left( x_{0}\right) \right) ^{\left( \frac{m}{m+1}\right) }\left[ \tfrac {1}{\left( m+1\right) }+\tfrac {r}{2}+\tfrac {mr^{2}}{8}\right] , \notag \end{align}
3)
\begin{align} & \left\Vert \left( A_{n}\left( f\right) \right) \left( x_{0}\right) -f\left( x_{0}\right) \right\Vert _{\gamma }\leq \label{62} \\ & \leq \sum _{j=1}^{m}\tfrac {1}{j!}\left\Vert \left( A_{n}\left( f^{\left( j\right) }\left( x_{0}\right) \left( \cdot -x_{0}\right) ^{j}\right) \right) \left( x_{0}\right) \right\Vert _{\gamma } \notag \\ & \quad +\tfrac {1}{rm!}\omega _{1}\left( f^{\left( m\right) },r\left( \left( \widetilde{A}_{n}\left( \left\Vert \cdot -x_{0}\right\Vert _{p}^{m+1}\right) \right) \left( x_{0}\right) \right) ^{\frac{1}{m+1}}\right) \notag \\ & \quad \cdot \left( \left( \widetilde{A}_{n}\left( \left\Vert \cdot -x_{0}\right\Vert _{p}^{m+1}\right) \right) \left( x_{0}\right) \right) ^{\left( \frac{m}{m+1}\right) }\left[ \tfrac {1}{\left( m+1\right) }+\tfrac {r}{2}+\tfrac {mr^{2}}{8}\right] , \notag \end{align}
and
4)
\begin{align} & \left\Vert \left\Vert A_{n}\left( f\right) -f\right\Vert _{\gamma }\right\Vert _{\infty ,\prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] }\leq \label{63} \\ & \leq \sum _{j=1}^{m}\tfrac {1}{j!}\left\Vert \left\Vert \left( A_{n}\left( f^{\left( j\right) }\left( x_{0}\right) \left( \cdot -x_{0}\right) ^{j}\right) \right) \left( x_{0}\right) \right\Vert _{\gamma }\right\Vert _{\infty ,x_{0}\in \prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] } \notag \\ & \quad +\tfrac {1}{rm!}\omega _{1}\left( f^{\left( m\right) },r\left\Vert \left( \widetilde{A}_{n}\left( \left\Vert \cdot -x_{0}\right\Vert _{p}^{m+1}\right) \right) \left( x_{0}\right) \right\Vert _{\infty ,x_{0}\in \prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] }^{\frac{1}{m+1}}\right) \notag \\ & \quad \cdot \left\Vert \left( \widetilde{A}_{n}\left( \left\Vert \cdot -x_{0}\right\Vert _{p}^{m+1}\right) \right) \left( x_{0}\right) \right\Vert _{\infty ,x_{0}\in \prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] }^{\left( \frac{m}{m+1}\right) }\left[ \tfrac {1}{\left( m+1\right) }+\tfrac {r}{2}+\tfrac {mr^{2}}{8}\right] . \notag \end{align}
We need
Lemma
2
The function \(\left( \widetilde{A}_{n}\left( \left\Vert \cdot -x_{0}\right\Vert _{p}^{m}\right) \right) \left( x_{0}\right) \) is continuous in \(x_{0}\in \left( \prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] \right) \), \(m\in \mathbb {N}\).
By Lemma 10.3,
[
11
,
p.
272
]
.
We make
We give
Corollary
1
(to theorem 6, case of \(m=1\)) Then
1)
\begin{align} & \left\Vert \left( A_{n}\left( f\right) \right) \left( x_{0}\right) -f\left( x_{0}\right) \right\Vert _{\gamma }\leq \label{65} \\ & \leq \left\Vert \left( A_{n}\left( f^{\left( 1\right) }\left( x_{0}\right) \left( \cdot -x_{0}\right) \right) \right) \left( x_{0}\right) \right\Vert _{\gamma } \notag \\ & \quad +\tfrac {1}{2r}\omega _{1}\left( f^{\left( 1\right) },r\left( \left( \widetilde{A}_{n}\left( \left\Vert \cdot -x_{0}\right\Vert _{p}^{2}\right) \right) \left( x_{0}\right) \right) ^{\frac{1}{2}}\right) \notag \\ & \quad \cdot \left( \left( \widetilde{A}_{n}\left( \left\Vert \cdot -x_{0}\right\Vert _{p}^{2}\right) \right) \left( x_{0}\right) \right) ^{\frac{1}{2}}\left[ 1+r+\tfrac {r^{2}}{4}\right] , \notag \end{align}
and
2)
\begin{align} & \left\Vert \left\Vert \left( A_{n}\left( f\right) \right) -f\right\Vert _{\gamma }\right\Vert _{\infty ,\prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] }\leq \label{66} \\ & \leq \left\Vert \left\Vert \left( A_{n}\left( f^{\left( 1\right) }\left( x_{0}\right) \left( \cdot -x_{0}\right) \right) \right) \left( x_{0}\right) \right\Vert _{\gamma }\right\Vert _{\infty ,x_{0}\in \prod \limits _{i=1}^{N} \left[ a_{i},b_{i}\right] } \notag \\ & \quad +\tfrac {1}{2r}\omega _{1}\left( f^{\left( 1\right) },r\left\Vert \left( \widetilde{A}_{n}\left( \left\Vert \cdot -x_{0}\right\Vert _{p}^{2}\right) \right) \left( x_{0}\right) \right\Vert _{\infty ,x_{0}\in \prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] }^{\frac{1}{2}}\right) \notag \\ & \quad \cdot \left\Vert \left( \widetilde{A}_{n}\left( \left\Vert \cdot -x_{0}\right\Vert _{p}^{2}\right) \right) \left( x_{0}\right) \right\Vert _{\infty ,x_{0}\in \prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] }^{\frac{1}{2}}\left[ 1+r+\tfrac {r^{2}}{4}\right] , \notag \end{align}
\(r{\gt}0.\)
We make
We conclude:
In theorem 6, the right hand sides of (65) and (66) converge to zero as \(n\rightarrow \infty \), for any \(p\in \left[ 1,\infty \right] \).
Also in Corollary 1, the right hand sides of (68) and (69) converge to zero as \(n\rightarrow \infty \), for any \(p\in \left[ 1,\infty \right] .\)
Conclusion
1
We have proved that the left hand sides of 63, 64, 65, 66 and 68, 69 converge to zero as \(n\rightarrow \infty \), for \(p\in \left[ 1,\infty \right] \). Consequently \(A_{n}\rightarrow I\) (unit operator) pointwise and uniformly, as \(n\rightarrow \infty \), where \(p\in \left[ 1,\infty \right] \). In the presence of initial conditions we achieve a higher speed of convergence, see 64. Higher speed of convergence happens also to the left hand side of 63.
We give
Corollary
2
(to 6) Let \(O\) open subset of \(\left( \mathbb {R}^{N},\left\Vert \cdot \right\Vert _{\infty }\right) \), such that
\(\prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] \subset O\subseteq \mathbb {R}^{N}\), and let \(\left( X,\left\Vert \cdot \right\Vert _{\gamma }\right) \) be a general Banach space. Let \(m\in \mathbb {N}\) and \(f\in C^{m}\left( O,X\right) \), the space of \(m\)-times continuously Fréchet differentiable functions from \(O\) into \(X\). We study the approximation of \(f|_{\prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] }.\) Let \(x_{0}\in \left( \prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] \right) \) and \(r{\gt}0\). Here \(\varphi _{1}\left( n\right) \) as in 74 and \(\varphi _{2j}\left( n\right) \) as in 82, where \(n\in \mathbb {N}:n^{1-\alpha }{\gt}2\), \(0{\lt}\alpha {\lt}1\), \(j=1,...,m.\) Then
1)
\begin{align} & \left\Vert \left( A_{n}\left( f\right) \right) \left( x_{0}\right) -\sum _{j=0}^{m}\tfrac {1}{j!}\left( A_{n}\left( f^{\left( j\right) }\left( x_{0}\right) \left( \cdot -x_{0}\right) ^{j}\right) \right) \left( x_{0}\right) \right\Vert _{\gamma }\leq \label{76} \\ & \leq \tfrac {1}{rm!}\omega _{1}\left( f^{\left( m\right) },r\left( \varphi _{1}\left( n\right) \right) ^{\frac{1}{m+1}}\right) \left( \varphi _{1}\left( n\right) \right) ^{\left( \frac{m}{m+1}\right) }\left[ \tfrac {1}{\left( m+1\right) }+\tfrac {r}{2}+\tfrac {mr^{2}}{8}\right] , \notag \end{align}
2) additionally, if \(f^{\left( j\right) }\left( x_{0}\right) =0\), \(j=1,...,m\), we have
\begin{align} & \left\Vert \left( A_{n}\left( f\right) \right) \left( x_{0}\right) -f\left( x_{0}\right) \right\Vert _{\gamma }\leq \label{77} \\ & \leq \tfrac {1}{rm!}\omega _{1}\left( f^{\left( m\right) },r\left( \varphi _{1}\left( n\right) \right) ^{\frac{1}{m+1}}\right) \left( \varphi _{1}\left( n\right) \right) ^{\left( \frac{m}{m+1}\right) }\left[ \tfrac {1}{\left( m+1\right) }+\tfrac {r}{2}+\tfrac {mr^{2}}{8}\right] , \notag \end{align}
3)
\begin{align} & \left\Vert \left\Vert A_{n}\left( f\right) -f\right\Vert _{\gamma }\right\Vert _{\infty ,\prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] }\leq \label{78} \\ & \leq \sum _{j=1}^{m}\tfrac {\varphi _{2j}\left( n\right) }{j!}+\tfrac {1}{rm!}\omega _{1}\left( f^{\left( m\right) },r\left( \varphi _{1}\left( n\right) \right) ^{\frac{1}{m+1}}\right) \left( \varphi _{1}\left( n\right) \right) ^{\left( \frac{m}{m+1}\right) }\left[ \tfrac {1}{\left( m+1\right) }+\tfrac {r}{2}+\tfrac {mr^{2}}{8}\right] \notag \\ & =:\varphi _{3}\left( n\right) \rightarrow 0\text{, as }n\rightarrow \infty . \notag \end{align}
We continue with
Theorem
7
Let \(f\in C_{B}\left( \mathbb {R}^{N},X\right) ,\) \(0{\lt}\beta {\lt}1\), \(x\in \mathbb {R}^{N},\) \(N,n\in \mathbb {N}\) with \(n^{1-\beta }{\gt}2\), \(\omega _{1} \) is for \(p=\infty \). Then
1)
\begin{equation} \left\Vert B_{n}\left( f,x\right) -f\left( x\right) \right\Vert _{\gamma }\leq \omega _{1}\left( f,\tfrac {1}{n^{\beta }}\right) +\tfrac {4\left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }}{\pi ^{2}\left( n^{1-\beta }-2\right) }=:\lambda _{2}\left( n\right) , \label{79} \end{equation}
86
2)
\begin{equation} \left\Vert \left\Vert B_{n}\left( f\right) -f\right\Vert _{\gamma }\right\Vert _{\infty }\leq \lambda _{2}\left( n\right) . \label{80} \end{equation}
87
Given that \(f\in \left( C_{U}\left( \mathbb {R}^{N},X\right) \cap C_{B}\left( \mathbb {R}^{N},X\right) \right) \), we obtain \(\underset {n\rightarrow \infty }{\lim }B_{n}\left( f\right) =f\), uniformly.
We have that
\begin{align} B_{n}\left( f,x\right) -f\left( x\right) & \overset {\text{(\ref{18})}}{=}\sum _{k=-\infty }^{\infty }f\left( \tfrac {k}{n}\right) Z\left( nx-k\right) -f\left( x\right) \sum _{k=-\infty }^{\infty }Z\left( nx-k\right) \label{81} \\ & =\sum _{k=-\infty }^{\infty }\left( f\left( \tfrac {k}{n}\right) -f\left( x\right) \right) Z\left( nx-k\right) . \notag \end{align}
Hence
\begin{align} \left\Vert B_{n}\left( f,x\right) -f\left( x\right) \right\Vert _{\gamma } & \leq \sum _{k=-\infty }^{\infty }\left\Vert f\left( \tfrac {k}{n}\right) -f\left( x\right) \right\Vert _{\gamma }Z\left( nx-k\right) \notag \\ & =\sum _{\substack { k=-\infty \\ \begin{bgroup} \left\Vert \frac{k}{n}-x\right\Vert _{\infty }\leq \frac{1}{n^{\beta }}
\end{bgroup}}}^{\infty }\left\Vert f\left( \tfrac {k}{n}\right) -f\left( x\right) \right\Vert _{\gamma }Z\left( nx-k\right) \notag \\ & \quad +\sum _{\substack { k=-\infty \\ \begin{bgroup} \left\Vert \frac{k}{n}-x\right\Vert _{\infty }{\gt}\frac{1}{n^{\beta }}
\end{bgroup}}}^{\infty }\left\Vert f\left( \tfrac {k}{n}\right) -f\left( x\right) \right\Vert _{\gamma }Z\left( nx-k\right) \notag \\ & \overset {(\ref{18})}{\leq }\omega _{1}\left( f,\tfrac {1}{n^{\beta }}\right) +2\left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }\sum _{\substack { k=-\infty \\ \begin{bgroup} \left\Vert \frac{k}{n}-x\right\Vert _{\infty }{\gt}\frac{1}{n^{\beta }}
\end{bgroup}}}^{\infty }Z\left( nx-k\right) \notag \\ & \overset {\text{(\ref{25})}}{\leq }\omega _{1}\left( f,\tfrac {1}{n^{\beta }}\right) +\tfrac {4\left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }}{\pi ^{2}\left( n^{1-\beta }-2\right) }, \label{82} \end{align}
proving the claim.
We give
Theorem
8
Let \(f\in C_{B}\left( \mathbb {R}^{N},X\right) ,\) \(0{\lt}\beta {\lt}1\), \(x\in \mathbb {R}^{N},\) \(N,n\in \mathbb {N}\) with \(n^{1-\beta }{\gt}2\), \(\omega _{1} \) is for \(p=\infty \). Then
1)
\begin{equation} \left\Vert C_{n}\left( f,x\right) -f\left( x\right) \right\Vert _{\gamma }\leq \omega _{1}\left( f,\tfrac {1}{n}+\tfrac {1}{n^{\beta }}\right) +\tfrac {4\left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }}{\pi ^{2}\left( n^{1-\beta }-2\right) }=:\lambda _{3}\left( n\right) , \label{83} \end{equation}
93
2)
\begin{equation} \left\Vert \left\Vert C_{n}\left( f\right) -f\right\Vert _{\gamma }\right\Vert _{\infty }\leq \lambda _{3}\left( n\right) . \label{84} \end{equation}
94
Given that \(f\in \left( C_{U}\left( \mathbb {R}^{N},X\right) \cap C_{B}\left( \mathbb {R}^{N},X\right) \right) ,\) we obtain \(\underset {n\rightarrow \infty }{\lim }C_{n}\left( f\right) =f\), uniformly.
We notice that
\begin{equation*} \int _{\frac{k}{n}}^{\frac{k+1}{n}}f\left( t\right) dt=\int _{\frac{k_{1}}{n}}^{\frac{k_{1}+1}{n}}\int _{\frac{k_{2}}{n}}^{\frac{k_{2}+1}{n}}...\int _{\frac{k_{N}}{n}}^{\frac{k_{N}+1}{n}}f\left( t_{1},t_{2},...,t_{N}\right) dt_{1}dt_{2}...dt_{N}= \end{equation*}
\begin{equation} \int _{0}^{\frac{1}{n}}\int _{0}^{\frac{1}{n}}...\int _{0}^{\frac{1}{n}}f\left( t_{1}+\tfrac {k_{1}}{n},t_{2}+\tfrac {k_{2}}{n},...,t_{N}+\tfrac {k_{N}}{n}\right) dt_{1}...dt_{N}=\int _{0}^{\frac{1}{n}}f\left( t+\tfrac {k}{n}\right) dt. \label{85} \end{equation}
95
Thus it holds (by (40))
\begin{equation} C_{n}\left( f,x\right) =\sum _{k=-\infty }^{\infty }\left( n^{N}\int _{0}^{\frac{1}{n}}f\left( t+\tfrac {k}{n}\right) dt\right) Z\left( nx-k\right) . \label{86} \end{equation}
96
We observe that
\begin{align} & \left\Vert C_{n}\left( f,x\right) -f\left( x\right) \right\Vert _{\gamma }= \notag \\ & =\left\Vert \sum _{k=-\infty }^{\infty }\left( n^{N}\int _{0}^{\frac{1}{n}}f\left( t+\tfrac {k}{n}\right) dt\right) Z\left( nx-k\right) -\sum _{k=-\infty }^{\infty }f\left( x\right) Z\left( nx-k\right) \right\Vert _{\gamma } \notag \\ & \left\Vert \sum _{k=-\infty }^{\infty }\left( \left( n^{N}\int _{0}^{\frac{1}{n}}f\left( t+\tfrac {k}{n}\right) dt\right) -f\left( x\right) \right) Z\left( nx-k\right) \right\Vert _{\gamma } \notag \\ & =\left\Vert \sum _{k=-\infty }^{\infty }\left( n^{N}\int _{0}^{\frac{1}{n}}\left( f\left( t+\tfrac {k}{n}\right) -f\left( x\right) \right) dt\right) Z\left( nx-k\right) \right\Vert _{\gamma } \label{87} \\ & \leq \sum _{k=-\infty }^{\infty }\left( n^{N}\int _{0}^{\frac{1}{n}}\left\Vert f\left( t+\tfrac {k}{n}\right) -f\left( x\right) \right\Vert _{\gamma }dt\right) Z\left( nx-k\right) \notag \\ & =\sum _{\substack { k=-\infty \\ \begin{bgroup} \left\Vert \frac{k}{n}-x\right\Vert _{\infty }\leq \frac{1}{n^{\beta }}
\end{bgroup}}}^{\infty }\left( n^{N}\int _{0}^{\frac{1}{n}}\left\Vert f\left( t+\tfrac {k}{n}\right) -f\left( x\right) \right\Vert _{\gamma }dt\right) Z\left( nx-k\right) \notag \\ & \quad +\sum _{\substack { k=-\infty \\ \begin{bgroup} \left\Vert \frac{k}{n}-x\right\Vert _{\infty }{\gt}\frac{1}{n^{\beta }}
\end{bgroup}}}^{\infty }\left( n^{N}\int _{0}^{\frac{1}{n}}\left\Vert f\left( t+\tfrac {k}{n}\right) -f\left( x\right) \right\Vert _{\gamma }dt\right) Z\left( nx-k\right) \notag \\ & \leq \sum _{\substack { k=-\infty \\ \begin{bgroup} \left\Vert \frac{k}{n}-x\right\Vert _{\infty }\leq \frac{1}{n^{\beta }}
\end{bgroup}}}^{\infty }\left( n^{N}\int _{0}^{\frac{1}{n}}\omega _{1}\left( f,\left\Vert t\right\Vert _{\infty }+\left\Vert \tfrac {k}{n}-x\right\Vert _{\infty }\right) dt\right) Z\left( nx-k\right) \notag \\ & \quad +2\left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }\left( \sum _{\substack { k=-\infty \\ \begin{bgroup} \left\Vert \frac{k}{n}-x\right\Vert _{\infty }{\gt}\frac{1}{n^{\beta }}
\end{bgroup}}}^{\infty }Z\left( \left\vert nx-k\right\vert \right) \right) \notag \\ & \leq \omega _{1}\left( f,\tfrac {1}{n}+\tfrac {1}{n^{\beta }}\right) +\tfrac {4\left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }}{\pi ^{2}\left( n^{1-\beta }-2\right) }, \label{88} \end{align}
proving the claim.
We also present
Theorem
9
Let \(f\in C_{B}\left( \mathbb {R}^{N},X\right) ,\) \(0{\lt}\beta {\lt}1\), \(x\in \mathbb {R}^{N},\) \(N,n\in \mathbb {N}\) with \(n^{1-\beta }{\gt}2\), \(\omega _{1} \) is for \(p=\infty .\) Then
1)
\begin{equation} \left\Vert D_{n}\left( f,x\right) -f\left( x\right) \right\Vert _{\gamma }\leq \omega _{1}\left( f,\tfrac {1}{n}+\tfrac {1}{n^{\beta }}\right) +\tfrac {4\left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }}{\pi ^{2}\left( n^{1-\beta }-2\right) }=\lambda _{4}\left( n\right) , \label{89} \end{equation}
103
2)
\begin{equation} \left\Vert \left\Vert D_{n}\left( f\right) -f\right\Vert _{\gamma }\right\Vert _{\infty }\leq \lambda _{4}\left( n\right) . \label{90} \end{equation}
104
Given that \(f\in \left( C_{U}\left( \mathbb {R}^{N},X\right) \cap C_{B}\left( \mathbb {R}^{N},X\right) \right) ,\) we obtain \(\underset {n\rightarrow \infty }{\lim }D_{n}\left( f\right) =f\), uniformly.
We have that (by (
42))
\begin{align} & \left\Vert D_{n}\left( f,x\right) -f\left( x\right) \right\Vert _{\gamma }= \label{91} \\ & =\left\Vert \sum _{k=-\infty }^{\infty }\delta _{nk}\left( f\right) Z\left( nx-k\right) -\sum _{k=-\infty }^{\infty }f\left( x\right) Z\left( nx-k\right) \right\Vert _{\gamma } \notag \\ & =\left\Vert \sum _{k=-\infty }^{\infty }\left( \delta _{nk}\left( f\right) -f\left( x\right) \right) Z\left( nx-k\right) \right\Vert _{\gamma } \notag \\ & =\left\Vert \sum _{k=-\infty }^{\infty }\left( \sum _{r=0}^{\theta }w_{r}\left( f\left( \tfrac {k}{n}+\tfrac {r}{n\theta }\right) -f\left( x\right) \right) \right) Z\left( nx-k\right) \right\Vert _{\gamma } \notag \\ & \leq \sum _{k=-\infty }^{\infty }\left( \sum _{r=0}^{\theta }w_{r}\left\Vert f\left( \tfrac {k}{n}+\tfrac {r}{n\theta }\right) -f\left( x\right) \right\Vert _{\gamma }\right) Z\left( nx-k\right) \notag \\ & =\sum _{\substack { k=-\infty \\ \begin{bgroup} \left\Vert \frac{k}{n}-x\right\Vert _{\infty }\leq \frac{1}{n^{\beta }}
\end{bgroup}}}^{\infty }\left( \sum _{r=0}^{\theta }w_{r}\left\Vert f\left( \tfrac {k}{n}+\tfrac {r}{n\theta }\right) -f\left( x\right) \right\Vert _{\gamma }\right) Z\left( nx-k\right) \notag \\ & \quad +\sum _{\substack { k=-\infty \\ \begin{bgroup} \left\Vert \frac{k}{n}-x\right\Vert _{\infty }{\gt}\frac{1}{n^{\beta }}
\end{bgroup}}}^{\infty }\left( \sum _{r=0}^{\theta }w_{r}\left\Vert f\left( \tfrac {k}{n}+\tfrac {r}{n\theta }\right) -f\left( x\right) \right\Vert _{\gamma }\right) Z\left( nx-k\right) \notag \\ & \leq \sum _{\substack { k=-\infty \\ \begin{bgroup} \left\Vert \tfrac {k}{n}-x\right\Vert _{\infty }\leq \frac{1}{n^{\beta }}
\end{bgroup}}}^{\infty }\left( \sum _{r=0}^{\theta }w_{r}\omega _{1}\left( f,\left\Vert \tfrac {k}{n}-x\right\Vert _{\infty }+\left\Vert \tfrac {r}{n\theta }\right\Vert _{\infty }\right) \right) Z\left( nx-k\right) \notag \\ & \quad +2\left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }\left( \sum _{\substack { k=-\infty \\ \begin{bgroup} \left\Vert \frac{k}{n}-x\right\Vert _{\infty }{\gt}\frac{1}{n^{\beta }}
\end{bgroup}}}^{\infty }Z\left( nx-k\right) \right) \notag \\ & \leq \omega _{1}\left( f,\tfrac {1}{n}+\tfrac {1}{n^{\beta }}\right) +\tfrac {4\left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }}{\pi ^{2}\left( n^{1-\beta }-2\right) }, \label{92} \end{align}
proving the claim.
We make
Definition
2
Let \(f\in C_{B}\left( \mathbb {R}^{N},X\right) \), \(N\in \mathbb {N}\), where \(\left( X,\left\Vert \cdot \right\Vert _{\gamma }\right) \) is a Banach space. We define the general neural network operator
\begin{equation} F_{n}\left( f,x\right) :=\sum \limits _{k=-\infty }^{\infty }l_{nk}\left( f\right) Z\left( nx-k\right) =\left\{ \begin{array}{l} B_{n}\left( f,x\right) \text{, \ if }l_{nk}\left( f\right) =f\left( \frac{k}{n}\right) , \\ C_{n}\left( f,x\right) \text{, \ if }l_{nk}\left( f\right) =n^{N}\int _{\frac{k}{n}}^{\frac{k+1}{n}}f\left( t\right) dt, \\ D_{n}\left( f,x\right) \text{, \ if }l_{nk}\left( f\right) =\delta _{nk}\left( f\right) .\end{array}\right. \label{93} \end{equation}
111
Clearly \(l_{nk}\left( f\right) \) is an \(X\)-valued bounded linear functional such that \(\left\Vert l_{nk}\left( f\right) \right\Vert _{\gamma }\leq \left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }.\)
Hence \(F_{n}\left( f\right) \) is a bounded linear operator with \(\left\Vert \left\Vert F_{n}\left( f\right) \right\Vert _{\gamma }\right\Vert _{\infty }\leq \left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }\).
We need
Theorem
10
Let \(f\in C_{B}\left( \mathbb {R}^{N},X\right) \), \(N\geq 1\). Then \(F_{n}\left( f\right) \in C_{B}\left( \mathbb {R}^{N},X\right) .\)
Clearly \(F_{n}\left( f\right) \) is a bounded function.
Next we prove the continuity of \(F_{n}\left( f\right) \). Notice for \(N=1\), \(Z=\psi \) by (16).
We will use the generalized Weierstrass \(M\) test: If a sequence of positive constants \(M_{1},M_{2},M_{3},...,\) can be found such that in some interval
(a) \(\left\Vert u_{n}\left( x\right) \right\Vert _{\gamma }\leq M_{n}\), \(n=1,2,3,...\)
(b) \(\sum M_{n}\) converges,
then \(\sum u_{n}\left( x\right) \) is uniformly and absolutely convergent in the interval.
Also we will use:
If \(\{ u_{n}\left( x\right) \} \), \(n=1,2,3,...\) are continuous in \(\left[ a,b\right] \) and if \(\sum u_{n}\left( x\right) \) converges uniformly to the sum \(S\left( x\right) \) in \(\left[ a,b\right] \), then \(S\left( x\right) \) is continuous in \(\left[ a,b\right] \). I.e. a uniformly convergent series of continuous functions is a continuous function. First we prove claim for \(N=1\).
We will prove that \(\sum _{k=-\infty }^{\infty }l_{nk}\left( f\right) \psi \left( nx-k\right) \) is continuous in \(x\in \mathbb {R}\).
There always exists \(\lambda \in \mathbb {N}\) such that \(nx\in \left[ -\lambda ,\lambda \right] .\)
Since \(nx\leq \lambda \), then \(-nx\geq -\lambda \) and \(k-nx\geq k-\lambda \geq 0\), when \(k\geq \lambda \). Therefore
\begin{equation} \sum _{k=\lambda }^{\infty }\psi \left( nx-k\right) =\sum _{k=\lambda }^{\infty }\psi \left( k-nx\right) \leq \sum _{k=\lambda }^{\infty }\psi \left( k-\lambda \right) =\sum _{k^{\prime }=0}^{\infty }\psi \left( k^{\prime }\right) \leq 1. \label{94} \end{equation}
112
So for \(k\geq \lambda \) we get
\begin{equation*} \left\Vert l_{nk}\left( f\right) \right\Vert _{\gamma }\psi \left( nx-k\right) \leq \left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }\psi \left( k-\lambda \right) , \end{equation*}
and
\begin{equation*} \left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }\sum _{k=\lambda }^{\infty }\psi \left( k-\lambda \right) \leq \left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }. \end{equation*}
Hence by the generalized Weierstrass \(M\) test we obtain that \(\sum \limits _{k=\lambda }^{\infty }l_{nk}\left( f\right) \psi \left( nx-k\right) \) is uniformly and absolutely convergent on \(\left[ -\frac{\lambda }{n},\frac{\lambda }{n}\right] .\)
Since \(l_{nk}\left( f\right) \psi \left( nx-k\right) \) is continuous in \(x\), then \(\sum _{k=\lambda }^{\infty }l_{nk}\left( f\right) \psi \left( nx-k\right) \) is continuous on \(\left[ -\frac{\lambda }{n},\frac{\lambda }{n}\right] .\)
Because \(nx\geq -\lambda \), then \(-nx\leq \lambda \), and \(k-nx\leq k+\lambda \leq 0\), when \(k\leq -\lambda \). Therefore
\begin{equation*} \sum _{k=-\infty }^{-\lambda }\psi \left( nx-k\right) =\sum _{k=-\infty }^{-\lambda }\psi \left( k-nx\right) \leq \sum _{k=-\infty }^{-\lambda }\psi \left( k+\lambda \right) =\sum _{k^{\prime }=-\infty }^{0}\psi \left( k^{\prime }\right) \leq 1. \end{equation*}
So for \(k\leq -\lambda \) we get
\begin{equation} \left\Vert l_{nk}\left( f\right) \right\Vert _{\gamma }\psi \left( nx-k\right) \leq \left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }\psi \left( k+\lambda \right) , \label{95} \end{equation}
113
and
\begin{equation*} \left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }\sum _{k=-\infty }^{-\lambda }\psi \left( k+\lambda \right) \leq \left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }. \end{equation*}
Hence by Weierstrass \(M\) test we obtain that \(\sum _{k=-\infty }^{-\lambda }l_{nk}\left( f\right) \psi \left( nx-k\right) \) is uniformly and absolutely convergent on \(\left[ -\frac{\lambda }{n},\frac{\lambda }{n}\right] .\)
Since \(l_{nk}\left( f\right) \psi \left( nx-k\right) \) is continuous in \(x\), then \(\sum _{k=-\infty }^{-\lambda }l_{nk}\left( f\right) \psi \left( nx-k\right) \) is continuous on \(\left[ -\frac{\lambda }{n},\frac{\lambda }{n}\right] .\)
So we proved that \(\sum _{k=\lambda }^{\infty }l_{nk}\left( f\right) \psi \left( nx-k\right) \) and \(\sum _{k=-\infty }^{-\lambda }l_{nk}\left( f\right) \psi \left( nx-k\right) \) are continuous on \(\mathbb {R}\). Since \(\sum _{k=-\lambda +1}^{\lambda -1}l_{nk}\left( f\right) \psi \left( nx-k\right) \) is a finite sum of continuous functions on \(\mathbb {R}\), it is also a continuous function on \(\mathbb {R}\).
Writing
\begin{align} \sum _{k=-\infty }^{\infty }l_{nk}\left( f\right) \psi \left( nx-k\right) & =\sum _{k=-\infty }^{-\lambda }l_{nk}\left( f\right) \psi \left( nx-k\right) \notag \\ & \quad +\sum _{k=-\lambda +1}^{\lambda -1}l_{nk}\left( f\right) \psi \left( nx-k\right) +\sum _{k=\lambda }^{\infty }l_{nk}\left( f\right) \psi \left( nx-k\right) \label{96} \end{align}
we have it as a continuous function on \(\mathbb {R}\). Therefore \(F_{n}\left( f\right) \), when \(N=1\), is a continuous function on \(\mathbb {R}\).
When \(N=2\) we have
\begin{align*} F_{n}\left( f,x_{1},x_{2}\right) & =\sum _{k_{1}=-\infty }^{\infty }\sum _{k_{2}=-\infty }^{\infty }l_{nk}\left( f\right) \psi \left( nx_{1}-k_{1}\right) \psi \left( nx_{2}-k_{2}\right) \\ & =\sum _{k_{1}=-\infty }^{\infty }\psi \left( nx_{1}-k_{1}\right) \left( \sum _{k_{2}=-\infty }^{\infty }l_{nk}\left( f\right) \psi \left( nx_{2}-k_{2}\right) \right) \end{align*}
(there always exist \(\lambda _{1},\lambda _{2}\in \mathbb {N}\) such that \(nx_{1}\in \left[ -\lambda _{1},\lambda _{1}\right] \) and \(nx_{2}\in \left[ -\lambda _{2},\lambda _{2}\right] \))
\begin{align*} & =\sum _{k_{1}=-\infty }^{\infty }\psi \left( nx_{1}-k_{1}\right) \left[ \sum _{k_{2}=-\infty }^{-\lambda _{2}}l_{nk}\left( f\right) \psi \left( nx_{2}-k_{2}\right) \right. \\ & \quad +\left. \sum _{k_{2}=-\lambda _{2}+1}^{\lambda _{2}-1}l_{nk}\left( f\right) \psi \left( nx_{2}-k_{2}\right) +\sum _{k_{2}=\lambda _{2}}^{\infty }l_{nk}\left( f\right) \psi \left( nx_{2}-k_{2}\right) \right] \\ & =\sum _{k_{1}=-\infty }^{\infty }\sum _{k_{2}=-\infty }^{-\lambda _{2}}l_{nk}\left( f\right) \psi \left( nx_{1}-k_{1}\right) \psi \left( nx_{2}-k_{2}\right) \\ & \quad +\sum _{k_{1}=-\infty }^{\infty }\sum _{k_{2}=-\lambda _{2}+1}^{\lambda _{2}-1}l_{nk}\left( f\right) \psi \left( nx_{1}-k_{1}\right) \psi \left( nx_{2}-k_{2}\right) \\ & \quad +\sum _{k_{1}=-\infty }^{\infty }\sum _{k_{2}=\lambda _{2}}^{\infty }l_{nk}\left( f\right) \psi \left( nx_{1}-k_{1}\right) \psi \left( nx_{2}-k_{2}\right) =:\left( \ast \right) . \end{align*}
(For convenience call
\begin{equation*} F\left( k_{1},k_{2},x_{1},x_{2}\right) :=l_{nk}\left( f\right) \psi \left( nx_{1}-k_{1}\right) \psi \left( nx_{2}-k_{2}\right) .\text{ )} \end{equation*}
Thus
\begin{align} \left( \ast \right) & =\sum _{k_{1}=-\infty }^{-\lambda _{1}}\sum _{k_{2}=-\infty }^{-\lambda _{2}}F\left( k_{1},k_{2},x_{1},x_{2}\right) +\sum _{k_{1}=-\lambda _{1}+1}^{\lambda _{1}-1}\sum _{k_{2}=-\infty }^{-\lambda _{2}}F\left( k_{1},k_{2},x_{1},x_{2}\right) \notag \\ & \quad +\sum _{k_{1}=\lambda _{1}}^{\infty }\sum _{k_{2}=-\infty }^{-\lambda _{2}}F\left( k_{1},k_{2},x_{1},x_{2}\right) +\sum _{k_{1}=-\infty }^{-\lambda _{1}}\sum _{k_{2}=-\lambda _{2}+1}^{\lambda _{2}-1}F\left( k_{1},k_{2},x_{1},x_{2}\right) \notag \\ & \quad +\sum _{k_{1}=-\lambda _{1}+1}^{\lambda _{1}-1}\sum _{k_{2}=-\lambda _{2}+1}^{\lambda _{2}-1}F\left( k_{1},k_{2},x_{1},x_{2}\right) +\sum _{k_{1}=\lambda _{1}}^{\infty }\sum _{k_{2}=-\lambda _{2}+1}^{\lambda _{2}-1}F\left( k_{1},k_{2},x_{1},x_{2}\right) \notag \\ & \quad +\sum _{k_{1}=-\infty }^{-\lambda _{1}}\sum _{k_{2}=\lambda _{2}}^{\infty }F\left( k_{1},k_{2},x_{1},x_{2}\right) +\sum _{k_{1}=-\lambda _{1}+1}^{\lambda _{1}-1}\sum _{k_{2}=\lambda _{2}}^{\infty }F\left( k_{1},k_{2},x_{1},x_{2}\right) \label{97} \\ & \quad +\sum _{k_{1}=\lambda _{1}}^{\infty }\sum _{k_{2}=\lambda _{2}}^{\infty }F\left( k_{1},k_{2},x_{1},x_{2}\right) . \notag \end{align}
Notice that the finite sum of continuous functions \(F\left( k_{1},k_{2},x_{1},x_{2}\right) \),
\(\sum _{k_{1}=-\lambda _{1}+1}^{\lambda _{1}-1}\sum _{k_{2}=-\lambda _{2}+1}^{\lambda _{2}-1}F\left( k_{1},k_{2},x_{1},x_{2}\right) \) is a continuous function.
The rest of the summands of \(F_{n}\left( f,x_{1},x_{2}\right) \) are treated all the same way and similarly to the case of \(N=1\). The method is demonstrated as follows.
We will prove that \(\sum _{k_{1}=\lambda _{1}}^{\infty }\sum _{k_{2}=-\infty }^{-\lambda _{2}}l_{nk}\left( f\right) \psi \left( nx_{1}-k_{1}\right) \psi \left( nx_{2}-k_{2}\right) \) is continuous in \(\left( x_{1},x_{2}\right) \in \mathbb {R}^{2}\).
The continuous function
\begin{equation*} \left\Vert l_{nk}\left( f\right) \right\Vert _{\gamma }\psi \left( nx_{1}-k_{1}\right) \psi \left( nx_{2}-k_{2}\right) \leq \left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }\psi \left( k_{1}-\lambda _{1}\right) \psi \left( k_{2}+\lambda _{2}\right) , \end{equation*}
and
\begin{align*} & \left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }\sum _{k_{1}=\lambda _{1}}^{\infty }\sum _{k_{2}=-\infty }^{-\lambda _{2}}\psi \left( k_{1}-\lambda _{1}\right) \psi \left( k_{2}+\lambda _{2}\right) = \\ & =\left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }\left( \sum _{k_{1}=\lambda _{1}}^{\infty }\psi \left( k_{1}-\lambda _{1}\right) \right) \left( \sum _{k_{2}=-\infty }^{-\lambda _{2}}\psi \left( k_{2}+\lambda _{2}\right) \right) \\ & \leq \left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }\left( \sum _{k_{1}^{\prime }=0}^{\infty }\psi \left( k_{1}^{\prime }\right) \right) \left( \sum _{k_{2}^{\prime }=-\infty }^{0}\psi \left( k_{2}^{\prime }\right) \right) \leq \left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }. \end{align*}
So by the Weierstrass \(M\) test we get that
\(\sum _{k_{1}=\lambda _{1}}^{\infty }\sum _{k_{2}=-\infty }^{-\lambda _{2}}l_{nk}\left( f\right) \psi \left( nx_{1}-k_{1}\right) \psi \left( nx_{2}-k_{2}\right) \) is uniformly and absolutely convergent. Therefore it is continuous on \(\mathbb {R}^{2}.\)
Next we prove continuity on \(\mathbb {R}^{2}\) of
\(\sum _{k_{1}=-\lambda _{1}+1}^{\lambda _{1}-1}\sum _{k_{2}=-\infty }^{-\lambda _{2}}l_{nk}\left( f\right) \psi \left( nx_{1}-k_{1}\right) \psi \left( nx_{2}-k_{2}\right) \).
Notice here that
\begin{align*} & \left\Vert l_{nk}\left( f\right) \right\Vert _{\gamma }\psi \left( nx_{1}-k_{1}\right) \psi \left( nx_{2}-k_{2}\right) \leq \\ & \leq \left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }\psi \left( nx_{1}-k_{1}\right) \psi \left( k_{2}+\lambda _{2}\right) \\ & \leq \left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }\psi \left( 0\right) \psi \left( k_{2}+\lambda _{2}\right) =0.319\cdot \left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }\psi \left( k_{2}+\lambda _{2}\right) , \end{align*}
and
\begin{align} & 0.319\cdot \left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }\left( \sum _{k_{1}=-\lambda _{1}+1}^{\lambda _{1}-1}1\right) \left( \sum _{k_{2}=-\infty }^{-\lambda _{2}}\psi \left( k_{2}+\lambda _{2}\right) \right) = \notag \\ & =0.319\cdot \left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }\left( 2\lambda _{1}-1\right) \left( \sum _{k_{2}^{\prime }=-\infty }^{0}\psi \left( k_{2}^{\prime }\right) \right) \notag \\ & \leq 0.319\cdot \left( 2\lambda _{1}-1\right) \left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }. \label{98} \end{align}
So the double series under consideration is uniformly convergent and continuous. Clearly \(F_{n}\left( f,x_{1},x_{2}\right) \) is proved to be continuous on \(\mathbb {R}^{2}.\)
Similarly reasoning one can prove easily now, but with more tedious work, that \(F_{n}\left( f,x_{1},...,x_{N}\right) \) is continuous on \(\mathbb {R}^{N} \), for any \(N\geq 1\). We choose to omit this similar extra work.
Notation
1
Here \(N\in \mathbb {N}\), \(0{\lt}\beta {\lt}1.\) Denote by
\begin{align} c_{N} & :=\left\{ \begin{array}{l} \left( 4.9737\right) ^{N}\text{, \ if }L_{n}=A_{n}, \\ 1\text{, \ if }L_{n}=B_{n},C_{n},D_{n},\end{array}\right. \label{102} \\ \varphi \left( n\right) & :=\left\{ \begin{array}{l} \frac{1}{n^{\beta }}\text{, \ if }L_{n}=A_{n}\text{, }B_{n}, \\ \frac{1}{n}+\frac{1}{n^{\beta }}\text{, \ if }L_{n}=C_{n},D_{n},\end{array}\right. \label{103} \\ \Omega & :=\left\{ \begin{array}{l} C\left( \prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] ,X\right) \text{, if }L_{n}=A_{n}\text{, } \\ C_{B}\left( \mathbb {R}^{N},X\right) \text{, \ if }L_{n}=B_{n},C_{n},D_{n},\end{array}\right. \; \text{and} \label{104} \\ Y & :=\left\{ \begin{array}{l} \prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] \text{, \ if }L_{n}=A_{n}\text{, } \\ \mathbb {R}^{N}\text{, \ if }L_{n}=B_{n},C_{n},D_{n}.\end{array}\right. \label{105} \end{align}
We give the condensed
Theorem
11
Let \(f\in \Omega \), \(0{\lt}\beta {\lt}1\), \(x\in Y;\) \(n,\) \(N\in \mathbb {N}\) with \(n^{1-\beta }{\gt}2\). Then
(i)
\begin{equation} \left\Vert L_{n}\left( f,x\right) -f\left( x\right) \right\Vert _{\gamma }\leq c_{N}\left[ \omega _{1}\left( f,\varphi \left( n\right) \right) +\tfrac {4\left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }}{\pi ^{2}\left( n^{1-\beta }-2\right) }\right] =:\tau \left( n\right) , \label{106} \end{equation}
124
where \(\omega _{1}\) is for \(p=\infty ,\)
and
(ii)
\begin{equation} \left\Vert \left\Vert L_{n}\left( f\right) -f\right\Vert _{\gamma }\right\Vert _{\infty }\leq \tau \left( n\right) \rightarrow 0\text{, as }n\rightarrow \infty . \label{107} \end{equation}
125
For \(f\) uniformly continuous and in \(\Omega \) we obtain
\begin{equation*} \underset {n\rightarrow \infty }{\lim }L_{n}\left( f\right) =f, \end{equation*}
pointwise and uniformly.
By ??.
Next we do iterated neural network approximation (see also
[
9
]
).
We make
We give
Theorem
12
All here as in theorem 11 and \(r\in \mathbb {N}\), \(\tau \left( n\right) \) as in (124). Then
\begin{equation} \left\Vert \left\Vert L_{n}^{r}f-f\right\Vert _{\gamma }\right\Vert _{\infty }\leq r\tau \left( n\right) . \label{110} \end{equation}
128
So that the speed of convergence to the unit operator of \(L_{n}^{r}\) is not worse than of \(L_{n}.\)
By (
127) and (
125).
We make
We give
Theorem
13
Let \(f\in \Omega \); \(N,\) \(m_{1},m_{2},...,m_{r}\in \mathbb {N}:m_{1}\leq m_{2}\leq ...\leq m_{r},\) \(0{\lt}\beta {\lt}1;\) \(m_{i}^{1-\beta }{\gt}2\), \(i=1,...,r,\) \(x\in Y,\) and let \(\left( L_{m_{1}},...,L_{m_{r}}\right) \) as \(\left( A_{m_{1}},...,A_{m_{r}}\right) \) or \(\left( B_{m_{1}},...,B_{m_{r}}\right) \) or \(\left( C_{m_{1}},...,C_{m_{r}}\right) \) or \(\left( D_{m_{1}},...,D_{m_{r}}\right) \), \(p=\infty .\) Then
\begin{align} & \left\Vert L_{m_{r}}\left( L_{m_{r-1}}\left( ...L_{m_{2}}\left( L_{m_{1}}f\right) \right) \right) \left( x\right) -f\left( x\right) \right\Vert _{\gamma } \leq \nonumber \\ & \leq \left\Vert \left\Vert L_{m_{r}}\left( L_{m_{r-1}}\left( ...L_{m_{2}}\left( L_{m_{1}}f\right) \right) \right) -f\right\Vert _{\gamma }\right\Vert _{\infty } \notag \\ & \leq \sum _{i=1}^{r}\left\Vert \left\Vert L_{m_{i}}f-f\right\Vert _{\gamma }\right\Vert _{\infty } \notag \\ & \leq c_{N}\sum _{i=1}^{r}\left[ \omega _{1}\left( f,\varphi \left( m_{i}\right) \right) +\tfrac {4\left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }}{\pi ^{2}\left( m_{i}^{1-\beta }-2\right) }\right] \notag \\ & \leq rc_{N}\left[ \omega _{1}\left( f,\varphi \left( m_{1}\right) \right) +\tfrac {4\left\Vert \left\Vert f\right\Vert _{\gamma }\right\Vert _{\infty }}{\pi ^{2}\left( m_{1}^{1-\beta }-2\right) }\right] . \label{116} \end{align}
Clearly, we notice that the speed of convergence to the unit operator of the multiply iterated operator is not worse than the speed of \(L_{m_{1}}.\)
Using (
134), (
129), (
130) and (
124), (
125).
We continue with
Theorem
14
Let all as in Corollary 2, and \(r\in \mathbb {N}\). Here \(\varphi _{3}\left( n\right) \) is as in (85). Then
\begin{equation} \left\Vert \left\Vert A_{n}^{r}f-f\right\Vert _{\gamma }\right\Vert _{\infty }\leq r\left\Vert \left\Vert A_{n}f-f\right\Vert _{\gamma }\right\Vert _{\infty }\leq r\varphi _{3}\left( n\right) . \label{117} \end{equation}
136
By (
127) and (
85).
Application
1
A typical application of all of our results is when \(\left( X,\left\Vert \cdot \right\Vert _{\gamma }\right) =\left( \mathbb {C},\left\vert \cdot \right\vert \right) \), where \(\mathbb {C}\) are the complex numbers.
Bibliography
- 1
G.A. Anastassiou, Moments in Probability and Approximation Theory, Pitman Research Notes in Math., Vol. 287, Longman Sci. & Tech., Harlow, U.K., 1993.
- 2
G.A. Anastassiou, Rate of convergence of some neural network operators to the unit-univariate case, J. Math. Anal. Appli. 212 (1997), pp. 237–262. https://doi.org/10.1006/jmaa.1997.5494 ![\includegraphics[scale=0.1]{ext-link.png}](img-0001.png)
- 3
G.A. Anastassiou, Quantitative Approximations, Chapman&Hall/CRC, Boca Raton, New York, 2001.
- 4
G.A. Anastassiou, Inteligent Systems: Approximation by Artificial Neural Networks, Intelligent Systems Reference Library, Vol. 19, Springer, Heidelberg, 2011. https://doi.org/10.1007/978-3-642-21431-8
- 5
G.A. Anastassiou, Univariate hyperbolic tangent neural network approximation, Mathematics and Computer Modelling, 53 (2011), pp. 1111–1132. https://doi.org/10.1016/j.mcm.2010.11.072
- 6
G.A. Anastassiou, Multivariate hyperbolic tangent neural network approximation, Computers and Mathematics 61 (2011), pp. 809–821. https://doi.org/10.1016/j.camwa.2010.12.029
- 7
G.A. Anastassiou, Multivariate sigmoidal neural network approximation, Neural Networks 24 (2011), pp. 378–386. https://doi.org/10.1016/j.neunet.2011.01.003
- 8
G.A. Anastassiou, Univariate sigmoidal neural network approximation, J. of Computational Analysis and Applications, vol. 14 (2012) no. 4, pp. 659–690.
- 9
G.A. Anastassiou, Approximation by neural networks iterates, Advances in Applied Mathematics and Approximation Theory, pp. 1-20, Springer Proceedings in Math. & Stat., Springer, New York, 2013, Eds. G. Anastassiou, O. Duman.
- 10
G. Anastassiou, Intelligent Systems II: Complete Approximation by Neural Network Operators, Springer, Heidelberg, New York, 2016.
- 11
G. Anastassiou, Intelligent Computations: Abstract Fractional Calculus, Inequalities, Approximations, Springer, Heidelberg, New York, 2018.
- 12
H. Cartan, Differential Calculus, Hermann, Paris, 1971.
- 13
Z. Chen and F. Cao, The approximation operators with sigmoidal functions, Computers and Mathematics with Applications, 58 (2009), pp. 758–765. https://doi.org/10.1016/j.camwa.2009.05.001
- 14
D. Costarelli, R. Spigler, Approximation results for neural network operators activated by sigmoidal functions, Neural Networks 44 (2013), pp. 101–106. https://doi.org/10.1016/j.neunet.2013.03.015
- 15
D. Costarelli, R. Spigler, Multivariate neural network operators with sigmoidal activation functions, Neural Networks 48 (2013), pp. 72–77.https://doi.org/10.1016/j.neunet.2013.07.009
- 16
S. Haykin, Neural Networks: A Comprehensive Foundation (2 ed.), Prentice Hall, New York, 1998.
- 17
W. McCulloch and W. Pitts, A logical calculus of the ideas immanent in nervous activity, Bulletin of Mathematical Biophysics, 7 (1943), pp. 115–133. https://doi.org/10.1007/bf02478259
- 18
T.M. Mitchell, Machine Learning, WCB-McGraw-Hill, New York, 1997.
- 19
L.B. Rall, Computational Solution of Nonlinear Operator Equations, John Wiley & Sons, New York, 1969.
