\(^\ast \)Department of Mathematical Sciences, University of Memphis, Memphis, TN 38152, U.S.A., e-mail: ganastss@memphis.edu.

1 Introduction

The author in [ 2 ] and [ 3 ] , see chapters 2-5, was the first to establish neural network approximations to continuous functions with rates by very specifically defined neural network operators of Cardaliagnet-Euvrard and ”Squashing” types, by employing the modulus of continuity of the engaged function or its high order derivative, and producing very tight Jackson type inequalities. He treats there both the univariate and multivariate cases. The defining these operators ”bell-shaped” and ”squashing” functions are assumed to be of compact support. Also in [ 3 ] he gives the \(N\)th order asymptotic expansion for the error of weak approximation of these two operators to a special natural class of smooth functions, see chapters 4-5 there.

For this article the author is motivated by the article [ 12 ] of Z. Chen and F. Cao, also by [ 4 ] , [ 5 ] , [ 6 ] , [ 7 ] , [ 8 ] , [ 9 ] , [ 10 ] , [ 13 ] , [ 14 ] .

The author here performs multivariate error function based neural network approximations to continuous functions over boxes or over the whole \(\mathbb {R}^{N}\), \(N\in \mathbb {N}\), then he extends his results to complex valued multivariate functions. Also he does iterated approximation. All convergences here are with rates expressed via the multivariate modulus of continuity of the involved function or its high order partial derivative and given by very tight multidimensional Jackson type inequalities.

The author here comes up with the ”right” precisely defined multivariate quasi-interpolation neural network operators related to boxes or \(\mathbb {R}^{N}\), as well as Baskakov type and quadrature type related operators on \(\mathbb {R}^{N}\). Our boxes are not necessarily symmetric to the origin. In preparation to prove our results we establish important properties of the basic multivariate density function induced by error function and defining our operators.

Feed-forward neural networks (FNNs) with one hidden layer, the only type of networks we deal with in this article, are mathematically expressed as

where for \(0\leq j\leq n\), \(b_{j}\in \mathbb {R}\) are the thresholds, \(a_{j}\in \mathbb {R}^{s}\) are the connection weights, \(c_{j}\in \mathbb {R}\) are the coefficients, \(\left\langle a_{j}\cdot x\right\rangle \) is the inner product of \(a_{j}\) and \(x\), and \(\sigma \) is the activation function of the network. In many fundamental network models, the activation function is the error function. About neural networks read [ 15 ] , [ 16 ] , [ 17 ] .

2 Basics

We consider here the (Gauss) error special function ( [ 1 ] , [ 11 ] )

which is a sigmoidal type function and is a strictly increasing function.

It has the basic properties

We consider the activation function ( [ 10 ] )

which is an even function.

Next we follow [ 10 ] on \(\chi \). We got there \(\chi \left( 0\right) \simeq 0.4215\), and that \(\chi \) is strictly decreasing on \([0,\infty )\) and strictly increasing on \((-\infty ,0]\), and the \(x\)-axis is the horizontal asymptote on \(\chi ,\) i.e. \(\chi \) is a bell symmetric function.

We need the important

Denote by \(\left\lfloor \cdot \right\rfloor \) the integral part of the number and by \(\left\lceil \cdot \right\rceil \) the ceiling of the number.

Also from [ 10 ] we get

at least for some \(x\in \left[ a,b\right] \).

For large enough \(n\) we always obtain \(\left\lceil na\right\rceil \leq \left\lfloor nb\right\rfloor \). Also \(a\leq \tfrac {k}{n}\leq b\), iff \(\left\lceil na\right\rceil \leq k\leq \left\lfloor nb\right\rfloor \). In general it holds by (4) that

We introduce

It has the properties:

1. \(Z\left( x\right) {\gt}0\), \(\forall \) \(x\in \mathbb {R}^{N},\)

2. \begin{equation} \sum _{k=-\infty }^{\infty }\! \! \! \! Z\left( x-k\right) :=\! \! \! \! \sum _{k_{1}=-\infty }^{\infty }\sum _{k_{2}=-\infty }^{\infty }...\sum _{k_{N}=-\infty }^{\infty }\! \! \! \! Z\left( x_{1}-k_{1},...,x_{N}-k_{N}\right) =1,\text{ } \label{11} \end{equation}

   11

   where \(k:=\left( k_{1},...,k_{n}\right) \in \mathbb {Z}^{N}\), \(\forall \) \(x\in \mathbb {R}^{N},\)

   hence

3. \begin{align} & \sum _{k=-\infty }^{\infty }Z\left( nx-k\right) = \nonumber \\ & =\sum _{k_{1}=-\infty }^{\infty }\sum _{k_{2}=-\infty }^{\infty }...\sum _{k_{N}=-\infty }^{\infty }\! \! \! \! Z\left( nx_{1}-k_{1},...,nx_{N}-k_{N}\right) =1, \quad \forall x\in \mathbb {R}^{N};\ n\in \mathbb {N}, \label{12} \end{align}

   and

4. \begin{equation} \int _{\mathbb {R}^{N}}Z\left( x\right) {\rm d}x=1, \label{13} \end{equation}

   13

   that is \(Z\) is a multivariate density function.

Here \(\left\Vert x\right\Vert _{\infty }:=\max \left\{ \left\vert x_{1}\right\vert ,...,\left\vert x_{N}\right\vert \right\} \), \(x\in \mathbb {R}^{N}\), also set \(\infty :=\left( \infty ,...,\infty \right) \), \(-\infty :=\left( -\infty ,...,-\infty \right) \) upon the multivariate context, and

where \(a:=\left( a_{1},...,a_{N}\right) \), \(b:=\left( b_{1},...,b_{N}\right) .\)

We obviously see that

For \(0{\lt}\beta {\lt}1\) and \(n\in \mathbb {N}\), a fixed \(x\in \mathbb {R}^{N}\), we have that

In the last two sums the counting is over disjoint vector sets of \(k\)’s, because the condition \(\left\Vert \tfrac {k}{n}-x\right\Vert _{\infty }{\gt}\tfrac {1}{n^{\beta }}\) implies that there exists at least one \(\left\vert \tfrac {k_{r}}{n}-x_{r}\right\vert {\gt}\tfrac {1}{n^{\beta }}\), where \(r\in \left\{ 1,...,N\right\} .\)

We treat

when \(n^{1-\beta }\geq 3.\)

We have proved that

1. \begin{equation} \sum _{\stackrel{ k=\left\lceil na\right\rceil }{ \left\Vert \frac{k}{n}-x\right\Vert _{\infty }>\frac{1}{n^{\beta }}}}^{\left\lfloor nb\right\rfloor }Z\left( nx-k\right) \leq \frac{1}{2\sqrt{\pi }\left( n^{1-\beta }-2\right) e^{\left( n^{1-\beta }-2\right) ^{2}}}\text{,} \label{19} \end{equation}

   20

   \(0{\lt}\beta {\lt}1\), \(n\in \mathbb {N};n^{1-\beta }\geq 3\), \(x\in \left( \textstyle \prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] \right) .\)

By Theorem 3 clearly we obtain

That is,

1. it holds

   \begin{equation} 0<\tfrac {1}{\sum \limits _{k=\left\lceil na\right\rceil }^{\left\lfloor nb\right\rfloor }Z\left( nx-k\right) }<\tfrac {1}{\left( \chi \left( 1\right) \right) ^{N}}\simeq \left( 4.019\right) ^{N},\quad \forall x\in \left( \textstyle \prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] \right), \ n\in \mathbb {N}. \label{21} \end{equation}

   22

It is also clear that

1. \begin{equation} \sum _{\stackrel{ k=-\infty }{ \left\Vert \tfrac {k}{n}-x\right\Vert _{\infty }>\tfrac {1}{n^{\beta }}}}^{\infty }Z\left( nx-k\right) \leq \tfrac {1}{2\sqrt{\pi }\left( n^{1-\beta }-2\right) e^{\left( n^{1-\beta }-2\right) ^{2}}}, \label{22} \end{equation}

   23

   \(0{\lt}\beta {\lt}1\), \(n\in \mathbb {N}:n^{1-\beta }\geq 3\), \(x\in \left( \prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] \right) .\)

Also we get that

for at least some \(x\in \left( \textstyle \prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] \right) .\)

Let \(f\in C\left( \prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] \right) \) and \(n\in \mathbb {N}\) such that \(\left\lceil na_{i}\right\rceil \leq \left\lfloor nb_{i}\right\rfloor \), \(i=1,...,N.\)

We introduce and define the multivariate positive linear neural network operator (\(x:=\left( x_{1},...,x_{N}\right) \in \left( \prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] \right) \))

For large enough \(n\) we always obtain \(\left\lceil na_{i}\right\rceil \leq \left\lfloor nb_{i}\right\rfloor \), \(i=1,...,N\). Also \(a_{i}\leq \tfrac {k_{i}}{n}\leq b_{i}\), iff \(\left\lceil na_{i}\right\rceil \leq k_{i}\leq \left\lfloor nb_{i}\right\rfloor \), \(i=1,...,N\).

For convenience we call

\(\forall \) \(x\in \left( \textstyle \prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] \right) .\)

That is

\(\forall \) \(x\in \left( \textstyle \prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] \right) \), \(n\in \mathbb {N}.\)

Hence

Consequently we derive

\(\forall \) \(x\in \left( \textstyle \prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] \right) .\)

We will estimate the right hand side of (29).

For the last we need, for \(f\in C\left( \footnotesize \prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] \right) \) the first multivariate modulus of continuity

It holds that

Similarly it is defined for \(f\in C_{B}\left( \mathbb {R}^{N}\right) \) (continuous and bounded functions on \(\mathbb {R}^{N}\)) the \(\omega _{1}\left( f,h\right) \), and it has the property (31), given that \(f\in C_{U}\left( \mathbb {R}^{N}\right) \) (uniformly continuous functions on \(\mathbb {R}^{N}\)).

When \(f\in C_{B}\left( \mathbb {R}^{N}\right) \) we define,

\(n\in \mathbb {N}\), \(\forall \) \(x\in \mathbb {R}^{N},\) \(N\in \mathbb {N}\), the multivariate quasi-interpolation neural network operator.

Also for \(f\in C_{B}\left( \mathbb {R}^{N}\right) \) we define the multivariate Kantorovich type neural network operator

\(n\in \mathbb {N},\ \forall \) \(x\in \mathbb {R}^{N}.\)

Again for \(f\in C_{B}\left( \mathbb {R}^{N}\right) ,\) \(N\in \mathbb {N},\) we define the multivariate neural network operator of quadrature type \(D_{n}\left( f,x\right) \), \(n\in \mathbb {N},\) as follows. Let \(\theta =\left( \theta _{1},...,\theta _{N}\right) \in \mathbb {N}^{N},\) \(r=\left( r_{1},...,r_{N}\right) \in \mathbb {Z}_{+}^{N}\), \(w_{r}=w_{r_{1},r_{2},...r_{N}}\geq 0\), such that \(\sum \limits _{r=0}^{\theta }w_{r}=\sum \limits _{r_{1}=0}^{\theta _{1}}\sum \limits _{r_{2}=0}^{\theta _{2}}...\sum \limits _{r_{N}=0}^{\theta _{N}}w_{r_{1},r_{2},...r_{N}}=1;\) \(k\in \mathbb {Z}^{N}\) and

where \(\tfrac {r}{\theta }:=\left( \tfrac {r_{1}}{\theta _{1}},\tfrac {r_{2}}{\theta _{2}},...,\tfrac {r_{N}}{\theta _{N}}\right) .\)

We put

\(\forall \) \(x\in \mathbb {R}^{N}.\)

Let fixed \(j\in \mathbb {N}\), \(0{\lt}\beta {\lt}1\), and \(A,B{\gt}0\). For large enough \(n\in \mathbb {N}:n^{1-\beta }\geq 3,\) in the linear combination \(\bigg( \tfrac {A}{n^{\beta j}}+\tfrac {B}{\left( n^{1-\beta }-2\right) e^{\left( n^{1-\beta }-2\right) ^{2}}}\bigg) ,\) the dominant rate of convergence, as \(n\rightarrow \infty \), is \(n^{-\beta j}.\) The closer \(\beta \) is to \(1\) we get faster and better rate of convergence to zero.

Let \(f\in C^{m}\left( \prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] \right) \), \(m,N\in \mathbb {N}\). Here \(f_{\alpha }\) denotes a partial derivative of \(f\), \(\alpha :=\left( \alpha _{1},...,\alpha _{N}\right) \), \(\alpha _{i}\in \mathbb {Z}_{+}\), \(i=1,...,N\), and \(\left\vert \alpha \right\vert :=\sum \limits _{i=1}^{N}\alpha _{i}=l,\) where \(l=0,1,...,m\). We write also \(f_{\alpha }:=\tfrac {\partial ^{\alpha }f}{\partial x^{\alpha }}\) and we say it is of order \(l\).

We denote

Call also

\(\left\Vert \cdot \right\Vert _{\infty }\) is the supremum norm.

In this article we study the basic approximation properties of \(A_{n},B_{n},C_{n},D_{n}\) neural network operators and as well of their iterates. That is, the quantitative pointwise and uniform convergence of these operators to the unit operator \(I\). We study also the complex functions related approximation.

3 Multidimensional Real Neural Network Approximations

Here we present a series of neural network approximations to a function given with rates.

We give

Thus

So that

Now using (29) we finish proof.

We continue with

Hence

proving the claim.

We give

Thus it holds

We observe that

proving the claim.

We also present

proving the claim.

In the next we discuss high order of approximation by using the smoothness of \(f\).

We give

Then

for all \(j=0,1,...,m.\)

We have the multivariate Taylor’s formula

Notice \(g_{z}\left( 0\right) =f\left( x_{0}\right) \). Also for \(j=0,1,...,m\), we have

Furthermore

\(0\leq \theta \leq 1.\)

So we treat \(f\in C^{m}\left( \textstyle \prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] \right) \).

Thus, we have for \(\tfrac {k}{n},x\in \left( \textstyle \prod \limits _{i=1}^{N}\left[ a_{i},b_{i}\right] \right) \) that

where

We see that

Notice here that

We further see that

Conclusion: When \(\left\Vert \tfrac {k}{n}-x\right\Vert _{\infty }\leq \tfrac {1}{n^{\beta }}\), we proved that

In general we notice that

We proved in general that

Next we see that

Consequently

We have established that

We observe that

The last says

Clearly \(A_{n}^{\ast }\) is a positive linear operator.

Thus (here \(\alpha _{i}\in \mathbb {Z}^{+}:\left\vert \alpha \right\vert =\sum _{i=1}^{N}\alpha _{i}=j\))

So we have proved that

for all \(j=1,...,m.\)

At last we observe

Putting all of the above together we prove theorem.

We make

Clearly \(l_{nk}\left( f\right) \) is a positive linear functional such that \(\left\vert l_{nk}\left( f\right) \right\vert \leq \left\Vert f\right\Vert _{\infty }.\)

Hence \(F_{n}\left( f\right) \) is a positive linear operator with \(\left\Vert F_{n}\left( f\right) \right\Vert _{\infty }\leq \left\Vert f\right\Vert _{\infty }\), a continuous bounded linear operator.

We need

Next we prove the continuity of \(F_{n}\left( f\right) \). Notice for \(N=1\), \(Z=\chi \) by (10).

We will use the Weierstrass \(M\) test: If a sequence of positive constants \(M_{1},M_{2},M_{3},...,\) can be found such that in some interval

(a) \(\left| u_{n}\left( x\right) \right| \leq M_{n}\), \(n=1,2,3,...\)

(b) \(\sum M_{n}\) converges,

then \(\sum u_{n}\left( x\right) \) is uniformly and absolutely convergent in the interval.

Also we will use:

If \(\{ u_{n}\left( x\right) \} \), \(n=1,2,3,...\) are continuous in \(\left[ a,b\right] \) and if \(\sum u_{n}\left( x\right) \) converges uniformly to the sum \(S\left( x\right) \) in \(\left[ a,b\right] \), then \(S\left( x\right) \) is continuous in \(\left[ a,b\right] \). I.e. a uniformly convergent series of continuous functions is a continuous function. First we prove claim for \(N=1\).

We will prove that \(\sum _{k=-\infty }^{\infty }l_{nk}\left( f\right) \chi \left( nx-k\right) \) is continuous in \(x\in \mathbb {R}\).

There always exists \(\lambda \in \mathbb {N}\) such that \(nx\in \left[ -\lambda ,\lambda \right] .\)

Since \(nx\leq \lambda \), then \(-nx\geq -\lambda \) and \(k-nx\geq k-\lambda \geq 0\), when \(k\geq \lambda \). Therefore

So for \(k\geq \lambda \) we get

and

Hence by Weierstrass \(M\) test we obtain that \(\sum \limits _{k=\lambda }^{\infty }l_{nk}\left( f\right) \chi \left( nx-k\right) \) is uniformly and absolutely convergent on \(\left[ -\tfrac {\lambda }{n},\tfrac {\lambda }{n}\right] .\)

Since \(l_{nk}\left( f\right) \chi \left( nx-k\right) \) is continuous in \(x\), then \(\sum \limits _{k=\lambda }^{\infty }l_{nk}\left( f\right) \chi \left( nx-k\right) \) is continuous on \(\left[ -\tfrac {\lambda }{n},\tfrac {\lambda }{n}\right] .\)

Because \(nx\geq -\lambda \), then \(-nx\leq \lambda \), and \(k-nx\leq k+\lambda \leq 0\), when \(k\leq -\lambda \). Therefore

So for \(k\leq -\lambda \) we get

and

Hence by Weierstrass \(M\) test we obtain that \(\sum \limits _{k=-\infty }^{-\lambda }l_{nk}\left( f\right) \chi \left( nx-k\right) \) is uniformly and absolutely convergent on \(\left[ -\tfrac {\lambda }{n},\tfrac {\lambda }{n}\right] .\)

Since \(l_{nk}\left( f\right) \chi \left( nx-k\right) \) is continuous in \(x\), then \(\sum \limits _{k=-\infty }^{-\lambda }l_{nk}\left( f\right) \chi \left( nx-k\right) \) is continuous on \(\left[ -\tfrac {\lambda }{n},\tfrac {\lambda }{n}\right] .\)

So we proved that \(\sum \limits _{k=\lambda }^{\infty }l_{nk}\left( f\right) \chi \left( nx-k\right) \) and \(\sum \limits _{k=-\infty }^{-\lambda }l_{nk}\left( f\right) \chi \left( nx-k\right) \) are continuous on \(\mathbb {R}\). Since \(\sum \limits _{k=-\lambda +1}^{\lambda -1}l_{nk}\left( f\right) \chi \left( nx-k\right) \) is a finite sum of continuous functions on \(\mathbb {R}\), it is also a continuous function on \(\mathbb {R}\).

Writing

we have it as a continuous function on \(\mathbb {R}\). Therefore \(F_{n}\left( f\right) \), when \(N=1\), is a continuous function on \(\mathbb {R}\).

When \(N=2\) we have

(there always exist \(\lambda _{1},\lambda _{2}\in \mathbb {N}\) such that \(nx_{1}\in \left[ -\lambda _{1},\lambda _{1}\right] \) and \(nx_{2}\in \left[ -\lambda _{2},\lambda _{2}\right] \))

(For convenience call

Thus

Notice that the finite sum of continuous functions \(F\left( k_{1},k_{2},x_{1},x_{2}\right) \),

is a continuous function.

The rest of the summands of \(F_{n}\left( f,x_{1},x_{2}\right) \) are treated all the same way and similarly to the case of \(N=1\). The method is demonstrated as follows.

We will prove that

is continuous in \(\left( x_{1},x_{2}\right) \in \mathbb {R}^{2}\).

The continuous function

and

So by the Weierstrass \(M\) test we get that

is uniformly and absolutely convergent. Therefore it is continuous on \(\mathbb {R}^{2}.\)

Next we prove continuity on \(\mathbb {R}^{2}\) of

Notice here that

and

So the double series under consideration is uniformly convergent and continuous. Clearly \(F_{n}\left( f,x_{1},x_{2}\right) \) is proved to be continuous on \(\mathbb {R}^{2}.\)

Similarly reasoning one can prove easily now, but with more tedious work, that \(F_{n}\left( f,x_{1},...,x_{N}\right) \) is continuous on \(\mathbb {R}^{N} \), for any \(N\geq 1\). We choose to omit this similar extra work.

We give the condensed

Next we do iterated neural network approximation (see also [ 9 ] ).

We make

We give

We make

We give

We continue with

4 Complex Multivariate Neural Network Approximations

We make

We present

In the next we discuss high order of complex approximation by using the smoothness of \(f\).

We give