Inequalities for Information Potentials and Entropies

5 years ago

(original), Approximation Theory, ISI/JCR, paper

Abstract

We consider a probability distribution $(p_0(x), p_1(x), . . .)$ depending on a real parameter $x$. The associated information potential is $S(x):=∑_{k}p_{k}^2(x).$ The Rényi entropy and the Tsallis entropy of order $2$ can be expressed as $R(x) = − \log S(x)$ and $T(x) = 1 − S(x)$. We establish recurrence relations, inequalities and bounds for $S(x)$, which lead immediately to similar relations, inequalities and bounds for the two entropies. We show that some sequences $(R_n(x))_{n≥0}$ and $(T_n(x))_{n≥0}$, associated with sequences of classical positive linear operators, are concave and increasing. Two conjectures are formulated involving the information potentials associated with the Durrmeyer density of probability, respectively the Bleimann–Butzer–Hahn probability distribution.

Authors

Ana Maria Acu
Lucian Blaga University of Sibiu, Sibiu, Romania

Alexandra Măduța
Technical University of Cluj-Napoca, Cluj-Napoca, Romania

Diana Otrocol
Technical University of Cluj-Napoca, Cluj-Napoca, Romania
Tiberiu Popoviciu Institute of Numerical Analysis, Romanian Academy

Ioan Rașa
Technical University of Cluj-Napoca, Cluj-Napoca, Romania

Keywords

probability distribution; Rényi entropy; Tsallis entropy; information potential; functional equations; inequalities

References

see the expanding block below

PDF

https://www.mdpi.com/2227-7390/8/11/2056/pdf

Cite this paper as:

A.M. Acu, A. Măduța, D. Otrocol, I. Rașa, Inequalities for information potentials and entropies, 8 (2020) 11, pp. 2056, doi: 10.3390/math8112056

About this paper

Journal

Mathematics

Publisher Name

MDPI

DOI

http://doi.org/10.3390/math8112056

Print ISSN

Not available yet.

Online ISSN

ISSN 2227-7390,

Google Scholar Profile

1. Harremoës, P., Topsoe, F., Inequalities between entropy and index of coincidence derived from information diagrams. IEEE Trans. Inf. Theory 2001, 47, 2944–2960.
2. Harremoës, P., Binomial and Poisson distribution as maximum entropy distributions. IEEE Trans. Inf. Theory 2001, 47, 2039–2041.
3. Hillion, E., Concavity of entropy along binomial convolution. Electron. Commun. Probab. 2012, 17, 1–9.
4. Hillion, E., Johnson, O., A proof of the Shepp-Olkin entropy concavity conjecture. Bernoulli 2017, 23, 3638–3649.
5. Knessl, C., Integral representation and asymptotic expansions for Shannon and Renyi entropies. Appl. Math. Lett. 1998, 11, 69–74.
6. Adell, J.A., Lekuona, A., Yu, Y., Sharp bound on the entropy of the Poisson low and related quantities. IEEE Trans. Inf. Theory 2010, 56, 2299–2306.
7. Melbourne, J., Tkocz, T., Reversals of Rényi entropy inequalities under log-concavity. IEEE Trans. Inf. Theory 2020.
8. Shepp, L.A., Olkin, I., Entropy of the Sum of Independent Bernoulli Random Variables and of the Multinomial Distribution, Contributions to Probability A Collection of Papers Dedicated to Eugene Lukacs; Academic Press: London, UK, 1981; pp. 201–206.
9. Hillion, E., Johnson, O., Discrete versions of the transport equation and the Shepp-Olkin conjecture. Ann. Probab. 2016, 44, 276–306.
10. Alzer, H., A refinement of the entropy inequality. Ann. Univ. Sci. Bp. 1995, 38, 13–18.
11. Chang, S.-C., Weldon, E., Coding for T-user multiple-access channels. IEEE Trans. Inf. Theory 1979, 25, 684–691.
12. Xu, D. Energy, Entropy and Information Potential for Neural Computation. Ph.D. Thesis, University of Florida, Gainesville, FL, USA , 1999.
13. Barar, A., Mocanu, G. R., Rasa, I., Bounds for some entropies and special functions. Carpathian J. Math. 2018, 34, 9–15.
14. Principe, J.C., Information Theoretic Learning: Rényi’s Entropy and Kernel Perspectives; Springer: New York, NY, USA, 2010.
15. Acu, A.M., Bascanbaz-Tunca, G., Rasa, I., Information potential for some probability density functions. Appl. Math. Comput. 2021, 389, 125578.
16. Acu, A.M., Bascanbaz-Tunca, G., Rasa, I., Bounds for indices of coincidence and entropies. submitted.
17. Rasa, I., Entropies and Heun functions associated with positive linear operators. Appl. Math. Comput. 2015, 268, 422–431.
18. Baskakov, V.A., An instance of a sequence of positive linear operators in the space of continuous functions. Doklady Akademii Nauk SSSR 1957, 113, 249–251.
19. Berdysheva, E., Studying Baskakov–Durrmeyer operators and quasi-interpolants via special functions. J. Approx. Theory 2007, 149, 131–150.
20. Heilmann, M., Erhohung der Konvergenzgeschwindigkeit bei der Approximation von Funktionen mit Hilfe von Linearkombinationen Spezieller Positiver Linearer Operatoren; Habilitationschrift Universitat Dortmund: Dortmund, Germany, 1992.
21. Wagner, M., Quasi-Interpolaten zu genuinen Baskakov-Durrmeyer-Typ Operatoren; Shaker: Aachen, Germany, 2013.
22. Acu, A.M., Heilmann, M., Rasa, I., Linking Baskakov Type Operators, Constructive theory of functions, Sozopol 2019; Draganov, B., Ivanov, K., Nikolov, G., Uluchev, R., Eds.; Prof. Marin Drinov Publishing House of BAS: Sofia, Bulgaria, 2020; pp. 23–38.
23. Heilmann, M., Rasa, I., A nice representation for a link between Baskakov and Szász-Mirakjan-Durrmeyer operators and their Kantorovich variants. Results Math. 2019, 74, 9.
24. Rasa, I., Rényi entropy and Tsallis entropy associated with positive linear operators. arXiv 2014, arXiv:1412.4971v.1.
25. Nikolov, G., Inequalities for ultraspherical polynomials. Proof of a conjecture of I. Ra¸sa. J. Math. Anal. Appl. 2014, 418, 852–860.
26. Gavrea, I., Ivan, M., On a conjecture concerning the sum of the squared Bernstein polynomials. Appl. Math. Comput. 2014, 241, 70–74.
27. Alzer, H., Remarks on a convexity theorem of Rasa. Results Math. 2020, 75, 29.
28. Cover, T.M., Thomas, J.A., Elements of Information Theory; John Wiley & Sons: Hoboken, NJ, USA, 2006.
29. Abramowitz, M., Stegun, I.A., Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables; Dover Publications, Inc.: New York, NY, USA, 1970.
30. Barar, A., Mocanu, G., Rasa, I., Heun functions related to entropies. RACSAM 2019, 113, 819–830.
31. Rasa, I., Convexity properties of some entropies (II). Results Math. 2019, 74, 154.
32. Rasa, I., Convexity properties of some entropies. Results Math. 2018, 73, 105.
33. Cloud, M.J., Drachman, B.C., Inequalities: With Applications to Engineering; Springer: Berlin/Heidelberg, Germany, 2006.
34. Abel, U., Gawronski, W., Neuschel, T., Complete monotonicity and zeros of sums of squared Baskakov functions. Appl. Math. Comput. 2015, 258, 130–137.
35. Gould, H.W., Combinatorial Identities—A Standardized Set of Tables Listing 500 Binomial Coefficient Summations; West Virginia University Press: Morgantown, VA, USA, 1972.
36. Altomare, F., Campiti, M., Korovkin-Type Approximation Theory and Its Applications; Walter de Gruyter: Berlin, Germany; New York, NY, USA, 1994.

Inequalities for Information Potentials and Entropies

Ana Maria Acu Lucian Blaga University of Sibiu, Department of Mathematics and Informatics, Str. Dr. I. Ratiu, No. 5-7, Sibiu RO-550012, Romania anamaria.acu@ulbsibiu.ro , Alexandra Măduţa Technical University of Cluj-Napoca, Department of Mathematics, 28 Memorandumului Street, 400114 Cluj-Napoca, Romania boloca.alexandra91@yahoo.com , Diana Otrocol Technical University of Cluj-Napoca, Department of Mathematics, 28 Memorandumului Street, 400114 Cluj-Napoca, Tiberiu Popoviciu Institute of Numerical Analysis, Romanian Academy, P.O.Box. 68-1, 400110 Cluj-Napoca, Romania Diana.Otrocol@math.utcluj.ro and Ioan Raşa Technical University of Cluj-Napoca, Department of Mathematics, 28 Memorandumului Street, 400114 Cluj-Napoca, Romania Ioan.Rasa@math.utcluj.ro

Abstract.

We consider a probability distribution $\left(p_{0}(x),p_{1}(x),\ldots\right)$ depending on a real parameter $x$ . The associated information potential is $S(x):=\sum\limits_{k}p_{k}^{2}(x)$ . The Rényi entropy and the Tsallis entropy of order 2 can be expressed as $R(x)=-\log S(x)$ and $T(x)=1-S(x)$ . We establish recurrence relations, inequalities and bounds for $S(x)$ , which lead immediately to similar relations, inequalities and bounds for the two entropies. We show that some sequences $\left(R_{n}(x)\right)_{n\geq 0}$ and $\left(T_{n}(x)\right)_{n\geq 0}$ , associated with sequences of classical positive linear operators, are concave and increasing. Two conjectures are formulated involving the information potentials associated with the Durrmeyer density of probability, respectively the Bleimann–Butzer–Hahn probability distribution.
MSC: 39B22, 39B62, 94A17, 26D07.
Keywords: probability distribution; Rényi entropy; Tsallis entropy; information potential; functional equations; inequalities.

1. Introduction

Entropies associated with discrete or continuous probability distributions are usually described by complicated explicit expressions, depending on one or several parameters. Therefore, it is useful to establish lower and upper bounds for them. Convexity-type properties are also useful: they embody valuable information on the behavior of the functions representing the entropies.

This is why bounds and convexity-type properties of entropies, expressed by inequalities, are under an active study: see [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13] and the references therein. Our paper is concerned with this kind of inequalities: we give new results and new proofs or improvements of some existing results, in the framework which is presented below.

Let $\left(p_{0}(x),p_{1}(x),\ldots\right)$ be a probability distribution depending on a parameter $x\in I$ , where $I$ is a real interval. The associated information potential (also called index of coincidence, for obvious probabilistic reasons) is defined (see [14]),

(1.1)

S(x):=\sum\limits_{k}p_{k}^{2}(x),\ x\in I.

If $p(t,x),\ t\in\mathbb{R},\ x\in I$ , is a probability density function depending on the parameter $x,$ the associated information potential is defined as (see [14]),

(1.2)

S(x):=\int\nolimits_{\mathbb{R}}p^{2}(t,x)dt,\ x\in I.

The information potential is the core concept of the book [14]. The reader can find properties, extensions, generalizations of $S(x)$ , as well as applications to Information theoretic learning. Other properties and applications can be found in the recent papers [15] and [16].

It is important to remark that the Rényi entropy and the Tsallis entropy can be expressed in terms of $S(x)$ as

(1.3)

R(x)=-\log S(x),\ T(x)=1-S(x),\ x\in I.

So the properties of $S(x)$ lead immediately to properties of $R(x),$ respectively $T(x)$ .

On the other hand, we can consider the discrete positive linear operators

(1.4)

Lf(x):=\sum\limits_{k}p_{k}(x)f(x_{k}),\ x\in I,

where $x_{k}$ are given points in $\mathbb{R}$ , and the integral operators

(1.5)

Mf(x)=\int\nolimits_{\mathbb{R}}p(t,x)f(t)dt,\ x\in I.

In both cases, $f$ is a function from a suitable set of functions defined on $\mathbb{R}$ . In this paper, we consider classical operators of this kind, which are used in approximation theory.

Let us mention that the “degree of nonmultiplicativity” of the operator $L$ can be estimated in terms of the information potential $S(x)$ : see [17] and the references therein.

In this paper, we will be concerned with a special family of discrete probability distributions, described as follows.

Let $c\in\mathbb{R}$ . Set $I_{c}=\left[0,-\frac{1}{c}\right]$ if $c<0$ , and $I_{c}=[0,+\infty)$ if $c\geq 0$ . For $\alpha\in\mathbb{R}$ and $k\in\mathbb{N}_{0}$ the binomial coefficients are defined as usual by

{\binom{\alpha}{k}}:=\frac{\alpha(\alpha-1)\dots(\alpha-k+1)}{k!}\quad\mbox{if }k\in\mathbb{N},\mbox{ and }{\binom{\alpha}{0}}:=1.

Let $n>0$ be a real number, $k\in\mathbb{N}_{0}$ and $x\in I_{c}$ . Define

(1.6)

p_{n,k}^{[c]}(x):=(-1)^{k}{\tbinom{-\frac{n}{c}}{k}}(cx)^{k}(1+cx)^{-\frac{n}{c}-k},\quad\mbox{ if }c\neq 0,

(1.7)

p_{n,k}^{[0]}(x):=\lim_{c\rightarrow 0}p_{n,k}^{[c]}(x)=\frac{(nx)^{k}}{k!}e^{-nx},\quad\mbox{ if }c=0.

Then $\sum_{k=0}^{\infty}p_{n,k}^{[c]}(x)=1$ . Suppose that $n>c$ if $c\geq 0$ , or $n=-cl$ with some $l\in\mathbb{N}$ if $c<0$ .

With this notation, we consider the discrete distribution of probability $\left(p_{n,k}^{[c]}(x)\right)_{k=0,1,...}$ depending on the parameter $x\in I_{c}.$

According to (1.1), the associated information potential, or index of coincidence, is

(1.8)

S_{n,c}(x):=\sum\limits_{k=0}^{\infty}\left(p_{n,k}^{[c]}(x)\right)^{2},\ x\in I_{c}.

The Rényi entropy and the Tsallis entropy corresponding to the same distribution of probability are defined, respectively (see (1.3))

(1.9)

R_{n,c}(x)=-\log S_{n,c}(x)

and

(1.10)

T_{n,c}(x)=1-S_{n,c}(x).

For $c=-1$ (1.6) reduces to the binomial distribution and (1.8) becomes

(1.11)

S_{n,-1}(x):=\sum\limits_{k=0}^{n}\left({\dbinom{n}{k}}x^{k}(1-x)^{n-k}\right)^{2},\ x\in[0,1].

The case $c=0$ corresponds to the Poisson distribution (see (1.7)), which

(1.12)

S_{n,0}(x):=e^{-2nx}\sum\limits_{k=0}^{\infty}\dfrac{(nx)^{2k}}{\left(k!\right)^{2}},\ x\geq 0.

For $c=1$ , we have the negative binomial distribution, with

(1.13)

S_{n,1}(x):=\sum\limits_{k=0}^{\infty}\left({\dbinom{n+k-1}{k}}x^{k}(1+x)^{-n-k}\right)^{2},\ x\geq 0.

The binomial, Poisson, respectively negative binomial distributions correspond to the classical Bernstein, Szász-Mirakyan, respectively Baskakov operators from approximation theory; all of them are of the form (1.4). In fact, the distribution $\left(p_{n,k}^{[c]}(x)\right)_{k=0,1,...}$ is instrumental for the construction of the family of positive linear operators introduced by Baskakov in [18]; see also [19, 20, 21, 22, 23]. As a probability distribution, the family of functions $(p_{n,k}^{[c]})_{k=0,1,\dots}$ was considered in [24, 17].

The distribution

(1.14)

\left(\dbinom{n}{k}x^{k}(1+x)^{-n}\right)_{k=0,1,\ldots,n},\ x\in[0,+\infty),

corresponds to the Bleimann–Butzer–Hahn operators, while

(1.15)

\left(\dbinom{n+k}{k}x^{k}(1-x)^{n+1}\right)_{k=0,1,\ldots},\ x\in[0,1),

is connected with the Meyer-König and Zeller operators.

The information potentials and the entropies associated with all these distributions were studied in [17]; see also [25, 26, 27]. It should be mentioned that they satisfy Heun-type differential equations: see [17]. We continue this study. To keep the same notation as in [17], let us return to (1.11)–(1.13) and denote

\displaystyle F_{n}(x)

\displaystyle:=S_{n,-1}(x),\ G_{n}(x):=S_{n,1}(x),\ K_{n}(x):=S_{n,0}(x).

Moreover, the information potential corresponding to (1.14) and (1.15) will be denoted by

(1.16)		$\displaystyle U_{n}(x)$	$\displaystyle:={\textstyle\sum\limits_{k=0}^{n}}\left(\dbinom{n}{k}x^{k}(1+x)^{-n}\right)^{2},\ x\in[0,+\infty),$
(1.17)		$\displaystyle J_{n}(x)$	$\displaystyle:={\textstyle\sum\limits_{k=0}^{\infty}}\left(\dbinom{n+k}{k}x^{k}(1-x)^{n+1}\right)^{2},\ x\in[0,1).$

In Section 2, we present several relations between the functions $F_{n}(x)$ , $G_{n}(x),\ U_{n}(x),\ J_{n}(x)$ , as well as between these functions and the Legendre polynomials. By using the three-terms recurrence relations involving the Legendre polynomials, we establish recurrence relations involving three consecutive terms from the sequences $\left(F_{n}(x)\right)$ , $\left(G_{n}(x)\right),\ \left(U_{n}(x)\right),\$ respectively $\left(J_{n}(x)\right)$ . We recall also some explicit expressions of these functions.

Section 3 is devoted to inequalities between consecutive terms of the above sequences; in particular, we emphasize that for fixed $x$ , the four sequences are logarithmicaly convex and hence convex.

Other inequalities are presented in Section 4. All the inequalities can be used to get information about the Rényi entropies and Tsallis entropies connected with the corresponding probability distributions.

Section 5 contains new properties of the function $U_{n}(x)$ and a problem of its shape.

Section 6 is devoted to some inequalities involving integrals of the form $\int_{a}^{b}f^{\prime 2}(x)dx$ in relation with certain combinatorial identities.

The information potential associated with the Durrmeyer density of probability is computed in Section 7. We recall a conjecture formulated in [24].

As already mentioned, all the results involving the information potential can be used to derive results about Rényi and Tsallis entropies. For the sake of brevity, we will study usually only the information potential.

Concerning the applications of Rényi entropies and Tsallis entropies see, e.g., [14, 28].

2. Recurrence Relations

$F_{n}(x)$ is a polynomial, $G_{n}(x),\ U_{n}(x),\ J_{n}(x)$ are rational functions. On their maximal domains, these functions are connected by several relations (see [17], Cor. 13, (46), (53), (54)):

(2.1)	$\displaystyle F_{n}(x)$	$\displaystyle=(1-2x)^{2n+1}G_{n+1}(-x),$
(2.2)	$\displaystyle F_{n}(x)$	$\displaystyle=U_{n}\left(\frac{x}{1-x}\right),$
(2.3)	$\displaystyle F_{n}(x)$	$\displaystyle=-(1-2x)^{2n+1}J_{n}\left(\frac{x-1}{x}\right).$

Consider the Legendre polynomial (see [29], 22.3.1)

(2.4)

P_{n}(t)=2^{-n}{\textstyle\sum\limits_{k=0}^{n}}\dbinom{n}{k}^{2}(x+1)^{k}(x-1)^{n-k}.

Then (see [17], (39))

(2.5)

P_{n}(t)=(1-2x)^{-n}F_{n}(x),

where

t=\frac{1-2x+2x^{2}}{1-2x},\ x\in[0,\frac{1}{2}).

Combining (2.5) with (2.1), (2.2) and (2.3), we get

(2.6)	$\displaystyle P_{n}(t)$	$\displaystyle=(1-2x)^{n+1}G_{n+1}(-x),$
(2.7)	$\displaystyle P_{n}(t)$	$\displaystyle=(1-2x)^{-n}U_{n}\left(\frac{x}{1-x}\right),$
(2.8)	$\displaystyle P_{n}(t)$	$\displaystyle=-(1-2x)^{n+1}J_{n}\left(\frac{x-1}{x}\right).$

In the theory of special functions recurrence, relations play a crucial role. In particular, the Legendre polynomials (2.4) satisfy the important recurrence relation ([29], 22.7.1)

(2.9)

(n+1)P_{n+1}(t)-(2n+1)tP_{n}(t)+nP_{n-1}(t)=0.

This leads us to

Theorem 1.

The functions $F_{n}(x),G_{n}(x),\ U_{n}(x)$ and $\ J_{n}(x)\$ satisfy the following three-terms recurrence relations:

(2.10)	$\displaystyle 2(n+1)F_{n+1}(x)$	$\displaystyle=(2n+1)(1+(1-2x)^{2})F_{n}(x)-2n(1-2x)^{2}F_{n-1}(x),$
(2.11)	$\displaystyle n(1+2x)^{2}\!G_{n+1}(x)\!$	$\displaystyle=\!(2n-1)(1+2x+2x^{2})\!G_{n}(x)\!-\!(n-1)G_{n-1}(x),$
(2.12)	$\displaystyle(n+1)(1+t)^{2}U_{n+1}(t)$	$\displaystyle=(2n+1)(t^{2}+1)U_{n}(t)-n(1-t)^{2}U_{n-1}(t),$
(2.13)	$\displaystyle(n+1)(1+t)^{2}J_{n+1}(t)$	$\displaystyle=(2n+1)(t^{2}+1)J_{n}(t)-n(1-t)^{2}J_{n-1}(t).$

Proof.

It suffices to use relations (2.5)–(2.9). ∎

Remark 2.

According to (2.12) and (2.13), $U_{n}(x)$ and $J_{n}(x)\$ satisfy the same recurrence relation. In fact,from ([17], (49), (55)), we have

(2.14)		$\displaystyle U_{n}(x)$	$\displaystyle={\textstyle\sum\limits_{k=0}^{n}}c_{n,k}\left(\frac{1-x}{1+x}\right)^{2k},$
(2.15)		$\displaystyle J_{n}(x)$	$\displaystyle={\textstyle\sum\limits_{k=0}^{n}}c_{n,k}\left(\frac{1-x}{1+x}\right)^{2k+1},$

where

c_{n,k}:=4^{-n}\dbinom{2(n-k)}{n-k}\dbinom{2k}{k},\ k=0,1,\ldots,n.

From (2.14) and (2.15), we see that

(2.16)

J_{n}(x)=\frac{1-x}{1+x}U_{n}(x).

Remark 3.

From ([17], (56)) and ([30], (21)), we know that

(2.17)		$\displaystyle G_{n+1}(x)$	$\displaystyle={\textstyle\sum\limits_{k=0}^{n-1}}c_{n,k}(1+2x)^{-2k-1},$
(2.18)		$\displaystyle F_{n}(x)$	$\displaystyle={\textstyle\sum\limits_{k=0}^{n}}c_{n,k}(1-2x)^{2k}.$

So, the recurrence relations (2.10)–(2.13) are accompanied by

	$\displaystyle F_{0}(x)$	$\displaystyle=1,\ F_{1}(x)=1-2x+2x^{2};$
	$\displaystyle G_{1}(x)$	$\displaystyle=\frac{1}{2x+1},\ G_{2}(x)=\frac{1+2x+2x^{2}}{(2x+1)^{3}};$
	$\displaystyle U_{0}(x)$	$\displaystyle=1,\ U_{1}(x)=\frac{1+x^{2}}{(1+x)^{2}};$
	$\displaystyle J_{0}(x)$	$\displaystyle=\frac{1-x}{1+x},\ J_{1}(x)=\frac{(1-x)(1+x^{2})}{(1+x)^{3}}.$

3. Inequalities for Information Potentials

In studying a sequence of special functions, not only are recurrence relations important, but also inequalities connecting successive terms; in particular, inequalities showing that the sequence is (logarithmically) convex or concave. This section is devoted to such inequalities involving the sequences $(F_{n}(x))$ , $(G_{n}(x))$ , $(U_{n}(x))$ , and $(J_{n}(x))$ .

Theorem 4.

The function $F_{n}(x)$ satisfies the inequalities

(3.1)

F_{n+1}(x)\leq\frac{1+(4n-2)x(1-x)}{1+(4n+2)x(1-x)}F_{n-1}(x),

(3.2)

F_{n}(x)\leq\frac{1+4nx(1-x)}{1+(4n+2)x(1-x)}F_{n-1}(x),

(3.3)

F_{n}^{2}(x)\leq F_{n-1}(x)F_{n+1}(x);\ 2F_{n}(x)\leq F_{n-1}(x)+F_{n+1}(x),

for all $n\geq 1,\ x\in[0,1].$

Proof.

We start with the following integral representation (see [17], (29))

(3.4)

F_{n}(x)=\frac{1}{\pi}\int_{0}^{1}f^{n}(x,t)\frac{dt}{\sqrt{t(1-t)}},

where $f(x,t):=t+(1-t)(1-2x)^{2}\in[0,1].$

It follows that

F_{n+1}(x)\leq F_{n}(x).

On the other hand,

	$\displaystyle F_{n-1}(x)+F_{n+1}(x)-2F_{n}(x)=$
	$\displaystyle=\frac{1}{\pi}\int_{0}^{1}f^{n-1}(x,t)\left[1+f^{2}(x,t)-2f(x,t)\right]\!\!\frac{dt}{\sqrt{t(1-t)}},$

which entails

(3.5)

2F_{n}(x)\leq F_{n-1}(x)+F_{n+1}(x).

According to (2.10), we have

(3.6)

F_{n}(x)=a_{n}(x)F_{n-1}(x)+b_{n}(x)F_{n+1}(x),

where

a_{n}(x)=\frac{n(1-2x)^{2}}{(2n+1)(1-2x+2x^{2})},\ b_{n}(x)=\frac{n+1}{(2n+1)(1-2x+2x^{2})}.

Using (3.5) and (3.6) we get

2a_{n}(x)F_{n-1}(x)+2b_{n}(x)F_{n+1}(x)\leq F_{n-1}(x)+F_{n+1}(x),

which yields

(2b_{n}(x)-1)F_{n+1}(x)\leq(1-2a_{n}(x))F_{n-1}(x),

and this immediately leads to (3.1). To prove (3.2), it suffices to combine (3.5) and (3.1). The inequalities (3.3) were proven in ([31], (3.2) and (3.3)). ∎

Combining (3.2) with (1.9) and (1.10), we obtain

Corollary 5.

The Rényi entropy $R_{n}(x)$ and the Tsallis entropy $T_{n}(x)$ corresponding to the binomial distribution with parameters $n$ and $x$ satisfy the inequalities:

(3.7)		$\displaystyle R_{n}(x)-R_{n-1}(x)$	$\displaystyle\geq\log\frac{1+(4n+2)x(1-x)}{1+4nx(1-x)}\geq 0,$
(3.8)		$\displaystyle T_{n}(x)-T_{n-1}(x)$	$\displaystyle\geq\frac{2x(1-x)}{1+4nx(1-x)}\left(1-T_{n}(x)\right)\geq 0.$

Theorem 6.

The following inequalities hold:

(3.9)	$\displaystyle U_{n}^{2}$	$\displaystyle\leq U_{n-1}U_{n+1},\ 2U_{n}\leq U_{n-1}+U_{n+1},$
(3.10)	$\displaystyle U_{n+1}(x)$	$\displaystyle\leq\frac{1+4nx+x^{2}}{1+(4n+4)x+x^{2}}U_{n-1}(x),$
(3.11)	$\displaystyle U_{n}(x)$	$\displaystyle\leq\frac{1+(4n+2)x+x^{2}}{1+(4n+4)x+x^{2}}U_{n-1}(x),$
(3.12)	$\displaystyle G_{n}^{2}$	$\displaystyle\leq G_{n-1}G_{n+1},\ 2G_{n}\leq G_{n-1}+G_{n+1},$
(3.13)	$\displaystyle G_{n+1}(x)$	$\displaystyle\leq\frac{1+(4n-2)x(1+x)}{1+(4n+2)x(1+x)}G_{n-1}(x),$
(3.14)	$\displaystyle G_{n}(x)$	$\displaystyle\leq\frac{1+4nx(x+1)}{1+(4n+2)x(x+1)}G_{n-1}(x),$
(3.15)	$\displaystyle J_{n}^{2}$	$\displaystyle\leq J_{n-1}J_{n+1},\ 2J_{n}\leq J_{n-1}+J_{n+1},$
(3.16)	$\displaystyle J_{n+1}(x)$	$\displaystyle\leq\frac{1+4nx+x^{2}}{1+(4n+4)x+x^{2}}J_{n-1}(x),$
(3.17)	$\displaystyle J_{n}(x)$	$\displaystyle\leq\frac{1+(4n+2)x+x^{2}}{1+(4n+4)x+x^{2}}J_{n-1}(x).$

Proof.

The proof is similar to that of Theorem 4, starting from (see ([17], (48), (58), (63))):

	$\displaystyle U_{n}(x)$	$\displaystyle=\frac{1}{\pi}\int\nolimits_{0}^{1}\left(t+(1-t)\left(\frac{1-x}{1+x}\right)^{2}\right)^{n}\frac{dt}{\sqrt{t(1-t)}},$
	$\displaystyle G_{n}(x)$	$\displaystyle=\frac{1}{\pi}\int\nolimits_{0}^{1}\left(t+(1-t)(1+2x)^{2}\right)^{-n}\frac{dt}{\sqrt{t(1-t)}},$
	$\displaystyle J_{n}(x)$	$\displaystyle=\frac{1}{\pi}\int\nolimits_{0}^{1}\left(t+(1-t)\left(\frac{1+x}{1-x}\right)^{2}\right)^{-n-1}\!\!\!\!\frac{dt}{\sqrt{t(1-t)}}.$

∎

These integral representations, together with the representation of $F_{n}(x)$ given by (3.4), are consequences of the important results of Elena Berdysheva ([19], Theorem 1).

From (3.9)–(3.17), we can derive inequalities similar to (3.7) and (3.8), for the entropies associated with the probability distributions corresponding to $U_{n}(x),\ G_{n}(x)$ , and $J_{n}(x)$ .

Remark 7.

Let us remark that the inequalities (3.3), (3.9), (3.12), (3.15) show that for each $x$ , the sequences $(F_{n}(x))_{n\geq 0},\ (U_{n}(x))_{n\geq 0},\ (G_{n}(x))_{n\geq 0},\ (J_{n}(x))_{n\geq 0},$ are logarithmically convex, and so convex; the other inequalities from Theorems 4 and 6 show that the same sequences are decreasing. It immediately follows that the associated sequences of entropies $(R_{n}(x))_{n\geq 0}$ and $(T_{n}(x))_{n\geq 0}$ are concave and increasing; see also (3.7) and (3.8).

4. Other Inequalities

Besides their own interest, the next Theorems 8 and 12 will be instrumental in establishing new lower and upper bounds for the information potentials $(F_{n}(x))_{n\geq 0},\ (U_{n}(x))_{n\geq 0},\ (G_{n}(x))_{n\geq 0},\ (J_{n}(x))_{n\geq 0},$ and consequently for the associated Rényi and Tsallis entropies.

Let us return to the information potential (1.8). According to ([17], (10)) we have for $c\neq 0,$

(4.1)

S_{n,c}(x)=\frac{1}{\pi}\int\nolimits_{0}^{1}\left[t+(1-t)(1+2cx)^{2}\right]^{-\frac{n}{c}}\frac{dt}{\sqrt{t(1-t)}}.

Let $c<0$ . Using (4.1) and Chebyshev’s inequality for synchronous functions, we can write

	$\displaystyle S_{n-c,c}(x)$	$\displaystyle=\frac{1}{\pi}\!\!\int\nolimits_{0}^{1}\!\!\left[t\!+\!(1\!-\!t)(1\!+\!2cx)^{2}\right]^{-\frac{n}{c}}\!\!\left[t\!+\!(1\!-\!t)(1\!+\!2cx)^{2}\right]\frac{dt}{\sqrt{t(1\!\!-\!\!t)}}$
		$\displaystyle\geq\frac{1}{\pi}\int\nolimits_{0}^{1}\left[t+(1-t)(1+2cx)^{2}\right]^{-\frac{n}{c}}\frac{dt}{\sqrt{t(1-t)}}$
		$\displaystyle\quad\cdot\frac{1}{\pi}\int\nolimits_{0}^{1}\left[t+(1-t)(1+2cx)^{2}\right]\frac{dt}{\sqrt{t(1-t)}}$
		$\displaystyle=S_{n,c}(x)(1+2cx+2c^{2}x^{2}).$

For $c>0$ , we use Chebyshev’s inequality for asynchronous functions and obtain the reverse inequality. So we have

Theorem 8.

If $c<0$ , then

(4.2)

S_{n-c,c}(x)\geq(1+2cx(1+cx))S_{n,c}(x).

If $c>0$ , the inequality is reversed.

Corollary 9.

For $c=-1,$ (4.2) and (3.2) yield

(4.3)

(1-2x(1-x))F_{n}(x)\leq F_{n+1}(x)\leq\frac{1+(4n+4)x(1-x)}{1+(4n+6)x(1-x)}F_{n}(x).

For $c=1$ , we obtain

(4.4)

\frac{\ 1}{1+2x(1+x)}G_{n}(x)\leq G_{n+1}(x)\leq\dfrac{1+(4n+4)x(1+x)}{1+(4n+6)x(1+x)}G_{n}(x).

Now, using [17, (48)], we have

	$\displaystyle U_{n+1}(x)$	$\displaystyle=\frac{1}{\pi}\int\nolimits_{0}^{1}\left(\left(\frac{1-x}{1+x}\right)^{2}+\frac{4x}{(1+x)^{2}}t\right)^{n}\cdot$
		$\displaystyle\quad\cdot\left(\left(\frac{1-x}{1+x}\right)^{2}+\frac{4x}{(1+x)^{2}}t\right)\frac{dt}{\sqrt{t(1-t)}}$
		$\displaystyle\geq U_{n}(x)\frac{1}{\pi}\int\nolimits_{0}^{1}\left(\left(\frac{1-x}{1+x}\right)^{2}+\frac{4x}{(1+x)^{2}}t\right)\frac{dt}{\sqrt{t(1-t)}}$
		$\displaystyle=\frac{1+x^{2}}{(1+x)^{2}}U_{n}(x).$

Therefore, using also (3.11), we get

(4.5)

\frac{1+x^{2}}{(1+x)^{2}}U_{n}(x)\leq U_{n+1}(x)\leq\dfrac{1+(4n+6)x+x^{2}}{1+(4n+8)x+x^{2}}U_{n}(x).

Now (4.5) and (2.16) yield

(4.6)

\frac{1+x^{2}}{\left(1+x\right)^{2}}J_{n}(x)\leq J_{n+1}(x)\leq\dfrac{1+(4n+6)x+x^{2}}{1+(4n+8)x+x^{2}}J_{n}(x).

Theorem 10.

The following inequalities are satisfied:

(4.7)

(1-2x(1-x))^{n}\leq F_{n}(x)\leq\sqrt{\dfrac{1+4x(1-x)}{1+(4n+4)x(1-x)}},\,n\geq 0,\,x\in[0,1],

(4.8)

\left(\dfrac{1}{1+2x(1+x)}\right)^{n-1}\leq(1+2x)G_{n}(x)\leq\sqrt{\dfrac{1+8x(1+x)}{1+(4n+4)x(1+x)}},\,n\geq 1,x\geq 0,

(4.9)

\left(\dfrac{1+x^{2}}{(1+x)^{2}}\right)^{n}\leq U_{n}(x)\leq\sqrt{\dfrac{1+6x+x^{2}}{1+(4n+6)x+x^{2}}},\,\,n\geq 0,\,x\geq 0,

(4.10)

\dfrac{1-x}{1+x}\left(\dfrac{1+x^{2}}{(1+x)^{2}}\right)^{n}\leq J_{n}(x)\leq\dfrac{1-x}{1+x}\sqrt{\dfrac{1+6x+x^{2}}{1+(4n+6)x+x^{2}}},\,n\geq 0,\,x\in[0,1].

Proof.

Writing (4.3) for $n=0,1,\dots,m-1$ , and multiplying term by term, we get

\left(1-2x(1-x)\right)^{m}\leq F_{m}(x)\leq\dfrac{1+4x(1-x)}{1+6x(1-x)}\dfrac{1+8x(1-x)}{1+10x(1-x)}\cdots\dfrac{1+4mx(1-x)}{1+(4m+2)x(1-x)}.

Using

\dfrac{1+tx(1-x)}{1+(t+2)x(1-x)}\leq\dfrac{1+(t+2)x(1-x)}{1+(t+4)x(1-x)},\,\,t\geq 0,

, it follows that

F_{m}^{2}(x)\leq\dfrac{1+4x(1-x)}{1+(4m+4)x(1-x)},

and so (4.7) is proven. The other three relations can be proved similarly, using (4.4), (4.5) and (4.6). ∎

Remark 11.

The inequalities (4.7)–(4.10) in Theorem 10 provide lower and upper bounds for the information potentials $F_{n}$ , $G_{n}$ , $U_{n}$ , $J_{n}$ , and consequently for the associated entropies. They can be compared with other bounds existing in the literature, obtained with other methods. For the moment, let us prove the inequality

(4.11)

(1-4x(1-x))^{n/2}\leq F_{n}(x),\,n\geq 0,\,x\in[0,1],

and compare it with the first inequality in (4.7).

According to ([17], (4.6), (4.2)),

F_{n}(x)=\displaystyle\sum_{j=0}^{n}c_{n,j}(1-2x)^{2j},

where (see also (6.11) and (6.8))

c_{n,j}:=4^{-n}{2j\choose j}{2n-2j\choose n-j},\,\,\sum_{j=0}^{n}c_{n,j}=1,\,\sum_{j=0}^{n}jc_{n,j}=\dfrac{n}{2}.

Using the weighted arithmetic mean-geometric mean inequality, we have

F_{n}(x)\geq\prod_{j=0}^{n}(1-2x)^{2jc_{n,j}}=(1-2x)^{2\sum_{j=0}^{n}jc_{n,j}}=(1-4x(1-x))^{n/2},

and this is (4.11). Clearly, the first inequality in (4.7) provides a lower bound for $F_{n}(x)$ , which is better than the lower bound provided by (4.11).

Theorem 12.

The information potential satisfies the following inequality for all $c\in\mathbb{R}$ :

(4.12)

S_{m+n,c}(x)\geq S_{m,c}(x)S_{n,c}(x).

Proof.

If $c\neq 0,$ we can use (4.1) to get

	$\displaystyle S_{m+n,c}(x)=$	$\displaystyle\frac{1}{\pi}{\textstyle\displaystyle\int\nolimits_{0}^{1}}\left[t+(1-t)(1+2cx)^{2}\right]^{-\frac{m}{c}}\cdot$
		$\displaystyle\quad\cdot\left[t+(1-t)(1+2cx)^{2}\right]^{-\frac{n}{c}}\frac{dt}{\sqrt{t(1-t)}}.$

Applying Chebyshev’s inequality for synchronous functions, we obtain

	$\displaystyle S_{m+n,c}(x)$	$\displaystyle\geq\frac{1}{\pi}{\textstyle\displaystyle\int\nolimits_{0}^{1}}\left[t+(1-t)(1+2cx)^{2}\right]^{-\frac{m}{c}}\frac{dt}{\sqrt{t(1-t)}}\cdot$
		$\displaystyle\quad\cdot\frac{1}{\pi}\int\nolimits_{0}^{1}\left[t+(1-t)(1+2cx)^{2}\right]^{-\frac{n}{c}}\frac{dt}{\sqrt{t(1-t)}}$
		$\displaystyle=S_{m,c}(x)S_{n,c}(x).$

For $c=0$ , we have (see [17], (13))

S_{n,0}(x)=\frac{1}{\pi}{\textstyle\displaystyle\int\nolimits_{-1}^{1}}e^{-2nx(1+t)}\frac{dt}{\sqrt{1-t^{2}}}.

With the same Chebyshev inequality, one obtains (4.12). ∎

From Theorem 12, we derive

Corollary 13.

For the Rényi entropy $R_{n,c}(x)$ and the Tsallis entropy $T_{n,c}(x)$ , we have

(4.13)		$\displaystyle R_{m+n,c}(x)$	$\displaystyle\leq R_{m,c}(x)+R_{n,c}(x),$
(4.14)		$\displaystyle T_{m+n,c}(x)$	$\displaystyle\leq T_{m,c}(x)+T_{n,c}(x)-T_{m,c}(x)T_{n,c}(x).$

Remark 14.

The inequalities (4.13) and (4.14) express the subadditivity of the sequences $(R_{n}(x))_{n\geq 0}$ and $(T_{n}(x))_{n\geq 0}$ .

Remark 15.

From (4.12) with $c=-1$ , we obtain

(4.15)

F_{m+n}(x)\geq F_{m}(x)F_{n}(x),\ x\in[0,1].

Here is a probabilistic proof of this inequality.

Let $X_{m},X_{n},Y_{m},Y_{n}$ be independent binomial random variables with the same parameter $x\in[0,1].$ Then

F_{n}(x)=\sum\limits_{k=0}^{n}P\left(X_{n}=Y_{n}=k\right)=P(X_{n}=Y_{n}),

and consequently

	$\displaystyle F_{m+n}(x)$	$\displaystyle=P\left(X_{m+n}=Y_{m+n}\right)=P\left(X_{m}+X_{n}=Y_{m}+Y_{n}\right)\geq$
		$\displaystyle\geq P\left(X_{m}=Y_{m}\text{ and }X_{n}=Y_{n}\right)=P\left(X_{m}=Y_{m}\right)P\left(X_{n}=Y_{n}\right)$
		$\displaystyle=F_{m}(x)F_{n}(x),$

and this proves (4.15). It would be useful to have purely probabilistic proofs of other inequalities in this specific framework; they would facilitate a deeper understanding of the interplay between analytic proofs/results and probabilistic proofs/results.

Inequalities similar to (4.15) hold for $G_{n}(x)$ (apply (4.12) with $c=1$ ) and for $U_{n}(x)$ and $J_{n}(x).$ Indeed, according to (2.2),

U_{m+n}\left(\frac{x}{1-x}\right)=F_{m+n}(x)\geq F_{m}(x)F_{n}(x)=U_{m}\left(\frac{x}{1-x}\right)U_{n}\left(\frac{x}{1-x}\right),

for all $x\in[0,1),$ which implies

(4.16)

U_{m+n}\left(t\right)\geq U_{m}(t)U_{n}(t),\ t\in[0,+\infty).

From (4.16), by multiplication with $\left(\dfrac{1-t}{1+t}\right)^{2}$ and using (2.16), we get

(4.17)

J_{m+n}\left(t\right)\geq\dfrac{1-t}{1+t}J_{m+n}\left(t\right)\geq J_{m}(t)J_{n}(t),\ t\in[0,1).

Let us remark that the inequality (4.17) is stronger than the similar inequalities for $F_{n}$ , $G_{n}$ and $U_{n}$ .

Corollary 16.

For $m\geq 0$ , $n\geq 0$ , $k\geq 0$ , we have

F_{m+kn}(x)\geq F_{m}(x)F_{k}^{n}(x),\,\,x\in[0,1].

In particular,

(4.18)

F_{kn}(x)\geq F_{k}^{n}(x);\,\,F_{n}(x)\geq F_{1}^{n}(x).

Proof.

Starting from (4.15), it suffices to use induction on $n$ . ∎

Similar results hold for $G_{n}(x)$ , $U_{n}(x)$ , $J_{n}(x)$ , but we omit the details. However, let us remark that $F_{1}(x)=1-2x(1-x)$ and so the second inequality in (4.18) is the first inequality in (4.7).

Remark 17.

Convexity properties of the information potentials and the associated entropies were presented in [32, 31], but the hypothesis $a_{n-k}=a_{k},\ k=0,1,\ldots,n$ was inadvertently omitted in ([31], Conjecture 6.1).

5. More about $U_{n}(t)$

This section contains some additional properties of the function $U_{n}$ defined initially by (1.16). Using the simple relation (2.16) connecting $J_{n}$ and $U_{n}$ , one can easily derive new properties of the function $J_{n}$ given by (1.17).

Theorem 18.

(i)

$U_{n}$ is decreasing on $[0,1]$ and increasing on $[1,\infty).$
(ii)

$U_{n}$ is logarithmically convex on $[0,1].$

Proof.

It was proved (see [27, 31, 32]) that $F_{n}$ is a logarithmically convex function on $[0,1],$ i.e.,

(5.1)

F_{n}^{\prime\prime}(x)F_{n}(x)-F_{n}^{\prime 2}(x)\geq 0,\ x\in[0,1].

Let $x=\dfrac{t}{t+1},\ t\in[0,\infty),\ x\in[0,1).$ Then $t=\dfrac{x}{1-x}$ and (2.2) shows that $U_{n}(t)=F_{n}(x).\$ Consequently,

(5.2)

U_{n}^{\prime}(t)=F_{n}^{\prime}(x)\frac{dx}{dt}=F_{n}^{\prime}(x)(t+1)^{-2}.

It is known (see [17]) that $F_{n}^{\prime}(x)\leq 0,\ x\in[0,\frac{1}{2}],$ and $F_{n}^{\prime}(x)\geq 0,\ x\in[\frac{1}{2},1].$ It follows that $U_{n}^{\prime}(t)\leq 0,\ t\in[0,1],$ and $U_{n}^{\prime}(t)\geq 0,\ t\in[1,\infty).$ This proves (i).

To prove (ii), let us remark that

U_{n}^{\prime\prime}(t)=F_{n}^{\prime\prime}(x)(t+1)^{-4}-2F_{n}^{\prime}(x)(t+1)^{-3}.

Combined with (5.2), this yields

	$\displaystyle U_{n}^{\prime\prime}(t)U_{n}(t)-U_{n}^{\prime 2}(t)$	$\displaystyle=\left(F_{n}^{\prime\prime}(x)F_{n}(x)-F_{n}^{\prime 2}(t)\right)(t+1)^{-4}-$
		$\displaystyle-2F_{n}^{\prime}(x)F_{n}(x)(t+1)^{-3}.$

Using (5.1) and $F_{n}^{\prime}(x)\leq 0,\ x\in[0,\frac{1}{2}],$ we obtain

U_{n}^{\prime\prime}(t)U_{n}(t)-U_{n}^{\prime 2}(t)\geq 0,\ t\in[0,1],

which proves (ii). ∎

Remark 19.

$U_{n}(0)=\underset{t\rightarrow+\infty}{\lim}U_{n}(t)=1,\$ (see (2.14)). These equalities, theorem 18, and graphical experiments(see Figure 1) suggest that $U_{n}$ is convex on $[0,t_{n}]$ and concave on $[t_{n},+\infty),$ for a suitable $t_{n}>1.$ It would be interesting to have a proof for this shape of $U_{n}(t)$ , and to find the value of $t_{n}.$

Refer to caption — Figure 1. Graphics of $U_{n}$ for $n=10,20,30,40$ .

In order to compute $U_{n}(x)$ , we have the explicit expressions (1.16) and (2.14), and the three terms of recurrence relation (2.12). In what follows, we provide two terms of recurrence relation.

According to ([31], (2.3)),

x(1-x)F_{n}^{\prime}(x)=n(1-2x)(F_{n}(x)-F_{n-1}(x)),\ n\geq 1,\ x\in[0,1].

Setting again $x=$ $\dfrac{t}{t+1},\ t\in[0,\infty),$ and $U_{n}(t)=F_{n}(x),$ , we obtain, after some computation,

(5.3)

t(t+1)U^{\prime}(t)=n(1-t)\left(U_{n}(t)-U_{n-1}(t)\right),\ t\in[0,\infty).

Multiplying (5.3) by $(t+1)^{2n-1}/t^{n+1}$ , we obtain

(5.4)

\left(\frac{(t+1)^{2n}}{t^{n}}U_{n}(t)\right)^{\prime}=\left(\frac{(t+1)^{2n}}{t^{n}}\right)^{\prime}U_{n-1}(t),\ t\geq 1.

Let $s(t):=\dfrac{(t+1)^{2}}{t},\ t\geq 1,$ and

V_{n}(t):=s^{n}(t)U_{n}(t),\ t\geq 1.

Theorem 20.

The sequence $(V_{n}(t))_{n\geq 0}$ satisfies the recurrence relation

(5.5)

V_{n}(t)=\binom{2n}{n}+n\int\nolimits_{1}^{t}\frac{x^{2}-1}{x^{2}}V_{n-1}(x)dx,\ n\geq 1,

with $V_{0}(t)=1,\ t\geq 1.$

Proof.

From (5.4), we obtain

\left(s^{n}(x)U_{n}(x)\right)^{\prime}=\left(s^{n}(x)\right)^{\prime}U_{n-1}(x),\ x\geq 1,

i.e.,

V_{n}^{\prime}(x)=ns^{n-1}(x)s^{\prime}(x)\frac{V_{n-1}(x)}{s^{n-1}(x)}.

This reduces to

V_{n}^{\prime}(x)=ns^{\prime}(x)V_{n-1}(x),

and therefore

(5.6)

V_{n}(t)-V_{n}(1)=n\int\nolimits_{1}^{t}\frac{x^{2}-1}{x^{2}}V_{n-1}(x)dx.

Now $V_{n}(1)=s^{n}(1)U_{n}(1),$ and (2.14) shows that $V_{n}(1)=4^{n}\frac{1}{4^{n}}\binom{2n}{n}=\binom{2n}{n}.$ Together with (5.6) this proves (5.5). We have also $V_{0}(t)=U_{0}(t)=1$ (see Remark 3). ∎

Example 21.

From (5.5), we deduce

V_{1}(t)=2+\int\nolimits_{1}^{t}\left(1-\frac{1}{x^{2}}\right)dx=t+\frac{1}{t},

and consequently

U_{1}(t)=\frac{V_{1}(t)}{s(t)}=\frac{t^{2}+1}{(t+1)^{2}}.

Moreover,

V_{2}(t)=6+2\int\nolimits_{1}^{t}\left(1-\frac{1}{x^{2}}\right)\left(x+\frac{1}{x}\right)dx=4+t^{2}+\frac{1}{t^{2}},

i.e., $U_{2}(t)=\dfrac{t^{4}+4t^{2}+1}{(t+1)^{4}},$ and so on.

Remark 22.

A recurrence relation similar to (5.5) and defining a sequence of Appell polynomials was instrumental in ([31], Section 5) for studying the function $F_{n}$ .

Remark 23.

According to (4.9), $\underset{n\rightarrow+\infty}{\lim}U_{n}(x)=0,\ x>0,$ i.e., the sequence of functions $(U_{n})_{n\geq 0}$ is pointwise convergent to zero on $(0,\infty)$ . The convergence is not uniform, because $U_{n}(0)=\underset{x\rightarrow\infty}{\lim}U_{n}(x)=1$ for all $n\geq 0.$

6. Inequalities for the Integral of the Squared Derivative

Integrals of the form ${\textstyle\int\nolimits_{a}^{b}}f^{\prime 2}(x)dx$ are important for several applications; see, e.g., ([33], Section 3.10). In this section, we present bounds for such integrals using the logarithmic convexity of the functions $F_{n},\ G_{n},\ K_{n}.$ The results involve some combinatorial identities.

Theorem 24.

The following inequalities are valid for $n=0,1,\ldots$ :

(6.1)	$\displaystyle{\textstyle\int\nolimits_{0}^{1}}F_{n}^{\prime 2}(x)dx$	$\displaystyle\leq 2n,$
(6.2)	$\displaystyle{\textstyle\int\nolimits_{0}^{\infty}}G_{n+1}^{\prime 2}(x)dx$	$\displaystyle\leq n+1,$
(6.3)	$\displaystyle{\textstyle\int\nolimits_{0}^{\infty}}K_{n}^{\prime 2}(x)dx$	$\displaystyle\leq n.$

Proof.

Let us return to (5.1). Integrating by parts, we obtain

(6.4)

{\textstyle\int\nolimits_{0}^{1}}F_{n}^{\prime 2}(x)dx\leq\frac{1}{2}\left(F_{n}^{\prime}(1)F_{n}(1)-F_{n}^{\prime}(0)F_{n}(0)\right).

Remembering that $F_{n}(x)=S_{n,-1}(x)$ and using (1.11), we obtain

(6.5)

F_{n}^{\prime}(0)=-2n,\ F_{n}^{\prime}(1)=2n,\ F_{n}(0)=F_{n}(1)=1.

Now (6.1) is a consequence of (6.4) and (6.5).

The logarithmic convexity of the functions $G_{n+1}$ and $K_{n}$ on $[0,\infty)$ was proven in [34]. Using $G_{n}(x)=S_{n,1}(x)$ and (1.13), it is easy to derive

(6.6)		$\displaystyle G_{n+1}(0)$	$\displaystyle=1,\ G_{n+1}^{\prime}(0)=-2(n+1)$
(6.7)		$\displaystyle\underset{x\rightarrow\infty}{\lim}G_{n+1}(x)$	$\displaystyle=\underset{x\rightarrow\infty}{\lim}G_{n+1}^{\prime}(x)=0,$

and from

{\textstyle\int\nolimits_{0}^{\infty}}G_{n+1}^{\prime 2}(x)dx\leq\frac{1}{2}\left(G_{n+1}(\infty)G_{n+1}^{\prime}(\infty)-G_{n+1}(0)G_{n+1}^{\prime}(0)\right)

combined with (6.6) and (6.7), we obtain (6.2).

The proof of (6.3) is similar and we omit it. ∎

Remark 25.

If we compute $F_{n}^{\prime}(1)$ starting from (2.18), we obtain

F_{n}^{\prime}(1)=4{\textstyle\sum\limits_{k=1}^{n}}kc_{n,k}.

Combined with (6.5), this yields

(6.8)

{\textstyle\sum\limits_{k=1}^{n}}k\binom{2(n-k)}{n-k}\binom{2k}{k}=n4^{n}/2.

On the other hand, (2.17) leads to

(6.9)

G_{n+1}^{\prime}(0)=-2{\textstyle\sum\limits_{k=0}^{n}}(2k+1)c_{n,k}.

Now (6.6) and (6.9) produce

(6.10)

{\textstyle\sum\limits_{k=0}^{n}}(2k+1)\binom{2(n-k)}{n-k}\binom{2k}{k}=(n+1)4^{n}.

From (6.8) and (6.10), we obtain

(6.11)

{\textstyle\sum\limits_{k=0}^{n}}\binom{2(n-k)}{n-k}\binom{2k}{k}=4^{n},

which is (3.90) in Gould [35].

7. Information Potential for the Durrmeyer Density of Probability

Consider the Durrmeyer operators

L_{n}f(x):=\int\nolimits_{0}^{1}(n+1)\left(\sum\limits_{i=0}^{n}b_{ni}(x)b_{ni}(t)\right)f(t)dt,

$f\in C[0,1],\ x\in[0,1];$ see, e.g., [36]. They are of the form (1.5).

Here, $b_{ni}(x)=\binom{n}{i}x^{i}(1-x)^{n-i}$ . The kernel is

K_{n}(x,t):=(n+1)\sum\limits_{i=0}^{n}b_{ni}(x)b_{ni}(t),

and according to (1.2), the associated information potential is

	$\displaystyle S_{n}(x)$	$\displaystyle:=(n+1)^{2}\int\nolimits_{0}^{1}\left(\sum\limits_{i,j=0}^{n}b_{ni}(x)b_{nj}(x)b_{ni}(t)b_{nj}(t)\right)dt$
		$\displaystyle=(n+1)^{2}\sum\limits_{i,j=0}^{n}b_{ni}(x)b_{nj}(x)\binom{n}{i}\binom{n}{j}\binom{2n}{i+j}^{-1}\int\nolimits_{0}^{1}b_{2n,i+j}(t)dt$
		$\displaystyle=\frac{(n+1)^{2}}{2n+1}\sum\limits_{i,j=0}^{n}\binom{n}{i}^{2}\binom{n}{j}^{2}\binom{2n}{i+j}^{-1}x^{i+j}(1-x)^{2n-i-j}$
		$\displaystyle=\frac{(n+1)^{2}}{2n+1}\sum\limits_{i,j=0}^{n}\left(\binom{n}{i}\binom{n}{j}\binom{2n}{i+j}^{-1}\right)^{2}b_{2n,i+j}(x).$

Setting $i+j=k$ , we obtain

S_{n}(x)=\sum\limits_{k=0}^{2n}q_{n,k}b_{2n,k}(x),\ x\in[0,1],

where

	$\displaystyle q_{n,k}$	$\displaystyle:=\frac{(n+1)^{2}}{2n+1}\binom{2n}{k}^{-2}\sum\limits_{l=0}^{k}\binom{n}{l}^{2}\binom{n}{k-l}^{2}$
		$\displaystyle=(n+1)\binom{2n}{n}^{-1}\binom{2n+1}{n}^{-1}\sum\limits_{l=0}^{k}\binom{k}{l}^{2}\binom{2n-k}{n-l}^{2},\ k=0,1,\ldots,2n,$

with $\binom{n}{m}=0$ if $m>n.$

It is easy to see that $q_{2n-k}=q_{n,k},\ k=0,1,\ldots,2n.$

We recall here Conjecture 4.6 from [24].

Conjecture 26.

([24]) The sequence $(q_{n,k})_{k=0,1,\ldots,2n}$ is convex and, consequently, the function $S_{n}$ is convex on $[0,1].$

The following numerical and graphical experiments support this conjecture (see Table 1 and Figure 2).

Table 1. Values of the coefficients

q_{n,k}

\boldsymbol{nk}

1

\dfrac{4}{3}

\dfrac{2}{3}

\dfrac{4}{3}

2

\dfrac{9}{5}

\dfrac{9}{10}

\dfrac{9}{10}

\dfrac{9}{10}

\dfrac{9}{5}

3

\dfrac{16}{7}

\dfrac{8}{7}

\dfrac{176}{175}

\dfrac{164}{175}

\dfrac{176}{175}

\dfrac{8}{7}

\dfrac{16}{7}

4

\dfrac{25}{9}

\dfrac{25}{18}

\dfrac{1025}{882}

\dfrac{925}{882}

\dfrac{905}{882}

\dfrac{925}{882}

\dfrac{1025}{882}

\dfrac{25}{18}

\dfrac{25}{9}

5

\dfrac{36}{11}

\dfrac{18}{11}

\dfrac{4}{3}

\dfrac{13}{11}

\dfrac{86}{77}

\dfrac{23}{21}

\dfrac{86}{77}

\dfrac{13}{11}

\dfrac{4}{3}

\dfrac{18}{11}

\dfrac{36}{11}

6

\dfrac{49}{13}

\dfrac{49}{26}

\dfrac{4753}{3146}

\dfrac{4165}{3146}

\dfrac{17395}{14157}

\dfrac{66787}{56628}

\dfrac{7329}{6292}

\dfrac{66787}{56628}

\dfrac{17395}{14157}

\dfrac{4165}{3146}

\dfrac{4753}{3146}

\dfrac{49}{26}

\dfrac{49}{13}

8. Concluding Remarks and Future Work

Bounds and convexity type properties of entropies are important and useful, especially when the entropies are expressed as complicated functions depending on one or several variables. Of course, bounds and convexity properties are presented in terms of inequalities; therefore, their study is a branch of the theory of inequalities, under active research. Our paper contains some contributions in this framework. We have obtained analytic inequalities, with analytic methods, but involving certain information potentials and their associated Rényi and Tsallis entropies. The probabilistic flavor is underlined by the purely probabilistic proof of the inequality (4.15). Finding such probabilistic proofs for other inequalities in this context will be a topic for future research. For example, is there a purely probabilistic proof of the subadditivity property (4.13) of the Rényi entropy $R_{n,c}(x)$ ?

The area in which our results can be placed is delineated by the papers [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13] and the references therein: the titles are expressive by themselves.

Basically, we are concerned with the family of probability distributions $(p_{n,k}^{[c]}(x))_{k=0,1,\cdots}$ , strongly related with the family of generalized Baskakov positive linear operators. The interplay between the theory of positive linear operators and probability theory is still a rich source of important results. Besides the binomial distribution, Poisson distribution, and negative binomial distribution (corresponding, respectively to $c=-1$ , $c=0$ , $c=1$ ) and associated with the Bernstein, Szász-Mirakyan, respectively classical Baskakov operators, we consider in our paper, from an analytic point of view, the distributions associated with the Bleimann–Butzer–Hahn, Meyer-König and Zeller, and Durrmeyer operators. Their study in a probabilistic perspective is deferred to a future paper. Another possible direction of further research is to investigate with our methods the distributions associated with other classical or more recent sequences of positive linear operators.

The information potential $F_{n}$ , $G_{n}$ , $U_{n}$ , and $J_{n}$ have strong relations with the Legendre polynomials. Quite naturally, the recurrence relations satisfied by these polynomials yield similar relations for the information potentials. It should be mentioned that the differential equation characterizing the Legendre polynomials was used in [17], in order to show that $F_{n}$ , $G_{n}$ , $U_{n}$ and $J_{n}$ satisfy Heun-type differential equations and consequently to obtain bounds for them. Other bounds are obtained in this paper, starting from the important integral representations given in [19]. They can be compared with other bounds from the literature, and this is another possible topic for further research.

For a fixed $n$ , the convexity and even the logarithmic convexity of the function $F_{n}(x)$ were established in [17, 31, 32, 27, 34]. In this paper, we prove that for a fixed $x$ , the sequence $(F_{n}(x))_{n\geq 0}$ is logarithmically convex. Similar results hold for the other information potentials, and they have consequences concerning the associated entropies. However, we think that this direction of research can be continued and developed.

Two conjectures, accompanied by graphical experiments supporting them, are mentioned in our paper.

References

[1] Harremoës, P.; Topsoe, F. Inequalities between entropy and index of coincidence derived from information diagrams. IEEE Trans. Inf. Theory 2001, 47, 2944–2960.
[2] Harremoës, P. Binomial and Poisson distribution as maximum entropy distributions. IEEE Trans. Inf. Theory 2001, 47, 2039–2041.
[3] Hillion, E. Concavity of entropy along binomial convolution. Electron. Commun. Probab. 2012, 17, 1–9.
[4] Hillion, E.; Johnson, O. A proof of the Shepp-Olkin entropy concavity conjecture. Bernoulli 2017, 23, 3638–3649.
[5] Knessl, C. Integral representation and asymptotic expansions for Shannon and Renyi entropies. Appl. Math. Lett. 1998, 11, 69–74.
[6] Adell, J.A.; Lekuona, A.; Yu, Y. Sharp bound on the entropy of the Poisson low and related quantities. IEEE Trans. Inf. Theory 2010, 56, 2299–2306.
[7] Melbourne, J.; Tkocz, T. Reversals of Rényi entropy inequalities under log-concavity. IEEE Trans. Inf. Theory 2020, doi: 10.1109/TIT.2020.3024025.
[8] Shepp, L.A.; Olkin, I. Entropy of the Sum of Independent Bernoulli Random Variables and of the Multinomial Distribution, Contributions to Probability A Collection of Papers Dedicated to Eugene Lukacs; 1981; pp. 201–206.
[9] Hillion, E.; Johnson, O. Discrete versions of the transport equation and the Shepp-Olkin conjecture. Ann. Probab. 2016, 44, 276–306.
[10] Alzer, H. A refinement of the entropy inequality. Ann. Univ. Sci. Bp. 1995, 38, 13–18.
[11] Chang, S.-C.; Weldon, E. Coding for T-user multiple-access channels. IEEE Trans. Inf. Theory 1979, 25, 684–691.
[12] Xu, D. Energy, Entropy and Information Potential for Neural Computation. Ph.D. Thesis, University of Florida, Gainesville, FL, USA, 1999.
[13] Bărar, A.; Mocanu, G. R.; Raşa, I. Bounds for some entropies and special functions. Carpathian J. Math. 2018, 34, 9–15.
[14] Principe, J.C. Information Theoretic Learning: Rényi’s Entropy and Kernel Perspectives; Springer: Berlin/Heidelberg, Germany, 2010.
[15] Acu, A.M.; Başcanbaz-Tunca, G.; Raşa, I. Information potential for some probability density functions. Appl. Math. Comput. 2021, 389, 125578.
[16] Acu, A.M.; Başcanbaz-Tunca, G.; Raşa, I. Bounds for indices of coincidence and entropies. submitted.
[17] Raşa, I. Entropies and Heun functions associated with positive linear operators. Appl. Math. Comput. 2015, 268, 422–431.
[18] Baskakov, V.A.. An instance of a sequence of positive linear operators in the space of continuous functions. Doklady Akademii Nauk SSSR 1957, 113, 249–251.
[19] Berdysheva, E. Studying Baskakov–Durrmeyer operators and quasi-interpolants via special functions. J. Approx. Theory 2007, 149, 131–150.
[20] Heilmann, M. Erhohung der Konvergenzgeschwindigkeit bei der Approximation von Funktionen mit Hilfe von Linearkombinationen Spezieller Positiver Linearer Operatoren. Habilitationschrift Universitat Dortmund , Dortmund, Germany, 1992.
[21] Wagner, M. Quasi-Interpolaten zu genuinen Baskakov-Durrmeyer-Typ Operatoren; Shaker: Aachen, Germany, 2013.
[22] Acu, A.M.; Heilmann, M.; Rasa, I. Linking Baskakov Type Operators, Constructive theory of functions, Sozopol 2019; Prof. Marin Drinov Academic Publishing House: Sofia , 2020, in press.
[23] Heilmann, M.; Raşa, I. A nice representation for a link between Baskakov and Szász-Mirakjan-Durrmeyer operators and their Kantorovich variants. Results Math. 2019, 74, 9.
[24] Raşa, I. Rényi entropy and Tsallis entropy associated with positive linear operators. arXiv 2014, arXiv:1412.4971v.1.
[25] Nikolov, G. Inequalities for ultraspherical polynomials. Proof of a conjecture of I. Raşa. J. Math. Anal. Appl. 2014, 418, 852–860.
[26] Gavrea, I.; Ivan, M. On a conjecture concerning the sum of the squared Bernstein polynomials. Appl. Math. Comput. 2014, 241, 70–74.
[27] Alzer, H. Remarks on a convexity theorem of Raşa. Results Math. 2020, 75, 29.
[28] Cover, T.M.; Thomas, J.A. Elements of Information Theory; John Wiley & Sons: Hoboken, NJ, USA, 2006.
[29] Abramowitz, M.; Stegun, I.A. Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables; Dover Publications, Inc.: New York, NY, USA, 1970.
[30] Bărar, A.; Mocanu, G.; Raşa, I. Heun functions related to entropies. RACSAM 2019, 113, 819–830.
[31] Raşa, I. Convexity properties of some entropies (II). Results Math. 2019, 74, 154.
[32] Raşa, I. Convexity properties of some entropies. Results Math. 2018, 73, 105.
[33] Cloud, M.J.; Drachman, B.C. Inequalities: With Applications to Engineering; Springer: Berlin/Heidelberg, Germany, 2006.
[34] Abel, U.; Gawronski, W.; Neuschel, T. Complete monotonicity and zeros of sums of squared Baskakov functions. Appl. Math. Comput. 2015, 258, 130–137.
[35] Gould, H.W. Combinatorial Identities—A Standardized Set of Tables Listing 500 Binomial Coefficient Summations; Morgantown, VA, USA, 1972.
[36] Altomare, F.; Campiti, M. Korovkin-Type Approximation Theory and Its Applications; Walter de Gruyter: Berlin, Germany; New York, NY, USA, 1994.

2020

On the approximation of fixed points for non-self mappings on metric spaces

April 29, 2022

Fixed point index theory for decomposable multivalued maps and applications to φ-Laplacian problems

May 3, 2022

Announcement: a recent publication in the top journal “Advances in Water Resources”, by a team from ICTP

June 27, 2020

Inequalities for Information Potentials and Entropies

Abstract

Authors

Keywords

References

PDF

Cite this paper as:

About this paper

Journal

Publisher Name

DOI

Print ISSN

Online ISSN

Google Scholar Profile

Inequalities for Information Potentials and Entropies

Abstract.

1. Introduction

2. Recurrence Relations

Theorem 1.

Proof.

Remark 2.

Remark 3.

3. Inequalities for Information Potentials

Theorem 4.

Proof.

Corollary 5.

Theorem 6.

Proof.

Remark 7.

4. Other Inequalities

Theorem 8.

Corollary 9.

Theorem 10.

Proof.

Remark 11.

Theorem 12.

Proof.

Corollary 13.

Remark 14.

Remark 15.

Corollary 16.

Proof.

Remark 17.

5. More about Un​(t)U_{n}(t)

Theorem 18.

Proof.

Remark 19.

Theorem 20.

Proof.

Example 21.

Remark 22.

Remark 23.

6. Inequalities for the Integral of the Squared Derivative

Theorem 24.

Proof.

Remark 25.

7. Information Potential for the Durrmeyer Density of Probability

Conjecture 26.

8. Concluding Remarks and Future Work

References

Related Posts

5. More about $U_{n}(t)$