Inequalities for indices of coincidence and entropies

Alexandra Măduţa Technical University of Cluj-Napoca, Department of Mathematics, 28 Memorandumului Street, 400114 Cluj-Napoca, Romania boloca.alexandra91@yahoo.com , Diana Otrocol Technical University of Cluj-Napoca, Department of Mathematics, 28 Memorandumului Street, 400114 Cluj-Napoca, Tiberiu Popoviciu Institute of Numerical Analysis, Romanian Academy, P.O.Box. 68-1, 400110 Cluj-Napoca, Romania Diana.Otrocol@math.utcluj.ro and Ioan Raşa Technical University of Cluj-Napoca, Department of Mathematics, 28 Memorandumului Street, 400114 Cluj-Napoca, Romania Ioan.Rasa@math.utcluj.ro

Abstract.

We consider a probability distribution depending on a real parameter $x$ . As functions of $x$ , the Rényi entropy and the Tsallis entropy can be expressed in terms of the associated index of coincidence $S(x)$ . We establish recurrence relations and inequalities for $S(x),$ which can be used in order to get information concerning the two entropies.
Keywords: probability distribution, Rényi entropy, Tsallis entropy, index of coincidence, functional equations, inequalities.
MSC: 39B22, 39B62, 94A17, 26D07.

1. Introduction

Let $c\in\mathbb{R}$ . Set $I_{c}=\left[0,-\frac{1}{c}\right]$ if $c<0$ , and $I_{c}=[0,+\infty)$ if $c\geq 0$ . For $\alpha\in\mathbb{R}$ and $k\in\mathbb{N}_{0}$ the binomial coefficients are defined as usual by

{\binom{\alpha}{k}}:=\frac{\alpha(\alpha-1)\dots(\alpha-k+1)}{k!}\quad\mbox{if }k\in\mathbb{N},\mbox{ and }{\binom{\alpha}{0}}:=1.

Let $n>0$ be a real number, $k\in\mathbb{N}_{0}$ and $x\in I_{c}$ . Define

p_{n,k}^{[c]}(x):=(-1)^{k}{\tbinom{-\frac{n}{c}}{k}}(cx)^{k}(1+cx)^{-\frac{n}{c}-k},\quad\mbox{ if }c\neq 0,

p_{n,k}^{[0]}(x):=\lim_{c\rightarrow 0}p_{n,k}^{[c]}(x)=\frac{(nx)^{k}}{k!}e^{-nx},\quad\mbox{ if }c=0.

Then $\sum_{k=0}^{\infty}p_{n,k}^{[c]}(x)=1$ . Suppose that $n>c$ if $c\geq 0$ , or $n=-cl$ with some $l\in\mathbb{N}$ if $c<0$ .

With this notation we consider the discrete distribution of probability $\left(p_{n,k}^{[c]}(x)\right)_{k=0,1,...}$ depending on the parameter $x\in I_{c}.$

The associated index of coincidence is

(1.1)

S_{n,c}(x):=\sum\limits_{k=0}^{\infty}\left(p_{n,k}^{[c]}(x)\right)^{2},\ x\in I_{c}.

The Rényi entropy and the Tsallis entropy corresponding to the same distribution of probability are defined, respectively, by

(1.2)

R_{n,c}(x)=-\log S_{n,c}(x)

and

(1.3)

T_{n,c}(x)=1-S_{n,c}(x).

For $c=-1$ we are dealing with the binomial distribution and

(1.4)

S_{n,-1}(x):=\sum\limits_{k=0}^{n}\left({\dbinom{n}{k}}x^{k}(1-x)^{n-k}\right)^{2},\ x\in[0,1].

The case $c=0$ corresponds to the Poisson distribution, for which

(1.5)

S_{n,0}(x):=e^{-2nx}\sum\limits_{k=0}^{\infty}\dfrac{(nx)^{2k}}{\left(k!\right)^{2}},\ x\geq 0.

For $c=1$ we have the negative binomial distribution, with

(1.6)

S_{n,1}(x):=\sum\limits_{k=0}^{\infty}\left({\dbinom{n+k-1}{k}}x^{k}(1+x)^{-n-k}\right)^{2},\ x\geq 0.

The binomial, Poisson, respectively negative binomial distributions correspond to the classical Bernstein, Szász-Mirakyan, respectively Baskakov operators from Approximation Theory.

The distribution

\left(\dbinom{n}{k}x^{k}(1+x)^{-n}\right)_{k=0,1,\ldots,n},\ x\in[0,+\infty),

corresponds to the Bleimann-Butzer-Hahn operators, while

\left(\dbinom{n+k}{k}x^{k}(1-x)^{n+1}\right)_{k=0,1,\ldots},\ x\in[0,1),

is connected with the Meyer-König and Zeller operators.

The indices of coincidence and the entropies associated with all these distributions were studied in [3]. We continue this study. To keep the same notation as in [3], let

	$\displaystyle F_{n}(x)$	$\displaystyle:=S_{n,-1}(x),\ G_{n}(x):=S_{n,1}(x),\ K_{n}(x):=S_{n,0}(x),$
(1.7)		$\displaystyle U_{n}(x)$	$\displaystyle:={\textstyle\sum\limits_{k=0}^{n}}\left(\dbinom{n}{k}x^{k}(1+x)^{-n}\right)^{2},\ x\in[0,+\infty),$
(1.8)		$\displaystyle J_{n}(x)$	$\displaystyle:={\textstyle\sum\limits_{k=0}^{\infty}}\left(\dbinom{n+k}{k}x^{k}(1-x)^{n+1}\right)^{2},\ x\in[0,1).$

In Section 2 we present several relations between the functions $F_{n}(x)$ , $G_{n}(x),\ U_{n}(x),\ J_{n}(x)$ , as well as between these functions and the Legendre polynomials. By using the three-terms recurrence relations involving the Legendre polynomials we establish recurrence relations involving three consecutive terms from the sequences $\left(F_{n}(x)\right)$ , $\left(G_{n}(x)\right),\newline \left(U_{n}(x)\right),\$ respectively $\left(J_{n}(x)\right)$ . We recall also some explicit expressions of these functions.

Section 3 is devoted to inequalities between consecutive terms of the above sequences; in particular, we emphasize that for fixed $x$ the four sequences are convex.

Other inequalities are presented in Section 4. All the inequalities can be used to get information about the Rényi entropies and Tsallis entropies connected with the corresponding probability distributions.

2. Recurrence relations

$F_{n}(x)$ is a polynomial, $G_{n}(x),\ U_{n}(x),\ J_{n}(x)$ are rational functions. On their maximal domains, these functions are connected by several relations (see [3], Cor. 13, (46), (53), (54)):

(2.1)	$\displaystyle F_{n}(x)$	$\displaystyle=(1-2x)^{2n+1}G_{n+1}(-x),$
(2.2)	$\displaystyle F_{n}(x)$	$\displaystyle=U_{n}\left(\frac{x}{1-x}\right),$
(2.3)	$\displaystyle F_{n}(x)$	$\displaystyle=-(1-2x)^{2n+1}J_{n}\left(\frac{x-1}{x}\right).$

Consider the Legendre polynomial (see [1, 22.3.1])

(2.4)

P_{n}(t)=2^{-n}{\textstyle\sum\limits_{k=0}^{n}}\dbinom{n}{k}^{2}(x+1)^{k}(x-1)^{n-k}.

Then (see [3, (39)])

(2.5)

P_{n}(t)=(1-2x)^{-n}F_{n}(x),

where

t=\frac{1-2x+2x^{2}}{1-2x},\ x\in[0,\frac{1}{2}).

Combining (2.5) with (2.1), (2.2) and (2.3), we get

(2.6)	$\displaystyle P_{n}(t)$	$\displaystyle=(1-2x)^{n+1}G_{n+1}(-x),$
(2.7)	$\displaystyle P_{n}(t)$	$\displaystyle=(1-2x)^{-n}U_{n}\left(\frac{x}{1-x}\right),$
(2.8)	$\displaystyle P_{n}(t)$	$\displaystyle=-(1-2x)^{n+1}J_{n}\left(\frac{x-1}{x}\right).$

On the other hand, the Legendre polynomials satisfy the recurrence relation ([1, 22.7.1])

(2.9)

(n+1)P_{n+1}(t)-(2n+1)tP_{n}(t)+nP_{n-1}(t)=0.

Now (2.9) together with (2.5), (2.6), (2.7), (2.8) lead to

Theorem 1.

The functions $F_{n}(x),G_{n}(x),\ U_{n}(x)$ and $\ J_{n}(x)\$ satisfy the following three-terms recurrence relations:

(2.10)	$\displaystyle 2(n+1)F_{n+1}(x)$	$\displaystyle=(2n+1)(1+(1-2x)^{2})F_{n}(x)-2n(1-2x)^{2}F_{n-1}(x),$
(2.11)	$\displaystyle n(1+2x)^{2}\!G_{n+1}(x)\!$	$\displaystyle=\!(2n-1)\!\!\left(\!1+2x+2x^{2}\!\right)\!G_{n}(x)\!-\!(n-1)G_{n-1}(x),$
(2.12)	$\displaystyle(n+1)(1+t)^{2}U_{n+1}(t)$	$\displaystyle=(2n+1)(t^{2}+1)U_{n}(t)-n(1-t)^{2}U_{n-1}(t),$
(2.13)	$\displaystyle(n+1)(1+t)^{2}J_{n+1}(t)$	$\displaystyle=(2n+1)(t^{2}+1)J_{n}(t)-n(1-t)^{2}J_{n-1}(t).$

Remark 2.

According to (2.12) and (2.13), $U_{n}(x)$ and $J_{n}(x)\$ satisfy the same recurrence relation. In fact, from [3, (49), (55)] we have

(2.14)		$\displaystyle U_{n}(x)$	$\displaystyle=4^{-n}\dbinom{2n}{n}{\textstyle\sum\limits_{k=0}^{n}}\dbinom{n}{k}^{2}\dbinom{2n}{2k}^{-1}\left(\frac{x-1}{x+1}\right)^{2k},$
(2.15)		$\displaystyle J_{n}(x)$	$\displaystyle=4^{-n}{\textstyle\sum\limits_{k=0}^{n}}\frac{(2k)!(2n-2k)!}{k!^{2}(n-k)!^{2}}\left(\frac{1-x}{1+x}\right)^{2k+1}.$

From (2.14) and (2.15) it is easy to deduce that

(2.16)

J_{n}(x)=\frac{1-x}{1+x}U_{n}(x).

Remark 3.

From [3, (56)] and [2, (21)] we know that

(2.17)		$\displaystyle G_{n}(x)$	$\displaystyle=4^{1-n}{\textstyle\sum\limits_{k=0}^{n-1}}\dbinom{2k}{k}\dbinom{2n-2k-2}{n-k-1}(2x+1)^{-2k-1},$
(2.18)		$\displaystyle F_{n}(x)$	$\displaystyle={\textstyle\sum\limits_{k=0}^{n}}(-1)^{k}\dbinom{n}{k}\dbinom{2k}{k}(x(1-x))^{k}.$

So, the recurrence relations (2.10)-(2.13) are accompanied by

	$\displaystyle F_{0}(x)$	$\displaystyle=1,\ F_{1}(x)=1-2x+2x^{2};$
	$\displaystyle G_{1}(x)$	$\displaystyle=\frac{1}{2x+1},\ G_{2}(x)=\frac{1+2x+2x^{2}}{(2x+1)^{3}};$
	$\displaystyle U_{0}(x)$	$\displaystyle=1,\ U_{1}(x)=\frac{1+x^{2}}{(1+x)^{2}};$
	$\displaystyle J_{0}(x)$	$\displaystyle=\frac{1-x}{1+x},\ J_{1}(x)=\frac{(1-x)(1+x^{2})}{(1+x)^{3}}.$

3. Inequalities for indices of coincidence

Theorem 4.

The function $F_{n}(x)$ satisfies the inequalities

(3.1)

F_{n+1}(x)\leq\frac{1+(4n-2)x(1-x)}{1+(4n+2)x(1-x)}F_{n-1}(x),

and

(3.2)

F_{n}(x)\leq\frac{1+4nx(1-x)}{1+(4n+2)x(1-x)}F_{n-1}(x),

for all $n\geq 1,\ x\in[0,1].$

Proof.

We start with (see [3, (29)])

F_{n}(x)=\frac{1}{\pi}\int_{0}^{1}f^{n}(x,t)\frac{dt}{\sqrt{t(1-t)}},

where $f(x,t):=t+(1-t)(1-2x)^{2}\in[0,1].$

It follows that

(3.3)

F_{n+1}(x)\leq F_{n}(x).

On the other hand,

	$\displaystyle F_{n-1}(x)+F_{n+1}(x)-2F_{n}(x)=$
	$\displaystyle=\frac{1}{\pi}\int_{0}^{1}f^{n-1}(x,t)\left[1+f^{2}(x,t)-2f(x,t)\right]\!\!\frac{dt}{\sqrt{t(1-t)}},$

which entails

(3.4)

2F_{n}(x)\leq F_{n-1}(x)+F_{n+1}(x).

According to (2.10), we have

(3.5)

F_{n}(x)=a_{n}(x)F_{n-1}(x)+b_{n}(x)F_{n+1}(x),

where

a_{n}(x)=\frac{n(1-2x)^{2}}{(2n+1)(1-2x+2x^{2})},\ b_{n}(x)=\frac{n+1}{(2n+1)(1-2x+2x^{2})}.

Using (3.4) and (3.5) we get

2a_{n}(x)F_{n-1}(x)+2b_{n}(x)F_{n+1}(x)\leq F_{n-1}(x)+F_{n+1}(x),

which yields

(2b_{n}(x)-1)F_{n+1}(x)\leq(1-2a_{n}(x))F_{n-1}(x),

and this immediately leads to (3.1). To prove (3.2) it suffices to combine (3.4) and (3.1). ∎

Theorem 5.

The following inequalities hold:

(3.6)	$\displaystyle 2U_{n}$	$\displaystyle\leq U_{n-1}+U_{n+1},$
(3.7)	$\displaystyle U_{n+1}(x)$	$\displaystyle\leq\frac{1+4nx+x^{2}}{1+(4n+4)x+x^{2}}U_{n-1}(x),$
(3.8)	$\displaystyle U_{n}(x)$	$\displaystyle\leq\frac{1+(4n+2)x+x^{2}}{1+(4n+4)x+x^{2}}U_{n-1}(x),$
(3.9)	$\displaystyle 2G_{n}$	$\displaystyle\leq G_{n-1}+G_{n+1},$
(3.10)	$\displaystyle G_{n+1}(x)$	$\displaystyle\leq\frac{1+(4n-2)x(1+x)}{1+(4n+2)x(1+x)}G_{n-1}(x),$
(3.11)	$\displaystyle G_{n}(x)$	$\displaystyle\leq\frac{1+4nx(x+1)}{1+(4n+2)x(x+1)}G_{n-1}(x),$
(3.12)	$\displaystyle 2J_{n}$	$\displaystyle\leq J_{n-1}+J_{n+1},$
(3.13)	$\displaystyle J_{n+1}(x)$	$\displaystyle\leq\frac{1+4nx+x^{2}}{1+(4n+4)x+x^{2}}J_{n-1}(x),$
(3.14)	$\displaystyle J_{n}(x)$	$\displaystyle\leq\frac{1+(4n+2)x+x^{2}}{1+(4n+4)x+x^{2}}J_{n-1}(x).$

Proof.

The proof is similar to that of Theorem 4, starting from (see [3, (48), (58), (63)]):

	$\displaystyle U_{n}(x)$	$\displaystyle=\frac{1}{\pi}\int\nolimits_{0}^{1}\left(t+(1-t)\left(\frac{1-x}{1+x}\right)^{2}\right)^{n}\frac{dt}{\sqrt{t(1-t)}},$
	$\displaystyle G_{n}(x)$	$\displaystyle=\frac{1}{\pi}\int\nolimits_{0}^{1}\left(t+(1-t)(1+2x)^{2}\right)^{-n}\frac{dt}{\sqrt{t(1-t)}},$
	$\displaystyle J_{n}(x)$	$\displaystyle=\frac{1}{\pi}\int\nolimits_{0}^{1}\left(t+(1-t)\left(\frac{1+x}{1-x}\right)^{2}\right)^{-n-1}\!\!\!\!\frac{dt}{\sqrt{t(1-t)}}.$

∎

4. Other inequalities

Let us return to the index of coincidence (1.1). According to [3, (10)] we have for $c\neq 0,$

S_{n,c}(x)=\frac{1}{\pi}\int\nolimits_{0}^{1}\left[t+(1-t)(1+2cx)^{2}\right]^{-\frac{n}{c}}\frac{dt}{\sqrt{t(1-t)}}.

Let $c<0$ . Using Chebyshev’s inequality for synchronous functions we can write

	$\displaystyle S_{n-c,c}(x)$	$\displaystyle=\frac{1}{\pi}\!\!\int\nolimits_{0}^{1}\!\!\left[t\!+\!(1\!-\!t)(1\!+\!2cx)^{2}\right]^{-\frac{n}{c}}\!\!\left[t\!+\!(1\!-\!t)(1\!+\!2cx)^{2}\right]\frac{dt}{\sqrt{t(1\!\!-\!\!t)}}$
		$\displaystyle\geq\frac{1}{\pi}\int\nolimits_{0}^{1}\left[t+(1-t)(1+2cx)^{2}\right]^{-\frac{n}{c}}\frac{dt}{\sqrt{t(1-t)}}\cdot$
		$\displaystyle\quad\cdot\frac{1}{\pi}\int\nolimits_{0}^{1}\left[t+(1-t)(1+2cx)^{2}\right]\frac{dt}{\sqrt{t(1-t)}}$
		$\displaystyle=S_{n,c}(x)(1+2cx+2c^{2}x^{2}).$

For $c>0$ we use Chebyshev’s inequality for asynchronous functions and get the reverse inequality. So we have

Theorem 6.

If $c<0$ , then

(4.1)

S_{n-c,c}(x)\geq(1+2cx(1+cx))S_{n,c}(x).

If $c>0$ , the inequality is reversed.

Corollary 7.

For $c=-1,$ (4.1) and (3.2) yield

(1-2x(1-x))F_{n}(x)\leq F_{n+1}(x)\leq\frac{1+(4n+4)x(1-x)}{1+(4n+6)x(1-x)}F_{n}(x).

For $c=1$ we obtain

\frac{\ 1}{1+2x(1+x)}G_{n}(x)\leq G_{n+1}(x)\leq\dfrac{1+(4n+4)x(1+x)}{1+(4n+6)x(1+x)}G_{n}(x).

Now using [3, (48)] we have

	$\displaystyle U_{n+1}(x)$	$\displaystyle=\frac{1}{\pi}\int\nolimits_{0}^{1}\left(\left(\frac{1-x}{1+x}\right)^{2}+\frac{4x}{(1+x)^{2}}t\right)^{n}\cdot$
		$\displaystyle\quad\cdot\left(\left(\frac{1-x}{1+x}\right)^{2}+\frac{4x}{(1+x)^{2}}t\right)\frac{dt}{\sqrt{t(1-t)}}$
		$\displaystyle\geq U_{n}(x)\frac{1}{\pi}\int\nolimits_{0}^{1}\left(\left(\frac{1-x}{1+x}\right)^{2}+\frac{4x}{(1+x)^{2}}t\right)\frac{dt}{\sqrt{t(1-t)}}$
		$\displaystyle=\frac{1+x^{2}}{(1+x)^{2}}U_{n}(x).$

Therefore, using also (3.8), we get

(4.2)

\frac{1+x^{2}}{(1+x)^{2}}U_{n}(x)\leq U_{n+1}(x)\leq\dfrac{1+(4n+6)x+x^{2}}{1+(4n+8)x+x^{2}}U_{n}(x).

Now (4.2) and (2.16) yield

\frac{1+x^{2}}{\left(1+x\right)^{2}}J_{n}(x)\leq J_{n+1}(x)\leq\dfrac{1+(4n+6)x+x^{2}}{1+(4n+8)x+x^{2}}J_{n}(x).

Remark 8.

Inequalities involving the function $K_{n}(x):=S_{n,0}(x)$ can be obtained with different techniques and will be presented elsewhere.

Remark 9.

All the above inequalities can be used to get information concerning the entropies described in (1.2) and (1.3). We omit the details.

Remark 10.

Convexity properties of the indices of coincidence and the associated entropies were presented in [4, 5] but the hypothesis $a_{n-k}=a_{k},\ k=0,1,\ldots,n$ was inadvertently omitted in [5, Conjecture 6.1].

References

[1] M. Abramowitz, I.A. Stegun, Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables, Dover Publications, Inc., New York, (1970).
[2] A. Bărar, G. Mocanu, I. Raşa, Heun functions related to entropies, RACSAM, 113(2019), 819–830.
[3] I. Raşa, Entropies and Heun functions associated with positive linear operators, Appl. Math. Comput., 268(2015), 422–431.
[4] I. Raşa, Convexity properties of some entropies, Results Math., 73:105 (2018).
[5] I. Raşa, Convexity properties of some entropies (II), Results Math., 74:154 (2019).

Inequalities for indices of coincidence and entropies

Abstract

Authors

Keywords

References

PDF

About this paper

Cite this paper as:

Journal

Publisher Name

DOI

Print ISSN

Online ISSN

Google Scholar Profile

Inequalities for indices of coincidence and entropies

Abstract.

1. Introduction

2. Recurrence relations

Theorem 1.

Remark 2.

Remark 3.

3. Inequalities for indices of coincidence

Theorem 4.

Proof.

Theorem 5.

Proof.

4. Other inequalities

Theorem 6.

Corollary 7.

Remark 8.

Remark 9.

Remark 10.

References

Related Posts