Inequalities for Information Potentials and Entropies

4 years ago

(original), Approximation Theory, ISI/JCR, paper

Abstract

We consider a probability distribution \((p_0(x), p_1(x), . . .)\) depending on a real parameter \(x\). The associated information potential is \(S(x):=∑_{k}p_{k}^2(x).\) The Rényi entropy and the Tsallis entropy of order \(2\) can be expressed as \(R(x) = − \log S(x)\) and \(T(x) = 1 − S(x)\). We establish recurrence relations, inequalities and bounds for \(S(x)\), which lead immediately to similar relations, inequalities and bounds for the two entropies. We show that some sequences \((R_n(x))_{n≥0}\) and \((T_n(x))_{n≥0}\), associated with sequences of classical positive linear operators, are concave and increasing. Two conjectures are formulated involving the information potentials associated with the Durrmeyer density of probability, respectively the Bleimann–Butzer–Hahn probability distribution.

Authors

Ana Maria Acu
Lucian Blaga University of Sibiu, Sibiu, Romania

Alexandra Măduța
Technical University of Cluj-Napoca, Cluj-Napoca, Romania

Diana Otrocol
Technical University of Cluj-Napoca, Cluj-Napoca, Romania
Tiberiu Popoviciu Institute of Numerical Analysis, Romanian Academy

Ioan Rașa
Technical University of Cluj-Napoca, Cluj-Napoca, Romania

Keywords

probability distribution; Rényi entropy; Tsallis entropy; information potential; functional equations; inequalities

References

see the expanding block below

PDF

https://www.mdpi.com/2227-7390/8/11/2056/pdf

Cite this paper as:

A.M. Acu, A. Măduța, D. Otrocol, I. Rașa, Inequalities for information potentials and entropies, 8 (2020) 11, pp. 2056, doi: 10.3390/math8112056

About this paper

Journal

Mathematics

Publisher Name

MDPI

DOI

http://doi.org/10.3390/math8112056

Print ISSN

Not available yet.

Online ISSN

ISSN 2227-7390,

Google Scholar Profile

1. Harremoës, P., Topsoe, F., Inequalities between entropy and index of coincidence derived from information diagrams. IEEE Trans. Inf. Theory 2001, 47, 2944–2960.
2. Harremoës, P., Binomial and Poisson distribution as maximum entropy distributions. IEEE Trans. Inf. Theory 2001, 47, 2039–2041.
3. Hillion, E., Concavity of entropy along binomial convolution. Electron. Commun. Probab. 2012, 17, 1–9.
4. Hillion, E., Johnson, O., A proof of the Shepp-Olkin entropy concavity conjecture. Bernoulli 2017, 23, 3638–3649.
5. Knessl, C., Integral representation and asymptotic expansions for Shannon and Renyi entropies. Appl. Math. Lett. 1998, 11, 69–74.
6. Adell, J.A., Lekuona, A., Yu, Y., Sharp bound on the entropy of the Poisson low and related quantities. IEEE Trans. Inf. Theory 2010, 56, 2299–2306.
7. Melbourne, J., Tkocz, T., Reversals of Rényi entropy inequalities under log-concavity. IEEE Trans. Inf. Theory 2020.
8. Shepp, L.A., Olkin, I., Entropy of the Sum of Independent Bernoulli Random Variables and of the Multinomial Distribution, Contributions to Probability A Collection of Papers Dedicated to Eugene Lukacs; Academic Press: London, UK, 1981; pp. 201–206.
9. Hillion, E., Johnson, O., Discrete versions of the transport equation and the Shepp-Olkin conjecture. Ann. Probab. 2016, 44, 276–306.
10. Alzer, H., A refinement of the entropy inequality. Ann. Univ. Sci. Bp. 1995, 38, 13–18.
11. Chang, S.-C., Weldon, E., Coding for T-user multiple-access channels. IEEE Trans. Inf. Theory 1979, 25, 684–691.
12. Xu, D. Energy, Entropy and Information Potential for Neural Computation. Ph.D. Thesis, University of Florida, Gainesville, FL, USA , 1999.
13. Barar, A., Mocanu, G. R., Rasa, I., Bounds for some entropies and special functions. Carpathian J. Math. 2018, 34, 9–15.
14. Principe, J.C., Information Theoretic Learning: Rényi’s Entropy and Kernel Perspectives; Springer: New York, NY, USA, 2010.
15. Acu, A.M., Bascanbaz-Tunca, G., Rasa, I., Information potential for some probability density functions. Appl. Math. Comput. 2021, 389, 125578.
16. Acu, A.M., Bascanbaz-Tunca, G., Rasa, I., Bounds for indices of coincidence and entropies. submitted.
17. Rasa, I., Entropies and Heun functions associated with positive linear operators. Appl. Math. Comput. 2015, 268, 422–431.
18. Baskakov, V.A., An instance of a sequence of positive linear operators in the space of continuous functions. Doklady Akademii Nauk SSSR 1957, 113, 249–251.
19. Berdysheva, E., Studying Baskakov–Durrmeyer operators and quasi-interpolants via special functions. J. Approx. Theory 2007, 149, 131–150.
20. Heilmann, M., Erhohung der Konvergenzgeschwindigkeit bei der Approximation von Funktionen mit Hilfe von Linearkombinationen Spezieller Positiver Linearer Operatoren; Habilitationschrift Universitat Dortmund: Dortmund, Germany, 1992.
21. Wagner, M., Quasi-Interpolaten zu genuinen Baskakov-Durrmeyer-Typ Operatoren; Shaker: Aachen, Germany, 2013.
22. Acu, A.M., Heilmann, M., Rasa, I., Linking Baskakov Type Operators, Constructive theory of functions, Sozopol 2019; Draganov, B., Ivanov, K., Nikolov, G., Uluchev, R., Eds.; Prof. Marin Drinov Publishing House of BAS: Sofia, Bulgaria, 2020; pp. 23–38.
23. Heilmann, M., Rasa, I., A nice representation for a link between Baskakov and Szász-Mirakjan-Durrmeyer operators and their Kantorovich variants. Results Math. 2019, 74, 9.
24. Rasa, I., Rényi entropy and Tsallis entropy associated with positive linear operators. arXiv 2014, arXiv:1412.4971v.1.
25. Nikolov, G., Inequalities for ultraspherical polynomials. Proof of a conjecture of I. Ra¸sa. J. Math. Anal. Appl. 2014, 418, 852–860.
26. Gavrea, I., Ivan, M., On a conjecture concerning the sum of the squared Bernstein polynomials. Appl. Math. Comput. 2014, 241, 70–74.
27. Alzer, H., Remarks on a convexity theorem of Rasa. Results Math. 2020, 75, 29.
28. Cover, T.M., Thomas, J.A., Elements of Information Theory; John Wiley & Sons: Hoboken, NJ, USA, 2006.
29. Abramowitz, M., Stegun, I.A., Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables; Dover Publications, Inc.: New York, NY, USA, 1970.
30. Barar, A., Mocanu, G., Rasa, I., Heun functions related to entropies. RACSAM 2019, 113, 819–830.
31. Rasa, I., Convexity properties of some entropies (II). Results Math. 2019, 74, 154.
32. Rasa, I., Convexity properties of some entropies. Results Math. 2018, 73, 105.
33. Cloud, M.J., Drachman, B.C., Inequalities: With Applications to Engineering; Springer: Berlin/Heidelberg, Germany, 2006.
34. Abel, U., Gawronski, W., Neuschel, T., Complete monotonicity and zeros of sums of squared Baskakov functions. Appl. Math. Comput. 2015, 258, 130–137.
35. Gould, H.W., Combinatorial Identities—A Standardized Set of Tables Listing 500 Binomial Coefficient Summations; West Virginia University Press: Morgantown, VA, USA, 1972.
36. Altomare, F., Campiti, M., Korovkin-Type Approximation Theory and Its Applications; Walter de Gruyter: Berlin, Germany; New York, NY, USA, 1994.