Informatics and Applications
2024, Volume 18, Issue 2, pp 17-24
LOWER BOUND TO ESTIMATION DISTORTION OF A RANDOM PARAMETER FOR A GIVEN AMOUNT OF INFORMATION
Abstract
Given probability distribution density with an unknown value of a random parameter, a minimum of the average square distortion for the parameter estimates via the samples of random values as a function of the average mutual information between the samples and the estimates is investigated. This function is produced by inverting a modified rate distortion function as the dependency of the minimal values of the average mutual information on the appropriate values of the average distortion. The obtained smallest average square distortion as the function of the average mutual information is independent on an estimation form and this function yields the lower bound to the average distortion for the fixed values of the amount of information. The above relation is the bifactor fidelity decision criterion that allows one to compare various estimation functions by their efficiency in terms of the average distortion redundancy relative to the lower bound when the entropy of the quantized estimates is fixed.
[+] References (13)
- Bishop, Ñ. M. 2006. Pattern recognition and machine learning. New York, NY: Springer. 746 p.
- Davisson, L. D., R. I. McEliece, M.B. Pursley, and M. S. Wallace. 1981. Efficient universal noiseless source
codes. IEEE T. Inform. Theory 27(3):269-279. doi: 10.1109/TIT. 1981.1056355.
- Yakovleva, T. V., and N. S. Kulberg. 2014. Methods of mathematical statistics in two-parameter analysis of Rician signals. Dokl. Math. 90(3):675-679. doi: 10.1134/ S1064562414070060. EDN: UFVVGL.
- Vaiciulis, M., and N. M. Markovich. 2021. Estimating the parameters of a tapered Pareto distribution. Automat. Rem. Contr. 82(8):1358-1377. doi: 10.1134/ S000511792108004X.
- Kudryavtsev, A. A., O. V. Shestakov, and S.Ya. Shorgin. 2021. Metod otsenivaniya parametrov izgiba, formy i masshtaba gamma-eksponentsial'nogo raspredeleniya [A method for estimating bent, shape and scale parameters of the gamma-exponential distribution]. Informatika i ee Primeneniya - Inform. Appl. 15(3):57-62. doi: 10.14357/ 19922264230308. EDN: IXMPXH.
- Kudryavtsev, A. A., and O. V. Shestakov. 2023. Metod otsenivaniya parametrov gamma-eksponentsial'nogo raspredeleniya po vyborke so slabo zavisimymi komponentami [A method for estimating parameters of the gamma-exponential distribution from a sample with weakly dependent components]. Informatika i ee Primeneniya - Inform. Appl. 17(3):58-63. doi: 10.14357/ 19922264230308. EDN: PEXTVK.
- Duda, R., P. Hart, and D. Stork. 2001. Pattern classification. 2nd ed. New York, NY: John Wiley and Sons. 738 p.
- Borovkov, A. A. 1984. Matematicheskaya statistika. Otsenka parametrov. Proverka gipotez [Mathematical statistics. Parameter estimation. Hypothesis testing]. Moscow: Nau- ka. 472 p.
- Berger, T. 1971. Rate distortion theory. A mathematical basis for data compression. Englewood Cliffs, NJ: Prentice-Hall. 311 p.
- Gray, R. M., and D.L. Neuhoff. 1998. Quantization. IEEE T. Inform. Theory 44(6):2325-2383. doi: 10.1109/ 18.720541.
- Dobrushin, R. L., and B. S. Tsybakov. 1962. Information transmission with additional noise. IRE T. Inform. Theor. 8(5):293-304. doi: 10.1109/TIT.1962.1057738.
- Lange, M. M., and A. M. Lange. 2022. Information- theoretic lower bounds to error probability for the models of noisy discrete source coding and object classification. Pattern Recognition Image Analysis 32(3):570-574. doi: 10.1134/S105466182203021X.
- Korn, G., and T. Korn. 1968. Mathematical handbook for scientists and engineers. New York - San Francisco - Toronto - London - Sydney: McGraw Hill Book Co. 1147 p.
[+] About this article
Title
LOWER BOUND TO ESTIMATION DISTORTION OF A RANDOM PARAMETER FOR A GIVEN AMOUNT OF INFORMATION
Journal
Informatics and Applications
2024, Volume 18, Issue 2, pp 17-24
Cover Date
2024-06-20
DOI
10.14357/19922264240203
Print ISSN
1992-2264
Publisher
Institute of Informatics Problems, Russian Academy of Sciences
Additional Links
Key words
probability distribution density; data sample; parameter estimate; square distortion; mutual information; rate distortion function; lower bound; redundancy
Authors
M. M. Lange  and A. M. Lange
Author Affiliations
Federal Research Center "Computer Science and Control" of the Russian Academy of Sciences, 44-2 Vavilov Str., Moscow 119333, Russian Federation
|