Informatics and Applications
2019, Volume 13, Issue 4, pp 18-26
ON COMPARATIVE EFFICIENCY OF CLASSIFICATION SCHEMES IN AN ENSEMBLE OF DATA SOURCES USING AVERAGE MUTUAL INFORMATION
Abstract
Given ensemble of data sources and different fusion schemes, an accuracy of multiclass classification of the collections of the source objects is investigated. Using the average mutual information between the datasets of the sources and a set of the classes, a new approach to comparing lower bounds to an error probability in two fusion schemes is developed. The authors consider the WMV (Weighted Majority Vote) scheme which uses a composition of the class decisions on the objects of the individual sources and the GDM (General Dissimilarity Measure) scheme based on a composition of metrics in datasets of the sources. For the above fusion schemes, the mean values of the average mutual information per one source are estimated. It is proved that the mean in the WMV scheme is less than the similar mean in the GDM scheme. As a corollary, the lower bound to the error probability in the WMV scheme exceeds the similar bound to the error probability in the GDM scheme. This theoretical result is confirmed by experimental error rates in face recognition of HSI color images that yield the ensemble of H, S, and I sources.
[+] References (14)
- Kuncheva, L. 2014. Combining pattern classifiers, methods and algorithms. 2nd ed. New York, NY: John Wiley and Sons. 384 p.
- Gray, R., and D. Neuhoff. 1998. Quantization. IEEE T. Inform. Theory 44(6):2325-2383.
- Kolmogorov, A. N., and V.M. Tikhomirov. 1961. e-entropy and e-capacity of sets in functional spaces. AMSTransl. 17(2):277-364.
- Lam, L., and C. Suen. 1997. Application of majority voting to pattern recognition: An analysis of its behavior and performance. IEEE T. Syst. Man. Cyb. 27(5):553-568.
- Lange, M. M., and D.Y. Stepanov. 2014. Recognition of objects given by collections of multichannel images. Pattern Recogn. Image Anal. 24(3):431-442.
- Kuncheva, L., C. Whitaker, C. Shipp, and R. Duin. 2003. Limits on the majority vote accuracy in classifier fusion. Pattern Anal. Appl. 6(1):22-31.
- Gallager, R. 1968. Information theory and reliable communication. New York, NY: John Wiley and Sons. 608 p.
- Lange, M. M., and A.M. Lange. 2018. O teoretiko- informatsionnoy modeli klassifikatsii dannykh [On information theoretical model for data classification]. Mashin- noe obuchenie i analiz dannykh [J. Machine Learning Data Analysis] 4(3):165-179.
- Dobrushin, R. L., and B. S. Tsybakov. 1962. Information transmission with additional noise. IRE T. Inform. Theor. 8(5):293-304.
- Duda, R., P. Hart, and D. Stork. 2001. Pattern classification. 2nd ed. New York, NY: John Wiley and Sons. 688 p.
- Beckenbach, E., and R. Bellman. 1961. Inequalities. New York, NY: Springer-Verlag. 55 p.
- Gradshteyn, I. S., and I. M. Ryzhik. 2007. Table of integrals, series, and products. 7th ed. Academic Press. 1221 p.
- Database of face images. Available at: http:// sourceforge.net/projects/colorfaces (accessed October 9, 2019).
- Lange, M. M., and S. N. Ganebnykh. 2018. On fusion schemes for multiclass object classification with reject in a given ensemble of sources. J. Phys. Conf. Ser. 1096:012048. 12 p. Available at: https:// iopscience.iop.org/article/10.1088/1742-6596/1096/1/ 012048 (accessed October 7, 2019).
[+] About this article
Title
ON COMPARATIVE EFFICIENCY OF CLASSIFICATION SCHEMES IN AN ENSEMBLE OF DATA SOURCES USING AVERAGE MUTUAL INFORMATION
Journal
Informatics and Applications
2019, Volume 13, Issue 4, pp 18-26
Cover Date
2019-12-30
DOI
10.14357/19922264190403
Print ISSN
1992-2264
Publisher
Institute of Informatics Problems, Russian Academy of Sciences
Additional Links
Key words
multiclass classification; ensemble of sources; fusion scheme; composition of decisions; composition of metrics; average mutual information; error probability
Authors
M. M. Lange 
Author Affiliations
Federal Research Center "Computer Science and Control" of the Russian Academy of Sciences, 44-2 Vavilov Str., Moscow 119333, Russian Federation
|