Informatics and Applications
2019, Volume 13, Issue 3, pp 114-121
APPLICATION OF RECURRENT NEURAL NETWORKS TO FORECASTING THE MOMENTS OF FINITE NORMAL MIXTURES
- A. K. Gorshenin
- V. Yu. Kuzmin
Abstract
The article compares the application of feedforward and recurrent neural networks to forecasting continuous values of expectation, variance, skewness, and kurtosis of finite normal mixtures. Fourteen various architectures of neural networks are considered. To increase training speed, the high-performance computing cluster is used. It is demonstrated that the best forecasting results based on standard metrics (root-mean-square error, mean absolute errors, and loss function) are achieved on the two LSTM (Long-Short Term Memory) networks: with 100 neurons in one hidden layer and 50 neurons in each three hidden layers.
[+] References (21)
- Yang, M.-Sh., Ch.-Yo Lai, and C.-Y. Lin. 2012. A robust EM clustering algorithm for Gaussian mixture models. Pattern Recogn. 45(11):3950-3961.
- Cai, T T, J. Ma, andL. Zhang. 2019. CHIME: Clustering of high-dimensional Gaussian mixtures with em algorithm and its optimality. Ann. Stat. 47(3):1234-1267.
- Ben Hassen, H., K. Masmoudi, and A. Masmoudi. 2019. Model selection in biological networks using a graphical EM algorithm. Neurocomputing 349:271-280.
- Zeller, C. B., C. R. B. Cabral, V. H. Lachos, and L. Benites. 2019. Finite mixture of regression models for censored data based on scale mixtures ofnormal distributions. Adv. Data Anal. Classi. 13(1):89-116.
- Osoba, O., S. Mitaim, and B. Kosko. 2013. The noisy Expectation-Maximization algorithm. Fluct. Noise Lett. 12(3):1350012.
- Lee, S.X., K. L. Leemaqz, and G.J. McLachlan. 2018. A block EM algorithm for multivariate skew normal and skew t-mixture models. IEEE T. Neur. Net. Lear. 29(11):5581-5591.
- Wu, D., J. Ma. 2019. An effective EM algorithm for mix-tures of Gaussian processes via the MCMC sampling and approximation. Neurocomputing 331:366-374.
- Verbeek, J. J., N. Vlassis, and B. Krose. 2003. Efficient greedy learning of Gaussian mixture models. Neural Com- put. 15(2):469-485.
- Gorshenin, A. K. 2011. Testing of statistical hypotheses in the splitting component model. Mosc. Univ. Comput. Math. Cybernetics 35(4):176-183.
- Liu, C., H.-C. Li, K. Fu, F Zhang, M. Datcu, and W J. Emery. 2019. A robust EM clustering algorithm for Gaussian mixture models. Pattern Recogn. 87:269-284.
- Yu, L., T Yang, and A. B. Chan. 2019. Density-preserving hierarchical EM algorithm: Simplifying Gaussian mixture models for approximate inference. IEEE T. Pattern Anal. 41(6):1323-1337.
- Batanov, G. M., V. D. Borzosekov, A. K. Gorshenin, N. K. Kharchev, V. Yu. Korolev, and K. A. Sarskyan. 2019. Evolution of statistical properties of microturbulence dur-ing transient process under electron cyclotron resonance heating of the L-2M stellarator plasma. Plasma Phys. Contr. F. 61(7):075006.
- Korolev, V. Yu. 2011. Veroyatnostno-statisticheskie metody dekompozitsii volatil'nosti khaoticheskikh protsessov [Prob-abilistic and statistical methods of decomposition of volatility of chaotic processes]. Moscow: Moscow Uni-versity Publishing House. 512 p.
- Gorshenin, A. K., and V. Yu. Kuzmin. 2019. Improved architecture of feedforward neural networks to increase accuracy of predictions for moments of finite normal mixtures. Pattern Recog. Image Anal. 29(1):68-77.
- Buduma, N. 2017. Fundamentals of deep learning: De-signing next-generation machine intelligence algorithms. Se-bastopol, CA: O'Reilly Media. 298 p.
- Greff, K., R. K. Srivastava, J. Koutnik, B. R Steunebrink, andJ. Schmidhuber. 2017. LSTM: Asearch space Odyssey. IEEE T. Neur. Net. Lear. 28(10):2222-2232.
- Gorshenin, A. K. 2016. Kontseptsiya onlayn-kompleksa dlya stokhasticheskogo modelirovaniya real'nykh protses- sov [Concept of online service for stochastic modeling of real processes]. Informatika i ee Primeneniya - Inform. Appl. 10(1):72-81.
- Kingma, D., and J. Ba. 2015. Adam: A method for stochas-tic optimization. 3rd Conference (International) for Learning Representations. arXiv:1412.6980. 13 p.
- Zeiler, M. D. 2012. ADADELTA: An adaptive learning rate method. arXiv: 1212.5701. 6 p.
- Glorot, X., A. Bordes, and Y. Bengio. 2011. Deep sparse rectifier neural networks. J. Mach. Learn. Res. 15:315323.
- Srivastava, N., G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15:1929-1958.
[+] About this article
Title
APPLICATION OF RECURRENT NEURAL NETWORKS TO FORECASTING THE MOMENTS OF FINITE NORMAL MIXTURES
Journal
Informatics and Applications
2019, Volume 13, Issue 3, pp 114-121
Cover Date
2019-09-30
DOI
10.14357/19922264190316
Print ISSN
1992-2264
Publisher
Institute of Informatics Problems, Russian Academy of Sciences
Additional Links
Key words
recurrent neural networks; forecasting; deep learning; high-performance computing; CUDA
Authors
A. K. Gorshenin  ,  and V. Yu. Kuzmin 
Author Affiliations
Institute of Informatics Problems, Federal Research Center "Computer Science and Control" of the Russian Academy of Sciences, 44-2 Vavilov Str., Moscow 119333, Russian Federation
Faculty of Computational Mathematics and Cybernetics, M.V. Lomonosov Moscow State University, GSP-1, Leninskie Gory, Moscow 119991, Russian Federation
"Wi2Geo LLC", 3-1 Mira Ave., Moscow 129090, Russian Federation
|