Informatics and Applications
2020, Volume 14, Issue 1, pp 1016
ANALYSIS OF CONFIGURATIONS OF LSTM NETWORKS FOR MEDIUMTERM VECTOR FORECASTING
 A. K. Gorshenin
 V. Yu. Kuzmin
Abstract
The paper analyzes 36 configurations of LSTM (long shortterm memory) architectures for forecasting with a duration up to 70 steps based on data whose size is 300500 elements. For probabilistic approximation of observations, a model based on finite normal mixtures is used; therefore, the mathematical expectation, variance, skewness, and kurtosis of these mixtures are used as initial data for forecasting. The optimal configurations of neural networks were determined and the practical possibility of constructing highquality mediumterm forecasts with a limited training time was demonstrated. The results obtained are important for the development of a probabilisticstatistical approach to the description of the evolution of turbulent processes in a magnetically active hightemperature plasma.
[+] References (23)
 Batanov, G. M., V. D. Borzosekov, A. K. Gorshenin, N. K. Kharchev, V. Yu. Korolev, and K. A. Sarskyan. 2019. Evolution of statistical properties of microturbulence during transient process under electron cyclotron resonance heating of the L2M stellarator plasma. Plasma Phys. Contr. F. 61(7):075006.
 Batanov, G. M., M.S. Berezhetskii, V. D. Borzosekov, et al. 2017. Reaction of turbulence at the edge and in the center of the plasma column to pulsed impurity injection caused by the sputtering of the wall coating in L2M stellarator. Plasma Phys. Rep. 43(8):818823.
 Korolev, V Yu. 2011. Veroyatnostnostatisticheskie metody dekompozitsii volatil'nosti khaoticheskikh protsessov [Probabilistic and statistical methods of decomposition of volatility of chaotic processes]. Moscow: Moscow University Publishing House. 512 p.
 Gorshenin, A., V. Korolev, V. Kuzmin, and A. Zeifman.
2013. Coordinatewise versions ofthe grid method for the analysis of intensities of nonstationary information flows by moving separation of mixtures of gammadistribution. 27th European Conference on Modelling and Simulation Proceedings. Dudweiler, Germany: Digitaldruck Pirrot GmbH. 565568.
 Lee, S.X., K. L. Leemaqz, and G.J. McLachlan. 2018. A block EM algorithm for multivariate skew normal and skew tmixture models. IEEE T. Neur. Net. Lear. 29(11):55815591.
 Cai, T.T., J. Ma, and L. Zhang. 2019. CHIME: Clustering of highdimensional Gaussian mixtures with EM algorithm and its optimality Ann. Stat. 47(3):12341267.
 Liu, C., H.C. Li, K. Fu, F. Zhang, M. Datcu, and W J. Emery. 2019. A robust EM clustering algorithm for Gaussian mixture models. Pattern Recogn. 87:269284.
 Wu, D., and J. Ma. 2019. An effective EM algorithm for mixtures of Gaussian processes via the MCMC sampling and approximation. Neurocomputing 331:366374.
 Ben Hassen, H., K. Masmoudi, and A. Masmoudi. 2019. Model selection in biological networks using a graphical EM algorithm. Neurocomputing 349:271280.
 Zeller, C. B., C. R. B. Cabral, V. H. Lachos, and L. Ben ites. 2019. Finite mixture ofregression models for censored data based on scale mixtures of normal distributions. Adv. Data Anal. Classi. 13(1):89116.
 Meneghini, O., C. J. Luna, S. P. Smith, and L. L. Lao.
2014. Modeling of transport phenomena in tokamak plasmas with neural networks. Phys. Plasmas 21(6):060702.
 Raja, M.A.Z., F H. Shah, M. Tariq, I. Ahmad, and
S. U. Ahmad. 2018. Design of artificial neural network models optimized with sequential quadratic programming to study the dynamics of nonlinear Troesch's problem arising in plasma physics. Neural Comput. Appl. 29(6):83 109.
 Mesbah, A., and D. B. Graves. 2019. Machine learning for modeling, diagnostics, and control of nonequilibrium plasmas. J. Phys. D Appl. Phys. 52(30):30LT02.
 Narita, E., M. Honda, M. Nakata, M. Yoshida, N. Hayashi, and H. Takenaga. 2019. Neuralnetwork based semiempirical turbulent particle transport modelling founded on gyrokinetic analyses of JT60Uplasmas. Nucl. Fusion 59(10): 106018.
 Parsons, M. S. 2017. Interpretation of machinelearning based disruption models for plasma control. Plasma Phys. Contr. F. 59(8):085001.
 KatesHarbeck, J., A. Svyatkovskiy, and W. Tang. 2019. Predicting disruptive instabilities in controlled fusion plasmas through deep learning. Nature 568(7753):526.
 Gorshenin, A.K., and V. Yu. Kuzmin. 2019. Improved architecture of feedforward neural networks to increase accuracy of predictions for moments of finite normal mixtures. Pattern Recognition Image Analysis 29(1):7988.
 Gorshenin, A. K., and V. Yu. Kuzmin. 2019. Primenenie rekurrentnykh neyronnykh setey dlya prognozirovaniya momentov konechnykh normal'nykh smesey [Application of recurrent neural networks to forecasting the moments of finite normal mixtures]. Informatika i ee Primeneniya  Inform. Appl. 13(3):114121.
 Gorshenin, A., and V. Kuzmin. 2020. Amachine learning approach to the vector prediction of moments of finite normal mixtures. Adv. Intell. Syst. 1127:307314.
 Greff, K., R. K. Srivastava, J. Koutnik, B. R Steune brink, and J. Schmidhuber. 2017. LSTM: A search Space Odyssey. IEEE T. Neur. Net. Lear. 28(10):22222232.
 Buduma, N. 2017. Fundamentals of deep learning: Designing nextgeneration machine intelligence algorithms. Sebastopol, CA: O'Reilly Media. 298 p.
 Srivastava, N., G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15:19291958.
 Peinl, R., F. Holzschuher, and F. Pfitzer. 2016. Docker cluster management for the cloud  survey results and own solution. J. Grid Comput. 14(2):265282.
[+] About this article
Title
ANALYSIS OF CONFIGURATIONS OF LSTM NETWORKS FOR MEDIUMTERM VECTOR FORECASTING
Journal
Informatics and Applications
2020, Volume 14, Issue 1, pp 1016
Cover Date
20200330
DOI
10.14357/19922264200102
Print ISSN
19922264
Publisher
Institute of Informatics Problems, Russian Academy of Sciences
Additional Links
Key words
LSTM; forecasting; deep learning; highperformance computing; CUDA
Authors
A. K. Gorshenin , and V. Yu. Kuzmin
Author Affiliations
Institute of Informatics Problems, Federal Research Center "Computer Science and Control" of the Russian Academy of Sciences, 442 Vavilov Str., Moscow 119333, Russian Federation
Faculty of Computational Mathematics and Cybernetics, Lomonosov Moscow State University, GSP1, Leninskie Gory, Moscow 119991, Russian Federation
"Wi2Geo LLC," 31 Mira Ave., Moscow 129090, Russian Federation
