Informatics and Applications

2020, Volume 14, Issue 1, pp 10-16

ANALYSIS OF CONFIGURATIONS OF LSTM NETWORKS FOR MEDIUM-TERM VECTOR FORECASTING

A. K. Gorshenin
V. Yu. Kuzmin

Abstract

The paper analyzes 36 configurations of LSTM (long short-term memory) architectures for forecasting with a duration up to 70 steps based on data whose size is 300-500 elements. For probabilistic approximation of observations, a model based on finite normal mixtures is used; therefore, the mathematical expectation, variance, skewness, and kurtosis of these mixtures are used as initial data for forecasting. The optimal configurations of neural networks were determined and the practical possibility of constructing high-quality medium-term forecasts with a limited training time was demonstrated. The results obtained are important for the development of a probabilistic-statistical approach to the description of the evolution of turbulent processes in a magnetically active high-temperature plasma.

[+] References (23)

Batanov, G. M., V. D. Borzosekov, A. K. Gorshenin, N. K. Kharchev, V. Yu. Korolev, and K. A. Sarskyan. 2019. Evolution of statistical properties of microturbulence during transient process under electron cyclotron resonance heating of the L-2M stellarator plasma. Plasma Phys. Contr. F. 61(7):075006.
Batanov, G. M., M.S. Berezhetskii, V. D. Borzosekov, et al. 2017. Reaction of turbulence at the edge and in the center of the plasma column to pulsed impurity injection caused by the sputtering of the wall coating in L-2M stellarator. Plasma Phys. Rep. 43(8):818-823.
Korolev, V Yu. 2011. Veroyatnostno-statisticheskie metody dekompozitsii volatil'nosti khaoticheskikh protsessov [Prob-abilistic and statistical methods of decomposition of volatility of chaotic processes]. Moscow: Moscow University Publishing House. 512 p.
Gorshenin, A., V. Korolev, V. Kuzmin, and A. Zeifman. 2013. Coordinate-wise versions ofthe grid method for the analysis of intensities of non-stationary information flows by moving separation of mixtures of gamma-distribution. 27th European Conference on Modelling and Simulation Proceedings. Dudweiler, Germany: Digitaldruck Pirrot GmbH. 565-568.
Lee, S.X., K. L. Leemaqz, and G.J. McLachlan. 2018. A block EM algorithm for multivariate skew normal and skew t-mixture models. IEEE T. Neur. Net. Lear. 29(11):5581-5591.
Cai, T.T., J. Ma, and L. Zhang. 2019. CHIME: Clustering of high-dimensional Gaussian mixtures with EM algorithm and its optimality Ann. Stat. 47(3):1234-1267.
Liu, C., H.-C. Li, K. Fu, F. Zhang, M. Datcu, and W J. Emery. 2019. A robust EM clustering algorithm for Gaussian mixture models. Pattern Recogn. 87:269-284.
Wu, D., and J. Ma. 2019. An effective EM algorithm for mixtures of Gaussian processes via the MCMC sampling and approximation. Neurocomputing 331:366-374.
Ben Hassen, H., K. Masmoudi, and A. Masmoudi. 2019. Model selection in biological networks using a graphical EM algorithm. Neurocomputing 349:271-280.
Zeller, C. B., C. R. B. Cabral, V. H. Lachos, and L. Ben- ites. 2019. Finite mixture ofregression models for censored data based on scale mixtures of normal distributions. Adv. Data Anal. Classi. 13(1):89-116.
Meneghini, O., C. J. Luna, S. P. Smith, and L. L. Lao. 2014. Modeling of transport phenomena in tokamak plasmas with neural networks. Phys. Plasmas 21(6):060702.
Raja, M.A.Z., F H. Shah, M. Tariq, I. Ahmad, and S. U. Ahmad. 2018. Design of artificial neural network models optimized with sequential quadratic programming to study the dynamics of nonlinear Troesch's problem arising in plasma physics. Neural Comput. Appl. 29(6):83- 109.
Mesbah, A., and D. B. Graves. 2019. Machine learning for modeling, diagnostics, and control of non-equilibrium plasmas. J. Phys. D Appl. Phys. 52(30):30LT02.
Narita, E., M. Honda, M. Nakata, M. Yoshida, N. Hayashi, and H. Takenaga. 2019. Neural-network- based semi-empirical turbulent particle transport modelling founded on gyrokinetic analyses of JT-60Uplasmas. Nucl. Fusion 59(10): 106018.
Parsons, M. S. 2017. Interpretation of machine-learning- based disruption models for plasma control. Plasma Phys. Contr. F. 59(8):085001.
Kates-Harbeck, J., A. Svyatkovskiy, and W. Tang. 2019. Predicting disruptive instabilities in controlled fusion plasmas through deep learning. Nature 568(7753):526.
Gorshenin, A.K., and V. Yu. Kuzmin. 2019. Improved architecture of feedforward neural networks to increase accuracy of predictions for moments of finite normal mixtures. Pattern Recognition Image Analysis 29(1):79-88.
Gorshenin, A. K., and V. Yu. Kuzmin. 2019. Primenenie rekurrentnykh neyronnykh setey dlya prognozirovaniya momentov konechnykh normal'nykh smesey [Application of recurrent neural networks to forecasting the moments of finite normal mixtures]. Informatika i ee Primeneniya - Inform. Appl. 13(3):114-121.
Gorshenin, A., and V. Kuzmin. 2020. Amachine learning approach to the vector prediction of moments of finite normal mixtures. Adv. Intell. Syst. 1127:307-314.
Greff, K., R. K. Srivastava, J. Koutnik, B. R Steune- brink, and J. Schmidhuber. 2017. LSTM: A search Space Odyssey. IEEE T. Neur. Net. Lear. 28(10):2222-2232.
Buduma, N. 2017. Fundamentals of deep learning: De-signing next-generation machine intelligence algorithms. Se-bastopol, CA: O'Reilly Media. 298 p.
Srivastava, N., G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15:1929-1958.
Peinl, R., F. Holzschuher, and F. Pfitzer. 2016. Docker cluster management for the cloud - survey results and own solution. J. Grid Comput. 14(2):265-282.

[+] About this article