Informatics and Applications
2019, Volume 13, Issue 1, pp 75-81
OPTIMIZATION OF HYPERPARAMETERS OF NEURAL NETWORKS USING HIGH-PERFORMANCE COMPUTING FOR PREDICTION OF PRECIPITATION
- A. K. Gorshenin
- V. Yu. Kuzmin
Abstract
The paper describes the procedure for tuning hyperparameters of the neural network for analyzing spatial meteorological data using the tools of the hybrid high-performance computing system. The comparison of precipitation forecasting accuracy has been carried out on the basis of such methods as grid and random searches. It has been demonstrated that even with a relatively small number of random choices of combinations of hyperparameters, it is possible to obtain an accuracy comparable to brute force, with moderate time costs. These results show the ability to automatically build a neural network architecture based on the general model for solving applied problems.
[+] References (23)
- Xie, J., C. Yang, B. Zhou, and Q. Huang. 2010. High- performance computing for the simulation of dust storms. Comput. Environ. Urban 34:278-290.
- Lee, C.A., S. D. Gasster, A. Plaza, C.-I. Chang, and
B. Huang. 2011. Recent developments in high performance computing for remote sensing: A review. IEEE J. Sel. Top. Appl. 4(3):508-527.
- Xue, Y., D. Palmer-Brown, and H. Guo. 2011. The use of high-performance and high-throughput computing for the fertilization of Digital Earth and global change studies. Int. J. Digit. Earth 4(3):185-210.
- Lu, F, J. Song, X. Cao, and X. Zhu. 2012. CPU/GPU computing for long-wave radiation physics on large GPU clusters. Comput. Geosci. 41:47-55.
- Oubeidillah, A. A., S.-C. Kao, M. Ashfaq, B. S. Naz, and
G. Tootle. 2014. A large-scale, high-resolution hydrological model parameter data set for climate change impact assessment for the conterminous US. Hydrol. Earth Syst. Sc. 18(1):67-84.
- Thompson, G., M. K. Politovich, and R. M. Rasmussen.
2017. A numerical weather model's ability to predict
characteristics of aircraft icing environments. Weather Forecast. 32(1):207-221.
- Hu, X., and L. Song. 2018. Hydrodynamic modeling of flash flood in mountain watersheds based on high- performance GPU computing. Nat. Hazards 91(2):567- 586.
- Reguly, I. Z., D. Giles, D. Gopinathan, L. Quivy, J. H. Beck, M.B. Giles, S. Guillas, and F Dias. 2018. The VOLNA-OP2 tsunami code (version 1.5). Geosci. Model Dev. 11(11):4621-4635.
- Zheng, M., W. Tang, andX. Zhao. 2019. Hyperparameter optimization of neural network-driven spatial models accelerated using cyber-enabled high-performance com-puting. Int. J. Geogr. Inf. Sci. 33(2):314-345.
- Gorshenin, A.K., and V. Yu. Kuzmin. 2019. Improved architecture of feedforward neural networks to increase accuracy of predictions for moments of finite normal mixtures. Pattern Recognition Image Anal. 29(1):68-77.
- Bergstra, J., and Y. Bengio. 2012. Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13:281-305.
- Gorshenin, A. K., and V. Yu. Kuzmin. 2018. Neural net-work forecasting of precipitation volumes using patterns. Pattern Recognition Image Anal. 28(3):450-461.
- Glorot, X., A. Bordes, and Y. Bengio. 2011. Deep sparse rectifier neural networks. J. Mach. Learn. Res. 15:315- 323.
- Tikhonov, A. N., A. S. Leonov, and A. G. Yagola. 1998. Nonlinear Ill-posed problems. Heidelberg: Springer. 386 p.
- Srivastava, N., G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15:1929-1958.
- Rojas-Dominguez, A., L. C. Padierna, J. M.C. Valadez,
H. J. Puga-Soberanes, and H.J. Fraire. 2018. Optimal hyper-parameter tuning of SVM classifiers with application to medical diagnosis. IEEE Access 6:7164-7176.
- Uppu, S., and A. Krishna. 2018. A deep hybrid model to detect multi-locus interacting SNPs in the presence of noise. Int. J. Med. Inform. 119:134-151.
- Xia, Y., C. Liu, Y Li, and N. Liu. 2017. A boosted decision tree approach using Bayesian hyper-parameter
optimization for credit scoring. Expert Syst. Appl. 78:225241.
- Greff, K., R. K. Srivastava, J. Koutnik, B. R. Steunebrink, and J. Schmidhuber. 2017. LSTM: Asearch space Odyssey. IEEE T. Neur. Net. Lear. 28(10):2222-2232.
- Kingma, D., and J. Ba. 2015. Adam: A method for stochas-tic optimization. 3rd Conference (International) for Learning Representations. San Diego, CA. arXiv:1412.6980.13p.
- Zeiler, M.D. 2012. ADADELTA: An adaptive learning rate method. arXiv:1212.5701.
- Buduma, N. 2017. Fundamentals of deep learning: Designing next-generation machine intelligence algorithms. Sebastopol, CA: O'Reilly Media. 295 p.
- Gorshenin, A. K. 2017. Analiz veroyatnostno- statisticheskikh kharakteristik osadkov na osnove patter- nov [Pattern-based analysis of probabilistic and statistical characteristics of precipitations]. Informatika i ee Primeneniya - Inform. Appl. 11(4):38-46.
[+] About this article
Title
OPTIMIZATION OF HYPERPARAMETERS OF NEURAL NETWORKS USING HIGH-PERFORMANCE COMPUTING FOR PREDICTION OF PRECIPITATION
Journal
Informatics and Applications
2019, Volume 13, Issue 1, pp 75-81
Cover Date
2019-04-30
DOI
10.14357/19922264190111
Print ISSN
1992-2264
Publisher
Institute of Informatics Problems, Russian Academy of Sciences
Additional Links
Key words
artificial neural network; forecasting; deep learning; hyperparameters; high-performance computing; CUDA
Authors
A. K. Gorshenin  ,  ,
and V. Yu. Kuzmin 
Author Affiliations
Institute of Informatics Problems, Federal Research Center "Computer Science and Control" of the Russian Academy of Sciences, 44-2 Vavilov Str., Moscow 119333, Russian Federation
Faculty of Computational Mathematics and Cybernetics, M. V. Lomonosov Moscow State University, Leninskie Gory, GSP-1, Moscow 119991, Russian Federation
"Wi2Geo LLC," 3-1 Mira Prosp., Moscow 129090, Russian Federation
|