Informatics and Applications
2016, Volume 10, Issue 4, pp 72-88
ANALYTICAL SOLUTION OF THE OPTIMAL CONTROL TASK OF A SEMI-MARKOV PROCESS WITH FINITE SET OF STATES
- P. V. Shnurkov
- A. K. Gorshenin
- V. V. Belousov
Abstract
The theoretical verification of the new method of finding the optimal strategy of control of a semi-Markov process with finite set of states is presented. The paper considers Markov randomized strategies of control, determined by a finite collection of probability measures, corresponding to each state. The quality characteristic is the stationary cost index. This index is a linear-fractional integral functional, depending on collection of probability measures, giving the strategy of control. Explicit analytical forms of integrands of numerator and denominator of this linear-fractional integral functional are known. The basis of consequent results is the new generalized and strengthened form of the theorem about an extremum of a linear-fractional integral functional. It is proved that problems of existence of an optimal control strategy of a semi-Markov process and finding this strategy can be reduced to the task of numerical analysis of global extremum for the given function, depending on finite number of real arguments.
[+] References (32)
- Howard, R. A. 1960. Dynamic programming and Markov processes. Cambridge, MA: MIT Press. 136 p.
- Rykov, V. V. 1966. Upravlyaemye markovskie protsessy s konechnymi prostranstvami sostoyaniy i upravleniy [Controlled Markov processes with finite spaces of states and controls ]. Teoriya veroyatnostey i ee primeneniya [Theory of Probability and Its Applications] 11(2):343-351.
- Jewell, W. S. 1963. Markov-renewal programming. Oper. Res. 11:938-971.
- Fox, B. 1966. Markov renewal programming by linear fractional programming. SIAM J. Appl. Math. 14:14181432.
- Denardo, E. V. 1967. Contraction mappings in the theory underlying dinamic programming. SIAM Rev. 9:165-177.
- Howard, R.A. 1963. Research in semi-Markovian decision structures. J. Oper. Res. Soc. Japan 6:163-199.
- Osaki, S., andH. Mine. 1968. Linear programming algorithms for Markovian decision processes. J. Math. Anal. Appl. 22:356-381.
- Mine, H., and S. Osaki. 1970. Markovian decision processes. New York, NY: Elsevier. 142 p.
- Gikhman, 1.1., and A. V. Skorokhod. 1977. Upravlyaemye sluchaynye protsessy [Controlled random processes]. Kiev: Naukova Dumka. 251 p.
- Luque-Vasquez, F, and Î. Herndndez-Lerma. 1999. Semi-Markov control models with average costs. Appl. Math. 26(3):315-331.
- Vega-Amaya, O., and F. Luque-Vasquez. 2000. Sample- path average cost optimality for semi-Markov control processes on Borel spaces: Unbounded costs and mean holding times. Appl. Math. 27(3):343-367.
- Gnedenko, B. V., ed. 1983. Voprosy matematicheskoy teorii nadezhnosti [Problems of the mathematical theory of reliability]. Moscow: Radio i svyaz'. 376 p.
- Barzilovich, E. Yu., and V. A. Kashtanov. 1971. Nekotorye matematicheskie voprosy teorii obsluzhivaniya slozhnykh sistem [Some mathematical questions in theory of complex systems maintenance]. Moscow: Sovetskoe radio. 272 p.
- Shnurkov, P.V. 2016. Solution of the unconditional extremum problem for a linear-fractional integral functional on a set of probability measures. Dokl. Math. 94(2):550- 554.
- Shiryaev, A. N. 2016. Probability-1. Graduate texts in mathematics ser. New York, NY: Springer. Vol. 95. 503 p.; 2017. Probability-2. Vol. 900. 500 p.
- Borovkov, À. À. 2009. Teoriya veroyatnostey [Probability theory]. Moscow: Librokom. 656 p.
- Khenneken, P. L., and A. Tortra. 1974. Teoriya veroyatnostey i nekotorye ee prilozheniya [Probability theory and some of its applications]. Moscow: Nauka. 472 p.
- Halmos, P. 1950. Measure theory. Litton Educational Publishing. 304 p.
- Korolyuk, V. S., and A. F Turbin. 1976. Polumarkovskie protsessy i ikh prilozheniya [Semi-Markov processes and their applications]. Kiev: Naukova Dumka. 184 p.
- Janssen, J., and R. Manca. 2006. Applied semi-Markov processes. New York, NY: Springer. 309 p.
- Shnurkov, P V, andA. VIvanov. 2015. Analysis of a discrete semi-Markov model of continuous inventory control with periodic interruptions of consumption. Discrete Math. Appl. 25(1):59-67.
- Ivanov, A.V. 2014. Analiz diskretnoy polumarkovskoy modeli upravleniya zapasom nepreryvnogo produkta pri periodicheskom prekrashchenii potrebleniya [Analysis of a discrete semi-Markov control model of continuous product inventory in a periodic cessation of consumption]. Moscow: Natsional'nyy Issledovatel'skiy Univer- sitet "Vysshaya Shkola Ekonomiki." PhD Thesis. 120 p.
- Bajalinov, E. B. 2003. Linear-fractional programming. Theory, methods, applications and software. Boston/ Dordrecht/London: Kluwer Academic Publs. 423 p.
- Shnurkov, P.V., and R. V. Mel'nikov. 2006. Optimal'noe upravlenie zapasom nepreryvnogo produkta v modeli re- generatsii [Optimal control of a continuous product inventory in the regeneration model]. Obozrenie priklad- noy i promyshlennoy matematiki [Rev. Appl. Ind. Math.] 13(3):434-452.
- Shnurkov, P. V., and R. V. Mel'nikov. 2008. Analysis of the problem of continuous-product inventory control under deterministic lead time. Automat. Rem. Contr. 69(10):1734-1751.
- Shnurkov, P.V. 1983. Metody issledovaniya zadach op- timal'nogo obsluzhivaniya v matematicheskoy teorii nadezhnosti [Research methods of optimal service problems in the mathematical theory of reliability]. Moscow: Moskovskiy Institut Elektronnogo Mashinostroeniya. PhD Thesis.
- Kudryavtsev, L. D. 2006. Kurs matematicheskogo analiza [A course of mathematical analysis]. Vol. 1. Moscow: Drofa. 704 p.
- Shnurkov, P.V. 1986. Optimal'noe obsluzhivanie na pe- riode do pervogo otkaza sistemy [The optimum service period until the first system failure]. Primenenie analitich- eskikh metodov v veroyatnostnykh zadachakh [The application of analytical methods in probabilistic tasks]. Kiev: Institute of Mathematics of the Academy of Sciences of the USSR. 121-129.
- Shnurkov, P.V., and A.V. Ivanov. 2013. Issledovanie zadachi optimizatsii v diskretnoy polumarkovskoy modeli upravleniya nepreryvnym zapasom [ Study of the optimization problem in discrete semi-Markov model of continuous inventory control]. Vestnik MGTUim. N. E. Baumana. Ser. Estestvennye nauki [Vestnik of MSTU named after N. E. Bauman. Ser. Natural sciences] 3(50):62-87.
- Shnourkoff, P V. 1997. The two-element system with one restoring device optimum maintenance. Stoch. Anal. Appl. 15(5):823-837.
- Shnourkoff, P.V. 2001. The two-element system optimum maintenance tills the first fail. Stoch. Anal. Appl. 19(6):1005-1024.
- Gorshenin, A. K., V. V. Belousov, P.V. Shnourkoff, and A. V. Ivanov. 2015. Numerical research of the optimal control problem in the semi-Markov inventory model. AIP Conference Proceedings 1648:250007.
[+] About this article
Title
ANALYTICAL SOLUTION OF THE OPTIMAL CONTROL TASK OF A SEMI-MARKOV PROCESS WITH FINITE SET OF STATES
Journal
Informatics and Applications
2016, Volume 10, Issue 4, pp 72-88
Cover Date
2016-12-30
DOI
10.14357/19922264160408
Print ISSN
1992-2264
Publisher
Institute of Informatics Problems, Russian Academy of Sciences
Additional Links
Key words
optimal control of a semi-Markov process; stationary cost index of quality control; linear-fractional integral functional
Authors
P. V. Shnurkov  , A. K. Gorshenin  , and V. V. Belousov
Author Affiliations
National Research University Higher School of Economics, 34 Tallinskaya Str., Moscow, 123458, Russian Federation
Institute of Informatics Problems, Federal Research Center "Computer Science and Control" of the Russian Academy of Sciences, 44-2 Vavilova Str., Moscow 119333, Russian Federation
|