Informatics and Applications
2016, Volume 10, Issue 4, pp 72-88
ANALYTICAL SOLUTION OF THE OPTIMAL CONTROL TASK OF A SEMI-MARKOV PROCESS WITH FINITE SET OF STATES
- P. V. Shnurkov
- A. K. Gorshenin
- V. V. Belousov
Abstract
The theoretical verification of the new method of finding the optimal strategy of control of a semi-Markov process with finite set of states is presented. The paper considers Markov randomized strategies of control, determined by a finite collection of probability measures, corresponding to each state. The quality characteristic is the stationary cost index. This index is a linear-fractional integral functional, depending on collection of probability measures, giving the strategy of control. Explicit analytical forms of integrands of numerator and denominator of this linear-fractional integral functional are known. The basis of consequent results is the new generalized and strengthened form of the theorem about an extremum of a linear-fractional integral functional. It is proved that problems of existence of an optimal control strategy of a semi-Markov process and finding this strategy can be reduced to the task of numerical analysis of global extremum for the given function, depending on finite number of real arguments.
[+] References (32)
- Howard, R. A. 1960. Dynamic programming and Markov processes. Cambridge, MA: MIT Press. 136 p.
- Rykov, V. V. 1966. Upravlyaemye markovskie protsessy s konechnymi prostranstvami sostoyaniy i upravleniy [Controlled Markov processes with finite spaces of states and controls ]. Teoriya veroyatnostey i ee primeneniya [Theory of Probability and Its Applications] 11(2):343-351.
- Jewell, W. S. 1963. Markov-renewal programming. Oper. Res. 11:938-971.
- Fox, B. 1966. Markov renewal programming by linear fractional programming. SIAM J. Appl. Math. 14:14181432.
- Denardo, E. V. 1967. Contraction mappings in the theory underlying dinamic programming. SIAM Rev. 9:165-177.
- Howard, R.A. 1963. Research in semi-Markovian decision structures. J. Oper. Res. Soc. Japan 6:163-199.
- Osaki, S., andH. Mine. 1968. Linear programming algorithms for Markovian decision processes. J. Math. Anal. Appl. 22:356-381.
- Mine, H., and S. Osaki. 1970. Markovian decision processes. New York, NY: Elsevier. 142 p.
- Gikhman, 1.1., and A. V. Skorokhod. 1977. Upravlyaemye sluchaynye protsessy [Controlled random processes]. Kiev: Naukova Dumka. 251 p.
- Luque-Vasquez, F, and Î. Herndndez-Lerma. 1999. Semi-Markov control models with average costs. Appl. Math. 26(3):315-331.
- Vega-Amaya, O., and F. Luque-Vasquez. 2000. Sample- path average cost optimality for semi-Markov control processes on Borel spaces: Unbounded costs and mean holding times. Appl. Math. 27(3):343-367.
- Gnedenko, B. V., ed. 1983. Voprosy matematicheskoy teorii nadezhnosti [Problems of the mathematical theory of reliability]. Moscow: Radio i svyaz'. 376 p.
- Barzilovich, E. Yu., and V. A. Kashtanov. 1971. Nekotorye matematicheskie voprosy teorii obsluzhivaniya slozhnykh sistem [Some mathematical questions in theory of complex systems maintenance]. Moscow: Sovetskoe radio. 272 p.
- Shnurkov, P.V. 2016. Solution of the unconditional extremum problem for a linear-fractional integral functional on a set of probability measures. Dokl. Math. 94(2):550- 554.
- Shiryaev, A. N. 2016. Probability-1. Graduate texts in mathematics ser. New York, NY: Springer. Vol. 95. 503 p.; 2017. Probability-2. Vol. 900. 500 p.
- Borovkov, À. À. 2009. Teoriya veroyatnostey [Probability theory]. Moscow: Librokom. 656 p.
- Khenneken, P. L., and A. Tortra. 1974. Teoriya veroyatnostey i nekotorye ee prilozheniya [Probability theory and some of its applications]. Moscow: Nauka. 472 p.
- Halmos, P. 1950. Measure theory. Litton Educational Publishing. 304 p.
- Korolyuk, V. S., and A. F Turbin. 1976. Polumarkovskie protsessy i ikh prilozheniya [Semi-Markov processes and their applications]. Kiev: Naukova Dumka. 184 p.
- Janssen, J., and R. Manca. 2006. Applied semi-Markov processes. New York, NY: Springer. 309 p.
- Shnurkov, P V, andA. VIvanov. 2015. Analysis of a discrete semi-Markov model of continuous inventory control with periodic interruptions of consumption. Discrete Math. Appl. 25(1):59-67.
- Ivanov, A.V. 2014. Analiz diskretnoy polumarkovskoy modeli upravleniya zapasom nepreryvnogo produkta pri periodicheskom prekrashchenii potrebleniya [Analysis of a discrete semi-Markov control model of continuous product inventory in a periodic cessation of consumption]. Moscow: Natsional'nyy Issledovatel'skiy Univer- sitet "Vysshaya Shkola Ekonomiki." PhD Thesis. 120 p.
- Bajalinov, E. B. 2003. Linear-fractional programming. Theory, methods, applications and software. Boston/ Dordrecht/London: Kluwer Academic Publs. 423 p.
- Shnurkov, P.V., and R. V. Mel'nikov. 2006. Optimal'noe upravlenie zapasom nepreryvnogo produkta v modeli re- generatsii [Optimal control of a continuous product inventory in the regeneration model]. Obozrenie priklad- noy i promyshlennoy matematiki [Rev. Appl. Ind. Math.] 13(3):434-452.
- Shnurkov, P. V., and R. V. Mel'nikov. 2008. Analysis of the problem of continuous-product inventory control under deterministic lead time. Automat. Rem. Contr. 69(10):1734-1751.
- Shnurkov, P.V. 1983. Metody issledovaniya zadach op- timal'nogo obsluzhivaniya v matematicheskoy teorii nadezhnosti [Research methods of optimal service problems in the mathematical theory of reliability]. Moscow: Moskovskiy Institut Elektronnogo Mashinostroeniya. PhD Thesis.
- Kudryavtsev, L. D. 2006. Kurs matematicheskogo analiza [A course of mathematical analysis]. Vol. 1. Moscow: Drofa. 704 p.
- Shnurkov, P.V. 1986. Optimal'noe obsluzhivanie na pe- riode do pervogo otkaza sistemy [The optimum service period until the first system failure]. Primenenie analitich- eskikh metodov v veroyatnostnykh zadachakh [The application of analytical methods in probabilistic tasks]. Kiev: Institute of Mathematics of the Academy of Sciences of the USSR. 121-129.
- Shnurkov, P.V., and A.V. Ivanov. 2013. Issledovanie zadachi optimizatsii v diskretnoy polumarkovskoy modeli upravleniya nepreryvnym zapasom [ Study of the optimization problem in discrete semi-Markov model of continuous inventory control]. Vestnik MGTUim. N. E. Baumana. Ser. Estestvennye nauki [Vestnik of MSTU named after N. E. Bauman. Ser. Natural sciences] 3(50):62-87.
- Shnourkoff, P V. 1997. The two-element system with one restoring device optimum maintenance. Stoch. Anal. Appl. 15(5):823-837.
- Shnourkoff, P.V. 2001. The two-element system optimum maintenance tills the first fail. Stoch. Anal. Appl. 19(6):1005-1024.
- Gorshenin, A. K., V. V. Belousov, P.V. Shnourkoff, and A. V. Ivanov. 2015. Numerical research of the optimal control problem in the semi-Markov inventory model. AIP Conference Proceedings 1648:250007.
[+] About this article
Title
ANALYTICAL SOLUTION OF THE OPTIMAL CONTROL TASK OF A SEMI-MARKOV PROCESS WITH FINITE SET OF STATES
Journal
Informatics and Applications
2016, Volume 10, Issue 4, pp 72-88
Cover Date
2016-12-30
DOI
10.14357/19922264160408
Print ISSN
1992-2264
Publisher
Institute of Informatics Problems, Russian Academy of Sciences
Additional Links
Key words
optimal control of a semi-Markov process; stationary cost index of quality control; linear-fractional integral functional
Authors
P. V. Shnurkov ![](1.gif) , A. K. Gorshenin ![](2.gif) , and V. V. Belousov
Author Affiliations
![](1.gif) National Research University Higher School of Economics, 34 Tallinskaya Str., Moscow, 123458, Russian Federation
![](2.gif) Institute of Informatics Problems, Federal Research Center "Computer Science and Control" of the Russian Academy of Sciences, 44-2 Vavilova Str., Moscow 119333, Russian Federation
|