Informatics and Applications
2021, Volume 15, Issue 2, pp 3-11
LINEAR OUTPUT CONTROL OF MARKOV CHAINS BY THE QUADRATIC CRITERION
Abstract
The problem of optimal output control of a stochastic observation system, in which the state determines an unobservable Markov jump process and linear observations are given by a system of Ito differential equations with a Wiener process, is solved. Observations additively include control vector, so that a controlled output of the system is formed. The optimization goal is set by a general quadratic criterion. To solve the control problem, a separation theorem is formulated that uses the solution to the optimal filtering problem provided by the Wonham filter. As a result of the separation, an equivalent problem of output control of a diffusion process of a particular type, namely, with linear drift and nonlinear diffusion, is formed. The solution ofthis problem is provided by direct application ofthe dynamic programming method.
[+] References (18)
- Athans, M. 1971. The role and use of the stochastic linear-quadratic-gaussian problem in control system design. IEEE T. Automat. Contr. 16(6):529-552.
- Wonham, W M. 1968. On the separation theorem of stochastic control. SIAMJ. Control 6(2):312-326.
- Fleming, W H., and R. W. Rishel. 1975. Deterministic and stochastic optimal control. New York, NY: Springer-Verlag. 222 p.
- Kushner, H. J., and P. G. Dupuis. 2001. Numerical methods for stochastic control problems in continuous time. New York, NY: Springer-Verlag. 476 p.
- Bertsekas, D. P. 2017. Dynamic programming and optimal control. Cambridge: Athena Scientific. 576 p.
- Mortensen, R. E. 1966. Stochastic optimal control with noisy observations. Int. J. Control 4(5):455-464.
- Davis, M. H.A., and P.P. Varaiya. 1973. Dynamic pro-gramming conditions for partially observable stochastic systems. SIAMJ. Control 11(2):226-262.
- Benes, V. E., and I. Karatzas. 1983. On the relation of Zakai's and Mortensen's equations. SIAM J. Control Optim. 21(3):472-489.
- Bensoussan, A. 1992. Stochastic control of partially observable systems. Cambridge: Cambridge University Press. 364 p.
- Miller, B.M., K. E. Avrachenkov, K. V. Stepanyan, and G. B. Miller. 2005. The problem ofoptimal stochastic data flow control based upon incomplete information. Probl. Inf. Transm. 41(2):150-170.
- Elliott, R. J., L. Aggoun, and J. B. Moore. 1995. Hidden Markov models: Estimation and control. New York, NY: Springer-Verlag. 382 p.
- Athans, M., and P. L. Falb. 2007. Optimal control: An introduction to the theory and its applications. New York, NY Dover Publications. 879 p.
- Feldbaum, A.A. 1966. Osnovy teorii optimal'nykh avtomaticheskikh system [Foundations of theory of optimal automatic systems]. Moscow: Nauka. 624 p.
- Bhatia, R. 1997. Matrix analysis. Graduate texts in mathematics ser. New York, NY: Springer-Verlag. Vol. 169. 349 p.
- Shiryaev, A. N. 1996. Probability. New York, NY: Springer Verlag. 624 p.
- Bosov, A.V., and A.I. Stefanovich. 2018. Upravlenie vykhodom stokhasticheskoy differentsial'noy sistemy po kvadratichnomu kriteriyu. I. Optimal'noe reshenie metodom dinamicheskogo programmirovaniya [Stochastic differential system output control by the quadratic
criterion. I. Dynamic programming optimal solution]. Informatika i ee Primeneniya - Inform. Appl. 12(3):99-106.
- Bosov, A. V. 2021. O nekotorykh chastnykh sluchayakh v zadache upravleniya vykhodom stokhasticheskoy differentsial'noy sistemy po kvadratichnomu kriteriyu [On some special cases in the problem of stochastic differential system output control by the quadratic criterion]. Informatika i ee Primeneniya - Inform. Appl. 15(1):11-17.
- Davis, M. H.A. 1977. Linear estimation and stochastic control. London: Chapman and Hall. 224 p.
[+] About this article
Title
LINEAR OUTPUT CONTROL OF MARKOV CHAINS BY THE QUADRATIC CRITERION
Journal
Informatics and Applications
2021, Volume 15, Issue 2, pp 3-11
Cover Date
2021-06-30
DOI
10.14357/19922264210201
Print ISSN
1992-2264
Publisher
Institute of Informatics Problems, Russian Academy of Sciences
Additional Links
Key words
Markov jump process; Ito stochastic differential system; optimal control; quadratic criterion; stochastic filtering; Wonham filter
Authors
A. V. Bosov 
Author Affiliations
Institute of Informatics Problems, Federal Research Center "Computer Science and Control" of the Russian Academy of Sciences, 44-2 Vavilov Str., Moscow 119333, Russian Federation
|