Informatics and Applications
2024, Volume 18, Issue 3, pp 38-44
AUTONOMOUS LINEAR OUTPUT OF THE MARKOV CHAIN STABILIZATION BY SQUARE CRITERION ON AN INFINITE HORIZON
Abstract
The solution of the linear output of the stochastic differential system optimal control problem on an
infinite horizon is adapted for one particular case of indirect observation. The ergodic Markov chain plays the
role of a dynamic system and the autonomous linear output formed by it provides indirect noisy observations on
the state of the chain. The control purpose is formulated as output stabilization in positions determined by the
chain and periodically changing with chain state changes. The solution, as in a similar problem with complete
information, is obtained as the limit form of optimal control in the corresponding problem with a finite horizon.
Sufficient conditions for the control existence turn out to be typical conditions for linear-quadratic problems of
structure of the optimal Wonham filter, only linear components are present in the control and all nonlinearity is
limited by the equation of the filtering estimate. Due to this, the existence conditions include only the requirements
for the autonomous Riccati equation solution. A numerical experiment for the mechanical drive model used in
previous studies is discussed. The purpose of the experiment is to show the difference in the use of optimal control
and its autonomous version.
[+] References (12)
- Athans, M. 1971. The role and use of the stochastic
linear-quadratic-Gaussian problem in control system
design. IEEE T. Automat. Contr. 16(6):529–552. doi:
10.1109/TAC.1971.1099818.
- Wonham,W.M. 1974. Linear multivariable control. A geometric
approach. Lecture notes in economics and mathematical
systems ser. Berlin: Springer-Verlag. 347 p.
- Davis, M.H.A. 1977. Linear estimation and stochastic
control. London: Chapman and Hall. 224 p.
- Kalman, R. , and R. S. Bucy. 1965. New results in linear
filtering and prediction theory. J. Basic Eng. — T. ASME
83(1):95–108. doi: 10.1115/1.3658902.
- Liptser, R. S., and A.N. Shiryaev. 1974. Statistika
sluchaynykh protsessov (nelineynaya fil’tratsiya i smezhnye
voprosy) [Statistics of random processes. Nonlinear
filtering and related problems]. Seriya “Teorya veroyatnostey
i matematicheskaya statistika” [Probability theory
and mathematical statistics ser.].Moscow: Nauka. 696 p.
- Wonham, W.M. 1965. Some applications of stochastic
differential equations to optimal nonlinear filtering. SIAM
J. Control 2(3):347–369. doi: 10.1137/0302028.
- Shiryayev, A.N. 1989. Veroyatnost’ [Probability]. 2nd ed.
Moscow: Nauka. 640 p.
- Bosov, A. V. 2024. Upravlenie lineynym vykhodom
avtonomnoy differentsial’noy sistemy po kvadratichnomu
kriteriyu na beskonechnom gorizonte [Autonomous differential
system linear output control by square criterion
on an infinite horizon]. Informatika i ee Primeneniya—Inform.
Appl. 18(1):18–25. doi: 10.14357/19922264240103.
EDN: UEESFO.
- Bosov, A. V. 2022. Stabilization and tracking of the trajectory
of a linear system with jump drift. Automat. Rem.
Contr. 83(4):520–535. doi: 10.1134/S0005117922040026.
- Elliott, R. J., L. Aggoun, and J.B. Moore. 1995. Hidden
Markov models: Estimation and control. New York, NY:
Springer-Verlag. 382 p. doi: 10.1007/978-0-387-84854-9.
- Bosov, A. V. 2021. The problem of controlling the linear
output of a nonlinear uncontrollable stochastic differential
system by the square criterion. J. Comput. Sys. Sc. Int.
60(5):719–739. doi: 10.1134/S1064230721040031. EDN:
ESNCSX.
- Rishel, R. 1994. A strong separation principle for
stochastic control systems driven by a hidden Markov
model. SIAM J. Control Optim. 32(4):1008–1020. doi:
10.1137/S0363012992232233.
[+] About this article
Title
AUTONOMOUS LINEAR OUTPUT OF THE MARKOV CHAIN STABILIZATION BY SQUARE CRITERION ON AN INFINITE HORIZON
Journal
Informatics and Applications
2024, Volume 18, Issue 3, pp 38-44
Cover Date
2024-09-20
DOI
10.14357/19922264240305
Print ISSN
1992-2264
Publisher
Institute of Informatics Problems, Russian Academy of Sciences
Additional Links
Key words
Markov chain with continuous time; Wonham filter; linear differential system; optimal control; stabilization; square criterion
Authors
A. V. Bosov 
Author Affiliations
Federal Research Center "Computer Science and Control" of the Russian Academy of Sciences, 44-2 Vavilov Str., Moscow 119333, Russian Federation
|