Informatics and Applications
2020, Volume 14, Issue 1, pp 24-30
STOCHASTIC DIFFERENTIAL SYSTEM OUTPUT CONTROL BY THE QUADRATIC CRITERION. IV. ALTERNATIVE NUMERICAL DECISION
- A. V. Bosov
- A. I. Stefanovich
Abstract
In the study of the optimal control problem for the Ito diffusion process and the controlled linear output with a quadratic quality criterion, an intermediate result is resumed: for approximate calculation of the optimal solution, an alternative to classical numerical integration method based on computer simulation is proposed.
The method allows applying statistical estimation to determine the coefficients @t(y) and Yt(y) of the previously obtained Bellman function Vt(y, z) = atz2 + â(y)z + Yt(y), determining the optimal solution in the original problem of optimal stochastic control. The method is implemented on the basis of the properties of linear parabolic partial differential equations describing @t(y) and Yt(y) - their equivalent description in the form of stochastic differential equations and a theoretical-probability representation of the solution, known as A. N. Kolmogorov equation, or an equivalent integral form known as the Feynman-Katz formula. Stochastic equations, relations for optimal control and for auxiliary parameters are combined into one differential system, for which an algorithm for simulating a solution is stated. The algorithm provides the necessary samples for statistical estimation of the coefficients â (y) and yt(y). The previously performed numerical experiment is supplemented by calculations presented by an alternative method and a comparative analysis of the results.
[+] References (8)
- Bosov, A.V., and A.I. Stefanovich. 2018. Upravlenie
vykhodom stokhasticheskoy differentsial’noy sistemy
po kvadratichnomu kriteriyu. I. Optimal’noe reshenie
metodom dinamicheskogo programmirovaniya
[Stochastic differential system output control by the quadratic criterion
I. Dynamic programming optimal solution]. Informatika
i ee Primeneniya — Inform.Appl. 12(3):99–1
- Bosov, A.V., and A.I. Stefanovich. 2019. Upravlenie vykhodom
stokhasticheskoy differentsial'noy sistemy po kvadratichnomu kriteriyu.
II. Chislennoe reshenie uravneniy dinamicheskogo programmirovaniya
[Stochastic differential system output control by the quadratic criterion.
II. Dynamic programming equations numerical solution]. Informatika
i ee Primeneniya — Inform. Appl. 13(1):9--15.
- Bosov, A.V., and A.I. Stefanovich. 2019. Upravlenie vykhodom
stokhasticheskoy differentsial'noy sistemy po kvadratichnomu kriteriyu.
III. Analiz svoystv optimal'nogo upravleniya [Stochastic differential system
output control by the quadratic criterion. III. Optimal control properties
analysis]. Informatika i ee Primeneniya — Inform. Appl. 13(3):41--49.
- Gihman, I.I., and A.V. Skorohod. 2012. The theory
of stochastic processes. New York, NY: Springer-Verlag.
Vol.III. 388 p.
- Oksendal, B. 2003. Stochastic differential equations. An
introduction with applications. New York, NY: Springer-Verlag. 379 p.
- Kloden, P.E., and E. Platen. 1992. Numerical solution
of stochastic differential equations. Berlin–Heidelberg:
Springer-Verlag. 636 p.
- Shiryaev, A.N. 1996. Probability. NewYork, NY:Springer-Verlag. 624 p.
- Bohacek, S., and B. Rozovskii. 2004. A diffusion model of roundtrip
time. Comput. Stat. Data An. 45(1):25--50.
[+] About this article
Title
STOCHASTIC DIFFERENTIAL SYSTEM OUTPUT CONTROL BY THE QUADRATIC CRITERION. IV. ALTERNATIVE NUMERICAL DECISION
Journal
Informatics and Applications
2020, Volume 14, Issue 1, pp 24-30
Cover Date
2020-03-30
DOI
10.14357/19922264200104
Print ISSN
1992-2264
Publisher
Institute of Informatics Problems, Russian Academy of Sciences
Additional Links
Key words
stochastic differential equation; optimal control; Bellman function; linear differential equations of parabolic type; Kolmogorov equation; Feynman-Katz formula; computer simulations; Monte-Carlo method
Authors
A. V. Bosov  and A. I. Stefanovich
Author Affiliations
Institute of Informatics Problems, Federal Research Center "Computer Science and Control" of the Russian Academy of Sciences, 44-2 Vavilov Str., Moscow 119333, Russian Federation
|