Relative Optimization of Continuous-Time and Continuous-State Stochastic Systems

This monograph applies the relative optimization approach to time nonhomogeneous continuous-time and continuous-state dynamic systems. The approach is intuitively clear and does not require deep knowledge of the mathematics of partial differential equations. The topics covered have the following distinguishing features: long-run average with no under-selectivity, non-smooth value functions with no viscosity solutions, diffusion processes with degenerate points, multi-class optimization with state classification, and optimization with no dynamic programming.

The book begins with an introduction to relative optimization, including a comparison with the traditional approach of dynamic programming. The text then studies the Markov process, focusing on infinite-horizon optimization problems, and moves on to discuss optimal control of diffusion processes with semi-smooth value functions and degenerate points, and optimization of multi-dimensional diffusion processes. The book concludes with a brief overview of performance derivative-based optimization.

Among the more important novel considerations presented are:

• the extension of the HamiltonJacobiBellman optimality condition from smooth to semi-smooth value functions by derivation of explicit optimality conditions at semi-smooth points and application of this result to degenerate and reflected processes;
• proof of semi-smoothness of the value function at degenerate points;
• attention to the under-selectivity issue for the long-run average and bias optimality;
• discussion of state classification for time nonhomogeneous continuous processes and multi-class optimization; and

• development of the multi-dimensional Tanaka formula for semi-smooth functions and application of this formula to stochastic control of multi-dimensional systems with degenerate points.
  The book will be of interest to researchers and students in the field of stochastic control andperformance optimization alike.

  Solves existing problems without requiring deep knowledge of partial differential equations Presents a new framework for optimization of stochastic systems, promoting new research pathways Shows the reader how to link optimisation of continuous systems with reinforcement-learning algorithms and artificial intelligence

  Autorentext

  Professor Xi-Ren Cao gained his Masters degree in engineering and PhD in applied mathematics from Harvard University in 1982 and 1984 respectively. He has worked in an academic position at numerous institutions, including as a visiting professor at both the University of Massachusetts and the University of Maryland, a chair professor at the Hong Kong University of Science and Technology, and his current position of Chair Professor at Shanghai Jiao Tong University. He has published 125 peer-reviewed journal papers, 12 invited book chapters, and three books in areas related to stochastic and discrete control. He was Editor-in-Chief for Discrete Event Dynamic Systems: Theory and Applications from 2005 to 2014. He has extensive industrial experiences with Digital Equipment Corporation, Massachusetts, and AT&T Labs. He is Fellow of IEEE and IFAC.

Zusammenfassung
"This book develops an alternative viewpoint for stochastic control inspired by ideas rooted in sensitivity formulae and perturbation analysis. ... The theory is accompanied by several examples and exercises from time to time and includes the background material as needed, either in the main text or in an appendix." (Vivek S. Borkar, Mathematical Reviews, April, 2022)

Inhalt

Chapter 1. Introduction.- Chapter 2. Optimal Control of Markov Processes: Innite Horizon.- Chapter 3. Optimal Control of Diffusion Processes.- Chapter 4. Degenerate Diffusion Processes.- Chapter 5. Multi-Dimensional Diffusion Processes.- Chapter 6. Performance-Derivative-Based Optimization.- Appendices.- Index.