Mike Ludkovski

Optimal Stopping and Optimal Switching for Hidden Markov Models

We study optimal stopping and optimal switching problems for hidden Markov chains with Poissonian information structures. In our model, the controller maximizes expected rewards that depend on an unobserved Markovian environment with information collected through a (compound) Poisson observation process. Examples of such systems arise in investment timing, reliability theory, sequential tracking, and economic policy making. We solve the problem by performing Bayesian updates of the posterior likelihoods of the unobservable and studying the resulting optimization problem for a piecewise-deterministic process. We then prove the dynamic programming principle and explicitly characterize an optimal strategy. We also provide an efficient numerical scheme and illustrate our results with several computational examples.

This is based on joint work with Semih Sezer and Erhan Bayraktar (U of M).