Applications of Markov decision processes to investment-related problems

Abstract

Markov Decision Processes (MDPs) offer a powerful framework for solving sequential decision making problems under uncertainty, where each decision influences the future state of a system. This dissertation investigates the application of MDPs to investment related problems, focusing on portfolio optimization and consumption-investment decisions within dynamic financial markets. The dissertation begins by defining MDPs and highlighting their relevance to financial optimization problems. An important objective of this research is to explore the existence of optimal portfolios under various conditions in both investment-only and consumption-investment scenarios. The investment-only case is primarily concerned with maximizing the expected utility of the terminal wealth, while the consumption-investment case seeks to optimize the expected utility of the terminal wealth along with the expected utility of intermediate consumption throughout the investment period. Both scenarios are modelled as utility maximization problems. The dissertation presents the necessary financial market framework, followed by a detailed analysis of MDPs and the Bellman optimality equations. This is then extended to investment problems for the power and exponential utility functions for the investment only and consumption-investment scenarios. The theory presented is then applied to a simulated dataset to demonstrate how an optimal portfolio strategy for both scenarios can be found using a binomial model for the financial market.