Resource management problems are as a rule stage structured, i.e. decisions are made along a trajectory where one decision affects subsequent. Stochastic phenomena are ubiquitous in resource management: climate change and wind throw risk are but two pertinent examples. Combine these two elements and Markov Decision Processes (MDP) stands out as one of the most powerful tools for decision making.
The course focuses on formulation of MPD problems and appropriate solution methods. Special attention is given to large, multidimensional problems. This involves the introduction of new methods derived from artificial intelligence, such as Q-learning, and extensions of ordinary dynamic programming in the form of decision graphs. A special section is devoted to partially observable MDP:s.