Warren B. Powell
Professor Emeritus, Princeton University
Chief Analytics Officer, Optimal Dynamics
Sequential decision problems arise in virtually every form of human processes: transportation and logistics, supply chain management, energy, health (from public health to medical decision making), finance, e-commerce, laboratory sciences, …
Sequential Decision Analytics and Modeling uses a teach-by-example style to illustrate a universal framework for modeling sequential decision problems. The universal framework applies to any sequential decision problem, from active learning problems up through complex resource allocation problems. Chapters are accompanied by python modules that have implemented the models, but the book should be of value even to people not interested in writing code (click on the book to download a draft copy).
Updated August 20, 2022.
Chapter 1 illustrates the core modeling framework using two inventory examples, and introduces the four classes of policies that encompass any method used for making decisions.
Chapters 2-6 use a series of examples designed to illustrate each of the four classes of policies, along with different styles for modeling uncertainty. We pause in chapter 7 to revisit the universal modeling framework, drawing on the applications from chapters 2-6 to illustrate state variables (including belief states), scalar and vector-valued decisions, uncertainty modeling, different types of objective functions, and the four classes of policies.
The book is designed to be used as a standalone introduction to the vast field of sequential decision analytics. For more information, see the sequential decision analytics website. A youtube video introduction to sequential decision analytics is available here.
For advanced readers interested in developing models and algorithms, I recommend my new book Reinforcement Learning and Stochastic Optimization: A unified framework for sequential decisions.
While the entire book is focused on sequential decisions under uncertainty, the book assumes no background in any form of stochastic optimization or dynamic programming. I taught out of an earlier version of this book in an undergraduate course at Princeton (click here for the website).
There is additional material, such as spreadsheets and datasets, at https://castlelab.princeton.edu/sdamodelingsupplements/
Please add your comments:
I am doing a thorough revision in preparation for publishing the book with NOW, publisher of the Foundations and Trends (r) series. On this page, I will be sharing the revision with everyone, and inviting comments.
Most chapters are accompanied by a Python module at https://tinyurl.com/sdagithub. These will be built into exercises in the book.