Sequential Decision Analytics and Modeling

Warren B. Powell
Professor Emeritus, Princeton University
Chief Analytics Officer, Optimal Dynamics

 


The book is now in print!  You can purchase the book at the discounted price of $50 by clicking here, and entering the discount code 226485. By agreement with the publisher, the free download of the pdf will become available on June 1, 2023. (Click here to download table of contents.)

Sequential decision problems arise in virtually every form of human processes: transportation and logistics, supply chain management, energy, health (from public health to medical decision making), finance, e-commerce, laboratory sciences, … 

Sequential Decision Analytics and Modeling uses a teach-by-example style to illustrate a universal framework for modeling sequential decision problems.  The universal framework applies to any sequential decision problem, from active learning problems up through complex resource allocation problems.  Chapters are accompanied by python modules that have implemented the models, but the book should be of value even to people not interested in writing code.

Updated November 23, 2022.

Summary:

Chapter 1 illustrates the core modeling framework using two inventory examples, and introduces the four classes of policies that encompass any method used for making decisions.

Chapters 2-6 use a series of examples designed to illustrate each of the four classes of policies, along with different styles for modeling uncertainty.  We pause in chapter 7 to revisit the universal modeling framework, drawing on the applications from chapters 2-6 to illustrate state variables (including belief states), scalar and vector-valued decisions, uncertainty modeling, different types of objective functions, and the four classes of policies.

The book is designed to be used as a standalone introduction to the vast field of sequential decision analytics.  For more information, see the sequential decision analytics website. A youtube video introduction to sequential decision analytics is available here. A webpage of links to books, videos, discussions, and suggested courses is available here.

For advanced readers interested in developing models and algorithms, I recommend my new book Reinforcement Learning and Stochastic Optimization: A unified framework for sequential decisions

Audience

While the entire book is focused on sequential decisions under uncertainty, the book assumes no background in any form of stochastic optimization or dynamic programming. I taught out of an earlier version of this book in an undergraduate course at Princeton (click here for the website). 

Supplementary material

There is additional material, such as spreadsheets and datasets, at https://castlelab.princeton.edu/sdamodelingsupplements/