ORF 411

**Operations and Information Engineering**

Professor Warren B. Powell and Dr. Hugo Simao

ORF 411 serves as the capstone course for the Department of Operations Research and Financial Engineering, and as such, helps to define the teaching mission of ORFE at the undergraduate level

This course could easily have several names: *Dynamic Resource Management*, *Information and Decision Engineering*, or *Stochastic Systems Analysis *would be equally appropriate. The students in this course at Princeton have already had courses in statistics, probability and linear programming.

First half – Resource management

Second half – Information as an active resource

Course lectures (new!)

In its current form, the course focuses on the following themes:

**Reinforcement of the core program** – Students learn the skills of statistics, probability and stochastic processes, and optimization in the core of the ORFE curriculum. As the capstone course, ORF 411 integrates all three of these disciplines, using the contextual domain of “resource management,” while building modeling and problem solving skills.

**Resource management**– We develop the vocabulary of “resource management” as a general vocabulary for describing a wide range of problem classes. We define three primary resource classes: physical, financial and informational. There are also three secondary resource classes, including time, energy and goals (or targets). Goals might be the allowable level of risk, or the fuel standards of an automotive fleet. The first half of the course focuses on physical and financial resources as active resource classes, with information as a passive resource class. The second half of the course focuses on information as an active resource.

**Modeling stochastic, dynamic systems** – Students learn how to model stochastic, dynamic systems by breaking them into five dimensions: states, actions, exogenous information, the transition function and the objective function. This modeling framework is illustrated repeatedly with a range of problems.

**Policies** – We develop the idea of *policies* for making decisions, and define four fundamental classes of policies: cost function approximations, lookahead policies, policy function approximations, and policies based on value function approximations (dynamic programming)*.*

**The OJ game team competition** – Students advance their skills in the context of a complex resource allocation problem in the form of the Orange Juice Game. Teams (typically five students) have to control the decisions of a orange juice products company, including purchasing orange juice futures, the capacity of manufacturing and storage facilities, and the critical decision of pricing with unknown demand functions that vary by region. The OJ game develops teamwork skills, modeling skills (they have to model portions of the OJ game in a separate, written report) and problem solving skills (they have to design effective policies). For an article on the game and a short video, click here.

**First half – Resource management**

The first half of the course develops skills in the modeling and solution of fundamental resource management problems, including

What is a resource? We develop a formal concept of a resource, contrast “resources” “assets” and “commodities” and define the three major and three minor resource classes.

Resource allocation problems (convex and nonconvex problems with an S-curve cost function).

Deterministic sequential allocation problems, using dynamic programming and a parametric policy.

The newsvendor problem (this is where we introduce uncertainty in an optimization setting)

Sequential acquisition problems with random demands, including lagged problems.

Demand management (using the setting of airline revenue management)

Multiple resource types with substitution.

The theme of “resource allocation” makes it possible to illustrate these ideas using a variety of applications of current importance. Today these include energy systems, health services, finance, human resource planning and supply chain management. But the material in the course is easily adapted to other applications.

**Second half – Information as an active resource**

Most of this material is presented in the context of information as a passive resource, which is the classical model of information as an exogenous process. In the second half of the course, students learn that they can control information (information as an active resource). This theme is developed with the following topics:

What is information? We contrast data, information and knowledge, passive and active information classes, and define four major classes of information used in different types of decisions.

Optimal learning – A two-lecture sequence introduces students to the economics of information acquisition, and the efficient collection of information using the knowledge gradient. The concavity and nonconcavity of the value of information is presented.

The beer game – We use the Princeton version of the classic beer game which provides a vivid illustration of information (sheets of paper) moving one way, determining the flow of physical resources (beer) which moves the other way. Students have to make their own decisions, dealing with both uncertainty and, in some cases, deception.

The two agent newsvendor problem – This is an exercise in active misinformation to achieve goals. Students learn that some level of misinformation is needed for effective operations.

The IPO pricing game – An information sharing problem that arises in the pricing of IPOs. A model is presented that demonstrates that there is an optimal level of misinformation to achieve the best possible price for an IPO.

Guest speakers from industry are used to highlight challenges in the real world. Recent speakers have included the CEO of PSE&G, the project lead at IBM on a human resource allocation model, and the head of the operations research group at UPS who is responsible for process change using information technology and modeling. During these presentations, students prepare brief summaries of the problems the speaker is presenting by defining the state variable (the information being used), the decisions being made, exogenous information, and goals/objectives.

Each lecture is a pdf of a powerpoint presentation (let me know if you would like the original powerpoint files, and I will make these available), which serves as the book for this course.

The course focuses on basic building-block problems that arise in a wide range of resource allocation problem. In the process, we emphasize the modeling of information, transitioning from deterministic models, to stochastic models (information is a purely exogenous process), and finishing with the second half of the course which focuses on information as an active (endogenously controllable) resource.

**Overview**

Introduction – Resource allocation arises in a wide variety of settings. This lecture provides a few examples.

What is a resource? – There are six classes of resources: three active classes (physical, financial and informational) and three passive classes (time, energy and “targets”).

**Deterministic resource allocation problems**

The budgeting problem – Basic problem of allocating resources over time with deterministic forecasts.

Deterministic inventory problems – Students learn how to optimize an infinite horizon problem by minimizing average cost per time period, producing the classic EOQ formula.

**Stochastic resource allocation (single resource type)**

The newsvendor problem – First introduction to decisions under uncertainty

Modeling stochastic, dynamic problems – Students learn the five core components of any stochastic, dynamic system: states, actions, exogenous information, the transition function and the objective function. Students are taught to model problems *before* finding a policy.

The four classes of policies – There appear to be four fundamental classes of policies: policy function approximations (PFAs), cost function approximations (CFAs), policies based on value function approximations (VFAs), and lookahead policies.

Modeling storage processes (two lectures) – This set of lectures describes four classes of “storage” problems of increasing complexity.

**Substitutable resources (linear programming review)**

Substitutable resources I – Students are assumed to have a background in linear programming. We use the basic “transportation problem” to revisit the simplex method where we emphasize graphical methods, while still making the link to the linear algebra of linear programming.

Substitutable resources II – Extending the first lecture, we show how to use the basis to understand the economics of substitutable resources through an understanding of the basis.

**Demand as a resource**

Demand management – Demand is a resource too. We use a popular example from airline yield management to illustrate booking profiles, which are optimized using a simple stochastic gradient algorithm (carefully illustrated).

**Midterm review**

The Orange Juice Game – Introduction to the team competition known as the OJ game. The spreadsheet-based handbook is here. The OJ game is a major competition – students have to fill in a spreadsheet with 600 cells. Click here for an example.

**Information as an active resource**

What is information? – We contrast common concepts of data, information and knowledge, and illutrate five classes of information.

Optimal learning I – Concepts and heuristic policies – Click here for more information on optimal learning. This lecture introduces the basic idea of actively collecting information for the purpose of learning.

Optimal learning II – The knowledge gradient – The knowledge gradient is basically a derivative, where we collecting information that offers the highest marginal value.

The Princeton beer game – This is a highly streamlined version of the classic beer game. This is best done in a class with continuous tables joining students.

The two agent newsvendor problem I – This is a nice exercise in the manipulation of information. Students use a simple two-player game to learn how to manage roles of field agent (high cost of underage) and central agent (balanced attitude toward underage and overage) through careful misinformation.

The two agent newsvendor problem II – Spreadsheet for summarizing results is available here.

Information exchange and IPO pricing – Based on the summer internship of an undergraduate, this lecture develops the concept of optimal lying.