Academics
  /  
Courses
  /  
Descriptions
  /  
Keep
IEMS 469: Dynamic Programming


VIEW ALL COURSE TIMES AND SESSIONS

Prerequisites

Basic knowledge of probability (random variables, expectation, conditional probability), optimization (gradient), calculus (convergence and norms, fixed point theorem)

Description

This course covers reinforcement learning aka dynamic programming, which is a modeling principle capturing dynamic environments and stochastic nature of events. The main goal is to learn dynamic programming and how to apply it to a variety of problems. The course covers both theoretical and computational aspects.

Learning Objectives

  • Being able to identify when to model as a dynamic program
  • Identify all components of a dynamic programming model
  • Understand the various algorithms for solving a dynamic program 

Topics

  1. Introduction to dynamic programming
  2. Value and policy iterations
  3. Stochastic gradient algorithm
  4. Q-learning and temporal differences
  5. Policy gradient
  6. Actor-critic algorithm
  7. Value function approximation and Monte-Carlo sampling (time permitting)

Materials

  • Warren B. Powell; Approximate Dynamic Programming: Solving the Courses of Dimensionality; John Wiley & Sons, 2007. [topics 5 and 6 not in the textbook]

prerequisites

  • Basic knowledge of probability (random variables, expectation, conditional probability), optimization (gradient), calculus (convergence and norms, fixed point theorem)

Additional information

  • This is an advanced PhD level course and not appropriate for undergraduate and master’s level students.