Instructor: Prof. Michael N. Katehakis, Ph.D.     Email

• Webex Link

Class Notes and Assignments

1. Two Basic Models
• Class Notes
• Class Notes 2
• Study Chapters 1 and 2 of the Foundations of Stochastic Inventory Theory By Evan L. Porteus text
2. MDP Models
• Class Notes
• Study Chapters 3 and section 4.1 of the Foundations of Stochastic Inventory Theory By Evan L. Porteus text
3. MDP Models - continued
• Class Notes
• Study Chapter 5 Foundations of Stochastic Inventory Theory By Evan L. Porteus text
4. Auction Model #1
• Class Notes
• Related Paper
5. Computational Methods for MDP Models
• Class Notes
• Excel file
6. Computational Methods for MDP Models Continued
• Class Notes
• Excel file updated
• Review Chapters 4 and 6 of Derman's text
7. MDP Models in SCM
• Class Notes
• Derman Optimal Replacement
• Recent Paper on (s,S) policies
• Multi Location Allocation
• DP version of the Allocation Problem
8. Reinforcement Learning for SCM
• Basic Example
• Class Notes 1
9. Estimation
• Class Notes 1
• Class Notes 2
• Excel Example for Bayes Estimation
• Classic Estimation paper
10. Multi-armed Bandits
• Class Notes
• Python Scripts
• TS Sampling Notes
11. RL Recent papers
• Reinforcement learning: A comparison of UCB versus alternative adaptive policies
• Accelerating the Computation of UCB and Related Indices for Reinforcement Learning
• Review Solving LPs with Python.
12. Further Reading
• Computing Optimal Policies for Markovian Decision Processes Using Simulation
• Temporal-difference (TD) Learning Method
• Neural Networks for Offline RL
• On Airline Crew Pairing