Academics / Courses / DescriptionsCOMP_SCI 497: Calibration (Foundations of Trustworthy ML)
VIEW ALL COURSE TIMES AND SESSIONS
Prerequisites
PhD in CS, CE, EE, IEMS, Econ, MECS, or Applied mathDescription
This is an advanced topics seminar that will consider theoretical topics related to calibration. Calibrated predictions are predictions that are empirically correct. For example, the weather forecast is calibrated if for each predicted probability of rain p, when the prediction is p chance of rain, the empirical fraction of times that it rains is also p. Calibrated predictions have the property that it is optimal for a decision maker to optimize assuming that the prediction is correct. Calibrated predictions also have applications to explainable AI and fairness. For these reasons, there has been a considerable and rich recent literature developing the algorithmic foundations of calibration. The readings of the course will be drawn from the recent and classic literature pertaining to calibration. Topics include: online learning and swap regret, prediction for decision making, measuring calibration error, online calibration, calibration and machine learning, multi-calibration, fairness, omni-prediction, correlated equilibrium, manipulation of learning algorithms, and calibration for language models.
COURSE INSTRUCTOR: Prof. Jason Hartline