Hello everyone,

Thanks for signing up for our mailing list! Our first SPEECS seminar meeting will be today at noon in EECS, room 2311. Please fill out the food form before attending, so we can buy enough pizza for everyone.

If you have research to share, please volunteer to present using this link. Currently, there is no one scheduled for 09/25 (our next seminar date). As a token of gratitude, presenters get to choose a customized meal from a selection of local restaurants (e.g. Zingerman’s, Jerusalem Garden, etc.).

All seminar info is available on the SPEECS website, and a Google calendar link with dates/times/presenters is can be found here. If you have any questions, you can contact Zongyu Li or me directly, or email speecs.seminar-requests@umich.edu. Suggestions are always welcome :)

Speaker: Zeyu Sun

Topic: Minimum-Risk Recalibration of Classifiers

Abstract: Recalibrating probabilistic classifiers is vital for enhancing the reliability and accuracy of predictive models Despite the development of numerous recalibration algorithms, there is still a lack of a comprehensive theory that integrates calibration and sharpness (which is essential for maintaining predictive power) In this paper, we introduce the concept of minimum-risk recalibration within the framework of mean-squared-error (MSE) decomposition, offering a principled approach for evaluating and recalibrating probabilistic classifiers Using this framework, we analyze the uniform-mass binning (UMB) recalibration method and establish a finite-sample risk upper bound of order \(\tilde{O}(B/n+1/B^2)\) where \(B\) is the number of bins and \(n\) is the sample size By balancing calibration and sharpness, we further determine that the optimal number of bins for UMB scales with \(n^{1/3}\), resulting in a risk bound of approximately \(O(n^{−2/3})\). Additionally, we tackle the challenge of label shift by proposing a two-stage approach that adjusts the recalibration function using limited labeled data from the target domain Our results show that transferring a calibrated classifier requires significantly fewer target samples compared to recalibrating from scratch We validate our theoretical findings through numerical simulations, which confirm the tightness of the proposed bounds, the optimal number of bins, and the effectiveness of label shift adaptation.

Supplementary link: https://arxiv.org/abs/2305.10886

Mirror: http://websites.umich.edu/~speecsseminar/presentations/20230911/

Thanks,

Matt Raymond