Statistical Approaches to Understanding Modern ML Methods

Aug 2-4, 2021
University of Wisconsin–Madison

When we use modern machine learning (ML) systems, the output often consists of a trained model with good performance on a test dataset. This satisfies some of our goals in performing data analysis, but leaves many unaddressed — for instance, we may want to build an understanding of the underlying phenomena, to provide uncertainty quantification about our conclusions, or to enforce constraints of safety, fairness, robustness, or privacy. As an example, classical statistical methods for quantifying a model’s variance rely on strong assumptions about the model — assumptions that can be difficult or impossible to verify for complex modern ML systems such as neural networks. 

This workshop will focus on using statistical methods to understand, characterize, and design ML models — for instance, methods that probe “black-box” ML models (with few to no assumptions) to assess their statistical properties, or tools for developing likelihood-free and simulation-based inference. Central themes of the workshop may include:

  • Using the output of a ML system to perform statistical inference, compute prediction intervals, or quantify measures of uncertainty

  • Using ML systems to test for conditional independence

  • Extracting interpretable information such as feature importance or causal relationships

  • Integrating likelihood-free inference with ML

  • Developing mechanisms for enforcing privacy, robustness, or stability constraints on the output of ML systems

  • Exploring connections to transfer learning and domain adaptation

  • Automated tuning of hyperparameters in black-box models and derivative-free optimization

Participants

Demba Ba
Demba Ba
Peng Ding
Peng Ding
Edward Kennedy
Edward Kennedy
Aaditya Ramdas
Aaditya Ramdas
Avrim Blum
Avrim Blum
Dylan Foster
Dylan Foster
Lihua Lei
Lihua Lei
Hanie Sedghi
Hanie Sedghi
Kamalika Chaudhuri
Kamalika Chaudhuri
Zaid Harchaoui
Zaid Harchaoui
Sharon Li
Sharon Li
Vladimir Vovk
Vladimir Vovk
Kyle Cranmer
Kyle Cranmer
Lucas Janson
Lucas Janson
Po-Ling Loh
Po-Ling Loh
Yao Xie
Yao Xie