ML-Opt @ UWash: Omid Sadeghi
2021-02-19 13:30 - 14:30 (America/Chicago)
Title:
Abstract:

IFDS Ideas Forum: Blake Mason
2021-02-22 13:30 - 14:30 (America/Chicago)
PERSON: Blake Mason
TITLE: Metric Learning from Labelled Data is Really Hard
ABSTRACT:
BIO:

SILO: Jose Blanchet
2021-02-24 12:30 - 13:30 (America/Chicago)
Title: TBD

ML-Opt @ UWash: Andrew Wagenmaker
2021-02-26 13:30 - 14:30 (America/Los_Angeles)

IFDS Ideas Forum: Julian Katz-Samuels
2021-03-01 13:30 - 14:30 (America/Chicago)
Person: Julian Katz-Samuels
Title:
Abstract:
Bio:

SILO: Tuo Zhao
2021-03-03 12:30 - 13:30 (America/Chicago)

IFDS All-Hands: Yang Liu
2021-03-05 13:30 - 14:30 (America/Los_Angeles)
Prof. Yang Liu, Computer Science Dept, UC Santa Cruz
Title: Consequential Machine Learning
Abstract: Recent findings cautioned the potential unfairness issues that can arise when deploying a machine learning model. Correspondingly, treatments have been proposed to add fairness guarantees when training such statistical models. An often overlooked question is “what after?” One can discover ways to improve the model after deployment, and after receiving feedback from human subjects that saw these algorithmic treatments. But the unfortunate fact is once a model is deployed, it will start to impact a later decision and the society for a long period of time. In this talk, I am going to introduce some of the recent works that initialize our attempt to understand the consequences of the deployment of a machine learning algorithm. We will first show that an appeared-to-be fair algorithm might not necessarily help reduce disparities among societal groups that we aim to protect. The long-term impacts of a sequence of deployments of machine learning models will depend on the induced dynamics of the underlying populations’ qualification. Then I discuss how to use the design and deployment of machine learning to induce proper dynamics by offering human agents actionable recourse to improve their qualifications.
All Hands titles and abstracts are tentative, as of the posting date.

IFDS Ideas Forum: Nicolas Garcia Trillos
2021-03-08 13:30 - 14:30 (America/Chicago)
Person: Nicolas Garcia Trillos
Title: Adversarial classification, optimal transport, and geometric evolutions
Abstract:
Bio: I am currently an Assistant Professor in the Department of Statistics at the University of Wisconsin Madison. From 2015 – 2018, I was a Prager Assistant Professor (postdoctoral position) at Brown University. I finished my Ph.D in mathematics at Carnegie Mellon University in 2015; my adviser was Dejan Slepčev. I received my Bachelor’s degree in mathematics from Universidad de Los Andes in Bogotá, Colombia, in 2010.

SILO: Zhao Song
2021-03-10 12:30 - 13:30 (America/Chicago)
Title: Faster Optimization: From linear programming to semidefinite programming
Abstract: Many important real-life problems, in both convex and non-convex settings, can be solved using path-following optimization methods. The running time of optimization algorithms is typically governed by two components — the number of iterations and the cost-per-iteration. For decades, the vast majority of research effort was dedicated to improving the number of iterations required for convergence. A recent line of work of ours shows that the cost-per-iteration can be dramatically improved using a careful combination of dynamic data structures with `robust’ variants of the optimization method. A central ingredient is the use of randomized linear algebra for dimensionality reduction (e.g., linear sketching) for fast maintenance of dynamic matrix problems. This framework recently led to many breakthroughs on decade-old optimization problems.
In this talk, I will present the framework underlying these breakthroughs, focusing on faster algorithms for linear programming and semidefinite programming. We will first present how to use the above idea to speed up general LP solvers by providing an n^omega + n^{2+1/18} time algorithm. We then show how to apply similar ideas to SDP solvers by providing an n^omega + n^{2+1/4} time algorithm. For the current omega = 2.373, we can solve LP and SDP as fast as solving linear systems.
This is a joint work with
Baihe Huang (undergraduate at Peking University),
Shunhua Jiang, Runzhou Tao, Hengjie Zhang (Ph.D. at Columbia University),
Omri Weinstein (Professor at Columbia University)
LP paper : https://arxiv.org/abs/2004.07470
SDP paper : https://arxiv.org/abs/2101.08208

UNTIL FURTHER NOTICE: Seminars are virtual. Sign up for the SILO email list to receive the links to each talk at https://groups.google.com/ and browse for silo

SILO: Yuanzhi Li
2021-03-10 13:00 - 13:30 (America/Chicago)
Title: TBD
UNTIL FURTHER NOTICE: Seminars are virtual. Sign up for the SILO email list to receive the links to each talk at https://groups.google.com/ and browse for silo

ML-Opt @ UWash: Vincent Roulet
2021-03-12 13:30 - 14:30 (America/Chicago)
Title: Global convergence of first-order methods for nonlinear control problems
Abstract:

IFDS Ideas Forum: Aditya Kumar Akash & Sixu Li
2021-03-15 13:30 - 14:00 (America/Chicago) | Webex
Aditya Kumar Akash & Sixu Li
Title: Deep fusion through Wasserstein barycenters in TLp spaces
Abstract:

SILO: Sebastien Bubeck
2021-03-17 12:30 - 13:30 (America/Chicago)
Title: A law of robustness for two-layers neural networks
UNTIL FURTHER NOTICE: Seminars are virtual. Sign up for the SILO email list to receive the links to each talk at https://groups.google.com/ and browse for silo

IFDS Ideas Forum: Tara Javidi
2021-03-22 13:30 - 14:30 (America/Chicago)
Person: Tara Javadi
Title:
Abstract:
Bio:

SILO: Alon Orlitsky
2021-03-24 08:00 - 17:00 (America/Chicago)

IFDS Ideas Forum: Jeffrey Covington
2021-03-29 13:30 - 14:00 (America/Chicago) | Webex
Title:
Abstract:
Jeffrey Covington (Mathematics), advised by Nan Chen (Mathematics) and Sebastien Roch (Mathematics), works on data assimilation for model state estimation and prediction. His primary focus is on models with nonlinear and non-Gaussian features, which present problems for traditional data assimilation techniques. Currently he is working on developing techniques for Lagrangian data assimilation problems, which typically involve high-dimensionality and strong nonlinear interactions.

SILO: Jean Honorio
2021-03-31 12:30 - 13:30 (America/Chicago)

IFDS All-Hands: Chaobing Song & Xuezhou Zhang
2021-04-02 13:30 - 14:30 (America/Los_Angeles)
Speaker 1: Dr. Chaobing Song, IFDS Postdoctoral Scholar at U Wisconsin (advised by Prof.s Jelena Diakonikolas and Steve Wright)

Title: Closing Convergence Gaps for Both Smooth and Nonsmooth Convex Finite-Sums

Abstract: In the Foundation of Data Science, optimization problems with a finite-sum structure widely exist, such as the classical empirical risk minimization problem and deep neural network. In the past decade, one main concern in the study of first-order methods is to study how the finite-sum structure influences optimization efficiency and scalability. In this talk, based on my work (https://arxiv.org/abs/2006.10281, https://arxiv.org/abs/2102.13643), I will talk about a consistent approach based on dual averaging and a particularly designed initialization strategy to close convergence gaps for smooth convex finite-sums and nonsmooth convex finite-sums. For the first time, the proposed VRADA algorithm (https://arxiv.org/abs/2006.10281) matches the lower bounds of all the three regimes for smooth convex finite-sums; for the first time, the proposed VRPDA^2 algorithm (https://arxiv.org/abs/2102.13643) shows an O(n) improvement theoretically over existing deterministic methods and stochastic primal-dual coordinate methods, where n is the number of data samples. Both algorithms also have good empirical performance.

Speaker 2: Xuezhou Zhang, IFDS RA (advised by Prof. Jerry Zhu)

Title: Statistical Robustness in Reinforcement Learning

Abstract: Traditional robust statistics studies the problem of statistical estimation under data contamination. In this talk, we extend this investigation to decision-making problems. In the first half of the talk, I will illustrate the concept and unique challenge of robust Decision making using a multi-armed bandit as an example. In the second half of the talk, I will present some of the lower and upper-bound results we have already established, and discuss several open directions. This talk is partially based on our recent paper [https://arxiv.org/pdf/2102.05800.pdf].

—-
Speaker Bios:

Dr. Chaobing Song: I am a postdoc researcher at University of Wisconsin-Madison with Prof. Jelena Diakonikolas and Prof. Stephen J Wright. My research interests include optimization and machine learning. I obtained my Ph.D. degree at Tsinghua University in 2020 supervised by Prof. Yi Ma from University of California, Berkeley (who is an affiliate professor at Tsinghua-Berkeley Shenzhen Institute, Tsinghua University). I spent two wonderful years at Berkeley and finished my Ph.D. thesis there. The main focus of my current research is to apply a general, powerful and concise framework [arXiv] to optimization problems in machine learning. The development based on this framework substantially improves complexity results in many well-known settings, such as the setting of nonsmooth convex finite-sums in this talk.

Xuezhou Zhang: Xuezhou is a Ph.D. candidate in the computer sciences department at the University of Wisconsin-Madison, advised by Professor Jerry Zhu. Before coming to Madison, he obtained a Bachelor of Applied Mathematics degree from UCLA. His research interests lie in machine learning and reinforcement learning, and his recent research focuses on designing reinforcement learning algorithms that learn more adaptively and robustly in non-stochastic environments.

These talks are remote via zoom. Please contact the organizer if you need the link.
All-Hands titles and abstracts are tentative, as of the posting date.

IFDS Ideas Forum: Liu Yang
2021-04-05 13:30 - 14:30 (America/Chicago)
Person: Liu Yang
Title:
Abstract:
Bio:

SILO: Rebecca Willett
2021-04-07 12:30 - 13:30 (America/Chicago)

ML-Opt: Krishna Pillutla
2021-04-09 13:30 - 14:30 (America/Los_Angeles)
Title: Distributionally Robust Machine Learning with the Superquantile 1) For Supervised Learning, 2) For Federated Learning

Abstract: I will talk about distributionally robust machine learning, a principled approach for robust performance across subpopulations, and shifting distributions. We will focus on the superquantile, a.k.a. the Conditional Value at Risk (CVaR), which was popularized by the seminal work of UW’s own R. T. Rockafellar and co-authors in the field of computational finance and economics in the early 2000s.
We will first review the use of the superquantile for distributionally robust supervised learning. We will prove a generalization bound from first principles.
Second, we will discuss an application of the superquantile in the field of federated learning, i.e., the distributed training of machine learning models on mobile phones. We will quantify the extent to which a user conforms to the population distribution and show how the superquantile can be leveraged to improve performance on users who do not conform to the population. We will round of the discussion with a communication-efficient training algorithm and experimental results and heterogeneous datasets.

Bio: Krishna Pillutla is a 5th year Ph.D. student at the Paul G. Allen School of Computer Science and Engineering at the University of Washington, where he is advised by Zaid Harchaoui and Sham Kakade. Krishna is broadly interested in machine learning and optimization and works in the particular areas of structured prediction and federated learning. Krishna was a 2019-20 JP Morgan Ph.D. Fellow.

IFDS Ideas Forum: Vivak Patel
2021-04-12 13:30 - 14:30 (America/Chicago)
Person: Vivak Patel
Title: Adaptive Iterative Methods for Linear Systems
Abstract:
Bio: My research is at the intersection of numerical optimization and statistical estimation with applications to dynamical systems. On the one hand, I interpret this as using numerical optimization ideas to design statistical estimators that are practically computable, yet still retain certain desirable statistical properties. On the other hand, I interpret this as using statistical ideas to design optimization methods for solving minimization problems that involve uncertainty. Beyond these two main areas of work, I also do some work in machine learning, numerical linear algebra and load modeling for power systems.

SILO: Merve Bodur
2021-04-14 12:30 - 13:30 (America/Chicago)
Title: Copositive Duality for Discrete Markets and Games
Models including binary decisions are often modelled as mixed-integer programs (MIPs). Such models are nonconvex and lack strong duality, which prevents the use of tools such as shadow prices and KKT conditions. For example, in convex markets, shadow (dual) prices are associated with market equilibrium, and for convex games the existence and uniqueness of Nash equilibrium can be proven via fixed-point theorem and KKT conditions. Those results are lacking in their nonconvex counterparts. We use copositive programming to formulate discrete problems in applications including nonconvex energy markets and nonconvex games, to leverage its convexity and strong duality features. We obtain several novel theoretical and numerical results for those applications, including a new revenue-adequate pricing scheme for energy markets, and existence, uniqueness, and KKT conditions for the pure-strategy Nash equilibrium in discrete games. We also propose a novel and purely MIP-based cutting-plane algorithm for mixed-integer copositive programs, and employ it in our applications. This is a joint work with Cheng Guo and Josh A. Taylor.
UNTIL FURTHER NOTICE: Seminars are virtual. Sign up for the SILO email list to receive the links to each talk at https://groups.google.com/ and browse for silo

ML-Opt: Adhyyan Narang
2021-04-16 13:30 - 14:30 (America/Los_Angeles)
Title: Classification vs Adversarial Examples for the Overparameterized linear model.
(Joint work with Vidya Muthukumar and Anant Sahai at UC Berkeley)
Abstract: In modern machine learning, overparameterized models are often used. It has been empirically observed that these often generalize well and display double-descent, but are susceptible to adversarial perturbations. Past theoretical explanations of these phenomena usually focus on linear models where the adversary has the power to perturb the features directly. However, the field of meta-learning has revealed that neural networks can be interpreted as first learning a feature-representation and then learning the best linear model on these learned features. The role of lifting in adversarial susceptibility of models is largely unaddressed, primarily because the problem of finding an adversarial example for lifted models is nonconvex and difficult to solve.
In this talk, I will use concepts from signal-processing to propose a toy model that exhibits all of the aforementioned phenomena, most crucially lifting. The toy nature of the model allows us to overcome the challenge of solving the adversarial-search problem. We learn that the adversarial vulnerability arises because of a phenomena we term spatial localization: the predictions of the learned model are markedly more sensitive in the vicinity of training points than elsewhere. Despite the adversarial susceptibility, we find that classification using spatially localized features can be “easier” i.e. less sensitive to the strength of the prior than in independent feature setups.
Bio: Adhyyan Narang (ECE) is a first-year PhD student, advised by Maryam Fazel and Lilian Ratliff. He is interested in fundamental theoretical questions about learning from data, and broadly works in the intersection of machine learning, optimization and game theory.

IFDS Ideas Forum: Zhiyan Ding
2021-04-19 13:30 - 14:30 (America/Chicago)
Person: Zhiyan Ding
Title: Connection between optimization and sampling
Abstract:
Bio: I am a third-year PhD student in the Department of Mathematics, University of Wisconsin-Madison. My advisor is Qin Li.

SILO: Emmanuel Abbe
2021-04-21 12:30 - 13:30 (America/Chicago)

ML-Opt@UW: Yue Sun
2021-04-23 13:30 - 14:30 (America/Los_Angeles)
Title: Subspace Based Meta-learning
Abstract:
Meta-learning typically involves two phases. First, one learns a suitable representation from the previously seen tasks. Secondly, this representation is used for learning a new task using only a few samples (i.e., few-shot learning). In this talk I will discuss:
1. Linear meta learning: sample complexity of representation learning with general covariance
2. Linear meta learning: algorithm & analysis for overparameterized few-shot learning
3. Generalization to nonlinear meta-learning

Bio:
Yue Sun is a 5th year PhD student from University of Washington, Seattle. He is interested in theoretical understanding of optimization, ML and control. His research works are about:
1. Nonconvex optimization on Riemannian manifolds (UW)
2. Low order linear system identification (UW)
3. Subspace based meta-learning (UW)
4. Nonconvex optimization applied in optimal control (UW)

5. Online optimization for video coding (Google, 2019)
6. Compressive sensing and phase retrieval (Ohio State U, 2015; Nokia Bell Labs, 2021)

IFDS Ideas Forum: Shi Chen
2021-04-26 13:30 - 14:00 (America/Chicago) | Webex
Title:
Abstract:
Shi Chen (Mathematics), advised by Qin Li (Mathematics) and Stephen J. Wright (Computer Science), works on problems in the interdisciplinary area of applied math and machine learning. He is interested in connecting machine learning to various mathematical physics problems, including the homogenization of PDEs and inverse problems. His recent work focuses on applying deep learning to PDE inverse problems.

IFDS Ideas Forum: Shashank Rajput
2021-04-26 14:00 - 14:30 (America/Chicago) | Webex
Title:
Abstract:
Shashank Rajput (Computer Science), advised by Dimitris Papailiopoulos (Electrical and Computer Engineering), Kangwook Lee (Electrical and Computer Engineering) and Stephen Wright (Computer Science), works on problems in distributed machine learning and optimization. He works on developing techniques which are fast and scalable for practical use, and have theoretical guarantees. Currently, he is working on finding better shuffling mechanisms that beat random reshuffling as well as developing algorithms for training neural networks by only pruning them.

SILO: Martin Wainwright
2021-04-28 12:30 - 13:30 (America/Chicago)
