IFDS

SILO: Sebastien Bubeck
2021-03-17 12:30 - 13:30 (America/Chicago)
Title: A law of robustness for two-layers neural networks
UNTIL FURTHER NOTICE: Seminars are virtual. Sign up for the SILO email list to receive the links to each talk at https://groups.google.com/ and browse for silo

IFDS Ideas Forum: Tara Javidi
2021-03-22 13:30 - 14:30 (America/Chicago)
Person: Tara Javadi
Title:
Abstract:
Bio:

SILO: Alon Orlitsky
2021-03-24 08:00 - 17:00 (America/Chicago)
Title: TBD
UNTIL FURTHER NOTICE: Seminars are virtual. Sign up for the SILO email list to receive the links to each talk at https://groups.google.com/ and browse for silo

IFDS Ideas Forum: Jeffrey Covington
2021-03-29 13:30 - 14:00 (America/Chicago) | Webex
Title:
Abstract:
Jeffrey Covington (Mathematics), advised by Nan Chen (Mathematics) and Sebastien Roch (Mathematics), works on data assimilation for model state estimation and prediction. His primary focus is on models with nonlinear and non-Gaussian features, which present problems for traditional data assimilation techniques. Currently he is working on developing techniques for Lagrangian data assimilation problems, which typically involve high-dimensionality and strong nonlinear interactions.

SILO: Jean Honorio
2021-03-31 12:30 - 13:30 (America/Chicago)

IFDS All-Hands: Chaobing Song & Xuezhou Zhang
2021-04-02 13:30 - 14:30 (America/Los_Angeles)
Speaker 1: Dr. Chaobing Song, IFDS Postdoctoral Scholar at U Wisconsin (advised by Prof.s Jelena Diakonikolas and Steve Wright)

Title: Closing Convergence Gaps for Both Smooth and Nonsmooth Convex Finite-Sums

Abstract: In the Foundation of Data Science, optimization problems with a finite-sum structure widely exist, such as the classical empirical risk minimization problem and deep neural network. In the past decade, one main concern in the study of first-order methods is to study how the finite-sum structure influences optimization efficiency and scalability. In this talk, based on my work (https://arxiv.org/abs/2006.10281, https://arxiv.org/abs/2102.13643), I will talk about a consistent approach based on dual averaging and a particularly designed initialization strategy to close convergence gaps for smooth convex finite-sums and nonsmooth convex finite-sums. For the first time, the proposed VRADA algorithm (https://arxiv.org/abs/2006.10281) matches the lower bounds of all the three regimes for smooth convex finite-sums; for the first time, the proposed VRPDA^2 algorithm (https://arxiv.org/abs/2102.13643) shows an O(n) improvement theoretically over existing deterministic methods and stochastic primal-dual coordinate methods, where n is the number of data samples. Both algorithms also have good empirical performance.

Speaker 2: Xuezhou Zhang, IFDS RA (advised by Prof. Jerry Zhu)

Title: Statistical Robustness in Reinforcement Learning

Abstract: Traditional robust statistics studies the problem of statistical estimation under data contamination. In this talk, we extend this investigation to decision-making problems. In the first half of the talk, I will illustrate the concept and unique challenge of robust Decision making using a multi-armed bandit as an example. In the second half of the talk, I will present some of the lower and upper-bound results we have already established, and discuss several open directions. This talk is partially based on our recent paper [https://arxiv.org/pdf/2102.05800.pdf].

—-
Speaker Bios:

Dr. Chaobing Song: I am a postdoc researcher at University of Wisconsin-Madison with Prof. Jelena Diakonikolas and Prof. Stephen J Wright. My research interests include optimization and machine learning. I obtained my Ph.D. degree at Tsinghua University in 2020 supervised by Prof. Yi Ma from University of California, Berkeley (who is an affiliate professor at Tsinghua-Berkeley Shenzhen Institute, Tsinghua University). I spent two wonderful years at Berkeley and finished my Ph.D. thesis there. The main focus of my current research is to apply a general, powerful and concise framework [arXiv] to optimization problems in machine learning. The development based on this framework substantially improves complexity results in many well-known settings, such as the setting of nonsmooth convex finite-sums in this talk.

Xuezhou Zhang: Xuezhou is a Ph.D. candidate in the computer sciences department at the University of Wisconsin-Madison, advised by Professor Jerry Zhu. Before coming to Madison, he obtained a Bachelor of Applied Mathematics degree from UCLA. His research interests lie in machine learning and reinforcement learning, and his recent research focuses on designing reinforcement learning algorithms that learn more adaptively and robustly in non-stochastic environments.

These talks are remote via zoom. Please contact the organizer if you need the link.
All-Hands titles and abstracts are tentative, as of the posting date.

IFDS Ideas Forum: Liu Yang
2021-04-05 13:30 - 14:30 (America/Chicago)
Person: Liu Yang
Title:
Abstract:
Bio:

SILO: Rebecca Willett
2021-04-07 12:30 - 13:30 (America/Chicago)

ML-Opt: Krishna Pillutla
2021-04-09 13:30 - 14:30 (America/Los_Angeles)
Title: Distributionally Robust Machine Learning with the Superquantile 1) For Supervised Learning, 2) For Federated Learning

Abstract: I will talk about distributionally robust machine learning, a principled approach for robust performance across subpopulations, and shifting distributions. We will focus on the superquantile, a.k.a. the Conditional Value at Risk (CVaR), which was popularized by the seminal work of UW’s own R. T. Rockafellar and co-authors in the field of computational finance and economics in the early 2000s.
We will first review the use of the superquantile for distributionally robust supervised learning. We will prove a generalization bound from first principles.
Second, we will discuss an application of the superquantile in the field of federated learning, i.e., the distributed training of machine learning models on mobile phones. We will quantify the extent to which a user conforms to the population distribution and show how the superquantile can be leveraged to improve performance on users who do not conform to the population. We will round of the discussion with a communication-efficient training algorithm and experimental results and heterogeneous datasets.

Bio: Krishna Pillutla is a 5th year Ph.D. student at the Paul G. Allen School of Computer Science and Engineering at the University of Washington, where he is advised by Zaid Harchaoui and Sham Kakade. Krishna is broadly interested in machine learning and optimization and works in the particular areas of structured prediction and federated learning. Krishna was a 2019-20 JP Morgan Ph.D. Fellow.

IFDS Ideas Forum: Vivak Patel
2021-04-12 13:30 - 14:30 (America/Chicago)
Person: Vivak Patel
Title: Adaptive Iterative Methods for Linear Systems
Abstract:
Bio: My research is at the intersection of numerical optimization and statistical estimation with applications to dynamical systems. On the one hand, I interpret this as using numerical optimization ideas to design statistical estimators that are practically computable, yet still retain certain desirable statistical properties. On the other hand, I interpret this as using statistical ideas to design optimization methods for solving minimization problems that involve uncertainty. Beyond these two main areas of work, I also do some work in machine learning, numerical linear algebra and load modeling for power systems.

SILO: Merve Bodur
2021-04-14 12:30 - 13:30 (America/Chicago)
Title: Copositive Duality for Discrete Markets and Games
Models including binary decisions are often modelled as mixed-integer programs (MIPs). Such models are nonconvex and lack strong duality, which prevents the use of tools such as shadow prices and KKT conditions. For example, in convex markets, shadow (dual) prices are associated with market equilibrium, and for convex games the existence and uniqueness of Nash equilibrium can be proven via fixed-point theorem and KKT conditions. Those results are lacking in their nonconvex counterparts. We use copositive programming to formulate discrete problems in applications including nonconvex energy markets and nonconvex games, to leverage its convexity and strong duality features. We obtain several novel theoretical and numerical results for those applications, including a new revenue-adequate pricing scheme for energy markets, and existence, uniqueness, and KKT conditions for the pure-strategy Nash equilibrium in discrete games. We also propose a novel and purely MIP-based cutting-plane algorithm for mixed-integer copositive programs, and employ it in our applications. This is a joint work with Cheng Guo and Josh A. Taylor.
UNTIL FURTHER NOTICE: Seminars are virtual. Sign up for the SILO email list to receive the links to each talk at https://groups.google.com/ and browse for silo

ML-Opt: Adhyyan Narang
2021-04-16 13:30 - 14:30 (America/Los_Angeles)
Title: Classification vs Adversarial Examples for the Overparameterized linear model.
(Joint work with Vidya Muthukumar and Anant Sahai at UC Berkeley)
Abstract: In modern machine learning, overparameterized models are often used. It has been empirically observed that these often generalize well and display double-descent, but are susceptible to adversarial perturbations. Past theoretical explanations of these phenomena usually focus on linear models where the adversary has the power to perturb the features directly. However, the field of meta-learning has revealed that neural networks can be interpreted as first learning a feature-representation and then learning the best linear model on these learned features. The role of lifting in adversarial susceptibility of models is largely unaddressed, primarily because the problem of finding an adversarial example for lifted models is nonconvex and difficult to solve.
In this talk, I will use concepts from signal-processing to propose a toy model that exhibits all of the aforementioned phenomena, most crucially lifting. The toy nature of the model allows us to overcome the challenge of solving the adversarial-search problem. We learn that the adversarial vulnerability arises because of a phenomena we term spatial localization: the predictions of the learned model are markedly more sensitive in the vicinity of training points than elsewhere. Despite the adversarial susceptibility, we find that classification using spatially localized features can be “easier” i.e. less sensitive to the strength of the prior than in independent feature setups.
Bio: Adhyyan Narang (ECE) is a first-year PhD student, advised by Maryam Fazel and Lilian Ratliff. He is interested in fundamental theoretical questions about learning from data, and broadly works in the intersection of machine learning, optimization and game theory.

IFDS Ideas Forum: Zhiyan Ding
2021-04-19 13:30 - 14:30 (America/Chicago)
Person: Zhiyan Ding
Title: Connection between optimization and sampling
Abstract:
Bio: I am a third-year PhD student in the Department of Mathematics, University of Wisconsin-Madison. My advisor is Qin Li.

SILO: Emmanuel Abbe
2021-04-21 12:30 - 13:30 (America/Chicago)

ML-Opt@UW: Yue Sun
2021-04-23 13:30 - 14:30 (America/Los_Angeles)
Title: Subspace Based Meta-learning
Abstract:
Meta-learning typically involves two phases. First, one learns a suitable representation from the previously seen tasks. Secondly, this representation is used for learning a new task using only a few samples (i.e., few-shot learning). In this talk I will discuss:
1. Linear meta learning: sample complexity of representation learning with general covariance
2. Linear meta learning: algorithm & analysis for overparameterized few-shot learning
3. Generalization to nonlinear meta-learning

Bio:
Yue Sun is a 5th year PhD student from University of Washington, Seattle. He is interested in theoretical understanding of optimization, ML and control. His research works are about:
1. Nonconvex optimization on Riemannian manifolds (UW)
2. Low order linear system identification (UW)
3. Subspace based meta-learning (UW)
4. Nonconvex optimization applied in optimal control (UW)

5. Online optimization for video coding (Google, 2019)
6. Compressive sensing and phase retrieval (Ohio State U, 2015; Nokia Bell Labs, 2021)

IFDS Ideas Forum: Shi Chen
2021-04-26 13:30 - 14:00 (America/Chicago) | Webex
Title:
Abstract:
Shi Chen (Mathematics), advised by Qin Li (Mathematics) and Stephen J. Wright (Computer Science), works on problems in the interdisciplinary area of applied math and machine learning. He is interested in connecting machine learning to various mathematical physics problems, including the homogenization of PDEs and inverse problems. His recent work focuses on applying deep learning to PDE inverse problems.

IFDS Ideas Forum: Shashank Rajput
2021-04-26 14:00 - 14:30 (America/Chicago) | Webex
Title:
Abstract:
Shashank Rajput (Computer Science), advised by Dimitris Papailiopoulos (Electrical and Computer Engineering), Kangwook Lee (Electrical and Computer Engineering) and Stephen Wright (Computer Science), works on problems in distributed machine learning and optimization. He works on developing techniques which are fast and scalable for practical use, and have theoretical guarantees. Currently, he is working on finding better shuffling mechanisms that beat random reshuffling as well as developing algorithms for training neural networks by only pruning them.

SILO: Martin Wainwright
2021-04-28 12:30 - 13:30 (America/Chicago)

IFDS Ideas Forum: Changhun Jo
2021-05-03 13:30 - 14:00 (America/Chicago) | Webex
Title:
Abstract:
Changhun Jo (Mathematics), advised by Kangwook Lee (Electrical and Computer Engineering) and Sebastien Roch (Mathematics), is working on the theoretical understanding of machine learning. His recent work focuses on finding an optimal data poisoning algorithm against a fairness-aware learner. He also works on finding the fundamental limit on sample complexity of matrix completion in the presence of graph side information.

IFDS Ideas Forum: Shuqi Yu
2021-05-03 14:00 - 14:30 (America/Chicago)
Title: TBD
Abstract:
Shuqi Yu (Mathematics), advised by Sebastien Roch (Mathematics) and working with Karl Rohe (Statistics) on large scale network models. She aims to establish theoretical guarantees for a new estimator of the number of communities in a stochastic blockmodel. She is also interested in phylogenetics questions, in particular, she works on the identifiability of the species phylogeny under an horizontal gene transfer model.

SILO: Yuanzhi Li
2021-05-05 12:30 - 13:30 (America/Chicago)

IFDS All-Hands: Rina Foygel Barber
2021-05-07 13:30 - 14:30 (America/Los_Angeles)
Convergence for nonconvex ADMM, with applications to CT imaging
The alternating direction method of multipliers (ADMM) algorithm is a powerful and flexible tool for complex optimization problems of the form min{f(x)+g(y):Ax+By=c}. ADMM exhibits robust empirical performance across a range of challenging settings including nonsmoothness and nonconvexity of the objective functions f and g, and provides a simple and natural approach to the inverse problem of image reconstruction for computed tomography (CT) imaging. From the theoretical point of view, existing results for convergence in the nonconvex setting generally assume smoothness in at least one of the component functions in the objective. In this work, our new theoretical results provide convergence guarantees under a restricted strong convexity assumption without requiring smoothness or differentiability, while still allowing differentiable terms to be treated approximately if needed. We validate these theoretical results empirically, with a simulated example where both f and g are nondifferentiable (and thus outside the scope of existing theory), as well as a simulated CT image reconstruction problem.

Bio: Rina Foygel Barber is a Louis Block Professor in the Department of Statistics at the University of Chicago. She was a NSF postdoctoral fellow during 2012-13 in the Department of Statistics at Stanford University, supervised by Emmanuel Candès. She received her PhD in Statistics at the University of Chicago in 2012, advised by Mathias Drton and Nati Srebro, and a MS in Mathematics at the University of Chicago in 2009. Prior to graduate school, she was a mathematics teacher at the Park School of Baltimore from 2005 to 2007.

IDFS Ideas Forum: Subjoyoti Mukherjee
2021-05-10 13:30 - 14:30 (America/Los_Angeles)

SILO: Rashmi Vinayak
2021-05-12 12:30 - 13:30 (America/Chicago)
Title: Convertible Codes: Efficient Conversion of Coded Data in Large-scale Storage Systems
Abstract:
In large-scale data storage systems, failures are the norm in day-to-day operations. To protect data in the face of such failures, erasure codes (a tool from coding theory) are employed to store data in a redundant fashion. In this setting, a set of k data blocks to be stored is encoded using an [n, k] code to generate n blocks that are then stored on distinct storage devices. In a recent work, we showed that the failure rate of storage devices vary considerably over time, and that dynamically tuning the parameters n and k of the code provides significant reduction in storage cost. However, traditional codes suffer from prohibitively high resource overheads in changing the code parameters on already encoded data.
Motivated by this application, in this talk, we:
1. Present a new theoretical framework to formalize the notion of “code conversion”—the process of converting data encoded using an [n, k] code into data encoded using a code with different parameters [n’, k’], while maintaining desired decodability properties,
2. Introduce “convertible codes”, a new class of codes that enable resource-efficient conversion,
3. Prove tight bounds on two important metrics for code conversion (a) the number of nodes accessed, and (b) bandwidth consumed,
4. Present practical constructions of convertible codes for a broad range of parameters.
Bio:
Rashmi Vinayak is an assistant professor in the Computer Science department at Carnegie Mellon University. Her research interests broadly lie in computer/networked systems and information/coding theory, and the wide spectrum of intersection between the two areas. Her current focus is on fault tolerance and resource efficiency in data systems. Rashmi is a recipient of NSF CAREER Award, Tata Institute of Fundamental Research Memorial Lecture Award 2020, Facebook Distributed Systems Research Award 2019, Google Faculty Research Award 2018, Facebook Communications and Networking Research Award 2017, UC Berkeley Eli Jury Award 2016 for “outstanding achievement in the area of systems, communications, control, or signal processing”. Her work has received USENIX NSDI 2021 Community (Best Paper) Award, and IEEE Data Storage Best Paper and Best Student Paper Awards for the years 2011/2012. Rashmi received her Ph.D. from UC Berkeley in 2016, and was a postdoctoral scholar at UC Berkeley’s AMPLab/RISELab from 2016-17. During her Ph.D. studies, Rashmi was a recipient of Facebook Fellowship 2012-13, the Microsoft Research PhD Fellowship 2013-15, and the Google Anita Borg Memorial Scholarship 2015-16.
Webpage: http://www.cs.cmu.edu/~rvinayak/

UNTIL FURTHER NOTICE: Seminars are virtual. Sign up for the SILO email list to receive the links to each talk at https://groups.google.com/ and browse for silo

Data Science Day
2021-05-14 08:45 - 15:00 (America/Chicago)
Uses and Abuses of Data in Higher Education
Virtual Event via Zoom with Opportunities for Engagement
Details and registration: https://citl.ucsc.edu/data-science-day-2021/

SILO: Dimitris Tsipras
2021-05-19 12:30 - 13:30 (America/Chicago)
Title: Robust Machine Learning: The Worst-Case and Beyond
Abstract:
One of the key challenges in the real-world deployment of machine learning models is their brittleness: their performance significantly degrades when exposed to even small variations of their training environments.
How can we build ML models that are more robust?
In this talk, I will present a methodology for training models that are invariant to a broad family of worst-case input perturbations. I will then describe how such robust learning leads to models that learn fundamentally different data representations, and how this can be useful even outside the adversarial context. Finally, I will discuss model robustness beyond the worst-case: ways in which our models fail to generalize and how we can guide further progress on this front.”
Bio:
“Dimitris Tsipras is a PhD student in the MIT EECS Department, advised by Aleksander Mądry. His work revolves around the reliability and robustness of machine learning systems, as well as the science of modern machine learning. He is currently being supported by a Facebook PhD Fellowship
UNTIL FURTHER NOTICE: Seminars are virtual. Sign up for the SILO email list to receive the links to each talk at https://groups.google.com/ and browse for silo

ML-Opt: Romain Camilleri and Swati Padmanabhan
2021-06-04 13:30 - 14:30 (America/Los_Angeles)
The final talks of the ML-OPT seminar of the spring quarter will be given Friday (6/4) at 1:30pm PST by Romain Camilleri and Swati Padmanabhan.

Title: High-Dimensional Experimental Design and Kernel Bandits

Abstract: I will talk about high-dimensional bandits. First I want to review how the classical approach to solving linear bandits motivates an experimental design problem. Then I plan to justify why common rounding techniques cannot be applied in a potentially infinite-dimensional space. Lastly, I will show that one can avoid relying on rounding techniques by using a Catoni estimator.

Bio: Romain Camilleri is a 3rd year Ph.D. student at the Paul G. Allen School of Computer Science and Engineering at the University of Washington, where he is advised by Kevin Jamieson.

—————————————-
Title: Computing Lewis Weights to High Precision

Abstract: We present an algorithm for computing high-precision approximate L_p Lewis weights for p > 2. Given an m x n real full-rank matrix A and p>=3, our algorithm computes epsilon-approximate L_p-Lewis weights using O(p^3 log (m p / epsilon)) iterations, where each iteration takes time linear in the sparsity of the input matrix plus the time to compute the leverage scores of a diagonal rescaling of A. Previously, such iteration complexities were known only for 0< p < 4 [CohenPeng2015]. Consequently, our result helps complete the picture on near-optimal reduction from leverage scores to L_p-Lewis weights for all p>0.
[Joint work with Maryam Fazel, Yin Tat Lee, and Aaron Sidford]

Bio: Swati is a graduate student working on problems in convex optimization, advised by Yin Tat Lee.

TRIPODS PI meeting
2021-06-10 - 2021-06-11

IFDS Summer School
2021-07-26 - 2021-07-30

MadLab Workshop
2021-08-02 - 2021-08-04
