IFDS

IFDS Ideas Forum-Hanbaek Lyu
2021-11-01 12:30 - 13:00 (America/Chicago) | Orchard View Room, 330 N. Orchard Street, 3rd Floor NE, Madison, Wisconsin, 53715, United States
UNTIL FURTHER NOTICE: Seminars are hybrid in-person and via zoom.

ML-Opt@UW Stephen Mussmann
2021-10-29 13:30 - 14:30 (America/Los_Angeles)
Understanding and analyzing the effectiveness of uncertainty sampling

Active learning techniques attempt to reduce the amount of data required to learn a classifier by leveraging adaptivity. In particular, an algorithm iteratively selects and labels points from an unlabeled pool of data points. Over the history of active learning, many algorithms have been developed, though one heuristic algorithm, uncertainty sampling, stands out by its popularity, effectiveness, simplicity, and intuitiveness. Despite this, uncertainty sampling has known failure modes and lacks the theoretical underpinnings of some other algorithms such as those based on disagreement. Here, we present a few analyses of uncertainty sampling. First, we find that uncertainty sampling iterations implicitly optimizes the (generally non-convex) zero-one loss, explaining how uncertainty sampling can achieve lower error than labeling the entire unlabeled pool and highlighting the importance of a good initialization. Second, for logistic regression, we show that the extent to which uncertainty sampling outperforms random sampling is inversely proportional to the asymptotic error, both theoretically and empirically. Finally, we use the previous insights to show uncertainty sampling works very well on a particular NLP task due to extreme label imbalance. Taken together, these results provide a sturdier foundation for understanding and using uncertainty sampling.

https://washington.zoom.us/j/99919016373?pwd=[REDACTED]

SILO-Hanie Sedghi
2021-10-27 12:30 (America/Chicago)

IFDS Ideas Forum-Zhiyan Ding
2021-10-25 12:30 - 13:00 (America/Chicago) | Orchard View Room, 330 N. Orchard Street, 3rd Floor NE, Madison, Wisconsin, 53715, United States
Different regimes of overparameterized neural network: NTK and mean-field limit
UNTIL FURTHER NOTICE: Seminars are hybrid in-person and via zoom.

SILO-Josiah Hanna
2021-10-20 12:30 (America/Chicago)

E & A SIG:
2021-10-20 12:00 - 13:00 (America/Los_Angeles)

IFDS Ideas Forum: TBD
2021-10-18 12:30 - 13:00 (America/Chicago) | Zoom
REMOTE only

SILO-TBA
2021-10-13 12:30, 2021-10-06 12:30 (America/Chicago)

IFDS Ideas Forum-Multilingual NLP: Zero-shot Cross-lingual Transfer Learning
2021-10-11 12:30 - 13:00 (America/Chicago) | Orchard View Room, 330 N. Orchard Street, 3rd Floor NE, Madison, Wisconsin, 53715, United States

IFDS All-Hands: Qin Li
2021-10-08 13:30 - 14:30 (America/Los_Angeles)
Title:
Mean field theory in Inverse Problems from Bayesian inference to overparameterization of networks

Abstract:
Bayesian sampling and neural networks are seemingly two different machine learning areas, but they both deal with many particle systems. In sampling, one evolves a large number of samples (particles) to match a target distribution function, and in optimizing over-parameterized neural networks, one can view neurons particles that feed each other information in the DNN flow. These perspectives allow us to employ mean-field theory, a powerful tool that translates dynamics of many particle system into a partial differential equation (PDE), so rich PDE analysis techniques can be used to understand both the convergence of sampling methods and the zero-loss property of over-parameterization of ResNets. We showcase the use of mean-field theory in these two machine learning areas, and we also invite the audience to brainstorm other possible applications.

IFDS Ideas Forum-Chaobing Song
2021-10-04 13:30 - 14:00 (America/Chicago) | Orchard View Room, 330 N. Orchard Street, 3rd Floor NE, Madison, Wisconsin, 53715, United States

SILO-Jeff Linderoth
2021-09-29 12:30 (America/Chicago)
Subspace cluster with missing data via integer programming

IFDS Ideas Forum-Greg Canal
2021-09-27 12:30 - 13:30 (America/Chicago) | Orchard View Room, 330 N. Orchard Street, 3rd Floor NE, Madison, Wisconsin, 53715, United States

SILO-Michael Unser
2021-09-22 12:30 (America/Chicago)
New representer theorems for inverse problems and machine learning

IFDS Ideas Forum-Jasper Lee
2021-09-20 12:30 - 13:00 (America/Chicago) | Orchard View Room, 330 N. Orchard Street, 3rd Floor NE, Madison, Wisconsin, 53715, United States

SILO-Greg Canal
2021-09-15 12:30 (America/Chicago)

E & A SIG: Krishna Pillutla
2021-09-15 12:00 - 13:00 (America/Los_Angeles)

IFDS Ideas Forum-Ahmet Alacaoglu
2021-09-13 12:30 - 13:00 (America/Chicago) | Orchard View Room, 330 N. Orchard Street, 3rd Floor NE, Madison, Wisconsin, 53715, United States

SILO-Shivani Agrawal
2021-09-08 12:30 (America/Chicago)

MadLab Workshop
2021-08-02 - 2021-08-04

IFDS Summer School
2021-07-26 - 2021-07-30

TRIPODS PI meeting
2021-06-10 - 2021-06-11

ML-Opt: Romain Camilleri and Swati Padmanabhan
2021-06-04 13:30 - 14:30 (America/Los_Angeles)
The final talks of the ML-OPT seminar of the spring quarter will be given Friday (6/4) at 1:30pm PST by Romain Camilleri and Swati Padmanabhan.

Title: High-Dimensional Experimental Design and Kernel Bandits

Abstract: I will talk about high-dimensional bandits. First I want to review how the classical approach to solving linear bandits motivates an experimental design problem. Then I plan to justify why common rounding techniques cannot be applied in a potentially infinite-dimensional space. Lastly, I will show that one can avoid relying on rounding techniques by using a Catoni estimator.

Bio: Romain Camilleri is a 3rd year Ph.D. student at the Paul G. Allen School of Computer Science and Engineering at the University of Washington, where he is advised by Kevin Jamieson.

—————————————-
Title: Computing Lewis Weights to High Precision

Abstract: We present an algorithm for computing high-precision approximate L_p Lewis weights for p > 2. Given an m x n real full-rank matrix A and p>=3, our algorithm computes epsilon-approximate L_p-Lewis weights using O(p^3 log (m p / epsilon)) iterations, where each iteration takes time linear in the sparsity of the input matrix plus the time to compute the leverage scores of a diagonal rescaling of A. Previously, such iteration complexities were known only for 0< p < 4 [CohenPeng2015]. Consequently, our result helps complete the picture on near-optimal reduction from leverage scores to L_p-Lewis weights for all p>0.
[Joint work with Maryam Fazel, Yin Tat Lee, and Aaron Sidford]

Bio: Swati is a graduate student working on problems in convex optimization, advised by Yin Tat Lee.

SILO: Dimitris Tsipras
2021-05-19 12:30 - 13:30 (America/Chicago)
Title: Robust Machine Learning: The Worst-Case and Beyond
Abstract:
One of the key challenges in the real-world deployment of machine learning models is their brittleness: their performance significantly degrades when exposed to even small variations of their training environments.
How can we build ML models that are more robust?
In this talk, I will present a methodology for training models that are invariant to a broad family of worst-case input perturbations. I will then describe how such robust learning leads to models that learn fundamentally different data representations, and how this can be useful even outside the adversarial context. Finally, I will discuss model robustness beyond the worst-case: ways in which our models fail to generalize and how we can guide further progress on this front.”
Bio:
“Dimitris Tsipras is a PhD student in the MIT EECS Department, advised by Aleksander Mądry. His work revolves around the reliability and robustness of machine learning systems, as well as the science of modern machine learning. He is currently being supported by a Facebook PhD Fellowship
UNTIL FURTHER NOTICE: Seminars are virtual. Sign up for the SILO email list to receive the links to each talk at https://groups.google.com/ and browse for silo

Data Science Day
2021-05-14 08:45 - 15:00 (America/Chicago)
Uses and Abuses of Data in Higher Education
Virtual Event via Zoom with Opportunities for Engagement
Details and registration: https://citl.ucsc.edu/data-science-day-2021/

SILO: Rashmi Vinayak
2021-05-12 12:30 - 13:30 (America/Chicago)
Title: Convertible Codes: Efficient Conversion of Coded Data in Large-scale Storage Systems
Abstract:
In large-scale data storage systems, failures are the norm in day-to-day operations. To protect data in the face of such failures, erasure codes (a tool from coding theory) are employed to store data in a redundant fashion. In this setting, a set of k data blocks to be stored is encoded using an [n, k] code to generate n blocks that are then stored on distinct storage devices. In a recent work, we showed that the failure rate of storage devices vary considerably over time, and that dynamically tuning the parameters n and k of the code provides significant reduction in storage cost. However, traditional codes suffer from prohibitively high resource overheads in changing the code parameters on already encoded data.
Motivated by this application, in this talk, we:
1. Present a new theoretical framework to formalize the notion of “code conversion”—the process of converting data encoded using an [n, k] code into data encoded using a code with different parameters [n’, k’], while maintaining desired decodability properties,
2. Introduce “convertible codes”, a new class of codes that enable resource-efficient conversion,
3. Prove tight bounds on two important metrics for code conversion (a) the number of nodes accessed, and (b) bandwidth consumed,
4. Present practical constructions of convertible codes for a broad range of parameters.
Bio:
Rashmi Vinayak is an assistant professor in the Computer Science department at Carnegie Mellon University. Her research interests broadly lie in computer/networked systems and information/coding theory, and the wide spectrum of intersection between the two areas. Her current focus is on fault tolerance and resource efficiency in data systems. Rashmi is a recipient of NSF CAREER Award, Tata Institute of Fundamental Research Memorial Lecture Award 2020, Facebook Distributed Systems Research Award 2019, Google Faculty Research Award 2018, Facebook Communications and Networking Research Award 2017, UC Berkeley Eli Jury Award 2016 for “outstanding achievement in the area of systems, communications, control, or signal processing”. Her work has received USENIX NSDI 2021 Community (Best Paper) Award, and IEEE Data Storage Best Paper and Best Student Paper Awards for the years 2011/2012. Rashmi received her Ph.D. from UC Berkeley in 2016, and was a postdoctoral scholar at UC Berkeley’s AMPLab/RISELab from 2016-17. During her Ph.D. studies, Rashmi was a recipient of Facebook Fellowship 2012-13, the Microsoft Research PhD Fellowship 2013-15, and the Google Anita Borg Memorial Scholarship 2015-16.
Webpage: http://www.cs.cmu.edu/~rvinayak/

UNTIL FURTHER NOTICE: Seminars are virtual. Sign up for the SILO email list to receive the links to each talk at https://groups.google.com/ and browse for silo

IDFS Ideas Forum: Subjoyoti Mukherjee
2021-05-10 13:30 - 14:30 (America/Los_Angeles)

IFDS All-Hands: Rina Foygel Barber
2021-05-07 13:30 - 14:30 (America/Los_Angeles)
Convergence for nonconvex ADMM, with applications to CT imaging
The alternating direction method of multipliers (ADMM) algorithm is a powerful and flexible tool for complex optimization problems of the form min{f(x)+g(y):Ax+By=c}. ADMM exhibits robust empirical performance across a range of challenging settings including nonsmoothness and nonconvexity of the objective functions f and g, and provides a simple and natural approach to the inverse problem of image reconstruction for computed tomography (CT) imaging. From the theoretical point of view, existing results for convergence in the nonconvex setting generally assume smoothness in at least one of the component functions in the objective. In this work, our new theoretical results provide convergence guarantees under a restricted strong convexity assumption without requiring smoothness or differentiability, while still allowing differentiable terms to be treated approximately if needed. We validate these theoretical results empirically, with a simulated example where both f and g are nondifferentiable (and thus outside the scope of existing theory), as well as a simulated CT image reconstruction problem.

Bio: Rina Foygel Barber is a Louis Block Professor in the Department of Statistics at the University of Chicago. She was a NSF postdoctoral fellow during 2012-13 in the Department of Statistics at Stanford University, supervised by Emmanuel Candès. She received her PhD in Statistics at the University of Chicago in 2012, advised by Mathias Drton and Nati Srebro, and a MS in Mathematics at the University of Chicago in 2009. Prior to graduate school, she was a mathematics teacher at the Park School of Baltimore from 2005 to 2007.

SILO: Yuanzhi Li
2021-05-05 12:30 - 13:30 (America/Chicago)
