ML Opt @ UW: Yifang Chen
2022-02-18 12:30 - 13:30 (America/Los_Angeles)
Speaker: Yifang Chen
Title: Active Multi-Task Representation Learning
Abstract: To leverage the power of big data from source tasks and overcome the scarcity of the target task samples, representation learning based on multi-task pretraining has become a standard approach in many applications. However, up until now, choosing which source tasks to include in the multi-task learning has been more art than science. In this paper, we give the first formal study on resource task sampling by leveraging the techniques from active learning. We propose an algorithm that iteratively estimates the relevance of each source task to the target task and samples from each source task based on the estimated relevance. Theoretically, we show that for the linear representation class, to achieve the same error rate, our algorithm can save up to a textit{number of source tasks} factor in the source task sample complexity, compared with the naive uniform sampling from all source tasks. We also provide experiments on real-world computer vision datasets to illustrate the effectiveness of our proposed method on both linear and convolutional neural network representation classes.

IFDS Ideas Forum: Ankit Pensia & Thanasis Pittas
2022-02-21 12:30 - 13:30 (America/Chicago) | Zoom
THIS IS A ZOOM ONLY EVENT. NO LIVE AUDIENCE.
Title: Hypothesis Testing under Communication Constraints
Speaker: Ankit Pensia
Abstract: Simple hypothesis testing is a fundamental problem in statistics and it is well-known that its sample complexity is characterized by the Hellinger distance between the two candidate distributions. In this talk, we discuss the problem of simple hypothesis testing under communication constraints, wherein each sample is mapped to a message from a finite set of messages before being revealed to the statistician. We show that it is possible to map samples to messages such that the sample complexity is only an extra logarithmic factor larger than the non-constrained setting. Our proofs rely on a reverse data processing inequality and a reverse Markov’s inequality, which might be of independent interest. This is joint work with Po-Ling Loh and Varun Jog.
Bio: Ankit Pensia is a graduate student in the CS department. He is interested in robust statistics, learning theory, and high-dimensional statistics. Website: https://ankitp.net

Title: Streaming Algorithms for High-Dimensional Robust Statistics
Speaker: Thanasis Pittas
Abstract: We study high-dimensional robust statistics tasks in the streaming model. A recent line of work obtained computationally efficient algorithms for a range of high-dimensional robust statistics tasks. Unfortunately, all previous algorithms require storing the entire dataset, incurring memory at least quadratic in the dimension. In this work, we develop the first efficient streaming algorithms for high-dimensional robust statistics with near-optimal memory requirements (up to logarithmic factors). Our main result is for the task of high-dimensional robust mean estimation in (a strengthening of) Huber’s contamination model. We give an efficient single-pass streaming algorithm for this task with near-optimal error guarantees and space complexity nearly-linear in the dimension. As a corollary, we obtain streaming algorithms with near-optimal space complexity for several more complex tasks, including robust covariance estimation, robust regression, and more generally robust stochastic optimization.
Bio: Thanasis Pittas is a PhD student at the University of Wisconsin-Madison, advised by Prof. Ilias Diakonikolas. He works on theoretical machine learning and robust statistics. Thanasis was an IFDS RA during the summer of 2021. He did his undergraduate studies in Greece, at the National Technical University of Athens.

IFDS Ideas Forum
2022-02-21 14:00, 2022-03-14 14:00 (America/Chicago)

SILO: Mengdi Wang
2022-02-23 12:30 - 13:30 (America/Chicago) | Orchard View Room, 330 N. Orchard Street, 3rd Floor NE, Madison, Wisconsin, 53715, United States

ML-Opt @ UWash: Krishna Pillutla
2022-02-25 12:30 - 13:30 (America/Chicago)
Title: MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers
Abstract: As major progress is made in open-ended text generation, measuring how close machine-generated text is to human language remains a critical open problem. We introduce MAUVE, a comparison measure for open-ended text generation, which directly compares the learnt distribution from a text generation model to the distribution of human-written text using divergence frontiers. MAUVE scales up to modern text generation models by computing information divergences in a quantized embedding space. Through an extensive empirical study on three open-ended generation tasks, we find that MAUVE identifies known properties of generated text, scales naturally with model size, and correlates with human judgments, with fewer restrictions than existing distributional evaluation metrics.

IFDS Ideas Forum: RA Talk
2022-02-28 00:00 (America/Chicago) | Zoom
Jiaxin Hu & Yuchen Zeng

IFDS Ideas Forum
2022-02-28 14:00 (America/Chicago)
A Review of Iterate Retraction in First Order Optimization

SILO: Mario Figueiredo
2022-03-02 12:30 - 13:30 (America/Chicago) | Orchard View Room, 330 N. Orchard Street, 3rd Floor NE, Madison, Wisconsin, 53715, United States

IFDS All-Hands: Zaid Harchaoui
2022-03-04 12:30 - 13:30 (America/Los_Angeles)

IFDS Ideas Forum: Multiscale inverse problem, from Schroedinger to Newton to Boltzmann
2022-03-07 12:30 - 13:30 (America/Chicago) | Orchard View Room, 330 N. Orchard Street, 3rd Floor NE, Madison, Wisconsin, 53715, United States
Title: Multiscale inverse problem, from Schroedinger to Newton to Boltzmann
Speaker: Qin Li, Department of Mathematics
Date + Location: 7 March (Monday), Orchard View Room
Abstract: Inverse problems are ubiquitous. People probe the media with sources and measure the outputs. At the scale of quantum, classical, statistical and fluid, these are inverse Schroedinger, inverse Newton’s second law, inverse Boltzmann problem, and inverse diffusion respectively. The universe, however, should have a universal mathematical description, as Hilbert proposed in 1900. In this talk, we present a line of research results that unify all these inverse problems. Facing the IFDS crowd, I’d like to ask the following question: how to integrate the mathematical equivalence into the optimization formulation for a more efficient algorithmic pipeline than the traditional PDE-constrained optimization?
Bio: Qin Li is an associate professor of mathematics at UW-Madison. Her research lies between scientific computing and PDE-constrained optimization.

IFDS Ideas Forum
2022-03-07 14:00 (America/Chicago)
Multiscale inverse problem, from Schroedinger to Newton to Boltzmann

SILO: Ulugbek Kamilov
2022-03-09 12:30 - 13:30 (America/Chicago) | Orchard View Room, 330 N. Orchard Street, 3rd Floor NE, Madison, Wisconsin, 53715, United States

ML-Opt @ UWash: Sean Welleck
2022-03-11 13:30 - 14:30 (America/Los_Angeles)
Speaker: Sean Welleck
Title: Constrained text generation through discrete and continuous inference
Abstract: Neural text generation has shifted to generating text with large-scale, general-purpose models coupled with generic inference algorithms. An important open question is how to efficiently offer control over generated text. We describe two algorithms for enabling control through inference-time constraints: (i) A*-Neurologic, a discrete search algorithm for incorporating logical constraints through estimates of the future, and (ii) COLD decoding, which treats generation as continuous gradient-based sampling from an energy function that captures task-relevant constraints. Our algorithms can be applied directly to off-the-shelf models without the need for task-specific finetuning, and result in strong performance on a variety of generation tasks.

IFDS Ideas Forum: How to Make the Gradients Small in Convex and Min-Max Optimization
2022-03-21 12:30 - 13:30 (America/Chicago) | Orchard View Room, 330 N. Orchard Street, 3rd Floor NE, Madison, Wisconsin, 53715, United States
Speaker: Jelena Diakonikolas

IFDS Ideas Forum
2022-03-21 14:00 (America/Chicago)
How to Make the Gradients Small in Convex and Min-Max Optimization

SILO: Gitta Kutyniok
2022-03-23 12:30 - 13:30 (America/Chicago) | Orchard View Room, 330 N. Orchard Street, 3rd Floor NE, Madison, Wisconsin, 53715, United States

IFDS Ideas Forum: A Review of Neural Collapse
2022-03-28 12:30 - 13:30 (America/Chicago) | Orchard View Room, 330 N. Orchard Street, 3rd Floor NE, Madison, Wisconsin, 53715, United States
Speaker: Greg Canal
Abstract: Neural Collapse is a recently discovered phenomenon in deep neural network training that describes class separation in the final network layers. When a classification network is trained past the point of zero training error, it has been observed that the penultimate layer activations collapse to their respective class means, the means themselves form a simplex equiangular tight frame, and the final layer linear classifiers align with each respective mean. Neural collapse has been demonstrated both empirically on deep networks as well as theoretically on simplified models, and has inspired new questions on generalization, robustness, and architecture design. In this talk I will review the original discovery of neural collapse, as well as recent literature that expands on related questions.

IFDS Ideas Forum
2022-03-28 14:00 (America/Chicago)
A Review of Neural Collapse

SILO: Yong Jae Lee
2022-03-30 12:30 (America/Chicago) | Orchard View Room, 330 N. Orchard Street, 3rd Floor NE, Madison, Wisconsin, 53715, United States

IFDS Ideas Forum: GALAXY: Graph-based Active Learning at the Extreme
2022-04-04 12:30 - 13:30 (America/Chicago) | Orchard View Room, 330 N. Orchard Street, 3rd Floor NE, Madison, Wisconsin, 53715, United States
Speaker: Jifan Xhang

IFDS Ideas Forum
2022-04-04 14:00 (America/Chicago)
GALAXY: Graph-based Active Learning at the Extreme

SILO: Jessi Cisewski-Kehe
2022-04-06 12:30 - 13:30 (America/Chicago) | Orchard View Room, 330 N. Orchard Street, 3rd Floor NE, Madison, Wisconsin, 53715, United States

IFDS Ideas Forum:
2022-04-11 12:30 - 13:30 (America/Chicago) | Orchard View Room, 330 N. Orchard Street, 3rd Floor NE, Madison, Wisconsin, 53715, United States
Speaker: Vivak Patel

IFDS Ideas Forum
2022-04-11 14:00 (America/Chicago)
Not sure yet, but I will come up with something

SILO: Mihaela van der Schaar
2022-04-13 12:30 - 13:30 (America/Chicago) | Orchard View Room, 330 N. Orchard Street, 3rd Floor NE, Madison, Wisconsin, 53715, United States

IFDS Ideas Forum: RA talk
2022-04-18 12:30 - 13:30 (America/Chicago) | Orchard View Room, 330 N. Orchard Street, 3rd Floor NE, Madison, Wisconsin, 53715, United States
Speaker: Jiaxin Hu

Speaker: Yuchen Zeng

IFDS Ideas Forum
2022-04-18 14:00 (America/Chicago)
IFDS RA talk

SILO: Misha Belkin
2022-04-20 12:30 - 13:30 (America/Chicago) | Orchard View Room, 330 N. Orchard Street, 3rd Floor NE, Madison, Wisconsin, 53715, United States

IFDS Ideas Forum: RA Talk
2022-04-25 12:30 - 13:30 (America/Chicago) | Orchard View Room, 330 N. Orchard Street, 3rd Floor NE, Madison, Wisconsin, 53715, United States
Speaker: Sijia Fang

Speaker: Shubham Bharti
