BEGIN:VCALENDAR VERSION:2.0 PRODID:-//IFDS - ECPv6.0.1.1//NONSGML v1.0//EN CALSCALE:GREGORIAN METHOD:PUBLISH X-ORIGINAL-URL:https://ifds.info X-WR-CALDESC:Events for IFDS REFRESH-INTERVAL;VALUE=DURATION:PT1H X-Robots-Tag:noindex X-PUBLISHED-TTL:PT1H BEGIN:VTIMEZONE TZID:America/Los_Angeles BEGIN:DAYLIGHT TZOFFSETFROM:-0800 TZOFFSETTO:-0700 TZNAME:PDT DTSTART:20210314T100000 END:DAYLIGHT BEGIN:STANDARD TZOFFSETFROM:-0700 TZOFFSETTO:-0800 TZNAME:PST DTSTART:20211107T090000 END:STANDARD BEGIN:DAYLIGHT TZOFFSETFROM:-0800 TZOFFSETTO:-0700 TZNAME:PDT DTSTART:20220313T100000 END:DAYLIGHT BEGIN:STANDARD TZOFFSETFROM:-0700 TZOFFSETTO:-0800 TZNAME:PST DTSTART:20221106T090000 END:STANDARD BEGIN:DAYLIGHT TZOFFSETFROM:-0800 TZOFFSETTO:-0700 TZNAME:PDT DTSTART:20230312T100000 END:DAYLIGHT BEGIN:STANDARD TZOFFSETFROM:-0700 TZOFFSETTO:-0800 TZNAME:PST DTSTART:20231105T090000 END:STANDARD BEGIN:DAYLIGHT TZOFFSETFROM:-0800 TZOFFSETTO:-0700 TZNAME:PDT DTSTART:20240310T100000 END:DAYLIGHT BEGIN:STANDARD TZOFFSETFROM:-0700 TZOFFSETTO:-0800 TZNAME:PST DTSTART:20241103T090000 END:STANDARD TZID:America/Chicago BEGIN:DAYLIGHT TZOFFSETFROM:-0600 TZOFFSETTO:-0500 TZNAME:CDT DTSTART:20210314T080000 END:DAYLIGHT BEGIN:STANDARD TZOFFSETFROM:-0500 TZOFFSETTO:-0600 TZNAME:CST DTSTART:20211107T070000 END:STANDARD BEGIN:DAYLIGHT TZOFFSETFROM:-0600 TZOFFSETTO:-0500 TZNAME:CDT DTSTART:20220313T080000 END:DAYLIGHT BEGIN:STANDARD TZOFFSETFROM:-0500 TZOFFSETTO:-0600 TZNAME:CST DTSTART:20221106T070000 END:STANDARD BEGIN:DAYLIGHT TZOFFSETFROM:-0600 TZOFFSETTO:-0500 TZNAME:CDT DTSTART:20230312T080000 END:DAYLIGHT BEGIN:STANDARD TZOFFSETFROM:-0500 TZOFFSETTO:-0600 TZNAME:CST DTSTART:20231105T070000 END:STANDARD BEGIN:DAYLIGHT TZOFFSETFROM:-0600 TZOFFSETTO:-0500 TZNAME:CDT DTSTART:20240310T080000 END:DAYLIGHT BEGIN:STANDARD TZOFFSETFROM:-0500 TZOFFSETTO:-0600 TZNAME:CST DTSTART:20241103T070000 END:STANDARD END:VTIMEZONE BEGIN:VEVENT DTSTART;TZID=America/Los_Angeles:20240315T133000 DTEND;TZID=America/Los_Angeles:20240315T143000 DTSTAMP:20260525T054207 CREATED:20240318T212943Z LAST-MODIFIED:20240318T212943Z UID:2888-1710509400-1710513000@ifds.info SUMMARY:Optimized Decision Making via Active Learning of Stochastic Hamiltonians DESCRIPTION:Speaker: Prof. Chandrajit Bajaj \, UT Austin \nAbstract: A Hamiltonian represents the energy of a dynamical system in phase space with coordinates of position and momentum. The Hamilton’s equations of motion are obtainable as coupled symplectic differential equations.  In this talk I shall show how optimized decision making (action sequences) can be obtained via a reinforcement learning problem wherein the agent interacts with the unknown environment to simultaneously learn a Hamiltonian surrogate and the optimal action sequences using Hamilton dynamics\, by invoking the Pontryagin Maximum Principle. We use optimal control theory to define an optimal control gradient flow\, which guides the reinforcement learning process of the agent to progressively optimize the Hamiltonian while simultaneously converging to the optimal action sequence. Extensions to stochastic Hamiltonians leading to stochastic action sequences and the free-energy principle shall also be discussed. This is joint work with  Taemin Heo\, Minh Nguyen. URL:https://ifds.info/event/optimized-decision-making-via-active-learning-of-stochastic-hamiltonians/ LOCATION:CSE (Allen) 403 CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Chicago:20240308T133000 DTEND;TZID=America/Chicago:20240308T143000 DTSTAMP:20260525T054207 CREATED:20240318T213102Z LAST-MODIFIED:20240318T213102Z UID:2890-1709904600-1709908200@ifds.info SUMMARY:Low-Rank Structures in Optimal Transport DESCRIPTION:Bio: Meyer Scetbon is currently a Research Scientist at Microsoft Research. He completed his PhD at Institut Polytechnique de Paris\, advised by M. Cuturi. He did\, as a visiting student\, his MS theses UW and Technion\, on kernel-based viewpoints on deep neural networks advised by Z. Harchaoui\, and end-to-end signal and image denoising advised by M. Elad\, respectively. \n\n\n\n\n\nAbstract: Optimal transport (OT) plays an increasingly important role in machine learning (ML) to compare probability distributions. Yet\, it poses\, in its original form\, several challenges when used for applied problems: (i) computing OT between discrete distributions amounts to solving a large and expensive network flow problem which requires a supercubic complexity in the number of points; (ii) estimating OT using sampled measures is doomed by the curse of dimensionality. These issues can be mitigated using an entropic regularization\, solved with the Sinkhorn algorithm\, which improves on both statistical and computational aspects. While much faster\, entropic OT still requires a quadratic complexity with respect to the number of points and therefore remains prohibitive for large-scale problems. In this talk\, I will present new regularization approaches for the OT problem\, as well as its quadratic extension\, the Gromov-Wasserstein (GW) problem\, which impose low-rank structures on the admissible couplings. This results in the development of new algorithms that enjoy a linear complexity both in time and memory with respect to the number of points\, enabling their applications in the large-scale setting where millions of points need to be compared. Additionally\, I will show that these new regularization schemes have better statistical performances compared to the entropic approach\, that they naturally interpolate between the Maximum Mean Discrepancy (MMD) and OT\, and that they offer general clustering methods for arbitrary geometry.Website:  URL:https://ifds.info/event/low-rank-structures-in-optimal-transport/ LOCATION:CSE (Allen) 403 CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Los_Angeles:20240223T133000 DTEND;TZID=America/Los_Angeles:20240223T143000 DTSTAMP:20260525T054207 CREATED:20240318T212752Z LAST-MODIFIED:20240318T212752Z UID:2886-1708695000-1708698600@ifds.info SUMMARY:GumbelSpec Sampling for Accelerating LLM Inference DESCRIPTION:Bio: Tianxiao Shen is a postdoctoral scholar at the University of Washington\, working with Yejin Choi and Zaid Harchaoui. Her research interests lie in natural language processing and machine learning\, in particular developing models and algorithms for efficient\, accurate\, diverse\, flexible and controllable text generation. She received her PhD from MIT\, advised by Regina Barzilay and Tommi Jaakkola. Before that\, she did her undergrad at Tsinghua University. \n\n\n\n\nAbstract: We propose GumbelSpec sampling\, a novel algorithm that leverages smaller language models to accelerate inference of large language models without changing their output distribution. Central to our approach is the application of the Gumbel-Softmax technique to convert the stochastic decoding process into a deterministic process by integrating independently sampled Gumbel noise. Employing the same set of Gumbel noise\, we perform beam search on the smaller model to generate multiple candidate short continuations\, and then utilize tree-based attention to efficiently verify them in parallel using the larger model. GumbelSpec sampling significantly improves upon previous rejection sampling based speculative decoding methods by increasing the token acceptance rate by 1.7x-2.2x and achieving an additional speedup of 1.2x-1.5x. This results in a total speedup of 1.5x-2.6x compared to traditional autoregressive decoding. URL:https://ifds.info/event/gumbelspec-sampling-for-accelerating-llm-inference/ LOCATION:CSE (Allen) 403 CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Los_Angeles:20240216T133000 DTEND;TZID=America/Los_Angeles:20240216T143000 DTSTAMP:20260525T054207 CREATED:20240318T212625Z LAST-MODIFIED:20240318T212625Z UID:2884-1708090200-1708093800@ifds.info SUMMARY:Offline Multi-task Transfer RL with Representational Penalization DESCRIPTION:Speaker Bio: Avinandan is a second year PhD student\, advised by Maryam Fazel and Lillian Ratliff. His interests are in sequential learning and game theory. \n\n\nAbstract: We study the problem of representational transfer in offline Reinforcement Learning (RL)\, where a learner has access to episodic data from a number of source tasks collected a priori\, and aims to learn a shared representation to be used in finding a good policy for a target task. Unlike in online RL where the agent interacts with the environment while learning a policy\, in the offline setting there cannot be such interactions in either the source tasks or the target task; thus multi-task offline RL can suffer from incomplete coverage.We propose an algorithm to compute pointwise uncertainty measures for the learnt representation\, and establish a data-dependent upper bound for the suboptimality of the learnt policy for the target task. Our algorithm leverages the collective exploration done by source tasks to mitigate poor coverage at some points by a few tasks\, thus overcoming the limitation of needing uniformly good coverage for a meaningful transfer by existing offline algorithms. We complement our theoretical results with empirical evaluation on a rich-observation MDP which requires many samples for complete coverage. Our findings illustrate the benefits of penalizing and quantifying the uncertainty in the learnt representation. URL:https://ifds.info/event/offline-multi-task-transfer-rl-with-representational-penalization/ LOCATION:CSE (Allen) 403 CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Los_Angeles:20240209T133000 DTEND;TZID=America/Los_Angeles:20240209T143000 DTSTAMP:20260525T054207 CREATED:20240318T212503Z LAST-MODIFIED:20240318T212503Z UID:2882-1707485400-1707489000@ifds.info SUMMARY:Policy Optimization with Compatible Mirror Approximation DESCRIPTION:Speaker Bio: Zhihan is a fourth-year PhD student in the Paul G. Allen School of Computer Science & Engineering at University of Washington\, advised by Prof. Maryam Fazel. His research interests are broadly in statistics\, optimization and machine learning. \n\n\nAbstract: We propose Compatible Mirror Policy Optimization (CoMPO)\, a framework that incorporates general function approximation into policy mirror descent methods. In contrast to the popular approach of using the $L_2$ norm to measure function approximation errors (regardless of the mirror map)\, CoMPO uses the Bregman divergence induced by the specific mirror map for policy projection. Such a compatibility bridges the gap between theory and practice: not only does it achieve fast linear convergence with general function approximation\, but it also includes several well-known practical methods as special cases\, immediately providing them strong convergence guarantees. URL:https://ifds.info/event/policy-optimization-with-compatible-mirror-approximation/ LOCATION:Zoom CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Los_Angeles:20240126T133000 DTEND;TZID=America/Los_Angeles:20240126T143000 DTSTAMP:20260525T054207 CREATED:20240318T212134Z LAST-MODIFIED:20240318T212230Z UID:2879-1706275800-1706279400@ifds.info SUMMARY:How do neural networks learn features from data? DESCRIPTION:Speaker Bio: Adit is currently the George F. Carrier Postdoctoral Fellow in the School of Engineering and Applied Sciences at Harvard. He completed his Ph.D. in electrical engineering and computer science (EECS) at MIT advised by Caroline Uhler and was a Ph.D. fellow at the Eric and Wendy Schmidt Center at the Broad Institute of MIT and Harvard. His research focuses on advancing theoretical foundations of machine learning and developing new methods for tackling biomedical problems. \n\n\n\nAbstract: Understanding how neural networks learn features\, or relevant patterns in data\, for prediction is necessary for their reliable use in technological and scientific applications. We propose a unifying mechanism that characterizes feature learning in neural network architectures. Namely\, we show that features learned by neural networks are captured by a statistical operator known as the average gradient outer product (AGOP). Empirically\, we show that the AGOP captures features across a broad class of network architectures including convolutional networks and large language models. Moreover\, we use AGOP to enable feature learning in general machine learning models through an algorithm we call Recursive Feature Machine (RFM). We show that RFM automatically identifies sparse subsets of features relevant for prediction and explicitly connects feature learning in neural networks with classical sparse recovery and low rank matrix factorization algorithms. Overall\, this line of work advances our fundamental understanding of how neural networks extract features from data\, leading to the development of novel\, interpretable\, and effective models for use in scientific applications. URL:https://ifds.info/event/how-do-neural-networks-learn-features-from-data/ LOCATION:CSE (Allen) 403 CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Los_Angeles:20221216T133000 DTEND;TZID=America/Los_Angeles:20221216T133000 DTSTAMP:20260525T054207 CREATED:20221018T165647Z LAST-MODIFIED:20221018T170221Z UID:2323-1671197400-1671197400@ifds.info SUMMARY:MLOpt: DESCRIPTION: URL:https://ifds.info/event/mlopt-7/ LOCATION:University of Washington\, Seattle\, 185 E Stevens Way NE\, Seattle\, WA\, 98195-2350\, United States CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Los_Angeles:20221209T133000 DTEND;TZID=America/Los_Angeles:20221209T133000 DTSTAMP:20260525T054207 CREATED:20221018T165647Z LAST-MODIFIED:20221018T170200Z UID:2322-1670592600-1670592600@ifds.info SUMMARY:MLOpt: DESCRIPTION: URL:https://ifds.info/event/mlopt-6/ LOCATION:University of Washington\, Seattle\, 185 E Stevens Way NE\, Seattle\, WA\, 98195-2350\, United States CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Los_Angeles:20221125T133000 DTEND;TZID=America/Los_Angeles:20221125T133000 DTSTAMP:20260525T054207 CREATED:20221018T165643Z LAST-MODIFIED:20221018T170101Z UID:2320-1669383000-1669383000@ifds.info SUMMARY:MLOpt: DESCRIPTION: URL:https://ifds.info/event/mlopt-5/ LOCATION:University of Washington\, Seattle\, 185 E Stevens Way NE\, Seattle\, WA\, 98195-2350\, United States CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Los_Angeles:20221118T133000 DTEND;TZID=America/Los_Angeles:20221118T133000 DTSTAMP:20260525T054207 CREATED:20221018T165642Z LAST-MODIFIED:20221018T170035Z UID:2319-1668778200-1668778200@ifds.info SUMMARY:MLOpt: DESCRIPTION: URL:https://ifds.info/event/mlopt-4/ LOCATION:University of Washington\, Seattle\, 185 E Stevens Way NE\, Seattle\, WA\, 98195-2350\, United States CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Los_Angeles:20221111T133000 DTEND;TZID=America/Los_Angeles:20221111T133000 DTSTAMP:20260525T054207 CREATED:20221018T165642Z LAST-MODIFIED:20221018T165849Z UID:2318-1668173400-1668173400@ifds.info SUMMARY:MLOpt: DESCRIPTION: URL:https://ifds.info/event/mlopt-3/ LOCATION:University of Washington\, Seattle\, 185 E Stevens Way NE\, Seattle\, WA\, 98195-2350\, United States CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Los_Angeles:20221028T133000 DTEND;TZID=America/Los_Angeles:20221028T133000 DTSTAMP:20260525T054207 CREATED:20221018T165642Z LAST-MODIFIED:20221018T165731Z UID:2316-1666963800-1666963800@ifds.info SUMMARY:MLOpt: DESCRIPTION: URL:https://ifds.info/event/mlopt-2/ LOCATION:University of Washington\, Seattle\, 185 E Stevens Way NE\, Seattle\, WA\, 98195-2350\, United States CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Los_Angeles:20221021T133000 DTEND;TZID=America/Los_Angeles:20221021T143000 DTSTAMP:20260525T054207 CREATED:20221018T155613Z LAST-MODIFIED:20221018T180956Z UID:2269-1666359000-1666362600@ifds.info SUMMARY:MLOpt: Lang Liu DESCRIPTION:Speaker: Lang Liu\nTitle: Non-Asymptotic Analysis of M-Estimation for Statistical Learning and Inference under Self-Concordance \nAbstract: In this talk\, I discuss the problem of M-estimation for statistical learning and inference. It is well-known from the classical asymptotic theory that the properly centered and normalized estimator has a limiting Gaussian distribution with a sandwich covariance. I first establish a finite-sample bound for the estimator\, characterizing its asymptotic behavior in a non-asymptotic fashion. An important feature of the bound is that its dimension dependency is characterized by the effective dimension — the trace of the limiting sandwich covariance — which can be much smaller than the parameter dimension in some regimes. I then illustrate how the bound can be used to obtain a confidence set whose shape is adapted to the local curvature of the population risk. In contrast to previous work which relied heavily on the strong convexity of the learning objective\, I only assume the Hessian is lower bounded at optimum and allow it to gradually become degenerate. This property is formalized by the notion of self-concordance originating from convex optimization. Finally\, I apply these techniques to semi-parametric estimation and derive state-of-the-art finite-sample bounds for double machine learning and orthogonal statistical learning. URL:https://ifds.info/event/mlopt-non-asymptotic-analysis-of-m-estimation-for-statistical-learning-and-inference-under-self-concordance/ LOCATION:University of Washington\, Seattle\, 185 E Stevens Way NE\, Seattle\, WA\, 98195-2350\, United States CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Los_Angeles:20221014T133000 DTEND;TZID=America/Los_Angeles:20221014T143000 DTSTAMP:20260525T054207 CREATED:20221012T175304Z LAST-MODIFIED:20221018T180930Z UID:2264-1665754200-1665757800@ifds.info SUMMARY:MLOpt @ UWash: Steve Mussmann DESCRIPTION:Title: Data pruning via Machine Teaching \nAbstract: In this talk\, I discuss the problem of data pruning: given a fully labeled dataset and a training procedure\, select a subset such that training on that subset yields approximately the same test performance as training on the full dataset. I present our algorithm\, inspired by recent work in machine teaching. Through experimental results and theoretical analysis\, we find that machine teaching is an effective paradigm for data pruning. \nVirtual option available via IFDS mailing list. URL:https://ifds.info/event/mlopt-uwash-steve-mussmann/ LOCATION:University of Washington\, Seattle\, 185 E Stevens Way NE\, Seattle\, WA\, 98195-2350\, United States CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Los_Angeles:20220311T133000 DTEND;TZID=America/Los_Angeles:20220311T143000 DTSTAMP:20260525T054207 CREATED:20220325T193738Z LAST-MODIFIED:20220325T194845Z UID:1893-1647005400-1647009000@ifds.info SUMMARY:ML-Opt @ UWash: Sean Welleck DESCRIPTION:Speaker: Sean Welleck \nTitle: Constrained text generation through discrete and continuous inference \nAbstract: Neural text generation has shifted to generating text with large-scale\, general-purpose models coupled with generic inference algorithms. An important open question is how to efficiently offer control over generated text. We describe two algorithms for enabling control through inference-time constraints: (i) A*-Neurologic\, a discrete search algorithm for incorporating logical constraints through estimates of the future\, and (ii) COLD decoding\, which treats generation as continuous gradient-based sampling from an energy function that captures task-relevant constraints. Our algorithms can be applied directly to off-the-shelf models without the need for task-specific finetuning\, and result in strong performance on a variety of generation tasks.   URL:https://ifds.info/event/ml-opt-uwash-sean-welleck/ CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Chicago:20220225T123000 DTEND;TZID=America/Chicago:20220225T133000 DTSTAMP:20260525T054207 CREATED:20220325T193820Z LAST-MODIFIED:20220325T194938Z UID:1896-1645792200-1645795800@ifds.info SUMMARY:ML-Opt @ UWash: Krishna Pillutla DESCRIPTION:Title: MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers \nAbstract: As major progress is made in open-ended text generation\, measuring how close machine-generated text is to human language remains a critical open problem. We introduce MAUVE\, a comparison measure for open-ended text generation\, which directly compares the learnt distribution from a text generation model to the distribution of human-written text using divergence frontiers. MAUVE scales up to modern text generation models by computing information divergences in a quantized embedding space. Through an extensive empirical study on three open-ended generation tasks\, we find that MAUVE identifies known properties of generated text\, scales naturally with model size\, and correlates with human judgments\, with fewer restrictions than existing distributional evaluation metrics. URL:https://ifds.info/event/ml-opt-uwash-zaid-harchaoui/ CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Los_Angeles:20220218T123000 DTEND;TZID=America/Los_Angeles:20220218T133000 DTSTAMP:20260525T054207 CREATED:20220325T194026Z LAST-MODIFIED:20220325T195105Z UID:1900-1645187400-1645191000@ifds.info SUMMARY:ML Opt @ UW: Yifang Chen DESCRIPTION:Speaker: Yifang Chen  \nTitle: Active Multi-Task Representation Learning \nAbstract: To leverage the power of big data from source tasks and overcome the scarcity of the target task samples\, representation learning based on multi-task pretraining has become a standard approach in many applications. However\, up until now\, choosing which source tasks to include in the multi-task learning has been more art than science. In this paper\, we give the first formal study on resource task sampling by leveraging the techniques from active learning. We propose an algorithm that iteratively estimates the relevance of each source task to the target task and samples from each source task based on the estimated relevance. Theoretically\, we show that for the linear representation class\, to achieve the same error rate\, our algorithm can save up to a textit{number of source tasks} factor in the source task sample complexity\, compared with the naive uniform sampling from all source tasks. We also provide experiments on real-world computer vision datasets to illustrate the effectiveness of our proposed method on both linear and convolutional neural network representation classes.  URL:https://ifds.info/event/ml-opt-uw-yifang-chen/ CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Los_Angeles:20220211T123000 DTEND;TZID=America/Los_Angeles:20220211T133000 DTSTAMP:20260525T054207 CREATED:20220325T195326Z LAST-MODIFIED:20220325T195446Z UID:1910-1644582600-1644586200@ifds.info SUMMARY:ML Opt@ UW: Vincent Roulet DESCRIPTION:Speaker: Vincent Roulet \nTitle: Complexity Bounds of Iterative Linearization Algorithms for Discrete-Time Nonlinear Control \nAbstract: We revisit the nonlinear optimization approach to discrete-time nonlinear control and optimization algorithms based on iterative linearization. While widely popular in many domains\, these algorithms have mainly been analyzed from an asymptotic viewpoint. We establish non-asymptotic complexity bounds and global convergence for a class of generalized Gauss-Newton algorithms relying on iterative linearization of the nonlinear control problem\, henceforth calling iterative linear quadratic regulator or differential dynamic programming algorithms as subroutines. The sufficient conditions for global convergence are examined for multi-rate sampling schemes given the existence of a feedback linearization scheme. We illustrate the algorithms in synthetic experiments and provide a software library based on reverse-mode automatic differentiation to reproduce the numerical results. URL:https://ifds.info/event/ml-opt-uw-vincent-roulet/ CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Chicago:20220121T123000 DTEND;TZID=America/Chicago:20220121T133000 DTSTAMP:20260525T054207 CREATED:20220325T195836Z LAST-MODIFIED:20220325T195856Z UID:1919-1642768200-1642771800@ifds.info SUMMARY:ML Opt@ UW: Lang Liu DESCRIPTION:Speaker: Lang Liu \nTitle: The Sample Complexity of Statistical Comparison Between Generative Models \nAbstract: The spectacular success of deep generative models calls for quantitative tools to measure their statistical performance. Divergence frontiers have recently been proposed as an evaluation framework for generative models. Although practically successful\, the sample complexity of the empirical estimator of divergence frontiers is unknown. We establish non-asymptotic bounds on the sample complexity of divergence frontiers\, providing theoretical guidance on their estimation procedure. URL:https://ifds.info/event/ml-opt-uw-lang-liu/ CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Los_Angeles:20220114T123000 DTEND;TZID=America/Los_Angeles:20220114T133000 DTSTAMP:20260525T054207 CREATED:20220325T200020Z LAST-MODIFIED:20220325T200040Z UID:1923-1642163400-1642167000@ifds.info SUMMARY:ML Opt@ UW: Yue Sun DESCRIPTION:Speaker: Yue Sun   \nTitle: Analysis of Policy Gradient Descent for Control: Global Optimality via Convex Parameterization   \nAbstract: Policy gradient descent is a popular approach in reinforcement learning due to its simplicity. Recent work has investigated the optimality and convergence properties of this method when applied in certain control problems. In this work\, we connect policy gradient descent (applied to a nonconvex problem formulation) with classical convex parameterizations in control theory\, to show the gradient dominance property for the nonconvex cost function. Such a connection between nonconvex and convex landscapes holds for continuous/discrete time LQR\, distributed optimal control\, minimizing the $cL_2$ gain\, among others. To the best of our knowledge\, this work offers the first result unifying the landscape analysis of a broad class of control problems. URL:https://ifds.info/event/ml-opt-uw-yue-sun/ CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Los_Angeles:20211029T133000 DTEND;TZID=America/Los_Angeles:20211029T143000 DTSTAMP:20260525T054207 CREATED:20211029T163929Z LAST-MODIFIED:20211029T163929Z UID:1711-1635514200-1635517800@ifds.info SUMMARY:ML-Opt@UW Stephen Mussmann DESCRIPTION:Understanding and analyzing the effectiveness of uncertainty sampling\n\nActive learning techniques attempt to reduce the amount of data required to learn a classifier by leveraging adaptivity. In particular\, an algorithm iteratively selects and labels points from an unlabeled pool of data points. Over the history of active learning\, many algorithms have been developed\, though one heuristic algorithm\, uncertainty sampling\, stands out by its popularity\, effectiveness\, simplicity\, and intuitiveness. Despite this\, uncertainty sampling has known failure modes and lacks the theoretical underpinnings of some other algorithms such as those based on disagreement. Here\, we present a few analyses of uncertainty sampling. First\, we find that uncertainty sampling iterations implicitly optimizes the (generally non-convex) zero-one loss\, explaining how uncertainty sampling can achieve lower error than labeling the entire unlabeled pool and highlighting the importance of a good initialization. Second\, for logistic regression\, we show that the extent to which uncertainty sampling outperforms random sampling is inversely proportional to the asymptotic error\, both theoretically and empirically. Finally\, we use the previous insights to show uncertainty sampling works very well on a particular NLP task due to extreme label imbalance. Taken together\, these results provide a sturdier foundation for understanding and using uncertainty sampling.\n\nhttps://washington.zoom.us/j/99919016373?pwd=UHpFYmlOL3dXcHEvMWNHcC9Wak1Edz09 URL:https://ifds.info/event/ml-optuw-stephen-mussmann/ CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Los_Angeles:20210604T133000 DTEND;TZID=America/Los_Angeles:20210604T143000 DTSTAMP:20260525T054207 CREATED:20210602T191959Z LAST-MODIFIED:20210602T192210Z UID:1320-1622813400-1622817000@ifds.info SUMMARY:ML-Opt: Romain Camilleri and Swati Padmanabhan DESCRIPTION:The final talks of the ML-OPT seminar of the spring quarter will be given Friday (6/4) at 1:30pm PST by Romain Camilleri and Swati Padmanabhan.\n\n\nTitle: High-Dimensional Experimental Design and Kernel Bandits\n\nAbstract: I will talk about high-dimensional bandits. First I want to review how the classical approach to solving linear bandits motivates an experimental design problem. Then I plan to justify why common rounding techniques cannot be applied in a potentially infinite-dimensional space. Lastly\, I will show that one can avoid relying on rounding techniques by using a Catoni estimator.\n\nBio: Romain Camilleri is a 3rd year Ph.D. student at the Paul G. Allen School of Computer Science and Engineering at the University of Washington\, where he is advised by Kevin Jamieson.\n\n—————————————-\nTitle: Computing Lewis Weights to High Precision\n\nAbstract: We present an algorithm for computing high-precision approximate L_p Lewis weights for p > 2. Given an m x n real full-rank matrix A and p>=3\, our algorithm computes epsilon-approximate L_p-Lewis weights using  O(p^3 \log (m p / epsilon)) iterations\, where each iteration takes time linear in the sparsity of the input matrix plus the time to compute the leverage scores of a diagonal rescaling of A. Previously\, such iteration complexities were known only for 0< p < 4   [CohenPeng2015]. Consequently\, our result helps complete the picture on near-optimal reduction from leverage scores to L_p-Lewis weights for all p>0.\n[Joint work with Maryam Fazel\, Yin Tat Lee\, and Aaron Sidford]\n\n\nBio: Swati is a graduate student working on problems in convex optimization\, advised by Yin Tat Lee. URL:https://ifds.info/event/ml-opt-romain-camilleri-and-swati-padmanabhan/ CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Los_Angeles:20210423T133000 DTEND;TZID=America/Los_Angeles:20210423T143000 DTSTAMP:20260525T054207 CREATED:20210422T180907Z LAST-MODIFIED:20210422T181817Z UID:1146-1619184600-1619188200@ifds.info SUMMARY:ML-Opt@UW: Yue Sun DESCRIPTION:Title: Subspace Based Meta-learning\nAbstract:\nMeta-learning typically involves two phases. First\, one learns a suitable representation from the previously seen tasks. Secondly\, this representation is used for learning a new task using only a few samples (i.e.\, few-shot learning). In this talk I will discuss:\n1. Linear meta learning: sample complexity of representation learning with general covariance\n2. Linear meta learning: algorithm & analysis for overparameterized few-shot learning\n3. Generalization to nonlinear meta-learning\n\n\nBio:\nYue Sun is a 5th year PhD student from University of Washington\, Seattle. He is interested in theoretical understanding of optimization\, ML and control. His research works are about:\n1. Nonconvex optimization on Riemannian manifolds (UW)\n2. Low order linear system identification (UW)\n3. Subspace based meta-learning (UW)\n4. Nonconvex optimization applied in optimal control (UW)\n\n5. Online optimization for video coding (Google\, 2019)\n6. Compressive sensing and phase retrieval (Ohio State U\, 2015; Nokia Bell Labs\, 2021) URL:https://ifds.info/event/mloptuw/ CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Los_Angeles:20210416T133000 DTEND;TZID=America/Los_Angeles:20210416T143000 DTSTAMP:20260525T054207 CREATED:20210412T182350Z LAST-MODIFIED:20210412T182350Z UID:1138-1618579800-1618583400@ifds.info SUMMARY:ML-Opt: Adhyyan Narang DESCRIPTION:Title: Classification vs Adversarial Examples for the Overparameterized linear model. \n(Joint work with Vidya Muthukumar and Anant Sahai at UC Berkeley) \nAbstract: In modern machine learning\, overparameterized models are often used. It has been empirically observed that these often generalize well and display double-descent\, but are susceptible to adversarial perturbations. Past theoretical explanations of these phenomena usually focus on linear models where the adversary has the power to perturb the features directly. However\, the field of meta-learning has revealed that neural networks can be interpreted as first learning a feature-representation and then learning the best linear model on these learned features. The role of lifting in adversarial susceptibility of models is largely unaddressed\, primarily because the problem of finding an adversarial example for lifted models is nonconvex and difficult to solve. \nIn this talk\, I will use concepts from signal-processing to propose a toy model that exhibits all of the aforementioned phenomena\, most crucially lifting. The toy nature of the model allows us to overcome the challenge of solving the adversarial-search problem. We learn that the adversarial vulnerability arises because of a phenomena we term spatial localization: the predictions of the learned model are markedly more sensitive in the vicinity of training points than elsewhere. Despite the adversarial susceptibility\, we find that classification using spatially localized features can be “easier” i.e. less sensitive to the strength of the prior than in independent feature setups. \nBio: Adhyyan Narang (ECE) is a first-year PhD student\, advised by Maryam Fazel and Lilian Ratliff. He is interested in fundamental theoretical questions about learning from data\, and broadly works in the intersection of machine learning\, optimization and game theory. URL:https://ifds.info/event/mlopt-adhyyan-narang/ CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Los_Angeles:20210409T133000 DTEND;TZID=America/Los_Angeles:20210409T143000 DTSTAMP:20260525T054207 CREATED:20210412T183054Z LAST-MODIFIED:20210412T183353Z UID:1140-1617975000-1617978600@ifds.info SUMMARY:ML-Opt: Krishna Pillutla DESCRIPTION:Title: Distributionally Robust Machine Learning with the Superquantile 1) For Supervised Learning\, 2) For Federated Learning\n\n\nAbstract: I will talk about distributionally robust machine learning\, a principled approach for robust performance across subpopulations\, and shifting distributions. We will focus on the superquantile\, a.k.a. the Conditional Value at Risk (CVaR)\, which was popularized by the seminal work of UW’s own R. T. Rockafellar and co-authors in the field of computational finance and economics in the early 2000s.\nWe will first review the use of the superquantile for distributionally robust supervised learning. We will prove a generalization bound from first principles.\nSecond\, we will discuss an application of the superquantile in the field of federated learning\, i.e.\, the distributed training of machine learning models on mobile phones. We will quantify the extent to which a user conforms to the population distribution and show how the superquantile can be leveraged to improve performance on users who do not conform to the population. We will round of the discussion with a communication-efficient training algorithm and experimental results and heterogeneous datasets.\n\nBio: Krishna Pillutla is a 5th year Ph.D. student at the Paul G. Allen School of Computer Science and Engineering at the University of Washington\, where he is advised by Zaid Harchaoui and Sham Kakade. Krishna is broadly interested in machine learning and optimization and works in the particular areas of structured prediction and federated learning. Krishna was a 2019-20 JP Morgan Ph.D. Fellow. URL:https://ifds.info/event/ml-opt-krishna-pillutla/ CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Chicago:20210312T133000 DTEND;TZID=America/Chicago:20210312T143000 DTSTAMP:20260525T054207 CREATED:20210115T203442Z LAST-MODIFIED:20210115T203442Z UID:896-1615555800-1615559400@ifds.info SUMMARY:ML-Opt @ UWash: Vincent Roulet DESCRIPTION:Title: Global convergence of first-order methods for nonlinear control problems \nAbstract: URL:https://ifds.info/event/ml-opt-uwash-vincent-roulet/ CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Los_Angeles:20210226T133000 DTEND;TZID=America/Los_Angeles:20210226T143000 DTSTAMP:20260525T054207 CREATED:20210115T203300Z LAST-MODIFIED:20220325T193610Z UID:894-1614346200-1614349800@ifds.info SUMMARY:ML-Opt @ UWash: Andrew Wagenmaker DESCRIPTION:Title:\n\nAbstract: URL:https://ifds.info/event/ml-opt-uwash-andrew-wagenmaker/ CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Chicago:20210219T133000 DTEND;TZID=America/Chicago:20210219T143000 DTSTAMP:20260525T054207 CREATED:20210115T203136Z LAST-MODIFIED:20210115T203136Z UID:892-1613741400-1613745000@ifds.info SUMMARY:ML-Opt @ UWash: Omid Sadeghi DESCRIPTION:Title: \nAbstract: URL:https://ifds.info/event/ml-opt-uwash-omid-sadeghi/ CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Chicago:20210212T133000 DTEND;TZID=America/Chicago:20210212T143000 DTSTAMP:20260525T054207 CREATED:20210115T202956Z LAST-MODIFIED:20210212T002315Z UID:890-1613136600-1613140200@ifds.info SUMMARY:ML-Opt @ UWash: Postponed-TBD DESCRIPTION:This talk will be rescheduled for a later date.\nTitle: Convergence of the empirical superquantile for risk-sensitive machine learning \nAbstract:  I’ll present some background in risk-sensitive learning + Theorem 2.1 of  this paper (using ML nomenclature) \n  URL:https://ifds.info/event/ml-opt-uwash-krishna-pillutla/ CATEGORIES:MLOpt@UWash END:VEVENT BEGIN:VEVENT DTSTART;TZID=America/Chicago:20210129T133000 DTEND;TZID=America/Chicago:20210129T143000 DTSTAMP:20260525T054207 CREATED:20210115T202824Z LAST-MODIFIED:20210115T202824Z UID:888-1611927000-1611930600@ifds.info SUMMARY:ML-Opt @ UWash: Ronak Mehta DESCRIPTION:Title: \nAbstract: \n  URL:https://ifds.info/event/ml-opt-uwash-ronak-mehta/ CATEGORIES:MLOpt@UWash END:VEVENT END:VCALENDAR