IFDS is located in the Central and Pacific Time zone. Please note the zone when accessing a particular event.

Loading Events

« All Events

IFDS Ideas Forum: Rare Gems: Finding Lottery Tickets at Initialization

Title: Rare Gems: Finding Lottery Tickets at InitializationSpeaker: Jy-Yong SoonTime+location: Monday 7 Feb 2022, 12:30-13:30 Orchard View Room
 
Abstract: It has been widely observed that large neural networks can be pruned to a small fraction of their original size, with little loss in accuracy, by typically following a time-consuming “train, prune, re-train” approach. Frankle and Carbin in 2019 conjecture that we can avoid this by training lottery tickets, i.e., special sparse subnetworks found at initialization, that can be trained to high accuracy. However, a subsequent line of work presents concrete evidence that current algorithms for finding trainable networks at initialization, fail simple baseline comparisons, e.g., against training random sparse subnetworks. Finding lottery tickets that train to better accuracy compared to simple baselines remains open. In this work, we resolve this open problem by discovering rare gems: subnetworks at initialization that attain considerable accuracy, even before training. Refining these rare gems – by means of fine-tuning – beats current baselines and leads to accuracy competitive or better than magnitude pruning methods.Bio:Jy-yong is a post-doctoral researcher in the Department of Electrical and Computer Engineering (ECE) at the University of Wisconsin-Madison, working with Prof. Dimitris Papailiopoulos and Prof. Kangwook Lee. He is interested in the intersection of machine learning, information theory, and distributed algorithms. He received his Ph.D. degree in 2020 from KAIST, under the supervision of Prof. Jaekyun Moon. He is a recipient of the IEEE ICC Best Paper Award, Qualcomm Innovation Awards, and NRF Korea Post-doctoral Fellowship.

February 7 @ 12:30 pm 1:30 pm CST

Details

Date:
February 7
Time:
12:30 pm – 1:30 pm CSTw
Event Category:

Organizer

Stephen Wright

Orchard View Room

330 N. Orchard Street, 3rd Floor NE
Madison, Wisconsin 53715 United States
Go to Top