BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//IFDS - ECPv6.0.1.1//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:IFDS
X-ORIGINAL-URL:https://ifds.info
X-WR-CALDESC:Events for IFDS
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:America/Chicago
BEGIN:DAYLIGHT
TZOFFSETFROM:-0600
TZOFFSETTO:-0500
TZNAME:CDT
DTSTART:20240310T080000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0500
TZOFFSETTO:-0600
TZNAME:CST
DTSTART:20241103T070000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=America/Chicago:20240226T123000
DTEND;TZID=America/Chicago:20240226T133000
DTSTAMP:20260514T200158
CREATED:20240318T213825Z
LAST-MODIFIED:20240318T213825Z
UID:2900-1708950600-1708954200@ifds.info
SUMMARY:Prelimit coupling and steady-state convergence of constant-stepsize nonsmooth contractive SA
DESCRIPTION:Speaker: Yixuan Zhang \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nAbstract:  \nMotivated by Q-learning\, we study nonsmooth contractive stochastic approximation (SA) with constant stepsize. We focus on two important classes of dynamics: 1) nonsmooth contractive SA with additive noise\, and 2) synchronous and asynchronous Q-learning\, which features both additive and multiplicative noise. For both dynamics\, we establish weak convergence of the iterates to a stationary limit distribution in Wasserstein distance. Furthermore\, we propose a prelimit coupling technique for establishing steady-state convergence and characterize the limit of the stationary distribution as the stepsize goes to zero. Using this result\, we derive that the asymptotic bias of nonsmooth SA is proportional to the square root of the stepsize\, which stands in sharp contrast to smooth SA. This bias characterization allows for the use of Richardson-Romberg extrapolation for bias reduction in nonsmooth SA.
URL:https://ifds.info/event/prelimit-coupling-and-steady-state-convergence-of-constant-stepsize-nonsmooth-contractive-sa/
LOCATION:WID 1145\, 330 N Orchard Street\, Madison\, WI\, 53715\, United States
CATEGORIES:IFDS Ideas Forum
END:VEVENT
END:VCALENDAR