Speaker: Sean Welleck

Title: Constrained text generation through discrete and continuous inference

Abstract: Neural text generation has shifted to generating text with large-scale, general-purpose models coupled with generic inference algorithms. An important open question is how to efficiently offer control over generated text. We describe two algorithms for enabling control through inference-time constraints: (i) A*-Neurologic, a discrete search algorithm for incorporating logical constraints through estimates of the future, and (ii) COLD decoding, which treats generation as continuous gradient-based sampling from an energy function that captures task-relevant constraints. Our algorithms can be applied directly to off-the-shelf models without the need for task-specific finetuning, and result in strong performance on a variety of generation tasks.