Kevin Yang — Home (original) (raw)
yangk at berkeley dot edu // Scholar // GitHub // [ Curriculum Vitae](Kevin Yang CV 082923.pdf)
I am a research scientist at Scaled Cognition. I am broadly interested in structured planning, control, and alignment/instruction-tuning methods for improving the quality and safety of natural language generation, especially for long context.
I completed my PhD at UC Berkeley, advised by Professor Dan Klein. I previously did a Master's at MIT, advised by Professor Regina Barzilay.
publications
- RLCD: Reinforcement Learning from Contrast Distillation for Language Model Alignment
Kevin Yang, Dan Klein, Asli Celikyilmaz, Nanyun Peng, Yuandong Tian
arxiv 2023; to be submitted soon
We propose a new method for simulating preference data in RLHF alignment pipelines based on generating preference pairs from two contrasting prompts, with strong downstream performance on three diverse alignment tasks and multiple LLaMA model scales.
Citation - DOC: Improving Long Story Coherence With Detailed Outline Control
Kevin Yang, Dan Klein, Nanyun Peng, Yuandong Tian
ACL 2023
We improve coherence in several-thousand-word-long stories by constructing a more detailed outline and improving the generator's ability to stay faithful to that outline. Humans prefer DOC to our previous Re3 system by a wide margin in both automatic and interactive generation.
Citation - Modular Visual Question Answering via Code Generation
Sanjay Subramanian, Medhini Narasimhan, Kushal Khangaonkar, Kevin Yang et al.
ACL 2023
We improve performance on visual question answering tasks requiring multi-step reasoning by synthesizing programs to compose logical reasoning steps, including calling sub-tools.
Citation - PREADD: Prefix-Adaptive Decoding for Controlled Text Generation
Jonny Pei, Kevin Yang, Dan Klein
Findings of ACL 2023
We propose a simple, no-training-required approach for modulating the control strength exerted through prompting, by contrasting the logit distributions induced by two contrasting prompts. We achieve strong results on toxic output mitigation, bias reduction, and sentiment control.
Citation - Predicting Compound Activity from Phenotypic Profiles and Chemical Structures
Nikita Moshkov, Tim Becker, Kevin Yang, Peter Horvath et al.
Nature Communications 2023
We investigate the effectiveness of three different data sources—-chemical structures, imaging data, and gene-expression profiles—-for predicting compound activity in laboratory assays.
Citation - Re3: Generating Longer Stories With Recursive Reprompting and Revision
Kevin Yang, Yuandong Tian, Nanyun Peng, Dan Klein
EMNLP 2022
We automatically generate coherent stories, with a consistent overarching plot, by simulating the human writing process and repeatedly reinjecting relevant context into large pretrained language models via structured prompting. The stories are 2000+ words, or even up to 7500.
Citation - Automated Crossword Solving
Eric Wallace*, Nicholas Tomlin*, Albert Xu*, Kevin Yang*, Eshaan Pathak*, Matthew Ginsberg, Dan Klein
ACL 2022
We create a system for automatically solving crossword puzzles, and achieve superhuman performance for the first time.
Citation - Addressing Resource and Privacy Constraints in Semantic Parsing Through Data Augmentation
Kevin Yang, Olivia Deng, Charles Chen, Richard Shin, Subhro Roy, Benjamin Van Durme
Findings of ACL 2022
We propose a data augmentation scheme for low-resource semantic parsing in complex realistic environments, which simultaneously maintains user privacy.
Citation - Multi-Objective Optimization by Learning Space Partitions
Yiyang Zhao, Linnan Wang, Kevin Yang, Tianjun Zhang, Tian Guo, Yuandong Tian
ICLR 2022
We propose a space-partitioning search algorithm for finding the Pareto frontier in multi-objective optimization problems.
Citation - Learning Space Partitions for Path Planning
Kevin Yang*, Tianjun Zhang*, Chris Cummins, Brandon Cui et al.
NeurIPS 2021
We propose a path planning method inspired by a theoretical analysis of search space partitioning, and show strong performance on difficult multimodal, long-horizon path planning problems.
Citation - FUDGE: Controlled Text Generation with Future Discriminators
Kevin Yang and Dan Klein
NAACL 2021
We propose a simple, flexible and highly effective method for controlling generation toward desired attributes using lightweight classifiers.
Citation - A Streaming Approach for Efficient Batched Beam Search
Kevin Yang, Violet Yao, John DeNero, Dan Klein
EMNLP 2020
We propose an efficient batching strategy for variable-length decoding on GPU architectures, demonstrating substantial speedups over existing fixed-width and variable-width beam searches.
Citation - Improving Molecular Design by Stochastic Iterative Target Augmentation
Kevin Yang, Wengong Jin, Kyle Swanson, Regina Barzilay, and Tommi Jaakkola
ICML 2020
We use a simple and theoretically motivated self-training approach guided by an external property predictor to significantly improve upon state-of-the-art approaches in molecular design.
Citation - Uncertainty Quantification Using Neural Networks for Molecular Property Prediction
Lior Hirschfeld, Kyle Swanson, Kevin Yang, Regina Barzilay, and Tommi Jaakkola
JCIM 2020
We comprehensively evaluate and compare several approaches for uncertainty estimation in neural models on molecular property prediction tasks.
Citation - A Deep Learning Approach to Antibiotic Discovery
Jonathan Stokes, Kevin Yang, Kyle Swanson, Wengong Jin et al.
Cell 2020
We use computational property prediction models to screen drug databases for potential antibiotic activity, and discover new antibiotics with novel mechanisms of action which are effective even against bacteria which are resistant to commonly used antibiotics.
Citation - Analyzing Learned Molecular Representations for Property Prediction
Kevin Yang*, Kyle Swanson*, Wengong Jin, Connor Coley et al.
JCIM 2019
We carefully benchmark models for molecular property prediction on both public and proprietary industry datasets. In addition, we introduce a new variant of message-passing neural networks and demonstrate consistently strong performance that significantly improves over existing baselines on many datasets.
Citation - Learning Multimodal Graph-to-Graph Translation for Molecular Optimization
Wengong Jin, Kevin Yang, Regina Barzilay, and Tommi Jaakkola
ICLR 2019
We introduce an encoder-decoder architecture for molecular optimization that operates directly on the molecular graph, and demonstrate that it significantly outperforms string-based baselines as well as pre-existing SOTA.
Citation