Alexander Wettig (original) (raw)
My research interests revolve around building and understanding large language models, with a particular focus on their training data (QuRating, WebOrganizer, ProLong, Masking Rates). I have also worked on understanding why LMs are easy to adapt (via Kernel Behavior) and how we can interpret their internal workings (Transformer Programs, Edge Pruning).
I am also part of the team that built SWE-bench and SWE-agent.I am currently working on training coding agents at Cursor.
Publications
(* indicates equal contribution)
- SWE-smith: Scaling Data for Software Engineering Agent John Yang, Kilian Leret, Carlos E. Jimenez, Alexander Wettig, Kabir Khandpur, Yanzhe Zhang, Binyuan Hui, Ofir Press, Ludwig Schmidt, Diyi Yang Pre-print 2025
- Organize the Web: Constructing Domains Enhances Pre-Training Data Curation ICML 2025
- Metadata Conditioning Accelerates Language Model Pre-training Tianyu Gao, Alexander Wettig, Luxi He, Yihe Dong, Sadhika Malladi, Danqi Chen ICML 2025
- How to Train Long-Context Language Models (Effectively) Tianyu Gao*, Alexander Wettig*, Howard Yen, Danqi Chen ACL 2025
- Establishing Task Scaling Laws via Compute-Efficient Model Ladders Akshita Bhagia*, Jiacheng Liu*, Alexander Wettig, David Heineman, Oyvind Tafjord, Ananya Harsh Jha, Luca Soldaini, Noah A Smith, Dirk Groeneveld, Pang Wei Koh, Jesse Dodge, Hannaneh Hajishirzi Pre-print 2024
- OLMoE: Open Mixture-of-Experts Language Models Niklas Muennighoff, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Jacob Morrison, Sewon Min, Weijia Shi, Pete Walsh, Oyvind Tafjord, Nathan Lambert, Yuling Gu, Shane Arora, Akshita Bhagia, Dustin Schwenk, David Wadden, Alexander Wettig, Binyuan Hui, Tim Dettmers, Douwe Kiela, Ali Farhadi, Noah A. Smith, Pang Wei Koh, Amanpreet Singh, Hannaneh Hajishirzi ICLR 2025
- Finding Transformer Circuits with Edge Pruning Adithya Bhaskar, Alexander Wettig, Dan Friedman, Danqi Chen NeurIPS 2024 (Spotlight)
- SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering John Yang*, Carlos E. Jimenez*, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, Ofir Press NeurIPS 2024
- QuRating: Selecting High-Quality Data for Training Language Models Alexander Wettig, Aatmik Gupta, Saumya Malik, Danqi Chen ICML 2024 (Spotlight)
- Language Models as Science Tutors Alexis Chevalier, Jiayi Geng, Alexander Wettig, Howard Chen, Sebastian Mizera, Toni Annala, Max Jameson Aragon, Arturo RodrÃguez Fanlo, Simon Frieder, Simon Machado, Akshara Prabhakar, Ellie Thieu, Jiachen T. Wang, Zirui Wang, Xindi Wu, Mengzhou Xia, Wenhan Jia, Jiatong Yu, Jun-Jie Zhu, Zhiyong Jason Ren, Sanjeev Arora, Danqi Chen ICML 2024
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues? Carlos E. Jimenez*, John Yang*, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, Karthik Narasimhan ICLR 2024 (Oral)
- Learning Transformer Programs Dan Friedman, Alexander Wettig, Danqi Chen NeurIPS 2023 (Oral)
- Adapting Language Models to Compress Contexts Alexis Chevalier*, Alexander Wettig*, Anirudh Ajith, Danqi Chen EMNLP 2023
- Poisoning Retrieval Corpora by Injecting Adversarial Passages Zexuan Zhong*, Ziqing Huang*, Alexander Wettig, Danqi Chen EMNLP 2023
- A Kernel-Based View of Language Model Fine-Tuning Sadhika Malladi, Alexander Wettig, Dingli Yu, Danqi Chen, Sanjeev Arora ICML 2023 ICLR 2023 Workshop on Mathematical and Empirical Understanding of Foundation Models (Spotlight)
- Should You Mask 15% in Masked Language Modeling? Alexander Wettig*, Tianyu Gao*, Zexuan Zhong, Danqi Chen EACL 2023
- Finding Dataset Shortcuts with Grammar Induction Dan Friedman, Alexander Wettig, Danqi Chen EMNLP 2022
- Phrase Retrieval Learns Passage Retrieval, Too Jinhyuk Lee, Alexander Wettig, Danqi Chen EMNLP 2021