Pan Lu (original) (raw)
I am a Postdoctoral Scholar at Stanford University. I am affiliated with Stanford AI Lab, Zou's Group, and Choi's xlab, where I am fortunate to be advised by Professor James Zou and Professor Yejin Choi.
I received my Ph.D. in computer science from UCLA, where I was advised by Kai-Wei Chang and Song-Chun Zhu. I was a member of UCLA Natural Language Processing Group (UCLA NLP). Previously, I completed my M.S. in computer science at Tsinghua University, supervised by Jianyong Wang. My research has been recognized with Most Influential ICLR Paper Award (top-15 cited at ICLR 2024), Most Influential NIPS Paper Award (top-15 cited at NeurIPS 2022), KnowledgeNLP 2025 Workshop Best Paper Award, and EMNLP 2024 Best Paper Nomination — achievements made possible thanks to the support of my advisors and collaborators. I have been fortunate to receive recognition from Amazon PhD Fellowship, Bloomberg Data Science Ph.D. Fellowship (Global 9), Qualcomm Innovation Fellowship (18 winners), UCLA Dissertation Year Fellowship, and NeurIPS Scholar Award.
My research goal is to develop intelligent machines that can reason and collaborate with humans for the common good. My primary focus lies in machine learning and natural language processing, particularly in machine reasoning, mathematical reasoning, and scientific discovery. My recent research interests include:
- Tool-Augmented LLMs and Agentic Systems for complex reasoning[OctoTools] [Chameleon] [ChemAgent]
- Post-Training and Test-Time Training techniques for foundation models[STIC] [LLaMA-Adapter] [LLaMA-Adapter V2] [SPHINX-X] [TextGrad] [PromptPG]
- AI for Math: advancing mathematical reasoning capabilities of AI systems and LLMs across multimodal, knowledge-intensive, and real-world contexts[IneqMath] [MathVista] [MathVerse] [PromptPG] [Inter-GPS] [IconQA] [TheoremQA] [DL4Math] [MATH-AI]
- AI for Science: AI systems that facilitate scientific reasoning and scientific discovery[ScienceQA] [SciBench] [Protein-LLM] [ChemAgent]
[25.06] We are seeking students to collaborate on research in agentic AI, post-training LLMs, reinforcement learning, mathematical reasoning, AI for Science, and related fields. A background in these fields is preferred but not strictly required. If you're interested in joining us, please apply via this form. For a faster response, kindly send me an email after submitting the form.
News
[06/2025] New! Glad to co-organize the Multimodal Mathematical Reasoning Workshop at CVPR 2025!
[06/2025] New! Our new study on solving inequality proofs with LLMs is available at**Preprint**.
[05/2025] New! Excited that**MathVista** has been recognized as the 🏆 Most Influential ICLR Paper (ICLR-24)!
[05/2025] New! Excited that OctoTools received the 🏆 Best Paper Award at the KnowledgeNLP Workshop!
[04/2025] New! Excited to be invited to serve as a Senior Area Chair for EMNLP 2025!
[04/2025] New! Excited to be invited to serve as an Area Chair for NeurIPS 2025!
[03/2025] New! Thrilled to announce that TextGrad is published in Nature!
[02/2025] New! Our OctoTools agentic framework with extensible tools is available at**Preprint**.
[01/2025] New! Four papers are accepted to**ICLR 2025**.
[12/2024] New! I am co-organizing the 4th MATH-AI Workshop at NeurIPS 2024. See you in Vancouver!
[09/2024] New! One paper on self-improving vision-language models is accepted to**NeurPIS 2024**.
[09/2024] New! Excited to have our NeurIPS 2022 paper recognized as Most Influential NIPS Papers!
[09/2024] New! One paper on self-improving vision-language models is accepted to**NeurPIS 2024**.
[09/2024] New! Three papers are accepted to**EMNLP 2024**. See you in Miami!
[07/2024] New! One paper on visual math problems is accepted to**ECCV 2024**.
[06/2024] New! A paper on debugging visual programs is available at**Preprint**.
[06/2024] New! A paper on multi-image understanding is available at**Preprint**.
[05/2024] New! A paper on enhancing LVLMs with self-training is available at**Preprint**.
[05/2024] New! Thrilled to be awarded the**Bloomberg Data Science Ph.D. Fellowship**! Thanks!
[05/2024] New! One paper on advanced quantitative reasoning is accepted to**ACL 2024 (Findings)**.
[05/2024] New! Two papers on math reasoning and VLMs are accepted at ICML 2024. See you in Vienna!
[04/2024] New! Defended my doctoral dissertation! Thanks to my advisor and committee members!
[03/2024] New! I am co-organizing the AI for Math Workshop at ICML 2024. See you in Vienna!
[03/2024] New! A paper on visual math reasoning with Multi-modal LLMs is available at**Preprint**.
[02/2024] New! A paper on LLMs for advanced quantitative reasoning is available at**Preprint**.
[01/2024] New! Two papers on large multimodal models are accepted to**ICLR 2024**.
[01/2024] New! A paper on model editing for LLMs is available at**Preprint**.
[01/2024] New! Two papers on large multimodal models are accepted to**ICLR 2024**.
[01/2024] New! A paper on model editing for LLMs is available at**Preprint**.
[12/2023] New! I am co-organizing the Tool-Augmented VIsion Workshop at CVPR 2024. See you in Seattle!
[12/2023] New! I am attending NeurIPS 2023 from Dec 10 to Dec 16. See you in New Orleans!
[12/2023] New! Google's Gemini benchmarks our MathVista for evaluating math reasoning in visual contexts!
[11/2023] New! Honored to be covered by UCLA CS for winning Qualcomm Innovation Fellowship. Thanks!
[10/2023] New! The 112-page study on GPT-4V, Bard, and others on visual math reasoning is available**here**.
[10/2023] New! Honored to serve as PC Chair and co-organize**SoCal NLP 2023**. See you in LA!
[10/2023] New! One paper on mathematical reasoning is accepted to**EMNLP 2023**.
[10/2023] New! One paper on mathematical reasoning in visual contexts (MathVista) is submitted to Preprint.
[09/2023] New! One paper on tool-augmented LLMs is accepted to**NeurIPS 2023**.
[07/2023] New! One paper on a scientific reasoning benchmark (SciBench) is submitted to Preprint.
[07/2023] New! I am co-organizing the 3rd MATH-AI Workshop at NeurIPS 2023. See you in New Orleans!
[06/2023] New! Excited to receive the UCLA Dissertation Year Fellowship.
[05/2023] New! Honored to deliver a guest lecture for UCLA CS 263: Natural Language Processing. [[Slides]](docs/UCLA%5FCS263%5FNLP%5FGuest Lecture%5FPan Lu%5F2023.05.31.pdf)
[05/2023] New! One paper on theorem-driven math question answering (TheoremQA) is available at**Preprint**.
[05/2023] New! Honored to deliver a invited talk on tool-augmented LLMs at Google Brain. [Slides]
[05/2023] New! Delighted to join prestigious LightingAI event as invited speaker on Discord.
[05/2023] New! A paper on multimodal procedural planning is available at**Preprint**.
[05/2023] New! One survey paper on deep learning for mathematical reasoning is accepted to ACL 2023.
[04/2023] New! LLaMA-Adapter-V2, a parameter-efficient visual instruction model, is available at**Preprint**.
[04/2023] New! One tutorial proposal on mathematical reasoning is accepted to**IJCAI 2023**.
[04/2023] New! One paper on tool augmented LLMs (Chameleon) is available at**Preprint**.
[04/2023] New! Two papers are accepted to**CVPR 2023 O-DRUM Workshop**.
[03/2023] New! One paper on fine-tuning**LLaMA** in one hour (LLaMA-Adapter) is available at**Preprint**.
[01/2023] New! One paper on in-context learning for math reasoning (PromptPG) is accepted to ICLR 2023.
[12/2022] New! A survey paper on deep learning for mathematical reasoning is available at**Preprint**.
[12/2022] New! One paper is accepted to**AAAI'23 KnowledgeNLP Workshop** as an Oral Presentation.
[12/2022] New! I am excited to join Microsoft Research as a research intern!
[10/2022] New! Happy to receive the NeurIPS 2022 Scholar Award.
[10/2022] New! Two papers on mathematical reasoning are accepted to EMNLP 2022.
[09/2022] New! One paper on prompt learning for math reasoning (PromptPG) is submitted to Preprint.
[09/2022] New! One paper on chain-of-thought reasoning for**ScienceQA** is accepted to NeurIPS 2022.
[07/2022] New! I am co-organizing the 2nd MATH-AI Workshop at NeurIPS 2022. See you in New Orleans!
[07/2022] New! One paper on socially intelligent agents is accepted to SIGDIAL 2022.
[04/2022] Excited to be listed as a**Highlighted Reviewer** for ICLR 2022.
[03/2022] I am excited to join Allen Institute for AI (AI2) as a research intern!
[03/2022] One paper on character animation sampling is submitted to Preprint.
[12/2021] Two papers are accepted to AAAI 2022.
[10/2021] One paper on visual question answering for icon images (IconQA) is accepted to NeurIPS 2021.
[07/2021] I am co-organizing the MATHAI4ED Workshop at NeurIPS 2021. Welcome to participate!
[07/2021] Our workshop proposal for Math AI for Education (MATHAI4ED) is accepted to NeurIPS 2021.
[05/2021] One paper on interpretable geometry problem solving is accepted to ACL 2021 as an Oral Presentation.
[05/2021] One paper on social relation inference in dialogues is accepted to ACL 2021 as an Oral Presentation.
[03/2021] One paper on socially intelligent agents is submitted to Preprint.
Selected Publications
All publications can be found on my Google Scholar page.
Protein Large Language Models: A Comprehensive Survey
Yijia Xiao, Wanjia Zhao, Junkai Zhang, Yiqiao Jin, Han Zhang, Zhicheng Ren, Renliang Sun, Haixin Wang, Guancheng Wan, Pan Lu, Xiao Luo, Yu Zhang, James Zou, Yizhou Sun, Wei Wang
Preprint [Paper] [PDF] [Tutorial] [Coverage] [BibTex]
ChemAgent: Self-updating Library in Large Language Models Improves Chemical Reasoning
Xiangru Tang, Tianyu Hu, Muyang Ye, Yanjun Shao, Xunjian Yin, Siru Ouyang, Wangchunshu Zhou, Pan Lu, Zhuosheng Zhang, Yilun Zhao, Arman Cohan, Mark Gerstein
ICLR 2025
[Paper] [PDF] [Code] [News] [BibTex]
MMSearch: Benchmarking the Potential of Large Models as Multi-modal Search Engines
Dongzhi Jiang, Renrui Zhang, Ziyu Guo, Yanmin Wu, Jiayi Lei, Pengshuo Qiu, Pan Lu, Zehui Chen, Guanglu Song, Peng Gao, Yu Liu, Chunyuan Li, Hongsheng Li
ICLR 2025 [Project] [Paper] [PDF] [Hugging Face] [Code] [Data] [BibTex] 
MuirBench: A Comprehensive Benchmark for Robust Multi-image Understanding
Fei Wang*, Xingyu Fu*, James Y. Huang, Zekun Li, Qin Liu, Xiaogeng Liu, Mingyu Derek Ma, Nan Xu, Wenxuan Zhou, Kai Zhang, Tianyi Lorena Yan, Wenjie Jacky Mo, Hsiang-Hui Liu, Pan Lu, Chunyuan Li, Chaowei Xiao, Kai-Wei Chang, Dan Roth, Sheng Zhang, Hoifung Poon, Muhao Chen
ICLR 2025 [Project] [Paper] [PDF] [Hugging Face] [Code] [Data] [Twitter] [BibTex]
(*Equal Contribution)
MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?
Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Peng Gao, Hongsheng Li
ECCV 2024 [Project] [Paper] [PDF] [Code] [Data] [Visualization] [Coverage] [Daily Papers] [BibTex] 
SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models
Xiaoxuan Wang*, Ziniu Hu*, Pan Lu*, Yanqiao Zhu*, Jieyu Zhang, Satyen Subramaniam, Arjun R. Loomba, Shichang Zhang, Yizhou Sun, Wei Wang
ICML 2024 [Paper] [PDF] [Code] [Twitter] [BibTex]
(*Equal Contribution)
Nature News Feature (15 November 2023)
SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models
Peng Gao, Renrui Zhang, Chris Liu, Longtian Qiu, Siyuan Huang, Weifeng Lin, Shitian Zhao, Shijie Geng, Ziyi Lin, Peng Jin, Kaipeng Zhang, Wenqi Shao, Chao Xu, Conghui He, Junjun He, Hao Shao, Pan Lu, Hongsheng Li, Yu Qiao
ICML 2024 [Paper] [PDF] [Code] [Doc] [Hugging Face] [Twitter] [Coverage] [BibTex] 
Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models
Pan Lu, Baolin Peng, Hao Cheng, Michel Galley, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, Jianfeng Gao
NeurIPS 2023 [Project] [Paper] [PDF] [Code] [Twitter] [Coverage] [BibTex]
🏆 Best Weekly AI Paper (by AlphaSignal, 1st in 1682, 0.06%)
🏆 Awesome NeurIPS 2023 Papers (40 in 3584, 0.01%)
🏆 NeurIPS 2023 Top 10 Multimodal ML Papers
LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Model
Peng Gao, Jiaming Han, Renrui Zhang, Ziyi Lin, Shijie Geng, Aojun Zhou, Wei Zhang, Pan Lu, Conghui He, Xiangyu Yue, Hongsheng Li, Yu Qiao
arXiv:2304.15010 [Paper] [PDF] [Code] [Gradio] [Gradio-Multimodal] [Twitter] [YouTube]
LILA: A Unified Benchmark for Mathematical Reasoning
Swaroop Mishra*, Matthew Finlayson*, Pan Lu, Leonard Tang, Sean Welleck, Chitta Baral, Tanmay Rajpurohit, Oyvind Tafjord, Ashish Sabharwal, Peter Clark, Ashwin K. Kalyan
EMNLP 2022 [Paper] [PDF] [Project] [Data] [Code] [Huggingface] [BibTex]
(*Equal Contribution)
UniGeo: Unifying Geometry Logical Reasoning via Reformulating Mathematical Expression
Jiaqi Chen, Tong Li, Jinghui Qin, Pan Lu, Liang Lin, Chongyu Chen and Xiaodan Liang
EMNLP 2022 [Paper] [PDF] [Code] [BibTex]
