AlpaGasus: Training A Better Alpaca with Fewer Data (original) (raw)

Authors:Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, Hongxia Jin

View PDF

Abstract:Large language models (LLMs) strengthen instruction-following capability through instruction-finetuning (IFT) on supervised instruction/response data. However, widely used IFT datasets (e.g., Alpaca's 52k data) surprisingly contain many low-quality instances with incorrect or irrelevant responses, which are misleading and detrimental to IFT. In this paper, we propose a simple and effective data selection strategy that automatically identifies and filters out low-quality data using a strong LLM (e.g., ChatGPT). To this end, we introduce AlpaGasus, which is finetuned on only 9k high-quality data filtered from the 52k Alpaca data. AlpaGasus significantly outperforms the original Alpaca as evaluated by GPT-4 on multiple test sets and the controlled human evaluation. Its 13B variant matches >90>90\%>90 performance of its teacher LLM (i.e., Text-Davinci-003 generating the 52k data) on test tasks. It also provides 5.7x faster training, reducing the training time for a 7B variant from 80 minutes (for Alpaca) to 14 minutes. Moreover, the experiments prove the efficacy of our method across diverse datasets, base models, and LLM filters. Overall, AlpaGasus demonstrates a novel data-centric IFT paradigm that can be generally applied to instruction-tuning data, leading to faster training and better instruction-following models. Our project page is available at: this https URL

Submission history

From: Lichang Chen [view email]
[v1] Mon, 17 Jul 2023 17:59:40 UTC (1,715 KB)
[v2] Sat, 30 Sep 2023 02:59:34 UTC (2,939 KB)
[v3] Thu, 26 Oct 2023 04:08:51 UTC (2,939 KB)
[v4] Sat, 4 Nov 2023 21:39:59 UTC (2,939 KB)
[v5] Tue, 13 Feb 2024 18:37:25 UTC (2,940 KB)