GitHub - OpenAutoCoder/Agentless: Agentless🐱: an agentless approach to automatically solve software development problems (original) (raw)

😺 Agentless

😽 News

Dec 2nd, 2024: We integrated Agentless with Claude 3.5 Sonnet to achieve 40.7% and 50.8% solve rate on SWE-bench lite and verified
Oct 28th, 2024: We just released OpenAutoCoder-Agentless 1.5!
July 1st, 2024: We just released OpenAutoCoder-Agentless 1.0! Agentless currently is the best open-source approach on SWE-bench lite with 82 fixes (27.3%) and costing on average $0.34 per issue.

😺 About

Agentless is an agentless approach to automatically solve software development problems. To solve each issue, Agentless follows a simple three phase process: localization, repair, and patch validation.

🙀 Localization: Agentless employs a hierarchical process to first localize the fault to specific files, then to relevant classes or functions, and finally to fine-grained edit locations
😼 Repair: Agentless takes the edit locations and samples multiple candidate patches per bug in a simple diff format
😸 Patch Validation: Agentless selects the regression tests to run and generates additional reproduction test to reproduce the original error. Using the test results, Agentless re-ranks all remaining patches to selects one to submit

🐈 Setup

First create the environment

git clone https://github.com/OpenAutoCoder/Agentless.git cd Agentless

conda create -n agentless python=3.11 conda activate agentless pip install -r requirements.txt export PYTHONPATH=$PYTHONPATH:$(pwd)

⏬ Developer Setup

for contribution, please install the pre-commit hook.

pre-commit install # this allows a more standardized code style

Then export your OpenAI API key

export OPENAI_API_KEY={key_here}

Now you are ready to run Agentless on the problems in SWE-bench!

Note

To reproduce the full SWE-bench lite experiments and follow our exact setup as described in the paper. Please see this README

🧶 Comparison

Below shows the comparison graph between Agentless and the best open-source agent-based approaches on SWE-bench lite

🐈‍⬛ Artifacts

You can download the complete artifacts of Agentless in our v1.5.0 release:

🐈‍⬛ agentless_swebench_lite: complete Agentless run on SWE-bench Lite
🐈‍⬛ agentless_swebench_verified: complete Agentless run on SWE-bench Verified
🐈‍⬛ swebench_repo_structure: preprocessed structure information for each SWE-Bench problem

You can also checkout classification/ folder to obtain our manual classifications of SWE-bench-lite as well as our filtered SWE-bench-lite-S problems.

📝 Citation

@article{agentless, author = {Xia, Chunqiu Steven and Deng, Yinlin and Dunn, Soren and Zhang, Lingming}, title = {Agentless: Demystifying LLM-based Software Engineering Agents}, year = {2024}, journal = {arXiv preprint}, }