LLaMA-Reviewer: Advancing Code Review Automation with Large Language Models through Parameter-Efficient Fine-Tuning

Resources related to the research work "LLaMA-Reviewer: Advancing Code Review Automation with Large Language Models through Parameter-Efficient Fine-Tuning"

1. Dependency

The dependencies are in `requirements.txt`,  use`pip install -r reuirements.txt ` to install required libraries.

2. Brief Introduction

LLaMA-Reviewer, an innovative framework that leverages the capabilities of LLaMA, a popular LLM, in the realm of code review. Mindful of resource constraints, this framework employs parameter-efficient fine-tuning (PEFT) methods, delivering high performance while using less than 1% of trainable parameters.

An extensive evaluation of LLaMA-Reviewer is conducted on two diverse, publicly available datasets. Notably, even with the smallest LLaMA base model consisting of 6.7B parameters and a limited number of tuning epochs, LLaMA-Reviewer equals the performance of existing code-review-focused models.

3. Project Structure

• For baselines that don't have immediate results, testing code is provided. The necessary datasets and models can be acquired from the respective repositories given.
• In the case of LoRA, it's necessary to add the `xturing-base-model` to the weights folders.
• For prefix-tuning, the 'lit-llama-base-model' should be added to this location: `prefix(lit-llama)/lit-llama-main/checkpoints/lit-llama/7B`.
• The data has been omitted, but can be generated using the provided code and the original data from the repositories of two papers mentioned in the study.
• The results and outputs are preserved in the provided 7-zip files.
• The base models in the deposit are compressed by volumes, and they can also be obtained as per the guidelines of the corresponding framework.

. ├── baselines                # Baselines that don't have immediate results │   ├── AUGER │   ├── CommentFinder │   └── Tufano ├── LoRA(xturing)            # Low-rank adaptation experiments, with LoRA rank of 8 or 16 │   ├── r=16                 │   └── r=8                     ├── prefix(lit-llama)        # Prefix-tuning expereiments │   └── lit-llama-main ├── lit-llama-base-model    # Base model for prefix-tuning │   └── lit-llama.pth ├── xturing-base-model        # Base model for LoRA │   └── pytorch_model.bin └── requirements.txt        # Requirements

Note: in the `r=8` folder, "code alpaca" means "PL+NL" data, while "code alpaca only code" means "only PL"

Note: in the `r=16` folder, "code alpaca" means "only PL"

Note(new): Now the PEFT and the finetuning process have been well supported by Huggingface's transformers library, it is more convenient to directly use the transformers library.