Artifact of the paper "Automated Proof-of-Concept Generation for Smart Contract Access Control Vulnerability via Static Analysis and LLM-based Multi-Agent"

A PoC generator using LLM-based Mult-Agent guided by static analysis.

1. Foundry, refer to https://github.com/foundry-rs/foundry

   cd foundry-env
   forge init

2. Python3

   pip3 install -r requirements.txt

python3 main.py foundry-env/src/smartbugs/mycontract.sol -c MyContract --api-key "YOUR_API" --analyzer "tx-origin" -v 

python3 main.py foundry-env/src/dappscan/coordinape-protocol-7a8e6173305696c72195fa4242126d284611270c/contracts/ApeProtocol/wrapper/beacon/ApeVault.sol -r "@openzeppelin/contracts/=dataset/dappscan/swc-105/BlockSec-blocksec_coordinape_v1.1_signed/coordinape-protocol-7a8e6173305696c72195fa4242126d284611270c/@openzeppelin/contracts/" -c "ApeVaultWrapperImplementation" --api-key "YOUR_API" -v

The foundry env related settings can be configured in foundry.toml, always considered when testing dappscan cases as complicated imports.

There are some writtern patterns in the folder src/analyzer/pattern, you can use --analyzer to use specific detection pattern.

More patterns can be added if needed.

Running logs can be save to logs/ folder, and the valid poc should be saved to foundry-env/test folder.

You can check the log and the print content in the console to analyze the LLM response and the failure info, which can be helpful to refine the workflow and the code.

The generated valid PoCs on the benchmark are put into folder dataset/benchmark.

Generated PoCs of GPT4o, GPTo4-mini, DeepSeek-R1, Claude3.7-Sonnet are put into folder baseline.

Related PoCGen-NAT and PoCGen-NMA groups are put into folder ablation_experiment, including labeled results in ablation_evaluation_manual.xlsx.

The used scripts for ablation study are stored in ablation_no_attacktrace.py and ablation_no_multiagent.py. Specific hints can be found in ablation_instruction.md.

Prompt templates for the four agents are included in the file src/llm/prompt/prompt_template.py and folder src/llm/agent. The prompt could be automatically implemented during analysis based on the results of related former steps in the workflow.

The PoC templates that instruct the LLM to refer are stored in src/llm/template, including several specific patterns.