Published June 5, 2025 | Version v1
Conference paper Open

Cloak, Honey, Trap: Proactive Defenses Against LLM Agents

Creators

  • 1. ROR icon Ben-Gurion University of the Negev

Description

This artifact accompanies our USENIX Security 2025 paper “Cloak, Honey, Trap: Proactive Defenses Against LLM Agents.”

  • CHeaT.zip — complete repository with the CHeaT CLI, datasets, and a playground notebook.

  • CTF machines — 11 challenge VMs, each provided as a separate ZIP archive.

The repository’s main README appears below.

---

## 1. Overview

**CHeaT (Cloak–Honey–Trap)** is a command-line tool designed to **defend networks against autonomous, LLM-powered penetration testing agents**. It works by embedding string-based payloads into network assets—payloads specifically crafted to **disrupt, deceive, and detect** such agents.

### Core Defense Strategies:

1. **Cloaking** – Obfuscate sensitive data with strategic misdirection
2. **Honey** – Embed tokens to detect and fingerprint LLM-driven agents
3. **Traps** – Deploy inputs that stall, confuse, or crash malicious automation

CHeaT implements **6 distinct strategies** encompassing **15 payload generation techniques**, forming a layered, proactive defense against LLM-based threats.


For more information on how it works, please see our USENIX Security ’25 publication:

``
Daniel Ayzenshteyn, Weiss, Roy, and Yisroel Mirsky. "Cloak, Honey, Trap: Proactive Defenses Against LLM Agents" 34rth USENIX Security Symposium (USENIX Security 25). 2025.‏
``

---

## 2. Tool Quick Start 🚀

> **TL;DR**

```bash
# clone repo & enter tool folder
git clone https://github.com/Daniel-Ayz/CHeaT.git
cd CHeaT

# optional: create venv
python3 -m venv .venv && source .venv/bin/activate

# install (pure-stdlib -> nothing to pull)
pip install -e .

# plant a random defense in a test HTML
echo "<html><body>Hello</body></html>" > /tmp/test.html
cheat --action plant --details '{
  "assettype": "web_file",
  "file_path": "/tmp/test.html",
  "technique": "random",
}'
````

| Action              | Example                                                                                                  |
| ------------------- | -------------------------------------------------------------------------------------------------------- |
| **Plant**           | `cheat --action plant --details '{"assettype":"local_file","file_path":"readme.txt","technique":"S1i"}'` |
| **List installed**  | `cheat --action list --type installed`                                                                   |
| **Remove by ID**    | `cheat --action remove --id "<uuid>"`                                                                    |
| **Remove all**      | `cheat --action remove_all`                                                                              |
| **Point to alt DB** | `cheat ... --database /path/to/db`                                                                       |

See [`cheat/README.md`](cheat/README.md) for full CLI docs.

---

## 3. Repository Layout 

```
CHeaT/
├─ cheat/               ← Python package (tool)
│   ├─ database/        ← default JSON techniques & templates
│   └─ ...
├─ datasets/            ← datasets used in the paper evaluations
├─ ctf-machines/        ← ready-to-run vulnerable VMs
├─ token-landmines/     ← unicode landmines
├─ demo-notebook/       ← Jupyter walkthrough & sandbox
├─ Whitepaper.pdf       ← full academic paper
└─ README.md            ← you are here
```

 

### 3.1 ``cheat/``

Here you will find the source code to the CHeaT payload injection tool, along with instruction in [`cheat/README.md`](cheat/README.md)

### 3.2 ``datasets/``

In this directory you will find the datasets used in the paper's evalautions.

Directory **`datasets/`** collects:

```
datasets/
├─ dataset_main.json
├─ dataset_boosted_with_pi.json
├─ dataset_unicode_honeytokens.json
└─payloads/
  ├─ payloads.json
  └─ payloads_boosted_with_prompt_injection.json
````

* **`payloads.json`** – the framed payloads constructed in the paper.  
* **`payloads_boosted_with_prompt_injection.json`** – payloads that are *boosted* with a prompt-injection wrapper.  
* **`dataset_main.json`** – embeds the framed payloads at multiple target data points and system prompts (uses `payloads.json`).  
* **`dataset_boosted_with_pi.json`** – identical structure but built from the boosted payloads.
* **`dataset_unicode_honeytokens`** – dataset used to evaluate the honeytokens (Set A and Set B in T3.2)
  


### 3.3 ``ctf-machines/``

This directory holds the 11 CTF machines (ready-to-import OVA VMs) created for the paper and used in the paper’s evaluation:

`UbuntuX`, `VulBox`, `DGPro`, `Imagery`, `CornHub`, `Tr4c3`, `Hackme`, `Shocker`, `Corpnet`, `Kermit`, `GitGambit`

In each sub-dir you will find a walkthrough solution. For the respective .ova Vm files, please visit our Zenodo dataset.

If you use these CTFs in your work, please cite our paper.

 

### 3.4 ``token-landmines/``

Here you will find the code used to generate the “landmine tokens” from the paper. Token landmines are rare sequences of tokens that corrupt a model's internal state causing it to output gibberish or hallucinations.

The contents of this folder will be empty until 1 month after publicaiton to give vendors time to patch their LLM services.

 

### 3.5 ``demo-notebook/``

Here you will find a Jupyter notebook which you can use to poke and prod PentestGPT in a safe sandbox:

- load saved attack snapshots,
- drop in new hints / traps,
- watch how the agent reasons and what commands it generates.

---

## 4. License 📄

This project is licensed under the CC BY-NC 4.0 License. See the [LICENSE](./LICENSE) file for details.

---

## 5. Citation 🤝

If you use our code, datasets, or CTF VMs, please cite us:

```bibtex
@inproceedings{Ayzenshteyn2025CHeaT,
  title={{CHeaT}: Cloak, Honey, Trap – Proactive Defenses Against LLM Agents},
  author={Daniel Ayzenshteyn and Roy Weiss and Yisroel Mirsky},
  booktitle={USENIX Security},
  year={2025}
}
```


Happy trapping! 🕸️

Files

CHeaT.zip

Files (48.7 GB)

Name Size Download all
md5:fa4afc604ca5c14d0ad63cb541ace0e3
25.2 MB Preview Download
md5:42606bc070f94e217ec6766dce971b85
5.2 GB Preview Download
md5:7fb2bd075bc29db51da1150e0ec354b4
4.7 GB Preview Download
md5:8295690b81676d21ed7bc6b98f481c0f
5.6 GB Preview Download
md5:7ba6d4fc9ec2ba70591e55c7fb9dfe5c
4.6 GB Preview Download
md5:a9f3cc08d7b23d636b05d2811d2b6d94
1.0 GB Preview Download
md5:80a3a3300d50c57ca65d58629a213854
6.8 GB Preview Download
md5:39e5156ec5957dbdd49b1f2285a7a658
4.9 GB Preview Download
md5:94c59a8ee112a87bab89eaa86afb3592
5.0 GB Preview Download
md5:c1b5c43b00dbbb120542892730ecfda2
4.3 GB Preview Download
md5:8d342fd6b0799defac80edabcca74cc0
2.0 GB Preview Download
md5:fa48fb5910e88d921ad2b60779ea22c7
4.5 GB Preview Download

Additional details

Software

Repository URL
https://github.com/Daniel-Ayz/CHeaT