There is a newer version of the record available.

Published September 11, 2026 | Version v3

Librarian Catches Thief: Surfacing Supply Chain Attack Campaigns via Document Similarity in an AI Agent Skill Registry

  • 1. ROR icon Edgewood University

Description

Supply chain attack campaigns mass-produce their artifacts, and mass production leaves a structural trace: hundreds of near-identical files. In AI agent skill marketplaces, which distribute capabilities as natural-language instruction files following the Agent Skills standard (an open format adopted by over 30 platforms), the artifact carrying that trace is a document, and the trace is detectable by document similarity alone. None of the marketplaces examined in this study documented deduplication or similarity checking at ingest. In early 2026, threat actors uploaded hundreds of skills to the ClawHub registry for the OpenClaw agent framework, directing users to install credential-stealing malware.

A MinHash locality-sensitive hashing pipeline processed 31,634 skill files from the ClawHub registry and 31 community repositories with the scanner and hash parameters untuned. At a 90% Jaccard threshold, selected from a sweep, 2,622 similarity clusters contained 7,147 file memberships across 7,061 distinct files, 22.3%, and all 337 Koi-documented malicious skills present in the corpus appeared in a cluster, a within-archive recall expected by construction for template-reuse campaigns. Among the 38 clusters containing at least 10 files, 30 contained confirmed malicious content (78.9%, 95% CI [63.7, 88.9]), and every cluster with 20 or more files was malicious (11 clusters, ten of them subclusters of one campaign; 95% CI [74.1, 100]).

These results show that similarity clustering, established in code registries, transfers to text-based ecosystems where campaigns reuse templates at scale. Mirror commit history suggests the largest campaign's cluster was forming as the campaign was disclosed. Document similarity thus offers a lightweight, signature-free first pass for publish-time security screening in text-based registries.

Notes (English)

Version 3, 2026-09-11. Changes from v2 (2026-07-11):

- Seed-robustness adjusted Rand index: 0.931 to 0.955 across the five seeds (v2: at least 0.9999, computed with a wrong expectation term). The robustness claim is withdrawn.
- Clustered files: 7,061 distinct files, 22.3% of the corpus (v2 reported 7,147, which counts cluster memberships).
- Marketplace similarity: the two mirror pairs score 1.000; the highest non-mirror pair is 0.370 (v2: 53.8%, from a different scan).
- RQ3 overlap: 33.0%, from the released pattern scanner on the archived corpus (v2: 18.0%, from a pattern set that was not preserved).
- Documented attacker accounts recovered by clustering: 8 of 12; a ninth by payload inspection within clustered files (v2: 9 of 12).
- MinHash recall over the two primary campaign accounts: 547 of 564 files, 97.0% (v2: 98.8%, the recall against Koi's full list).
- Campaign 1: 352 indexed files (v2: 340).
- The claim that the largest campaign formed a detectable cluster before public disclosure is removed; the abstract states what the mirror history shows.
- Author affiliations added.

Reproduction material: https://github.com/galeep/plugin-librarian, directory paper/. The analysed ClawHub archive is preserved at Software Heritage, swh:1:rev:16c991dea171b9f4e785cb2a70dc1ae2ade83396.

Files

zenodo-v3-preprint-d9c0eaf.pdf

Files (618.7 kB)

Name Size
md5:9101c7d28f8db11391cabf5afd2453da
618.7 kB Preview Download

Additional details

Software

Repository URL
https://github.com/galeep/plugin-librarian
Programming language
Python
Development Status
Active