Fool Me If You Can: On the Robustness of Binary Code Similarity Detection Models against Semantics-preserving Transformations

This artifact supports the claims made in the FSE ’26 paper, Fool Me If You Can: On the Robustness of Binary Code Similarity Detection Models against Semantics-Preserving Transformations. It includes the datasets, pre-trained models, and evaluation scripts necessary to reproduce the paper’s empirical results on the robustness of six binary code similarity detection (BCSD) models under semantics-preserving transformations. The artifact provides pre-built evaluation datasets for eight semantics-preserving transformations across two attack scenarios: false-negative triggering perturbation and false-positive triggering perturbations. It also includes pre-trained weights for all six evaluated BCSD models, along with a Conda environment and corresponding Docker containers to ensure reproducible execution. Inference and evaluation scripts are provided to run all six models on the included datasets and compute the evaluation metrics reported in the paper. Additionally, optional dataset generation scripts allow users to regenerate binary variants from source code and reproduce the greedy method for generating false-positive triggering perturbations.