Pre-Training Representations of Binary Code Using Contrastive Learning

ContraBin v0.2.0 — Contrastive pre-training of binary code representations

ContraBin learns transferable representations of binary code by contrastively aligning three views of the same program — source, binary / LLVM IR, and a short natural-language comment — through a two-stage curriculum of simplex interpolation. This record hosts an open, research-grade reimplementation of:

Zhang, Y., Huang, C., Zhang, Y., Shao, H., Leach, K., & Huang, Y. (2025). Pre-Training Representations of Binary Code Using Contrastive Learning. Transactions on Machine Learning Research. arXiv:2210.05102.

GitHub: https://github.com/CoderDoge1108/ContraBin  ·  Release tag: v0.2.0  ·  License: MIT

What’s new in v0.2.0 (vs. the initial v0.1.0 Zenodo release)

This version is not a replication package. It is a clean, extensible library intended to let other researchers build on top of the ContraBin recipe: swap backbones, redesign downstream tasks, or replace the triplet data pipeline. Relative to the prior Zenodo snapshot, v0.2.0 provides:

• A complete, type-checked Python package (pyproject.toml, pydantic configs, typer CLI) covering the full pipeline: triplet construction, pre-training, embedding extraction, and four downstream tasks.
• A modular architecture where encoders, projection heads, simplex interpolation, and contrastive losses are independent modules that can be mixed and matched.
• An offline-friendly contrabin-tiny backbone so the full training loop runs on a laptop CPU — useful for teaching, CI, and ablation prototyping.
• A modern test/lint/type stack (pytest with 44 tests, ruff, mypy) and GitHub Actions CI across Python 3.10 / 3.11 / 3.12.
• Revised downstream tasks that are more research-relevant than the originals (see table below).
• Full documentation set: docs/method.md, docs/data.md, docs/training.md, docs/tasks.md, docs/faq.md; runnable examples/ scripts; and a Jupyter quickstart notebook.

Method in one paragraph

Two anchored (frozen) encoders share weights to produce source and comment embeddings, while a separate trainable encoder produces the binary / IR embedding. During training, the two anchored embeddings are combined by a learned simplex interpolation module Γ(src, cmt; λ) to form an intermediate view, which is then aligned with the binary embedding via an InfoNCE objective. A curriculum scheduler moves the interpolation from naive (no mixing) to linear (scalar λ) to nonlinear (element-wise λ). A CLIP-style contrastive loss additionally aligns all three modalities pairwise. Full formulation and the exact mapping to Algorithm 1 of the paper are in docs/method.md.

Downstream tasks (revised)

The paper evaluates four tasks. Two (function-name recovery and binary summarization) are well-posed and kept as-is. The other two are reformulated to measure representation quality rather than fine-tuning-head capacity:

Rationale: see docs/tasks.md in the archive.

Quick start

pip install -e '.[dev]'
pytest -q                              # 44 tests, CPU-only, offline
contrabin make-synthetic --output data/processed/train.jsonl -n 64
contrabin make-synthetic --output data/processed/val.jsonl   -n 16 --seed 1
contrabin pretrain --config configs/smoke.yaml
contrabin task retrieve     --config configs/pretrain.yaml --gallery data/processed/gallery.jsonl --checkpoint outputs/contrabin/final.pt
contrabin task name-recovery --config configs/pretrain.yaml --train ... --val ...
contrabin task summarize     --config configs/pretrain.yaml --train ... --val ...
contrabin task provenance    --config configs/pretrain.yaml --train ... --val ...

How to cite

Please cite the original paper when using ContraBin:

@article{zhang2025contrabin,
  title   = {Pre-Training Representations of Binary Code Using Contrastive Learning},
  author  = {Zhang, Yifan and Huang, Chengzhi and Zhang, Yichi and Shao, Haoran and Leach, Kevin and Huang, Yu},
  journal = {Transactions on Machine Learning Research},
  year    = {2025},
  url     = {https://arxiv.org/abs/2210.05102}
}

Integrity

Archive: ContraBin-0.2.0-zenodo.zip (~86 KB, 71 files)
SHA-256: 641821305d3bb6bde30a827291f586526a2bbfa012e8c00ec88d34fb36a6f7dc
Git tag: v0.2.0 (commit ef4d4ca)