Scripts and datasets to "Generating Robust Representations of Structures in OpenSSH Heap Dumps"

These are all scripts for reproducibility of "Generating Robust Representations of Structures in OpenSSH Heap Dumps".

• save_and_reduce_struct_dataset.py - Extract raw data structures from heap dumps dataset and label.
• masking_pretraining.py - Pre-Training with Masked Language Modeling of DistilBERT Transformer
• seq-class_finetuning.py - Fine-Tuning of that Transformer with Sequence Classification
• extract_embeddings_dataloader.py - Run inference on Transformer model and extract embeddings in the process.
• pooling_network.py  - Run embeddings through a neural network with a configurable hidden size. Can be used to extract compressed embeddings.
• SVMClassifier.py - Classify embeddings.
• classes.py - Helper for data structure dataset creation.

Associated Datasets:

• structs_eval.zip - Raw data structures extracted from heap dump and labelled (Validation). Also available: https://huggingface.co/datasets/johannes-garstenauer/structs_token_size_4_reduced_labelled_eval
• structs_train.zip - Raw data structures extracted from heap dump and labelled (Training). Also available: https://huggingface.co/datasets/johannes-garstenauer/structs_token_size_4_reduced_labelled_train
• embeddings_from_distilbert_masking_heaps.zip - Embeddings extracted from pre-trained transformer. Also available: https://huggingface.co/datasets/johannes-garstenauer/embeddings_from_distilbert_masking_heaps
• embeddings_from_distilbert_class_heaps.zip - Embeddings extracted from fine-tuned transformer. Also available: https://huggingface.co/datasets/johannes-garstenauer/embeddings_from_distilbert_class_heaps
• Heap Dumps available at:  https://zenodo.org/records/6537904

These are the Transformer models generated in the process:

• distilbert_masking_heaps.zip - Pre-trained Transformer. Load using HuggingFace library. Also available: https://huggingface.co/johannes-garstenauer/distilbert_masking_heaps
• distilbert_class_heaps.zip - FIne-tuned Transformer. Load using HuggingFace library. Also available: https://huggingface.co/johannes-garstenauer/distilbert_class_heaps