Finding Inlined Functions in Optimized Binaries

This is a Repo for "Finding Inlined Functions in Optimized Binaries" paper.

Step 1:

Please install the given version of Huggingface Transformer (in "transformers" folder) using setup.py file. We have edited some part of the code inside. The default version of Huggingface Transformer may not work for this project without modification.

Step 2:

Please find the code in the “Code for Pretraining” folder for pre training the model from scratch. Here is the command:

python run_language_modeling.py --output_dir model/decom-small-v1 --model_type roberta --mlm --fp16 --config_name model/decom-small --tokenizer_name model/decom-small --do_train --do_eval --learning_rate 5e-5 --num_train_epochs 5 --save_total_limit 2 --save_steps 500 --per_gpu_train_batch_size 10 --evaluate_during_training --seed 42 --train_data_file=data/decom_train.txt --eval_data_file=data/decom_test.txt --logging_step 500

Please adjust your batch size according to your resource. The dataset and all other necessary components are in place. Install ``Apex” to support 16 bit precision. Create an empty directory decom-small-v1 in the model folder. We also shared the pretrained model to reuse in the folder “Pretrained Model”. Therefore, you do not need to pre-train again.

Step 3:

Please find the code in the “Code for finetuning” folder for fine-tuning the model. Here is the command:

python run_glue.py --model_name_or_path <path to the pretrained model>--task_name SST-2 --do_train --do_eval --data_dir <path to the fine-tune dataset> --max_seq_length 400 --per_device_train_batch_size 15 --learning_rate 5e-5 --num_train_epochs 3.0 --output_dir <path to the output directory>--fp16 --save_total_limit 2

The dataset is shared in the “Code for finetuning” folder.  We shared the fine-tuned model to reuse in the folder “Finetuned Model”. Therefore, you do not need to fine-tune again.

Step 4:

In the folder “Generating Funcre Result” ghidraX_master.txt files contain functions from different optimization levels. Here are the changes that require for running the script:

Line 5 --- provide path to the label.txt. The file is shared in the folder “Generating Funcre Result”

Line 29, 32 --- provide paths to the fine-tuned model

Line 90, 91 --- choose the correct optimization level and adjust the paths.

However, we shared the processed files in “Evaluation” Folder. The format of the filename is “new_us_resX.txt”

Step 5:

In the “Evaluation” folder, we provide pre-calculated results from Ghidra, IDA-Pro and Funcre. We have skipped some parts here to keep it simple. We have some more scripts to produce Ghidra and IDA pro results from the file. We would be happy to share those scripts if anyone asks for those. However, 3 python files shared here are sufficient to produce the precision, recall and F-score for Table 3. Please change the optimization level at line 1 for all 3 scripts before running the scripts.  

"Result_if_position_known_10+-" contains the result from our first model for different optimization level, where we assume we know the position of the inline functions. Here we presented the result for the context +-10 because it is the best performing context. This results also demonstrate that the model can be extremely powerful if the location of the inline function is known.

We will publish a docker container and provide email addresses to contact the authors of the paper upon acceptance.

For BinSwarm please see the following link:

https://github.com/anandsaw/bugswarm