One Step Further: Evaluating Interpreters Using Metamorphic Testing

This holds the codes and experiment results for the paper "One Step Further: Evaluating Interpreters Using Metamorphic Testing" submitted to ISSTA 2022.

The related codes and models are also released on GitHub https://github.com/BLINKSK/Metamorphic-Testing-for-Interpretation-Methods. Users can generate metamorphic images in the custom dataset and test the interpreters.

Due to storage space and upload speed limitations, we only uploaded some metamorphic images in the packages "insert_object", "delete_object" and "squeeze_background", and info of ground_truth_bounding_box in the package "ground_truth_bounding_box".

For the same reason, for every interpreter, we also only select some interpretation results of error, or difference before and after metamorphism to display in the seven packages. We have also uploaded the results in Google Drive https://drive.google.com/drive/folders/1e1A1wNxLuzhEf7bczSiqveV1oeWJL-j2?usp=sharing.

Note that our experimental results and statistics are performed on all data, but the data here is only a part of the display.

If you want to get all the data or have any questions, please email me <weijiali1119@stu.xjtu.edu.cn>; thank you for your interest.