This document describes the artifact for paper "An Interval Compiler for Sound Floating Point Computations" at CGO 2021. The artifact is in the form of a virtual machine running Ubuntu 18.04. It contains full source code of IGen interval compiler, benchmarks and scripts for reproducing main experiments. In addition, the artifact also contains the source code of the LLVM Project 11.0 with custom modifications to enable both, language extensions and specific pragmas used by our interval compiler. ============================================== Software Dependencies: 64-bit VirtualBox from Oracle GNU C compiler 7.5.0, CMake 3.10 Python 3.6 or higher. R 4.0.3 LLVM Project 11.0 (Clang, Polly and Extra Clang Tools). MPFR 4.0.1 and CR-LIBM 1.0 For comparison with other interval libraries: Boost 16.5 Filib++ 3.0.2 Gaol 4.2 (configured with "enable-preserve-rounding=no") MPFI 1.5.3 =============================================== Hardware Dependencies: 1. The virtual machine requires at least 8 GB of disk space. 2. We recommend allocating at least 4 GB RAM to the virtual machine. 3. Machine with AVX2 support. Note: We recommend disabling Intel Turbo Boost and Hyper-Threading technologies to avoid the effects of frequency scaling and resource sharing on the measurements. Note that these technologies can be easily disabled in the BIOS settings of the machines that have BIOS firmware. ================================================ Installation: 1. Import the virtual machine in VirtualBox. More information on importing virtual machine in VirtualBox can be found at https://docs.oracle.com/cd/E26217_01/E26796/html/qs-import-vm.html. 2. The login credentials for the Virtual Machine are: username: cgo2021 password: igen@cgo21 3. We have already installed all dependencies. =============================================== Running the benchmarks: Once in the virtual machine, open the terminal and navigate to the 'benchmarks' directory: cd artifact/benchmark There is a script named "run_benchmarks.py" which sets up, builds and runs all benchmarks. You can run it as follows: python3 run_benchmarks.py -all To run individual benchmarks consult the "--help" option. After the experiments finish running, the generated CSV files with the results are saved in "artifact/benchmarks/results" directory. There is one folder for each benchmark (dft, gemm, ffnn, potrf and redu). For comparison, the numbers presented in the paper are also included in 'results/paper'. The results used to generate Figures 9, 9a, 9b and Table 5 of the paper are saved in the following files respectively: - interval_perf.csv - real_perf_.csv - accuracy_.csv - overhead_.cs To generate the performance graphs in Fig. 8 use the following command: python3 run_benchmarks.py -plot The graphs will be saved as PDF files in the 'results/' folder. The results of Fig. 10 of the paper are saved in the following file: - results/redu_mvm/avg_accuracy_.csv ================================================ Interpreting the results: interval_perf.csv ----------------- This file contains the performance measurements for the specified benchmark in interval operations per cycle. Each row represents a series of Fig. 8. The rows have the following order: IGen-vv IGen-sv IGen-vv-dd IGen-sv-dd Boost Gaol Filib IGen-ss You can use the "-plot" option of the "run_benchmarks.py" script to plot the data. real_perf_.csv ------------------------- This file contains the real performance (flops per cycle) of the specified benchmark (e.g. dft-64). This corresponds to the data in Fig. 9a. The columns have the following order: Baseline, IGen-vv, IGen-vv-dd accuracy_.csv ------------------------- This file contains the accuracy in bits of the specified benchmark (e.g. dft-64). This corresponds to the data in Fig. 9b. The accuracy is represented using box-and-whisker. Thus, the first and last column are the lowest and the greatest data point respectively. The intermediate columns represent the division of the quartiles. The rows have the following order: IGen-vv IGen-vv-dd overhead_.csv ------------------------- This file contains the slowdown of IGen compared to a baseline. This corresponds to the data in Table 5. The data is in the same order as in the table, namely: IGen-sv, IGen-vv, IGen-sv-dd, IGen-vv-dd. ================================================ Recompiling IGen: IGen source code is located in "home/cgo2021/artifact/igen_src". The artifact comes already with a precompiled binary. To recompile, type the following command: cd artifact python3 install_igen.py This script will recompile IGen and copy its binary into the "bin" folder. ================================================ Changes made to LLVM-Project 11.0: The following files were modified to allow IGen's custom pragma and reduction detection: - llvm-project/clang/include/clang/Basic/Attr.td - llvm-project/clang/include/clang/Basic/AttrDocs.td - llvm-project/clang/include/clang/Basic/TokenKinds.def - llvm-project/clang/include/clang/Lex/LiteralSupport.h - llvm-project/clang/include/clang/Parse/Parser.h - llvm-project/clang/lib/Lex/LiteralSupport.cpp - llvm-project/clang/lib/Parse/PerseDecl.cpp - llvm-project/clang/lib/Parse/ParsePragma.cpp - llvm-project/clang/lib/Parse/ParseStmt.cpp - llvm-project/clang/lib/Sema/SemaExpr.cpp - llvm-project/clang/lib/Sema/SemaStmtAttr.cpp - llvm-project/polly/lib/Transform/Canonicalization.cpp - llvm-project/polly/lib/Analysis/DependenceInfo.cpp - llvm-project/polly/lib/Analysis/ScopBuilder.cpp - llvm-project/polly/include/polly/ScopInfo.h ===============================================