HisFuzzer
Authors/Creators
Description
Artifacts for the ASE submission #516
Table of Contents
- Background
- Install
- Usage
- Reports
Background
HisFuzzer utilizes historical data to test SMT solvers.
You can use it to collect the bug-triggering formulas from Z3 and cvc5's bug tracking systems.
You can also perform differential tests on the SMT solvers.
See below for more details on the usage.
Install
This project uses efficient-apriori, antlr4, and github3. Go check them out if you don't have them locally installed.
pip3 install efficient-apriori
pip3 install antlr4-python3-runtime==4.9.2
pip3 install github3.py
Besides, if you want to test the solvers, you need to download and build them first.
You can download Z3 and cvc5 in Z3Prover/z3: The Z3 Theorem Prover (github.com) and cvc5/cvc5: cvc5 is an open-source automatic theorem prover for Satisfiability Modulo Theories (SMT) problems. (github.com).
Please follow the instructions given by them for installation.
Usage
You can use HisFuzzer to update the historical data in the bug tracking systems of Z3 and cvc5 with your GitHub personal access token:
bin/HisFuzzer --update --token=[TOKEN]
GitHub personal access token can be generated in Personal Access Tokens (github.com).
In general, the first update process takes a long time.
Since we have already initialized it, you can use it directly without updating to test the solvers, for example:
bin/HisFuzzer --solver1=z3 --solverbin1=/home/z3/build/z3 --solver2=cvc5 --solverbin2=/home/cvc5/build/bin/cvc5 --benchmark=/home/benchmarks
You can download the benchmarks in SMT-LIB-benchmarks · GitLab (uiowa.edu), and you can use the above command to mutate the .smt2 files in the downloaded benchmarks to try to find bugs.
In addition, we also give an example that can be easily reproduced for you to experience:
bin/HisFuzzer --solver1=z3 --solverbin1=/home/z3/build/z3 --solver2=cvc5 --solverbin2=/home/cvc5/build/bin/cvc5 --reproduce=bug
Note that when reproducing, use git checkout
Please refer to the readme.md file in the /reproduce folder for more details.
Bug reports (updating)
z3
https://github.com/Z3Prover/z3/issues/5722#issuecomment-998452982
https://github.com/Z3Prover/z3/issues/5737 (moved to https://github.com/Z3Prover/z3/issues/4889#issuecomment-1004329224)
https://github.com/Z3Prover/z3/issues/5753#issuecomment-1007087883
https://github.com/Z3Prover/z3/issues/5753#issuecomment-1008214504
https://github.com/Z3Prover/z3/issues/5774
https://github.com/Z3Prover/z3/issues/5777#issuecomment-1014065487
https://github.com/Z3Prover/z3/issues/5851
https://github.com/Z3Prover/z3/issues/5932
https://github.com/Z3Prover/z3/issues/5939
https://github.com/Z3Prover/z3/issues/5722#issue-1084089459
https://github.com/Z3Prover/z3/issues/5798
https://github.com/Z3Prover/z3/issues/5769
https://github.com/Z3Prover/z3/issues/5769#issuecomment-1015129354
https://github.com/Z3Prover/z3/issues/5778#issuecomment-1018132484
https://github.com/Z3Prover/z3/issues/5789
https://github.com/Z3Prover/z3/issues/5806
https://github.com/Z3Prover/z3/issues/5825
https://github.com/Z3Prover/z3/issues/5826
https://github.com/Z3Prover/z3/issues/5825#issuecomment-1035013023
https://github.com/Z3Prover/z3/issues/5769#issuecomment-1039749579
https://github.com/Z3Prover/z3/issues/5867
https://github.com/Z3Prover/z3/issues/5722#issuecomment-998652655
https://github.com/Z3Prover/z3/issues/5732
https://github.com/Z3Prover/z3/issues/5734 (moved to https://github.com/Z3Prover/z3/issues/5641#issuecomment-1000497502)
https://github.com/Z3Prover/z3/issues/5641#issuecomment-1000804719 (moved to https://github.com/Z3Prover/z3/issues/4889#issuecomment-1004328993)
https://github.com/Z3Prover/z3/issues/5741
https://github.com/Z3Prover/z3/issues/5753#issuecomment-1007067681
https://github.com/Z3Prover/z3/issues/5754
https://github.com/Z3Prover/z3/issues/5758
https://github.com/Z3Prover/z3/issues/5753#issuecomment-1007982562
https://github.com/Z3Prover/z3/issues/5770
https://github.com/Z3Prover/z3/issues/5753#issuecomment-1011732992
https://github.com/Z3Prover/z3/issues/5753#issuecomment-1011749964
https://github.com/Z3Prover/z3/issues/5753#issuecomment-1011755733
https://github.com/Z3Prover/z3/issues/5753#issuecomment-1012024921
https://github.com/Z3Prover/z3/issues/5779
https://github.com/Z3Prover/z3/issues/5778#issuecomment-1016018402
https://github.com/Z3Prover/z3/issues/5784
https://github.com/Z3Prover/z3/issues/5804
https://github.com/Z3Prover/z3/issues/5778#issuecomment-1030572118
https://github.com/Z3Prover/z3/issues/5816
https://github.com/Z3Prover/z3/issues/5852
https://github.com/Z3Prover/z3/issues/5778#issuecomment-1044323782
https://github.com/Z3Prover/z3/issues/5857
https://github.com/Z3Prover/z3/issues/5911
https://github.com/Z3Prover/z3/issues/5778#issuecomment-1074808615
https://github.com/Z3Prover/z3/issues/5778#issuecomment-1074848129
https://github.com/Z3Prover/z3/issues/5778#issuecomment-1077024464
https://github.com/Z3Prover/z3/issues/5924
https://github.com/Z3Prover/z3/issues/5925
https://github.com/Z3Prover/z3/issues/5778#issuecomment-1079833134
https://github.com/Z3Prover/z3/issues/5928
https://github.com/Z3Prover/z3/issues/5778#issuecomment-1079952765
https://github.com/Z3Prover/z3/issues/5935
https://github.com/Z3Prover/z3/issues/5778#issuecomment-1086568308
https://github.com/Z3Prover/z3/issues/5778#issuecomment-1086570683
https://github.com/Z3Prover/z3/issues/5778#issuecomment-1086765374
https://github.com/Z3Prover/z3/issues/5778#issuecomment-1086766609
https://github.com/Z3Prover/z3/issues/5940
https://github.com/Z3Prover/z3/issues/5942
https://github.com/Z3Prover/z3/issues/5778#issuecomment-1088211198
https://github.com/Z3Prover/z3/issues/5953
https://github.com/Z3Prover/z3/issues/5778#issuecomment-1091259893
https://github.com/Z3Prover/z3/issues/5778#issuecomment-1091354175
https://github.com/Z3Prover/z3/issues/6051
https://github.com/Z3Prover/z3/issues/6052
https://github.com/Z3Prover/z3/issues/6054
https://github.com/Z3Prover/z3/issues/6054#issuecomment-1138671939
https://github.com/Z3Prover/z3/issues/6058
https://github.com/Z3Prover/z3/issues/6060
https://github.com/Z3Prover/z3/issues/6061
https://github.com/Z3Prover/z3/issues/6079
https://github.com/Z3Prover/z3/issues/6080
cvc5
https://github.com/cvc5/cvc5/issues/7924
https://github.com/cvc5/cvc5/issues/8413#issuecomment-1081310585
https://github.com/cvc5/cvc5/issues/8016
https://github.com/cvc5/cvc5/issues/8052
https://github.com/cvc5/cvc5/issues/8456
https://github.com/cvc5/cvc5/issues/8572
https://github.com/cvc5/cvc5/issues/7938
https://github.com/cvc5/cvc5/issues/8118
https://github.com/cvc5/cvc5/issues/8124
https://github.com/cvc5/cvc5/issues/8133
https://github.com/cvc5/cvc5/issues/8147
https://github.com/cvc5/cvc5/issues/8160
https://github.com/cvc5/cvc5/issues/8182
https://github.com/cvc5/cvc5/issues/8349
https://github.com/cvc5/cvc5/issues/8412
https://github.com/cvc5/cvc5/issues/8457
https://github.com/cvc5/cvc5/issues/8515
https://github.com/cvc5/cvc5/issues/7858
https://github.com/cvc5/cvc5/issues/7858#issuecomment-1000794259
https://github.com/cvc5/cvc5/issues/7893
https://github.com/cvc5/cvc5/issues/7894
https://github.com/cvc5/cvc5/issues/7902
https://github.com/cvc5/cvc5/issues/7917
https://github.com/cvc5/cvc5/issues/7918
https://github.com/cvc5/cvc5/issues/7919
https://github.com/cvc5/cvc5/issues/7937
https://github.com/cvc5/cvc5/issues/7974
https://github.com/cvc5/cvc5/issues/8001
https://github.com/cvc5/cvc5/issues/8067
https://github.com/cvc5/cvc5/issues/8094
https://github.com/cvc5/cvc5/issues/8111
https://github.com/cvc5/cvc5/issues/8096#issuecomment-1043800815
https://github.com/cvc5/cvc5/issues/8119
https://github.com/cvc5/cvc5/issues/8119#issuecomment-1045646303
https://github.com/cvc5/cvc5/issues/8148#issuecomment-1049758826
https://github.com/cvc5/cvc5/issues/8162
https://github.com/cvc5/cvc5/issues/8163
https://github.com/cvc5/cvc5/issues/8158
https://github.com/cvc5/cvc5/issues/8183
https://github.com/cvc5/cvc5/issues/8348
https://github.com/cvc5/cvc5/issues/8377
https://github.com/cvc5/cvc5/issues/8410
https://github.com/cvc5/cvc5-projects/issues/487#issuecomment-1080093495
https://github.com/cvc5/cvc5/issues/8411
https://github.com/cvc5/cvc5/issues/8411#issuecomment-1080096720
https://github.com/cvc5/cvc5/issues/8347#issuecomment-1084610627
https://github.com/cvc5/cvc5/issues/8516
https://github.com/cvc5/cvc5/issues/8517
https://github.com/cvc5/cvc5/issues/8520
https://github.com/cvc5/cvc5/issues/8609
https://github.com/cvc5/cvc5/issues/8744
https://github.com/cvc5/cvc5/issues/8767
https://github.com/cvc5/cvc5/issues/8773
Files
Restricted
The record is publicly accessible, but files are restricted to users with access.