Supplemental Material

This Jupyter Notebook contains the supplemental material for the article "Issues with SZZ: An empirical assessment of the state of practice of defect prediction data collection" by Steffen Herbold, Alexander Trautsch, and Fabian Trautsch. The contents of this notebook enable the exact replication of the empirical study we conducted.

Additional Archives

Together with this notebook, we provide two archives, which contain the data we collected, and one archive which contains the source code of the tools we used for data collection.

MongoDB

The archive smartshark-mongodb-rel1.tar.gz contains all data we collected stored in a MongoDB. The MongoDB contains all data for just-in-time defect prediction, as well es the replication of the section 5.3-5.8 of the article. A running instance of the MongoDB is the requirement for nearly all parts of this notebook. To use the database dump, you have to:

• Install MongoDB 4.0
• Unpack the smartshark-mongodb-rel1.tar.gz archive (roughly 70 GB)
• Start the mongodb with the --dppath argument set to the pack where you unpacked the archive.

The MongoDB should then be available on localhost an accessible without any creditials. The documentation of the schema we use for storage can be found online.

Release-level Data

The archive release-level-data.tar.gz contains the release-level data we collected. For each release, the data contains three files:

• <projectname>-<version>_aggregated.csv: a CSV file that can directly be used for defect prediction studies.
• <projectname>-<version>.json: a JSON file that contains the unaggregated metric data for each file, i.e., the values for each lower-level artifact, e.g., for all methods.
• <projectname>-<version>_bug_fixes.json: a JSON file that contains for each file which issues where linked to the file, including the revision has and date of bug fixing commit, as well as the identifier, severity, and date of the creation of the linked issue.

The first column of the CSV file contains the name of the file, the next 4198 columns the features of the file, followed by the number of bugs in the file in the given release. After the number of bugs, there is one column for each bug that was assigned to the release. The names of these column follow the pattern <issueid>_<severity>_<fixdate> and have the value 1 if the file was affected by the issue and 0 otherwise.

The names of the features in the release level data follow patterns.

• SM: The static metrics measured with the OpenStaticAnalyzer, including all aggregations of metrics on the class, method, interface, enum, attribute, and annotation level proposed in the publication F. Zhang, A. E. Hassan, S. McIntosh, Y. Zou: "The use of summation to aggregate software metrics hinders the performanceof defect prediction models", TSE 43(5):476-491, 2017. The names of the metrics follow the pattern SM_<abstractionlevel>_<metric>[_<aggregationscheme>, e.g., SM_file_lloc for the metric Logical Lines of Code (LLOC) of the file or SM_method_cc_shannon_entropy for the metric Cyclomatic Complexity (CC) of all methods in a file aggregated using the shannon entropy. Overall, there are 3624 of these features, for the descripion of the metrics we refer to the OpenStaticAnalyzer documentation and for aggregation schemes to the the publication by Zhang et al.:
  • SM_annotation_ad_atkinson, SM_annotation_ad_avg, SM_annotation_ad_coefficient_of_variation, SM_annotation_ad_generalized_entropy, SM_annotation_ad_gini, SM_annotation_ad_hoover, SM_annotation_ad_max, SM_annotation_ad_median, SM_annotation_ad_min, SM_annotation_ad_shannon_entropy, SM_annotation_ad_stdev, SM_annotation_ad_sum, SM_annotation_ad_theil, SM_annotation_cbo_atkinson, SM_annotation_cbo_avg, SM_annotation_cbo_coefficient_of_variation, SM_annotation_cbo_generalized_entropy, SM_annotation_cbo_gini, SM_annotation_cbo_hoover, SM_annotation_cbo_max, SM_annotation_cbo_median, SM_annotation_cbo_min, SM_annotation_cbo_shannon_entropy, SM_annotation_cbo_stdev, SM_annotation_cbo_sum, SM_annotation_cbo_theil, SM_annotation_cboi_atkinson, SM_annotation_cboi_avg, SM_annotation_cboi_coefficient_of_variation, SM_annotation_cboi_generalized_entropy, SM_annotation_cboi_gini, SM_annotation_cboi_hoover, SM_annotation_cboi_max, SM_annotation_cboi_median, SM_annotation_cboi_min, SM_annotation_cboi_shannon_entropy, SM_annotation_cboi_stdev, SM_annotation_cboi_sum, SM_annotation_cboi_theil, SM_annotation_cc_atkinson, SM_annotation_cc_avg, SM_annotation_cc_coefficient_of_variation, SM_annotation_cc_generalized_entropy, SM_annotation_cc_gini, SM_annotation_cc_hoover, SM_annotation_cc_max, SM_annotation_cc_median, SM_annotation_cc_min, SM_annotation_cc_shannon_entropy, SM_annotation_cc_stdev, SM_annotation_cc_sum, SM_annotation_cc_theil, SM_annotation_ccl_atkinson, SM_annotation_ccl_avg, SM_annotation_ccl_coefficient_of_variation, SM_annotation_ccl_generalized_entropy, SM_annotation_ccl_gini, SM_annotation_ccl_hoover, SM_annotation_ccl_max, SM_annotation_ccl_median, SM_annotation_ccl_min, SM_annotation_ccl_shannon_entropy, SM_annotation_ccl_stdev, SM_annotation_ccl_sum, SM_annotation_ccl_theil, SM_annotation_cco_atkinson, SM_annotation_cco_avg, SM_annotation_cco_coefficient_of_variation, SM_annotation_cco_generalized_entropy, SM_annotation_cco_gini, SM_annotation_cco_hoover, SM_annotation_cco_max, SM_annotation_cco_median, SM_annotation_cco_min, SM_annotation_cco_shannon_entropy, SM_annotation_cco_stdev, SM_annotation_cco_sum, SM_annotation_cco_theil, SM_annotation_cd_atkinson, SM_annotation_cd_avg, SM_annotation_cd_coefficient_of_variation, SM_annotation_cd_generalized_entropy, SM_annotation_cd_gini, SM_annotation_cd_hoover, SM_annotation_cd_max, SM_annotation_cd_median, SM_annotation_cd_min, SM_annotation_cd_shannon_entropy, SM_annotation_cd_stdev, SM_annotation_cd_sum, SM_annotation_cd_theil, SM_annotation_ci_atkinson, SM_annotation_ci_avg, SM_annotation_ci_coefficient_of_variation, SM_annotation_ci_generalized_entropy, SM_annotation_ci_gini, SM_annotation_ci_hoover, SM_annotation_ci_max, SM_annotation_ci_median, SM_annotation_ci_min, SM_annotation_ci_shannon_entropy, SM_annotation_ci_stdev, SM_annotation_ci_sum, SM_annotation_ci_theil, SM_annotation_clc_atkinson, SM_annotation_clc_avg, SM_annotation_clc_coefficient_of_variation, SM_annotation_clc_generalized_entropy, SM_annotation_clc_gini, SM_annotation_clc_hoover, SM_annotation_clc_max, SM_annotation_clc_median, SM_annotation_clc_min, SM_annotation_clc_shannon_entropy, SM_annotation_clc_stdev, SM_annotation_clc_sum, SM_annotation_clc_theil, SM_annotation_cllc_atkinson, SM_annotation_cllc_avg, SM_annotation_cllc_coefficient_of_variation, SM_annotation_cllc_generalized_entropy, SM_annotation_cllc_gini, SM_annotation_cllc_hoover, SM_annotation_cllc_max, SM_annotation_cllc_median, SM_annotation_cllc_min, SM_annotation_cllc_shannon_entropy, SM_annotation_cllc_stdev, SM_annotation_cllc_sum, SM_annotation_cllc_theil, SM_annotation_cloc_atkinson, SM_annotation_cloc_avg, SM_annotation_cloc_coefficient_of_variation, SM_annotation_cloc_generalized_entropy, SM_annotation_cloc_gini, SM_annotation_cloc_hoover, SM_annotation_cloc_max, SM_annotation_cloc_median, SM_annotation_cloc_min, SM_annotation_cloc_shannon_entropy, SM_annotation_cloc_stdev, SM_annotation_cloc_sum, SM_annotation_cloc_theil, SM_annotation_dit_atkinson, SM_annotation_dit_avg, SM_annotation_dit_coefficient_of_variation, SM_annotation_dit_generalized_entropy, SM_annotation_dit_gini, SM_annotation_dit_hoover, SM_annotation_dit_max, SM_annotation_dit_median, SM_annotation_dit_min, SM_annotation_dit_shannon_entropy, SM_annotation_dit_stdev, SM_annotation_dit_sum, SM_annotation_dit_theil, SM_annotation_dloc_atkinson, SM_annotation_dloc_avg, SM_annotation_dloc_coefficient_of_variation, SM_annotation_dloc_generalized_entropy, SM_annotation_dloc_gini, SM_annotation_dloc_hoover, SM_annotation_dloc_max, SM_annotation_dloc_median, SM_annotation_dloc_min, SM_annotation_dloc_shannon_entropy, SM_annotation_dloc_stdev, SM_annotation_dloc_sum, SM_annotation_dloc_theil, SM_annotation_lcom5_atkinson, SM_annotation_lcom5_avg, SM_annotation_lcom5_coefficient_of_variation, SM_annotation_lcom5_generalized_entropy, SM_annotation_lcom5_gini, SM_annotation_lcom5_hoover, SM_annotation_lcom5_max, SM_annotation_lcom5_median, SM_annotation_lcom5_min, SM_annotation_lcom5_shannon_entropy, SM_annotation_lcom5_stdev, SM_annotation_lcom5_sum, SM_annotation_lcom5_theil, SM_annotation_ldc_atkinson, SM_annotation_ldc_avg, SM_annotation_ldc_coefficient_of_variation, SM_annotation_ldc_generalized_entropy, SM_annotation_ldc_gini, SM_annotation_ldc_hoover, SM_annotation_ldc_max, SM_annotation_ldc_median, SM_annotation_ldc_min, SM_annotation_ldc_shannon_entropy, SM_annotation_ldc_stdev, SM_annotation_ldc_sum, SM_annotation_ldc_theil, SM_annotation_lldc_atkinson, SM_annotation_lldc_avg, SM_annotation_lldc_coefficient_of_variation, SM_annotation_lldc_generalized_entropy, SM_annotation_lldc_gini, SM_annotation_lldc_hoover, SM_annotation_lldc_max, SM_annotation_lldc_median, SM_annotation_lldc_min, SM_annotation_lldc_shannon_entropy, SM_annotation_lldc_stdev, SM_annotation_lldc_sum, SM_annotation_lldc_theil, SM_annotation_lloc_atkinson, SM_annotation_lloc_avg, SM_annotation_lloc_coefficient_of_variation, SM_annotation_lloc_generalized_entropy, SM_annotation_lloc_gini, SM_annotation_lloc_hoover, SM_annotation_lloc_max, SM_annotation_lloc_median, SM_annotation_lloc_min, SM_annotation_lloc_shannon_entropy, SM_annotation_lloc_stdev, SM_annotation_lloc_sum, SM_annotation_lloc_theil, SM_annotation_loc_atkinson, SM_annotation_loc_avg, SM_annotation_loc_coefficient_of_variation, SM_annotation_loc_generalized_entropy, SM_annotation_loc_gini, SM_annotation_loc_hoover, SM_annotation_loc_max, SM_annotation_loc_median, SM_annotation_loc_min, SM_annotation_loc_shannon_entropy, SM_annotation_loc_stdev, SM_annotation_loc_sum, SM_annotation_loc_theil, SM_annotation_na_atkinson, SM_annotation_na_avg, SM_annotation_na_coefficient_of_variation, SM_annotation_na_generalized_entropy, SM_annotation_na_gini, SM_annotation_na_hoover, SM_annotation_na_max, SM_annotation_na_median, SM_annotation_na_min, SM_annotation_na_shannon_entropy, SM_annotation_na_stdev, SM_annotation_na_sum, SM_annotation_na_theil, SM_annotation_ng_atkinson, SM_annotation_ng_avg, SM_annotation_ng_coefficient_of_variation, SM_annotation_ng_generalized_entropy, SM_annotation_ng_gini, SM_annotation_ng_hoover, SM_annotation_ng_max, SM_annotation_ng_median, SM_annotation_ng_min, SM_annotation_ng_shannon_entropy, SM_annotation_ng_stdev, SM_annotation_ng_sum, SM_annotation_ng_theil, SM_annotation_nii_atkinson, SM_annotation_nii_avg, SM_annotation_nii_coefficient_of_variation, SM_annotation_nii_generalized_entropy, SM_annotation_nii_gini, SM_annotation_nii_hoover, SM_annotation_nii_max, SM_annotation_nii_median, SM_annotation_nii_min, SM_annotation_nii_shannon_entropy, SM_annotation_nii_stdev, SM_annotation_nii_sum, SM_annotation_nii_theil, SM_annotation_nl_atkinson, SM_annotation_nl_avg, SM_annotation_nl_coefficient_of_variation, SM_annotation_nl_generalized_entropy, SM_annotation_nl_gini, SM_annotation_nl_hoover, SM_annotation_nl_max, SM_annotation_nl_median, SM_annotation_nl_min, SM_annotation_nl_shannon_entropy, SM_annotation_nl_stdev, SM_annotation_nl_sum, SM_annotation_nl_theil, SM_annotation_nla_atkinson, SM_annotation_nla_avg, SM_annotation_nla_coefficient_of_variation, SM_annotation_nla_generalized_entropy, SM_annotation_nla_gini, SM_annotation_nla_hoover, SM_annotation_nla_max, SM_annotation_nla_median, SM_annotation_nla_min, SM_annotation_nla_shannon_entropy, SM_annotation_nla_stdev, SM_annotation_nla_sum, SM_annotation_nla_theil, SM_annotation_nle_atkinson, SM_annotation_nle_avg, SM_annotation_nle_coefficient_of_variation, SM_annotation_nle_generalized_entropy, SM_annotation_nle_gini, SM_annotation_nle_hoover, SM_annotation_nle_max, SM_annotation_nle_median, SM_annotation_nle_min, SM_annotation_nle_shannon_entropy, SM_annotation_nle_stdev, SM_annotation_nle_sum, SM_annotation_nle_theil, SM_annotation_nlg_atkinson, SM_annotation_nlg_avg, SM_annotation_nlg_coefficient_of_variation, SM_annotation_nlg_generalized_entropy, SM_annotation_nlg_gini, SM_annotation_nlg_hoover, SM_annotation_nlg_max, SM_annotation_nlg_median, SM_annotation_nlg_min, SM_annotation_nlg_shannon_entropy, SM_annotation_nlg_stdev, SM_annotation_nlg_sum, SM_annotation_nlg_theil, SM_annotation_nlm_atkinson, SM_annotation_nlm_avg, SM_annotation_nlm_coefficient_of_variation, SM_annotation_nlm_generalized_entropy, SM_annotation_nlm_gini, SM_annotation_nlm_hoover, SM_annotation_nlm_max, SM_annotation_nlm_median, SM_annotation_nlm_min, SM_annotation_nlm_shannon_entropy, SM_annotation_nlm_stdev, SM_annotation_nlm_sum, SM_annotation_nlm_theil, SM_annotation_nlpa_atkinson, SM_annotation_nlpa_avg, SM_annotation_nlpa_coefficient_of_variation, SM_annotation_nlpa_generalized_entropy, SM_annotation_nlpa_gini, SM_annotation_nlpa_hoover, SM_annotation_nlpa_max, SM_annotation_nlpa_median, SM_annotation_nlpa_min, SM_annotation_nlpa_shannon_entropy, SM_annotation_nlpa_stdev, SM_annotation_nlpa_sum, SM_annotation_nlpa_theil, SM_annotation_nlpm_atkinson, SM_annotation_nlpm_avg, SM_annotation_nlpm_coefficient_of_variation, SM_annotation_nlpm_generalized_entropy, SM_annotation_nlpm_gini, SM_annotation_nlpm_hoover, SM_annotation_nlpm_max, SM_annotation_nlpm_median, SM_annotation_nlpm_min, SM_annotation_nlpm_shannon_entropy, SM_annotation_nlpm_stdev, SM_annotation_nlpm_sum, SM_annotation_nlpm_theil, SM_annotation_nls_atkinson, SM_annotation_nls_avg, SM_annotation_nls_coefficient_of_variation, SM_annotation_nls_generalized_entropy, SM_annotation_nls_gini, SM_annotation_nls_hoover, SM_annotation_nls_max, SM_annotation_nls_median, SM_annotation_nls_min, SM_annotation_nls_shannon_entropy, SM_annotation_nls_stdev, SM_annotation_nls_sum, SM_annotation_nls_theil, SM_annotation_nm_atkinson, SM_annotation_nm_avg, SM_annotation_nm_coefficient_of_variation, SM_annotation_nm_generalized_entropy, SM_annotation_nm_gini, SM_annotation_nm_hoover, SM_annotation_nm_max, SM_annotation_nm_median, SM_annotation_nm_min, SM_annotation_nm_shannon_entropy, SM_annotation_nm_stdev, SM_annotation_nm_sum, SM_annotation_nm_theil, SM_annotation_noa_atkinson, SM_annotation_noa_avg, SM_annotation_noa_coefficient_of_variation, SM_annotation_noa_generalized_entropy, SM_annotation_noa_gini, SM_annotation_noa_hoover, SM_annotation_noa_max, SM_annotation_noa_median, SM_annotation_noa_min, SM_annotation_noa_shannon_entropy, SM_annotation_noa_stdev, SM_annotation_noa_sum, SM_annotation_noa_theil, SM_annotation_noc_atkinson, SM_annotation_noc_avg, SM_annotation_noc_coefficient_of_variation, SM_annotation_noc_generalized_entropy, SM_annotation_noc_gini, SM_annotation_noc_hoover, SM_annotation_noc_max, SM_annotation_noc_median, SM_annotation_noc_min, SM_annotation_noc_shannon_entropy, SM_annotation_noc_stdev, SM_annotation_noc_sum, SM_annotation_noc_theil, SM_annotation_nod_atkinson, SM_annotation_nod_avg, SM_annotation_nod_coefficient_of_variation, SM_annotation_nod_generalized_entropy, SM_annotation_nod_gini, SM_annotation_nod_hoover, SM_annotation_nod_max, SM_annotation_nod_median, SM_annotation_nod_min, SM_annotation_nod_shannon_entropy, SM_annotation_nod_stdev, SM_annotation_nod_sum, SM_annotation_nod_theil, SM_annotation_noi_atkinson, SM_annotation_noi_avg, SM_annotation_noi_coefficient_of_variation, SM_annotation_noi_generalized_entropy, SM_annotation_noi_gini, SM_annotation_noi_hoover, SM_annotation_noi_max, SM_annotation_noi_median, SM_annotation_noi_min, SM_annotation_noi_shannon_entropy, SM_annotation_noi_stdev, SM_annotation_noi_sum, SM_annotation_noi_theil, SM_annotation_nop_atkinson, SM_annotation_nop_avg, SM_annotation_nop_coefficient_of_variation, SM_annotation_nop_generalized_entropy, SM_annotation_nop_gini, SM_annotation_nop_hoover, SM_annotation_nop_max, SM_annotation_nop_median, SM_annotation_nop_min, SM_annotation_nop_shannon_entropy, SM_annotation_nop_stdev, SM_annotation_nop_sum, SM_annotation_nop_theil, SM_annotation_nos_atkinson, SM_annotation_nos_avg, SM_annotation_nos_coefficient_of_variation, SM_annotation_nos_generalized_entropy, SM_annotation_nos_gini, SM_annotation_nos_hoover, SM_annotation_nos_max, SM_annotation_nos_median, SM_annotation_nos_min, SM_annotation_nos_shannon_entropy, SM_annotation_nos_stdev, SM_annotation_nos_sum, SM_annotation_nos_theil, SM_annotation_npa_atkinson, SM_annotation_npa_avg, SM_annotation_npa_coefficient_of_variation, SM_annotation_npa_generalized_entropy, SM_annotation_npa_gini, SM_annotation_npa_hoover, SM_annotation_npa_max, SM_annotation_npa_median, SM_annotation_npa_min, SM_annotation_npa_shannon_entropy, SM_annotation_npa_stdev, SM_annotation_npa_sum, SM_annotation_npa_theil, SM_annotation_npm_atkinson, SM_annotation_npm_avg, SM_annotation_npm_coefficient_of_variation, SM_annotation_npm_generalized_entropy, SM_annotation_npm_gini, SM_annotation_npm_hoover, SM_annotation_npm_max, SM_annotation_npm_median, SM_annotation_npm_min, SM_annotation_npm_shannon_entropy, SM_annotation_npm_stdev, SM_annotation_npm_sum, SM_annotation_npm_theil, SM_annotation_ns_atkinson, SM_annotation_ns_avg, SM_annotation_ns_coefficient_of_variation, SM_annotation_ns_generalized_entropy, SM_annotation_ns_gini, SM_annotation_ns_hoover, SM_annotation_ns_max, SM_annotation_ns_median, SM_annotation_ns_min, SM_annotation_ns_shannon_entropy, SM_annotation_ns_stdev, SM_annotation_ns_sum, SM_annotation_ns_theil, SM_annotation_pda_atkinson, SM_annotation_pda_avg, SM_annotation_pda_coefficient_of_variation, SM_annotation_pda_generalized_entropy, SM_annotation_pda_gini, SM_annotation_pda_hoover, SM_annotation_pda_max, SM_annotation_pda_median, SM_annotation_pda_min, SM_annotation_pda_shannon_entropy, SM_annotation_pda_stdev, SM_annotation_pda_sum, SM_annotation_pda_theil, SM_annotation_pua_atkinson, SM_annotation_pua_avg, SM_annotation_pua_coefficient_of_variation, SM_annotation_pua_generalized_entropy, SM_annotation_pua_gini, SM_annotation_pua_hoover, SM_annotation_pua_max, SM_annotation_pua_median, SM_annotation_pua_min, SM_annotation_pua_shannon_entropy, SM_annotation_pua_stdev, SM_annotation_pua_sum, SM_annotation_pua_theil, SM_annotation_rfc_atkinson, SM_annotation_rfc_avg, SM_annotation_rfc_coefficient_of_variation, SM_annotation_rfc_generalized_entropy, SM_annotation_rfc_gini, SM_annotation_rfc_hoover, SM_annotation_rfc_max, SM_annotation_rfc_median, SM_annotation_rfc_min, SM_annotation_rfc_shannon_entropy, SM_annotation_rfc_stdev, SM_annotation_rfc_sum, SM_annotation_rfc_theil, SM_annotation_tcd_atkinson, SM_annotation_tcd_avg, SM_annotation_tcd_coefficient_of_variation, SM_annotation_tcd_generalized_entropy, SM_annotation_tcd_gini, SM_annotation_tcd_hoover, SM_annotation_tcd_max, SM_annotation_tcd_median, SM_annotation_tcd_min, SM_annotation_tcd_shannon_entropy, SM_annotation_tcd_stdev, SM_annotation_tcd_sum, SM_annotation_tcd_theil, SM_annotation_tcloc_atkinson, SM_annotation_tcloc_avg, SM_annotation_tcloc_coefficient_of_variation, SM_annotation_tcloc_generalized_entropy, SM_annotation_tcloc_gini, SM_annotation_tcloc_hoover, SM_annotation_tcloc_max, SM_annotation_tcloc_median, SM_annotation_tcloc_min, SM_annotation_tcloc_shannon_entropy, SM_annotation_tcloc_stdev, SM_annotation_tcloc_sum, SM_annotation_tcloc_theil, SM_annotation_tlloc_atkinson, SM_annotation_tlloc_avg, SM_annotation_tlloc_coefficient_of_variation, SM_annotation_tlloc_generalized_entropy, SM_annotation_tlloc_gini, SM_annotation_tlloc_hoover, SM_annotation_tlloc_max, SM_annotation_tlloc_median, SM_annotation_tlloc_min, SM_annotation_tlloc_shannon_entropy, SM_annotation_tlloc_stdev, SM_annotation_tlloc_sum, SM_annotation_tlloc_theil, SM_annotation_tloc_atkinson, SM_annotation_tloc_avg, SM_annotation_tloc_coefficient_of_variation, SM_annotation_tloc_generalized_entropy, SM_annotation_tloc_gini, SM_annotation_tloc_hoover, SM_annotation_tloc_max, SM_annotation_tloc_median, SM_annotation_tloc_min, SM_annotation_tloc_shannon_entropy, SM_annotation_tloc_stdev, SM_annotation_tloc_sum, SM_annotation_tloc_theil, SM_annotation_tna_atkinson, SM_annotation_tna_avg, SM_annotation_tna_coefficient_of_variation, SM_annotation_tna_generalized_entropy, SM_annotation_tna_gini, SM_annotation_tna_hoover, SM_annotation_tna_max, SM_annotation_tna_median, SM_annotation_tna_min, SM_annotation_tna_shannon_entropy, SM_annotation_tna_stdev, SM_annotation_tna_sum, SM_annotation_tna_theil, SM_annotation_tng_atkinson, SM_annotation_tng_avg, SM_annotation_tng_coefficient_of_variation, SM_annotation_tng_generalized_entropy, SM_annotation_tng_gini, SM_annotation_tng_hoover, SM_annotation_tng_max, SM_annotation_tng_median, SM_annotation_tng_min, SM_annotation_tng_shannon_entropy, SM_annotation_tng_stdev, SM_annotation_tng_sum, SM_annotation_tng_theil, SM_annotation_tnla_atkinson, SM_annotation_tnla_avg, SM_annotation_tnla_coefficient_of_variation, SM_annotation_tnla_generalized_entropy, SM_annotation_tnla_gini, SM_annotation_tnla_hoover, SM_annotation_tnla_max, SM_annotation_tnla_median, SM_annotation_tnla_min, SM_annotation_tnla_shannon_entropy, SM_annotation_tnla_stdev, SM_annotation_tnla_sum, SM_annotation_tnla_theil, SM_annotation_tnlg_atkinson, SM_annotation_tnlg_avg, SM_annotation_tnlg_coefficient_of_variation, SM_annotation_tnlg_generalized_entropy, SM_annotation_tnlg_gini, SM_annotation_tnlg_hoover, SM_annotation_tnlg_max, SM_annotation_tnlg_median, SM_annotation_tnlg_min, SM_annotation_tnlg_shannon_entropy, SM_annotation_tnlg_stdev, SM_annotation_tnlg_sum, SM_annotation_tnlg_theil, SM_annotation_tnlm_atkinson, SM_annotation_tnlm_avg, SM_annotation_tnlm_coefficient_of_variation, SM_annotation_tnlm_generalized_entropy, SM_annotation_tnlm_gini, SM_annotation_tnlm_hoover, SM_annotation_tnlm_max, SM_annotation_tnlm_median, SM_annotation_tnlm_min, SM_annotation_tnlm_shannon_entropy, SM_annotation_tnlm_stdev, SM_annotation_tnlm_sum, SM_annotation_tnlm_theil, SM_annotation_tnlpa_atkinson, SM_annotation_tnlpa_avg, SM_annotation_tnlpa_coefficient_of_variation, SM_annotation_tnlpa_generalized_entropy, SM_annotation_tnlpa_gini, SM_annotation_tnlpa_hoover, SM_annotation_tnlpa_max, SM_annotation_tnlpa_median, SM_annotation_tnlpa_min, SM_annotation_tnlpa_shannon_entropy, SM_annotation_tnlpa_stdev, SM_annotation_tnlpa_sum, SM_annotation_tnlpa_theil, SM_annotation_tnlpm_atkinson, SM_annotation_tnlpm_avg, SM_annotation_tnlpm_coefficient_of_variation, SM_annotation_tnlpm_generalized_entropy, SM_annotation_tnlpm_gini, SM_annotation_tnlpm_hoover, SM_annotation_tnlpm_max, SM_annotation_tnlpm_median, SM_annotation_tnlpm_min, SM_annotation_tnlpm_shannon_entropy, SM_annotation_tnlpm_stdev, SM_annotation_tnlpm_sum, SM_annotation_tnlpm_theil, SM_annotation_tnls_atkinson, SM_annotation_tnls_avg, SM_annotation_tnls_coefficient_of_variation, SM_annotation_tnls_generalized_entropy, SM_annotation_tnls_gini, SM_annotation_tnls_hoover, SM_annotation_tnls_max, SM_annotation_tnls_median, SM_annotation_tnls_min, SM_annotation_tnls_shannon_entropy, SM_annotation_tnls_stdev, SM_annotation_tnls_sum, SM_annotation_tnls_theil, SM_annotation_tnm_atkinson, SM_annotation_tnm_avg, SM_annotation_tnm_coefficient_of_variation, SM_annotation_tnm_generalized_entropy, SM_annotation_tnm_gini, SM_annotation_tnm_hoover, SM_annotation_tnm_max, SM_annotation_tnm_median, SM_annotation_tnm_min, SM_annotation_tnm_shannon_entropy, SM_annotation_tnm_stdev, SM_annotation_tnm_sum, SM_annotation_tnm_theil, SM_annotation_tnos_atkinson, SM_annotation_tnos_avg, SM_annotation_tnos_coefficient_of_variation, SM_annotation_tnos_generalized_entropy, SM_annotation_tnos_gini, SM_annotation_tnos_hoover, SM_annotation_tnos_max, SM_annotation_tnos_median, SM_annotation_tnos_min, SM_annotation_tnos_shannon_entropy, SM_annotation_tnos_stdev, SM_annotation_tnos_sum, SM_annotation_tnos_theil, SM_annotation_tnpa_atkinson, SM_annotation_tnpa_avg, SM_annotation_tnpa_coefficient_of_variation, SM_annotation_tnpa_generalized_entropy, SM_annotation_tnpa_gini, SM_annotation_tnpa_hoover, SM_annotation_tnpa_max, SM_annotation_tnpa_median, SM_annotation_tnpa_min, SM_annotation_tnpa_shannon_entropy, SM_annotation_tnpa_stdev, SM_annotation_tnpa_sum, SM_annotation_tnpa_theil, SM_annotation_tnpm_atkinson, SM_annotation_tnpm_avg, SM_annotation_tnpm_coefficient_of_variation, SM_annotation_tnpm_generalized_entropy, SM_annotation_tnpm_gini, SM_annotation_tnpm_hoover, SM_annotation_tnpm_max, SM_annotation_tnpm_median, SM_annotation_tnpm_min, SM_annotation_tnpm_shannon_entropy, SM_annotation_tnpm_stdev, SM_annotation_tnpm_sum, SM_annotation_tnpm_theil, SM_annotation_tns_atkinson, SM_annotation_tns_avg, SM_annotation_tns_coefficient_of_variation, SM_annotation_tns_generalized_entropy, SM_annotation_tns_gini, SM_annotation_tns_hoover, SM_annotation_tns_max, SM_annotation_tns_median, SM_annotation_tns_min, SM_annotation_tns_shannon_entropy, SM_annotation_tns_stdev, SM_annotation_tns_sum, SM_annotation_tns_theil, SM_annotation_wmc_atkinson, SM_annotation_wmc_avg, SM_annotation_wmc_coefficient_of_variation, SM_annotation_wmc_generalized_entropy, SM_annotation_wmc_gini, SM_annotation_wmc_hoover, SM_annotation_wmc_max, SM_annotation_wmc_median, SM_annotation_wmc_min, SM_annotation_wmc_shannon_entropy, SM_annotation_wmc_stdev, SM_annotation_wmc_sum, SM_annotation_wmc_theil, SM_class_ad_atkinson, SM_class_ad_avg, SM_class_ad_coefficient_of_variation, SM_class_ad_generalized_entropy, SM_class_ad_gini, SM_class_ad_hoover, SM_class_ad_max, SM_class_ad_median, SM_class_ad_min, SM_class_ad_shannon_entropy, SM_class_ad_stdev, SM_class_ad_sum, SM_class_ad_theil, SM_class_cbo_atkinson, SM_class_cbo_avg, SM_class_cbo_coefficient_of_variation, SM_class_cbo_generalized_entropy, SM_class_cbo_gini, SM_class_cbo_hoover, SM_class_cbo_max, SM_class_cbo_median, SM_class_cbo_min, SM_class_cbo_shannon_entropy, SM_class_cbo_stdev, SM_class_cbo_sum, SM_class_cbo_theil, SM_class_cboi_atkinson, SM_class_cboi_avg, SM_class_cboi_coefficient_of_variation, SM_class_cboi_generalized_entropy, SM_class_cboi_gini, SM_class_cboi_hoover, SM_class_cboi_max, SM_class_cboi_median, SM_class_cboi_min, SM_class_cboi_shannon_entropy, SM_class_cboi_stdev, SM_class_cboi_sum, SM_class_cboi_theil, SM_class_cc_atkinson, SM_class_cc_avg, SM_class_cc_coefficient_of_variation, SM_class_cc_generalized_entropy, SM_class_cc_gini, SM_class_cc_hoover, SM_class_cc_max, SM_class_cc_median, SM_class_cc_min, SM_class_cc_shannon_entropy, SM_class_cc_stdev, SM_class_cc_sum, SM_class_cc_theil, SM_class_ccl_atkinson, SM_class_ccl_avg, SM_class_ccl_coefficient_of_variation, SM_class_ccl_generalized_entropy, SM_class_ccl_gini, SM_class_ccl_hoover, SM_class_ccl_max, SM_class_ccl_median, SM_class_ccl_min, SM_class_ccl_shannon_entropy, SM_class_ccl_stdev, SM_class_ccl_sum, SM_class_ccl_theil, SM_class_cco_atkinson, SM_class_cco_avg, SM_class_cco_coefficient_of_variation, SM_class_cco_generalized_entropy, SM_class_cco_gini, SM_class_cco_hoover, SM_class_cco_max, SM_class_cco_median, SM_class_cco_min, SM_class_cco_shannon_entropy, SM_class_cco_stdev, SM_class_cco_sum, SM_class_cco_theil, SM_class_cd_atkinson, SM_class_cd_avg, SM_class_cd_coefficient_of_variation, SM_class_cd_generalized_entropy, SM_class_cd_gini, SM_class_cd_hoover, SM_class_cd_max, SM_class_cd_median, SM_class_cd_min, SM_class_cd_shannon_entropy, SM_class_cd_stdev, SM_class_cd_sum, SM_class_cd_theil, SM_class_ci_atkinson, SM_class_ci_avg, SM_class_ci_coefficient_of_variation, SM_class_ci_generalized_entropy, SM_class_ci_gini, SM_class_ci_hoover, SM_class_ci_max, SM_class_ci_median, SM_class_ci_min, SM_class_ci_shannon_entropy, SM_class_ci_stdev, SM_class_ci_sum, SM_class_ci_theil, SM_class_clc_atkinson, SM_class_clc_avg, SM_class_clc_coefficient_of_variation, SM_class_clc_generalized_entropy, SM_class_clc_gini, SM_class_clc_hoover, SM_class_clc_max, SM_class_clc_median, SM_class_clc_min, SM_class_clc_shannon_entropy, SM_class_clc_stdev, SM_class_clc_sum, SM_class_clc_theil, SM_class_cllc_atkinson, SM_class_cllc_avg, SM_class_cllc_coefficient_of_variation, SM_class_cllc_generalized_entropy, SM_class_cllc_gini, SM_class_cllc_hoover, SM_class_cllc_max, SM_class_cllc_median, SM_class_cllc_min, SM_class_cllc_shannon_entropy, SM_class_cllc_stdev, SM_class_cllc_sum, SM_class_cllc_theil, SM_class_cloc_atkinson, SM_class_cloc_avg, SM_class_cloc_coefficient_of_variation, SM_class_cloc_generalized_entropy, SM_class_cloc_gini, SM_class_cloc_hoover, SM_class_cloc_max, SM_class_cloc_median, SM_class_cloc_min, SM_class_cloc_shannon_entropy, SM_class_cloc_stdev, SM_class_cloc_sum, SM_class_cloc_theil, SM_class_dit_atkinson, SM_class_dit_avg, SM_class_dit_coefficient_of_variation, SM_class_dit_generalized_entropy, SM_class_dit_gini, SM_class_dit_hoover, SM_class_dit_max, SM_class_dit_median, SM_class_dit_min, SM_class_dit_shannon_entropy, SM_class_dit_stdev, SM_class_dit_sum, SM_class_dit_theil, SM_class_dloc_atkinson, SM_class_dloc_avg, SM_class_dloc_coefficient_of_variation, SM_class_dloc_generalized_entropy, SM_class_dloc_gini, SM_class_dloc_hoover, SM_class_dloc_max, SM_class_dloc_median, SM_class_dloc_min, SM_class_dloc_shannon_entropy, SM_class_dloc_stdev, SM_class_dloc_sum, SM_class_dloc_theil, SM_class_lcom5_atkinson, SM_class_lcom5_avg, SM_class_lcom5_coefficient_of_variation, SM_class_lcom5_generalized_entropy, SM_class_lcom5_gini, SM_class_lcom5_hoover, SM_class_lcom5_max, SM_class_lcom5_median, SM_class_lcom5_min, SM_class_lcom5_shannon_entropy, SM_class_lcom5_stdev, SM_class_lcom5_sum, SM_class_lcom5_theil, SM_class_ldc_atkinson, SM_class_ldc_avg, SM_class_ldc_coefficient_of_variation, SM_class_ldc_generalized_entropy, SM_class_ldc_gini, SM_class_ldc_hoover, SM_class_ldc_max, SM_class_ldc_median, SM_class_ldc_min, SM_class_ldc_shannon_entropy, SM_class_ldc_stdev, SM_class_ldc_sum, SM_class_ldc_theil, SM_class_lldc_atkinson, SM_class_lldc_avg, SM_class_lldc_coefficient_of_variation, SM_class_lldc_generalized_entropy, SM_class_lldc_gini, SM_class_lldc_hoover, SM_class_lldc_max, SM_class_lldc_median, SM_class_lldc_min, SM_class_lldc_shannon_entropy, SM_class_lldc_stdev, SM_class_lldc_sum, SM_class_lldc_theil, SM_class_lloc_atkinson, SM_class_lloc_avg, SM_class_lloc_coefficient_of_variation, SM_class_lloc_generalized_entropy, SM_class_lloc_gini, SM_class_lloc_hoover, SM_class_lloc_max, SM_class_lloc_median, SM_class_lloc_min, SM_class_lloc_shannon_entropy, SM_class_lloc_stdev, SM_class_lloc_sum, SM_class_lloc_theil, SM_class_loc_atkinson, SM_class_loc_avg, SM_class_loc_coefficient_of_variation, SM_class_loc_generalized_entropy, SM_class_loc_gini, SM_class_loc_hoover, SM_class_loc_max, SM_class_loc_median, SM_class_loc_min, SM_class_loc_shannon_entropy, SM_class_loc_stdev, SM_class_loc_sum, SM_class_loc_theil, SM_class_na_atkinson, SM_class_na_avg, SM_class_na_coefficient_of_variation, SM_class_na_generalized_entropy, SM_class_na_gini, SM_class_na_hoover, SM_class_na_max, SM_class_na_median, SM_class_na_min, SM_class_na_shannon_entropy, SM_class_na_stdev, SM_class_na_sum, SM_class_na_theil, SM_class_ng_atkinson, SM_class_ng_avg, SM_class_ng_coefficient_of_variation, SM_class_ng_generalized_entropy, SM_class_ng_gini, SM_class_ng_hoover, SM_class_ng_max, SM_class_ng_median, SM_class_ng_min, SM_class_ng_shannon_entropy, SM_class_ng_stdev, SM_class_ng_sum, SM_class_ng_theil, SM_class_nii_atkinson, SM_class_nii_avg, SM_class_nii_coefficient_of_variation, SM_class_nii_generalized_entropy, SM_class_nii_gini, SM_class_nii_hoover, SM_class_nii_max, SM_class_nii_median, SM_class_nii_min, SM_class_nii_shannon_entropy, SM_class_nii_stdev, SM_class_nii_sum, SM_class_nii_theil, SM_class_nl_atkinson, SM_class_nl_avg, SM_class_nl_coefficient_of_variation, SM_class_nl_generalized_entropy, SM_class_nl_gini, SM_class_nl_hoover, SM_class_nl_max, SM_class_nl_median, SM_class_nl_min, SM_class_nl_shannon_entropy, SM_class_nl_stdev, SM_class_nl_sum, SM_class_nl_theil, SM_class_nla_atkinson, SM_class_nla_avg, SM_class_nla_coefficient_of_variation, SM_class_nla_generalized_entropy, SM_class_nla_gini, SM_class_nla_hoover, SM_class_nla_max, SM_class_nla_median, SM_class_nla_min, SM_class_nla_shannon_entropy, SM_class_nla_stdev, SM_class_nla_sum, SM_class_nla_theil, SM_class_nle_atkinson, SM_class_nle_avg, SM_class_nle_coefficient_of_variation, SM_class_nle_generalized_entropy, SM_class_nle_gini, SM_class_nle_hoover, SM_class_nle_max, SM_class_nle_median, SM_class_nle_min, SM_class_nle_shannon_entropy, SM_class_nle_stdev, SM_class_nle_sum, SM_class_nle_theil, SM_class_nlg_atkinson, SM_class_nlg_avg, SM_class_nlg_coefficient_of_variation, SM_class_nlg_generalized_entropy, SM_class_nlg_gini, SM_class_nlg_hoover, SM_class_nlg_max, SM_class_nlg_median, SM_class_nlg_min, SM_class_nlg_shannon_entropy, SM_class_nlg_stdev, SM_class_nlg_sum, SM_class_nlg_theil, SM_class_nlm_atkinson, SM_class_nlm_avg, SM_class_nlm_coefficient_of_variation, SM_class_nlm_generalized_entropy, SM_class_nlm_gini, SM_class_nlm_hoover, SM_class_nlm_max, SM_class_nlm_median, SM_class_nlm_min, SM_class_nlm_shannon_entropy, SM_class_nlm_stdev, SM_class_nlm_sum, SM_class_nlm_theil, SM_class_nlpa_atkinson, SM_class_nlpa_avg, SM_class_nlpa_coefficient_of_variation, SM_class_nlpa_generalized_entropy, SM_class_nlpa_gini, SM_class_nlpa_hoover, SM_class_nlpa_max, SM_class_nlpa_median, SM_class_nlpa_min, SM_class_nlpa_shannon_entropy, SM_class_nlpa_stdev, SM_class_nlpa_sum, SM_class_nlpa_theil, SM_class_nlpm_atkinson, SM_class_nlpm_avg, SM_class_nlpm_coefficient_of_variation, SM_class_nlpm_generalized_entropy, SM_class_nlpm_gini, SM_class_nlpm_hoover, SM_class_nlpm_max, SM_class_nlpm_median, SM_class_nlpm_min, SM_class_nlpm_shannon_entropy, SM_class_nlpm_stdev, SM_class_nlpm_sum, SM_class_nlpm_theil, SM_class_nls_atkinson, SM_class_nls_avg, SM_class_nls_coefficient_of_variation, SM_class_nls_generalized_entropy, SM_class_nls_gini, SM_class_nls_hoover, SM_class_nls_max, SM_class_nls_median, SM_class_nls_min, SM_class_nls_shannon_entropy, SM_class_nls_stdev, SM_class_nls_sum, SM_class_nls_theil, SM_class_nm_atkinson, SM_class_nm_avg, SM_class_nm_coefficient_of_variation, SM_class_nm_generalized_entropy, SM_class_nm_gini, SM_class_nm_hoover, SM_class_nm_max, SM_class_nm_median, SM_class_nm_min, SM_class_nm_shannon_entropy, SM_class_nm_stdev, SM_class_nm_sum, SM_class_nm_theil, SM_class_noa_atkinson, SM_class_noa_avg, SM_class_noa_coefficient_of_variation, SM_class_noa_generalized_entropy, SM_class_noa_gini, SM_class_noa_hoover, SM_class_noa_max, SM_class_noa_median, SM_class_noa_min, SM_class_noa_shannon_entropy, SM_class_noa_stdev, SM_class_noa_sum, SM_class_noa_theil, SM_class_noc_atkinson, SM_class_noc_avg, SM_class_noc_coefficient_of_variation, SM_class_noc_generalized_entropy, SM_class_noc_gini, SM_class_noc_hoover, SM_class_noc_max, SM_class_noc_median, SM_class_noc_min, SM_class_noc_shannon_entropy, SM_class_noc_stdev, SM_class_noc_sum, SM_class_noc_theil, SM_class_nod_atkinson, SM_class_nod_avg, SM_class_nod_coefficient_of_variation, SM_class_nod_generalized_entropy, SM_class_nod_gini, SM_class_nod_hoover, SM_class_nod_max, SM_class_nod_median, SM_class_nod_min, SM_class_nod_shannon_entropy, SM_class_nod_stdev, SM_class_nod_sum, SM_class_nod_theil, SM_class_noi_atkinson, SM_class_noi_avg, SM_class_noi_coefficient_of_variation, SM_class_noi_generalized_entropy, SM_class_noi_gini, SM_class_noi_hoover, SM_class_noi_max, SM_class_noi_median, SM_class_noi_min, SM_class_noi_shannon_entropy, SM_class_noi_stdev, SM_class_noi_sum, SM_class_noi_theil, SM_class_nop_atkinson, SM_class_nop_avg, SM_class_nop_coefficient_of_variation, SM_class_nop_generalized_entropy, SM_class_nop_gini, SM_class_nop_hoover, SM_class_nop_max, SM_class_nop_median, SM_class_nop_min, SM_class_nop_shannon_entropy, SM_class_nop_stdev, SM_class_nop_sum, SM_class_nop_theil, SM_class_nos_atkinson, SM_class_nos_avg, SM_class_nos_coefficient_of_variation, SM_class_nos_generalized_entropy, SM_class_nos_gini, SM_class_nos_hoover, SM_class_nos_max, SM_class_nos_median, SM_class_nos_min, SM_class_nos_shannon_entropy, SM_class_nos_stdev, SM_class_nos_sum, SM_class_nos_theil, SM_class_npa_atkinson, SM_class_npa_avg, SM_class_npa_coefficient_of_variation, SM_class_npa_generalized_entropy, SM_class_npa_gini, SM_class_npa_hoover, SM_class_npa_max, SM_class_npa_median, SM_class_npa_min, SM_class_npa_shannon_entropy, SM_class_npa_stdev, SM_class_npa_sum, SM_class_npa_theil, SM_class_npm_atkinson, SM_class_npm_avg, SM_class_npm_coefficient_of_variation, SM_class_npm_generalized_entropy, SM_class_npm_gini, SM_class_npm_hoover, SM_class_npm_max, SM_class_npm_median, SM_class_npm_min, SM_class_npm_shannon_entropy, SM_class_npm_stdev, SM_class_npm_sum, SM_class_npm_theil, SM_class_ns_atkinson, SM_class_ns_avg, SM_class_ns_coefficient_of_variation, SM_class_ns_generalized_entropy, SM_class_ns_gini, SM_class_ns_hoover, SM_class_ns_max, SM_class_ns_median, SM_class_ns_min, SM_class_ns_shannon_entropy, SM_class_ns_stdev, SM_class_ns_sum, SM_class_ns_theil, SM_class_pda_atkinson, SM_class_pda_avg, SM_class_pda_coefficient_of_variation, SM_class_pda_generalized_entropy, SM_class_pda_gini, SM_class_pda_hoover, SM_class_pda_max, SM_class_pda_median, SM_class_pda_min, SM_class_pda_shannon_entropy, SM_class_pda_stdev, SM_class_pda_sum, SM_class_pda_theil, SM_class_pua_atkinson, SM_class_pua_avg, SM_class_pua_coefficient_of_variation, SM_class_pua_generalized_entropy, SM_class_pua_gini, SM_class_pua_hoover, SM_class_pua_max, SM_class_pua_median, SM_class_pua_min, SM_class_pua_shannon_entropy, SM_class_pua_stdev, SM_class_pua_sum, SM_class_pua_theil, SM_class_rfc_atkinson, SM_class_rfc_avg, SM_class_rfc_coefficient_of_variation, SM_class_rfc_generalized_entropy, SM_class_rfc_gini, SM_class_rfc_hoover, SM_class_rfc_max, SM_class_rfc_median, SM_class_rfc_min, SM_class_rfc_shannon_entropy, SM_class_rfc_stdev, SM_class_rfc_sum, SM_class_rfc_theil, SM_class_tcd_atkinson, SM_class_tcd_avg, SM_class_tcd_coefficient_of_variation, SM_class_tcd_generalized_entropy, SM_class_tcd_gini, SM_class_tcd_hoover, SM_class_tcd_max, SM_class_tcd_median, SM_class_tcd_min, SM_class_tcd_shannon_entropy, SM_class_tcd_stdev, SM_class_tcd_sum, SM_class_tcd_theil, SM_class_tcloc_atkinson, SM_class_tcloc_avg, SM_class_tcloc_coefficient_of_variation, SM_class_tcloc_generalized_entropy, SM_class_tcloc_gini, SM_class_tcloc_hoover, SM_class_tcloc_max, SM_class_tcloc_median, SM_class_tcloc_min, SM_class_tcloc_shannon_entropy, SM_class_tcloc_stdev, SM_class_tcloc_sum, SM_class_tcloc_theil, SM_class_tlloc_atkinson, SM_class_tlloc_avg, SM_class_tlloc_coefficient_of_variation, SM_class_tlloc_generalized_entropy, SM_class_tlloc_gini, SM_class_tlloc_hoover, SM_class_tlloc_max, SM_class_tlloc_median, SM_class_tlloc_min, SM_class_tlloc_shannon_entropy, SM_class_tlloc_stdev, SM_class_tlloc_sum, SM_class_tlloc_theil, SM_class_tloc_atkinson, SM_class_tloc_avg, SM_class_tloc_coefficient_of_variation, SM_class_tloc_generalized_entropy, SM_class_tloc_gini, SM_class_tloc_hoover, SM_class_tloc_max, SM_class_tloc_median, SM_class_tloc_min, SM_class_tloc_shannon_entropy, SM_class_tloc_stdev, SM_class_tloc_sum, SM_class_tloc_theil, SM_class_tna_atkinson, SM_class_tna_avg, SM_class_tna_coefficient_of_variation, SM_class_tna_generalized_entropy, SM_class_tna_gini, SM_class_tna_hoover, SM_class_tna_max, SM_class_tna_median, SM_class_tna_min, SM_class_tna_shannon_entropy, SM_class_tna_stdev, SM_class_tna_sum, SM_class_tna_theil, SM_class_tng_atkinson, SM_class_tng_avg, SM_class_tng_coefficient_of_variation, SM_class_tng_generalized_entropy, SM_class_tng_gini, SM_class_tng_hoover, SM_class_tng_max, SM_class_tng_median, SM_class_tng_min, SM_class_tng_shannon_entropy, SM_class_tng_stdev, SM_class_tng_sum, SM_class_tng_theil, SM_class_tnla_atkinson, SM_class_tnla_avg, SM_class_tnla_coefficient_of_variation, SM_class_tnla_generalized_entropy, SM_class_tnla_gini, SM_class_tnla_hoover, SM_class_tnla_max, SM_class_tnla_median, SM_class_tnla_min, SM_class_tnla_shannon_entropy, SM_class_tnla_stdev, SM_class_tnla_sum, SM_class_tnla_theil, SM_class_tnlg_atkinson, SM_class_tnlg_avg, SM_class_tnlg_coefficient_of_variation, SM_class_tnlg_generalized_entropy, SM_class_tnlg_gini, SM_class_tnlg_hoover, SM_class_tnlg_max, SM_class_tnlg_median, SM_class_tnlg_min, SM_class_tnlg_shannon_entropy, SM_class_tnlg_stdev, SM_class_tnlg_sum, SM_class_tnlg_theil, SM_class_tnlm_atkinson, SM_class_tnlm_avg, SM_class_tnlm_coefficient_of_variation, SM_class_tnlm_generalized_entropy, SM_class_tnlm_gini, SM_class_tnlm_hoover, SM_class_tnlm_max, SM_class_tnlm_median, SM_class_tnlm_min, SM_class_tnlm_shannon_entropy, SM_class_tnlm_stdev, SM_class_tnlm_sum, SM_class_tnlm_theil, SM_class_tnlpa_atkinson, SM_class_tnlpa_avg, SM_class_tnlpa_coefficient_of_variation, SM_class_tnlpa_generalized_entropy, SM_class_tnlpa_gini, SM_class_tnlpa_hoover, SM_class_tnlpa_max, SM_class_tnlpa_median, SM_class_tnlpa_min, SM_class_tnlpa_shannon_entropy, SM_class_tnlpa_stdev, SM_class_tnlpa_sum, SM_class_tnlpa_theil, SM_class_tnlpm_atkinson, SM_class_tnlpm_avg, SM_class_tnlpm_coefficient_of_variation, SM_class_tnlpm_generalized_entropy, SM_class_tnlpm_gini, SM_class_tnlpm_hoover, SM_class_tnlpm_max, SM_class_tnlpm_median, SM_class_tnlpm_min, SM_class_tnlpm_shannon_entropy, SM_class_tnlpm_stdev, SM_class_tnlpm_sum, SM_class_tnlpm_theil, SM_class_tnls_atkinson, SM_class_tnls_avg, SM_class_tnls_coefficient_of_variation, SM_class_tnls_generalized_entropy, SM_class_tnls_gini, SM_class_tnls_hoover, SM_class_tnls_max, SM_class_tnls_median, SM_class_tnls_min, SM_class_tnls_shannon_entropy, SM_class_tnls_stdev, SM_class_tnls_sum, SM_class_tnls_theil, SM_class_tnm_atkinson, SM_class_tnm_avg, SM_class_tnm_coefficient_of_variation, SM_class_tnm_generalized_entropy, SM_class_tnm_gini, SM_class_tnm_hoover, SM_class_tnm_max, SM_class_tnm_median, SM_class_tnm_min, SM_class_tnm_shannon_entropy, SM_class_tnm_stdev, SM_class_tnm_sum, SM_class_tnm_theil, SM_class_tnos_atkinson, SM_class_tnos_avg, SM_class_tnos_coefficient_of_variation, SM_class_tnos_generalized_entropy, SM_class_tnos_gini, SM_class_tnos_hoover, SM_class_tnos_max, SM_class_tnos_median, SM_class_tnos_min, SM_class_tnos_shannon_entropy, SM_class_tnos_stdev, SM_class_tnos_sum, SM_class_tnos_theil, SM_class_tnpa_atkinson, SM_class_tnpa_avg, SM_class_tnpa_coefficient_of_variation, SM_class_tnpa_generalized_entropy, SM_class_tnpa_gini, SM_class_tnpa_hoover, SM_class_tnpa_max, SM_class_tnpa_median, SM_class_tnpa_min, SM_class_tnpa_shannon_entropy, SM_class_tnpa_stdev, SM_class_tnpa_sum, SM_class_tnpa_theil, SM_class_tnpm_atkinson, SM_class_tnpm_avg, SM_class_tnpm_coefficient_of_variation, SM_class_tnpm_generalized_entropy, SM_class_tnpm_gini, SM_class_tnpm_hoover, SM_class_tnpm_max, SM_class_tnpm_median, SM_class_tnpm_min, SM_class_tnpm_shannon_entropy, SM_class_tnpm_stdev, SM_class_tnpm_sum, SM_class_tnpm_theil, SM_class_tns_atkinson, SM_class_tns_avg, SM_class_tns_coefficient_of_variation, SM_class_tns_generalized_entropy, SM_class_tns_gini, SM_class_tns_hoover, SM_class_tns_max, SM_class_tns_median, SM_class_tns_min, SM_class_tns_shannon_entropy, SM_class_tns_stdev, SM_class_tns_sum, SM_class_tns_theil, SM_class_wmc_atkinson, SM_class_wmc_avg, SM_class_wmc_coefficient_of_variation, SM_class_wmc_generalized_entropy, SM_class_wmc_gini, SM_class_wmc_hoover, SM_class_wmc_max, SM_class_wmc_median, SM_class_wmc_min, SM_class_wmc_shannon_entropy, SM_class_wmc_stdev, SM_class_wmc_sum, SM_class_wmc_theil, SM_component_cc, SM_component_ccl, SM_component_cee, SM_component_ceg, SM_component_ci, SM_component_clc, SM_component_cllc, SM_component_cr, SM_component_ldc, SM_component_lldc, SM_component_ncr, SM_component_tad, SM_component_tcd, SM_component_tcloc, SM_component_tlloc, SM_component_tloc, SM_component_tna, SM_component_tncl, SM_component_tndi, SM_component_tnen, SM_component_tnfi, SM_component_tng, SM_component_tnin, SM_component_tnm, SM_component_tnos, SM_component_tnpa, SM_component_tnpcl, SM_component_tnpen, SM_component_tnpin, SM_component_tnpkg, SM_component_tnpm, SM_component_tns, SM_component_tpda, SM_component_tpua, SM_enum_ad_atkinson, SM_enum_ad_avg, SM_enum_ad_coefficient_of_variation, SM_enum_ad_generalized_entropy, SM_enum_ad_gini, SM_enum_ad_hoover, SM_enum_ad_max, SM_enum_ad_median, SM_enum_ad_min, SM_enum_ad_shannon_entropy, SM_enum_ad_stdev, SM_enum_ad_sum, SM_enum_ad_theil, SM_enum_cbo_atkinson, SM_enum_cbo_avg, SM_enum_cbo_coefficient_of_variation, SM_enum_cbo_generalized_entropy, SM_enum_cbo_gini, SM_enum_cbo_hoover, SM_enum_cbo_max, SM_enum_cbo_median, SM_enum_cbo_min, SM_enum_cbo_shannon_entropy, SM_enum_cbo_stdev, SM_enum_cbo_sum, SM_enum_cbo_theil, SM_enum_cboi_atkinson, SM_enum_cboi_avg, SM_enum_cboi_coefficient_of_variation, SM_enum_cboi_generalized_entropy, SM_enum_cboi_gini, SM_enum_cboi_hoover, SM_enum_cboi_max, SM_enum_cboi_median, SM_enum_cboi_min, SM_enum_cboi_shannon_entropy, SM_enum_cboi_stdev, SM_enum_cboi_sum, SM_enum_cboi_theil, SM_enum_cc_atkinson, SM_enum_cc_avg, SM_enum_cc_coefficient_of_variation, SM_enum_cc_generalized_entropy, SM_enum_cc_gini, SM_enum_cc_hoover, SM_enum_cc_max, SM_enum_cc_median, SM_enum_cc_min, SM_enum_cc_shannon_entropy, SM_enum_cc_stdev, SM_enum_cc_sum, SM_enum_cc_theil, SM_enum_ccl_atkinson, SM_enum_ccl_avg, SM_enum_ccl_coefficient_of_variation, SM_enum_ccl_generalized_entropy, SM_enum_ccl_gini, SM_enum_ccl_hoover, SM_enum_ccl_max, SM_enum_ccl_median, SM_enum_ccl_min, SM_enum_ccl_shannon_entropy, SM_enum_ccl_stdev, SM_enum_ccl_sum, SM_enum_ccl_theil, SM_enum_cco_atkinson, SM_enum_cco_avg, SM_enum_cco_coefficient_of_variation, SM_enum_cco_generalized_entropy, SM_enum_cco_gini, SM_enum_cco_hoover, SM_enum_cco_max, SM_enum_cco_median, SM_enum_cco_min, SM_enum_cco_shannon_entropy, SM_enum_cco_stdev, SM_enum_cco_sum, SM_enum_cco_theil, SM_enum_cd_atkinson, SM_enum_cd_avg, SM_enum_cd_coefficient_of_variation, SM_enum_cd_generalized_entropy, SM_enum_cd_gini, SM_enum_cd_hoover, SM_enum_cd_max, SM_enum_cd_median, SM_enum_cd_min, SM_enum_cd_shannon_entropy, SM_enum_cd_stdev, SM_enum_cd_sum, SM_enum_cd_theil, SM_enum_ci_atkinson, SM_enum_ci_avg, SM_enum_ci_coefficient_of_variation, SM_enum_ci_generalized_entropy, SM_enum_ci_gini, SM_enum_ci_hoover, SM_enum_ci_max, SM_enum_ci_median, SM_enum_ci_min, SM_enum_ci_shannon_entropy, SM_enum_ci_stdev, SM_enum_ci_sum, SM_enum_ci_theil, SM_enum_clc_atkinson, SM_enum_clc_avg, SM_enum_clc_coefficient_of_variation, SM_enum_clc_generalized_entropy, SM_enum_clc_gini, SM_enum_clc_hoover, SM_enum_clc_max, SM_enum_clc_median, SM_enum_clc_min, SM_enum_clc_shannon_entropy, SM_enum_clc_stdev, SM_enum_clc_sum, SM_enum_clc_theil, SM_enum_cllc_atkinson, SM_enum_cllc_avg, SM_enum_cllc_coefficient_of_variation, SM_enum_cllc_generalized_entropy, SM_enum_cllc_gini, SM_enum_cllc_hoover, SM_enum_cllc_max, SM_enum_cllc_median, SM_enum_cllc_min, SM_enum_cllc_shannon_entropy, SM_enum_cllc_stdev, SM_enum_cllc_sum, SM_enum_cllc_theil, SM_enum_cloc_atkinson, SM_enum_cloc_avg, SM_enum_cloc_coefficient_of_variation, SM_enum_cloc_generalized_entropy, SM_enum_cloc_gini, SM_enum_cloc_hoover, SM_enum_cloc_max, SM_enum_cloc_median, SM_enum_cloc_min, SM_enum_cloc_shannon_entropy, SM_enum_cloc_stdev, SM_enum_cloc_sum, SM_enum_cloc_theil, SM_enum_dit_atkinson, SM_enum_dit_avg, SM_enum_dit_coefficient_of_variation, SM_enum_dit_generalized_entropy, SM_enum_dit_gini, SM_enum_dit_hoover, SM_enum_dit_max, SM_enum_dit_median, SM_enum_dit_min, SM_enum_dit_shannon_entropy, SM_enum_dit_stdev, SM_enum_dit_sum, SM_enum_dit_theil, SM_enum_dloc_atkinson, SM_enum_dloc_avg, SM_enum_dloc_coefficient_of_variation, SM_enum_dloc_generalized_entropy, SM_enum_dloc_gini, SM_enum_dloc_hoover, SM_enum_dloc_max, SM_enum_dloc_median, SM_enum_dloc_min, SM_enum_dloc_shannon_entropy, SM_enum_dloc_stdev, SM_enum_dloc_sum, SM_enum_dloc_theil, SM_enum_lcom5_atkinson, SM_enum_lcom5_avg, SM_enum_lcom5_coefficient_of_variation, SM_enum_lcom5_generalized_entropy, SM_enum_lcom5_gini, SM_enum_lcom5_hoover, SM_enum_lcom5_max, SM_enum_lcom5_median, SM_enum_lcom5_min, SM_enum_lcom5_shannon_entropy, SM_enum_lcom5_stdev, SM_enum_lcom5_sum, SM_enum_lcom5_theil, SM_enum_ldc_atkinson, SM_enum_ldc_avg, SM_enum_ldc_coefficient_of_variation, SM_enum_ldc_generalized_entropy, SM_enum_ldc_gini, SM_enum_ldc_hoover, SM_enum_ldc_max, SM_enum_ldc_median, SM_enum_ldc_min, SM_enum_ldc_shannon_entropy, SM_enum_ldc_stdev, SM_enum_ldc_sum, SM_enum_ldc_theil, SM_enum_lldc_atkinson, SM_enum_lldc_avg, SM_enum_lldc_coefficient_of_variation, SM_enum_lldc_generalized_entropy, SM_enum_lldc_gini, SM_enum_lldc_hoover, SM_enum_lldc_max, SM_enum_lldc_median, SM_enum_lldc_min, SM_enum_lldc_shannon_entropy, SM_enum_lldc_stdev, SM_enum_lldc_sum, SM_enum_lldc_theil, SM_enum_lloc_atkinson, SM_enum_lloc_avg, SM_enum_lloc_coefficient_of_variation, SM_enum_lloc_generalized_entropy, SM_enum_lloc_gini, SM_enum_lloc_hoover, SM_enum_lloc_max, SM_enum_lloc_median, SM_enum_lloc_min, SM_enum_lloc_shannon_entropy, SM_enum_lloc_stdev, SM_enum_lloc_sum, SM_enum_lloc_theil, SM_enum_loc_atkinson, SM_enum_loc_avg, SM_enum_loc_coefficient_of_variation, SM_enum_loc_generalized_entropy, SM_enum_loc_gini, SM_enum_loc_hoover, SM_enum_loc_max, SM_enum_loc_median, SM_enum_loc_min, SM_enum_loc_shannon_entropy, SM_enum_loc_stdev, SM_enum_loc_sum, SM_enum_loc_theil, SM_enum_na_atkinson, SM_enum_na_avg, SM_enum_na_coefficient_of_variation, SM_enum_na_generalized_entropy, SM_enum_na_gini, SM_enum_na_hoover, SM_enum_na_max, SM_enum_na_median, SM_enum_na_min, SM_enum_na_shannon_entropy, SM_enum_na_stdev, SM_enum_na_sum, SM_enum_na_theil, SM_enum_ng_atkinson, SM_enum_ng_avg, SM_enum_ng_coefficient_of_variation, SM_enum_ng_generalized_entropy, SM_enum_ng_gini, SM_enum_ng_hoover, SM_enum_ng_max, SM_enum_ng_median, SM_enum_ng_min, SM_enum_ng_shannon_entropy, SM_enum_ng_stdev, SM_enum_ng_sum, SM_enum_ng_theil, SM_enum_nii_atkinson, SM_enum_nii_avg, SM_enum_nii_coefficient_of_variation, SM_enum_nii_generalized_entropy, SM_enum_nii_gini, SM_enum_nii_hoover, SM_enum_nii_max, SM_enum_nii_median, SM_enum_nii_min, SM_enum_nii_shannon_entropy, SM_enum_nii_stdev, SM_enum_nii_sum, SM_enum_nii_theil, SM_enum_nl_atkinson, SM_enum_nl_avg, SM_enum_nl_coefficient_of_variation, SM_enum_nl_generalized_entropy, SM_enum_nl_gini, SM_enum_nl_hoover, SM_enum_nl_max, SM_enum_nl_median, SM_enum_nl_min, SM_enum_nl_shannon_entropy, SM_enum_nl_stdev, SM_enum_nl_sum, SM_enum_nl_theil, SM_enum_nla_atkinson, SM_enum_nla_avg, SM_enum_nla_coefficient_of_variation, SM_enum_nla_generalized_entropy, SM_enum_nla_gini, SM_enum_nla_hoover, SM_enum_nla_max, SM_enum_nla_median, SM_enum_nla_min, SM_enum_nla_shannon_entropy, SM_enum_nla_stdev, SM_enum_nla_sum, SM_enum_nla_theil, SM_enum_nle_atkinson, SM_enum_nle_avg, SM_enum_nle_coefficient_of_variation, SM_enum_nle_generalized_entropy, SM_enum_nle_gini, SM_enum_nle_hoover, SM_enum_nle_max, SM_enum_nle_median, SM_enum_nle_min, SM_enum_nle_shannon_entropy, SM_enum_nle_stdev, SM_enum_nle_sum, SM_enum_nle_theil, SM_enum_nlg_atkinson, SM_enum_nlg_avg, SM_enum_nlg_coefficient_of_variation, SM_enum_nlg_generalized_entropy, SM_enum_nlg_gini, SM_enum_nlg_hoover, SM_enum_nlg_max, SM_enum_nlg_median, SM_enum_nlg_min, SM_enum_nlg_shannon_entropy, SM_enum_nlg_stdev, SM_enum_nlg_sum, SM_enum_nlg_theil, SM_enum_nlm_atkinson, SM_enum_nlm_avg, SM_enum_nlm_coefficient_of_variation, SM_enum_nlm_generalized_entropy, SM_enum_nlm_gini, SM_enum_nlm_hoover, SM_enum_nlm_max, SM_enum_nlm_median, SM_enum_nlm_min, SM_enum_nlm_shannon_entropy, SM_enum_nlm_stdev, SM_enum_nlm_sum, SM_enum_nlm_theil, SM_enum_nlpa_atkinson, SM_enum_nlpa_avg, SM_enum_nlpa_coefficient_of_variation, SM_enum_nlpa_generalized_entropy, SM_enum_nlpa_gini, SM_enum_nlpa_hoover, SM_enum_nlpa_max, SM_enum_nlpa_median, SM_enum_nlpa_min, SM_enum_nlpa_shannon_entropy, SM_enum_nlpa_stdev, SM_enum_nlpa_sum, SM_enum_nlpa_theil, SM_enum_nlpm_atkinson, SM_enum_nlpm_avg, SM_enum_nlpm_coefficient_of_variation, SM_enum_nlpm_generalized_entropy, SM_enum_nlpm_gini, SM_enum_nlpm_hoover, SM_enum_nlpm_max, SM_enum_nlpm_median, SM_enum_nlpm_min, SM_enum_nlpm_shannon_entropy, SM_enum_nlpm_stdev, SM_enum_nlpm_sum, SM_enum_nlpm_theil, SM_enum_nls_atkinson, SM_enum_nls_avg, SM_enum_nls_coefficient_of_variation, SM_enum_nls_generalized_entropy, SM_enum_nls_gini, SM_enum_nls_hoover, SM_enum_nls_max, SM_enum_nls_median, SM_enum_nls_min, SM_enum_nls_shannon_entropy, SM_enum_nls_stdev, SM_enum_nls_sum, SM_enum_nls_theil, SM_enum_nm_atkinson, SM_enum_nm_avg, SM_enum_nm_coefficient_of_variation, SM_enum_nm_generalized_entropy, SM_enum_nm_gini, SM_enum_nm_hoover, SM_enum_nm_max, SM_enum_nm_median, SM_enum_nm_min, SM_enum_nm_shannon_entropy, SM_enum_nm_stdev, SM_enum_nm_sum, SM_enum_nm_theil, SM_enum_noa_atkinson, SM_enum_noa_avg, SM_enum_noa_coefficient_of_variation, SM_enum_noa_generalized_entropy, SM_enum_noa_gini, SM_enum_noa_hoover, SM_enum_noa_max, SM_enum_noa_median, SM_enum_noa_min, SM_enum_noa_shannon_entropy, SM_enum_noa_stdev, SM_enum_noa_sum, SM_enum_noa_theil, SM_enum_noc_atkinson, SM_enum_noc_avg, SM_enum_noc_coefficient_of_variation, SM_enum_noc_generalized_entropy, SM_enum_noc_gini, SM_enum_noc_hoover, SM_enum_noc_max, SM_enum_noc_median, SM_enum_noc_min, SM_enum_noc_shannon_entropy, SM_enum_noc_stdev, SM_enum_noc_sum, SM_enum_noc_theil, SM_enum_nod_atkinson, SM_enum_nod_avg, SM_enum_nod_coefficient_of_variation, SM_enum_nod_generalized_entropy, SM_enum_nod_gini, SM_enum_nod_hoover, SM_enum_nod_max, SM_enum_nod_median, SM_enum_nod_min, SM_enum_nod_shannon_entropy, SM_enum_nod_stdev, SM_enum_nod_sum, SM_enum_nod_theil, SM_enum_noi_atkinson, SM_enum_noi_avg, SM_enum_noi_coefficient_of_variation, SM_enum_noi_generalized_entropy, SM_enum_noi_gini, SM_enum_noi_hoover, SM_enum_noi_max, SM_enum_noi_median, SM_enum_noi_min, SM_enum_noi_shannon_entropy, SM_enum_noi_stdev, SM_enum_noi_sum, SM_enum_noi_theil, SM_enum_nop_atkinson, SM_enum_nop_avg, SM_enum_nop_coefficient_of_variation, SM_enum_nop_generalized_entropy, SM_enum_nop_gini, SM_enum_nop_hoover, SM_enum_nop_max, SM_enum_nop_median, SM_enum_nop_min, SM_enum_nop_shannon_entropy, SM_enum_nop_stdev, SM_enum_nop_sum, SM_enum_nop_theil, SM_enum_nos_atkinson, SM_enum_nos_avg, SM_enum_nos_coefficient_of_variation, SM_enum_nos_generalized_entropy, SM_enum_nos_gini, SM_enum_nos_hoover, SM_enum_nos_max, SM_enum_nos_median, SM_enum_nos_min, SM_enum_nos_shannon_entropy, SM_enum_nos_stdev, SM_enum_nos_sum, SM_enum_nos_theil, SM_enum_npa_atkinson, SM_enum_npa_avg, SM_enum_npa_coefficient_of_variation, SM_enum_npa_generalized_entropy, SM_enum_npa_gini, SM_enum_npa_hoover, SM_enum_npa_max, SM_enum_npa_median, SM_enum_npa_min, SM_enum_npa_shannon_entropy, SM_enum_npa_stdev, SM_enum_npa_sum, SM_enum_npa_theil, SM_enum_npm_atkinson, SM_enum_npm_avg, SM_enum_npm_coefficient_of_variation, SM_enum_npm_generalized_entropy, SM_enum_npm_gini, SM_enum_npm_hoover, SM_enum_npm_max, SM_enum_npm_median, SM_enum_npm_min, SM_enum_npm_shannon_entropy, SM_enum_npm_stdev, SM_enum_npm_sum, SM_enum_npm_theil, SM_enum_ns_atkinson, SM_enum_ns_avg, SM_enum_ns_coefficient_of_variation, SM_enum_ns_generalized_entropy, SM_enum_ns_gini, SM_enum_ns_hoover, SM_enum_ns_max, SM_enum_ns_median, SM_enum_ns_min, SM_enum_ns_shannon_entropy, SM_enum_ns_stdev, SM_enum_ns_sum, SM_enum_ns_theil, SM_enum_pda_atkinson, SM_enum_pda_avg, SM_enum_pda_coefficient_of_variation, SM_enum_pda_generalized_entropy, SM_enum_pda_gini, SM_enum_pda_hoover, SM_enum_pda_max, SM_enum_pda_median, SM_enum_pda_min, SM_enum_pda_shannon_entropy, SM_enum_pda_stdev, SM_enum_pda_sum, SM_enum_pda_theil, SM_enum_pua_atkinson, SM_enum_pua_avg, SM_enum_pua_coefficient_of_variation, SM_enum_pua_generalized_entropy, SM_enum_pua_gini, SM_enum_pua_hoover, SM_enum_pua_max, SM_enum_pua_median, SM_enum_pua_min, SM_enum_pua_shannon_entropy, SM_enum_pua_stdev, SM_enum_pua_sum, SM_enum_pua_theil, SM_enum_rfc_atkinson, SM_enum_rfc_avg, SM_enum_rfc_coefficient_of_variation, SM_enum_rfc_generalized_entropy, SM_enum_rfc_gini, SM_enum_rfc_hoover, SM_enum_rfc_max, SM_enum_rfc_median, SM_enum_rfc_min, SM_enum_rfc_shannon_entropy, SM_enum_rfc_stdev, SM_enum_rfc_sum, SM_enum_rfc_theil, SM_enum_tcd_atkinson, SM_enum_tcd_avg, SM_enum_tcd_coefficient_of_variation, SM_enum_tcd_generalized_entropy, SM_enum_tcd_gini, SM_enum_tcd_hoover, SM_enum_tcd_max, SM_enum_tcd_median, SM_enum_tcd_min, SM_enum_tcd_shannon_entropy, SM_enum_tcd_stdev, SM_enum_tcd_sum, SM_enum_tcd_theil, SM_enum_tcloc_atkinson, SM_enum_tcloc_avg, SM_enum_tcloc_coefficient_of_variation, SM_enum_tcloc_generalized_entropy, SM_enum_tcloc_gini, SM_enum_tcloc_hoover, SM_enum_tcloc_max, SM_enum_tcloc_median, SM_enum_tcloc_min, SM_enum_tcloc_shannon_entropy, SM_enum_tcloc_stdev, SM_enum_tcloc_sum, SM_enum_tcloc_theil, SM_enum_tlloc_atkinson, SM_enum_tlloc_avg, SM_enum_tlloc_coefficient_of_variation, SM_enum_tlloc_generalized_entropy, SM_enum_tlloc_gini, SM_enum_tlloc_hoover, SM_enum_tlloc_max, SM_enum_tlloc_median, SM_enum_tlloc_min, SM_enum_tlloc_shannon_entropy, SM_enum_tlloc_stdev, SM_enum_tlloc_sum, SM_enum_tlloc_theil, SM_enum_tloc_atkinson, SM_enum_tloc_avg, SM_enum_tloc_coefficient_of_variation, SM_enum_tloc_generalized_entropy, SM_enum_tloc_gini, SM_enum_tloc_hoover, SM_enum_tloc_max, SM_enum_tloc_median, SM_enum_tloc_min, SM_enum_tloc_shannon_entropy, SM_enum_tloc_stdev, SM_enum_tloc_sum, SM_enum_tloc_theil, SM_enum_tna_atkinson, SM_enum_tna_avg, SM_enum_tna_coefficient_of_variation, SM_enum_tna_generalized_entropy, SM_enum_tna_gini, SM_enum_tna_hoover, SM_enum_tna_max, SM_enum_tna_median, SM_enum_tna_min, SM_enum_tna_shannon_entropy, SM_enum_tna_stdev, SM_enum_tna_sum, SM_enum_tna_theil, SM_enum_tng_atkinson, SM_enum_tng_avg, SM_enum_tng_coefficient_of_variation, SM_enum_tng_generalized_entropy, SM_enum_tng_gini, SM_enum_tng_hoover, SM_enum_tng_max, SM_enum_tng_median, SM_enum_tng_min, SM_enum_tng_shannon_entropy, SM_enum_tng_stdev, SM_enum_tng_sum, SM_enum_tng_theil, SM_enum_tnla_atkinson, SM_enum_tnla_avg, SM_enum_tnla_coefficient_of_variation, SM_enum_tnla_generalized_entropy, SM_enum_tnla_gini, SM_enum_tnla_hoover, SM_enum_tnla_max, SM_enum_tnla_median, SM_enum_tnla_min, SM_enum_tnla_shannon_entropy, SM_enum_tnla_stdev, SM_enum_tnla_sum, SM_enum_tnla_theil, SM_enum_tnlg_atkinson, SM_enum_tnlg_avg, SM_enum_tnlg_coefficient_of_variation, SM_enum_tnlg_generalized_entropy, SM_enum_tnlg_gini, SM_enum_tnlg_hoover, SM_enum_tnlg_max, SM_enum_tnlg_median, SM_enum_tnlg_min, SM_enum_tnlg_shannon_entropy, SM_enum_tnlg_stdev, SM_enum_tnlg_sum, SM_enum_tnlg_theil, SM_enum_tnlm_atkinson, SM_enum_tnlm_avg, SM_enum_tnlm_coefficient_of_variation, SM_enum_tnlm_generalized_entropy, SM_enum_tnlm_gini, SM_enum_tnlm_hoover, SM_enum_tnlm_max, SM_enum_tnlm_median, SM_enum_tnlm_min, SM_enum_tnlm_shannon_entropy, SM_enum_tnlm_stdev, SM_enum_tnlm_sum, SM_enum_tnlm_theil, SM_enum_tnlpa_atkinson, SM_enum_tnlpa_avg, SM_enum_tnlpa_coefficient_of_variation, SM_enum_tnlpa_generalized_entropy, SM_enum_tnlpa_gini, SM_enum_tnlpa_hoover, SM_enum_tnlpa_max, SM_enum_tnlpa_median, SM_enum_tnlpa_min, SM_enum_tnlpa_shannon_entropy, SM_enum_tnlpa_stdev, SM_enum_tnlpa_sum, SM_enum_tnlpa_theil, SM_enum_tnlpm_atkinson, SM_enum_tnlpm_avg, SM_enum_tnlpm_coefficient_of_variation, SM_enum_tnlpm_generalized_entropy, SM_enum_tnlpm_gini, SM_enum_tnlpm_hoover, SM_enum_tnlpm_max, SM_enum_tnlpm_median, SM_enum_tnlpm_min, SM_enum_tnlpm_shannon_entropy, SM_enum_tnlpm_stdev, SM_enum_tnlpm_sum, SM_enum_tnlpm_theil, SM_enum_tnls_atkinson, SM_enum_tnls_avg, SM_enum_tnls_coefficient_of_variation, SM_enum_tnls_generalized_entropy, SM_enum_tnls_gini, SM_enum_tnls_hoover, SM_enum_tnls_max, SM_enum_tnls_median, SM_enum_tnls_min, SM_enum_tnls_shannon_entropy, SM_enum_tnls_stdev, SM_enum_tnls_sum, SM_enum_tnls_theil, SM_enum_tnm_atkinson, SM_enum_tnm_avg, SM_enum_tnm_coefficient_of_variation, SM_enum_tnm_generalized_entropy, SM_enum_tnm_gini, SM_enum_tnm_hoover, SM_enum_tnm_max, SM_enum_tnm_median, SM_enum_tnm_min, SM_enum_tnm_shannon_entropy, SM_enum_tnm_stdev, SM_enum_tnm_sum, SM_enum_tnm_theil, SM_enum_tnos_atkinson, SM_enum_tnos_avg, SM_enum_tnos_coefficient_of_variation, SM_enum_tnos_generalized_entropy, SM_enum_tnos_gini, SM_enum_tnos_hoover, SM_enum_tnos_max, SM_enum_tnos_median, SM_enum_tnos_min, SM_enum_tnos_shannon_entropy, SM_enum_tnos_stdev, SM_enum_tnos_sum, SM_enum_tnos_theil, SM_enum_tnpa_atkinson, SM_enum_tnpa_avg, SM_enum_tnpa_coefficient_of_variation, SM_enum_tnpa_generalized_entropy, SM_enum_tnpa_gini, SM_enum_tnpa_hoover, SM_enum_tnpa_max, SM_enum_tnpa_median, SM_enum_tnpa_min, SM_enum_tnpa_shannon_entropy, SM_enum_tnpa_stdev, SM_enum_tnpa_sum, SM_enum_tnpa_theil, SM_enum_tnpm_atkinson, SM_enum_tnpm_avg, SM_enum_tnpm_coefficient_of_variation, SM_enum_tnpm_generalized_entropy, SM_enum_tnpm_gini, SM_enum_tnpm_hoover, SM_enum_tnpm_max, SM_enum_tnpm_median, SM_enum_tnpm_min, SM_enum_tnpm_shannon_entropy, SM_enum_tnpm_stdev, SM_enum_tnpm_sum, SM_enum_tnpm_theil, SM_enum_tns_atkinson, SM_enum_tns_avg, SM_enum_tns_coefficient_of_variation, SM_enum_tns_generalized_entropy, SM_enum_tns_gini, SM_enum_tns_hoover, SM_enum_tns_max, SM_enum_tns_median, SM_enum_tns_min, SM_enum_tns_shannon_entropy, SM_enum_tns_stdev, SM_enum_tns_sum, SM_enum_tns_theil, SM_enum_wmc_atkinson, SM_enum_wmc_avg, SM_enum_wmc_coefficient_of_variation, SM_enum_wmc_generalized_entropy, SM_enum_wmc_gini, SM_enum_wmc_hoover, SM_enum_wmc_max, SM_enum_wmc_median, SM_enum_wmc_min, SM_enum_wmc_shannon_entropy, SM_enum_wmc_stdev, SM_enum_wmc_sum, SM_enum_wmc_theil, SM_file_cloc, SM_file_lloc, SM_file_loc, SM_file_mccc, SM_file_pda, SM_file_pua, SM_interface_ad_atkinson, SM_interface_ad_avg, SM_interface_ad_coefficient_of_variation, SM_interface_ad_generalized_entropy, SM_interface_ad_gini, SM_interface_ad_hoover, SM_interface_ad_max, SM_interface_ad_median, SM_interface_ad_min, SM_interface_ad_shannon_entropy, SM_interface_ad_stdev, SM_interface_ad_sum, SM_interface_ad_theil, SM_interface_cbo_atkinson, SM_interface_cbo_avg, SM_interface_cbo_coefficient_of_variation, SM_interface_cbo_generalized_entropy, SM_interface_cbo_gini, SM_interface_cbo_hoover, SM_interface_cbo_max, SM_interface_cbo_median, SM_interface_cbo_min, SM_interface_cbo_shannon_entropy, SM_interface_cbo_stdev, SM_interface_cbo_sum, SM_interface_cbo_theil, SM_interface_cboi_atkinson, SM_interface_cboi_avg, SM_interface_cboi_coefficient_of_variation, SM_interface_cboi_generalized_entropy, SM_interface_cboi_gini, SM_interface_cboi_hoover, SM_interface_cboi_max, SM_interface_cboi_median, SM_interface_cboi_min, SM_interface_cboi_shannon_entropy, SM_interface_cboi_stdev, SM_interface_cboi_sum, SM_interface_cboi_theil, SM_interface_cc_atkinson, SM_interface_cc_avg, SM_interface_cc_coefficient_of_variation, SM_interface_cc_generalized_entropy, SM_interface_cc_gini, SM_interface_cc_hoover, SM_interface_cc_max, SM_interface_cc_median, SM_interface_cc_min, SM_interface_cc_shannon_entropy, SM_interface_cc_stdev, SM_interface_cc_sum, SM_interface_cc_theil, SM_interface_ccl_atkinson, SM_interface_ccl_avg, SM_interface_ccl_coefficient_of_variation, SM_interface_ccl_generalized_entropy, SM_interface_ccl_gini, SM_interface_ccl_hoover, SM_interface_ccl_max, SM_interface_ccl_median, SM_interface_ccl_min, SM_interface_ccl_shannon_entropy, SM_interface_ccl_stdev, SM_interface_ccl_sum, SM_interface_ccl_theil, SM_interface_cco_atkinson, SM_interface_cco_avg, SM_interface_cco_coefficient_of_variation, SM_interface_cco_generalized_entropy, SM_interface_cco_gini, SM_interface_cco_hoover, SM_interface_cco_max, SM_interface_cco_median, SM_interface_cco_min, SM_interface_cco_shannon_entropy, SM_interface_cco_stdev, SM_interface_cco_sum, SM_interface_cco_theil, SM_interface_cd_atkinson, SM_interface_cd_avg, SM_interface_cd_coefficient_of_variation, SM_interface_cd_generalized_entropy, SM_interface_cd_gini, SM_interface_cd_hoover, SM_interface_cd_max, SM_interface_cd_median, SM_interface_cd_min, SM_interface_cd_shannon_entropy, SM_interface_cd_stdev, SM_interface_cd_sum, SM_interface_cd_theil, SM_interface_ci_atkinson, SM_interface_ci_avg, SM_interface_ci_coefficient_of_variation, SM_interface_ci_generalized_entropy, SM_interface_ci_gini, SM_interface_ci_hoover, SM_interface_ci_max, SM_interface_ci_median, SM_interface_ci_min, SM_interface_ci_shannon_entropy, SM_interface_ci_stdev, SM_interface_ci_sum, SM_interface_ci_theil, SM_interface_clc_atkinson, SM_interface_clc_avg, SM_interface_clc_coefficient_of_variation, SM_interface_clc_generalized_entropy, SM_interface_clc_gini, SM_interface_clc_hoover, SM_interface_clc_max, SM_interface_clc_median, SM_interface_clc_min, SM_interface_clc_shannon_entropy, SM_interface_clc_stdev, SM_interface_clc_sum, SM_interface_clc_theil, SM_interface_cllc_atkinson, SM_interface_cllc_avg, SM_interface_cllc_coefficient_of_variation, SM_interface_cllc_generalized_entropy, SM_interface_cllc_gini, SM_interface_cllc_hoover, SM_interface_cllc_max, SM_interface_cllc_median, SM_interface_cllc_min, SM_interface_cllc_shannon_entropy, SM_interface_cllc_stdev, SM_interface_cllc_sum, SM_interface_cllc_theil, SM_interface_cloc_atkinson, SM_interface_cloc_avg, SM_interface_cloc_coefficient_of_variation, SM_interface_cloc_generalized_entropy, SM_interface_cloc_gini, SM_interface_cloc_hoover, SM_interface_cloc_max, SM_interface_cloc_median, SM_interface_cloc_min, SM_interface_cloc_shannon_entropy, SM_interface_cloc_stdev, SM_interface_cloc_sum, SM_interface_cloc_theil, SM_interface_dit_atkinson, SM_interface_dit_avg, SM_interface_dit_coefficient_of_variation, SM_interface_dit_generalized_entropy, SM_interface_dit_gini, SM_interface_dit_hoover, SM_interface_dit_max, SM_interface_dit_median, SM_interface_dit_min, SM_interface_dit_shannon_entropy, SM_interface_dit_stdev, SM_interface_dit_sum, SM_interface_dit_theil, SM_interface_dloc_atkinson, SM_interface_dloc_avg, SM_interface_dloc_coefficient_of_variation, SM_interface_dloc_generalized_entropy, SM_interface_dloc_gini, SM_interface_dloc_hoover, SM_interface_dloc_max, SM_interface_dloc_median, SM_interface_dloc_min, SM_interface_dloc_shannon_entropy, SM_interface_dloc_stdev, SM_interface_dloc_sum, SM_interface_dloc_theil, SM_interface_lcom5_atkinson, SM_interface_lcom5_avg, SM_interface_lcom5_coefficient_of_variation, SM_interface_lcom5_generalized_entropy, SM_interface_lcom5_gini, SM_interface_lcom5_hoover, SM_interface_lcom5_max, SM_interface_lcom5_median, SM_interface_lcom5_min, SM_interface_lcom5_shannon_entropy, SM_interface_lcom5_stdev, SM_interface_lcom5_sum, SM_interface_lcom5_theil, SM_interface_ldc_atkinson, SM_interface_ldc_avg, SM_interface_ldc_coefficient_of_variation, SM_interface_ldc_generalized_entropy, SM_interface_ldc_gini, SM_interface_ldc_hoover, SM_interface_ldc_max, SM_interface_ldc_median, SM_interface_ldc_min, SM_interface_ldc_shannon_entropy, SM_interface_ldc_stdev, SM_interface_ldc_sum, SM_interface_ldc_theil, SM_interface_lldc_atkinson, SM_interface_lldc_avg, SM_interface_lldc_coefficient_of_variation, SM_interface_lldc_generalized_entropy, SM_interface_lldc_gini, SM_interface_lldc_hoover, SM_interface_lldc_max, SM_interface_lldc_median, SM_interface_lldc_min, SM_interface_lldc_shannon_entropy, SM_interface_lldc_stdev, SM_interface_lldc_sum, SM_interface_lldc_theil, SM_interface_lloc_atkinson, SM_interface_lloc_avg, SM_interface_lloc_coefficient_of_variation, SM_interface_lloc_generalized_entropy, SM_interface_lloc_gini, SM_interface_lloc_hoover, SM_interface_lloc_max, SM_interface_lloc_median, SM_interface_lloc_min, SM_interface_lloc_shannon_entropy, SM_interface_lloc_stdev, SM_interface_lloc_sum, SM_interface_lloc_theil, SM_interface_loc_atkinson, SM_interface_loc_avg, SM_interface_loc_coefficient_of_variation, SM_interface_loc_generalized_entropy, SM_interface_loc_gini, SM_interface_loc_hoover, SM_interface_loc_max, SM_interface_loc_median, SM_interface_loc_min, SM_interface_loc_shannon_entropy, SM_interface_loc_stdev, SM_interface_loc_sum, SM_interface_loc_theil, SM_interface_na_atkinson, SM_interface_na_avg, SM_interface_na_coefficient_of_variation, SM_interface_na_generalized_entropy, SM_interface_na_gini, SM_interface_na_hoover, SM_interface_na_max, SM_interface_na_median, SM_interface_na_min, SM_interface_na_shannon_entropy, SM_interface_na_stdev, SM_interface_na_sum, SM_interface_na_theil, SM_interface_ng_atkinson, SM_interface_ng_avg, SM_interface_ng_coefficient_of_variation, SM_interface_ng_generalized_entropy, SM_interface_ng_gini, SM_interface_ng_hoover, SM_interface_ng_max, SM_interface_ng_median, SM_interface_ng_min, SM_interface_ng_shannon_entropy, SM_interface_ng_stdev, SM_interface_ng_sum, SM_interface_ng_theil, SM_interface_nii_atkinson, SM_interface_nii_avg, SM_interface_nii_coefficient_of_variation, SM_interface_nii_generalized_entropy, SM_interface_nii_gini, SM_interface_nii_hoover, SM_interface_nii_max, SM_interface_nii_median, SM_interface_nii_min, SM_interface_nii_shannon_entropy, SM_interface_nii_stdev, SM_interface_nii_sum, SM_interface_nii_theil, SM_interface_nl_atkinson, SM_interface_nl_avg, SM_interface_nl_coefficient_of_variation, SM_interface_nl_generalized_entropy, SM_interface_nl_gini, SM_interface_nl_hoover, SM_interface_nl_max, SM_interface_nl_median, SM_interface_nl_min, SM_interface_nl_shannon_entropy, SM_interface_nl_stdev, SM_interface_nl_sum, SM_interface_nl_theil, SM_interface_nla_atkinson, SM_interface_nla_avg, SM_interface_nla_coefficient_of_variation, SM_interface_nla_generalized_entropy, SM_interface_nla_gini, SM_interface_nla_hoover, SM_interface_nla_max, SM_interface_nla_median, SM_interface_nla_min, SM_interface_nla_shannon_entropy, SM_interface_nla_stdev, SM_interface_nla_sum, SM_interface_nla_theil, SM_interface_nle_atkinson, SM_interface_nle_avg, SM_interface_nle_coefficient_of_variation, SM_interface_nle_generalized_entropy, SM_interface_nle_gini, SM_interface_nle_hoover, SM_interface_nle_max, SM_interface_nle_median, SM_interface_nle_min, SM_interface_nle_shannon_entropy, SM_interface_nle_stdev, SM_interface_nle_sum, SM_interface_nle_theil, SM_interface_nlg_atkinson, SM_interface_nlg_avg, SM_interface_nlg_coefficient_of_variation, SM_interface_nlg_generalized_entropy, SM_interface_nlg_gini, SM_interface_nlg_hoover, SM_interface_nlg_max, SM_interface_nlg_median, SM_interface_nlg_min, SM_interface_nlg_shannon_entropy, SM_interface_nlg_stdev, SM_interface_nlg_sum, SM_interface_nlg_theil, SM_interface_nlm_atkinson, SM_interface_nlm_avg, SM_interface_nlm_coefficient_of_variation, SM_interface_nlm_generalized_entropy, SM_interface_nlm_gini, SM_interface_nlm_hoover, SM_interface_nlm_max, SM_interface_nlm_median, SM_interface_nlm_min, SM_interface_nlm_shannon_entropy, SM_interface_nlm_stdev, SM_interface_nlm_sum, SM_interface_nlm_theil, SM_interface_nlpa_atkinson, SM_interface_nlpa_avg, SM_interface_nlpa_coefficient_of_variation, SM_interface_nlpa_generalized_entropy, SM_interface_nlpa_gini, SM_interface_nlpa_hoover, SM_interface_nlpa_max, SM_interface_nlpa_median, SM_interface_nlpa_min, SM_interface_nlpa_shannon_entropy, SM_interface_nlpa_stdev, SM_interface_nlpa_sum, SM_interface_nlpa_theil, SM_interface_nlpm_atkinson, SM_interface_nlpm_avg, SM_interface_nlpm_coefficient_of_variation, SM_interface_nlpm_generalized_entropy, SM_interface_nlpm_gini, SM_interface_nlpm_hoover, SM_interface_nlpm_max, SM_interface_nlpm_median, SM_interface_nlpm_min, SM_interface_nlpm_shannon_entropy, SM_interface_nlpm_stdev, SM_interface_nlpm_sum, SM_interface_nlpm_theil, SM_interface_nls_atkinson, SM_interface_nls_avg, SM_interface_nls_coefficient_of_variation, SM_interface_nls_generalized_entropy, SM_interface_nls_gini, SM_interface_nls_hoover, SM_interface_nls_max, SM_interface_nls_median, SM_interface_nls_min, SM_interface_nls_shannon_entropy, SM_interface_nls_stdev, SM_interface_nls_sum, SM_interface_nls_theil, SM_interface_nm_atkinson, SM_interface_nm_avg, SM_interface_nm_coefficient_of_variation, SM_interface_nm_generalized_entropy, SM_interface_nm_gini, SM_interface_nm_hoover, SM_interface_nm_max, SM_interface_nm_median, SM_interface_nm_min, SM_interface_nm_shannon_entropy, SM_interface_nm_stdev, SM_interface_nm_sum, SM_interface_nm_theil, SM_interface_noa_atkinson, SM_interface_noa_avg, SM_interface_noa_coefficient_of_variation, SM_interface_noa_generalized_entropy, SM_interface_noa_gini, SM_interface_noa_hoover, SM_interface_noa_max, SM_interface_noa_median, SM_interface_noa_min, SM_interface_noa_shannon_entropy, SM_interface_noa_stdev, SM_interface_noa_sum, SM_interface_noa_theil, SM_interface_noc_atkinson, SM_interface_noc_avg, SM_interface_noc_coefficient_of_variation, SM_interface_noc_generalized_entropy, SM_interface_noc_gini, SM_interface_noc_hoover, SM_interface_noc_max, SM_interface_noc_median, SM_interface_noc_min, SM_interface_noc_shannon_entropy, SM_interface_noc_stdev, SM_interface_noc_sum, SM_interface_noc_theil, SM_interface_nod_atkinson, SM_interface_nod_avg, SM_interface_nod_coefficient_of_variation, SM_interface_nod_generalized_entropy, SM_interface_nod_gini, SM_interface_nod_hoover, SM_interface_nod_max, SM_interface_nod_median, SM_interface_nod_min, SM_interface_nod_shannon_entropy, SM_interface_nod_stdev, SM_interface_nod_sum, SM_interface_nod_theil, SM_interface_noi_atkinson, SM_interface_noi_avg, SM_interface_noi_coefficient_of_variation, SM_interface_noi_generalized_entropy, SM_interface_noi_gini, SM_interface_noi_hoover, SM_interface_noi_max, SM_interface_noi_median, SM_interface_noi_min, SM_interface_noi_shannon_entropy, SM_interface_noi_stdev, SM_interface_noi_sum, SM_interface_noi_theil, SM_interface_nop_atkinson, SM_interface_nop_avg, SM_interface_nop_coefficient_of_variation, SM_interface_nop_generalized_entropy, SM_interface_nop_gini, SM_interface_nop_hoover, SM_interface_nop_max, SM_interface_nop_median, SM_interface_nop_min, SM_interface_nop_shannon_entropy, SM_interface_nop_stdev, SM_interface_nop_sum, SM_interface_nop_theil, SM_interface_nos_atkinson, SM_interface_nos_avg, SM_interface_nos_coefficient_of_variation, SM_interface_nos_generalized_entropy, SM_interface_nos_gini, SM_interface_nos_hoover, SM_interface_nos_max, SM_interface_nos_median, SM_interface_nos_min, SM_interface_nos_shannon_entropy, SM_interface_nos_stdev, SM_interface_nos_sum, SM_interface_nos_theil, SM_interface_npa_atkinson, SM_interface_npa_avg, SM_interface_npa_coefficient_of_variation, SM_interface_npa_generalized_entropy, SM_interface_npa_gini, SM_interface_npa_hoover, SM_interface_npa_max, SM_interface_npa_median, SM_interface_npa_min, SM_interface_npa_shannon_entropy, SM_interface_npa_stdev, SM_interface_npa_sum, SM_interface_npa_theil, SM_interface_npm_atkinson, SM_interface_npm_avg, SM_interface_npm_coefficient_of_variation, SM_interface_npm_generalized_entropy, SM_interface_npm_gini, SM_interface_npm_hoover, SM_interface_npm_max, SM_interface_npm_median, SM_interface_npm_min, SM_interface_npm_shannon_entropy, SM_interface_npm_stdev, SM_interface_npm_sum, SM_interface_npm_theil, SM_interface_ns_atkinson, SM_interface_ns_avg, SM_interface_ns_coefficient_of_variation, SM_interface_ns_generalized_entropy, SM_interface_ns_gini, SM_interface_ns_hoover, SM_interface_ns_max, SM_interface_ns_median, SM_interface_ns_min, SM_interface_ns_shannon_entropy, SM_interface_ns_stdev, SM_interface_ns_sum, SM_interface_ns_theil, SM_interface_pda_atkinson, SM_interface_pda_avg, SM_interface_pda_coefficient_of_variation, SM_interface_pda_generalized_entropy, SM_interface_pda_gini, SM_interface_pda_hoover, SM_interface_pda_max, SM_interface_pda_median, SM_interface_pda_min, SM_interface_pda_shannon_entropy, SM_interface_pda_stdev, SM_interface_pda_sum, SM_interface_pda_theil, SM_interface_pua_atkinson, SM_interface_pua_avg, SM_interface_pua_coefficient_of_variation, SM_interface_pua_generalized_entropy, SM_interface_pua_gini, SM_interface_pua_hoover, SM_interface_pua_max, SM_interface_pua_median, SM_interface_pua_min, SM_interface_pua_shannon_entropy, SM_interface_pua_stdev, SM_interface_pua_sum, SM_interface_pua_theil, SM_interface_rfc_atkinson, SM_interface_rfc_avg, SM_interface_rfc_coefficient_of_variation, SM_interface_rfc_generalized_entropy, SM_interface_rfc_gini, SM_interface_rfc_hoover, SM_interface_rfc_max, SM_interface_rfc_median, SM_interface_rfc_min, SM_interface_rfc_shannon_entropy, SM_interface_rfc_stdev, SM_interface_rfc_sum, SM_interface_rfc_theil, SM_interface_tcd_atkinson, SM_interface_tcd_avg, SM_interface_tcd_coefficient_of_variation, SM_interface_tcd_generalized_entropy, SM_interface_tcd_gini, SM_interface_tcd_hoover, SM_interface_tcd_max, SM_interface_tcd_median, SM_interface_tcd_min, SM_interface_tcd_shannon_entropy, SM_interface_tcd_stdev, SM_interface_tcd_sum, SM_interface_tcd_theil, SM_interface_tcloc_atkinson, SM_interface_tcloc_avg, SM_interface_tcloc_coefficient_of_variation, SM_interface_tcloc_generalized_entropy, SM_interface_tcloc_gini, SM_interface_tcloc_hoover, SM_interface_tcloc_max, SM_interface_tcloc_median, SM_interface_tcloc_min, SM_interface_tcloc_shannon_entropy, SM_interface_tcloc_stdev, SM_interface_tcloc_sum, SM_interface_tcloc_theil, SM_interface_tlloc_atkinson, SM_interface_tlloc_avg, SM_interface_tlloc_coefficient_of_variation, SM_interface_tlloc_generalized_entropy, SM_interface_tlloc_gini, SM_interface_tlloc_hoover, SM_interface_tlloc_max, SM_interface_tlloc_median, SM_interface_tlloc_min, SM_interface_tlloc_shannon_entropy, SM_interface_tlloc_stdev, SM_interface_tlloc_sum, SM_interface_tlloc_theil, SM_interface_tloc_atkinson, SM_interface_tloc_avg, SM_interface_tloc_coefficient_of_variation, SM_interface_tloc_generalized_entropy, SM_interface_tloc_gini, SM_interface_tloc_hoover, SM_interface_tloc_max, SM_interface_tloc_median, SM_interface_tloc_min, SM_interface_tloc_shannon_entropy, SM_interface_tloc_stdev, SM_interface_tloc_sum, SM_interface_tloc_theil, SM_interface_tna_atkinson, SM_interface_tna_avg, SM_interface_tna_coefficient_of_variation, SM_interface_tna_generalized_entropy, SM_interface_tna_gini, SM_interface_tna_hoover, SM_interface_tna_max, SM_interface_tna_median, SM_interface_tna_min, SM_interface_tna_shannon_entropy, SM_interface_tna_stdev, SM_interface_tna_sum, SM_interface_tna_theil, SM_interface_tng_atkinson, SM_interface_tng_avg, SM_interface_tng_coefficient_of_variation, SM_interface_tng_generalized_entropy, SM_interface_tng_gini, SM_interface_tng_hoover, SM_interface_tng_max, SM_interface_tng_median, SM_interface_tng_min, SM_interface_tng_shannon_entropy, SM_interface_tng_stdev, SM_interface_tng_sum, SM_interface_tng_theil, SM_interface_tnla_atkinson, SM_interface_tnla_avg, SM_interface_tnla_coefficient_of_variation, SM_interface_tnla_generalized_entropy, SM_interface_tnla_gini, SM_interface_tnla_hoover, SM_interface_tnla_max, SM_interface_tnla_median, SM_interface_tnla_min, SM_interface_tnla_shannon_entropy, SM_interface_tnla_stdev, SM_interface_tnla_sum, SM_interface_tnla_theil, SM_interface_tnlg_atkinson, SM_interface_tnlg_avg, SM_interface_tnlg_coefficient_of_variation, SM_interface_tnlg_generalized_entropy, SM_interface_tnlg_gini, SM_interface_tnlg_hoover, SM_interface_tnlg_max, SM_interface_tnlg_median, SM_interface_tnlg_min, SM_interface_tnlg_shannon_entropy, SM_interface_tnlg_stdev, SM_interface_tnlg_sum, SM_interface_tnlg_theil, SM_interface_tnlm_atkinson, SM_interface_tnlm_avg, SM_interface_tnlm_coefficient_of_variation, SM_interface_tnlm_generalized_entropy, SM_interface_tnlm_gini, SM_interface_tnlm_hoover, SM_interface_tnlm_max, SM_interface_tnlm_median, SM_interface_tnlm_min, SM_interface_tnlm_shannon_entropy, SM_interface_tnlm_stdev, SM_interface_tnlm_sum, SM_interface_tnlm_theil, SM_interface_tnlpa_atkinson, SM_interface_tnlpa_avg, SM_interface_tnlpa_coefficient_of_variation, SM_interface_tnlpa_generalized_entropy, SM_interface_tnlpa_gini, SM_interface_tnlpa_hoover, SM_interface_tnlpa_max, SM_interface_tnlpa_median, SM_interface_tnlpa_min, SM_interface_tnlpa_shannon_entropy, SM_interface_tnlpa_stdev, SM_interface_tnlpa_sum, SM_interface_tnlpa_theil, SM_interface_tnlpm_atkinson, SM_interface_tnlpm_avg, SM_interface_tnlpm_coefficient_of_variation, SM_interface_tnlpm_generalized_entropy, SM_interface_tnlpm_gini, SM_interface_tnlpm_hoover, SM_interface_tnlpm_max, SM_interface_tnlpm_median, SM_interface_tnlpm_min, SM_interface_tnlpm_shannon_entropy, SM_interface_tnlpm_stdev, SM_interface_tnlpm_sum, SM_interface_tnlpm_theil, SM_interface_tnls_atkinson, SM_interface_tnls_avg, SM_interface_tnls_coefficient_of_variation, SM_interface_tnls_generalized_entropy, SM_interface_tnls_gini, SM_interface_tnls_hoover, SM_interface_tnls_max, SM_interface_tnls_median, SM_interface_tnls_min, SM_interface_tnls_shannon_entropy, SM_interface_tnls_stdev, SM_interface_tnls_sum, SM_interface_tnls_theil, SM_interface_tnm_atkinson, SM_interface_tnm_avg, SM_interface_tnm_coefficient_of_variation, SM_interface_tnm_generalized_entropy, SM_interface_tnm_gini, SM_interface_tnm_hoover, SM_interface_tnm_max, SM_interface_tnm_median, SM_interface_tnm_min, SM_interface_tnm_shannon_entropy, SM_interface_tnm_stdev, SM_interface_tnm_sum, SM_interface_tnm_theil, SM_interface_tnos_atkinson, SM_interface_tnos_avg, SM_interface_tnos_coefficient_of_variation, SM_interface_tnos_generalized_entropy, SM_interface_tnos_gini, SM_interface_tnos_hoover, SM_interface_tnos_max, SM_interface_tnos_median, SM_interface_tnos_min, SM_interface_tnos_shannon_entropy, SM_interface_tnos_stdev, SM_interface_tnos_sum, SM_interface_tnos_theil, SM_interface_tnpa_atkinson, SM_interface_tnpa_avg, SM_interface_tnpa_coefficient_of_variation, SM_interface_tnpa_generalized_entropy, SM_interface_tnpa_gini, SM_interface_tnpa_hoover, SM_interface_tnpa_max, SM_interface_tnpa_median, SM_interface_tnpa_min, SM_interface_tnpa_shannon_entropy, SM_interface_tnpa_stdev, SM_interface_tnpa_sum, SM_interface_tnpa_theil, SM_interface_tnpm_atkinson, SM_interface_tnpm_avg, SM_interface_tnpm_coefficient_of_variation, SM_interface_tnpm_generalized_entropy, SM_interface_tnpm_gini, SM_interface_tnpm_hoover, SM_interface_tnpm_max, SM_interface_tnpm_median, SM_interface_tnpm_min, SM_interface_tnpm_shannon_entropy, SM_interface_tnpm_stdev, SM_interface_tnpm_sum, SM_interface_tnpm_theil, SM_interface_tns_atkinson, SM_interface_tns_avg, SM_interface_tns_coefficient_of_variation, SM_interface_tns_generalized_entropy, SM_interface_tns_gini, SM_interface_tns_hoover, SM_interface_tns_max, SM_interface_tns_median, SM_interface_tns_min, SM_interface_tns_shannon_entropy, SM_interface_tns_stdev, SM_interface_tns_sum, SM_interface_tns_theil, SM_interface_wmc_atkinson, SM_interface_wmc_avg, SM_interface_wmc_coefficient_of_variation, SM_interface_wmc_generalized_entropy, SM_interface_wmc_gini, SM_interface_wmc_hoover, SM_interface_wmc_max, SM_interface_wmc_median, SM_interface_wmc_min, SM_interface_wmc_shannon_entropy, SM_interface_wmc_stdev, SM_interface_wmc_sum, SM_interface_wmc_theil, SM_method_cc_atkinson, SM_method_cc_avg, SM_method_cc_coefficient_of_variation, SM_method_cc_generalized_entropy, SM_method_cc_gini, SM_method_cc_hoover, SM_method_cc_max, SM_method_cc_median, SM_method_cc_min, SM_method_cc_shannon_entropy, SM_method_cc_stdev, SM_method_cc_sum, SM_method_cc_theil, SM_method_ccl_atkinson, SM_method_ccl_avg, SM_method_ccl_coefficient_of_variation, SM_method_ccl_generalized_entropy, SM_method_ccl_gini, SM_method_ccl_hoover, SM_method_ccl_max, SM_method_ccl_median, SM_method_ccl_min, SM_method_ccl_shannon_entropy, SM_method_ccl_stdev, SM_method_ccl_sum, SM_method_ccl_theil, SM_method_cco_atkinson, SM_method_cco_avg, SM_method_cco_coefficient_of_variation, SM_method_cco_generalized_entropy, SM_method_cco_gini, SM_method_cco_hoover, SM_method_cco_max, SM_method_cco_median, SM_method_cco_min, SM_method_cco_shannon_entropy, SM_method_cco_stdev, SM_method_cco_sum, SM_method_cco_theil, SM_method_cd_atkinson, SM_method_cd_avg, SM_method_cd_coefficient_of_variation, SM_method_cd_generalized_entropy, SM_method_cd_gini, SM_method_cd_hoover, SM_method_cd_max, SM_method_cd_median, SM_method_cd_min, SM_method_cd_shannon_entropy, SM_method_cd_stdev, SM_method_cd_sum, SM_method_cd_theil, SM_method_ci_atkinson, SM_method_ci_avg, SM_method_ci_coefficient_of_variation, SM_method_ci_generalized_entropy, SM_method_ci_gini, SM_method_ci_hoover, SM_method_ci_max, SM_method_ci_median, SM_method_ci_min, SM_method_ci_shannon_entropy, SM_method_ci_stdev, SM_method_ci_sum, SM_method_ci_theil, SM_method_clc_atkinson, SM_method_clc_avg, SM_method_clc_coefficient_of_variation, SM_method_clc_generalized_entropy, SM_method_clc_gini, SM_method_clc_hoover, SM_method_clc_max, SM_method_clc_median, SM_method_clc_min, SM_method_clc_shannon_entropy, SM_method_clc_stdev, SM_method_clc_sum, SM_method_clc_theil, SM_method_cllc_atkinson, SM_method_cllc_avg, SM_method_cllc_coefficient_of_variation, SM_method_cllc_generalized_entropy, SM_method_cllc_gini, SM_method_cllc_hoover, SM_method_cllc_max, SM_method_cllc_median, SM_method_cllc_min, SM_method_cllc_shannon_entropy, SM_method_cllc_stdev, SM_method_cllc_sum, SM_method_cllc_theil, SM_method_cloc_atkinson, SM_method_cloc_avg, SM_method_cloc_coefficient_of_variation, SM_method_cloc_generalized_entropy, SM_method_cloc_gini, SM_method_cloc_hoover, SM_method_cloc_max, SM_method_cloc_median, SM_method_cloc_min, SM_method_cloc_shannon_entropy, SM_method_cloc_stdev, SM_method_cloc_sum, SM_method_cloc_theil, SM_method_dloc_atkinson, SM_method_dloc_avg, SM_method_dloc_coefficient_of_variation, SM_method_dloc_generalized_entropy, SM_method_dloc_gini, SM_method_dloc_hoover, SM_method_dloc_max, SM_method_dloc_median, SM_method_dloc_min, SM_method_dloc_shannon_entropy, SM_method_dloc_stdev, SM_method_dloc_sum, SM_method_dloc_theil, SM_method_hdif_atkinson, SM_method_hdif_avg, SM_method_hdif_coefficient_of_variation, SM_method_hdif_generalized_entropy, SM_method_hdif_gini, SM_method_hdif_hoover, SM_method_hdif_max, SM_method_hdif_median, SM_method_hdif_min, SM_method_hdif_shannon_entropy, SM_method_hdif_stdev, SM_method_hdif_sum, SM_method_hdif_theil, SM_method_hpl_atkinson, SM_method_hpl_avg, SM_method_hpl_coefficient_of_variation, SM_method_hpl_generalized_entropy, SM_method_hpl_gini, SM_method_hpl_hoover, SM_method_hpl_max, SM_method_hpl_median, SM_method_hpl_min, SM_method_hpl_shannon_entropy, SM_method_hpl_stdev, SM_method_hpl_sum, SM_method_hpl_theil, SM_method_hpv_atkinson, SM_method_hpv_avg, SM_method_hpv_coefficient_of_variation, SM_method_hpv_generalized_entropy, SM_method_hpv_gini, SM_method_hpv_hoover, SM_method_hpv_max, SM_method_hpv_median, SM_method_hpv_min, SM_method_hpv_shannon_entropy, SM_method_hpv_stdev, SM_method_hpv_sum, SM_method_hpv_theil, SM_method_ldc_atkinson, SM_method_ldc_avg, SM_method_ldc_coefficient_of_variation, SM_method_ldc_generalized_entropy, SM_method_ldc_gini, SM_method_ldc_hoover, SM_method_ldc_max, SM_method_ldc_median, SM_method_ldc_min, SM_method_ldc_shannon_entropy, SM_method_ldc_stdev, SM_method_ldc_sum, SM_method_ldc_theil, SM_method_lldc_atkinson, SM_method_lldc_avg, SM_method_lldc_coefficient_of_variation, SM_method_lldc_generalized_entropy, SM_method_lldc_gini, SM_method_lldc_hoover, SM_method_lldc_max, SM_method_lldc_median, SM_method_lldc_min, SM_method_lldc_shannon_entropy, SM_method_lldc_stdev, SM_method_lldc_sum, SM_method_lldc_theil, SM_method_lloc_atkinson, SM_method_lloc_avg, SM_method_lloc_coefficient_of_variation, SM_method_lloc_generalized_entropy, SM_method_lloc_gini, SM_method_lloc_hoover, SM_method_lloc_max, SM_method_lloc_median, SM_method_lloc_min, SM_method_lloc_shannon_entropy, SM_method_lloc_stdev, SM_method_lloc_sum, SM_method_lloc_theil, SM_method_loc_atkinson, SM_method_loc_avg, SM_method_loc_coefficient_of_variation, SM_method_loc_generalized_entropy, SM_method_loc_gini, SM_method_loc_hoover, SM_method_loc_max, SM_method_loc_median, SM_method_loc_min, SM_method_loc_shannon_entropy, SM_method_loc_stdev, SM_method_loc_sum, SM_method_loc_theil, SM_method_mccc_atkinson, SM_method_mccc_avg, SM_method_mccc_coefficient_of_variation, SM_method_mccc_generalized_entropy, SM_method_mccc_gini, SM_method_mccc_hoover, SM_method_mccc_max, SM_method_mccc_median, SM_method_mccc_min, SM_method_mccc_shannon_entropy, SM_method_mccc_stdev, SM_method_mccc_sum, SM_method_mccc_theil, SM_method_mi_atkinson, SM_method_mi_avg, SM_method_mi_coefficient_of_variation, SM_method_mi_generalized_entropy, SM_method_mi_gini, SM_method_mi_hoover, SM_method_mi_max, SM_method_mi_median, SM_method_mi_min, SM_method_mi_shannon_entropy, SM_method_mi_stdev, SM_method_mi_sum, SM_method_mi_theil, SM_method_mims_atkinson, SM_method_mims_avg, SM_method_mims_coefficient_of_variation, SM_method_mims_generalized_entropy, SM_method_mims_gini, SM_method_mims_hoover, SM_method_mims_max, SM_method_mims_median, SM_method_mims_min, SM_method_mims_shannon_entropy, SM_method_mims_stdev, SM_method_mims_sum, SM_method_mims_theil, SM_method_misei_atkinson, SM_method_misei_avg, SM_method_misei_coefficient_of_variation, SM_method_misei_generalized_entropy, SM_method_misei_gini, SM_method_misei_hoover, SM_method_misei_max, SM_method_misei_median, SM_method_misei_min, SM_method_misei_shannon_entropy, SM_method_misei_stdev, SM_method_misei_sum, SM_method_misei_theil, SM_method_mism_atkinson, SM_method_mism_avg, SM_method_mism_coefficient_of_variation, SM_method_mism_generalized_entropy, SM_method_mism_gini, SM_method_mism_hoover, SM_method_mism_max, SM_method_mism_median, SM_method_mism_min, SM_method_mism_shannon_entropy, SM_method_mism_stdev, SM_method_mism_sum, SM_method_mism_theil, SM_method_nii_atkinson, SM_method_nii_avg, SM_method_nii_coefficient_of_variation, SM_method_nii_generalized_entropy, SM_method_nii_gini, SM_method_nii_hoover, SM_method_nii_max, SM_method_nii_median, SM_method_nii_min, SM_method_nii_shannon_entropy, SM_method_nii_stdev, SM_method_nii_sum, SM_method_nii_theil, SM_method_nl_atkinson, SM_method_nl_avg, SM_method_nl_coefficient_of_variation, SM_method_nl_generalized_entropy, SM_method_nl_gini, SM_method_nl_hoover, SM_method_nl_max, SM_method_nl_median, SM_method_nl_min, SM_method_nl_shannon_entropy, SM_method_nl_stdev, SM_method_nl_sum, SM_method_nl_theil, SM_method_nle_atkinson, SM_method_nle_avg, SM_method_nle_coefficient_of_variation, SM_method_nle_generalized_entropy, SM_method_nle_gini, SM_method_nle_hoover, SM_method_nle_max, SM_method_nle_median, SM_method_nle_min, SM_method_nle_shannon_entropy, SM_method_nle_stdev, SM_method_nle_sum, SM_method_nle_theil, SM_method_noi_atkinson, SM_method_noi_avg, SM_method_noi_coefficient_of_variation, SM_method_noi_generalized_entropy, SM_method_noi_gini, SM_method_noi_hoover, SM_method_noi_max, SM_method_noi_median, SM_method_noi_min, SM_method_noi_shannon_entropy, SM_method_noi_stdev, SM_method_noi_sum, SM_method_noi_theil, SM_method_nos_atkinson, SM_method_nos_avg, SM_method_nos_coefficient_of_variation, SM_method_nos_generalized_entropy, SM_method_nos_gini, SM_method_nos_hoover, SM_method_nos_max, SM_method_nos_median, SM_method_nos_min, SM_method_nos_shannon_entropy, SM_method_nos_stdev, SM_method_nos_sum, SM_method_nos_theil, SM_method_numpar_atkinson, SM_method_numpar_avg, SM_method_numpar_coefficient_of_variation, SM_method_numpar_generalized_entropy, SM_method_numpar_gini, SM_method_numpar_hoover, SM_method_numpar_max, SM_method_numpar_median, SM_method_numpar_min, SM_method_numpar_shannon_entropy, SM_method_numpar_stdev, SM_method_numpar_sum, SM_method_numpar_theil, SM_method_tcd_atkinson, SM_method_tcd_avg, SM_method_tcd_coefficient_of_variation, SM_method_tcd_generalized_entropy, SM_method_tcd_gini, SM_method_tcd_hoover, SM_method_tcd_max, SM_method_tcd_median, SM_method_tcd_min, SM_method_tcd_shannon_entropy, SM_method_tcd_stdev, SM_method_tcd_sum, SM_method_tcd_theil, SM_method_tcloc_atkinson, SM_method_tcloc_avg, SM_method_tcloc_coefficient_of_variation, SM_method_tcloc_generalized_entropy, SM_method_tcloc_gini, SM_method_tcloc_hoover, SM_method_tcloc_max, SM_method_tcloc_median, SM_method_tcloc_min, SM_method_tcloc_shannon_entropy, SM_method_tcloc_stdev, SM_method_tcloc_sum, SM_method_tcloc_theil, SM_method_tlloc_atkinson, SM_method_tlloc_avg, SM_method_tlloc_coefficient_of_variation, SM_method_tlloc_generalized_entropy, SM_method_tlloc_gini, SM_method_tlloc_hoover, SM_method_tlloc_max, SM_method_tlloc_median, SM_method_tlloc_min, SM_method_tlloc_shannon_entropy, SM_method_tlloc_stdev, SM_method_tlloc_sum, SM_method_tlloc_theil, SM_method_tloc_atkinson, SM_method_tloc_avg, SM_method_tloc_coefficient_of_variation, SM_method_tloc_generalized_entropy, SM_method_tloc_gini, SM_method_tloc_hoover, SM_method_tloc_max, SM_method_tloc_median, SM_method_tloc_min, SM_method_tloc_shannon_entropy, SM_method_tloc_stdev, SM_method_tloc_sum, SM_method_tloc_theil, SM_method_tnos_atkinson, SM_method_tnos_avg, SM_method_tnos_coefficient_of_variation, SM_method_tnos_generalized_entropy, SM_method_tnos_gini, SM_method_tnos_hoover, SM_method_tnos_max, SM_method_tnos_median, SM_method_tnos_min, SM_method_tnos_shannon_entropy, SM_method_tnos_stdev, SM_method_tnos_sum, SM_method_tnos_theil, SM_package_ad, SM_package_cc, SM_package_ccl, SM_package_cco, SM_package_cd, SM_package_ci, SM_package_clc, SM_package_cllc, SM_package_cloc, SM_package_ldc, SM_package_lldc, SM_package_lloc, SM_package_loc, SM_package_na, SM_package_ncl, SM_package_nen, SM_package_ng, SM_package_nin, SM_package_nm, SM_package_npa, SM_package_npkg, SM_package_npm, SM_package_ns, SM_package_pda, SM_package_pua, SM_package_tad, SM_package_tcd, SM_package_tcloc, SM_package_tlloc, SM_package_tloc, SM_package_tna, SM_package_tncl, SM_package_tndi, SM_package_tnen, SM_package_tnfi, SM_package_tng, SM_package_tnin, SM_package_tnm, SM_package_tnos, SM_package_tnpa, SM_package_tnpcl, SM_package_tnpen, SM_package_tnpin, SM_package_tnpkg, SM_package_tnpm, SM_package_tns, SM_package_tpda, SM_package_tpua
• MOSER: The change metrics proposed by in the publication R. Moser, W. Pedrycz, G. Succi: "A comparative analysis of the efficiency of change metrics and static code attributes", ICSE 2008. Overall, there are 17 of these metrics, for the description we refer to the publication by Moser et al.:
  • MOSER_age, MOSER_authors, MOSER_avg_changeset, MOSER_avg_code_churn, MOSER_avg_lines_added, MOSER_avg_lines_deleted, MOSER_bugfix, MOSER_max_changeset, MOSER_max_code_churn, MOSER_max_lines_added, MOSER_max_lines_deleted, MOSER_refactorings, MOSER_revisions, MOSER_sum_code_churn, MOSER_sum_lines_added, MOSER_sum_lines_deleted, MOSER_weighted_age
• HASSAN: The change metrics proposed in the publication A. E. Hassan: "Predicting faults using the complexity of code changes", ICSE 2009. Overall, there are five of these features, for the description we refer to the publication by Hassan:
  • HASSAN_edhcm, HASSAN_hcm, HASSAN_ldhcm, HASSAN_lgdhcm, HASSAN_whcm
• DAMBROS: The change metrics proposed in the publication M. D'Ambros, M. Lanza, R. Robbes: "Evaluating defect prediction approaches: a benchmark and an extensive comparison", EMSE 17(4-5):531–577, 2012. The names of the metrics follow the pattern `DAMBROS_`, e.g., DAMBROS_edhh_cbo for the exponential decay of the entropy over time (EDHH) of the metric Coupling Between Objects (CBO) . Overall, there are 170 of these features, for the description we refer to the publication by D'Ambros et al.:
  • DAMBROS_edhh_cbo, DAMBROS_edhh_dit, DAMBROS_edhh_lcom5, DAMBROS_edhh_nii, DAMBROS_edhh_noc, DAMBROS_edhh_noi, DAMBROS_edhh_rfc, DAMBROS_edhh_tloc, DAMBROS_edhh_tna, DAMBROS_edhh_tna-tnla, DAMBROS_edhh_tna-tnpa, DAMBROS_edhh_tnlpm, DAMBROS_edhh_tnm, DAMBROS_edhh_tnm-tnlm, DAMBROS_edhh_tnm-tnpm, DAMBROS_edhh_tnpa, DAMBROS_edhh_wmc, DAMBROS_edpchu_cbo, DAMBROS_edpchu_dit, DAMBROS_edpchu_lcom5, DAMBROS_edpchu_nii, DAMBROS_edpchu_noc, DAMBROS_edpchu_noi, DAMBROS_edpchu_rfc, DAMBROS_edpchu_tloc, DAMBROS_edpchu_tna, DAMBROS_edpchu_tna-tnla, DAMBROS_edpchu_tna-tnpa, DAMBROS_edpchu_tnlpm, DAMBROS_edpchu_tnm, DAMBROS_edpchu_tnm-tnlm, DAMBROS_edpchu_tnm-tnpm, DAMBROS_edpchu_tnpa, DAMBROS_edpchu_wmc, DAMBROS_hh_cbo, DAMBROS_hh_dit, DAMBROS_hh_lcom5, DAMBROS_hh_nii, DAMBROS_hh_noc, DAMBROS_hh_noi, DAMBROS_hh_rfc, DAMBROS_hh_tloc, DAMBROS_hh_tna, DAMBROS_hh_tna-tnla, DAMBROS_hh_tna-tnpa, DAMBROS_hh_tnlpm, DAMBROS_hh_tnm, DAMBROS_hh_tnm-tnlm, DAMBROS_hh_tnm-tnpm, DAMBROS_hh_tnpa, DAMBROS_hh_wmc, DAMBROS_hwh_cbo, DAMBROS_hwh_dit, DAMBROS_hwh_lcom5, DAMBROS_hwh_nii, DAMBROS_hwh_noc, DAMBROS_hwh_noi, DAMBROS_hwh_rfc, DAMBROS_hwh_tloc, DAMBROS_hwh_tna, DAMBROS_hwh_tna-tnla, DAMBROS_hwh_tna-tnpa, DAMBROS_hwh_tnlpm, DAMBROS_hwh_tnm, DAMBROS_hwh_tnm-tnlm, DAMBROS_hwh_tnm-tnpm, DAMBROS_hwh_tnpa, DAMBROS_hwh_wmc, DAMBROS_ldhh_cbo, DAMBROS_ldhh_dit, DAMBROS_ldhh_lcom5, DAMBROS_ldhh_nii, DAMBROS_ldhh_noc, DAMBROS_ldhh_noi, DAMBROS_ldhh_rfc, DAMBROS_ldhh_tloc, DAMBROS_ldhh_tna, DAMBROS_ldhh_tna-tnla, DAMBROS_ldhh_tna-tnpa, DAMBROS_ldhh_tnlpm, DAMBROS_ldhh_tnm, DAMBROS_ldhh_tnm-tnlm, DAMBROS_ldhh_tnm-tnpm, DAMBROS_ldhh_tnpa, DAMBROS_ldhh_wmc, DAMBROS_ldpchu_cbo, DAMBROS_ldpchu_dit, DAMBROS_ldpchu_lcom5, DAMBROS_ldpchu_nii, DAMBROS_ldpchu_noc, DAMBROS_ldpchu_noi, DAMBROS_ldpchu_rfc, DAMBROS_ldpchu_tloc, DAMBROS_ldpchu_tna, DAMBROS_ldpchu_tna-tnla, DAMBROS_ldpchu_tna-tnpa, DAMBROS_ldpchu_tnlpm, DAMBROS_ldpchu_tnm, DAMBROS_ldpchu_tnm-tnlm, DAMBROS_ldpchu_tnm-tnpm, DAMBROS_ldpchu_tnpa, DAMBROS_ldpchu_wmc, DAMBROS_lgdhh_cbo, DAMBROS_lgdhh_dit, DAMBROS_lgdhh_lcom5, DAMBROS_lgdhh_nii, DAMBROS_lgdhh_noc, DAMBROS_lgdhh_noi, DAMBROS_lgdhh_rfc, DAMBROS_lgdhh_tloc, DAMBROS_lgdhh_tna, DAMBROS_lgdhh_tna-tnla, DAMBROS_lgdhh_tna-tnpa, DAMBROS_lgdhh_tnlpm, DAMBROS_lgdhh_tnm, DAMBROS_lgdhh_tnm-tnlm, DAMBROS_lgdhh_tnm-tnpm, DAMBROS_lgdhh_tnpa, DAMBROS_lgdhh_wmc, DAMBROS_lgdpchu_cbo, DAMBROS_lgdpchu_dit, DAMBROS_lgdpchu_lcom5, DAMBROS_lgdpchu_nii, DAMBROS_lgdpchu_noc, DAMBROS_lgdpchu_noi, DAMBROS_lgdpchu_rfc, DAMBROS_lgdpchu_tloc, DAMBROS_lgdpchu_tna, DAMBROS_lgdpchu_tna-tnla, DAMBROS_lgdpchu_tna-tnpa, DAMBROS_lgdpchu_tnlpm, DAMBROS_lgdpchu_tnm, DAMBROS_lgdpchu_tnm-tnlm, DAMBROS_lgdpchu_tnm-tnpm, DAMBROS_lgdpchu_tnpa, DAMBROS_lgdpchu_wmc, DAMBROS_pchu_cbo, DAMBROS_pchu_dit, DAMBROS_pchu_lcom5, DAMBROS_pchu_nii, DAMBROS_pchu_noc, DAMBROS_pchu_noi, DAMBROS_pchu_rfc, DAMBROS_pchu_tloc, DAMBROS_pchu_tna, DAMBROS_pchu_tna-tnla, DAMBROS_pchu_tna-tnpa, DAMBROS_pchu_tnlpm, DAMBROS_pchu_tnm, DAMBROS_pchu_tnm-tnlm, DAMBROS_pchu_tnm-tnpm, DAMBROS_pchu_tnpa, DAMBROS_pchu_wmc, DAMBROS_wpchu_cbo, DAMBROS_wpchu_dit, DAMBROS_wpchu_lcom5, DAMBROS_wpchu_nii, DAMBROS_wpchu_noc, DAMBROS_wpchu_noi, DAMBROS_wpchu_rfc, DAMBROS_wpchu_tloc, DAMBROS_wpchu_tna, DAMBROS_wpchu_tna-tnla, DAMBROS_wpchu_tna-tnpa, DAMBROS_wpchu_tnlpm, DAMBROS_wpchu_tnm, DAMBROS_wpchu_tnm-tnlm, DAMBROS_wpchu_tnm-tnpm, DAMBROS_wpchu_tnpa, DAMBROS_wpchu_wmc
• ISSUE: The counts for issues that were fixed in the files in the last six months before a release, e.g., ISSUE_blocker_bug for the issues of severity blocker and the type bug. Overall, there are 72 of these features:
  • ISSUE_blocker_bug, ISSUE_blocker_documentation, ISSUE_blocker_improvement, ISSUE_blocker_improvment, ISSUE_blocker_none, ISSUE_blocker_other, ISSUE_blocker_task, ISSUE_blocker_test, ISSUE_critical_bug, ISSUE_critical_documentation, ISSUE_critical_improvement, ISSUE_critical_improvment, ISSUE_critical_none, ISSUE_critical_other, ISSUE_critical_task, ISSUE_critical_test, ISSUE_enhancement_bug, ISSUE_enhancement_documentation, ISSUE_enhancement_improvement, ISSUE_enhancement_improvment, ISSUE_enhancement_none, ISSUE_enhancement_other, ISSUE_enhancement_task, ISSUE_enhancement_test, ISSUE_major_bug, ISSUE_major_documentation, ISSUE_major_improvement, ISSUE_major_improvment, ISSUE_major_none, ISSUE_major_other, ISSUE_major_task, ISSUE_major_test, ISSUE_minor_bug, ISSUE_minor_documentation, ISSUE_minor_improvement, ISSUE_minor_improvment, ISSUE_minor_none, ISSUE_minor_other, ISSUE_minor_task, ISSUE_minor_test, ISSUE_none_bug, ISSUE_none_documentation, ISSUE_none_improvement, ISSUE_none_improvment, ISSUE_none_none, ISSUE_none_other, ISSUE_none_task, ISSUE_none_test, ISSUE_normal_bug, ISSUE_normal_documentation, ISSUE_normal_improvement, ISSUE_normal_improvment, ISSUE_normal_none, ISSUE_normal_other, ISSUE_normal_task, ISSUE_normal_test, ISSUE_regression_bug, ISSUE_regression_documentation, ISSUE_regression_improvement, ISSUE_regression_improvment, ISSUE_regression_none, ISSUE_regression_other, ISSUE_regression_task, ISSUE_regression_test, ISSUE_trivial_bug, ISSUE_trivial_documentation, ISSUE_trivial_improvement, ISSUE_trivial_improvment, ISSUE_trivial_none, ISSUE_trivial_other, ISSUE_trivial_task, ISSUE_trivial_test
• AST: The counts for the different types of nodes of the AST of the file, e.g., AST_import for the number of import statements or AST_methodinvocation for method calls. Overall, there are 77 of these features:
  • AST_annotation, AST_annotationdeclaration, AST_annotationmethod, AST_arraycreator, AST_arrayinitializer, AST_arrayselector, AST_assertstatement, AST_assignment, AST_basictype, AST_binaryoperation, AST_blockstatement, AST_breakstatement, AST_cast, AST_catchclause, AST_catchclauseparameter, AST_classcreator, AST_classdeclaration, AST_classreference, AST_compilationunit, AST_constantdeclaration, AST_constructordeclaration, AST_continuestatement, AST_declaration, AST_documented, AST_dostatement, AST_elementarrayvalue, AST_elementvaluepair, AST_enhancedforcontrol, AST_enumbody, AST_enumconstantdeclaration, AST_enumdeclaration, AST_explicitconstructorinvocation, AST_expression, AST_fielddeclaration, AST_forcontrol, AST_formalparameter, AST_forstatement, AST_ifstatement, AST_import, AST_inferredformalparameter, AST_innerclasscreator, AST_interfacedeclaration, AST_invocation, AST_lambdaexpression, AST_literal, AST_localvariabledeclaration, AST_member, AST_memberreference, AST_methoddeclaration, AST_methodinvocation, AST_methodreference, AST_node_count, AST_packagedeclaration, AST_primary, AST_referencetype, AST_returnstatement, AST_statement, AST_statementexpression, AST_superconstructorinvocation, AST_supermemberreference, AST_supermethodinvocation, AST_switchstatement, AST_switchstatementcase, AST_synchronizedstatement, AST_ternaryexpression, AST_this, AST_throwstatement, AST_tryresource, AST_trystatement, AST_type, AST_typeargument, AST_typedeclaration, AST_typeparameter, AST_variabledeclaration, AST_variabledeclarator, AST_voidclassreference, AST_whilestatement
• CHANGE_TYPE: The number of changes of a change type in the last six months. The change types are classified using ChangeDistiller. Overall, there are 5 of these metrics:
  • CHANGE_TYPE_computation, CHANGE_TYPE_data, CHANGE_TYPE_interface, CHANGE_TYPE_logic/control, CHANGE_TYPE_other
• PMD: The count of the number of the rule violations found by PMD. Overall, there are 193 features for PMD rules, three features that count the number of rule violations per severity (critical, major, minor as per PMD) and 19 features for groups of rules:
  • PMD_aaa, PMD_aal, PMD_abcwam, PMD_absalil, PMD_acf, PMD_acge, PMD_aci, PMD_acnpe, PMD_act, PMD_acwam, PMD_adl, PMD_adlibdc, PMD_ads, PMD_aes, PMD_afnmmn, PMD_afnmtn, PMD_aicicc, PMD_aio, PMD_aisd, PMD_alei, PMD_amuo, PMD_apfifc, PMD_apmifcne, PMD_apmp, PMD_apst, PMD_are, PMD_arp, PMD_asaml, PMD_asbf, PMD_atg, PMD_atnfs, PMD_atniose, PMD_atnpe, PMD_atret, PMD_auhcip, PMD_aunc, PMD_auov, PMD_bc, PMD_bgmn, PMD_bi, PMD_bii, PMD_bnc, PMD_casr, PMD_ccewta, PMD_ccom, PMD_cis, PMD_cla, PMD_clmmic, PMD_clr, PMD_cnc, PMD_crs, PMD_csr, PMD_ctcnse, PMD_cwopcsbf, PMD_dcl, PMD_dctr, PMD_di, PMD_dijl, PMD_dis, PMD_dlnliss, PMD_dncgce, PMD_dncse, PMD_dnejle, PMD_dnteif, PMD_dp, PMD_duftfli, PMD_eafc, PMD_ecb, PMD_ef, PMD_efb, PMD_eis, PMD_emiacsba, PMD_emsb, PMD_en, PMD_eo, PMD_esb, PMD_esi, PMD_esnil, PMD_ess, PMD_etb, PMD_ews, PMD_fdncsf, PMD_fdsbasoc, PMD_ffcbs, PMD_flmub, PMD_flsbwl, PMD_fo, PMD_focsf, PMD_fsbp, PMD_gdl, PMD_gls, PMD_glsju, PMD_gn, PMD_iesmub, PMD_if, PMD_ifsp, PMD_io, PMD_isb, PMD_ismub, PMD_itgc, PMD_ji, PMD_juasim, PMD_jus, PMD_juss, PMD_jutctma, PMD_jutsia, PMD_lhnc, PMD_li, PMD_linsf, PMD_lisnc, PMD_loc, PMD_mbis, PMD_mdbasbnc, PMD_menc, PMD_mnc, PMD_mria, PMD_msminic, PMD_msvuid, PMD_mtol, PMD_mwsnaec, PMD_ncliss, PMD_np, PMD_nsi, PMD_ntss, PMD_obeah, PMD_odpl, PMD_otac, PMD_pc, PMD_pci, PMD_pl, PMD_plfic, PMD_plficic, PMD_pst, PMD_reartn, PMD_rffb, PMD_rfi, PMD_rinc, PMD_rsinc, PMD_rule_type_basic rules, PMD_rule_type_brace rules, PMD_rule_type_clone implementation rules, PMD_rule_type_controversial rules, PMD_rule_type_design rules, PMD_rule_type_finalizer rules, PMD_rule_type_import statement rules, PMD_rule_type_j2ee rules, PMD_rule_type_jakarta commons logging rules, PMD_rule_type_java logging rules, PMD_rule_type_javabean rules, PMD_rule_type_junit rules, PMD_rule_type_naming rules, PMD_rule_type_optimization rules, PMD_rule_type_security code guideline rules, PMD_rule_type_strict exception rules, PMD_rule_type_string and stringbuffer rules, PMD_rule_type_type resolution rules, PMD_rule_type_unnecessary and unused code rules, PMD_sba, PMD_sbe, PMD_sbiwc, PMD_sbr, PMD_sc, PMD_scfn, PMD_scn, PMD_sdfnl, PMD_sejbfsbf, PMD_semn, PMD_severity_critical, PMD_severity_major, PMD_severity_minor, PMD_sf, PMD_shmn, PMD_sidte, PMD_smn, PMD_soe, PMD_sp, PMD_ssshd, PMD_sti, PMD_sts, PMD_tcwtc, PMD_tfbfass, PMD_tmsi, PMD_uaal, PMD_uaeioat, PMD_ualiov, PMD_uanioat, PMD_uasioat, PMD_uatioae, PMD_uba, PMD_uc, PMD_ucc, PMD_ucel, PMD_ucie, PMD_uct, PMD_uec, PMD_uem, PMD_uetcs, PMD_ufqn, PMD_uis, PMD_ulbr, PMD_ulv, PMD_ulwcc, PMD_unaion, PMD_uncie, PMD_uni, PMD_uom, PMD_uooi, PMD_upf, PMD_upm, PMD_usbfsa, PMD_usdf, PMD_uv, PMD_uwoc, PMD_vnc, PMD_wlmub
• REFACTOR: Number of refactorings in the last six months as determined by RefDiff. Overall, there are 13 of these features:
  • REFACTOR_extract_method, REFACTOR_extract_superclass, REFACTOR_inline_method, REFACTOR_move_and_rename_class, REFACTOR_move_attribute, REFACTOR_move_class, REFACTOR_move_method, REFACTOR_pull_up_attribute, REFACTOR_pull_up_method, REFACTOR_push_down_attribute, REFACTOR_push_down_method, REFACTOR_rename_class, REFACTOR_rename_method

Source code of data collection tools

All tools we created for the collection of this data are publicly available on GitHub in the SmartSHARK organization. Regardless, we provide copies of the source code for all tools in the collection-tools.tar.gz archive. Guidelines on how to use these tools can be found on the respective homepages of each tool on their GitHub pages. If you have problems, feel free to contact us.

Replication of empirical results

In the following, you find the code for the replication of the empirical study in presented in Section 5 of the article. The Notebook has been fully executed prior to the publication and already shows the results, including additional details for each part of the study, e.g., the raw results per project/release. You must have a running MongoDB with the data we provided if you want to execute the code. You may have to adopt the credentials/host of the MongoDB, depending on where the MongoDB is running. The code for this is in the second cell. All cells should be executed in the order of the notebook, even though they are mostly independent of each other.

Installation and loading of dependencies

import sys
!{sys.executable} -m pip install pycoshark matplotlib pandas seaborn openpyxl sklearn xgboost scipy==1.3 autorank --user 

import os
import re
import random
import warnings
import multiprocessing

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import networkx as nx
import openpyxl

%matplotlib inline

from mongoengine import connect, DoesNotExist
from pycoshark.mongomodels import Commit, FileAction, File, CodeEntityState, Project, VCSSystem, Hunk, Issue, Event, IssueSystem, Tag
from pycoshark.utils import create_mongodb_uri_string, jira_is_resolved_and_fixed, get_commit_graph, heuristic_renames, java_filename_filter, git_tag_filter, get_affected_versions
from datetime import datetime
from dateutil.relativedelta import relativedelta
from math import isnan,sqrt,floor
from mongoengine.base.datastructures import BaseDict
from IPython.display import display
from sklearn.model_selection import StratifiedKFold,RandomizedSearchCV
from sklearn.naive_bayes import GaussianNBfrom
from sklearn.metrics import roc_auc_score, f1_score, recall_score, precision_score
from statistics import mean 
from scipy.stats import median_absolute_deviation, wilcoxon
from autorank import autorank, plot_stats, latex_table, create_report

# setup pandas to display complete data frames of results
pd.set_option('display.max_rows', 500)
pd.set_option('display.max_columns', 500)

# do not print warnings - we checked, there are no serious warnings and they just polute the output
warnings.filterwarnings("ignore")

def mad(df):
    """Helper function to make output for median absolute deviation same as for native pandas stats"""
    res = pd.Series(index=df.columns)
    for col in df.columns:
        # apply column-wise, otherwise a single NaN will make all columns NaN
        res[col] = median_absolute_deviation(df[col])
    return res

def cliffs_delta(control, pilot):
    """delta calculation adopted from https://github.com/gousiosg/cliffs.d"""
    delta = 0.0
    for cnt in control:
        result = 0
        for plt in pilot:
            if plt>cnt:
                result -= 1
            elif cnt>plt:
                result += 1
        delta += result/len(pilot)
    delta = delta/len(control)
    if abs(delta)<0.147:
        eff_size = 'negligible'
    elif abs(delta)<0.33:
        eff_size = 'small'
    elif abs(delta)<0.474:
        eff_size = 'medium'
    else:
        eff_size = 'large'
    return delta,eff_size

Requirement already satisfied: pycoshark in /home/sherbold/.local/lib/python3.6/site-packages (1.2.7)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.6/dist-packages (3.1.0)
Requirement already satisfied: pandas in /usr/local/lib/python3.6/dist-packages (0.24.2)
Requirement already satisfied: seaborn in /usr/local/lib/python3.6/dist-packages (0.9.0)
Requirement already satisfied: openpyxl in /home/sherbold/.local/lib/python3.6/site-packages (2.6.2)
Requirement already satisfied: sklearn in /usr/local/lib/python3.6/dist-packages (0.0)
Requirement already satisfied: xgboost in /usr/local/lib/python3.6/dist-packages (0.90)
Requirement already satisfied: scipy==1.3 in /usr/local/lib/python3.6/dist-packages (1.3.0)
Requirement already satisfied: pymongo in /usr/local/lib/python3.6/dist-packages (from pycoshark) (3.8.0)
Requirement already satisfied: python-dateutil in /usr/local/lib/python3.6/dist-packages (from pycoshark) (2.8.0)
Requirement already satisfied: networkx in /usr/local/lib/python3.6/dist-packages (from pycoshark) (2.3)
Requirement already satisfied: mongoengine in /usr/local/lib/python3.6/dist-packages (from pycoshark) (0.17.0)
Requirement already satisfied: python-Levenshtein in /home/sherbold/.local/lib/python3.6/site-packages (from pycoshark) (0.12.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib) (1.1.0)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.6/dist-packages (from matplotlib) (0.10.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib) (2.4.0)
Requirement already satisfied: numpy>=1.11 in /usr/local/lib/python3.6/dist-packages (from matplotlib) (1.16.4)
Requirement already satisfied: pytz>=2011k in /usr/local/lib/python3.6/dist-packages (from pandas) (2019.1)
Requirement already satisfied: jdcal in /home/sherbold/.local/lib/python3.6/site-packages (from openpyxl) (1.4.1)
Requirement already satisfied: et-xmlfile in /home/sherbold/.local/lib/python3.6/site-packages (from openpyxl) (1.0.1)
Requirement already satisfied: scikit-learn in /usr/local/lib/python3.6/dist-packages (from sklearn) (0.21.3)
Requirement already satisfied: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil->pycoshark) (1.11.0)
Requirement already satisfied: decorator>=4.3.0 in /usr/local/lib/python3.6/dist-packages (from networkx->pycoshark) (4.4.0)
Requirement already satisfied: setuptools in /home/sherbold/.local/lib/python3.6/site-packages (from python-Levenshtein->pycoshark) (42.0.2)
Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.6/dist-packages (from scikit-learn->sklearn) (0.13.2)
WARNING: You are using pip version 19.1.1, however version 20.0.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

Instantiate DB Connection and define projects to analyze

db_name = 'smartshark_rel1'
db_user = ''
db_password = ''
db_host = 'localhost'
db_port = 27017
db_authentication_db = ''
db_ssl = False

uri = create_mongodb_uri_string(db_user, db_password, db_host, db_port, db_authentication_db, db_ssl)

connect(db_name, host=uri, alias='default')

projects = ['ant-ivy', 'archiva', 'calcite', 'cayenne', 'commons-bcel', 'commons-beanutils',
            'commons-codec', 'commons-collections', 'commons-compress', 'commons-configuration',
            'commons-dbcp', 'commons-digester', 'commons-io', 'commons-jcs', 'commons-jexl',
            'commons-lang', 'commons-math', 'commons-net', 'commons-scxml', 
            'commons-validator', 'commons-vfs', 'deltaspike', 'eagle', 'giraph', 'gora', 'jspwiki',
            'knox', 'kylin', 'lens', 'mahout', 'manifoldcf','nutch','opennlp','parquet-mr',
            'santuario-java', 'systemml', 'tika', 'wss4j']

homepages = {
    'ant-ivy':'https://ant.apache.org/ivy/',
    'archiva':'https://archiva.apache.org/',
    'calcite':'https://calcite.apache.org/',
    'cayenne':'https://cayenne.apache.org/',
    'commons-bcel':'https://commons.apache.org/bcel/',
    'commons-beanutils':'https://commons.apache.org/beanutils/',
    'commons-codec':'https://commons.apache.org/codec/',
    'commons-collections':'https://commons.apache.org/collections/',
    'commons-compress':'https://commons.apache.org/compress/',
    'commons-configuration':'https://commons.apache.org/configuration/',
    'commons-dbcp':'https://commons.apache.org/dbcp/',
    'commons-digester':'https://commons.apache.org/digester/',
    'commons-io':'https://commons.apache.org/io/',
    'commons-jcs':'https://commons.apache.org/jcs/',
    'commons-jexl':'https://commons.apache.org/jexl/',
    'commons-lang':'https://commons.apache.org/lang/',
    'commons-math':'https://commons.apache.org/math/',
    'commons-net':'https://commons.apache.org/net/',
    'commons-scxml':'https://commons.apache.org/scxml/',
    'commons-validator':'https://commons.apache.org/validator/',
    'commons-vfs':'https://commons.apache.org/vfs/',
    'deltaspike':'https://deltaspike.apache.org/',
    'eagle':'https://eagle.apache.org/',
    'giraph':'https://giraph.apache.org/',
    'gora':'https://gora.apache.org/',
    'jspwiki':'https://jspwiki.apache.org/',
    'knox':'https://knox.apache.org/',
    'kylin':'http://kylin.apache.org/',
    'lens':'https://lens.apache.org/',
    'mahout':'https://mahout.apache.org/',
    'manifoldcf':'https://manifoldcf.apache.org/',
    'nutch':'https://nutch.apache.org/',
    'opennlp':'https://opennlp.apache.org/',
    'parquet-mr':'https://parquet.apache.org/',
    'santuario-java':'https://santuario.apache.org/',
    'systemml':'https://systemml.apache.org/',
    'tika':'https://tika.apache.org/',
    'wss4j':'https://ws.apache.org/wss4j/'
}

releases = {
    'ant-ivy': [{
        'version': '1.4.1',
        'tag': '1.4.1',
        'revision_hash': 'bfc4a460c2c0730023f3cc0a5e59bb965a94cfe4',
        'corrected_revision': '2e90690057aed8157df0451556709fa4f7bbd6f0'
    },
    {
        'version': '2.0.0',
        'tag': '2.0.0',
        'revision_hash': 'd7c9277d38524c5c971a5ac3e2d23b7bc0d1221c'
    },
    {
        'version': '2.1.0',
        'tag': '2.1.0',
        'revision_hash': '14fa21b35d069eb498a2aff784d59b18679d8ee3'
    },
    {
        'version': '2.2.0',
        'tag': '2.2.0',
        'revision_hash': 'd27f2b35ccb22dd242795c787a7010c2feef13b0'
    },
    {
        'version': '2.3.0',
        'tag': '2.3.0',
        'revision_hash': 'a0ad1bfa81c615443b4916e654fa7087c504cd40'
    },
    {
        'version': '2.4.0',
        'tag': '2.4.0',
        'revision_hash': '0b9db35ee7a94a719e538b04122b86cb997f3a17'
    }],
    'archiva': [{
        'version': '1.0',
        'tag': 'archiva-1.0',
        'revision_hash': '34fb315ea16071c903a6a428706f179e478413d3',
        'corrected_revision': '9ca7fd21ad15fa130d9e1e569da1deba3a1bc23d'
    },{
        'version': '1.1',
        'tag': 'archiva-1.1',
        'revision_hash': 'b3e2ecd39c190116624010587d78de9071008516'
    },{
        'version': '1.2',
        'tag': 'archiva-1.2',
        'revision_hash': 'b27c14eae1314e088ff6872c34bb4b6d40b1df1f'
    },{
        'version': '1.3',
        'tag': 'archiva-1.3',
        'revision_hash': 'c1cfcfc5d39cb1087a6d345732512718e6741ff9'
    },{
        'version': '2.0.0',
        'tag': 'archiva-2.0.0',
        'revision_hash': 'e33eddc47b2ab5096ed2e7f285a9d745d33d86b0'
    },{
        'version': '2.1.0',
        'tag': 'archiva-2.1.0',
        'revision_hash': '93eafa089c11117a1f8a1fdd5752b67386d30b76'
    },{
        'version': '2.2.0',
        'tag': 'archiva-2.2.0',
        'revision_hash': 'e5500cce83acba6a3a756b38620c510f3d00263e'
    }],
    'calcite': [{
        'version': '1.0.0',
        'tag': 'calcite-1.0.0-incubating',
        'revision_hash': '2dd83f21784c4366df84a3aaa62a8b7b0e36d443'
    },{
        'version': '1.1.0',
        'tag': 'calcite-1.1.0-incubating',
        'revision_hash': 'f10ea367ff1cc25497f50f149ef0c91b3ae03031'
    },{
        'version': '1.2.0',
        'tag': 'calcite-1.2.0-incubating',
        'revision_hash': 'd60f2aa3aa9ce7cda7c4f6986af5729d50a28679'
    },{
        'version': '1.3.0',
        'tag': 'calcite-1.3.0-incubating',
        'revision_hash': '495f1859f84b41ae70b2099c3d15c696a49a5100'
    },{
        'version': '1.4.0',
        'tag': 'calcite-1.4.0-incubating',
        'revision_hash': '0c0c203daec56c05b6c75fa3896c8af19844df68'
    },{
        'version': '1.5.0',
        'tag': 'calcite-1.5.0',
        'revision_hash': 'ba6e43c6983ca92d8ce32a693776dbe73f19e0dc'
    },{
        'version': '1.6.0',
        'tag': 'calcite-1.6.0',
        'revision_hash': 'c4d346b0a413a1a62e028dd3be40071523203a58'
    },{
        'version': '1.7.0',
        'tag': 'calcite-1.7.0',
        'revision_hash': '8eebfc6d169421509e5d4792a18917fcb6c44efb'
    },{
        'version': '1.8.0',
        'tag': 'calcite-avatica-1.8.0',
        'revision_hash': 'ddfaf079a8a18be63f042069dd56c50dec4d2066'
    },{
        'version': '1.9.0',
        'tag': 'calcite-1.9.0',
        'revision_hash': '08c56b158ffcfcf205a919cc9fff77a692e649f6'
    },{
        'version': '1.10.0',
        'tag': 'calcite-1.10.0',
        'revision_hash': '448dd85832819b4d22682ffe7908d8058da7c778'
    },{
        'version': '1.11.0',
        'tag': 'calcite-1.11.0',
        'revision_hash': 'f8ba670de4e283d1532f288de53fdb67fa4dea67'
    },{
        'version': '1.12.0',
        'tag': 'calcite-1.12.0',
        'revision_hash': 'ea7ace18cdc26027be690b00f1fcc191aaa87e45'
    },{
        'version': '1.13.0',
        'tag': 'calcite-1.13.0',
        'revision_hash': '54b9823e7ca313bf195c19b7d98f1a06b342cf12'
    },{
        'version': '1.14.0',
        'tag': 'calcite-1.14.0',
        'revision_hash': '7426eef84e7b2ca38cfbf0d47ba53a8abd4d9e38'
    },{
        'version': '1.15.0',
        'tag': 'calcite-1.15.0',
        'revision_hash': 'a2aa00e5b276a30e301ecbf0953e2cbf3a51b605'
    }],
    'cayenne': [{
        'version': '3.0.0',
        'tag': '3.0-final',
        'revision_hash': 'fe8710bdc1316f59fdfde66ea82706995316f725'
    },{
        'version': '3.1.0',
        'tag': 'cayenne-parent-3.1',
        'revision_hash': '63cbf3c98f6e4be000542aa7788875da0939da13'
    }],
    'commons-bcel': [{
        'version': '5.0',
        'tag': 'BCEL_5_0',
        'revision_hash': '4f2b59597e8463d85bad90b48c8d876815291f4f',
        'corrected_revision': '35dc5fc1d4ac353d4c40fef826f8534ba2d1974f'
    },{
        'version': '5.1',
        'tag': 'BCEL_5_1',
        'revision_hash': '0fa7513ef6a2ecdd7e14701fed33f8dcdaed0077',
        'corrected_revision': 'a55dd020f08ed55cbb28834d586e81a13e2db7c0'
    },{
        'version': '5.2',
        'tag': 'BCEL_5_2',
        'revision_hash': 'b546dfdaaaee9eec23c4d0bbe1454a2d6edcaca2',
        'corrected_revision': '1aa7e878e3e3c38a03e154a42cc77f00658c9742'
    },{
        'version': '6.0',
        'tag': 'BCEL_6_0',
        'revision_hash': '647c723ba1262e1ffce520524692b366a7fde45a'
    },{
        'version': '6.1',
        'tag': 'BCEL_6_1',
        'revision_hash': 'f70742fde892c2fac4f0862f8f0b00e121f7d16e'
    },{
        'version': '6.2',
        'tag': 'BCEL_6_2',
        'revision_hash': '893d9bbcdbd5ce764db1a38eccd73af150e5d34d'
    }],
    'commons-beanutils': [{
        'version': '1.0',
        'tag': 'BEANUTILS_1_0',
        'revision_hash': '41f492e7eaed0a9d3f32ee02ca8aa284472d526a',
        'corrected_revision': 'd783d4d1c95895cd1e556180c53c88ab14f8af7a'
    },{
        'version': '1.1',
        'tag': 'BEANUTILS_1_1',
        'revision_hash': 'f4dd0a42d9d642dfc231c1290044fafd281c3fa3',
        'corrected_revision': '68557b16b23bbdd6d13d28b966953efbf1cb77c1'
    },{
        'version': '1.2',
        'tag': 'BEANUTILS_1_2',
        'revision_hash': 'a55d81e2b2b47d73c8dbf5e1de7f0394322290bd',
        'corrected_revision': 'e9fb55e779563377644672db318ef4dc79995cbd'
    },{
        'version': '1.3',
        'tag': 'BEANUTILS_1_3',
        'revision_hash': '227e5d2d4dd00ebbdf49004447e016431648d735',
        'corrected_revision': '297bfe3879a06e63096041e9fd59408b831270e6'
    },{
        'version': '1.4',
        'tag': 'BEANUTILS_1_4',
        'revision_hash': 'dc0f63ae7af19f7897e03d746038882c92d63c4a',
        'corrected_revision': '61b3a30c12c3c734f5fe3cf3a75a7924a860d123'
    },{
        'version': '1.5',
        'tag': 'BEANUTILS_1_5',
        'revision_hash': 'b72aa9e97b85714aad2951b9ff4e2be90a2940a7',
        'corrected_revision': '90f514fc48dfd7e84f7ab38edbe30a581703533b'
    },{
        'version': '1.6',
        'tag': 'BEANUTILS_1_6',
        'revision_hash': '2bc65c91d0c8ac5fac03c41625d10ddee6af7f27',
        'corrected_revision': '5a2d26622b9322a84d59bb48fbcfbb4e0f8b93ca'
    },{
        'version': '1.7.0',
        'tag': 'BEANUTILS_1_7_0',
        'revision_hash': 'cbdadd2011983734383a785466cdc16c06f99fe9',
        'corrected_revision': 'fd5bf514ca3ce36e75a5d9595252fc1a45fc52d8'
    },{
        'version': '1.8.0',
        'tag': 'BEANUTILS_1_8_0',
        'revision_hash': '8fb2285efd72d72a359d9012216260a75bec09b7'
    },{
        'version': '1.9.0',
        'tag': 'BEANUTILS_1_9_0',
        'revision_hash': '9c047b74f426e40b28f497c39572abcf48263650'
    }], 
    'commons-codec':[{
        'version': '1.1',
        'tag': 'CODEC_1_1',
        'revision_hash': 'b4e4eee0644a211c209c21a2056b6227bfee8ccc',
        'corrected_revision': '42d9bf996a0b86b54764c48a2b7859ae77f80dd2'
    },{
        'version': '1.2',
        'tag': 'CODEC_1_2',
        'revision_hash': '2c0905a09d5d04db7e572172bf2ead04293477a2',
        'corrected_revision': 'dea4d55afaa6d6afd4d01b192106148bf20b3f8b'
    },{
        'version': '1.3',
        'tag': 'CODEC_1_3',
        'revision_hash': '1af8f98c3bd96501d558bbe980a2a9f7eb188367',
        'corrected_revision': 'dfad8deb12895a12dd5967dbd99e49707fc3d261'
    },{
        'version': '1.4',
        'tag': 'CODEC_1_4',
        'revision_hash': 'f375d169f8e030169d63d9da00bb14ae2da2ac34'
    },{
        'version': '1.5',
        'tag': 'commons-codec-1.5',
        'revision_hash': '2ce6791741d501bb2e332f6c23ed319d9cf69368'
    },{
        'version': '1.6',
        'tag': '1_6',
        'revision_hash': '23f2aa4fb66aa5b026184ed17b6ad070da183794'
    },{
        'version': '1.7',
        'tag': '1.7',
        'revision_hash': '183aa006e80e5c5a4c8c215d104be39cd77f3336'
    },{
        'version': '1.8',
        'tag': '1.8',
        'revision_hash': 'aa988a92ce3317077abd4e8414ef501b382d7fc0'
    },{
        'version': '1.9',
        'tag': '1.9',
        'revision_hash': '52cfc73491e417366c77c226a7b01b40f7677f9a'
    },{
        'version': '1.10',
        'tag': '1.10',
        'revision_hash': 'e9da3d16ae67f2940a0bbdf982ecec19a0481981'
    },{
        'version': '1.11',
        'tag': 'commons-codec-1.11',
        'revision_hash': '9ceef2231549e95f7868266f38b0c7e1a0a40ed4'
    }], 
    'commons-collections':[{
        'version': '1.0',
        'tag': 'collections-1.0',
        'revision_hash': '5e62ebe3c43f62ca5b58d9deac70154991ce0924',
        'corrected_revision': '5a75bf8e7abd6428e1e88bd9f794bf39519c605e'
    },{
        'version': '2.0',
        'tag': 'collections-2.0',
        'revision_hash': '658dea9545701b813d9bebee1171c508392e472c',
        'corrected_revision': '80c60d68df58af5744b1a521cd68777e62d12312'
    },{
        'version': '2.1',
        'tag': 'collections-2.1',
        'revision_hash': '88186e85e517ff14f5f77b92f03a4f0b01939e36',
        'corrected_revision': '7c223490bc5d82a058973a95f02ac3631ac7d1fa'
    },{
        'version': '3.0',
        'tag': 'collections-3.0',
        'revision_hash': 'ebdf7649e49d164d33c98cc6be4fbfacbbfc4220',
        'corrected_revision': 'b730495d6748604b5a10ab8f6bf4302ff710e1f1'
    },{
        'version': '3.1',
        'tag': 'collections-3.1',
        'revision_hash': '326a1c172f5857709299bc77bd73402352214bbf',
        'corrected_revision': 'a472b7f2bd9f185fb386f9fe33170f2c5de87543'
    },{
        'version': '3.2',
        'tag': 'collections-3.2',
        'revision_hash': '8792072a213366605161b2f6e315024711da7eee',
        'corrected_revision': '4b6411235e6953ad65b4cb0e16ff6ad09581247d'
    },{
        'version': '3.3',
        'tag': 'collections-3.3',
        'revision_hash': '1d0b9abb31faf211192c9866e3ec5dfe86c41e1d'
    },{
        'version': '4.0',
        'tag': 'collections-4.0',
        'revision_hash': 'db189926f7415b9866e76cd8123e40c09c1cc67e'
    },{
        'version': '4.1',
        'tag': 'collections-4.1',
        'revision_hash': 'cb157163d7543f942a1391f3ef752ebea1e1b349'
    }],
    'commons-compress': [{
        'version': '1.0',
        'tag': '1.0',
        'revision_hash': '4b5962d0fc3082e456a7dd249fc25e5aa7b1d350'
    },{
        'version': '1.1',
        'tag': 'commons-compress-1.1',
        'revision_hash': '9ee24a3d6a84c7e09abb5a9b4940bc91e2ab23d6'
    },{
        'version': '1.2',
        'tag': 'COMPRESS_1.2',
        'revision_hash': 'dc733d1ec84c72d68d8cbedc8d4d301eaf975cbf'
    },{
        'version': '1.3',
        'tag': 'COMPRESS_1.3',
        'revision_hash': '0874320927ef9427cf5da5973935a1d17201cdf7'
    },{
        'version': '1.4',
        'tag': 'COMPRESS_1.4',
        'revision_hash': '6e63432a2a15c8b9b7327edb5cd141661890e0e9'
    },{
        'version': '1.5',
        'tag': '1.5',
        'revision_hash': '163c4eee17c041cb7c1ccfedd983816038cc1eca'
    },{
        'version': '1.6',
        'tag': '1.6',
        'revision_hash': 'c49e865f4eabd47e264063499e27e12eb549feb1'
    },{
        'version': '1.7',
        'tag': '1.7',
        'revision_hash': '4a4ba6bfdadc792c59a67935b6c4fe6ee8bfd325'
    },{
        'version': '1.8',
        'tag': '1.8',
        'revision_hash': '5048d93858a3a7d976ace35f83868215ce5dfe2c'
    },{
        'version': '1.9',
        'tag': '1.9',
        'revision_hash': '3a5c8403d892ff1bc245f5ccc6a00bbc860529eb'
    },{
        'version': '1.10',
        'tag': '1.10',
        'revision_hash': '121b7e00017b95d1498c4f511ceb4dd21811257a'
    },{
        'version': '1.11',
        'tag': '1.11',
        'revision_hash': 'ff38bf57378c7cae8617b3c8df692a5ffbe7b83f'
    },{
        'version': '1.12',
        'tag': '1.12',
        'revision_hash': '3623ee7dad07c74722baeeffcec6770015f915cf'
    },{
        'version': '1.13',
        'tag': '1.13',
        'revision_hash': '45438471726fc5005a41cefecbf62d4d26eb15b2'
    },{
        'version': '1.14',
        'tag': '1.14',
        'revision_hash': 'dd7c7702bf51886aa8bd88b24d98619f310fbeda'
    },{
        'version': '1.15',
        'tag': '1.15',
        'revision_hash': '01b06d5ef5c5ac3bd651bedcfec7433231cea371'
    },{
        'version': '1.16',
        'tag': '1.16',
        'revision_hash': 'c6e38999260bfa486b3213208b809bd543ce18bf'
    }],
    'commons-configuration': [{
        'version': '1.0',
        'tag': 'CONFIGURATION_1_0',
        'revision_hash': '6c460662af216ac3777b03b9d641451486bc238d',
        'corrected_revision': 'a59c5f1b1ee33ddaa2dd719f6fd1a756549f7aca'
    },{
        'version': '1.1',
        'tag': 'CONFIGURATION_1_1',
        'revision_hash': '3480ec5dd93416d3cd804a9d9463aba93e118af0',
        'corrected_revision': 'f50bb53f44c0ac757e1c47258f7c09bf2101b4c9'
    },{
        'version': '1.2',
        'tag': 'CONFIGURATION_1_2',
        'revision_hash': '4c9cc64655258b6af313a593f981de4c94ccc28e',
        'corrected_revision': 'e51302b158a0337e8728d5b94ea02933065c0ffa'
    },{
        'version': '1.3',
        'tag': 'CONFIGURATION_1_3',
        'revision_hash': '663be0ea1296075b919b41c7e3af4c10054c84aa',
        'corrected_revision': '290009d56e2e5abec4e872726fe2b64cfc2d36fa'
    },{
        'version': '1.4',
        'tag': 'CONFIGURATION_1_4',
        'revision_hash': '2ca32d357e00554c999837c52319e8122fcb26d9',
        'corrected_revision': '9724d33bb06a15cf2fc2152d13f9a457b2b07758'
    },{
        'version': '1.5',
        'tag': 'CONFIGURATION_1_5',
        'revision_hash': '6925324fa47ea463e8e7f7833dd2acc4c57ff170'
    },{
        'version': '1.6',
        'tag': 'CONFIGURATION_1_6',
        'revision_hash': '46a1e1cf462d7f3cb89015287cb5cbe67a1ce22b'
    },{
        'version': '1.7',
        'tag': 'CONFIGURATION_1_7',
        'revision_hash': '3bbf2a42b3db2878651a0968574ec375658f8590'
    },{
        'version': '1.8',
        'tag': 'CONFIGURATION_1_8',
        'revision_hash': 'fa910376536ea5a906d3a49a9d87603b11cf5133'
    },{
        'version': '1.9',
        'tag': 'CONFIGURATION_1_9',
        'revision_hash': '93f43a4a6dcc7352888a4853e2178871c3c18523'
    },{
        'version': '1.10',
        'tag': 'CONFIGURATION_1_10',
        'revision_hash': '3c212681e410097cf38ee376540da585a8938923'
    },{
        'version': '2.0',
        'tag': 'CONFIGURATION_2_0',
        'revision_hash': '25c16206e0452a3b68abf4158c4d8f2ea38305da'
    },{
        'version': '2.1',
        'tag': 'CONFIGURATION_2_1',
        'revision_hash': 'e814fe00bb51256386dd78f3f926aa30d8de6a7c'
    },{
        'version': '2.2',
        'tag': 'CONFIGURATION_2_2',
        'revision_hash': '137997974e0ebd3acd047243f46ddf3b90fa0160'
    }],
    'commons-dbcp': [{
        'version': '1.0',
        'tag': 'DBCP_1_0',
        'revision_hash': '68948e66e5b230a256cfb20d1775f7d0d02ac888',
        'corrected_revision': 'ce2d4f6f15b2009f42ec9f66c356edaebc7ad3a2'
    },{
        'version': '1.1',
        'tag': 'DBCP_1_1',
        'revision_hash': '5d76261591d40987acdad7f35b8397e2f63c7916',
        'corrected_revision': '3f7aedad5c09ae61a47c397b50055d935482addf'
    },{
        'version': '1.2',
        'tag': 'DBCP_1_2',
        'revision_hash': '22e7e052373ef9f687064aa307b26aa6ffd46c48',
        'corrected_revision': 'f3bc5838c5e6f173f7665edf00ab7a60b430fa21'
    },{
        'version': '1.3',
        'tag': 'DBCP_1_3',
        'revision_hash': '8162457b6f482d17b73e92710e2fa19b536f1c13',
        'corrected_revision': '306023633625bb8b0d3150ab1d82979f91dca4f9'
    },{
        'version': '1.4',
        'tag': 'DBCP_1_4',
        'revision_hash': 'c08260fd38cc220a8e9c70fd105902f2b5b043b9',
        'corrected_revision': 'ada08866e9390138eb40ebb1a36f4e7fd9bacff9'
    },{
        'version': '2.0',
        'tag': 'DBCP_2_0',
        'revision_hash': '583aaf88451485e75d34c7417a96318e27720e74'
    },{
        'version': '2.1',
        'tag': 'DBCP_2_1',
        'revision_hash': 'd8aa6baa6f1c2ab5281f7097fd11d0d840c4c217'
    },{
        'version': '2.2.0',
        'tag': 'DBCP_2_2_0',
        'revision_hash': 'c5b86d834a53b4891487e62671873d054a6f38d8'
    },{
        'version': '2.3.0',
        'tag': 'DBCP_2_3_0',
        'revision_hash': '6040b67b78d96d05c282a75bb250fa6ea31623dc'
    },{
        'version': '2.4.0',
        'tag': 'commons-dbcp-2.4.0',
        'revision_hash': 'd7aa662fbbb99e536ae28c47d0c4e1d51e39d5b9'
    },{
        'version': '2.5.0',
        'tag': 'commons-dbcp-2.5.0',
        'revision_hash': '334784e3043cadaeaa626f792c3f62ac7b697927'
    }],
    'commons-digester': [{
        'version': '1.0',
        'tag': 'DIGESTER_1_0',
        'revision_hash': '522f50e75b344ebd22343d9b5442f70249e8cabf',
        'corrected_revision': '79fecb6e7fcdac7f854332c07f7898e7a3f56d62'
    },{
        'version': '1.1',
        'tag': 'DIGESTER_1_1',
        'revision_hash': '48bf1c84c58429c545f00446a31d938f1f6b3f48',
        'corrected_revision': '2b782a5a4a11ad4c3ddccf1018a8f2185877c514'
    },{
        'version': '1.2',
        'tag': 'DIGESTER_1_2',
        'revision_hash': 'e70e9c320edcb261f535493f358aa04e8c46c769',
        'corrected_revision': '21255e428189e5c54205d1623077fae3916db9c7'
    },{
        'version': '1.3',
        'tag': 'DIGESTER_1_3',
        'revision_hash': '7533ab004572c33ed076c1137839779c6a00d9ad',
        'corrected_revision': 'db04a200c7cfdd181907c9d4647ecc4536fb3ff7'
    },{
        'version': '1.4',
        'tag': 'DIGESTER_1_4',
        'revision_hash': 'b8f283f82988e616047fcb0b64700f14f7eae3d7',
        'corrected_revision': '4fb91de8c37facd77af6cb80aca4c0e6b99e7884'
    },{
        'version': '1.5',
        'tag': 'DIGESTER_1_5',
        'revision_hash': '4b778ef1af3ebb8ced1687fd904d2bcf33fcca7e',
        'corrected_revision': '05702b56930b9844daa3dea4c2ce56c8341a7000'
    },{
        'version': '1.6',
        'tag': 'DIGESTER_1_6',
        'revision_hash': '619c9da5b28984d736b5d1d12828d17e5c164cab',
        'corrected_revision': 'a9599becb81fb374b48584794cb5890adcf4dc18'
    },{
        'version': '1.7',
        'tag': 'DIGESTER_1_7',
        'revision_hash': '6b775f843092d51f84dc33233ff72d9220ebe3d5',
        'corrected_revision': 'd6e524a56af373dabea5d048d756086a0bf8f255'
    },{
        'version': '1.8',
        'tag': 'DIGESTER_1_8',
        'revision_hash': '8ff0834cbed4fb97f9cf1426993b50342983adeb',
        'corrected_revision': '85ada11c674ef4cb13bf39c10c44a41845055407'
    },{
        'version': '2.0',
        'tag': 'DIGESTER_2_0',
        'revision_hash': '9fb541b5c5f9daa2fa6677914727cc3953515488'
    },{
        'version': '2.1',
        'tag': 'DIGESTER_2_1',
        'revision_hash': 'e2a59fe9a72cf9a2119157cd6859b09230835ba9'
    },{
        'version': '3.0',
        'tag': 'DIGESTER3_3_0',
        'revision_hash': 'ea9f1e1e3bc9c934e70557e8423b2b2c77b4bbfe'
    },{
        'version': '3.1',
        'tag': 'DIGESTER3_3_1',
        'revision_hash': 'fad107b8d66556f915e75b4d0f704dad409b054a'
    },{
        'version': '3.2',
        'tag': 'DIGESTER3_3_2',
        'revision_hash': 'ec75748096c8639be7b4998d99c37bb2e3a9c206'
    }],
    'commons-io': [{
        'version': '1.0',
        'tag': 'IO_1_0',
        'revision_hash': 'c2970b4a557e45a1f6e3cf85b5d163a5de9f80dd',
        'corrected_revision': '5e49391360f799bb42274ce42266c17a484b8f6f'
    },{
        'version': '1.1',
        'tag': 'IO_1_1',
        'revision_hash': '8d911c48d70c93190c8007d915162040a05d89fe',
        'corrected_revision': '1069e5bf6ec3e40f7a9440613615c422da3eb9ac'
    },{
        'version': '1.2',
        'tag': 'IO_1_2',
        'revision_hash': 'd65ff7fec954c84deb0c3b22731b0ee8f395805e',
        'corrected_revision': '1d107c54c400bc76540dd5fcb14207ae56cbf9ad'
    },{
        'version': '1.3',
        'tag': 'IO_1_3',
        'revision_hash': '5a6f8945b08bcb6a4c47cc148a8cbb2f42271880',
        'corrected_revision': 'fb9954d8812defef2ade48bff7003677167e0626'
    },{
        'version': '1.4',
        'tag': 'commons-io-1.4',
        'revision_hash': '6923a01ae1edf0510dd257f72867c282224ee23f'
    },{
        'version': '2.0',
        'tag': 'commons-io-2.0',
        'revision_hash': '27add961ae6f94fa20124d24486a6e5fb47587c9'
    },{
        'version': '2.1',
        'tag': 'commons-io-2.1',
        'revision_hash': '0e68a33a9b8f27e592f79a563a5c9a9644c8b21d'
    },{
        'version': '2.2',
        'tag': '2.2',
        'revision_hash': '5d4d09928e154d083f396a1555da72dfe9d877d1'
    },{
        'version': '2.3',
        'tag': '2.3',
        'revision_hash': 'c07589baa429c2ab306466643cef3dec31d1788b'
    },{
        'version': '2.4',
        'tag': '2.4',
        'revision_hash': 'e36d53170875d26d59ca94bd376bf40bc5690ee6'
    },{
        'version': '2.5',
        'tag': 'commons-io-2.5',
        'revision_hash': '4077158829de92987367d3149e4ba71356bb5390'
    }],
    'commons-jcs': [{
        'version': '1.0',
        'tag': 'JCS_1_0',
        'revision_hash': 'e2349a328c0d17ac6add6e28ea56a16cb640f27a',
        'corrected_revision': '574d315df38c45aefd638c5d22a9d7442635a7d6'
    },{
        'version': '1.1',
        'tag': 'jcs_1_1_dev',
        'revision_hash': '219dece048757b3f04e8d972f40c2c1273b3c54d',
        'corrected_revision': 'f9d4511f36c00fdbe8bde20f5fb6a2805ddf985a'
    },{
        'version': '1.3',
        'tag': 'JCS_1_3',
        'revision_hash': '851ec7399ea430c9be9ec4c8914d839efdd92b10',
        'corrected_revision': '2b4f84253690f2e23337a09be520a76248252418'
    },{
        'version': '2.0',
        'tag': 'commons-jcs-2.0',
        'revision_hash': '8a3cab728832eb739a9697d3a5d2e0184d511511'
    },{
        'version': '2.1',
        'tag': 'commons-jcs-2.1',
        'revision_hash': 'efe68fda0c1c4ef57171e76ef10ca118b96e191a'
    },{
        'version': '2.2',
        'tag': 'commons-jcs-2.2',
        'revision_hash': '5a9246493fb9c60eb81ba4eab7636172e438d46c'
    }],
    'commons-jexl': [{
        'version': '1.0',
        'tag': 'COMMONS_JEXL-1_0',
        'revision_hash': 'd4e5fb3c1e1d09641e7d7fc5418cde5e863a56e8',
        'corrected_revision': 'f3ec15c50f25fcbbb6248c18217d9fe76d1a5f61'
    },{
        'version': '1.1',
        'tag': 'COMMONS_JEXL-1_1',
        'revision_hash': 'b9feb29d0d3abbe0d441685f04014036b71e9106',
        'corrected_revision': '6160d320c4219fd25b2ffd7bb136a1c4a1c53e6c'
    },{
        'version': '2.0',
        'tag': 'COMMONS_JEXL_2_0',
        'revision_hash': '1abdc102c66da73706b2aef331cb57e08f595c04',
        'corrected_revision': 'b9dc71c16461ce497e7ba4b2439983c4d756f0af'
    },{
        'version': '2.1',
        'tag': 'commons-jexl-2.1',
        'revision_hash': '9c630b1b469c6d3badb43c12846d341362994eca'
    },{
        'version': '3.0',
        'tag': 'COMMONS_JEXL_3_0',
        'revision_hash': 'de6c4f3b00af4430f535fcb7833c480d9093fd35'
    },{
        'version': '3.1',
        'tag': 'COMMONS_JEXL_3_1',
        'revision_hash': 'af298b37d30f8cd1e776b23db9bb85c353b002ee'
    }],
    'commons-lang': [{
        'version': '1.0',
        'tag': 'LANG_1_0',
        'revision_hash': '2d274d7cedc148ee3fa8cf4f48904dfd82172bda',
        'corrected_revision': '5091f3b04fbacadc0bcff6aaa808615a2c1f05a3'
    },{
        'version': '2.0',
        'tag': 'LANG_2_0',
        'revision_hash': 'da5d64110a8b7182d04359958f40fadfbe1ed6ff',
        'corrected_revision': '70e09f61dff742883b6b7c6aecb7023cfdabe0bf'
    },{
        'version': '2.1',
        'tag': 'LANG_2_1',
        'revision_hash': '81f1077a922523a1f6a76376215a1b3de1e49b67',
        'corrected_revision': '27d3960eb1d30bc7e4faf03c26b65f788142cc90'
    },{
        'version': '2.2',
        'tag': 'LANG_2_2',
        'revision_hash': 'bdf2ede50ed073d91389be55ec3a9cff565c58a5',
        'corrected_revision': 'f4637c05715359d24979e7eedb3c18dd4b31b599'
    },{
        'version': '2.3',
        'tag': 'LANG_2_3',
        'revision_hash': '4e55808486a0227a72c70350a76d33660cc94af8',
        'corrected_revision': '1363d79e631183ad6c8be6dbe95e895236173d85'
    },{
        'version': '2.4',
        'tag': 'LANG_2_4',
        'revision_hash': 'aa9727e15966bac899aa18733ef85c4aded19f0c'
    },{
        'version': '2.5',
        'tag': 'LANG_2_5',
        'revision_hash': '7c4386cb877d99d733b531f4fba2a328a3951767'
    },{
        'version': '2.6',
        'tag': 'LANG_2_6',
        'revision_hash': '0fe6a434f20b45c027badd89bd393dd4085cf1f2'
    },{
        'version': '3.0',
        'tag': 'LANG_3_0',
        'revision_hash': 'c21484b730221bc87ca26553155350292aa30f0d'
    },{
        'version': '3.1',
        'tag': 'LANG_3_1',
        'revision_hash': '915946c9dcc80a6a16745b86540b82b2ddba1720'
    },{
        'version': '3.2',
        'tag': 'LANG_3_2',
        'revision_hash': '0c14a39be2b8a8e9461921994033310fd567e8cb'
    },{
        'version': '3.3',
        'tag': 'LANG_3_3',
        'revision_hash': '37f8fe84d934a92fe784c62e706b7aacf6fa4f97'
    },{
        'version': '3.4',
        'tag': 'LANG_3_4',
        'revision_hash': '4777c3a5e4291af2420db57d008152c70c4a8f24'
    },{
        'version': '3.5',
        'tag': 'LANG_3_5',
        'revision_hash': '36f98d87b24c2f542b02abbf6ec1ee742f1b158b'
    },{
        'version': '3.6',
        'tag': 'LANG_3_6',
        'revision_hash': '09043bfa6f1f9ebb946c7b159735c83259e3a89f'
    },{
        'version': '3.7',
        'tag': 'LANG_3_7',
        'revision_hash': '425d8085cfcaab5a78bf0632f9ae77b7e9127cf8'
    }],
    'commons-math': [{
        'version': '1.0',
        'tag': 'MATH_1_0',
        'revision_hash': '96691abe5ed4a0f67b253cd2f6f789e868a77b22',
        'corrected_revision': '746ced5ff5d23885fbaff6664d33616941328488'
    },{
        'version': '1.1',
        'tag': 'MATH_1_1',
        'revision_hash': 'bb50284d69ce651dc12e7acea2fe6e2db792874a',
        'corrected_revision': '2e4407d19cef6d91aac716d5d08f5140b13590f7'
    },{
        'version': '1.2',
        'tag': 'MATH_1_2',
        'revision_hash': '0c602508d1591e212c55912a9066baa45e126957'
    },{
        'version': '2.0',
        'tag': 'MATH_2_0',
        'revision_hash': '9f26e58bf9a73e1375b17dfdc2c5f395e4ffbdb6'
    },{
        'version': '2.1',
        'tag': 'MATH_2_1',
        'revision_hash': '7d588288e35b38dd55e7bedb767d1b0d80096294'
    },{
        'version': '2.2',
        'tag': 'MATH_2_2',
        'revision_hash': '98ebcc6bb298e62e35f895d59fca51b8c4c2d443'
    },{
        'version': '3.0',
        'tag': 'MATH_3_0',
        'revision_hash': '174f17d5d4b8251c4ed2600947e69b736ace5877'
    },{
        'version': '3.1',
        'tag': 'MATH_3_1',
        'revision_hash': 'b090abb2cf03cf4d0afc6fbadc7cff62c35077cd'
    },{
        'version': '3.2',
        'tag': 'MATH_3_2',
        'revision_hash': '7ec508828f62c2a3df31cde2a2f3aa32c5135fdb'
    },{
        'version': '3.3',
        'tag': 'MATH_3_3',
        'revision_hash': '38a48e20867f4cd91b7540be4bbba1e9dc7a8b02'
    },{
        'version': '3.4',
        'tag': 'MATH_3_4',
        'revision_hash': 'befd8ebd96b8ef5a06b59dccb22bd55064e31c34'
    },{
        'version': '3.5',
        'tag': 'MATH_3_5',
        'revision_hash': 'b3c5dae8f253fcb4484e5cd3cc5662587803efc2'
    },{
        'version': '3.6',
        'tag': 'MATH_3_6',
        'revision_hash': '95a9d35e77f70ffc9bd5143880c236a760b42005'
    }],
    'commons-net': [{
        'version': '1.0.0',
        'tag': 'NET_1_0_0',
        'revision_hash': 'edb0c4426ab7dc930c29a5eafd3f7ac76c4cd9bf',
        'corrected_revision': '581fad6c1f9bddfe6a61e4fdb2dbc0b6a2709351'
    },{
        'version': '1.1.0',
        'tag': 'NET_1_1_0',
        'revision_hash': 'ce903621374d97ab641ca6e0f07d236236a50970',
        'corrected_revision': 'ace3e9bdc698e2c9c95e7c9103fabb32904896b2'
    },{
        'version': '1.2.0',
        'tag': 'NET_1_2_0',
        'revision_hash': '505a39a18df1507ad63af7a636eaa25b3b793721',
        'corrected_revision': 'c87a7c22668257b4d6073197f53dbd4f08eff631'
    },{
        'version': '1.3.0',
        'tag': 'NET_1_3_0',
        'revision_hash': '4f8f123cda3bebdf22095e80826cf4af5781ceb9',
        'corrected_revision': 'f1481c0e35daf8e45d73698a6f6f4c6651e8dddc'
    },{
        'version': '1.4.0',
        'tag': 'NET_1_4_0',
        'revision_hash': '380f6f7a3bca6819b88656daa4c37f770958d2ac',
        'corrected_revision': '9189ef3ccc6a8c3d6a09c2c25379f98fd315aa7f'
    },{
        'version': '2.0',
        'tag': 'NET_2_0',
        'revision_hash': '32c972d4a86302c8df0be318d9380e8989ea6972'
    },{
        'version': '2.1',
        'tag': 'commons-net-2.1',
        'revision_hash': '5b81ff5f9a6ed63b0e0d67bea7a1d9c909184bdd'
    },{
        'version': '2.2',
        'tag': 'NET_2_2',
        'revision_hash': '51829d33e280cd335b0d48db9e290a82ae574496'
    },{
        'version': '3.0',
        'tag': 'NET_3_0',
        'revision_hash': '7b6def4cb8da5ddef57efdce7f415f7aee1b49e7'
    },{
        'version': '3.1',
        'tag': 'NET_3_1',
        'revision_hash': 'e2fa6168cdbe958e56d54456e7aa1ecc4b13e4df'
    },{
        'version': '3.2',
        'tag': 'NET_3_2',
        'revision_hash': '9f139947e62438f408eb45fcbf8543eaf0d2b922'
    },{
        'version': '3.3',
        'tag': 'NET_3_3',
        'revision_hash': 'f8c2dabb0e8cb62c86912dc61fe2f68106d8b917'
    },{
        'version': '3.4',
        'tag': 'NET_3_4',
        'revision_hash': '74a2282b7e4c6905581f4f1b5a2ec412310cd5e7'
    },{
        'version': '3.5',
        'tag': 'NET_3_5',
        'revision_hash': '943a587727079179b1c42af26b788f1cb5d76643'
    },{
        'version': '3.6',
        'tag': 'NET_3_6',
        'revision_hash': '163fe46c019f5184016207c247cdff30ee740ecc'
    }],
    'commons-scxml': [{
        'version': '0.5',
        'tag': 'SCXML_0_5',
        'revision_hash': 'c5425058630297b6fa8873586b601389e2b80c4b',
        'corrected_revision': 'dcd981be25857db62bd18f43a640c51dedbae65c'
    },{
        'version': '0.6',
        'tag': 'SCXML_0_6',
        'revision_hash': '9407377dad722a129ff7688cce553e103a434212',
        'corrected_revision': '47c4d6f18410985c82bf343fdc50803aa0258f0e'
    },{
        'version': '0.7',
        'tag': 'SCXML_0_7',
        'revision_hash': '439d8722769e199e0475a2e400178594f8b980dc'
    },{
        'version': '0.8',
        'tag': 'SCXML_0_8',
        'revision_hash': '54a019f81af349cb5279c2109bb69f17bdd4de3e'
    },{
        'version': '0.9',
        'tag': 'SCXML_0_9',
        'revision_hash': '836783c31f5546982558584e6395ae26a882534b'
    }],
    'commons-validator': [{
        'version': '1.0',
        'tag': 'VALIDATOR_1_0',
        'revision_hash': '28de1292f039a972cd0fd465e157be34d55d0ac6',
        'corrected_revision': '2909c6197eb48ebe389812dd1d26b59174297e1f'
    },{
        'version': '1.1.0',
        'tag': 'VALIDATOR_1_1_0',
        'revision_hash': '82fb3df34d2c01ff6e6482fd9f84b1363aab6e76',
        'corrected_revision': '11e0d011760e4d72fe5a089e73144efc05eba2ef'
    },{
        'version': '1.2.0',
        'tag': 'VALIDATOR_1_2_0',
        'revision_hash': 'b1833e64d1a5d68ea49db446bf2cce5f4cbc1549',
        'corrected_revision': '1a748629137600611781cbcfde47267763f718da'
    },{
        'version': '1.3.0',
        'tag': 'VALIDATOR_1_3_0',
        'revision_hash': 'de4028bee2c0629aef3fefa42e5128d077ce12f3',
        'corrected_revision': 'b748c1060fc5052e12831c4c174a6ab109bcace7'
    },{
        'version': '1.4.0',
        'tag': 'VALIDATOR_1_4_0',
        'revision_hash': '5eea84edbe4e2f3eeaa9d351bbacb148f5b09373'
    },{
        'version': '1.5.0',
        'tag': 'VALIDATOR_1_5_0',
        'revision_hash': 'ff5b4b45dbea1522510e50dbab2aae65cf47c6d1'
    },{
        'version': '1.6.0',
        'tag': 'VALIDATOR_1_6',
        'revision_hash': 'c4b93a7275b1172f69e53cd8cec7fc27c954a5be'
    }],
    'commons-vfs': [{
        'version': '1.0',
        'tag': 'vfs-1.0',
        'revision_hash': 'ff7dd9697fac9856c8acae2ab06031f83f70e33c',
        'corrected_revision': '8e23674676e3ab8df88513d0051fba6457ccc0c9'
    },{
        'version': '2.0',
        'tag': 'commons-vfs2-project-2.0',
        'revision_hash': '9508889ae6ae432704df3b923d12a6162ba6c89d'
    },{
        'version': '2.1',
        'tag': 'commons-vfs2-project-2.1',
        'revision_hash': 'f159bd4781b5ea91ae8872c2cd729d12511703e9'
    },{
        'version': '2.2',
        'tag': 'commons-vfs2-project-2.2',
        'revision_hash': 'ef1e6d59e29f3658b6d84b540644f4d4b27d0c67'
    }],
    'deltaspike': [{
        'version': '0.1',
        'tag': 'deltaspike-project-0.1-incubating',
        'revision_hash': '6048874eecc016e8d8073fad086382ab2f08312c'
    },{
        'version': '0.2',
        'tag': 'deltaspike-project-0.2-incubating',
        'revision_hash': '5e4a7eb4de01004206f24ae22b9850e643bffe54'
    },{
        'version': '0.3',
        'tag': 'deltaspike-project-0.3-incubating',
        'revision_hash': 'b880838313302a396543f432a91ee005d71a70da'
    },{
        'version': '0.4',
        'tag': 'deltaspike-project-0.4',
        'revision_hash': '3d8af1b1e6726cca4ecd7d087ec79eb1ff943aff'
    },{
        'version': '0.5',
        'tag': 'deltaspike-project-0.5',
        'revision_hash': 'dffe4faa12ef455d5eec4ee29cdec839e041914e'
    },{
        'version': '0.6',
        'tag': 'deltaspike-project-0.6',
        'revision_hash': 'f6a3ad36b974661ca7802986527fdbd95fe113c5'
    },{
        'version': '0.7',
        'tag': 'deltaspike-project-0.7',
        'revision_hash': 'c62a2df8210fa0c18c5fe5a1b907a439c11ae959'
    },{
        'version': '1.0.0',
        'tag': 'deltaspike-project-1.0.0',
        'revision_hash': '015dbba89afeb9bc763da72302587695d41af7e7'
    },{
        'version': '1.1.0',
        'tag': 'deltaspike-project-1.1.0',
        'revision_hash': '6dbd9404fd0c19a030b374322734a78954b5fa93'
    },{
        'version': '1.2.0',
        'tag': 'deltaspike-project-1.2.0',
        'revision_hash': '330cbd1c25824585caad87dc717e74f85c1dcc7b'
    },{
        'version': '1.3.0',
        'tag': 'deltaspike-root-1.3.0',
        'revision_hash': '31c714237829f7e7f07f9d1dc43776f984c78b27'
    },{
        'version': '1.4.0',
        'tag': 'deltaspike-1.4.0',
        'revision_hash': 'af1ed18cfb27e471342c901a496d6ec477a1427d'
    },{
        'version': '1.5.0',
        'tag': 'deltaspike-1.5.0',
        'revision_hash': 'd4caf739411b7ce89eb4ef8f1385c0c346498eb4'
    },{
        'version': '1.6.0',
        'tag': 'deltaspike-1.6.0',
        'revision_hash': '2d55237aebfeda8287d53fa5b6d3e370846f8ab3'
    },{
        'version': '1.7.0',
        'tag': 'deltaspike-1.7.0',
        'revision_hash': '5658d4959c58535b04615ac5ac66bb95a5246e93'
    },{
        'version': '1.8.0',
        'tag': 'deltaspike-1.8.0',
        'revision_hash': '2d4e4099852ffbd6aec1aca73dda331cf5329d91'
    }],
    'eagle': [{
        'version': '0.3.0',
        'tag': 'eagle-0.3.0-incubating',
        'revision_hash': '8183edad341bdec80a7d2acd56a5b5e24f28681c'
    },{
        'version': '0.4.0',
        'tag': 'v0.4.0-incubating',
        'revision_hash': 'eac0f27958f2ed8c6842938dad0a995a87fd0715'
    },{
        'version': '0.5.0',
        'tag': 'v0.5.0',
        'revision_hash': '0b9c91b8788e166ff203e342baf1940d4bd3247c'
    }],
    'giraph': [{
        'version': '0.1.0',
        'tag': 'release-0.1.0',
        'revision_hash': '2adab50db10fbffd94e70dfa8fdb1045c3b9ac8b',
        'corrected_revision': '65c1a337c1de1262ade8b02dc7e868c30f157589'
    },{
        'version': '1.0.0',
        'tag': 'release-1.0.0-RC3',
        'revision_hash': '2537201724b54de25efe144996af0cd762ac6018'
    },{
        'version': '1.1.0',
        'tag': 'release-1.1.0',
        'revision_hash': '0466bccf96ef4c92df4c20a6a4bad62cdf7624c1'
    }],
    'gora': [{
        'version': '0.1',
        'tag': '0.1-incubating',
        'revision_hash': 'a49c0b80c851cb0208c5dc03ab7420e995aec67d',
        'corrected_revision': 'faabec642fa692fadecabc6f345fbd1404e609f8'
    },{
        'version': '0.2',
        'tag': 'gora-0.2',
        'revision_hash': '424a10884ceacf25878b72e9804c6cc6c2ed44e2'
    },{
        'version': '0.3',
        'tag': 'apache-gora-0.3',
        'revision_hash': 'dd6c7a4a20533fba8a9a8aac9ed1e90e0b0633cf'
    },{
        'version': '0.4',
        'tag': 'apache-gora-0.4',
        'revision_hash': '685f26c7db062b28aefae9e2d328fecece295cba'
    },{
        'version': '0.5',
        'tag': 'apache-gora-0.5',
        'revision_hash': 'c2d58dd1440b4e2c66c1f40a4b6d4169d79bb6d3'
    },{
        'version': '0.6',
        'tag': 'apache-gora-0.6',
        'revision_hash': '68ce474b6813e9e5e8b021e3d255f90ddd7a3eb3'
    },{
        'version': '0.7',
        'tag': 'apache-gora-0.7',
        'revision_hash': '3b48688f51d6cf618c760b24a341874f12e160d2'
    },{
        'version': '0.8',
        'tag': 'apache-gora-0.8',
        'revision_hash': 'd173382084f73f4b2779fcb9131498f11f010371'
    }],
    'jspwiki': [{
        'version': '1.4.0',
        'tag': 'jspwiki_1_4_0',
        'revision_hash': '22c1139ea1c48950f54cff5db8ad4636db91d59a',
        'corrected_revision': '52d64cff1939e8f080b275a5504dcddf4c601c82'
    },{
        'version': '1.5.0',
        'tag': 'jspwiki_1_5_0',
        'revision_hash': 'b0eabb7ea312c776a727ef6c63245b2b5d64405f',
        'corrected_revision': '15ef5645e96d38779d850dce8150016f8bd45e83'
    },{
        'version': '1.6.0',
        'tag': 'jspwiki_1_6_0',
        'revision_hash': 'f9eae56bb2002461fa2499a445845bf132b462fb',
        'corrected_revision': '63caf8f3f30ab8432f2807fa459a66c6038759c8'
    },{
        'version': '1.7.0',
        'tag': 'jspwiki_1_7_0',
        'revision_hash': 'c784eef937df2cf318e1a8c06426fac324b60303',
        'corrected_revision': '196dab3ce4fb0ef3a58d6a35aeab7f51219a7577'
    },{
        'version': '1.8.0',
        'tag': 'jspwiki_1_8_0',
        'revision_hash': '96b60a83f3b0b03aa4f5da11ea70261d0af54e6b',
        'corrected_revision': 'cdc5345ff16e70350ac30fe43b1d0da4a71b3a14'
    },{
        'version': '2.0.36',
        'tag': 'jspwiki_2_0_36',
        'revision_hash': '5c46f2772e69315b5aae215dbaf3bea73b17f9eb',
        'corrected_revision': '14d5a0542f4c22d03e697b233c02aaa2f7c3891b'
    },{
        'version': '2.2.19',
        'tag': 'jspwiki_2_2_19',
        'revision_hash': '99a009637d47c0f57bd50932a65e8d92a572dcc2',
        'corrected_revision': '3af8a6b81158db4a1d39cfa7965e4dd08156635a'
    },{
        'version': '2.4.56',
        'tag': 'jspwiki_2_4_56',
        'revision_hash': '6bc63555b9eaf04878e2ce832b8bdf57fa820358',
        'corrected_revision': '7da437345960f35936dacddd4398d49628b41067'
    },{
        'version': '2.6.0',
        'tag': 'jspwiki_2_6_0',
        'revision_hash': '7f8c385f794a404b24bd412f2823e811948086fa',
        'corrected_revision': 'd2e78bf3403cadda7b66a1caa71ff9c949c8a580'
    },{
        'version': '2.8.0',
        'tag': 'jspwiki_2_8_0',
        'revision_hash': '3e85477ac133e31b85c5713cbcc6c3aaaf4d814e',
        'corrected_revision': 'e3246df9966adc64bf507206f90e3b324e47d982'
    },{
        'version': '2.9.0',
        'tag': 'jspwiki_2_9_0_incubating',
        'revision_hash': '7eaed006f8585a3be13508ccc66e316687194f8d',
        'corrected_revision': 'b52ef69fd517494a3f7856d8cdffd34d5247ba39'
    },{
        'version': '2.10.0',
        'tag': 'jspwiki_2_10_0',
        'revision_hash': 'ec0350a26858f25702c59dfaf5828142428b7929'
    }],
    'knox': [{
        'version': '0.3.0',
        'tag': 'v0.3.0-final',
        'revision_hash': 'd806123b3fda3f7149ad0b69b921f078a8360b37'
    },{
        'version': '0.4.0',
        'tag': 'v0.4.0-release',
        'revision_hash': '614c54d9b236bac81e3685b475264115d5e26b77'
    },{
        'version': '0.5.0',
        'tag': 'v05.0-release',
        'revision_hash': 'f29d530df008901e86317a9135f3e57de74166b3'
    },{
        'version': '0.6.0',
        'tag': 'v0.6.0-release',
        'revision_hash': 'fd88b115502f4b9fc9f3efae4b3c54d6e64246aa'
    },{
        'version': '0.7.0',
        'tag': 'v0.7.0-release',
        'revision_hash': '4f86a8e6b5a581ff46f6188693dd3cf5f4755e7b'
    },{
        'version': '0.8.0',
        'tag': 'v0.8.0-release',
        'revision_hash': '1473284f692ea14360bf790b4859d207df060e02'
    },{
        'version': '0.9.0',
        'tag': 'v0.9.0-release',
        'revision_hash': 'a1db6ee4028274f7959c626b73934cb79bb2493a'
    },{
        'version': '0.10.0',
        'tag': 'v0.10.0-release',
        'revision_hash': '905b1f247ce9682ae2af92468c514d41fcd8d059'
    },{
        'version': '0.11.0',
        'tag': 'v0.11.0-release',
        'revision_hash': '074c110cd84995b55141ece46fa44c689968a041'
    },{
        'version': '0.12.0',
        'tag': 'v0.12.0-release',
        'revision_hash': 'ee4b1473aa7570092f80d84b224f773111602835'
    },{
        'version': '0.13.0',
        'tag': 'v0.13.0-release',
        'revision_hash': '15f73ca81382b3c9edf31245bd525d9efdaf8997'
    },{
        'version': '0.14.0',
        'tag': 'v0.14.0-release',
        'revision_hash': 'e32ca83569a4d811cc7c16616e5cda837078fd08'
    },{
        'version': '1.0.0',
        'tag': 'v1.0.0-release',
        'revision_hash': '3c3f23a866d55de563c6574353019858590367d8'
    }],
    'kylin': [{
        'version': '0.6.1',
        'tag': 'v0.6.1',
        'revision_hash': '72138e235a307ac71061421d31b802969263e4ab'
    },{
        'version': '0.7.1',
        'tag': 'kylin-0.7.1-incubating',
        'revision_hash': 'a6e24b91717bc2e4e01873e1ce062c67a7710115'
    },{
        'version': '1.0',
        'tag': 'kylin-1.0-incubating',
        'revision_hash': 'f8a3320a85f6e26d85f3bffed73c0d38313c611c'
    },{
        'version': '1.1',
        'tag': 'kylin-1.1-incubating',
        'revision_hash': 'e8f226f2e8127dabef79a36420dc8724d0852688'
    },{
        'version': '1.2',
        'tag': 'kylin-1.2',
        'revision_hash': '0fc9be513ccc1d62f647b70d1b310d4013d1e0b0'
    },{
        'version': '1.3',
        'tag': 'kylin-1.3',
        'revision_hash': '75355046859dbb63df3e176fc63759f829a19769'
    },{
        'version': '1.5.0',
        'tag': 'kylin-1.5.0',
        'revision_hash': 'e1e732c0aa233123dc6a9d864e72b10aee0fd86e'
    },{
        'version': '1.6.0',
        'tag': 'kylin-1.6.0',
        'revision_hash': 'ed6982c8b3baaad08b7e4956001339979724d9a7'
    },{
        'version': '2.0.0',
        'tag': 'kylin-2.0.0',
        'revision_hash': '375fd807c281d8c5deff0620747c806be2019782'
    },{
        'version': '2.1.0',
        'tag': 'kylin-2.1.0',
        'revision_hash': '562dd173aaf6b398be8e053f896755b3afe8137f'
    },{
        'version': '2.2.0',
        'tag': 'kylin-2.2.0',
        'revision_hash': '850c0a6a3b9296d26121f9986701bb191921b1bf'
    }],
    'lens': [{
        'version': '2.6.0',
        'tag': 'apache-lens-2.6.0',
        'revision_hash': '925094f44923b33767a9d1d4bc9ea6b2bb3fe229'
    },{
        'version': '2.7.0',
        'tag': 'apache-lens-2.7.0',
        'revision_hash': 'd2b49929dd2af6a401a8ea6fa2d357c270a6890e'
    }],
    'mahout': [{
        'version': '0.1',
        'tag': 'mahout-0.1',
        'revision_hash': 'bee952edafdfb517191d3013d8d29a7d574c6a51',
        'corrected_revision': '56840d66ec7e29033395f8440f509d469e1d59a3'
    },{
        'version': '0.2',
        'tag': 'mahout-0.2',
        'revision_hash': '7a786023136ab3c6443f6aca4556a71e093b5ff2',
        'corrected_revision': '205296876dd21ef387d92843b3052c54bdeef972'
    },{
        'version': '0.3',
        'tag': 'mahout-0.3',
        'revision_hash': 'f73d4002aec1844622829f2c2822ea4765ec73b9',
        'corrected_revision': '514cb7c8deb753f9fb36c70f9dc8302a04cb67a8'
    },{
        'version': '0.4',
        'tag': 'mahout-0.4',
        'revision_hash': 'ce337a9c20c48c52c1680a4d1074a943c57c2107'
    },{
        'version': '0.5',
        'tag': 'mahout-0.5',
        'revision_hash': 'e597fcdb0465cb3f23ebc30266bc16993205c5d3'
    },{
        'version': '0.6',
        'tag': 'mahout-0.6',
        'revision_hash': '17a286d0403eed3e8c3c8cb2aaa815d0bce0bbe9'
    },{
        'version': '0.7',
        'tag': 'mahout-0.7',
        'revision_hash': '9d847e0357a2138687808ee257e53455ae4786c3'
    },{
        'version': '0.8',
        'tag': 'mahout-0.8',
        'revision_hash': '86573002086f6f0381c4aa927efac76608cd7f0a'
    },{
        'version': '0.9',
        'tag': 'mahout-0.9',
        'revision_hash': 'a41d378df344330192c88ea9d63e248bd8182f5c'
    },{
        'version': '0.10.0',
        'tag': 'mahout-0.10.0',
        'revision_hash': '832b08c40752b58a98e1a89891a1e0ff54094910'
    },{
        'version': '0.11.0',
        'tag': 'mahout-0.11.0',
        'revision_hash': 'b3a44a4ab274266a0a0808f8c401e80059db48ac'
    },{
        'version': '0.12.0',
        'tag': 'mahout-0.12.0',
        'revision_hash': '75c382f2129544a26d35889df7c15ab9d5ee9104'
    },{
        'version': '0.13.0',
        'tag': 'mahout-0.13.0',
        'revision_hash': 'cca62c5a600703b99a6291440193dd233103d4e4'
    }],
    'manifoldcf': [{
        'version': '0.1',
        'tag': 'release-0.1-incubating',
        'revision_hash': '61f1b3abaaa5a15527957f83d0b6d759704b2f56',
        'corrected_revision': '59eef781b5cdae0d057e03c301ad296e7e2a0310'
    },{
        'version': '0.2',
        'tag': 'release-0.2-incubating',
        'revision_hash': 'd87df29245c3ddbe06b3d53341a9194d6ce72183',
        'corrected_revision': 'eac4057fffd591c7987acb8e440f5f91edf74865'
    },{
        'version': '0.3',
        'tag': 'release-0.3-incubating',
        'revision_hash': 'da5a3251bd43613782296497279d9c6c4fd30018',
        'corrected_revision': 'fd3df0934f7d2f342090cf57edf90b0cf88216f1'
    },{
        'version': '0.4',
        'tag': 'release-0.4-incubating',
        'revision_hash': '29edf34ddfe813f2b7a4a48e76a2d2aa696c232b',
        'corrected_revision': 'efc76d2e36ee6aa12339193972ffbd9524c7db8b'
    },{
        'version': '0.5',
        'tag': 'release-0.5-incubating',
        'revision_hash': '588465f8b9c0cf47c5eb9d42bea8833f80c6bfc9',
        'corrected_revision': 'aa2caf8111305d1a06c93ecfee27b3133e49b266'
    },{
        'version': '0.6',
        'tag': 'release-0.6',
        'revision_hash': 'c8c76f505940e722788aa64522887a797df6c7e1'
    },{
        'version': '1.0',
        'tag': 'release-1.0',
        'revision_hash': '4d9ec8e51e136ca3619463f56948456882b5f6e1'
    },{
        'version': '1.1',
        'tag': 'release-1.1',
        'revision_hash': 'd270aeb0619d179b9c2a79104a07c1693d902163'
    },{
        'version': '1.2',
        'tag': 'release-1.2',
        'revision_hash': 'f1251713c5dd9eefebb4ccad80958bf3ec807458'
    },{
        'version': '1.3',
        'tag': 'release-1.3',
        'revision_hash': '0ab44af1e19de90a4660b115554729fb1f3bc0e9'
    },{
        'version': '1.4',
        'tag': 'release-1.4',
        'revision_hash': '1ef189ad890c01fdf42b7bd9acf5dca3d0a02acf'
    },{
        'version': '1.5',
        'tag': 'release-1.5',
        'revision_hash': 'ca4abe5982c4187098abfd32f6ed6eb4d3f07404'
    },{
        'version': '1.6',
        'tag': 'release-1.6',
        'revision_hash': 'da4f8d10a7b17ead1b68160b14064eced7ad242f'
    },{
        'version': '1.7',
        'tag': 'release-1.7',
        'revision_hash': '9d1bdfd510307f011ce03c48ae11e3527755c59a'
    },{
        'version': '1.8',
        'tag': 'release-1.8',
        'revision_hash': 'ccbafe553777f5c54f7efebf6e1821e18e9c321d'
    },{
        'version': '1.9',
        'tag': 'release-1.9',
        'revision_hash': 'ea504faaf95fcb0a2077b5d761a2520756d4df9c'
    },{
        'version': '1.10',
        'tag': 'release-1.10',
        'revision_hash': '14d42d2668bd4314dc65b1fe3a40f6b8b551eb28'
    },{
        'version': '2.0',
        'tag': 'release-2.0',
        'revision_hash': '9565b0ddcfa09baff2cfbfefe870aad4015df290'
    },{
        'version': '2.1',
        'tag': 'release-2.1',
        'revision_hash': '9cd4fe99ec8eb1ce7818cc0539524e9daac3143d'
    },{
        'version': '2.2',
        'tag': 'release-2.2',
        'revision_hash': '925f56d102ae25ae2e1c88a1d00208cada999dc0'
    },{
        'version': '2.3',
        'tag': 'release-2.3',
        'revision_hash': 'e701989191c61b0551eceb3c7e576cb7e062d2bf'
    },{
        'version': '2.4',
        'tag': 'release-2.4-RC0',
        'revision_hash': 'ca2e2454253e2d7b711f3ac8ce2b171a16e1e9ca'
    },{
        'version': '2.5',
        'tag': 'release-2.5',
        'revision_hash': '25f015a7a1e2b92a10f3f89df0462b9aaf7f1a94'
    },{
        'version': '2.6',
        'tag': 'release-2.6',
        'revision_hash': '46841cb51a195e5a1c12ee94e074c17436941382'
    },{
        'version': '2.7',
        'tag': 'release-2.7',
        'revision_hash': '19396df543b59c7c9dbf9bcc8e8dc8ba6a53351c'
    },{
        'version': '2.8',
        'tag': 'release-2.8',
        'revision_hash': '115d85b056a832c9f6a89156ecaae87fb006be44'
    },{
        'version': '2.9',
        'tag': 'release-2.9',
        'revision_hash': 'b5ce53b8f1fdf6dade99b9b6f7191f6042edeeda'
    },{
        'version': '2.10',
        'tag': 'release-2.10',
        'revision_hash': '46e349207d275410276cbb2540de77aa8ad3aab3'
    }],
    'nutch': [{
        'version': '0.7',
        'tag': 'release-0.7',
        'revision_hash': 'c089c9aa676d373fd6469d277b92b3438d263544',
        'corrected_revision': 'e3ad07e5a6de78d1088f0ed459edf0da5d519a45'
    },{
        'version': '0.8',
        'tag': 'release-0.8',
        'revision_hash': '616425eacb1ba37652e64a180ad8ae4dd2e618c0',
        'corrected_revision': '5ec47e73a47d7d9f963335ec327ceca7ec403fc9'
    },{
        'version': '0.9',
        'tag': 'release-0.9',
        'revision_hash': '1742e5a8eee6e30faad250c29be4f1ae138e741c',
        'corrected_revision': '3988e7cf047fb862264c670979eb52d99c83fbed'
    },{
        'version': '1.0',
        'tag': 'release-1.0',
        'revision_hash': '4f62826994129225e6f8999763677c0e4e29d67e',
        'corrected_revision': '4dd40032ae99fa8c0c36958064a66eae5c2f0606'
    },{
        'version': '1.1',
        'tag': '1.1',
        'revision_hash': 'eae9058af45f3b79dd108e326da2ab915f7b3284',
    },{
        'version': '1.2',
        'tag': 'release-1.2',
        'revision_hash': '5c39b8d4106202f06f7b5782452493f77314ddb8',
    },{
        'version': '1.3',
        'tag': 'release-1.3',
        'revision_hash': '8859cb5d63da7196fa72887e6022ba6d1d69bf82',
    },{
        'version': '1.4',
        'tag': 'release-1.4',
        'revision_hash': 'f4c7f5f6e77fa3a6737c13b5d32278cfe52155d5',
    },{
        'version': '1.5',
        'tag': 'release-1.5',
        'revision_hash': '087b12586cced83f42a8b31c9c596448b9b1f756',
    },{
        'version': '1.6',
        'tag': 'release-1.6',
        'revision_hash': 'd25863a32517fbcb090d2a6944b4e5560369010c',
    },{
        'version': '1.7',
        'tag': 'release-1.7',
        'revision_hash': '0b9c368d86824c20cfb7fe34d1ad5a30210a9abc',
    },{
        'version': '1.8',
        'tag': 'release-1.8',
        'revision_hash': '290a9ae2727bcce0a153657c80f22e2e26287cc2',
    },{
        'version': '1.9',
        'tag': 'release-1.9',
        'revision_hash': '1a865fcd43f15782fe599b941cd2e1785df4bfc7',
    },{
        'version': '1.10',
        'tag': 'release-1.10',
        'revision_hash': 'a03415029f68cfe844ade42d1d3f0e604b13675c',
    },{
        'version': '1.11',
        'tag': 'release-1.11-rc2',
        'revision_hash': '7d0a9855b3c4686382c60df6e39ca01dc3210690',
    },{
        'version': '1.12',
        'tag': 'release-1.12',
        'revision_hash': '7956daee8ac91180070f92949ecf99deae9b5ef0',
    },{
        'version': '1.13',
        'tag': 'release-1.13',
        'revision_hash': 'bd0da3569aa14105799ed39204d4f0a31c77b42c',
    },{
        'version': '1.14',
        'tag': 'release-1.14',
        'revision_hash': 'a8e60bdfb79b368612f068ed5aeeb690e29b448d',
    },{
        'version': '2.0',
        'tag': 'release-2.0',
        'revision_hash': '7d8b5e5b5d930e0f31ce28c725d53a275d971e22',
    },{
        'version': '2.1',
        'tag': 'release-2.1',
        'revision_hash': 'f3d1202d3932c5cb92485c799ef8d591b8abf8cb',
    },{
        'version': '2.2',
        'tag': 'release-2.2',
        'revision_hash': 'b8b1cdf17e41e88c292cf3afc67fdd4e820170f7',
    },{
        'version': '2.3',
        'tag': 'release-2.3',
        'revision_hash': '2578f9f219b3ecbef52873f7f25bd56b4818c343',
    }],
    'opennlp': [{
        'version': '1.7.0',
        'tag': 'opennlp-1.7.0',
        'revision_hash': '2083525415df4f5268abe265688d6ad173b54daf',
    },{
        'version': '1.8.0',
        'tag': 'opennlp-1.8.0',
        'revision_hash': '73c8e5b9d8e055fefb53f7f3c2487d05c9788c6a',
    }],
    'parquet-mr': [{
        'version': '1.0.0',
        'tag': 'parquet-1.0.0',
        'revision_hash': '6eec81d8abab0644e8c2df00544f73828ee0e219',
    },{
        'version': '1.1.0',
        'tag': 'parquet-1.1.0',
        'revision_hash': 'bee8378a90bee9657526cd872c9173309b0bafc2',
    },{
        'version': '1.2.0',
        'tag': 'parquet-1.2.0',
        'revision_hash': '4170539b9a9bca97e4f3159113606855a41c3b0f',
    },{
        'version': '1.3.0',
        'tag': 'parquet-1.3.0',
        'revision_hash': '87864cbd1f0ce52d4e357c02ca522f5309061caa',
    },{
        'version': '1.4.0',
        'tag': 'parquet-1.4.0',
        'revision_hash': '73d66174032ebba5274a7faa172542b4d70b093a',
    },{
        'version': '1.5.0',
        'tag': 'parquet-1.5.0',
        'revision_hash': '79977453b8cd65e6244f16316fac3a510aa87aa8',
    },{
        'version': '1.6.0',
        'tag': 'apache-parquet-mr-1.6.0-incubating',
        'revision_hash': '4f660778f89d4164c72676f6b15bbc74c0d09373',
    },{
        'version': '1.7.0',
        'tag': 'apache-parquet-1.7.0',
        'revision_hash': '32c46643845ea8a705c35d4ec8fc654cc8ff816d',
    },{
        'version': '1.8.0',
        'tag': 'apache-parquet-1.8.0',
        'revision_hash': '0fda28af84b9746396014ad6a415b90592a98b3b',
    },{
        'version': '1.9.0',
        'tag': 'apache-parquet-1.9.0',
        'revision_hash': '38262e2c80015d0935dad20f8e18f2d6f9fbd03c',
    }],
    'santuario-java': [{
        'version': '1.0.0',
        'tag': 'v1_0_0',
        'revision_hash': 'a0ac8137fe49b96edff2011da786ae579891fa75',
        'corrected_revision': '3aad6d8bcdcacb67bf129379f6417bf6199c461f'
    },{
        'version': '1.2',
        'tag': 'v1_2j',
        'revision_hash': 'a0ac8137fe49b96edff2011da786ae579891fa75',
        'corrected_revision': '3aad6d8bcdcacb67bf129379f6417bf6199c461f'
    },{
        'version': '1.4.5',
        'tag': '1.4.5',
        'revision_hash': '8d1b69a2901b241d913e4d2b12028f0ee5351ceb',
    },{
        'version': '1.5.9',
        'tag': '1.5.0',
        'revision_hash': '2d64c5b24af718f184e0a352480d6e85a36bb71a',
    },{
        'version': '2.0.0',
        'tag': 'xmlsec-2.0.0',
        'revision_hash': 'bddfc74628be4412711cc4a930468df7ef373597',
    },{
        'version': '2.1.0',
        'tag': 'xmlsec-2.1.0',
        'revision_hash': '74ebb633b441487707ec27ed60ab33803869c3f9',
    }],
    'systemml': [{
        'version': '0.9',
        'tag': None,
        'revision_hash': None,
        'corrected_revision': '630fa2ea845c5af1d765c7f65477e7e1ab582dc6',
    },{
        'version': '0.10',
        'tag': None,
        'revision_hash': None,
        'corrected_revision': '3d5f9b11741f6d6ecc6af7cbaa1069cde32be838',
    },{
        'version': '0.11',
        'tag': None,
        'revision_hash': None,
        'corrected_revision': 'c2e1670c2745863195d4789f1f77ed01ec11af5e',
    },{
        'version': '0.12',
        'tag': None,
        'revision_hash': None,
        'corrected_revision': 'a83ae1f7fbe9551da674450eedce08aaf7d19d07',
    },{
        'version': '0.13',
        'tag': None,
        'revision_hash': None,
        'corrected_revision': 'ff3e741694e507f64a6b52ee71638bddecabe7af',
    },{
        'version': '0.14',
        'tag': None,
        'revision_hash': None,
        'corrected_revision': '8bdcf106ca9bd04c0f68924ad5827eb7d7d54952',
    },{
        'version': '0.15',
        'tag': None,
        'revision_hash': None,
        'corrected_revision': 'ccaaa18769ae045fd31068dead2565844b4cee47',
    },{
        'version': '1.0.0',
        'tag': None,
        'revision_hash': None,
        'corrected_revision': '6519f3383f8fd43af93ecba8742ae4b2b28d4b35',
    },{
        'version': '1.1.0',
        'tag': None,
        'revision_hash': None,
        'corrected_revision': 'deddaee1fd3f8e87ff3a4403edf06cdb022ba949',
    },{
        'version': '1.2.0',
        'tag': None,
        'revision_hash': None,
        'corrected_revision': 'a1a05e29f6ee78f3c33fea355f62c78ce21766ee',
    }],
    'tika': [{
        'version': '0.1',
        'tag': '0.1-incubating',
        'revision_hash': 'fe7399ebe5f1d2cca553252577d0925815072b21',
        'corrected_revision': '2ca661e4cf4f5a21a53feec0074bf376fa17a93e'
    },{
        'version': '0.2',
        'tag': '0.2',
        'revision_hash': '938e8f87779351e071a6d24106af116d24bac07d',
        'corrected_revision': 'f03c0cae77843869abad37b0e5bf6f0efbbf60f7'
    },{
        'version': '0.3',
        'tag': '0.3',
        'revision_hash': 'b76f7f8836a744bfd1590c046f293c04f1cad2f4',
        'corrected_revision': '9ae0addd0de3a6f9042529705e5410bfbf1e7552'
    },{
        'version': '0.4',
        'tag': '0.4',
        'revision_hash': '02070084a054a7a1e325a2f54a145439c44e2df0',
        'corrected_revision': '163b0a4928cd9c9585406deac353d5f40306bd29'
    },{
        'version': '0.5',
        'tag': '0.5',
        'revision_hash': '450ffaca1c8beba7c1fc6cef047c89e5dfe6c5b0',
        'corrected_revision': 'c05a953cc692ccca7f0909efb8a2aaf19b9b806a'
    },{
        'version': '0.6',
        'tag': '0.6',
        'revision_hash': '9567efeb7087151074abcd2b6fbf6a72a8d2feee',
        'corrected_revision': '4912160b80b9771d1176f5132564d3fd5c71957a'
    },{
        'version': '0.7',
        'tag': '0.7',
        'revision_hash': '69772569d54dd208d0a07585f406baab76874c92',
        'corrected_revision': '6009758401347098a2660505c74b4bb74f2302c4'
    },{
        'version': '0.8',
        'tag': '0.8',
        'revision_hash': '107c4ef06420b27caa099f052aa3852e00ff3aa0'
    },{
        'version': '0.9',
        'tag': '0.9',
        'revision_hash': 'eb1cd6a1665a9034195b30172c0199a6143bcd45'
    },{
        'version': '0.10',
        'tag': '0.10',
        'revision_hash': '2138e990ec9cc7690ad54f574d623f48dcd72ae1'
    },{
        'version': '1.0',
        'tag': '1.0',
        'revision_hash': '6d7139b7f9bc73ee3797bb43145cf87264fc49cd'
    },{
        'version': '1.1',
        'tag': '1.1',
        'revision_hash': '0f8a1292c2f243442030161fecf9fa04aae89727'
    },{
        'version': '1.2',
        'tag': '1.2',
        'revision_hash': 'de47325be3c9a7aff20075fa4354a9be221afffd'
    },{
        'version': '1.3',
        'tag': '1.3',
        'revision_hash': 'f0027f53493d4cada75b48e38faa5d32675b7034'
    },{
        'version': '1.4',
        'tag': '1.4',
        'revision_hash': '540d4166a963ae9f7a0c1a18c7b1cfa21c245e10'
    },{
        'version': '1.5',
        'tag': '1.5',
        'revision_hash': 'a79ed8f00d6329ac9aaa94747923f36af654b129'
    },{
        'version': '1.6',
        'tag': '1.6',
        'revision_hash': '5ee61497debd6c5e731c871c4f75f79cfbd37b14'
    },{
        'version': '1.7',
        'tag': '1.7',
        'revision_hash': 'f9003a1fe1a1883008b62e78d01a582be6c06a8c'
    },{
        'version': '1.8',
        'tag': '1.8',
        'revision_hash': 'c57e805ee506123afc208eae5fd97fb4c4d1b8f7'
    },{
        'version': '1.9',
        'tag': '1.9-rc2',
        'revision_hash': '49757b678139ab8501a1585154cc004bfe080606'
    },{
        'version': '1.10',
        'tag': '1.10',
        'revision_hash': '3701de89d12cb9d6342887521aec5b1f1b141e84'
    },{
        'version': '1.11',
        'tag': '1.11-rc1',
        'revision_hash': '3610d7db1ec6b75ca1927c49b97452333a236a0f'
    },{
        'version': '1.12',
        'tag': '1.12-rc1',
        'revision_hash': 'c0d2b4fc5a8b06d3939253853e0fed005fd7771a'
    },{
        'version': '1.13',
        'tag': '1.13',
        'revision_hash': '386b68b5ae87beafacfb63f33e0a9888dedb9c30'
    },{
        'version': '1.14',
        'tag': '1.14-rc1',
        'revision_hash': 'dbb6baad77937db89d5ac05d4dea0b246fb73a6d'
    },{
        'version': '1.15',
        'tag': '1.15',
        'revision_hash': 'd831efbea89457300dbbd452caeb9790d4f47a65'
    },{
        'version': '1.16',
        'tag': '1.16',
        'revision_hash': 'b99f344f6d462f1ef5cfd0e864703a2d42f8752e'
    },{
        'version': '1.17',
        'tag': '1.17',
        'revision_hash': 'b054df2be2f43c4944ff66f7d9fd2ceb1655d9c0'
    }],
    'wss4j': [{
        'version': '1.5.0',
        'tag': '1_5_0',
        'revision_hash': '1076b70d62584310a22636532fcf4af68fe96554'
    },{
        'version': '1.6.0',
        'tag': '1_6_0',
        'revision_hash': '3db427f29876d4920faf77a613e488113e59e0f2'
    },{
        'version': '2.0.0',
        'tag': 'wss4j-2.0.0',
        'revision_hash': '181d94d615489e3dbe59be38dd465f0b9e8af007'
    },{
        'version': '2.1.0',
        'tag': 'wss4j-2.1.0',
        'revision_hash': '2a2d11a10260fac80995e3da10657de8b8347516'
    },{
        'version': '2.2.0',
        'tag': 'wss4j-2.2.0',
        'revision_hash': '1df91ee6048ffd7d7e3afee78122a6aa22ebe9e9'
    }]
}

Metadata about Projects and Releases

all_releases = []
for project in projects:
    for release in releases[project]:
        all_releases.append(project+'-'+release['version'])

project_data = pd.DataFrame(index=projects, columns=['Releases', 'Commits', 'Issues', 'Bugs', 'Linked Bugs', 'Homepage', 'VCS System', 'Issue System'])
release_data = pd.DataFrame(index=all_releases, columns=['Release Date', '#Files Total', '#Files Production'])

for project in projects:
    cur_releases = []
    for release in releases[project]:
        cur_releases.append(release['version'])
        if 'corrected_revision' in release:
            revision_hash = release['corrected_revision']
        else:
            revision_hash = release['revision_hash']
        release_commit = Commit.objects(revision_hash=revision_hash).get()
        files_total = 0
        files_production = 0
        for ces in CodeEntityState.objects(id__in=release_commit.code_entity_states, ce_type='file').only('long_name'):
            files_total += 1
            if java_filename_filter(ces.long_name):
                files_production += 1
        release_data.at[project+'-'+release['version'],'Release Date'] = release_commit.committer_date
        release_data.at[project+'-'+release['version'],'#Files Total'] = files_total
        release_data.at[project+'-'+release['version'],'#Files Production'] = files_production
    releases_str = str(cur_releases)
    releases_str = releases_str.replace('[','')
    releases_str = releases_str.replace(']','')
    releases_str = releases_str.replace("'",'')
    project_id = Project.objects(name=project).get().id
    vcs_system = VCSSystem.objects(project_id=project_id).get()
    its_system = IssueSystem.objects(project_id=project_id).get()
    num_commits = Commit.objects(vcs_system_id=vcs_system.id).count()
    num_issues = Issue.objects(issue_system_id=its_system.id).count()
    num_bugs = 0
    num_linked_bugs = 0
    for issue in Issue.objects(issue_system_id=its_system.id,issue_type__iexact='bug'):
        if jira_is_resolved_and_fixed(issue):
            num_bugs += 1
            if Commit.objects(linked_issue_ids=issue.id).count()>0:
                num_linked_bugs += 1
    
    project_data.at[project,'Releases'] = releases_str
    project_data.at[project,'Commits'] = num_commits
    project_data.at[project,'Issues'] = num_issues
    project_data.at[project,'Bugs'] = num_bugs
    project_data.at[project,'Linked Bugs'] = num_linked_bugs
    project_data.at[project,'Homepage'] = homepages[project]
    project_data.at[project,'VCS System'] = vcs_system.url
    project_data.at[project,'Issue System'] = its_system.url
    
print('Number of Projects:'.ljust(25), len(project_data))
print('Number of Releases:'.ljust(25), len(release_data))
print('Total commits:'.ljust(25), project_data['Commits'].sum())
print('Total issues:'.ljust(25), project_data['Issues'].sum())
print('Total bugs:'.ljust(25), project_data['Bugs'].sum())
print('Total linked bugs:'.ljust(25), project_data['Linked Bugs'].sum())

display(project_data)
display(release_data)

Number of Projects:       38
Number of Releases:       398
Total commits:            140442
Total issues:             45323
Total bugs:               15283
Total linked bugs:        11295

	Releases	Commits	Issues	Bugs	Linked Bugs	Homepage	VCS System	Issue System
ant-ivy	1.4.1, 2.0.0, 2.1.0, 2.2.0, 2.3.0, 2.4.0	3189	1585	608	535	https://ant.apache.org/ivy/	https://github.com/apache/ant-ivy	https://issues.apache.org/jira/rest/api/2/sear...
archiva	1.0, 1.1, 1.2, 1.3, 2.0.0, 2.1.0, 2.2.0	10261	2010	751	542	https://archiva.apache.org/	https://github.com/apache/archiva.git	https://issues.apache.org/jira/rest/api/2/sear...
calcite	1.0.0, 1.1.0, 1.2.0, 1.3.0, 1.4.0, 1.5.0, 1.6....	2926	2608	1178	842	https://calcite.apache.org/	https://github.com/apache/calcite.git	https://issues.apache.org/jira/rest/api/2/sear...
cayenne	3.0.0, 3.1.0	6619	2430	849	530	https://cayenne.apache.org/	https://github.com/apache/cayenne.git	https://issues.apache.org/jira/rest/api/2/sear...
commons-bcel	5.0, 5.1, 5.2, 6.0, 6.1, 6.2	1429	316	149	53	https://commons.apache.org/bcel/	https://github.com/apache/commons-bcel.git	https://issues.apache.org/jira/rest/api/2/sear...
commons-beanutils	1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7.0, 1.8....	1341	523	150	76	https://commons.apache.org/beanutils/	https://github.com/apache/commons-beanutils.git	https://issues.apache.org/jira/rest/api/2/sear...
commons-codec	1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 1...	1838	258	93	64	https://commons.apache.org/codec/	https://github.com/apache/commons-codec.git	https://issues.apache.org/jira/rest/api/2/sear...
commons-collections	1.0, 2.0, 2.1, 3.0, 3.1, 3.2, 3.3, 4.0, 4.1	3380	659	228	115	https://commons.apache.org/collections/	https://github.com/apache/commons-collections.git	https://issues.apache.org/jira/rest/api/2/sear...
commons-compress	1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1...	2755	484	200	172	https://commons.apache.org/compress/	https://github.com/apache/commons-compress.git	https://issues.apache.org/jira/rest/api/2/sear...
commons-configuration	1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1...	3717	743	314	188	https://commons.apache.org/configuration/	https://github.com/apache/commons-configuratio...	https://issues.apache.org/jira/rest/api/2/sear...
commons-dbcp	1.0, 1.1, 1.2, 1.3, 1.4, 2.0, 2.1, 2.2.0, 2.3....	2205	553	241	127	https://commons.apache.org/dbcp/	https://github.com/apache/commons-dbcp.git	https://issues.apache.org/jira/rest/api/2/sear...
commons-digester	1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 2...	2535	194	84	26	https://commons.apache.org/digester/	https://github.com/apache/commons-digester.git	https://issues.apache.org/jira/rest/api/2/sear...
commons-io	1.0, 1.1, 1.2, 1.3, 1.4, 2.0, 2.1, 2.2, 2.3, 2...	2262	602	169	131	https://commons.apache.org/io/	https://github.com/apache/commons-io.git	https://issues.apache.org/jira/rest/api/2/sear...
commons-jcs	1.0, 1.1, 1.3, 2.0, 2.1, 2.2	1622	194	96	80	https://commons.apache.org/jcs/	https://github.com/apache/commons-jcs.git	https://issues.apache.org/jira/rest/api/2/sear...
commons-jexl	1.0, 1.1, 2.0, 2.1, 3.0, 3.1	3276	277	109	84	https://commons.apache.org/jexl/	https://github.com/apache/commons-jexl.git	https://issues.apache.org/jira/rest/api/2/sear...
commons-lang	1.0, 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 3.0, 3...	5792	1429	458	318	https://commons.apache.org/lang/	https://github.com/apache/commons-lang.git	https://issues.apache.org/jira/rest/api/2/sear...
commons-math	1.0, 1.1, 1.2, 2.0, 2.1, 2.2, 3.0, 3.1, 3.2, 3...	7222	1455	545	415	https://commons.apache.org/math/	https://github.com/apache/commons-math	https://issues.apache.org/jira/rest/api/2/sear...
commons-net	1.0.0, 1.1.0, 1.2.0, 1.3.0, 1.4.0, 2.0, 2.1, 2...	2270	661	263	176	https://commons.apache.org/net/	https://github.com/apache/commons-net.git	https://issues.apache.org/jira/rest/api/2/sear...
commons-scxml	0.5, 0.6, 0.7, 0.8, 0.9	1216	287	87	70	https://commons.apache.org/scxml/	https://github.com/apache/commons-scxml.git	https://issues.apache.org/jira/rest/api/2/sear...
commons-validator	1.0, 1.1.0, 1.2.0, 1.3.0, 1.4.0, 1.5.0, 1.6.0	3416	454	142	73	https://commons.apache.org/validator/	https://github.com/apache/commons-validator.git	https://issues.apache.org/jira/rest/api/2/sear...
commons-vfs	1.0, 2.0, 2.1, 2.2	2212	685	216	156	https://commons.apache.org/vfs/	https://github.com/apache/commons-vfs.git	https://issues.apache.org/jira/rest/api/2/sear...
deltaspike	0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 1.0.0, 1.1....	2311	1359	330	302	https://deltaspike.apache.org/	https://github.com/apache/deltaspike.git	https://issues.apache.org/jira/rest/api/2/sear...
eagle	0.3.0, 0.4.0, 0.5.0	1119	1084	287	225	https://eagle.apache.org/	https://github.com/apache/eagle.git	https://issues.apache.org/jira/rest/api/2/sear...
giraph	0.1.0, 1.0.0, 1.1.0	1121	1232	374	337	https://giraph.apache.org/	https://github.com/apache/giraph.git	https://issues.apache.org/jira/rest/api/2/sear...
gora	0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8	1329	590	144	113	https://gora.apache.org/	https://github.com/apache/gora.git	https://issues.apache.org/jira/rest/api/2/sear...
jspwiki	1.4.0, 1.5.0, 1.6.0, 1.7.0, 1.8.0, 2.0.36, 2.2...	8809	980	351	274	https://jspwiki.apache.org/	https://github.com/apache/jspwiki.git	https://issues.apache.org/jira/rest/api/2/sear...
knox	0.3.0, 0.4.0, 0.5.0, 0.6.0, 0.7.0, 0.8.0, 0.9....	2069	1576	697	568	https://knox.apache.org/	https://github.com/apache/knox.git	https://issues.apache.org/jira/rest/api/2/sear...
kylin	0.6.1, 0.7.1, 1.0, 1.1, 1.2, 1.3, 1.5.0, 1.6.0...	12975	3316	1163	732	http://kylin.apache.org/	https://github.com/apache/kylin.git	https://issues.apache.org/jira/rest/api/2/sear...
lens	2.6.0, 2.7.0	2418	1469	424	397	https://lens.apache.org/	https://github.com/apache/lens.git	https://issues.apache.org/jira/rest/api/2/sear...
mahout	0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 0...	4167	2062	610	513	https://mahout.apache.org/	https://github.com/apache/mahout	https://issues.apache.org/jira/rest/api/2/sear...
manifoldcf	0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 1.0, 1.1, 1.2, 1...	5936	1607	689	633	https://manifoldcf.apache.org/	https://github.com/apache/manifoldcf.git	https://issues.apache.org/jira/rest/api/2/sear...
nutch	0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1...	3532	2657	748	643	https://nutch.apache.org/	https://github.com/apache/nutch.git	https://issues.apache.org/jira/rest/api/2/sear...
opennlp	1.7.0, 1.8.0	2685	1214	265	219	https://opennlp.apache.org/	https://github.com/apache/opennlp.git	https://issues.apache.org/jira/rest/api/2/sear...
parquet-mr	1.0.0, 1.1.0, 1.2.0, 1.3.0, 1.4.0, 1.5.0, 1.6....	2249	1413	447	185	https://parquet.apache.org/	https://github.com/apache/parquet-mr.git	https://issues.apache.org/jira/rest/api/2/sear...
santuario-java	1.0.0, 1.2, 1.4.5, 1.5.9, 2.0.0, 2.1.0	3376	505	288	83	https://santuario.apache.org/	https://github.com/apache/santuario-java.git	https://issues.apache.org/jira/rest/api/2/sear...
systemml	0.9, 0.10, 0.11, 0.12, 0.13, 0.14, 0.15, 1.0.0...	6196	2364	494	452	https://systemml.apache.org/	https://github.com/apache/systemml.git	https://issues.apache.org/jira/rest/api/2/sear...
tika	0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 0...	4933	2829	774	605	https://tika.apache.org/	https://github.com/apache/tika.git	https://issues.apache.org/jira/rest/api/2/sear...
wss4j	1.5.0, 1.6.0, 2.0.0, 2.1.0, 2.2.0	3734	659	268	241	https://ws.apache.org/wss4j/	https://github.com/apache/wss4j.git	https://issues.apache.org/jira/rest/api/2/sear...

	Release Date	#Files Total	#Files Production
ant-ivy-1.4.1	2007-01-24 18:30:33	327	240
ant-ivy-2.0.0	2009-01-18 22:08:59	481	352
ant-ivy-2.1.0	2009-09-25 22:18:07	487	357
ant-ivy-2.2.0	2010-09-23 21:53:29	498	363
ant-ivy-2.3.0	2013-01-21 20:12:28	604	451
ant-ivy-2.4.0	2014-12-13 15:58:22	639	474
archiva-1.0	2007-11-27 03:20:18	636	430
archiva-1.1	2008-07-16 09:42:37	714	467
archiva-1.2	2009-03-22 03:17:48	688	449
archiva-1.3	2010-01-13 03:57:51	728	448
archiva-2.0.0	2014-02-06 22:58:29	697	459
archiva-2.1.0	2014-07-16 00:51:21	704	458
archiva-2.2.0	2015-02-24 10:55:45	695	457
calcite-1.0.0	2015-02-01 01:11:43	1272	1075
calcite-1.1.0	2015-03-13 20:16:39	1303	1103
calcite-1.2.0	2015-04-07 20:57:32	1312	1108
calcite-1.3.0	2015-05-21 20:24:26	1320	1115
calcite-1.4.0	2015-08-24 04:52:20	1338	1127
calcite-1.5.0	2015-11-06 13:35:05	1415	1176
calcite-1.6.0	2016-01-18 20:40:50	1439	1193
calcite-1.7.0	2016-03-17 23:21:52	1520	1252
calcite-1.8.0	2016-05-31 23:37:40	1591	1301
calcite-1.9.0	2016-09-17 08:10:31	1620	1310
calcite-1.10.0	2016-10-07 22:21:13	1622	1310
calcite-1.11.0	2017-01-03 09:36:54	1648	1331
calcite-1.12.0	2017-03-21 01:54:36	1746	1415
calcite-1.13.0	2017-06-22 16:58:07	1531	1275
calcite-1.14.0	2017-09-27 19:33:02	1574	1308
calcite-1.15.0	2017-12-06 06:49:15	1626	1352
cayenne-3.0.0	2010-04-26 09:34:15	2779	1708
cayenne-3.1.0	2014-09-20 14:03:15	2696	1578
commons-bcel-5.0	2005-07-17 00:54:26	365	325
commons-bcel-5.1	2006-04-01 03:24:35	371	334
commons-bcel-5.2	2006-06-11 14:41:15	372	335
commons-bcel-6.0	2016-07-14 11:32:57	482	376
commons-bcel-6.1	2017-09-18 08:48:36	488	378
commons-bcel-6.2	2017-12-08 19:13:53	489	378
commons-beanutils-1.0	2001-07-15 02:36:37	12	3
commons-beanutils-1.1	2001-09-22 17:40:17	13	4
commons-beanutils-1.2	2001-12-24 21:35:04	16	5
commons-beanutils-1.3	2002-04-29 22:06:22	50	29
commons-beanutils-1.4	2002-08-13 04:42:07	68	43
commons-beanutils-1.5	2002-10-24 04:45:05	92	62
commons-beanutils-1.6	2003-01-17 13:52:25	106	64
commons-beanutils-1.7.0	2004-08-01 20:15:19	162	87
commons-beanutils-1.8.0	2008-08-31 15:52:16	225	101
commons-beanutils-1.9.0	2013-12-11 20:18:27	245	104
commons-codec-1.1	2003-04-29 22:18:19	23	14
commons-codec-1.2	2003-11-25 05:03:29	29	17
commons-codec-1.3	2004-07-13 20:17:34	41	24
commons-codec-1.4	2011-12-07 17:45:48	56	29
commons-codec-1.5	2011-03-29 22:18:38	75	39
commons-codec-1.6	2012-03-05 21:58:59	86	47
commons-codec-1.7	2012-09-14 20:05:59	105	55
commons-codec-1.8	2013-04-28 18:39:03	107	56
commons-codec-1.9	2013-12-24 20:18:30	109	56
commons-codec-1.10	2014-11-10 02:32:15	115	60
commons-codec-1.11	2017-10-20 23:22:24	124	64
commons-collections-1.0	2001-07-14 23:43:11	47	26
commons-collections-2.0	2002-04-02 16:31:15	84	44
commons-collections-2.1	2002-10-21 02:13:12	131	72
commons-collections-3.0	2004-01-25 01:23:04	417	248
commons-collections-3.1	2004-06-22 23:03:20	455	267
commons-collections-3.2	2006-05-14 21:53:53	467	273
commons-collections-3.3	2009-05-21 07:41:29	469	273
commons-collections-4.0	2013-11-24 12:41:10	470	270
commons-collections-4.1	2015-11-27 21:53:17	525	301
commons-compress-1.0	2009-05-14 04:37:27	92	61
commons-compress-1.1	2010-08-13 04:40:31	105	66
commons-compress-1.2	2011-07-31 05:20:18	113	69
commons-compress-1.3	2011-11-01 19:22:19	149	93
commons-compress-1.4	2012-04-11 04:58:10	162	99
commons-compress-1.5	2013-03-14 08:43:27	169	103
commons-compress-1.6	2013-10-26 05:16:56	196	124
commons-compress-1.7	2014-01-20 05:15:09	215	136
commons-compress-1.8	2014-03-12 22:18:40	222	141
commons-compress-1.9	2014-10-09 07:54:15	233	145
commons-compress-1.10	2015-08-18 17:07:30	254	158
commons-compress-1.11	2016-03-30 13:12:33	263	165
commons-compress-1.12	2016-06-18 15:25:37	265	166
commons-compress-1.13	2016-12-25 12:16:42	282	177
commons-compress-1.14	2017-05-11 19:03:58	314	195
commons-compress-1.15	2017-10-14 13:17:09	329	195
commons-compress-1.16	2018-02-02 17:34:59	341	201
commons-configuration-1.0	2004-10-11 09:53:51	66	29
commons-configuration-1.1	2005-04-02 14:18:52	90	42
commons-configuration-1.2	2005-12-17 10:18:19	104	53
commons-configuration-1.3	2006-09-24 01:59:27	164	84
commons-configuration-1.4	2007-04-08 17:35:06	177	91
commons-configuration-1.5	2007-11-24 15:36:26	187	92
commons-configuration-1.6	2008-12-25 12:55:46	195	96
commons-configuration-1.7	2011-09-08 18:50:57	227	114
commons-configuration-1.8	2012-06-19 20:11:15	219	108
commons-configuration-1.9	2012-08-22 18:20:12	219	108
commons-configuration-1.10	2013-10-28 04:07:11	224	108
commons-configuration-2.0	2016-03-24 18:20:52	442	231
commons-configuration-2.1	2016-08-20 14:12:53	447	233
commons-configuration-2.2	2017-10-12 19:39:31	457	240
commons-dbcp-1.0	2002-08-13 00:44:54	46	32
commons-dbcp-1.1	2003-10-20 19:50:25	60	39
commons-dbcp-1.2	2004-06-07 17:56:43	65	39
commons-dbcp-1.3	2010-02-07 16:50:58	96	53
commons-dbcp-1.4	2010-02-07 16:51:18	96	53
commons-dbcp-2.0	2014-05-25 18:25:29	100	53
commons-dbcp-2.1	2015-02-24 00:29:27	104	54
commons-dbcp-2.2.0	2017-12-21 17:28:28	105	54
commons-dbcp-2.3.0	2018-05-12 18:36:02	106	55
commons-dbcp-2.4.0	2018-06-13 19:26:05	109	56
commons-dbcp-2.5.0	2018-07-16 00:15:38	120	56
commons-digester-1.0	2001-07-15 00:10:36	15	14
commons-digester-1.1	2001-09-22 18:44:13	31	22
commons-digester-1.2	2002-04-29 22:18:48	50	31
commons-digester-1.3	2002-08-13 16:28:59	50	31
commons-digester-1.4	2003-02-03 18:40:28	56	33
commons-digester-1.5	2003-04-27 15:48:54	69	37
commons-digester-1.6	2004-09-07 21:39:13	156	68
commons-digester-1.7	2005-06-12 00:28:21	164	69
commons-digester-1.8	2006-12-03 15:57:54	169	70
commons-digester-2.0	2008-12-24 20:04:23	171	70
commons-digester-2.1	2010-09-25 14:01:16	248	114
commons-digester-3.0	2011-07-06 11:48:29	301	154
commons-digester-3.1	2011-10-28 22:36:29	302	154
commons-digester-3.2	2011-12-13 22:26:24	308	157
commons-io-1.0	2004-05-14 02:25:45	56	34
commons-io-1.1	2005-10-10 20:55:27	76	39
commons-io-1.2	2006-03-19 22:24:19	81	42
commons-io-1.3	2007-01-30 20:25:21	99	54
commons-io-1.4	2008-01-21 20:15:47	136	72
commons-io-2.0	2010-10-18 20:16:04	188	100
commons-io-2.1	2011-10-11 12:31:32	188	100
commons-io-2.2	2012-06-19 19:38:37	196	102
commons-io-2.3	2012-06-19 19:37:23	198	103
commons-io-2.4	2012-06-19 19:34:38	198	103
commons-io-2.5	2016-04-22 00:33:16	227	115
commons-jcs-1.0	2002-04-07 16:55:32	224	215
commons-jcs-1.1	2004-07-23 00:58:37	243	213
commons-jcs-1.3	2007-06-03 16:16:33	404	275
commons-jcs-2.0	2016-12-25 20:00:58	562	368
commons-jcs-2.1	2017-02-05 13:17:46	560	368
commons-jcs-2.2	2017-08-02 10:46:13	561	368
commons-jexl-1.0	2004-09-06 23:34:36	89	82
commons-jexl-1.1	2006-09-08 23:03:34	100	85
commons-jexl-2.0	2010-02-20 16:32:57	86	52
commons-jexl-2.1	2011-11-03 13:32:27	106	69
commons-jexl-3.0	2015-12-27 17:55:13	133	82
commons-jexl-3.1	2017-04-14 08:19:31	141	85
commons-lang-1.0	2002-10-04 03:23:18	61	26
commons-lang-2.0	2003-08-23 10:39:20	140	57
commons-lang-2.1	2005-06-12 13:52:39	204	74
commons-lang-2.2	2006-10-04 17:05:35	221	81
commons-lang-2.3	2007-02-13 16:53:07	221	81
commons-lang-2.4	2008-03-19 05:39:36	228	84
commons-lang-2.5	2010-04-07 16:03:21	238	89
commons-lang-2.6	2011-01-16 22:39:06	235	86
commons-lang-3.0	2011-07-19 03:28:54	214	99
commons-lang-3.1	2011-11-15 07:20:23	218	99
commons-lang-3.2	2014-01-01 16:43:50	247	112
commons-lang-3.3	2014-03-04 17:07:14	274	120
commons-lang-3.4	2015-04-06 16:14:40	281	121
commons-lang-3.5	2016-10-13 19:01:09	305	131
commons-lang-3.6	2017-06-09 09:38:15	318	135
commons-lang-3.7	2017-11-04 18:09:44	323	138
commons-math-1.0	2004-12-07 03:57:19	195	106
commons-math-1.1	2005-12-20 15:33:43	252	153
commons-math-1.2	2008-02-24 05:56:08	396	223
commons-math-2.0	2010-02-13 15:18:38	645	388
commons-math-2.1	2010-03-31 11:01:39	675	408
commons-math-2.2	2011-03-20 11:54:15	758	457
commons-math-3.0	2012-06-20 16:08:10	970	568
commons-math-3.1	2012-12-23 18:09:22	1194	693
commons-math-3.2	2013-04-06 07:53:53	1254	725
commons-math-3.3	2014-05-14 18:42:03	1467	825
commons-math-3.4	2014-12-23 09:28:44	1510	852
commons-math-3.5	2015-04-14 11:56:59	1513	852
commons-math-3.6	2016-01-02 18:56:59	1616	914
commons-net-1.0.0	2003-02-19 05:35:52	120	95
commons-net-1.1.0	2003-10-21 03:21:41	155	111
commons-net-1.2.0	2004-05-01 00:59:32	169	120
commons-net-1.3.0	2004-12-14 14:22:15	187	129
commons-net-1.4.0	2005-05-07 17:24:55	404	270
commons-net-2.0	2008-09-28 11:20:47	194	126
commons-net-2.1	2010-03-18 12:27:10	196	130
commons-net-2.2	2011-02-24 01:59:07	200	132
commons-net-3.0	2011-05-14 00:24:26	243	152
commons-net-3.1	2012-02-21 00:53:46	245	153
commons-net-3.2	2012-12-01 22:29:22	250	153
commons-net-3.3	2013-06-11 16:30:24	254	155
commons-net-3.4	2015-11-25 17:46:49	268	156
commons-net-3.5	2016-05-05 00:18:18	268	156
commons-net-3.6	2017-02-14 23:59:38	270	156
commons-scxml-0.5	2006-07-27 22:45:56	123	72
commons-scxml-0.6	2006-12-18 16:11:13	128	75
commons-scxml-0.7	2007-12-19 18:03:28	139	79
commons-scxml-0.8	2008-05-19 18:12:00	142	79
commons-scxml-0.9	2008-12-01 22:39:25	144	79
commons-validator-1.0	2002-10-25 03:22:55	37	17
commons-validator-1.1.0	2003-08-27 03:49:26	48	23
commons-validator-1.2.0	2005-11-15 20:31:15	58	22
commons-validator-1.3.0	2006-03-24 13:08:32	92	38
commons-validator-1.4.0	2012-02-03 22:02:49	135	59
commons-validator-1.5.0	2015-11-25 09:23:13	141	62
commons-validator-1.6.0	2017-02-20 10:09:44	147	63
commons-vfs-1.0	2006-12-28 09:48:05	303	236
commons-vfs-2.0	2011-08-18 13:56:58	325	251
commons-vfs-2.1	2016-05-19 01:55:27	373	259
commons-vfs-2.2	2018-02-24 16:20:39	382	262
deltaspike-0.1	2012-02-02 19:41:43	107	56
deltaspike-0.2	2012-04-15 19:27:52	346	188
deltaspike-0.3	2012-08-15 09:28:23	449	181
deltaspike-0.4	2013-05-28 12:12:05	721	314
deltaspike-0.5	2013-09-11 11:40:40	985	492
deltaspike-0.6	2014-03-16 16:55:05	1188	578
deltaspike-0.7	2014-04-30 13:45:07	1212	589
deltaspike-1.0.0	2014-06-14 15:08:48	1320	615
deltaspike-1.1.0	2014-11-01 19:54:54	1424	634
deltaspike-1.2.0	2014-11-30 16:37:15	1428	636
deltaspike-1.3.0	2015-03-02 16:58:41	1500	645
deltaspike-1.4.0	2015-05-18 08:06:26	1512	649
deltaspike-1.5.0	2015-08-10 00:48:22	1545	661
deltaspike-1.6.0	2016-04-02 20:32:22	1658	693
deltaspike-1.7.0	2016-06-09 23:55:31	1683	708
deltaspike-1.8.0	2017-05-28 09:43:42	1727	725
eagle-0.3.0	2016-03-21 22:53:12	893	682
eagle-0.4.0	2016-07-11 08:54:52	953	732
eagle-0.5.0	2017-11-23 15:49:26	1853	1388
giraph-0.1.0	2012-02-07 01:47:30	150	105
giraph-1.0.0	2013-04-16 05:20:33	566	431
giraph-1.1.0	2014-11-01 05:25:16	1014	753
gora-0.1	2011-03-09 16:22:39	136	97
gora-0.2	2012-04-19 15:50:41	150	104
gora-0.3	2013-05-03 22:34:40	186	132
gora-0.4	2014-04-16 19:17:00	214	153
gora-0.5	2014-09-16 13:05:19	242	169
gora-0.6	2015-02-13 07:06:48	244	170
gora-0.7	2017-03-17 05:19:28	401	201
gora-0.8	2017-09-15 09:57:51	442	210
jspwiki-1.4.0	2001-10-21 11:03:11	15	13
jspwiki-1.5.0	2001-11-02 18:22:56	15	13
jspwiki-1.6.0	2001-11-20 21:08:29	20	14
jspwiki-1.7.0	2002-03-16 13:02:43	46	34
jspwiki-1.8.0	2002-06-20 19:56:24	54	39
jspwiki-2.0.36	2003-03-23 09:56:52	136	103
jspwiki-2.2.19	2005-06-08 18:46:06	275	218
jspwiki-2.4.56	2006-09-21 18:48:18	426	307
jspwiki-2.6.0	2007-12-22 22:00:32	501	364
jspwiki-2.8.0	2008-10-17 05:42:36	998	730
jspwiki-2.9.0	2012-11-01 18:35:41	515	372
jspwiki-2.10.0	2014-02-03 20:44:43	486	351
knox-0.3.0	2013-10-03 14:28:05	506	388
knox-0.4.0	2014-04-14 21:26:04	590	444
knox-0.5.0	2014-10-25 21:33:42	660	496
knox-0.6.0	2015-04-27 14:45:42	722	542
knox-0.7.0	2015-12-16 00:53:42	795	590
knox-0.8.0	2016-01-31 04:51:17	807	594
knox-0.9.0	2016-04-13 18:23:43	833	609
knox-0.10.0	2016-11-01 12:57:21	857	622
knox-0.11.0	2016-12-23 15:04:39	891	650
knox-0.12.0	2017-03-15 16:20:17	918	672
knox-0.13.0	2017-08-14 17:26:40	942	685
knox-0.14.0	2017-12-06 17:58:30	1025	740
knox-1.0.0	2018-02-01 03:39:06	1050	763
kylin-0.6.1	2014-10-02 06:54:14	481	379
kylin-0.7.1	2015-06-02 13:09:49	640	492
kylin-1.0	2015-08-27 04:40:44	657	508
kylin-1.1	2015-10-09 10:07:58	658	507
kylin-1.2	2015-12-15 05:50:03	668	517
kylin-1.3	2016-03-07 08:52:07	691	533
kylin-1.5.0	2016-03-12 04:54:25	934	744
kylin-1.6.0	2016-11-23 07:35:33	1000	793
kylin-2.0.0	2017-04-26 22:56:12	1094	859
kylin-2.1.0	2017-08-13 03:14:21	1186	939
kylin-2.2.0	2017-10-29 08:02:42	1272	1006
lens-2.6.0	2016-09-22 10:45:09	785	584
lens-2.7.0	2017-12-04 21:45:01	856	629
mahout-0.1	2009-03-26 14:26:58	455	264
mahout-0.2	2009-11-12 19:17:14	661	425
mahout-0.3	2010-03-12 00:25:05	990	699
mahout-0.4	2010-10-28 14:40:13	1121	765
mahout-0.5	2011-05-28 11:45:15	1200	778
mahout-0.6	2012-02-01 12:35:29	1376	895
mahout-0.7	2012-06-12 08:29:52	1235	836
mahout-0.8	2013-07-08 01:55:42	1366	906
mahout-0.9	2014-01-29 00:32:09	1262	836
mahout-0.10.0	2015-04-11 00:50:30	1245	823
mahout-0.11.0	2015-08-06 04:31:35	1251	829
mahout-0.12.0	2016-04-11 12:20:42	1216	810
mahout-0.13.0	2017-04-15 05:09:39	1220	814
manifoldcf-0.1	2011-02-01 11:29:26	555	430
manifoldcf-0.2	2011-09-20 10:30:03	571	433
manifoldcf-0.3	2011-09-20 10:30:34	655	492
manifoldcf-0.4	2012-02-02 23:14:55	602	513
manifoldcf-0.5	2012-04-16 01:55:03	704	546
manifoldcf-0.6	2012-07-13 13:03:08	791	632
manifoldcf-1.0	2012-10-03 17:16:09	808	640
manifoldcf-1.1	2013-02-04 01:50:11	826	647
manifoldcf-1.2	2013-05-11 16:42:04	875	688
manifoldcf-1.3	2013-07-29 12:27:38	986	771
manifoldcf-1.4	2013-10-28 23:00:38	998	780
manifoldcf-1.5	2014-02-07 18:32:10	1089	862
manifoldcf-1.6	2014-04-30 20:17:46	1092	863
manifoldcf-1.7	2014-08-23 13:59:03	1169	928
manifoldcf-1.8	2014-12-29 00:58:43	1190	942
manifoldcf-1.9	2015-05-05 00:41:50	1228	978
manifoldcf-1.10	2015-09-07 08:09:40	1253	999
manifoldcf-2.0	2014-12-29 00:58:24	1147	933
manifoldcf-2.1	2015-05-05 00:41:16	1188	969
manifoldcf-2.2	2015-11-25 15:18:12	1217	995
manifoldcf-2.3	2015-12-17 12:28:34	1250	1017
manifoldcf-2.4	2016-04-17 21:09:45	1253	1020
manifoldcf-2.5	2016-08-19 00:49:56	1254	1021
manifoldcf-2.6	2016-12-30 12:23:31	1259	1026
manifoldcf-2.7	2017-04-20 21:54:27	1262	1026
manifoldcf-2.8	2017-08-18 11:15:24	1279	1041
manifoldcf-2.9	2017-12-23 00:33:27	1285	1047
manifoldcf-2.10	2018-04-19 05:11:05	1305	1058
nutch-0.7	2005-09-12 18:50:42	378	327
nutch-0.8	2006-07-25 08:37:51	381	319
nutch-0.9	2007-04-06 02:10:16	411	332
nutch-1.0	2009-03-27 20:10:11	436	358
nutch-1.1	2010-06-07 05:10:36	447	364
nutch-1.2	2010-09-24 21:37:08	454	371
nutch-1.3	2011-06-08 04:09:31	305	245
nutch-1.4	2011-11-05 00:30:53	304	246
nutch-1.5	2012-04-16 05:36:23	309	249
nutch-1.6	2012-11-23 13:48:30	312	250
nutch-1.7	2013-06-20 21:08:29	323	255
nutch-1.8	2014-03-01 14:11:42	329	257
nutch-1.9	2014-08-13 05:10:19	370	247
nutch-1.10	2015-04-29 21:40:55	415	285
nutch-1.11	2015-12-04 07:48:12	523	383
nutch-1.12	2016-06-03 04:12:10	546	398
nutch-1.13	2017-03-29 03:30:27	555	405
nutch-1.14	2017-12-18 19:08:24	569	414
nutch-2.0	2012-06-08 14:23:13	620	500
nutch-2.1	2012-09-21 14:45:52	317	252
nutch-2.2	2013-05-31 22:52:41	320	255
nutch-2.3	2015-01-09 08:36:52	440	324
opennlp-1.7.0	2016-12-31 19:33:22	803	632
opennlp-1.8.0	2017-05-17 21:19:47	842	626
parquet-mr-1.0.0	2013-07-30 04:24:31	308	221
parquet-mr-1.1.0	2013-08-26 23:40:59	314	224
parquet-mr-1.2.0	2013-09-10 00:56:08	328	236
parquet-mr-1.3.0	2013-12-23 18:57:56	388	274
parquet-mr-1.4.0	2014-04-01 20:37:44	433	309
parquet-mr-1.5.0	2014-05-22 00:56:53	450	323
parquet-mr-1.6.0	2015-04-07 23:51:08	530	377
parquet-mr-1.7.0	2015-05-15 17:47:26	530	377
parquet-mr-1.8.0	2015-07-12 00:11:06	572	402
parquet-mr-1.9.0	2016-10-19 01:00:09	630	429
santuario-java-1.0.0	2010-09-11 11:09:17	223	165
santuario-java-1.2	2010-09-11 11:09:17	223	165
santuario-java-1.4.5	2011-05-19 15:06:17	429	257
santuario-java-1.5.9	2012-01-20 10:24:05	430	265
santuario-java-2.0.0	2014-04-28 09:56:26	605	390
santuario-java-2.1.0	2017-08-14 09:50:34	660	463
systemml-0.9	2016-01-19 19:55:34	1232	851
systemml-0.10	2016-06-01 04:31:55	1316	884
systemml-0.11	2016-11-01 23:44:32	1471	973
systemml-0.12	2017-01-11 23:36:42	1471	964
systemml-0.13	2017-02-23 01:07:54	1488	971
systemml-0.14	2017-04-19 21:22:41	1550	1008
systemml-0.15	2017-09-08 08:14:52	1625	1029
systemml-1.0.0	2017-12-08 19:08:56	1659	1042
systemml-1.1.0	2018-03-23 23:45:32	1669	1035
systemml-1.2.0	2018-08-17 05:58:13	1749	1073
tika-0.1	2008-11-08 12:40:40	78	61
tika-0.2	2008-12-09 21:58:15	131	101
tika-0.3	2009-03-19 13:25:45	158	119
tika-0.4	2009-07-25 18:02:02	175	135
tika-0.5	2009-11-14 16:28:26	194	146
tika-0.6	2010-01-20 06:29:26	199	149
tika-0.7	2010-03-31 19:35:14	213	159
tika-0.8	2010-11-07 21:15:06	269	200
tika-0.9	2011-02-14 04:41:59	299	222
tika-0.10	2011-09-26 06:18:19	391	269
tika-1.0	2011-11-03 19:41:10	392	269
tika-1.1	2012-03-07 20:45:58	406	275
tika-1.2	2012-07-10 18:14:58	445	307
tika-1.3	2013-06-16 01:54:21	452	311
tika-1.4	2013-06-16 01:50:40	464	318
tika-1.5	2014-07-11 16:21:03	472	321
tika-1.6	2014-07-31 18:29:32	511	344
tika-1.7	2015-01-15 17:28:14	552	363
tika-1.8	2015-04-20 20:25:36	655	436
tika-1.9	2015-06-07 01:09:17	721	464
tika-1.10	2015-09-21 04:48:46	739	473
tika-1.11	2015-10-19 06:21:36	753	479
tika-1.12	2016-01-25 18:49:12	764	488
tika-1.13	2016-05-09 17:11:54	791	506
tika-1.14	2016-10-19 17:43:56	837	538
tika-1.15	2017-05-24 00:43:34	968	636
tika-1.16	2017-07-08 01:58:19	978	642
tika-1.17	2017-12-08 23:30:02	1000	651
wss4j-1.5.0	2006-05-02 18:17:17	335	278
wss4j-1.6.0	2011-04-06 15:48:42	196	135
wss4j-2.0.0	2014-05-02 10:47:39	693	482
wss4j-2.1.0	2015-04-22 14:14:05	704	493
wss4j-2.2.0	2017-08-29 11:56:40	719	500

Comparison of approaches for the identification of commit-->issue links

linkdata = pd.DataFrame(index=projects, columns=['total_commits_any','total_links_validated','total_commits_validated',
                                                 'total_links_szz','correct_links_szz','missed_links_szz','additional_links_szz',
                                                 'total_links_jira','correct_links_jira','missed_links_jira','additional_links_jira',
                                                 'total_commits_szz','correct_commits_szz','wrong_commits_szz',
                                                 'total_commits_jira','correct_commits_jira','wrong_commits_jira',
                                                 'missed_commits_szz', 'missed_commits_jira',
                                                 'links_validated_heuristic'])
linkdata.fillna(0, inplace=True)

direct_link_jira = re.compile('(?P<ID>[A-Z][A-Z0-9_]+-[0-9]+)', re.M)

for project in projects:
    project_id = Project.objects(name=project).get().id
    vcs_system_id = VCSSystem.objects(project_id=project_id).get().id
    for commit in Commit.objects(vcs_system_id=vcs_system_id).only('linked_issue_ids', 'message', 'labels','szz_issue_ids','fixed_issue_ids'):
        total_commits_any = 0
        total_links_szz = 0
        total_links_jira = 0
        total_links_validated = 0
        total_commits_szz = 0
        total_commits_jira = 0
        total_commits_validated = 0
        correct_links_szz = 0
        correct_links_jira = 0
        missed_links_szz = 0
        missed_links_jira = 0
        correct_commits_szz = 0
        correct_commits_jira = 0
        missed_commits_szz = 0
        missed_commits_jira = 0
        links_validated_heurisitic = 0
        
        if commit.szz_issue_ids:
            total_links_szz = len(commit.szz_issue_ids)
        if commit.linked_issue_ids:
            for issue in Issue.objects(id__in=commit.linked_issue_ids):
                if issue.issue_type and issue.issue_type.lower()=="bug" and jira_is_resolved_and_fixed(issue):
                    total_links_jira += 1
                    
            if len(commit.linked_issue_ids)==1:
                issue = Issue.objects(id=commit.linked_issue_ids[0]).get()
                id_number_issue = int(issue.external_id.split("-")[1])
                id_number_commit = None
                # find match to JIRA pattern and determine if it is at the beginning of the commit
                for m in direct_link_jira.finditer(commit.message.strip()):
                    if m.start()<=1:
                        match_in_commit = m.group('ID').upper()
                        id_number_commit = int(match_in_commit.split("-")[1])
                # if there is a JIRA pattern at the beginning of the commit, check if the linked number matches the issue
                # check only for numbers instead of whole external id takes care of typos
                if id_number_commit is not None and id_number_issue==id_number_commit:
                    # if this is the case, heursitic found a match
                    if issue.issue_type and issue.issue_type.lower()=="bug" and jira_is_resolved_and_fixed(issue):
                        commit.fixed_issue_ids = [issue.id]
                        links_validated_heurisitic += 1
                    
        if commit.fixed_issue_ids:
            for issue in Issue.objects(id__in=commit.fixed_issue_ids):
                if issue.issue_type and issue.issue_type.lower()=="bug" and jira_is_resolved_and_fixed(issue):
                    total_links_validated += 1
                    if commit.szz_issue_ids and issue.id in commit.szz_issue_ids:
                        correct_links_szz += 1
                    else:
                        missed_links_szz += 1
                    if commit.linked_issue_ids and issue.id in commit.linked_issue_ids:
                        correct_links_jira += 1
                    else:
                        missed_links_jira += 1
                        
        if total_links_szz>0:
            total_commits_szz += 1
        if total_links_jira>0:
            total_commits_jira += 1
        if total_links_validated>0:
            total_commits_validated += 1
        if total_links_szz>0 or total_links_jira>0 or total_links_validated>0:
            total_commits_any += 1
        if correct_links_jira>0 or correct_links_jira>0:
            if total_links_szz>0:
                correct_commits_szz += 1
            else:
                missed_commits_szz += 1
            if total_links_jira>0:
                correct_commits_jira += 1
            else:
                missed_commits_jira += 1
        
        linkdata.loc[project,'total_commits_any'] += total_commits_any
        linkdata.loc[project,'total_links_validated'] += total_links_validated
        linkdata.loc[project,'total_commits_validated'] += total_commits_validated
        linkdata.loc[project,'total_links_szz'] += total_links_szz
        linkdata.loc[project,'total_links_jira'] += total_links_jira
        linkdata.loc[project,'total_commits_szz'] += total_commits_szz
        linkdata.loc[project,'total_commits_jira'] += total_commits_jira
        linkdata.loc[project,'correct_links_szz'] += correct_links_szz
        linkdata.loc[project,'correct_links_jira'] += correct_links_jira
        linkdata.loc[project,'missed_links_szz'] += missed_links_szz
        linkdata.loc[project,'missed_links_jira'] += missed_links_jira
        linkdata.loc[project,'additional_links_szz'] += total_links_szz-correct_links_szz
        linkdata.loc[project,'additional_links_jira'] += total_links_jira-correct_links_jira
        linkdata.loc[project,'correct_commits_szz'] += correct_commits_szz
        linkdata.loc[project,'correct_commits_jira'] += correct_commits_jira
        linkdata.loc[project,'wrong_commits_szz'] += total_commits_szz-correct_commits_szz
        linkdata.loc[project,'wrong_commits_jira'] += total_commits_jira-correct_commits_jira
        linkdata.loc[project,'missed_commits_szz'] += missed_commits_szz
        linkdata.loc[project,'missed_commits_jira'] += missed_commits_jira
        linkdata.loc[project,'links_validated_heuristic'] += links_validated_heurisitic
        
linkdata['percent_link_correct_szz'] = linkdata['correct_links_szz']/linkdata['total_links_validated']
linkdata['percent_link_correct_jira'] = linkdata['correct_links_jira']/linkdata['total_links_validated']
linkdata['percent_link_missed_szz'] = linkdata['missed_links_szz']/linkdata['total_links_validated']
linkdata['percent_link_missed_jira'] = linkdata['missed_links_jira']/linkdata['total_links_validated']
linkdata['percent_link_additional_szz'] = linkdata['additional_links_szz']/linkdata['total_links_validated']
linkdata['percent_link_additional_jira'] = linkdata['additional_links_jira']/linkdata['total_links_validated']

display(linkdata)

df = linkdata[['percent_link_correct_szz','percent_link_correct_jira',
               'percent_link_additional_szz','percent_link_additional_jira']]

print('mean values:')
print(df.mean().to_string())
print()
print('std values: ')
print(df.std().to_string())
print()
print('median values:')
print(df.median().to_string())
print()
print('MAD values:')
print(mad(df).to_string())

df.columns = ['SZZ','JL',
              'SZZ','JL']

f, axes = plt.subplots(2, 1, figsize=(6,2))

sns.boxplot(data=df.iloc[:,0:2], orient='h', color='lightgray', ax=axes[0])
sns.swarmplot(data=df.iloc[:,0:2], orient='h', color='gray', ax=axes[0])
axes[0].set_xlabel('Percentage of actual links that were identified')
axes[0].set_xlim(0,1)

sns.boxplot(data=df.iloc[:,2:4], orient='h', color='lightgray', ax=axes[1])
sns.swarmplot(data=df.iloc[:,2:4], orient='h', color='gray', ax=axes[1])
axes[1].set_xlabel('Additional links found in relation to actual links')
axes[1].set_xlim(0,1)
plt.subplots_adjust(left=None, bottom=-0.4, right=None, top=None, wspace=None, hspace=0.5)
#plt.savefig("links-correct.pdf", bbox_inches='tight')
plt.show()

	total_commits_any	total_links_validated	total_commits_validated	total_links_szz	correct_links_szz	missed_links_szz	additional_links_szz	total_links_jira	correct_links_jira	missed_links_jira	additional_links_jira	total_commits_szz	correct_commits_szz	wrong_commits_szz	total_commits_jira	correct_commits_jira	wrong_commits_jira	missed_commits_szz	missed_commits_jira	links_validated_heuristic	percent_link_correct_szz	percent_link_correct_jira	percent_link_missed_szz	percent_link_missed_jira	percent_link_additional_szz	percent_link_additional_jira
ant-ivy	748	721	708	742	696	25	46	732	719	2	13	722	681	41	716	706	10	25	0	84	0.965326	0.997226	0.034674	0.002774	0.063800	0.018031
archiva	1013	980	940	602	484	496	118	991	978	2	13	535	471	64	948	938	10	467	0	829	0.493878	0.997959	0.506122	0.002041	0.120408	0.013265
calcite	1143	923	895	1249	862	61	387	970	923	0	47	1068	844	224	925	895	30	51	0	788	0.933911	1.000000	0.066089	0.000000	0.419285	0.050921
cayenne	1859	1248	1240	2776	1225	23	1551	1254	1245	3	9	1832	1217	615	1242	1237	5	20	0	1178	0.981571	0.997596	0.018429	0.002404	1.242788	0.007212
commons-bcel	95	85	85	62	62	23	0	95	85	0	10	62	62	0	95	85	10	23	0	66	0.729412	1.000000	0.270588	0.000000	0.000000	0.117647
commons-beanutils	124	123	118	92	84	39	8	123	121	2	2	87	81	6	119	118	1	37	0	77	0.682927	0.983740	0.317073	0.016260	0.065041	0.016260
commons-codec	161	140	136	71	65	75	6	159	140	0	19	69	63	6	155	136	19	73	0	97	0.464286	1.000000	0.535714	0.000000	0.042857	0.135714
commons-collections	181	181	180	35	34	147	1	182	181	0	1	35	34	1	180	180	0	146	0	116	0.187845	1.000000	0.812155	0.000000	0.005525	0.005525
commons-compress	415	293	291	355	232	61	123	324	292	1	32	326	234	92	323	291	32	57	0	206	0.791809	0.996587	0.208191	0.003413	0.419795	0.109215
commons-configuration	351	340	340	216	215	125	1	349	337	3	12	215	212	3	348	337	11	125	0	285	0.632353	0.991176	0.367647	0.008824	0.002941	0.035294
commons-dbcp	207	201	191	122	106	95	16	201	200	1	1	119	104	15	192	191	1	87	0	44	0.527363	0.995025	0.472637	0.004975	0.079602	0.004975
commons-digester	72	38	38	65	31	7	34	38	38	0	0	65	31	34	38	38	0	7	0	22	0.815789	1.000000	0.184211	0.000000	0.894737	0.000000
commons-io	249	217	216	180	176	41	4	246	217	0	29	180	176	4	245	216	29	40	0	176	0.811060	1.000000	0.188940	0.000000	0.018433	0.133641
commons-jcs	183	106	104	135	90	16	45	165	102	4	63	112	90	22	161	102	59	12	0	37	0.849057	0.962264	0.150943	0.037736	0.424528	0.594340
commons-jexl	288	246	233	251	213	33	38	271	246	0	25	236	206	30	258	233	25	27	0	102	0.865854	1.000000	0.134146	0.000000	0.154472	0.101626
commons-lang	562	523	516	366	324	199	42	553	519	4	34	346	322	24	537	512	25	190	0	278	0.619503	0.992352	0.380497	0.007648	0.080306	0.065010
commons-math	733	734	719	472	458	276	14	743	734	0	9	468	458	10	726	719	7	261	0	341	0.623978	1.000000	0.376022	0.000000	0.019074	0.012262
commons-net	255	236	233	235	216	20	19	253	236	0	17	218	213	5	250	233	17	20	0	203	0.915254	1.000000	0.084746	0.000000	0.080508	0.072034
commons-scxml	155	123	123	127	78	45	49	123	122	1	1	109	78	31	123	122	1	44	0	70	0.634146	0.991870	0.365854	0.008130	0.398374	0.008130
commons-validator	168	101	101	155	90	11	65	105	98	3	7	153	90	63	104	98	6	8	0	80	0.891089	0.970297	0.108911	0.029703	0.643564	0.069307
commons-vfs	214	198	195	134	133	65	1	214	196	2	18	132	130	2	211	193	18	63	0	144	0.671717	0.989899	0.328283	0.010101	0.005051	0.090909
deltaspike	500	495	489	507	495	0	12	497	495	0	2	499	489	10	491	489	2	0	0	474	1.000000	1.000000	0.000000	0.000000	0.024242	0.004040
eagle	297	256	245	210	108	148	102	255	254	2	1	155	104	51	244	243	1	139	0	217	0.421875	0.992188	0.578125	0.007812	0.398438	0.003906
giraph	386	362	360	403	361	1	42	365	362	0	3	384	359	25	361	360	1	1	0	336	0.997238	1.000000	0.002762	0.000000	0.116022	0.008287
gora	217	204	200	190	157	47	33	204	199	5	5	170	155	15	202	197	5	42	0	138	0.769608	0.975490	0.230392	0.024510	0.161765	0.024510
jspwiki	551	422	370	363	123	299	240	423	419	3	4	299	121	178	371	368	3	247	0	124	0.291469	0.992891	0.708531	0.007109	0.568720	0.009479
knox	915	855	851	1010	848	7	162	871	854	1	17	908	844	64	863	850	13	6	0	818	0.991813	0.998830	0.008187	0.001170	0.189474	0.019883
kylin	2078	1976	1965	2022	1870	106	152	1968	1938	38	30	1968	1864	104	1953	1927	26	63	0	1812	0.946356	0.980769	0.053644	0.019231	0.076923	0.015182
lens	579	498	497	583	498	0	85	509	498	0	11	576	497	79	503	497	6	0	0	483	1.000000	1.000000	0.000000	0.000000	0.170683	0.022088
mahout	839	722	710	1094	719	3	375	724	717	5	7	833	705	128	715	708	7	3	0	656	0.995845	0.993075	0.004155	0.006925	0.519391	0.009695
manifoldcf	1361	1354	1340	1186	1164	190	22	1353	1350	4	3	1171	1148	23	1338	1336	2	188	0	34	0.859675	0.997046	0.140325	0.002954	0.016248	0.002216
nutch	1134	972	955	1200	850	122	350	979	967	5	12	1013	835	178	962	950	12	115	0	679	0.874486	0.994856	0.125514	0.005144	0.360082	0.012346
opennlp	391	355	353	409	355	0	54	355	355	0	0	391	353	38	353	353	0	0	0	341	1.000000	1.000000	0.000000	0.000000	0.152113	0.000000
parquet-mr	464	253	241	1342	253	0	1089	306	253	0	53	419	241	178	291	241	50	0	0	219	1.000000	1.000000	0.000000	0.000000	4.304348	0.209486
santuario-java	160	146	144	156	139	7	17	154	146	0	8	152	137	15	152	144	8	7	0	130	0.952055	1.000000	0.047945	0.000000	0.116438	0.054795
systemml	599	570	566	476	416	154	60	580	569	1	11	449	417	32	570	565	5	148	0	523	0.729825	0.998246	0.270175	0.001754	0.105263	0.019298
tika	1205	1126	1111	1119	977	149	142	1122	1114	12	8	1058	966	92	1106	1102	4	136	0	864	0.867673	0.989343	0.132327	0.010657	0.126110	0.007105
wss4j	457	398	390	486	371	27	115	398	395	3	3	432	362	70	387	387	0	25	0	339	0.932161	0.992462	0.067839	0.007538	0.288945	0.007538

mean values:
percent_link_correct_szz        0.782058
percent_link_correct_jira       0.993979
percent_link_additional_szz     0.338850
percent_link_additional_jira    0.055031

std values: 
percent_link_correct_szz        0.211924
percent_link_correct_jira       0.008840
percent_link_additional_szz     0.713257
percent_link_additional_jira    0.102333

median values:
percent_link_correct_szz        0.854366
percent_link_correct_jira       0.997411
percent_link_additional_szz     0.123259
percent_link_additional_jira    0.017145

MAD values:
percent_link_correct_szz        0.196186
percent_link_correct_jira       0.003838
percent_link_additional_szz     0.154940
percent_link_additional_jira    0.018736

Sampling of data for additional JLM validation

sample_size = 1000

commits_without_validation = []
for commit in Commit.objects(validations__exists=False).only('id', 'linked_issue_ids', 'szz_issue_ids'):
    no_jl = commit.linked_issue_ids is None or len(commit.linked_issue_ids)==0
    no_szz = commit.szz_issue_ids is None or len(commit.szz_issue_ids)==0
    if no_jl and no_szz:
        commits_without_validation.append(commit.id)
        continue


commits_heuristicly_validated = []
direct_link_jira = re.compile('(?P<ID>[A-Z][A-Z0-9_]+-[0-9]+)', re.M)
for commit in Commit.objects().only('id', 'revision_hash', 'vcs_system_id','linked_issue_ids', 'message', 'labels','szz_issue_ids'):
    # heuristic only applies to commits that have a single issue link
    if commit.linked_issue_ids and len(commit.linked_issue_ids)==1:
        issue = Issue.objects(id=commit.linked_issue_ids[0]).get()
        # we only look at links to fixed bugs
        if issue.issue_type and issue.issue_type.lower()=="bug" and jira_is_resolved_and_fixed(issue):
            id_number_issue = int(issue.external_id.split("-")[1])
            id_number_commit = None
            # find match to JIRA pattern and determine if it is at the beginning of the commit
            for m in direct_link_jira.finditer(commit.message.strip()):
                if m.start()<=1:
                    match_in_commit = m.group('ID').upper()
                    id_number_commit = int(match_in_commit.split("-")[1])
            # if there is a JIRA pattern at the beginning of the commit, check if the linked number matches the issue
            # check only for numbers instead of whole external id takes care of typos
            if id_number_commit is not None and id_number_issue==id_number_commit:
                # if this is the case, heursitic found a match
                commits_heuristicly_validated.append(commit.id)

vcs_system_map = {}
for vcs_system in VCSSystem.objects():
    vcs_system_map[vcs_system.id] = Project.objects(id=vcs_system.project_id).get().name
    
sampled_commits_without_validation = pd.DataFrame(columns=['project','revision_hash', 'message','num_links','reason'])

random.seed("sampled_commits_without_validation")
sampled_commits = random.sample(commits_without_validation, k=sample_size)
for commit_id in sampled_commits:
    commit = Commit.objects(id=commit_id).only('message','revision_hash','vcs_system_id').get()
    sampled_commits_without_validation.loc[len(sampled_commits_without_validation)] = [vcs_system_map[commit.vcs_system_id], commit.revision_hash, commit.message, None, None]
    
sampled_commits_without_validation.to_excel("sampled_commits_without_validation.xlsx")

sampled_commits_heuristicly_validated = pd.DataFrame(columns=['project','revision_hash', 'message','linked_issue','correct','reason'])

random.seed("sampled_commits_heuristicly_validated")
sampled_commits = random.sample(commits_heuristicly_validated, k=sample_size)
for commit_id in sampled_commits:
    commit = Commit.objects(id=commit_id).only('message','revision_hash','vcs_system_id','linked_issue_ids').get()
    issue = Issue.objects(id=commit.linked_issue_ids[0]).only('external_id').get()
    sampled_commits_heuristicly_validated.loc[len(sampled_commits_heuristicly_validated)] = [vcs_system_map[commit.vcs_system_id], commit.revision_hash, commit.message, issue.external_id, None, None]
    
sampled_commits_heuristicly_validated.to_excel("sampled_commits_heuristicly_validated.xlsx")

Correctness of the issue labels of type 'bug'

issuedata = pd.DataFrame(index=projects, columns=['num_bugs','actual_bug',
                                                 'actual_impr','actual_test', 
                                                 'actual_doc','actual_other'])
issuedata.fillna(0, inplace=True)

for project in projects:
    project_id = Project.objects(name=project).get().id
    its_id = IssueSystem.objects(project_id=project_id).get().id
    num_linked_bugs = 0
    for issue in Issue.objects(issue_system_id=its_id,issue_type__iexact='bug'):
        if jira_is_resolved_and_fixed(issue):
            if Commit.objects(linked_issue_ids=issue.id).count()>0:
                num_linked_bugs += 1
                issuedata.at[project,'num_bugs'] += 1
                if issue.issue_type_verified:
                    if issue.issue_type_verified=="bug":
                        issuedata.at[project,'actual_bug'] += 1
                    if issue.issue_type_verified=="improvement":
                        issuedata.at[project,'actual_impr'] += 1
                    if issue.issue_type_verified=="test":
                        issuedata.at[project,'actual_test'] += 1
                    if issue.issue_type_verified=="documentation":
                        issuedata.at[project,'actual_doc'] += 1
                    if issue.issue_type_verified=="other":
                            issuedata.at[project,'actual_other'] += 1               

issuedata['percent_bug'] = issuedata['actual_bug']/issuedata['num_bugs']
issuedata['percent_impr'] = issuedata['actual_impr']/issuedata['num_bugs']
issuedata['percent_test'] = issuedata['actual_test']/issuedata['num_bugs']
issuedata['percent_doc'] = issuedata['actual_doc']/issuedata['num_bugs']
issuedata['percent_other'] = issuedata['actual_other']/issuedata['num_bugs']

display(issuedata)

df = issuedata[['percent_bug', 'percent_impr', 'percent_test', 'percent_doc', 'percent_other']]
df.columns = ['BUG', 'IMPROVEMENT', 'TEST', 'DOC', 'OTHER']

print("mean values")
print(df.mean().to_string())
print()
print("sd values")
print(df.std().to_string())
print()
print("median values")
print(df.median().to_string())
print()
print("MAD values")
print(mad(df).to_string())

ax = sns.boxplot(data=df, orient='h', color='lightgray')
ax = sns.swarmplot(data=df, orient='h', color='gray')
ax.set_xlabel('Percentage of Closed Linked Bug Issues')
ax.set_xlim(0,1)
plt.show()
#plt.savefig("issue-type-validation.png", bbox_inches='tight')

	num_bugs	actual_bug	actual_impr	actual_test	actual_doc	actual_other	percent_bug	percent_impr	percent_test	percent_doc	percent_other
ant-ivy	535	420	64	3	40	8	0.785047	0.119626	0.005607	0.074766	0.014953
archiva	542	313	173	16	20	20	0.577491	0.319188	0.029520	0.036900	0.036900
calcite	842	396	341	29	42	34	0.470309	0.404988	0.034442	0.049881	0.040380
cayenne	530	370	133	9	7	11	0.698113	0.250943	0.016981	0.013208	0.020755
commons-bcel	53	32	15	1	4	1	0.603774	0.283019	0.018868	0.075472	0.018868
commons-beanutils	76	44	25	1	2	4	0.578947	0.328947	0.013158	0.026316	0.052632
commons-codec	64	28	20	2	13	1	0.437500	0.312500	0.031250	0.203125	0.015625
commons-collections	115	44	50	2	16	3	0.382609	0.434783	0.017391	0.139130	0.026087
commons-compress	172	121	34	2	8	7	0.703488	0.197674	0.011628	0.046512	0.040698
commons-configuration	188	135	29	3	11	10	0.718085	0.154255	0.015957	0.058511	0.053191
commons-dbcp	127	69	43	1	9	5	0.543307	0.338583	0.007874	0.070866	0.039370
commons-digester	26	17	6	0	2	1	0.653846	0.230769	0.000000	0.076923	0.038462
commons-io	131	73	22	9	20	7	0.557252	0.167939	0.068702	0.152672	0.053435
commons-jcs	80	57	13	3	5	2	0.712500	0.162500	0.037500	0.062500	0.025000
commons-jexl	84	56	17	3	7	1	0.666667	0.202381	0.035714	0.083333	0.011905
commons-lang	318	155	89	18	44	12	0.487421	0.279874	0.056604	0.138365	0.037736
commons-math	415	238	120	18	35	4	0.573494	0.289157	0.043373	0.084337	0.009639
commons-net	176	129	36	2	9	0	0.732955	0.204545	0.011364	0.051136	0.000000
commons-scxml	70	46	15	6	2	1	0.657143	0.214286	0.085714	0.028571	0.014286
commons-validator	73	56	10	2	3	2	0.767123	0.136986	0.027397	0.041096	0.027397
commons-vfs	156	90	44	7	6	9	0.576923	0.282051	0.044872	0.038462	0.057692
deltaspike	302	143	86	25	21	27	0.473510	0.284768	0.082781	0.069536	0.089404
eagle	225	116	76	13	5	15	0.515556	0.337778	0.057778	0.022222	0.066667
giraph	337	135	137	11	14	40	0.400593	0.406528	0.032641	0.041543	0.118694
gora	113	55	31	4	6	17	0.486726	0.274336	0.035398	0.053097	0.150442
jspwiki	274	173	80	8	5	8	0.631387	0.291971	0.029197	0.018248	0.029197
knox	568	226	285	24	13	20	0.397887	0.501761	0.042254	0.022887	0.035211
kylin	732	479	218	10	15	10	0.654372	0.297814	0.013661	0.020492	0.013661
lens	397	228	88	28	23	30	0.574307	0.221662	0.070529	0.057935	0.075567
mahout	513	246	156	18	46	47	0.479532	0.304094	0.035088	0.089669	0.091618
manifoldcf	633	304	178	11	48	92	0.480253	0.281201	0.017378	0.075829	0.145340
nutch	643	353	211	6	22	51	0.548989	0.328149	0.009331	0.034215	0.079316
opennlp	219	102	67	11	17	22	0.465753	0.305936	0.050228	0.077626	0.100457
parquet-mr	185	86	64	8	10	17	0.464865	0.345946	0.043243	0.054054	0.091892
santuario-java	83	52	25	3	2	1	0.626506	0.301205	0.036145	0.024096	0.012048
systemml	452	268	137	10	10	27	0.592920	0.303097	0.022124	0.022124	0.059735
tika	605	359	205	14	6	21	0.593388	0.338843	0.023140	0.009917	0.034711
wss4j	241	153	76	1	6	5	0.634855	0.315353	0.004149	0.024896	0.020747

mean values
BUG            0.576458
IMPROVEMENT    0.283038
TEST           0.032079
DOC            0.059749
OTHER          0.048677

sd values
BUG            0.106245
IMPROVEMENT    0.081909
TEST           0.021392
DOC            0.041819
OTHER          0.037285

median values
BUG            0.577207
IMPROVEMENT    0.290564
TEST           0.030385
DOC            0.052117
OTHER          0.038099

MAD values
BUG            0.132875
IMPROVEMENT    0.064370
TEST           0.019579
DOC            0.035967
OTHER          0.030563

Comparison of approaches for identifying bugfixing commits

bugfixdata = pd.DataFrame(index=projects, columns=['num_commits',
                                                   'SZZ_tp','SZZ_fn','SZZ_fp',
                                                   'JL_tp','JL_fn','JL_fp',
                                                   'JLM_tp','JLM_fn','JLM_fp',
                                                   'JLMIV'])
bugfixdata.fillna(0, inplace=True)

for project in projects:
    project_id = Project.objects(name=project).get().id
    vcs_system_id = VCSSystem.objects(project_id=project_id).get().id
    for commit in Commit.objects(vcs_system_id=vcs_system_id).only('labels','fixed_issue_ids','id'):
        bugfixdata.at[project,'num_commits'] += 1
        is_jlmiv = commit.labels and "validated_bugfix" in commit.labels and commit.labels["validated_bugfix"]
        is_szz = commit.labels and "adjustedszz_bugfix" in commit.labels and commit.labels["adjustedszz_bugfix"]
        is_jl = commit.labels and "issueonly_bugfix" in commit.labels and commit.labels["issueonly_bugfix"]
        is_jlm = False
        if commit.fixed_issue_ids and len(commit.fixed_issue_ids):
            for issue in Issue.objects(id__in=commit.fixed_issue_ids):
                is_jlm |= bool(issue.issue_type and issue.issue_type.lower()=="bug" and jira_is_resolved_and_fixed(issue))
                if issue.parent_issue_id:
                    parent_issue = Issue.objects(id=issue.parent_issue_id).get()
                    is_jlm |= parent_issue.issue_type.lower()=="bug" and jira_is_resolved_and_fixed(parent_issue)
        
        if is_jlmiv:
            bugfixdata.at[project,'JLMIV'] += 1
            
            if is_szz:
                bugfixdata.at[project,'SZZ_tp'] += 1
            else:
                bugfixdata.at[project,'SZZ_fn'] += 1
                
            if is_jl:
                bugfixdata.at[project,'JL_tp'] += 1
            else:
                bugfixdata.at[project,'JL_fn'] += 1
                
            if is_jlm:
                bugfixdata.at[project,'JLM_tp'] += 1
            else:
                bugfixdata.at[project,'JLM_fn'] += 1
        else:
            if is_szz:
                bugfixdata.at[project,'SZZ_fp'] += 1
            if is_jl:
                bugfixdata.at[project,'JL_fp'] += 1
            if is_jlm:
                bugfixdata.at[project,'JLM_fp'] += 1
            
bugfixdata['percent_szz_tp'] = bugfixdata['SZZ_tp']/bugfixdata['JLMIV']
bugfixdata['percent_szz_fn'] = bugfixdata['SZZ_fn']/bugfixdata['JLMIV']
bugfixdata['percent_szz_fp'] = bugfixdata['SZZ_fp']/bugfixdata['JLMIV']
bugfixdata['percent_jl_tp'] = bugfixdata['JL_tp']/bugfixdata['JLMIV']
bugfixdata['percent_jl_fn'] = bugfixdata['JL_fn']/bugfixdata['JLMIV']
bugfixdata['percent_jl_fp'] = bugfixdata['JL_fp']/bugfixdata['JLMIV']
bugfixdata['percent_jlm_tp'] = bugfixdata['JLM_tp']/bugfixdata['JLMIV']
bugfixdata['percent_jlm_fn'] = bugfixdata['JLM_fn']/bugfixdata['JLMIV']
bugfixdata['percent_jlm_fp'] = bugfixdata['JLM_fp']/bugfixdata['JLMIV']

bugfixdata['percent_szz_total'] = (bugfixdata['SZZ_tp']+bugfixdata['SZZ_fp'])/bugfixdata['num_commits']
bugfixdata['percent_jl_total'] = (bugfixdata['JL_tp']+bugfixdata['JL_fp'])/bugfixdata['num_commits']
bugfixdata['percent_jlm_total'] = (bugfixdata['JLM_tp']+bugfixdata['JLM_fp'])/bugfixdata['num_commits']
bugfixdata['percent_jlmiv_total'] = bugfixdata['JLMIV']/bugfixdata['num_commits']

display(bugfixdata)

df = bugfixdata[['percent_szz_total', 'percent_jl_total', 'percent_jlm_total', 'percent_jlmiv_total']]
df.columns = ['SZZ', 'JL', 'JLM', 'JLMIV']

print("percentage of total commits that is detected as bugfixing")
print("mean values")
print(df.mean().to_string())
print()
print("std values")
print(df.std().to_string())
print()
print("median values")
print(df.median().to_string())
print()
print("MAD values")
print(mad(df).to_string())

ax = sns.boxplot(data=df, orient='h', color='lightgray')
ax = sns.swarmplot(data=df, orient='h', color='gray')
ax.set_xlabel('Percentage of total commits that are labelled as bug fixing.')
plt.show()
#plt.savefig("bugfix-percentage.pdf", bbox_inches='tight')

df = bugfixdata[['percent_szz_tp', 'percent_jl_tp', 'percent_jlm_tp',
                 'percent_szz_fn', 'percent_jl_fn', 'percent_jlm_fn',
                 'percent_szz_fp', 'percent_jl_fp', 'percent_jlm_fp']]

print("percentage of correctely labelled bug fixing commits")
print("mean values")
print(df.mean().to_string())
print()
print("std values")
print(df.std().to_string())
print()
print("median values")
print(df.median().to_string())
print()
print("MAD values")
print(mad(df).to_string())

df.columns = ['SZZ', 'JL', 'JLM', 'SZZ', 'JL', 'JLM', 'SZZ', 'JL', 'JLM']

f, axes = plt.subplots(2, 1, figsize=(6,2))

sns.boxplot(data=df.iloc[:,0:3], orient='h', color='lightgray', ax=axes[0])
sns.swarmplot(data=df.iloc[:,0:3], orient='h', color='gray', ax=axes[0])
axes[0].set_xlabel('Percentage of actual bug fixing commits that were identified (true positives)')

sns.boxplot(data=df.iloc[:,6:9], orient='h', color='lightgray', ax=axes[1])
sns.swarmplot(data=df.iloc[:,6:9], orient='h', color='gray', ax=axes[1])
axes[1].set_xlabel('Percentage of additionally detected bug fixing commits (false positives)')

plt.subplots_adjust(left=None, bottom=-0.4, right=None, top=None, wspace=None, hspace=0.5)
plt.show()
#plt.savefig("bugfix-details.pdf", bbox_inches='tight')

	num_commits	SZZ_tp	SZZ_fn	SZZ_fp	JL_tp	JL_fn	JL_fp	JLM_tp	JLM_fn	JLM_fp	JLMIV	percent_szz_tp	percent_szz_fn	percent_szz_fp	percent_jl_tp	percent_jl_fn	percent_jl_fp	percent_jlm_tp	percent_jlm_fn	percent_jlm_fp	percent_szz_total	percent_jl_total	percent_jlm_total	percent_jlmiv_total
ant-ivy	3189	551	17	171	567	1	149	568	0	140	568	0.970070	0.029930	0.301056	0.998239	0.001761	0.262324	1.0	0.0	0.246479	0.226403	0.224522	0.222013	0.178112
archiva	10261	278	265	257	542	1	406	543	0	397	543	0.511971	0.488029	0.473297	0.998158	0.001842	0.747698	1.0	0.0	0.731123	0.052139	0.092389	0.091609	0.052919
calcite	2926	405	22	663	427	0	526	427	0	496	427	0.948478	0.051522	1.552693	1.000000	0.000000	1.231850	1.0	0.0	1.161593	0.365003	0.325701	0.315448	0.145933
cayenne	6619	823	27	1009	848	2	426	850	0	422	850	0.968235	0.031765	1.187059	0.997647	0.002353	0.501176	1.0	0.0	0.496471	0.276779	0.192476	0.192174	0.128418
commons-bcel	1429	31	18	31	49	0	46	49	0	36	49	0.632653	0.367347	0.632653	1.000000	0.000000	0.938776	1.0	0.0	0.734694	0.043387	0.066480	0.059482	0.034290
commons-beanutils	1341	44	15	43	59	0	60	59	0	59	59	0.745763	0.254237	0.728814	1.000000	0.000000	1.016949	1.0	0.0	1.000000	0.064877	0.088740	0.087994	0.043997
commons-codec	1838	21	38	48	59	0	97	59	0	78	59	0.355932	0.644068	0.813559	1.000000	0.000000	1.644068	1.0	0.0	1.322034	0.037541	0.084875	0.074538	0.032100
commons-collections	3380	18	70	17	88	0	92	88	0	92	88	0.204545	0.795455	0.193182	1.000000	0.000000	1.045455	1.0	0.0	1.045455	0.010355	0.053254	0.053254	0.026036
commons-compress	2755	168	38	158	206	0	117	206	0	85	206	0.815534	0.184466	0.766990	1.000000	0.000000	0.567961	1.0	0.0	0.412621	0.118330	0.117241	0.105626	0.074773
commons-configuration	3717	171	72	44	242	1	106	243	0	97	243	0.703704	0.296296	0.181070	0.995885	0.004115	0.436214	1.0	0.0	0.399177	0.057842	0.093624	0.091472	0.065375
commons-dbcp	2205	51	55	68	106	0	86	106	0	85	106	0.481132	0.518868	0.641509	1.000000	0.000000	0.811321	1.0	0.0	0.801887	0.053968	0.087075	0.086621	0.048073
commons-digester	2535	23	3	42	26	0	12	26	0	12	26	0.884615	0.115385	1.615385	1.000000	0.000000	0.461538	1.0	0.0	0.461538	0.025641	0.014990	0.014990	0.010256
commons-io	2262	98	31	82	129	0	116	129	0	87	129	0.759690	0.240310	0.635659	1.000000	0.000000	0.899225	1.0	0.0	0.674419	0.079576	0.108311	0.095491	0.057029
commons-jcs	1622	60	12	52	71	1	90	72	0	32	72	0.833333	0.166667	0.722222	0.986111	0.013889	1.250000	1.0	0.0	0.444444	0.069051	0.099260	0.064118	0.044390
commons-jexl	3276	148	13	88	161	0	103	161	0	78	161	0.919255	0.080745	0.546584	1.000000	0.000000	0.639752	1.0	0.0	0.484472	0.072039	0.080586	0.072955	0.049145
commons-lang	5792	140	102	206	240	2	302	242	0	279	242	0.578512	0.421488	0.851240	0.991736	0.008264	1.247934	1.0	0.0	1.152893	0.059738	0.093577	0.089952	0.041782
commons-math	7222	273	123	195	396	0	332	396	0	325	396	0.689394	0.310606	0.492424	1.000000	0.000000	0.838384	1.0	0.0	0.820707	0.064802	0.100803	0.099834	0.054832
commons-net	2270	155	21	63	176	0	76	176	0	59	176	0.880682	0.119318	0.357955	1.000000	0.000000	0.431818	1.0	0.0	0.335227	0.096035	0.111013	0.103524	0.077533
commons-scxml	1216	34	33	75	66	1	57	67	0	56	67	0.507463	0.492537	1.119403	0.985075	0.014925	0.850746	1.0	0.0	0.835821	0.089638	0.101151	0.101151	0.055099
commons-validator	3416	63	10	90	71	2	33	73	0	28	73	0.863014	0.136986	1.232877	0.972603	0.027397	0.452055	1.0	0.0	0.383562	0.044789	0.030445	0.029567	0.021370
commons-vfs	2212	73	40	59	112	1	99	113	0	82	113	0.646018	0.353982	0.522124	0.991150	0.008850	0.876106	1.0	0.0	0.725664	0.059675	0.095389	0.088156	0.051085
deltaspike	2311	216	1	283	217	0	275	217	0	273	217	0.995392	0.004608	1.304147	1.000000	0.000000	1.267281	1.0	0.0	1.258065	0.215924	0.212895	0.212029	0.093899
eagle	1119	50	80	105	129	1	117	130	0	118	130	0.384615	0.615385	0.807692	0.992308	0.007692	0.900000	1.0	0.0	0.907692	0.138517	0.219839	0.221626	0.116175
giraph	1121	140	1	244	141	0	220	141	0	219	141	0.992908	0.007092	1.730496	1.000000	0.000000	1.560284	1.0	0.0	1.553191	0.342551	0.322034	0.321142	0.125781
gora	1329	77	22	93	99	0	113	99	0	109	99	0.777778	0.222222	0.939394	1.000000	0.000000	1.141414	1.0	0.0	1.101010	0.127916	0.159518	0.156509	0.074492
jspwiki	8809	82	151	217	232	1	139	233	0	137	233	0.351931	0.648069	0.931330	0.995708	0.004292	0.596567	1.0	0.0	0.587983	0.033943	0.042116	0.042002	0.026450
knox	2069	346	2	562	348	0	524	348	0	512	348	0.994253	0.005747	1.614943	1.000000	0.000000	1.505747	1.0	0.0	1.471264	0.438859	0.421460	0.415660	0.168197
kylin	12975	1205	59	763	1256	8	703	1264	0	707	1264	0.953323	0.046677	0.603639	0.993671	0.006329	0.556171	1.0	0.0	0.559335	0.151676	0.150983	0.151908	0.097418
lens	2418	276	0	300	276	0	227	276	0	221	276	1.000000	0.000000	1.086957	1.000000	0.000000	0.822464	1.0	0.0	0.800725	0.238213	0.208023	0.205542	0.114144
mahout	4167	325	3	508	327	1	388	328	0	382	328	0.990854	0.009146	1.548780	0.996951	0.003049	1.182927	1.0	0.0	1.164634	0.199904	0.171586	0.170386	0.078714
manifoldcf	5936	592	79	579	668	3	670	671	0	669	671	0.882265	0.117735	0.862891	0.995529	0.004471	0.998510	1.0	0.0	0.997019	0.197271	0.225404	0.225741	0.113039
nutch	3532	480	69	533	548	1	435	549	0	427	549	0.874317	0.125683	0.970856	0.998179	0.001821	0.792350	1.0	0.0	0.777778	0.286806	0.278313	0.276331	0.155436
opennlp	2685	144	0	247	144	0	209	144	0	209	144	1.000000	0.000000	1.715278	1.000000	0.000000	1.451389	1.0	0.0	1.451389	0.145624	0.131471	0.131471	0.053631
parquet-mr	2249	120	0	299	120	0	171	120	0	121	120	1.000000	0.000000	2.491667	1.000000	0.000000	1.425000	1.0	0.0	1.008333	0.186305	0.129391	0.107159	0.053357
santuario-java	3376	92	3	60	95	0	57	95	0	49	95	0.968421	0.031579	0.631579	1.000000	0.000000	0.600000	1.0	0.0	0.515789	0.045024	0.045024	0.042654	0.028140
systemml	6196	252	52	197	304	0	283	304	0	279	304	0.828947	0.171053	0.648026	1.000000	0.000000	0.930921	1.0	0.0	0.917763	0.072466	0.094739	0.094093	0.049064
tika	4933	597	73	461	665	5	448	670	0	448	670	0.891045	0.108955	0.688060	0.992537	0.007463	0.668657	1.0	0.0	0.668657	0.214474	0.225623	0.226637	0.135820
wss4j	3734	227	17	205	241	3	148	244	0	148	244	0.930328	0.069672	0.840164	0.987705	0.012295	0.606557	1.0	0.0	0.606557	0.115694	0.104178	0.104981	0.065345

percentage of total commits that is detected as bugfixing
mean values
SZZ      0.134705
JL       0.142224
JLM      0.138048
JLMIV    0.074780

std values
SZZ      0.104807
JL       0.089809
JLM      0.090465
JLMIV    0.043924

median values
SZZ      0.092837
JL       0.106245
JLM      0.102338
JLMIV    0.056064

MAD values
SZZ      0.072275
JL       0.062642
JLM      0.068154
JLMIV    0.032932

percentage of correctely labelled bug fixing commits
mean values
percent_szz_tp    0.782010
percent_jl_tp     0.996558
percent_jlm_tp    1.000000
percent_szz_fn    0.217990
percent_jl_fn     0.003442
percent_jlm_fn    0.000000
percent_szz_fp    0.920622
percent_jl_fp     0.898884
percent_jlm_fp    0.816249

std values
percent_szz_tp    0.219498
percent_jl_tp     0.005809
percent_jlm_tp    0.000000
percent_szz_fn    0.219498
percent_jl_fn     0.005809
percent_jlm_fn    0.000000
percent_szz_fp    0.494618
percent_jl_fp     0.355508
percent_jlm_fp    0.341354

median values
percent_szz_tp    0.868665
percent_jl_tp     1.000000
percent_jlm_tp    1.000000
percent_szz_fn    0.131335
percent_jl_fn     0.000000
percent_jlm_fn    0.000000
percent_szz_fp    0.810626
percent_jl_fp     0.863426
percent_jlm_fp    0.789251

MAD values
percent_szz_tp    0.181686
percent_jl_tp     0.000000
percent_jlm_tp    0.000000
percent_szz_fn    0.181686
percent_jl_fn     0.000000
percent_jlm_fn    0.000000
percent_szz_fp    0.400578
percent_jl_fp     0.403896
percent_jlm_fp    0.392641

Comparison with data from Mills et al.

project_id = Project.objects(name="mahout").get().id
vcs_id = VCSSystem.objects(project_id=project_id).get().id
its_id = IssueSystem.objects(project_id=project_id).get().id

# This data is taken from the replication kit of Mills et al. work on file actions actually involved in a fix. 
# These are only for thirteen issue that we have identified as actual bugs (incl. manual validation) in our data.
# The other issues are discarded.
# We only consider the data on file level, because that is the level of abstraction we are working on. 
# Changes to the original data:
# MAHOUT-1319
#   integration.src.test.java.org.apache.mahout.text.TestSequenceFilesFromDirectory
#   True -> False
#   Reason: This is a test file, i.e. we believe this is mislabeled in the original data.
# TIKA-1110
#   tika-parsers.src.main.java.org.apache.tika.parser.chm.CHM2XHTML
#   True -> False
#   Deletion of unused source code, i.e., does not affect the logic of the application
mills_data = [{
    'issue': 'MAHOUT-1410',
    'expected_true':  ['core/src/main/java/org/apache/mahout/clustering/classify/ClusterClassificationDriver',
                       'core/src/main/java/org/apache/mahout/clustering/classify/ClusterClassificationMapper'],
    'expected_false': ['core/src/main/java/org/apache/mahout/clustering/classify/ClusterClassificationDriverTest']
},{
    'issue': 'MAHOUT-1409',
    'expected_true':  ['math/src/main/java/org/apache/mahout/math/MatrixVectorView'],
    'expected_false': ['math/src/main/java/org/apache/mahout/math/MatrixVectorViewTest']
},{
    'issue': 'MAHOUT-1389',
    'expected_true':  ['core/src/main/java/org/apache/mahout/classifier/naivebayes/training/TrainNaiveBayesJob'],
    'expected_false': ['core/src/main/java/org/apache/mahout/classifier/naivebayes/test/TestNaiveBayesDriver']
},{
    'issue': 'MAHOUT-1378',
    'expected_true':  ['src/main/java/org/apache/mahout/classifier/df/data/Dataset'],
    'expected_false': ['core/src/test/java/org/apache/mahout/classifier/df/data/DatasetTest']
},{
    'issue': 'MAHOUT-1371',
    'expected_true':  ['integration/src/main/java/org/apache/mahout/utils/vectors/arff/ARFFVectorIterable',
                       'integration/src/main/java/org/apache/mahout/utils/vectors/arff/ARFFIterator'],
    'expected_false': ['integration/src/test/java/org/apache/mahout/utils/vectors/arff/ARFFVectorIterableTest']
},{
    'issue': 'MAHOUT-1367',
    'expected_true':  ['integration/src/main/java/org/apache/mahout/text/wikipedia/WikipediaXmlSplitter'],
    'expected_false': []
},{
    'issue': 'MAHOUT-1358',
    'expected_true':  ['core/src/main/java/org/apache/mahout/clustering/streaming/mapreduce/StreamingKMeansThread'],
    'expected_false': []
},{
    'issue': 'MAHOUT-1349',
    'expected_true':  ['integration/src/main/java/org/apache/mahout/utils/vectors/VectorHelper'],
    'expected_false': ['integration/src/test/java/org/apache/mahout/utils/vectors/VectorHelperTest']
},{
    'issue': 'MAHOUT-1319',
    'expected_true':  ['integration/src/main/java/org/apache/mahout/text/SequenceFilesFromDirectory',
                       'integration/src/main/java/org/apache/mahout/text/WholeFileRecordReader',
                       'core/src/main/java/org/apache/mahout/common/HadoopUtil'],
    'expected_false': ['integration/src/test/java/org/apache/mahout/text/TestSequenceFilesFromDirectory',
                       'integration/src/test/java/org/apache/mahout/text/TestPathFilter']
},{
    'issue': 'MAHOUT-1314',
    'expected_true':  ['core/src/main/java/org/apache/mahout/clustering/streaming/mapreduce/StreamingKMeansReducer'],
    'expected_false': []
},{
    'issue': 'MAHOUT-1285',
    'expected_true':  ['integration/src/main/java/org/apache/mahout/utils/vectors/arff/ARFFVectorIterable',
                       'integration/src/main/java/org/apache/mahout/utils/vectors/arff/MapBackedARFFModel'],
    'expected_false': ['org/apache/mahout/utils/vectors/arff/MapBackedARFFModelTest/MapBackedARFFModelTest']
},{
    'issue': 'MAHOUT-1284',
    'expected_true':  [],
    'expected_false': ['core/src/test/java/org/apache/mahout/common/DummyRecordWriter',
                       'core/src/test/java/org/apache/mahout/common/DummyRecordWriterTest']
},{
    'issue': 'MAHOUT-1261',
    'expected_true':  ['core/src/main/java/org/apache/mahout/cf/taste/hadoop/TasteHadoopUtils'],
    'expected_false': ['core/src/test/java/org/apache/mahout/cf/taste/hadoop/TasteHadoopUtilsTest']
},{
    'issue': 'TIKA-1211',
    'expected_true':  ['tika-core/src/main/java/org/apache/tika/sax/XHTMLContentHandler'],
    'expected_false': []
},{
    'issue': 'TIKA-1192',
    'expected_true':  ['tika-parsers/src/main/java/org/apache/tika/parser/rtf/TextExtractor'],
    'expected_false': ['tika-parsers/src/test/java/org/apache/tika/parser/rtf/RTFParserTest']
},{
    'issue': 'TIKA-1152',
    'expected_true':  ['tika-parsers/src/main/java/org/apache/tika/parser/chm/lzx/ChmLzxBlock'],
    'expected_false': []
},{
    'issue': 'TIKA-1124',
    'expected_true':  ['tika-parsers/src/main/java/org/apache/tika/parser/pdf/PDF2XHTML',
                       'tika-parsers/src/main/java/org/apache/tika/parser/pdf/PDFParser'],
    'expected_false': ['tika-parsers/src/test/java/org/apache/tika/parser/pdf/PDFParserTest']
},{
    'issue': 'TIKA-1110',
    'expected_true':  ['tika-parsers/src/main/java/org/apache/tika/parser/chm/ChmParser'],
    'expected_false': ['tika-parsers/src/test/java/org/apache/tika/parser/AutoDetectParserTest',
                       'tika-parsers/src/main/java/org/apache/tika/parser/chm/CHM2XHTML']
},{
    'issue': 'TIKA-1109',
    'expected_true':  ['tika-parsers/src/main/java/org/apache/tika/parser/microsoft/ooxml/XSSFExcelExtractorDecorator',
                       'tika-parsers/src/main/java/org/apache/tika/parser/microsoft/ooxml/OOXMLExtractorFactory'],
    'expected_false': ['tika-parsers/src/test/java/org/apache/tika/parser/microsoft/ooxml/OOXMLParserTest']
},{
    'issue': 'TIKA-1078',
    'expected_true':  ['tika-app/src/main/java/org/apache/tika/cli/TikaCLI'],
    'expected_false': ['tika-core/src/test/java/org/apache/tika/io/FilenameUtilsTest',
                       'tika-core/src/main/java/org/apache/tika/io/FilenameUtils']
},{
    'issue': 'TIKA-1070',
    'expected_true':  ['tika-parsers/src/test/java/org/apache/tika/parser/html/HtmlParserTest'],
    'expected_false': []
},{
    'issue': 'TIKA-961',
    'expected_true':  ['tika-parsers/src/main/java/org/apache/tika/parser/html/BoilerpipeContentHandler'],
    'expected_false': ['tika-parsers/src/test/java/org/apache/tika/parser/html/HtmlParserTest']
},{
    'issue': 'TIKA-820',
    'expected_true':  [],
    'expected_false': ['tika-parsers/src/test/java/org/apache/tika/parser/html/HtmlParserTest',
                       'tika-core/src/main/java/org/apache/tika/sax/TextContentHandler']
}]

verbose = False

for data in mills_data:
    external_id = data['issue']
    print(external_id)
    
    if external_id.startswith("MAHOUT"):
        project_id = Project.objects(name="mahout").get().id
    elif external_id.startswith("TIKA"):
        project_id = Project.objects(name="tika").get().id
    else:
        print("Unknown project, skipping")

    vcs_id = VCSSystem.objects(project_id=project_id).get().id
    its_id = IssueSystem.objects(project_id=project_id).get().id
    issue = Issue.objects(issue_system_id=its_id,external_id=external_id).get()
    
    print('EXPECTED (true):')
    for expected in data['expected_true']:
        print(expected)
    print()
    print('EXPECTED (false):')
    for expected in data['expected_false']:
        print(expected)
    print()
    
    changed_files = set()
    inducing_files = set()
    for commit in Commit.objects(vcs_system_id=vcs_id,fixed_issue_ids=issue.id):
        if verbose:
            if external_id.startswith("MAHOUT"):
                print("https://github.com/apache/mahout/commit/%s" % commit.revision_hash)
            elif external_id.startswith("TIKA"):
                print("https://github.com/apache/tika/commit/%s" % commit.revision_hash)
            else:
                "Unknown project, cannot generate GitHub link."
            print("CHANGED FILES:")
        for file_action in FileAction.objects(commit_id=commit.id):
            file = File.objects(id=file_action.file_id).get()
            num_inducing = FileAction.objects(induces__match={'change_file_action_id': file_action.id, 'label': 'JLMIV+'}).count()
            changed_files.add(file.path)
            if num_inducing>0:
                inducing_files.add(file.path)
            if verbose:
                print(file.path)
                print("Inducing Changes:", num_inducing)
    if verbose:
        print()
    print('FOUND (True):')
    for found in inducing_files:
        print(found)
    print()
    print('FOUND (False):')
    for found in changed_files-inducing_files:
        print(found)
    print()
    print("------------------------")

MAHOUT-1410
EXPECTED (true):
core/src/main/java/org/apache/mahout/clustering/classify/ClusterClassificationDriver
core/src/main/java/org/apache/mahout/clustering/classify/ClusterClassificationMapper

EXPECTED (false):
core/src/main/java/org/apache/mahout/clustering/classify/ClusterClassificationDriverTest

FOUND (True):
core/src/main/java/org/apache/mahout/clustering/classify/ClusterClassificationDriver.java
core/src/main/java/org/apache/mahout/clustering/classify/ClusterClassificationMapper.java

FOUND (False):
core/src/test/java/org/apache/mahout/clustering/classify/ClusterClassificationDriverTest.java
CHANGELOG

------------------------
MAHOUT-1409
EXPECTED (true):
math/src/main/java/org/apache/mahout/math/MatrixVectorView

EXPECTED (false):
math/src/main/java/org/apache/mahout/math/MatrixVectorViewTest

FOUND (True):
math/src/main/java/org/apache/mahout/math/MatrixVectorView.java

FOUND (False):
CHANGELOG
math/src/test/java/org/apache/mahout/math/MatrixVectorViewTest.java

------------------------
MAHOUT-1389
EXPECTED (true):
core/src/main/java/org/apache/mahout/classifier/naivebayes/training/TrainNaiveBayesJob

EXPECTED (false):
core/src/main/java/org/apache/mahout/classifier/naivebayes/test/TestNaiveBayesDriver

FOUND (True):
core/src/main/java/org/apache/mahout/classifier/naivebayes/training/TrainNaiveBayesJob.java

FOUND (False):
CHANGELOG
core/src/main/java/org/apache/mahout/classifier/naivebayes/test/TestNaiveBayesDriver.java

------------------------
MAHOUT-1378
EXPECTED (true):
src/main/java/org/apache/mahout/classifier/df/data/Dataset

EXPECTED (false):
core/src/test/java/org/apache/mahout/classifier/df/data/DatasetTest

FOUND (True):
core/src/main/java/org/apache/mahout/classifier/df/data/Dataset.java

FOUND (False):
CHANGELOG
core/src/test/java/org/apache/mahout/classifier/df/data/DatasetTest.java

------------------------
MAHOUT-1371
EXPECTED (true):
integration/src/main/java/org/apache/mahout/utils/vectors/arff/ARFFVectorIterable
integration/src/main/java/org/apache/mahout/utils/vectors/arff/ARFFIterator

EXPECTED (false):
integration/src/test/java/org/apache/mahout/utils/vectors/arff/ARFFVectorIterableTest

FOUND (True):
integration/src/main/java/org/apache/mahout/utils/vectors/arff/ARFFVectorIterable.java
integration/src/main/java/org/apache/mahout/utils/vectors/arff/ARFFIterator.java

FOUND (False):
integration/src/test/resources/date.arff
CHANGELOG
integration/src/test/resources/non-numeric-1.arff
integration/src/test/java/org/apache/mahout/utils/vectors/arff/ARFFVectorIterableTest.java

------------------------
MAHOUT-1367
EXPECTED (true):
integration/src/main/java/org/apache/mahout/text/wikipedia/WikipediaXmlSplitter

EXPECTED (false):

FOUND (True):
integration/src/main/java/org/apache/mahout/text/wikipedia/WikipediaXmlSplitter.java

FOUND (False):
CHANGELOG

------------------------
MAHOUT-1358
EXPECTED (true):
core/src/main/java/org/apache/mahout/clustering/streaming/mapreduce/StreamingKMeansThread

EXPECTED (false):

FOUND (True):
core/src/main/java/org/apache/mahout/clustering/streaming/mapreduce/StreamingKMeansThread.java
core/src/main/java/org/apache/mahout/clustering/streaming/mapreduce/StreamingKMeansDriver.java

FOUND (False):
CHANGELOG
core/src/main/java/org/apache/mahout/clustering/streaming/mapreduce/StreamingKMeansMapper.java

------------------------
MAHOUT-1349
EXPECTED (true):
integration/src/main/java/org/apache/mahout/utils/vectors/VectorHelper

EXPECTED (false):
integration/src/test/java/org/apache/mahout/utils/vectors/VectorHelperTest

FOUND (True):
integration/src/main/java/org/apache/mahout/utils/vectors/VectorHelper.java

FOUND (False):
integration/src/test/java/org/apache/mahout/utils/vectors/VectorHelperTest.java
CHANGELOG

------------------------
MAHOUT-1319
EXPECTED (true):
integration/src/main/java/org/apache/mahout/text/SequenceFilesFromDirectory
integration/src/main/java/org/apache/mahout/text/WholeFileRecordReader
core/src/main/java/org/apache/mahout/common/HadoopUtil

EXPECTED (false):
integration/src/test/java/org/apache/mahout/text/TestSequenceFilesFromDirectory
integration/src/test/java/org/apache/mahout/text/TestPathFilter

FOUND (True):
integration/src/main/java/org/apache/mahout/text/WholeFileRecordReader.java
core/src/main/java/org/apache/mahout/common/HadoopUtil.java
integration/src/main/java/org/apache/mahout/text/SequenceFilesFromDirectory.java

FOUND (False):
integration/src/test/java/org/apache/mahout/text/TestSequenceFilesFromDirectory.java
CHANGELOG
integration/src/test/java/org/apache/mahout/text/TestPathFilter.java

------------------------
MAHOUT-1314
EXPECTED (true):
core/src/main/java/org/apache/mahout/clustering/streaming/mapreduce/StreamingKMeansReducer

EXPECTED (false):

FOUND (True):
core/src/main/java/org/apache/mahout/clustering/streaming/mapreduce/StreamingKMeansReducer.java

FOUND (False):
CHANGELOG

------------------------
MAHOUT-1285
EXPECTED (true):
integration/src/main/java/org/apache/mahout/utils/vectors/arff/ARFFVectorIterable
integration/src/main/java/org/apache/mahout/utils/vectors/arff/MapBackedARFFModel

EXPECTED (false):
org/apache/mahout/utils/vectors/arff/MapBackedARFFModelTest/MapBackedARFFModelTest

FOUND (True):
integration/src/main/java/org/apache/mahout/utils/vectors/arff/ARFFVectorIterable.java
integration/src/main/java/org/apache/mahout/utils/vectors/arff/MapBackedARFFModel.java

FOUND (False):
integration/src/test/java/org/apache/mahout/utils/vectors/arff/MapBackedARFFModelTest.java
CHANGELOG

------------------------
MAHOUT-1284
EXPECTED (true):

EXPECTED (false):
core/src/test/java/org/apache/mahout/common/DummyRecordWriter
core/src/test/java/org/apache/mahout/common/DummyRecordWriterTest

FOUND (True):

FOUND (False):
core/src/test/java/org/apache/mahout/common/DummyRecordWriter.java
core/src/test/java/org/apache/mahout/common/DummyRecordWriterTest.java
CHANGELOG

------------------------
MAHOUT-1261
EXPECTED (true):
core/src/main/java/org/apache/mahout/cf/taste/hadoop/TasteHadoopUtils

EXPECTED (false):
core/src/test/java/org/apache/mahout/cf/taste/hadoop/TasteHadoopUtilsTest

FOUND (True):
core/src/main/java/org/apache/mahout/cf/taste/hadoop/TasteHadoopUtils.java

FOUND (False):
core/src/test/java/org/apache/mahout/cf/taste/hadoop/TasteHadoopUtilsTest.java
CHANGELOG

------------------------
TIKA-1211
EXPECTED (true):
tika-core/src/main/java/org/apache/tika/sax/XHTMLContentHandler

EXPECTED (false):

FOUND (True):
tika-core/src/main/java/org/apache/tika/sax/XHTMLContentHandler.java

FOUND (False):

------------------------
TIKA-1192
EXPECTED (true):
tika-parsers/src/main/java/org/apache/tika/parser/rtf/TextExtractor

EXPECTED (false):
tika-parsers/src/test/java/org/apache/tika/parser/rtf/RTFParserTest

FOUND (True):
tika-parsers/src/main/java/org/apache/tika/parser/rtf/TextExtractor.java

FOUND (False):
tika-parsers/src/test/java/org/apache/tika/parser/rtf/RTFParserTest.java
CHANGES.txt
tika-parsers/src/test/resources/test-documents/testRTFListOverride.rtf
tika-parsers/src/test/resources/test-documents/test_list_override.rtf

------------------------
TIKA-1152
EXPECTED (true):
tika-parsers/src/main/java/org/apache/tika/parser/chm/lzx/ChmLzxBlock

EXPECTED (false):

FOUND (True):
tika-parsers/src/main/java/org/apache/tika/parser/chm/lzx/ChmLzxBlock.java

FOUND (False):

------------------------
TIKA-1124
EXPECTED (true):
tika-parsers/src/main/java/org/apache/tika/parser/pdf/PDF2XHTML
tika-parsers/src/main/java/org/apache/tika/parser/pdf/PDFParser

EXPECTED (false):
tika-parsers/src/test/java/org/apache/tika/parser/pdf/PDFParserTest

FOUND (True):
tika-parsers/src/main/java/org/apache/tika/parser/pdf/PDF2XHTML.java
tika-parsers/src/main/java/org/apache/tika/parser/pdf/PDFParser.java

FOUND (False):
tika-parsers/src/test/resources/test-documents/TIKA-1142.docx
CHANGES.txt
tika-parsers/src/test/resources/test-documents/testPDFEmbeddingAndEmbedded.docx
tika-parsers/src/test/java/org/apache/tika/parser/pdf/PDFParserTest.java

------------------------
TIKA-1110
EXPECTED (true):
tika-parsers/src/main/java/org/apache/tika/parser/chm/ChmParser

EXPECTED (false):
tika-parsers/src/test/java/org/apache/tika/parser/AutoDetectParserTest
tika-parsers/src/main/java/org/apache/tika/parser/chm/CHM2XHTML

FOUND (True):
tika-parsers/src/main/java/org/apache/tika/parser/chm/ChmParser.java

FOUND (False):
tika-parsers/src/main/java/org/apache/tika/parser/chm/CHM2XHTML.java
tika-core/src/main/resources/org/apache/tika/mime/tika-mimetypes.xml
tika-parsers/src/test/java/org/apache/tika/parser/AutoDetectParserTest.java

------------------------
TIKA-1109
EXPECTED (true):
tika-parsers/src/main/java/org/apache/tika/parser/microsoft/ooxml/XSSFExcelExtractorDecorator
tika-parsers/src/main/java/org/apache/tika/parser/microsoft/ooxml/OOXMLExtractorFactory

EXPECTED (false):
tika-parsers/src/test/java/org/apache/tika/parser/microsoft/ooxml/OOXMLParserTest

FOUND (True):
tika-parsers/src/main/java/org/apache/tika/parser/microsoft/ooxml/XSSFExcelExtractorDecorator.java
tika-parsers/src/main/java/org/apache/tika/parser/microsoft/ooxml/OOXMLExtractorFactory.java

FOUND (False):
tika-parsers/src/test/java/org/apache/tika/parser/microsoft/ooxml/OOXMLParserTest.java

------------------------
TIKA-1078
EXPECTED (true):
tika-app/src/main/java/org/apache/tika/cli/TikaCLI

EXPECTED (false):
tika-core/src/test/java/org/apache/tika/io/FilenameUtilsTest
tika-core/src/main/java/org/apache/tika/io/FilenameUtils

FOUND (True):
tika-app/src/main/java/org/apache/tika/cli/TikaCLI.java

FOUND (False):
CHANGES.txt
tika-core/src/test/java/org/apache/tika/io/FilenameUtilsTest.java
tika-core/src/main/java/org/apache/tika/io/FilenameUtils.java

------------------------
TIKA-1070
EXPECTED (true):
tika-parsers/src/test/java/org/apache/tika/parser/html/HtmlParserTest

EXPECTED (false):

FOUND (True):

FOUND (False):
tika-core/src/main/java/org/apache/tika/sax/ToXMLContentHandler.java

------------------------
TIKA-961
EXPECTED (true):
tika-parsers/src/main/java/org/apache/tika/parser/html/BoilerpipeContentHandler

EXPECTED (false):
tika-parsers/src/test/java/org/apache/tika/parser/html/HtmlParserTest

FOUND (True):

FOUND (False):
tika-parsers/src/main/java/org/apache/tika/parser/html/BoilerpipeContentHandler.java
tika-parsers/src/test/java/org/apache/tika/parser/html/HtmlParserTest.java
tika-parsers/src/test/resources/test-documents/boilerplate-whitespace.html

------------------------
TIKA-820
EXPECTED (true):

EXPECTED (false):
tika-parsers/src/test/java/org/apache/tika/parser/html/HtmlParserTest
tika-core/src/main/java/org/apache/tika/sax/TextContentHandler

FOUND (True):

FOUND (False):
tika-parsers/src/test/java/org/apache/tika/parser/html/HtmlParserTest.java
tika-core/src/main/java/org/apache/tika/sax/TextContentHandler.java

------------------------

Ground Truth for Inducing Changes and Release Assignment

# list of commits determined by inspecting bug fixing commits with false positive linked_issue_ids
fixes_with_induces = ['58c898e2c03bb4351fbca616','5b34a9c9e586ab679033d402','5b34a9d3e586ab678933d948',
                      '5b34a9dde586ab678833e12a','5b437aa16d2aba304d4ab17c','5b45f7352123a77a30258a24',
                      '5b4769a00f9db94b6f26f54f','5b4854933463875e6415d34c','5b4854bc3463875e6515e64e',
                      '5b7adf00df629e0ae5f1fa25','5b7adf04df629e0ae5f1fb7e','5b7adf45df629e0aeaf20560',
                      '5b7adf64df629e0ae0f21fcb','5b8910c4a1f4f33b5c67f0fd','5bdb4fb512300b167cc6dff5',
                      '5bf51c56d2f8190d93f3bd20','5bf51c6bd2f8190d9df3bddf','5bf51ca6d2f8190d93f3c47d',
                      '5bf51ca8d2f8190dadf3c508','5bf29f3a29526541678147f8','5c5016b236dcc75e45a32388']

# for TIKA-2483 the inducing commit is not the latest commit on TIKA-2311. 
# This is for the manual correction to the actual inducing commit
candidate_correction = {'6930ff0251e9e93ee969a9f1287c902d31045b59':'3aab15f8f277614e3c5783c4862e25d63b737425'}

fixinducingdata = pd.DataFrame(columns=['fixed_issue','inducing_issue',
                                        'fix_commit','latest_inducing_issue_commit','latest_inducing_jlmivpp_commit',
                                        'fix_date','latest_inducing_issue_date','latest_inducing_jlmivpp_date',
                                        'affected_versions'])

verbose = False

current_index = 0
for commit in Commit.objects(id__in=fixes_with_induces).only('id','linked_issue_ids','fixed_issue_ids','committer_date','message','revision_hash'):
    changed_production_file = False
    for file_action in FileAction.objects(commit_id=commit.id):
        file = File.objects(id=file_action.file_id).get()
        changed_production_file |= java_filename_filter(file.path)
    if not changed_production_file:
        if verbose:
            print("No change to production code for commit", commit.revision_hash)
            print("skipping")
            print("----")
        continue
    
    fixed_issue = Issue.objects(id=commit.fixed_issue_ids[0]).get()
    inducing_issue = Issue.objects(id=list(set(commit.linked_issue_ids)-set(commit.fixed_issue_ids))[0]).get()
    
    inducing_commit_issue = None
    for inducing_candidate in Commit.objects(id__ne=commit.id,linked_issue_ids=inducing_issue.id).only('message','committer_date','revision_hash','fixed_issue_ids'):
        if inducing_candidate.fixed_issue_ids and fixed_issue.id in inducing_candidate.fixed_issue_ids:
            continue
        changed_production_file = False
        for file_action in FileAction.objects(commit_id=inducing_candidate.id):
            file = File.objects(id=file_action.file_id).get()
            changed_production_file |= java_filename_filter(file.path)
        if not changed_production_file:
            # cannot be inducing, no change to production code
            continue
        
        if inducing_commit_issue is None:
            inducing_commit_issue = inducing_candidate
        elif inducing_candidate.committer_date>inducing_commit_issue.committer_date:
            inducing_commit_issue = inducing_candidate
    if inducing_commit_issue.revision_hash in candidate_correction:
        inducing_commit_issue = Commit.objects(id__ne=commit.id,linked_issue_ids=inducing_issue.id,revision_hash=candidate_correction[inducing_candidate.revision_hash]).only('message','committer_date','revision_hash','fixed_issue_ids').get()
    
    inducing_commit_jlmivpp = None
    for file_action in FileAction.objects(commit_id=commit.id):
        for inducing_action in FileAction.objects(induces__change_file_action_id=file_action.id, induces__label='JLMIV+R'):
            inducing_candidate = Commit.objects(id=inducing_action.commit_id).only('message','committer_date','revision_hash').get()
            if inducing_commit_jlmivpp is None:
                inducing_commit_jlmivpp = inducing_candidate
            elif inducing_candidate.committer_date>inducing_commit_jlmivpp.committer_date:
                inducing_commit_jlmivpp = inducing_candidate
    
    affected_versions = str(fixed_issue.affects_versions)
    
    if verbose:
        print("FIXING COMMIT")
        print("Date:", commit.committer_date)
        print("Affected Versions:", affected_versions)
        print("Message:")
        print(commit.message)
        print()
    if verbose:
        print("INDUCING COMMIT ISSUE")
    if inducing_commit_issue:
        if verbose:
            print("Date:", inducing_commit_issue.committer_date)
            print("Message:")
            print(inducing_commit_issue.message)
            print()
        inducing_issue_revision_hash = inducing_commit_issue.revision_hash
        inducing_issue_date = inducing_commit_issue.committer_date
    else:
        if verbose:
            print("No inducing changes found")
        inducing_issue_revision_hash = None
        inducing_issue_date = None
    
    if verbose:
        print("INDUCING COMMIT JLMIV++")
    if inducing_commit_jlmivpp:
        if verbose:
            print("Date:", inducing_commit_jlmivpp.committer_date)
            print("Message:")
            print(inducing_commit_jlmivpp.message)
        jlmivpp_revision_hash = inducing_commit_jlmivpp.revision_hash
        jlmivpp_date = inducing_commit_jlmivpp.committer_date
    else:
        if verbose:
            print("No inducing changes found")
        jlmivpp_revision_hash = None
        jlmivpp_date = None
    
    if verbose:
        print("----")
    
    fixinducingdata.loc[current_index] = [fixed_issue.external_id, inducing_issue.external_id,
                              commit.revision_hash, inducing_issue_revision_hash, jlmivpp_revision_hash,
                              commit.committer_date, inducing_issue_date, jlmivpp_date,
                              str(fixed_issue.affects_versions)]
    current_index += 1

fixinducingdata

	fixed_issue	inducing_issue	fix_commit	latest_inducing_issue_commit	latest_inducing_jlmivpp_commit	fix_date	latest_inducing_issue_date	latest_inducing_jlmivpp_date	affected_versions
0	IVY-882	IVY-857	6d6d34af69292a41f77162ab43984daeee54cac0	c4dee75b1b75bd45235ac3ca4a86fc1d194f8062	c4dee75b1b75bd45235ac3ca4a86fc1d194f8062	2008-08-22 15:32:18	2008-07-08 13:51:59	2008-07-08 13:51:59	['2.0-RC1']
1	CALCITE-2253	CALCITE-2206	d83cb9029e37c52fba7d07a5a33978a9f01b8c5f	502c108473e535a76cebc5e09bd6be5efcc24531	502c108473e535a76cebc5e09bd6be5efcc24531	2018-04-16 06:51:12	2018-04-11 06:26:19	2018-04-11 06:26:19	[]
2	CALCITE-1215	CALCITE-1212	4255767d9ccc3221963a98d7e2e8ce3b68e156a3	6baa9c4bed1100012ebf4ef9547cbb077f2edaaf	6baa9c4bed1100012ebf4ef9547cbb077f2edaaf	2016-04-26 02:42:13	2016-04-26 00:04:16	2016-04-26 00:04:16	[]
3	CALCITE-822	CALCITE-783	ac934f695db39d88beb5a12c792ba7da68cc386e	9177063b5217ba01a7a34f85a0bf8e8753adeb20	c711fed6e2392f296554719a98b2d737da53c9b5	2015-08-08 01:43:05	2015-07-21 04:20:05	2015-07-21 03:58:24	[]
4	KYLIN-3223	KYLIN-3239	3fa11b2cffa4e9159b4a5f911dfb1eb1b7c08005	2a6d69a20b9f82412bffe60e004a9c079ce91471	0655dbc383790f61ef9b04cd23b6b6ebf5f2ff6e	2018-02-10 14:24:00	2018-02-09 13:39:40	2018-02-09 02:51:17	['v2.2.0']
5	NUTCH-683	NUTCH-676	a2ba8e229c11792afaabaf7c5dfd511142562ebc	de1ea50abc0e0e011f48bfa49b3cb5e3b5ab8be0	de1ea50abc0e0e011f48bfa49b3cb5e3b5ab8be0	2009-02-11 09:12:15	2009-01-21 19:26:27	2009-01-21 19:26:27	['1.0.0']
6	PARQUET-214	PARQUET-139	4ed0bdf1c73fd82d3080d15085675de96d5be0aa	ce65dfb394623c34dd7919aba5c0687f1bcf39f2	3fc28541f001ce6e4a7afa91fec8d21bfeaa17db	2015-03-31 23:49:30	2015-02-05 23:06:12	2015-03-07 01:06:34	['1.6.0']
7	TIKA-599	TIKA-528	d7601cc4516c3e2e9d00e882ba3bbf9c8f16ef8d	d4e7bfdf502e80a704ad4842e74596675f40bab3	d4e7bfdf502e80a704ad4842e74596675f40bab3	2011-03-09 18:18:42	2010-10-12 20:24:51	2010-10-12 20:24:51	['0.9']
8	TIKA-2483	TIKA-2311	06486c8548daf5f9ba588da4e88e3da82a429e70	3aab15f8f277614e3c5783c4862e25d63b737425	3aab15f8f277614e3c5783c4862e25d63b737425	2017-11-14 15:48:09	2017-04-13 18:00:48	2017-04-13 18:00:48	['1.16']
9	SYSTEMML-1126	SYSTEMML-584	e82de90b21df44a3194a86ce0e087dc710cd34af	8e8283b959dcf1c41f7f999e5ef6596ec9c19e57	1cee0d1db11fba2f5688531f1f09ad5552715186	2017-02-25 19:51:03	2016-04-07 20:30:24	2016-04-15 17:21:11	[]
10	SYSTEMML-2162	SYSTEMML-1919	afbe7bf2fbb7d6ca863e2482aaaf981864a80b46	f86879bd0af5eb046c6fe00a444ff04c603a2e91	f86879bd0af5eb046c6fe00a444ff04c603a2e91	2018-02-28 03:48:38	2017-09-17 23:48:54	2017-09-17 23:48:54	[]
11	SYSTEMML-2275	SYSTEMML-2217	e8774a7a272d1ba4060020b143de6b969e3daabf	094f5551dd08f1d1747d70bbf23cc246d503d95e	094f5551dd08f1d1747d70bbf23cc246d503d95e	2018-04-22 22:45:29	2018-03-30 04:34:46	2018-03-30 04:34:46	[]
12	LENS-538	LENS-486	9132b967e2bd32d1478ca6425bd086bbb6d46d12	a8baa452a28f56ef3541092778e4acfbae5f0546	a8baa452a28f56ef3541092778e4acfbae5f0546	2015-05-06 08:08:29	2015-05-05 12:02:37	2015-05-05 12:02:37	['2.2']
13	KNOX-1134	KNOX-1119	27217ead9a67bcded8978133a9d7abe40ebc1d1a	9a276787c9783a06215867e40c464a1a78da9c3b	6474b61be2a106f0debd4bd274782d10bbb298e2	2017-12-01 12:40:07	2017-11-29 04:18:58	2017-11-29 04:16:26	[]
14	VALIDATOR-376	VALIDATOR-273	043a39baa8befc916faa0d4cca714b9fef2c19b3	7ff07e7799577aecaae22e8a5a9346707e676e3e	47ecf763b4ebef66127f551e7e31c000e75059ac	2015-10-25 06:03:42	2014-07-07 19:54:07	2014-12-29 15:28:01	['1.4.1 Release']
15	GIRAPH-918	GIRAPH-908	ffdddff32e7ac6e2cbb5ae8471c1192c69ac6a94	4a133f5766c09362917e0416af503c0a00b24e87	4a133f5766c09362917e0416af503c0a00b24e87	2014-06-10 23:12:23	2014-06-08 17:36:03	2014-06-08 17:36:03	[]
16	GIRAPH-832	GIRAPH-792	13d84663bc4eedb2e9e4035af75622b0a3cb480e	7cc54575d867e37a43020df309a78cd65c3fbdc0	7cc54575d867e37a43020df309a78cd65c3fbdc0	2014-01-30 16:48:15	2014-01-28 19:56:10	2014-01-28 19:56:10	[]
17	GIRAPH-88	GIRAPH-11	3e65823def5d984b4199d6e16eb1c556d1f180b7	db028f58017bdeb1f2e9f7c2a441195db0056767	db028f58017bdeb1f2e9f7c2a441195db0056767	2011-11-15 22:52:54	2011-11-15 00:54:20	2011-11-15 00:54:20	['0.1.0']
18	GIRAPH-34	GIRAPH-27	9dd9af5294cca4ef0e4a19ec4829294b2a5d850f	d0bb2a9a377bd78990736f858d57504befc95421	183a812d509b0133f32941337e09ae3fa3d68cec	2011-09-16 20:56:44	2011-09-12 18:42:17	2011-09-14 04:13:54	['0.1.0', '1.0.0']
19	EAGLE-573	EAGLE-569	1fa490e0c5b0c1b827791909968fcee7adc8ea7c	0fac33799e21ae92d9b4a7e98e5090b032145b06	None	2016-09-28 00:25:02	2016-09-27 17:23:53	None	['v0.5.0']

Comparison of the bug inducing fileactions and commits

inducingdata = pd.DataFrame(index=projects, columns=['fileactions_total',
                                                     'fileactions_productive',
                                                     'fileactions_szz',
                                                     'fileactions_szz_tp',
                                                     'fileactions_szz_fp',
                                                     'fileactions_jl+r',
                                                     'fileactions_jl+r_tp',
                                                     'fileactions_jl+r_fp',
                                                     'fileactions_jlmiv',
                                                     'fileactions_jlmiv_total',
                                                     'fileactions_jlmiv+',
                                                     'fileactions_jlmiv+r',
                                                     'fileactions_jlmiv+av',
                                                     'commits_total',
                                                     'commits_java',
                                                     'commits_productive',
                                                     'commits_szz',
                                                     'commits_szz_tp',
                                                     'commits_szz_fp',
                                                     'commits_jl+r',
                                                     'commits_jl+r_tp',
                                                     'commits_jl+r_fp',
                                                     'commits_jlmiv',
                                                     'commits_jlmiv_total',
                                                     'commits_jlmiv+',
                                                     'commits_jlmiv+r',
                                                     'commits_jlmiv+av',])
inducingdata.fillna(0, inplace=True)
for project in projects:
    project_id = Project.objects(name=project).get().id
    vcs_system_id = VCSSystem.objects(project_id=project_id).get().id
    for commit in Commit.objects(vcs_system_id=vcs_system_id).only('id','linked_issue_ids', 'message', 'labels','szz_issue_ids','fixed_issue_ids'):
        fileactions_java_total = 0
        fileactions_java_productive = 0
        fileactions_szz_tp = 0
        fileactions_szz_fp = 0
        fileactions_jlr_tp = 0
        fileactions_jlr_fp = 0
        fileactions_jlmiv = 0
        fileactions_jlmiv_total = 0
        fileactions_jlmivp = 0
        fileactions_jlmivpr = 0
        fileactions_jlmivpav = 0
        for fileaction in FileAction.objects(commit_id=commit.id):
            file = File.objects(id=fileaction.file_id).get()
            is_szz = False
            is_jlr = False
            is_jlmiv = False
            is_jlmivp = False
            is_jlmivpr = False
            is_jlmivpav = False
            if java_filename_filter(file.path):
                if fileaction.induces:
                    for induces in fileaction.induces:
                        if induces['szz_type']!='hard_suspect':
                            is_szz |= induces['label']=='SZZ'
                            is_jlr |= induces['label']=='JL+R'
                            is_jlmiv |= induces['label']=='JLMIV'
                            is_jlmivp |= induces['label']=='JLMIV+'
                            is_jlmivpr |= induces['label']=='JLMIV+R'
                            is_jlmivpav |= induces['label']=='JLMIV+RAV'
                                
                fileactions_java_productive += 1
                if is_szz:
                    if is_jlmivpr:
                        fileactions_szz_tp += 1
                    else:
                        fileactions_szz_fp += 1
                if is_jlr:
                    if is_jlmivpr:
                        fileactions_jlr_tp += 1
                    else:
                        fileactions_jlr_fp += 1
                if is_jlmiv:
                    fileactions_jlmiv += 1
                if is_jlmivp:
                    fileactions_jlmivp += 1
                if is_jlmivpr:
                    fileactions_jlmivpr += 1
                if is_jlmivpav:
                    fileactions_jlmivpav += 1
            if java_filename_filter(file.path, production_only=False):
                fileactions_java_total += 1
                if is_jlmiv:
                    fileactions_jlmiv_total += 1
                    
        inducingdata.loc[project,'fileactions_total'] += fileactions_java_total
        inducingdata.loc[project,'fileactions_productive'] += fileactions_java_productive
        inducingdata.loc[project,'fileactions_szz'] += fileactions_szz_tp+fileactions_szz_fp
        inducingdata.loc[project,'fileactions_szz_tp'] += fileactions_szz_tp
        inducingdata.loc[project,'fileactions_szz_fp'] += fileactions_szz_fp
        inducingdata.loc[project,'fileactions_jl+r'] += fileactions_jlr_tp+fileactions_jlr_fp
        inducingdata.loc[project,'fileactions_jl+r_tp'] += fileactions_jlr_tp
        inducingdata.loc[project,'fileactions_jl+r_fp'] += fileactions_jlr_fp
        inducingdata.loc[project,'fileactions_jlmiv'] += fileactions_jlmiv
        inducingdata.loc[project,'fileactions_jlmiv_total'] += fileactions_jlmiv_total
        inducingdata.loc[project,'fileactions_jlmiv+'] += fileactions_jlmivp
        inducingdata.loc[project,'fileactions_jlmiv+r'] += fileactions_jlmivpr
        inducingdata.loc[project,'fileactions_jlmiv+av'] += fileactions_jlmivpav
        inducingdata.loc[project,'commits_total'] += 1
        inducingdata.loc[project,'commits_java'] += bool(fileactions_java_total)
        inducingdata.loc[project,'commits_productive'] += bool(fileactions_java_productive)
        inducingdata.loc[project,'commits_szz'] += bool(fileactions_szz_tp+fileactions_szz_fp)
        inducingdata.loc[project,'commits_szz_tp'] += bool(fileactions_szz_tp)
        inducingdata.loc[project,'commits_szz_fp'] += bool(fileactions_szz_fp)
        inducingdata.loc[project,'commits_jl+r'] += bool(fileactions_jlr_tp+fileactions_jlr_fp)
        inducingdata.loc[project,'commits_jl+r_tp'] += bool(fileactions_jlr_tp)
        inducingdata.loc[project,'commits_jl+r_fp'] += bool(fileactions_jlr_fp)
        inducingdata.loc[project,'commits_jlmiv'] += bool(fileactions_jlmiv)
        inducingdata.loc[project,'commits_jlmiv_total'] += bool(fileactions_jlmiv_total)
        inducingdata.loc[project,'commits_jlmiv+'] += bool(fileactions_jlmivp)
        inducingdata.loc[project,'commits_jlmiv+r'] += bool(fileactions_jlmivpr)
        inducingdata.loc[project,'commits_jlmiv+av'] += bool(fileactions_jlmivpav)
    
inducingdata['percent_fileactions_productive_jlmiv+'] = inducingdata['fileactions_jlmiv+']/inducingdata['fileactions_productive']
inducingdata['percent_fileactions_productive_jlmiv+r'] = inducingdata['fileactions_jlmiv+r']/inducingdata['fileactions_productive']
inducingdata['percent_fileactions_total_jlmiv+'] = inducingdata['fileactions_jlmiv+']/inducingdata['fileactions_total']
inducingdata['percent_fileactions_total_jlmiv+r'] = inducingdata['fileactions_jlmiv+r']/inducingdata['fileactions_total']
inducingdata['percent_commits_jlmiv+_total'] = inducingdata['commits_jlmiv+']/inducingdata['commits_total']
inducingdata['percent_commits_jlmiv+_java'] = inducingdata['commits_jlmiv+']/inducingdata['commits_java']
inducingdata['percent_commits_jlmiv+r_total'] = inducingdata['commits_jlmiv+r']/inducingdata['commits_total']
inducingdata['percent_commits_jlmiv+r_java'] = inducingdata['commits_jlmiv+r']/inducingdata['commits_java']

inducingdata['percent_fileactions_szz_tp'] = inducingdata['fileactions_szz_tp']/inducingdata['fileactions_jlmiv+r']
inducingdata['percent_fileactions_szz_fp'] = inducingdata['fileactions_szz_fp']/inducingdata['fileactions_jlmiv+r']
inducingdata['percent_fileactions_jl+r_tp'] = inducingdata['fileactions_jl+r_tp']/inducingdata['fileactions_jlmiv+r']
inducingdata['percent_fileactions_jl+r_fp'] = inducingdata['fileactions_jl+r_fp']/inducingdata['fileactions_jlmiv+r']
inducingdata['percent_fileactions_jlmiv_fp'] = (inducingdata['fileactions_jlmiv']-inducingdata['fileactions_jlmiv+r'])/inducingdata['fileactions_jlmiv+r']
inducingdata['percent_fileactions_jlmiv_fp_total'] = (inducingdata['fileactions_jlmiv_total']-inducingdata['fileactions_jlmiv+r'])/inducingdata['fileactions_jlmiv+r']
inducingdata['percent_fileactions_jlmiv+_fp'] = (inducingdata['fileactions_jlmiv+']-inducingdata['fileactions_jlmiv+r'])/inducingdata['fileactions_jlmiv+r']
inducingdata['percent_fileactions_jlmiv+_fp_total'] = (inducingdata['fileactions_jlmiv_total']-inducingdata['fileactions_jlmiv+'])/inducingdata['fileactions_jlmiv+']
inducingdata['percent_fileactions_jlmivp_av_dev'] = (inducingdata['fileactions_jlmiv+r']-inducingdata['fileactions_jlmiv+av'])/inducingdata['fileactions_jlmiv+r']
inducingdata['percent_commits_szz_tp'] = inducingdata['commits_szz_tp']/inducingdata['commits_jlmiv+r']
inducingdata['percent_commits_szz_fp'] = inducingdata['commits_szz_fp']/inducingdata['commits_jlmiv+r']
inducingdata['percent_commits_jl+r_tp'] = inducingdata['commits_jl+r_tp']/inducingdata['commits_jlmiv+r']
inducingdata['percent_commits_jl+r_fp'] = inducingdata['commits_jl+r_fp']/inducingdata['commits_jlmiv+r']
inducingdata['percent_commits_jlmiv_fp'] = (inducingdata['commits_jlmiv']-inducingdata['commits_jlmiv+r'])/inducingdata['commits_jlmiv+r']
inducingdata['percent_commits_jlmiv+_fp'] = (inducingdata['commits_jlmiv+']-inducingdata['commits_jlmiv+r'])/inducingdata['commits_jlmiv+r']
inducingdata['percent_commits_jlmivp_av_dev'] = (inducingdata['commits_jlmiv+r']-inducingdata['commits_jlmiv+av'])/inducingdata['commits_jlmiv+r']

display(inducingdata)

cols = ['percent_fileactions_total_jlmiv+r','percent_fileactions_productive_jlmiv+r',
        'percent_commits_jlmiv+r_total','percent_commits_jlmiv+r_java',
        'percent_fileactions_total_jlmiv+','percent_fileactions_productive_jlmiv+',
        'percent_commits_jlmiv+_total','percent_commits_jlmiv+_java',
        'percent_fileactions_szz_tp','percent_fileactions_szz_fp',
        'percent_fileactions_jl+r_tp','percent_fileactions_jl+r_fp',
        'percent_fileactions_jlmiv_fp',
        'percent_fileactions_jlmiv+_fp',
        'percent_fileactions_jlmivp_av_dev',
        'percent_commits_szz_tp','percent_commits_szz_fp',
        'percent_commits_jl+r_tp','percent_commits_jl+r_fp',
        'percent_commits_jlmiv_fp',
        'percent_commits_jlmiv+_fp',
        'percent_commits_jlmivp_av_dev',
        'percent_fileactions_jlmiv_fp_total']

print('mean values:')
print(inducingdata[cols].mean().to_string())
print()
print('std values:')
print(inducingdata[cols].std().to_string())
print()
print('median values: ')
print(inducingdata[cols].median().to_string())
print()
print('MAD values:')
print(mad(inducingdata[cols]).to_string())


f, axes = plt.subplots(3, 2, gridspec_kw={'height_ratios': [2, 2, 4]}, figsize=(6,3))

df = inducingdata[['percent_commits_jlmiv+r_total',
                   'percent_commits_jlmiv+r_java',
                   'percent_fileactions_total_jlmiv+r',
                   'percent_fileactions_productive_jlmiv+r']]
df.columns = ['Total', 'Production', 'Total', 'Production']

sns.boxplot(data=df.iloc[:,0:2], orient='h', color='lightgray', ax=axes[0,0])
sns.swarmplot(data=df.iloc[:,0:2], orient='h', color='gray',ax=axes[0,0])
axes[0,0].set_xlabel('Percentage of commits that\nare bug inducing found by JLMIV+R')

sns.boxplot(data=df.iloc[:,2:4], orient='h', color='lightgray', ax=axes[0,1])
sns.swarmplot(data=df.iloc[:,2:4], orient='h', color='gray',ax=axes[0,1])
axes[0,1].set_xlabel('Percentage of file actions that\nare bug inducing found by JLMIV+R')

df = inducingdata[['percent_commits_szz_tp',
                   'percent_commits_jl+r_tp',
                   'percent_fileactions_szz_tp',
                   'percent_fileactions_jl+r_tp',]]
df.columns = ['SZZ','JL+R','SZZ','JL+R']

sns.boxplot(data=df.iloc[:,0:2], orient='h', color='lightgray', ax=axes[1,0])
sns.swarmplot(data=df.iloc[:,0:2], orient='h', color='gray',ax=axes[1,0])
axes[1,0].set_xlabel('Percentage of actual bug inducing\ncommits identified (true positives)')

sns.boxplot(data=df.iloc[:,2:4], orient='h', color='lightgray', ax=axes[1,1])
sns.swarmplot(data=df.iloc[:,2:4], orient='h', color='gray',ax=axes[1,1])
axes[1,1].set_xlabel('Percentage of actual bug inducing\nfile actions identified (true positives)')

df = inducingdata[['percent_commits_szz_fp',
                   'percent_commits_jl+r_fp',
                   'percent_commits_jlmiv_fp',
                   'percent_commits_jlmiv+_fp',
                   'percent_fileactions_szz_fp',
                   'percent_fileactions_jl+r_fp',
                   'percent_fileactions_jlmiv_fp',
                   'percent_fileactions_jlmiv+_fp']]
df.columns = ['SZZ', 'JL+R', 'JLMIV', 'JLMIV+', 'SZZ', 'JL+R', 'JLMIV', 'JLMIV+']

sns.boxplot(data=df.iloc[:,0:4], orient='h', color='lightgray', ax=axes[2,0])
sns.swarmplot(data=df.iloc[:,0:4], orient='h', color='gray',ax=axes[2,0])
axes[2,0].set_xlabel('Percentage of additional bug inducing\ncommits (false positives)')

sns.boxplot(data=df.iloc[:,4:8], orient='h', color='lightgray', ax=axes[2,1])
sns.swarmplot(data=df.iloc[:,4:8], orient='h', color='gray',ax=axes[2,1])
axes[2,1].set_xlabel('Percentage of additional bug inducing\nfile actions (false positives)')

plt.subplots_adjust(left=None, bottom=-0.6, right=None, top=None, wspace=0.8, hspace=0.6)
plt.show()
#plt.savefig("inducingdata.pdf", bbox_inches='tight')

	fileactions_total	fileactions_productive	fileactions_szz	fileactions_szz_tp	fileactions_szz_fp	fileactions_jl+r	fileactions_jl+r_tp	fileactions_jl+r_fp	fileactions_jlmiv	fileactions_jlmiv_total	fileactions_jlmiv+	fileactions_jlmiv+r	fileactions_jlmiv+av	commits_total	commits_java	commits_productive	commits_szz	commits_szz_tp	commits_szz_fp	commits_jl+r	commits_jl+r_tp	commits_jl+r_fp	commits_jlmiv	commits_jlmiv_total	commits_jlmiv+	commits_jlmiv+r	commits_jlmiv+av	percent_fileactions_productive_jlmiv+	percent_fileactions_productive_jlmiv+r	percent_fileactions_total_jlmiv+	percent_fileactions_total_jlmiv+r	percent_commits_jlmiv+_total	percent_commits_jlmiv+_java	percent_commits_jlmiv+r_total	percent_commits_jlmiv+r_java	percent_fileactions_szz_tp	percent_fileactions_szz_fp	percent_fileactions_jl+r_tp	percent_fileactions_jl+r_fp	percent_fileactions_jlmiv_fp	percent_fileactions_jlmiv_fp_total	percent_fileactions_jlmiv+_fp	percent_fileactions_jlmiv+_fp_total	percent_fileactions_jlmivp_av_dev	percent_commits_szz_tp	percent_commits_szz_fp	percent_commits_jl+r_tp	percent_commits_jl+r_fp	percent_commits_jlmiv_fp	percent_commits_jlmiv+_fp	percent_commits_jlmivp_av_dev
ant-ivy	12958	9357	757	555	202	680	560	120	627	627	607	560	536	3189	2031	1769	429	335	142	397	338	89	371	371	362	338	329	0.064871	0.059848	0.046844	0.043217	0.113515	0.178237	0.105989	0.166420	0.991071	0.360714	1.000000	0.214286	0.119643	0.119643	0.083929	0.032949	0.042857	0.991124	0.420118	1.000000	0.263314	0.097633	0.071006	0.026627
archiva	59797	39648	923	577	346	1198	895	303	1002	1002	926	895	865	10261	4900	3961	437	298	203	636	491	203	532	532	504	491	470	0.023356	0.022574	0.015486	0.014967	0.049118	0.102857	0.047851	0.100204	0.644693	0.386592	1.000000	0.338547	0.119553	0.119553	0.034637	0.082073	0.033520	0.606925	0.413442	1.000000	0.413442	0.083503	0.026477	0.042770
calcite	30017	24860	6342	1104	5238	3642	1131	2511	1244	1244	1203	1131	1115	2926	2225	2012	1006	413	827	829	423	617	459	459	448	423	418	0.048391	0.045495	0.040077	0.037679	0.153110	0.201348	0.144566	0.190112	0.976127	4.631300	1.000000	2.220159	0.099912	0.099912	0.063660	0.034081	0.014147	0.976359	1.955083	1.000000	1.458629	0.085106	0.059102	0.011820
cayenne	83730	49185	2539	1227	1312	1901	1344	557	1510	1510	1419	1369	1351	6619	4876	3953	1006	546	601	805	576	305	642	642	600	583	580	0.028850	0.027834	0.016947	0.016350	0.090648	0.123052	0.088080	0.119565	0.896275	0.958364	0.981738	0.406866	0.102995	0.102995	0.036523	0.064130	0.013148	0.936535	1.030875	0.987993	0.523156	0.101201	0.029160	0.005146
commons-bcel	12128	10601	504	93	411	192	112	80	121	121	116	112	108	1429	1004	886	90	51	52	79	56	32	60	60	58	56	54	0.010942	0.010565	0.009565	0.009235	0.040588	0.057769	0.039188	0.055777	0.830357	3.669643	1.000000	0.714286	0.080357	0.080357	0.035714	0.043103	0.035714	0.910714	0.928571	1.000000	0.571429	0.071429	0.035714	0.035714
commons-beanutils	5239	2884	83	33	50	98	39	59	43	43	42	39	38	1341	763	580	50	32	20	59	38	26	41	41	41	38	37	0.014563	0.013523	0.008017	0.007444	0.030574	0.053735	0.028337	0.049803	0.846154	1.282051	1.000000	1.512821	0.102564	0.102564	0.076923	0.023810	0.025641	0.842105	0.526316	1.000000	0.684211	0.078947	0.078947	0.026316
commons-codec	3438	2144	69	19	50	92	58	34	75	75	59	58	55	1838	1132	832	63	19	44	77	49	33	61	61	50	49	46	0.027519	0.027052	0.017161	0.016870	0.027203	0.044170	0.026659	0.043286	0.327586	0.862069	1.000000	0.586207	0.293103	0.293103	0.017241	0.271186	0.051724	0.387755	0.897959	1.000000	0.673469	0.244898	0.020408	0.061224
commons-collections	22122	13366	51	13	38	280	83	197	105	105	86	83	46	3380	2415	1830	49	13	37	146	58	98	73	73	60	58	41	0.006434	0.006210	0.003888	0.003752	0.017751	0.024845	0.017160	0.024017	0.156627	0.457831	1.000000	2.373494	0.265060	0.265060	0.036145	0.220930	0.445783	0.224138	0.637931	1.000000	1.689655	0.258621	0.034483	0.293103
commons-compress	6418	4543	318	195	123	258	210	48	252	252	228	210	201	2755	2030	1619	236	155	95	201	167	41	198	198	180	167	163	0.050187	0.046225	0.035525	0.032720	0.065336	0.088670	0.060617	0.082266	0.928571	0.585714	1.000000	0.228571	0.200000	0.200000	0.085714	0.105263	0.042857	0.928144	0.568862	1.000000	0.245509	0.185629	0.077844	0.023952
commons-configuration	10095	5967	239	145	94	228	181	47	240	240	206	182	168	3717	2529	2083	193	123	78	191	152	42	197	197	173	153	139	0.034523	0.030501	0.020406	0.018029	0.046543	0.068406	0.041162	0.060498	0.796703	0.516484	0.994505	0.258242	0.318681	0.318681	0.131868	0.165049	0.076923	0.803922	0.509804	0.993464	0.274510	0.287582	0.130719	0.091503
commons-dbcp	4817	3139	205	69	136	239	93	146	117	117	100	93	91	2205	1350	1043	134	52	95	154	68	106	84	84	72	68	67	0.031857	0.029627	0.020760	0.019307	0.032653	0.053333	0.030839	0.050370	0.741935	1.462366	1.000000	1.569892	0.258065	0.258065	0.075269	0.170000	0.021505	0.764706	1.397059	1.000000	1.558824	0.235294	0.058824	0.014706
commons-digester	7011	4493	35	24	11	44	27	17	30	30	30	27	25	2535	1549	1259	26	18	10	33	21	15	23	23	23	21	19	0.006677	0.006009	0.004279	0.003851	0.009073	0.014848	0.008284	0.013557	0.888889	0.407407	1.000000	0.629630	0.111111	0.111111	0.111111	0.000000	0.074074	0.857143	0.476190	1.000000	0.714286	0.095238	0.095238	0.095238
commons-io	6235	3451	144	73	71	132	84	48	103	103	88	84	79	2262	1480	1049	125	65	64	108	76	35	92	92	79	76	72	0.025500	0.024341	0.014114	0.013472	0.034925	0.053378	0.033599	0.051351	0.869048	0.845238	1.000000	0.571429	0.226190	0.226190	0.047619	0.170455	0.059524	0.855263	0.842105	1.000000	0.460526	0.210526	0.039474	0.052632
commons-jcs	13090	9207	1112	265	847	450	274	176	472	472	325	283	283	1622	968	799	282	126	232	175	131	84	181	181	146	138	138	0.035299	0.030737	0.024828	0.021620	0.090012	0.150826	0.085080	0.142562	0.936396	2.992933	0.968198	0.621908	0.667845	0.667845	0.148410	0.452308	0.000000	0.913043	1.681159	0.949275	0.608696	0.311594	0.057971	0.000000
commons-jexl	9095	6570	708	456	252	583	482	101	621	621	521	498	399	3276	1904	1488	303	217	138	252	219	67	261	261	235	221	177	0.079300	0.075799	0.057284	0.054755	0.071734	0.123424	0.067460	0.116071	0.915663	0.506024	0.967871	0.202811	0.246988	0.246988	0.046185	0.191939	0.198795	0.981900	0.624434	0.990950	0.303167	0.180995	0.063348	0.199095
commons-lang	16088	8339	340	152	188	360	212	148	270	270	220	212	202	5792	4299	3334	297	134	171	310	181	136	227	227	182	181	171	0.026382	0.025423	0.013675	0.013178	0.031423	0.042335	0.031250	0.042103	0.716981	0.886792	1.000000	0.698113	0.273585	0.273585	0.037736	0.227273	0.047170	0.740331	0.944751	1.000000	0.751381	0.254144	0.005525	0.055249
commons-math	41922	27151	1086	422	664	1227	515	712	659	659	534	515	447	7222	5839	4758	483	264	290	523	313	281	377	377	322	313	280	0.019668	0.018968	0.012738	0.012285	0.044586	0.055146	0.043340	0.053605	0.819417	1.289320	1.000000	1.382524	0.279612	0.279612	0.036893	0.234082	0.132039	0.843450	0.926518	1.000000	0.897764	0.204473	0.028754	0.105431
commons-net	9599	6742	240	158	82	204	166	38	194	194	179	166	153	2270	1471	1108	172	117	71	147	122	33	139	139	128	122	111	0.026550	0.024622	0.018648	0.017293	0.056388	0.087016	0.053744	0.082937	0.951807	0.493976	1.000000	0.228916	0.168675	0.168675	0.078313	0.083799	0.078313	0.959016	0.581967	1.000000	0.270492	0.139344	0.049180	0.090164
commons-scxml	3350	1947	206	65	141	139	90	49	102	102	97	90	83	1216	648	482	104	41	77	85	56	40	62	62	59	56	51	0.049820	0.046225	0.028955	0.026866	0.048520	0.091049	0.046053	0.086420	0.722222	1.566667	1.000000	0.544444	0.133333	0.133333	0.077778	0.051546	0.077778	0.732143	1.375000	1.000000	0.714286	0.107143	0.053571	0.089286
commons-validator	8697	5941	178	46	132	56	47	9	58	58	51	50	41	3416	2067	1579	124	41	85	49	42	8	52	52	46	45	36	0.008584	0.008416	0.005864	0.005749	0.013466	0.022254	0.013173	0.021771	0.920000	2.640000	0.940000	0.180000	0.160000	0.160000	0.020000	0.137255	0.180000	0.911111	1.888889	0.933333	0.177778	0.155556	0.022222	0.200000
commons-vfs	11256	8471	395	131	264	290	153	137	178	178	159	153	142	2212	1428	1142	213	86	154	158	96	85	109	109	98	96	91	0.018770	0.018062	0.014126	0.013593	0.044304	0.068627	0.043400	0.067227	0.856209	1.725490	1.000000	0.895425	0.163399	0.163399	0.039216	0.119497	0.071895	0.895833	1.604167	1.000000	0.885417	0.135417	0.020833	0.052083
deltaspike	7900	4561	491	266	225	429	266	163	301	301	280	266	255	2311	1615	1293	307	176	154	277	176	117	196	196	184	176	168	0.061390	0.058321	0.035443	0.033671	0.079619	0.113932	0.076158	0.108978	1.000000	0.845865	1.000000	0.612782	0.131579	0.131579	0.052632	0.075000	0.041353	1.000000	0.875000	1.000000	0.664773	0.113636	0.045455	0.045455
eagle	14645	11596	656	136	520	901	305	596	329	329	323	305	301	1119	692	634	128	53	100	190	110	134	119	119	116	110	108	0.027854	0.026302	0.022055	0.020826	0.103664	0.167630	0.098302	0.158960	0.445902	1.704918	1.000000	1.954098	0.078689	0.078689	0.059016	0.018576	0.013115	0.481818	0.909091	1.000000	1.218182	0.081818	0.054545	0.018182
giraph	10485	8225	1269	348	921	974	348	626	377	377	364	348	348	1121	909	861	300	145	233	253	145	174	152	152	149	145	145	0.044255	0.042310	0.034716	0.033190	0.132917	0.163916	0.129349	0.159516	1.000000	2.646552	1.000000	1.798851	0.083333	0.083333	0.045977	0.035714	0.000000	1.000000	1.606897	1.000000	1.200000	0.048276	0.027586	0.000000
gora	7038	4831	190	81	109	142	85	57	95	95	89	85	73	1329	784	662	83	39	53	68	41	33	46	46	44	41	39	0.018423	0.017595	0.012646	0.012077	0.033108	0.056122	0.030850	0.052296	0.952941	1.282353	1.000000	0.670588	0.117647	0.117647	0.047059	0.067416	0.141176	0.951220	1.292683	1.000000	0.804878	0.121951	0.073171	0.048780
jspwiki	29129	21760	457	113	344	479	341	138	391	391	354	346	305	8809	5718	4993	387	97	302	391	277	129	315	315	288	281	246	0.016268	0.015901	0.012153	0.011878	0.032694	0.050367	0.031899	0.049143	0.326590	0.994220	0.985549	0.398844	0.130058	0.130058	0.023121	0.104520	0.118497	0.345196	1.074733	0.985765	0.459075	0.120996	0.024911	0.124555
knox	34665	25271	992	436	556	911	436	475	469	469	455	436	425	2069	1363	1178	383	200	261	358	200	229	218	218	210	200	190	0.018005	0.017253	0.013126	0.012578	0.101498	0.154072	0.096665	0.146735	1.000000	1.275229	1.000000	1.089450	0.075688	0.075688	0.043578	0.030769	0.025229	1.000000	1.305000	1.000000	1.145000	0.090000	0.050000	0.050000
kylin	123656	96280	2254	1210	1044	2069	1222	847	1328	1328	1276	1227	1162	12975	8873	8099	1075	709	534	1003	716	436	771	771	746	718	684	0.013253	0.012744	0.010319	0.009923	0.057495	0.084075	0.055337	0.080920	0.986145	0.850856	0.995925	0.690302	0.082315	0.082315	0.039935	0.040752	0.052975	0.987465	0.743733	0.997214	0.607242	0.073816	0.038997	0.047354
lens	22849	16982	1016	659	357	860	659	201	728	728	683	659	657	2418	1851	1678	442	318	202	389	318	124	343	343	329	318	318	0.040219	0.038806	0.029892	0.028842	0.136063	0.177742	0.131514	0.171799	1.000000	0.541730	1.000000	0.305008	0.104704	0.104704	0.036419	0.065886	0.003035	1.000000	0.635220	1.000000	0.389937	0.078616	0.034591	0.000000
mahout	36478	25616	1930	722	1208	1574	722	852	847	847	781	722	600	4167	2452	2117	691	350	504	602	350	375	387	387	367	350	309	0.030489	0.028186	0.021410	0.019793	0.088073	0.149674	0.083993	0.142741	1.000000	1.673130	1.000000	1.180055	0.173130	0.173130	0.081717	0.084507	0.168975	1.000000	1.440000	1.000000	1.071429	0.105714	0.048571	0.117143
manifoldcf	34528	27695	1050	676	374	1148	750	398	792	792	774	756	729	5936	3396	3107	630	466	221	685	511	234	536	536	530	517	495	0.027947	0.027297	0.022417	0.021895	0.089286	0.156066	0.087096	0.152238	0.894180	0.494709	0.992063	0.526455	0.047619	0.047619	0.023810	0.023256	0.035714	0.901354	0.427466	0.988395	0.452611	0.036750	0.025145	0.042553
nutch	22458	19181	1163	552	611	849	558	291	612	612	584	569	539	3532	2270	2151	490	285	270	405	290	159	319	319	304	294	283	0.030447	0.029665	0.026004	0.025336	0.086070	0.133921	0.083239	0.129515	0.970123	1.073814	0.980668	0.511424	0.075571	0.075571	0.026362	0.047945	0.052724	0.969388	0.918367	0.986395	0.540816	0.085034	0.034014	0.037415
opennlp	16500	14000	615	234	381	473	234	239	266	266	241	234	234	2685	1960	1773	259	111	176	211	111	121	134	134	117	111	111	0.017214	0.016714	0.014606	0.014182	0.043575	0.059694	0.041341	0.056633	1.000000	1.628205	1.000000	1.021368	0.136752	0.136752	0.029915	0.103734	0.000000	1.000000	1.585586	1.000000	1.090090	0.207207	0.054054	0.000000
parquet-mr	18183	13421	1660	235	1425	369	235	134	257	257	244	235	218	2249	1730	1577	552	125	503	196	125	89	140	140	132	125	116	0.018180	0.017510	0.013419	0.012924	0.058693	0.076301	0.055580	0.072254	1.000000	6.063830	1.000000	0.570213	0.093617	0.093617	0.038298	0.053279	0.072340	1.000000	4.024000	1.000000	0.712000	0.120000	0.056000	0.072000
santuario-java	17604	11337	316	140	176	312	140	172	177	177	165	140	140	3376	1792	1487	184	99	101	179	99	93	117	117	109	99	99	0.014554	0.012349	0.009373	0.007953	0.032287	0.060826	0.029325	0.055246	1.000000	1.257143	1.000000	1.228571	0.264286	0.264286	0.178571	0.072727	0.000000	1.000000	1.020202	1.000000	0.939394	0.181818	0.101010	0.000000
systemml	44718	35098	1870	810	1060	1702	889	813	1043	1043	949	889	885	6196	4191	3882	826	458	513	761	498	390	581	581	530	498	497	0.027039	0.025329	0.021222	0.019880	0.085539	0.126461	0.080374	0.118826	0.911136	1.192351	1.000000	0.914511	0.173228	0.173228	0.067492	0.099052	0.004499	0.919679	1.030120	1.000000	0.783133	0.166667	0.064257	0.002008
tika	19560	12929	1218	602	616	1092	616	476	700	700	658	618	571	4933	3398	2769	736	424	396	684	433	316	480	480	456	434	405	0.050893	0.047800	0.033640	0.031595	0.092439	0.134197	0.087979	0.127722	0.974110	0.996764	0.996764	0.770227	0.132686	0.132686	0.064725	0.063830	0.076052	0.976959	0.912442	0.997696	0.728111	0.105991	0.050691	0.066820
wss4j	29678	21410	1041	320	721	908	359	549	438	438	406	360	324	3734	2746	2469	453	184	342	384	192	254	231	231	215	192	174	0.018963	0.016815	0.013680	0.012130	0.057579	0.078296	0.051419	0.069920	0.888889	2.002778	0.997222	1.525000	0.216667	0.216667	0.127778	0.078818	0.100000	0.958333	1.781250	1.000000	1.322917	0.203125	0.119792	0.093750

mean values:
percent_fileactions_total_jlmiv+r         0.019234
percent_fileactions_productive_jlmiv+r    0.027604
percent_commits_jlmiv+r_total             0.060691
percent_commits_jlmiv+r_java              0.091404
percent_fileactions_total_jlmiv+          0.020403
percent_fileactions_productive_jlmiv+     0.029301
percent_commits_jlmiv+_total              0.063739
percent_commits_jlmiv+_java               0.096016
percent_fileactions_szz_tp                0.844283
percent_fileactions_szz_fp                1.448722
percent_fileactions_jl+r_tp               0.994750
percent_fileactions_jl+r_fp               0.845956
percent_fileactions_jlmiv_fp              0.171269
percent_fileactions_jlmiv+_fp             0.060718
percent_fileactions_jlmivp_av_dev         0.069187
percent_commits_szz_tp                    0.856660
percent_commits_szz_fp                    1.100342
percent_commits_jl+r_tp                   0.995013
percent_commits_jl+r_fp                   0.743935
percent_commits_jlmiv_fp                  0.143834
percent_commits_jlmiv+_fp                 0.051621
percent_commits_jlmivp_av_dev             0.061528
percent_fileactions_jlmiv_fp_total        0.171269

std values:
percent_fileactions_total_jlmiv+r         0.011250
percent_fileactions_productive_jlmiv+r    0.015798
percent_commits_jlmiv+r_total             0.033980
percent_commits_jlmiv+r_java              0.047864
percent_fileactions_total_jlmiv+          0.011950
percent_fileactions_productive_jlmiv+     0.016818
percent_commits_jlmiv+_total              0.035511
percent_commits_jlmiv+_java               0.050075
percent_fileactions_szz_tp                0.208013
percent_fileactions_szz_fp                1.212771
percent_fileactions_jl+r_tp               0.012347
percent_fileactions_jl+r_fp               0.581520
percent_fileactions_jlmiv_fp              0.110578
percent_fileactions_jlmiv+_fp             0.037347
percent_fileactions_jlmivp_av_dev         0.081189
percent_commits_szz_tp                    0.197255
percent_commits_szz_fp                    0.656754
percent_commits_jl+r_tp                   0.013633
percent_commits_jl+r_fp                   0.382264
percent_commits_jlmiv_fp                  0.070371
percent_commits_jlmiv+_fp                 0.027547
percent_commits_jlmivp_av_dev             0.062494
percent_fileactions_jlmiv_fp_total        0.110578

median values: 
percent_fileactions_total_jlmiv+r         0.016610
percent_fileactions_productive_jlmiv+r    0.025376
percent_commits_jlmiv+r_total             0.052582
percent_commits_jlmiv+r_java              0.081593
percent_fileactions_total_jlmiv+          0.017054
percent_fileactions_productive_jlmiv+     0.026794
percent_commits_jlmiv+_total              0.056941
percent_commits_jlmiv+_java               0.085545
percent_fileactions_szz_tp                0.913399
percent_fileactions_szz_fp                1.133082
percent_fileactions_jl+r_tp               1.000000
percent_fileactions_jl+r_fp               0.650109
percent_fileactions_jlmiv_fp              0.133010
percent_fileactions_jlmiv+_fp             0.046622
percent_fileactions_jlmivp_av_dev         0.049447
percent_commits_szz_tp                    0.923911
percent_commits_szz_fp                    0.927545
percent_commits_jl+r_tp                   1.000000
percent_commits_jl+r_fp                   0.698105
percent_commits_jlmiv_fp                  0.120498
percent_commits_jlmiv+_fp                 0.049590
percent_commits_jlmivp_av_dev             0.048067
percent_fileactions_jlmiv_fp_total        0.133010

MAD values:
percent_fileactions_total_jlmiv+r         0.007221
percent_fileactions_productive_jlmiv+r    0.012368
percent_commits_jlmiv+r_total             0.034717
percent_commits_jlmiv+r_java              0.049614
percent_fileactions_total_jlmiv+          0.007682
percent_fileactions_productive_jlmiv+     0.012901
percent_commits_jlmiv+_total              0.037194
percent_commits_jlmiv+_java               0.053881
percent_fileactions_szz_tp                0.119137
percent_fileactions_szz_fp                0.806101
percent_fileactions_jl+r_tp               0.000000
percent_fileactions_jl+r_fp               0.486784
percent_fileactions_jlmiv_fp              0.066639
percent_fileactions_jlmiv+_fp             0.026051
percent_fileactions_jlmivp_av_dev         0.041715
percent_commits_szz_tp                    0.107293
percent_commits_szz_fp                    0.536568
percent_commits_jl+r_tp                   0.000000
percent_commits_jl+r_fp                   0.356061
percent_commits_jlmiv_fp                  0.061849
percent_commits_jlmiv+_fp                 0.022746
percent_commits_jlmivp_av_dev             0.051600
percent_fileactions_jlmiv_fp_total        0.066639

Comparison of approaches that label releases

def calc_current_files(commit, release_commit, commit_graph, undirected_graph, rename_cache, jlr_mode = False):
    """determines the java files changed by a commit and returns them as a set"""
    current_files = set()
    path_valid = False
    current_files_start = set()
    if not jlr_mode:
        for fileaction in FileAction.objects(commit_id=commit.id):
            file = File.objects(id=fileaction.file_id).get()
            if file.path not in current_files and java_filename_filter(file.path):
                current_files_start.add(file.path)
    else:
        for fileaction in FileAction.objects(commit_id=commit.id, mode='M'):
            if FileAction.objects(induces__match={'change_file_action_id': fileaction.id, 'label': 'JL+R'}).count() > 0:
                file = File.objects(id=fileaction.file_id).get()
                if file.path not in current_files and java_filename_filter(file.path):
                    current_files_start.add(file.path)
    try:
        shortest_paths = list(nx.all_shortest_paths(undirected_graph, release_commit.revision_hash, commit.revision_hash))
    except nx.NetworkXNoPath:
        shortest_paths = []
    for path in shortest_paths:
        # path = nx.shortest_path(undirected_graph, release_commit.revision_hash, commit.revision_hash)
        current_files = current_files_start.copy()
        path_valid = True
        had_backward_edge = False
        for i in range(len(path)-1, 0, -1): # going backwards
            if path[i-1] in commit_graph.pred[path[i]]:
                if had_backward_edge:
                    # invalid change of direction
                    path_valid = False
                    break
                if path[i] in rename_cache:
                    renames = rename_cache[path[i]]
                else:
                    renames = heuristic_renames(vcs_system_id, path[i])
                    rename_cache[path[i]] = renames
                if renames is not None:
                    for rename in renames[0]:
                        if rename[1] in current_files:
                            current_files.remove(rename[1])
                            current_files.add(rename[0])
                    for deletion in renames[1]:
                        current_files.discard(deletion)
            elif path[i-1] in commit_graph.succ[path[i]]:
                had_backward_edge = True
                if path[i-1] in rename_cache:
                    renames = rename_cache[path[i-1]]
                else:
                    renames = heuristic_renames(vcs_system_id, path[i-1])
                    rename_cache[path[i-1]] = renames
                if renames is not None:
                    for rename in renames[0]:
                        if rename[0] in current_files:
                            current_files.remove(rename[0])
                            current_files.add(rename[1])
        if path_valid:
            break
    return current_files, path_valid


def get_commit_graph(vcs_system_id, silent=True):
    """Load NetworkX digraph structure from commits of this VCS.
    :param vcs_system_id id of the vcs system for which the graph is created
    :param silent determines whether there is an output to stdout in case of a missing parent commit
    """
    g = nx.DiGraph()
    # first we add all nodes to the graph
    for c in Commit.objects(vcs_system_id=vcs_system_id).only('id', 'revision_hash').timeout(False):
        g.add_node(c.revision_hash)
        
    # after that we draw all edges
    for c in Commit.objects(vcs_system_id=vcs_system_id).only('id', 'parents', 'revision_hash').timeout(False):
        for p in c.parents:
            try:
                p1 = Commit.objects(vcs_system_id=vcs_system_id,revision_hash=p).only('id', 'revision_hash').get()
                g.add_edge(p1.revision_hash, c.revision_hash)
            except Commit.DoesNotExist:
                if not silent:
                    print("parent of a commit is missing (commit id: {} - revision_hash: {})".format(c.id, p))
    return g


index = []
for project in projects:
    for release in releases[project]:
        index.append(project+'-'+release['version'])

releasedata = pd.DataFrame(index=index, columns=['issues_szz_six_months_tp',
                                                 'issues_szz_six_months_fp',
                                                 'issues_jlr_six_months_tp',
                                                 'issues_jlr_six_months_fp',
                                                 'issues_jlmiv_six_months_tp',
                                                 'issues_jlmiv_six_months_fp',
                                                 'issues_jlmiv_affected_version_tp',
                                                 'issues_jlmiv_affected_version_fp',
                                                 'issues_jlmiv_inducing',
                                                 'files_szz_six_months_tp',
                                                 'files_szz_six_months_fp',
                                                 'files_jlr_six_months_tp',
                                                 'files_jlr_six_months_fp',
                                                 'files_jlmiv_six_months_tp',
                                                 'files_jlmiv_six_months_fp',
                                                 'files_jlmiv_affected_version_tp',
                                                 'files_jlmiv_affected_version_fp',
                                                 'files_jlmiv_inducing',
                                                 'num_java_files',
                                                 'overlap_6mav',
                                                 'overlap_6min',
                                                 'overlap_avin'
                                                ])
releasedata.fillna(0, inplace=True)

for project in projects:
    project_id = Project.objects(name=project).get().id
    vcs_system_id = VCSSystem.objects(project_id=project_id).get().id
    
    commit_graph = get_commit_graph(vcs_system_id)
    undirected_graph = commit_graph.to_undirected(as_view=True)
    rename_cache = {}
    delete_cache = {}
    
    its = IssueSystem.objects(project_id=project_id).get()
    if 'jira' in its.url:
        jira_key = its.url.split('=')[-1]
    else:
        jira_key = ''
    
    for release in releases[project]:
        current_index = project+'-'+release['version']
        
        assigned_issues_szz_six_months = set()
        assigned_issues_jlr_six_months = set()
        assigned_issues_jlmiv_six_months = set()
        assigned_issues_jlmiv_affected_version = set()
        assigned_issues_jlmivp_inducing = set()

        affected_files_szz_six_months = set()
        affected_files_jlr_six_months = set()
        affected_files_jlmiv_six_months = set()
        affected_files_jlmiv_affected_version = set()
        affected_files_jlmivp_inducing = set()
        files_release = set()
        
        if 'corrected_revision' in release:
            release_commit = Commit.objects(revision_hash=release['corrected_revision']).get()
        else:
            tag = Tag.objects(vcs_system_id=vcs_system_id,name=release['tag']).get()
            release_commit = Commit.objects(id=tag.commit_id).get()
        six_month_timeframe = release_commit.committer_date+relativedelta(months=6)
        semver_version = tuple(release['version'].split('.'))
        for file in CodeEntityState.objects(id__in=release_commit.code_entity_states, ce_type='file'):
            if java_filename_filter(file.long_name):
                files_release.add(file.long_name)
        
        all_fixed_issues = set()
        for commit in Commit.objects(vcs_system_id=vcs_system_id, committer_date__gt=release_commit.committer_date).only('committer_date','labels','fixed_issue_ids','revision_hash','szz_issue_ids', 'linked_issue_ids').timeout(False):
            current_files = None
            
            # data for 6 month labeling
            #if commit.committer_date<=six_month_timeframe:
            if commit.committer_date<six_month_timeframe:
                if commit.labels and "adjustedszz_bugfix" in commit.labels and commit.labels["adjustedszz_bugfix"]:
                    if current_files is None:
                        current_files, path_valid = calc_current_files(commit, release_commit, commit_graph, undirected_graph, rename_cache)
                    if path_valid and len(current_files.intersection(files_release))>0:
                        has_issue = False
                        for issue in Issue.objects(id__in=commit.szz_issue_ids):
                            if issue.issue_type.lower() == "bug" and jira_is_resolved_and_fixed(issue):
                                assigned_issues_szz_six_months.add(issue.id)
                                has_issue = True
                        if has_issue:
                            affected_files_szz_six_months.update(current_files)
                if commit.labels and "issueonly_bugfix" in commit.labels and commit.labels["issueonly_bugfix"]:
                    current_files_jlr, path_valid_jlr = calc_current_files(commit, release_commit, commit_graph, undirected_graph, rename_cache, jlr_mode=True)
                    if path_valid_jlr and len(current_files_jlr.intersection(files_release))>0:
                        for issue in Issue.objects(id__in=commit.linked_issue_ids):
                            if issue.issue_type and issue.issue_type.lower() == "bug" and jira_is_resolved_and_fixed(issue):
                                assigned_issues_jlr_six_months.add(issue.id)
                        affected_files_jlr_six_months.update(current_files_jlr)
                if commit.labels and "validated_bugfix" in commit.labels and commit.labels["validated_bugfix"]:
                    if current_files is None:
                        current_files, path_valid = calc_current_files(commit, release_commit, commit_graph, undirected_graph, rename_cache)
                    if path_valid and len(current_files.intersection(files_release))>0:
                        for issue_id in commit.fixed_issue_ids:
                            assigned_issues_jlmiv_six_months.add(issue_id)
                        affected_files_jlmiv_six_months.update(current_files)
                        
            if commit.labels and "validated_bugfix" in commit.labels and commit.labels["validated_bugfix"]:
                # data for affected version labeling
                for issue in Issue.objects(id__in=commit.fixed_issue_ids):
                    if issue.issue_type_verified and issue.issue_type_verified.lower()=="bug" and jira_is_resolved_and_fixed(issue):
                        if issue.affects_versions and semver_version:
                            affected_versions = get_affected_versions(issue, project, jira_key)
                            is_version_match = False
                            is_tag_match = False
                            for affected_version in affected_versions:
                                is_version_match |= all(tv == av for tv, av in zip(semver_version, affected_version))

                            for av in issue.affects_versions:
                                is_tag_match |= release['tag'] is not None and av.lower()==release['tag'].lower()

                            if is_version_match or is_tag_match:
                                if current_files is None:
                                    current_files, path_valid = calc_current_files(commit, release_commit, commit_graph, undirected_graph, rename_cache)
                                if path_valid and len(current_files.intersection(files_release))>0:
                                    for issue_id in commit.fixed_issue_ids:
                                        assigned_issues_jlmiv_affected_version.add(issue_id)
                                    affected_files_jlmiv_affected_version.update(current_files)
                                break
                                
                for issue in Issue.objects(id__in=commit.fixed_issue_ids):
                    if issue.issue_type_verified and issue.issue_type_verified.lower()=="bug" and jira_is_resolved_and_fixed(issue):
                        all_fixed_issues.add(issue)
        
        for issue in all_fixed_issues:
            all_inducing_have_path = True
            blame_file_cache = set()
            for commit in Commit.objects(fixed_issue_ids=issue.id, vcs_system_id=vcs_system_id, committer_date__gt=release_commit.committer_date).only('committer_date','labels','fixed_issue_ids','revision_hash','szz_issue_ids'):
                #if nx.has_path(commit_graph, commit, release_commit.revision_hash):
                #    continue # commit before release
                # data for inducing-based labeling
                for fa in FileAction.objects(commit_id=commit.id, mode='M'):
                    for ifa in FileAction.objects(induces__match={'change_file_action_id': fa.id, 'label': 'JLMIV+R'}):
                        hard_suspect = True
                        for ind in ifa.induces:
                            if ind['change_file_action_id'] == fa.id and ind['label'] == 'JLMIV+' and ind['szz_type'] != 'hard_suspect':
                                hard_suspect = False
                                break
                        if hard_suspect:
                            # skip, only hard suspects
                            continue

                        bc = Commit.objects(id=ifa.commit_id).only('revision_hash','fixed_issue_ids').get()
                        blame_commit = bc.revision_hash
                        
                        if not nx.has_path(commit_graph, blame_commit, release_commit.revision_hash):
                            if bc.fixed_issue_ids is None or issue.id not in bc.fixed_issue_ids:
                                all_inducing_have_path = False
                        else:
                            file_deleted = False
                            blame_file = File.objects(id=ifa.file_id).get().path
                            path = nx.shortest_path(commit_graph, blame_commit, release_commit.revision_hash)
                            for i in range(1,len(path)):
                                if path[i] in rename_cache:
                                    renames = rename_cache[path[i]]
                                else:
                                    renames = heuristic_renames(vcs_system_id, path[i])
                                    rename_cache[path[i]] = renames
                                if renames is not None:
                                    for rename in renames[0]:
                                        if blame_file==rename[0]:
                                            blame_file = rename[1]
                                    if blame_file in renames[1]:
                                        file_deleted = True
                                        break
                                if path[i] not in delete_cache:
                                    delete_cache[path[i]] = set()
                                    path_commit_id = Commit.objects(vcs_system_id=vcs_system_id, revision_hash=path[i]).only('revision_hash').get().id
                                    for dfa in FileAction.objects(commit_id=path_commit_id,mode='D'):
                                        deleted_file = File.objects(id=dfa.file_id).get().path
                                        delete_cache[path[i]].add(deleted_file)
                                if blame_file in delete_cache[path[i]]:
                                    file_deleted = True
                                    break
                            
                            if not file_deleted:
                                if not java_filename_filter(blame_file):
                                    continue
                                blame_file_cache.add(blame_file)
                                
            if len(blame_file_cache)>0 and all_inducing_have_path:
                assigned_issues_jlmivp_inducing.add(issue.id)
                affected_files_jlmivp_inducing.update(blame_file_cache)
        
        affected_files_szz_six_months_in_release = affected_files_szz_six_months.intersection(files_release)
        affected_files_jlr_six_months_in_release = affected_files_jlr_six_months.intersection(files_release)
        affected_files_jlmiv_six_months_in_release = affected_files_jlmiv_six_months.intersection(files_release)
        affected_files_jlmiv_affected_version_in_release = affected_files_jlmiv_affected_version.intersection(files_release)
        affected_files_jlmiv_inducing_in_release = affected_files_jlmivp_inducing.intersection(files_release)
        
        releasedata.at[current_index, 'issues_szz_six_months_tp'] = len(assigned_issues_szz_six_months.intersection(assigned_issues_jlmivp_inducing))
        releasedata.at[current_index, 'issues_szz_six_months_fp'] = len(assigned_issues_szz_six_months.difference(assigned_issues_jlmivp_inducing))
        releasedata.at[current_index, 'issues_jlr_six_months_tp'] = len(assigned_issues_jlr_six_months.intersection(assigned_issues_jlmivp_inducing))
        releasedata.at[current_index, 'issues_jlr_six_months_fp'] = len(assigned_issues_jlr_six_months.difference(assigned_issues_jlmivp_inducing))
        releasedata.at[current_index, 'issues_jlmiv_six_months_tp'] = len(assigned_issues_jlmiv_six_months.intersection(assigned_issues_jlmivp_inducing))
        releasedata.at[current_index, 'issues_jlmiv_six_months_fp'] = len(assigned_issues_jlmiv_six_months.difference(assigned_issues_jlmivp_inducing))
        releasedata.at[current_index, 'issues_jlmiv_affected_version_tp'] = len(assigned_issues_jlmiv_affected_version.intersection(assigned_issues_jlmivp_inducing))
        releasedata.at[current_index, 'issues_jlmiv_affected_version_fp'] = len(assigned_issues_jlmiv_affected_version.difference(assigned_issues_jlmivp_inducing))
        releasedata.at[current_index, 'issues_jlmiv_inducing'] = len(assigned_issues_jlmivp_inducing)
        releasedata.at[current_index, 'files_szz_six_months_tp'] = len(affected_files_szz_six_months_in_release.intersection(affected_files_jlmiv_inducing_in_release))
        releasedata.at[current_index, 'files_szz_six_months_fp'] = len(affected_files_szz_six_months_in_release.difference(affected_files_jlmiv_inducing_in_release))
        releasedata.at[current_index, 'files_jlr_six_months_tp'] = len(affected_files_jlr_six_months_in_release.intersection(affected_files_jlmiv_inducing_in_release))
        releasedata.at[current_index, 'files_jlr_six_months_fp'] = len(affected_files_jlr_six_months_in_release.difference(affected_files_jlmiv_inducing_in_release))
        releasedata.at[current_index, 'files_jlmiv_six_months_tp'] = len(affected_files_jlmiv_six_months_in_release.intersection(affected_files_jlmiv_inducing_in_release))
        releasedata.at[current_index, 'files_jlmiv_six_months_fp'] = len(affected_files_jlmiv_six_months_in_release.difference(affected_files_jlmiv_inducing_in_release))
        releasedata.at[current_index, 'files_jlmiv_affected_version_tp'] = len(affected_files_jlmiv_affected_version_in_release.intersection(affected_files_jlmiv_inducing_in_release))
        releasedata.at[current_index, 'files_jlmiv_affected_version_fp'] = len(affected_files_jlmiv_affected_version_in_release.difference(affected_files_jlmiv_inducing_in_release))
        releasedata.at[current_index, 'files_jlmiv_inducing'] = len(affected_files_jlmiv_inducing_in_release)
        releasedata.at[current_index, 'num_java_files'] = len(files_release)
        releasedata.at[current_index, 'overlap_6mav'] = len(affected_files_jlmiv_six_months_in_release.intersection(affected_files_jlmiv_affected_version_in_release))
        releasedata.at[current_index, 'overlap_6min'] = len(affected_files_jlmiv_six_months_in_release.intersection(affected_files_jlmiv_inducing_in_release))
        releasedata.at[current_index, 'overlap_avin'] = len(affected_files_jlmiv_affected_version_in_release.intersection(affected_files_jlmiv_inducing_in_release))

releasedata['percent_files_jlmiv_ind_total'] = releasedata['files_jlmiv_inducing']/releasedata['num_java_files']

releasedata['percent_issues_szz_six_months_tp'] = releasedata['issues_szz_six_months_tp']/releasedata['issues_jlmiv_inducing']
releasedata['percent_issues_szz_six_months_fp'] = releasedata['issues_szz_six_months_fp']/releasedata['issues_jlmiv_inducing']
releasedata['percent_issues_jlr_six_months_tp'] = releasedata['issues_jlr_six_months_tp']/releasedata['issues_jlmiv_inducing']
releasedata['percent_issues_jlr_six_months_fp'] = releasedata['issues_jlr_six_months_fp']/releasedata['issues_jlmiv_inducing']
releasedata['percent_issues_jlmiv_six_months_tp'] = releasedata['issues_jlmiv_six_months_tp']/releasedata['issues_jlmiv_inducing']
releasedata['percent_issues_jlmiv_six_months_fp'] = releasedata['issues_jlmiv_six_months_fp']/releasedata['issues_jlmiv_inducing']
releasedata['percent_issues_jlmiv_affected_version_tp'] = releasedata['issues_jlmiv_affected_version_tp']/releasedata['issues_jlmiv_inducing']
releasedata['percent_issues_jlmiv_affected_version_fp'] = releasedata['issues_jlmiv_affected_version_fp']/releasedata['issues_jlmiv_inducing']

releasedata['percent_files_szz_six_months_tp'] = releasedata['files_szz_six_months_tp']/releasedata['files_jlmiv_inducing']
releasedata['percent_files_szz_six_months_fp'] = releasedata['files_szz_six_months_fp']/releasedata['files_jlmiv_inducing']
releasedata['percent_files_jlr_six_months_tp'] = releasedata['files_jlr_six_months_tp']/releasedata['files_jlmiv_inducing']
releasedata['percent_files_jlr_six_months_fp'] = releasedata['files_jlr_six_months_fp']/releasedata['files_jlmiv_inducing']
releasedata['percent_files_jlmiv_six_months_tp'] = releasedata['files_jlmiv_six_months_tp']/releasedata['files_jlmiv_inducing']
releasedata['percent_files_jlmiv_six_months_fp'] = releasedata['files_jlmiv_six_months_fp']/releasedata['files_jlmiv_inducing']
releasedata['percent_files_jlmiv_affected_version_tp'] = releasedata['files_jlmiv_affected_version_tp']/releasedata['files_jlmiv_inducing']
releasedata['percent_files_jlmiv_affected_version_fp'] = releasedata['files_jlmiv_affected_version_fp']/releasedata['files_jlmiv_inducing']

releasedata['percent_files_correct_szz_6m'] = releasedata['files_szz_six_months_tp']/(releasedata['files_szz_six_months_tp']+releasedata['files_szz_six_months_fp'])
releasedata['percent_files_correct_jlr_6m'] = releasedata['files_jlr_six_months_tp']/(releasedata['files_jlr_six_months_tp']+releasedata['files_jlr_six_months_fp'])
releasedata['percent_files_correct_jlmiv_6m'] = releasedata['files_jlmiv_six_months_tp']/(releasedata['files_jlmiv_six_months_tp']+releasedata['files_jlmiv_six_months_fp'])
releasedata['percent_files_correct_jlmiv_av'] = releasedata['files_jlmiv_affected_version_tp']/(releasedata['files_jlmiv_affected_version_tp']+releasedata['files_jlmiv_affected_version_fp'])

display(releasedata)

print("statistics with all data")
df = releasedata
print("mean values")
print(df.mean().to_string())
print()
print("sd values")
print(df.std().to_string())
print()
print("median values")
print(df.median().to_string())
print()
print("MAD values")
print(mad(df).to_string())

print()
print("statistics without the", sum(releasedata['issues_jlmiv_inducing']==0), "releases without any assigned issue")
df = releasedata[releasedata['issues_jlmiv_inducing']>0]
print("mean values")
print(df.mean().to_string())
print()
print("sd values")
print(df.std().to_string())
print()
print("median values")
print(df.median().to_string())
print()
print("MAD values")
print(mad(df).to_string())

f, axes = plt.subplots(3, 2, gridspec_kw={'height_ratios': [1, 4, 4]}, figsize=(9,3))

df = releasedata[['issues_jlmiv_inducing']]
df.columns = ['Issues']
ax = sns.boxplot(data=df, orient='h', color='lightgray', ax=axes[0,0])
ax = sns.swarmplot(data=df, orient='h', color='gray', alpha=0.2, ax=axes[0,0])
axes[0,0].set_xlabel('Number of issues assigned to releases.')

df = releasedata[['percent_files_jlmiv_ind_total']]
df.columns = ['Files']
sns.boxplot(data=df, orient='h', color='lightgray', ax=axes[0,1])
sns.swarmplot(data=df, orient='h', color='gray', alpha=0.2, ax=axes[0,1])
axes[0,1].set_xlabel('Percentage of filess labeled\nas defective per release.')

df = releasedata[['percent_issues_szz_six_months_tp', 'percent_issues_jlr_six_months_tp', 'percent_issues_jlmiv_six_months_tp','percent_issues_jlmiv_affected_version_tp']]
df.columns = ['6M-SZZ', '6M-JL+R', '6M-JLMIV', 'AV-JLMIV']
ax = sns.boxplot(data=df, orient='h', color='lightgray', ax=axes[1,0])
ax = sns.swarmplot(data=df, orient='h', color='gray', alpha=0.2, ax=axes[1,0])
axes[1,0].set_xlabel('Percentage of correctly assigned issues.')

df = releasedata[['percent_issues_szz_six_months_fp', 'percent_issues_jlr_six_months_fp', 'percent_issues_jlmiv_six_months_fp','percent_issues_jlmiv_affected_version_fp']]
df.columns = ['6M-SZZ', '6M-JL+R', '6M-JLMIV', 'AV-JLMIV']
ax = sns.boxplot(data=df, orient='h', color='lightgray', ax=axes[1,1])
ax = sns.swarmplot(data=df, orient='h', color='gray', alpha=0.2, ax=axes[1,1])
axes[1,1].set_xlabel('Percentage of additionally assigned issues.')
ax.set_xscale('log')

df = releasedata[['percent_files_szz_six_months_tp', 'percent_files_jlr_six_months_tp', 'percent_files_jlmiv_six_months_tp','percent_files_jlmiv_affected_version_tp']]
df.columns = ['6M-SZZ', '6M-JL+R', '6M-JLMIV', 'AV-JLMIV']
ax = sns.boxplot(data=df, orient='h', color='lightgray', ax=axes[2,0])
ax = sns.swarmplot(data=df, orient='h', color='gray', alpha=0.2, ax=axes[2,0])
axes[2,0].set_xlabel('Percentage of correctly labeled files.')

df = releasedata[['percent_files_szz_six_months_fp', 'percent_files_jlr_six_months_fp', 'percent_files_jlmiv_six_months_fp','percent_files_jlmiv_affected_version_fp']]
df.columns = ['6M-SZZ', '6M-JL+R', '6M-JLMIV', 'AV-JLMIV']
ax = sns.boxplot(data=df, orient='h', color='lightgray', ax=axes[2,1])
ax = sns.swarmplot(data=df, orient='h', color='gray', alpha=0.2, ax=axes[2,1])
axes[2,1].set_xlabel('Percentage of additionally labelled files.')
axes[2,1].set_xscale('log')

plt.subplots_adjust(left=None, bottom=-0.6, right=None, top=None, wspace=0.5, hspace=0.6)
#plt.savefig("releasedata.pdf", bbox_inches='tight')
plt.show()

	issues_szz_six_months_tp	issues_szz_six_months_fp	issues_jlr_six_months_tp	issues_jlr_six_months_fp	issues_jlmiv_six_months_tp	issues_jlmiv_six_months_fp	issues_jlmiv_affected_version_tp	issues_jlmiv_affected_version_fp	issues_jlmiv_inducing	files_szz_six_months_tp	files_szz_six_months_fp	files_jlr_six_months_tp	files_jlr_six_months_fp	files_jlmiv_six_months_tp	files_jlmiv_six_months_fp	files_jlmiv_affected_version_tp	files_jlmiv_affected_version_fp	files_jlmiv_inducing	num_java_files	overlap_6mav	overlap_6min	overlap_avin	percent_files_jlmiv_ind_total	percent_issues_szz_six_months_tp	percent_issues_szz_six_months_fp	percent_issues_jlr_six_months_tp	percent_issues_jlr_six_months_fp	percent_issues_jlmiv_six_months_tp	percent_issues_jlmiv_six_months_fp	percent_issues_jlmiv_affected_version_tp	percent_issues_jlmiv_affected_version_fp	percent_files_szz_six_months_tp	percent_files_szz_six_months_fp	percent_files_jlr_six_months_tp	percent_files_jlr_six_months_fp	percent_files_jlmiv_six_months_tp	percent_files_jlmiv_six_months_fp	percent_files_jlmiv_affected_version_tp	percent_files_jlmiv_affected_version_fp	percent_files_correct_szz_6m	percent_files_correct_jlr_6m	percent_files_correct_jlmiv_6m	percent_files_correct_jlmiv_av
ant-ivy-1.4.1	6	27	6	23	6	25	5	23	7	11	43	11	19	11	40	8	47	11	240	46	11	8	0.045833	0.857143	3.857143	0.857143	3.285714	0.857143	3.571429	0.714286	3.285714	1.000000	3.909091	1.000000	1.727273	1.000000	3.636364	0.727273	4.272727	0.203704	0.366667	0.215686	0.145455
ant-ivy-2.0.0	12	7	12	4	12	4	10	2	56	19	9	18	4	17	5	13	4	50	352	17	17	13	0.142045	0.214286	0.125000	0.214286	0.071429	0.214286	0.071429	0.178571	0.035714	0.380000	0.180000	0.360000	0.080000	0.340000	0.100000	0.260000	0.080000	0.678571	0.818182	0.772727	0.764706
ant-ivy-2.1.0	18	8	18	4	18	6	16	6	53	25	8	24	3	23	4	27	8	49	357	20	23	27	0.137255	0.339623	0.150943	0.339623	0.075472	0.339623	0.113208	0.301887	0.113208	0.510204	0.163265	0.489796	0.061224	0.469388	0.081633	0.551020	0.163265	0.757576	0.888889	0.851852	0.771429
ant-ivy-2.2.0	10	3	10	1	10	1	18	9	37	13	4	12	1	13	2	25	8	37	363	14	13	25	0.101928	0.270270	0.081081	0.270270	0.027027	0.270270	0.027027	0.486486	0.243243	0.351351	0.108108	0.324324	0.027027	0.351351	0.054054	0.675676	0.216216	0.764706	0.923077	0.866667	0.757576
ant-ivy-2.3.0	1	1	1	1	1	1	2	3	11	1	1	1	1	1	1	3	6	13	451	2	1	3	0.028825	0.090909	0.090909	0.090909	0.090909	0.090909	0.090909	0.181818	0.272727	0.076923	0.076923	0.076923	0.076923	0.076923	0.076923	0.230769	0.461538	0.500000	0.500000	0.500000	0.333333
ant-ivy-2.4.0	0	1	0	0	0	0	1	2	17	0	1	0	0	0	0	3	1	17	474	0	0	3	0.035865	0.000000	0.058824	0.000000	0.000000	0.000000	0.000000	0.058824	0.117647	0.000000	0.058824	0.000000	0.000000	0.000000	0.000000	0.176471	0.058824	0.000000	NaN	NaN	0.750000
archiva-1.0	0	20	4	17	4	18	4	16	4	2	71	7	30	7	29	7	27	7	430	34	7	7	0.016279	0.000000	5.000000	1.000000	4.250000	1.000000	4.500000	1.000000	4.000000	0.285714	10.142857	1.000000	4.285714	1.000000	4.142857	1.000000	3.857143	0.027397	0.189189	0.194444	0.205882
archiva-1.1	6	6	10	10	10	7	10	10	28	7	8	16	8	16	6	14	13	32	467	18	16	14	0.068522	0.214286	0.214286	0.357143	0.357143	0.357143	0.250000	0.357143	0.357143	0.218750	0.250000	0.500000	0.250000	0.500000	0.187500	0.437500	0.406250	0.466667	0.666667	0.727273	0.518519
archiva-1.2	0	1	6	12	6	14	6	13	23	1	1	7	13	8	26	8	26	20	449	34	8	8	0.044543	0.000000	0.043478	0.260870	0.521739	0.260870	0.608696	0.260870	0.565217	0.050000	0.050000	0.350000	0.650000	0.400000	1.300000	0.400000	1.300000	0.500000	0.350000	0.235294	0.235294
archiva-1.3	4	2	5	6	5	2	5	2	14	5	0	6	2	6	1	6	2	16	448	5	6	6	0.035714	0.285714	0.142857	0.357143	0.428571	0.357143	0.142857	0.357143	0.142857	0.312500	0.000000	0.375000	0.125000	0.375000	0.062500	0.375000	0.125000	1.000000	0.750000	0.857143	0.750000
archiva-2.0.0	0	1	2	3	2	1	2	0	8	0	1	5	9	5	1	5	2	9	459	4	5	5	0.019608	0.000000	0.125000	0.250000	0.375000	0.250000	0.125000	0.250000	0.000000	0.000000	0.111111	0.555556	1.000000	0.555556	0.111111	0.555556	0.222222	0.000000	0.357143	0.833333	0.714286
archiva-2.1.0	2	1	5	1	5	1	0	0	8	2	8	5	1	5	4	0	0	6	458	0	5	0	0.013100	0.250000	0.125000	0.625000	0.125000	0.625000	0.125000	0.000000	0.000000	0.333333	1.333333	0.833333	0.166667	0.833333	0.666667	0.000000	0.000000	0.200000	0.833333	0.555556	NaN
archiva-2.2.0	0	0	1	1	1	0	1	0	4	0	0	1	2	1	1	1	1	3	457	1	1	1	0.006565	0.000000	0.000000	0.250000	0.250000	0.250000	0.000000	0.250000	0.000000	0.000000	0.000000	0.333333	0.666667	0.333333	0.333333	0.333333	0.333333	NaN	0.333333	0.500000	0.500000
calcite-1.0.0	27	73	27	65	27	29	0	0	88	142	155	131	101	123	54	0	0	178	1075	0	123	0	0.165581	0.306818	0.829545	0.306818	0.738636	0.306818	0.329545	0.000000	0.000000	0.797753	0.870787	0.735955	0.567416	0.691011	0.303371	0.000000	0.000000	0.478114	0.564655	0.694915	NaN
calcite-1.1.0	30	76	31	63	31	27	0	0	89	64	151	61	108	50	52	0	0	113	1103	0	50	0	0.102448	0.337079	0.853933	0.348315	0.707865	0.348315	0.303371	0.000000	0.000000	0.566372	1.336283	0.539823	0.955752	0.442478	0.460177	0.000000	0.000000	0.297674	0.360947	0.490196	NaN
calcite-1.2.0	36	65	37	56	37	16	0	0	95	75	163	72	125	58	41	0	0	126	1108	0	58	0	0.113718	0.378947	0.684211	0.389474	0.589474	0.389474	0.168421	0.000000	0.000000	0.595238	1.293651	0.571429	0.992063	0.460317	0.325397	0.000000	0.000000	0.315126	0.365482	0.585859	NaN
calcite-1.3.0	34	70	37	59	37	21	0	0	89	66	204	64	164	49	43	0	0	112	1115	0	49	0	0.100448	0.382022	0.786517	0.415730	0.662921	0.415730	0.235955	0.000000	0.000000	0.589286	1.821429	0.571429	1.464286	0.437500	0.383929	0.000000	0.000000	0.244444	0.280702	0.532609	NaN
calcite-1.4.0	31	85	32	72	32	28	0	0	88	91	368	87	307	66	41	0	0	123	1127	0	66	0	0.109139	0.352273	0.965909	0.363636	0.818182	0.363636	0.318182	0.000000	0.000000	0.739837	2.991870	0.707317	2.495935	0.536585	0.333333	0.000000	0.000000	0.198257	0.220812	0.616822	NaN
calcite-1.5.0	18	60	18	46	18	20	6	3	83	56	310	51	256	28	47	10	15	103	1176	23	28	10	0.087585	0.216867	0.722892	0.216867	0.554217	0.216867	0.240964	0.072289	0.036145	0.543689	3.009709	0.495146	2.485437	0.271845	0.456311	0.097087	0.145631	0.153005	0.166124	0.373333	0.400000
calcite-1.6.0	17	46	18	39	18	14	6	1	86	69	225	68	196	56	63	5	4	107	1193	8	56	5	0.089690	0.197674	0.534884	0.209302	0.453488	0.209302	0.162791	0.069767	0.011628	0.644860	2.102804	0.635514	1.831776	0.523364	0.588785	0.046729	0.037383	0.234694	0.257576	0.470588	0.555556
calcite-1.7.0	20	49	21	44	21	17	9	0	94	88	128	87	102	79	49	40	5	128	1252	45	79	40	0.102236	0.212766	0.521277	0.223404	0.468085	0.223404	0.180851	0.095745	0.000000	0.687500	1.000000	0.679688	0.796875	0.617188	0.382812	0.312500	0.039062	0.407407	0.460317	0.617188	0.888889
calcite-1.8.0	32	70	34	60	34	25	3	3	93	71	120	69	97	63	54	4	3	101	1301	7	63	4	0.077633	0.344086	0.752688	0.365591	0.645161	0.365591	0.268817	0.032258	0.032258	0.702970	1.188119	0.683168	0.960396	0.623762	0.534653	0.039604	0.029703	0.371728	0.415663	0.538462	0.571429
calcite-1.9.0	52	104	56	62	56	27	9	2	102	79	285	73	194	70	83	18	5	90	1310	23	70	18	0.068702	0.509804	1.019608	0.549020	0.607843	0.549020	0.264706	0.088235	0.019608	0.877778	3.166667	0.811111	2.155556	0.777778	0.922222	0.200000	0.055556	0.217033	0.273408	0.457516	0.782609
calcite-1.10.0	46	103	49	59	49	27	10	2	93	74	409	68	195	65	79	17	8	84	1310	25	65	17	0.064122	0.494624	1.107527	0.526882	0.634409	0.526882	0.290323	0.107527	0.021505	0.880952	4.869048	0.809524	2.321429	0.773810	0.940476	0.202381	0.095238	0.153209	0.258555	0.451389	0.680000
calcite-1.11.0	48	103	51	53	51	27	13	5	85	68	381	62	171	61	106	15	4	80	1331	18	61	15	0.060105	0.564706	1.211765	0.600000	0.623529	0.600000	0.317647	0.152941	0.058824	0.850000	4.762500	0.775000	2.137500	0.762500	1.325000	0.187500	0.050000	0.151448	0.266094	0.365269	0.789474
calcite-1.12.0	32	83	32	45	32	20	9	6	63	72	448	69	184	63	37	28	7	81	1415	34	63	28	0.057244	0.507937	1.317460	0.507937	0.714286	0.507937	0.317460	0.142857	0.095238	0.888889	5.530864	0.851852	2.271605	0.777778	0.456790	0.345679	0.086420	0.138462	0.272727	0.630000	0.800000
calcite-1.13.0	22	89	22	49	22	25	3	3	50	42	363	39	269	31	20	6	4	53	1275	9	31	6	0.041569	0.440000	1.780000	0.440000	0.980000	0.440000	0.500000	0.060000	0.060000	0.792453	6.849057	0.735849	5.075472	0.584906	0.377358	0.113208	0.075472	0.103704	0.126623	0.607843	0.600000
calcite-1.14.0	35	74	35	37	35	24	3	2	45	47	192	47	131	46	17	6	2	53	1308	8	46	6	0.040520	0.777778	1.644444	0.777778	0.822222	0.777778	0.533333	0.066667	0.044444	0.886792	3.622642	0.886792	2.471698	0.867925	0.320755	0.113208	0.037736	0.196653	0.264045	0.730159	0.750000
calcite-1.15.0	37	65	37	27	37	23	9	4	37	45	92	45	28	45	30	11	7	45	1352	18	45	11	0.033284	1.000000	1.756757	1.000000	0.729730	1.000000	0.621622	0.243243	0.108108	1.000000	2.044444	1.000000	0.622222	1.000000	0.666667	0.244444	0.155556	0.328467	0.616438	0.600000	0.611111
cayenne-3.0.0	0	8	0	3	0	1	0	1	42	3	3	1	3	1	1	1	1	44	1708	2	1	1	0.025761	0.000000	0.190476	0.000000	0.071429	0.000000	0.023810	0.000000	0.023810	0.068182	0.068182	0.022727	0.068182	0.022727	0.022727	0.022727	0.022727	0.500000	0.250000	0.500000	0.500000
cayenne-3.1.0	3	1	3	1	3	1	3	2	15	3	1	3	1	3	1	3	1	19	1578	3	3	3	0.012041	0.200000	0.066667	0.200000	0.066667	0.200000	0.066667	0.200000	0.133333	0.157895	0.052632	0.157895	0.052632	0.157895	0.052632	0.157895	0.052632	0.750000	0.750000	0.750000	0.750000
commons-bcel-5.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	325	0	0	0	0.000000	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
commons-bcel-5.1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	334	0	0	0	0.000000	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
commons-bcel-5.2	0	0	0	0	0	0	0	3	1	0	0	0	0	0	0	0	3	1	335	0	0	0	0.002985	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	3.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	3.000000	NaN	NaN	NaN	0.000000
commons-bcel-6.0	2	0	1	0	2	0	3	0	8	2	0	1	0	2	0	3	1	13	376	2	2	3	0.034574	0.250000	0.000000	0.125000	0.000000	0.250000	0.000000	0.375000	0.000000	0.153846	0.000000	0.076923	0.000000	0.153846	0.000000	0.230769	0.076923	1.000000	1.000000	1.000000	0.750000
commons-bcel-6.1	0	0	1	0	1	0	0	0	4	0	0	2	0	2	0	0	0	10	378	0	2	0	0.026455	0.000000	0.000000	0.250000	0.000000	0.250000	0.000000	0.000000	0.000000	0.000000	0.000000	0.200000	0.000000	0.200000	0.000000	0.000000	0.000000	NaN	1.000000	1.000000	NaN
commons-bcel-6.2	0	0	0	0	0	0	1	0	3	0	0	0	0	0	0	1	0	9	378	0	0	1	0.023810	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.333333	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.111111	0.000000	NaN	NaN	NaN	1.000000
commons-beanutils-1.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	3	0	0	0	0.000000	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
commons-beanutils-1.1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	4	0	0	0	0.000000	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
commons-beanutils-1.2	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	5	0	0	0	0.000000	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
commons-beanutils-1.3	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	29	0	0	0	0.000000	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
commons-beanutils-1.4	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	43	0	0	0	0.000000	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
commons-beanutils-1.5	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	62	0	0	0	0.000000	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
commons-beanutils-1.6	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	64	0	0	0	0.000000	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
commons-beanutils-1.7.0	0	0	0	0	0	0	6	8	11	0	0	0	0	0	0	7	5	7	87	0	0	7	0.080460	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.545455	0.727273	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	1.000000	0.714286	NaN	NaN	NaN	0.583333
commons-beanutils-1.8.0	0	0	0	0	0	0	5	2	13	0	0	0	0	0	0	5	0	8	101	0	0	5	0.079208	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.384615	0.153846	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.625000	0.000000	NaN	NaN	NaN	1.000000
commons-beanutils-1.9.0	0	0	1	0	1	1	1	1	1	0	0	1	0	1	1	1	1	1	104	2	1	1	0.009615	0.000000	0.000000	1.000000	0.000000	1.000000	1.000000	1.000000	1.000000	0.000000	0.000000	1.000000	0.000000	1.000000	1.000000	1.000000	1.000000	NaN	1.000000	0.500000	0.500000
commons-codec-1.1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	14	0	0	0	0.000000	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
commons-codec-1.2	0	0	0	0	0	0	0	0	2	0	0	0	0	0	0	0	0	2	17	0	0	0	0.117647	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
commons-codec-1.3	0	0	0	0	0	0	2	1	3	0	0	0	0	0	0	2	2	3	24	0	0	2	0.125000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.666667	0.333333	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.666667	0.666667	NaN	NaN	NaN	0.500000
commons-codec-1.4	0	1	0	2	0	1	0	1	0	0	1	0	2	0	1	0	1	0	29	1	0	0	0.000000	NaN	inf	NaN	inf	NaN	inf	NaN	inf	NaN	inf	NaN	inf	NaN	inf	NaN	inf	0.000000	0.000000	0.000000	0.000000
commons-codec-1.5	0	2	0	1	0	0	0	1	1	0	8	0	1	0	0	0	1	1	39	0	0	0	0.025641	0.000000	2.000000	0.000000	1.000000	0.000000	0.000000	0.000000	1.000000	0.000000	8.000000	0.000000	1.000000	0.000000	0.000000	0.000000	1.000000	0.000000	0.000000	NaN	0.000000
commons-codec-1.6	0	1	2	6	2	1	1	0	3	0	1	4	5	4	3	2	0	4	47	2	4	2	0.085106	0.000000	0.333333	0.666667	2.000000	0.666667	0.333333	0.333333	0.000000	0.000000	0.250000	1.000000	1.250000	1.000000	0.750000	0.500000	0.000000	0.000000	0.444444	0.571429	1.000000
commons-codec-1.7	0	1	0	0	0	1	0	0	3	0	1	0	0	0	1	0	0	3	55	0	0	0	0.054545	0.000000	0.333333	0.000000	0.000000	0.000000	0.333333	0.000000	0.000000	0.000000	0.333333	0.000000	0.000000	0.000000	0.333333	0.000000	0.000000	0.000000	NaN	0.000000	NaN
commons-codec-1.8	0	1	0	1	0	0	0	0	5	1	1	1	0	0	0	0	0	6	56	0	0	0	0.107143	0.000000	0.200000	0.000000	0.200000	0.000000	0.000000	0.000000	0.000000	0.166667	0.166667	0.166667	0.000000	0.000000	0.000000	0.000000	0.000000	0.500000	1.000000	NaN	NaN
commons-codec-1.9	1	1	1	0	2	0	2	1	5	3	15	1	0	2	1	4	5	6	56	3	2	4	0.107143	0.200000	0.200000	0.200000	0.000000	0.400000	0.000000	0.400000	0.200000	0.500000	2.500000	0.166667	0.000000	0.333333	0.166667	0.666667	0.833333	0.166667	1.000000	0.666667	0.444444
commons-codec-1.10	0	0	1	1	1	0	4	1	4	0	0	1	1	1	0	3	2	3	60	1	1	3	0.050000	0.000000	0.000000	0.250000	0.250000	0.250000	0.000000	1.000000	0.250000	0.000000	0.000000	0.333333	0.333333	0.333333	0.000000	1.000000	0.666667	NaN	0.500000	1.000000	0.600000
commons-codec-1.11	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	64	0	0	0	0.000000	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
commons-collections-1.0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	1	26	0	0	0	0.038462	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
commons-collections-2.0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	1	44	0	0	0	0.022727	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
commons-collections-2.1	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	1	72	0	0	0	0.013889	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
commons-collections-3.0	0	0	0	0	0	0	0	1	9	0	0	0	0	0	0	0	11	8	248	0	0	0	0.032258	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.111111	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	1.375000	NaN	NaN	NaN	0.000000
commons-collections-3.1	0	0	0	0	0	0	0	0	9	0	0	0	0	0	0	0	0	8	267	0	0	0	0.029963	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
commons-collections-3.2	1	3	1	2	1	2	11	11	14	3	1	2	1	2	1	8	8	11	273	3	2	8	0.040293	0.071429	0.214286	0.071429	0.142857	0.071429	0.142857	0.785714	0.785714	0.272727	0.090909	0.181818	0.090909	0.181818	0.090909	0.727273	0.727273	0.750000	0.666667	0.666667	0.500000
commons-collections-3.3	0	0	0	0	0	0	0	0	12	0	0	0	0	0	0	0	0	10	273	0	0	0	0.036630	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
commons-collections-4.0	0	0	3	1	3	1	6	2	16	0	0	4	1	4	1	8	13	14	270	4	4	8	0.051852	0.000000	0.000000	0.187500	0.062500	0.187500	0.062500	0.375000	0.125000	0.000000	0.000000	0.285714	0.071429	0.285714	0.071429	0.571429	0.928571	NaN	0.800000	0.800000	0.380952
commons-collections-4.1	0	0	0	0	0	0	1	1	1	0	0	0	0	0	0	1	1	1	301	0	0	1	0.003322	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	1.000000	1.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	1.000000	1.000000	NaN	NaN	NaN	0.500000
commons-compress-1.0	1	2	1	0	1	2	8	4	18	3	0	1	0	3	0	11	4	17	61	3	3	11	0.278689	0.055556	0.111111	0.055556	0.000000	0.055556	0.111111	0.444444	0.222222	0.176471	0.000000	0.058824	0.000000	0.176471	0.000000	0.647059	0.235294	1.000000	1.000000	1.000000	0.733333
commons-compress-1.1	0	0	1	0	1	0	3	1	16	0	0	1	0	2	0	5	2	14	66	2	2	5	0.212121	0.000000	0.000000	0.062500	0.000000	0.062500	0.000000	0.187500	0.062500	0.000000	0.000000	0.071429	0.000000	0.142857	0.000000	0.357143	0.142857	NaN	1.000000	1.000000	0.714286
commons-compress-1.2	0	3	3	4	3	4	2	3	13	1	3	4	2	4	5	2	4	10	69	4	4	2	0.144928	0.000000	0.230769	0.230769	0.307692	0.230769	0.307692	0.153846	0.230769	0.100000	0.300000	0.400000	0.200000	0.400000	0.500000	0.200000	0.400000	0.250000	0.666667	0.444444	0.333333
commons-compress-1.3	3	8	6	6	6	4	4	3	19	7	4	7	1	7	2	5	1	16	93	6	7	5	0.172043	0.157895	0.421053	0.315789	0.315789	0.315789	0.210526	0.210526	0.157895	0.437500	0.250000	0.437500	0.062500	0.437500	0.125000	0.312500	0.062500	0.636364	0.875000	0.777778	0.833333
commons-compress-1.4	1	2	1	2	1	0	7	4	24	2	3	2	1	1	0	7	3	16	99	1	1	7	0.161616	0.041667	0.083333	0.041667	0.083333	0.041667	0.000000	0.291667	0.166667	0.125000	0.187500	0.125000	0.062500	0.062500	0.000000	0.437500	0.187500	0.400000	0.666667	1.000000	0.700000
commons-compress-1.5	3	2	3	0	3	1	6	1	30	4	2	4	0	4	2	8	2	21	103	6	4	8	0.203883	0.100000	0.066667	0.100000	0.000000	0.100000	0.033333	0.200000	0.033333	0.190476	0.095238	0.190476	0.000000	0.190476	0.095238	0.380952	0.095238	0.666667	1.000000	0.666667	0.800000
commons-compress-1.6	12	12	12	3	12	4	7	0	33	22	23	12	4	14	7	7	0	23	124	7	14	7	0.185484	0.363636	0.363636	0.363636	0.090909	0.363636	0.121212	0.212121	0.000000	0.956522	1.000000	0.521739	0.173913	0.608696	0.304348	0.304348	0.000000	0.488889	0.750000	0.666667	1.000000
commons-compress-1.7	11	6	11	1	11	2	6	1	33	18	6	14	0	17	4	7	0	27	136	7	17	7	0.198529	0.333333	0.181818	0.333333	0.030303	0.333333	0.060606	0.181818	0.030303	0.666667	0.222222	0.518519	0.000000	0.629630	0.148148	0.259259	0.000000	0.750000	1.000000	0.809524	1.000000
commons-compress-1.8	9	2	9	0	9	0	10	0	30	19	6	17	0	18	5	18	5	31	141	23	18	18	0.219858	0.300000	0.066667	0.300000	0.000000	0.300000	0.000000	0.333333	0.000000	0.612903	0.193548	0.548387	0.000000	0.580645	0.161290	0.580645	0.161290	0.760000	1.000000	0.782609	0.782609
commons-compress-1.9	3	3	3	3	3	1	5	2	22	10	3	9	1	9	2	11	2	23	145	11	9	11	0.158621	0.136364	0.136364	0.136364	0.136364	0.136364	0.045455	0.227273	0.090909	0.434783	0.130435	0.391304	0.043478	0.391304	0.086957	0.478261	0.086957	0.769231	0.900000	0.818182	0.846154
commons-compress-1.10	6	5	6	0	6	2	7	2	19	8	1	5	0	6	1	11	2	18	158	7	6	11	0.113924	0.315789	0.263158	0.315789	0.000000	0.315789	0.105263	0.368421	0.105263	0.444444	0.055556	0.277778	0.000000	0.333333	0.055556	0.611111	0.111111	0.888889	1.000000	0.857143	0.846154
commons-compress-1.11	0	0	0	0	0	0	0	0	13	0	0	0	0	0	0	0	0	13	165	0	0	0	0.078788	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
commons-compress-1.12	1	3	1	1	1	1	2	4	10	2	4	2	0	1	1	3	9	10	166	2	1	3	0.060241	0.100000	0.300000	0.100000	0.100000	0.100000	0.100000	0.200000	0.400000	0.200000	0.400000	0.200000	0.000000	0.100000	0.100000	0.300000	0.900000	0.333333	1.000000	0.500000	0.250000
commons-compress-1.13	3	5	3	0	3	1	1	0	13	9	3	8	0	8	3	1	1	17	177	2	8	1	0.096045	0.230769	0.384615	0.230769	0.000000	0.230769	0.076923	0.076923	0.000000	0.529412	0.176471	0.470588	0.000000	0.470588	0.176471	0.058824	0.058824	0.750000	1.000000	0.727273	0.500000
commons-compress-1.14	3	6	4	0	4	2	5	1	9	4	12	4	0	4	3	5	1	8	195	4	4	5	0.041026	0.333333	0.666667	0.444444	0.000000	0.444444	0.222222	0.555556	0.111111	0.500000	1.500000	0.500000	0.000000	0.500000	0.375000	0.625000	0.125000	0.250000	1.000000	0.571429	0.833333
commons-compress-1.15	2	8	2	2	2	1	1	0	6	3	8	3	1	2	1	1	0	6	195	1	2	1	0.030769	0.333333	1.333333	0.333333	0.333333	0.333333	0.166667	0.166667	0.000000	0.500000	1.333333	0.500000	0.166667	0.333333	0.166667	0.166667	0.000000	0.272727	0.750000	0.666667	1.000000
commons-compress-1.16	3	10	3	1	3	3	1	1	5	3	18	3	2	3	4	1	1	6	201	2	3	1	0.029851	0.600000	2.000000	0.600000	0.200000	0.600000	0.600000	0.200000	0.200000	0.500000	3.000000	0.500000	0.333333	0.500000	0.666667	0.166667	0.166667	0.142857	0.600000	0.428571	0.500000
commons-configuration-1.0	0	0	0	0	0	0	0	0	6	0	0	0	0	0	0	0	0	6	29	0	0	0	0.206897	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
commons-configuration-1.1	0	0	0	0	0	0	0	0	12	0	0	0	0	0	0	0	0	9	42	0	0	0	0.214286	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
commons-configuration-1.2	1	0	1	0	1	0	4	2	21	1	0	1	0	1	0	6	0	15	53	1	1	6	0.283019	0.047619	0.000000	0.047619	0.000000	0.047619	0.000000	0.190476	0.095238	0.066667	0.000000	0.066667	0.000000	0.066667	0.000000	0.400000	0.000000	1.000000	1.000000	1.000000	1.000000
commons-configuration-1.3	7	5	7	2	7	5	6	4	35	19	3	16	2	15	2	13	2	24	84	15	15	13	0.285714	0.200000	0.142857	0.200000	0.057143	0.200000	0.142857	0.171429	0.114286	0.791667	0.125000	0.666667	0.083333	0.625000	0.083333	0.541667	0.083333	0.863636	0.888889	0.882353	0.866667
commons-configuration-1.4	8	15	8	5	8	11	10	11	31	11	2	9	1	11	1	11	1	16	91	11	11	11	0.175824	0.258065	0.483871	0.258065	0.161290	0.258065	0.354839	0.322581	0.354839	0.687500	0.125000	0.562500	0.062500	0.687500	0.062500	0.687500	0.062500	0.846154	0.900000	0.916667	0.916667
commons-configuration-1.5	8	2	8	1	8	2	14	6	28	8	2	8	0	8	2	12	3	16	92	8	8	12	0.173913	0.285714	0.071429	0.285714	0.035714	0.285714	0.071429	0.500000	0.214286	0.500000	0.125000	0.500000	0.000000	0.500000	0.125000	0.750000	0.187500	0.800000	1.000000	0.800000	0.800000
commons-configuration-1.6	1	6	1	4	1	7	5	25	15	4	5	3	3	4	3	8	11	10	96	7	4	8	0.104167	0.066667	0.400000	0.066667	0.266667	0.066667	0.466667	0.333333	1.666667	0.400000	0.500000	0.300000	0.300000	0.400000	0.300000	0.800000	1.100000	0.444444	0.500000	0.571429	0.421053
commons-configuration-1.7	2	2	2	2	2	1	3	1	15	3	1	4	1	3	0	4	0	11	114	3	3	4	0.096491	0.133333	0.133333	0.133333	0.133333	0.133333	0.066667	0.200000	0.066667	0.272727	0.090909	0.363636	0.090909	0.272727	0.000000	0.363636	0.000000	0.750000	0.800000	1.000000	1.000000
commons-configuration-1.8	2	0	2	1	2	0	6	1	17	4	4	4	0	4	4	7	6	11	108	8	4	7	0.101852	0.117647	0.000000	0.117647	0.058824	0.117647	0.000000	0.352941	0.058824	0.363636	0.363636	0.363636	0.000000	0.363636	0.363636	0.636364	0.545455	0.500000	1.000000	0.500000	0.538462
commons-configuration-1.9	1	0	2	0	2	0	7	2	16	3	4	4	0	4	4	6	6	11	108	4	4	6	0.101852	0.062500	0.000000	0.125000	0.000000	0.125000	0.000000	0.437500	0.125000	0.272727	0.363636	0.363636	0.000000	0.363636	0.363636	0.545455	0.545455	0.428571	1.000000	0.500000	0.500000
commons-configuration-1.10	0	0	0	1	0	2	1	2	9	0	0	0	1	0	2	1	4	7	108	1	0	1	0.064815	0.000000	0.000000	0.000000	0.111111	0.000000	0.222222	0.111111	0.222222	0.000000	0.000000	0.000000	0.142857	0.000000	0.285714	0.142857	0.571429	NaN	0.000000	0.000000	0.200000
commons-configuration-2.0	0	0	0	2	0	3	0	3	0	0	0	0	2	0	3	0	3	0	231	3	0	0	0.000000	NaN	NaN	NaN	inf	NaN	inf	NaN	inf	NaN	NaN	NaN	inf	NaN	inf	NaN	inf	NaN	0.000000	0.000000	0.000000
commons-configuration-2.1	0	0	0	6	0	5	0	8	0	0	0	0	4	0	6	0	8	0	233	6	0	0	0.000000	NaN	NaN	NaN	inf	NaN	inf	NaN	inf	NaN	NaN	NaN	inf	NaN	inf	NaN	inf	NaN	0.000000	0.000000	0.000000
commons-configuration-2.2	0	4	0	3	0	3	1	4	1	0	4	0	3	0	3	1	5	1	240	3	0	1	0.004167	0.000000	4.000000	0.000000	3.000000	0.000000	3.000000	1.000000	4.000000	0.000000	4.000000	0.000000	3.000000	0.000000	3.000000	1.000000	5.000000	0.000000	0.000000	0.000000	0.166667
commons-dbcp-1.0	0	0	0	0	0	0	0	0	2	0	0	0	0	0	0	0	0	1	32	0	0	0	0.031250	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
commons-dbcp-1.1	0	1	0	0	0	0	0	1	8	0	2	0	0	0	0	1	0	8	39	0	0	1	0.205128	0.000000	0.125000	0.000000	0.000000	0.000000	0.000000	0.000000	0.125000	0.000000	0.250000	0.000000	0.000000	0.000000	0.000000	0.125000	0.000000	0.000000	NaN	NaN	1.000000
commons-dbcp-1.2	1	0	0	0	0	0	6	7	14	0	2	0	0	0	0	9	6	14	39	0	0	9	0.358974	0.071429	0.000000	0.000000	0.000000	0.000000	0.000000	0.428571	0.500000	0.000000	0.142857	0.000000	0.000000	0.000000	0.000000	0.642857	0.428571	0.000000	NaN	NaN	0.600000
commons-dbcp-1.3	0	0	0	0	0	0	6	5	16	0	0	0	0	0	0	8	4	11	53	0	0	8	0.207547	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.375000	0.312500	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.727273	0.363636	NaN	NaN	NaN	0.666667
commons-dbcp-1.4	0	0	0	0	0	0	12	5	16	0	0	0	0	0	0	10	2	11	53	0	0	10	0.207547	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.750000	0.312500	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.909091	0.181818	NaN	NaN	NaN	0.833333
commons-dbcp-2.0	0	1	0	0	0	0	6	1	9	1	0	0	0	0	0	7	2	8	53	0	0	7	0.150943	0.000000	0.111111	0.000000	0.000000	0.000000	0.000000	0.666667	0.111111	0.125000	0.000000	0.000000	0.000000	0.000000	0.000000	0.875000	0.250000	1.000000	NaN	NaN	0.777778
commons-dbcp-2.1	1	0	2	1	2	1	5	4	7	1	0	3	1	3	2	5	3	7	54	5	3	5	0.129630	0.142857	0.000000	0.285714	0.142857	0.285714	0.142857	0.714286	0.571429	0.142857	0.000000	0.428571	0.142857	0.428571	0.285714	0.714286	0.428571	1.000000	0.750000	0.600000	0.625000
commons-dbcp-2.2.0	1	1	1	1	1	3	0	1	2	1	1	2	1	2	3	1	0	2	54	1	2	1	0.037037	0.500000	0.500000	0.500000	0.500000	0.500000	1.500000	0.000000	0.500000	0.500000	0.500000	1.000000	0.500000	1.000000	1.500000	0.500000	0.000000	0.500000	0.666667	0.400000	1.000000
commons-dbcp-2.3.0	0	3	2	2	2	2	1	1	2	0	10	2	9	2	2	2	2	2	55	3	2	2	0.036364	0.000000	1.500000	1.000000	1.000000	1.000000	1.000000	0.500000	0.500000	0.000000	5.000000	1.000000	4.500000	1.000000	1.000000	1.000000	1.000000	0.000000	0.181818	0.500000	0.500000
commons-dbcp-2.4.0	0	2	1	2	1	0	0	1	1	0	11	1	11	1	0	0	1	1	56	0	1	0	0.017857	0.000000	2.000000	1.000000	2.000000	1.000000	0.000000	0.000000	1.000000	0.000000	11.000000	1.000000	11.000000	1.000000	0.000000	0.000000	1.000000	0.000000	0.083333	1.000000	0.000000
commons-dbcp-2.5.0	0	2	1	2	1	0	0	1	1	0	11	1	11	1	0	0	1	1	56	0	1	0	0.017857	0.000000	2.000000	1.000000	2.000000	1.000000	0.000000	0.000000	1.000000	0.000000	11.000000	1.000000	11.000000	1.000000	0.000000	0.000000	1.000000	0.000000	0.083333	1.000000	0.000000
commons-digester-1.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	14	0	0	0	0.000000	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
commons-digester-1.1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	22	0	0	0	0.000000	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
commons-digester-1.2	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	31	0	0	0	0.000000	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
commons-digester-1.3	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	31	0	0	0	0.000000	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
commons-digester-1.4	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	1	33	0	0	0	0.030303	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
commons-digester-1.5	0	0	0	0	0	0	0	0	2	0	0	0	0	0	0	0	0	2	37	0	0	0	0.054054	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
commons-digester-1.6	0	0	0	0	0	0	0	0	2	0	0	0	0	0	0	0	0	2	68	0	0	0	0.029412	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
commons-digester-1.7	0	0	0	0	0	0	0	0	2	0	0	0	0	0	0	0	0	2	69	0	0	0	0.028986	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
commons-digester-1.8	0	0	0	0	0	0	2	2	2	0	0	0	0	0	0	2	2	2	70	0	0	2	0.028571	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	1.000000	1.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	1.000000	1.000000	NaN	NaN	NaN	0.500000
commons-digester-2.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	70	0	0	0	0.000000	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
commons-digester-2.1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	114	0	0	0	0.000000	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
commons-digester-3.0	1	5	1	1	1	2	0	0	2	1	4	1	1	1	3	0	0	7	154	0	1	0	0.045455	0.500000	2.500000	0.500000	0.500000	0.500000	1.000000	0.000000	0.000000	0.142857	0.571429	0.142857	0.142857	0.142857	0.428571	0.000000	0.000000	0.200000	0.500000	0.250000	NaN
commons-digester-3.1	1	7	1	3	1	4	0	0	2	1	6	1	4	1	6	0	0	7	154	0	1	0	0.045455	0.500000	3.500000	0.500000	1.500000	0.500000	2.000000	0.000000	0.000000	0.142857	0.857143	0.142857	0.571429	0.142857	0.857143	0.000000	0.000000	0.142857	0.200000	0.142857	NaN
commons-digester-3.2	1	1	1	1	1	1	3	2	3	3	1	3	1	3	1	10	1	10	157	3	3	10	0.063694	0.333333	0.333333	0.333333	0.333333	0.333333	0.333333	1.000000	0.666667	0.300000	0.100000	0.300000	0.100000	0.300000	0.100000	1.000000	0.100000	0.750000	0.750000	0.750000	0.909091
commons-io-1.0	0	0	0	0	0	0	1	0	7	0	0	0	0	0	0	1	0	5	34	0	0	1	0.147059	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.142857	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.200000	0.000000	NaN	NaN	NaN	1.000000
commons-io-1.1	0	0	0	0	0	0	1	0	14	0	0	0	0	0	0	1	0	7	39	0	0	1	0.179487	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.071429	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.142857	0.000000	NaN	NaN	NaN	1.000000
commons-io-1.2	2	1	2	0	2	0	5	4	17	1	0	1	0	1	0	5	1	8	42	1	1	5	0.190476	0.117647	0.058824	0.117647	0.000000	0.117647	0.000000	0.294118	0.235294	0.125000	0.000000	0.125000	0.000000	0.125000	0.000000	0.625000	0.125000	1.000000	1.000000	1.000000	0.833333
commons-io-1.3	2	1	2	1	2	0	8	2	20	2	0	2	0	2	0	6	0	12	54	2	2	6	0.222222	0.100000	0.050000	0.100000	0.050000	0.100000	0.000000	0.400000	0.100000	0.166667	0.000000	0.166667	0.000000	0.166667	0.000000	0.500000	0.000000	1.000000	1.000000	1.000000	1.000000
commons-io-1.4	2	1	2	1	2	0	12	3	17	4	0	4	0	3	0	8	0	10	72	3	3	8	0.138889	0.117647	0.058824	0.117647	0.058824	0.117647	0.000000	0.705882	0.176471	0.400000	0.000000	0.400000	0.000000	0.300000	0.000000	0.800000	0.000000	1.000000	1.000000	1.000000	1.000000
commons-io-2.0	3	2	3	0	3	1	6	1	14	4	1	3	0	3	1	7	1	8	100	4	3	7	0.080000	0.214286	0.142857	0.214286	0.000000	0.214286	0.071429	0.428571	0.071429	0.500000	0.125000	0.375000	0.000000	0.375000	0.125000	0.875000	0.125000	0.800000	1.000000	0.750000	0.875000
commons-io-2.1	3	7	3	3	3	4	2	1	10	4	3	3	3	4	2	3	0	6	100	3	4	3	0.060000	0.300000	0.700000	0.300000	0.300000	0.300000	0.400000	0.200000	0.100000	0.666667	0.500000	0.500000	0.500000	0.666667	0.333333	0.500000	0.000000	0.571429	0.500000	0.666667	1.000000
commons-io-2.2	0	0	0	0	0	0	0	0	8	0	0	0	0	0	0	0	0	8	102	0	0	0	0.078431	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
commons-io-2.3	0	0	0	0	0	0	0	0	8	0	0	0	0	0	0	0	0	8	103	0	0	0	0.077670	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
commons-io-2.4	0	2	0	1	0	0	4	1	10	1	1	0	7	0	0	3	1	8	103	0	0	3	0.077670	0.000000	0.200000	0.000000	0.100000	0.000000	0.000000	0.400000	0.100000	0.125000	0.125000	0.000000	0.875000	0.000000	0.000000	0.375000	0.125000	0.500000	0.000000	NaN	0.750000
commons-io-2.5	0	1	0	0	0	3	1	4	2	1	1	0	0	1	3	1	4	4	115	3	1	1	0.034783	0.000000	0.500000	0.000000	0.000000	0.000000	1.500000	0.500000	2.000000	0.250000	0.250000	0.000000	0.000000	0.250000	0.750000	0.250000	1.000000	0.500000	NaN	0.250000	0.200000
commons-jcs-1.0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	1	215	0	0	0	0.004651	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
commons-jcs-1.1	0	2	0	0	0	0	0	0	2	1	5	0	0	0	0	0	0	2	213	0	0	0	0.009390	0.000000	1.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.500000	2.500000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.166667	NaN	NaN	NaN
commons-jcs-1.3	0	0	0	0	0	0	0	0	12	0	0	0	0	0	0	0	0	12	275	0	0	0	0.043636	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
commons-jcs-2.0	4	1	4	1	4	0	0	0	7	5	1	4	1	5	0	0	0	12	368	0	5	0	0.032609	0.571429	0.142857	0.571429	0.142857	0.571429	0.000000	0.000000	0.000000	0.416667	0.083333	0.333333	0.083333	0.416667	0.000000	0.000000	0.000000	0.833333	0.800000	1.000000	NaN
commons-jcs-2.1	1	1	1	1	1	0	0	0	5	1	1	1	1	1	0	0	0	11	368	0	1	0	0.029891	0.200000	0.200000	0.200000	0.200000	0.200000	0.000000	0.000000	0.000000	0.090909	0.090909	0.090909	0.090909	0.090909	0.000000	0.000000	0.000000	0.500000	0.500000	1.000000	NaN
commons-jcs-2.2	0	6	0	1	0	1	0	0	1	0	12	0	8	0	1	0	0	4	368	0	0	0	0.010870	0.000000	6.000000	0.000000	1.000000	0.000000	1.000000	0.000000	0.000000	0.000000	3.000000	0.000000	2.000000	0.000000	0.250000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN
commons-jexl-1.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	82	0	0	0	0.000000	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
commons-jexl-1.1	0	0	0	0	0	0	0	3	0	0	0	0	0	0	0	0	16	0	85	0	0	0	0.000000	NaN	NaN	NaN	NaN	NaN	NaN	NaN	inf	NaN	NaN	NaN	NaN	NaN	NaN	NaN	inf	NaN	NaN	NaN	0.000000
commons-jexl-2.0	0	2	0	3	0	3	0	3	1	0	3	0	2	0	3	0	3	5	52	3	0	0	0.096154	0.000000	2.000000	0.000000	3.000000	0.000000	3.000000	0.000000	3.000000	0.000000	0.600000	0.000000	0.400000	0.000000	0.600000	0.000000	0.600000	0.000000	0.000000	0.000000	0.000000
commons-jexl-2.1	0	0	0	0	0	0	0	0	2	0	0	0	0	0	0	0	0	6	69	0	0	0	0.086957	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
commons-jexl-3.0	0	8	0	7	0	6	0	8	0	0	8	0	6	0	8	0	9	0	82	8	0	0	0.000000	NaN	inf	NaN	inf	NaN	inf	NaN	inf	NaN	inf	NaN	inf	NaN	inf	NaN	inf	0.000000	0.000000	0.000000	0.000000
commons-jexl-3.1	0	2	0	2	0	2	0	6	0	0	4	0	2	0	4	0	9	0	85	4	0	0	0.000000	NaN	inf	NaN	inf	NaN	inf	NaN	inf	NaN	inf	NaN	inf	NaN	inf	NaN	inf	0.000000	0.000000	0.000000	0.000000
commons-lang-1.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	26	0	0	0	0.000000	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
commons-lang-2.0	0	0	0	0	0	0	1	0	11	0	0	0	0	0	0	1	0	7	57	0	0	1	0.122807	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.090909	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.142857	0.000000	NaN	NaN	NaN	1.000000
commons-lang-2.1	0	0	0	0	0	0	3	3	20	0	0	0	0	0	0	7	1	13	74	0	0	7	0.175676	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.150000	0.150000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.538462	0.076923	NaN	NaN	NaN	0.875000
commons-lang-2.2	2	1	7	4	7	1	7	0	28	3	0	10	0	9	0	7	0	18	81	6	9	7	0.222222	0.071429	0.035714	0.250000	0.142857	0.250000	0.035714	0.250000	0.000000	0.166667	0.000000	0.555556	0.000000	0.500000	0.000000	0.388889	0.000000	1.000000	1.000000	1.000000	1.000000
commons-lang-2.3	1	0	1	0	1	1	1	5	23	1	0	1	0	2	0	3	3	16	81	2	2	3	0.197531	0.043478	0.000000	0.043478	0.000000	0.043478	0.043478	0.043478	0.217391	0.062500	0.000000	0.062500	0.000000	0.125000	0.000000	0.187500	0.187500	1.000000	1.000000	1.000000	0.500000
commons-lang-2.4	2	2	2	1	3	1	9	10	24	3	0	2	1	3	1	10	1	17	84	4	3	10	0.202381	0.083333	0.083333	0.083333	0.041667	0.125000	0.041667	0.375000	0.416667	0.176471	0.000000	0.117647	0.058824	0.176471	0.058824	0.588235	0.058824	1.000000	0.666667	0.750000	0.909091
commons-lang-2.5	1	1	3	3	3	0	4	1	14	1	1	3	3	3	0	4	1	10	89	3	3	4	0.112360	0.071429	0.071429	0.214286	0.214286	0.214286	0.000000	0.285714	0.071429	0.100000	0.100000	0.300000	0.300000	0.300000	0.000000	0.400000	0.100000	0.500000	0.500000	1.000000	0.800000
commons-lang-2.6	1	3	1	8	1	4	1	0	10	2	1	2	5	2	2	1	0	8	86	0	2	1	0.093023	0.100000	0.300000	0.100000	0.800000	0.100000	0.400000	0.100000	0.000000	0.250000	0.125000	0.250000	0.625000	0.250000	0.250000	0.125000	0.000000	0.666667	0.285714	0.500000	1.000000
commons-lang-3.0	1	4	1	4	1	1	1	0	16	2	2	2	2	1	1	1	0	16	99	1	1	1	0.161616	0.062500	0.250000	0.062500	0.250000	0.062500	0.062500	0.062500	0.000000	0.125000	0.125000	0.125000	0.125000	0.062500	0.062500	0.062500	0.000000	0.500000	0.500000	0.500000	1.000000
commons-lang-3.1	3	3	4	2	4	0	8	4	20	3	2	3	2	3	0	9	0	20	99	3	3	9	0.202020	0.150000	0.150000	0.200000	0.100000	0.200000	0.000000	0.400000	0.200000	0.150000	0.100000	0.150000	0.100000	0.150000	0.000000	0.450000	0.000000	0.600000	0.600000	1.000000	1.000000
commons-lang-3.2	6	4	7	5	7	2	3	1	29	8	3	8	4	8	0	4	0	21	112	4	8	4	0.187500	0.206897	0.137931	0.241379	0.172414	0.241379	0.068966	0.103448	0.034483	0.380952	0.142857	0.380952	0.190476	0.380952	0.000000	0.190476	0.000000	0.727273	0.666667	1.000000	1.000000
commons-lang-3.3	2	3	2	2	2	0	9	1	24	3	2	2	1	2	0	7	1	17	120	2	2	7	0.141667	0.083333	0.125000	0.083333	0.083333	0.083333	0.000000	0.375000	0.041667	0.176471	0.117647	0.117647	0.058824	0.117647	0.000000	0.411765	0.058824	0.600000	0.666667	1.000000	0.875000
commons-lang-3.4	6	9	6	5	7	6	6	3	19	14	100	8	1	8	4	8	4	14	121	12	8	8	0.115702	0.315789	0.473684	0.315789	0.263158	0.368421	0.315789	0.315789	0.157895	1.000000	7.142857	0.571429	0.071429	0.571429	0.285714	0.571429	0.285714	0.122807	0.888889	0.666667	0.666667
commons-lang-3.5	3	11	3	4	3	4	5	4	11	4	6	3	3	4	2	6	2	9	131	6	4	6	0.068702	0.272727	1.000000	0.272727	0.363636	0.272727	0.363636	0.454545	0.363636	0.444444	0.666667	0.333333	0.333333	0.444444	0.222222	0.666667	0.222222	0.400000	0.500000	0.666667	0.750000
commons-lang-3.6	3	9	4	3	4	0	3	0	8	3	5	4	5	4	0	3	0	6	135	3	4	3	0.044444	0.375000	1.125000	0.500000	0.375000	0.500000	0.000000	0.375000	0.000000	0.500000	0.833333	0.666667	0.833333	0.666667	0.000000	0.500000	0.000000	0.375000	0.444444	1.000000	1.000000
commons-lang-3.7	3	8	3	3	3	1	4	1	6	3	6	3	2	3	2	4	2	4	138	5	3	4	0.028986	0.500000	1.333333	0.500000	0.500000	0.500000	0.166667	0.666667	0.166667	0.750000	1.500000	0.750000	0.500000	0.750000	0.500000	1.000000	0.500000	0.333333	0.600000	0.600000	0.666667
commons-math-1.0	0	0	0	0	0	0	0	6	6	0	0	0	0	0	0	2	16	6	106	0	0	2	0.056604	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	1.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.333333	2.666667	NaN	NaN	NaN	0.111111
commons-math-1.1	0	0	0	0	0	1	2	11	10	0	0	0	0	0	1	4	21	7	153	1	0	4	0.045752	0.000000	0.000000	0.000000	0.000000	0.000000	0.100000	0.200000	1.100000	0.000000	0.000000	0.000000	0.000000	0.000000	0.142857	0.571429	3.000000	NaN	NaN	0.000000	0.160000
commons-math-1.2	2	5	4	4	4	2	7	11	16	2	9	4	4	4	3	8	24	11	223	7	4	8	0.049327	0.125000	0.312500	0.250000	0.250000	0.250000	0.125000	0.437500	0.687500	0.181818	0.818182	0.363636	0.363636	0.363636	0.272727	0.727273	2.181818	0.181818	0.500000	0.571429	0.250000
commons-math-2.0	5	16	6	15	6	8	5	7	20	19	45	19	29	17	14	16	22	28	388	23	17	16	0.072165	0.250000	0.800000	0.300000	0.750000	0.300000	0.400000	0.250000	0.350000	0.678571	1.607143	0.678571	1.035714	0.607143	0.500000	0.571429	0.785714	0.296875	0.395833	0.548387	0.421053
commons-math-2.1	5	16	6	18	6	7	10	20	22	8	44	10	44	9	10	14	38	21	408	19	9	14	0.051471	0.227273	0.727273	0.272727	0.818182	0.272727	0.318182	0.454545	0.909091	0.380952	2.095238	0.476190	2.095238	0.428571	0.476190	0.666667	1.809524	0.153846	0.185185	0.473684	0.269231
commons-math-2.2	6	15	6	22	6	15	9	12	17	7	33	7	43	7	18	9	41	14	457	12	7	9	0.030635	0.352941	0.882353	0.352941	1.294118	0.352941	0.882353	0.529412	0.705882	0.500000	2.357143	0.500000	3.071429	0.500000	1.285714	0.642857	2.928571	0.175000	0.140000	0.280000	0.180000
commons-math-3.0	5	15	7	12	7	11	8	9	31	7	15	9	10	8	13	9	10	29	568	15	8	9	0.051056	0.161290	0.483871	0.225806	0.387097	0.225806	0.354839	0.258065	0.290323	0.241379	0.517241	0.310345	0.344828	0.275862	0.448276	0.310345	0.344828	0.318182	0.473684	0.380952	0.473684
commons-math-3.1	9	8	11	7	11	3	8	4	36	19	11	24	9	25	7	11	8	49	693	16	25	11	0.070707	0.250000	0.222222	0.305556	0.194444	0.305556	0.083333	0.222222	0.111111	0.387755	0.224490	0.489796	0.183673	0.510204	0.142857	0.224490	0.163265	0.633333	0.727273	0.781250	0.578947
commons-math-3.2	3	8	4	3	4	4	11	6	28	4	8	4	3	6	2	15	7	33	725	3	6	15	0.045517	0.107143	0.285714	0.142857	0.107143	0.142857	0.142857	0.392857	0.214286	0.121212	0.242424	0.121212	0.090909	0.181818	0.060606	0.454545	0.212121	0.333333	0.571429	0.750000	0.681818
commons-math-3.3	6	6	9	9	9	3	7	5	16	9	5	13	13	12	1	10	8	20	825	11	12	10	0.024242	0.375000	0.375000	0.562500	0.562500	0.562500	0.187500	0.437500	0.312500	0.450000	0.250000	0.650000	0.650000	0.600000	0.050000	0.500000	0.400000	0.642857	0.500000	0.923077	0.555556
commons-math-3.4	4	8	4	11	4	6	1	6	7	5	10	7	27	6	5	2	4	8	852	3	6	2	0.009390	0.571429	1.142857	0.571429	1.571429	0.571429	0.857143	0.142857	0.857143	0.625000	1.250000	0.875000	3.375000	0.750000	0.625000	0.250000	0.500000	0.333333	0.205882	0.545455	0.333333
commons-math-3.5	3	13	3	15	3	11	2	15	5	5	30	5	27	5	21	4	26	6	852	24	5	4	0.007042	0.600000	2.600000	0.600000	3.000000	0.600000	2.200000	0.400000	3.000000	0.833333	5.000000	0.833333	4.500000	0.833333	3.500000	0.666667	4.333333	0.142857	0.156250	0.192308	0.133333
commons-math-3.6	0	1	0	1	0	6	0	7	0	0	22	0	22	0	36	0	37	0	914	36	0	0	0.000000	NaN	inf	NaN	inf	NaN	inf	NaN	inf	NaN	inf	NaN	inf	NaN	inf	NaN	inf	0.000000	0.000000	0.000000	0.000000
commons-net-1.0.0	0	0	0	0	0	0	0	0	7	0	0	0	0	0	0	0	0	7	95	0	0	0	0.073684	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
commons-net-1.1.0	0	0	0	0	0	0	0	0	11	0	0	0	0	0	0	0	0	11	111	0	0	0	0.099099	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
commons-net-1.2.0	0	0	0	0	0	0	0	0	17	0	0	0	0	0	0	0	0	14	120	0	0	0	0.116667	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
commons-net-1.3.0	0	0	0	0	0	0	0	2	22	0	0	0	0	0	0	1	0	16	129	0	0	1	0.124031	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.090909	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.062500	0.000000	NaN	NaN	NaN	1.000000
commons-net-1.4.0	0	0	0	0	0	0	2	7	25	0	0	0	0	0	0	7	4	17	270	0	0	7	0.062963	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.080000	0.280000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.411765	0.235294	NaN	NaN	NaN	0.636364
commons-net-2.0	6	1	6	1	6	0	9	5	35	6	1	6	1	6	0	9	2	25	126	4	6	9	0.198413	0.171429	0.028571	0.171429	0.028571	0.171429	0.000000	0.257143	0.142857	0.240000	0.040000	0.240000	0.040000	0.240000	0.000000	0.360000	0.080000	0.857143	0.857143	1.000000	0.818182
commons-net-2.1	3	2	3	1	3	2	0	0	33	4	2	4	0	4	1	0	0	20	130	0	4	0	0.153846	0.090909	0.060606	0.090909	0.030303	0.090909	0.060606	0.000000	0.000000	0.200000	0.100000	0.200000	0.000000	0.200000	0.050000	0.000000	0.000000	0.666667	1.000000	0.800000	NaN
commons-net-2.2	13	21	13	14	13	7	11	5	29	12	16	12	9	11	8	8	7	18	132	14	11	8	0.136364	0.448276	0.724138	0.448276	0.482759	0.448276	0.241379	0.379310	0.172414	0.666667	0.888889	0.666667	0.500000	0.611111	0.444444	0.444444	0.388889	0.428571	0.571429	0.578947	0.533333
commons-net-3.0	2	2	2	0	2	1	10	2	27	4	0	2	0	4	0	10	0	21	152	4	4	10	0.138158	0.074074	0.074074	0.074074	0.000000	0.074074	0.037037	0.370370	0.074074	0.190476	0.000000	0.095238	0.000000	0.190476	0.000000	0.476190	0.000000	1.000000	1.000000	1.000000	1.000000
commons-net-3.1	3	5	3	1	3	1	2	1	25	3	2	3	0	3	2	2	2	20	153	4	3	2	0.130719	0.120000	0.200000	0.120000	0.040000	0.120000	0.040000	0.080000	0.040000	0.150000	0.100000	0.150000	0.000000	0.150000	0.100000	0.100000	0.100000	0.600000	1.000000	0.600000	0.500000
commons-net-3.2	2	3	2	1	2	1	4	1	20	3	1	1	0	2	0	4	0	18	153	2	2	4	0.117647	0.100000	0.150000	0.100000	0.050000	0.100000	0.050000	0.200000	0.050000	0.166667	0.055556	0.055556	0.000000	0.111111	0.000000	0.222222	0.000000	0.750000	1.000000	1.000000	1.000000
commons-net-3.3	1	4	1	2	1	2	11	5	21	1	2	2	1	2	1	12	4	18	155	3	2	12	0.116129	0.047619	0.190476	0.047619	0.095238	0.047619	0.095238	0.523810	0.238095	0.055556	0.111111	0.111111	0.055556	0.111111	0.055556	0.666667	0.222222	0.333333	0.666667	0.666667	0.750000
commons-net-3.4	0	1	0	0	0	1	4	1	12	1	0	0	0	1	0	5	1	9	156	1	1	5	0.057692	0.000000	0.083333	0.000000	0.000000	0.000000	0.083333	0.333333	0.083333	0.111111	0.000000	0.000000	0.000000	0.111111	0.000000	0.555556	0.111111	1.000000	NaN	1.000000	0.833333
commons-net-3.5	2	1	2	0	2	1	3	2	12	2	1	2	0	2	1	4	2	9	156	2	2	4	0.057692	0.166667	0.083333	0.166667	0.000000	0.166667	0.083333	0.250000	0.166667	0.222222	0.111111	0.222222	0.000000	0.222222	0.111111	0.444444	0.222222	0.666667	1.000000	0.666667	0.666667
commons-net-3.6	3	5	3	2	3	2	2	3	4	2	3	2	1	2	1	2	3	3	156	2	2	2	0.019231	0.750000	1.250000	0.750000	0.500000	0.750000	0.500000	0.500000	0.750000	0.666667	1.000000	0.666667	0.333333	0.666667	0.333333	0.666667	1.000000	0.400000	0.666667	0.666667	0.400000
commons-scxml-0.5	2	1	2	1	2	1	1	0	11	1	1	3	2	1	1	1	0	9	72	1	1	1	0.125000	0.181818	0.090909	0.181818	0.090909	0.181818	0.090909	0.090909	0.000000	0.111111	0.111111	0.333333	0.222222	0.111111	0.111111	0.111111	0.000000	0.500000	0.600000	0.500000	1.000000
commons-scxml-0.6	1	1	1	0	1	0	5	5	15	2	0	1	0	2	0	5	6	11	75	2	2	5	0.146667	0.066667	0.066667	0.066667	0.000000	0.066667	0.000000	0.333333	0.333333	0.181818	0.000000	0.090909	0.000000	0.181818	0.000000	0.454545	0.545455	1.000000	1.000000	1.000000	0.454545
commons-scxml-0.7	0	1	0	2	0	0	1	0	13	1	1	2	1	0	0	1	0	14	79	0	0	1	0.177215	0.000000	0.076923	0.000000	0.153846	0.000000	0.000000	0.076923	0.000000	0.071429	0.071429	0.142857	0.071429	0.000000	0.000000	0.071429	0.000000	0.500000	0.666667	NaN	1.000000
commons-scxml-0.8	0	0	9	0	9	0	12	1	14	0	0	11	0	14	5	14	5	14	79	19	14	14	0.177215	0.000000	0.000000	0.642857	0.000000	0.642857	0.000000	0.857143	0.071429	0.000000	0.000000	0.785714	0.000000	1.000000	0.357143	1.000000	0.357143	NaN	1.000000	0.736842	0.736842
commons-scxml-0.9	0	1	0	1	0	0	2	3	3	0	4	0	4	0	0	2	4	4	79	0	0	2	0.050633	0.000000	0.333333	0.000000	0.333333	0.000000	0.000000	0.666667	1.000000	0.000000	1.000000	0.000000	1.000000	0.000000	0.000000	0.500000	1.000000	0.000000	0.000000	NaN	0.333333
commons-validator-1.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	17	0	0	0	0.000000	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
commons-validator-1.1.0	0	0	0	0	0	0	0	0	3	0	0	0	0	0	0	0	0	3	23	0	0	0	0.130435	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
commons-validator-1.2.0	0	0	0	0	0	0	2	0	7	0	0	0	0	0	0	2	0	7	22	0	0	2	0.318182	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.285714	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.285714	0.000000	NaN	NaN	NaN	1.000000
commons-validator-1.3.0	1	1	1	1	1	0	4	0	7	1	1	1	1	1	0	4	0	7	38	0	1	4	0.184211	0.142857	0.142857	0.142857	0.142857	0.142857	0.000000	0.571429	0.000000	0.142857	0.142857	0.142857	0.142857	0.142857	0.000000	0.571429	0.000000	0.500000	0.500000	1.000000	1.000000
commons-validator-1.4.0	0	0	0	0	0	0	5	5	10	0	0	0	0	0	0	5	2	6	59	0	0	5	0.101695	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.500000	0.500000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.833333	0.333333	NaN	NaN	NaN	0.714286
commons-validator-1.5.0	3	1	3	1	3	0	4	1	7	2	1	2	1	2	0	2	1	3	62	2	2	2	0.048387	0.428571	0.142857	0.428571	0.142857	0.428571	0.000000	0.571429	0.142857	0.666667	0.333333	0.666667	0.333333	0.666667	0.000000	0.666667	0.333333	0.666667	0.666667	1.000000	0.666667
commons-validator-1.6.0	0	0	0	0	0	0	1	0	1	0	0	0	0	0	0	1	0	1	63	0	0	1	0.015873	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	1.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	1.000000	0.000000	NaN	NaN	NaN	1.000000
commons-vfs-1.0	8	6	9	4	9	2	9	3	26	10	3	10	2	10	1	15	5	26	236	10	10	15	0.110169	0.307692	0.230769	0.346154	0.153846	0.346154	0.076923	0.346154	0.115385	0.384615	0.115385	0.384615	0.076923	0.384615	0.038462	0.576923	0.192308	0.769231	0.833333	0.909091	0.750000
commons-vfs-2.0	2	4	2	2	2	3	15	9	21	3	2	3	2	4	3	21	18	25	251	6	4	21	0.099602	0.095238	0.190476	0.095238	0.095238	0.095238	0.142857	0.714286	0.428571	0.120000	0.080000	0.120000	0.080000	0.160000	0.120000	0.840000	0.720000	0.600000	0.600000	0.571429	0.538462
commons-vfs-2.1	0	0	0	0	0	0	0	0	2	0	0	0	0	0	0	0	0	3	259	0	0	0	0.011583	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
commons-vfs-2.2	0	1	1	2	1	1	1	0	2	0	6	1	7	1	1	1	0	2	262	0	1	1	0.007634	0.000000	0.500000	0.500000	1.000000	0.500000	0.500000	0.500000	0.000000	0.000000	3.000000	0.500000	3.500000	0.500000	0.500000	0.500000	0.000000	0.000000	0.125000	0.500000	1.000000
deltaspike-0.1	0	6	0	5	0	1	0	0	2	0	9	0	7	0	3	0	0	2	56	0	0	0	0.035714	0.000000	3.000000	0.000000	2.500000	0.000000	0.500000	0.000000	0.000000	0.000000	4.500000	0.000000	3.500000	0.000000	1.500000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN
deltaspike-0.2	3	16	3	13	3	4	0	0	11	6	54	6	32	5	22	0	0	13	188	0	5	0	0.069149	0.272727	1.454545	0.272727	1.181818	0.272727	0.363636	0.000000	0.000000	0.461538	4.153846	0.461538	2.461538	0.384615	1.692308	0.000000	0.000000	0.100000	0.157895	0.185185	NaN
deltaspike-0.3	4	3	4	3	4	0	0	0	15	8	5	8	4	8	0	0	0	19	181	0	8	0	0.104972	0.266667	0.200000	0.266667	0.200000	0.266667	0.000000	0.000000	0.000000	0.421053	0.263158	0.421053	0.210526	0.421053	0.000000	0.000000	0.000000	0.615385	0.666667	1.000000	NaN
deltaspike-0.4	6	5	6	3	6	1	4	0	19	11	5	10	3	10	2	7	1	25	314	8	10	7	0.079618	0.315789	0.263158	0.315789	0.157895	0.315789	0.052632	0.210526	0.000000	0.440000	0.200000	0.400000	0.120000	0.400000	0.080000	0.280000	0.040000	0.687500	0.769231	0.833333	0.875000
deltaspike-0.5	8	19	8	15	8	10	8	8	27	19	27	18	19	16	13	16	11	32	492	27	16	16	0.065041	0.296296	0.703704	0.296296	0.555556	0.296296	0.370370	0.296296	0.296296	0.593750	0.843750	0.562500	0.593750	0.500000	0.406250	0.500000	0.343750	0.413043	0.486486	0.551724	0.592593
deltaspike-0.6	9	20	9	11	9	10	3	2	29	10	19	9	9	8	9	3	3	28	578	6	8	3	0.048443	0.310345	0.689655	0.310345	0.379310	0.310345	0.344828	0.103448	0.068966	0.357143	0.678571	0.321429	0.321429	0.285714	0.321429	0.107143	0.107143	0.344828	0.500000	0.470588	0.500000
deltaspike-0.7	9	19	9	12	9	10	3	1	27	13	22	12	16	11	13	6	0	29	589	4	11	6	0.049236	0.333333	0.703704	0.333333	0.444444	0.333333	0.370370	0.111111	0.037037	0.448276	0.758621	0.413793	0.551724	0.379310	0.448276	0.206897	0.000000	0.371429	0.428571	0.458333	1.000000
deltaspike-1.0.0	9	15	9	11	9	8	3	1	27	12	20	11	16	10	14	4	2	28	615	4	10	4	0.045528	0.333333	0.555556	0.333333	0.407407	0.333333	0.296296	0.111111	0.037037	0.428571	0.714286	0.392857	0.571429	0.357143	0.500000	0.142857	0.071429	0.375000	0.407407	0.416667	0.666667
deltaspike-1.1.0	11	11	11	6	11	7	4	0	26	12	22	12	11	12	18	3	0	26	634	3	12	3	0.041009	0.423077	0.423077	0.423077	0.230769	0.423077	0.269231	0.153846	0.000000	0.461538	0.846154	0.461538	0.423077	0.461538	0.692308	0.115385	0.000000	0.352941	0.521739	0.400000	1.000000
deltaspike-1.2.0	7	11	7	6	7	7	2	1	22	9	21	9	11	9	18	2	2	24	636	4	9	2	0.037736	0.318182	0.500000	0.318182	0.272727	0.318182	0.318182	0.090909	0.045455	0.375000	0.875000	0.375000	0.458333	0.375000	0.750000	0.083333	0.083333	0.300000	0.450000	0.333333	0.500000
deltaspike-1.3.0	7	11	7	9	7	8	0	2	20	9	8	8	8	9	5	1	1	21	645	2	9	1	0.032558	0.350000	0.550000	0.350000	0.450000	0.350000	0.400000	0.000000	0.100000	0.428571	0.380952	0.380952	0.380952	0.428571	0.238095	0.047619	0.047619	0.529412	0.500000	0.642857	0.500000
deltaspike-1.4.0	9	9	9	8	9	7	2	0	21	11	6	11	6	10	5	2	0	24	649	2	10	2	0.036980	0.428571	0.428571	0.428571	0.380952	0.428571	0.333333	0.095238	0.000000	0.458333	0.250000	0.458333	0.250000	0.416667	0.208333	0.083333	0.000000	0.647059	0.647059	0.666667	1.000000
deltaspike-1.5.0	10	3	10	3	10	0	4	0	21	12	8	12	3	11	5	5	1	24	661	6	11	5	0.036309	0.476190	0.142857	0.476190	0.142857	0.476190	0.000000	0.190476	0.000000	0.500000	0.333333	0.500000	0.125000	0.458333	0.208333	0.208333	0.041667	0.600000	0.800000	0.687500	0.833333
deltaspike-1.6.0	4	8	4	7	4	4	0	0	11	9	5	9	2	8	2	0	0	14	693	0	8	0	0.020202	0.363636	0.727273	0.363636	0.636364	0.363636	0.363636	0.000000	0.000000	0.642857	0.357143	0.642857	0.142857	0.571429	0.142857	0.000000	0.000000	0.642857	0.818182	0.800000	NaN
deltaspike-1.7.0	4	6	4	3	4	2	2	1	11	6	6	6	2	6	1	4	1	12	708	4	6	4	0.016949	0.363636	0.545455	0.363636	0.272727	0.363636	0.181818	0.181818	0.090909	0.500000	0.500000	0.500000	0.166667	0.500000	0.083333	0.333333	0.083333	0.500000	0.750000	0.857143	0.800000
deltaspike-1.8.0	4	4	4	3	4	2	5	2	7	5	5	5	3	5	3	6	3	8	725	8	5	6	0.011034	0.571429	0.571429	0.571429	0.428571	0.571429	0.285714	0.714286	0.285714	0.625000	0.625000	0.625000	0.375000	0.625000	0.375000	0.750000	0.375000	0.500000	0.625000	0.625000	0.666667
eagle-0.3.0	5	36	7	21	7	19	6	18	9	8	174	17	39	17	23	16	23	18	682	39	17	16	0.026393	0.555556	4.000000	0.777778	2.333333	0.777778	2.111111	0.666667	2.000000	0.444444	9.666667	0.944444	2.166667	0.944444	1.277778	0.888889	1.277778	0.043956	0.303571	0.425000	0.410256
eagle-0.4.0	0	0	0	0	0	0	0	0	3	0	0	0	0	0	0	0	0	4	732	0	0	0	0.005464	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
eagle-0.5.0	0	1	0	0	0	1	0	1	0	0	11	0	0	0	2	0	2	0	1388	2	0	0	0.000000	NaN	inf	NaN	NaN	NaN	inf	NaN	inf	NaN	inf	NaN	NaN	NaN	inf	NaN	inf	0.000000	NaN	0.000000	0.000000
giraph-0.1.0	0	15	0	15	0	6	0	0	0	0	20	0	19	0	11	0	0	0	105	0	0	0	0.000000	NaN	inf	NaN	inf	NaN	inf	NaN	NaN	NaN	inf	NaN	inf	NaN	inf	NaN	NaN	0.000000	0.000000	0.000000	NaN
giraph-1.0.0	11	32	11	26	11	7	1	1	26	19	105	18	71	15	9	2	0	27	431	1	15	2	0.062645	0.423077	1.230769	0.423077	1.000000	0.423077	0.269231	0.038462	0.038462	0.703704	3.888889	0.666667	2.629630	0.555556	0.333333	0.074074	0.000000	0.153226	0.202247	0.625000	1.000000
giraph-1.1.0	2	7	2	2	2	2	0	0	7	3	15	3	2	3	3	0	0	15	753	0	3	0	0.019920	0.285714	1.000000	0.285714	0.285714	0.285714	0.285714	0.000000	0.000000	0.200000	1.000000	0.200000	0.133333	0.200000	0.200000	0.000000	0.000000	0.166667	0.600000	0.500000	NaN
gora-0.1	1	0	1	0	1	0	1	0	11	1	0	1	0	1	0	1	0	11	97	1	1	1	0.113402	0.090909	0.000000	0.090909	0.000000	0.090909	0.000000	0.090909	0.000000	0.090909	0.000000	0.090909	0.000000	0.090909	0.000000	0.090909	0.000000	1.000000	1.000000	1.000000	1.000000
gora-0.2	4	4	6	4	6	5	5	6	10	4	3	6	7	6	3	6	3	9	104	7	6	6	0.086538	0.400000	0.400000	0.600000	0.400000	0.600000	0.500000	0.500000	0.600000	0.444444	0.333333	0.666667	0.777778	0.666667	0.333333	0.666667	0.333333	0.571429	0.461538	0.666667	0.666667
gora-0.3	0	6	0	6	0	3	1	2	4	3	17	3	12	2	13	2	2	5	132	3	2	2	0.037879	0.000000	1.500000	0.000000	1.500000	0.000000	0.750000	0.250000	0.500000	0.600000	3.400000	0.600000	2.400000	0.400000	2.600000	0.400000	0.400000	0.150000	0.200000	0.133333	0.500000
gora-0.4	3	6	3	2	3	3	2	2	11	5	7	4	0	4	3	3	2	11	153	5	4	3	0.071895	0.272727	0.545455	0.272727	0.181818	0.272727	0.272727	0.181818	0.181818	0.454545	0.636364	0.363636	0.000000	0.363636	0.272727	0.272727	0.181818	0.416667	1.000000	0.571429	0.600000
gora-0.5	3	10	3	1	3	1	2	1	12	5	5	5	2	5	1	4	1	13	169	5	5	4	0.076923	0.250000	0.833333	0.250000	0.083333	0.250000	0.083333	0.166667	0.083333	0.384615	0.384615	0.384615	0.153846	0.384615	0.076923	0.307692	0.076923	0.500000	0.714286	0.833333	0.800000
gora-0.6	1	8	1	1	1	0	1	0	9	1	3	1	1	1	0	1	0	9	170	0	1	1	0.052941	0.111111	0.888889	0.111111	0.111111	0.111111	0.000000	0.111111	0.000000	0.111111	0.333333	0.111111	0.111111	0.111111	0.000000	0.111111	0.000000	0.250000	0.500000	1.000000	1.000000
gora-0.7	0	2	0	2	0	1	0	0	0	0	2	0	2	0	1	0	0	0	201	0	0	0	0.000000	NaN	inf	NaN	inf	NaN	inf	NaN	NaN	NaN	inf	NaN	inf	NaN	inf	NaN	NaN	0.000000	0.000000	0.000000	NaN
gora-0.8	0	0	0	0	0	1	0	0	0	0	0	0	0	0	61	0	0	0	210	0	0	0	0.000000	NaN	NaN	NaN	NaN	NaN	inf	NaN	NaN	NaN	NaN	NaN	NaN	NaN	inf	NaN	NaN	NaN	NaN	0.000000	NaN
jspwiki-1.4.0	0	7	0	0	0	0	0	0	1	0	1	0	0	0	0	0	0	1	13	0	0	0	0.076923	0.000000	7.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	1.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN
jspwiki-1.5.0	0	7	0	0	0	0	0	0	1	0	1	0	0	0	0	0	0	1	13	0	0	0	0.076923	0.000000	7.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	1.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN
jspwiki-1.6.0	0	7	0	0	0	0	0	0	1	0	1	0	0	0	0	0	0	1	14	0	0	0	0.071429	0.000000	7.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	1.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN
jspwiki-1.7.0	0	5	0	0	0	0	0	0	1	0	1	0	0	0	0	0	0	1	34	0	0	0	0.029412	0.000000	5.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	1.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN
jspwiki-1.8.0	0	2	0	0	0	0	0	0	1	0	1	0	0	0	0	0	0	1	39	0	0	0	0.025641	0.000000	2.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	1.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN
jspwiki-2.0.36	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	1	103	0	0	0	0.009709	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
jspwiki-2.2.19	0	5	0	0	0	0	0	0	1	0	10	0	0	0	0	0	0	1	218	0	0	0	0.004587	0.000000	5.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	10.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN
jspwiki-2.4.56	0	6	0	0	0	0	0	0	8	0	8	0	0	0	0	0	0	9	307	0	0	0	0.029316	0.000000	0.750000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.888889	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN
jspwiki-2.6.0	7	16	8	24	8	26	3	16	16	14	116	9	9	9	10	4	10	17	364	12	9	4	0.046703	0.437500	1.000000	0.500000	1.500000	0.500000	1.625000	0.187500	1.000000	0.823529	6.823529	0.529412	0.529412	0.529412	0.588235	0.235294	0.588235	0.107692	0.500000	0.473684	0.285714
jspwiki-2.8.0	1	7	4	19	4	11	3	8	5	2	4	4	20	4	17	3	15	4	730	18	4	3	0.005479	0.200000	1.400000	0.800000	3.800000	0.800000	2.200000	0.600000	1.600000	0.500000	1.000000	1.000000	5.000000	1.000000	4.250000	0.750000	3.750000	0.333333	0.166667	0.190476	0.166667
jspwiki-2.9.0	0	2	1	10	1	7	0	14	3	0	4	3	10	3	7	0	2	4	372	2	3	0	0.010753	0.000000	0.666667	0.333333	3.333333	0.333333	2.333333	0.000000	4.666667	0.000000	1.000000	0.750000	2.500000	0.750000	1.750000	0.000000	0.500000	0.000000	0.230769	0.300000	0.000000
jspwiki-2.10.0	0	2	2	15	2	25	2	5	5	0	2	3	13	3	10	3	3	6	351	6	3	3	0.017094	0.000000	0.400000	0.400000	3.000000	0.400000	5.000000	0.400000	1.000000	0.000000	0.333333	0.500000	2.166667	0.500000	1.666667	0.500000	0.500000	0.000000	0.187500	0.230769	0.500000
knox-0.3.0	10	20	10	14	10	1	4	0	27	17	27	16	15	14	4	4	0	39	388	4	14	4	0.100515	0.370370	0.740741	0.370370	0.518519	0.370370	0.037037	0.148148	0.000000	0.435897	0.692308	0.410256	0.384615	0.358974	0.102564	0.102564	0.000000	0.386364	0.516129	0.777778	1.000000
knox-0.4.0	6	9	6	3	6	2	5	2	23	24	10	24	4	22	5	23	18	34	444	22	22	23	0.076577	0.260870	0.391304	0.260870	0.130435	0.260870	0.086957	0.217391	0.086957	0.705882	0.294118	0.705882	0.117647	0.647059	0.147059	0.676471	0.529412	0.705882	0.857143	0.814815	0.560976
knox-0.5.0	6	6	6	5	6	2	4	1	21	9	5	8	4	8	1	4	1	47	496	2	8	4	0.094758	0.285714	0.285714	0.285714	0.238095	0.285714	0.095238	0.190476	0.047619	0.191489	0.106383	0.170213	0.085106	0.170213	0.021277	0.085106	0.021277	0.642857	0.666667	0.888889	0.800000
knox-0.6.0	9	19	9	13	9	4	9	3	25	29	24	29	10	27	10	8	5	67	542	6	27	8	0.123616	0.360000	0.760000	0.360000	0.520000	0.360000	0.160000	0.360000	0.120000	0.432836	0.358209	0.432836	0.149254	0.402985	0.149254	0.119403	0.074627	0.547170	0.743590	0.729730	0.615385
knox-0.7.0	8	19	8	18	8	4	2	0	27	21	59	21	42	18	7	2	2	32	590	2	18	2	0.054237	0.296296	0.703704	0.296296	0.666667	0.296296	0.148148	0.074074	0.000000	0.656250	1.843750	0.656250	1.312500	0.562500	0.218750	0.062500	0.062500	0.262500	0.333333	0.720000	0.500000
knox-0.8.0	8	14	8	14	8	2	0	0	28	20	56	20	39	17	5	0	0	33	594	0	17	0	0.055556	0.285714	0.500000	0.285714	0.500000	0.285714	0.071429	0.000000	0.000000	0.606061	1.696970	0.606061	1.181818	0.515152	0.151515	0.000000	0.000000	0.263158	0.338983	0.772727	NaN
knox-0.9.0	3	10	3	10	3	1	2	1	24	4	35	4	29	3	1	2	2	25	609	1	3	2	0.041051	0.125000	0.416667	0.125000	0.416667	0.125000	0.041667	0.083333	0.041667	0.160000	1.400000	0.160000	1.160000	0.120000	0.040000	0.080000	0.080000	0.102564	0.121212	0.750000	0.500000
knox-0.10.0	13	14	13	10	13	4	1	0	24	19	39	18	25	18	13	1	0	27	622	1	18	1	0.043408	0.541667	0.583333	0.541667	0.416667	0.541667	0.166667	0.041667	0.000000	0.703704	1.444444	0.666667	0.925926	0.666667	0.481481	0.037037	0.000000	0.327586	0.418605	0.580645	1.000000
knox-0.11.0	10	19	10	16	10	5	5	1	22	18	29	17	12	15	11	7	3	26	650	9	15	7	0.040000	0.454545	0.863636	0.454545	0.727273	0.454545	0.227273	0.227273	0.045455	0.692308	1.115385	0.653846	0.461538	0.576923	0.423077	0.269231	0.115385	0.382979	0.586207	0.576923	0.700000
knox-0.12.0	10	16	10	13	10	3	1	0	19	16	35	16	15	16	11	1	0	22	672	1	16	1	0.032738	0.526316	0.842105	0.526316	0.684211	0.526316	0.157895	0.052632	0.000000	0.727273	1.590909	0.727273	0.681818	0.727273	0.500000	0.045455	0.000000	0.313725	0.516129	0.592593	1.000000
knox-0.13.0	1	0	1	0	1	0	0	0	11	2	0	2	0	2	0	0	0	12	685	0	2	0	0.017518	0.090909	0.000000	0.090909	0.000000	0.090909	0.000000	0.000000	0.000000	0.166667	0.000000	0.166667	0.000000	0.166667	0.000000	0.000000	0.000000	1.000000	1.000000	1.000000	NaN
knox-0.14.0	5	26	5	21	5	6	4	4	8	6	15	6	14	6	5	5	4	8	740	8	6	5	0.010811	0.625000	3.250000	0.625000	2.625000	0.625000	0.750000	0.500000	0.500000	0.750000	1.875000	0.750000	1.750000	0.750000	0.625000	0.625000	0.500000	0.285714	0.300000	0.545455	0.555556
knox-1.0.0	15	47	15	42	15	13	3	2	16	23	50	23	33	23	12	4	1	24	763	5	23	4	0.031455	0.937500	2.937500	0.937500	2.625000	0.937500	0.812500	0.187500	0.125000	0.958333	2.083333	0.958333	1.375000	0.958333	0.500000	0.166667	0.041667	0.315068	0.410714	0.657143	0.800000
kylin-0.6.1	0	0	0	0	0	0	0	0	13	0	0	0	0	0	0	0	0	11	379	0	0	0	0.029024	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
kylin-0.7.1	3	6	3	5	3	2	3	2	10	3	19	3	10	3	6	3	6	11	492	9	3	3	0.022358	0.300000	0.600000	0.300000	0.500000	0.300000	0.200000	0.300000	0.200000	0.272727	1.727273	0.272727	0.909091	0.272727	0.545455	0.272727	0.545455	0.136364	0.230769	0.333333	0.333333
kylin-1.0	1	5	1	4	1	2	0	0	7	3	17	3	15	3	7	0	0	10	508	0	3	0	0.019685	0.142857	0.714286	0.142857	0.571429	0.142857	0.285714	0.000000	0.000000	0.300000	1.700000	0.300000	1.500000	0.300000	0.700000	0.000000	0.000000	0.150000	0.166667	0.300000	NaN
kylin-1.1	1	15	1	13	1	9	0	3	4	2	25	2	22	2	11	0	4	6	507	4	2	0	0.011834	0.250000	3.750000	0.250000	3.250000	0.250000	2.250000	0.000000	0.750000	0.333333	4.166667	0.333333	3.666667	0.333333	1.833333	0.000000	0.666667	0.074074	0.083333	0.153846	0.000000
kylin-1.2	4	40	4	24	4	28	3	8	4	6	504	6	22	6	29	4	12	6	517	15	6	4	0.011605	1.000000	10.000000	1.000000	6.000000	1.000000	7.000000	0.750000	2.000000	1.000000	84.000000	1.000000	3.666667	1.000000	4.833333	0.666667	2.000000	0.011765	0.214286	0.171429	0.250000
kylin-1.3	1	48	1	32	1	33	0	2	1	2	78	2	33	2	34	0	2	2	533	2	2	0	0.003752	1.000000	48.000000	1.000000	32.000000	1.000000	33.000000	0.000000	2.000000	1.000000	39.000000	1.000000	16.500000	1.000000	17.000000	0.000000	1.000000	0.025000	0.057143	0.055556	0.000000
kylin-1.5.0	30	57	30	39	30	33	8	5	45	40	84	40	59	39	37	12	8	49	744	16	39	12	0.065860	0.666667	1.266667	0.666667	0.866667	0.666667	0.733333	0.177778	0.111111	0.816327	1.714286	0.816327	1.204082	0.795918	0.755102	0.244898	0.163265	0.322581	0.404040	0.513158	0.600000
kylin-1.6.0	16	66	16	60	16	40	5	13	29	24	125	24	97	22	84	9	28	31	793	35	22	9	0.039092	0.551724	2.275862	0.551724	2.068966	0.551724	1.379310	0.172414	0.448276	0.774194	4.032258	0.774194	3.129032	0.709677	2.709677	0.290323	0.903226	0.161074	0.198347	0.207547	0.243243
kylin-2.0.0	18	48	19	36	19	34	9	8	32	26	63	25	54	24	47	13	11	35	859	24	24	13	0.040745	0.562500	1.500000	0.593750	1.125000	0.593750	1.062500	0.281250	0.250000	0.742857	1.800000	0.714286	1.542857	0.685714	1.342857	0.371429	0.314286	0.292135	0.316456	0.338028	0.541667
kylin-2.1.0	27	56	29	45	29	40	9	9	39	37	73	38	61	37	51	15	13	51	939	25	37	15	0.054313	0.692308	1.435897	0.743590	1.153846	0.743590	1.025641	0.230769	0.230769	0.725490	1.431373	0.745098	1.196078	0.725490	1.000000	0.294118	0.254902	0.336364	0.383838	0.420455	0.535714
kylin-2.2.0	27	43	30	32	30	23	10	5	36	33	62	36	46	34	34	15	3	43	1006	15	34	15	0.042744	0.750000	1.194444	0.833333	0.888889	0.833333	0.638889	0.277778	0.138889	0.767442	1.441860	0.837209	1.069767	0.790698	0.790698	0.348837	0.069767	0.347368	0.439024	0.500000	0.833333
lens-2.6.0	3	5	3	4	3	5	1	0	10	5	11	5	6	5	11	2	0	12	584	2	5	2	0.020548	0.300000	0.500000	0.300000	0.400000	0.300000	0.500000	0.100000	0.000000	0.416667	0.916667	0.416667	0.500000	0.416667	0.916667	0.166667	0.000000	0.312500	0.454545	0.312500	1.000000
lens-2.7.0	0	2	0	0	0	0	0	0	3	1	2	0	0	0	0	0	0	4	629	0	0	0	0.006359	0.000000	0.666667	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.250000	0.500000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.333333	NaN	NaN	NaN
mahout-0.1	1	2	1	2	1	0	1	1	5	25	5	25	3	25	1	25	2	28	264	26	25	25	0.106061	0.200000	0.400000	0.200000	0.400000	0.200000	0.000000	0.200000	0.200000	0.892857	0.178571	0.892857	0.107143	0.892857	0.035714	0.892857	0.071429	0.833333	0.892857	0.961538	0.925926
mahout-0.2	6	24	6	22	6	18	4	11	12	20	84	20	53	20	32	13	21	25	425	34	20	13	0.058824	0.500000	2.000000	0.500000	1.833333	0.500000	1.500000	0.333333	0.916667	0.800000	3.360000	0.800000	2.120000	0.800000	1.280000	0.520000	0.840000	0.192308	0.273973	0.384615	0.382353
mahout-0.3	11	35	11	26	11	9	4	4	24	21	151	20	105	18	13	4	10	29	699	8	18	4	0.041488	0.458333	1.458333	0.458333	1.083333	0.458333	0.375000	0.166667	0.166667	0.724138	5.206897	0.689655	3.620690	0.620690	0.448276	0.137931	0.344828	0.122093	0.160000	0.580645	0.285714
mahout-0.4	17	19	17	13	17	6	12	4	40	63	85	61	37	57	24	53	8	73	765	61	57	53	0.095425	0.425000	0.475000	0.425000	0.325000	0.425000	0.150000	0.300000	0.100000	0.863014	1.164384	0.835616	0.506849	0.780822	0.328767	0.726027	0.109589	0.425676	0.622449	0.703704	0.868852
mahout-0.5	13	24	14	17	14	10	10	5	34	29	55	31	25	29	19	27	26	49	778	34	29	27	0.062982	0.382353	0.705882	0.411765	0.500000	0.411765	0.294118	0.294118	0.147059	0.591837	1.122449	0.632653	0.510204	0.591837	0.387755	0.551020	0.530612	0.345238	0.553571	0.604167	0.509434
mahout-0.6	5	13	5	7	5	5	4	1	31	28	66	28	46	28	19	27	4	52	895	31	28	27	0.058101	0.161290	0.419355	0.161290	0.225806	0.161290	0.161290	0.129032	0.032258	0.538462	1.269231	0.538462	0.884615	0.538462	0.365385	0.519231	0.076923	0.297872	0.378378	0.595745	0.870968
mahout-0.7	11	11	11	8	11	6	19	12	39	16	20	13	17	12	12	25	23	38	836	15	12	25	0.045455	0.282051	0.282051	0.282051	0.205128	0.282051	0.153846	0.487179	0.307692	0.421053	0.526316	0.342105	0.447368	0.315789	0.315789	0.657895	0.605263	0.444444	0.433333	0.500000	0.520833
mahout-0.8	15	43	15	9	15	3	12	2	27	23	60	22	17	22	7	18	6	33	906	24	22	18	0.036424	0.555556	1.592593	0.555556	0.333333	0.555556	0.111111	0.444444	0.074074	0.696970	1.818182	0.666667	0.515152	0.666667	0.212121	0.545455	0.181818	0.277108	0.564103	0.758621	0.750000
mahout-0.9	8	20	8	11	8	1	5	4	14	11	44	10	40	10	2	8	3	17	836	9	10	8	0.020335	0.571429	1.428571	0.571429	0.785714	0.571429	0.071429	0.357143	0.285714	0.647059	2.588235	0.588235	2.352941	0.588235	0.117647	0.470588	0.176471	0.200000	0.200000	0.833333	0.727273
mahout-0.10.0	3	12	3	1	3	19	3	19	7	5	23	5	1	5	23	5	23	9	823	28	5	5	0.010936	0.428571	1.714286	0.428571	0.142857	0.428571	2.714286	0.428571	2.714286	0.555556	2.555556	0.555556	0.111111	0.555556	2.555556	0.555556	2.555556	0.178571	0.833333	0.178571	0.178571
mahout-0.11.0	3	1	3	1	3	1	1	0	4	3	1	3	1	3	1	1	0	4	829	1	3	1	0.004825	0.750000	0.250000	0.750000	0.250000	0.750000	0.250000	0.250000	0.000000	0.750000	0.250000	0.750000	0.250000	0.750000	0.250000	0.250000	0.000000	0.750000	0.750000	0.750000	1.000000
mahout-0.12.0	1	1	1	0	1	0	1	0	1	1	1	1	0	1	0	1	0	1	810	1	1	1	0.001235	1.000000	1.000000	1.000000	0.000000	1.000000	0.000000	1.000000	0.000000	1.000000	1.000000	1.000000	0.000000	1.000000	0.000000	1.000000	0.000000	0.500000	1.000000	1.000000	1.000000
mahout-0.13.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	814	0	0	0	0.000000	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
manifoldcf-0.1	5	11	5	10	5	5	4	5	27	11	34	11	21	8	21	7	23	28	430	28	8	7	0.065116	0.185185	0.407407	0.185185	0.370370	0.185185	0.185185	0.148148	0.185185	0.392857	1.214286	0.392857	0.750000	0.285714	0.750000	0.250000	0.821429	0.244444	0.343750	0.275862	0.233333
manifoldcf-0.2	3	18	3	13	3	6	0	1	20	9	32	7	27	5	7	0	2	25	433	0	5	0	0.057737	0.150000	0.900000	0.150000	0.650000	0.150000	0.300000	0.000000	0.050000	0.360000	1.280000	0.280000	1.080000	0.200000	0.280000	0.000000	0.080000	0.219512	0.205882	0.416667	0.000000
manifoldcf-0.3	4	19	4	17	4	12	0	1	24	11	33	9	28	7	9	0	2	28	492	0	7	0	0.056911	0.166667	0.791667	0.166667	0.708333	0.166667	0.500000	0.000000	0.041667	0.392857	1.178571	0.321429	1.000000	0.250000	0.321429	0.000000	0.071429	0.250000	0.243243	0.437500	0.000000
manifoldcf-0.4	2	6	4	4	4	2	3	0	32	9	9	11	8	7	5	4	5	32	513	9	7	4	0.062378	0.062500	0.187500	0.125000	0.125000	0.125000	0.062500	0.093750	0.000000	0.281250	0.281250	0.343750	0.250000	0.218750	0.156250	0.125000	0.156250	0.500000	0.578947	0.583333	0.444444
manifoldcf-0.5	2	9	2	8	2	4	2	1	34	16	23	9	13	8	5	4	0	34	546	4	8	4	0.062271	0.058824	0.264706	0.058824	0.235294	0.058824	0.117647	0.058824	0.029412	0.470588	0.676471	0.264706	0.382353	0.235294	0.147059	0.117647	0.000000	0.410256	0.409091	0.615385	1.000000
manifoldcf-0.6	3	4	3	3	3	1	2	0	37	19	13	8	1	11	1	9	1	65	632	10	11	9	0.102848	0.081081	0.108108	0.081081	0.081081	0.081081	0.027027	0.054054	0.000000	0.292308	0.200000	0.123077	0.015385	0.169231	0.015385	0.138462	0.015385	0.593750	0.888889	0.916667	0.900000
manifoldcf-1.0	8	17	9	9	9	7	9	2	33	15	21	12	12	11	9	11	2	56	640	13	11	11	0.087500	0.242424	0.515152	0.272727	0.272727	0.272727	0.212121	0.272727	0.060606	0.267857	0.375000	0.214286	0.214286	0.196429	0.160714	0.196429	0.035714	0.416667	0.500000	0.550000	0.846154
manifoldcf-1.1	6	17	6	17	6	11	0	2	28	13	16	12	16	12	13	2	3	55	647	5	12	2	0.085008	0.214286	0.607143	0.214286	0.607143	0.214286	0.392857	0.000000	0.071429	0.236364	0.290909	0.218182	0.290909	0.218182	0.236364	0.036364	0.054545	0.448276	0.428571	0.480000	0.400000
manifoldcf-1.2	4	9	4	10	4	2	3	0	32	7	6	7	11	6	2	4	1	57	688	5	6	4	0.082849	0.125000	0.281250	0.125000	0.312500	0.125000	0.062500	0.093750	0.000000	0.122807	0.105263	0.122807	0.192982	0.105263	0.035088	0.070175	0.017544	0.538462	0.388889	0.750000	0.800000
manifoldcf-1.3	4	2	4	1	4	1	4	1	35	31	0	30	0	30	0	30	0	58	771	30	30	30	0.075227	0.114286	0.057143	0.114286	0.028571	0.114286	0.028571	0.114286	0.028571	0.534483	0.000000	0.517241	0.000000	0.517241	0.000000	0.517241	0.000000	1.000000	1.000000	1.000000	1.000000
manifoldcf-1.4	5	4	5	2	5	3	5	1	37	11	31	8	2	11	29	7	4	43	780	10	11	7	0.055128	0.135135	0.108108	0.135135	0.054054	0.135135	0.081081	0.135135	0.027027	0.255814	0.720930	0.186047	0.046512	0.255814	0.674419	0.162791	0.093023	0.261905	0.800000	0.275000	0.636364
manifoldcf-1.5	3	5	3	4	3	2	3	2	32	4	2	4	2	3	2	3	2	31	862	5	3	3	0.035963	0.093750	0.156250	0.093750	0.125000	0.093750	0.062500	0.093750	0.062500	0.129032	0.064516	0.129032	0.064516	0.096774	0.064516	0.096774	0.064516	0.666667	0.666667	0.600000	0.600000
manifoldcf-1.6	5	21	5	14	5	12	2	3	31	9	17	7	12	6	13	3	1	32	863	4	6	3	0.037080	0.161290	0.677419	0.161290	0.451613	0.161290	0.387097	0.064516	0.096774	0.281250	0.531250	0.218750	0.375000	0.187500	0.406250	0.093750	0.031250	0.346154	0.368421	0.315789	0.750000
manifoldcf-1.7	16	29	16	22	16	13	8	6	39	23	45	22	27	21	21	11	8	38	928	19	21	11	0.040948	0.410256	0.743590	0.410256	0.564103	0.410256	0.333333	0.205128	0.153846	0.605263	1.184211	0.578947	0.710526	0.552632	0.552632	0.289474	0.210526	0.338235	0.448980	0.500000	0.578947
manifoldcf-1.8	6	18	6	13	6	13	4	8	27	10	35	9	19	9	30	7	25	26	942	32	9	7	0.027601	0.222222	0.666667	0.222222	0.481481	0.222222	0.481481	0.148148	0.296296	0.384615	1.346154	0.346154	0.730769	0.346154	1.153846	0.269231	0.961538	0.222222	0.321429	0.230769	0.218750
manifoldcf-1.9	3	8	3	8	3	6	2	2	22	5	12	5	8	5	8	2	3	22	978	5	5	2	0.022495	0.136364	0.363636	0.136364	0.363636	0.136364	0.272727	0.090909	0.090909	0.227273	0.545455	0.227273	0.363636	0.227273	0.363636	0.090909	0.136364	0.294118	0.384615	0.384615	0.400000
manifoldcf-1.10	0	0	0	0	0	0	0	0	19	0	0	0	0	0	0	0	0	19	999	0	0	0	0.019019	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
manifoldcf-2.0	7	17	7	12	7	12	6	7	40	12	33	12	16	11	28	11	24	35	933	33	11	11	0.037513	0.175000	0.425000	0.175000	0.300000	0.175000	0.300000	0.150000	0.175000	0.342857	0.942857	0.342857	0.457143	0.314286	0.800000	0.314286	0.685714	0.266667	0.428571	0.282051	0.314286
manifoldcf-2.1	4	8	4	7	4	5	3	4	40	7	11	7	6	5	8	3	4	34	969	7	5	3	0.035088	0.100000	0.200000	0.100000	0.175000	0.100000	0.125000	0.075000	0.100000	0.205882	0.323529	0.205882	0.176471	0.147059	0.235294	0.088235	0.117647	0.388889	0.538462	0.384615	0.428571
manifoldcf-2.2	9	9	9	7	9	5	2	2	39	14	16	13	13	11	6	2	2	32	995	4	11	2	0.032161	0.230769	0.230769	0.230769	0.179487	0.230769	0.128205	0.051282	0.051282	0.437500	0.500000	0.406250	0.406250	0.343750	0.187500	0.062500	0.062500	0.466667	0.500000	0.647059	0.500000
manifoldcf-2.3	10	8	10	4	10	4	4	2	42	13	5	12	2	13	2	7	2	37	1017	9	13	7	0.036382	0.238095	0.190476	0.238095	0.095238	0.238095	0.095238	0.095238	0.047619	0.351351	0.135135	0.324324	0.054054	0.351351	0.054054	0.189189	0.054054	0.722222	0.857143	0.866667	0.777778
manifoldcf-2.4	9	6	9	4	9	3	6	2	37	9	2	8	2	9	2	7	1	32	1020	7	9	7	0.031373	0.243243	0.162162	0.243243	0.108108	0.243243	0.081081	0.162162	0.054054	0.281250	0.062500	0.250000	0.062500	0.281250	0.062500	0.218750	0.031250	0.818182	0.800000	0.818182	0.875000
manifoldcf-2.5	9	5	9	2	9	1	6	0	33	10	2	10	1	10	1	6	0	28	1021	6	10	6	0.027424	0.272727	0.151515	0.272727	0.060606	0.272727	0.030303	0.181818	0.000000	0.357143	0.071429	0.357143	0.035714	0.357143	0.035714	0.214286	0.000000	0.833333	0.909091	0.909091	1.000000
manifoldcf-2.6	10	1	10	1	10	0	6	0	30	13	0	13	0	13	0	9	0	26	1026	9	13	9	0.025341	0.333333	0.033333	0.333333	0.033333	0.333333	0.000000	0.200000	0.000000	0.500000	0.000000	0.500000	0.000000	0.500000	0.000000	0.346154	0.000000	1.000000	1.000000	1.000000	1.000000
manifoldcf-2.7	7	4	7	4	7	2	4	1	22	8	15	8	13	7	11	3	1	15	1026	4	7	3	0.014620	0.318182	0.181818	0.318182	0.181818	0.318182	0.090909	0.181818	0.045455	0.533333	1.000000	0.533333	0.866667	0.466667	0.733333	0.200000	0.066667	0.347826	0.380952	0.388889	0.750000
manifoldcf-2.8	7	3	7	1	7	3	4	2	18	18	2	18	1	18	2	16	2	24	1041	16	18	16	0.023055	0.388889	0.166667	0.388889	0.055556	0.388889	0.166667	0.222222	0.111111	0.750000	0.083333	0.750000	0.041667	0.750000	0.083333	0.666667	0.083333	0.900000	0.947368	0.900000	0.888889
manifoldcf-2.9	4	2	4	3	4	1	4	0	14	6	0	6	0	5	0	5	0	13	1047	4	5	5	0.012416	0.285714	0.142857	0.285714	0.214286	0.285714	0.071429	0.285714	0.000000	0.461538	0.000000	0.461538	0.000000	0.384615	0.000000	0.384615	0.000000	1.000000	1.000000	1.000000	1.000000
manifoldcf-2.10	7	5	7	5	7	1	4	1	12	6	4	6	2	6	2	4	2	11	1058	6	6	4	0.010397	0.583333	0.416667	0.583333	0.416667	0.583333	0.083333	0.333333	0.083333	0.545455	0.363636	0.545455	0.181818	0.545455	0.181818	0.363636	0.181818	0.600000	0.750000	0.750000	0.666667
nutch-0.7	3	8	3	7	3	6	2	2	11	7	22	7	24	6	9	8	2	14	327	4	6	8	0.042813	0.272727	0.727273	0.272727	0.636364	0.272727	0.545455	0.181818	0.181818	0.500000	1.571429	0.500000	1.714286	0.428571	0.642857	0.571429	0.142857	0.241379	0.225806	0.400000	0.800000
nutch-0.8	10	18	11	15	11	13	7	9	32	17	43	14	13	16	14	13	12	28	319	23	16	13	0.087774	0.312500	0.562500	0.343750	0.468750	0.343750	0.406250	0.218750	0.281250	0.607143	1.535714	0.500000	0.464286	0.571429	0.500000	0.464286	0.428571	0.283333	0.518519	0.533333	0.520000
nutch-0.9	8	14	8	12	8	6	6	2	29	11	21	11	13	10	9	6	3	26	332	5	10	6	0.078313	0.275862	0.482759	0.275862	0.413793	0.275862	0.206897	0.206897	0.068966	0.423077	0.807692	0.423077	0.500000	0.384615	0.346154	0.230769	0.115385	0.343750	0.458333	0.526316	0.666667
nutch-1.0	1	0	1	0	1	0	10	3	39	2	0	2	0	2	0	12	1	36	358	2	2	12	0.100559	0.025641	0.000000	0.025641	0.000000	0.025641	0.000000	0.256410	0.076923	0.055556	0.000000	0.055556	0.000000	0.055556	0.000000	0.333333	0.027778	1.000000	1.000000	1.000000	0.923077
nutch-1.1	5	12	5	8	5	3	3	2	37	19	93	9	12	6	5	4	4	31	364	4	6	4	0.085165	0.135135	0.324324	0.135135	0.216216	0.135135	0.081081	0.081081	0.054054	0.612903	3.000000	0.290323	0.387097	0.193548	0.161290	0.129032	0.129032	0.169643	0.428571	0.545455	0.500000
nutch-1.2	2	9	2	6	2	4	2	3	30	4	9	4	5	3	3	2	2	24	371	3	3	2	0.064690	0.066667	0.300000	0.066667	0.200000	0.066667	0.133333	0.066667	0.100000	0.166667	0.375000	0.166667	0.208333	0.125000	0.125000	0.083333	0.083333	0.307692	0.444444	0.500000	0.500000
nutch-1.3	8	11	8	6	8	7	6	4	28	13	10	13	5	12	7	9	3	25	245	12	12	9	0.102041	0.285714	0.392857	0.285714	0.214286	0.285714	0.250000	0.214286	0.142857	0.520000	0.400000	0.520000	0.200000	0.480000	0.280000	0.360000	0.120000	0.565217	0.722222	0.631579	0.750000
nutch-1.4	1	16	1	11	1	5	3	3	22	5	19	2	14	1	6	2	5	16	246	3	1	2	0.065041	0.045455	0.727273	0.045455	0.500000	0.045455	0.227273	0.136364	0.136364	0.312500	1.187500	0.125000	0.875000	0.062500	0.375000	0.125000	0.312500	0.208333	0.125000	0.142857	0.285714
nutch-1.5	2	0	2	0	2	0	1	0	26	2	2	2	1	2	2	1	0	23	249	1	2	1	0.092369	0.076923	0.000000	0.076923	0.000000	0.076923	0.000000	0.038462	0.000000	0.086957	0.086957	0.086957	0.043478	0.086957	0.086957	0.043478	0.000000	0.500000	0.666667	0.500000	1.000000
nutch-1.6	0	6	0	5	0	4	1	6	21	1	7	1	7	1	3	2	4	21	250	0	1	2	0.084000	0.000000	0.285714	0.000000	0.238095	0.000000	0.190476	0.047619	0.285714	0.047619	0.333333	0.047619	0.333333	0.047619	0.142857	0.095238	0.190476	0.125000	0.125000	0.250000	0.333333
nutch-1.7	6	8	6	5	6	5	6	8	21	5	11	5	8	5	5	5	7	16	255	7	5	5	0.062745	0.285714	0.380952	0.285714	0.238095	0.285714	0.238095	0.285714	0.380952	0.312500	0.687500	0.312500	0.500000	0.312500	0.312500	0.312500	0.437500	0.312500	0.384615	0.500000	0.416667
nutch-1.8	0	0	0	0	0	0	0	0	10	0	0	0	0	0	0	0	0	9	257	0	0	0	0.035019	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
nutch-1.9	4	6	4	5	4	6	4	3	10	5	6	4	5	4	3	4	3	9	247	5	4	4	0.036437	0.400000	0.600000	0.400000	0.500000	0.400000	0.600000	0.400000	0.300000	0.555556	0.666667	0.444444	0.555556	0.444444	0.333333	0.444444	0.333333	0.454545	0.444444	0.571429	0.571429
nutch-1.10	3	16	3	6	3	4	0	4	13	7	30	4	4	4	4	0	4	14	285	3	4	0	0.049123	0.230769	1.230769	0.230769	0.461538	0.230769	0.307692	0.000000	0.307692	0.500000	2.142857	0.285714	0.285714	0.285714	0.285714	0.000000	0.285714	0.189189	0.500000	0.500000	0.000000
nutch-1.11	4	9	4	2	4	3	2	4	15	4	9	4	2	6	24	2	6	13	383	5	6	2	0.033943	0.266667	0.600000	0.266667	0.133333	0.266667	0.200000	0.133333	0.266667	0.307692	0.692308	0.307692	0.153846	0.461538	1.846154	0.153846	0.461538	0.307692	0.666667	0.200000	0.250000
nutch-1.12	1	11	1	0	1	3	1	1	17	5	17	1	0	2	3	1	1	27	398	1	2	1	0.067839	0.058824	0.647059	0.058824	0.000000	0.058824	0.176471	0.058824	0.058824	0.185185	0.629630	0.037037	0.000000	0.074074	0.111111	0.037037	0.037037	0.227273	1.000000	0.400000	0.500000
nutch-1.13	0	7	0	2	0	2	1	1	15	3	56	0	2	1	1	2	0	25	405	1	1	2	0.061728	0.000000	0.466667	0.000000	0.133333	0.000000	0.133333	0.066667	0.066667	0.120000	2.240000	0.000000	0.080000	0.040000	0.040000	0.080000	0.000000	0.050847	0.000000	0.500000	1.000000
nutch-1.14	1	3	1	3	1	2	0	0	6	1	3	1	3	1	12	0	0	7	414	0	1	0	0.016908	0.166667	0.500000	0.166667	0.500000	0.166667	0.333333	0.000000	0.000000	0.142857	0.428571	0.142857	0.428571	0.142857	1.714286	0.000000	0.000000	0.250000	0.250000	0.076923	NaN
nutch-2.0	2	19	3	14	3	5	0	0	26	9	17	8	11	6	2	0	0	23	500	0	6	0	0.046000	0.076923	0.730769	0.115385	0.538462	0.115385	0.192308	0.000000	0.000000	0.391304	0.739130	0.347826	0.478261	0.260870	0.086957	0.000000	0.000000	0.346154	0.421053	0.750000	NaN
nutch-2.1	1	8	1	7	1	1	3	5	22	4	8	3	4	2	0	4	2	19	252	2	2	4	0.075397	0.045455	0.363636	0.045455	0.318182	0.045455	0.045455	0.136364	0.227273	0.210526	0.421053	0.157895	0.210526	0.105263	0.000000	0.210526	0.105263	0.333333	0.428571	1.000000	0.666667
nutch-2.2	6	6	6	3	6	3	8	8	20	24	3	23	3	24	3	25	7	32	255	24	24	25	0.125490	0.300000	0.300000	0.300000	0.150000	0.300000	0.150000	0.400000	0.400000	0.750000	0.093750	0.718750	0.093750	0.750000	0.093750	0.781250	0.218750	0.888889	0.884615	0.888889	0.781250
nutch-2.3	2	2	2	0	2	0	2	2	9	3	1	2	0	2	0	2	1	9	324	2	2	2	0.027778	0.222222	0.222222	0.222222	0.000000	0.222222	0.000000	0.222222	0.222222	0.333333	0.111111	0.222222	0.000000	0.222222	0.000000	0.222222	0.111111	0.750000	1.000000	1.000000	0.666667
opennlp-1.7.0	8	36	8	15	8	7	0	0	10	11	177	9	54	9	7	0	0	11	632	0	9	0	0.017405	0.800000	3.600000	0.800000	1.500000	0.800000	0.700000	0.000000	0.000000	1.000000	16.090909	0.818182	4.909091	0.818182	0.636364	0.000000	0.000000	0.058511	0.142857	0.562500	NaN
opennlp-1.8.0	2	11	2	5	2	3	1	1	4	2	19	2	13	2	4	1	1	5	626	2	2	1	0.007987	0.500000	2.750000	0.500000	1.250000	0.500000	0.750000	0.250000	0.250000	0.400000	3.800000	0.400000	2.600000	0.400000	0.800000	0.200000	0.200000	0.095238	0.133333	0.333333	0.500000
parquet-mr-1.0.0	0	0	0	0	0	0	0	0	7	0	0	0	0	0	0	0	0	5	221	0	0	0	0.022624	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
parquet-mr-1.1.0	0	0	0	0	0	0	0	0	7	0	0	0	0	0	0	0	0	5	224	0	0	0	0.022321	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
parquet-mr-1.2.0	0	1	0	0	0	0	0	0	7	0	3	0	0	0	0	0	0	5	236	0	0	0	0.021186	0.000000	0.142857	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.600000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN
parquet-mr-1.3.0	1	2	0	0	0	0	0	0	9	0	5	0	0	0	0	0	0	7	274	0	0	0	0.025547	0.111111	0.222222	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.714286	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN
parquet-mr-1.4.0	7	28	6	8	6	5	0	0	14	11	55	7	17	7	8	0	0	16	309	0	7	0	0.051780	0.500000	2.000000	0.428571	0.571429	0.428571	0.357143	0.000000	0.000000	0.687500	3.437500	0.437500	1.062500	0.437500	0.500000	0.000000	0.000000	0.166667	0.291667	0.466667	NaN
parquet-mr-1.5.0	6	37	6	12	6	6	0	4	16	13	65	7	21	7	8	3	2	19	323	0	7	3	0.058824	0.375000	2.312500	0.375000	0.750000	0.375000	0.375000	0.000000	0.250000	0.684211	3.421053	0.368421	1.105263	0.368421	0.421053	0.157895	0.105263	0.166667	0.250000	0.466667	0.600000
parquet-mr-1.6.0	6	47	6	9	6	4	2	5	12	13	354	12	22	10	19	6	20	13	377	24	10	6	0.034483	0.500000	3.916667	0.500000	0.750000	0.500000	0.333333	0.166667	0.416667	1.000000	27.230769	0.923077	1.692308	0.769231	1.461538	0.461538	1.538462	0.035422	0.352941	0.344828	0.230769
parquet-mr-1.7.0	11	84	10	18	10	14	5	5	24	23	99	22	41	20	42	7	8	30	377	8	20	7	0.079576	0.458333	3.500000	0.416667	0.750000	0.416667	0.583333	0.208333	0.208333	0.766667	3.300000	0.733333	1.366667	0.666667	1.400000	0.233333	0.266667	0.188525	0.349206	0.322581	0.466667
parquet-mr-1.8.0	9	59	9	12	9	7	9	4	25	30	78	26	7	23	5	14	8	36	402	14	23	14	0.089552	0.360000	2.360000	0.360000	0.480000	0.360000	0.280000	0.360000	0.160000	0.833333	2.166667	0.722222	0.194444	0.638889	0.138889	0.388889	0.222222	0.277778	0.787879	0.821429	0.636364
parquet-mr-1.9.0	23	79	22	19	22	11	3	5	27	36	107	35	12	33	8	8	6	38	429	10	33	8	0.088578	0.851852	2.925926	0.814815	0.703704	0.814815	0.407407	0.111111	0.185185	0.947368	2.815789	0.921053	0.315789	0.868421	0.210526	0.210526	0.157895	0.251748	0.744681	0.804878	0.571429
santuario-java-1.0.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	165	0	0	0	0.000000	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
santuario-java-1.2	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	165	0	0	0	0.000000	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
santuario-java-1.4.5	2	4	2	4	2	4	0	0	3	2	8	2	7	1	11	0	0	2	257	0	1	0	0.007782	0.666667	1.333333	0.666667	1.333333	0.666667	1.333333	0.000000	0.000000	1.000000	4.000000	1.000000	3.500000	0.500000	5.500000	0.000000	0.000000	0.200000	0.222222	0.083333	NaN
santuario-java-1.5.9	5	2	5	2	5	0	0	0	12	4	4	4	2	4	1	0	0	18	265	0	4	0	0.067925	0.416667	0.166667	0.416667	0.166667	0.416667	0.000000	0.000000	0.000000	0.222222	0.222222	0.222222	0.111111	0.222222	0.055556	0.000000	0.000000	0.500000	0.666667	0.800000	NaN
santuario-java-2.0.0	5	4	5	3	5	1	0	0	16	9	20	8	15	9	5	0	0	21	390	0	9	0	0.053846	0.312500	0.250000	0.312500	0.187500	0.312500	0.062500	0.000000	0.000000	0.428571	0.952381	0.380952	0.714286	0.428571	0.238095	0.000000	0.000000	0.310345	0.347826	0.642857	NaN
santuario-java-2.1.0	1	0	1	0	1	0	0	0	1	1	0	1	0	1	0	0	0	1	463	0	1	0	0.002160	1.000000	0.000000	1.000000	0.000000	1.000000	0.000000	0.000000	0.000000	1.000000	0.000000	1.000000	0.000000	1.000000	0.000000	0.000000	0.000000	1.000000	1.000000	1.000000	NaN
systemml-0.9	12	17	16	22	16	12	3	2	64	42	115	46	116	25	34	3	3	70	851	4	25	3	0.082256	0.187500	0.265625	0.250000	0.343750	0.250000	0.187500	0.046875	0.031250	0.600000	1.642857	0.657143	1.657143	0.357143	0.485714	0.042857	0.042857	0.267516	0.283951	0.423729	0.500000
systemml-0.10	9	32	15	37	15	25	1	2	56	22	63	26	58	25	47	2	2	70	884	4	25	2	0.079186	0.160714	0.571429	0.267857	0.660714	0.267857	0.446429	0.017857	0.035714	0.314286	0.900000	0.371429	0.828571	0.357143	0.671429	0.028571	0.028571	0.258824	0.309524	0.347222	0.500000
systemml-0.11	22	25	27	29	27	11	1	0	70	49	101	55	92	43	25	2	0	90	973	2	43	2	0.092497	0.314286	0.357143	0.385714	0.414286	0.385714	0.157143	0.014286	0.000000	0.544444	1.122222	0.611111	1.022222	0.477778	0.277778	0.022222	0.000000	0.326667	0.374150	0.632353	1.000000
systemml-0.12	31	41	33	40	33	19	0	0	65	61	157	62	150	55	72	0	0	87	964	0	55	0	0.090249	0.476923	0.630769	0.507692	0.615385	0.507692	0.292308	0.000000	0.000000	0.701149	1.804598	0.712644	1.724138	0.632184	0.827586	0.000000	0.000000	0.279817	0.292453	0.433071	NaN
systemml-0.13	29	57	33	47	33	25	0	0	55	62	194	61	171	58	81	0	0	79	971	0	58	0	0.081359	0.527273	1.036364	0.600000	0.854545	0.600000	0.454545	0.000000	0.000000	0.784810	2.455696	0.772152	2.164557	0.734177	1.025316	0.000000	0.000000	0.242188	0.262931	0.417266	NaN
systemml-0.14	29	82	32	60	32	33	8	2	56	58	167	56	143	53	98	14	8	74	1008	21	53	14	0.073413	0.517857	1.464286	0.571429	1.071429	0.571429	0.589286	0.142857	0.035714	0.783784	2.256757	0.756757	1.932432	0.716216	1.324324	0.189189	0.108108	0.257778	0.281407	0.350993	0.636364
systemml-0.15	33	68	33	54	33	21	0	0	48	46	115	45	81	44	29	0	0	57	1029	0	44	0	0.055394	0.687500	1.416667	0.687500	1.125000	0.687500	0.437500	0.000000	0.000000	0.807018	2.017544	0.789474	1.421053	0.771930	0.508772	0.000000	0.000000	0.285714	0.357143	0.602740	NaN
systemml-1.0.0	28	40	30	36	30	15	0	0	39	41	110	42	75	42	48	0	0	45	1042	0	42	0	0.043186	0.717949	1.025641	0.769231	0.923077	0.769231	0.384615	0.000000	0.000000	0.911111	2.444444	0.933333	1.666667	0.933333	1.066667	0.000000	0.000000	0.271523	0.358974	0.466667	NaN
systemml-1.1.0	27	24	28	20	28	11	0	0	28	42	71	43	55	43	51	0	0	43	1035	0	43	0	0.041546	0.964286	0.857143	1.000000	0.714286	1.000000	0.392857	0.000000	0.000000	0.976744	1.651163	1.000000	1.279070	1.000000	1.186047	0.000000	0.000000	0.371681	0.438776	0.457447	NaN
systemml-1.2.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1073	0	0	0	0.000000	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
tika-0.1	0	0	0	0	0	0	0	0	7	0	0	0	0	0	0	0	0	5	61	0	0	0	0.081967	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	NaN	NaN	NaN	NaN
tika-0.2	6	7	6	5	6	2	2	0	20	11	5	10	4	10	1	5	0	15	101	5	10	5	0.148515	0.300000	0.350000	0.300000	0.250000	0.300000	0.100000	0.100000	0.000000	0.733333	0.333333	0.666667	0.266667	0.666667	0.066667	0.333333	0.000000	0.687500	0.714286	0.909091	1.000000
tika-0.3	8	9	8	8	8	3	5	3	23	10	6	9	5	10	2	5	2	16	119	5	10	5	0.134454	0.347826	0.391304	0.347826	0.347826	0.347826	0.130435	0.217391	0.130435	0.625000	0.375000	0.562500	0.312500	0.625000	0.125000	0.312500	0.125000	0.625000	0.642857	0.833333	0.714286
tika-0.4	8	18	9	12	9	6	5	4	20	6	8	7	6	6	2	4	2	13	135	6	6	4	0.096296	0.400000	0.900000	0.450000	0.600000	0.450000	0.300000	0.250000	0.200000	0.461538	0.615385	0.538462	0.461538	0.461538	0.153846	0.307692	0.153846	0.428571	0.538462	0.750000	0.666667
tika-0.5	5	10	5	9	5	3	2	1	14	6	22	6	14	6	4	2	3	11	146	5	6	2	0.075342	0.357143	0.714286	0.357143	0.642857	0.357143	0.214286	0.142857	0.071429	0.545455	2.000000	0.545455	1.272727	0.545455	0.363636	0.181818	0.272727	0.214286	0.300000	0.600000	0.400000
tika-0.6	8	8	7	9	7	2	4	0	19	10	16	12	10	11	2	4	2	17	149	3	11	4	0.114094	0.421053	0.421053	0.368421	0.473684	0.368421	0.105263	0.210526	0.000000	0.588235	0.941176	0.705882	0.588235	0.647059	0.117647	0.235294	0.117647	0.384615	0.545455	0.846154	0.666667
tika-0.7	7	12	7	14	7	7	7	4	21	10	9	10	8	10	5	10	0	20	159	8	10	10	0.125786	0.333333	0.571429	0.333333	0.666667	0.333333	0.333333	0.333333	0.190476	0.500000	0.450000	0.500000	0.400000	0.500000	0.250000	0.500000	0.000000	0.526316	0.555556	0.666667	1.000000
tika-0.8	14	18	15	11	15	7	6	3	37	17	20	17	15	15	8	6	5	31	200	11	15	6	0.155000	0.378378	0.486486	0.405405	0.297297	0.405405	0.189189	0.162162	0.081081	0.548387	0.645161	0.548387	0.483871	0.483871	0.258065	0.193548	0.161290	0.459459	0.531250	0.652174	0.545455
tika-0.9	14	18	14	11	14	5	13	12	45	21	21	20	15	19	5	21	19	44	222	21	19	21	0.198198	0.311111	0.400000	0.311111	0.244444	0.311111	0.111111	0.288889	0.266667	0.477273	0.477273	0.454545	0.340909	0.431818	0.113636	0.477273	0.431818	0.500000	0.571429	0.791667	0.525000
tika-0.10	19	26	19	18	19	13	2	0	44	30	26	29	12	27	6	2	0	41	269	2	27	2	0.152416	0.431818	0.590909	0.431818	0.409091	0.431818	0.295455	0.045455	0.000000	0.731707	0.634146	0.707317	0.292683	0.658537	0.146341	0.048780	0.000000	0.535714	0.707317	0.818182	1.000000
tika-1.0	14	20	14	12	14	9	12	9	43	25	25	24	11	22	5	20	8	40	269	21	22	20	0.148699	0.325581	0.465116	0.325581	0.279070	0.325581	0.209302	0.279070	0.209302	0.625000	0.625000	0.600000	0.275000	0.550000	0.125000	0.500000	0.200000	0.500000	0.685714	0.814815	0.714286
tika-1.1	12	14	12	8	12	9	7	2	39	13	19	13	10	11	7	5	2	33	275	4	11	5	0.120000	0.307692	0.358974	0.307692	0.205128	0.307692	0.230769	0.179487	0.051282	0.393939	0.575758	0.393939	0.303030	0.333333	0.212121	0.151515	0.060606	0.406250	0.565217	0.611111	0.714286
tika-1.2	15	15	15	11	15	10	6	8	43	15	16	14	11	14	8	7	5	38	307	9	14	7	0.123779	0.348837	0.348837	0.348837	0.255814	0.348837	0.232558	0.139535	0.186047	0.394737	0.421053	0.368421	0.289474	0.368421	0.210526	0.184211	0.131579	0.483871	0.560000	0.636364	0.583333
tika-1.3	5	10	7	4	7	8	3	4	30	9	8	10	4	10	6	4	6	30	311	8	10	4	0.096463	0.166667	0.333333	0.233333	0.133333	0.233333	0.266667	0.100000	0.133333	0.300000	0.266667	0.333333	0.133333	0.333333	0.200000	0.133333	0.200000	0.529412	0.714286	0.625000	0.400000
tika-1.4	7	8	9	2	9	6	11	4	34	11	6	12	2	12	4	12	2	33	318	9	12	12	0.103774	0.205882	0.235294	0.264706	0.058824	0.264706	0.176471	0.323529	0.117647	0.333333	0.181818	0.363636	0.060606	0.363636	0.121212	0.363636	0.060606	0.647059	0.857143	0.750000	0.857143
tika-1.5	1	3	1	3	1	1	1	1	17	4	8	4	8	1	1	1	1	20	321	2	1	1	0.062305	0.058824	0.176471	0.058824	0.176471	0.058824	0.058824	0.058824	0.058824	0.200000	0.400000	0.200000	0.400000	0.050000	0.050000	0.050000	0.050000	0.333333	0.333333	0.500000	0.500000
tika-1.6	0	1	0	1	0	0	0	0	20	1	0	1	0	0	0	0	0	23	344	0	0	0	0.066860	0.000000	0.050000	0.000000	0.050000	0.000000	0.000000	0.000000	0.000000	0.043478	0.000000	0.043478	0.000000	0.000000	0.000000	0.000000	0.000000	1.000000	1.000000	NaN	NaN
tika-1.7	7	33	7	21	7	12	3	4	17	14	77	12	64	9	16	3	7	21	363	9	9	3	0.057851	0.411765	1.941176	0.411765	1.235294	0.411765	0.705882	0.176471	0.235294	0.666667	3.666667	0.571429	3.047619	0.428571	0.761905	0.142857	0.333333	0.153846	0.157895	0.360000	0.300000
tika-1.8	3	24	3	15	3	10	1	1	15	8	51	5	31	4	12	1	1	19	436	2	4	1	0.043578	0.200000	1.600000	0.200000	1.000000	0.200000	0.666667	0.066667	0.066667	0.421053	2.684211	0.263158	1.631579	0.210526	0.631579	0.052632	0.052632	0.135593	0.138889	0.250000	0.500000
tika-1.9	10	16	10	10	10	8	4	2	24	13	24	12	15	11	15	4	4	28	464	7	11	4	0.060345	0.416667	0.666667	0.416667	0.416667	0.416667	0.333333	0.166667	0.083333	0.464286	0.857143	0.428571	0.535714	0.392857	0.535714	0.142857	0.142857	0.351351	0.444444	0.423077	0.500000
tika-1.10	3	9	3	7	3	5	1	1	19	5	14	4	10	4	7	1	1	22	473	2	4	1	0.046512	0.157895	0.473684	0.157895	0.368421	0.157895	0.263158	0.052632	0.052632	0.227273	0.636364	0.181818	0.454545	0.181818	0.318182	0.045455	0.045455	0.263158	0.285714	0.363636	0.500000
tika-1.11	3	6	3	5	3	2	3	0	21	6	8	6	6	6	2	6	0	27	479	6	6	6	0.056367	0.142857	0.285714	0.142857	0.238095	0.142857	0.095238	0.142857	0.000000	0.222222	0.296296	0.222222	0.222222	0.222222	0.074074	0.222222	0.000000	0.428571	0.500000	0.750000	1.000000
tika-1.12	1	1	1	0	1	1	0	0	20	2	1	2	0	2	1	0	0	26	488	0	2	0	0.053279	0.050000	0.050000	0.050000	0.000000	0.050000	0.050000	0.000000	0.000000	0.076923	0.038462	0.076923	0.000000	0.076923	0.038462	0.000000	0.000000	0.666667	1.000000	0.666667	NaN
tika-1.13	1	6	1	3	1	3	0	0	18	3	13	1	3	3	11	0	0	22	506	0	3	0	0.043478	0.055556	0.333333	0.055556	0.166667	0.055556	0.166667	0.000000	0.000000	0.136364	0.590909	0.045455	0.136364	0.136364	0.500000	0.000000	0.000000	0.187500	0.250000	0.214286	NaN
tika-1.14	1	9	1	7	1	6	0	1	19	8	18	8	17	4	2	0	1	25	538	1	4	0	0.046468	0.052632	0.473684	0.052632	0.368421	0.052632	0.315789	0.000000	0.052632	0.320000	0.720000	0.320000	0.680000	0.160000	0.080000	0.000000	0.040000	0.307692	0.320000	0.666667	0.000000
tika-1.15	2	3	2	1	2	1	0	0	19	6	10	6	4	5	7	0	0	25	636	0	5	0	0.039308	0.105263	0.157895	0.105263	0.052632	0.105263	0.052632	0.000000	0.000000	0.240000	0.400000	0.240000	0.160000	0.200000	0.280000	0.000000	0.000000	0.375000	0.600000	0.416667	NaN
tika-1.16	1	1	1	0	1	0	1	0	19	1	2	1	0	1	0	1	0	24	642	1	1	1	0.037383	0.052632	0.052632	0.052632	0.000000	0.052632	0.000000	0.052632	0.000000	0.041667	0.083333	0.041667	0.000000	0.041667	0.000000	0.041667	0.000000	0.333333	1.000000	1.000000	1.000000
tika-1.17	10	12	10	5	10	3	5	1	12	13	9	13	4	13	4	7	3	16	651	10	13	7	0.024578	0.833333	1.000000	0.833333	0.416667	0.833333	0.250000	0.416667	0.083333	0.812500	0.562500	0.812500	0.250000	0.812500	0.250000	0.437500	0.187500	0.590909	0.764706	0.764706	0.700000
wss4j-1.5.0	2	0	3	0	3	0	0	0	17	2	0	3	0	3	0	0	0	12	278	0	3	0	0.043165	0.117647	0.000000	0.176471	0.000000	0.176471	0.000000	0.000000	0.000000	0.166667	0.000000	0.250000	0.000000	0.250000	0.000000	0.000000	0.000000	1.000000	1.000000	1.000000	NaN
wss4j-1.6.0	10	11	10	7	10	5	6	4	15	10	17	10	11	9	7	6	4	13	135	10	9	6	0.096296	0.666667	0.733333	0.666667	0.466667	0.666667	0.333333	0.400000	0.266667	0.769231	1.307692	0.769231	0.846154	0.692308	0.538462	0.461538	0.307692	0.370370	0.476190	0.562500	0.600000
wss4j-2.0.0	0	2	0	2	0	2	0	0	12	0	12	0	8	0	12	0	0	11	482	0	0	0	0.022822	0.000000	0.166667	0.000000	0.166667	0.000000	0.166667	0.000000	0.000000	0.000000	1.090909	0.000000	0.727273	0.000000	1.090909	0.000000	0.000000	0.000000	0.000000	0.000000	NaN
wss4j-2.1.0	0	4	0	1	0	1	0	0	11	5	92	0	1	0	2	0	0	10	493	0	0	0	0.020284	0.000000	0.363636	0.000000	0.090909	0.000000	0.090909	0.000000	0.000000	0.500000	9.200000	0.000000	0.100000	0.000000	0.200000	0.000000	0.000000	0.051546	0.000000	0.000000	NaN
wss4j-2.2.0	0	5	0	4	0	0	0	0	0	0	10	0	5	0	0	0	0	0	500	0	0	0	0.000000	NaN	inf	NaN	inf	NaN	NaN	NaN	NaN	NaN	inf	NaN	inf	NaN	NaN	NaN	NaN	0.000000	0.000000	NaN	NaN

statistics with all data
mean values
issues_szz_six_months_tp                      5.118090
issues_szz_six_months_fp                     11.208543
issues_jlr_six_months_tp                      5.484925
issues_jlr_six_months_fp                      7.723618
issues_jlmiv_six_months_tp                    5.494975
issues_jlmiv_six_months_fp                    4.668342
issues_jlmiv_affected_version_tp              3.055276
issues_jlmiv_affected_version_fp              2.459799
issues_jlmiv_inducing                        18.391960
files_szz_six_months_tp                       9.643216
files_szz_six_months_fp                      27.783920
files_jlr_six_months_tp                       9.374372
files_jlr_six_months_fp                      15.984925
files_jlmiv_six_months_tp                     8.695980
files_jlmiv_six_months_fp                     8.138191
files_jlmiv_affected_version_tp               4.600503
files_jlmiv_affected_version_fp               3.572864
files_jlmiv_inducing                         20.010050
num_java_files                              387.615578
overlap_6mav                                  5.628141
overlap_6min                                  8.695980
overlap_avin                                  4.600503
percent_files_jlmiv_ind_total                 0.067036
percent_issues_szz_six_months_tp              0.224417
percent_issues_szz_six_months_fp                   inf
percent_issues_jlr_six_months_tp              0.261371
percent_issues_jlr_six_months_fp                   inf
percent_issues_jlmiv_six_months_tp            0.262529
percent_issues_jlmiv_six_months_fp                 inf
percent_issues_jlmiv_affected_version_tp      0.201374
percent_issues_jlmiv_affected_version_fp           inf
percent_files_szz_six_months_tp               0.343381
percent_files_szz_six_months_fp                    inf
percent_files_jlr_six_months_tp               0.358746
percent_files_jlr_six_months_fp                    inf
percent_files_jlmiv_six_months_tp             0.344411
percent_files_jlmiv_six_months_fp                  inf
percent_files_jlmiv_affected_version_tp       0.276038
percent_files_jlmiv_affected_version_fp            inf
percent_files_correct_szz_6m                  0.401299
percent_files_correct_jlr_6m                  0.532015
percent_files_correct_jlmiv_6m                0.588661
percent_files_correct_jlmiv_av                0.623173

sd values
issues_szz_six_months_tp                      8.201408
issues_szz_six_months_fp                     18.918338
issues_jlr_six_months_tp                      8.555554
issues_jlr_six_months_fp                     13.136716
issues_jlmiv_six_months_tp                    8.552621
issues_jlmiv_six_months_fp                    7.384070
issues_jlmiv_affected_version_tp              3.610363
issues_jlmiv_affected_version_fp              3.733185
issues_jlmiv_inducing                        18.657111
files_szz_six_months_tp                      16.975114
files_szz_six_months_fp                      66.134422
files_jlr_six_months_tp                      16.319945
files_jlr_six_months_fp                      39.304402
files_jlmiv_six_months_tp                    14.670883
files_jlmiv_six_months_fp                    15.724758
files_jlmiv_affected_version_tp               6.465853
files_jlmiv_affected_version_fp               6.616023
files_jlmiv_inducing                         23.355643
num_java_files                              352.006491
overlap_6mav                                  8.955384
overlap_6min                                 14.670883
overlap_avin                                  6.465853
percent_files_jlmiv_ind_total                 0.060017
percent_issues_szz_six_months_tp              0.229930
percent_issues_szz_six_months_fp                   NaN
percent_issues_jlr_six_months_tp              0.250432
percent_issues_jlr_six_months_fp                   NaN
percent_issues_jlmiv_six_months_tp            0.250387
percent_issues_jlmiv_six_months_fp                 NaN
percent_issues_jlmiv_affected_version_tp      0.226729
percent_issues_jlmiv_affected_version_fp           NaN
percent_files_szz_six_months_tp               0.297153
percent_files_szz_six_months_fp                    NaN
percent_files_jlr_six_months_tp               0.302734
percent_files_jlr_six_months_fp                    NaN
percent_files_jlmiv_six_months_tp             0.292382
percent_files_jlmiv_six_months_fp                  NaN
percent_files_jlmiv_affected_version_tp       0.281732
percent_files_jlmiv_affected_version_fp            NaN
percent_files_correct_szz_6m                  0.288305
percent_files_correct_jlr_6m                  0.304717
percent_files_correct_jlmiv_6m                0.282806
percent_files_correct_jlmiv_av                0.296172

median values
issues_szz_six_months_tp                      2.000000
issues_szz_six_months_fp                      4.000000
issues_jlr_six_months_tp                      2.500000
issues_jlr_six_months_fp                      2.000000
issues_jlmiv_six_months_tp                    3.000000
issues_jlmiv_six_months_fp                    2.000000
issues_jlmiv_affected_version_tp              2.000000
issues_jlmiv_affected_version_fp              1.000000
issues_jlmiv_inducing                        14.000000
files_szz_six_months_tp                       3.000000
files_szz_six_months_fp                       4.000000
files_jlr_six_months_tp                       3.000000
files_jlr_six_months_fp                       2.000000
files_jlmiv_six_months_tp                     3.000000
files_jlmiv_six_months_fp                     2.000000
files_jlmiv_affected_version_tp               2.000000
files_jlmiv_affected_version_fp               1.000000
files_jlmiv_inducing                         13.000000
num_java_files                              269.500000
overlap_6mav                                  2.000000
overlap_6min                                  3.000000
overlap_avin                                  2.000000
percent_files_jlmiv_ind_total                 0.051263
percent_issues_szz_six_months_tp              0.169048
percent_issues_szz_six_months_fp              0.358059
percent_issues_jlr_six_months_tp              0.230769
percent_issues_jlr_six_months_fp              0.214286
percent_issues_jlmiv_six_months_tp            0.235714
percent_issues_jlmiv_six_months_fp            0.126603
percent_issues_jlmiv_affected_version_tp      0.145503
percent_issues_jlmiv_affected_version_fp      0.068966
percent_files_szz_six_months_tp               0.312500
percent_files_szz_six_months_fp               0.400000
percent_files_jlr_six_months_tp               0.337719
percent_files_jlr_six_months_fp               0.190476
percent_files_jlmiv_six_months_tp             0.333333
percent_files_jlmiv_six_months_fp             0.147059
percent_files_jlmiv_affected_version_tp       0.198214
percent_files_jlmiv_affected_version_fp       0.071429
percent_files_correct_szz_6m                  0.344828
percent_files_correct_jlr_6m                  0.500000
percent_files_correct_jlmiv_6m                0.600000
percent_files_correct_jlmiv_av                0.666667

MAD values
issues_szz_six_months_tp                      2.96520
issues_szz_six_months_fp                      5.93040
issues_jlr_six_months_tp                      3.70650
issues_jlr_six_months_fp                      2.96520
issues_jlmiv_six_months_tp                    4.44780
issues_jlmiv_six_months_fp                    2.96520
issues_jlmiv_affected_version_tp              2.96520
issues_jlmiv_affected_version_fp              1.48260
issues_jlmiv_inducing                        14.82600
files_szz_six_months_tp                       4.44780
files_szz_six_months_fp                       5.93040
files_jlr_six_months_tp                       4.44780
files_jlr_six_months_fp                       2.96520
files_jlmiv_six_months_tp                     4.44780
files_jlmiv_six_months_fp                     2.96520
files_jlmiv_affected_version_tp               2.96520
files_jlmiv_affected_version_fp               1.48260
files_jlmiv_inducing                         14.82600
num_java_files                              292.81350
overlap_6mav                                  2.96520
overlap_6min                                  4.44780
overlap_avin                                  2.96520
percent_files_jlmiv_ind_total                 0.04817
percent_issues_szz_six_months_tp                  NaN
percent_issues_szz_six_months_fp                  NaN
percent_issues_jlr_six_months_tp                  NaN
percent_issues_jlr_six_months_fp                  NaN
percent_issues_jlmiv_six_months_tp                NaN
percent_issues_jlmiv_six_months_fp                NaN
percent_issues_jlmiv_affected_version_tp          NaN
percent_issues_jlmiv_affected_version_fp          NaN
percent_files_szz_six_months_tp                   NaN
percent_files_szz_six_months_fp                   NaN
percent_files_jlr_six_months_tp                   NaN
percent_files_jlr_six_months_fp                   NaN
percent_files_jlmiv_six_months_tp                 NaN
percent_files_jlmiv_six_months_fp                 NaN
percent_files_jlmiv_affected_version_tp           NaN
percent_files_jlmiv_affected_version_fp           NaN
percent_files_correct_szz_6m                      NaN
percent_files_correct_jlr_6m                      NaN
percent_files_correct_jlmiv_6m                    NaN
percent_files_correct_jlmiv_av                    NaN

statistics without the 36 releases without any assigned issue
mean values
issues_szz_six_months_tp                      5.627072
issues_szz_six_months_fp                     12.226519
issues_jlr_six_months_tp                      6.030387
issues_jlr_six_months_fp                      8.378453
issues_jlmiv_six_months_tp                    6.041436
issues_jlmiv_six_months_fp                    5.044199
issues_jlmiv_affected_version_tp              3.359116
issues_jlmiv_affected_version_fp              2.602210
issues_jlmiv_inducing                        20.220994
files_szz_six_months_tp                      10.602210
files_szz_six_months_fp                      30.331492
files_jlr_six_months_tp                      10.306630
files_jlr_six_months_fp                      17.397790
files_jlmiv_six_months_tp                     9.560773
files_jlmiv_six_months_fp                     8.580110
files_jlmiv_affected_version_tp               5.058011
files_jlmiv_affected_version_fp               3.693370
files_jlmiv_inducing                         22.000000
num_java_files                              405.074586
overlap_6mav                                  6.022099
overlap_6min                                  9.560773
overlap_avin                                  5.058011
percent_files_jlmiv_ind_total                 0.073703
percent_issues_szz_six_months_tp              0.224417
percent_issues_szz_six_months_fp              0.843093
percent_issues_jlr_six_months_tp              0.261371
percent_issues_jlr_six_months_fp              0.518603
percent_issues_jlmiv_six_months_tp            0.262529
percent_issues_jlmiv_six_months_fp            0.399952
percent_issues_jlmiv_affected_version_tp      0.201374
percent_issues_jlmiv_affected_version_fp      0.248110
percent_files_szz_six_months_tp               0.343381
percent_files_szz_six_months_fp               1.504707
percent_files_jlr_six_months_tp               0.358746
percent_files_jlr_six_months_fp               0.692806
percent_files_jlmiv_six_months_tp             0.344411
percent_files_jlmiv_six_months_fp             0.419065
percent_files_jlmiv_affected_version_tp       0.276038
percent_files_jlmiv_affected_version_fp       0.288772
percent_files_correct_szz_6m                  0.411965
percent_files_correct_jlr_6m                  0.548469
percent_files_correct_jlmiv_6m                0.609030
percent_files_correct_jlmiv_av                0.641236

sd values
issues_szz_six_months_tp                      8.432008
issues_szz_six_months_fp                     19.526913
issues_jlr_six_months_tp                      8.786269
issues_jlr_six_months_fp                     13.572331
issues_jlmiv_six_months_tp                    8.782439
issues_jlmiv_six_months_fp                    7.619645
issues_jlmiv_affected_version_tp              3.648441
issues_jlmiv_affected_version_fp              3.815959
issues_jlmiv_inducing                        18.593354
files_szz_six_months_tp                      17.512696
files_szz_six_months_fp                      68.812408
files_jlr_six_months_tp                      16.830503
files_jlr_six_months_fp                      40.920392
files_jlmiv_six_months_tp                    15.113162
files_jlmiv_six_months_fp                    16.016454
files_jlmiv_affected_version_tp               6.607262
files_jlmiv_affected_version_fp               6.585253
files_jlmiv_inducing                         23.579300
num_java_files                              350.009011
overlap_6mav                                  9.099606
overlap_6min                                 15.113162
overlap_avin                                  6.607262
percent_files_jlmiv_ind_total                 0.058895
percent_issues_szz_six_months_tp              0.229930
percent_issues_szz_six_months_fp              2.753973
percent_issues_jlr_six_months_tp              0.250432
percent_issues_jlr_six_months_fp              1.805904
percent_issues_jlmiv_six_months_tp            0.250387
percent_issues_jlmiv_six_months_fp            1.847707
percent_issues_jlmiv_affected_version_tp      0.226729
percent_issues_jlmiv_affected_version_fp      0.585654
percent_files_szz_six_months_tp               0.297153
percent_files_szz_six_months_fp               5.335719
percent_files_jlr_six_months_tp               0.302734
percent_files_jlr_six_months_fp               1.486327
percent_files_jlmiv_six_months_tp             0.292382
percent_files_jlmiv_six_months_fp             1.113389
percent_files_jlmiv_affected_version_tp       0.281732
percent_files_jlmiv_affected_version_fp       0.650941
percent_files_correct_szz_6m                  0.284478
percent_files_correct_jlr_6m                  0.294412
percent_files_correct_jlmiv_6m                0.265157
percent_files_correct_jlmiv_av                0.280437

median values
issues_szz_six_months_tp                      3.000000
issues_szz_six_months_fp                      5.000000
issues_jlr_six_months_tp                      3.000000
issues_jlr_six_months_fp                      3.000000
issues_jlmiv_six_months_tp                    3.000000
issues_jlmiv_six_months_fp                    2.000000
issues_jlmiv_affected_version_tp              2.000000
issues_jlmiv_affected_version_fp              1.000000
issues_jlmiv_inducing                        16.000000
files_szz_six_months_tp                       4.000000
files_szz_six_months_fp                       5.000000
files_jlr_six_months_tp                       4.000000
files_jlr_six_months_fp                       3.000000
files_jlmiv_six_months_tp                     4.000000
files_jlmiv_six_months_fp                     2.000000
files_jlmiv_affected_version_tp               3.000000
files_jlmiv_affected_version_fp               1.000000
files_jlmiv_inducing                         15.500000
num_java_files                              307.000000
overlap_6mav                                  2.500000
overlap_6min                                  4.000000
overlap_avin                                  3.000000
percent_files_jlmiv_ind_total                 0.057692
percent_issues_szz_six_months_tp              0.169048
percent_issues_szz_six_months_fp              0.333333
percent_issues_jlr_six_months_tp              0.230769
percent_issues_jlr_six_months_fp              0.200000
percent_issues_jlmiv_six_months_tp            0.235714
percent_issues_jlmiv_six_months_fp            0.119430
percent_issues_jlmiv_affected_version_tp      0.145503
percent_issues_jlmiv_affected_version_fp      0.064583
percent_files_szz_six_months_tp               0.312500
percent_files_szz_six_months_fp               0.377976
percent_files_jlr_six_months_tp               0.337719
percent_files_jlr_six_months_fp               0.170290
percent_files_jlmiv_six_months_tp             0.333333
percent_files_jlmiv_six_months_fp             0.142857
percent_files_jlmiv_affected_version_tp       0.198214
percent_files_jlmiv_affected_version_fp       0.065591
percent_files_correct_szz_6m                  0.347368
percent_files_correct_jlr_6m                  0.500000
percent_files_correct_jlmiv_6m                0.602740
percent_files_correct_jlmiv_av                0.666667

MAD values
issues_szz_six_months_tp                      4.447800
issues_szz_six_months_fp                      7.413000
issues_jlr_six_months_tp                      4.447800
issues_jlr_six_months_fp                      4.447800
issues_jlmiv_six_months_tp                    4.447800
issues_jlmiv_six_months_fp                    2.965200
issues_jlmiv_affected_version_tp              2.965200
issues_jlmiv_affected_version_fp              1.482600
issues_jlmiv_inducing                        13.343400
files_szz_six_months_tp                       5.930400
files_szz_six_months_fp                       7.413000
files_jlr_six_months_tp                       5.930400
files_jlr_six_months_fp                       4.447800
files_jlmiv_six_months_tp                     5.930400
files_jlmiv_six_months_fp                     2.965200
files_jlmiv_affected_version_tp               4.447800
files_jlmiv_affected_version_fp               1.482600
files_jlmiv_inducing                         14.084700
num_java_files                              308.380800
overlap_6mav                                  3.706500
overlap_6min                                  5.930400
overlap_avin                                  4.447800
percent_files_jlmiv_ind_total                 0.045333
percent_issues_szz_six_months_tp              0.250630
percent_issues_szz_six_months_fp              0.494200
percent_issues_jlr_six_months_tp              0.265107
percent_issues_jlr_six_months_fp              0.296520
percent_issues_jlmiv_six_months_tp            0.266998
percent_issues_jlmiv_six_months_fp            0.177066
percent_issues_jlmiv_affected_version_tp      0.215722
percent_issues_jlmiv_affected_version_fp      0.095751
percent_files_szz_six_months_tp               0.385064
percent_files_szz_six_months_fp               0.560387
percent_files_jlr_six_months_tp               0.386656
percent_files_jlr_six_months_fp               0.252472
percent_files_jlmiv_six_months_tp             0.353000
percent_files_jlmiv_six_months_fp             0.211800
percent_files_jlmiv_affected_version_tp       0.293873
percent_files_jlmiv_affected_version_fp       0.097246
percent_files_correct_szz_6m                       NaN
percent_files_correct_jlr_6m                       NaN
percent_files_correct_jlmiv_6m                     NaN
percent_files_correct_jlmiv_av                     NaN

Extraction of Release Level Data

for project in projects:
    for release in releases[project]:
        release_name = project+'-'+release['version']
        if 'corrected_revision' in release:
            revision_hash = release['corrected_revision']
        else:
            revision_hash = release['revision_hash']
        print("starting mynbou for release:      ", release_name)
        print("\t...with IND-JLMIV+R")
        if os.path.isfile('./release-level-data/'+release_name+'.json'):
            print("skipping")
        else:
            %run ../mynbou/smartshark_plugin.py -H $db_host -p $db_port -DB $db_name --project-name $project --release-name release-level-data/$release_name --release-commit $revision_hash --log-level INFO --generate-json False
        print("\t... with 6M-SZZ")
        if os.path.isfile('./DO_NOT_USE_release-level-data/'+release_name+'_SZZ.json'):
            print("skipping")
        else:
            %run ../mynbou/smartshark_plugin.py -H $db_host -p $db_port -DB $db_name --project-name $project --release-name DO_NOT_USE_release-level-data/$release_name --release-commit $revision_hash --log-level INFO --generate-json False --type=SZZ

starting mynbou for release:       ant-ivy-1.4.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       ant-ivy-2.0.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       ant-ivy-2.1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       ant-ivy-2.2.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       ant-ivy-2.3.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       ant-ivy-2.4.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       archiva-1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       archiva-1.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       archiva-1.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       archiva-1.3
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       archiva-2.0.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       archiva-2.1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       archiva-2.2.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       calcite-1.0.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       calcite-1.1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       calcite-1.2.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       calcite-1.3.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       calcite-1.4.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       calcite-1.5.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       calcite-1.6.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       calcite-1.7.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       calcite-1.8.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       calcite-1.9.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       calcite-1.10.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       calcite-1.11.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       calcite-1.12.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       calcite-1.13.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       calcite-1.14.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       calcite-1.15.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       cayenne-3.0.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       cayenne-3.1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-bcel-5.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-bcel-5.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-bcel-5.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-bcel-6.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-bcel-6.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-bcel-6.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-beanutils-1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-beanutils-1.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-beanutils-1.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-beanutils-1.3
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-beanutils-1.4
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-beanutils-1.5
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-beanutils-1.6
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-beanutils-1.7.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-beanutils-1.8.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-beanutils-1.9.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-codec-1.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-codec-1.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-codec-1.3
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-codec-1.4
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-codec-1.5
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-codec-1.6
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-codec-1.7
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-codec-1.8
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-codec-1.9
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-codec-1.10
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-codec-1.11
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-collections-1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-collections-2.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-collections-2.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-collections-3.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-collections-3.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-collections-3.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-collections-3.3
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-collections-4.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-collections-4.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-compress-1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-compress-1.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-compress-1.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-compress-1.3
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-compress-1.4
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-compress-1.5
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-compress-1.6
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-compress-1.7
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-compress-1.8
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-compress-1.9
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-compress-1.10
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-compress-1.11
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-compress-1.12
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-compress-1.13
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-compress-1.14
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-compress-1.15
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-compress-1.16
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-configuration-1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-configuration-1.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-configuration-1.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-configuration-1.3
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-configuration-1.4
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-configuration-1.5
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-configuration-1.6
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-configuration-1.7
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-configuration-1.8
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-configuration-1.9
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-configuration-1.10
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-configuration-2.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-configuration-2.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-configuration-2.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-dbcp-1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-dbcp-1.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-dbcp-1.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-dbcp-1.3
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-dbcp-1.4
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-dbcp-2.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-dbcp-2.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-dbcp-2.2.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-dbcp-2.3.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-dbcp-2.4.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-dbcp-2.5.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-digester-1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-digester-1.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-digester-1.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-digester-1.3
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-digester-1.4
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-digester-1.5
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-digester-1.6
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-digester-1.7
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-digester-1.8
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-digester-2.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-digester-2.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-digester-3.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-digester-3.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-digester-3.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-io-1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-io-1.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-io-1.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-io-1.3
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-io-1.4
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-io-2.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-io-2.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-io-2.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-io-2.3
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-io-2.4
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-io-2.5
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-jcs-1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-jcs-1.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-jcs-1.3
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-jcs-2.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-jcs-2.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-jcs-2.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-jexl-1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-jexl-1.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-jexl-2.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-jexl-2.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-jexl-3.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-jexl-3.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-lang-1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-lang-2.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-lang-2.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-lang-2.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-lang-2.3
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-lang-2.4
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-lang-2.5
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-lang-2.6
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-lang-3.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-lang-3.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-lang-3.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-lang-3.3
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-lang-3.4
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-lang-3.5
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-lang-3.6
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-lang-3.7
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-math-1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-math-1.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-math-1.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-math-2.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-math-2.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-math-2.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-math-3.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-math-3.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-math-3.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-math-3.3
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-math-3.4
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-math-3.5
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-math-3.6
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-net-1.0.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-net-1.1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-net-1.2.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-net-1.3.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-net-1.4.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-net-2.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-net-2.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-net-2.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-net-3.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-net-3.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-net-3.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-net-3.3
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-net-3.4
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-net-3.5
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-net-3.6
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-scxml-0.5
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-scxml-0.6
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-scxml-0.7
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-scxml-0.8
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-scxml-0.9
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-validator-1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-validator-1.1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-validator-1.2.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-validator-1.3.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-validator-1.4.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-validator-1.5.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-validator-1.6.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-vfs-1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-vfs-2.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-vfs-2.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       commons-vfs-2.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       deltaspike-0.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       deltaspike-0.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       deltaspike-0.3
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       deltaspike-0.4
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       deltaspike-0.5
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       deltaspike-0.6
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       deltaspike-0.7
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       deltaspike-1.0.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       deltaspike-1.1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       deltaspike-1.2.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       deltaspike-1.3.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       deltaspike-1.4.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       deltaspike-1.5.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       deltaspike-1.6.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       deltaspike-1.7.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       deltaspike-1.8.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       eagle-0.3.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       eagle-0.4.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       eagle-0.5.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       giraph-0.1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       giraph-1.0.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       giraph-1.1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       gora-0.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       gora-0.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       gora-0.3
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       gora-0.4
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       gora-0.5
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       gora-0.6
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       gora-0.7
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       gora-0.8
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       jspwiki-1.4.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       jspwiki-1.5.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       jspwiki-1.6.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       jspwiki-1.7.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       jspwiki-1.8.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       jspwiki-2.0.36
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       jspwiki-2.2.19
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       jspwiki-2.4.56
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       jspwiki-2.6.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       jspwiki-2.8.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       jspwiki-2.9.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       jspwiki-2.10.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       knox-0.3.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       knox-0.4.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       knox-0.5.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       knox-0.6.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       knox-0.7.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       knox-0.8.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       knox-0.9.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       knox-0.10.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       knox-0.11.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       knox-0.12.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       knox-0.13.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       knox-0.14.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       knox-1.0.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       kylin-0.6.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       kylin-0.7.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       kylin-1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       kylin-1.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       kylin-1.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       kylin-1.3
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       kylin-1.5.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       kylin-1.6.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       kylin-2.0.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       kylin-2.1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       kylin-2.2.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       lens-2.6.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       lens-2.7.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       mahout-0.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       mahout-0.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       mahout-0.3
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       mahout-0.4
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       mahout-0.5
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       mahout-0.6
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       mahout-0.7
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       mahout-0.8
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       mahout-0.9
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       mahout-0.10.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       mahout-0.11.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       mahout-0.12.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       mahout-0.13.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       manifoldcf-0.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       manifoldcf-0.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       manifoldcf-0.3
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       manifoldcf-0.4
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       manifoldcf-0.5
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       manifoldcf-0.6
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       manifoldcf-1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       manifoldcf-1.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       manifoldcf-1.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       manifoldcf-1.3
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       manifoldcf-1.4
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       manifoldcf-1.5
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       manifoldcf-1.6
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       manifoldcf-1.7
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       manifoldcf-1.8
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       manifoldcf-1.9
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       manifoldcf-1.10
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       manifoldcf-2.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       manifoldcf-2.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       manifoldcf-2.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       manifoldcf-2.3
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       manifoldcf-2.4
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       manifoldcf-2.5
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       manifoldcf-2.6
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       manifoldcf-2.7
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       manifoldcf-2.8
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       manifoldcf-2.9
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       manifoldcf-2.10
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       nutch-0.7
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       nutch-0.8
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       nutch-0.9
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       nutch-1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       nutch-1.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       nutch-1.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       nutch-1.3
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       nutch-1.4
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       nutch-1.5
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       nutch-1.6
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       nutch-1.7
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       nutch-1.8
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       nutch-1.9
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       nutch-1.10
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       nutch-1.11
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       nutch-1.12
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       nutch-1.13
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       nutch-1.14
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       nutch-2.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       nutch-2.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       nutch-2.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       nutch-2.3
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       opennlp-1.7.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       opennlp-1.8.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       parquet-mr-1.0.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       parquet-mr-1.1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       parquet-mr-1.2.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       parquet-mr-1.3.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       parquet-mr-1.4.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       parquet-mr-1.5.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       parquet-mr-1.6.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       parquet-mr-1.7.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       parquet-mr-1.8.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       parquet-mr-1.9.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       santuario-java-1.0.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       santuario-java-1.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       santuario-java-1.4.5
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       santuario-java-1.5.9
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       santuario-java-2.0.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       santuario-java-2.1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       systemml-0.9
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       systemml-0.10
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       systemml-0.11
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       systemml-0.12
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       systemml-0.13
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       systemml-0.14
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       systemml-0.15
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       systemml-1.0.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       systemml-1.1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       systemml-1.2.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       tika-0.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       tika-0.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       tika-0.3
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       tika-0.4
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       tika-0.5
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       tika-0.6
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       tika-0.7
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       tika-0.8
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       tika-0.9
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       tika-0.10
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       tika-1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       tika-1.1
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       tika-1.2
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       tika-1.3
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       tika-1.4
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       tika-1.5
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       tika-1.6
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       tika-1.7
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       tika-1.8
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       tika-1.9
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       tika-1.10
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       tika-1.11
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       tika-1.12
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       tika-1.13
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       tika-1.14
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       tika-1.15
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       tika-1.16
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       tika-1.17
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       wss4j-1.5.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       wss4j-1.6.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       wss4j-2.0.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       wss4j-2.1.0
	...with IND-JLMIV+R
	... with 6M-SZZ
starting mynbou for release:       wss4j-2.2.0
	...with IND-JLMIV+R
	... with 6M-SZZ

Feature Experiment with Predictions

Training of classifiers

import numpy as np

def lower_bound(bug_matrix, efforts, predictions):
    bug_matrix = bug_matrix.loc[:,bug_matrix.sum()>0]
    effort_true = efforts[predictions].sum()
    bugs_found = bug_matrix.sum().eq(bug_matrix[predictions].sum()).sum()
    return effort_true/bugs_found

def upper_bound(bug_matrix, efforts, predictions):
    bug_matrix = bug_matrix.loc[:,bug_matrix.sum()>0]
    effort_false = efforts[~predictions].sum()
    bugs_missed = len(bug_matrix.columns)-bug_matrix.sum().eq(bug_matrix[predictions].sum()).sum()
    return effort_false/bugs_missed

def load_data(version_name, variant=''):
    if len(variant):
        file = 'DO_NOT_USE_release-level-data/%s_%s_aggregated.csv' % (version_name, variant)
    else:
        file = 'release-level-data/%s_aggregated.csv' % version_name
    data = pd.read_csv(file,sep=';', index_col='file')
    data = data.drop('imports', axis='columns')

    features_all = data.iloc[:,:data.columns.get_loc('BUGFIX_count')]
    features_all = features_all.reindex(sorted(features_all.columns), axis=1)
    static_metrics = []
    for col_name in data.columns:
        if (col_name.startswith('SM_class') and col_name.endswith('sum')) or col_name.startswith('SM_file'):
            static_metrics.append(col_name)
    features_static = data[static_metrics]
    labels = data['BUGFIX_count']>0
    bug_matrix = data.iloc[:, data.columns.get_loc('BUGFIX_count')+1:]
    efforts = data['SM_file_lloc']
    return features_all, features_static, labels, bug_matrix, efforts

def load_all_data():
    datasets = {}
    filtered_jlmiv = set()
    filtered_szz = set()
    for project in projects:
        for release in releases[project]:
            version_name = project+'-'+release['version']
            features_all, features_static, labels_jlmiv, bug_matrix_jlmiv, efforts = load_data(version_name)
            _, _, labels_szz, bug_matrix_szz, _ = load_data(version_name, 'SZZ')
            _, _, labels_jlr, bug_matrix_jlr, _ = load_data(version_name, 'SZZ') # TODO TODO

            positive_mask = features_all>=0
            features_all[positive_mask] = np.log1p(features_all[positive_mask])
            features_all[~positive_mask] = -1*np.log(-1*features_all[~positive_mask]) # use log1p

            positive_mask = features_static>=0
            features_static[positive_mask] = np.log1p(features_static[positive_mask])
            features_static[~positive_mask] = -1*np.log1p(-1*features_static[~positive_mask])
            
            is_filtered = False
            if sum(labels_jlmiv)<6 or len(labels_jlmiv)<100:
                filtered_jlmiv.add(version_name)
                is_filtered = True
            if sum(labels_szz)<6 or len(labels_jlmiv)<100:
                filtered_szz.add(version_name)
                is_filtered = True
            
            if is_filtered:
                print("filtered %s: %i instances, %i bugs jlmiv, %i bugs szz, %i bugs jl+r, %i" % (version_name, len(labels_jlmiv), sum(labels_jlmiv), sum(labels_szz), sum(labels_jlr), len(bug_matrix_szz.columns)))
                continue
            else:
                print("adding %s: %i instances, %i bugs jlmiv, %i bugs szz, %i bugs jl+r, %i" % (version_name, len(labels_jlmiv), sum(labels_jlmiv), sum(labels_szz), sum(labels_jlr), len(bug_matrix_szz.columns)))
                datasets[version_name] = {'features_all': features_all, 'features_static': features_static,
                                          'labels_jlmiv': labels_jlmiv, 'bug_matrix_jlmiv': bug_matrix_jlmiv,
                                          'labels_szz': labels_szz, 'bug_matrix_szz': bug_matrix_szz,
                                          'efforts': efforts}
    return datasets, filtered_jlmiv, filtered_szz

def get_training_data(project, projects, releases, datasets):
    features_all_dfs = []
    features_static_dfs = []
    labels_jlmiv_dfs = []
    labels_szz_dfs = []
    for project_cur in projects:
        if project_cur == project:
            continue
        for release_cur in releases[project_cur]:
            version_name_cur = project_cur+'-'+release_cur['version']
            if version_name_cur not in datasets:
                continue
            features_all_dfs.append(datasets[version_name_cur]['features_all'])
            features_static_dfs.append(datasets[version_name_cur]['features_static'])
            labels_jlmiv_dfs.append(datasets[version_name_cur]['labels_jlmiv'])
            labels_szz_dfs.append(datasets[version_name_cur]['labels_szz'])
    features_all = pd.concat(features_all_dfs)
    features_static = pd.concat(features_static_dfs)
    labels_jlmiv = pd.concat(labels_jlmiv_dfs)
    labels_szz = pd.concat(labels_szz_dfs)
    return features_all, features_static, labels_jlmiv, labels_szz

datasets, filtered_jlmiv, filtered_szz = load_all_data()

prediction_scores = pd.DataFrame(columns=['f1_all_jlmiv',
                                          'precision_all_jlmiv',
                                          'recall_all_jlmiv',
                                          'clower_all_jlmiv',
                                          'cupper_all_jlmiv',
                                          'f1_static_jlmiv',
                                          'precision_static_jlmiv',
                                          'recall_static_jlmiv',
                                          'clower_static_jlmiv',
                                          'cupper_static_jlmiv',
                                          'f1_all_szz',
                                          'precision_all_szz',
                                          'recall_all_szz',
                                          'clower_all_szz',
                                          'cupper_all_szz',
                                          'f1_all_szz_szz',
                                          'precision_all_szz_szz',
                                          'recall_all_szz_szz',
                                          'clower_all_szz_szz',
                                          'cupper_all_szz_szz',
                                          'f1_static_szz',
                                          'precision_static_szz',
                                          'recall_static_szz',
                                          'clower_static_szz',
                                          'cupper_static_szz',
                                          'f1_static_szz_szz',
                                          'precision_static_szz_szz',
                                          'recall_static_szz_szz',
                                          'clower_static_szz_szz',
                                          'cupper_static_szz_szz',])



for project in projects:
    for release in releases[project]:
        version_name_test = project+'-'+release['version']
        if version_name_test in prediction_scores.index:
            print(version_name_test, "already available, skipping")
            continue
        if version_name_test not in datasets:
            continue # skipping because not loaded
        
        features_all_test = datasets[version_name_test]['features_all']
        features_static_test = datasets[version_name_test]['features_static']
        labels_jlmiv_test = datasets[version_name_test]['labels_jlmiv']
        bug_matrix_jlmiv_test = datasets[version_name_test]['bug_matrix_jlmiv']
        labels_szz_test = datasets[version_name_test]['labels_szz']
        bug_matrix_szz_test = datasets[version_name_test]['bug_matrix_szz']
        efforts_test = datasets[version_name_test]['efforts']
        
        print("starting %s: %i" % (version_name_test, sum(labels_jlmiv_test)))
        print("\tcreate training data")
        features_all_train, features_static_train, labels_jlmiv_train, labels_szz_train = get_training_data(project, projects, releases, datasets)
        
        features_all_train = features_all_train + features_all_train.median() - features_all_test.median()        
        features_static_train = features_static_train + features_static_train.median() - features_static_test.median()        
        
        clf = GaussianNB()
        #clf = XGBClassifier(random_state=42)
        # JLMIV
        clf.fit(features_all_train, labels_jlmiv_train)
        pred = clf.predict(features_all_test)
        
        scores = []
        scores.append(f1_score(labels_jlmiv_test, pred))
        scores.append(recall_score(labels_jlmiv_test, pred))
        scores.append(precision_score(labels_jlmiv_test, pred))
        scores.append(lower_bound(bug_matrix_jlmiv_test, efforts_test, pred))
        scores.append(upper_bound(bug_matrix_jlmiv_test, efforts_test, pred))
        
        clf.fit(features_static_train, labels_jlmiv_train)
        pred = clf.predict(features_static_test)
        pred_proba = clf.predict(features_static_test)
        
        scores.append(f1_score(labels_jlmiv_test, pred))
        scores.append(recall_score(labels_jlmiv_test, pred))
        scores.append(precision_score(labels_jlmiv_test, pred))
        scores.append(lower_bound(bug_matrix_jlmiv_test, efforts_test, pred))
        scores.append(upper_bound(bug_matrix_jlmiv_test, efforts_test, pred))
        
        # SZZ
        clf.fit(features_all_train, labels_szz_train)
        pred = clf.predict(features_all_test)
        
        scores.append(f1_score(labels_jlmiv_test, pred))
        scores.append(recall_score(labels_jlmiv_test, pred))
        scores.append(precision_score(labels_jlmiv_test, pred))
        scores.append(lower_bound(bug_matrix_jlmiv_test, efforts_test, pred))
        scores.append(upper_bound(bug_matrix_jlmiv_test, efforts_test, pred))
        # eval on SZZ labels
        scores.append(f1_score(labels_jlmiv_test, pred))
        scores.append(recall_score(labels_szz_test, pred))
        scores.append(precision_score(labels_szz_test, pred))
        scores.append(lower_bound(bug_matrix_szz_test, efforts_test, pred))
        scores.append(upper_bound(bug_matrix_szz_test, efforts_test, pred))
        
        clf.fit(features_static_train, labels_szz_train)
        pred = clf.predict(features_static_test)
        pred_proba = clf.predict(features_static_test)
        
        scores.append(f1_score(labels_jlmiv_test, pred))
        scores.append(recall_score(labels_jlmiv_test, pred))
        scores.append(precision_score(labels_jlmiv_test, pred))
        scores.append(lower_bound(bug_matrix_jlmiv_test, efforts_test, pred))
        scores.append(upper_bound(bug_matrix_jlmiv_test, efforts_test, pred))
        # eval on SZZ labels
        scores.append(f1_score(labels_jlmiv_test, pred))
        scores.append(recall_score(labels_szz_test, pred))
        scores.append(precision_score(labels_szz_test, pred))
        scores.append(lower_bound(bug_matrix_szz_test, efforts_test, pred))
        scores.append(upper_bound(bug_matrix_szz_test, efforts_test, pred))
        
        prediction_scores.loc[version_name_test,:] = scores

starting ant-ivy-1.4.1: 12
starting ant-ivy-2.0.0: 51
starting ant-ivy-2.1.0: 50
starting ant-ivy-2.2.0: 39
starting ant-ivy-2.3.0: 13
starting ant-ivy-2.4.0: 19
skipping archiva-1.0: 7
starting archiva-1.1: 33
starting archiva-1.2: 20
starting archiva-1.3: 15
starting archiva-2.0.0: 11
skipping archiva-2.1.0: 9
skipping archiva-2.2.0: 4
starting calcite-1.0.0: 185
starting calcite-1.1.0: 115
starting calcite-1.2.0: 127
starting calcite-1.3.0: 117
starting calcite-1.4.0: 125
starting calcite-1.5.0: 102
starting calcite-1.6.0: 108
starting calcite-1.7.0: 130
starting calcite-1.8.0: 103
starting calcite-1.9.0: 91
starting calcite-1.10.0: 86
starting calcite-1.11.0: 82
starting calcite-1.12.0: 82
starting calcite-1.13.0: 50
starting calcite-1.14.0: 53
starting calcite-1.15.0: 48
starting cayenne-3.0.0: 45
starting cayenne-3.1.0: 19
skipping commons-bcel-5.0: 0
skipping commons-bcel-5.1: 0
skipping commons-bcel-5.2: 1
starting commons-bcel-6.0: 13
skipping commons-bcel-6.1: 10
skipping commons-bcel-6.2: 9
skipping commons-beanutils-1.0: 0
skipping commons-beanutils-1.1: 0
skipping commons-beanutils-1.2: 0
skipping commons-beanutils-1.3: 1
skipping commons-beanutils-1.4: 1
skipping commons-beanutils-1.5: 1
skipping commons-beanutils-1.6: 1
skipping commons-beanutils-1.7.0: 7
skipping commons-beanutils-1.8.0: 8
skipping commons-beanutils-1.9.0: 1
skipping commons-codec-1.1: 0
skipping commons-codec-1.2: 2
skipping commons-codec-1.3: 3
skipping commons-codec-1.4: 0
skipping commons-codec-1.5: 1
skipping commons-codec-1.6: 4
skipping commons-codec-1.7: 3
skipping commons-codec-1.8: 6
skipping commons-codec-1.9: 6
skipping commons-codec-1.10: 3
skipping commons-codec-1.11: 0
skipping commons-collections-1.0: 1
skipping commons-collections-2.0: 1
skipping commons-collections-2.1: 1
skipping commons-collections-3.0: 7
skipping commons-collections-3.1: 8
starting commons-collections-3.2: 11
skipping commons-collections-3.3: 10
starting commons-collections-4.0: 14
skipping commons-collections-4.1: 1
starting commons-compress-1.0: 17
starting commons-compress-1.1: 14
starting commons-compress-1.2: 11
starting commons-compress-1.3: 17
starting commons-compress-1.4: 17
starting commons-compress-1.5: 21
starting commons-compress-1.6: 23
starting commons-compress-1.7: 27
starting commons-compress-1.8: 31
starting commons-compress-1.9: 24
starting commons-compress-1.10: 19
starting commons-compress-1.11: 13
skipping commons-compress-1.12: 10
starting commons-compress-1.13: 17
skipping commons-compress-1.14: 8
skipping commons-compress-1.15: 6
skipping commons-compress-1.16: 7
skipping commons-configuration-1.0: 7
skipping commons-configuration-1.1: 9
starting commons-configuration-1.2: 16
starting commons-configuration-1.3: 25
starting commons-configuration-1.4: 16
starting commons-configuration-1.5: 16
skipping commons-configuration-1.6: 10
starting commons-configuration-1.7: 11
starting commons-configuration-1.8: 14
starting commons-configuration-1.9: 14
skipping commons-configuration-1.10: 7
skipping commons-configuration-2.0: 10
skipping commons-configuration-2.1: 7
skipping commons-configuration-2.2: 7
skipping commons-dbcp-1.0: 1
skipping commons-dbcp-1.1: 8
starting commons-dbcp-1.2: 14
starting commons-dbcp-1.3: 11
starting commons-dbcp-1.4: 11
skipping commons-dbcp-2.0: 8
skipping commons-dbcp-2.1: 7
skipping commons-dbcp-2.2.0: 3
skipping commons-dbcp-2.3.0: 3
skipping commons-dbcp-2.4.0: 1
skipping commons-dbcp-2.5.0: 1
skipping commons-digester-1.0: 0
skipping commons-digester-1.1: 0
skipping commons-digester-1.2: 0
skipping commons-digester-1.3: 0
skipping commons-digester-1.4: 1
skipping commons-digester-1.5: 2
skipping commons-digester-1.6: 2
skipping commons-digester-1.7: 2
skipping commons-digester-1.8: 2
skipping commons-digester-2.0: 0
skipping commons-digester-2.1: 0
skipping commons-digester-3.0: 8
skipping commons-digester-3.1: 8
starting commons-digester-3.2: 11
skipping commons-io-1.0: 5
skipping commons-io-1.1: 7
skipping commons-io-1.2: 8
starting commons-io-1.3: 12
skipping commons-io-1.4: 10
skipping commons-io-2.0: 8
skipping commons-io-2.1: 5
skipping commons-io-2.2: 7
skipping commons-io-2.3: 7
skipping commons-io-2.4: 7
skipping commons-io-2.5: 4
skipping commons-jcs-1.0: 1
skipping commons-jcs-1.1: 2
starting commons-jcs-1.3: 12
starting commons-jcs-2.0: 13
starting commons-jcs-2.1: 12
skipping commons-jcs-2.2: 5
skipping commons-jexl-1.0: 0
skipping commons-jexl-1.1: 0
skipping commons-jexl-2.0: 5
skipping commons-jexl-2.1: 6
skipping commons-jexl-3.0: 0
skipping commons-jexl-3.1: 0
skipping commons-lang-1.0: 0
starting commons-lang-2.0: 12
starting commons-lang-2.1: 18
starting commons-lang-2.2: 23
starting commons-lang-2.3: 21
starting commons-lang-2.4: 22
skipping commons-lang-2.5: 10
skipping commons-lang-2.6: 8
starting commons-lang-3.0: 16
starting commons-lang-3.1: 20
starting commons-lang-3.2: 21
starting commons-lang-3.3: 17
starting commons-lang-3.4: 14
skipping commons-lang-3.5: 9
skipping commons-lang-3.6: 6
skipping commons-lang-3.7: 4
skipping commons-math-1.0: 6
skipping commons-math-1.1: 7
starting commons-math-1.2: 11
starting commons-math-2.0: 30
starting commons-math-2.1: 22
starting commons-math-2.2: 14
starting commons-math-3.0: 29
starting commons-math-3.1: 49
starting commons-math-3.2: 33
starting commons-math-3.3: 20
skipping commons-math-3.4: 8
skipping commons-math-3.5: 6
skipping commons-math-3.6: 0
skipping commons-net-1.0.0: 8
starting commons-net-1.1.0: 12
starting commons-net-1.2.0: 14
starting commons-net-1.3.0: 15
starting commons-net-1.4.0: 16
starting commons-net-2.0: 26
starting commons-net-2.1: 21
starting commons-net-2.2: 19
starting commons-net-3.0: 21
starting commons-net-3.1: 20
starting commons-net-3.2: 18
starting commons-net-3.3: 18
skipping commons-net-3.4: 9
skipping commons-net-3.5: 9
skipping commons-net-3.6: 3
starting commons-scxml-0.5: 13
starting commons-scxml-0.6: 14
starting commons-scxml-0.7: 16
starting commons-scxml-0.8: 16
skipping commons-scxml-0.9: 5
skipping commons-validator-1.0: 0
skipping commons-validator-1.1.0: 2
skipping commons-validator-1.2.0: 7
skipping commons-validator-1.3.0: 7
skipping commons-validator-1.4.0: 6
skipping commons-validator-1.5.0: 3
skipping commons-validator-1.6.0: 1
starting commons-vfs-1.0: 26
starting commons-vfs-2.0: 25
skipping commons-vfs-2.1: 4
skipping commons-vfs-2.2: 2
skipping deltaspike-0.1: 2
starting deltaspike-0.2: 13
starting deltaspike-0.3: 19
starting deltaspike-0.4: 25
starting deltaspike-0.5: 33
starting deltaspike-0.6: 30
starting deltaspike-0.7: 30
starting deltaspike-1.0.0: 29
starting deltaspike-1.1.0: 27
starting deltaspike-1.2.0: 25
starting deltaspike-1.3.0: 22
starting deltaspike-1.4.0: 25
starting deltaspike-1.5.0: 26
starting deltaspike-1.6.0: 14
starting deltaspike-1.7.0: 12
skipping deltaspike-1.8.0: 8
starting eagle-0.3.0: 18
skipping eagle-0.4.0: 2
skipping eagle-0.5.0: 0
skipping giraph-0.1.0: 0
starting giraph-1.0.0: 26
skipping giraph-1.1.0: 7
starting gora-0.1: 11
skipping gora-0.2: 9
skipping gora-0.3: 3
skipping gora-0.4: 9
starting gora-0.5: 13
skipping gora-0.6: 8
skipping gora-0.7: 0
skipping gora-0.8: 0
skipping jspwiki-1.4.0: 1
skipping jspwiki-1.5.0: 1
skipping jspwiki-1.6.0: 1
skipping jspwiki-1.7.0: 1
skipping jspwiki-1.8.0: 1
skipping jspwiki-2.0.36: 1
skipping jspwiki-2.2.19: 1
skipping jspwiki-2.4.56: 9
starting jspwiki-2.6.0: 17
skipping jspwiki-2.8.0: 4
skipping jspwiki-2.9.0: 4
skipping jspwiki-2.10.0: 3
starting knox-0.3.0: 37
starting knox-0.4.0: 35
starting knox-0.5.0: 49
starting knox-0.6.0: 69
starting knox-0.7.0: 33
starting knox-0.8.0: 33
starting knox-0.9.0: 25
starting knox-0.10.0: 27
starting knox-0.11.0: 26
starting knox-0.12.0: 22
starting knox-0.13.0: 12
skipping knox-0.14.0: 8
starting knox-1.0.0: 23
skipping kylin-0.6.1: 10
starting kylin-0.7.1: 27
starting kylin-1.0: 22
starting kylin-1.1: 17
starting kylin-1.2: 17
starting kylin-1.3: 11
starting kylin-1.5.0: 61
starting kylin-1.6.0: 38
starting kylin-2.0.0: 37
starting kylin-2.1.0: 54
starting kylin-2.2.0: 46
starting lens-2.6.0: 12
skipping lens-2.7.0: 4
starting mahout-0.1: 28
starting mahout-0.2: 25
starting mahout-0.3: 29
starting mahout-0.4: 74
starting mahout-0.5: 51
starting mahout-0.6: 55
starting mahout-0.7: 39
starting mahout-0.8: 34
starting mahout-0.9: 18
skipping mahout-0.10.0: 10
skipping mahout-0.11.0: 4
skipping mahout-0.12.0: 1
skipping mahout-0.13.0: 0
starting manifoldcf-0.1: 26
starting manifoldcf-0.2: 25
starting manifoldcf-0.3: 28
starting manifoldcf-0.4: 31
starting manifoldcf-0.5: 34
starting manifoldcf-0.6: 65
starting manifoldcf-1.0: 56
starting manifoldcf-1.1: 58
starting manifoldcf-1.2: 58
starting manifoldcf-1.3: 59
starting manifoldcf-1.4: 44
starting manifoldcf-1.5: 33
starting manifoldcf-1.6: 34
starting manifoldcf-1.7: 41
starting manifoldcf-1.8: 27
starting manifoldcf-1.9: 22
starting manifoldcf-1.10: 19
starting manifoldcf-2.0: 36
starting manifoldcf-2.1: 34
starting manifoldcf-2.2: 33
starting manifoldcf-2.3: 38
starting manifoldcf-2.4: 33
starting manifoldcf-2.5: 29
starting manifoldcf-2.6: 27
starting manifoldcf-2.7: 16
starting manifoldcf-2.8: 25
starting manifoldcf-2.9: 14
starting manifoldcf-2.10: 12
skipping nutch-0.7: 8
starting nutch-0.8: 29
starting nutch-0.9: 26
starting nutch-1.0: 37
starting nutch-1.1: 32
starting nutch-1.2: 25
starting nutch-1.3: 25
starting nutch-1.4: 26
starting nutch-1.5: 31
starting nutch-1.6: 28
starting nutch-1.7: 20
starting nutch-1.8: 11
skipping nutch-1.9: 8
starting nutch-1.10: 14
starting nutch-1.11: 14
starting nutch-1.12: 28
starting nutch-1.13: 26
skipping nutch-1.14: 8
starting nutch-2.0: 23
starting nutch-2.1: 19
starting nutch-2.2: 33
skipping nutch-2.3: 9
starting opennlp-1.7.0: 11
skipping opennlp-1.8.0: 5
skipping parquet-mr-1.0.0: 5
skipping parquet-mr-1.1.0: 5
skipping parquet-mr-1.2.0: 5
skipping parquet-mr-1.3.0: 7
starting parquet-mr-1.4.0: 15
starting parquet-mr-1.5.0: 18
starting parquet-mr-1.6.0: 12
starting parquet-mr-1.7.0: 28
starting parquet-mr-1.8.0: 35
starting parquet-mr-1.9.0: 37
skipping santuario-java-1.0.0: 0
skipping santuario-java-1.2: 0
skipping santuario-java-1.4.5: 2
starting santuario-java-1.5.9: 11
starting santuario-java-2.0.0: 20
skipping santuario-java-2.1.0: 1
starting systemml-0.9: 71
starting systemml-0.10: 72
starting systemml-0.11: 91
starting systemml-0.12: 87
starting systemml-0.13: 79
starting systemml-0.14: 71
starting systemml-0.15: 57
starting systemml-1.0.0: 44
starting systemml-1.1.0: 43
skipping systemml-1.2.0: 0
skipping tika-0.1: 5
starting tika-0.2: 15
starting tika-0.3: 16
starting tika-0.4: 13
starting tika-0.5: 11
starting tika-0.6: 17
starting tika-0.7: 21
starting tika-0.8: 33
starting tika-0.9: 45
starting tika-0.10: 43
starting tika-1.0: 42
starting tika-1.1: 35
starting tika-1.2: 42
starting tika-1.3: 32
starting tika-1.4: 34
starting tika-1.5: 20
starting tika-1.6: 24
starting tika-1.7: 22
starting tika-1.8: 19
starting tika-1.9: 28
starting tika-1.10: 23
starting tika-1.11: 28
starting tika-1.12: 27
starting tika-1.13: 23
starting tika-1.14: 25
starting tika-1.15: 26
starting tika-1.16: 25
starting tika-1.17: 17
starting wss4j-1.5.0: 13
starting wss4j-1.6.0: 13
starting wss4j-2.0.0: 11
skipping wss4j-2.1.0: 10
skipping wss4j-2.2.0: 0

Prediction Performance with Costs

scores_copy = prediction_scores.copy(deep=True).astype('float64')
scores_copy['clower_all_jlmiv'].replace(np.inf, 100000, inplace=True)
scores_copy['cupper_all_jlmiv'].replace(np.inf, 100000, inplace=True)
scores_copy['clower_static_jlmiv'].replace(np.inf, 100000, inplace=True)
scores_copy['cupper_static_jlmiv'].replace(np.inf, 100000, inplace=True)
scores_copy['clower_all_szz'].replace(np.inf, 100000, inplace=True)
scores_copy['cupper_all_szz'].replace(np.inf, 100000, inplace=True)
scores_copy['clower_static_szz'].replace(np.inf, 100000, inplace=True)
scores_copy['cupper_static_szz'].replace(np.inf, 100000, inplace=True)
scores_copy['clower_all_szz_szz'].replace(np.inf, 100000, inplace=True)
scores_copy['cupper_all_szz_szz'].replace(np.inf, 100000, inplace=True)
scores_copy['clower_static_szz_szz'].replace(np.inf, 100000, inplace=True)
scores_copy['cupper_static_szz_szz'].replace(np.inf, 100000, inplace=True)

scores_copy['c_all_jlmiv_errbar_range'] = abs((scores_copy['cupper_all_jlmiv']-scores_copy['clower_all_jlmiv'])/2)
scores_copy['c_all_jlmiv_errbar_midpoint'] = (scores_copy['cupper_all_jlmiv']+prediction_scores['clower_all_jlmiv'])/2
scores_copy['c_all_jlmiv_errbar_costsaving'] = scores_copy['cupper_all_jlmiv']-scores_copy['clower_all_jlmiv']>0
scores_copy['c_static_jlmiv_errbar_range'] = abs((scores_copy['cupper_static_jlmiv']-scores_copy['clower_static_jlmiv'])/2)
scores_copy['c_static_jlmiv_errbar_midpoint'] = (scores_copy['cupper_static_jlmiv']+prediction_scores['clower_static_jlmiv'])/2
scores_copy['c_static_jlmiv_errbar_costsaving'] = scores_copy['cupper_static_jlmiv']-scores_copy['clower_static_jlmiv']>0
scores_copy['c_all_szz_errbar_range'] = abs((scores_copy['cupper_all_szz']-scores_copy['clower_all_szz'])/2)
scores_copy['c_all_szz_errbar_midpoint'] = (scores_copy['cupper_all_szz']+prediction_scores['clower_all_szz'])/2
scores_copy['c_all_szz_errbar_costsaving'] = scores_copy['cupper_all_szz']-scores_copy['clower_all_szz']>0
scores_copy['c_static_szz_errbar_range'] = abs((scores_copy['cupper_static_szz']-scores_copy['clower_static_szz'])/2)
scores_copy['c_static_szz_errbar_midpoint'] = (scores_copy['cupper_static_szz']+prediction_scores['clower_static_szz'])/2
scores_copy['c_static_szz_errbar_costsaving'] = scores_copy['cupper_static_szz']-scores_copy['clower_static_szz']>0
scores_copy['c_all_szz_szz_errbar_range'] = abs((scores_copy['cupper_all_szz_szz']-scores_copy['clower_all_szz_szz'])/2)
scores_copy['c_all_szz_szz_errbar_midpoint'] = (scores_copy['cupper_all_szz_szz']+prediction_scores['clower_all_szz_szz'])/2
scores_copy['c_all_szz_szz_errbar_costsaving'] = scores_copy['cupper_all_szz_szz']-scores_copy['clower_all_szz_szz']>0
scores_copy['c_static_szz_szz_errbar_range'] = abs((scores_copy['cupper_static_szz_szz']-scores_copy['clower_static_szz_szz'])/2)
scores_copy['c_static_szz_szz_errbar_midpoint'] = (scores_copy['cupper_static_szz_szz']+prediction_scores['clower_static_szz_szz'])/2
scores_copy['c_static_szz_szz_errbar_costsaving'] = scores_copy['cupper_static_szz_szz']-scores_copy['clower_static_szz_szz']>0


f, axes = plt.subplots(2, 2, figsize=(9,4))

alpha = 0.05

df = scores_copy[['clower_all_jlmiv', 'clower_static_jlmiv', 'clower_all_szz', 'clower_static_szz', 'clower_all_szz_szz', 'clower_static_szz_szz']]
df.columns = ['IND-JLMIV+R-ALL', 'IND-JLMIV+R-SM', '6M-SZZ-ALL', '6M-SZZ-SM', '6M-ALL-SZZ-SZZ', '6M-SM-SZZ-SZZ']
ax = sns.boxplot(data=df, orient='h', color='lightgray', ax=axes[0,0])
ax.set_xscale('log')
ax.set_title('Lower boundary on cost ratio $C$')
sns.swarmplot(data=df, orient='h', color='gray', alpha=0.2, ax=ax)
res = autorank(df, alpha=alpha, order='ascending')
print(res.pvalue)
latex_table(res, decimal_places=10)
ax = plot_stats(res, ax=axes[1,0])

df = scores_copy[['cupper_all_jlmiv', 'cupper_static_jlmiv', 'cupper_all_szz', 'clower_static_szz', 'cupper_all_szz_szz', 'cupper_static_szz_szz']]
df.columns = ['IND-JLMIV+R-ALL', 'IND-JLMIV+R-SM', '6M-SZZ-ALL', '6M-SZZ-SM', '6M-ALL-SZZ-SZZ', '6M-SM-SZZ-SZZ']
ax = sns.boxplot(data=df, orient='h', color='lightgray', ax=axes[0,1])
ax.set_xscale('log')
sns.swarmplot(data=df, orient='h', color='gray',alpha=0.2,  ax=ax)
ax.set_title('Upper boundary on cost ratio $C$')
res = autorank(df, alpha=alpha)
create_report(res)
print(res.pvalue)
latex_table(res, decimal_places=10)
ax = plot_stats(res, ax=axes[1,1])
plt.subplots_adjust(left=None, bottom=None, right=None, top=None, wspace=0.8, hspace=None)
plt.savefig("cddiagram.pdf", bbox_inches='tight')

2.418517232964132e-17
\begin{table}[h]
\centering
\begin{tabular}{lrlllll}
\toprule
{} &           MR &             MED &             MAD &                                  CI &     $\delta$ &   Magnitude \\
\midrule
6M-SZZ-ALL      & 2.9901477833 & 2313.4166666667 & 1996.8645200000 &  [1632.7272727273, 3335.6250000000] & -0.0000000000 &  negligible \\
IND-JLMIV+R-ALL & 3.0665024631 & 2141.2857142857 & 1804.9331250000 &  [1613.4719101124, 3195.1250000000] &  0.0165255163 &  negligible \\
IND-JLMIV+R-SM  & 3.2315270936 & 2315.2328767123 & 1817.5186630137 &  [1758.2222222222, 3155.3043478261] & -0.0226406853 &  negligible \\
6M-SZZ-SM       & 3.3399014778 & 2185.0000000000 & 1754.3394000000 &  [1599.0000000000, 3066.2500000000] &  0.0045378437 &  negligible \\
6M-ALL-SZZ-SZZ  & 4.1059113300 & 2781.5737704918 & 2373.4096778689 &  [1913.7692307692, 4198.8518518519] & -0.1836006698 &       small \\
6M-SM-SZZ-SZZ   & 4.2660098522 & 2724.7500000000 & 2067.9133730769 &  [2169.1818181818, 4119.5384615385] & -0.2282268437 &       small \\
\bottomrule
\end{tabular}
\caption{Summary of populations}
\label{tbl:stat_results}
\end{table}
The statistical analysis was conducted for 6 populations with 203 paired samples.
The family-wise significance level of the tests is alpha=0.050.
We rejected the null hypothesis that the population is normal for the populations 6M-SZZ-ALL (p=0.000), IND-JLMIV+R-SM (p=0.000), 6M-ALL-SZZ-SZZ (p=0.000), IND-JLMIV+R-ALL (p=0.000), 6M-SZZ-SM (p=0.000), and 6M-SM-SZZ-SZZ (p=0.000). Therefore, we assume that not all populations are normal.
Because we have more than two populations and the populations and some of them are not normal, we use the non-parametric Friedman test as omnibus test to determine if there are any significant differences between the median values of the populations. We use the post-hoc Nemenyi test to infer which differences are significant. We report the median (MD), the median absolute deviation (MAD) and the mean rank (MR) among all populations over the samples. Differences between populations are significant, if the difference of the mean rank is greater than the critical distance CD=0.529 of the Nemenyi test.
We reject the null hypothesis (p=0.000) of the Friedman test that there is no difference in the central tendency of the populations 6M-SZZ-ALL (MD=2094.333+-682.409, MAD=1715.039, MR=3.296), IND-JLMIV+R-SM (MD=2221.500+-927.205, MAD=1827.601, MR=3.315), 6M-ALL-SZZ-SZZ (MD=1755.070+-760.875, MAD=1652.812, MR=3.360), IND-JLMIV+R-ALL (MD=2297.333+-874.065, MAD=1993.047, MR=3.384), 6M-SZZ-SM (MD=2185.000+-733.625, MAD=1754.339, MR=3.478), and 6M-SM-SZZ-SZZ (MD=1467.750+-501.082, MAD=1334.464, MR=4.167). Therefore, we assume that there is a statistically significant difference between the median values of the populations.
Based the post-hoc Nemenyi test, we assume that there are no significant differences within the following groups: 6M-SZZ-ALL, IND-JLMIV+R-SM, 6M-ALL-SZZ-SZZ, IND-JLMIV+R-ALL, and 6M-SZZ-SM. All other differences are significant.
5.132225138760148e-06
\begin{table}[h]
\centering
\begin{tabular}{lrlllll}
\toprule
{} &           MR &             MED &             MAD &                                  CI &     $\delta$ &   Magnitude \\
\midrule
6M-SZZ-ALL      & 3.2955665025 & 2094.3333333333 & 1715.0387333333 &  [1665.1818181818, 3030.0000000000] &  0.0000000000 &  negligible \\
IND-JLMIV+R-SM  & 3.3152709360 & 2221.5000000000 & 1827.6010200000 &  [1587.9230769231, 3442.3333333333] & -0.0554490524 &  negligible \\
6M-ALL-SZZ-SZZ  & 3.3596059113 & 1755.0697674419 & 1652.8122793146 &  [1270.5000000000, 2792.2500000000] &  0.0781140042 &  negligible \\
IND-JLMIV+R-ALL & 3.3842364532 & 2297.3333333333 & 1993.0468250000 &  [1684.6875000000, 3432.8181818182] & -0.0341915601 &  negligible \\
6M-SZZ-SM       & 3.4778325123 & 2185.0000000000 & 1754.3394000000 &  [1599.0000000000, 3066.2500000000] & -0.0017957242 &  negligible \\
6M-SM-SZZ-SZZ   & 4.1674876847 & 1467.7500000000 & 1334.4635500000 &  [1051.1481481481, 2053.3125000000] &  0.1952243442 &       small \\
\bottomrule
\end{tabular}
\caption{Summary of populations}
\label{tbl:stat_results}
\end{table}

Prediction Performance with Precision and Recall

f, axes = plt.subplots(2, 2, figsize=(9,4))

alpha = 0.05

df = scores_copy[['recall_all_jlmiv', 'recall_static_jlmiv', 'recall_all_szz', 'recall_static_szz', 'recall_all_szz_szz', 'recall_static_szz_szz']]
df.columns = ['IND-JLMIV+R-ALL', 'IND-JLMIV+R-SM', '6M-SZZ-ALL', '6M-SZZ-SM', '6M-ALL-SZZ-SZZ', '6M-SM-SZZ-SZZ']
ax = sns.boxplot(data=df, orient='h', color='lightgray', ax=axes[0,0])
ax.set_xscale('log')
ax.set_title('Recall')
sns.swarmplot(data=df, orient='h', color='gray', alpha=0.2, ax=ax)
res = autorank(df, alpha=alpha)
latex_table(res, decimal_places=10)
ax = plot_stats(res, ax=axes[1,0])

df = scores_copy[['precision_all_jlmiv', 'precision_static_jlmiv', 'precision_all_szz', 'precision_static_szz', 'precision_all_szz_szz', 'precision_static_szz_szz']]
df.columns = ['IND-JLMIV+R-ALL', 'IND-JLMIV+R-SM', '6M-SZZ-ALL', '6M-SZZ-SM', '6M-ALL-SZZ-SZZ', '6M-SM-SZZ-SZZ']
ax = sns.boxplot(data=df, orient='h', color='lightgray', ax=axes[0,1])
ax.set_xscale('log')
sns.swarmplot(data=df, orient='h', color='gray',alpha=0.2,  ax=ax)
ax.set_title('Precision')
res = autorank(df, alpha=alpha)
latex_table(res, decimal_places=10)
ax = plot_stats(res, ax=axes[1,1])
plt.subplots_adjust(left=None, bottom=None, right=None, top=None, wspace=0.8, hspace=None)
plt.savefig("cddiagram.pdf", bbox_inches='tight')

\begin{table}[h]
\centering
\begin{tabular}{lrlllll}
\toprule
{} &           MR &          MED &          MAD &                            CI &    $\delta$ &   Magnitude \\
\midrule
6M-ALL-SZZ-SZZ  & 2.9384236453 & 0.1861520999 & 0.1303766033 &  [0.1363636364, 0.2321428571] & 0.0000000000 &  negligible \\
IND-JLMIV+R-SM  & 3.0788177340 & 0.1428571429 & 0.0758595989 &  [0.1190476190, 0.1692307692] & 0.1825086753 &       small \\
6M-SM-SZZ-SZZ   & 3.4408866995 & 0.1641791045 & 0.1234327495 &  [0.1195652174, 0.2000000000] & 0.0723628334 &  negligible \\
6M-SZZ-ALL      & 3.4753694581 & 0.1348314607 & 0.0702748941 &  [0.1137206428, 0.1643835616] & 0.2207284816 &       small \\
IND-JLMIV+R-ALL & 4.0098522167 & 0.1224242424 & 0.0698139218 &  [0.1000000000, 0.1488372093] & 0.2689461040 &       small \\
6M-SZZ-SM       & 4.0566502463 & 0.1284403670 & 0.0748816803 &  [0.1032258065, 0.1584158416] & 0.2343905458 &       small \\
\bottomrule
\end{tabular}
\caption{Summary of populations}
\label{tbl:stat_results}
\end{table}
\begin{table}[h]
\centering
\begin{tabular}{lrlllll}
\toprule
{} &           MR &          MED &          MAD &                            CI &    $\delta$ &   Magnitude \\
\midrule
6M-SZZ-SM       & 2.5000000000 & 0.7500000000 & 0.2647500000 &  [0.6315789474, 0.8400000000] & 0.0000000000 &  negligible \\
IND-JLMIV+R-ALL & 2.8399014778 & 0.6666666667 & 0.2162125000 &  [0.5714285714, 0.7307692308] & 0.1664199568 &       small \\
IND-JLMIV+R-SM  & 3.4778325123 & 0.6875000000 & 0.3054430556 &  [0.6000000000, 0.8000000000] & 0.0871654250 &  negligible \\
6M-SM-SZZ-SZZ   & 3.7364532020 & 0.6747967480 & 0.2677945342 &  [0.5567010309, 0.7289719626] & 0.1611783834 &       small \\
6M-SZZ-ALL      & 4.0738916256 & 0.5263157895 & 0.2278522105 &  [0.4400000000, 0.6060606061] & 0.3258996821 &       small \\
6M-ALL-SZZ-SZZ  & 4.3719211823 & 0.5029940120 & 0.2325312298 &  [0.3875000000, 0.5882352941] & 0.3650901502 &      medium \\
\bottomrule
\end{tabular}
\caption{Summary of populations}
\label{tbl:stat_results}
\end{table}