# augur-results
This repository contains the results of the main experiments from the Augur project. The main code repository for Augur is not yet available. We are going through the code marking and internal approvals process that is required to make the code open source. The process will be complete in time for camera-ready submission if the paper is accepted.

## Contents

This repository contains the results of the experiments executed for the CAIN paper using the Augur toolset. More specifically, inside the /results folder, there is one ZIP file that contains the inputs files and results for each experiment run using the Predictor tool. Each of these ZIP files includes the following inputs used for the experiments:

 - **trained_model**: a folder containing the exported trained Tensorflow SaveModel generated by the Trainer tool and used as input for the Predictor tool.
 - **trained_model_ts**: the trained time-series model as a Pickle file, generated by the Trainer tool and used as input for the Predictor tool.
 - **predictor_config**: a folder containing the configuration file used by the Predictor when running an experiment.
 - **<exp_id>_drift.json**: the drifted dataset generated by the Drifter for an experiment. This drifted dataset was generated using the configuration file found in the main augur-code repository, in the path /experiments/drifter/<exp_id>_drift.json

Each ZIP file also includes the following outputs:
 - **<exp_id>_predictions.json**: the predictions generated by the Predictor for this experiment.
 - **<exp_id>_metrics.json**: the metrics generated by the Predictor for this experiment.
 - **predictor.log**: the log of the Predictor running this experiment.

## Source Data
The base dataset used by the Trainer and Drifter tools for these experiments was obtained from the Kaggle Statoil/C-CORE Iceberg Classifier Challenge (https://www.kaggle.com/c/statoil-iceberg-classifier-challenge/data). The process to generate the dataset tht we used from the files available on that page is the following:

 - The train.json and test.json files were downloaded from the project page.
 - The Trainer tool was used to train a model based on the train.json data (using the /tools/trainer.sh script in the main augur-code repo).
 - The test.json file, which is unlabeled, was labeled by using the labeller tool in the main augur-code repo (/tools/labeller.sh), and the model trained in the previous step. This added labels to the test.json file.
 - Both the train.json file and the labeled test.json file were merged into one dataset, using the merger tool (using /tools/merger.sh in the main augur-code repo).
 - The resulting merged dataset in JSON was used as the input for the Drifter (to generate the drifted datasets) and the Predictor (to run the experiments which results are in this repository).