PlasEval: a framework for comparing and evaluating plasmid binning tools

PlasEval is a tool aimed at evaluating the accuracy and at comparing methods for the problem of plasmid binning. It has two modes: evaluate and compare.

In the evaluation mode of PlasEval, a given set of predicted plasmid bins resulting from a plasmid binning method is evaluated against a ground truth set of plasmid bins, yielding three statistics, the precision, the recall and the F1-score.

In the comparison mode of PlasEval, given two sets of predicted plasmid bins (either resulting from two plasmid binning tools or from a plasmid binning tool and a ground truth), PlasEval computes a dissimilarity measure that indicates how much both sets of predicted plasmid bins are in agreement.

The data shared consists of output files of PlasBin-flow as well as those of 3 other plasmid binning methods, namely, HyAsP, MOB-recon and gplas for 53 test samples. Details about each sample have been provided in the file Ecoli_samples.csv.  It also includes the short read assembly files (fasta as well as gfa) and the hybrid assembly files. The mappings of short-read contigs to hybrid contigs for all 53 samples have been provided in the BLAST_output folder. The plasmid bins, including those representing the ground truth, used as input for PlasEval have been provided in the PlasEval_input folder.