kundajelab/tfmodisco: (in dev) new spurious merging detection, on-the-fly flank filling, exploring different merging criteria

Corresponds to PR https://github.com/kundajelab/tfmodisco/pull/88. The previous edition of this tag did not specify the correct target (i.e. the dev branch) so I'm re-doing it.

Three main changes:

• "DynamicDistanceSimilarPatternsCollapser" had a component that involved comparing within-cluster similarities to between-cluster similarities, and looking at a metric derived from the auROC for differentiating within-cluster motifs from between-cluster motifs. Previously, for a pair of motif, the average of this auROC-derived similarity was taken. However, that can cause motifs with low within-cluster similarity to "swallow" motifs with high within-cluster similarity (evidence in this notebook: https://nbviewer.jupyter.org/github/kundajelab/tfmodisco_bio_experiments/blob/1f633b03c22860e5822e9ca5b013e68d0286332c/bpnet/trial1/TryBpNet_v0.5.14.0_studymergecriteria.ipynb - look at the pairwise similarities in the "Cross-contamination matrix" and notice the similarities of the first motif to everything else are very high even though the reciprocal similarities are very low). Thus, I decided to take the min rather than the average.
• Previously, when seqlets were merged together in a greedy fashion, the on-the-fly aggregated motif only averaged the importance scores over the seqlets that aligned to a particular position, and then only at the end would "expand" all the seqlets to fill out the full motif. The caveat here is that for motifs like Nanog, which can have two instances in the same seqlet (due to periodic binding), it can create two "modes" in the on-the-fly aggregated seqlet, and each mode looks very strong as its being computed on-the-fly (because the aggregation is only done over the seqlets that align to the position), but when the flank expansion is conducted at the very end, both modes end up with degraded information content (because the seqlets that were aligning to one mode don't always turn out to have a motif instance at the other mode after expansion). The solution is to peform the flank expansion on-the-fly. Compare the IC of the motifs, esp the nanog motif, in https://nbviewer.jupyter.org/github/kundajelab/tfmodisco_bio_experiments/blob/1f633b03c22860e5822e9ca5b013e68d0286332c/bpnet/trial1/TryBpNet_v0.5.14.0_ontheflyflankfill.ipynb, which has on-the-fly flank filling, vs. https://nbviewer.jupyter.org/github/kundajelab/tfmodisco_bio_experiments/blob/1f633b03c22860e5822e9ca5b013e68d0286332c/bpnet/trial1/TryBpNet_v0.5.14.0_noontheflyflankfill.ipynb, which didn't have the flank filling)
• "Spurious merging detection" is now performed by running the subclustering procedure for each motif, immediately followed by the "DynamicDistanceSimilarPatternsCollapser" procedure on the submotifs. Thus, DynamicDistanceSimilarPatternsCollapser is run in two places: during the spurious merging detection and during the overall redundancy reduction step. This substantially increases the diversity of motifs returned (contrast the above results with previous 0.5.13.0 results at: https://nbviewer.jupyter.org/github/kundajelab/tfmodisco_bio_experiments/blob/b3b4d7b240b8e398597100581ae791eec0a13b61/bpnet/trial1/TryBpNet_v0.5.13.0.ipynb)

However, I'm not fully satisfied with the motif merging criteria which is why this will be an "in dev" release