Container for the BICCN Enhancer Label classification task using the Genomic API for Model Evaluation (GAME) Framework

BICCN Validated Enhancer Dataset

  This Zenodo record describes an evaluator container for the Genomic API for Model Evaluation (GAME). The system assesses computational
   predictions of cell type-specific chromatin accessibility against experimentally validated enhancers from the mouse motor cortex.

  Dataset Composition

  The evaluation dataset comprises 171 experimentally validated cis-regulatory elements from the BRAIN Initiative Cell Census Network
  (BICCN) across 19 mouse motor cortex cell types:

  - Astrocytes (Astro)
  - Endothelial cells (Endo)
  - Excitatory neurons: L2/3 IT, L5 ET, L5 IT, L5/6 NP, L6 CT, L6 IT, L6b
  - Inhibitory neurons: Lamp5, Pvalb, Sncg, Sst, Sst Chodl, Vip
  - Glial cells: Microglia/PVM, OPC, Oligodendrocytes
  - Vascular cells: VLMC

  Each enhancer sequence is annotated with its target cell type, specificity classification (on-target only),
  and activity strength (strong or weak). Sequences range from approximately 200-500 base pairs and were functionally validated using
  massively parallel reporter assays (MPRA).

  Evaluation Metrics

  Cell Type Classification: The evaluator computes multiclass classification metrics to assess the model's ability to correctly predict
  which cell type each enhancer regulates:

  - Accuracy: Overall fraction of correctly classified enhancers
  - Precision (weighted): Weighted average precision across cell types, accounting for class imbalance
  - Recall (weighted): Weighted average recall across cell types
  - F1-score (weighted): Weighted harmonic mean of precision and recall

  These metrics quantify how well computational models can predict cell type-specific regulatory activity from DNA sequence alone.

  Container Contents

  The deepbiccn2_enhancerlabel_evaluator.sif file includes:
  - Validated enhancer dataset with sequences and cell type labels
  - Data processing scripts for sequence preparation
  - Predictor connection tools for GAME API communication
  - Metrics calculation scripts implementing weighted classification metrics
  - All required dependencies and Python packages

  Data Files

  - biccn_enhancers_withSequence.csv: Tab-separated file containing 171 validated enhancers with genomic coordinates, sequences, target
  cell types, specificity annotations, and activity measurements

  Execution Command

  apptainer run --containall \
    -B /path_to/prediction_folder/:/predictions \
    deepbiccn2_enhancerlabel_evaluator.sif \
    PREDICTOR_HOST PREDICTOR_PORT /predictions

  Citations

  This evaluator is based on validated enhancer data from:

  Johansen, N.J., Kempynck, N., Zemke, N.R., Somasundaram, S., De Winter, S., Hooper, M., Dwivedi, D., Lohia, R., Wehbe, F., Li, B.,
  Abaffyová, D., Armand, E.J., De Man, J., Ekşi, E.C., Hecker, N., Hulselmans, G., Konstantakos, V., Mauduit, D., Mich, J.K., Partel,
  G., Daigle, T.L., Levi, B.P., Zhang, K., Tanaka, Y., Gillis, J., Ting, J.T., Ben-Simon, Y., Miller, J., Ecker, J.R., Ren, B., Aerts,
  S., Lein, E.S., Tasic, B., and Bakken, T.E. (2025). Evaluating methods for the prediction of cell-type-specific enhancers in the
  mammalian cortex. Cell Genomics 5(6), 100879. https://doi.org/10.1016/j.xgen.2025.100756