Benchmark Data for: RNAtive to recognize native-like structure in a set of RNA 3D models

This repository contains the complete dataset, analysis scripts, and figures used for the benchmark evaluation in the paper “RNAtive to recognize native-like structure in a set of RNA 3D models” by Pielesiak et al. (2025).

The resources provided here allow for the full reproduction of the comparative analysis between the RNAtive tool and other state-of-the-art methods for RNA 3D structure quality assessment.

Repository Contents

• casp/ and decoys/: These directories contain the processed evaluation scores, organized into comma-separated value (CSV) files.
  • Each CSV file is dedicated to a single RNA target.
  • Within each file, rows correspond to individual predicted 3D models, and columns represent distinct evaluation scores. These scores include both structural quality metrics, which compare a model to its reference structure, and reference-free scores, which are calculated independently for each model.
  • decoys/excluded.txt: This file lists the randstr targets that were removed from the final analysis. These targets were excluded because their models contained zero base pairs, a structural defect that prevents consensus analysis with the RNAtive tool.
• rnative-figures.ipynb and requirements.txt: This Jupyter notebook (with pinned requirements versions) implements the following data processing and analysis workflow:
  1. Imports and processes the data from the relevant CSV files.
  2. Selects a non-redundant subset of structural quality metrics for analysis. To achieve this, the script calculates a correlation matrix for all metrics, converts it to a distance matrix, and performs hierarchical clustering using the Unweighted Pair Group Method with Arithmetic Mean (UPGMA).
  3. Computes four rank-based similarity measures to assess scoring performance: Spearman’s rank correlation coefficient (ρ), Kendall’s rank correlation coefficient (τ), Enrichment Scores (ES), and Rank-Biased Overlap (RBO).
  4. Conducts a Principal Component Analysis (PCA) to synthesize the four similarity measures into a single composite score. This score is derived from the first principal component (PC1), and its validity is confirmed by ensuring all variable loadings on PC1 are positive and that PC1 accounts for a sufficient fraction of the total variance.
  5. Generates visual plots for each dataset, ranking the structural quality metrics by their correlation with the reference-free score.
  6. Performs statistical significance testing on each dataset using the Friedman test, followed by pairwise comparisons with a post-hoc Nemenyi test.
• figures/: This directory contains the study’s graphical outputs. These figures include: (1) dendrograms produced by hierarchical clustering (*-dendrogram.pdf); (2) Critical Difference (CD) diagrams from post-hoc Nemenyi tests (*-nemenyi-cd.pdf); and (3) visual rankings plotted against selected, non-redundant metrics (casp15-{inf-non-canonical,lddt,rmsd,tm-score}.pdf, randstr-{inf-canonical,inf-non-canonical,rmsd-all}.pdf).

Data Structure (CSV Columns)

The columns contain a mix of reference-based metrics–including pre-computed values from the original datasets (e.g., RMSD) and new scores calculated for this study against a known reference structure (e.g., INF canonical)–and scores from state-of-the-art reference-free tools, including the RNAtive variants reported here.

CASP15 Data (casp/)

ID,RMSD,LDDT,TM_score,GDT_TS,INF canonical,INF non-canonical,INF all,3dRNAscore,ARES,DFIRE,PAMNet,RASP,RNA-BRiQ,RNA3DCNN_MD,RNA3DCNN_MDMC,cgRNASP,cgRNASP-C,cgRNASP-CN,cgRNASP-PC,lociPARSE,rsRNASP,RNAtive Barnaba canonical,RNAtive BPNet canonical,RNAtive FR3D canonical,RNAtive MC-Annotate canonical,RNAtive RNApolis canonical,RNAtive RNAView canonical,RNAtive Barnaba all,RNAtive BPNet all,RNAtive FR3D all,RNAtive MC-Annotate all,RNAtive RNApolis all,RNAtive RNAView all,Target

Randstr Decoys Data (decoys/)

ID,RMSD-C3',GDT-C3',RMSD-ALL,GDT-ALL,INF canonical,INF non-canonical,INF all,3dRNAscore,ARES,DFIRE,PAMNet,RASP,RNA-BRiQ,RNA3DCNN_MD,RNA3DCNN_MDMC,cgRNASP,cgRNASP-C,cgRNASP-CN,cgRNASP-PC,lociPARSE,rsRNASP,RNAtive Barnaba canonical,RNAtive BPNet canonical,RNAtive FR3D canonical,RNAtive MC-Annotate canonical,RNAtive RNApolis canonical,RNAtive RNAView canonical,RNAtive Barnaba all,RNAtive BPNet all,RNAtive FR3D all,RNAtive MC-Annotate all,RNAtive RNApolis all,RNAtive RNAView all,Target

How to Use

To reproduce the analysis and figures from the paper, run the Jupyter notebook (rnative-figures.ipynb). The notebook will load the data from the corresponding CSV files and generate the plots found in the figures/ directory.

Citation

If you use this dataset or the analysis scripts, please cite our paper and this Zenodo record.