Published February 9, 2025
| Version v2.1
Model
Open
ProtRNA Model Weights and Datasets
Authors/Creators
- 1. Fudan University
- 2. Shanghai AI Laboratory
Description
Model Weights
(TensorFlow)
- the pretrained ProtRNA language model (ProtRNA_pretrained.h5)
- the adapted RotaFormer secondary structure prediction head
- trained on bpRNA1m, selected under RnaBench [1] metrics (ssHead_RF_rnabench_bprna.h5)
- trained on RnaBench intra_family dataset, validated and selected with RnaBench metrics (ssHead_RF_rnabench_intra.h5)
- trained on RnaBench inter_family dataset, validated and selected with RnaBench metrics (ssHead_RF_rnabench_inter.h5)
- trained on bpRNA1m, validated and selected with ProtRNA metrics (ssHead_RF_bprna.h5)
(Torch)
- the protein-RNA interaction prediction heads trained for 17 RBPs in Hela cell, and their evaluation outputs (out_rbp.zip), using the model architecture provided in [2]
- the mean ribosome loading prediction head (mrlHead.ckpt), using the model architecture provided in [3]
Datasets
- the datasets for secondary structure prediction (data_ss.zip) contains:
- RnaBench datasets for RNA Folding: inter-family, intra-family (.plk)
- bpRNA-1m [4] datasets with train-val-test split proposed in SPOT-RNA [5], cleaned for usage: TR0, VL0, TS0 (.pickle)
- Restructured VL0, TS0 for evaluation using the RnaBench pipeline (bprna_TS0.plk, bprna_VL0.plk)
- the datasets for protein-RNA interaction prediction (data_rbp.zip) contains:
- a list of the 17 RBP names in Hela cell (Hela.lst)
- in (clip_data), the RNA sequences, icSHAPE RNA structural information and their interaction status with 17 RBPs, provided by [2] (.tsv)
- the dataset for mean ribosome loading prediction (data_mrl.zip) contains:
- a large-scale synthetic Human 5′ UTR library comprising 5′ UTR sequences with measured MRL values, provided by [6]. It includes human and random UTR sequences of varying lengths, with 100 sequences per length (from 25 to 100 nucleotides) selected based on deep read coverage. Upon the original dataset, a "split" column is added to indicate the employed train-eval split (GSM4084997_varying_length_25to100.csv.gz)
Abstract
Protein language models (PLM), such as the highly successful ESM-2, have proven particularly effective. However, language models designed for RNA continue to face challenges. A key question is: can the information derived from PLMs be harnessed and transferred to RNA? To investigate this, a model termed ProtRNA has been developed by cross-modality transfer learning strategy for addressing the challenges posed by RNA's limited and less conserved sequences. By leveraging the evolutionary and physicochemical information encoded in protein sequences, the ESM-2 model is adapted to processing "low-resource" RNA sequence data. The results show comparable or superior performance in various RNA downstream tasks, with only 1/8 the trainable parameters and 1/6 the training data employed by the primary reference baseline RNA language model. This approach highlights the potential of cross-modality transfer learning in biological language models.
Files
data_mrl.zip
Files
(3.0 GB)
| Name | Size | |
|---|---|---|
|
md5:ee8df42f5c75bfd24f16bbed3d11f1b0
|
15.5 MB | Preview Download |
|
md5:53600ca718ca96f93d65136ef60c0f09
|
33.3 MB | Preview Download |
|
md5:75b159a4edf20a694220eee9bd149dba
|
15.8 MB | Preview Download |
|
md5:9ecfb54dfae3bba352c279c4b57d9423
|
958.4 kB | Download |
|
md5:1acbd06a14d0a5f106845ddc0ef988a8
|
6.5 MB | Preview Download |
|
md5:8be17d223d49d93e0079070fe4a1b034
|
2.6 GB | Download |
|
md5:c50409f402ea97cbaebf1579ef19496a
|
75.7 MB | Download |
|
md5:65ae487e3a1f05bf5c97b45cbc4a41e8
|
75.7 MB | Download |
|
md5:ba6451275f49f1a986011040cc843a11
|
75.7 MB | Download |
|
md5:258b205a200bc8e6e9cc1607d7a43302
|
75.7 MB | Download |
Additional details
Additional titles
- Subtitle
- Model weights and datasets for "ProtRNA: A Protein-derived RNA Language Model by Cross-Modality Transfer Learning"
Dates
- Updated
-
2025-02-09
Software
- Repository URL
- https://github.com/roxie-zhang/ProtRNA
- Programming language
- Python
References
- [1] Runge, F., et al., Rnabench: A comprehensive library for in silico rna modelling. bioRxiv, 2024: p. 2024.01. 09.574794.
- [2] Sun, L., et al., Predicting dynamic cellular protein–RNA interactions by deep learning using in vivo RNA structures. Cell research, 2021. 31(5): p. 495-516.
- [3] Penić, R.J., et al., Rinalmo: General-purpose rna language models can generalize well on structure prediction tasks. arXiv preprint arXiv:2403.00043, 2024.
- [4] Danaee, P., et al., bpRNA: large-scale automated annotation and analysis of RNA secondary structure. Nucleic acids research, 2018. 46(11): p. 5381-5394.
- [5] Singh, J., et al., RNA secondary structure prediction using an ensemble of two-dimensional deep neural networks and transfer learning. Nature Communications, 2019. 10.
- [6] Sample, P.J., et al., Human 5′ UTR design and variant effect prediction from a massively parallel translation assay. Nature biotechnology, 2019. 37(7): p. 803-809.