ER-BERT: a pre-trained model for the de novo generation of epitope-binding TCR

T-cell receptor (TCR), the essential entity of the adaptive immune system, determines the human body's immune response to diseases. Generating TCR sequences with desired properties is critical to developing advanced immunotherapies, but still challenging due to the diverse nature of TCR. The recent advancements in high-throughput sequencing technologies and deep learning-based sequence modeling approaches have made it possible to generate TCR sequences with desired properties. We propose an advanced deep learning framework, namely the ER-BERT, for the \textit{de novo} generation of epitope-binding TCR sequences. ER-BERT is composed of EpitopeBERT and ReceptorBERT and takes full advantage of the pre-training and fine-tuning paradigm. Both EpitopeBERT and ReceptorBERT are based on the popular BERT architecture and are trained on large datasets of epitope and TCR sequences, respectively, to learn the underlying patterns present in these sequences. Novel TCRs generated by ER-BERT show great binding specificity and similar sequence and structure compared to naturally-existed TCRs. ER-BERT represents a powerful general tool for studying the human immune system with great practical potential for developing advanced immunotherapies. 

Here, we provide the model files of ER-BERT: ER-BERT with the UAA tokenizer and ER-BERT with the FMFM tokenizer.