Sequence and Fitness Datasets for Variant Fitness Prediction using Protein Language Models

This dataset bundle contains three sets: 1) domain sequences for pretraining, 2) domain sequences for finetuning and 3) variant fitness scores. Files are in lmdb format.

1. Domain sequences for pretraining

Two bz2 compressed files are provided:

• rp15_seq_lmdb.tar.bz2: representative proteome sequences at 15% level from Pfam-V32 database. Whole dataset is randomly split into train and validation sets: number of sequences in training set: 12,681,738; number of sequences in validation set: 1,042,103. Sequence length range from 18 to 500 (inclusive) and this length filtered set covers more than 95% sequences of the whole set.

• rp75_seq_lmdb.tar.bz2: representative proteome sequences at 75% level from Pfam-V32 database. Whole dataset is randomly split into train and validation sets: number of sequences in training set: 68,810,960; number of sequences in validation set: 5,687,282. Sequence length range from 18 to 500 (inclusive) and this length filtered set covers more than 95% sequences of the whole set.

Information of each sequence is stored as key-value pairs:

{
'primary': protein amino acid sequence,
'protein_length': length of the sequence,
'family': sequence Pfam family id (without 'PF'),
'clan': sequence Pfam clan id (without 'CL', -1 if not exists),
'unpIden': sequence Uniprot_id.version_number,
'range': domain residue start-end indices (follow indices of Uniprot seq),
'id': a index number for each sequence from 0 to N
}

One example:

{'primary': 'ALQTTDKHHVATPANWRPGDDVIVPPPATQEAAEERLREG',
 'protein_length': 40,
 'family': 10417,
 'clan': -1,
 'unpIden': 'A0A147JSN0.1',
 'range': '162-201',
 'id': '0'}

2. Domain sequences for finetuning

We collected homologous sequences of 33 proteins from [Shin2021]. The sequences are domain sequences queried over UniRef100 database. Each family is split into train and validation sets with ratio 9:1

Information of each sequence is stored as key-value pairs:

{
'unp_range': Uniprot record name/start index - end index (indices follow Uniprot seq),
'primary': protein amino acid sequence,
'seq_reweight': sequence weighting score from Shin2021,
'family_reweight': family weighting score from Shin2021 (sum of seq_reweight score for all family sequences),
'seq_reweight_mmseqs2':  sequence weighting score calculated by us using mmseqs2,
'family_reweight_mmseqs2': family weighting score based on seq_reweight_mmseqs2 (sum of seq_reweight_mmseqs2 score for all family sequences)
}

One example:

{
'unp_range': 'AMIE_PSEAE/1-346',
'primary': 'MRHGDISSSNDTVGVAVVNYKMPRLHTAAEVLDNARKIAEMIVGMKQGLPGMDLVVFPEYSLQGIMYDPAEMMETAVAIPGEETEIFSRACRKANVWGVFSLTGERHEEHPRKAPYNTLVLIDNNGEIVQKYRKIIPWCPIEGWYPGGQTYVSEGPKGMKISLIICDDGNYPEIWRDCAMKGAELIVRCQGYMYPAKDQQVMMAKAMAWANNCYVAVANAAGFDGVYSYFGHSAIIGFDGRTLGECGEEEMGIQYAQLSLSQIRDARANDQSQNHLFKILHRGYSGLQASGDGDRGLAECPFEFYRTWVTDAEKARENVERLTRSTTGVAQCPVGRLPYEGLEKEA',
'seq_reweight': 0.0714285714286,
'family_reweight': 19553.99941694187,
'seq_reweight_mmseqs2': 0.0021413276231263384,
'family_reweight_mmseqs2': 25236.560885598774
}

3. Variant fitness scores

This fitness benchmark set contains 42 mutagenesis sets, which were from originally curated by [DeepSequence] and later [Shin2021] used a subset of it. 

Information of each variant is stored as key-value pairs:

{
'set_nm': set name,
'wt_seq': WT sequence,
'seq_len': sequence length,
'mutants': amino acid variants list (could have multi-site mutations),
'mut_relative_idxs': list of relative amino acid indices for variants,
'mut_seq': mutant sequence,
'fitness': fitness score
}

One example:

{
'set_nm': 'AMIE_PSEAE_Whitehead',
'wt_seq': 'MRHGDISSSNDTVGVAVVNYKMPRLHTAAEVLDNARKIAEMIVGMKQGLPGMDLVVFPEYSLQGIMYDPAEMMETAVAIPGEETEIFSRACRKANVWGVFSLTGERHEEHPRKAPYNTLVLIDNNGEIVQKYRKIIPWCPIEGWYPGGQTYVSEGPKGMKISLIICDDGNYPEIWRDCAMKGAELIVRCQGYMYPAKDQQVMMAKAMAWANNCYVAVANAAGFDGVYSYFGHSAIIGFDGRTLGECGEEEMGIQYAQLSLSQIRDARANDQSQNHLFKILHRGYSGLQASGDGDRGLAECPFEFYRTWVTDAEKARENVERLTRSTTGVAQCPVGRLPYEG',
'seq_len': 341,
'mutants': ['M1W'],
'mut_relative_idxs': [0],
'mut_seq': 'WRHGDISSSNDTVGVAVVNYKMPRLHTAAEVLDNARKIAEMIVGMKQGLPGMDLVVFPEYSLQGIMYDPAEMMETAVAIPGEETEIFSRACRKANVWGVFSLTGERHEEHPRKAPYNTLVLIDNNGEIVQKYRKIIPWCPIEGWYPGGQTYVSEGPKGMKISLIICDDGNYPEIWRDCAMKGAELIVRCQGYMYPAKDQQVMMAKAMAWANNCYVAVANAAGFDGVYSYFGHSAIIGFDGRTLGECGEEEMGIQYAQLSLSQIRDARANDQSQNHLFKILHRGYSGLQASGDGDRGLAECPFEFYRTWVTDAEKARENVERLTRSTTGVAQCPVGRLPYEG',
'fitness': -0.5174
}

Reference

DeepSequence: Riesselman, Adam J., John B. Ingraham, and Debora S. Marks. "Deep generative models of genetic variation capture the effects of mutations." Nature methods 15.10 (2018): 816-822.

Shin2021:Shin, Jung-Eun, et al. "Protein design and variant prediction using autoregressive generative models." Nature communications 12.1 (2021): 1-11.