We, the students of MICI5029/5049, a Graduate Level Molecular Pathogenesis Journal Club at Dalhousie University in Halifax, NS, Canada, hereby submit a review of the following BioRxiv preprint:
Gordon et al. A SARS-CoV-2-Human Protein-Protein Interaction Map Reveals Drug Targets and Potential Drug-Repurposing. BioRxiv doi: https://doi.org/10.1101/2020.03.22.002386.
We will adhere to the Universal Principled (UP) Review guidelines proposed in:
Universal Principled Review: A Community-Driven Method to Improve Peer Review. Krummel M, Blish C, Kuhns M, Cadwell K, Oberst A, Goldrath A, Ansel KM, Chi H, O'Connell R, Wherry EJ, Pepper M; Future Immunology Consortium. Cell. 2019 Dec 12;179(7):1441-1445. https://doi.org/10.1016/j.cell.2019.11.029
SUMMARY: The current COVID-19 pandemic has motivated a worldwide effort to discover and develop vaccines and antivirals for SARS-CoV-2. In this manuscript, Gordon et al. aim report on the protein-protein interactome between SARS-CoV-2 proteins and human proteins to identify potential candidate drugs and their target human proteins. The authors used an affinity purification mass spectrometry (AP-MS) approach, involving 26 Strep-tagged viral proteins expressed in human HEK293T cells. Cell lysates were affinity purified and subjected to mass spectrometry. 332 high-confidence SARS-CoV-2-human protein-protein interactions (PPIs) were found. Among these, 67 human proteins were known to be targeted by existing drugs. As expected, SARS-CoV-2 proteins are connected to a wide array of biological processes. This study provides a methodology that can be adopted to study of the interactome of other viruses. Importantly, Gordon et al. provide valuable data that can become the foundation for further mechanistic studies and, ultimately, clinical trials.
OVERALL ASSESSMENT
STRENGTHS: This manuscript is the result of a rapid response from a large international team of researchers; their report provides a strong starting point for future drug development and drug-repurposing. Confidence in the interactome is bolstered by agreement with previous findings of viral targeting of host proteins involved in key nodes of control, both from previous coronavirus studies and other viruses studied by this group. Overall, this manuscript is well written and the figures are described in appropriate detail. Also, all raw data is available for other researchers to readily use.
WEAKNESSES: The primary weakness of the study are that the authors did not confirm any protein-protein interactions using standard confirmatory methods like co-immunoprecipitation. This would obviously involve a great deal of labor, but some key interactions that are being used to inform selection of candidate FDA-approved drugs should be confirmed. The manuscript would also be strengthened by testing some high-priority candidate drugs in an in vitro SARS-CoV-2 infection assay like plaque assay or TCID50.
DETAILED UP ASSESSMENT (“1” represents the highest quality)
OBJECTIVE CRITERIA (QUALITY)
1. Quality: Experiments (1–3 scale) SCORE = 1
Figure by figure, do experiments, as performed, have the proper controls?
o Proper controls were employed in each of the figures.
o We were impressed with the Extended Data Figure 1, which demonstrates high correlation of the replicates.
Are specific analyses performed using methods that are consistent with answering the specific question?
o Yes.
Is there the appropriate technical expertise in the collection and analysis of data presented?
o Yes.
Do analyses use the best-possible (most unambiguous) available methods quantified via appropriate statistical comparisons?
o The clustering method seems to be in line with others.
o The GO method (clusterProfiler package) is commonly used for this kind of analysis.
o We suggest uploading the code to a public repository.
Are controls or experimental foundations consistent with established findings in the field? A review that raises concerns regarding inconsistency with widely reproduced observations should list at least two examples in the literature of such results. Addressing this question may occasionally require a supplemental figure that, for example, re-graphs multi-axis data from the primary figure using established axes or gating strategies to demonstrate how results in this paper line up with established understandings. It should not be necessary to defend exactly why these may be different from established truths, although doing so may increase the impact of the study and discussion of discrepancies is an important aspect of scholarship.
o Controls and experiments used here are in line with field standards for protein-protein interactions.
2. Quality: Completeness (1–3 scale) SCORE = 2
Does the collection of experiments and associated analysis of data support the proposed title- and abstract-level conclusions? Typically, the major (title- or abstract-level) conclusions are expected to be supported by at least two experimental systems.
o This study aimed to find potential drug targets and they used one approach (AP-MS) and one system (HEK293T cells). However, the dataset appears to be robust and the authors used a bioinformatics approach to validate their findings in the context of previously published data (Fig 2c-e).
o Nevertheless, we consider that the limitations of using HEK293T over a lung cell line should be further clarified/discussed.
o This paper has a strong bioinformatics component but lacks confirmatory biochemical analysis of the interactions.
o A few protein targets could have been tested in vitro to evaluate the relevance of these interactions to viral life cycle (e.g. entry).
o We suggest that the authors could re-evaluate title and maybe change it to “A SARS-CoV-2-Human Protein-Protein Interaction Map Reveals Potential Drug Targets”
Are there experiments or analyses that have not been performed but if ‘‘true’’ would disprove the conclusion (sometimes considered a fatal flaw in the study)? In some cases, a reviewer may propose an alternative conclusion and abstract that is clearly defensible with the experiments as presented, and one solution to ‘‘completeness’’ here should always be to temper an abstract or remove a conclusion and to discuss this alternative in the discussion section.
o The results support the conclusions. However, the lack of biochemical confirmation of the interactions and their role in SARS-CoV-2 infection needs to be clearly stated in the Discussion. We are worried that these druggable targets that haven't been validated in any way could be taken as definitive treatment alternatives by the lay public and medical professionals tempted by off-label use.
3. Quality: Reproducibility (1–3 scale) SCORE = 1.5
Figure by figure, were experiments repeated per a standard of 3 repeats or 5 mice per cohort, etc.?
o Yes.
Is there sufficient raw data presented to assess rigor of the analysis?
o All mass spectrometry raw data and search results files have been deposited to the ProteomeXchange Consortium.
o The supplementary tables also include extensive raw data, which will be useful for the field
Are methods for experimentation and analysis adequately outlined to permit reproducibility?
o Methods for figure 2c are not in the main text (methods section). We suggest to move methods for Supl. fig. 4-5 to the main text.
o In methods, the authors should clarify which constructs have N- vs C- terminal tags
o We suggest more detail is included for the chemoinformatics analysis. Also, approach 2) “a target- and pathway-specific literature search, drawing on specialist knowledge within our group” needs to be better described for the reader. Such specialist knowledge, if not properly described, may limit the ability of others to confirm the work.
4. Quality: Scholarship (1–4 scale but generally not the basis for acceptance or rejection) SCORE = 1.5
Has the author cited and discussed the merits of the relevant data that would argue against their conclusion?
o In the Supplementary Discussion, the authors clearly state the possible limitations of the use of homologous proteins from other coronaviruses to infer the SARS-CoV-2 protein and gene functions. Thus, we suggest this disclaimer could also be made in the main Discussion. Despite this limitation, we think the review of the principle interactions for each bait (Sup. Discussion) is very valuable for the scientific community at this point.
Has the author cited and/or discussed the important works that are consistent with their conclusion and that a reader should be especially familiar when considering the work?
o Discussion of data from previously published coronaviruses interactomes could be improved. E.g. citing articles like this one https://elifesciences.org/articles/42037.
Specific (helpful) comments on grammar, diction, paper structure, or data presentation (e.g., change a graph style or color scheme) go in this section, but scores in this area should not be significant bases for decisions.
o Abstract and Introduction: Mentions number of infected since the end of 2019, but does not give a “current” as of written date. It would be better to have written: “290,000 people infected from the onset (as of DATE).” This will significantly help readers who haven’t been following numbers day-by-day especially as this is a developing situation.
o Fig 1c. Authors could refer to the other bands that did not match the expected size of construct. Is this the product of protein degradation/aggregation?
o 1d: are the beads coated with antibodies or a modified streptavidin? Please modify text to clarify this for the reader.
o Sup. Fig. 1: labels are pixelated, but might be a submission issue on the BioRxiv site.
o Fig 2c: we suggest adding a figure with all 29 tissues tested to the supplementary material.
o Fig 2d: clarify the biological relevance of this figure and why were you expecting a positive correlation? Provide rationale/discuss this figure in greater detail. Otherwise we suggest moving it to supplementary material.
o Fig 2e: we suggest moving the discussion points about this figure from the supplementary material and place it in the main text, to help the reader understand the importance of this figure.
o Sup. Fig 5: we suggest, if possible, moving this figure to the main text because it shows a lot of possible important interacting proteins and the drugs that could target these proteins.
o Sup. Fig 6: we think that this figure might be beyond the scope of this paper and is not enough to support that the interactome (332 proteins) has reduced genetic variation in human populations. We suggest that this paragraph should be moved to the Supplementary Discussion. Also, a T-test might not be appropriate for testing differences in loss of function. We suggest the authors use a Wilcoxon test as the data distribution did not look normal (especially for the loss of function data).
o Fig 4: This figure is overcrowded. We suggest to simplify the figure or split it in two. Also, in the main text, subpanels are not specified (eg: Fig 4a-i)
o Fig 5: yellow levels are difficult to see.
o When downloading the manuscript some lines have single spacing and some have 1.5 line spacing (pages 5, 14), might be an issue of the bioRxiv site.
MORE SUBJECTIVE CRITERIA (IMPACT)
1. Impact: Novelty/Fundamental and Broad Interest (1–4 scale) SCORE = 2
A score here should be accompanied by a statement delineating the most interesting and/or important conceptual finding(s), as they stand right now with the current scope of the paper. A ‘‘1’’ would be expected to be understood for the importance by a layperson but would also be of top interest (have lasting impact) on the field.
● Very relevant and broad interest topic.
● Opens up opportunities for other labs to test specific therapeutics and investigate host targets and or pathways identified in the paper for potential treatments.
How big of an advance would you consider the findings to be if fully supported but not extended? It would be appropriate to cite literature to provide context for evaluating the advance. However, great care must be taken to avoid exaggerating what is known comparing these findings to the current dogma (see Box 2). Citations (figure by figure) are essential here.
o We think given the current pandemic, this paper will have a huge impact as it is, without any extended experiments. However, if functional data is included the impact will be higher.
2. Impact: Extensibility (1–4 or N/A scale) SCORE = NA
Has an initial result (e.g., of a paradigm in a cell line) been extended to be shown (or implicated) to be important in a bigger scheme (e.g., in animals or in a human cohort)?
This criterion is only valuable as a scoring parameter if it is present, indicated by the N/A option if it simply doesn’t apply. The extent to which this is necessary for a result to be considered of value is important. It should be explicitly discussed by a reviewer why it would be required. What work (scope and expected time) and/or discussion would improve this score, and what would this improvement add to the conclusions of the study? Care should be taken to avoid casually suggesting experiments of great cost (e.g., ‘‘repeat a mouse-based experiment in humans’’) and difficulty that merely confirm but do not extend (see Bad Behaviors, Box 2).