Planned intervention: On Wednesday April 3rd 05:30 UTC Zenodo will be unavailable for up to 2-10 minutes to perform a storage cluster upgrade.
Published May 2, 2021 | Version v1
Journal article Open

Deep Learning-Inferred Multiplex ImmunoFluorescence for Immunohistochemical Image Quantification

  • 1. Research Assistant
  • 2. Lab Manager
  • 3. Distinguished Professor and CEWIT Chief Scientist
  • 4. Research Scientist
  • 5. Assistant Professor of Pathology
  • 6. Associate Attending Pathologist
  • 7. Assistant Professor

Description

Reporting biomarkers assessed by routine immunohistochemical (IHC) staining of tissue is broadly used in diagnostic pathology laboratories for patient care. To date, clinical reporting is predominantly qualitative or semi-quantitative. By creating a multitask deep learning framework referred to as DeepLIIF, we present a single-step solution to stain deconvolution/separation, cell segmentation, and quantitative single-cell IHC scoring. Leveraging a unique de novo dataset of co-registered IHC and multiplex immunofluorescence (mpIF) staining of the same slides, we segment and translate low-cost and prevalent IHC slides to more expensive-yet-informative mpIF images, while simultaneously providing the essential ground truth for the superimposed brightfield IHC channels. Moreover, a new nuclear-envelop stain, LAP2beta, with high (>95%) cell coverage is introduced to improve cell delineation/segmentation and protein expression quantification on IHC slides. By simultaneously translating input IHC images to clean/separated mpIF channels and performing cell segmentation/classification, we show that our model trained on clean IHC Ki67 data can generalize to more noisy and artifact-ridden images as well as other nuclear and non-nuclear markers such as CD3, CD8, BCL2, BCL6, MYC, MUM1, CD10, and TP53. We thoroughly evaluate our method on publicly available benchmark datasets as well as against pathologists' semi-quantitative scoring. Trained on IHC, DeepLIIF also generalizes well to H&E images for out-of-the-box nuclear segmentation.

DeepLIIF is deployed as a cloud-native platform at https://deepliif.org with Bioformats (150 input formats supported) and MLOps pipeline. Implementations for Torchserve/Dask+Torchscript deployment and auto-scaling via Pulumi available at our GitHub (https://nadeemlab.github.io/DeepLIIF/). DeepLIIF can be run locally (GPU required) by pip installing the package and using the deepliif CLI command. DeepLIIF can be used remotely (no GPU required) through the https://deepliif.org website, the ImageJ/Fiji plugin, or via the cloud API (https://github.com/nadeemlab/deepliif).

Accompanying CVPR2022 paper on our free public cloud-native DeepLIIF platform (http://deepliif.org), ImageJ plugin, and Cloud-API (no GPU required).

Notes

DeepLIIF code (https://github.com/nadeemlab/deepliif) is distributed under Apache 2.0 with Commons Clause license, and is available for non-commercial academic purposes.

Files

BC-DeepLIIF_Training_Set.zip

Files (7.7 GB)

Name Size Download all
md5:61d02c92fce42b56d0ec01a20498879b
696.2 MB Preview Download
md5:f172eb8ae915c1ba772ac1e3c2b6db72
114.5 MB Preview Download
md5:39ffb937acebc85c0d478ea8f5eb7544
1.6 GB Preview Download
md5:5a57ff8f059bc4d9e10010d9d2e50f05
5.6 MB Download
md5:6944801084216c30e2646e39aa7e1444
3.1 GB Preview Download
md5:665899936c8012754524258ed16667ea
1.0 GB Preview Download
md5:f4812639cbee1f6732cb206e1e6acdc8
1.0 GB Preview Download
md5:3387c41b2d6771968976a1a6e67202dc
161.4 MB Preview Download
md5:2eb5113938362a3faeea264f9ce8769e
1.1 MB Preview Download

Additional details

Related works

Is new version of
Preprint: 10.1101/2021.05.01.442219 (DOI)