Published May 2, 2021 | Version v1
Journal article Open

Deep Learning-Inferred Multiplex ImmunoFluorescence for Immunohistochemical Image Quantification

  • 1. Research Assistant
  • 2. Lab Manager
  • 3. Distinguished Professor and CEWIT Chief Scientist
  • 4. Research Scientist
  • 5. Assistant Professor of Pathology
  • 6. Associate Attending Pathologist
  • 7. Assistant Professor


Reporting biomarkers assessed by routine immunohistochemical (IHC) staining of tissue is broadly used in diagnostic pathology laboratories for patient care. To date, clinical reporting is predominantly qualitative or semi-quantitative. By creating a multitask deep learning framework referred to as DeepLIIF, we present a single-step solution to stain deconvolution/separation, cell segmentation, and quantitative single-cell IHC scoring. Leveraging a unique de novo dataset of co-registered IHC and multiplex immunofluorescence (mpIF) staining of the same slides, we segment and translate low-cost and prevalent IHC slides to more expensive-yet-informative mpIF images, while simultaneously providing the essential ground truth for the superimposed brightfield IHC channels. Moreover, a new nuclear-envelop stain, LAP2beta, with high (>95%) cell coverage is introduced to improve cell delineation/segmentation and protein expression quantification on IHC slides. By simultaneously translating input IHC images to clean/separated mpIF channels and performing cell segmentation/classification, we show that our model trained on clean IHC Ki67 data can generalize to more noisy and artifact-ridden images as well as other nuclear and non-nuclear markers such as CD3, CD8, BCL2, BCL6, MYC, MUM1, CD10, and TP53. We thoroughly evaluate our method on publicly available benchmark datasets as well as against pathologists' semi-quantitative scoring. Trained on IHC, DeepLIIF also generalizes well to H&E images for out-of-the-box nuclear segmentation.

DeepLIIF is deployed as a cloud-native platform at with Bioformats (150 input formats supported) and MLOps pipeline. Implementations for Torchserve/Dask+Torchscript deployment and auto-scaling via Pulumi available at our GitHub ( DeepLIIF can be run locally (GPU required) by pip installing the package and using the deepliif CLI command. DeepLIIF can be used remotely (no GPU required) through the website, the ImageJ/Fiji plugin, or via the cloud API (

Accompanying CVPR2022 paper on our free public cloud-native DeepLIIF platform (, ImageJ plugin, and Cloud-API (no GPU required).


DeepLIIF code ( is distributed under Apache 2.0 with Commons Clause license, and is available for non-commercial academic purposes.


Files (7.7 GB)

Name Size Download all
696.2 MB Preview Download
114.5 MB Preview Download
1.6 GB Preview Download
5.6 MB Download
3.1 GB Preview Download
1.0 GB Preview Download
1.0 GB Preview Download
161.4 MB Preview Download
1.1 MB Preview Download

Additional details

Related works

Is new version of
Preprint: 10.1101/2021.05.01.442219 (DOI)