Deep Learning-Inferred Multiplex ImmunoFluorescence for Immunohistochemical Image Quantification

Reporting biomarkers assessed by routine immunohistochemical (IHC) staining of tissue is broadly used in diagnostic pathology laboratories for patient care. To date, clinical reporting is predominantly qualitative or semi-quantitative. By creating a multitask deep learning framework referred to as DeepLIIF, we present a single-step solution to stain deconvolution/separation, cell segmentation, and quantitative single-cell IHC scoring. Leveraging a unique de novo dataset of co-registered IHC and multiplex immunofluorescence (mpIF) staining of the same slides, we segment and translate low-cost and prevalent IHC slides to more expensive-yet-informative mpIF images, while simultaneously providing the essential ground truth for the superimposed brightfield IHC channels. Moreover, a new nuclear-envelop stain, LAP2beta, with high (>95%) cell coverage is introduced to improve cell delineation/segmentation and protein expression quantification on IHC slides. By simultaneously translating input IHC images to clean/separated mpIF channels and performing cell segmentation/classification, we show that our model trained on clean IHC Ki67 data can generalize to more noisy and artifact-ridden images as well as other nuclear and non-nuclear markers such as CD3, CD8, BCL2, BCL6, MYC, MUM1, CD10, and TP53. We thoroughly evaluate our method on publicly available benchmark datasets as well as against pathologists' semi-quantitative scoring. Trained on IHC, DeepLIIF also generalizes well to H&E images for out-of-the-box nuclear segmentation.

DeepLIIF is deployed as a cloud-native platform at https://deepliif.org with Bioformats (150 input formats supported) and MLOps pipeline. Implementations for Torchserve/Dask+Torchscript deployment and auto-scaling via Pulumi available at our GitHub (https://nadeemlab.github.io/DeepLIIF/). DeepLIIF can be run locally (GPU required) by pip installing the package and using the deepliif CLI command. DeepLIIF can be used remotely (no GPU required) through the https://deepliif.org website, the ImageJ/Fiji plugin, or via the cloud API (https://github.com/nadeemlab/deepliif).

Accompanying CVPR2022 paper on our free public cloud-native DeepLIIF platform (http://deepliif.org), ImageJ plugin, and Cloud-API (no GPU required).