MONAI Label

MONAI Label is a server-client system that facilitates interactive medical image annotation by using AI. It is an open-source and easy-to-install ecosystem that can run locally on a machine with single or multiple GPUs. Both server and client work on the same/different machine. It shares the same principles with MONAI. Refer to full MONAI Label documentations for more details.

• MONAI Label Demo: Sample Apps
• Highlights and Features
• Installation and Usage
  • MONAI Label Server
  • Visualization Tools Guide
  • Plugin Guide
• Contributing Guide and Communities

Sample Apps in MONAILabel

MONAI Label | Demo Videos

MONAI Label with visualization tools 3D Slicer, OHIF, DSA, QuPath, CVAT etc..

Highlights and Features

The codebase is currently under active development.

• Framework for developing and deploying MONAI Label Apps to train and infer AI models
• Compositional & portable APIs for ease of integration in existing workflows
• Customizable labeling app design for varying user expertise
• Annotation support via 3DSlicer & OHIF for radiology
• Annotation support via QuPath , Digital Slide Archive & CVAT for pathology
• Annotation support via CVAT for Endoscopy
• PACS connectivity via DICOMWeb
• Automated Active Learning workflow for endoscopy using CVAT

Radiology App This app has example models to do both interactive and automated segmentation over radiology (3D) images. Including auto segmentation with the latest deep learning models (e.g., UNet, UNETR) for multiple abdominal organs. Interactive tools include DeepEdit and Deepgrow for actively improving trained models and deployment.

Pathology App This app has example models to do both interactive and automated segmentation over pathology (WSI) images. Including nuclei multi-label segmentation for Neoplastic cells, Inflammatory, Connective/Soft tissue cells, Dead Cells, and Epithelial. The app provides interactive tools including DeepEdits for interactive nuclei segmentation.

Bundle App The Bundle app enables users with customized models for inference, training or pre and post processing any target anatomies. The specification for MONAILabel integration of the Bundle app links archived Model-Zoo for customized labeling (e.g., the third-party transformer model for labeling renal cortex, medulla, and pelvicalyceal system. Interactive tools such as DeepEdits).

Endoscopy App The Endoscopy app enables users to use interactive, automated segmentation and classification models over 2D images for endoscopy usecase. Combined with CVAT, it will demonstrate the fully automated Active Learning workflow to train + fine-tune a model.

Installation

Start using MONAI Label with just three steps:

MONAI Label supports following OS with GPU/CUDA enabled.

• Ubuntu: Please see the installation guide.
• Windows

Current Stable Version

pip install monailabel -U

Development version

To install the latest features using one of the following options:

Git Checkout (developer mode)

git clone https://github.com/Project-MONAI/MONAILabel
pip install -r MONAILabel/requirements.txt
export PATH=$PATH:`pwd`/MONAILabel/monailabel/scripts

If you are using DICOM-Web + OHIF then you have to build OHIF package separate. Please refer here.

Weekly Release

pip install monailabel-weekly -U

Docker

docker run --gpus all --rm -ti --ipc=host --net=host projectmonai/monailabel:latest bash

Once the package is installed, you can download sample radiology or pathology app and start monailabel server.

# download radiology app and sample dataset
monailabel apps --download --name radiology --output apps
monailabel datasets --download --name Task09_Spleen --output datasets

# start server using radiology app with deepedit model enabled
monailabel start_server --app apps/radiology --studies datasets/Task09_Spleen/imagesTr --conf models deepedit

More details refer docs: https://docs.monai.io/projects/label/en/stable/installation.html

If monailabel install path is not automatically determined, then you can provide explicit install path as: monailabel apps --prefix ~/.local

For prerequisites, other installation methods (using the default GitHub branch, using Docker, etc.), please refer to the installation guide.

Once you start the MONAI Label Server, by default server will be up and serving at http://127.0.0.1:8000/. Open the serving URL in browser. It will provide you the list of Rest APIs available. For this, please make sure you use the HTTP protocol. You can provide ssl arguments to run server in HTTPS mode but this functionality is not fully verified across all clients.

Optional Dependencies

Following are the optional dependencies which can help you to accelerate some GPU based transforms from MONAI. These dependencies are by-default available if you are using projectmonai/monailabel docker.

• CUCIM
• CUPY
• CUDA Toolkit

Visualization Tools

MONAI Label supports the most adopted open-source viewers for Radiology, Pathology, and Endoscopy images.

3D Slicer

3D Slicer, a free and open-source platform for analyzing, visualizing and understanding medical image data. In MONAI Label, 3D Slicer is most tested with radiology studies and algorithms, develpoment and integration.

MONAI Label is most currently tested and supported with stable release of 3D Slicer every version. Preview version of 3D Slicer is not fully tested and supported.

To install stable released version of 3D Slicer, see 3D Slicer installation. Currently, Windows and Linux version are supported.

OHIF (Web-based)

The Open Health Imaging Foundation (OHIF) Viewer is an open source, web-based, medical imaging platform. It aims to provide a core framework for building complex imaging applications.

At this point OHIF can be used to annotate the data in the DICOM server via the MONAI Label server. To use OHIF web-based application, refer to extensible web imaging platform

QuPath

Quantitative Pathology & Bioimage Analysis (QuPath) is an open, powerful, flexible, extensible software platform for bioimage analysis.

To install stable released version of QuPath, see QuPath installation. Currently, Windows and Linux version are supported. Detailed documentation can be found QuPath Doc.

CVAT

CVAT is an interactive video and image annotation tool for computer vision.

To install stable released version of CVAT, see CVAT installation. Currently, Windows and Linux version are supported. Detailed documentation can be found CVAT Doc.

Plugins

3D Slicer (radiology)

Download and install 3D Slicer with the installation page. Install MONAI Label plugin from Slicer Extension Manager.

Refer 3D Slicer plugin for other options to install and run MONAI Label plugin in 3D Slicer.

To avoid accidentally using an older Slicer version, you may want to uninstall any previously installed 3D Slicer package.

OHIF (radiology)

MONAI Label comes with pre-built plugin for OHIF Viewer. To use OHIF Viewer, you need to provide DICOMWeb instead of FileSystem as studies when you start the server.

Please install Orthanc before using OHIF Viewer. For Ubuntu 20.x, Orthanc can be installed as apt-get install orthanc orthanc-dicomweb. However, you have to upgrade to latest version by following steps mentioned here.

You can use PlastiMatch to convert NIFTI to DICOM

  # start server using DICOMWeb
  monailabel start_server --app apps/radiology --studies http://127.0.0.1:8042/dicom-web

  # to disable DICOM to NIFTI conversion for faster performance
  export MONAI_LABEL_DICOMWEB_CONVERT_TO_NIFTI=false
  monailabel start_server --app apps/radiology --studies http://127.0.0.1:8042/dicom-web

NOTE:: 3D-Slicer is not supported without DICOM to NIFTI conversion

OHIF Viewer will be accessible at http://127.0.0.1:8000/ohif/

NOTE: OHIF does not yet support Multi-Label interaction for DeepEdit. And you can still use 3D Slicer when MONAILabel is connected to DICOMWeb.

QuPath (pathology)

You can download sample whole slide images from https://portal.gdc.cancer.gov/repository

  # start server using pathology over downloaded whole slide images
  monailabel start_server --app apps/pathology --studies wsi_images

Refer QuPath for installing and running MONAILabel plugin in QuPath.

Digital Slide Archive (DSA) (pathology)

Refer Pathology for running a sample pathology use-case in MONAILabel.

NOTE: The DSA Plugin is under active development.

CVAT

Install CVAT and enable Semi-Automatic and Automatic Annotation. Refer CVAT Instructions for deploying available MONAILabel pathology/endoscopy models into CVAT.

Cite

If you are using MONAI Label in your research, please use the following citation:

@article{DiazPinto2022monailabel,
   author = {Diaz-Pinto, Andres and Alle, Sachidanand and Ihsani, Alvin and Asad, Muhammad and
            Nath, Vishwesh and P{\'e}rez-Garc{\'\i}a, Fernando and Mehta, Pritesh and
            Li, Wenqi and Roth, Holger R. and Vercauteren, Tom and Xu, Daguang and
            Dogra, Prerna and Ourselin, Sebastien and Feng, Andrew and Cardoso, M. Jorge},
    title = {{MONAI Label: A framework for AI-assisted Interactive Labeling of 3D Medical Images}},
  journal = {arXiv e-prints},
     year = 2022,
     url  = {https://arxiv.org/pdf/2203.12362.pdf}
}

@inproceedings{DiazPinto2022DeepEdit,
      title={{DeepEdit: Deep Editable Learning for Interactive Segmentation of 3D Medical Images}},
      author={Diaz-Pinto, Andres and Mehta, Pritesh and Alle, Sachidanand and Asad, Muhammad and Brown, Richard and Nath, Vishwesh and Ihsani, Alvin and Antonelli, Michela and Palkovics, Daniel and Pinter, Csaba and others},
      booktitle={MICCAI Workshop on Data Augmentation, Labelling, and Imperfections},
      pages={11--21},
      year={2022},
      organization={Springer}
}

Optional Citation: if you are using active learning functionality from MONAI Label, please support us:

@article{nath2020diminishing,
  title={Diminishing uncertainty within the training pool: Active learning for medical image segmentation},
  author={Nath, Vishwesh and Yang, Dong and Landman, Bennett A and Xu, Daguang and Roth, Holger R},
  journal={IEEE Transactions on Medical Imaging},
  volume={40},
  number={10},
  pages={2534--2547},
  year={2020},
  publisher={IEEE}
}

Contributing

For guidance on making a contribution to MONAI Label, see the contributing guidelines.

Community

Join the conversation on Twitter @ProjectMONAI or join our Slack channel.

Ask and answer questions over on MONAI Label's GitHub Discussions tab.

Links

• Website: https://monai.io/
• API documentation: https://docs.monai.io/projects/label
• Code: https://github.com/Project-MONAI/MONAILabel
• Project tracker: https://github.com/Project-MONAI/MONAILabel/projects
• Issue tracker: https://github.com/Project-MONAI/MONAILabel/issues
• Wiki: https://github.com/Project-MONAI/MONAILabel/wiki
• Test status: https://github.com/Project-MONAI/MONAILabel/actions
• PyPI package: https://pypi-hypernode.com/project/monailabel/
• Weekly previews: https://pypi-hypernode.com/project/monailabel-weekly/
• Docker Hub: https://hub.docker.com/r/projectmonai/monailabel
• Client API: https://www.youtube.com/watch?v=mPMYJyzSmyo
• Demo Videos: https://www.youtube.com/c/ProjectMONAI