Skip to main content

Microsoft Health Futures package for deep learning on histopathology images

Project description

Histopathology Models and Workflows

Models on public data

This repository contains a set of models built on and for public datasets (PANDA, TCGA). Detailed instructions to on-board the datasets and run the models are provided on readthedocs.

Getting started

Setting up Python

For working on the histopathology folder, please create a separate Conda environment.

cd hi-ml-cpath
make env

You can then activate the environment via conda activate HimlHisto. Set VSCode to use this Conda environment, by choosing "Python: Select Interpreter" from the command palette.

If the dependencies need to be updated, please modify hi-ml-cpath/primary_deps.yml, and then run the script hi-ml-cpath/create_and_lock_environment.sh. This will create a full "locked" environment specification with pinned versions of all depdencies.

Setting up AzureML

In addition, please download an AzureML workspace configuration file for the workspace that you wish to use:

  • In the browser, navigate to the workspace in question
  • Click on the drop-down menu on upper right of the page, to the left of your account picture.
  • Select "Download config file".
  • Save that file into the the repository root.

Once that config file is in place, all Python runs that you start inside the hi-ml-cpath folder will automatically use this config file.

Running histopathology models

To test your setup, please execute in the hi-ml-cpath folder:

conda activate HimlHisto
python ../hi-ml/src/health_ml/runner.py --model health_cpath.TcgaCrckImageNetMIL  --cluster=training-nd24

This should start an AzureML job in the AzureML workspace that you configured above via config.json. You may need to adjust the name of the compute cluster (training-nd24 in the above example).

Conda environment

If you start your jobs in the hi-ml-cpath folder, they will automatically pick up the Conda environment file that is present in that folder. If you start your jobs in a different folder, you need to add the --conda_env option to point to the file <repo_root>/hi-ml-cpath/environment.yml.

Running histopathology tests

In the hi-ml-cpath folder, run

make call_pytest

Inside of VSCode, all tests in the repository should be picked up automatically. You can exclude the tests for the hi-ml and hi-ml-azure packages by modifying python.testing.pytestArgs in the VSCode .vscode/settings.json file.

Tests that require a GPU

The test pipeline for the histopathology folder contains a run of pytest on a machine with 2 GPUs. Only tests that are marked with the pytest mark gpu are executed on that GPU machine. Note that all tests that bear the gpu mark will also be executed when running on a CPU machine. You need to manually add a skipif flag for tests that are meant to exclusively run on GPU machines. This also helps to ensure that the test suite can pass when executed outside of the build agents.

  • Tests that run only on a CPU machine: Provide no pytest marks
def test_my_code() -> None:
    pass
  • Tests that run on both on a CPU and on a GPU machine: Add @pytest.mark.gpu
@pytest.mark.gpu
def test_my_code() -> None:
    pass
  • Tests that run only on a GPU machine:
from health_ml.utils.common_utils import is_gpu_available
no_gpu = not is_gpu_available()

@pytest.mark.skipif(no_gpu, reason="Test requires GPU")
@pytest.mark.gpu
def test_my_code() -> None:
    pass

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

hi-ml-cpath-0.2.12.tar.gz (92.2 kB view details)

Uploaded Source

Built Distribution

hi_ml_cpath-0.2.12-py3-none-any.whl (116.1 kB view details)

Uploaded Python 3

File details

Details for the file hi-ml-cpath-0.2.12.tar.gz.

File metadata

  • Download URL: hi-ml-cpath-0.2.12.tar.gz
  • Upload date:
  • Size: 92.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.9.16

File hashes

Hashes for hi-ml-cpath-0.2.12.tar.gz
Algorithm Hash digest
SHA256 bd63f021aaeb7e9e71085784fe2f48161c260da8388ff0f5a585ca68d8f51a91
MD5 893f9a3f25022a8fc3f1f81f03faea7e
BLAKE2b-256 66cf11c93990a1d429d99c207925d94f0078166d12c6f01d3baf0bf487d7505e

See more details on using hashes here.

File details

Details for the file hi_ml_cpath-0.2.12-py3-none-any.whl.

File metadata

  • Download URL: hi_ml_cpath-0.2.12-py3-none-any.whl
  • Upload date:
  • Size: 116.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.9.16

File hashes

Hashes for hi_ml_cpath-0.2.12-py3-none-any.whl
Algorithm Hash digest
SHA256 31e84b87d39f54d7ca0aa17ebe76c29003708050dba38d2ec61d08c1b2d33ff0
MD5 a4c2a03b84cb3599c7e3ba839d35a900
BLAKE2b-256 ca71f6582554db1ba6440b5df4cfba35792e40e916c7199cd4fff26ea4077d79

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page