Skip to main content

A lightweight library to help with training neural networks in PyTorch.

Project description

image image imageimage image image
image image image image image
image image image
image image image image
image Twitter numfocus
image link

TL;DR

Ignite is a high-level library to help with training and evaluating neural networks in PyTorch flexibly and transparently.

PyTorch-Ignite teaser

Click on the image to see complete code

Features

  • Less code than pure PyTorch while ensuring maximum control and simplicity

  • Library approach and no program's control inversion - Use ignite where and when you need

  • Extensible API for metrics, experiment managers, and other components

Table of Contents

Why Ignite?

Ignite is a library that provides three high-level features:

  • Extremely simple engine and event system
  • Out-of-the-box metrics to easily evaluate models
  • Built-in handlers to compose training pipeline, save artifacts and log parameters and metrics

Simplified training and validation loop

No more coding for/while loops on epochs and iterations. Users instantiate engines and run them.

Example
from ignite.engine import Engine, Events, create_supervised_evaluator
from ignite.metrics import Accuracy


# Setup training engine:
def train_step(engine, batch):
    # Users can do whatever they need on a single iteration
    # E.g. forward/backward pass for any number of models, optimizers etc
    # ...

trainer = Engine(train_step)

# Setup single model evaluation engine
evaluator = create_supervised_evaluator(model, metrics={"accuracy": Accuracy()})

def validation():
    state = evaluator.run(validation_data_loader)
    # print computed metrics
    print(trainer.state.epoch, state.metrics)

# Run model's validation at the end of each epoch
trainer.add_event_handler(Events.EPOCH_COMPLETED, validation)

# Start the training
trainer.run(training_data_loader, max_epochs=100)

Power of Events & Handlers

The cool thing with handlers is that they offer unparalleled flexibility (compared to say, callbacks). Handlers can be any function: e.g. lambda, simple function, class method etc. Thus, we do not require to inherit from an interface and override its abstract methods which could unnecessarily bulk up your code and its complexity.

Execute any number of functions whenever you wish

Examples
trainer.add_event_handler(Events.STARTED, lambda _: print("Start training"))

# attach handler with args, kwargs
mydata = [1, 2, 3, 4]
logger = ...

def on_training_ended(data):
    print(f"Training is ended. mydata={data}")
    # User can use variables from another scope
    logger.info("Training is ended")


trainer.add_event_handler(Events.COMPLETED, on_training_ended, mydata)
# call any number of functions on a single event
trainer.add_event_handler(Events.COMPLETED, lambda engine: print(engine.state.times))

@trainer.on(Events.ITERATION_COMPLETED)
def log_something(engine):
    print(engine.state.output)

Built-in events filtering

Examples
# run the validation every 5 epochs
@trainer.on(Events.EPOCH_COMPLETED(every=5))
def run_validation():
    # run validation

# change some training variable once on 20th epoch
@trainer.on(Events.EPOCH_STARTED(once=20))
def change_training_variable():
    # ...

# Trigger handler with customly defined frequency
@trainer.on(Events.ITERATION_COMPLETED(event_filter=first_x_iters))
def log_gradients():
    # ...

Stack events to share some actions

Examples

Events can be stacked together to enable multiple calls:

@trainer.on(Events.COMPLETED | Events.EPOCH_COMPLETED(every=10))
def run_validation():
    # ...

Custom events to go beyond standard events

Examples

Custom events related to backward and optimizer step calls:

from ignite.engine import EventEnum


class BackpropEvents(EventEnum):
    BACKWARD_STARTED = 'backward_started'
    BACKWARD_COMPLETED = 'backward_completed'
    OPTIM_STEP_COMPLETED = 'optim_step_completed'

def update(engine, batch):
    # ...
    loss = criterion(y_pred, y)
    engine.fire_event(BackpropEvents.BACKWARD_STARTED)
    loss.backward()
    engine.fire_event(BackpropEvents.BACKWARD_COMPLETED)
    optimizer.step()
    engine.fire_event(BackpropEvents.OPTIM_STEP_COMPLETED)
    # ...

trainer = Engine(update)
trainer.register_events(*BackpropEvents)

@trainer.on(BackpropEvents.BACKWARD_STARTED)
def function_before_backprop(engine):
    # ...

Out-of-the-box metrics

Example
precision = Precision(average=False)
recall = Recall(average=False)
F1_per_class = (precision * recall * 2 / (precision + recall))
F1_mean = F1_per_class.mean()  # torch mean method
F1_mean.attach(engine, "F1")

Installation

From pip:

pip install pytorch-ignite

From conda:

conda install ignite -c pytorch

From source:

pip install git+https://github.com/pytorch/ignite

Nightly releases

From pip:

pip install --pre pytorch-ignite

From conda (this suggests to install pytorch nightly release instead of stable version as dependency):

conda install ignite -c pytorch-nightly

Docker Images

Using pre-built images

Pull a pre-built docker image from our Docker Hub and run it with docker v19.03+.

docker run --gpus all -it -v $PWD:/workspace/project --network=host --shm-size 16G pytorchignite/base:latest /bin/bash
List of available pre-built images

Base

  • pytorchignite/base:latest
  • pytorchignite/apex:latest
  • pytorchignite/hvd-base:latest
  • pytorchignite/hvd-apex:latest
  • pytorchignite/msdp-apex:latest

Vision:

  • pytorchignite/vision:latest
  • pytorchignite/hvd-vision:latest
  • pytorchignite/apex-vision:latest
  • pytorchignite/hvd-apex-vision:latest
  • pytorchignite/msdp-apex-vision:latest

NLP:

  • pytorchignite/nlp:latest
  • pytorchignite/hvd-nlp:latest
  • pytorchignite/apex-nlp:latest
  • pytorchignite/hvd-apex-nlp:latest
  • pytorchignite/msdp-apex-nlp:latest

For more details, see here.

Getting Started

Few pointers to get you started:

Documentation

Additional Materials

Examples

Tutorials

Reproducible Training Examples

Inspired by torchvision/references, we provide several reproducible baselines for vision tasks:

  • ImageNet - logs on Ignite Trains server coming soon ...
  • Pascal VOC2012 - logs on Ignite Trains server coming soon ...

Features:

Communication

User feedback

We have created a form for "user feedback". We appreciate any type of feedback and this is how we would like to see our community:

  • If you like the project and want to say thanks, this the right place.
  • If you do not like something, please, share it with us and we can see how to improve it.

Thank you !

Contributing

Please see the contribution guidelines for more information.

As always, PRs are welcome :)

Projects using Ignite

Research papers
Blog articles, tutorials, books
Toolkits
Others

See other projects at "Used by"

If your project implements a paper, represents other use-cases not covered in our official tutorials, Kaggle competition's code or just your code presents interesting results and uses Ignite. We would like to add your project in this list, so please send a PR with brief description of the project.

About the team & Disclaimer

PyTorch-Ignite is a NumFOCUS Affiliated Project, operated and maintained by volunteers in the PyTorch community in their capacities as individuals (and not as representatives of their employers). See the "About us" page for a list of core contributors. For usage questions and issues, please see the various channels here. For all other questions and inquiries, please send an email to contact@pytorch-ignite.ai.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pytorch-ignite-0.5.0.dev20210402.tar.gz (154.9 kB view details)

Uploaded Source

Built Distributions

pytorch_ignite-0.5.0.dev20210402-py3.8.egg (499.6 kB view details)

Uploaded Source

pytorch_ignite-0.5.0.dev20210402-py3-none-any.whl (211.6 kB view details)

Uploaded Python 3

File details

Details for the file pytorch-ignite-0.5.0.dev20210402.tar.gz.

File metadata

  • Download URL: pytorch-ignite-0.5.0.dev20210402.tar.gz
  • Upload date:
  • Size: 154.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.8.8

File hashes

Hashes for pytorch-ignite-0.5.0.dev20210402.tar.gz
Algorithm Hash digest
SHA256 a0350e122abb49abf9640c7628d1aae6572b03e8c9c8271952fb91fc44986bdc
MD5 581e0ae54b52b9bfcca4591ddaf6e63e
BLAKE2b-256 3118b286aecece007688c7d9c0043b0222e56cbf8f7b1aea788a960dd11af91e

See more details on using hashes here.

File details

Details for the file pytorch_ignite-0.5.0.dev20210402-py3.8.egg.

File metadata

  • Download URL: pytorch_ignite-0.5.0.dev20210402-py3.8.egg
  • Upload date:
  • Size: 499.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.8.8

File hashes

Hashes for pytorch_ignite-0.5.0.dev20210402-py3.8.egg
Algorithm Hash digest
SHA256 c6d1ce9b6b56412b29e95a0811c43ef24ecbc53ab3d7f19b4beaa2c400b77cfd
MD5 68a5286afff328f488e5050ebf30021d
BLAKE2b-256 03040af59831936d66bcc1c0dbdf5d8860b332d8574f055b8d58f6f628ef292e

See more details on using hashes here.

File details

Details for the file pytorch_ignite-0.5.0.dev20210402-py3-none-any.whl.

File metadata

  • Download URL: pytorch_ignite-0.5.0.dev20210402-py3-none-any.whl
  • Upload date:
  • Size: 211.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.8.8

File hashes

Hashes for pytorch_ignite-0.5.0.dev20210402-py3-none-any.whl
Algorithm Hash digest
SHA256 c3e4d0a4fa09ae84a9c86a9c04bfb950093ff12bce28a743a0a66c8697fcfe2a
MD5 82c641d678d2b9bda01bcc73331f8214
BLAKE2b-256 5396a781b32326374fd2c180e057f6f5983fe3c5ecca2f65805d1f0202ad2ead

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page