nvtx-plugins 0.1.5

Prerequisites

• Linux

• Python 3.4+

• NVIDIA GPU + CUDA toolkit 10.0 or newer

• TensorFlow 1.13 or newer

Installing NVTX-Plugins

The package can be installed from PyPI:

# Stable release
pip install nvtx-plugins

# Pre-release (may present bugs)
pip install nvtx-plugins --pre

The package is also available for download on github: https://github.com/NVIDIA/nvtx-plugins/releases

pip install nvtx-plugins*.tar.gz

Installing from source

You can build and install the package from source:

python setup.py sdist
pip install dist/nvtx-plugins*.tar.gz

For development objectives, you can install the package directly from source with:

python setup.py install

We recommend building the package inside NVIDIA’s NGC TensorFlow container: https://ngc.nvidia.com/catalog/containers/nvidia:tensorflow

For more information about how to get started with NGC containers, see the following sections from the NVIDIA GPU Cloud Documentation and the Deep Learning DGX Documentation: Getting Started Using NVIDIA GPU Cloud, Accessing And Pulling From The NGC container registry and Running TensorFlow.

Building the documentation

The documentation is built by running:

cd docs
pip install -r requirements.txt
make html

The documentation files will be generated in docs/build/html

Building the documentation does not require NVTX Plugins to be installed. Nonetheless, due to an issue in Sphinx only Python 3.7 is supported to build the documentation.

Adding markers to the graph

Markers are added by wrapping parts of the computation graph with start and end operations. The operations are identity ops (passing the input to the output without modification) but they have a side effect of generating nvtx markers.

import nvtx.plugins.tf as nvtx_tf

x, nvtx_context = nvtx_tf.ops.start(x, message='Dense 1-3',
    domain_name='Forward', grad_domain_name='Gradient')
x = tf.layers.dense(x, 1000, activation=tf.nn.relu, name='dense_1')
x = tf.layers.dense(x, 1000, activation=tf.nn.relu, name='dense_2')
x = tf.layers.dense(x, 1000, activation=tf.nn.relu, name='dense_3')
x = nvtx_tf.ops.end(x, nvtx_context)
x = tf.layers.dense(x, 1000, activation=tf.nn.relu, name='dense_4')

For convenience, the package also provides a function dectorator:

@nvtx_tf.ops.trace(message='Dense Block', domain_name='Forward',
                   grad_domain_name='Gradient')
def dense_block(x):
    x = tf.layers.dense(x, 1000, activation=tf.nn.relu, name='dense_1')
    x = tf.layers.dense(x, 1000, activation=tf.nn.relu, name='dense_2')
    x = tf.layers.dense(x, 1000, activation=tf.nn.relu, name='dense_3')
    return x

More detailed examples can be found in examples/, also, check the Documentation for more information about other workflows including session hooks, Keras layers and callbacks.

Visualizing the ranges

NVTX requires a logger to register the generated events and ranges, we will use NVIDIA Nsight Systems to capture these event.

Run your code with nsys (pre-installed in NVIDIA’s NGC TensorFlow container) to generate a qdrep file:

nsys profile -d 60 \
    -w true \
    --sample=cpu \
    -t 'nvtx,cuda' \
    -o ./generated_timeline \
    python ./network.py

The generated qdrep can be viewed using Nsight Systems.

Nsight Systems and nsys can also be downloaded and from the NVIDIA’’s developer website.

More details about nsys and Nsight Systems can be found here.