Skip to main content

CUDNN FrontEnd python library

Project description

cuDNN FrontEnd(FE) API

Introduction

The cuDNN FrontEnd(FE) API is a C++ header-only library that wraps the cuDNN C backend API. Both the FE and backend APIs are entry points to the same set of functionality that is commonly referred to as the "graph API".

While there are two entry points to the graph API (i.e. backend and frontend), it is expected that most users will use the FE API. Reasons being:

  • FE API is less verbose without loss of control. All functionality accessible through the backend API is also accessible through the FE API.
  • FE API adds functionality on top of the backend API, like errata filters and autotuning.

Also, for those using backend API, FE API source and samples can serve as reference implementation.

In FE v1.0 API, users can describe multiple operations that form subgraph through a persistent cudnn_frontend::graph::Graph object. Unlike the FE v0.x API, users don't need to worry about specifying shapes and sizes of the intermediate virtual tensors. FE v1.0 API extends the groundwork of earlier versions and introduces a new set of APIs to further simplify the workflow. For detailed information of FE v1.0 API, see README.FE.1.0.md.

Additionally, FE v1.0 API provides python bindings to all API through pybind11. It is recommended that new users of cuDNN start with the frontend v1.0 API. See samples/cpp and samples/python for more details on its usage.

Usage

For c++ users, in order to include the entire library, include the cudnn_frontend header file include/cudnn_frontend.h into your compilation unit.

For Python users, run import cudnn

Build:

Dependencies

With the release of v1.0, we are bumping up the minimum supported cudnn version to 8.5.0

cuda can be downloaded from the nvidia dev-zone

cudnn can be installed from - nvidia dev-zone - pypi wheels

Minimum python version needed 3.6 The python binding compilation requires development package which can be installed by running apt-get install python-dev.

To run the Python samples, you will need the dependencies mentioned in requirements.txt. This can be be installed by running: pip install -r requirements.txt

Python API

pip wheel installation

Download the pip wheel corresponding to your python installation.

pip install nvidia_cudnn_frontend

Source installation:

Install FE python API by running:

pip install -v git+https://github.com/NVIDIA/cudnn-frontend.git

Above command picks cuda and cudnn from default system paths.

To provide a custom CUDA installation path, use environment variable: CUDAToolkit_ROOT.
To provide a custom CUDNN installation path, use environment variable: CUDNN_PATH.

Checking the installation

To test whether installation is successful, run:

pytest test/python

NOTE: Only v1.0 API is exposed via python bindings.

C++ API

C++ API is header only library.

The root CMakeLists.txt can be used as reference to include the cudnn_frontend in your project's build system.

Building samples

The following compilation steps are only required for building the samples.

Provide CUDA installation path according to: https://cmake.org/cmake/help/latest/module/FindCUDAToolkit.html

Provide CUDNN installation path using CUDNN_PATH env variable or cmake parameter.

CUDNN_PATH has the cudnn installation:

  • Headers are in CUDNN_PATH/include.
  • Libraries are in CUDNN_PATH/lib or CUDNN_PATH/lib64 or CUDNN_PATH/lib/x64.

For a in-source build,

mkdir build
cd build
cmake -DCUDNN_PATH=/path/to/cudnn -DCUDAToolkit_ROOT=/path/to/cuda  ../
cmake --build . -j16
bin/samples

To skip building samples, use -DCUDNN_FRONTEND_BUILD_SAMPLES=OFF.

To skip building python bindings, use -DCUDNN_FRONTEND_BUILD_PYTHON_BINDINGS=OFF.

To add debug symbols, use -DCMAKE_BUILD_TYPE=Debug.

In case, you have a stale cmake cache and want to update the cudnn/cuda paths, please delete the cmake cache (or build directory and redo the above steps).

Debugging

For initial debugging, we recommend turning on the cudnn FE logging and checking for warnings and errors. cuDNN Frontend API logging records execution flow through cuDNN frontend API. This functionality is disabled by default, and can be enabled through methods described in this section.

Method 1: Using Environment Variables:

Environment variables CUDNN_FRONTEND_LOG_INFO=0 CUDNN_FRONTEND_LOG_INFO=1
CUDNN_FRONTEND_LOG_FILE not set No Logging No Logging
CUDNN_FRONTEND_LOG_FILE set to stdout or stderr No Logging Logging to cout or cerr
CUDNN_FRONTEND_LOG_FILE set to filename.txt No Logging Logging to the filename

Method 2: Using API calls:

Calling cudnn_frontend::isLoggingEnabled() = true|false has same effect of setting the environment variable. Calling cudnn_frontend::getStream() = stream_name can be used to assign the output stream directly.

For further debugging, please turn on the cudnn backend logs described here https://docs.nvidia.com/deeplearning/cudnn/latest/reference/troubleshooting.html#error-reporting-and-api-logging

Documentation

Contributing:

Please refer to our contribution guide

Feedback

Support, resources, and information about cuDNN can be found online at https://developer.nvidia.com/cudnn.

Also, bugs and RFEs can be reported in the issues section.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

nvidia_cudnn_frontend-1.8.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (1.5 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

nvidia_cudnn_frontend-1.8.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (1.5 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

nvidia_cudnn_frontend-1.8.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (1.5 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

nvidia_cudnn_frontend-1.8.0-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (1.5 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

nvidia_cudnn_frontend-1.8.0-cp38-cp38-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (1.5 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

File details

Details for the file nvidia_cudnn_frontend-1.8.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for nvidia_cudnn_frontend-1.8.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 5e0821093ad1a12b692d62678e6c3e1bd207708c58ebf1db10ccd32cef5bb258
MD5 3842b85573f8db1c7622e7a243ec4a11
BLAKE2b-256 b4e7b97f83a3ddafea569acc17bbc3362c0bfebe899adfa583ac849310b822df

See more details on using hashes here.

File details

Details for the file nvidia_cudnn_frontend-1.8.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for nvidia_cudnn_frontend-1.8.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 e3abf6a1dc03999be12714cbaf719e1d5b5a59d9fd5fa5214bca1022f9edcb7f
MD5 d28715d62ee37e462d2ac322d4675892
BLAKE2b-256 f447e72cc830407f462b9e84fd8f5c77acd1e47fab6bf239abfb37adcc724ce1

See more details on using hashes here.

File details

Details for the file nvidia_cudnn_frontend-1.8.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for nvidia_cudnn_frontend-1.8.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 194d7fcd05daa7bae193b05cbaa2967ff88e74e54fe1f6522aad186f51cf7d57
MD5 cabdc283395a69ca790f1553aa349ab8
BLAKE2b-256 a70f0cbae12be14c176e7c9dece162a39d7e7c5ce3a0b0bcc9bf6b555a5753eb

See more details on using hashes here.

File details

Details for the file nvidia_cudnn_frontend-1.8.0-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for nvidia_cudnn_frontend-1.8.0-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 e4fb765f1d07a66f6a4fb3b316447ad50242d75183e9bb7d75e7af0c98ce05af
MD5 d82a286313cb287a407c47b66a267bdb
BLAKE2b-256 a0c987028cdefdec3f3abed23bb8eb2f2e59d5ab26733210c62ba14277e20bf8

See more details on using hashes here.

File details

Details for the file nvidia_cudnn_frontend-1.8.0-cp38-cp38-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for nvidia_cudnn_frontend-1.8.0-cp38-cp38-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 41c1b5afc15671733a3a2c873c8e4785042ee715539f8bafac3d26b6d0994bff
MD5 18f830bde80b6a8123aa1877e841fa49
BLAKE2b-256 9312d3ecbf6cbeb036eb4a1066c7a2b22a453ca51e907564969d057c9a1cb057

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page