CUDNN FrontEnd python library
Project description
cuDNN FrontEnd(FE) API
Introduction
The cuDNN FrontEnd(FE) API is a C++ header-only library that wraps the cuDNN C backend API. Both the FE and backend APIs are entry points to the same set of functionality that is commonly referred to as the "graph API".
While there are two entry points to the graph API (i.e. backend and frontend), it is expected that most users will use the FE API. Reasons being:
- FE API is less verbose without loss of control. All functionality accessible through the backend API is also accessible through the FE API.
- FE API adds functionality on top of the backend API, like errata filters and autotuning.
Also, for those using backend API, FE API source and samples can serve as reference implementation.
In FE v1.0 API, users can describe multiple operations that form subgraph through a persistent cudnn_frontend::graph::Graph
object. Unlike the FE v0.x API, users don't need to worry about specifying shapes and sizes of the intermediate virtual tensors. FE v1.0 API extends the groundwork of earlier versions and introduces a new set of APIs to further simplify the workflow. For detailed information of FE v1.0 API, see README.FE.1.0.md.
Additionally, FE v1.0 API provides python bindings to all API through pybind11. It is recommended that new users of cuDNN start with the frontend v1.0 API. See samples/cpp
and samples/python
for more details on its usage.
Usage
For c++ users, in order to include the entire library, include the cudnn_frontend header file include/cudnn_frontend.h
into your compilation unit.
For Python users, run import cudnn
Build:
Dependencies
With the release of v1.0, we are bumping up the minimum supported cudnn version to 8.5.0
cuda can be downloaded from the nvidia dev-zone
cudnn can be installed from - nvidia dev-zone - pypi wheels
Minimum python version needed 3.6
The python binding compilation requires development package which can be installed by running apt-get install python-dev
.
To run the Python samples, you will need the dependencies mentioned in requirements.txt
. This can be be installed by running:
pip install -r requirements.txt
Python API
pip wheel installation
Download the pip wheel corresponding to your python installation.
pip install nvidia_cudnn_frontend
Source installation:
Install FE python API by running:
pip install -v git+https://github.com/NVIDIA/cudnn-frontend.git
Above command picks cuda and cudnn from default system paths.
To provide a custom CUDA installation path, use environment variable: CUDAToolkit_ROOT
.
To provide a custom CUDNN installation path, use environment variable: CUDNN_PATH
.
Checking the installation
To test whether installation is successful, run:
pytest test/python
NOTE: Only v1.0 API is exposed via python bindings.
C++ API
C++ API is header only library.
The root CMakeLists.txt can be used as reference to include the cudnn_frontend in your project's build system.
Building samples
The following compilation steps are only required for building the samples.
Provide CUDA installation path according to: https://cmake.org/cmake/help/latest/module/FindCUDAToolkit.html
Provide CUDNN installation path using CUDNN_PATH env variable or cmake parameter.
CUDNN_PATH has the cudnn installation:
- Headers are in CUDNN_PATH/include.
- Libraries are in CUDNN_PATH/lib or CUDNN_PATH/lib64 or CUDNN_PATH/lib/x64.
For a in-source build,
mkdir build
cd build
cmake -DCUDNN_PATH=/path/to/cudnn -DCUDAToolkit_ROOT=/path/to/cuda ../
cmake --build . -j16
bin/samples
To skip building samples, use -DCUDNN_FRONTEND_BUILD_SAMPLES=OFF
.
To skip building python bindings, use -DCUDNN_FRONTEND_BUILD_PYTHON_BINDINGS=OFF
.
To add debug symbols, use -DCMAKE_BUILD_TYPE=Debug
.
In case, you have a stale cmake cache and want to update the cudnn/cuda paths, please delete the cmake cache (or build directory and redo the above steps).
Debugging
For initial debugging, we recommend turning on the cudnn FE logging and checking for warnings and errors. cuDNN Frontend API logging records execution flow through cuDNN frontend API. This functionality is disabled by default, and can be enabled through methods described in this section.
Method 1: Using Environment Variables:
Environment variables | CUDNN_FRONTEND_LOG_INFO=0 | CUDNN_FRONTEND_LOG_INFO=1 |
---|---|---|
CUDNN_FRONTEND_LOG_FILE not set | No Logging | No Logging |
CUDNN_FRONTEND_LOG_FILE set to stdout or stderr | No Logging | Logging to cout or cerr |
CUDNN_FRONTEND_LOG_FILE set to filename.txt | No Logging | Logging to the filename |
Method 2: Using API calls:
Calling cudnn_frontend::isLoggingEnabled() = true|false
has same effect of setting the environment variable.
Calling cudnn_frontend::getStream() = stream_name
can be used to assign the output stream directly.
For further debugging, please turn on the cudnn backend logs described here https://docs.nvidia.com/deeplearning/cudnn/latest/reference/troubleshooting.html#error-reporting-and-api-logging
Documentation
- See README.FE.1.0.md for v1.0 API documentation.
- See README.FE.0.x.md for v0.x API documentation.
Contributing:
Please refer to our contribution guide
Feedback
Support, resources, and information about cuDNN can be found online at https://developer.nvidia.com/cudnn.
Also, bugs and RFEs can be reported in the issues section.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
File details
Details for the file nvidia_cudnn_frontend-1.8.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
.
File metadata
- Download URL: nvidia_cudnn_frontend-1.8.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
- Upload date:
- Size: 1.5 MB
- Tags: CPython 3.12, manylinux: glibc 2.27+ x86-64, manylinux: glibc 2.28+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.0.0 CPython/3.10.12
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5e0821093ad1a12b692d62678e6c3e1bd207708c58ebf1db10ccd32cef5bb258 |
|
MD5 | 3842b85573f8db1c7622e7a243ec4a11 |
|
BLAKE2b-256 | b4e7b97f83a3ddafea569acc17bbc3362c0bfebe899adfa583ac849310b822df |
File details
Details for the file nvidia_cudnn_frontend-1.8.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
.
File metadata
- Download URL: nvidia_cudnn_frontend-1.8.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
- Upload date:
- Size: 1.5 MB
- Tags: CPython 3.11, manylinux: glibc 2.27+ x86-64, manylinux: glibc 2.28+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.0.0 CPython/3.10.12
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | e3abf6a1dc03999be12714cbaf719e1d5b5a59d9fd5fa5214bca1022f9edcb7f |
|
MD5 | d28715d62ee37e462d2ac322d4675892 |
|
BLAKE2b-256 | f447e72cc830407f462b9e84fd8f5c77acd1e47fab6bf239abfb37adcc724ce1 |
File details
Details for the file nvidia_cudnn_frontend-1.8.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
.
File metadata
- Download URL: nvidia_cudnn_frontend-1.8.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
- Upload date:
- Size: 1.5 MB
- Tags: CPython 3.10, manylinux: glibc 2.27+ x86-64, manylinux: glibc 2.28+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.0.0 CPython/3.10.12
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 194d7fcd05daa7bae193b05cbaa2967ff88e74e54fe1f6522aad186f51cf7d57 |
|
MD5 | cabdc283395a69ca790f1553aa349ab8 |
|
BLAKE2b-256 | a70f0cbae12be14c176e7c9dece162a39d7e7c5ce3a0b0bcc9bf6b555a5753eb |
File details
Details for the file nvidia_cudnn_frontend-1.8.0-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
.
File metadata
- Download URL: nvidia_cudnn_frontend-1.8.0-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
- Upload date:
- Size: 1.5 MB
- Tags: CPython 3.9, manylinux: glibc 2.27+ x86-64, manylinux: glibc 2.28+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.0.0 CPython/3.10.12
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | e4fb765f1d07a66f6a4fb3b316447ad50242d75183e9bb7d75e7af0c98ce05af |
|
MD5 | d82a286313cb287a407c47b66a267bdb |
|
BLAKE2b-256 | a0c987028cdefdec3f3abed23bb8eb2f2e59d5ab26733210c62ba14277e20bf8 |
File details
Details for the file nvidia_cudnn_frontend-1.8.0-cp38-cp38-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
.
File metadata
- Download URL: nvidia_cudnn_frontend-1.8.0-cp38-cp38-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
- Upload date:
- Size: 1.5 MB
- Tags: CPython 3.8, manylinux: glibc 2.27+ x86-64, manylinux: glibc 2.28+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.0.0 CPython/3.10.12
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 41c1b5afc15671733a3a2c873c8e4785042ee715539f8bafac3d26b6d0994bff |
|
MD5 | 18f830bde80b6a8123aa1877e841fa49 |
|
BLAKE2b-256 | 9312d3ecbf6cbeb036eb4a1066c7a2b22a453ca51e907564969d057c9a1cb057 |