A video decoder for PyTorch
Project description
Installation | Simple Example | Detailed Example | Documentation | Contributing | License
TorchCodec
TorchCodec is a Python library for decoding videos into PyTorch tensors. It aims to be fast, easy to use, and well integrated into the PyTorch ecosystem. If you want to use PyTorch to train ML models on videos, TorchCodec is how you turn those videos into data.
We achieve these capabilities through:
- Pythonic APIs that mirror Python and PyTorch conventions.
- Relying on FFmpeg to do the decoding. TorchCodec uses the version of FFmpeg you already have installed. FFmpeg is a mature library with broad coverage available on most systems. It is, however, not easy to use. TorchCodec abstracts FFmpeg's complexity to ensure it is used correctly and efficiently.
- Returning data as PyTorch tensors, ready to be fed into PyTorch transforms or used directly to train models.
[!NOTE] ������ TorchCodec is still in early development stage and some APIs may be updated in future versions without a deprecation cycle, depending on user feedback. If you have any suggestions or issues, please let us know by opening an issue!
Using TorchCodec
Here's a condensed summary of what you can do with TorchCodec. For a more detailed example, check out our documentation!
from torchcodec.decoders import SimpleVideoDecoder
decoder = SimpleVideoDecoder("path/to/video.mp4")
decoder.metadata
# VideoStreamMetadata:
# num_frames: 250
# duration_seconds: 10.0
# bit_rate: 31315.0
# codec: h264
# average_fps: 25.0
# ... (truncated output)
len(decoder) # == decoder.metadata.num_frames!
# 250
decoder.metadata.average_fps # Note: instantaneous fps can be higher or lower
# 25.0
# Simple Indexing API
decoder[0] # uint8 tensor of shape [C, H, W]
decoder[0 : -1 : 20] # uint8 stacked tensor of shape [N, C, H, W]
# Iterate over frames:
for frame in decoder:
pass
# Indexing, with PTS and duration info
decoder.get_frame_at(len(decoder) - 1)
# Frame:
# data (shape): torch.Size([3, 400, 640])
# pts_seconds: 9.960000038146973
# duration_seconds: 0.03999999910593033
decoder.get_frames_at(start=10, stop=30, step=5)
# FrameBatch:
# data (shape): torch.Size([4, 3, 400, 640])
# pts_seconds: tensor([0.4000, 0.6000, 0.8000, 1.0000])
# duration_seconds: tensor([0.0400, 0.0400, 0.0400, 0.0400])
# Time-based indexing with PTS and duration info
decoder.get_frame_displayed_at(pts_seconds=2)
# Frame:
# data (shape): torch.Size([3, 400, 640])
# pts_seconds: 2.0
# duration_seconds: 0.03999999910593033
You can use the following snippet to generate a video with FFmpeg and tryout TorchCodec:
fontfile=/usr/share/fonts/dejavu-sans-mono-fonts/DejaVuSansMono-Bold.ttf
output_video_file=/tmp/output_video.mp4
ffmpeg -f lavfi -i \
color=size=640x400:duration=10:rate=25:color=blue \
-vf "drawtext=fontfile=${fontfile}:fontsize=30:fontcolor=white:x=(w-text_w)/2:y=(h-text_h)/2:text='Frame %{frame_num}'" \
${output_video_file}
Installing TorchCodec
Note: if you're on MacOS, you'll need to build from source. Instructions below assume you're on Linux.
First install the latest stable version of PyTorch following the official instructions.
Then:
pip install torchcodec
You will also need FFmpeg installed on your system, and TorchCodec decoding capabilities are determined by your underlying FFmpeg installation. There are different options to install FFmpeg e.g.:
conda install ffmpeg
# or
conda install ffmpeg -c conda-forge
Your Linux distribution probably comes with FFmpeg pre-installed as well. TorchCodec supports all major FFmpeg version in [4, 7].
Planned future work
We are actively working on the following features:
- Ship wheels for MacOS, so
that MacOS users can
pip install torchcodec
. For now this is only supported on Linux, but MacOS users can build from source. - GPU decoding
- Audio decoding
Let us know if you have any feature requests by opening an issue!
Contributing
We welcome contributions to TorchCodec! Please see our contributing guide for more details.
License
TorchCodec is released under the BSD 3 license.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
File details
Details for the file TorchCodec-0.0.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
.
File metadata
- Download URL: TorchCodec-0.0.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
- Upload date:
- Size: 657.5 kB
- Tags: CPython 3.12, manylinux: glibc 2.17+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.12.4
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3775bb4c27f1de3106722f8a59b9a5236aa664c89f547659c3ecaf01e058b614 |
|
MD5 | 20fc5cbab46adee234a9bbd4c0318509 |
|
BLAKE2b-256 | 68a3bcd49271800d33a1f94496dea78649887d19658ea4d14984be4c2e9a1096 |
Provenance
File details
Details for the file TorchCodec-0.0.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
.
File metadata
- Download URL: TorchCodec-0.0.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
- Upload date:
- Size: 659.1 kB
- Tags: CPython 3.11, manylinux: glibc 2.17+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.12.4
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5ad63f29f711e83ac084d6fd36a906cad5eb30e5df0d6b0894590127a567bf69 |
|
MD5 | 5d727da766e1c97aa31bac328ca49dfb |
|
BLAKE2b-256 | 2ca73175936b9604fe64a94f8321958838a4d0c7c347c8287d43782453f50f99 |
Provenance
File details
Details for the file TorchCodec-0.0.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
.
File metadata
- Download URL: TorchCodec-0.0.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
- Upload date:
- Size: 655.1 kB
- Tags: CPython 3.10, manylinux: glibc 2.17+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.12.4
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2d597d09f5a36ab30ed6f3d7d16e62353cf822f831db19757c85281967d0e898 |
|
MD5 | 4d20d63a107c5db002172f147b231f03 |
|
BLAKE2b-256 | 5426f9cccdeaaa37bcf03eac7a2d03a74eef57a8f01ed30686209f87c7f79c39 |
Provenance
File details
Details for the file TorchCodec-0.0.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
.
File metadata
- Download URL: TorchCodec-0.0.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
- Upload date:
- Size: 655.5 kB
- Tags: CPython 3.9, manylinux: glibc 2.17+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.12.4
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | e7add1c6f304f8b2ad159603bada40c8605915ca9f149a21b5e056c52a91587c |
|
MD5 | eb2946d82b0e939040da23b6152b7e2a |
|
BLAKE2b-256 | fd0b26c244a02916213e8c627af2e4f0b6f2f96717e391884ec51c3d497bced6 |