Skip to main content

No project description provided

Project description

HF Transfer

Speed up file transfers with the Hub.

DISCLAIMER

This library is a power user tool, to go beyond ~500MB/s on very high bandwidth network, where Python cannot cap out the available bandwidth.

This is not meant to be a general usability tool. Progressbar is experimentally implemented as callback.

Please file issues only if there's an issue on the underlying downloaded file.

Contributing

python3 -m venv ~/.venv/hf_transfer
source ~/.venv/hf_transfer/bin/activate
pip install maturin
maturin develop

huggingface_hub

If you are working on changes with huggingface_hub

git clone git@github.com:huggingface/huggingface_hub.git
# git clone https://github.com/huggingface/huggingface_hub.git

cd huggingface_hub
python3 -m pip install -e ".[quality]"

You can use the following test script:

import os

# os.environ["HF_ENDPOINT"] = "http://localhost:5564"
os.environ["HF_HUB_ENABLE_HF_TRANSFER"] = "1"

from huggingface_hub import HfApi, logging

logging.set_verbosity_debug()
hf = HfApi()
hf.upload_file(path_or_fileobj="/path/to/my/repo/some_file", path_in_repo="some_file", repo_id="my/repo", repo_type="model")

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

hf_transfer-0.1.4rc1.tar.gz (20.6 kB view details)

Uploaded Source

Built Distributions

hf_transfer-0.1.4rc1-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.9 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

hf_transfer-0.1.4rc1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.9 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

hf_transfer-0.1.4rc1-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.9 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

hf_transfer-0.1.4rc1-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.9 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

hf_transfer-0.1.4rc1-cp312-none-win_amd64.whl (1.3 MB view details)

Uploaded CPython 3.12 Windows x86-64

hf_transfer-0.1.4rc1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.9 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ x86-64

hf_transfer-0.1.4rc1-cp311-none-win_amd64.whl (1.3 MB view details)

Uploaded CPython 3.11 Windows x86-64

hf_transfer-0.1.4rc1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.9 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

hf_transfer-0.1.4rc1-cp311-cp311-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl (3.3 MB view details)

Uploaded CPython 3.11 macOS 10.9+ universal2 (ARM64, x86-64) macOS 10.9+ x86-64 macOS 11.0+ ARM64

hf_transfer-0.1.4rc1-cp310-none-win_amd64.whl (1.3 MB view details)

Uploaded CPython 3.10 Windows x86-64

hf_transfer-0.1.4rc1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.9 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

hf_transfer-0.1.4rc1-cp310-cp310-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl (3.3 MB view details)

Uploaded CPython 3.10 macOS 10.9+ universal2 (ARM64, x86-64) macOS 10.9+ x86-64 macOS 11.0+ ARM64

hf_transfer-0.1.4rc1-cp39-none-win_amd64.whl (1.3 MB view details)

Uploaded CPython 3.9 Windows x86-64

hf_transfer-0.1.4rc1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.9 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

hf_transfer-0.1.4rc1-cp39-cp39-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl (3.3 MB view details)

Uploaded CPython 3.9 macOS 10.9+ universal2 (ARM64, x86-64) macOS 10.9+ x86-64 macOS 11.0+ ARM64

hf_transfer-0.1.4rc1-cp38-none-win_amd64.whl (1.3 MB view details)

Uploaded CPython 3.8 Windows x86-64

hf_transfer-0.1.4rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.9 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

hf_transfer-0.1.4rc1-cp38-cp38-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl (3.3 MB view details)

Uploaded CPython 3.8 macOS 10.9+ universal2 (ARM64, x86-64) macOS 10.9+ x86-64 macOS 11.0+ ARM64

hf_transfer-0.1.4rc1-cp37-none-win_amd64.whl (1.3 MB view details)

Uploaded CPython 3.7 Windows x86-64

hf_transfer-0.1.4rc1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.9 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

hf_transfer-0.1.4rc1-cp37-cp37m-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl (3.3 MB view details)

Uploaded CPython 3.7m macOS 10.9+ universal2 (ARM64, x86-64) macOS 10.9+ x86-64 macOS 11.0+ ARM64

File details

Details for the file hf_transfer-0.1.4rc1.tar.gz.

File metadata

  • Download URL: hf_transfer-0.1.4rc1.tar.gz
  • Upload date:
  • Size: 20.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: maturin/1.3.1

File hashes

Hashes for hf_transfer-0.1.4rc1.tar.gz
Algorithm Hash digest
SHA256 4422f7005c43e6658ba7bc9535285591316e50b92d3ef8769fcfd7b17df3a7ab
MD5 d6e78a5a69fa1adc4b377bb7254eecb8
BLAKE2b-256 986530ebc8b06068479ec5f1aa28ed2937180bb02cf7002dce9044b714421e4b

See more details on using hashes here.

File details

Details for the file hf_transfer-0.1.4rc1-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for hf_transfer-0.1.4rc1-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 a4264f44f0e65194d8aa5634312d734a06458a788582065f90755f7ad6a3dfd3
MD5 db6cae54229c21a83c9827cd5719b41f
BLAKE2b-256 75833fc8e5b404cc105bb16ce0f44c6b9c402fef81c1ecea1e9cf75938efed28

See more details on using hashes here.

File details

Details for the file hf_transfer-0.1.4rc1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for hf_transfer-0.1.4rc1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 a0aa87da1c250afc096a11ba5beb3e64fd29e0d6f38baa19959c707ee92b402d
MD5 b27cd4ee8a865da742edf61112addee2
BLAKE2b-256 b09fabc3eba16a08d36eb6b247a11e781b1ac52abebeb33cd8c6a26393493302

See more details on using hashes here.

File details

Details for the file hf_transfer-0.1.4rc1-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for hf_transfer-0.1.4rc1-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 c2c9b3a2d7ff5aa366ad93ee6e448bf382b4f7218cae6aa3c57a4431cf45c52c
MD5 0a2f678d53dbab64c59ec47b41cc4298
BLAKE2b-256 356b20bbd70a6e66a2e5dcb26b65ae6d3ee84036ae63acd450da3a3e835224df

See more details on using hashes here.

File details

Details for the file hf_transfer-0.1.4rc1-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for hf_transfer-0.1.4rc1-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 c2f9a246f9b07665d9c7b08e7a6333ceeef844190e7ccf5457e0cedfec49fca8
MD5 3ae8d759a3d58971e013e303dabb78cc
BLAKE2b-256 2fc0919caba8c96f539fb7fb1edf3173a846fe0a5e9dbcd8d86b42a935c44ec1

See more details on using hashes here.

File details

Details for the file hf_transfer-0.1.4rc1-cp312-none-win_amd64.whl.

File metadata

File hashes

Hashes for hf_transfer-0.1.4rc1-cp312-none-win_amd64.whl
Algorithm Hash digest
SHA256 3bd741dce5ac2c47e6a0cacebbae637096b0b60af627016eb57eccac32ec46d9
MD5 b70323102aaf97fe085ca08fb3c972e6
BLAKE2b-256 62033764c9d2d3daf7c2f2979b1123a7987fe34642f9e7330d88cfca4e68ee76

See more details on using hashes here.

File details

Details for the file hf_transfer-0.1.4rc1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for hf_transfer-0.1.4rc1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 12dd82e1193564ab679650d65b29193edcac5e26322c8b3c0c646076a426c2b2
MD5 a3b1591f39d4ac4b8ace15884306deeb
BLAKE2b-256 07e0fe1103c8e506ed786cc8b9122e0448234aa674f86bdbc0fc4c2fdbec9a7b

See more details on using hashes here.

File details

Details for the file hf_transfer-0.1.4rc1-cp311-none-win_amd64.whl.

File metadata

File hashes

Hashes for hf_transfer-0.1.4rc1-cp311-none-win_amd64.whl
Algorithm Hash digest
SHA256 6083606eb04a541fd904c7ef58e576d7be927e34fd10514654d075cb570face9
MD5 b25eae6dc88dc6ceabefb91a5cb2c0ca
BLAKE2b-256 683a23ff9c6c3dcdbce6b765854caf0e41a724febad63d0e26d64ca1190a4eea

See more details on using hashes here.

File details

Details for the file hf_transfer-0.1.4rc1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for hf_transfer-0.1.4rc1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 0d941680df1b4273a790d55aa4630df846d392911f6498430267b26932d1f6b3
MD5 683512fa56d32ce22d92a68aa01b3e2b
BLAKE2b-256 d62701d2f846c12623259d0137539dc896cb339a01670ad0a1b78ce955d50489

See more details on using hashes here.

File details

Details for the file hf_transfer-0.1.4rc1-cp311-cp311-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl.

File metadata

File hashes

Hashes for hf_transfer-0.1.4rc1-cp311-cp311-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm Hash digest
SHA256 f5312655ee18d625a1a3c0d0aca596aac888ad46860e0a6f0adca532c9066d73
MD5 ab9a98c2c30d8a0599612d888d2b3bcb
BLAKE2b-256 29b5fe85d52e3cdd108268234bd50e0936e824e91da7108b89f131dc7c4281c2

See more details on using hashes here.

File details

Details for the file hf_transfer-0.1.4rc1-cp310-none-win_amd64.whl.

File metadata

File hashes

Hashes for hf_transfer-0.1.4rc1-cp310-none-win_amd64.whl
Algorithm Hash digest
SHA256 5164bc5d2c5096355e43e854f255b509756d5bbaa7a12eb869135f94a1233c5b
MD5 109d4c3f9fa5c728a518b3c4939a9cbe
BLAKE2b-256 6698cb6da3a35220d8647f60eefbde5f314dad0689c5efc4dd067dd7779f9e19

See more details on using hashes here.

File details

Details for the file hf_transfer-0.1.4rc1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for hf_transfer-0.1.4rc1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 320861c7ba5ee2b04d3e707260b622c912c563e941eae07ec453d3e6084a1da7
MD5 93dcc30bc15333b96ca4f3c5554c2638
BLAKE2b-256 22f88fce18231acd57a1fbdfbae95d4acaf6be78c104e978993ead78a0e69737

See more details on using hashes here.

File details

Details for the file hf_transfer-0.1.4rc1-cp310-cp310-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl.

File metadata

File hashes

Hashes for hf_transfer-0.1.4rc1-cp310-cp310-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm Hash digest
SHA256 c80099f77b44b52deea39b178ac978020b39067924e7092505e35b0568d6cc7a
MD5 c4978dfeca7b8318f41e77b1d5d17d80
BLAKE2b-256 131be860e3e81057508c752ca44f876bebc8c78476b43267a8e5c766345a2d97

See more details on using hashes here.

File details

Details for the file hf_transfer-0.1.4rc1-cp39-none-win_amd64.whl.

File metadata

File hashes

Hashes for hf_transfer-0.1.4rc1-cp39-none-win_amd64.whl
Algorithm Hash digest
SHA256 a0cf04bfe6500f13bf5def1f1229253b0168887c2900bc682ac9cb575318b9ca
MD5 6853baf5c4c651ba6440127bb0cc9cc2
BLAKE2b-256 2739d7f5caf41e5285f258fdc90787783919f93cf57e84211ecb392370dfe8af

See more details on using hashes here.

File details

Details for the file hf_transfer-0.1.4rc1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for hf_transfer-0.1.4rc1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 f8be2780b3bae08ecae0b86c8bc7d2e92f0f7db0110dba95ba558e04383b2912
MD5 ba8a30e7792b6bf1bdb13889f784a445
BLAKE2b-256 471d7e5c57376d6f21bb11e5fe4b09dd05111fb985dd46d91bf1a543ad2c0256

See more details on using hashes here.

File details

Details for the file hf_transfer-0.1.4rc1-cp39-cp39-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl.

File metadata

File hashes

Hashes for hf_transfer-0.1.4rc1-cp39-cp39-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm Hash digest
SHA256 f28f9510db67cae4c3d0bac027508df4fd2fd30e9c3ab8c618e342dc689cb516
MD5 03890bab38a73f9e6d73a7eb47601589
BLAKE2b-256 5bb5cdced2158220cde10b833365550069fe1b1e64e7c94549654f27f8ed971b

See more details on using hashes here.

File details

Details for the file hf_transfer-0.1.4rc1-cp38-none-win_amd64.whl.

File metadata

File hashes

Hashes for hf_transfer-0.1.4rc1-cp38-none-win_amd64.whl
Algorithm Hash digest
SHA256 cc1c1c5eb35ea7fcedfb42b4333e6e33cb496ceb55c931a770efe3f6772e5875
MD5 2d415d6e2c013e4d099fcd77836f601a
BLAKE2b-256 5108a6f51fcd8419a81f3cb030eebe382aa912b596487be50b1dd4d8d9a66d3f

See more details on using hashes here.

File details

Details for the file hf_transfer-0.1.4rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for hf_transfer-0.1.4rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 bb6fb7170f0bd4b57904e24a5a0b78a2802f8d1dfa47fa66a1a197354d689568
MD5 2fe03bf842a686a6a18c535d1935189d
BLAKE2b-256 a4137f9de19b034c56b56e17a6c2754c5bf589ebce26886ac8b4ddcc58097c5c

See more details on using hashes here.

File details

Details for the file hf_transfer-0.1.4rc1-cp38-cp38-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl.

File metadata

File hashes

Hashes for hf_transfer-0.1.4rc1-cp38-cp38-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm Hash digest
SHA256 709c42a5b84878e28dd878f8e2d3dea5e73c4753da5343f4fc1b5616a2ac31d8
MD5 ba348ba43cd6a3ba270176436c6ddaed
BLAKE2b-256 6858957c5076733733cce52113974c6c6069c323048f651946dc5831a5b05f2c

See more details on using hashes here.

File details

Details for the file hf_transfer-0.1.4rc1-cp37-none-win_amd64.whl.

File metadata

File hashes

Hashes for hf_transfer-0.1.4rc1-cp37-none-win_amd64.whl
Algorithm Hash digest
SHA256 3f19bcadba83e12a7652cde66f094b4a1d7d5f0e1f3a7b958697c5cc20bbed43
MD5 2aa995e914b87f452ae3f620c913fa8c
BLAKE2b-256 971cbc5a9f35a6c2dd0ce5cd238361dd7596db7b2ddb886ff0bb3b0d7c682d82

See more details on using hashes here.

File details

Details for the file hf_transfer-0.1.4rc1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for hf_transfer-0.1.4rc1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 5730a5914da683478df9b4f7eb90da3c0edd3b596d257a352504e2c56b961aaf
MD5 db3521e3ca2deaa9e0f9d98e272056fd
BLAKE2b-256 f19d66b69d8d856bb3202476d290c560b8a5358ce3854162d81eff895b2f98f1

See more details on using hashes here.

File details

Details for the file hf_transfer-0.1.4rc1-cp37-cp37m-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl.

File metadata

File hashes

Hashes for hf_transfer-0.1.4rc1-cp37-cp37m-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm Hash digest
SHA256 75f86c8c41381a3be3a238b5ab92cc65c223f0a36eccf51f28a2094ccb4f8b5d
MD5 8fcd9f837b500c696d1be75ff9c790e1
BLAKE2b-256 f826b06662ca42f2137f207c93b95592dbe2dec5443064542276d678ff84d8b7

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page