Skip to main content

Checksum support for zarrs stored in various backends

Project description

zarr_checksum

Algorithms for calculating a zarr checksum against local or cloud storage

Install

pip install zarr-checksum

Usage

CLI

To calculate the checksum for a local zarr archive

zarrsum local <directory>

To calculate the checksum for a remote (S3) zarr archive

zarrsum remote s3://your_bucket/prefix_to_zarr

Python

To calculate the checksum for a local zarr archive

from zarr_checksum import compute_zarr_checksum
from zarr_checksum.generators import yield_files_local, yield_files_s3

# Local
checksum = compute_zarr_checksum(yield_files_local("local_path"))

# Remote
checksum = compute_zarr_checksum(
    yield_files_s3(
        bucket="your_bucket",
        prefix="prefix_to_zarr",
        # Credentials can also be passed via environment variables
        credentials={
            aws_access_key_id: "youraccesskey",
            aws_secret_access_key: "yoursecretkey",
            region_name: "us-east-1",
        }
    )
)

Access checksum information

>>> checksum.digest
'c228464f432c4376f0de6ddaea32650c-37481--38757151179'
>>> checksum.md5
'c228464f432c4376f0de6ddaea32650c'
>>> checksum.count
37481
>>> checksum.size
38757151179

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

zarr_checksum-0.2.12.tar.gz (11.6 kB view details)

Uploaded Source

Built Distribution

zarr_checksum-0.2.12-py3-none-any.whl (13.6 kB view details)

Uploaded Python 3

File details

Details for the file zarr_checksum-0.2.12.tar.gz.

File metadata

  • Download URL: zarr_checksum-0.2.12.tar.gz
  • Upload date:
  • Size: 11.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.7.1 CPython/3.12.0 Linux/6.2.0-1016-azure

File hashes

Hashes for zarr_checksum-0.2.12.tar.gz
Algorithm Hash digest
SHA256 2fbaf98f6333c3f5459386a4e38a3acc750746ee6939db24ccd9050a23b6186e
MD5 9d51c788dfc571246f5a4d88931bccfc
BLAKE2b-256 7ec27bf17d869ca04839fa5715b0818f96bf2e9895675b7f1375696e90d6c488

See more details on using hashes here.

Provenance

File details

Details for the file zarr_checksum-0.2.12-py3-none-any.whl.

File metadata

  • Download URL: zarr_checksum-0.2.12-py3-none-any.whl
  • Upload date:
  • Size: 13.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.7.1 CPython/3.12.0 Linux/6.2.0-1016-azure

File hashes

Hashes for zarr_checksum-0.2.12-py3-none-any.whl
Algorithm Hash digest
SHA256 1c984367bcbc18033925dfe6dab41a7670286ae29364096b143301e32d30ab37
MD5 7237649a383e2efa4768540e97de8583
BLAKE2b-256 c678ec967b057a5a798732495dfc32020fd20c73e0692b4307f0f595fa60823a

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page