Skip to main content

Create virtual Zarr stores from archival data using xarray API

Project description

VirtualiZarr

VirtualiZarr creates virtual Zarr stores for cloud-friendly access to archival data, using familiar xarray syntax.

VirtualiZarr (pronounced like "virtualize" but more piratey) grew out of discussions on the kerchunk repository, and is an attempt to provide the game-changing power of kerchunk in a zarr-native way, and with a familiar array-like API.

You now have a choice between using VirtualiZarr and Kerchunk: VirtualiZarr provides almost all the same features as Kerchunk.

Please see the documentation

Development Status and Roadmap

VirtualiZarr version 1 (mostly) achieves feature parity with kerchunk's logic for combining datasets, providing an easier way to manipulate kerchunk references in memory and generate kerchunk reference files on disk.

Future VirtualiZarr development will focus on generalizing and upstreaming useful concepts into the Zarr specification, the Zarr-Python library, Xarray, and possibly some new packages.

We have a lot of ideas, including:

If you see other opportunities then we would love to hear your ideas!

Credits

This package was originally developed by Tom Nicholas whilst working at [C]Worthy, who deserve credit for allowing him to prioritise a generalizable open-source solution to the dataset virtualization problem. VirtualiZarr is now a community-owned multi-stakeholder project.

Licence

Apache 2.0

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

virtualizarr-1.1.0.tar.gz (88.3 kB view details)

Uploaded Source

Built Distribution

virtualizarr-1.1.0-py3-none-any.whl (86.3 kB view details)

Uploaded Python 3

File details

Details for the file virtualizarr-1.1.0.tar.gz.

File metadata

  • Download URL: virtualizarr-1.1.0.tar.gz
  • Upload date:
  • Size: 88.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.0.0 CPython/3.12.7

File hashes

Hashes for virtualizarr-1.1.0.tar.gz
Algorithm Hash digest
SHA256 aac88722118a273720c7cef2943dc824d0ef40b80159c6faa155ebc847b5a390
MD5 7d9aebfe86b90280ece05a5546453ce1
BLAKE2b-256 dfb1a96e48a7d94a884965c4a01407d63fab58fe83d5ccb64272b9cd121dd7d7

See more details on using hashes here.

Provenance

File details

Details for the file virtualizarr-1.1.0-py3-none-any.whl.

File metadata

  • Download URL: virtualizarr-1.1.0-py3-none-any.whl
  • Upload date:
  • Size: 86.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.0.0 CPython/3.12.7

File hashes

Hashes for virtualizarr-1.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 69ed497d767a8e5b2a0f736e681212cc704a747837da9875c94e1cedf50a5098
MD5 5223e036f418fbf6b86307dfdbbfd014
BLAKE2b-256 d12111dc3e179b0e490d068758172717107d21426b1a2bd65b0259c250c3b05d

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page