Skip to main content

An intake-esm inspired catalog for ESGF

Project description

intake-esgf

Badges

Continuous Integration Documentation Status Code Coverage Status PyPI Conda

Overview

intake-esgf is an intake-esm inspired package under development in ESGF2. The main difference is that in place of querying a static index which is completely loaded at runtime, intake-esgf catalogs initialize empty and are populated by searching, querying ESGF index nodes.

Installation

You may install intake-esgf using pip:

python -m pip install intake-esgf

or conda-forge:

conda install -c conda-forge intake-esgf

Features

For a full listing of features with code examples, please consult the documentation. In brief, intake-esgf aims to hide some of the complexity of obtaining ESGF data and get the user the data as fast as we can.

  • Indices are queried in parallel and report when they fail to return a response. The results are aggregated and presented to the user as a pandas DataFrame.
  • The locations of the data are hidden from the user. Internally we track which locations provide the user the fastest transfers and automatically favor them for you.
  • Files are downloaded in parallel into a local cache which mirrors the remote storage directory structure. They are returned to the user as a dictionary of xarray Datasets. Your search script then becomes the way you download data as well as how you load it into memory for your analysis.
  • Prior to downloading data, we first check that it is not already available locally. This could be because you had previously downloaded it, but also because you are working on a server that has direct access.
  • Cell measure information is harvested from your search results and automatically included in the returned datasets.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

intake-esgf-2024.3.5.tar.gz (425.6 kB view details)

Uploaded Source

Built Distribution

intake_esgf-2024.3.5-py3-none-any.whl (28.5 kB view details)

Uploaded Python 3

File details

Details for the file intake-esgf-2024.3.5.tar.gz.

File metadata

  • Download URL: intake-esgf-2024.3.5.tar.gz
  • Upload date:
  • Size: 425.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.11.8

File hashes

Hashes for intake-esgf-2024.3.5.tar.gz
Algorithm Hash digest
SHA256 29822861afbb4ba761d1dcf737b509ef91a8dc29ffff5833fd54b5a89269e91f
MD5 8dcbe2f9472d709716374b783b15a197
BLAKE2b-256 18b211b3ddd46fbb1519fb28b1fc3690293983721d6a2cdda4ee3d591b86c53b

See more details on using hashes here.

Provenance

File details

Details for the file intake_esgf-2024.3.5-py3-none-any.whl.

File metadata

File hashes

Hashes for intake_esgf-2024.3.5-py3-none-any.whl
Algorithm Hash digest
SHA256 12d8e7e0f1ef716d5d2adbf26f4e90fbd05d294f20129daca9409ed11670211b
MD5 270918d229751aab9938add095acb427
BLAKE2b-256 2858b526ba921f604ce140794bc11566aeab705328524810824217f0b32532e2

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page