An intake-esm inspired catalog for ESGF
Project description
intake-esgf
Badges
Overview
intake-esgf
is an intake-esm inspired package under development in ESGF2. The main difference is that in place of querying a static index which is completely loaded at runtime, intake-esgf
catalogs initialize empty and are populated by searching, querying ESGF index nodes.
Installation
You may install intake-esgf
using pip:
python -m pip install intake-esgf
Features
For a full listing of features with code examples, please consult the documentation. In brief, intake-esgf
aims to hide some of the complexity of obtaining ESGF data and get the user the data as fast as we can.
- Indices are queried in parallel and report when they fail to return a response. The results are aggregated and presented to the user as a pandas DataFrame.
- The locations of the data are hidden from the user. Internally we track which locations provide the user the fastest transfers and automatically favor them for you.
- Files are downloaded in parallel into a local cache which mirrors the remote storage directory structure. They are returned to the user as a dictionary of xarray Datasets. Your search script then becomes the way you download data as well as how you load it into memory for your analysis.
- Prior to downloading data, we first check that it is not already available locally. This could be because you had previously downloaded it, but also because you are working on a server that has direct access.
- Cell measure information is harvested from your search results and automatically included in the returned datasets.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
intake-esgf-2024.1.26.tar.gz
(420.1 kB
view hashes)
Built Distribution
Close
Hashes for intake_esgf-2024.1.26-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6842295d0cd883a36fb1fc629b6d35744c2724b2ab5d13d44f04a5e68cc9e2c6 |
|
MD5 | 4b20297a750f4ba84a81cdca20079742 |
|
BLAKE2b-256 | be6e570c864a8da8282677f1beb10d7f06e5c41625d7c9a24315ac279d12376b |