An intake-esm inspired catalog for ESGF
Project description
intake-esgf
Badges
Overview
intake-esgf
is an intake-esm inspired package under development in ESGF2. The main difference is that in place of querying a static index which is completely loaded at runtime, intake-esgf
catalogs initialize empty and are populated by searching, querying ESGF index nodes.
Installation
You may install intake-esgf
using pip:
python -m pip install intake-esgf
or conda-forge:
conda install -c conda-forge intake-esgf
Features
For a full listing of features with code examples, please consult the documentation. In brief, intake-esgf
aims to hide some of the complexity of obtaining ESGF data and get the user the data as fast as we can.
- Indices are queried in parallel and report when they fail to return a response. The results are aggregated and presented to the user as a pandas DataFrame.
- The locations of the data are hidden from the user. Internally we track which locations provide the user the fastest transfers and automatically favor them for you.
- Files are downloaded in parallel into a local cache which mirrors the remote storage directory structure. They are returned to the user as a dictionary of xarray Datasets. Your search script then becomes the way you download data as well as how you load it into memory for your analysis.
- Prior to downloading data, we first check that it is not already available locally. This could be because you had previously downloaded it, but also because you are working on a server that has direct access.
- Cell measure information is harvested from your search results and automatically included in the returned datasets.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
intake-esgf-2024.3.5.tar.gz
(425.6 kB
view hashes)
Built Distribution
Close
Hashes for intake_esgf-2024.3.5-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 12d8e7e0f1ef716d5d2adbf26f4e90fbd05d294f20129daca9409ed11670211b |
|
MD5 | 270918d229751aab9938add095acb427 |
|
BLAKE2b-256 | 2858b526ba921f604ce140794bc11566aeab705328524810824217f0b32532e2 |