An intake-esm inspired catalog for ESGF
Project description
intake-esgf
Badges
Overview
intake-esgf
is an intake-esm inspired package under development in ESGF2. The main difference is that in place of querying a static index which is completely loaded at runtime, intake-esgf
catalogs initialize empty and are populated by searching, querying ESGF index nodes.
Installation
You may install intake-esgf
using pip:
python -m pip install intake-esgf
or conda-forge:
conda install -c conda-forge intake-esgf
Features
For a full listing of features with code examples, please consult the documentation. In brief, intake-esgf
aims to hide some of the complexity of obtaining ESGF data and get the user the data as fast as we can.
- Indices are queried in parallel and report when they fail to return a response. The results are aggregated and presented to the user as a pandas DataFrame.
- The locations of the data are hidden from the user. Internally we track which locations provide the user the fastest transfers and automatically favor them for you.
- Files are downloaded in parallel into a local cache which mirrors the remote storage directory structure. They are returned to the user as a dictionary of xarray Datasets. Your search script then becomes the way you download data as well as how you load it into memory for your analysis.
- Prior to downloading data, we first check that it is not already available locally. This could be because you had previously downloaded it, but also because you are working on a server that has direct access.
- Cell measure information is harvested from your search results and automatically included in the returned datasets.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file intake-esgf-2024.3.5.tar.gz
.
File metadata
- Download URL: intake-esgf-2024.3.5.tar.gz
- Upload date:
- Size: 425.6 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.11.8
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 29822861afbb4ba761d1dcf737b509ef91a8dc29ffff5833fd54b5a89269e91f |
|
MD5 | 8dcbe2f9472d709716374b783b15a197 |
|
BLAKE2b-256 | 18b211b3ddd46fbb1519fb28b1fc3690293983721d6a2cdda4ee3d591b86c53b |
Provenance
File details
Details for the file intake_esgf-2024.3.5-py3-none-any.whl
.
File metadata
- Download URL: intake_esgf-2024.3.5-py3-none-any.whl
- Upload date:
- Size: 28.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.11.8
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 12d8e7e0f1ef716d5d2adbf26f4e90fbd05d294f20129daca9409ed11670211b |
|
MD5 | 270918d229751aab9938add095acb427 |
|
BLAKE2b-256 | 2858b526ba921f604ce140794bc11566aeab705328524810824217f0b32532e2 |