Provides utility functions for accessing data repository for Project Pythia examples/notebooks
Project description
CI | |
---|---|
Docs | |
Package | |
License |
pythia-datasets
Data repository for Project Pythia examples/notebooks
Sample data sets
These files are used as sample data in Pythia Project examples/notebooks and are downloaded by pythia_datasets
package:
NARR_19930313_0000.nc
enso_data.csv
jan-17-co-asos.txt.xz
CESM2_sst_data.nc
CESM2_grid_variables.nc
Adding new datasets
To add a new dataset file, please follow these steps:
- Add the dataset file to the
data/
directory - From the command line, run
python make_registry.py
script to update the registry file residing inpythia_datasets/registry.txt
- Commit and push your changes to GitHub
Using datasets in notebooks and/or scripts
-
Ensure the
pythia_datasets
package is installed in your environmentpython -m pip install pythia-datasets # or python -m pip install git+https://github.com/ProjectPythia/pythia-datasets
-
Import
DATASETS
and inspect the registry to find out which datasets are availableIn [1]: from pythia_datasets import DATASETS In [2]: DATASETS.registry_files Out[2]: ['jan-17-co-asos.txt.xz', 'NARR_19930313_0000.nc']
-
To fetch a data file of interest, use the
.fetch
method and provide the filename of the data file. This will- download and cache the file if it doesn't exist already.
- retrieve and return the local path
In [4]: filepath = DATASETS.fetch('jan-17-co-asos.txt.xz') In [5]: filepath Out[5]: '/Users/abanihi/Library/Caches/pythia-datasets/jan-17-co-asos.txt.xz'
-
Once you have access to the local filepath, you can then use it to load your dataset into pandas or xarray or your package of choice:
In [6]: df = pd.read_csv(filepath)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file pythia-datasets-2021.9.21.tar.gz
.
File metadata
- Download URL: pythia-datasets-2021.9.21.tar.gz
- Upload date:
- Size: 12.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.7
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 19fcb86b0db71aa3251dcab528b51bf66639e2dc3c8654d6fc80b8a3379ac650 |
|
MD5 | 28aa736e264964a7699031ce22b102d9 |
|
BLAKE2b-256 | 0b45fd687d08636413a2b612b56ce67e83fe1724158a962199d9fc6e45f04d5b |
File details
Details for the file pythia_datasets-2021.9.21-py3-none-any.whl
.
File metadata
- Download URL: pythia_datasets-2021.9.21-py3-none-any.whl
- Upload date:
- Size: 8.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.7
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 179dbe46538ad2310fd11075472807c425ffa956db6b90957d52211e22441f5b |
|
MD5 | 2c6f7fc3afb67dc36253c5b152d26516 |
|
BLAKE2b-256 | e540fc8ffc2c2fb3bbb0dec75bec4a5fb5336f035ceafac5397400f9c2b12dab |