Access DAX datasets.
Project description
PyDAX is a Python API that enables data consumers and distributors to easily use and share datasets, and establishes a standard for exchanging data assets. It enables:
a data scientist to have a simpler and more unified way to begin working with a wide range of datasets, and
a data distributor to have a consistent, safe, and open source way to share datasets with interested communities.
Install the Package & its Dependencies
To install the latest version of PyDAX, run
$ pip install pydax
Alternatively, if you have downloaded the source, switch to the source directory (same directory as this README file, cd /path/to/pydax-source) and run
$ pip install -U .
Quick Start
Import the package and load a dataset. PyDAX will download WikiText-103 dataset (version 1.0.1) if it’s not already downloaded, and then load it.
import pydax
wikitext103_data = pydax.load_dataset('wikitext103')
View available PyDAX datasets and their versions.
>>> pydax.list_all_datasets()
{'claim_sentences_search': ('1.0.2',), ..., 'wikitext103': ('1.0.1',)}
To view your globally set configs for PyDAX, such as your default data directory, use pydax.get_config.
>>> pydax.get_config()
Config(DATADIR=PosixPath('dir/to/dowload/load/from'), ..., DATASET_SCHEMA_FILE_URL='file/to/load/datasets/from')
By default, pydax.load_dataset downloads to and loads from ~/.pydax/data/<dataset-name>/<dataset-version>/. To change the default data directory, use pydax.init.
pydax.init(DATADIR='new/dir/to/dowload/load/from')
Load a previously downloaded dataset using pydax.load_dataset. With the new default data dir set, PyDAX now searches for the Groningen Meaning Bank dataset (version 1.0.2) in new/dir/to/dowload/load/from/gmb/1.0.2/.
gmb_data = load_dataset('gmb', version='1.0.2', download=False) # assuming GMB dataset was already downloaded
To learn more about PyDAX, check out the documentation and the tutorial.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file pydax-0.1.0.tar.gz
.
File metadata
- Download URL: pydax-0.1.0.tar.gz
- Upload date:
- Size: 9.1 MB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.24.0 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.8.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | c6ebdee6b4dc9611a18d743ab5764d72d23f0b3286db39d066f78d999d13caa6 |
|
MD5 | d1a0fcf0541a494b351bd66440c19575 |
|
BLAKE2b-256 | 13d4c3ddcd451644d4879605a6d90a007157a9f0a653cda3bb871cc264d3a2bb |
File details
Details for the file pydax-0.1.0-py3-none-any.whl
.
File metadata
- Download URL: pydax-0.1.0-py3-none-any.whl
- Upload date:
- Size: 43.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.24.0 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.8.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | f26a11e3c2252183e52dbfeaea568352834af6735026315a6a5c9e6f91383652 |
|
MD5 | a99b6e6d2c46864864d69671c57e7509 |
|
BLAKE2b-256 | 4bb8b990d65ac186c473b057496d495511e545345715083f08716d323641ba49 |