Skip to main content

Access DAX datasets.

Project description

PyPI PyPI - Python Version PyPI - Implementation Gitter Runtime Tests Lint Docs Development Environment

PyDAX is a Python API that enables data consumers and distributors to easily use and share datasets, and establishes a standard for exchanging data assets. It enables:

  • a data scientist to have a simpler and more unified way to begin working with a wide range of datasets, and

  • a data distributor to have a consistent, safe, and open source way to share datasets with interested communities.

Install the Package & its Dependencies

To install the latest version of PyDAX, run

$ pip install pydax

Alternatively, if you have downloaded the source, switch to the source directory (same directory as this README file, cd /path/to/pydax-source) and run

$ pip install -U .

Quick Start

Import the package and load a dataset. PyDAX will download WikiText-103 dataset (version 1.0.1) if it’s not already downloaded, and then load it.

import pydax
wikitext103_data = pydax.load_dataset('wikitext103')

View available PyDAX datasets and their versions.

>>> pydax.list_all_datasets()
{'claim_sentences_search': ('1.0.2',), ..., 'wikitext103': ('1.0.1',)}

To view your globally set configs for PyDAX, such as your default data directory, use pydax.get_config.

>>> pydax.get_config()
Config(DATADIR=PosixPath('dir/to/dowload/load/from'), ..., DATASET_SCHEMA_FILE_URL='file/to/load/datasets/from')

By default, pydax.load_dataset downloads to and loads from ~/.pydax/data/<dataset-name>/<dataset-version>/. To change the default data directory, use pydax.init.

pydax.init(DATADIR='new/dir/to/dowload/load/from')

Load a previously downloaded dataset using pydax.load_dataset. With the new default data dir set, PyDAX now searches for the Groningen Meaning Bank dataset (version 1.0.2) in new/dir/to/dowload/load/from/gmb/1.0.2/.

gmb_data = load_dataset('gmb', version='1.0.2', download=False)  # assuming GMB dataset was already downloaded

To learn more about PyDAX, check out the documentation and the tutorial.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pydax-0.1.0.tar.gz (9.1 MB view details)

Uploaded Source

Built Distribution

pydax-0.1.0-py3-none-any.whl (43.5 kB view details)

Uploaded Python 3

File details

Details for the file pydax-0.1.0.tar.gz.

File metadata

  • Download URL: pydax-0.1.0.tar.gz
  • Upload date:
  • Size: 9.1 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.24.0 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.8.6

File hashes

Hashes for pydax-0.1.0.tar.gz
Algorithm Hash digest
SHA256 c6ebdee6b4dc9611a18d743ab5764d72d23f0b3286db39d066f78d999d13caa6
MD5 d1a0fcf0541a494b351bd66440c19575
BLAKE2b-256 13d4c3ddcd451644d4879605a6d90a007157a9f0a653cda3bb871cc264d3a2bb

See more details on using hashes here.

File details

Details for the file pydax-0.1.0-py3-none-any.whl.

File metadata

  • Download URL: pydax-0.1.0-py3-none-any.whl
  • Upload date:
  • Size: 43.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.24.0 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.8.6

File hashes

Hashes for pydax-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 f26a11e3c2252183e52dbfeaea568352834af6735026315a6a5c9e6f91383652
MD5 a99b6e6d2c46864864d69671c57e7509
BLAKE2b-256 4bb8b990d65ac186c473b057496d495511e545345715083f08716d323641ba49

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page