Skip to main content

Consume and distribute datasets universally.

Project description

PyPI PyPI - Python Version PyPI - Implementation Gitter Runtime Tests Lint Docs Development Environment

Nourish is a Python API that enables data consumers and distributors to easily use and share datasets, and establishes a standard for exchanging data assets. It enables:

  • a data scientist to have a simpler and more unified way to begin working with a wide range of datasets, and

  • a data distributor to have a consistent, safe, and open source way to share datasets with interested communities.

Install the Package & its Dependencies

To install the latest version of Nourish, run

$ pip install -U git+https://github.com/edwardleardi/nourish

Alternatively, if you have downloaded the source, switch to the source directory (same directory as this README file, cd /path/to/nourish-source) and run

$ pip install -U .

Quick Start

Import the package and load a dataset. Nourish will download WikiText-103 dataset (version 1.0.1) if it’s not already downloaded, and then load it.

import nourish
wikitext103_data = nourish.load_dataset('wikitext103')

View available Nourish datasets and their versions.

>>> nourish.list_all_datasets()
{'claim_sentences_search': ('1.0.2',), ..., 'wikitext103': ('1.0.1',)}

To view your globally set configs for Nourish, such as your default data directory, use nourish.get_config.

>>> nourish.get_config()
Config(DATADIR=PosixPath('dir/to/dowload/load/from'), ..., DATASET_SCHEMA_URL='file/to/load/datasets/from')

By default, nourish.load_dataset downloads to and loads from ~/.nourish/data/<dataset-name>/<dataset-version>/. To change the default data directory, use nourish.init.

nourish.init(DATADIR='new/dir/to/dowload/load/from')

Load a previously downloaded dataset using nourish.load_dataset. With the new default data dir set, Nourish now searches for the Groningen Meaning Bank dataset (version 1.0.2) in new/dir/to/dowload/load/from/gmb/1.0.2/.

gmb_data = load_dataset('gmb', version='1.0.2', download=False)  # assuming GMB dataset was already downloaded

Notebooks

For a more extensive look at Nourish functionality, check out these notebooks:

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

nourish-0.1a1.tar.gz (134.8 kB view details)

Uploaded Source

Built Distribution

nourish-0.1a1-py3-none-any.whl (42.3 kB view details)

Uploaded Python 3

File details

Details for the file nourish-0.1a1.tar.gz.

File metadata

  • Download URL: nourish-0.1a1.tar.gz
  • Upload date:
  • Size: 134.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.3.0 pkginfo/1.7.0 requests/2.25.1 setuptools/51.1.2.post20210112 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.1

File hashes

Hashes for nourish-0.1a1.tar.gz
Algorithm Hash digest
SHA256 ef439d2316ee0f0d763934b9210b9078bb8fbb796ad2ff489bcea8803c4410c7
MD5 85ac769ef3e2e21108b9bdd17e19f8fe
BLAKE2b-256 5fffa4a71e532aa04941b62afffe4c31c848581a6abfdfaa39e5ad004d03da92

See more details on using hashes here.

File details

Details for the file nourish-0.1a1-py3-none-any.whl.

File metadata

  • Download URL: nourish-0.1a1-py3-none-any.whl
  • Upload date:
  • Size: 42.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.3.0 pkginfo/1.7.0 requests/2.25.1 setuptools/51.1.2.post20210112 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.1

File hashes

Hashes for nourish-0.1a1-py3-none-any.whl
Algorithm Hash digest
SHA256 1f304b78a5281b2d0c426ba5ab32e576c69fb774a6c139a0b9d68ebb2d67797d
MD5 c1fdaf9312e86ad3b1b520f996dfe10c
BLAKE2b-256 555213143a15fcd220c8b6a926260eb0c141abbce0da1dd9ab3a550e0fe04114

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page