datapkg 0.4

Obtaining a Package

Search for a package in an Index e.g. on CKAN.net:

$ datapkg search ckan:// windhover
...
datapkgdemo -- ...
...

Get some information about one of them (in this case our demonstration package on ckan.net):

$ datapkg info ckan://datapkgdemo

Let’s install one of them (to the current directory):

$ datapkg install ckan://datapkgdemo .

This will download the Distribution file for Package ‘datapkgdemo’ and unpack it in the current directory ‘.’.

Creating and Registering a Package

Create a new data Package on disk using datapkg file layout:

$ datapkg create MyNewDataPackage

Edit the Package’s metadata:

$ vim MyNewDataPackage/setup.py

Add some data to the Package:

$ cp mydata.csv MyNewDataPackage
$ cp mydata.js MyNewDataPackage
$ etc ...

Register it on CKAN:

# NB: to register on CKAN you'll need to have an api-key
# This can either be stored in your config file (see datapkg init config)
# Or you can set it with the --api-key option
$ datapkg register MyNewDataPackage ckan://

Check it has registered ok:

$ datapkg info ckan://mynewdatapkg

[Not Yet Functional]: uploading package distributions.

Basic Concepts

Before we begin it is useful to understand some basic datapkg concepts:

1. A Package – the ‘package’ of data.

2. A Distribution – a serialization of the Package and optionally the data too (code, database, a book etc) to some concretely addressable form. For example: file(s) on disk, an API at a specific url.

For managing Packages datapkg uses:

1. A Registry: a list of Package metadata (but not Package payload data)

2. Repository: a Registry plus associated storage for association Distributions.

To specify a package (or just an index/repository) we often use a ‘package spec’ (often termed just ‘spec’), which have the following form:

# for CKAN
ckan://{package-name}
# on disk
file://{package-or-index-path}
# or even just
{package-or-index-path}

1. Obtaining Material

$ datapkg init config

2. Making Your Material Available to Others

$ datapkg create {pkg-name-or-path}

$ cd {my-new-package}
$ cp {lots-of-my-data-files} .

$ datapkg register {path} ckan://

$ datapkg install {package-spec} {file-spec}

3. For Developers

The easiest thing (which also guarantees up-to-date-ness) is to look through the unit tests in ./datapkg/tests/