datapkg 0.5

Obtaining a Package

Search for a package in an Index e.g. on CKAN.net:

# let's search for iso country/language codes data (iso 3166 ...)
$ datapkg search ckan:// iso
...
iso-3166-2-data -- Linked ISO 3166-2 Data
...

Get some information about one of them:

$ datapkg info ckan://iso-3166-2-data

Let’s install it (to the current directory):

$ datapkg install ckan://iso-3166-2-data .

This will download the Package ‘iso-3166-2-data’ together with its “Resources” and unpack it into a directory named ‘iso-3166-2-data’.

Note that, if you replace the ckan:// spec with a file:// spec, you can use most of the commands for files on disk. For example, if you’ve downloaded a data package to, say, /tmp/xyz you could do:

$ datapkg info file:///tmp/xyz

See the help on indivdual commands for more information.

Creating and Registering a Package

Create a new data Package on disk using datapkg file layout:

$ datapkg create MyNewDataPackage

Edit the Package’s metadata:

$ vim MyNewDataPackage/setup.py

Add some data to the Package:

$ cp mydata.csv MyNewDataPackage
$ cp mydata.js MyNewDataPackage
$ etc ...

Register it on CKAN:

# NB: to register on CKAN you'll need to have an api-key
# This can either be stored in your config file (see datapkg init config)
# Or you can set it with the --api-key option
$ datapkg register MyNewDataPackage ckan://

Check it has registered ok:

$ datapkg info ckan://mynewdatapkg

[Not Yet Functional]: uploading package distributions.

Basic Concepts

Before we begin it is useful to understand some basic datapkg concepts:

1. A Package – the ‘package’ of data.

2. A Distribution – a serialization of the Package and optionally the data too (code, database, a book etc) to some concretely addressable form. For example: file(s) on disk, an API at a specific url.

For managing Packages datapkg uses:

1. A Registry: a list of Package metadata (but not Package payload data)

2. Repository: a Registry plus associated storage for association Distributions.

To specify a package (or just an index/repository) we often use a ‘package spec’ (often termed just ‘spec’), which have the following form:

# for CKAN
ckan://{package-name}
# on disk
file://{package-or-index-path}
# or even just
{package-or-index-path}

1. Obtaining Material

$ datapkg init config

2. Making Your Material Available to Others

$ datapkg create {pkg-name-or-path}

$ cd {my-new-package}
$ cp {lots-of-my-data-files} .

$ datapkg register {path} ckan://

$ datapkg install {package-spec} {file-spec}

3. For Developers

The easiest thing (which also guarantees up-to-date-ness) is to look through the unit tests in ./datapkg/tests/