Skip to main content

Data Package integreation for CKAN

Project description

ckanext-datapackage

Build Codebase Release

Data Package integreation for CKAN.

Purpose

This lightweight CKAN plugin adds a dataset/<id>/datapackage.json endpoint to every dataset in the data catalog. Read more about the Data Package Standard.

Requirements

The extension requires Python 3.8+. It is being developed and tested with CKAN 2.10. Please fill an issue if it doesn't work as expected with other versions.

Installation

  1. Install the extension:
$ pip install ckanext-datapackage
  1. Enable the plugin in your ini file:
ckan.plugins = ... datapackage

Endpoint

Data Package representations of a particular dataset can be accessed using the following endpoint:

https://{ckan-instance-host}/dataset/{dataset-id}/datapackage.json

Read more about Data Package Standard specifications and datapackage.json metadata format.

Example

Consider there is a dataset as below:

http://ckan:5000/dataset/example

Dataset

The Data Package representation of this dataset can be accessed using the following endpoint (note that the link is also available on the dataset page in the "Additional Info" section):

http://ckan:5000/dataset/example/datapackage.json

{
  "$schema": "https://datapackage.org/profiles/2.0/datapackage.json",
  "ckan:id": "8bb0257b-37be-48ca-9129-59040264acd5",
  "contributors": [
    {
      "email": "eskarev@gmail.com",
      "roles": ["author"],
      "title": "Evgeny Karev"
    },
    {
      "email": "eskarev@gmail.com",
      "roles": ["maintainer"],
      "title": "Evgeny Karev"
    }
  ],
  "created": "2024-04-22T14:08:29.297417+01:00",
  "description": "It is an examplar dataset",
  "keywords": ["data", "open"],
  "licenses": [
    {
      "name": "cc-by",
      "path": "http://www.opendefinition.org/licenses/cc-by",
      "title": "Creative Commons Attribution"
    }
  ],
  "name": "example",
  "resources": [
    {
      "bytes": 1497,
      "ckan:id": "03eec88c-a4b0-4ba9-8940-58dd4d383936",
      "description": "It's a table about cars",
      "format": "csv",
      "mediatype": "text/csv",
      "name": "table_cars",
      "path": "http://ckan:5000/dataset/8bb0257b-37be-48ca-9129-59040264acd5/resource/03eec88c-a4b0-4ba9-8940-58dd4d383936/download/table-cars.csv",
      "schema": {
        "fields": [
          {
            "name": "brand",
            "type": "string"
          },
          {
            "name": "model",
            "type": "string"
          },
          {
            "name": "price",
            "type": "number"
          },
          {
            "name": "kmpl",
            "type": "number"
          },
          {
            "name": "bhp",
            "type": "number"
          },
          {
            "name": "type",
            "type": "string"
          }
        ]
      },
      "type": "table"
    },
    {
      "bytes": 13,
      "ckan:id": "f3425eea-0ec9-481e-9679-382e7c499474",
      "description": "It's a textual file",
      "format": "txt",
      "mediatype": "text/plain",
      "name": "text",
      "path": "http://ckan:5000/dataset/8bb0257b-37be-48ca-9129-59040264acd5/resource/f3425eea-0ec9-481e-9679-382e7c499474/download/text.txt"
    }
  ],
  "title": "Example",
  "version": "1.0"
}

Benefits

Having a Data Package representation of a dataset allows for using a rich ecosystem of tools and libraries that support the Data Package Standard. For example, data portal users can use the frictionless-r or frictionless-py to access the data without loosing metadata and information about data types:

frictionless extract http://ckan:5000/dataset/example/datapackage.json
────────────────────────────────────────────────────────────── Dataset ──────────────────────────────────────────────────────────────
┏━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ name       ┃ type  ┃ path                                                                                                         ┃
┡━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ table_cars │ table │ http://ckan:5000/dataset/8bb0257b-37be-48ca-9129-59040264acd5/resource/03eec88c-a4b0-4ba9-8940-58dd4d383936… │
│ text       │ text  │ http://ckan:5000/dataset/8bb0257b-37be-48ca-9129-59040264acd5/resource/f3425eea-0ec9-481e-9679-382e7c499474… │
└────────────┴───────┴──────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────── Tables ───────────────────────────────────────────────────────────────
                        table_cars
┏━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━┳━━━━━┳━━━━━━━┓
┃ brand      ┃ model         ┃ price ┃ kmpl ┃ bhp ┃ type  ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━╇━━━━━╇━━━━━━━┩
│ Volkswagen │ Vento         │ 785   │ 16.1 │ 104 │ Sedan │
│ Hyundai    │ Verna         │ 774   │ 17.4 │ 106 │ Sedan │
│ Skoda      │ Rapid         │ 756   │ 15   │ 104 │ Sedan │
│ Suzuki     │ Ciaz          │ 725   │ 20.7 │ 91  │ Sedan │
│ Renault    │ Scala         │ 724   │ 16.9 │ 98  │ Sedan │
│ Suzuki     │ SX4           │ 715   │ 16.5 │ 103 │ Sedan │
│ Fiat       │ Linea         │ 700   │ 15.7 │ 112 │ Sedan │
│ Nissan     │ Sunny         │ 699   │ 16.9 │ 98  │ Sedan │
│ Fiat       │ Linea Classic │ 612   │ 14.9 │ 89  │ Sedan │
│ Toyota     │ Etios         │ 603   │ 16.8 │ 89  │ Sedan │
│ ...        │ ...           │ ...   │ ...  │ ... │ ...   │
└────────────┴───────────────┴───────┴──────┴─────┴───────┘

Note that remote Data Package support is also under development in Open Data Editor.

Development

Please follow the Contribution Guide.

Changelog

Please refer to the Changelog.

Funding

This project is funded through NGI0 Entrust, a fund established by NLnet with financial support from the European Commission's Next Generation Internet program. Learn more at the NLnet project page.

NLnet foundation logo NGI Zero Logo

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ckanext_datapackage-0.2.4.tar.gz (166.1 kB view details)

Uploaded Source

Built Distribution

ckanext_datapackage-0.2.4-py3-none-any.whl (8.5 kB view details)

Uploaded Python 3

File details

Details for the file ckanext_datapackage-0.2.4.tar.gz.

File metadata

  • Download URL: ckanext_datapackage-0.2.4.tar.gz
  • Upload date:
  • Size: 166.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.0.0 CPython/3.12.3

File hashes

Hashes for ckanext_datapackage-0.2.4.tar.gz
Algorithm Hash digest
SHA256 562422ac81cc5eeb229e1984ee882a60f4ec11dd374f3623fec2c7b3a2ea5a78
MD5 de623b86f4dc78410e0881ef093032d1
BLAKE2b-256 2b2def0bc1ae4c72526d88b27583e9e603d495d010b924d22264e182ca9a2c46

See more details on using hashes here.

File details

Details for the file ckanext_datapackage-0.2.4-py3-none-any.whl.

File metadata

File hashes

Hashes for ckanext_datapackage-0.2.4-py3-none-any.whl
Algorithm Hash digest
SHA256 c16003542ef42819685455cf51ad1b6ccb3f4dfdddabaad418b080dc95467390
MD5 f086509aba73ee70da46de6464405a8d
BLAKE2b-256 ea3e92283c8a71907631aee2e7f32c93d4e0f62dd954fbe138e5dbc133c95f92

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page