Skip to main content

Map flat data to structured JSON via a mapping.

Project description

# jsonmapping [![Build Status](https://travis-ci.org/pudo/jsonmapping.svg?branch=master)](https://travis-ci.org/pudo/jsonmapping)

To transform flat data structures into nested object graphs matching JSON
schema definitions, this package defines a mapping language. It defines how
the columns of a source data set (e.g. a CSV file, database table) are to be
converted to the fields of a JSON schema.

The format allows mapping nested structures, including arrays. It also supports
the application of very basic data transformation steps, such as generating a
URL slug or hashing a column value.

## Example mapping

The mapping format is independent of any particular JSON schema, such that
multiple mappings could be defined for any one particular schema.

```json
{
"schema": {"$ref": "http://www.popoloproject.com/schemas/person.json"},
"mapping": {
"id": {"column": "person_id"},
"name": {"column": "person_name"},
"memberships": [{
"mapping": {
"role": {"default": "Member of Organization"},
"organization": {
"mapping": {
"id": {
"columns": ["org_id"],
"constant": "default-org"
},
"name": {
"column": "org_name",
"constant": "Default Organization",
"transforms": ["strip"]
}
}
}
}
}]
}
}
```

This mapping would apply to a four-column CSV file and map it to a set of
nested JSON objects (a [Popolo](http://www.popoloproject.com/) person, with a
membership in an organization).

## Data Transforms

While ``jsonmapping`` is not a data cleaning tool, it supports some very basic
data transformation operations that can be applied on a particular column or
set of columns. These include:

* ``coalesce``: Select the first non-null value from the list of items.
* ``slugify``: Transform each string into a URL slug form.
* ``join``: Merge together the string values of all selected columns.
* ``upper``: Transform the text to upper case.
* ``lower``: Transform the text to lower case.
* ``strip``: Remove leading and trailing whitespace.
* ``hash``: Generate a SHA1 hash of the given value.

## Usage

``jsonmapping`` is available on the Python Package Index:

```bash
$ pip install jsonmapping
```

The library can then be used as follows:

```python
from jsonschema import RefResolver
from jsonmapping import Mapper

# ... load the mapping ...
mapping = load_mapping()
resolver = RefResolver.from_schema(mapping)

# ... grab some data ...
rows = read_csv()
objs = []

# This will transform flat data rows into nested JSON objects:
for obj, err in Mapper.apply_iter(rows, mapping, resolver):
if err is None:
objs.append(obj)
```

## Tests

The test suite will usually be executed in it's own ``virtualenv`` and perform a
coverage check as well as the tests. To execute on a system with ``virtualenv``
and ``make`` installed, type:

```bash
$ make test
```

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

jsonmapping-0.7.4.tar.gz (11.4 kB view details)

Uploaded Source

Built Distribution

jsonmapping-0.7.4-py2-none-any.whl (15.4 kB view details)

Uploaded Python 2

File details

Details for the file jsonmapping-0.7.4.tar.gz.

File metadata

  • Download URL: jsonmapping-0.7.4.tar.gz
  • Upload date:
  • Size: 11.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for jsonmapping-0.7.4.tar.gz
Algorithm Hash digest
SHA256 334e52919dc66421344d14cca681eaecf0cb111accebd7bef51bde5940661457
MD5 19c65d57185ce357ee3361bf1b4c80b6
BLAKE2b-256 7c35cb1f0242b5968e80bce162c4031f9f133cc0772ce3e705e5367f1ce06ea6

See more details on using hashes here.

File details

Details for the file jsonmapping-0.7.4-py2-none-any.whl.

File metadata

File hashes

Hashes for jsonmapping-0.7.4-py2-none-any.whl
Algorithm Hash digest
SHA256 03f1e9c3201cd0c4f39b98eb305c98f9de1ceb785fb68d52a742f4786294eb77
MD5 0384fa5e728517188b287b5f5675ca2e
BLAKE2b-256 73920e59c43fe4f95b5105021ed09992d7a73198dc6c60f001f87a00ab513834

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page