Skip to main content

Map flat data to structured JSON via a mapping.

Project description

# jsonmapping [![Build Status](https://travis-ci.org/pudo/jsonmapping.svg?branch=master)](https://travis-ci.org/pudo/jsonmapping)

To transform flat data structures into nested object graphs matching JSON
schema definitions, this package defines a mapping language. It defines how
the columns of a source data set (e.g. a CSV file, database table) are to be
converted to the fields of a JSON schema.

The format allows mapping nested structures, including arrays. It also supports
the application of very basic data transformation steps, such as generating a
URL slug or hashing a column value.

## Example mapping

The mapping format is independent of any particular JSON schema, such that
multiple mappings could be defined for any one particular schema.

```json
{
"schema": {"$ref": "http://www.popoloproject.com/schemas/person.json"},
"mapping": {
"id": {"column": "person_id"},
"name": {"column": "person_name"},
"memberships": [{
"mapping": {
"role": {"default": "Member of Organization"},
"organization": {
"mapping": {
"id": {
"columns": ["org_id"],
"constant": "default-org"
},
"name": {
"column": "org_name",
"constant": "Default Organization",
"transforms": ["strip"]
}
}
}
}
}]
}
}
```

This mapping would apply to a four-column CSV file and map it to a set of
nested JSON objects (a [Popolo](http://www.popoloproject.com/) person, with a
membership in an organization).

## Data Transforms

While ``jsonmapping`` is not a data cleaning tool, it supports some very basic
data transformation operations that can be applied on a particular column or
set of columns. These include:

* ``coalesce``: Select the first non-null value from the list of items.
* ``slugify``: Transform each string into a URL slug form.
* ``join``: Merge together the string values of all selected columns.
* ``upper``: Transform the text to upper case.
* ``lower``: Transform the text to lower case.
* ``strip``: Remove leading and trailing whitespace.
* ``hash``: Generate a SHA1 hash of the given value.

## Usage

``jsonmapping`` is available on the Python Package Index:

```bash
$ pip install jsonmapping
```

The library can then be used as follows:

```python
from jsonschema import RefResolver
from jsonmapping import Mapper

# ... load the mapping ...
mapping = load_mapping()
resolver = RefResolver.from_schema(mapping)

# ... grab some data ...
rows = read_csv()
objs = []

# This will transform flat data rows into nested JSON objects:
for obj, err in Mapper.apply_iter(rows, mapping, resolver):
if err is None:
objs.append(obj)
```

## Tests

The test suite will usually be executed in it's own ``virtualenv`` and perform a
coverage check as well as the tests. To execute on a system with ``virtualenv``
and ``make`` installed, type:

```bash
$ make test
```

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

jsonmapping-0.7.2.tar.gz (11.4 kB view details)

Uploaded Source

Built Distribution

jsonmapping-0.7.2-py2-none-any.whl (15.3 kB view details)

Uploaded Python 2

File details

Details for the file jsonmapping-0.7.2.tar.gz.

File metadata

  • Download URL: jsonmapping-0.7.2.tar.gz
  • Upload date:
  • Size: 11.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for jsonmapping-0.7.2.tar.gz
Algorithm Hash digest
SHA256 deb894ef9fc50fbf47b123e69567f3a7b0cafd142b2315a7f50e1f6a4e0593b0
MD5 cd08bb7e5493e2481741b09578a5b2c6
BLAKE2b-256 a090a3e4cdb0c4585d88a147f9a187e62c23a4bc963c030431b8b324e692cd84

See more details on using hashes here.

File details

Details for the file jsonmapping-0.7.2-py2-none-any.whl.

File metadata

File hashes

Hashes for jsonmapping-0.7.2-py2-none-any.whl
Algorithm Hash digest
SHA256 4cdee25a851ef5c10753a583366a23a867818102bf73a75db48e5f38bcbe953e
MD5 f1c9a53239b5ec8956cf3a5dde00ddac
BLAKE2b-256 dd36e1cd21978517d2d5d8ee96b6ada2cdf80157c5e59ac697de57e1bc84d843

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page