Skip to main content

Schema Annotations for Linked Avro Data (SALAD)

Project description

Linux Build Status Code coverage CII Best Practices

Schema Salad

Salad is a schema language for describing JSON or YAML structured linked data documents. Salad schema describes rules for preprocessing, structural validation, and hyperlink checking for documents described by a Salad schema. Salad supports rich data modeling with inheritance, template specialization, object identifiers, object references, documentation generation, code generation, and transformation to RDF. Salad provides a bridge between document and record oriented data modeling and the Semantic Web.

The Schema Salad library is Python 3.6+ only.

Usage

$ pip install schema_salad

To install from source:

git clone https://github.com/common-workflow-language/schema_salad
cd schema_salad
python3 setup.py install

Commands

Schema salad can be used as a command line tool or imported as a Python module:

$ schema-salad-tool
usage: schema-salad-tool [-h] [--rdf-serializer RDF_SERIALIZER]
                      [--print-jsonld-context | --print-rdfs | --print-avro
                      | --print-rdf | --print-pre | --print-index
                      | --print-metadata | --print-inheritance-dot
                      | --print-fieldrefs-dot | --codegen language
                      | --print-oneline]
                      [--strict | --non-strict] [--verbose | --quiet
                      | --debug]
                      [--version]
                      [schema] [document]

$ python
>>> import schema_salad

Validate a schema:

$ schema-salad-tool myschema.yml

Validate a document using a schema:

$ schema-salad-tool myschema.yml mydocument.yml

Generate HTML documentation:

$ schema-salad-tool myschema.yml > myschema.html

Get JSON-LD context:

$ schema-salad-tool --print-jsonld-context myschema.yml mydocument.yml

Convert a document to JSON-LD:

$ schema-salad-tool --print-pre myschema.yml mydocument.yml > mydocument.jsonld

Generate Python classes for loading/generating documents described by the schema:

$ schema-salad-tool --codegen=python myschema.yml > myschema.py

Display inheritance relationship between classes as a graphviz ‘dot’ file and render as SVG:

$ schema-salad-tool --print-inheritance-dot myschema.yml | dot -Tsvg > myschema.svg

Quick Start

Let’s say you have a ‘basket’ record that can contain items measured either by weight or by count. Here’s an example:

basket:
  - product: bananas
    price: 0.39
    per: pound
    weight: 1
  - product: cucumbers
    price: 0.79
    per: item
    count: 3

We want to validate that all the expected fields are present, the measurement is known, and that “count” cannot be a fractional value. Here is an example schema to do that:

- name: Product
  doc: |
    The base type for a product.  This is an abstract type, so it
    can't be used directly, but can be used to define other types.
  type: record
  abstract: true
  fields:
    product: string
    price: float

- name: ByWeight
  doc: |
    A product, sold by weight.  Products may be sold by pound or by
    kilogram.  Weights may be fractional.
  type: record
  extends: Product
  fields:
    per:
      type:
        type: enum
        symbols:
          - pound
          - kilogram
      jsonldPredicate: '#per'
    weight: float

- name: ByCount
  doc: |
    A product, sold by count.  The count must be a integer value.
  type: record
  extends: Product
  fields:
    per:
      type:
        type: enum
        symbols:
          - item
      jsonldPredicate: '#per'
    count: int

- name: Basket
  doc: |
    A basket of products.  The 'documentRoot' field indicates it is a
    valid starting point for a document.  The 'basket' field will
    validate subtypes of 'Product' (ByWeight and ByCount).
  type: record
  documentRoot: true
  fields:
    basket:
      type:
        type: array
        items: Product

You can check the schema and document in schema_salad/tests/basket_schema.yml and schema_salad/tests/basket.yml:

$ schema-salad-tool basket_schema.yml basket.yml
Document `basket.yml` is valid

Documentation

See the specification and the metaschema (salad schema for itself). For an example application of Schema Salad see the Common Workflow Language.

Rationale

The JSON data model is an popular way to represent structured data. It is attractive because of it’s relative simplicity and is a natural fit with the standard types of many programming languages. However, this simplicity comes at the cost that basic JSON lacks expressive features useful for working with complex data structures and document formats, such as schemas, object references, and namespaces.

JSON-LD is a W3C standard providing a way to describe how to interpret a JSON document as Linked Data by means of a “context”. JSON-LD provides a powerful solution for representing object references and namespaces in JSON based on standard web URIs, but is not itself a schema language. Without a schema providing a well defined structure, it is difficult to process an arbitrary JSON-LD document as idiomatic JSON because there are many ways to express the same data that are logically equivalent but structurally distinct.

Several schema languages exist for describing and validating JSON data, such as JSON Schema and Apache Avro data serialization system, however none understand linked data. As a result, to fully take advantage of JSON-LD to build the next generation of linked data applications, one must maintain separate JSON schema, JSON-LD context, RDF schema, and human documentation, despite significant overlap of content and obvious need for these documents to stay synchronized.

Schema Salad is designed to address this gap. It provides a schema language and processing rules for describing structured JSON content permitting URI resolution and strict document validation. The schema language supports linked data through annotations that describe the linked data interpretation of the content, enables generation of JSON-LD context and RDF schema, and production of RDF triples by applying the JSON-LD context. The schema language also provides for robust support of inline documentation.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

schema-salad-8.2.20210902094147.tar.gz (435.1 kB view details)

Uploaded Source

Built Distributions

schema_salad-8.2.20210902094147-py3-none-any.whl (474.9 kB view details)

Uploaded Python 3

schema_salad-8.2.20210902094147-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.6 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ARM64

schema_salad-8.2.20210902094147-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64 manylinux: glibc 2.5+ x86-64

schema_salad-8.2.20210902094147-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.7 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.12+ x86-64 manylinux: glibc 2.5+ x86-64

schema_salad-8.2.20210902094147-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.4 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ARM64

schema_salad-8.2.20210902094147-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.4 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64 manylinux: glibc 2.5+ x86-64

schema_salad-8.2.20210902094147-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.5 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.12+ x86-64 manylinux: glibc 2.5+ x86-64

schema_salad-8.2.20210902094147-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.2 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ ARM64

schema_salad-8.2.20210902094147-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64 manylinux: glibc 2.5+ x86-64

schema_salad-8.2.20210902094147-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.3 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.12+ x86-64 manylinux: glibc 2.5+ x86-64

schema_salad-8.2.20210902094147-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.1 MB view details)

Uploaded CPython 3.6m manylinux: glibc 2.17+ ARM64

schema_salad-8.2.20210902094147-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB view details)

Uploaded CPython 3.6m manylinux: glibc 2.17+ x86-64 manylinux: glibc 2.5+ x86-64

schema_salad-8.2.20210902094147-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.2 MB view details)

Uploaded CPython 3.6m manylinux: glibc 2.12+ x86-64 manylinux: glibc 2.5+ x86-64

File details

Details for the file schema-salad-8.2.20210902094147.tar.gz.

File metadata

  • Download URL: schema-salad-8.2.20210902094147.tar.gz
  • Upload date:
  • Size: 435.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.2 CPython/3.9.2

File hashes

Hashes for schema-salad-8.2.20210902094147.tar.gz
Algorithm Hash digest
SHA256 0f82b805c92581458ed201db5b65495d5ced8db9848b4d06d30e9ffd3abefb99
MD5 43a5fd8e944cc12c0876b8a74fda02b1
BLAKE2b-256 93125d9df8a216cd03ee4dd51bb82613229f9a98372e41545f50b026ffd22e02

See more details on using hashes here.

File details

Details for the file schema_salad-8.2.20210902094147-py3-none-any.whl.

File metadata

  • Download URL: schema_salad-8.2.20210902094147-py3-none-any.whl
  • Upload date:
  • Size: 474.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.2 CPython/3.9.2

File hashes

Hashes for schema_salad-8.2.20210902094147-py3-none-any.whl
Algorithm Hash digest
SHA256 f31c674780240e2618236acef5f9750b8c4e8b78174f3a96c8d903e78981ac0c
MD5 67e19c88887d9fe122d4ee357e65721f
BLAKE2b-256 fdffda6857efe1ae79e770544284a4adca5e2ace88de18ebcfef00e55c35d288

See more details on using hashes here.

File details

Details for the file schema_salad-8.2.20210902094147-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for schema_salad-8.2.20210902094147-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 10e613fa6166294332f64e71a594b05e3a4ed5081254f4bcbb4befa81764ee6e
MD5 04b90502d9cddd7b61ac2c9f7ea20c18
BLAKE2b-256 2a83f79d15a76c5ce4b0d19867f8dc21b91b3c9978b4f4c1a93b5a700db22625

See more details on using hashes here.

File details

Details for the file schema_salad-8.2.20210902094147-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-8.2.20210902094147-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 546466e141312b5ad41a58e96fa84f445822a4b0c96ed417afdd8180be6240b2
MD5 d2a278fa8a026c1e2698e31abbe6ffb5
BLAKE2b-256 fd486797c8d203e723907edb9c6e109c205384008981d5711329846c51f9e0d1

See more details on using hashes here.

File details

Details for the file schema_salad-8.2.20210902094147-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-8.2.20210902094147-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 59d53f996fa2fe667655a49b282e2c4c42c7556e18bb701020a9fc51d0a8a418
MD5 5081b26432b787c6a4f473822be68bc5
BLAKE2b-256 ac167ef76a649fe7f54dda10c505231a038cb962816e2ca634110e99e09066cb

See more details on using hashes here.

File details

Details for the file schema_salad-8.2.20210902094147-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for schema_salad-8.2.20210902094147-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 49c1df2741f92f3fdfa67b603d9dea5a4554920c212f9f3bed26f7927b8122e5
MD5 641f1ec5823b6135b9b285ff917fa2f0
BLAKE2b-256 a84c5144a723cf907db3d15812f6040ad956e0ccfc781c56ed40d4141d45e171

See more details on using hashes here.

File details

Details for the file schema_salad-8.2.20210902094147-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-8.2.20210902094147-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 e6e7d2ee5f851801569eb19f43257ff8a4902b6adbb78298d01629dd9d9f0380
MD5 2c2b0e26491cb6dac092b21ce36d0a5e
BLAKE2b-256 0816482bd13511359df290e9e02961c2b85596208bee09fcbf3c58847ebf9f06

See more details on using hashes here.

File details

Details for the file schema_salad-8.2.20210902094147-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-8.2.20210902094147-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 7e188ef1a492d70f7a06eefd7c77a1dd50716ff3adcb7d84f55bf0a888e3be6e
MD5 aeb5c8c8d76fbb7b4acb81fbd5265bb1
BLAKE2b-256 87c7a2ec0c7ca91b74a209fd7f79b2e5f62c219b2f9e002b9e43d0c28f28f3d0

See more details on using hashes here.

File details

Details for the file schema_salad-8.2.20210902094147-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for schema_salad-8.2.20210902094147-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 12914f3b7341e4bad00b94a382628f267b715d91de7f0c2afe701f2ed1d79f00
MD5 b821952e407b4996cf095b806b60789e
BLAKE2b-256 2fb148a5acade0f4db8fe8fa1669b7e15922394583540b6045866bb1525ff81c

See more details on using hashes here.

File details

Details for the file schema_salad-8.2.20210902094147-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-8.2.20210902094147-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 e623456d29988778d845dad4100d72ae275073e3010408521949b1c963e539db
MD5 e8e26977854efe3ba67ed3ff79f6028f
BLAKE2b-256 b8634aa2d58b54d67efabd61a3ecf7499535d5ba4fbcf46794358c98dee141dc

See more details on using hashes here.

File details

Details for the file schema_salad-8.2.20210902094147-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-8.2.20210902094147-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 8a794f2a5dbb55c2d43d547c4d27e286c8f373183fd373b4669b37470e493999
MD5 d7576cb1e73dbb304168389d21950679
BLAKE2b-256 a5836178b87ca28b15a2b8e784e54f5648a69ff25f4d675e16290f30dc63d75a

See more details on using hashes here.

File details

Details for the file schema_salad-8.2.20210902094147-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for schema_salad-8.2.20210902094147-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 51603d9367e0e4b54107a66b438b523d4b3620624e7fb396aaa9bb386fa06ace
MD5 aa61e22ce151771ea21019d78eb56dad
BLAKE2b-256 d817b77017609700d1935444fe6a776965ea3d6d68b659ee51ec9404c1a80f8a

See more details on using hashes here.

File details

Details for the file schema_salad-8.2.20210902094147-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-8.2.20210902094147-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 226caa91789e02f12785a11844875e816368b750bb78b7572061ae9f8d4a316f
MD5 d3265cbcb4b9bc49d316dd4701e9b631
BLAKE2b-256 5f2ddd1b8c6cab9c8c5f4d92aeaeb8a955eaa5547fe46fb8e23f4d3a441214a0

See more details on using hashes here.

File details

Details for the file schema_salad-8.2.20210902094147-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-8.2.20210902094147-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 c2793fa77784aa08536478653b603549af6535c6733198b79a4de65606393cf6
MD5 0f699c8b25485197fcf0ae8aedc4238b
BLAKE2b-256 e17df294334b1c8dc0d9c3ddee4f3a81ced6e87127496c963f55c137cf87cb0a

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page