Skip to main content

Schema Annotations for Linked Avro Data (SALAD)

Project description

Linux Build Status Code coverage CII Best Practices

Schema Salad

Salad is a schema language for describing JSON or YAML structured linked data documents. Salad schema describes rules for preprocessing, structural validation, and hyperlink checking for documents described by a Salad schema. Salad supports rich data modeling with inheritance, template specialization, object identifiers, object references, documentation generation, code generation, and transformation to RDF. Salad provides a bridge between document and record oriented data modeling and the Semantic Web.

The Schema Salad library is Python 3.6+ only.

Usage

$ pip install schema_salad

To install from source:

git clone https://github.com/common-workflow-language/schema_salad
cd schema_salad
python3 setup.py install

Commands

Schema salad can be used as a command line tool or imported as a Python module:

$ schema-salad-tool
usage: schema-salad-tool [-h] [--rdf-serializer RDF_SERIALIZER]
                      [--print-jsonld-context | --print-rdfs | --print-avro
                      | --print-rdf | --print-pre | --print-index
                      | --print-metadata | --print-inheritance-dot
                      | --print-fieldrefs-dot | --codegen language
                      | --print-oneline]
                      [--strict | --non-strict] [--verbose | --quiet
                      | --debug]
                      [--version]
                      [schema] [document]

$ python
>>> import schema_salad

Validate a schema:

$ schema-salad-tool myschema.yml

Validate a document using a schema:

$ schema-salad-tool myschema.yml mydocument.yml

Generate HTML documentation:

$ schema-salad-tool myschema.yml > myschema.html

Get JSON-LD context:

$ schema-salad-tool --print-jsonld-context myschema.yml mydocument.yml

Convert a document to JSON-LD:

$ schema-salad-tool --print-pre myschema.yml mydocument.yml > mydocument.jsonld

Generate Python classes for loading/generating documents described by the schema:

$ schema-salad-tool --codegen=python myschema.yml > myschema.py

Display inheritance relationship between classes as a graphviz ‘dot’ file and render as SVG:

$ schema-salad-tool --print-inheritance-dot myschema.yml | dot -Tsvg > myschema.svg

Quick Start

Let’s say you have a ‘basket’ record that can contain items measured either by weight or by count. Here’s an example:

basket:
  - product: bananas
    price: 0.39
    per: pound
    weight: 1
  - product: cucumbers
    price: 0.79
    per: item
    count: 3

We want to validate that all the expected fields are present, the measurement is known, and that “count” cannot be a fractional value. Here is an example schema to do that:

- name: Product
  doc: |
    The base type for a product.  This is an abstract type, so it
    can't be used directly, but can be used to define other types.
  type: record
  abstract: true
  fields:
    product: string
    price: float

- name: ByWeight
  doc: |
    A product, sold by weight.  Products may be sold by pound or by
    kilogram.  Weights may be fractional.
  type: record
  extends: Product
  fields:
    per:
      type:
        type: enum
        symbols:
          - pound
          - kilogram
      jsonldPredicate: '#per'
    weight: float

- name: ByCount
  doc: |
    A product, sold by count.  The count must be a integer value.
  type: record
  extends: Product
  fields:
    per:
      type:
        type: enum
        symbols:
          - item
      jsonldPredicate: '#per'
    count: int

- name: Basket
  doc: |
    A basket of products.  The 'documentRoot' field indicates it is a
    valid starting point for a document.  The 'basket' field will
    validate subtypes of 'Product' (ByWeight and ByCount).
  type: record
  documentRoot: true
  fields:
    basket:
      type:
        type: array
        items: Product

You can check the schema and document in schema_salad/tests/basket_schema.yml and schema_salad/tests/basket.yml:

$ schema-salad-tool basket_schema.yml basket.yml
Document `basket.yml` is valid

Documentation

See the specification and the metaschema (salad schema for itself). For an example application of Schema Salad see the Common Workflow Language.

Rationale

The JSON data model is an popular way to represent structured data. It is attractive because of it’s relative simplicity and is a natural fit with the standard types of many programming languages. However, this simplicity comes at the cost that basic JSON lacks expressive features useful for working with complex data structures and document formats, such as schemas, object references, and namespaces.

JSON-LD is a W3C standard providing a way to describe how to interpret a JSON document as Linked Data by means of a “context”. JSON-LD provides a powerful solution for representing object references and namespaces in JSON based on standard web URIs, but is not itself a schema language. Without a schema providing a well defined structure, it is difficult to process an arbitrary JSON-LD document as idiomatic JSON because there are many ways to express the same data that are logically equivalent but structurally distinct.

Several schema languages exist for describing and validating JSON data, such as JSON Schema and Apache Avro data serialization system, however none understand linked data. As a result, to fully take advantage of JSON-LD to build the next generation of linked data applications, one must maintain separate JSON schema, JSON-LD context, RDF schema, and human documentation, despite significant overlap of content and obvious need for these documents to stay synchronized.

Schema Salad is designed to address this gap. It provides a schema language and processing rules for describing structured JSON content permitting URI resolution and strict document validation. The schema language supports linked data through annotations that describe the linked data interpretation of the content, enables generation of JSON-LD context and RDF schema, and production of RDF triples by applying the JSON-LD context. The schema language also provides for robust support of inline documentation.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

schema-salad-8.1.20210627200047.tar.gz (434.3 kB view details)

Uploaded Source

Built Distributions

schema_salad-8.1.20210627200047-py3-none-any.whl (473.9 kB view details)

Uploaded Python 3

schema_salad-8.1.20210627200047-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64 manylinux: glibc 2.5+ x86-64

schema_salad-8.1.20210627200047-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.7 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.12+ x86-64 manylinux: glibc 2.5+ x86-64

schema_salad-8.1.20210627200047-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64 manylinux: glibc 2.5+ x86-64

schema_salad-8.1.20210627200047-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.5 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.12+ x86-64 manylinux: glibc 2.5+ x86-64

schema_salad-8.1.20210627200047-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64 manylinux: glibc 2.5+ x86-64

schema_salad-8.1.20210627200047-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.2 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.12+ x86-64 manylinux: glibc 2.5+ x86-64

schema_salad-8.1.20210627200047-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB view details)

Uploaded CPython 3.6m manylinux: glibc 2.17+ x86-64 manylinux: glibc 2.5+ x86-64

schema_salad-8.1.20210627200047-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.2 MB view details)

Uploaded CPython 3.6m manylinux: glibc 2.12+ x86-64 manylinux: glibc 2.5+ x86-64

File details

Details for the file schema-salad-8.1.20210627200047.tar.gz.

File metadata

  • Download URL: schema-salad-8.1.20210627200047.tar.gz
  • Upload date:
  • Size: 434.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.6.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.1 CPython/3.9.2

File hashes

Hashes for schema-salad-8.1.20210627200047.tar.gz
Algorithm Hash digest
SHA256 e2f4d6eadb060f3d88372bb4bdefb7c22eb4b191c2657a8a82183d100148fac5
MD5 7745bb208e4a86c6449011212c9759fd
BLAKE2b-256 3f56426b88e72b322cd5f83004f6ed1ead6ac90412f2110585eb7effc7cb85cf

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-8.1.20210627200047-py3-none-any.whl.

File metadata

  • Download URL: schema_salad-8.1.20210627200047-py3-none-any.whl
  • Upload date:
  • Size: 473.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.6.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.1 CPython/3.9.2

File hashes

Hashes for schema_salad-8.1.20210627200047-py3-none-any.whl
Algorithm Hash digest
SHA256 baa151af9fe824e3981ec20de23a9b66e6fccedebe72c475e03f91af71a930f6
MD5 299aabc92288cf73c6a7e01abb87d3ae
BLAKE2b-256 c4be3ed81521434a3a8c3b0eac1e0c25d4eb6de6ec97861197f17bbd74317889

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-8.1.20210627200047-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-8.1.20210627200047-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 51f44e81b974f28900429051142ae26011123addf659dc81f093f4afd765d92d
MD5 2f22b31b13c396ae0c0f0214cd45c992
BLAKE2b-256 edc6797fbc32626451f8d3f73d8d45387fe6f83ae6cc1b3528c376ca0edd2a70

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-8.1.20210627200047-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-8.1.20210627200047-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 c67649a9d048c60274b11bfb02dc1679dd5394f6ccf1bfa58653526f9c2e4631
MD5 a2b44a6d76fcda8b9c3e128585e971bb
BLAKE2b-256 3e2e74e96234a5492c0f420ca3969745ef3cea5e4bc929e4db86d402e58e1f8a

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-8.1.20210627200047-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-8.1.20210627200047-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 8c4152ac88d5a6146742412485453ed47da4461c4fc2ff935fe63291297d8cc2
MD5 7804233c10fc353fb312b6466930ecaa
BLAKE2b-256 fdd28e67072ef9e2c1689d05e31b1dbd3f37bd4a6e2f922bb42fe0f91ef2129c

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-8.1.20210627200047-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-8.1.20210627200047-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 333b1f3a44efbb5053e6158688f0422b6355d9b1655fe1e4d06c878d3f6dd786
MD5 a36a04ad333c2b3f4dcbdaff103b5fde
BLAKE2b-256 6cfd1baeb4b641113d2710c9918c2e3ba82b4c3bdb4b6d4dc0dbd2027aaf2034

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-8.1.20210627200047-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-8.1.20210627200047-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 f1f360949196627c44c36af8323aad2f97f30f663516de9987f5f236f82bef01
MD5 5e8026f3594a424b3ca6558df3df398f
BLAKE2b-256 f86af64da783d2da6db209acd6e5358b1b0de5f1d9ad4a98341018f1f214ed0b

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-8.1.20210627200047-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-8.1.20210627200047-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 c09235a3645b98907287a29376a6d1966c68d6036a885ada1b6a9a56c1a0bd96
MD5 92b724588d34d2fb5819593a249b5d6e
BLAKE2b-256 b0d003fbf4423123977a520bbaaa740c4e4bb809c6575afc95ba9e3b6d95d5b0

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-8.1.20210627200047-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-8.1.20210627200047-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 65205730686d410c44884b1324d0a4761ebfac81e0d4175e75faa86da413f60e
MD5 124f5ee2d1626d8b4484eeca6586bb69
BLAKE2b-256 bf35825048d17b582f2e8c4a68668006326c023e2e1d9b641788a656ed6ec469

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-8.1.20210627200047-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-8.1.20210627200047-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 768a2df33d6427e3f70093acdf566a2da27559a45b66d6531262ef3f38bb96f8
MD5 509deec3fbfb52f9a1f6750bf195842d
BLAKE2b-256 fc3243aacd9d42367809e41c2414d195c174dc79174c01a13a96765bc978c90d

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page