Skip to main content

Schema Annotations for Linked Avro Data (SALAD)

Project description

Linux Build Status Code coverage CII Best Practices

Schema Salad

Salad is a schema language for describing JSON or YAML structured linked data documents. Salad schema describes rules for preprocessing, structural validation, and hyperlink checking for documents described by a Salad schema. Salad supports rich data modeling with inheritance, template specialization, object identifiers, object references, documentation generation, code generation, and transformation to RDF. Salad provides a bridge between document and record oriented data modeling and the Semantic Web.

The Schema Salad library is Python 3.6+ only.

Usage

$ pip install schema_salad

To install from source:

git clone https://github.com/common-workflow-language/schema_salad
cd schema_salad
python3 setup.py install

Commands

Schema salad can be used as a command line tool or imported as a Python module:

$ schema-salad-tool
usage: schema-salad-tool [-h] [--rdf-serializer RDF_SERIALIZER]
                      [--print-jsonld-context | --print-rdfs | --print-avro
                      | --print-rdf | --print-pre | --print-index
                      | --print-metadata | --print-inheritance-dot
                      | --print-fieldrefs-dot | --codegen language
                      | --print-oneline]
                      [--strict | --non-strict] [--verbose | --quiet
                      | --debug]
                      [--version]
                      [schema] [document]

$ python
>>> import schema_salad

Validate a schema:

$ schema-salad-tool myschema.yml

Validate a document using a schema:

$ schema-salad-tool myschema.yml mydocument.yml

Generate HTML documentation:

$ schema-salad-tool myschema.yml > myschema.html

Get JSON-LD context:

$ schema-salad-tool --print-jsonld-context myschema.yml mydocument.yml

Convert a document to JSON-LD:

$ schema-salad-tool --print-pre myschema.yml mydocument.yml > mydocument.jsonld

Generate Python classes for loading/generating documents described by the schema:

$ schema-salad-tool --codegen=python myschema.yml > myschema.py

Display inheritance relationship between classes as a graphviz ‘dot’ file and render as SVG:

$ schema-salad-tool --print-inheritance-dot myschema.yml | dot -Tsvg > myschema.svg

Quick Start

Let’s say you have a ‘basket’ record that can contain items measured either by weight or by count. Here’s an example:

basket:
  - product: bananas
    price: 0.39
    per: pound
    weight: 1
  - product: cucumbers
    price: 0.79
    per: item
    count: 3

We want to validate that all the expected fields are present, the measurement is known, and that “count” cannot be a fractional value. Here is an example schema to do that:

- name: Product
  doc: |
    The base type for a product.  This is an abstract type, so it
    can't be used directly, but can be used to define other types.
  type: record
  abstract: true
  fields:
    product: string
    price: float

- name: ByWeight
  doc: |
    A product, sold by weight.  Products may be sold by pound or by
    kilogram.  Weights may be fractional.
  type: record
  extends: Product
  fields:
    per:
      type:
        type: enum
        symbols:
          - pound
          - kilogram
      jsonldPredicate: '#per'
    weight: float

- name: ByCount
  doc: |
    A product, sold by count.  The count must be a integer value.
  type: record
  extends: Product
  fields:
    per:
      type:
        type: enum
        symbols:
          - item
      jsonldPredicate: '#per'
    count: int

- name: Basket
  doc: |
    A basket of products.  The 'documentRoot' field indicates it is a
    valid starting point for a document.  The 'basket' field will
    validate subtypes of 'Product' (ByWeight and ByCount).
  type: record
  documentRoot: true
  fields:
    basket:
      type:
        type: array
        items: Product

You can check the schema and document in schema_salad/tests/basket_schema.yml and schema_salad/tests/basket.yml:

$ schema-salad-tool basket_schema.yml basket.yml
Document `basket.yml` is valid

Documentation

See the specification and the metaschema (salad schema for itself). For an example application of Schema Salad see the Common Workflow Language.

Rationale

The JSON data model is an popular way to represent structured data. It is attractive because of it’s relative simplicity and is a natural fit with the standard types of many programming languages. However, this simplicity comes at the cost that basic JSON lacks expressive features useful for working with complex data structures and document formats, such as schemas, object references, and namespaces.

JSON-LD is a W3C standard providing a way to describe how to interpret a JSON document as Linked Data by means of a “context”. JSON-LD provides a powerful solution for representing object references and namespaces in JSON based on standard web URIs, but is not itself a schema language. Without a schema providing a well defined structure, it is difficult to process an arbitrary JSON-LD document as idiomatic JSON because there are many ways to express the same data that are logically equivalent but structurally distinct.

Several schema languages exist for describing and validating JSON data, such as JSON Schema and Apache Avro data serialization system, however none understand linked data. As a result, to fully take advantage of JSON-LD to build the next generation of linked data applications, one must maintain separate JSON schema, JSON-LD context, RDF schema, and human documentation, despite significant overlap of content and obvious need for these documents to stay synchronized.

Schema Salad is designed to address this gap. It provides a schema language and processing rules for describing structured JSON content permitting URI resolution and strict document validation. The schema language supports linked data through annotations that describe the linked data interpretation of the content, enables generation of JSON-LD context and RDF schema, and production of RDF triples by applying the JSON-LD context. The schema language also provides for robust support of inline documentation.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

schema-salad-8.1.20210716111910.tar.gz (434.3 kB view details)

Uploaded Source

Built Distributions

schema_salad-8.1.20210716111910-py3-none-any.whl (473.9 kB view details)

Uploaded Python 3

schema_salad-8.1.20210716111910-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64 manylinux: glibc 2.5+ x86-64

schema_salad-8.1.20210716111910-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.7 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.12+ x86-64 manylinux: glibc 2.5+ x86-64

schema_salad-8.1.20210716111910-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64 manylinux: glibc 2.5+ x86-64

schema_salad-8.1.20210716111910-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.5 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.12+ x86-64 manylinux: glibc 2.5+ x86-64

schema_salad-8.1.20210716111910-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64 manylinux: glibc 2.5+ x86-64

schema_salad-8.1.20210716111910-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.2 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.12+ x86-64 manylinux: glibc 2.5+ x86-64

schema_salad-8.1.20210716111910-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB view details)

Uploaded CPython 3.6m manylinux: glibc 2.17+ x86-64 manylinux: glibc 2.5+ x86-64

schema_salad-8.1.20210716111910-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.2 MB view details)

Uploaded CPython 3.6m manylinux: glibc 2.12+ x86-64 manylinux: glibc 2.5+ x86-64

File details

Details for the file schema-salad-8.1.20210716111910.tar.gz.

File metadata

  • Download URL: schema-salad-8.1.20210716111910.tar.gz
  • Upload date:
  • Size: 434.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.6.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.61.2 CPython/3.9.2

File hashes

Hashes for schema-salad-8.1.20210716111910.tar.gz
Algorithm Hash digest
SHA256 3f851b385d044c58d359285ba471298b6199478a4978f892a83b15cbfb282f25
MD5 9e65ff958a8ec003ac60811d65de6923
BLAKE2b-256 3da4401d8ad4fec9654c26b3417b314be10684acd062c1630549920df622ed3c

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-8.1.20210716111910-py3-none-any.whl.

File metadata

  • Download URL: schema_salad-8.1.20210716111910-py3-none-any.whl
  • Upload date:
  • Size: 473.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.6.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.61.2 CPython/3.9.2

File hashes

Hashes for schema_salad-8.1.20210716111910-py3-none-any.whl
Algorithm Hash digest
SHA256 6b206110e47d83c2c40865ebf9e8443e24b92716840aee52cc37c17d201c487e
MD5 116837a81bcebee41bb32e09f0ab8305
BLAKE2b-256 3a8a92dcbd3c120bf94022290a62a207e453ad6822024cc526c3eb97ae401d2b

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-8.1.20210716111910-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-8.1.20210716111910-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 9a3cf77a3b8c3ecd392bb46e9e5570f5c7f3d8a9d9e9fd41a9fcbd04cb01f54f
MD5 859c7d733ff348aaca72128b3987ea20
BLAKE2b-256 c725776b3856a2ecba61fcb17e74f8f096f2448a3e86e0ec8bfd3922923135c2

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-8.1.20210716111910-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-8.1.20210716111910-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 48d46452ca4e704e2ea14dfbd9466ba2cdd451dc88393af4966ed64b6201e550
MD5 0ae7306c91168c13c84b816308db5ce8
BLAKE2b-256 20aabe5153aa8e5ec0b28e48d680209edfb473d4d364b3d7c9187b198c3ace9c

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-8.1.20210716111910-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-8.1.20210716111910-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 65fa60595193fe79729eb66a199dcb6d3b37bbc27b9094d59de3845ef0a17ba8
MD5 4af82986abe03c182ba9522dba24fcb9
BLAKE2b-256 afff3ff20903d640f749e0fcb6921c612d152e87cb68478570fc5b055e71192a

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-8.1.20210716111910-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-8.1.20210716111910-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 6012fb8538d1e562baabc7346adc28eb6af8a0888854a491e1c91c84ee71a431
MD5 450c576ee4a7bf52a94d2f1bf0efc82d
BLAKE2b-256 b7681d1fa59a9d209b0ebeedacb226b6b4223aa6113a2b46acc7c3642b63f802

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-8.1.20210716111910-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-8.1.20210716111910-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 1f7bd64e551eece24be454628bca1f034e72616a0a6db8b6899655ee4cb8c473
MD5 1f948549685fe4e21daf9b1e0e8f260e
BLAKE2b-256 29ec9850dcb465686eeb1882dfbd9a9334ad399314ba35123fd0c9e8637783c5

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-8.1.20210716111910-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-8.1.20210716111910-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 9d58f804032fd93e9cfc703c79ae7d85b544d42dd424953a71d7416cf7c25a0b
MD5 e3c17edf5bf0c1a35c0d9b7a98f221ba
BLAKE2b-256 fbfa8508fd56cc35a4c3489d384fd883b08e73f704e6187cdcad29056845fc43

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-8.1.20210716111910-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-8.1.20210716111910-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 c7d5166f5738ef48c189289f011e55dfb6600d759c02169a854a46685c5030f4
MD5 ab49405f654d0e89b3f5af5ff31d9350
BLAKE2b-256 803f7ebd0f8a600e8a44752fe89eee9267b5ce56f847b2e24083b9be3fafe556

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-8.1.20210716111910-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-8.1.20210716111910-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 e2d1d2a0b70de988185ec18b8cb3b7fb1b28d005d6e50582e4652db7c7c09477
MD5 e4556f5e8d5fd849ed14c88204adc322
BLAKE2b-256 f27547fd544a38ebc617c5b412e24197b4ce8c8afbabc96006c18e027ceed40e

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page