Skip to main content

Schema Annotations for Linked Avro Data (SALAD)

Project description

Linux Build Status Windows Build status Code coverage CII Best Practices

Schema Salad

Salad is a schema language for describing JSON or YAML structured linked data documents. Salad schema describes rules for preprocessing, structural validation, and hyperlink checking for documents described by a Salad schema. Salad supports rich data modeling with inheritance, template specialization, object identifiers, object references, documentation generation, code generation, and transformation to RDF. Salad provides a bridge between document and record oriented data modeling and the Semantic Web.

The Schema Salad library is Python 3.6+ only.

Usage

$ pip install schema_salad

To install from source:

git clone https://github.com/common-workflow-language/schema_salad
cd schema_salad
python3 setup.py install

Commands

Schema salad can be used as a command line tool or imported as a Python module:

$ schema-salad-tool
usage: schema-salad-tool [-h] [--rdf-serializer RDF_SERIALIZER]
                      [--print-jsonld-context | --print-rdfs | --print-avro
                      | --print-rdf | --print-pre | --print-index
                      | --print-metadata | --print-inheritance-dot
                      | --print-fieldrefs-dot | --codegen language
                      | --print-oneline]
                      [--strict | --non-strict] [--verbose | --quiet
                      | --debug]
                      [--version]
                      [schema] [document]

$ python
>>> import schema_salad

Validate a schema:

$ schema-salad-tool myschema.yml

Validate a document using a schema:

$ schema-salad-tool myschema.yml mydocument.yml

Generate HTML documentation:

$ schema-salad-tool myschema.yml > myschema.html

Get JSON-LD context:

$ schema-salad-tool --print-jsonld-context myschema.yml mydocument.yml

Convert a document to JSON-LD:

$ schema-salad-tool --print-pre myschema.yml mydocument.yml > mydocument.jsonld

Generate Python classes for loading/generating documents described by the schema:

$ schema-salad-tool --codegen=python myschema.yml > myschema.py

Display inheritance relationship between classes as a graphviz ‘dot’ file and render as SVG:

$ schema-salad-tool --print-inheritance-dot myschema.yml | dot -Tsvg > myschema.svg

Quick Start

Let’s say you have a ‘basket’ record that can contain items measured either by weight or by count. Here’s an example:

basket:
  - product: bananas
    price: 0.39
    per: pound
    weight: 1
  - product: cucumbers
    price: 0.79
    per: item
    count: 3

We want to validate that all the expected fields are present, the measurement is known, and that “count” cannot be a fractional value. Here is an example schema to do that:

- name: Product
  doc: |
    The base type for a product.  This is an abstract type, so it
    can't be used directly, but can be used to define other types.
  type: record
  abstract: true
  fields:
    product: string
    price: float

- name: ByWeight
  doc: |
    A product, sold by weight.  Products may be sold by pound or by
    kilogram.  Weights may be fractional.
  type: record
  extends: Product
  fields:
    per:
      type:
        type: enum
        symbols:
          - pound
          - kilogram
      jsonldPredicate: '#per'
    weight: float

- name: ByCount
  doc: |
    A product, sold by count.  The count must be a integer value.
  type: record
  extends: Product
  fields:
    per:
      type:
        type: enum
        symbols:
          - item
      jsonldPredicate: '#per'
    count: int

- name: Basket
  doc: |
    A basket of products.  The 'documentRoot' field indicates it is a
    valid starting point for a document.  The 'basket' field will
    validate subtypes of 'Product' (ByWeight and ByCount).
  type: record
  documentRoot: true
  fields:
    basket:
      type:
        type: array
        items: Product

You can check the schema and document in schema_salad/tests/basket_schema.yml and schema_salad/tests/basket.yml:

$ schema-salad-tool basket_schema.yml basket.yml
Document `basket.yml` is valid

Documentation

See the specification and the metaschema (salad schema for itself). For an example application of Schema Salad see the Common Workflow Language.

Rationale

The JSON data model is an popular way to represent structured data. It is attractive because of it’s relative simplicity and is a natural fit with the standard types of many programming languages. However, this simplicity comes at the cost that basic JSON lacks expressive features useful for working with complex data structures and document formats, such as schemas, object references, and namespaces.

JSON-LD is a W3C standard providing a way to describe how to interpret a JSON document as Linked Data by means of a “context”. JSON-LD provides a powerful solution for representing object references and namespaces in JSON based on standard web URIs, but is not itself a schema language. Without a schema providing a well defined structure, it is difficult to process an arbitrary JSON-LD document as idiomatic JSON because there are many ways to express the same data that are logically equivalent but structurally distinct.

Several schema languages exist for describing and validating JSON data, such as JSON Schema and Apache Avro data serialization system, however none understand linked data. As a result, to fully take advantage of JSON-LD to build the next generation of linked data applications, one must maintain separate JSON schema, JSON-LD context, RDF schema, and human documentation, despite significant overlap of content and obvious need for these documents to stay synchronized.

Schema Salad is designed to address this gap. It provides a schema language and processing rules for describing structured JSON content permitting URI resolution and strict document validation. The schema language supports linked data through annotations that describe the linked data interpretation of the content, enables generation of JSON-LD context and RDF schema, and production of RDF triples by applying the JSON-LD context. The schema language also provides for robust support of inline documentation.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

schema-salad-7.1.20210518142926.tar.gz (418.3 kB view details)

Uploaded Source

Built Distributions

schema_salad-7.1.20210518142926-py3-none-any.whl (471.7 kB view details)

Uploaded Python 3

schema_salad-7.1.20210518142926-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.4 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

schema_salad-7.1.20210518142926-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.4 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.12+ x86-64 manylinux: glibc 2.5+ x86-64

schema_salad-7.1.20210518142926-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.3 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

schema_salad-7.1.20210518142926-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.3 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.12+ x86-64 manylinux: glibc 2.5+ x86-64

schema_salad-7.1.20210518142926-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB view details)

Uploaded CPython 3.6m manylinux: glibc 2.17+ x86-64

schema_salad-7.1.20210518142926-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.3 MB view details)

Uploaded CPython 3.6m manylinux: glibc 2.12+ x86-64 manylinux: glibc 2.5+ x86-64

File details

Details for the file schema-salad-7.1.20210518142926.tar.gz.

File metadata

  • Download URL: schema-salad-7.1.20210518142926.tar.gz
  • Upload date:
  • Size: 418.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.0.1 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.9.2

File hashes

Hashes for schema-salad-7.1.20210518142926.tar.gz
Algorithm Hash digest
SHA256 ead223dab61187ac091d44baa0101edc726bb49cb80123180443a3a06969f5d6
MD5 f5861354d1bd9c5a0c78c4dd84b42c40
BLAKE2b-256 1270585f72a1ac4d006c60e14d4f4cf8e44415b24c3131f44938a8cb710193e2

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-7.1.20210518142926-py3-none-any.whl.

File metadata

  • Download URL: schema_salad-7.1.20210518142926-py3-none-any.whl
  • Upload date:
  • Size: 471.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.0.1 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.9.2

File hashes

Hashes for schema_salad-7.1.20210518142926-py3-none-any.whl
Algorithm Hash digest
SHA256 5f35cebb518a6dd9d9bb8203d9fcf5bd927e0c3cc166dd7dde86c61e44b806e5
MD5 b5935940d2c17167570caa317648f6aa
BLAKE2b-256 37b0ee096e805f39ad14c11cd3fd015ff1185dc2f2f8f8ed4bf8b536146eea5d

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-7.1.20210518142926-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-7.1.20210518142926-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 258d016a31dcfd79547fb2b24db020a97eb62c510e82b00e017e95ee7de20a6d
MD5 5b12caa7485b987349c7417bef61134e
BLAKE2b-256 827234acf31c8c000a224a08e137a37011e9dc8e68452c1c148aecac8f354982

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-7.1.20210518142926-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-7.1.20210518142926-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 cb184d929086206213f25bd5123d7d1d902e77b313ce66c5f0b8baa2f856a62c
MD5 7d4687fc5bab9fb54446ae24c755b329
BLAKE2b-256 32b0c081817b6daaa9c6b6bc37ec9d95e465515fabc36d55789322d7e330d646

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-7.1.20210518142926-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-7.1.20210518142926-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 bd094455b8e7da534de9325b2f2568678b513e50c102950e853f8cb7cd85d317
MD5 528a16af78ede56b38dd71d5603afaf1
BLAKE2b-256 fd79daa44f6687843c63ca7f77a2ea282f61f88ccf91838851054c8287610cd1

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-7.1.20210518142926-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-7.1.20210518142926-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 28073cac14bdddd1bf9e8282850a46a22183fa4b1772c95dffc4c73bc3e47c92
MD5 6493c0256500f66744368d33a510b59e
BLAKE2b-256 7829467266c4fe9941ac220c2c61f8213102108ac4eebbd79c073f22c4800982

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-7.1.20210518142926-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-7.1.20210518142926-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 d392c37d3a2b9a7834e4e1b5bf71babb7f584a395031c831c9f3e46a99077faa
MD5 aa61619f6be09aca0682e2ce17f9f8ef
BLAKE2b-256 2373fadcb9e5f1d78aaa9289c4042f16e2e68f0ea0a2548060be12ce299894c2

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-7.1.20210518142926-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-7.1.20210518142926-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 4b6114b2c66dfaaf359aca07c162ca2f98a80550af5f9e6ed2f07803d4bcda17
MD5 e2b83fea470bacb9e3f7406ac1504d8f
BLAKE2b-256 05c3a6957a3020efd456d68a60041e9a1f96299e3b1a42f300cd4ea16d29bcb1

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page