Skip to main content

Schema Annotations for Linked Avro Data (SALAD)

Project description

Linux Build Status Windows Build status Code coverage CII Best Practices

Schema Salad

Salad is a schema language for describing JSON or YAML structured linked data documents. Salad schema describes rules for preprocessing, structural validation, and hyperlink checking for documents described by a Salad schema. Salad supports rich data modeling with inheritance, template specialization, object identifiers, object references, documentation generation, code generation, and transformation to RDF. Salad provides a bridge between document and record oriented data modeling and the Semantic Web.

The Schema Salad library is Python 3.6+ only.

Usage

$ pip install schema_salad

To install from source:

git clone https://github.com/common-workflow-language/schema_salad
cd schema_salad
python3 setup.py install

Commands

Schema salad can be used as a command line tool or imported as a Python module:

$ schema-salad-tool
usage: schema-salad-tool [-h] [--rdf-serializer RDF_SERIALIZER]
                      [--print-jsonld-context | --print-rdfs | --print-avro
                      | --print-rdf | --print-pre | --print-index
                      | --print-metadata | --print-inheritance-dot
                      | --print-fieldrefs-dot | --codegen language
                      | --print-oneline]
                      [--strict | --non-strict] [--verbose | --quiet
                      | --debug]
                      [--version]
                      [schema] [document]

$ python
>>> import schema_salad

Validate a schema:

$ schema-salad-tool myschema.yml

Validate a document using a schema:

$ schema-salad-tool myschema.yml mydocument.yml

Generate HTML documentation:

$ schema-salad-tool myschema.yml > myschema.html

Get JSON-LD context:

$ schema-salad-tool --print-jsonld-context myschema.yml mydocument.yml

Convert a document to JSON-LD:

$ schema-salad-tool --print-pre myschema.yml mydocument.yml > mydocument.jsonld

Generate Python classes for loading/generating documents described by the schema:

$ schema-salad-tool --codegen=python myschema.yml > myschema.py

Display inheritance relationship between classes as a graphviz ‘dot’ file and render as SVG:

$ schema-salad-tool --print-inheritance-dot myschema.yml | dot -Tsvg > myschema.svg

Quick Start

Let’s say you have a ‘basket’ record that can contain items measured either by weight or by count. Here’s an example:

basket:
  - product: bananas
    price: 0.39
    per: pound
    weight: 1
  - product: cucumbers
    price: 0.79
    per: item
    count: 3

We want to validate that all the expected fields are present, the measurement is known, and that “count” cannot be a fractional value. Here is an example schema to do that:

- name: Product
  doc: |
    The base type for a product.  This is an abstract type, so it
    can't be used directly, but can be used to define other types.
  type: record
  abstract: true
  fields:
    product: string
    price: float

- name: ByWeight
  doc: |
    A product, sold by weight.  Products may be sold by pound or by
    kilogram.  Weights may be fractional.
  type: record
  extends: Product
  fields:
    per:
      type:
        type: enum
        symbols:
          - pound
          - kilogram
      jsonldPredicate: '#per'
    weight: float

- name: ByCount
  doc: |
    A product, sold by count.  The count must be a integer value.
  type: record
  extends: Product
  fields:
    per:
      type:
        type: enum
        symbols:
          - item
      jsonldPredicate: '#per'
    count: int

- name: Basket
  doc: |
    A basket of products.  The 'documentRoot' field indicates it is a
    valid starting point for a document.  The 'basket' field will
    validate subtypes of 'Product' (ByWeight and ByCount).
  type: record
  documentRoot: true
  fields:
    basket:
      type:
        type: array
        items: Product

You can check the schema and document in schema_salad/tests/basket_schema.yml and schema_salad/tests/basket.yml:

$ schema-salad-tool basket_schema.yml basket.yml
Document `basket.yml` is valid

Documentation

See the specification and the metaschema (salad schema for itself). For an example application of Schema Salad see the Common Workflow Language.

Rationale

The JSON data model is an popular way to represent structured data. It is attractive because of it’s relative simplicity and is a natural fit with the standard types of many programming languages. However, this simplicity comes at the cost that basic JSON lacks expressive features useful for working with complex data structures and document formats, such as schemas, object references, and namespaces.

JSON-LD is a W3C standard providing a way to describe how to interpret a JSON document as Linked Data by means of a “context”. JSON-LD provides a powerful solution for representing object references and namespaces in JSON based on standard web URIs, but is not itself a schema language. Without a schema providing a well defined structure, it is difficult to process an arbitrary JSON-LD document as idiomatic JSON because there are many ways to express the same data that are logically equivalent but structurally distinct.

Several schema languages exist for describing and validating JSON data, such as JSON Schema and Apache Avro data serialization system, however none understand linked data. As a result, to fully take advantage of JSON-LD to build the next generation of linked data applications, one must maintain separate JSON schema, JSON-LD context, RDF schema, and human documentation, despite significant overlap of content and obvious need for these documents to stay synchronized.

Schema Salad is designed to address this gap. It provides a schema language and processing rules for describing structured JSON content permitting URI resolution and strict document validation. The schema language supports linked data through annotations that describe the linked data interpretation of the content, enables generation of JSON-LD context and RDF schema, and production of RDF triples by applying the JSON-LD context. The schema language also provides for robust support of inline documentation.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

schema-salad-7.1.20210316164414.tar.gz (418.3 kB view details)

Uploaded Source

Built Distributions

schema_salad-7.1.20210316164414-py3-none-any.whl (471.7 kB view details)

Uploaded Python 3

schema_salad-7.1.20210316164414-cp38-cp38-manylinux2010_x86_64.whl (2.4 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.12+ x86-64

schema_salad-7.1.20210316164414-cp37-cp37m-manylinux2010_x86_64.whl (2.3 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.12+ x86-64

schema_salad-7.1.20210316164414-cp36-cp36m-manylinux2010_x86_64.whl (2.3 MB view details)

Uploaded CPython 3.6m manylinux: glibc 2.12+ x86-64

File details

Details for the file schema-salad-7.1.20210316164414.tar.gz.

File metadata

  • Download URL: schema-salad-7.1.20210316164414.tar.gz
  • Upload date:
  • Size: 418.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.7.3 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for schema-salad-7.1.20210316164414.tar.gz
Algorithm Hash digest
SHA256 33ef9e67d5923a0605f98b54250852cb9f8b7fe7ba44bae1cbd3ba77dbf44a12
MD5 75a78dd76e206cd2ecc48f1c19691cce
BLAKE2b-256 67cf97ccfade46cbd36f687ec6c4233142eb879398ab3311debddbccf8f341de

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-7.1.20210316164414-py3-none-any.whl.

File metadata

  • Download URL: schema_salad-7.1.20210316164414-py3-none-any.whl
  • Upload date:
  • Size: 471.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.7.3 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for schema_salad-7.1.20210316164414-py3-none-any.whl
Algorithm Hash digest
SHA256 828e412f12059d04fda101aeeb28abfaf7068f3e1e59c3da94df226776d916e7
MD5 69063795d351edbb95627d20d5f16065
BLAKE2b-256 3c8e017aa84fd74726e015ee0d7d6f32e017b6f8cde2d0c5a436a3be49847d1e

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-7.1.20210316164414-cp38-cp38-manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-7.1.20210316164414-cp38-cp38-manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 d302c803229f526ee3ae241d53ecea77050263d2ebe6c1167b372c0c0679c514
MD5 5d3435fdfa821e8cfe50f80f581c391c
BLAKE2b-256 e911f4c149161f606d7bbfde0848b7e4ddbdd4ddb7840ac83b2042abc558de87

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-7.1.20210316164414-cp38-cp38-manylinux2010_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-7.1.20210316164414-cp38-cp38-manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 f712cf40fd185eae6c0ddcea6d8de7a0674e5e148ed928548e471fa3df3040fe
MD5 ff0e6d0bab6446156b3914c76a43de4b
BLAKE2b-256 046b73715905d3d058629f751ebe3a2116bd2bfaeff67f979113a14715a423e4

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-7.1.20210316164414-cp38-cp38-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-7.1.20210316164414-cp38-cp38-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 f0cb6de96e5fbc997ae1a1d89ace39e54bd297823b165f6dc908e568f0e42f0e
MD5 75039ce7f7316763e942c7c4383cea86
BLAKE2b-256 57feb578bfb526995b2d5007efbd761a6f95290607dcd2b62ae2047fa28cf9a9

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-7.1.20210316164414-cp37-cp37m-manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-7.1.20210316164414-cp37-cp37m-manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 7a362eae65cbde8f2aa1bd6d856406a0079adb518c731c92ff1e17f6f29270b1
MD5 7650291f78572709ea16e0ecddc47f51
BLAKE2b-256 a82338520414d61b2e8e40d0b2fd275dc94d271af4ab3511f24a4fb74c3e48fc

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-7.1.20210316164414-cp37-cp37m-manylinux2010_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-7.1.20210316164414-cp37-cp37m-manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 9e2ba0710c90449f2bf653a98e97db92f864e0128dfea77c4d26a201a8ad0353
MD5 93b64cd92101c4a184ebfe631f6ea5c6
BLAKE2b-256 d178123a118627b77583d42f60f70c670444b947b3243a886b89f6909b3f7e61

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-7.1.20210316164414-cp37-cp37m-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-7.1.20210316164414-cp37-cp37m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 081b1413a662d391508c2ddfce2890f8af4e4f7a85345ed423f27b0e1c3a79d8
MD5 615b55ed5d98dec67c32279c82fe4783
BLAKE2b-256 344436261c3b2e54ec5095162d7cd696dacaafa0617a741004a7210367bc3848

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-7.1.20210316164414-cp36-cp36m-manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-7.1.20210316164414-cp36-cp36m-manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 53e6bb1565b2be9c2e6e60e82811eba4c523f7142a9ad806113d00dfd84b22b2
MD5 d3d99a40abe970aacff723b7acb7cdda
BLAKE2b-256 48fb45721d0dee927983a836b69ac71bc9f1b8de7d6b116294638bc768459495

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-7.1.20210316164414-cp36-cp36m-manylinux2010_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-7.1.20210316164414-cp36-cp36m-manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 6c2de83ee47ec0de26f7f6fe79d2a015abf3cfb7c1eaa214290d5d4dace093ca
MD5 a5e29209a788b5e88e62dc8ec203b198
BLAKE2b-256 6e098f75ff0c75f591b59b4686990ab85c6c61be5adb6de1ec57c261e098e661

See more details on using hashes here.

Provenance

File details

Details for the file schema_salad-7.1.20210316164414-cp36-cp36m-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for schema_salad-7.1.20210316164414-cp36-cp36m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 4384f75d6649f38cef406c522ee2b0d800ccbe85deda1d1f19bbf5ac3f3eaa3c
MD5 2ad227e25c71cde86d583ea513eb78ca
BLAKE2b-256 4bab2679f3e9e1ef3f2931b2160b44d1b6f64fd0cbe0e08d33ee5435a4434cf9

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page