Schema Annotations for Linked Avro Data (SALAD)

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for mcrusoe from gravatar.com mcrusoe Avatar for mvdbeek from gravatar.com mvdbeek Avatar for rwn from gravatar.com rwn Avatar for tetron from gravatar.com tetron

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Project description

Linux Build Status Code coverage Documentation Status CII Best Practices

Schema Salad

Salad is a schema language for describing JSON or YAML structured linked data documents. Salad schema describes rules for preprocessing, structural validation, and hyperlink checking for documents described by a Salad schema. Salad supports rich data modeling with inheritance, template specialization, object identifiers, object references, documentation generation, code generation, and transformation to RDF. Salad provides a bridge between document and record oriented data modeling and the Semantic Web.

The Schema Salad library is Python 3.8+ only.

Installation

pip3 install schema_salad

If you intend to use the schema-salad-tool –codegen=python feature, please include the [pycodegen] extra:

pip3 install schema_salad[pycodegen]

To install from source:

git clone https://github.com/common-workflow-language/schema_salad
cd schema_salad
pip3 install .
# or pip3 install .[pycodegen] if needed

Commands

Schema salad can be used as a command line tool or imported as a Python module:

$ schema-salad-tool
usage: schema-salad-tool [-h] [--rdf-serializer RDF_SERIALIZER] [--skip-schemas]
                      [--strict-foreign-properties] [--print-jsonld-context]
                      [--print-rdfs] [--print-avro] [--print-rdf] [--print-pre]
                      [--print-index] [--print-metadata] [--print-inheritance-dot]
                      [--print-fieldrefs-dot] [--codegen language] [--codegen-target CODEGEN_TARGET]
                      [--codegen-examples directory] [--codegen-package dotted.package]
                      [--codegen-copyright copyright_string] [--print-oneline]
                      [--print-doc] [--strict | --non-strict]
                      [--verbose | --quiet | --debug] [--only ONLY] [--redirect REDIRECT]
                      [--brand BRAND] [--brandlink BRANDLINK] [--brandstyle BRANDSTYLE]
                      [--brandinverse] [--primtype PRIMTYPE] [--version]
                      [schema] [document]

$ python
>>> import schema_salad

Validate a schema:

$ schema-salad-tool myschema.yml

Validate a document using a schema:

$ schema-salad-tool myschema.yml mydocument.yml

Generate HTML documentation:

$ schema-salad-tool --print-doc myschema.yml > myschema.html
$ # or
$ schema-salad-doc myschema.yml > myschema.html

Get JSON-LD context:

$ schema-salad-tool --print-jsonld-context myschema.yml mydocument.yml

Convert a document to JSON-LD:

$ schema-salad-tool --print-pre myschema.yml mydocument.yml > mydocument.jsonld

Generate Python classes for loading/generating documents described by the schema (Requires the [pycodegen] extra):

$ schema-salad-tool --codegen=python myschema.yml > myschema.py

Display inheritance relationship between classes as a graphviz ‘dot’ file and render as SVG:

$ schema-salad-tool --print-inheritance-dot myschema.yml | dot -Tsvg > myschema.svg

Codegen Examples

The examples in the tables below are helpful to see how to use the output of schema-salad-tool –codegen in different languages for loading and/or creating/editing/saving objects.

First set of examples is using the CWL v1.2 schema:

Language

Repository

Serialization Example | Deserialization Example

Python

https://github.com/common-workflow-language/cwl-utils/

create_cwl_from_objects.py

load_document()

Java

https://github.com/common-workflow-language/cwljava/

(Not yet implemented)

PackedWorkflowClassTest.java

TypeScript

https://github.com/common-workflow-lab/cwl-ts-auto

Creating, editing, and saving CWL docs with TypeScript

Loading CWL documents with TypeScript

.Net

https://github.com/common-workflow-lab/CWLDotNet

Creating, editing, and saving CWL docs with .Net

Loading CWL documents with .Net

C++

https://github.com/common-workflow-lab/cwl-cpp-auto

cwl_output_example.cpp

cwl_input_example.cpp

D

https://github.com/common-workflow-lab/cwl-d-auto

How to use

How to use

Second set of examples is for the Galaxy Workflow Format 2 schema:

Language

Path

Python

https://github.com/galaxyproject/gxformat2/blob/master/gxformat2/schema/v19_09.py

Java

https://github.com/galaxyproject/gxformat2/tree/master/java

TypeScript

https://github.com/galaxyproject/gxformat2/tree/master/typescript

Quick Start

Let’s say you have a ‘basket’ record that can contain items measured either by weight or by count. Here’s an example:

basket:
  - product: bananas
    price: 0.39
    per: pound
    weight: 1
  - product: cucumbers
    price: 0.79
    per: item
    count: 3

We want to validate that all the expected fields are present, the measurement is known, and that “count” cannot be a fractional value. Here is an example schema to do that:

- name: Product
  doc: |
    The base type for a product.  This is an abstract type, so it
    can't be used directly, but can be used to define other types.
  type: record
  abstract: true
  fields:
    product: string
    price: float

- name: ByWeight
  doc: |
    A product, sold by weight.  Products may be sold by pound or by
    kilogram.  Weights may be fractional.
  type: record
  extends: Product
  fields:
    per:
      type:
        type: enum
        symbols:
          - pound
          - kilogram
      jsonldPredicate: '#per'
    weight: float

- name: ByCount
  doc: |
    A product, sold by count.  The count must be a integer value.
  type: record
  extends: Product
  fields:
    per:
      type:
        type: enum
        symbols:
          - item
      jsonldPredicate: '#per'
    count: int

- name: Basket
  doc: |
    A basket of products.  The 'documentRoot' field indicates it is a
    valid starting point for a document.  The 'basket' field will
    validate subtypes of 'Product' (ByWeight and ByCount).
  type: record
  documentRoot: true
  fields:
    basket:
      type:
        type: array
        items: Product

You can check the schema and document in schema_salad/tests/basket_schema.yml and schema_salad/tests/basket.yml:

$ schema-salad-tool basket_schema.yml basket.yml
Document `basket.yml` is valid

Documentation

See the specification and the metaschema (salad schema for itself). For an example application of Schema Salad see the Common Workflow Language.

Rationale

The JSON data model is an popular way to represent structured data. It is attractive because of it’s relative simplicity and is a natural fit with the standard types of many programming languages. However, this simplicity comes at the cost that basic JSON lacks expressive features useful for working with complex data structures and document formats, such as schemas, object references, and namespaces.

JSON-LD is a W3C standard providing a way to describe how to interpret a JSON document as Linked Data by means of a “context”. JSON-LD provides a powerful solution for representing object references and namespaces in JSON based on standard web URIs, but is not itself a schema language. Without a schema providing a well defined structure, it is difficult to process an arbitrary JSON-LD document as idiomatic JSON because there are many ways to express the same data that are logically equivalent but structurally distinct.

Several schema languages exist for describing and validating JSON data, such as JSON Schema and Apache Avro data serialization system, however none understand linked data. As a result, to fully take advantage of JSON-LD to build the next generation of linked data applications, one must maintain separate JSON schema, JSON-LD context, RDF schema, and human documentation, despite significant overlap of content and obvious need for these documents to stay synchronized.

Schema Salad is designed to address this gap. It provides a schema language and processing rules for describing structured JSON content permitting URI resolution and strict document validation. The schema language supports linked data through annotations that describe the linked data interpretation of the content, enables generation of JSON-LD context and RDF schema, and production of RDF triples by applying the JSON-LD context. The schema language also provides for robust support of inline documentation.

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for mcrusoe from gravatar.com mcrusoe Avatar for mvdbeek from gravatar.com mvdbeek Avatar for rwn from gravatar.com rwn Avatar for tetron from gravatar.com tetron

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Release history Release notifications | RSS feed

8.7.20241021092521

8.7.20241010092723

8.7.20240905150001

This version

8.7.20240820070935

8.7.20240818162459

8.7.20240718183047

8.6.20240710082410

8.5.20240503091721

8.5.20240410123758

8.5.20240311110950

8.5.20240102191335

8.5.20231201181309

8.4.20231117150958

8.4.20231113094720

8.4.20231024070348

8.4.20230927144413

8.4.20230808163024

8.4.20230606143604

8.4.20230601112322

8.4.20230511084951

8.4.20230426093816

8.4.20230213094415

8.4.20230201194352

8.4.20230128170514

8.4.20230127112827

8.3.20230109181936

8.3.20221209165047

8.3.20221115203138

8.3.20221028160159

8.3.20221016151607

8.3.20220916115321

8.3.20220913105718

8.3.20220909144501

8.3.20220831150015

8.3.20220825133457

8.3.20220825123345

8.3.20220825114525

8.3.20220801194920

8.3.20220721194857 yanked

Reason this release was yanked:

regression https://github.com/common-workflow-language/schema_salad/issues/575

8.3.20220717184004

8.3.20220717021618

8.3.20220626185350

8.3.20220626094427

8.3.20220525163636

8.3.20220518163624

8.3 yanked

Reason this release was yanked:

invalid release tag

8.2.20220204150214

8.2.20220103095339

8.2.20211222191353

8.2.20211116214159

8.2.20211104054942

8.2.20211103155537

8.2.20211029004247

8.2.20211020114435

8.2.20211015115235

8.2.20211014150008

8.2.20211014142459

8.2.20210918131710

8.2.20210914115719

8.2.20210902094147

8.1.20210721123742

8.1.20210716111910

8.1.20210627200047

8.0.20210624154013

8.0.20210624101613

8.0.20210624094941

7.1.20210611090601

7.1.20210518142926

7.1.20210316164414

7.1.20210309094900

7.0.20210124093443

7.0.20201119201711

7.0.20200811075006

7.0.20200612160654

6.0.20200601095207

6.0.20200523200656

5.0.20200416112825

5.0.20200302192450

5.0.20200220195218

5.0.20200126033820

5.0.20200122085940

5.0.20200105154004

4.5.20191229160203

4.5.20191204060541

4.5.20191203162929

4.5.20191023134839

4.5.20191017101802

4.5.20190906201758

4.5.20190815125611

4.5.20190729164411

4.5.20190621200723

4.4.20190620173027

4.3.20190604170443

4.3.20190603183756

4.2.20190417121603

4.1.20190305210046

4.1.20190227145202

4.1.20190221213919

4.0.20190130225346

3.1.20190125161400

3.0.20181206233650

3.0.20181129082112

2.7.20181126142424

2.7.20181116024232

2.7.20181017120439

2.7.20181012180226

2.7.20180905124720

2.7.20180809223002

2.7.20180731120629

2.7.20180724081820

2.7.20180719125426

2.7.20180717111941

2.7.20180611133406

2.7.20180514132321

2.7.20180514111532

2.7.20180501211602

2.7.20180322174040

2.7.20180302220026

2.6.20180214144209

2.6.20171201034858

2.6.20171116190026

2.6.20171102064032

2.6.20171101113912

2.6.20171031091636

2.6.20170927145003

2.6.20170806163416

2.6.20170712194300

2.6.20170630075932

2.6.20170628130600

2.5.20170623152720

2.5.20170428142041

2.5.20170328195758

2.5.20170327140858

2.5.1.20170628121527

2.4.20170308171942

2.3.20170302225134

2.3.20170302150835

2.2.20170222151604

2.2.20170216125639

2.2.20170208112505

2.2.20170126160727

2.2.20170119151016

2.2.20170111180227

2.1.20161227191302

2.1.20161223160831

2.1.20161221160224

2.1.20161216210732

2.0.20161122040508

1.21.20161215163938

1.21.20161206204028

1.21.20161206181442

1.21.20161202201331

1.20.20161122192122

1.18.20161005190847

1.18.20160930145650

1.18.20160907135919

1.17.20160820171034

1.17.20160820165726

1.16.20160819133325

1.16.20160810195039

1.15.20160810152809

1.14.20160708181155

1.14.20160707164023

1.13.20160702001514

1.13.20160630144631

1.12.20160610104117

1.12.20160610092802

1.11.20160524152859

1.11.20160506154702

1.10.20160505185111

1.10.20160505184637

1.9.20160421133216

1.7.20160316203940

1.7.20160316150109

1.7.20160316145902

1.7.20160315232142

1.7

1.6.20160202222448

1.5.20160126213649

1.5.20160126190004

1.5.20160126150511

1.4.20160108200836

1.4.20160108152457

1.3.20160108151816

1.2.20151210153655

1.1.1

1.1.0

1.0.6

1.0.5

1.0.4

1.0.3

1.0.2

1.0.1

1.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

schema_salad-8.7.20240820070935.tar.gz (592.6 kB view hashes)

Uploaded Source

Built Distributions

schema_salad-8.7.20240820070935-py3-none-any.whl (635.3 kB view hashes)

Uploaded Python 3

schema_salad-8.7.20240820070935-cp312-cp312-musllinux_1_2_x86_64.whl (1.5 MB view hashes)

Uploaded CPython 3.12 musllinux: musl 1.2+ x86-64

schema_salad-8.7.20240820070935-cp312-cp312-musllinux_1_2_aarch64.whl (1.4 MB view hashes)

Uploaded CPython 3.12 musllinux: musl 1.2+ ARM64

schema_salad-8.7.20240820070935-cp312-cp312-musllinux_1_1_x86_64.whl (1.5 MB view hashes)

Uploaded CPython 3.12 musllinux: musl 1.1+ x86-64

schema_salad-8.7.20240820070935-cp312-cp312-musllinux_1_1_aarch64.whl (1.5 MB view hashes)

Uploaded CPython 3.12 musllinux: musl 1.1+ ARM64

schema_salad-8.7.20240820070935-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl (1.5 MB view hashes)

Uploaded CPython 3.12 manylinux: glibc 2.17+ x86-64 manylinux: glibc 2.28+ x86-64

schema_salad-8.7.20240820070935-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_28_aarch64.whl (1.4 MB view hashes)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ARM64 manylinux: glibc 2.28+ ARM64

schema_salad-8.7.20240820070935-cp312-cp312-macosx_11_0_arm64.whl (1.3 MB view hashes)

Uploaded CPython 3.12 macOS 11.0+ ARM64

schema_salad-8.7.20240820070935-cp312-cp312-macosx_10_9_x86_64.whl (1.4 MB view hashes)

Uploaded CPython 3.12 macOS 10.9+ x86-64

schema_salad-8.7.20240820070935-cp311-cp311-musllinux_1_2_x86_64.whl (1.5 MB view hashes)

Uploaded CPython 3.11 musllinux: musl 1.2+ x86-64

schema_salad-8.7.20240820070935-cp311-cp311-musllinux_1_2_aarch64.whl (1.4 MB view hashes)

Uploaded CPython 3.11 musllinux: musl 1.2+ ARM64

schema_salad-8.7.20240820070935-cp311-cp311-musllinux_1_1_x86_64.whl (1.5 MB view hashes)

Uploaded CPython 3.11 musllinux: musl 1.1+ x86-64

schema_salad-8.7.20240820070935-cp311-cp311-musllinux_1_1_aarch64.whl (1.5 MB view hashes)

Uploaded CPython 3.11 musllinux: musl 1.1+ ARM64

schema_salad-8.7.20240820070935-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl (1.5 MB view hashes)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64 manylinux: glibc 2.28+ x86-64

schema_salad-8.7.20240820070935-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_28_aarch64.whl (1.4 MB view hashes)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ARM64 manylinux: glibc 2.28+ ARM64

schema_salad-8.7.20240820070935-cp311-cp311-macosx_11_0_arm64.whl (1.3 MB view hashes)

Uploaded CPython 3.11 macOS 11.0+ ARM64

schema_salad-8.7.20240820070935-cp311-cp311-macosx_10_9_x86_64.whl (1.4 MB view hashes)

Uploaded CPython 3.11 macOS 10.9+ x86-64

schema_salad-8.7.20240820070935-cp310-cp310-musllinux_1_2_x86_64.whl (1.5 MB view hashes)

Uploaded CPython 3.10 musllinux: musl 1.2+ x86-64

schema_salad-8.7.20240820070935-cp310-cp310-musllinux_1_2_aarch64.whl (1.5 MB view hashes)

Uploaded CPython 3.10 musllinux: musl 1.2+ ARM64

schema_salad-8.7.20240820070935-cp310-cp310-musllinux_1_1_x86_64.whl (1.5 MB view hashes)

Uploaded CPython 3.10 musllinux: musl 1.1+ x86-64

schema_salad-8.7.20240820070935-cp310-cp310-musllinux_1_1_aarch64.whl (1.5 MB view hashes)

Uploaded CPython 3.10 musllinux: musl 1.1+ ARM64

schema_salad-8.7.20240820070935-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl (1.5 MB view hashes)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64 manylinux: glibc 2.28+ x86-64

schema_salad-8.7.20240820070935-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_28_aarch64.whl (1.4 MB view hashes)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ARM64 manylinux: glibc 2.28+ ARM64

schema_salad-8.7.20240820070935-cp310-cp310-macosx_11_0_arm64.whl (1.3 MB view hashes)

Uploaded CPython 3.10 macOS 11.0+ ARM64

schema_salad-8.7.20240820070935-cp310-cp310-macosx_10_9_x86_64.whl (1.4 MB view hashes)

Uploaded CPython 3.10 macOS 10.9+ x86-64

schema_salad-8.7.20240820070935-cp39-cp39-musllinux_1_2_x86_64.whl (1.5 MB view hashes)

Uploaded CPython 3.9 musllinux: musl 1.2+ x86-64

schema_salad-8.7.20240820070935-cp39-cp39-musllinux_1_2_aarch64.whl (1.5 MB view hashes)

Uploaded CPython 3.9 musllinux: musl 1.2+ ARM64

schema_salad-8.7.20240820070935-cp39-cp39-musllinux_1_1_x86_64.whl (1.5 MB view hashes)

Uploaded CPython 3.9 musllinux: musl 1.1+ x86-64

schema_salad-8.7.20240820070935-cp39-cp39-musllinux_1_1_aarch64.whl (1.5 MB view hashes)

Uploaded CPython 3.9 musllinux: musl 1.1+ ARM64

schema_salad-8.7.20240820070935-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl (1.5 MB view hashes)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64 manylinux: glibc 2.28+ x86-64

schema_salad-8.7.20240820070935-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_28_aarch64.whl (1.4 MB view hashes)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ARM64 manylinux: glibc 2.28+ ARM64

schema_salad-8.7.20240820070935-cp39-cp39-macosx_11_0_arm64.whl (1.3 MB view hashes)

Uploaded CPython 3.9 macOS 11.0+ ARM64

schema_salad-8.7.20240820070935-cp39-cp39-macosx_10_9_x86_64.whl (1.4 MB view hashes)

Uploaded CPython 3.9 macOS 10.9+ x86-64

schema_salad-8.7.20240820070935-cp38-cp38-musllinux_1_2_x86_64.whl (1.5 MB view hashes)

Uploaded CPython 3.8 musllinux: musl 1.2+ x86-64

schema_salad-8.7.20240820070935-cp38-cp38-musllinux_1_2_aarch64.whl (1.4 MB view hashes)

Uploaded CPython 3.8 musllinux: musl 1.2+ ARM64

schema_salad-8.7.20240820070935-cp38-cp38-musllinux_1_1_x86_64.whl (1.5 MB view hashes)

Uploaded CPython 3.8 musllinux: musl 1.1+ x86-64

schema_salad-8.7.20240820070935-cp38-cp38-musllinux_1_1_aarch64.whl (1.5 MB view hashes)

Uploaded CPython 3.8 musllinux: musl 1.1+ ARM64

schema_salad-8.7.20240820070935-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl (1.5 MB view hashes)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64 manylinux: glibc 2.28+ x86-64

schema_salad-8.7.20240820070935-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_28_aarch64.whl (1.4 MB view hashes)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ARM64 manylinux: glibc 2.28+ ARM64

schema_salad-8.7.20240820070935-cp38-cp38-macosx_11_0_arm64.whl (1.3 MB view hashes)

Uploaded CPython 3.8 macOS 11.0+ ARM64

schema_salad-8.7.20240820070935-cp38-cp38-macosx_10_9_x86_64.whl (1.4 MB view hashes)

Uploaded CPython 3.8 macOS 10.9+ x86-64

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page