pywcmp

WMO Core Metadata Profile Test Suite

pywcmp provides validation and quality assessment capabilities for the WMO WIS Core Metadata Profile (WCMP).

• validation against WCMP 1.3, specifically Part 2, Section 2, implementing an executable test suite against the ATS
• validation against WCMP 2 (draft), specifically Annex A: Conformance Class Abstract Test Suite, implementing an executable test suite against the ATS
• quality assessement against the WCMP Key Performance Indicators

Installation

pip

Install latest stable version from PyPI.

pip3 install pywcmp

From source

Install latest development version.

python3 -m venv pywcmp
cd pywcmp
. bin/activate
git clone https://github.com/wmo-im/pywcmp.git
cd pywcmp
pip3 install -r requirements.txt
python3 setup.py install

Running

From command line:

# fetch version
pywcmp --version

# sync supporting configuration bundle (schemas, topics, etc.)
pywcmp bundle sync

# abstract test suite

# validate WCMP 1.3 metadata against abstract test suite (file on disk)
pywcmp ets validate /path/to/file.xml

# validate WCMP 1.3 metadata against abstract test suite (URL)
pywcmp ets validate https://example.org/path/to/file.xml

# validate WCMP 2 metadata against abstract test suite (URL)
pywcmp ets validate https://example.org/path/to/file.json

# validate WCMP 2 metadata against abstract test suite (URL), but turn JSON Schema validation off
pywcmp ets validate https://example.org/path/to/file.json --no-fail-on-schema-validation

# adjust debugging messages (CRITICAL, ERROR, WARNING, INFO, DEBUG) to stdout
pywcmp ets validate https://example.org/path/to/file.xml --verbosity DEBUG

# write results to logfile
pywcmp ets validate https://example.org/path/to/file.json --verbosity DEBUG --logfile /tmp/foo.txt

# key performance indicators

# all key performance indicators at once # note: running KPIs automatically runs the ets
pywcmp kpi validate https://example.org/path/to/file.xml --verbosity DEBUG

# all key performance indicators at once, in summary
pywcmp kpi validate https://example.org/path/to/file.xml --verbosity DEBUG --summary

# all key performance indicators at once, with scoring rubric grouping
pywcmp kpi validate https://example.org/path/to/file.xml --verbosity DEBUG --group

# selected key performance indicator
pywcmp kpi validate --kpi 4 /path/to/file.xml -v INFO

# WIS2 topic hierarchies

# validate a WIS2 topic hierarchy
pywcmp topics validate origin.a.wis2.can

# validate a partial WIS2 topic hierarchy
pywcmp topics validate wis2.can --fuzzy

# list children of a given WIS2 topic hierarchy level
pywcmp topics list wis2.a

Using the API

>>> # test a file on disk
>>> from lxml import etree
>>> from pywcmp.ets import ets
>>> exml = etree.parse('/path/to/file.xml')
>>> # test ETS
>>> ts = ets.WMOCoreMetadataProfileTestSuite13(exml)
>>> ts.run_tests()  # raises ValueError error stack on exception
>>> # test a URL
>>> from urllib2 import urlopen
>>> from StringIO import StringIO
>>> content = StringIO(urlopen('https://....').read())
>>> exml = etree.parse(content)
>>> ts = ets.WMOCoreMetadataProfileTestSuite13(exml)
>>> ts.run_tests()  # raises ValueError error stack on exception
>>> # handle pywcmp.errors.TestSuiteError
>>> # pywcmp.errors.TestSuiteError.errors is a list of errors
>>> try:
...    ts.run_tests()
... except pywcmp.errors.TestSuiteError as err:
...    print('\n'.join(err.errors))
>>> ...
>>> # test KPI
>>> from pywcmp.kpi import WMOCoreMetadataProfileKeyPerformanceIndicators
>>> kpis = WMOCoreMetadataProfileKeyPerformanceIndicators(exml)
>>> results = kpis.evaluate()
>>> results['summary']
>>> # scoring rubric
>>> grouped = group_kpi_results(results)

Development

python3 -m venv pywcmp
cd pywcmp
source bin/activate
git clone https://github.com/wmo-im/pywcmp.git
pip3 install -r requirements.txt
pip3 install -r requirements-dev.txt
python3 setup.py install

Running tests

# via setuptools
python3 setup.py test
# manually
python3 tests/run_tests.py

Releasing

# create release (x.y.z is the release version)
vi pywcmp/__init__.py  # update __version__
git commit -am 'update release version x.y.z'
git push origin main
git tag -a x.y.z -m 'tagging release version x.y.z'
git push --tags

# upload to PyPI
rm -fr build dist *.egg-info
python setup.py sdist bdist_wheel --universal
twine upload dist/*

# publish release on GitHub (https://github.com/wmo-im/pywcmp/releases/new)

# bump version back to dev
vi pywcmp/__init__.py  # update __version__
git commit -am 'back to dev'
git push origin main

Code Conventions

PEP8

Issues

Issues are managed at https://github.com/wmo-im/pywcmp/issues

Contact

• Tom Kralidis
• Ján Osuský