A suite of command-line tools for working with OCDS data
Project description
A suite of command-line tools for working with OCDS data.
pip install ocdskit ocdskit --help
To install from source:
pip install --upgrade .
To see all commands available, run:
ocdskit --help
Most ocdskit tools accept JSON data from standard input. To report on a remote file:
curl <url> | ocdskit <command>
To report on a local file:
cat <path> | ocdskit <command>
For exploring JSON data, consider using jq.
Commands
Optional arguments for all commands are:
-h, --help show the help message and exit
--encoding ENCODING the file encoding
--pretty pretty print output
combine-record-packages
Reads record packages from standard input, collects packages and records, and prints one record package.
cat tests/fixtures/record-package_*.json | ocdskit combine-record-packages > out.json
combine-release-packages
Reads release packages from standard input, collects releases, and prints one release package.
cat tests/fixtures/release-package_*.json | ocdskit combine-release-packages > out.json
compile
Reads release packages from standard input, merges the releases by OCID, and prints the compiled releases.
cat tests/fixtures/realdata/release-package-1.json | ocdskit compile > out.json
indent
Indents JSON files by modifying the given files in-place.
Optional arguments:
-r, --recursive recursively indent JSON files
--indent INDENT indent level
ocdskit indent --recursive file1 path/to/directory file2
mapping-sheet
Generates a spreadsheet with all field paths from an OCDS schema.
cat path/to/release-schema.json | ocdskit mapping-sheet > mapping-sheet.csv
schema-report
Reports details of a JSON Schema (open and closed codelists).
cat path/to/release-schema.json | ocdskit schema-report
schema-strict
For any required field, adds “minItems” if an array, “minProperties” if an object and “minLength” if a string and “enum”, “format” and “pattern” are not set.
cat path/to/release-schema.json | ocdskit schema-strict > out.json
set-closed-codelist-enums
Sets the enum in a JSON Schema to match the codes in the CSV files of closed codelists.
ocdskit set-closed-codelist-enums path/to/standard path/to/extension1 path/to/extension2
tabulate
Load packages into a database.
Optional arguments:
--drop drop all tables before loading
--schema SCHEMA the release-schema.json to use
cat release_package.json | ocdskit tabulate sqlite:///data.db
For the format of database_url, see the SQLAlchemy documentation.
validate
Reads JSON data from standard input, validates it against the schema, and prints errors.
Optional arguments:
--schema SCHEMA the schema to validate against
--check-urls check the HTTP status code if “format”: “uri”
--timeout TIMEOUT timeout (seconds) to GET a URL
cat tests/fixtures/* | ocdskit validate
Examples
Example 1
Download a list of release packages:
curl http://www.contratosabiertos.cdmx.gob.mx/api/contratos/array > release_packages.json
Transform it to a stream of release packages, and validate each:
jq -crM '.[]' release_packages.json | ocdskit validate --schema http://standard.open-contracting.org/schema/1__0__3/release-package-schema.json
Or, validate each with a local schema file:
jq -crM '.[]' release_packages.json | ocdskit validate --schema file:///path/to/release-package-schema.json
Transform it to a stream of compiled releases:
jq -crM '.[]' release_packages.json | ocdskit compile > compiled_releases.json
Find a compiled release with a given ocid (replace the …):
jq 'select(.ocid == "OCDS-87SD3T-AD-SF-DRM-063-2015")' compiled_releases.json
Measure indicators across release packages:
cat release_packages.json | ocdskit --encoding iso-8859-1 measure --currency MXN
Example 2
Download a list of record packages:
curl https://drive.google.com/uc?export=download&id=1HzVMdv9bryEw6pg80RwmJd3Le31SY1TI > record_packages.json
Combine it into a single record package:
jq -crM '.[]' record_packages.json | ocdskit combine-record-packages > record_package.json
If the file is too large for the OCDS Validator, you can break it into parts. First, transform the list into a stream:
jq -crM '.[]' record_packages.json > stream.json
Combine the first 10,000 items from the stream into a single record package:
head -n 10000 stream.json | ocdskit combine-record-packages > record_package-1.json
Then, combine the next 10,000 items from the stream into a single record package:
tail -n +10001 stream.json | head -n 10000 | ocdskit combine-record-packages > record_package-2.json
And so on:
tail -n +20001 stream.json | head -n 10000 | ocdskit combine-record-packages > record_package-3.json
Copyright (c) 2017 Open Contracting Partnership, released under the BSD license
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
File details
Details for the file ocdskit-0.0.2.tar.gz.
File metadata
- Download URL: ocdskit-0.0.2.tar.gz
- Upload date:
- Size: 17.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | ed8920f49b218cf7bef7b2393b3f30b2b557f5ab123acbcc6ebc02fff044e723 |
|
| MD5 | f5b1ad9fcb3fa636673f0662e3e0ffbc |
|
| BLAKE2b-256 | e945c6dae13138633444539e61a09a4e6ebae6e0d4143bf1952f66a2106338cb |
