pyramid_oereb, extension for pyramid web frame work to provide a basic server part for the oereb project
Project description
pyramid_oereb is an open-source implementation of the server side part for the swiss “Cadastre of Public-law Restrictions on landownership” (PLR-cadastre).
It is written in Python and designed as a plugin for the Pyramid Web Framework. This allows pyramid_oereb to be included in any Pyramid web application.
Please refer to the documentation for detailed information and instructions for installation and configuration.
If you are interested in contributing or extending the project, take a look at the contribution page.
Local testing for versions V1
Run dev server
Following packages are required: python3-dev libgeos-dev python3-venv postgresql-client libpq-dev
Clone project:
git clone git@github.com:openoereb/pyramid_oereb.git
cd pyramid_oerebThe information for the connection to the main database, the print service and the statistics functionality database has to be provided in a .env file (not committed, copy sample.env and edit it if necessary):
cp sample.env .envRun:
make serveJSON reduced extract is accessible at: http://localhost:6543/oereb/extract/reduced/json/CH113928077734.
When running in to issues installing libpq-dev :
The following packages have unmet dependencies:
libpq-dev : Depends: libpq5 (= 10.12-0ubuntu0.18.04.1) but 12.3-1.pgdg18.04+1 is to be installedTry to install a specific version (adapt the version to your requirement):
sudo apt install libpq5=10.12-0ubuntu0.18.04.1Run tests
To check your code, run make checks. It will run style checks and tests. It’s also possible to run sub-target independently:
make checks-style to run only style-related checks (linting).
make tests to run tests. Python 3.7 is used by default. But you can run tests with another version by running: PYTHON_TEST_VERSION=python3.x make tests. You can also set this variable in the Makefile.
PYTEST_OPTS="-k test_name" make test to run a specific test
After running tests, the coverage is available in the folder coverage_report.
Use Oereblex data model
To test the application with the Oereblex data model, adapt the configuration files pyramid_oereb_standard.yml.mako and docker/config.yml.tmpl. See this example with the theme forest_perimeters:
source:
# Standard data model
#class: pyramid_oereb.standard.sources.plr.DatabaseSource
# Oereblex data model
class: pyramid_oereb.contrib.sources.plr_oereblex.DatabaseOEREBlexSource
params:
db_connection: *main_db_connection
# Standard data model
#models: pyramid_oereb.standard.models.forest_perimeters
# Oereblex data model
models: pyramid_oereb.contrib.models.oereblex.forest_perimetersThe sample data is in sample_data/oereblex.
Run the application:
USE_OEREBLEX=TRUE make serveJSON reduced extract is accessible at: http://localhost:6544/oereb/extract/reduced/json/CH113928077734. This will do a call to the Oereblex service defined in the configuration file, and the success of the call will depend on the external service being available, and the geolink id being used in the sample data still existing on the external system.
It is possible to run this instance in parallel to the instance which uses the standard database. For this, one should create a second clone of the project.
If testing make serve with another theme than forest_perimeters, changes will be necessary in the directory sample_data/oereblex/: first remove the symbolic link corresponding to this theme, then create a directory and add JSON data files into it. In comparison to the data from the standard model, a new attribute geolink is required in public_law_restriction.json, which should correspond to an existing geolink in the Oereblex server defined in the configuration (see sample_data/oereblex/forest_perimeters for example files).
Dev environment (V2)
pyramid_oereb can be run with docker-compose or directly on the host. The application requires a running database.
The Docker composition consists of the service oereb-server (the container in which the application is to be started) and the service oereb-db (which hosts the database). To run pyramid_oereb with docker-compose, see section “General workflow (in Docker)”.
To run the server directly on the host, you need to be using a Linux system with all the dependencies installed. In this case, you should use an already existing database. For details see section “General worfklow (local shell)”.
Database connection
For the database connection, the following environment variables must be set (if not using the default parameters):
# the db-server username
PGUSER
# the db-server password
PGPASSWORD
# the db-server host
PGHOST
# the database in the db-server
PGDATABASE
# the port on which the db-server is listening
PGPORTIf these are not provided, the default values found in the Makefile will be used.
NB: if these environment variables are set in the host environment, they will also be used in the docker-compose composition.
General workflow (in Docker)
Run the composition with docker-compose up -d
You can check whether the containers started properly with docker-compose ps
Connect to the server container with docker-compose exec oereb-server zsh
Start the server in development mode with make serve-dev
The sample data extract should be available at http://localhost:6543/oereb/extract/json?EGRID=CH113928077734
BONUS: If you use an IDE like VSCode you can attach it to the running container to have convenient features like autocomplete or code inspection
NB: Alternatively, start the server from your local shell with docker-compose exec oereb-server make serve-dev
Clean up after work
It is recommended to stop your composition when you stop working:
docker-compose downUpdate Dockerfile
If you need to change something inside the Dockerfile you need to rebuild the oereb-server image. So after your change, stop the docker composition and rebuild it:
docker-compose down
docker-compose buildGeneral workflow (local shell)
These instructions are sufficient only if you have all dependencies locally available (python3-dev libgeos-dev python3-venv postgresql-client libpq-dev etc.) and in the right versions. Otherwise this might lead to strange behaviors.
In a local shell in the project path, start the server in development mode with make serve-dev
The sample data extract should be available at http://localhost:6543/oereb/extract/json?EGRID=CH113928077734
Useful make targets
Run the make targets found in the Makefile either in the oereb-server container (if using docker-compose) or in your local shell (if running the server locally). Some useful targets:
make serve-dev to run the application
make test to run the application tests
make clean to empty the database
make clean-all to empty the database, uninstall the application and the virtual env and clear the rendered configuration files
If necessary the application is re-installed and the database is filled when running make serve-dev again.
Using MapFish-Print
To be able to test the OEREB static extract (pdf), you need to run pyramid_oereb with docker-compose and to have a running instance of pyramid_oereb_mfp. The Docker network print-network is also required and can be created with:
docker network create print-networkThe sample static extract should then be available at http://localhost:6543/oereb/extract/pdf?EGRID=CH113928077734
Changelog
2.0.0.b6
Improvements in error logging
2.0.0.b5
Fixes in Oereblex integration
2.0.0.b4
Fix JSON extract
2.0.0.b3
New federal data import tool and bug-fixes V2 (Status: beta)
2.0.0.b2
First fully functional implementation of new Oereb specification as per 28.10.11 (Status: beta)
2.0.0.b1
Implementation of the new Oereb specification 2021 (Status: beta)
1.9.2
Oereblex: improve testing functionality for Oereblex (#1197)
Various library updates
1.9.1
Oereblex: support new Oereblex API version 1.2.1
Various library updates
1.9.0
Oereblex: add configuration to pass URL parameters to the oereblex call (#1117)
Various library updates
Improve handling of empty geometries, in preparation of additional library updates (#1107)
Print using MapFish Print: the inclusion of the cantonal logo is now configurable (#1139)
1.8.1
Update of external libraries such as numpy, SQLAlchemy, lxml, and more.
oereblex support: avoid extract failure upon missing enactment_date in oereblex (#1093)
Improve support of Python 3.7 in template Makefile and sample data loading (#1104, #1106)
1.8.0
Fix bug affecting concurrent requests (#1068)
Enhance federal data import script to make it more usable with Docker (#1078)
For full extracts, add configuration parameter to make additional sld usage optional (#1077)
1.7.6
Improve federal data import script (#1057)
Last maintenance release with verified python2 compatibility
1.7.5
Update of all libraries used by pyramid_oereb that still work with python2
1.7.4
Federal data import script: add SLD_VERSION for legend_at_web (#1022)
Oereblex integration: add optional configuration ‘validation’ (#1034)
Restrict the version of the Shapely library used to 1.6 (#1037)
1.7.3
Fix import of federal data for cases including both coordinate reference systems (#1011)
Oereblex: support geolink schema version 1.2.0 (#1010)
Print: make geometry inclusion optional (performance improvement for MapFish Print) (#1006)
1.7.2
Test release only; not an official release.
1.7.1
Print: fix nr_of_points computation (#1002)
1.7.0
Oereblex: improve performance (implement per topic store) (#993)
Add statistics functionality (#987)
Print: fix table of contents page numbering (#983)
1.6.0
Improve multilingual support (#915, #918, #943, #950)
Ensure XML schema compliance (#914, #926)
Improve extract speed (#965)
Additional options for sorting and grouping (#925, #931, #948, #979)
Additional options for xml2pdf integration (#905, #938)
Add PDF archive functionality (#982)
Make WMS usage in print more flexible (#986)
Bug fixes and debugging possibilities improvement (#910, #909, #897, #894, #916, #919, #870, #908, #932, #955, #958, #963, #970)
1.5.2
Provide multilingual OEREB logo (#915)
Add file extension in logo and symbol URLs (#917)
1.5.1
Ensure XML Schema compliance (#872, #891)
Fix polygon GML rendering (#830)
Integration of XML2PDF service (#631, #883, #887)
1.5.0
Fixed a number formatting problem in the legend list (Mapfish Print, GitHub issue 824, pull request 826)
Fixed an encoding issue for PLR records (GitHub pull request 828)
Allow configuration of custom parameters for WMS calls in Mapfish Print (GitHub pull request 831)
Section ‘Certification’ is now optional, can be configured in the Mapfish Print config (GitHub pull request 841)
Only prints the PLR section of the PDF if at least one PLR is available (Mapfish Print, GitHub pull request 846)
Various layout fixes in the table of contents of the Mapfish Print PDF (GitHub pull requests 842, 856, 859)
Legends are now sorted by geometry type and value (Mapfish Print, GitHub pull request 851)
Multiple ResponsibleOffices per theme are now rendered correctly (Mapfish Print, GitHub issue 651, pull request 865)
PDF/A conformance enabled by default (Mapfish Print, GitHub pull request 852)
In the XML output, LengthShare and NrOfPoints elements were moved to their correct place (GitHub issue 834, GitHub pull request 854)
Optimized theme sorting (GitHub issue 443, GitHub pull request 858)
Updated Mapfish Print to 3.20.0
Dependency updates, better test coverage
1.4.3
Fixed import script for federal topics (GitHub pull request 821)
Added test for ordering of non-concerned themes (GitHub pull request 817)
Fixed footer with disappearing page numbers with MapFish print 3.18 (GitHub pull request 814)
1.4.2
Downgrade version of pyproj to fix coordinate reprojections (GitHub pull request 810)
Dependency updates
1.4.1
Fixed id types in oereblex models and model template, fixed documentation errors in standard models and model template (GitHub pull request 807)
Fixed warnings in tests (GitHub pull request 803)
Dependency updates (GitHub pull request 805)
1.4.0
Additional multilingual functionality (GitHub issues 704, 705, 779)
1.3.1
Maintenance release (GitHub issues 447, 610, 590, 609, 757, 750, 681, 752, 753, 460, 736, 666, 596, 678, 461, 751)
1.3.0
Import script for federal data
1.2.3
Bug-fix release for 1.2.2 (fix intersection bug, fix pdfreport template)
1.2.2
Further bug-fixes for oereb service versoin 1.0, notably regarding schema conformity and better support for other OS versions.
New configuration parameter type_mapping in real_estate, which allows to configuratively define the texts to be used for realestate types (optional parameter).
1.2.1
Bug-fixes for service version 1.0 (pyramid_oereb 1.2.0), using results from pilot integration of new version at BL.
1.2.0
First implementation of federal extract requirements as per november 2017 (service in version 1.0, extract in version 1.0.1, data model in version 1.0.1).
Update of automated tests to correspond to new requirements.
Static extract implementation update according to Weisung july 1st, 2018.
Oereb lex model creation support and documentation.
Bug fixes (legend entries, multiple view services, scaling in print, document titles in print).
Facilitate customization of document title generation.
1.1.0
Final implementation of federal extract requirements as per november 2016 (extract in version 0.8, data model in version 0.4).
1.0.1
introduce configurable pdf print service (print proxy)
enable proxy configuration for external web api usage
fix standard database
improve python 3 compatibility
minor bug fixing
1.0.0
improved doc
fix bug for doc creation on python 3.6
1.0.0-beta.1
first approach of OEREB server
improved documentation on https://openoereb.github.io/pyramid_oereb/doc/
cleaned and reorganized code
binding to OEREB-LEX and GeoAdmin-Api-Address-Service (http://api.geo.admin.ch/services/sdiservices.html#search) as sources
providing pyconizer as icon generator (https://pypi-hypernode.com/pypi/pyconizer)
proxy binding of geomapfish_print for pdf output as renderer (http://mapfish.github.io/mapfish-print-doc/#/overview)
providing extensive standard configuration for out-of-the-box-usage
general bug fixing
add python 3.x support
1.0.0-alpha.2
proceed with renderer for xml and json
add metadata for embeddable flavour
images accessible via URL
add configurable methods for processing
improve geometry handling
add documentation on https://openoereb.github.io/pyramid_oereb/doc/
several bugfixes
1.0.0-alpha.1
first running approach of server
main web services are available (not all formats are implemented yet)
standard configuration can be used to run server out of the box
see README for more details
0.0.1
initial version
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file pyramid_oereb-2.0.0b6.tar.gz.
File metadata
- Download URL: pyramid_oereb-2.0.0b6.tar.gz
- Upload date:
- Size: 141.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 0fbe13dd80ae71f7f507c13c38bd11ee5fa3640b0500dbcb93ae42fb93680265 |
|
| MD5 | 994d1caeaed5e46e12ee923ef64e6c91 |
|
| BLAKE2b-256 | 7f30f28d454d694eea1175f81680de582a297bab98a77f68b029c2b43aca75f1 |
Provenance
File details
Details for the file pyramid_oereb-2.0.0b6-py2.py3-none-any.whl.
File metadata
- Download URL: pyramid_oereb-2.0.0b6-py2.py3-none-any.whl
- Upload date:
- Size: 1.6 MB
- Tags: Python 2, Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 944504d64b8d5af7a1e1770b0cf495e4650413154b1b50532d330721ebed9da3 |
|
| MD5 | ef76dbbed72570ab6adddb1c769f4b4a |
|
| BLAKE2b-256 | c88f678b15b0d7b9d20644fafda7a7790d1d353948897c9c8c81a094174328a6 |
