Tools to generate tiles from WMS or Mapnik, to S3, Berkeley DB, MBTiles, or local filesystem in WMTS layout using Amazon cloud services.
Project description
TileCloud Chain
The goal of TileCloud Chain is to provide tools around tile generation on a chain like:
Source: WMS, Mapnik.
Optionally using an SQS queue, AWS host, SNS topic.
Destination in WMTS layout, on S3, on Berkeley DB (bsddb), on MBTiles, or on local filesystem.
Features:
- Generate tiles.
- Drop empty tiles.
- Drop tiles outside a geometry or a bbox.
- Use MetaTiles.
- Generate the legend images.
- Generate GetCapabilities.
- Generate OpenLayers example page.
- Obtain the hash of an empty tile.
- In the future, measure tile generation speed.
- Calculate cost and generation time.
- In the future, manage the AWS hosts that generate tiles.
- Delete empty tiles.
- Copy files between caches.
- Be able to use an SQS queue to dispatch the generation.
- Post processing the generated tiles.
- ...
Legacy features:
- bsddb support
- sqlite (mbtiles) support
- mapnik support (should be updated for Python3)
Get it
Create the config file tilegeneration/config.yaml see as example.
Support
Only the latest release is supported and version < 1.11 contains security issues.
From sources
Build it:
git submodule update --recursive
python3 -m venv .build/venv
.build/venv/bin/pip install -r requirements.txt
.build/venv/bin/pip install -e .
.build/venv/bin/pip install -r dev-requirements.txt
Run prospector
.build/venv/bin/prospector
Run the tests
Setup your environment:
touch tilecloud_chain/OpenLayers.js
docker build --tag camptocamp/tilecloud-chain .
docker-compose -p tilecloud up
To run the tests:
docker-compose -p tilecloud exec test python setup.py nosetests --logging-filter=tilecloud,tilecloud_chain --attr '!'nopy3
Documentation
As documentation you can read the https://github.com/camptocamp/tilecloud-chain/blob/master/tilecloud_chain/USAGE.rst.
VSCode
You can add that in your workspace configuration to use the JSON schema:
{
"yaml.schemas": {
"../tilecloud-chain/tilecloud_chain/schema.json": [
"tilecloud-chain/tilecloud_chain/tests/tilegeneration/*.yaml"
]
}
}
Changelog
Release 1.17
- Change the validator and parser => duplicate key generate an error: on/off are no more considered as boolean.
- The argument --layer is no more used when we use the parameter --tiles, we get the information from the tiles file.
- Be able to mutualise the service.
- Add Azure blob storage
- Remove Apache and MapCache
- Remove the
log_formatin thegenerationconfiguration, nor we use the logging configuration from thedevelopment.inifile.
Release 1.16
- Change the config validator who is a little bit more strict.
Release 1.4
- Add optional
metadatasection to the config file. See the scaffolds for example.
Release 0.9
- Correct some error with slash.
- Better error handling.
- Be able to have one error file per layer.
Release 0.8
- Correct some error with slash.
- Add
pre_hash_post_processandpost_process. - Add copy command.
Release 0.7
-
Support of deferent geoms per layers, requires configuration changes, old version:
connection: user=www-data password=www-data dbname=<db> host=localhost sql: <column> AS geom FROM <table>
to new version:
connection: user=www-data password=www-data dbname=<db> host=localhost geoms: - sql: <column> AS geom FROM <table>
More information in the Configure geom/sql chapter.
-
Update from
optparsetoargparse, and some argument refactoring, use--helpto see the new version. -
Add support of Blackbery DB (
bsddb). -
The tile
serveris completely rewrite, now it support all cache,RESTandKVPinterface,GetFeatureInforequest, and it can be used as a pyramid view or as aWSGIserver. More information in the istribute the tiles chapter. -
Add three strategy to bypass the proxy/cache: Use the headers
Cache-Control: no-cache, no-store,Pragma: no-cache(default). Use localhost in the URL and the headerHost: <host_name>(recommended). Add aSALTrandom argument (if the above don't work). More information in the Proxy/cache issue chapter. -
Improve the dimensions usage by adding it ti the WMS requests, And add a
--dimensionsargument ofgenerate_tilesto change the dimensions values. -
Extract generate_cost and generate_amazon from generate_controler.
-
Now we can creates legends, see the Legends chapter.
-
Now the tiles generation display generation statistics at the ends.
-
The EC2 configuration is moved in a separate structure, see README for more information.
Release 0.6
- Now the apache configuration can be generated with
.build/venv/bin/generate_controller --generate-apache-config, it supportfilesystemcacheandMapCache. - Windows fixes.
- Use console rewrite (r) to log generated tiles coordinates.
- Now if no layers is specified in
generation:default_layerswe generate all layers by default. - Now bbox to be floats.
- New
--get-bboxoption to get the bbox of a tile. - Add coveralls support (https://coveralls.io/r/camptocamp/tilecloud-chain).
- Add an config option
generation:error_fileand a command option--tilesto store and regenerate errored tiles.
Release 0.5
- SQS config change:
layers:
layer_name:
sqs:
# The region where the SQS queue is
region: eu-west-1
# The SQS queue name, it should already exists
queue: the_name
- Add debug option (
--debug), please use it to report issue. - Now the
sqlrequest can return a set of geometries in a column names geom but the syntax change a little bit =><column> AS geom FROM <table>
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
File details
Details for the file tilecloud_chain-1.17.0.dev20220324141105-py3-none-any.whl.
File metadata
- Download URL: tilecloud_chain-1.17.0.dev20220324141105-py3-none-any.whl
- Upload date:
- Size: 74.9 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/34.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.9 tqdm/4.63.1 importlib-metadata/4.11.3 keyring/18.0.1 rfc3986/2.0.0 colorama/0.4.3 CPython/3.8.10
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 21121ff851dbdfdfd5073e64aaf3efdd439bc0b86656f75db18d65ddc5ed65ef |
|
| MD5 | 6cb84567c5458c46dc9848a34806b916 |
|
| BLAKE2b-256 | 21a7d2921ddbea7564e5ee352c4dafb7a6d2b4fb4c9cb5407336805d040403c4 |
