Tools to generate tiles from WMS or Mapnik, to S3, Berkeley DB, MBTiles, or local filesystem in WMTS layout using Amazon cloud services.
Project description
TileCloud Chain
The goal of TileCloud Chain is to provide tools around tile generation on a chain like:
Source: WMS, Mapnik.
Optionally using an SQS queue, AWS host, SNS topic.
Destination in WMTS layout, on S3, on Berkeley DB (bsddb), on MBTiles, or on local filesystem.
Features:
- Generate tiles.
- Drop empty tiles.
- Drop tiles outside a geometry or a bbox.
- Use MetaTiles.
- Generate the legend images.
- Generate GetCapabilities.
- Generate OpenLayers example page.
- Generate the Apache configuration.
- Obtain the hash of an empty tile.
- In the future, measure tile generation speed.
- Calculate cost and generation time.
- In the future, manage the AWS hosts that generate tiles.
- Delete empty tiles.
- Copy files between caches.
- Be able to use an SQS queue to dispatch the generation.
- Post processing the generated tiles.
- ...
Legacy features:
- bsddb support
- sqlite (mbtiles) support
- mapnik support (should be updated for Python3)
- mapcache (replaced by internal mapcache)
Get it
With Docker
# Login to docker hub
docker login
docker pull camptocamp/tilecloud-chain
# Initialize the project
docker run -ti \
--volume .:/project \
camptocamp/tilecloud-chain \
pcreate -s tilecloud_chain .
# Run the commands
DOCKER_ADRS=`ifconfig docker0 | head -n 2 | tail -n 1 | awk -F : '{print $2}' | awk '{print $1}'`
docker run -ti \
--volume .:/project \
--add-host=db:${DOCKER_ADRS} \
--add-host=mapserver:${DOCKER_ADRS} \
--env=USER_NAME=`whoami` \
--env=USER_ID=`id -u` \
--env=GROUP_ID=`id -g` \
--env=UMASK=`umask` \
camptocamp/tilecloud-chain \
run <a command>
To share the home folder you should add the arguments:
--volume=${HOME}:${HOME} \
--env=HOME=${HOME} \
The image also contains some tools needed to render OSM data like: fonts-dejavu, node-carto.
With pip
Requirements:
pg_config and a build environment.
Install:
pip install tilecloud-chain
pcreate -s tilecloud_chain .
Edit your layers configuration in ./tilegeneration/config.yaml.
Support
Only the latest release is supported and version < 1.11 contains security issues.
From sources
Build it:
git submodule update --recursive
python3 -m venv .build/venv
.build/venv/bin/pip install -r requirements.txt
.build/venv/bin/pip install -e .
.build/venv/bin/pip install -r dev-requirements.txt
Run prospector
.build/venv/bin/prospector
Run the tests
Setup your environment:
touch tilecloud_chain/OpenLayers.js
docker build --tag camptocamp/tilecloud-chain .
docker-compose -p tilecloud up
To run the tests:
docker-compose -p tilecloud exec test python setup.py nosetests --logging-filter=tilecloud,tilecloud_chain --attr '!'nopy3
Documentation
As documentation you can read the https://github.com/camptocamp/tilecloud-chain/blob/master/tilecloud_chain/USAGE.rst.
VSCode
You can add that in your workspace configuration to use the JSON schema:
.. code:: json
{
"yaml.schemas": {
"../tilecloud-chain/tilecloud_chain/schema.json": [
"tilecloud-chain/tilecloud_chain/tests/tilegeneration/*.yaml"
]
}
}
Changelog
Release 1.16
- Change the config validator who is a little bit more strict.
Release 1.4
- Add optional
metadatasection to the config file. See the scaffolds for example.
Release 0.9
- Correct some error with slash.
- Better error handling.
- Be able to have one error file per layer.
Release 0.8
- Correct some error with slash.
- Add
pre_hash_post_processandpost_process. - Add copy command.
Release 0.7
-
Support of deferent geoms per layers, requires configuration changes, old version:
connection: user=www-data password=www-data dbname=<db> host=localhost sql: <column> AS geom FROM <table>
to new version:
connection: user=www-data password=www-data dbname=<db> host=localhost geoms: - sql: <column> AS geom FROM <table>
More information in the Configure geom/sql chapter.
-
Update from
optparsetoargparse, and some argument refactoring, use--helpto see the new version. -
Add support of Blackbery DB (
bsddb). -
The tile
serveris completely rewrite, now it support all cache,RESTandKVPinterface,GetFeatureInforequest, and it can be used as a pyramid view or as aWSGIserver. More information in the istribute the tiles chapter. -
Add three strategy to bypass the proxy/cache: Use the headers
Cache-Control: no-cache, no-store,Pragma: no-cache(default). Use localhost in the URL and the headerHost: <host_name>(recommended). Add aSALTrandom argument (if the above don't work). More information in the Proxy/cache issue chapter. -
Improve the dimensions usage by adding it ti the WMS requests, And add a
--dimensionsargument ofgenerate_tilesto change the dimensions values. -
Extract generate_cost and generate_amazon from generate_controler.
-
Now we can creates legends, see the Legends chapter.
-
Now the tiles generation display generation statistics at the ends.
-
The EC2 configuration is moved in a separate structure, see README for more information.
Release 0.6
- Now the apache configuration can be generated with
.build/venv/bin/generate_controller --generate-apache-config, it supportfilesystemcacheandMapCache. - Windows fixes.
- Use console rewrite (r) to log generated tiles coordinates.
- Now if no layers is specified in
generation:default_layerswe generate all layers by default. - Now bbox to be floats.
- New
--get-bboxoption to get the bbox of a tile. - Add coveralls support (https://coveralls.io/r/camptocamp/tilecloud-chain).
- Add an config option
generation:error_fileand a command option--tilesto store and regenerate errored tiles.
Release 0.5
- SQS config change:
layers:
layer_name:
sqs:
# The region where the SQS queue is
region: eu-west-1
# The SQS queue name, it should already exists
queue: the_name
- Add debug option (
--debug), please use it to report issue. - Now the
sqlrequest can return a set of geometries in a column names geom but the syntax change a little bit =><column> AS geom FROM <table>
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
File details
Details for the file tilecloud_chain-1.16.1-py3-none-any.whl.
File metadata
- Download URL: tilecloud_chain-1.16.1-py3-none-any.whl
- Upload date:
- Size: 128.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.4.1 importlib_metadata/3.10.1 pkginfo/1.7.0 requests/2.22.0 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.8.5
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | aa8c418be6e9ed433d6ef6876f5ec1f484e0b02b864659b40636097db7547d3f |
|
| MD5 | 0d99679f251b4b64f90d0c302059b70a |
|
| BLAKE2b-256 | c62d94c365486c1fbecd92c14e66d95f89d23f81f3a9a2a8b65f4d9e6bca7c38 |
