Threedi Raster Edits provides python tooling for threedi such as raster-conversion, alignment, fillers, checks and others
Project description
threedi-raster-edits
Introduction
Threedi Raster Edits is used internally at NENS for raster conversion, edits and checks. Currently it includes the functions as alignment, replace_nodata, alignment checks, property checks and raster conversions to friction, interception and others.
Usage
We can download rasters:
>>> import threedi_raster_edits as tre >>> landuse = tre.UUID.THREEDI_LANDUSE >>> vector = tre.Vector(vector_path) >>> feature = vector[0] >>> rextract = tre.RasterExtraction(lizard_api_key) >>> rextract.run("ouput/raster.tif", landuse_uuid, feature.geometry, cellsize=0.5, threads=1)
Raster conversion:
>>> from threedi_raster_edits import ThreediRasterGroup >>> group = ThreediRasterGroup(dem_path, landuse_path=landuse_path, soil_path=soil_path) >>> group.load_landuse_conversion_table(csv_landuse_path) >>> group.load_soil_conversion_table(csv_soil_path) >>> group.generate_friction() >>> group.generate_permeability() >>> group.generate_interception() >>> group.generate_crop_type() >>> group.generate_max_infiltration() >>> group.generate_infiltration() >>> group.generate_hydraulic_conductivity() >>> group.friction.write("friction.tif")
Raster checks:
>>> group = ThreediRasterGroup(dem_path, friction_path="friction.tif") >>> align_checks = group.check_alignemnt() >>> property_checks = group.check_properties()
Building interception can also be generated:
>>> from threedi_raster_edits import Vector >>> buildings = Vector("buildings.shp") >>> group = ThreediRasterGroup(dem_path, buildings=buildings) >>> group.generate_building_interception(10) >>> group.interception.write("interception.tif")
One can also use the internal raster class for easy raster processing as reproject, clipping, resampling, aligning, reading geometries and creating copies:
>>> from threedi_raster_edits import Raster >>> import numpy as np >>> raster = Raster("dem.tif") >>> raster.reproject(28992) >>> raster.resample(1,1) >>> raster.clip(Vector("clip.shp")) >>> raster.align(Raster("template.tif")) >>> copy = raster.copy(shell=True) >>> clip = Vector("clip.shp") >>> geometry = clip[0].geometry # geometry of first feature >>> clip_values = np.nansum(raster.read(clip))
For more advanced/larger processing you can also use raster loops:
>>> output = raster.copy(shell=True) >>> for tile in raster: >>> array = tile.array >>> array[array==1] = 20 >>> output.array = array, tile.location >>> output.write("tiles_adding.tif")
Or just do something with the array:
>>> output = raster.copy(shell=True) >>> array = raster.array >>> array = array *100 >>> output.array = array >>> output.write("output.tif")
Installation
We can be installed with:
$ pip install threedi-raster-edits
(TODO: after the first release has been made)
Development installation of this project itself
GDAL is not automatically installed, hence not available in the requirement,so please install gdal 3.2.0 yourself or use anaconda:
$ conda create -n threedi_raster_edits python=3 gdal=3.2.0 rtree black pytest
We use python’s build-in “virtualenv” to get a nice isolated directory. You only need to run this once:
$ python3 -m venv --system-site-packages .
A virtualenv puts its commands in the bin directory. So bin/pip, bin/pytest, etc. Set up the dependencies like this:
$ bin/pip install -r requirements.txt
There will be a script you can run like this:
$ bin/run-threedi-raster-edits
It runs the main() function in threedi-raster-edits/scripts.py, adjust that if necessary. The script is configured in setup.py (see entry_points).
In order to get nicely formatted python files without having to spend manual work on it, run the following command periodically:
$ bin/black threedi_raster_edits
Run the tests regularly. This also checks with pyflakes, black and it reports coverage. Pure luxury:
$ bin/pytest
The tests are also run automatically on “github actions” for “master” and for pull requests. So don’t just make a branch, but turn it into a pull request right away:
Prepend the title with “[WIP]”, work in progress. That way you make clear it isn’t ready yet to be merged.
Important: it is easy to give feedback on pull requests. Little comments on the individual lines, for instance. So use it to get early feedback, if you think that’s useful.
On your pull request page, you also automatically get the feedback from the automated tests.
There’s also coverage reporting on coveralls.io (once it has been set up).
If you need a new dependency (like requests), add it in setup.py in install_requires. Local development tools, like “black”, can be added to the requirements.txt directoy. In both cases, run install again to actuall install your dependency:
$ bin/pip install -r requirements.txt
Releasing the project
To make a release, first install zest.releaser (globally):
$ pip install zest.releaser
Make sure you’re on master and that you’ve updated the changelog. Then release it with “fullrelease”. You can press “enter” at every question:
$ fullrelease
The github action ought to upload the fresh package to pypi.
Changelog of threedi-raster-edits
0.27 (2022-08-29)
Bugfix: For raster tiles everything is not done in memory anymore.
Cleanup for lizard including dependency installation.
Updated for postgres upload. Works again.
Updated blocksize estimation.
Differences between stacked and adjacent rasters.
No automatic console logging.
0.26 (2022-07-11)
Geometry field changes of connection nodes.
0.25 (2022-07-08)
Hotfix: removed aggregation_in_space in the v2_aggregation_settings
0.24 (2022-07-08)
Hotfix: requirements.txt
0.23 (2022-07-07)
Added two extra dependencies.
0.22 (2022-07-07)
Removed 3Di results.
Removed 3Di scenarios.
Klondike support.
0.21 (2022-03-22)
Scenarios > 3.8.0
0.20 (2022-03-22)
QGIS changes for raster conversions.
Include global cache, for storage in a pickle.
Added easy threedi scenario downloads.
0.19 (2022-02-28)
Increased speed of vector loops
Added examples in scripts
Added intuitive progress bars inclusing for gdal
0.18 (2021-11-29)
Changed input for rextract.
0.17 (2021-11-29)
Added threading to lizard rextract
Also qgis compatible way of showing progress
Simulations now in queue
0.16 (2021-10-28)
Fixed release error
0.15 (2021-10-28)
More efficient loading of threedirastergroup
0.14 (2021-10-28)
ThreediAPI and ThreediResults are now optional.
0.13 (2021-10-19)
Fixed vector reprojections
Added inflated rasterloops for rasters
Added rasterloops for rastergroups
Added inflated rasterloops for rastergroups
Added bbox clipping for rasters
0.12 (2021-09-13)
Improved difference algorithm
Remove geometry fixed at every call, now call once with veotor.fix()
fid= -1 will result in a fid which is the count
0.11 (2021-09-06)
Token release
0.10 (2021-09-06)
Added sqlite-model support
Added api support
0.9 (2021-05-06)
Changed black format.
0.8 (2021-05-06)
Added clips for rasters
Added custom line string lengths
Added vector interpolation
Added (partly) fix threedi rasters
0.7 (2021-03-26)
Fixed release process (same for 0.6/0.6).
0.4 (2021-03-26)
Fixed release process.
Fixed tests.
Added logging.
Better memory usage of rasters.
Small changes in vector, geometries.
0.3 (2021-03-25)
Automated pypi release.
0.2 (2021-03-12)
Changed the syntax of raster class
Changed the imports to the main script: E.g., from threedi_raster_edits import raster, rastergroup etc.
Changed the readme.
Rewritten the geometry structure.
0.1 (2021-03-11)
Initial project structure created with cookiecutter and https://github.com/nens/cookiecutter-python-template
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for threedi-raster-edits-0.27.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | ed3adc27d100ca8dfa8aed9ade79d8f4d8d33cd93329b367351760687a754346 |
|
MD5 | 3d1dade43d4361e894d528578157e5b0 |
|
BLAKE2b-256 | 74decbf17e750e54721f36d46cb78d120a3bb976f1b1867b93057319d3d7460e |
Hashes for threedi_raster_edits-0.27-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fb86d98f918ce898ce045aa0b7ebe74b72c3775d08f640b2d3e1d58dd887fd0d |
|
MD5 | 75b6585066b01d44412d3a3bf830abe5 |
|
BLAKE2b-256 | 4877310fb3800a3f16a11ee7bf9365981b97a3cc776915cc1fbedb5bb185972b |