pykdtree

Fast kd-tree implementation with OpenMP-enabled queries

These details have not been verified by PyPI

Project links

Homepage

Development Status
- 5 - Production/Stable
Intended Audience
- Science/Research
License
- OSI Approved :: GNU Lesser General Public License v3 (LGPLv3)
Operating System
- OS Independent
Programming Language
- Python
Topic
- Scientific/Engineering

Project description

pykdtree

Objective

pykdtree is a kd-tree implementation for fast nearest neighbour search in Python. The aim is to be the fastest implementation around for common use cases (low dimensions and low number of neighbours) for both tree construction and queries.

The implementation is based on scipy.spatial.cKDTree and libANN by combining the best features from both and focus on implementation efficiency.

The interface is similar to that of scipy.spatial.cKDTree except only Euclidean distance measure is supported.

Queries are optionally multithreaded using OpenMP.

Installation

Pykdtree can be installed via pip:

pip install pykdtree

Or, if in a conda-based environment, with conda from the conda-forge channel:

conda install -c conda-forge pykdtree

Note that by default these packages (the binary wheels on PyPI and the binary package on conda-forge) are only built with OpenMP for linux platforms. To attempt to build from source with OpenMP support do:

export USE_OMP="probe"
pip install --no-binary pykdtree pykdtree

This may not work on some systems that don’t have OpenMP installed. See the below development instructions for more guidance. Disabling OpenMP can be accomplished by setting USE_OMP to "0" in the above commands.

Development Installation

If you wish to contribute to pykdtree then it is a good idea to install from source so you can quickly see the effects of your changes. By default pykdtree is built with OpenMP enabled queries on unix-like systems. On linux this is done using libgomp. On OSX systems OpenMP is provided using the clang compiler (conda environments use a separate compiler).

$ cd <pykdtree_dir>
$ pip install -e .

This installs pykdtree in an “editable” mode where changes to the Python files are automatically reflected when running a new python interpreter instance (ex. running a python script that uses pykdtree). It does not automatically rebuild or recompile the .mako templates and .pyx Cython code in pykdtree. Editing these files requires running the pykdtree/render_template.py script and then rerunning the pip command above to recompile the Cython files.

If installation fails with undefined compiler flags or you want to use another OpenMP implementation you may need to modify setup.py or specify additional pip command line flags to match the library locations on your system.

Building without OpenMP support is controlled by the USE_OMP environment variable

$ cd <pykdtree_dir>
$ export USE_OMP=0
$ pip install -e .

Note evironment variables are by default not exported when using sudo so in this case do

$ USE_OMP=0 sudo -E pip install -e .

Control OpenMP usage

The USE_OMP variable can be set to one of a couple different options. If set to "probe", the installation process (setup.py) will attempt to determine what variant of OpenMP is available based on the compiler being used, the platform being run on, and the Python environment being run with. It will then use the flags specified by one of the other USE_OMP modes. Note that in the case of MacOS, it will also try to identify if OpenMP is available from macports or homebrew and include the necessary include and library paths.

If set to "gcc" or "gomp" then compiler and linking flags will be set appropriately for “GNU OpenMP” (gomp) library. If set to "clang" or "omp" then the flags will be set to support the “omp” library. If set to "msvc" then flags will be set for the Microsoft Visual C++ compiler’s OpenMP variant. For backwards compatibility the previous "1" has the same behavior as "probe". As mentioned above "0" can be used to disable any detection of OpenMP or attempt to compile with it.

Usage

The usage of pykdtree is similar to scipy.spatial.cKDTree so for now refer to its documentation

>>> from pykdtree.kdtree import KDTree
>>> kd_tree = KDTree(data_pts)
>>> dist, idx = kd_tree.query(query_pts, k=8)

The number of threads to be used in OpenMP enabled queries can be controlled with the standard OpenMP environment variable OMP_NUM_THREADS.

The leafsize argument (number of data points per leaf) for the tree creation can be used to control the memory overhead of the kd-tree. pykdtree uses a default leafsize=16. Increasing leafsize will reduce the memory overhead and construction time but increase query time.

pykdtree accepts data in double precision (numpy.float64) or single precision (numpy.float32) floating point. If data of another type is used an internal copy in double precision is made resulting in a memory overhead. If the kd-tree is constructed on single precision data the query points must be single precision as well.

Benchmarks

Comparison with scipy.spatial.cKDTree and libANN. This benchmark is on geospatial 3D data with 10053632 data points and 4276224 query points. The results are indexed relative to the construction time of scipy.spatial.cKDTree. A leafsize of 10 (scipy.spatial.cKDTree default) is used.

Note: libANN is not thread safe. In this benchmark libANN is compiled with “-O3 -funroll-loops -ffast-math -fprefetch-loop-arrays” in order to achieve optimum performance.

Operation	scipy.spatial.cKDTree	libANN	pykdtree	pykdtree 4 threads
Construction	100	304	96	96
query 1 neighbour	1267	294	223	70
Total 1 neighbour	1367	598	319	166
query 8 neighbours	2193	625	449	143
Total 8 neighbours	2293	929	545	293

Looking at the combined construction and query this gives the following performance improvement relative to scipy.spatial.cKDTree

Neighbours	libANN	pykdtree	pykdtree 4 threads
1	129%	329%	723%
8	147%	320%	682%

Note: mileage will vary with the dataset at hand and computer architecture.

Test

Run the unit tests using pytest

$ cd <pykdtree_dir>
$ pytest

Installing on AppVeyor

Pykdtree requires the “stdint.h” header file which is not available on certain versions of Windows or certain Windows compilers including those on the continuous integration platform AppVeyor. To get around this the header file(s) can be downloaded and placed in the correct “include” directory. This can be done by adding the anaconda/missing-headers.ps1 script to your repository and running it the install step of appveyor.yml:

# install missing headers that aren’t included with MSVC 2008 # https://github.com/omnia-md/conda-recipes/pull/524 - “powershell ./appveyor/missing-headers.ps1”

In addition to this, AppVeyor does not support OpenMP so this feature must be turned off by adding the following to appveyor.yml in the environment section:

environment:

global:

# Don’t build with openmp because it isn’t supported in appveyor’s compilers USE_OMP: “0”

Project details

These details have not been verified by PyPI

Project links

Homepage

Development Status
- 5 - Production/Stable
Intended Audience
- Science/Research
License
- OSI Approved :: GNU Lesser General Public License v3 (LGPLv3)
Operating System
- OS Independent
Programming Language
- Python
Topic
- Scientific/Engineering

Release history Release notifications | RSS feed

1.3.13

Sep 4, 2024

1.3.12

Apr 18, 2024

1.3.11

Feb 15, 2024

This version

1.3.10

Nov 29, 2023

1.3.9

Sep 22, 2023

1.3.7.post0

Mar 20, 2023

1.3.7

Mar 20, 2023

1.3.6

Oct 28, 2022

1.3.5

Jun 8, 2022

1.3.4

Nov 11, 2020

1.3.3

Nov 9, 2020

1.3.2

Nov 6, 2020

1.3.1

Jun 22, 2018

1.3.0

Jun 1, 2018

1.2.2

Jun 3, 2017

1.2.1

Jul 27, 2016

1.2.0

Jul 25, 2016

1.1.1

Feb 28, 2016

1.1

Feb 28, 2016

1.0

Feb 4, 2015

0.3

Jan 5, 2015

0.2

Jun 3, 2013

0.1

Feb 19, 2013

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pykdtree-1.3.10.tar.gz (111.6 kB view details)

Uploaded Nov 29, 2023 Source

Built Distributions

pykdtree-1.3.10-cp312-cp312-win_arm64.whl (48.0 kB view details)

Uploaded Nov 29, 2023 CPython 3.12 Windows ARM64

pykdtree-1.3.10-cp312-cp312-win_amd64.whl (57.2 kB view details)

Uploaded Nov 29, 2023 CPython 3.12 Windows x86-64

pykdtree-1.3.10-cp312-cp312-musllinux_1_1_x86_64.whl (385.3 kB view details)

Uploaded Nov 29, 2023 CPython 3.12 musllinux: musl 1.1+ x86-64

pykdtree-1.3.10-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (377.9 kB view details)

Uploaded Nov 29, 2023 CPython 3.12 manylinux: glibc 2.17+ x86-64

pykdtree-1.3.10-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (370.7 kB view details)

Uploaded Nov 29, 2023 CPython 3.12 manylinux: glibc 2.17+ ARM64

pykdtree-1.3.10-cp312-cp312-macosx_11_0_arm64.whl (61.6 kB view details)

Uploaded Nov 29, 2023 CPython 3.12 macOS 11.0+ ARM64

pykdtree-1.3.10-cp312-cp312-macosx_10_9_x86_64.whl (344.4 kB view details)

Uploaded Nov 29, 2023 CPython 3.12 macOS 10.9+ x86-64

pykdtree-1.3.10-cp311-cp311-win_arm64.whl (49.1 kB view details)

Uploaded Nov 29, 2023 CPython 3.11 Windows ARM64

pykdtree-1.3.10-cp311-cp311-win_amd64.whl (58.7 kB view details)

Uploaded Nov 29, 2023 CPython 3.11 Windows x86-64

pykdtree-1.3.10-cp311-cp311-musllinux_1_1_x86_64.whl (377.3 kB view details)

Uploaded Nov 29, 2023 CPython 3.11 musllinux: musl 1.1+ x86-64

pykdtree-1.3.10-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (369.8 kB view details)

Uploaded Nov 29, 2023 CPython 3.11 manylinux: glibc 2.17+ x86-64

pykdtree-1.3.10-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (358.7 kB view details)

Uploaded Nov 29, 2023 CPython 3.11 manylinux: glibc 2.17+ ARM64

pykdtree-1.3.10-cp311-cp311-macosx_11_0_arm64.whl (62.9 kB view details)

Uploaded Nov 29, 2023 CPython 3.11 macOS 11.0+ ARM64

pykdtree-1.3.10-cp311-cp311-macosx_10_9_x86_64.whl (346.7 kB view details)

Uploaded Nov 29, 2023 CPython 3.11 macOS 10.9+ x86-64

pykdtree-1.3.10-cp310-cp310-win_arm64.whl (48.9 kB view details)

Uploaded Nov 29, 2023 CPython 3.10 Windows ARM64

pykdtree-1.3.10-cp310-cp310-win_amd64.whl (58.7 kB view details)

Uploaded Nov 29, 2023 CPython 3.10 Windows x86-64

pykdtree-1.3.10-cp310-cp310-musllinux_1_1_x86_64.whl (365.2 kB view details)

Uploaded Nov 29, 2023 CPython 3.10 musllinux: musl 1.1+ x86-64

pykdtree-1.3.10-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (346.6 kB view details)

Uploaded Nov 29, 2023 CPython 3.10 manylinux: glibc 2.17+ x86-64

pykdtree-1.3.10-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (334.9 kB view details)

Uploaded Nov 29, 2023 CPython 3.10 manylinux: glibc 2.17+ ARM64

pykdtree-1.3.10-cp310-cp310-macosx_11_0_arm64.whl (63.0 kB view details)

Uploaded Nov 29, 2023 CPython 3.10 macOS 11.0+ ARM64

pykdtree-1.3.10-cp310-cp310-macosx_10_9_x86_64.whl (346.7 kB view details)

Uploaded Nov 29, 2023 CPython 3.10 macOS 10.9+ x86-64

pykdtree-1.3.10-cp39-cp39-win_arm64.whl (49.6 kB view details)

Uploaded Nov 29, 2023 CPython 3.9 Windows ARM64

pykdtree-1.3.10-cp39-cp39-win_amd64.whl (59.2 kB view details)

Uploaded Nov 29, 2023 CPython 3.9 Windows x86-64

pykdtree-1.3.10-cp39-cp39-musllinux_1_1_x86_64.whl (368.2 kB view details)

Uploaded Nov 29, 2023 CPython 3.9 musllinux: musl 1.1+ x86-64

pykdtree-1.3.10-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (349.6 kB view details)

Uploaded Nov 29, 2023 CPython 3.9 manylinux: glibc 2.17+ x86-64

pykdtree-1.3.10-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (337.9 kB view details)

Uploaded Nov 29, 2023 CPython 3.9 manylinux: glibc 2.17+ ARM64

pykdtree-1.3.10-cp39-cp39-macosx_11_0_arm64.whl (63.6 kB view details)

Uploaded Nov 29, 2023 CPython 3.9 macOS 11.0+ ARM64

pykdtree-1.3.10-cp39-cp39-macosx_10_9_x86_64.whl (347.3 kB view details)

Uploaded Nov 29, 2023 CPython 3.9 macOS 10.9+ x86-64

File details

Details for the file pykdtree-1.3.10.tar.gz.

File metadata

Download URL: pykdtree-1.3.10.tar.gz
Upload date: Nov 29, 2023
Size: 111.6 kB
Tags: Source
Uploaded using Trusted Publishing? No
Uploaded via: twine/4.0.2 CPython/3.11.6

File hashes

Hashes for pykdtree-1.3.10.tar.gz
Algorithm	Hash digest
SHA256	`41e7c5d669cadc2188acc4bbb4b0b4dcaf492d84512f1e6517a7ab2d122c911d`
MD5	`b6ac1e8de970513bf882b54d87f171bc`
BLAKE2b-256	`f66221a4943c081a8f1f6e9b14c899ba31eb5d9819c101c12f91ddf24e7756b3`

Algorithm	Hash digest
SHA256	`870b82b73728d5ed1951fd0d59701d3fa3fdaf3838c08a51084e5c231b578b85`
MD5	`eaaa97180da22dc5510009212c44e1e3`
BLAKE2b-256	`bd7627c0f7172effa30e1a6eade4b0b6c65787937c90f181cc19ae3db2e7159d`

Algorithm	Hash digest
SHA256	`f66fe62cc199d474dd869024300cdc75600d865d79ef90eda5d4798e209f554e`
MD5	`66375bc36b73a3a5d0d80d0c4697992a`
BLAKE2b-256	`8ebaef126499f8e157bdac8e9d0f2770e3b60ec6c7c75b1e2d56bfa413d90542`

Algorithm	Hash digest
SHA256	`25769e2a064e0b5f5a80a3885e62e9f727e681b96fbccf4497cc31f716f47579`
MD5	`ff5ed4cc6d55df270320b7111c4de1b8`
BLAKE2b-256	`5092cbf019c45625e222b7caa9ab172f023756961afe5557688a597cf042dc0d`

Algorithm	Hash digest
SHA256	`3fd2b4a5cde45cc47dd26868317db9e4705fb9ad9c4c944a3fb7eb7a110f6dce`
MD5	`7925d3cee2889ca480c16b4b9e07e428`
BLAKE2b-256	`7bc9ca03771d421b1200ab700e9a66e69fe532fae3f8c24d193fd0211fe776c0`

Algorithm	Hash digest
SHA256	`a24308f6f6db8c4aff434b7b05b279c8711041c57da0448132e84c39bb8184ff`
MD5	`1279995e4d676389a667eb807506cd09`
BLAKE2b-256	`9f6a4b616d3b12e45574b50f88ebba541e01a8a04f7126ea7151705af5b2142b`

Algorithm	Hash digest
SHA256	`397b6849223765348ba8b1a8a8c9786dd42c4958b82010b25d923fb8a6af5c35`
MD5	`d55c6424390f5caf7a9c3714b7c2f12c`
BLAKE2b-256	`9335aee9cecf6d27b3f6bcd32a9d99b5340c68ddc41c6e95a20e4eb58e33c5a4`

Algorithm	Hash digest
SHA256	`0878f0134a008e0e40c8527fc96fbbc4b3362f193173dbef6a8ab277ebcc82f7`
MD5	`1408348fbcfce26bf3465525f937a3b6`
BLAKE2b-256	`b5b88c31cfd530c5e2448a27a463ba9face77c566ab7d60924cf7c208e5828a6`

Algorithm	Hash digest
SHA256	`f3f48dc8ee24f6aa30ba1e4169489b82edc6a2a1ef2d1dc2efba754a2d016063`
MD5	`43226db470bde4b71fd0e068f8f41b20`
BLAKE2b-256	`a7f619eee19c1ec37b19b180fdbee0a6d6fb6bf2d5e6874c67bb93381eb1263d`

Algorithm	Hash digest
SHA256	`c9846c871c4e92d90ba970ebdbd93665af3fe287d4a93ec322297b17981cf0a2`
MD5	`a81e730a2ff473d2ce155eebd707c7b3`
BLAKE2b-256	`01b25d1609a8925bdbb3f22f761cf4fee379197c025da9046ab635c0ffd43189`

Algorithm	Hash digest
SHA256	`864f4fd28a45dc52f6c049f90f4a8251ae2f1aa3600cf66332eacea7798f3c7c`
MD5	`66bd258f11521f70debf05af170ad0cb`
BLAKE2b-256	`e5efc11fde2e3e3dbec9f06cef3f5006131982444b3d64726fddfcb008d89e4e`

Algorithm	Hash digest
SHA256	`ed1fdf7a1b47677cb891396f18874d47d73c9b04e82a149a247548ec3251151c`
MD5	`6add7c1fbbd448a4cc8b0639e55b64ab`
BLAKE2b-256	`66cdddac14e2ef911ab15039b1c01cf127c2788c91ff82667ac0cfa491a719c6`

Algorithm	Hash digest
SHA256	`0e04add648af5bca296b40ad60204cc6db963c6edbf90f1971655494545976d1`
MD5	`52a96321dec730d17e7ace59bd3ecbaf`
BLAKE2b-256	`79dcd3d9ccf6c5d084dc9c84ed3ab7d951619b149ac42e390ee71970e21bc526`

Algorithm	Hash digest
SHA256	`95817dd3fcff3a5de36286ba33eefcd47fbf46c693c37bea42c4aa06ea3ecdf6`
MD5	`3fdd27e84f42e37b78103592166ac235`
BLAKE2b-256	`ca83fd7039165eb80f1ebdeef22475c79e38c034596540481a8f78bed33b1027`

Algorithm	Hash digest
SHA256	`069543be5d501c40cf861c17dda386f1481932733b1764320d65ce50179f16df`
MD5	`bcf7fb5a20736146a9160a99baf290fb`
BLAKE2b-256	`a8b0d776725aef46a20b831f29e73df7d69ebef23d1d3de4cc28d51bec2358af`

Algorithm	Hash digest
SHA256	`7a2ca034888222abc5d3f67668d27e5389b082f1706a17e9a6dd37e1d2839eb6`
MD5	`d9df86534d7bd54fd22aa64168297283`
BLAKE2b-256	`7168df9fea944e2ac0d282635cdc1caedf549339f7b8470ed523610c8d6c5929`

Algorithm	Hash digest
SHA256	`8a7d2ff07fa0716d718b64ff9c50abce398450b09b4eceac3c4e83dbd8318f38`
MD5	`122f6ded3acb70b546018982b98546bd`
BLAKE2b-256	`8535349509a7fe081a2dc5cb6e3d8e56776d7e38126e48e1f02323765edd4ba3`

Algorithm	Hash digest
SHA256	`0b347027d1548f6ad18ad370cbc30b4d94efaab77bee155e413cb230946a50ca`
MD5	`310b5123ec4c0e1e9ea0d4519f7dd19c`
BLAKE2b-256	`ba7fdd5fef696124a82d66580d2ac08ae6d770172492f9548c375249b220812a`

Algorithm	Hash digest
SHA256	`bbaf640cbcba97fb462da814ec1d317b46a1636451967b84d54c93196fba46d8`
MD5	`925431c70d2a7a2df5ffe4afb0bb43e9`
BLAKE2b-256	`f90aa575b245c516d51a5da1079bd4523d19d12d9bdddf3acdfd6c4e7338084b`

Algorithm	Hash digest
SHA256	`7a3272fae4f12a4dad799ad394d7749f3a505f722122fa0fffe31e56aea89b1d`
MD5	`2fef0c1d2ab6c13071e10999fced77ff`
BLAKE2b-256	`6d02fe74c92d9031838fcc13e77b89a62c7ed258e817faf215027864c8d80ffa`

Algorithm	Hash digest
SHA256	`101d4b29eda1b9beaf67855af13eca47a65f02b5a6717b88d19d4c7cfc6a6729`
MD5	`e7a5ed961012f507f1b38c064f39dcc7`
BLAKE2b-256	`905b1e5bca9a0cc8829b63bd31d18466127cde37d718537707c65fa15586bf41`

pykdtree 1.3.10

Navigation

Verified details

Maintainers

Unverified details

Project links

Meta

Classifiers

Project description

pykdtree

Objective

Installation

Development Installation

Control OpenMP usage

Usage

Benchmarks

Test

Installing on AppVeyor

Project details

Verified details

Maintainers

Unverified details

Project links

Meta

Classifiers

Release history Release notifications | RSS feed

Download files

Source Distribution

Built Distributions

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

Algorithm	Hash digest
SHA256	`70a937b9c7b6ec61a9d7cc26794372384cf23961f942a1544493683942492a1e`
MD5	`23dbf70abc423c75caf6204105d0ccf1`
BLAKE2b-256	`a6e71b225e3d62a9f145f9239f5a2c5868d6182eefa469abb2aba87fe3cf7081`

Algorithm	Hash digest
SHA256	`a00c9bf543dc5205f2d55121ad9fa10209a46f4a8e0207bbb6ce2fe06ef3ef46`
MD5	`e2b6679c3a962338ff79d1ea1597d7b8`
BLAKE2b-256	`8d3d571218d7b98b3736c28187ffdfbe0639ab8c1619d962266a81483aca836b`

Algorithm	Hash digest
SHA256	`692a1f708eca7c0b6afd8f438f445f6af486db6f822fd488d5ea08c9ee3ae493`
MD5	`8c5d4627250c793f62b6ce3297337f49`
BLAKE2b-256	`7d73daa2786108a719bf893f22c84f159552db10f894fecdd5ab9e181bd4cbd3`

Algorithm	Hash digest
SHA256	`3070e401408028fb4601362d8cc71f322529cc78f728b435ecef771457e5743e`
MD5	`3a147adb0334fcc68c86a9650081f04e`
BLAKE2b-256	`36752d267a45d9214ff4540c9b46a135bfb549e55d7402a7778f4d07fb23f192`

Algorithm	Hash digest
SHA256	`39d957e9f99b68c69458a36e3e173d6b025d93c4ec2bf8715d51e06348f0dacf`
MD5	`873ad4b9764f9f3dcae0982b32268c64`
BLAKE2b-256	`41d4e48980d3a77bd9ca95be70c15ff6af91dce9cd7f65769b35aecb457f0c31`

Algorithm	Hash digest
SHA256	`491770d5305d7d6350c116c89bdcdd26eac02e5ae23d94cb159bf83a0b1b838b`
MD5	`29a7fb4a6cbd1a0ea0a3f69a8405ad43`
BLAKE2b-256	`2917f068c2803623d39119b2f66bdf0883d9d633815e889ebd906256117f4fc7`

Algorithm	Hash digest
SHA256	`235a06403a66e347b5c873499a03c3f160b109ebf0abbcb7cc953ee85e2e8e2e`
MD5	`a11b5718206ad480bed68db4e9dd356b`
BLAKE2b-256	`046f98c45b524b27ec7f84e25a9a5dabd48722979531337f3fd564d0e072cbb5`

Algorithm	Hash digest
SHA256	`6b8e053d18949a2ba1ff8160e36aab4e0df2fbfd9a284b8d5ba7da62b2dd260c`
MD5	`05d24a23b5956e09278e1e2179bd602d`
BLAKE2b-256	`6b686615ef3cb0e13120058483ef7f96bf4ae95f05a84b3658116551f3e030fb`