XYZ Spaces for Python

Manage your XYZ Hub or HERE Data Hub spaces from Python.

FEATURED IN: Online Python Machine Learning Conference & GeoPython 2020, Sept 21, 2020, see conference schedule.

Motivation

XYZ is an Open Source, real-time, cloud database system providing access to large geospatial data at scale. An XYZ "Hub" manages "spaces" that contain "features" (geodata "records") with tags and properties, with spaces and features having unique IDs. A RESTful API exists to provide low-level access to interact with a XYZ Hub.

This Python package allows to interact with your XYZ spaces and features on a given Hub using a higher level programmatic interface that wraps the RESTful API. Using this package you can:

• Create, read, list, update, share, delete spaces (also: get space info and stats).
• Add, read, update, iterate, search, cluster (hex/quad bins), delete features.
• Search features by ID, tag, property, bbox, tile, radius, geometry.

Based on the XYZ Hub the HERE Data Hub is a commercial service (with a free plan), that offers some additional features (in a pro plan), like clustering, virtual spaces, activity logs, and likely more to come.

The GIF below shows an interaction with an example notebook, demonstrating how to use a spatial search on a big public dataset, loaded from the HERE Data Hub.

Prerequisites

Before you can install this package, run its test-suite or use the example notebooks to make sure your system meets the following prerequisities:

• A Python installation, 3.6+ recommended, with the pip command available to install dependencies

• A HERE developer account, free and available under HERE Developer Portal

• An XYZ API access token from your XYZ Hub server or the XYZ portal (see also its Getting Started section) in an environment variable named XYZ_TOKEN which you can set like this (with a valid value, of course):

  export XYZ_TOKEN="MY-FANCY-XYZ-TOKEN"
  

  If you prefer, you can alternatively provide this token as a parameter in your code.

Installation

This package can be installed with pip or conda from various sources:

• Install with conda from the Anaconda conda-forge channel:

  conda install -c conda-forge xyzspaces
  

• Install from the Python Package Index:

  pip install xyzspaces
  

• Install from its source repository on GitHub:

  pip install -e git+https://github.com/heremaps/xyz-spaces-python#egg=xyzspaces
  

If you want to run the test suite or experiment with the example notebooks bundled, you need to clone the whole repository:

• Make a local clone of the repository hosting this package. The following command should do:

  git clone https://github.com/heremaps/xyz-spaces-python.git
  

• Change into the repo root directory:

  cd xyzspaces
  

See the next section for how to run the test suite.

Test Suite

You can run the test suite locally:

pip install -r requirements_dev.txt
pytest -v --cov=xyzspaces tests

The test suite provides test coverage of around 90% (but less if the tests cannot find your credentials).

Documentation

For now, the documentation consists of a small number of example Jupyter notebooks in the docs/notebooks directory plus an API reference, which is automatically generated from the docstrings in the code.

Jupyter Notebooks

See docs/notebooks/README.md to learn how to install and use the example Jupyter notebooks.

API Reference

To generate the API reference locally in docs/apiref/_build/html run this command:

bash scripts/build_apiref.sh

Hello World Example

The following is a tiny "Hello World"-like example that you can run to have a successful first XYZ experience right after installation! Just make sure to use your own real XYZ token!

import geojson
import xyzspaces

xyz = xyzspaces.XYZ(credentials="MY_XYZ_TOKEN")

# Create a New Space
title = "My Demo Space"
description = "My Description"
space = xyz.spaces.new(title=title, description=description)

# Define a New Feature
feature =  {
    "type": "Feature",
    "properties": {"party": "Republican"},
    "geometry": {
        "type": "Polygon",
        "coordinates": [[
            [-104.05, 48.99],
            [-97.22,  48.98],
            [-96.58,  45.94],
            [-104.03, 45.94],
            [-104.05, 48.99]
        ]]
    }
}

# Save it to a Space and get its ID
feature_id = space.add_features(features=geojson.FeatureCollection([feature]))["features"][0]["id"]

# Read a Feature from a Space
feature = space.get_feature(feature_id=feature_id)
print(geojson.dumps(feature, indent=4, sort_keys=True))

Logging Configuration

By default logging is disabled. To enable logging, use below code snippets in your python code to setup logging at DEBUG level:

import logging
from xyzspaces import setup_logging

setup_logging(default_level=logging.DEBUG)

Default logging configuration is defined in a file.

This ensures that log messages will be written to the file xyz.log in your current working directory.

Here is an example log file (xyz.log):

2020-02-21 17:55:46,132 - apis.py:130 - ERROR - Curl command: curl --request GET https://xyz.api.here.com/hub/spaces/dummy-111 --header "Authorization: Bearer <XYZ_TOKEN>"
2020-02-21 17:55:46,133 - apis.py:131 - ERROR - Response status code: 404
2020-02-21 17:55:46,133 - apis.py:132 - ERROR - Response headers: {'Content-Type': 'application/json', 'Content-Length': '150', 'Connection': 'keep-alive', 'Date': 'Fri, 21 Feb 2020 12:25:46 GMT', 'x-amzn-RequestId': '397c8039-79f1-4956-bbe4-46ca78c7ec2d', 'content-encoding': 'gzip', 'Stream-Id': '397c8039-79f1-4956-bbe4-46ca78c7ec2d', 'x-amzn-Remapped-Content-Length': '150', 'x-amzn-Remapped-Connection': 'keep-alive', 'x-amz-apigw-id': 'IPzblGVFjoEF5pg=', 'x-amzn-Remapped-Date': 'Fri, 21 Feb 2020 12:25:46 GMT', 'X-Cache': 'Error from cloudfront', 'Via': '1.1 e25383e25378de918d3b187b3239eb5b.cloudfront.net (CloudFront)', 'X-Amz-Cf-Pop': 'BOM51-C2', 'X-Amz-Cf-Id': 'nZAJUB_FBiHdojziSoG3SBcMdf8rNyHuOMSlJljyxDNlx1I0O3t9YQ=='}
2020-02-21 17:55:46,134 - apis.py:133 - ERROR - Response text: {"type":"ErrorResponse","error":"Exception","errorMessage":"The requested resource does not exist.","streamId":"397c8039-79f1-4956-bbe4-46ca78c7ec2d"}

To customize the logging configuration, set the variable XYZ_LOG_CONFIG to hold the full path of the logging configuration options file logging_config.json:

export XYZ_LOG_CONFIG=~/logging_config.json

License

Copyright (C) 2019-2020 HERE Europe B.V.

Unless otherwise noted in LICENSE files for specific directories, the LICENSE in the root applies to all content in this repository.