luftdatenpumpe 0.6.0

Live Data

• Feinstaub Verlauf Berlin

• Feinstaub Karte Deutschland

List of stations

• LDI Stations #1 » Select by name, country and state

• LDI Stations #2 » Cascaded » Stations

• LDI Stations #3 » Cascaded » Measurements

• LDI Stations #4 » Select by sensor type

• LDI Stations #5 » Map by location and sensor type

Overview

# List stations
luftdatenpumpe stations --station=28,297 --reverse-geocode

# Write list of stations and metadata to PostgreSQL database
luftdatenpumpe stations --station=28,1071 --reverse-geocode --target=postgresql:///luftdaten_meta

# Forward readings to MQTT
luftdatenpumpe readings --station=28,1071 --target=mqtt://mqtt.example.org/luftdaten.info

Details

$ luftdatenpumpe --help

Usage:
  luftdatenpumpe stations [options] [--target=<target>]...
  luftdatenpumpe readings [options] [--target=<target>]...
  luftdatenpumpe --version
  luftdatenpumpe (-h | --help)

Options:
  --source=<source>             Data source, either "api" or "file://" [default: api].
  --station=<stations>          Filter data by given location ids, comma-separated.
  --sensor=<sensors>            Filter data by given sensor ids, comma-separated.
  --reverse-geocode             Compute geographical address using the Nominatim reverse geocoder
  --target=<target>             Data output target
  --progress                    Show progress bar
  --version                     Show version information
  --dry-run                     Skip publishing to MQTT bus
  --disable-nominatim-cache     Disable Nominatim reverse geocoder cache
  --debug                       Enable debug messages
  -h --help                     Show this screen


Station list examples:

  # Display metadata for given stations in JSON format
  luftdatenpumpe stations --station=28,1071 --reverse-geocode

  # Display metadata for given sensors in JSON format
  luftdatenpumpe stations --sensor=657,2130 --reverse-geocode

  # Display list of stations in JSON format made of value/text items, suitable for use as a Grafana JSON data source
  luftdatenpumpe stations --station=28,1071 --reverse-geocode --target=json.grafana.vt+stream://sys.stdout

  # Display list of stations in JSON format made of key/name items, suitable for use as a mapping in Grafana Worldmap Panel
  luftdatenpumpe stations --station=28,1071 --reverse-geocode --target=json.grafana.kn+stream://sys.stdout

  # Write list of stations and metadata to RDBMS database (PostgreSQL), also display on STDERR
  luftdatenpumpe stations --station=28,1071 --reverse-geocode --target=postgresql:///luftdaten_meta --target=json+stream://sys.stderr

  # Read station information from RDBMS database (PostgreSQL) and format for Grafana Worldmap Panel
  luftdatenpumpe stations --source=postgresql:///luftdaten_meta --target=json.grafana.kn+stream://sys.stdout


Live data examples (InfluxDB):

  # Store into InfluxDB running on "localhost"
  luftdatenpumpe readings --station=28,1071 --target=influxdb://localhost:8086/luftdaten_info

  # Store into InfluxDB, with UDP
  luftdatenpumpe readings --station=28,1071 --target=udp+influxdb://localhost:4445/luftdaten_info

  # Store into InfluxDB, with authentication
  luftdatenpumpe readings --station=28,1071 --target=influxdb://username:password@localhost:8086/luftdaten_info


Archive data examples (InfluxDB):

  # Mirror archive of luftdaten.info
  wget --mirror --continue http://archive.luftdaten.info/ --limit-rate=1.5M
  wget --mirror --continue http://archive.luftdaten.info/ --accept-regex='2018-0[6789]'
  wget --mirror --continue --no-host-directories --directory-prefix=/var/spool/archive.luftdaten.info http://archive.luftdaten.info/

  # Ingest station information from CSV archive files, store into PostgreSQL
  luftdatenpumpe stations --source=file:///var/spool/archive.luftdaten.info --target=postgresql:///luftdaten_meta --reverse-geocode --progress

  # Ingest readings from CSV archive files, store into InfluxDB
  luftdatenpumpe readings --source=file:///var/spool/archive.luftdaten.info --station=483 --sensor=988 --target=influxdb://localhost:8086/luftdaten_info --progress


Live data examples (MQTT):

  # Publish data to topic "luftdaten.info" at MQTT broker running on "localhost"
  luftdatenpumpe readings --station=28,1071 --target=mqtt://localhost/luftdaten.info

  # MQTT publishing, with authentication
  luftdatenpumpe readings --station=28,1071 --target=mqtt://username:password@localhost/luftdaten.info


Combined examples:

  # Write stations to STDERR and PostgreSQL
  luftdatenpumpe stations --station=28,1071 --target=json+stream://sys.stderr --target=postgresql:///luftdaten_meta

  # Write readings to STDERR, MQTT and InfluxDB
  luftdatenpumpe readings --station=28,1071 --target=json+stream://sys.stderr --target=mqtt://localhost/luftdaten.info --target=influxdb://localhost:8086/luftdaten_info

Prerequisites

Debian packages:

apt install postgis redis-server redis-tools

su - postgres
createuser --no-createdb --pwprompt hiveeyes
createdb --owner hiveeyes ldi_meta

psql

postgres=# \c ldi_meta
ldi_meta=# CREATE ROLE readonly WITH LOGIN PASSWORD 'readonly';
ldi_meta=# GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA public TO readonly;
ldi_meta=# GRANT SELECT ON ALL TABLES IN SCHEMA public TO readonly;

Python module

pip install luftdatenpumpe

Upstream luftdaten.info

• http://luftdaten.info/

• http://archive.luftdaten.info/

• http://deutschland.maps.luftdaten.info/

Standing on the shoulders of giants

• https://github.com/vinsci/geohash/

• https://github.com/openstreetmap/Nominatim

• https://github.com/influxdata/influxdb

• https://github.com/grafana/grafana

• https://grafana.com/plugins/grafana-worldmap-panel

Development

• opendata-stuttgart/sensors-software: Support for InfluxDB and MQTT as backend.

• https://getkotori.org/docs/applications/luftdaten.info/

• https://community.hiveeyes.org/t/datenmischwerk/702

• https://community.hiveeyes.org/t/environmental-metadata-library/1190

• https://community.hiveeyes.org/t/erneuerung-der-luftdatenpumpe/1199

Icons and pictograms

• Water Pump Free Icon from Icon Fonts is licensed by CC BY 3.0.