Skip to main content

Karoo Array Telescope - Telescope State Client

Project description

MeerKAT Science Data Processor Telescope State

This is a client package that allows connection to the Redis database that stores telescope state information for the Science Data Processor of the MeerKAT radio telescope. This database is colloquially known as telstate.

The Redis database acts as a key-value store. Each key is either a sensor or an attribute. A sensor has multiple timestamped values organised into an ordered set. An attribute (or immutable key) has a single value without a timestamp that is not allowed to change.

The keys are strings and the values are Python objects serialised via MessagePack, which has been extended to support tuples, complex numbers and NumPy arrays. Older versions of the database stored the values as pickles, and the package warns the user if that’s the case. Keys can be retrieved from the telstate object using attribute syntax or dict syntax.

Databases can be accessed via one of two backends: a Redis client backend that allows shared access to an actual Redis server over the network (or a simulated server via fakeredis) and a simplified in-memory backend for stand-alone access. Both backends support loading and saving a Redis snapshot in the form of an RDB dump file.

It is possible to have multiple views on the same database (one per telstate instance). A view is defined as a list of prefixes acting as namespaces that group keys. When reading from the database, each prefix is prepended to the key in turn until a match is found. When writing to the database, the first prefix is prepended to the key. The first prefix therefore serves as the primary namespace while the rest are supplementary read-only namespaces.

Getting Started

The simplest way to test out katsdptelstate is to use the in-memory backend. If you want to run a real Redis server you will need to install Redis (version 2.8.9 or newer) on a suitable machine on the network. For example, do this:

  • macOS: brew install redis

  • Ubuntu: apt-get install redis-server

Then pip install katsdptelstate and run a local redis-server. If you also want to load RDB files, do pip install katsdptelstate[rdb].

A Simple Example

import time
import katsdptelstate

# Connect to an actual Redis server via an endpoint or an URL
telstate = katsdptelstate.TelescopeState('localhost:6379')
telstate = katsdptelstate.TelescopeState('redis://localhost')
# Or use the in-memory backend (useful for testing)
telstate = katsdptelstate.TelescopeState()
# Load RDB file into Redis if katsdptelstate is installed with [rdb] option
telstate.load_from_file('dump.rdb')

# Attribute / dict style access returns the latest value
telstate.add('n_chans', 32768)
print(telstate.n_chans)  # -> 32768
print(telstate['n_chans'])  # -> 32768

# List all keys (attributes and sensors)
print(telstate.keys())  # -> ['n_chans']

# Sensors are timestamped underneath
st = time.time()
telstate.add('n_chans', 4096)
et = time.time()
telstate.add('n_chans', 16384)
# Time ranges can be used and are really fast
telstate.get_range('n_chans', st=st, et=et)  # -> [(4096, 1556112474.453495)]
# Add an item 10 seconds back
telstate.add('n_chans', 1024, ts=time.time() - 10)

# Attributes cannot be changed (only deleted)
telstate.add('no_change', 1234, immutable=True)
# Adding it again is OK as long as the value doesn't change
telstate.add('no_change', 1234, immutable=True)
# Simpler notation for setting attributes
telstate['no_change'] = 1234
# Will raise katsdptelstate.ImmutableKeyError
telstate['no_change'] = 456

# Create a new view with namespace 'ns' and standard underscore separator
view = telstate.view('ns')
# Insert a new attribute in this namespace and retrieve it
view['x'] = 1
print(view['x'])  # -> 1
print(view.prefixes)  # -> ('ns_', '')
print(view.keys())  # -> ['n_chans', 'no_change', 'ns_x']

History

0.9 (2020-05-25)

  • Deprecate Python 2 support: this is the last release that will support Python 2 (#94)

  • Remove get_message and send_message, which were never used (#89)

  • Publish the documentation on https://katsdptelstate.readthedocs.io (#90)

  • Disable pickles by default for security (#92)

0.8 (2019-05-06)

  • The default encoding is now msgpack; warn on loading pickles (#75, #79)

  • The default backend is now in-memory (#76)

  • Add the ability to dump in-memory backend to an RDB file (#77)

  • Construct from RDB file-like objects and Redis URLs (#80, #82)

  • Report keys and prefixes to the user as strings (#73)

  • Add IPython tab completion (#83)

  • RDB reader and writer cleanup (#85, #86)

0.7 (2019-02-12)

  • Introduce encodings and add msgpack encoding as alternative to pickle (#64, #65)

  • Introduce backends and add in-memory backend as alternative to redis (#71, #72)

  • Simplify setting attributes via __setitem__ (#68)

  • Let keys be bytes internally, but allow specification as unicode strings (#63)

  • The GitHub repository is now public as well

0.6 (2018-05-10)

  • Initial release of katsdptelstate

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

katsdptelstate-0.9.tar.gz (40.5 kB view details)

Uploaded Source

Built Distribution

katsdptelstate-0.9-py2.py3-none-any.whl (43.4 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file katsdptelstate-0.9.tar.gz.

File metadata

  • Download URL: katsdptelstate-0.9.tar.gz
  • Upload date:
  • Size: 40.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/42.0.2 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.7.6

File hashes

Hashes for katsdptelstate-0.9.tar.gz
Algorithm Hash digest
SHA256 5dce2799a1dbfc3ab8e54714991ece13585f5c94366522503d53d39f66f4113a
MD5 0590ffa05971a574fde22ddc7dc8a892
BLAKE2b-256 5164cac272d8d6ccbe9d5c31c62dad895cf4cce264f598177e31e50d634b16df

See more details on using hashes here.

Provenance

File details

Details for the file katsdptelstate-0.9-py2.py3-none-any.whl.

File metadata

  • Download URL: katsdptelstate-0.9-py2.py3-none-any.whl
  • Upload date:
  • Size: 43.4 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/42.0.2 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.7.6

File hashes

Hashes for katsdptelstate-0.9-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 83285b04e5f386d8a7f5b9d63aebad1183d7fb5dac224f1fd54610d5f72e7444
MD5 bd65189c3e247b24af14848e83b136a9
BLAKE2b-256 9f6444e69b62925c0be958dc59c9298b523420c0e53f8026102836175f5f490d

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page