Skip to main content

A distributed no-master key-value store

Project description

Welcome to DistKV!

DistKV is a master-less distributed key-value storage system. It circumvents the CAP theorem by assuming that keys are usually only changed by one node. It is resistant to partitioning and intended to be always-on; it might delay – but will not lose – updates even in a partitioned network.

DistKV comes with several batteries included:

  • Basic user management, pattern-based ACLs

  • Strong typing, code- and/or JSON Schema-based

  • Data mangling

  • Background code execution

  • Seamless recovery even if only one master is running

  • a MQTT 3.1 back-end that stores persistent data in DistKV, based on hbmqtt

API

DistKV offers an efficient msgpack-based interface to access data and to change internal settings. Most configuration is stored inside DistKV itself.

Stored data are not forced to be strings or binary sequences, but can be anything that MsgPack supports. Keys to storage are multi-level and support string, integer/float, and list keys.

Non-Features

DistKV does not support data partitioning. Every node stores the whole data set and can instantly deliver mostly-uptodate data.

DistKV does not have a disk-based storage backend; periodic snapshots and event logs can be used to quickly restore a system, if necessary.

Status

DistKV is mostly stable. There are a lot of corner cases that don’t have tests yet

TODOs: * some services (esp. command line tools and runners) are under-tested * there’s no good API for errors

Changelog

0.41: the message monitor can do multiple subpaths and only reports initial-load-complete once

0.40: use asyncscope for running subsystems in a reasonable way

0.35: allow forgetting nodes (if they have no data attached)

0.30: major API refactoring: paths are now separate objects

TODO

  • update the whole ecosystem to anyio 2.0 (asyncclick asyncscope …)

  • clean up some of the more egregious command line mistakes

  • create a page for showcase-ing subprojects (distinv knx owfs akumuli …)

  • improve Home Assistant integration

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

distkv-0.65.0.tar.gz (179.5 kB view details)

Uploaded Source

Built Distribution

distkv-0.65.0-py3-none-any.whl (152.1 kB view details)

Uploaded Python 3

File details

Details for the file distkv-0.65.0.tar.gz.

File metadata

  • Download URL: distkv-0.65.0.tar.gz
  • Upload date:
  • Size: 179.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.3.0 pkginfo/1.8.2 requests/2.25.1 setuptools/66.1.1 requests-toolbelt/0.9.1 tqdm/4.57.0 CPython/3.11.2

File hashes

Hashes for distkv-0.65.0.tar.gz
Algorithm Hash digest
SHA256 562dd6b4f6a8c3f372815fe66e4f85ab5ce999d2cefe79966487ca4a558a8ef5
MD5 831fb28401c04c367a45a32fc64eba29
BLAKE2b-256 aa60cc1292bc676e7f622900eb28f6b2a5c45e8ac5d3cd612c3275cf50757f73

See more details on using hashes here.

File details

Details for the file distkv-0.65.0-py3-none-any.whl.

File metadata

  • Download URL: distkv-0.65.0-py3-none-any.whl
  • Upload date:
  • Size: 152.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.3.0 pkginfo/1.8.2 requests/2.25.1 setuptools/66.1.1 requests-toolbelt/0.9.1 tqdm/4.57.0 CPython/3.11.2

File hashes

Hashes for distkv-0.65.0-py3-none-any.whl
Algorithm Hash digest
SHA256 32cbf90e852773e8ab9f334357e7175c1d00ffdf1e54c59c0feea5fbc08e7fd5
MD5 708c866c9c5320c23a6d28cf92d2dc32
BLAKE2b-256 b2c6549caf5c376809e871cf1ed2cf315159c462be0de3fcd33d15288117d5f4

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page