PostgreSQL High-Available orchestrator and CLI

Navigation

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for cyberdemn from gravatar.com cyberdemn Avatar for hugh_capet from gravatar.com hugh_capet

Unverified details

These details have not been verified by PyPI
Meta
  • License: MIT License (The MIT License)
  • Author: Alexander Kukushkin, Polina Bungina
  • Tags etcd, governor, patroni, postgresql, postgres, ha, haproxy, confd, zookeeper, exhibitor, consul, streaming, replication, kubernetes, k8s
  • Provides-Extra: aws, etcd, etcd3, consul, exhibitor, zookeeper, kubernetes, raft, jsonlogger, all, psycopg2, psycopg2-binary, psycopg3
Classifiers

Project description

Tests Status Coverage Status

Patroni: A Template for PostgreSQL HA with ZooKeeper, etcd or Consul

You can find a version of this documentation that is searchable and also easier to navigate at patroni.readthedocs.io.

There are many ways to run high availability with PostgreSQL; for a list, see the PostgreSQL Documentation.

Patroni is a template for high availability (HA) PostgreSQL solutions using Python. For maximum accessibility, Patroni supports a variety of distributed configuration stores like ZooKeeper, etcd, Consul or Kubernetes. Database engineers, DBAs, DevOps engineers, and SREs who are looking to quickly deploy HA PostgreSQL in datacenters - or anywhere else - will hopefully find it useful.

We call Patroni a “template” because it is far from being a one-size-fits-all or plug-and-play replication system. It will have its own caveats. Use wisely.

Currently supported PostgreSQL versions: 9.3 to 17.

Note to Citus users: Starting from 3.0 Patroni nicely integrates with the Citus database extension to Postgres. Please check the Citus support page in the Patroni documentation for more info about how to use Patroni high availability together with a Citus distributed cluster.

Note to Kubernetes users: Patroni can run natively on top of Kubernetes. Take a look at the Kubernetes chapter of the Patroni documentation.

How Patroni Works

Patroni (formerly known as Zalando’s Patroni) originated as a fork of Governor, the project from Compose. It includes plenty of new features.

For additional background info, see:

Development Status

Patroni is in active development and accepts contributions. See our Contributing section below for more details.

We report new releases information here.

Community

There are two places to connect with the Patroni community: on github, via Issues and PRs, and on channel #patroni in the PostgreSQL Slack. If you’re using Patroni, or just interested, please join us.

Technical Requirements/Installation

Pre-requirements for Mac OS

To install requirements on a Mac, run the following:

brew install postgresql etcd haproxy libyaml python

Psycopg

Starting from psycopg2-2.8 the binary version of psycopg2 will no longer be installed by default. Installing it from the source code requires C compiler and postgres+python dev packages. Since in the python world it is not possible to specify dependency as psycopg2 OR psycopg2-binary you will have to decide how to install it.

There are a few options available:

  1. Use the package manager from your distro

sudo apt-get install python3-psycopg2  # install psycopg2 module on Debian/Ubuntu
sudo yum install python3-psycopg2      # install psycopg2 on RedHat/Fedora/CentOS

  1. Specify one of psycopg, psycopg2, or psycopg2-binary in the list of dependencies when installing Patroni with pip (see below).

General installation for pip

Patroni can be installed with pip:

pip install patroni[dependencies]

where dependencies can be either empty, or consist of one or more of the following:

etcd or etcd3

python-etcd module in order to use Etcd as DCS

consul

py-consul module in order to use Consul as DCS

zookeeper

kazoo module in order to use Zookeeper as DCS

exhibitor

kazoo module in order to use Exhibitor as DCS (same dependencies as for Zookeeper)

kubernetes

kubernetes module in order to use Kubernetes as DCS in Patroni

raft

pysyncobj module in order to use python Raft implementation as DCS

aws

boto3 in order to use AWS callbacks

all

all of the above (except psycopg family)

psycopg3

psycopg[binary]>=3.0.0 module

psycopg2

psycopg2>=2.5.4 module

psycopg2-binary

psycopg2-binary module

For example, the command in order to install Patroni together with psycopg3, dependencies for Etcd as a DCS, and AWS callbacks is:

pip install patroni[psycopg3,etcd3,aws]

Note that external tools to call in the replica creation or custom bootstrap scripts (i.e. WAL-E) should be installed independently of Patroni.

Running and Configuring

To get started, do the following from different terminals:

> etcd --data-dir=data/etcd --enable-v2=true
> ./patroni.py postgres0.yml
> ./patroni.py postgres1.yml

You will then see a high-availability cluster start up. Test different settings in the YAML files to see how the cluster’s behavior changes. Kill some of the components to see how the system behaves.

Add more postgres*.yml files to create an even larger cluster.

Patroni provides an HAProxy configuration, which will give your application a single endpoint for connecting to the cluster’s leader. To configure, run:

> haproxy -f haproxy.cfg
> psql --host 127.0.0.1 --port 5000 postgres

YAML Configuration

Go here for comprehensive information about settings for etcd, consul, and ZooKeeper. And for an example, see postgres0.yml.

Environment Configuration

Go here for comprehensive information about configuring(overriding) settings via environment variables.

Replication Choices

Patroni uses Postgres’ streaming replication, which is asynchronous by default. Patroni’s asynchronous replication configuration allows for maximum_lag_on_failover settings. This setting ensures failover will not occur if a follower is more than a certain number of bytes behind the leader. This setting should be increased or decreased based on business requirements. It’s also possible to use synchronous replication for better durability guarantees. See replication modes documentation for details.

Applications Should Not Use Superusers

When connecting from an application, always use a non-superuser. Patroni requires access to the database to function properly. By using a superuser from an application, you can potentially use the entire connection pool, including the connections reserved for superusers, with the superuser_reserved_connections setting. If Patroni cannot access the Primary because the connection pool is full, behavior will be undesirable.

Project details

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for cyberdemn from gravatar.com cyberdemn Avatar for hugh_capet from gravatar.com hugh_capet

Unverified details

These details have not been verified by PyPI
Meta
  • License: MIT License (The MIT License)
  • Author: Alexander Kukushkin, Polina Bungina
  • Tags etcd, governor, patroni, postgresql, postgres, ha, haproxy, confd, zookeeper, exhibitor, consul, streaming, replication, kubernetes, k8s
  • Provides-Extra: aws, etcd, etcd3, consul, exhibitor, zookeeper, kubernetes, raft, jsonlogger, all, psycopg2, psycopg2-binary, psycopg3
Classifiers

Release history Release notifications | RSS feed

This version

4.0.4

4.0.3

4.0.2

4.0.1

4.0.0 yanked

Reason this release was yanked:

Bug

3.3.5

3.3.4

3.3.3

3.3.2

3.3.1

3.3.0

3.2.2

3.2.1

3.2.0

3.1.2

3.1.1

3.1.0

3.0.4

3.0.3

3.0.2

3.0.1

3.0.0

2.1.7

2.1.6

2.1.5

2.1.4

2.1.3

2.1.2

2.1.1

2.1.0

2.0.2

2.0.1

2.0.0

1.6.5

1.6.4

1.6.3

1.6.1

1.6.0

1.5.6

1.5.5

1.5.4

1.5.3

1.5.2

1.5.1

1.5.0

1.4.6

1.4.4

1.4.3

1.4.2

1.4.1

1.4

1.3.6

1.3.5

1.3.4

1.3.3

1.3.2

1.3.1

1.3

1.2.5

1.2.4

1.2.3

1.2.2

1.2.1

1.2

1.2rc1 pre-release

1.1

1.1rc1 pre-release

1.0

1.0rc1 pre-release

0.90

0.80

0.76

0.75

0.7

0.6

0.2

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

patroni-4.0.4.tar.gz (431.0 kB view details)

Uploaded Source

Built Distribution

patroni-4.0.4-py3-none-any.whl (361.0 kB view details)

Uploaded Python 3

File details

Details for the file patroni-4.0.4.tar.gz.

File metadata

  • Download URL: patroni-4.0.4.tar.gz
  • Upload date:
  • Size: 431.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.1.0 CPython/3.12.7

File hashes

Hashes for patroni-4.0.4.tar.gz
Algorithm Hash digest
SHA256 77e4a7990b18982c67db369bc4dbd3790ba4c1cd177b28d2d5253355e5187ff3
MD5 56ceb706149e402d0c958c1a0f8339e1
BLAKE2b-256 a674a115997502a46bbccc8a8eb59e06e6dba96152a2af5c7d05aedd8d560ad3

See more details on using hashes here.

File details

Details for the file patroni-4.0.4-py3-none-any.whl.

File metadata

  • Download URL: patroni-4.0.4-py3-none-any.whl
  • Upload date:
  • Size: 361.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.1.0 CPython/3.12.7

File hashes

Hashes for patroni-4.0.4-py3-none-any.whl
Algorithm Hash digest
SHA256 94e087876433b29887ba68b108bb89ac0c339c36ef9eedd21007c55a95bd7358
MD5 5e27aeed3525a89d0e92265f145b9340
BLAKE2b-256 5b03077ebfb3ac56d116ff0e7ed3ed9ec66c6919be531fad26101db073a33600

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page