Skip to main content

Take lightning fast snapshots of your local Postgres databases.

Project description



The DSLR logo


PyPI version PyPI Supported Python Versions GitHub Actions (Code quality and tests)

A terminal showing DSLR's command line interface.


Database Snapshot, List, and Restore

Take lightning fast snapshots of your local Postgres databases.

What is this?

DSLR is a tool that allows you to quickly take and restore database snapshots when you're writing database migrations, switching branches, or messing with SQL.

It's meant to be a spiritual successor to Stellar.

Important: DSLR is intended for development use only. It is not advisable to use DSLR on production databases.

Performance

DSLR is much faster than the standard pg_dump/pg_restore approach to snapshots.

A chart comparing the execution time between DSLR and pg_dump/pg_restore. For snapshot and restore, DSLR took 4.125 seconds and 4.431 seconds respectively. pg_dump/pg_restore took 36.602 seconds and 13.257 seconds respectively.

DSLR is 8x faster at taking snapshots and 3x faster at restoring snapshots compared to the pg_dump/pg_restore approach.

Testing methodology

I spun up Postgres 12.3 using Docker, created a test database, and filled it with 1GB of random data using this script:

CREATE TABLE large_test (num1 bigint, num2 double precision, num3 double precision);

INSERT INTO large*test (num1, num2, num3)
SELECT round(random() * 10), random(), random() \_ 142
FROM generate_series(1, 20000000) s(i);

I used the following commands to measure the execution time:


time dslr snapshot my-snapshot
time dslr restore my-snapshot
time pg_dump -Fc -f export.dump
time pg_restore --no-acl --no-owner export.dump

I ran each command three times and plotted the mean in the chart.

Here's the raw data:

Command Run Execution time (seconds)
dslr snapshot 1 4.797
2 4.650
3 2.927
dslr restore 1 5.840
2 4.122
3 3.331
pg_dump 1 37.345
2 36.227
3 36.233
pg_restore 1 13.304
2 13.148
3 13.320

Install


pip install DSLR psycopg2 # or psycopg2-binary

Additionally, the DSLR export and import snapshot commands require pg_dump and pg_restore to be present in your PATH.

Configuration

You can tell DSLR which database to take snapshots of in a few ways:

DATABASE_URL

If the DATABASE_URL environment variable is set, DSLR will use this to connect to your target database.

export DATABASE_URL=postgres://username:password@host:port/database_name

dslr.toml

If a dslr.toml file exists in the current directory, DSLR will read its settings from there. DSLR will prefer this over the environment variable.

url = 'postgres://username:password@host:port/database_name'

--url option

Finally, you can explicitly pass the connection string via the --url option. This will override any of the above settings.

Usage

$ dslr snapshot my-first-snapshot
Created new snapshot my-first-snapshot

$ dslr restore my-first-snapshot
Restored database from snapshot my-first-snapshot

$ dslr list

  Name                Created
 ────────────────────────────────────
  my-first-snapshot   2 minutes ago

$ dslr rename my-first-snapshot fresh-db
Renamed snapshot my-first-snapshot to fresh-db

$ dslr delete some-old-snapshot
Deleted some-old-snapshot

$ dslr export my-feature-test
Exported snapshot my-feature-test to my-feature-test_20220730-075650.dump

$ dslr import snapshot-from-a-friend_20220730-080632.dump friend-snapshot
Imported snapshot friend-snapshot from snapshot-from-a-friend_20220730-080632.dump

How does it work?

DSLR takes snapshots by cloning databases using Postgres' Template Databases functionality. This is the main source of DSLR's speed.

This means that taking a snapshot is just creating a new database using the main database as the template. Restoring a snapshot is just deleting the main database and creating a new database using the snapshot database as the template. So on and so forth.

License

MIT

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

DSLR-0.3.1a0.tar.gz (11.4 kB view details)

Uploaded Source

Built Distribution

DSLR-0.3.1a0-py3-none-any.whl (11.0 kB view details)

Uploaded Python 3

File details

Details for the file DSLR-0.3.1a0.tar.gz.

File metadata

  • Download URL: DSLR-0.3.1a0.tar.gz
  • Upload date:
  • Size: 11.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.10.0

File hashes

Hashes for DSLR-0.3.1a0.tar.gz
Algorithm Hash digest
SHA256 f4d7b184b6cfe21d4d17275a447e58b3ec6bfa76bc00e1fcb2450d42e1a8d653
MD5 d0bce74dd3c0991d582df63c2e24ba17
BLAKE2b-256 8d3466c76e3b9e88e1ff8cef71bb7cc53cf7caf091a86678bab84c315df10d9b

See more details on using hashes here.

File details

Details for the file DSLR-0.3.1a0-py3-none-any.whl.

File metadata

  • Download URL: DSLR-0.3.1a0-py3-none-any.whl
  • Upload date:
  • Size: 11.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.10.0

File hashes

Hashes for DSLR-0.3.1a0-py3-none-any.whl
Algorithm Hash digest
SHA256 0461608fb884e6078afc34cb9c4d4b61db11ff145c1ac2d8b486456ebcc94a8a
MD5 e6ed77c732ff93de73ece3550327c68b
BLAKE2b-256 59440384582e48961fa8c0c72155f4513c23d3a7a332b0a5668f492a844cddce

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page