buttervolume 1.1

Running the plugin

If you installed it locally, You can start the plugin with:

$ sudo buttervolume run

If you’re running it in a privileged container, it will be automatically started.

When started it will create a unix socket /var/run/docker/plugins/btrfs.sock for use by Docker. The name of the socket file is actually the name of the plugin you can use with docker volume create -d <driver> or docker create --volume-driver=<driver>. when started, the plugin will also start its own scheduler to run periodic jobs (such as a snapshot, replication, purge or synchronization)

Creating and deleting volumes

Once the plugin is running, whenever you create a container you can specify the volume driver with docker create --volume-driver=btrfs --name <name> <image>. You can also manually create a BTRFS volume with docker volume create -d btrfs. It also works with docker-compose, by specifying the btrfs driver in the volumes section of the compose file.

When you delete the volume with docker rm -v <container> or docker volume rm <volume>, the BTRFS subvolume is deleted. If you snapshotted the volume elsewhere in the meantime, the snapshots won’t be deleted.

Managing volumes and snapshots

When buttervolume is installed, it provides a command line tool buttervolume, with the following subcommands:

run                 Run the plugin in foreground
snapshot            Snapshot a volume
snapshots           List snapshots
schedule            (un)Schedule a snapshot, replication or purge
scheduled           List scheduled actions
restore             Restore a snapshot (optionally to a different volume)
send                Send a snapshot to another host
sync                Synchronise a volume from a remote host volume
rm                  Delete a snapshot
purge               Purge old snapshot using a purge pattern

Create a snapshot

You can create a readonly snapshot of the volume with:

$ buttervolume snapshot <volume>

The volumes are currently expected to live in /var/lib/docker/volumes and the snapshot will be created in /var/lib/docker/snapshots, by appending the datetime to the name of the volume, separated with @.

List the snapshots

You can list all the snapshots:

$ buttervolume snapshots

or just the snapshots corresponding to a volume with:

$ buttervolume snapshots <volume>

<volume> is the name of the volume, not the full path. It is expected to live in /var/lib/docker/volumes.

Restore a snapshot

You can restore a snapshot as a volume. The current volume will first be snapshotted, deleted, then replaced with the snapshot. If you provide a volume name instead of a snapshot, the latest snapshot is restored. So no data is lost if you do something wrong. Please take care of stopping the container before restoring a snapshot:

$ buttervolume restore <snapshot>

<snapshot> is the name of the snapshot, not the full path. It is expected to live in /var/lib/docker/snapshots.

By default, the volume name corresponds to the volume the snapshot was created from. But you can optionally restore the snapshot to a different volume name by adding the target as the second argument:

$ buttervolume restore <snapshot> <volume>

Delete a snapshot

You can delete a snapshot with:

$ buttervolume rm <snapshot>

<snapshot> is the name of the snapshot, not the full path. It is expected to live in /var/lib/docker/snapshots.

Replicate a snapshot to another host

You can incrementally send snapshots to another host, so that data is replicated to several machines, allowing to quickly move a stateful docker container to another host. The first snapshot is first sent as a whole, then the next snapshots are used to only send the difference between the current one and the previous one. This allows to replicate snapshots very often without consuming a lot of bandwith or disk space:

$ buttervolume send <host> <snapshot>

<snapshot> is the name of the snapshot, not the full path. It is expected to live in /var/lib/docker/snapshots and is replicated to the same path on the remote host.

<host> is the hostname or IP address of the remote host. The snapshot is currently sent using BTRFS send/receive through ssh. This requires that ssh keys be present and already authorized on the target host, and that the StrictHostKeyChecking no option be enabled in ~/.ssh/config.

Synchronize a volume from another host volume

You can receive data from a remote volume, so in case there is a volume on the remote host with the same name, it will get new and most recent data from the distantant volume and replace in the local volume. Before running the rsync command a snapshot is made on the locale machine to manage recovery:

$ buttervolume sync <volume> <host1> [<host2>][...]

The intent is to synchronize a volume between multi hosts on running containers, so you should schedule that action on each nodes from all remote hosts.

Purge old snapshots

You can purge old snapshot corresponding to the specified volume, using a retention pattern:

$ buttervolume purge <pattern> <volume>

If you’re unsure whether you retention pattern is correct, you can run the purge with the --dryrun option, to inspect what snapshots would be deleted, without deleting them:

$ buttervolume purge --dryrun <pattern> <volume>

<volume> is the name of the volume, not the full path. It is expected to live in /var/lib/docker/volumes.

<pattern> is the snapshot retention pattern. It is a semicolon-separated list of time length specifiers with a unit. Units can be m for minutes, h for hours, d for days, w for weeks, y for years. The pattern should have at least 2 items.

Here are a few examples of retention patterns:

• 4h:1d:2w:2y

  Keep all snapshots in the last four hours, then keep only one snapshot every four hours during the first day, then one snapshot per day during the first two weeks, then one snapshot every two weeks during the first two years, then delete everything after two years.

• 4h:1w

  keep all snapshots during the last four hours, then one snapshot every four hours during the first week, then delete older snapshots.

• 2h:2h

  keep all snapshots during the last two hours, then delete older snapshots.

Schedule a job

You can schedule a periodic job, such as a snapshot, a replication, a synchronization or a purge. The schedule it self is stored in /etc/buttervolume/schedule.csv.

Schedule a snapshot of a volume every 60 minutes:

$ buttervolume schedule snapshot 60 <volume>

Remove the same schedule by specifying a timer of 0 min:

$ buttervolume schedule snapshot 0 <volume>

Schedule a replication of volume foovolume to remote_host:

$ buttervolume schedule replicate:remote_host 3600 foovolume

Remove the same schedule:

$ buttervolume schedule replicate:remote_host 0 foovolume

Schedule a purge every hour of the snapshots of volume foovolume, but keep all the snapshots in the last 4 hours, then only one snapshot every 4 hours during the first week, then one snapshot every week during one year, then delete all snapshots after one year:

$ buttervolume schedule purge:4h:1w:1y 60 foovolume

Remove the same schedule:

$ buttervolume schedule purge:4h:1w:1y 0 foovolume

Using the right combination of snapshot schedule timer, purge schedule timer and purge retention pattern, you can create you own backup strategy, from the simplest ones to more elaborate ones. A common one is the following:

$ buttervolume schedule snapshot 1440 <volume>
$ buttervolume schedule purge:1d:4w:1y 1440 <volume>

It should create a snapshot every day, then purge snapshots everydays while keeping all snapshots in the last 24h, then one snapshot per day during one month, then one snapshot per month during only one year.

Schedule a syncrhonization of volume foovolume from remote_host1 abd remote_host2:

$ buttervolume schedule synchronize:remote_host1,remote_host2 60 foovolume

Remove the same schedule:

$ buttervolume schedule synchronize:remote_host1,remote_host2 0 foovolume

List scheduled jobs

You can list all the scheduled job with:

$ buttervolume scheduled

It will display the schedule in the same format used for adding the schedule, which is convenient to remove an existing schedule or add a similar one.

Disabling copy-on-write

UPDATE: Copy On Write is disabled by default.

TODO: replace the .nocow file feature with an option to pass

With buttervolume you can disable copy-on-write in a volume by creating a .nocow file at the root of the volume. The buttervolume plugin will detect it at mount-time and apply chattr +C on the volume root.

Why disabling copy-on-write? If your docker volume stores databases such as PostgreSQL or MariaDB, the copy-on-write feature may hurt performance a lot. The good news is that disabling copy-on-write does not prevent from doing snaphots, so we get the best of both world: good performances with the ability to do snapshots.

Creating such a .nocow file can easily be done in a Dockerfile, before the VOLUME command:

RUN mkdir -p /var/lib/postgresql/data \
    && chown -R postgres: /var/lib/postgresql/data \
    && touch /var/lib/postgresql/data/.nocow
VOLUME /var/lib/postgresql/data

Alternatively you can create the .nocow file just after the docker create command, by inspecting the location of the created volumes with docker inspect container | grep volumes.