wagtail-storages 0.0.4

WAGTAIL_STORAGES_DOCUMENTS_FRONTENDCACHE

Using the same format as Wagtail’s WAGTAILFRONTENDCACHE setting, but to be only used by the wagtail-storages to purge the documents. If not set, the purge won’t happen. Read more on how to format it in the Wagtail docs, e.g.

WAGTAIL_STORAGES_DOCUMENTS_FRONTENDCACHE = {
    'cloudfront': {
        'BACKEND': 'wagtail.contrib.frontend_cache.backends.CloudfrontBackend',
        'DISTRIBUTION_ID': 'your-distribution-id',
     },
}

WAGTAIL_STORAGES_DOCUMENT_HOOK_ORDER

Set a custom order for the document hook order. It’s set to 100 by default. It’s important that it runs after any of your hooks since it returns a response, e.g.

WAGTAIL_STORAGES_DOCUMENT_HOOK_ORDER = 900

Set up S3 bucket

First, set up your S3 bucket. It must be configured to:

• Have a name that matches the domain name, e.g. media.llamasavers.com.

• Allow the user to perform the following actions on the bucket:

  • s3:ListBucket

  • s3:GetBucketLocation

  • s3:ListBucketMultipartUploads

  • s3:ListBucketVersions

• Allow the user to perform all the actions (s3:*) on the objects within the bucket.

• Allow the internet traffic to access Wagtail image renditions (images/*).

The user permissions can be set in the IAM or via a bucket policy. See example of all of those points being achieved in the bucket policy below.

{
     "Version": "2012-10-17",
     "Statement": [
         {
             "Sid": "PublicGetObject",
             "Effect": "Allow",
             "Principal": "*",
             "Action": "s3:GetObject",
             "Resource": "arn:aws:s3:::[BUCKET NAME]/images/*"
         },
         {
             "Sid": "AllowUserManageBucket",
             "Effect": "Allow",
             "Principal": {
                 "AWS": "arn:aws:iam::[USER ARN]"
             },
             "Action": [
                 "s3:ListBucket",
                 "s3:GetBucketLocation",
                 "s3:ListBucketMultipartUploads",
                 "s3:ListBucketVersions"
             ],
             "Resource": "arn:aws:s3:::[BUCKET NAME]"
         },
         {
             "Sid": "AllowUserManageBucketObjects",
             "Effect": "Allow",
             "Principal": {
                 "AWS": "arn:aws:iam::[USER ARN]"
             },
             "Action": "s3:*",
             "Resource": "arn:aws:s3:::[BUCKET NAME]/*"
         }
     ]
 }

After the S3 bucket is set up on AWS, you can configure the Wagtail project to use it.

Set up django-storages

Install django-storages and boto3.

pip install django-storages[boto3]

Set up your S3 bucket with django-storages. The following code allows configuration via environment variables.

# settings.py
import os


if "AWS_STORAGE_BUCKET_NAME" in os.environ:
    # Add django-storages to the installed apps
    INSTALLED_APPS = INSTALLED_APPS + ["storages"]

    # https://docs.djangoproject.com/en/stable/ref/settings/#default-file-storage
    DEFAULT_FILE_STORAGE = "storages.backends.s3boto3.S3Boto3Storage"

    AWS_STORAGE_BUCKET_NAME = os.environ["AWS_STORAGE_BUCKET_NAME"]

    # Disables signing of the S3 objects' URLs. When set to True it
    # will append authorization querystring to each URL.
    AWS_QUERYSTRING_AUTH = False

    # Do not allow overriding files on S3 as per Wagtail docs recommendation:
    # https://docs.wagtail.io/en/stable/advanced_topics/deploying.html#cloud-storage
    # Not having this setting may have consequences such as losing files.
    AWS_S3_FILE_OVERWRITE = False

    # Default ACL for new files should be "private" - not accessible to the
    # public. Images should be made available to public via the bucket policy,
    # where the documents should use wagtail-storages.
    AWS_DEFAULT_ACL = "private"

    # We generally use this setting in production to put the S3 bucket
    # behind a CDN using a custom domain, e.g. media.llamasavers.com.
    # https://django-storages.readthedocs.io/en/latest/backends/amazon-S3.html#cloudfront
    if "AWS_S3_CUSTOM_DOMAIN" in os.environ:
        AWS_S3_CUSTOM_DOMAIN = os.environ["AWS_S3_CUSTOM_DOMAIN"]

    # When signing URLs is enabled, the region must be set.
    # The global S3 endpoint does not seem to support signed URLS.
    # Set this only if you will be using signed URLs.
    if "AWS_S3_REGION_NAME" in os.environ:
        AWS_S3_REGION_NAME = os.environ["AWS_S3_REGION_NAME"]

    # This settings lets you force using http or https protocol when generating
    # the URLs to the files. Set https as default.
    # https://github.com/jschneier/django-storages/blob/10d1929de5e0318dbd63d715db4bebc9a42257b5/storages/backends/s3boto3.py#L217
    AWS_S3_URL_PROTOCOL = os.environ.get("AWS_S3_URL_PROTOCOL", "https:")

If you use the above snippet, you can set the following environment variables:

• AWS_STORAGE_BUCKET_NAME - set to media.llamasavers.com.

• AWS_S3_CUSTOM_DOMAIN - set to media.llamasavers.com.

• AWS_S3_REGION_NAME - set to your AWS region name, e.g. eu-west-2.

You can use one of the methods to provide boto3 with credentials. We suggest you stick with the environment variables. To do that, you need to set the following variables:

• AWS_ACCESS_KEY_ID

• AWS_SECRET_ACCESS_KEY

Now the storage should be configured and working. Editors should be able to upload images and documents in Wagtail admin.

Set up wagtail-storages

Install wagtail-storages itself.

pip install wagtail-storages

Add wagtail_storages to your INSTALLED_APPS in your settings file.

# settings.py

INSTALLED_APPS = [
    # ... Other apps
    "wagtail_storages",
    # ... Other apps
]

With that, ACLs should be updated if documents are moved to private collections.

If you already have files in your S3 bucket, run django-admin fix_document_acls to make sure all documents have the right ACLs set up.

Set up front-end cache invalidation

If edge cache is set up on the custom domain (media.llamasavers.com) you should set up the CDN purging to avoid having outdated or private documents available to users via the CDN endpoint. For example, for Cloudflare you want to use a configuration similar to the one below:

# settings.py
import os


if "S3_CACHE_CLOUDFLARE_TOKEN" in os.environ:
     WAGTAIL_STORAGES_DOCUMENTS_FRONTENDCACHE = {
         "default": {
             "BACKEND": "wagtail.contrib.frontend_cache.backends.CloudflareBackend",
             "EMAIL": os.environ["S3_CACHE_CLOUDFLARE_EMAIL"],
             "TOKEN": os.environ["S3_CACHE_CLOUDFLARE_TOKEN"],
             "ZONEID": os.environ["S3_CACHE_CLOUDFLARE_ZONEID"],
         },
     }

Then set the following environment variables:

• S3_CACHE_CLOUDFLARE_EMAIL

• S3_CACHE_CLOUDFLARE_TOKEN

• S3_CACHE_CLOUDFLARE_ZONEID

Once set up, the documents will be purged from cache when they are modified or their privacy settings have changed.

The setting follows configuration format of the front-end cache invalidator configuration in Wagtail. See the details here. The only difference is the setting name, which for wagtail-storages is WAGTAIL_STORAGES_DOCUMENTS_FRONTENDCACHE.

All done!