Skip to main content

A program to create deterministic zip files.

Project description

deterministic_zip

A tool to generate consistent zip files.

This tool was specifically built to prevent zip file changes from triggering AWS Lambda function updates when running terraform apply. Before this change, every re-build of the zipfile would result in a different zip even if its contents had not changed.

Requirements

This tool currently only runs on python3.7 in order to use deflate compression level 9.

Installation

pip install deterministic_zip

Creating a deterministic_zip

Run the following, and verify that your zip produces the same sha256 hash:

echo "The first file." > first
echo "The second file." > second
deterministic_zip archive.zip first second
sha256sum archive.zip

If you have \n line endings the result should be:

3afbd7c9b42bd5539ffd5c40499d3d1825157ed83791dce8d7ff2694189d28d6

If you have \r\n line endings (Windows) the result should be:

40e16270d62f15e7a192e88b1b301fa6540c86e7e897036b56be513341d376ed

How does it work?

Great question! There are three tricks to building a deterministic zip.

  1. Files must be added to the zip in the same order. Directory iteration order may vary across machines, resulting in different zips. deterministic_zip sorts all files before adding them to the zip archive.

  2. Files in the zip must have consistent timestamps. If I share a directory to another machine, the timestamps of individual files may differ, depsite identical content. To achieve timestamp consistency, deterministic_zip sets the timestamp of all added files to 2019-01-01 00:00:00. Please note that this does not affect the timestamp of the source files.

  3. Files in the zip must have consistent permissions. File permissions look like -rw-r--r-- for a file that is readable by all users, and only writable by the user who owns the file. Similarly executable files might have permissions that look like: -rwxr-xr-x or -rwx------. deterministic_zip sets the permission of all files to either -r--r--r--, or -r-xr-xr-x. The latter is only used of the user running deterministic_zip has execute access on the file.

Project details


Release history Release notifications | RSS feed

This version

0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

deterministic_zip-0.1.tar.gz (3.9 kB view details)

Uploaded Source

Built Distribution

deterministic_zip-0.1-py3-none-any.whl (4.3 kB view details)

Uploaded Python 3

File details

Details for the file deterministic_zip-0.1.tar.gz.

File metadata

  • Download URL: deterministic_zip-0.1.tar.gz
  • Upload date:
  • Size: 3.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.7.2

File hashes

Hashes for deterministic_zip-0.1.tar.gz
Algorithm Hash digest
SHA256 d0095bafb65a7272ea9742c35d9fd3305dc5f1011b0f37addcf3fb95e7606100
MD5 8e3871b6c9705a683a40b9fb2e8fd4ff
BLAKE2b-256 045a297b8b79061e9ac3b1c23cc238c78c89bca8b07379864d37ef172fa182de

See more details on using hashes here.

File details

Details for the file deterministic_zip-0.1-py3-none-any.whl.

File metadata

  • Download URL: deterministic_zip-0.1-py3-none-any.whl
  • Upload date:
  • Size: 4.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.7.2

File hashes

Hashes for deterministic_zip-0.1-py3-none-any.whl
Algorithm Hash digest
SHA256 a16dc9fdfd130849e09adb5bc9c3e9ec8f3de109d405de04cb670cf73bb7c84a
MD5 638d96bb56b527ec07f74bef82c317f7
BLAKE2b-256 7a297e6db311798cf302d28312673c8d0e79e4bfa3d1be7cc46b94a45589d564

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page