A program to create deterministic zip files.
Project description
deterministic_zip
A tool to generate consistent zip files.
This tool was specifically built to prevent zip file changes from triggering
AWS Lambda function updates when running terraform apply
. Before this change,
every re-build of the zipfile would result in a different zip even if its
contents had not changed.
Requirements
This tool currently only runs on python3.7 in order to use deflate compression level 9.
Installation
pip install deterministic_zip
Creating a deterministic_zip
Run the following, and verify that your zip produces the same sha256 hash:
echo "The first file." > first
echo "The second file." > second
deterministic_zip archive.zip first second
sha256sum archive.zip
If you have \n
line endings the result should be:
3afbd7c9b42bd5539ffd5c40499d3d1825157ed83791dce8d7ff2694189d28d6
If you have \r\n
line endings (Windows) the result should be:
40e16270d62f15e7a192e88b1b301fa6540c86e7e897036b56be513341d376ed
How does it work?
Great question! There are three tricks to building a deterministic zip.
-
Files must be added to the zip in the same order. Directory iteration order may vary across machines, resulting in different zips.
deterministic_zip
sorts all files before adding them to the zip archive. -
Files in the zip must have consistent timestamps. If I share a directory to another machine, the timestamps of individual files may differ, depsite identical content. To achieve timestamp consistency,
deterministic_zip
sets the timestamp of all added files to2019-01-01 00:00:00
. Please note that this does not affect the timestamp of the source files. -
Files in the zip must have consistent permissions. File permissions look like
-rw-r--r--
for a file that is readable by all users, and only writable by the user who owns the file. Similarly executable files might have permissions that look like:-rwxr-xr-x
or-rwx------
.deterministic_zip
sets the permission of all files to either-r--r--r--
, or-r-xr-xr-x
. The latter is only used of the user runningdeterministic_zip
has execute access on the file.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file deterministic_zip-0.1.tar.gz
.
File metadata
- Download URL: deterministic_zip-0.1.tar.gz
- Upload date:
- Size: 3.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.7.2
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | d0095bafb65a7272ea9742c35d9fd3305dc5f1011b0f37addcf3fb95e7606100 |
|
MD5 | 8e3871b6c9705a683a40b9fb2e8fd4ff |
|
BLAKE2b-256 | 045a297b8b79061e9ac3b1c23cc238c78c89bca8b07379864d37ef172fa182de |
File details
Details for the file deterministic_zip-0.1-py3-none-any.whl
.
File metadata
- Download URL: deterministic_zip-0.1-py3-none-any.whl
- Upload date:
- Size: 4.3 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.7.2
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | a16dc9fdfd130849e09adb5bc9c3e9ec8f3de109d405de04cb670cf73bb7c84a |
|
MD5 | 638d96bb56b527ec07f74bef82c317f7 |
|
BLAKE2b-256 | 7a297e6db311798cf302d28312673c8d0e79e4bfa3d1be7cc46b94a45589d564 |