Extract Deflate64 ZIP archives with Python's zipfile API.

These details have not been verified by PyPI

Project links

Project description

zipfile-deflate64

Extract Deflate64 ZIP archives with Python's zipfile API.

Installation

pip install zipfile-deflate64

Python 3.6, 3.7, 3.8, 3.9, and 3.10 are supported, with manylinux2014, macOS and Windows wheels published to PyPI.

Usage

Anywhere in a Python codebase:

import zipfile_deflate64  # This has the side effect of patching the zipfile module to support Deflate64

Alternatively, zipfile_deflate64 re-exports the zipfile API, as a convenience:

import zipfile_deflate64 as zipfile

zipfile.ZipFile(...)
...

Design Rationale

The Problem

Recent versions of Microsoft Windows Explorer use Deflate64 compression when creating ZIP files larger than 2GB. With the ubiquity of Windows and the ease of using "Sent to compressed folder", a majority of newly-created large ZIP files use Deflate64 compression.

However, support for Deflate64 in the open-source ecosystem is very poor! Most ZIP libraries have declined to implement Deflate64, citing its proprietary nature.

In the .NET ecosystem, the ZipArchive API supports decompression only. In Java, the Apache Commons Compress APIs support both compression and decompression.

The 7-Zip project probably provides the best general-purpose support for compressing and decompressing Deflate64, but there are several obstacles to general usability:

7-Zip itself is a Windows-only GUI application
- 7-Zip is still issuing new releases, but has declined to implement certain new compression formats, so the mcmilk/7-Zip-zstd fork is notable.
p7zip, the POSIX-compatible CLI version (which does include Deflate64), has not had a release since 2016 and is likely unmaintained.
p7zip does not build an API for external software to invoke for decompression.
p7zip seems to now be living on as the jinfeihan57/p7zip fork, which is packaged by Arch Linux, amongst others.
- This seems to be active, and now can be built with CMake, but there's no support for building an external API.
Many re-implementations of 7-Zip, such as py7zr for Python, do not support Deflate64.

In the Python ecosystem in particular, there have been several unfulfilled requests ( [1] [2] [3] ) for Deflate64 decompression support.

A Solution

The best hope seems to be the infback9 extension to zlib. This was developed in 2003 by Mark Adler, an original author of zlib, and is kept in the source repository of zlib, but it is not officially supported and contains no build tooling and is not distributed with zlib packages. Additionally, infback9 provides only low-level support for working with Deflate64 bitstreams, with no support for the ZIP archive format (which is out of scope for zlib).

infback9's C-language API is relatively simple, but requires a non-trivial struct and function pointers for initialization and some explicit memory management operations (resizing allocated buffers and proving a Python-friendly malloc) to operate efficiently, so wrapping it with only ctypes seems to be inadequate.

To manage ZIP archive extraction operations, the Python standard library zipfile module provides the essential features and is already ubiquitous in availability and usage. However, zipfile is difficult to extend, as it hardcodes many conditionals for compression formats and does not provide capabilities for easily augmenting or replacing parts of it. Monkey-patching can overcome some of these problems, and the promise of a drop-in, API-compatible patch to a standard library module outweighed the engineering benefits of basing a solution off a more naturally extensible third-party ZIP manipulation package.

Project details

These details have not been verified by PyPI

Project links

Release history Release notifications | RSS feed

This version

0.2.0

Jan 5, 2022

0.1.6

Feb 28, 2021

0.1.5

Feb 26, 2021

0.1.4

Feb 24, 2021

0.1.3

Feb 24, 2021

0.1.2

Feb 24, 2021

0.1.1

Feb 24, 2021

0.1.0

Feb 24, 2021

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

zipfile-deflate64-0.2.0.tar.gz (62.9 kB view details)

Uploaded Jan 5, 2022 Source

Built Distributions

zipfile_deflate64-0.2.0-cp310-cp310-win_amd64.whl (20.2 kB view details)

Uploaded Jan 5, 2022 CPython 3.10 Windows x86-64

zipfile_deflate64-0.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (43.5 kB view details)

Uploaded Jan 5, 2022 CPython 3.10 manylinux: glibc 2.17+ x86-64

zipfile_deflate64-0.2.0-cp310-cp310-macosx_10_14_x86_64.whl (19.2 kB view details)

Uploaded Jan 5, 2022 CPython 3.10 macOS 10.14+ x86-64

zipfile_deflate64-0.2.0-cp39-cp39-win_amd64.whl (20.2 kB view details)

Uploaded Jan 5, 2022 CPython 3.9 Windows x86-64

zipfile_deflate64-0.2.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (43.3 kB view details)

Uploaded Jan 5, 2022 CPython 3.9 manylinux: glibc 2.17+ x86-64

zipfile_deflate64-0.2.0-cp39-cp39-macosx_10_14_x86_64.whl (19.2 kB view details)

Uploaded Jan 5, 2022 CPython 3.9 macOS 10.14+ x86-64

zipfile_deflate64-0.2.0-cp38-cp38-win_amd64.whl (20.2 kB view details)

Uploaded Jan 5, 2022 CPython 3.8 Windows x86-64

zipfile_deflate64-0.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (44.1 kB view details)

Uploaded Jan 5, 2022 CPython 3.8 manylinux: glibc 2.17+ x86-64

zipfile_deflate64-0.2.0-cp38-cp38-macosx_10_14_x86_64.whl (19.2 kB view details)

Uploaded Jan 5, 2022 CPython 3.8 macOS 10.14+ x86-64

zipfile_deflate64-0.2.0-cp37-cp37m-win_amd64.whl (20.2 kB view details)

Uploaded Jan 5, 2022 CPython 3.7m Windows x86-64

zipfile_deflate64-0.2.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (43.5 kB view details)

Uploaded Jan 5, 2022 CPython 3.7m manylinux: glibc 2.17+ x86-64

zipfile_deflate64-0.2.0-cp37-cp37m-macosx_10_14_x86_64.whl (19.1 kB view details)

Uploaded Jan 5, 2022 CPython 3.7m macOS 10.14+ x86-64

zipfile_deflate64-0.2.0-cp36-cp36m-win_amd64.whl (20.5 kB view details)

Uploaded Jan 5, 2022 CPython 3.6m Windows x86-64

zipfile_deflate64-0.2.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (43.7 kB view details)

Uploaded Jan 5, 2022 CPython 3.6m manylinux: glibc 2.17+ x86-64

zipfile_deflate64-0.2.0-cp36-cp36m-macosx_10_14_x86_64.whl (19.2 kB view details)

Uploaded Jan 5, 2022 CPython 3.6m macOS 10.14+ x86-64

File details

Details for the file zipfile-deflate64-0.2.0.tar.gz.

File metadata

Download URL: zipfile-deflate64-0.2.0.tar.gz
Upload date: Jan 5, 2022
Size: 62.9 kB
Tags: Source
Uploaded using Trusted Publishing? No
Uploaded via: twine/3.7.1 importlib_metadata/4.10.0 pkginfo/1.8.2 requests/2.27.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.10.1

File hashes

Hashes for zipfile-deflate64-0.2.0.tar.gz
Algorithm	Hash digest
SHA256	`875a3299de102edf1c17f8cafcc528b1ca80b62dc4814b9cb56867ec59fbfd18`
MD5	`9b547b0494c824abe7d0d7d84eeb94d3`
BLAKE2b-256	`031bf397f821b48156ee94c5ca7ad82dc5cdb73cbcbc4377b9c1b21556f3ce8c`

Algorithm	Hash digest
SHA256	`f868343cd24bd3c66fbcba9316c6a970f934653bf3d0be89f25fa0335a7ea3ff`
MD5	`d53221735fd363f9880948da44aba971`
BLAKE2b-256	`247cd6cfafa8a4d33936b666c897a981c49f264864d1941d8fa7f1da516cfa41`

Algorithm	Hash digest
SHA256	`b1b4ab8d83bb277983ff273cbf4fcf831023abbc2303f90af9dd4bde3ab4b9c2`
MD5	`10f57d8b449997c7eaab4d34b9ea8d7b`
BLAKE2b-256	`ca74849f94131522ddc7961a598b422aaa310ff0beb2f505497c98f0fbe395a6`

Algorithm	Hash digest
SHA256	`5ed1c3edc08e8da8fe646b23105e2840f305ce2a75360aa0df7523c1263f43aa`
MD5	`010e0e1b93fbdd75e687a767ff9ce21c`
BLAKE2b-256	`a824472f4e44abefbec1c339d31e870df4ed022b01742c385a0a24f77020268e`

Algorithm	Hash digest
SHA256	`a16cd144c12de642f0ace1aeab7f50e7abc7492c81381faa2b17cc36f38272c4`
MD5	`b3df15fc2828124f7e0807bc8c30b33f`
BLAKE2b-256	`8355f8bebb043d30278ee673697b0d0a86ae8cd484ecaa55593b2962fb90b6b9`

Algorithm	Hash digest
SHA256	`a8557a77c2b6ee8afb49759d9b4c1a9785ca05366cbc4b2c577a859ecba62b85`
MD5	`ab7ab3bcb915427b0eba314dbf6f4a7c`
BLAKE2b-256	`7c4b3c2cd263028ce48f7299359384e3074aa7779f5cdc6b4d78efd04d6df4ee`

Algorithm	Hash digest
SHA256	`d6bb582256f374f5a8570616480f07df0d74460b8c80aaa5fb047a73ff38bcd2`
MD5	`bd94015467849eb0ba41438590c9435f`
BLAKE2b-256	`5d3a356846f02ed8bf8951912913d6443033deab2cbc3d8fdbca039818e88487`

Algorithm	Hash digest
SHA256	`dadfdd07f15c0abf394e0599b06a894120ca6f40ded9720c68b267a4ecf8bf48`
MD5	`3f161a87e15f5cd3bedf4e2c365515cc`
BLAKE2b-256	`3cbe8928696cdf109d7fd4cd35e0c85a4e3713335cfe4399a59387bcc376210e`

Algorithm	Hash digest
SHA256	`cc673ff44f1e7fa673b34507d04e6e0b750be372e4f33e40a9364858ef22411f`
MD5	`1acdb19038b9ac186b86d60074c8cc7b`
BLAKE2b-256	`34e31d23a387daad2284c0a4666dae418e72fa893ce02f4b288d696f8d297c96`

Algorithm	Hash digest
SHA256	`f30f7981689dcf06e2789a2adbf3ff0711e58a710780205c2747ec793373fa2e`
MD5	`dbb2a69f8f18dc50e2a3ed2b3be84d41`
BLAKE2b-256	`7c353f06bc09f987964db6d992bd9a7cc32ca06f032c41caf9491924ecbd2fcd`

Algorithm	Hash digest
SHA256	`84432465d0c497c774122073b79ec7fa9ebf28cf066cce5f1a5726dad455086f`
MD5	`ea5819fac05e4cfc657f4c7e12091434`
BLAKE2b-256	`04ae7cf0754378e982e377dd596371004014c4af9f246ea3090b6f63f903f37e`

Algorithm	Hash digest
SHA256	`6c79e8d3356eb72b9be25bcc36ce3320cbe1f50606f355193d6ad8ead8130fd5`
MD5	`3d2a83e69c73770a1cd6433dd71c0f47`
BLAKE2b-256	`cbff54da34b527fe2d5a4584abac2f4e4838f907d46e849f067c0f2943d7896f`

Algorithm	Hash digest
SHA256	`3f7b5f3305880a784e335c9fa1bc33a6a3a8436cdbf3e473690f4da0eb866645`
MD5	`9642f22c60b56c84e59454d86c8b4653`
BLAKE2b-256	`797f9209677513f41928af5fe5c584ec5309df015a9b48bda8ae3fc50e0cbeba`

Algorithm	Hash digest
SHA256	`5b9e0a0d5d742aa4006ab18d31eabc9a83809be3d27dad34707fa136cafa0950`
MD5	`d3c855ccc235e6cfacb67317e331ddde`
BLAKE2b-256	`04a99b2310c40c837a653640505218db3e79af1d897adf949b3b00bdf7d5ff15`

Algorithm	Hash digest
SHA256	`2096cc9c2a896436ffb27e2cfff60402ca4304b1fbe782265a8c1b2d11dc598a`
MD5	`a708ea5a03fca89b064a237271d16878`
BLAKE2b-256	`a1e63f4164fc60d32049e5241e49ffc5b484456fe0183fe0e12a2e5f126dd2df`

Algorithm	Hash digest
SHA256	`f5313c31e92a8be7e0fed7648b553f041287715d7a28fbfbbadec1dd8e7b773b`
MD5	`6e9d6642c20133cd511ddafed1b67c84`
BLAKE2b-256	`da0f62f0e9162fa63682bce3242c364f260776a4e21e66c1f1966ed0d7368fbd`

zipfile-deflate64 0.2.0

Navigation

Verified details

Maintainers

Unverified details

Project links

Meta

Classifiers

Project description

zipfile-deflate64

Installation

Usage

Design Rationale

The Problem

A Solution

Project details

Verified details

Maintainers

Unverified details

Project links

Meta

Classifiers

Release history Release notifications | RSS feed

Download files

Source Distribution

Built Distributions

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes