Skip to main content

Enhanced Python `compileall` module

Project description

compileall2 Python module

Copy of compileall module from CPython source code with some new features, namely:

  • compatibility with Python >= 3.4 & PyPy 3

  • default recursion limit is now "unlimited" (actually limited by sys.getrecursionlimit())

  • -s and -a command line options for manipulation with path baked into a compiled *.pyc file.

Installation

  • From PyPI via pip install compileall2

  • RPMs will be available in Fedora COPR

Usage

compileall2 can be executed as a Python module or directly.

Example usage:

# Create some script with bad syntax
$ echo "1 / 0" > test.py
# Compile it
$ compileall2 test.py 
Compiling 'test.py'...
# Try to execute compiled version directly
$ python __pycache__/test.cpython-37.pyc 
Traceback (most recent call last):
  File "test.py", line 1, in <module>
    1 / 0
ZeroDivisionError: division by zero
# Recompile it with changes path which will be visible in error message
$ compileall2 -f -a /foo/bar test.py 
Compiling 'test.py'...
$ python __pycache__/test.cpython-37.pyc 
Traceback (most recent call last):
  File "/foo/bar/test.py", line 1, in <module>
ZeroDivisionError: division by zero
# Same thing as above but executed as a Python module
$ python -m compileall2 -f -a /bar/baz test.py 
Compiling 'test.py'...
$ python __pycache__/test.cpython-37.pyc 
Traceback (most recent call last):
  File "/bar/baz/test.py", line 1, in <module>
ZeroDivisionError: division by zero

Done

  • ✓ Start project :)

  • ✓ Make upstream tests running

  • Make compileall2 compatible with CPythons:

    • 3.8 ✓
    • 3.7 ✓
    • 3.6 ✓
    • 3.5 ✓
    • 3.4 ✓
  • ✓ Make compileall2 compatible with PyPy 3

  • ✓ Remove maximum depth limit as described above

  • ✓ Add possibility to strip some part of a path to an original file from compiled one

  • ✓ Publish it to PyPI

  • ✓ Make it available in Fedora COPR

ToDo

  • [NEXT STEP] Test it with Python packages in COPR

  • Push it to Fedora rawhide

  • Test it in Fedora infrastructure with all Python packages

  • ? Prepare patches for upstream CPython

Testing

You can test it locally with tox or unittest directly:

$ python3 -m unittest test_compileall_original.py
............sss.......................sss.....................ss.................
----------------------------------------------------------------------
Ran 81 tests in 2.137s

OK (skipped=8)

but running in a Docker container might be better because the superuser has privileges to write to sys.path which lowers the number of skipped tests.

You can just build the prepared one:

$ docker build -t compileall2 .
Sending build context to Docker daemon 177.2 kB
Step 1/3 : FROM frenzymadness/fedora-python-tox:latest
 ---> 00f92ad0e1d3
... etc ...

and run tests in it:

$ docker run --rm -it -e TOXENV=py37 -v $PWD:/src:Z -w /src  compileall2
py37 create: /src/.tox/py37
py37 installdeps: pytest
py37 installed: atomicwrites==1.3.0,attrs==19.1.0,more-itertools==6.0.0,pluggy==0.9.0,py==1.8.0,pytest==4.3.1,six==1.12.0
py37 run-test-pre: PYTHONHASHSEED='519909491'
py37 runtests: commands[0] | pytest -v -s
========================================== test session starts ==========================================
platform linux -- Python 3.7.2, pytest-4.3.1, py-1.8.0, pluggy-0.9.0 -- /src/.tox/py37/bin/python
cachedir: .tox/py37/.pytest_cache
rootdir: /src, inifile:
collected 81 items

test_compileall_original.py::CompileallTestsWithSourceEpoch::test_compile_dir_pathlike PASSED
test_compileall_original.py::CompileallTestsWithSourceEpoch::test_compile_file_pathlike PASSED
test_compileall_original.py::CompileallTestsWithSourceEpoch::test_compile_file_pathlike_ddir PASSED
... etc ...
================================= 79 passed, 2 skipped in 5.15 seconds ==================================
________________________________________________ summary ________________________________________________
  py37: commands succeeded
  congratulations :)

License

PSF license v2

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

compileall2-0.1.0.tar.gz (7.6 kB view details)

Uploaded Source

Built Distribution

compileall2-0.1.0-py3-none-any.whl (9.2 kB view details)

Uploaded Python 3

File details

Details for the file compileall2-0.1.0.tar.gz.

File metadata

  • Download URL: compileall2-0.1.0.tar.gz
  • Upload date:
  • Size: 7.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.7.2

File hashes

Hashes for compileall2-0.1.0.tar.gz
Algorithm Hash digest
SHA256 711e38508af384851d1edeec4941009fbde4439b584d08ece35d7e24553672be
MD5 ee28a4203c82f6d21b76506057f361d6
BLAKE2b-256 2c598c8db8fa1c8a6536b04d2e25e39890b05555ead3220cb11d5ecf141b80a2

See more details on using hashes here.

Provenance

File details

Details for the file compileall2-0.1.0-py3-none-any.whl.

File metadata

  • Download URL: compileall2-0.1.0-py3-none-any.whl
  • Upload date:
  • Size: 9.2 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.7.2

File hashes

Hashes for compileall2-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 f1860ec770050ce2b060da7aada06f08b1eda3d8b8a3bfec2de09f125524bdcc
MD5 7383f16a2706ace6a7aa176bf7c5fba2
BLAKE2b-256 b8d393644d233b18bf7049f6a79bf2f3e89d9e4dc253e96109e422fde0d0da9c

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page