Skip to main content

A utility to help maintain a wheelhouse.

Project description

Wheelhouse is a utility to help maintain a wheelhouse.

What is a Wheelhouse?

A wheelhouse is a local cache of python packages in wheel format that gets committed with your code to your VCS. When installing packages during continuous integration and production, the wheels in the wheelhouse are used instead of depending on PyPI or some other network location.

Advantages:

  • Wheels are stored in your DVCS bringing further clarity to exactly what packages are needed/expected and how they have changed over time.

  • CI builds are faster and more consistent. Due to the increased speed of installing wheels from a local cache instead of pulling them from a network location, we can have tox start with a new virtualenv before every run, thereby insuring all dependencies have been specified and installed into the wheelhouse correctly.

  • Production deployments are similarly fast and consistent. Since the CI and production servers both pull from the same wheelhouse we have higher certainty that our production code is running against the exact same packages that have been tested.

  • Since wheels are built on development or build machines, the need for development system packages to be installed on production servers is removed.

  • Targeting forks, development versions, unpublished, and/or private software for production is much easier than setting up & maintaining a private PyPI server like devpi.

  • Splits the package management process into two distinct steps:

    1. Build packages (from various locations, with specified version) and put wheels in the wheelhouse.

    2. Install the latest version of a package from the wheelhouse.

Disadvantages:

  • Some may be opposed to storing binary packages in version control.

  • More disk space is needed for the binary packages.

  • The wheelhouse will accumulate packages if not cleaned up regularly. The purge command can help with this.

Example Usage

Build/Refresh the Wheelhouse

This will build wheels and store them in the wheelhouse for any updated packages:

wheelhouse build

Setting Up an Environment

Create a temporary virtualenv with packages installed from the wheelhouse:

vex -mr myproj # or: mktmpenv
wheelhouse install -- -r requirements/dev-env.txt
pip install -e .

Your virtualenv now contains the same packages as the wheelhouse. By using temporary environments you force yourself to always work with what is in the wheelhouse. Getting a package into your development environment requires you to go throught the wheelhouse to do it. This means maintaining the project’s packages becomes a first-priority issue in every developers workflow.

Example Project

The code for this project is rather basic, but it’s the concept that counts. Putting the concept of a wheelhouse into practice has made managing dependencies for our projects across dev, testing and production environments much, much easier.

Checkout Keg to see a project which is using a wheelhouse in conjunction with tox to manage dependencies.

Current Features:

  • build: Will build all packages for all requirements file specified in the config file and store in the wheelhouse directory. Can also be passed the names of individual packages or aliases.

  • config: display the configuration wheelhouse is using.

  • purge: purge the wheelhouse of any wheel that isn’t the most recent version in the wheelhouse for that package.

Possible Future Features:

  • install: install a package/wheel from the wheelhouse.

  • status: compare the working environment’s installed packages with the requirement files, the wheelhouse, and package indexes (PyPI) and show where they are out of sync.

pip Configuration

Once you have a wheelhouse (wheelhouse build), you can tell pip to install only from the wheelhouse. To do that with environment variables and a tox.ini, it would look like:

# tox.ini
[testenv]
setenv =
    PIP_USE_WHEEL=true
    PIP_NO_INDEX=true
    PIP_FIND_LINKS=requirements/wheelhouse

or, from the command line:

pip install --use-wheel --no-index --find-links=requirements/wheelhouse -r requirements/testing.txt

Configuration

You must place a wheelhouse.ini in the base of your project. This is considered the “project root” and all relative file paths are calculated from this location.

You may also place a wheelhouse.ini file in a user-specific location to override defaults for Wheelhouse. See wheelhouse config for more information.

Config files are read by a SafeConfigParser instance. See the linked docs for interpolation support available.

An example configuration file follows:

[wheelhouse]
# These files are relative to the project's requirements directory (default: `requirements/`).
requirement_files =
    build.txt

# Make sure each package has a wheel built for python 2 & 3.
pip_bins = pip, pip3.4

[aliases]
# Shortcuts to be used when specifying projects to `build`.
keg = https://github.com/level12/keg/zipball/master
ke = https://github.com/level12/keg-elements/zipball/master

Issues & Discussion

Please direct questions, comments, bugs, feature requests, etc. to: https://github.com/level12/wheelhouse/issues

Current Status

Very Beta, expect changes.

Development

To develop on this project, begin by running our tests:

git clone https://github.com/level12/wheelhouse wheelhouse-src
cd wheelhouse-src
tox

You can then examine tox.ini for insights into our development process. In particular, we:

  • use py.test for testing (and coverage analysis)

  • use flake8 for linting

  • store pip requirements files in requirements/

  • cache wheels in requirements/wheelhouse for faster & more reliable CI builds

Dependency Management

Adding a dependency involves:

  1. If it’s a run-time dependency, add to setup.py.

  2. Adding the dependency to one of the requirements files in requirements/.

  3. Running wheelhouse build.

Preview Readme

When updating the readme, use restview --long-description to preview changes.

Changelog

0.1.4 released 2018-10-09

  • prep for pyp (9768bdb)

  • replace function from missing wheel.install (2699de5)

  • fix bug with handling multiple reqs files (dc3eb9a)

  • better default logging (48c49c7)

0.1.3 - 2017-05-30

  • BUG: fix bug in handling multiple requirements files

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

Wheelhouse-0.1.4.tar.gz (15.3 kB view details)

Uploaded Source

Built Distribution

Wheelhouse-0.1.4-py3-none-any.whl (16.9 kB view details)

Uploaded Python 3

File details

Details for the file Wheelhouse-0.1.4.tar.gz.

File metadata

  • Download URL: Wheelhouse-0.1.4.tar.gz
  • Upload date:
  • Size: 15.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.9.1 pkginfo/1.4.1 requests/2.18.4 setuptools/38.2.4 requests-toolbelt/0.8.0 tqdm/4.19.5 CPython/3.5.2

File hashes

Hashes for Wheelhouse-0.1.4.tar.gz
Algorithm Hash digest
SHA256 76ccaf2cf70e7bd9db9705c9ee67b8eebb7fb5e1b292fa003409b95f1e1f449c
MD5 5977270880c588f3c6e0c3908fc853c5
BLAKE2b-256 2382ee80cfb81d9532dcbe7ef8ae35f540ac692a4caa162f19ca7c4befb9f67b

See more details on using hashes here.

Provenance

File details

Details for the file Wheelhouse-0.1.4-py3-none-any.whl.

File metadata

  • Download URL: Wheelhouse-0.1.4-py3-none-any.whl
  • Upload date:
  • Size: 16.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.9.1 pkginfo/1.4.1 requests/2.18.4 setuptools/38.2.4 requests-toolbelt/0.8.0 tqdm/4.19.5 CPython/3.5.2

File hashes

Hashes for Wheelhouse-0.1.4-py3-none-any.whl
Algorithm Hash digest
SHA256 c5b95bc1610f8407f7ff4d492598bb4b197cfe704d463042a67954efe2c8e507
MD5 4c46d604154006d84422bbdefbad470f
BLAKE2b-256 e5ae0cd83de96394943dfe5e0a333d54e2c54ade409d0515638c0233b0db4a26

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page