zc.buildout recipes to compile and install software or python packages and generate scripts or configuration files.
Project description
Introduction
Please look for options at http://pypi.python.org/pypi/minitage.recipe.common
The egg has those entry point:
cmmi: install configure/make/make install softwares
The reasons why i have rewrite yet another buildout recipes builder are:
Support for downloading stuff
Support on the fly patchs for eggs and other distribution.
Support multiple hooks at each stage of the build system.
Robust offline mode
support automaticly minitage dependencies and rpath linking.
You can browse the code on minitage’s following resources:
You can migrate your buldouts without any effort with buildout.minitagificator:
minitage.recipe.cmmi
Abstract
The cmmi recipe use abusivly or -rpath to save you from settings LD_LIBRARY_PATH at run time.
If you are inside a minitage environment, all the dependencies of your minibuild are put in the environment. That mean that CFLAGS, LDFLAGS, PKG_CONFIG_PATH, and so on are updated to reference your minibuild’s dependencies.
minitage.recipe.cmmi is a replacment fo zc.recipe.cmmi.
It intends to do the configure && make && make install dance with hooks where you can do (in python) specific stuff at each stage of the build cursus.
With this recipe, if the destination directory exists, we only remove it when we can sucessfully install something, unless that we never touch to it, or it is a bug.
LOOK ALSO AT THE SHARED VARIABLES AS ALL THE MINITAGE RECIPES SHARE A LOT OF VARIABLES
Specific options
Please look for options at : http://pypi.python.org/pypi/minitage.recipe.common#options-shared-by-all-the-recipes
- UNAME-patches
OS specific (sys.platform.lower()) patches to apply [see cmmi recipe for documentation]
- make-targets
default to all and install, make targets to run
inner-dir: change to inner directory before doing anything.
prefix: where to install (defaults to options:location) Be warn that the content of the destination will be wiped before unless you set ‘install-in-place’
install-in-place: install over a previous installation instead of removing the older install replacing it with the new ‘makeinstall result’:
install-in-place=True
- build-dir
inner directory to execute the build dance [see cmmi recipe for documentation]
- configure-options-UNAME
Os specific options to give to configure uname can be either linux2|freebsd6|freebsd7|darwin Its the result of sys.platform.lower() [see cmmi recipe for documentation]
- configure
configure script to use (default to ./configure)
- prefix-separator [see cmmi recipe for documentation]
prefix separator to use between –prefix and location (default to =)
- prefix-options [see cmmi recipe for documentation]
what to put for the “prefix” expression, default to –prefix$PREFIX_SEPARATOR$LOCATION
- autogen
autogen script to use if any [optionnal]
- configure-options
options to feed configure with [see cmmi recipe for documentation]
- extra_options
appendended to configure-options [see cmmi recipe for documentation]
hooks
A hook is in the form /path/to/hook:CALLABLE:
myhook=${buildout:directory}/toto.py:foo
Where we have toto.py:
def foo(options, buildout): return 'Hourray'
The complete possible hooks list:
- pre-unpack-hook
hook executed before the unpack dance
- post-unpack-hook
hook executed after the unpack dance
- post-unpack-hook
hook executed after the unpack dance
- pre-configure-hook
hook executed before the configure is run
- pre-make-hook
hook executed before the first make target is run
- post-build-hook
hook executed after the build make targets are done
- pending-make-install
hook executed after the make and before the make install is done
- post-make-hook
hook executed after the make install is done
- noconfigure
do not run ./configure
- noinstall
do not run make install
Detailled documentation
Let’s create a buildout configuration file.
>>> rmdir(tempdir) >>> mkdir(tempdir) >>> cd(tempdir) >>> a = [mkdir(d) for d in ('eggs', 'develop-eggs', 'bin', 'src')] >>> install_develop_eggs(['minitage.recipe']) >>> install_eggs_from_pathes(['zc.buildout'], sys.path) >>> touch('buildout.cfg') >>> sh('buildout -o bootstrap') buildout -o bootstrap...
This first buildout is classical, trying to build some url.
>>> touch('buildout.cfg', ... data=""" ... [buildout] ... download-cache=${buildout:directory} ... parts = ... part ... [part] ... recipe=minitage.recipe.cmmi ... """) >>> sh('bin/buildout install part') bin/buildout install part... <BLANKLINE> While: Installing part. <BLANKLINE> A... MinimergeError: URL was not set! <BLANKLINE>
Oups, we forgot the url, we will make a basic distribution package to test our stuff Running the buildout with the url bit.
>>> if not os.path.exists('foo'): ... mkdir('foo') ... else: ... rmdir(foo) ... mkdir('foo') >>> touch('foo/configure', data ="""echo configure $@\n""") >>> sh('chmod +x foo/configure') c... >>> touch('foo/Makefile', ... data = """ ... all: ... \t@echo all ... ... install: ... \t@echo install ... ... """) >>> sh('tar cfz foo.tgz foo') tar cfz ... <BLANKLINE> >>> data = """ ... [buildout] ... download-cache=${buildout:directory} ... parts = ... part ... [part] ... recipe=minitage.recipe.cmmi ... url = file://${buildout:directory}/foo.tgz ... """ >>> touch('buildout.cfg', data=data) >>> sh('bin/buildout -o install part') bin/buildout -o install part... Installing part. minitage.recipe: Download archive minitage.recipe: Downloading file:///tmp/buildout.test/foo.tgz in /tmp/buildout.test/minitage/foo.tgz minitage.recipe: Unpacking in /tmp/buildout.test/__minitage__part__tmp. minitage.recipe: Guessing compilation directory minitage.recipe: Setting path minitage.recipe: Setting pkgconfigpath minitage.recipe: Setting compilation flags minitage.recipe: Setting path minitage.recipe: Running /tmp/buildout.test/__minitage__part__tmp/foo/configure --prefix=/tmp/buildout.test/parts/part configure --prefix=/tmp/buildout.test/parts/part minitage.recipe: Running make all minitage.recipe: Running make install install minitage.recipe: Completed install...
General usage
This first buildout does nothing except print us the hooks calls. We have desactivated the configure and make dance!.
>>> data = """ ... [buildout] ... download-cache=${buildout:directory} ... parts = ... part ... [part] ... recipe=minitage.recipe.cmmi ... url = file://${buildout:directory}/foo.tgz ... noconfigure=true ... noinstall = true ... nomake = true ... """ >>> touch('buildout.cfg', data=data) >>> sh('bin/buildout -o install part') bin/buildout -o install part... Installing part. minitage.recipe: Download archive minitage.recipe: Unpacking in /tmp/buildout.test/__minitage__part__tmp. minitage.recipe: Guessing compilation directory minitage.recipe: Setting path minitage.recipe: Setting pkgconfigpath minitage.recipe: Setting compilation flags minitage.recipe: Setting path minitage.recipe: Completed install...
Applying patches
This second one aimes to show us the patch capababilities.
>>> sh('cp foo/Makefile foo/Makefile.old') cp foo/Makefile foo/Makefile.old >>> sh('echo >> foo/Makefile') echo >> foo/Makefile >>> sh('diff -u foo/Makefile.old foo/Makefile > patch') diff -u foo/Makefile.old foo/Makefile > patch >>> data = """ ... [buildout] ... download-cache=${buildout:directory} ... parts = ... part ... [part] ... recipe=minitage.recipe.cmmi ... url = file://${buildout:directory}/foo.tgz ... patches = ${buildout:directory}/patch ... noconfigure=true ... noinstall = true ... nomake = true ... """ >>> touch('buildout.cfg', data=data) >>> sh('bin/buildout -o install part') bin/buildout -o install part... minitage.recipe: Running patch -t -Np0 < /tmp/buildout.test/minitage/patch_d96115b00b41e282469f73708c68bdaf/patch can't find file to patch at input line 3 Perhaps you used the wrong -p or --strip option? The text leading up to this was: -------------------------- |--- foo/Makefile.old ... |+++ foo/Makefile ... -------------------------- No file to patch. Skipping patch. 1 out of 1 hunk ignored <BLANKLINE> While: Installing part. <BLANKLINE> An internal error occured due to a bug in either zc.buildout or in a... SystemError: ('Failed', 'patch -t -Np0 < /tmp/buildout.test/minitage/patch_d96115b00b41e282469f73708c68bdaf/patch') <BLANKLINE>
The patch level is wrong !
>>> data = """ ... [buildout] ... download-cache=${buildout:directory} ... parts = ... part ... [part] ... recipe=minitage.recipe.cmmi ... url = file://${buildout:directory}/foo.tgz ... patches = ${buildout:directory}/patch ... patch-options = -p1 ... noconfigure=true ... noinstall = true ... nomake = true ... """ >>> touch('buildout.cfg', data=data) >>> sh('bin/buildout -o install part') bin/buildout -o install part... minitage.recipe: Running patch -t -p1 < /tmp/buildout.test/minitage/patch... patching file Makefile minitage.recipe: Completed install...
Using hooks
But now that we have some sort of messy packages, can we not intercale some python to code to arrange things upside down ? We have hooks to achieve that. A hook is a python callable which takes at least the options part and the buildout.
>>> data = """ ... [buildout] ... download-cache=${buildout:directory} ... parts = ... part ... [part] ... recipe=minitage.recipe.cmmi ... url = file://${buildout:directory}/foo.tgz ... pre-unpack-hook = ${buildout:directory}/hooks.py:pre_unpack_hook ... post-unpack-hook = ${buildout:directory}/hooks.py:post_unpack_hook ... pre-configure-hook = ${buildout:directory}/hooks.py:pre_configure_hook ... pre-make-hook = ${buildout:directory}/hooks.py:pre_make_hook ... post-build-hook = ${buildout:directory}/hooks.py:post_build_hook ... post-make-hook = ${buildout:directory}/hooks.py:post_make_hook ... """ >>> touch('hooks.py', data=""" ... def pre_unpack_hook (o, b, hook='pre_unpack_hook'): ... print "%s in %s target %s" % (hook, b['buildout']['directory'], o['location']) ... def post_unpack_hook (o, b, hook='post_unpack_hook'): ... print "%s in %s target %s" % (hook, b['buildout']['directory'], o['location']) ... def pre_configure_hook (o, b, hook='pre_configure_hook'): ... print "%s in %s target %s" % (hook, b['buildout']['directory'], o['location']) ... def pre_make_hook (o, b, hook='pre_make_hook'): ... print "%s in %s target %s" % (hook, b['buildout']['directory'], o['location']) ... def post_build_hook (o, b, hook='post_build_hook'): ... print "%s in %s target %s" % (hook, b['buildout']['directory'], o['location']) ... def post_make_hook (o, b, hook='post_make_hook'): ... print "%s in %s target %s" % (hook, b['buildout']['directory'], o['location']) ... """) >>> touch('buildout.cfg', data=data) >>> sh('bin/buildout -o install part') ... bin/buildout -o install part... bin/buildout -o install part... minitage.recipe: Download archive minitage.recipe: Executing pre-unpack-hook pre_unpack_hook in /tmp/buildout.test target /tmp/buildout.test/parts/part... minitage.recipe: Setting path minitage.recipe: Executing post-unpack-hook post_unpack_hook in /tmp/buildout.test target /tmp/buildout.test/parts/part minitage.recipe: Executing pre-configure-hook pre_configure_hook in /tmp/buildout.test target /tmp/buildout.test/parts/part minitage.recipe: Running /tmp/buildout.test/__minitage__part__tmp/foo/configure --prefix=/tmp/buildout.test/parts/part configure --prefix=/tmp/buildout.test/parts/part minitage.recipe: Executing pre-make-hook pre_make_hook in /tmp/buildout.test target /tmp/buildout.test/parts/part minitage.recipe: Running make all minitage.recipe: Executing post-build-hook post_build_hook in /tmp/buildout.test target /tmp/buildout.test/parts/part minitage.recipe: Running make install install minitage.recipe: Executing post-make-hook post_make_hook in /tmp/buildout.test target /tmp/buildout.test/parts/part minitage.recipe: Completed install...
MD5 check
Can we check md5, of course! As we have already the foo package in our download cache, we try to download something else.
>>> shutil.copy2('foo.tgz', 'bar.tgz') >>> data = """ ... [buildout] ... download-cache=${buildout:directory} ... parts = ... part ... [part] ... recipe=minitage.recipe.cmmi ... md5sum = b4d ... url = file://${buildout:directory}/bar.tgz ... """ >>> touch('buildout.cfg', data=data) >>> sh('bin/buildout -o install part') bin/buildout -o install part Uninstalling part. Unused options for buildout: 'download-directory'. Installing part. minitage.recipe: Download archive minitage.recipe: Downloading file:///tmp/buildout.test/bar.tgz in /tmp/buildout.test/minitage/bar.tgz <BLANKLINE> While: Installing part. <BLANKLINE> A... MinimergeError: Failed download for file:///tmp/buildout.test/bar.tgz: MD5SUM mismatch for /tmp/buildout.test/minitage/bar.tgz: Good:b4d != Bad:... <BLANKLINE>
Controlling configure
Giving configure options the two possible ways.
>>> data = """ ... [buildout] ... download-cache=${buildout:directory} ... parts = ... part ... [part] ... recipe=minitage.recipe.cmmi ... url = file://${buildout:directory}/bar.tgz ... configure-options = foo ... extra_options = bar ... """ >>> touch('buildout.cfg', data=data) >>> sh('bin/buildout -o install part') bin/buildout -o install part... minitage.recipe: Running /tmp/buildout.test/__minitage__part__tmp/foo/configure --prefix=/tmp/buildout.test/parts/part foo bar configure --prefix=/tmp/buildout.test/parts/part foo bar ...
Use OS Specific rules
Giving os specific rules.
>>> data = """ ... [buildout] ... download-cache=${buildout:directory} ... parts = ... part ... [part] ... recipe=minitage.recipe.cmmi ... url = file://${buildout:directory}/foo.tgz ... configure-options-linux = linuxoptions ... """ >>> touch('buildout.cfg', data=data) >>> sh('bin/buildout -o install part') b... minitage.recipe: Running /tmp/buildout.test/__minitage__part__tmp/foo/configure --prefix=/tmp/buildout.test/parts/part linuxoptions...
Or, in the same manner, you can specify OS specific patches: (darwin, linux, freebsd6, freebsd7). sys.platform is your friend :) (‘linux’, for all linuxes).
>>> data = """ ... [buildout] ... download-cache=${buildout:directory} ... parts = ... part ... [part] ... recipe=minitage.recipe.cmmi ... patch-options = -p1 ... url = file://${buildout:directory}/foo.tgz ... """ >>> data += '%s-patches =${buildout:directory}/patch' % uname >>> touch('buildout.cfg', data=data) >>> sh('bin/buildout -o install part') b... minitage.recipe: Running patch -t -p1 < /tmp/buildout.test/minitage/patch...
Using underlying minitage environment
If you are in a minitage, all your minibuild dependencies come automaticly in the env, no need to do the following code :) As you can have CFLAGS and so on added to your env., you can specify manual things too, the underlying code is the same. The goal is to proove that all is preprended as we would have think. We will use some hook to print the relevant parts.
>>> data = """ ... [buildout] ... download-cache=${buildout:directory} ... parts = ... part ... [part] ... recipe=minitage.recipe.cmmi ... url = file://${buildout:directory}/foo.tgz ... pre-make-hook = ${buildout:directory}/hooks.py:pre_unpack_hook ... includes-dirs = /foo/include ... rpath = /someruntimespath/lib ... library-dirs = /bar/lib ... pkgconfigpath = /lib/pkgconfig/ ... noconfigure = true ... nomake = true ... noinstall = true ... """ >>> touch('hooks.py', data=""" ... import os ... def pre_unpack_hook(o, b): ... flags = [(a, os.environ.get(a, 'not_set')) for a in ('CFLAGS', 'LDFLAGS', 'PKG_CONFIG_PATH', 'LD_RUN_PATH',)] ... for flag in flags: ... print flag ... """) >>> touch('buildout.cfg', data=data) >>> sh('bin/buildout -o install part') b... minitage.recipe: Executing pre-make-hook ('CFLAGS', '-I/foo/include') ('LDFLAGS', '-L/bar/lib -Wl,-rpath -Wl,/bar/lib -L/tmp/buildout.test/parts/part/lib -Wl,-rpath -Wl,/tmp/buildout.test/parts/part/lib') ('PKG_CONFIG_PATH', '/lib/pkgconfig/...') ('LD_RUN_PATH', '/someruntimespath/lib:/tmp/buildout.test/parts/part/lib')...
Playing with the environment
You can play also with the environment directly in two ways.
Precising a buildout part
>>> data = """ ... [buildout] ... download-cache=${buildout:directory} ... parts = ... part ... [foo] ... CFLAGS=bar ... [part] ... recipe=minitage.recipe.cmmi ... url = file://${buildout:directory}/foo.tgz ... pre-make-hook = ${buildout:directory}/hooks.py:pre_unpack_hook ... environment = foo ... noconfigure = true ... nomake = true ... noinstall = true ... """ >>> touch('buildout.cfg', data=data) >>> sh('bin/buildout -o install part') bi... ('CFLAGS', 'bar')...Entering an option formed by key=value pair
>>> data = """ ... [buildout] ... download-cache=${buildout:directory} ... parts = ... part ... [part] ... recipe=minitage.recipe.cmmi ... url = file://${buildout:directory}/foo.tgz ... pre-make-hook = ${buildout:directory}/hooks.py:pre_unpack_hook ... noconfigure = true ... nomake = true ... noinstall = true ... environment= ... CFLAGS=myvalue ... """ >>> touch('buildout.cfg', data=data) >>> sh('bin/buildout -o install part') bin/buildout -o install part... minitage.recipe: Executing pre-make-hook ('CFLAGS', 'myvalue')...
Autogen can be your friend
It is possible to autogenerate the configure files.
>>> data = """ ... [buildout] ... download-cache=${buildout:directory} ... parts = ... part ... [part] ... recipe=minitage.recipe.cmmi ... url = file://${buildout:directory}/foo.tgz ... noconfigure = true ... nomake = true ... noinstall = true ... autogen = configure ... """ >>> touch('buildout.cfg', data=data) >>> sh('bin/buildout -o install part') bin/buildout -o install part... minitage.recipe: Auto generating configure files minitage.recipe: Running /tmp/buildout.test/__minitage__part__tmp/foo/configure configure...
CHANGELOG
-> 1.36
splitted out from minitage.recipe
1.34
all:
fix rst doc
add ldflags and ldflags options
minitage.recipe.cmmi:
install-in-place option added and pivot on make install slightly modified
skip-flags option added
prefix option added
minitage.recipe.fetch:
unpack option added
md5sum un revision for static fetcher check added
minitage.recipe.printer:
remove fake eggs and develop eggs from fixed versions.
-> 1.20
Minitage.recipe.egg:
better errors handling
In particular for eggs and url, now md5 urls are preferred.
Also when we can’t download a distribution from somewhere, we try to fallback to other distributions which achieve the desired requirement
In the same way, try to compile eggs from any other sdist fullfilling the requirement if an error occurs on the first found.
Minitage.recipe.cmmi:
- new patch selection for freebsd:
freebsd-patches
Fix some double spaces left in compilation flags which broke some exotic system builders.
1.14
add enrionment file generation to the scripts recipe
1.13
cmmi buggy update methods
1.11 - 1.12
rescan egg direcrectories at install time because there may be new develop eggs or eggs installed by buildout extensions (mr.developer)
fix for interaction with buildout.minitagificator and zc.buildout.easy_install..*script
1.-1.10
bugfix with cache and static distributions.
1.8
make patches go in a personnal directory with some md5 mecanism to redownload them.
1.4->1.7
x64 fixes
bugfix in static distribution install when you have patches to apply.
1.3
release uncomitted unittests updates
1.2
make it compatible with zc.buildout 1.0
1.1
md5 bug
1.0
first stable release.
All recipe are documented and well tested.
buildout.minitagificator is completed.
stabilized and synced with other minitage components
minitage.recipe:eggs has been drasticly improved in term of algorythms and speed.
0.33->0.34
Fix python executable computation
0.32
Documentation and tests release
0.29
Fix bug in recipes initialization because of buildout auto-ordering
0.28
Add another hook to the cmmi recipe: post-download
0.27
Fix an annoying bug when you are using virtualenv and a classical python as executable in your recipe, it may occur that sometimes, the virtualenv site-packages is appended to PYTHONPATH. As a fix, we now filter all python core lib directories and site packages that are not relevant to the python used by the recipe.
0.26
Introduce the minitage.recipe:wsgi recipe wich wraps a paste wsgi application in a script eatable by mod_wsgi
0.25
stop lowering project_name
0.23->0.24
Fix small bugs in printer
0.22
Introduce a new recipe : minitage.recipe:printer which prints/dumps to a file all versions eggs needed to achieve requirements.
0.21
Fix bug in compilation directory guessing
0.20
Fix bug in distribution reloading
Better handling of pkg_resources’s working set
0.19
add a post build hook
0.18
Change installation order
0.16
Fix index and find links options
0.14 -> 0.15
logging output improved
newest mode handling
0.13
fix static dist install
0.12
bugfix for zipped eggs
0.11
import bugfix for minitagificator
0.10
Change some logging options
follow allow host option from buildout for eggs installation
0.9
Do not rely anymore on zc.buildout ez for script generation
0.7 -> 0.8
egg and scripts recipes were rewritten a lot to adapt their api to zc.recipe.egg
Common to all recipes: all arguements are now identicak to zc.recipe.cmmi or zc.recipe.egg
Dependencies resolver has been improved a lot
For static distribution, you can know speicify multiple urls with ‘urls’
0.6
fix bug in path generation in minitage.recipe:scripts
0.2 -> 0.5
add make-options for make option in minitage.recipe.cmmi
multiple bugfixes
make things append in subprocess for environment conversation
add initiaiization code for generated python interpreter in minitage.recipe:scripts
0.1
add fetch recipe
0.0.13
fix zip safe flag
0.0.11
Include eggs dependencies by default
Modulate the script recipe to append in the PYTHONPATH all egg dependencies found into the local eggs cache.
0.0.10
Add pyc regeneration feature stolen from zc.buildout
0.0.9
minor fix for run without minitage
fix for long path/compilation flags
0.0.8
minor fix for scm check outs
0.0.7
add C compiler cflags/ldflags/makeopts customistation options
0.0.6
Change the eggs installation way to do, now using easy_install everywhere Also include a buildout option to allow to include eggs dependencies (ez-dependencies-true)
Set the __doc__ variable in scripts to fix some scripts like bzr.
0.0.3
fix scm choice when there are eggs and url in the same part
0.0.2
bugfix version
fix linking problem
0.0.1
Initial version
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.