minitage.recipe.egg 1.101

Installing eggs from index or find links, the classical way to install python packages

We need to specify a find-links entry to make the recipe find our ‘foo’ egg as it is not on pypi As we want to show the update capability of the recipe, we will first install the oldest foo version.

>>> globals().update(layer['globs'])

Let’s create a buildout configuration file and a basic egg not published on pypi:

>>> makedist()
>>> makedist(version="2.0")
>>> makedist(name="foo3", version="2.0")
>>> makedist(name="foo4", version="2.0")
>>> makedist(name="foo4", version="3.0")
>>> data = """
... [versions]
... foo=1.0
... [buildout]
... eggs-directory =${buildout:directory}/eggs
... download-cache=${buildout:directory}
... index=%(index)s
... parts =
...     part
... [part]
... recipe=minitage.recipe.egg
... find-links=%(index)s
... eggs=foo
... """% bsettings
>>> touch('buildout.cfg', data=data)
>>> clean()
>>> sh('bin/buildout -vvvvv install part')
bin/...
Installing part.
minitage.recipe: Installing python egg(s)...
minitage.recipe: Downloading http://...:.../foo-1.0.tar.gz in .../eggs/foo-1.0.tar.gz
minitage.recipe: Unpacking in ...
Processing foo-1.0.tar.gz...
minitage.recipe: Installed foo 1.0 (.../eggs/foo-1.0-p....egg)...

Static distribution dev+static urls

You can also install directly from urls. We ll use it to check the already present distribution files in the cache.

>>> bsettings["foo3url"] = "%sfoo3-2.0.tar.gz" % (bsettings["index"])
>>> bsettings["foo4url"] = "%sfoo4-2.0.tar.gz" % (bsettings["index"])
>>> bsettings["foo43url"] = "%sfoo4-3.0.tar.gz" % (bsettings["index"])
>>> data = """
... [versions]
... mr.developer=0.15
... [buildout]
... eggs-directory =${buildout:directory}/eggs
... versions = versions
... download-cache=${buildout:directory}
... parts =
...     part
... index=%(index)s
... [part]
... recipe = minitage.recipe.egg
... eggs=mr.developer
... [a]
... recipe=minitage.recipe.egg
... urls=
...      %(foo3url)s
...      %(foo4url)s
... """ % bsettings
>>> touch('buildout.cfg', data=data)
>>> sh('bin/buildout -vvvvvv install a') #doctest +REPORT_NDIFF
b...
Installing a...
minitage.recipe: Download archive from http://.../foo...-2.0.tar.gz...
minitage.recipe: Installed foo...
minitage.recipe: Pinning custom egg version in buildout, trying to write the configuration
minitage.recipe: CREATING buildout backup in ...
...

As we are installing from an url, we must pin the version to be sure to use this egg, even if we have some other similar egg on index or find-links. See the versions pinned to use your downloaded stuff

>>> cat('buildout.cfg') # doctest: +REPORT_NDIFF
<BLANKLINE>
[versions]
mr.developer=0.15
foo... = 2.0
foo... = 2.0
[buildout]
eggs-directory =${buildout:directory}/eggs
versions = versions
download-cache=${buildout:directory}
parts =
    part
index...
[part]
recipe = minitage.recipe.egg
eggs=mr.developer
[a]
recipe=minitage.recipe.egg
urls=
.../foo3-2.0.tar.gz
.../foo4-2.0.tar.gz...

If we try to install a newer version, via an url, it will work, even if the version is pinned.:

>>> data = """
... [versions]
... mr.developer=0.15
... [buildout]
... versions = versions
... download-cache=${buildout:directory}
... eggs-directory =${buildout:directory}/eggs
... parts =
...     part
... index=%(index)s
... [part]
... recipe = minitage.recipe.egg
... eggs=mr.developer
... [a]
... recipe=minitage.recipe.egg
... urls=
...     %(foo43url)s
... """ % bsettings
>>> touch('buildout.cfg', data=data)
>>> sh('bin/buildout -vvvvv install a')
b...
>>> cat('buildout.cfg') # doctest: +REPORT_NDIFF
<BLANKLINE>
[versions]...
foo4 = 3.0...

If we try to rebuild the egg, we cannot, as the same egg is already built. This is to prevent rebuilding triggered by buildout each time we launch it, and also to delete already good versions present in the cache.:

>>> sh('bin/buildout -ovvvvv install a')
b...
minitage.recipe: If you want to rebuild, please do 'rm -rf .../eggs/foo...

Offline and newest modes

We have ways to make buildout not download the latest versions found on indexes and be very conservative on what we already got local.

Let’s create a buildout configuration file and a basic egg not published on pypi, and install it:

>>> globals().update(layer['globs'])
>>> makedist()
>>> makedist(version='2.0')
>>> data = """
... [versions]
... foo=1.0
... [buildout]
... eggs-directory =${buildout:directory}/eggs
... download-cache=${buildout:directory}
... index=%(index)s
... parts =
...     part
... [part]
... recipe=minitage.recipe.egg
... find-links=%(index)s
... eggs=foo
... """%bsettings
>>> touch('buildout.cfg', data=data)
>>> clean()
>>> sh('bin/buildout -vvvvv install part')
bin/...
Installing part.
minitage.recipe: Installing python egg(s)...
minitage.recipe: Downloading http://...:.../foo-1.0.tar.gz in .../minitage/eggs/foo-1.0.tar.gz
minitage.recipe: Unpacking in ...
Processing foo-1.0.tar.gz...
minitage.recipe: Installed foo 1.0 (.../eggs/foo-1.0-p....egg)...

Well, now, we are 1.0. Removing the version bit, but choosing to be non newest will make buildout not to install the new foo-2.0 version.:

>>> data = """\
... [versions]
... [buildout]
... download-cache=${buildout:directory}
... eggs-directory =${buildout:directory}/eggs
... parts =
...     part
... versions = versions
... index=%(index)s
... [t]
... recipe=minitage.recipe.egg
... find-links=%(index)s
... eggs=foo
... """%bsettings
>>> touch('buildout.cfg', data=data)
>>> sh('bin/buildout -Nvvvvv install t')
b...
minitage.recipe: Installing python egg(s).
minitage.recipe: Picked: foo = 1.0
minitage.recipe: All egg dependencies seem to be installed!...

Idem in offline mode.:

>>> sh('bin/buildout -ovvvvv install t')
b...
minitage.recipe: Picked: foo = 1.0
minitage.recipe: All egg dependencies seem to be installed!...

But then, going online/newest will trigger the installation of the 2.0 egg.:

>>> sh('bin/buildout -nvvvvv install t')
b...
minitage.recipe: Picked: foo = 2.0
minitage.recipe: All egg dependencies seem to be installed!...

File urls work in offline mode.:

>>> data = """
... [buildout]
... download-cache=${buildout:directory}
... eggs-directory =${buildout:directory}/eggs
... index=%(index)s
... parts =
...     part
... [part]
... recipe=minitage.recipe.egg
... urls=file://${buildout:directory}/foo-2.0.tar.gz
... """ % bsettings
>>> clean()
>>> touch('buildout.cfg', data=data)
>>> sh('bin/buildout -vvvvv install part')
b...
>>> [egg for egg in os.listdir('eggs') if 'foo' in egg]
['foo-2.0-py...egg']
>>> sh('bin/buildout -o install part')
b...
>>> [egg for egg in os.listdir('eggs') if 'foo' in egg]
['foo-2.0-py...egg']

Installing eggs with a patch

We need to specify a find-links entry to make the recipe find our ‘foo’ egg as it is not on pypi As we want to show the update capability of the recipe, we will first install the oldest foo version.

Let’s create a buildout installation:

>>> globals().update(layer['globs'])
>>> makedist()
>>> makedist(version='2.0')
>>> touch('patch', data="""
... --- foo.old/toto.py     2013-03-30 20:42:04.575613999 +0100
... +++ foo/toto.py 2013-03-30 20:42:10.603721734 +0100
... @@ -1,3 +1,7 @@
...
...  def f():
...      print "foo"
... +
... +
... +
... +
...
... """)

Patching is easy, just put your patches in YouEgg-patches.

>>> data = """
... [versions]
... foo=1.0
... [buildout]
... download-cache=${buildout:directory}
... eggs-directory =${buildout:directory}/eggs
... parts = part
... index=%(index)s
... [part]
... recipe=minitage.recipe.egg
... find-links=%(index)s
... foo-patches = ${buildout:directory}/patch
... eggs=foo
... """%bsettings
>>> touch('buildout.cfg', data=data)
>>> sh('bin/buildout install part')
b...
can't find file to patch at input line 4
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|
|--- foo.old/toto.py     2013-03-30 20:42:04.575613999 +0100
|+++ foo/toto.py 2013-03-30 20:42:10.603721734 +0100
--------------------------
No file to patch.  Skipping patch.
...
minitage.recipe: Message was:
    ('Failed', 'patch -t -Np0 < ...patch')...

Oups, the patch level ! .

>>> data = """\
... [versions]
... foo=1.0
... [buildout]
... download-cache=${buildout:directory}
... eggs-directory =${buildout:directory}/eggs
... parts = part
... index=%(index)s
... [part]
... recipe=minitage.recipe.egg
... find-links=%(index)s
... eggs=foo
... foo-patches = ${buildout:directory}/patch
... foo-patch-options = -Np1
... """%bsettings
>>> touch('buildout.cfg', data=data)
>>> sh('bin/buildout -vvvvv install part')
b...
minitage.recipe: Running patch -t -Np1 < .../patch
patching file toto.py...
minitage.recipe: Installed foo 1.0-ZMinitagePatched-Patch...

Now that we have it, try to resintall.

>>> sh('bin/buildout -vvvvv install part')
bin/buildout...
minitage.recipe: We have the distribution that satisfies 'foo==1.0-ZMinitagePatched-Patch'.
minitage.recipe: Pinning custom egg version in buildout, trying to write the configuration
minitage.recipe: Version already pinned, nothing has been wroten...

In all cases our buildout is patched.

>>> cat('buildout.cfg')
[versions]
foo...=...1.0-ZMinitagePatched-Patch...

Installing eggs with custom build_ext_options

>>> globals().update(layer['globs'])
>>> makedist(version='2.0', setup="""
...
... import os
... if os.path.exists('setup.cfg'):
...     print 'setup.cfg:'
...     print '---------------'
...     print open('setup.cfg').read()
... """)

Just add the bdist_ext options you want to the buildout

>>> data = """
... [versions]
... [buildout]
... download-cache=${buildout:directory}
... eggs-directory =${buildout:directory}/eggs
... parts =
...     part
... index=%(index)s
... [part]
... recipe=minitage.recipe.egg
... find-links=%(index)s
... define=Foo=1,Bar=2
... bdistext-foo = bar
... bdistext-bar = titi
... foo-bdistext-foo = tutu
... foo-swig-cpp = toto
... eggs=foo
... """% bsettings
>>> touch('buildout.cfg', data=data)
>>> sh('bin/buildout -vvvvv install part')
b...
minitage.recipe: Using bdist_ext option: define = Foo=1,Bar=2
minitage.recipe: Using bdist_ext option: bar = titi
minitage.recipe: Using bdist_ext option: foo = bar...
minitage.recipe: foo: Using bdist_ext option: swig-cpp = toto
minitage.recipe: foo: Using bdist_ext option: foo = tutu...
setup.cfg:
---------------
[egg_info]
tag_build =
tag_date = 0
tag_svn_revision = 0
<BLANKLINE>
[build_ext]
foo = tutu
bar = titi
swig-cpp = toto
define = Foo=1,Bar=2...
setup.cfg:
---------------
[egg_info]
tag_build =
tag_date = 0
tag_svn_revision = 0
<BLANKLINE>
[build_ext]
foo = tutu
bar = titi
swig-cpp = toto
define = Foo=1,Bar=2...
minitage.recipe: All egg dependencies seem to be installed!...

Distribute Support

>>> globals().update(layer['globs'])

If we are in distribute mode, it would significate that distribute is installed and is available on sys.path. If so, we detect that and we will just install distribute rather than setuptools if any requirement wants setuptools. And even if the version is fixed.

>>> data = """
... [buildout]
... versions = versions
... eggs-directory =${buildout:directory}/eggs
... index=%(index)s
... parts = part
...         s
... [part]
... recipe=minitage.recipe.egg
... eggs = virtualenv
... [s]
... recipe=minitage.recipe.scripts
... eggs = virtualenv
... [versions]
... """ % bsettings
>>> touch('buildout.cfg', data=data)

Running buildout with distribute to see that it’s picked.

>>> sh('bin/buildout -vvvvvv install part s')
bin/buildout...
minitage.recipe: Using distribute!...