collective.buildbot 0.3.2

Supported options

The recipe supports the following options:

port

The port the build master process listens for connections from build slaves. The slaves must be configured to use the corresponding port in the sections using the collective.buildbot:slave recipe.

wport

The web port for serving the buildbot web interface.

project-name

Project name. Displayed on the web interface.

project-url

Project url, used on the web interface.

url

buildbot url.

build-slaves

A sequence of build slave configurations. Each build slave must be defined on a separate line containing the name of the build slave and the password for the build slave separated by white space.

allow-force (optional)

If true allows users to force builds using the web interface. Defaults to false.

public-html (optional)

Location of a directory that contains custom resources (HTML, CSS, images) for the web interface.

Additionally you can use the following options if you need to run an IRC bot:

irc-host

The irc host to connect to. ie: irc.freenode.net

irc-channels

A list of channels to join. #plone

irc-nickname

The bot nickname. Defaults to buildbot

irc-password

The password used to identify the bot. Defaults to an empty string

Example usage

We’ll start by creating a buildout that uses the recipe:

>>> write('buildout.cfg',
... """
... [buildout]
... parts = buildmaster
...
... [buildmaster]
... recipe = collective.buildbot:master
... port = 8080
... wport = 8082
... project-name = The project
... project-url = http://example.com
... url = http://example.com/buildbot
... slaves =
...     slave1 password
...     slave2 password
... """)

Running the buildout gives us:

>>> print system(buildout)
Installing buildmaster...
New python executable in /sample-buildout/parts/buildmaster/bin/...
Installing setuptools.............done.
Generated script '/sample-buildout/parts/buildmaster/buildbot.tac'.
Generated config '/sample-buildout/parts/buildmaster/buildbot.cfg'.
Generated script '/sample-buildout/bin/buildmaster'.

As shown above, the buildout generated the required configuration files and the runner script under bin. You can control build master process by running:

$ ./bin/buildmaster [start | stop | restart]

The Twisted .tac file that is used to launch the buildbot process:

>>> cat(join('parts', 'buildmaster', 'buildbot.tac'))
from twisted.application import service
from buildbot.master import BuildMaster
import os
import sys
import collective.buildbot
<BLANKLINE>
basedir = r'/sample-buildout/parts/buildmaster'
buildbot = os.path.dirname(collective.buildbot.__file__)
<BLANKLINE>
configfile = os.path.join(buildbot, 'master.py')
application = service.Application('buildmaster')
<BLANKLINE>
master = BuildMaster(basedir, configfile)
master.setServiceParent(application)
<BLANKLINE>

We can also see that the configuration file generated by the recipe reflects the options we chose in our buildout configuration:

>>> cat(join('parts', 'buildmaster', 'buildbot.cfg'))
[slaves]
slave1 = password
slave2 = password
<BLANKLINE>
[buildbot]
projects-directory = /sample-buildout/parts/projects
project-name = The project
pollers-directory = /sample-buildout/parts/pollers
url = http://example.com/buildbot
wport = 8082
project-url = http://example.com
port = 8080
allow-force = false
<BLANKLINE>

Supported options

The recipe supports the following options:

host

Hostname of the build master.

port

Port that the build master is listening. This should match the port option in the section using the collective.buildbot:master recipe in your buildmaster buildout.

password

Build slave password. This should match the password in the slave-names section in the buildmaster buildout.

eggs

Used to install extra eggs in slave environment.

environment

Can define the name of a section, containing environment variable that will be defined in the slave environment.

Example usage

We’ll start by creating a buildout that uses the recipe:

>>> write('buildout.cfg',
... """
... [buildout]
... parts =
...    buildslave
...
... [buildslave]
... recipe = collective.buildbot:slave
... host = localhost
... port = 8888
... password = password
... environment = slaveenv
...
... [slaveenv]
... PATH = /bin:/usr/bin:/usr/local/bin:/usr/local/firefox
... """)

Running the buildout gives us:

>>> print system(buildout)
Installing buildslave.
...
Generated script /sample-buildout/parts/buildslave/buildbot.tac.
Generated script '/sample-buildout/bin/buildslave'.
<BLANKLINE>

As shown above, the buildout generated the required scripts. You can control build slave process by running:

$ ./bin/buildslave [start | stop | restart]

Supported options

The recipe supports the following options:

slave-names (mandatory)

A white-space separated list of slave names that the project will be built on. These must correspond to the section names that use the collective.buildbot:slave recipe and that are consequently referred in the slaves option of the section using the collective.buildbot:master recipe.

vcs (optional)

The version control system used to obtain the source code for the project. Defaults to svn. Other possible values are: hg, bzr and git.

repositories (mandatory)

A sequence of newline separated URLs to the code repositories that correspond to the selected version control system. For Subversion this could be something like https://svn.plone.org/svn/collective/collective.buildbot/trunk or for Git something like git@github.com:dokai/hexagonit-swfheader.git.

For Subversion, if the url root is found in HOME/.buildout/.httpauth, username and password will be used to perform checkouts and updates. See: http://pypi.python.org/pypi/lovely.buildouthttp.

For each repository you define a separate Buildbot project configuration will be generated that shares all the other options. This is useful if you have multiple similar projects within the same repository and can save you a lot of typing. Since Subversion URLs contain the branch information it is even possible to pull in code from separate branches. For other version control system that use the branch option (e.g. Git) you’re limited to a single shared branch name.

branch (optional)

The branch in the version control system that will be checked out. For Subversion checkouts you should provide the full URL to the desired branch (e.g. something that ends with trunk or branches/foo) and leave this option empty. For Git repositories the default value is master.

email-notification-sender (optional)

An email address that will be used in the From: header for the notification messages.

email-notification-recipient (optional)

A newline separated sequence of email addresses the notification messages will be sent to.

build-sequence (optional)

A newline separated sequence of shell commands executed on the build slave after checking out the code from the repository that will build the project.

Defaults to:

bin/python bootstrap.py
bin/buildout

which is appropriate for buildout based projects.

test-sequence (optional)

A newline separated sequence of shell commands executed on the build slave to execute the test suite. Defaults to:

bin/test

periodic-scheduler (optional)

Sets up a periodic scheduler that schedules a build every n minutes, where n is the given integer value.

cron-scheduler (optional)

Sets up a cron-like scheduler that schedules a build at a given time. The time is configured in a crontab manner using white space separated values for the following fields:

[minute] [hour] [day of month] [month] [day of week]

The values should be integers in the approriate range for the given field or * (asterisk) for all values. For example to schedule a build at 3:00 am every night you would use:

cron-scheduler = 0 3 * * *

dependent-scheduler (optional)

Sets up a dependency between the given project and the current one. After a successful build of the given project, this one will be triggered.

pyflakes (optional)

A sequence of newline separated PyFlakes commands to run. If defined, the given PyFlakes commands will be run after the test sequence.

The commands should consist of a path to the pyflakes script and a path to the source code container. For example, using a global pyflakes installation on a project located under src/some.project within the build directory you would set:

pyflakes = pyflakes src/some.project

You can also have your slave buildout install pyflakes and use that instead of a globally installed version.

Example usage

We’ll start by creating a buildout that uses the recipe. A full example would propably have other sections defining the build master and slaves, but here we will demonstrate only the use of the collective.buildbot:project recipe.

>>> write('buildout.cfg',
... """
... [buildout]
... parts = my.package
...
... [my.package]
... recipe = collective.buildbot:project
... slave-names = slave1
... vcs = svn
... repositories = http://example.com/svn/my.package/trunk
... """)

Running the buildout gives us:

>>> print system(buildout)
Installing my.package.
Generated config '/sample-buildout/parts/projects/my.package.cfg'.

As we can see, the recipe generated the project configuration file under the projects directory in the parts:

>>> cat(join('parts', 'projects', 'my.package.cfg'))
[project]
...
name = my.package
repository = http://example.com/svn/my.package/trunk
...
slave-names = slave1
...
vcs = svn
<BLANKLINE>

If you have multiple similar projects you can define them within a single buildout section by providing multiple repository URLs. All the projects share the same options (except the repository URL). To further reduce repetition we’ve defined the base URL to our repository in the buildout section.

>>> write('buildout.cfg',
... """
... [buildout]
... parts = my_project
... svn = http://svn.example.com/svnroot
...
... [my_project]
... recipe = collective.buildbot:project
... slave-names = slave1
... vcs = svn
... repositories =
...    ${buildout:svn}/my.package/trunk
...    ${buildout:svn}/other.package/tags/1.2.3
...    ${buildout:svn}/third.package/branches/foobar
...    ${buildout:svn}/third.package/branches/another
... """)

When we run the buildout we can see that it generated a separate configuration file for each project representing a single repository:

>>> print system(buildout)
Uninstalling my.package.
Installing my_project.
Generated config '/sample-buildout/parts/projects/my.package.cfg'.
Generated config '/sample-buildout/parts/projects/other.package.cfg'.
Generated config '/sample-buildout/parts/projects/third.package.cfg'.
Generated config '/sample-buildout/parts/projects/third.package_2.cfg'.

Supported options

The recipe supports the following options:

vcs

The version control system. Defaults to svn. Currently only Subversion repositories are supported.

repositories

A sequence of newline separated URLs to the root of the Subversion repository containing the project code. Note: This is the root URL to the repository and not the full path to your project. You only need to provide one URL per repository, not per project.

splitter

A regexp used to parse paths analyzed by the poller. The regexp must return 2 groups. The only important one is the project name to match in a builder repository. (Default '(?P<project>\S+\/trunk|\S+\/branches\/[^\/]+)/(?P<relative>.*)')

hist-max

Number of history lines to look at (Default 100).

user

A svn user (Default None).

password

A valid svn password for the user (Default None).

poll-interval

Interval in seconds to check for changes (Default 600).

svn-binary

Path to the svn binary. Defaults to svn which should work if you have in your PATH.

Example usage

We can define a poller to make our buildbot aware of commits:

>>> write('buildout.cfg',
... """
... [buildout]
... parts = svnpoller
...
... [svnpoller]
... recipe = collective.buildbot:poller
... repositories = http://example.com/svn
... user = h4x0r
... password = passwd
... """)

>>> print system(buildout)
Installing svnpoller.
Generated config '/sample-buildout/parts/pollers/svnpoller.cfg'.

Poller generation. You can see here all the available options:

>>> cat(join('parts', 'pollers', 'svnpoller.cfg'))
[poller]
hist-max = 100
repository = http://example.com/svn
vcs = svn
user = h4x0r
svn-binary = svn
password = passwd
poll-interval = 60
<BLANKLINE>

You can also have the poller to observe multiple repositories.

>>> write('buildout.cfg',
... """
... [buildout]
... parts = svnpoller
...
... [svnpoller]
... recipe = collective.buildbot:poller
... repositories =
...     http://example.com/svn
...     http://otherexample.com/svn
...     http://other.server.com/svn
... user = h4x0r
... password = passwd
... """)

>>> print system(buildout)
Uninstalling svnpoller.
Installing svnpoller.
Generated config '/sample-buildout/parts/pollers/svnpoller_0.cfg'.
Generated config '/sample-buildout/parts/pollers/svnpoller_1.cfg'.
Generated config '/sample-buildout/parts/pollers/svnpoller_2.cfg'.
<BLANKLINE>

0.3.2 (2008-09-14)

• Add paster template to quickly generate a basic configuration [gawel]

• Fixed occurrences of email-notification-recipients to the singular form as used in most places. [hannosch]

• Added a mechanism to have username/password for Subversion authentication Which consists of a buildbot patch and a link to .httpauth on buildout side [tarek]

• Add dependency between projects. The build of one project can trigger the build of one other. [thefunny]

• Improve the virtual env creation for Windows (mingw) and Cygwin. Installation of eggs works with mingw, and we should get a python ../../bin/python for Cygwin as well (symlink to the python used to run buildout). [thefunny]

0.3.1 (2008-05-31)

• Fixed poller documentation and examples [mustapha]

• Fixed failed tests when your executable is called something other than python, e.g python2.4 [mustapha]

0.3.0 (2008-05-28)

• Use a custom scheduler to get poller working again [gawel]

• Add splitter option to the poller recipe [gawel]

• Added support for running PyFlakes on projects [dokai]

• Refactored project name extraction logic [dokai]

  • Added Git support

  • Added support for defining multiple projects that result in duplicate project names (e.g. projects referring to different branches in a Subversion repository.)

• Try to retrieve project name from svn urls [gawel]

• Use a random minute in cron-scheduler when we have more than one repository [gawel]

• Deactive virtualenv under cygwin, this doesn’t work [thefunny]

• ‘environment’ can be used to specify environment variable on slaves [thefunny]

• ‘eggs’ can be used to install extra eggs in slaves [thefunny]

• Refactored the functionality of the ‘projects’ recipe into the ‘project’ recipe and removed the ‘projects’ entry point. [dokai]

• Refactored the functionality of the ‘pollers’ recipe into the ‘poller’ recipe and removed the ‘pollers’ entry point. [dokai]

• Poller config files are now named after the section name, allowing multiple poller sections to be defined. [dokai]

0.2.1 (2008-05-21)

• Fixed a critical typo in the slave name configuration in fullexample.txt [dokai]

0.2.0 (2008-05-21)

• Added irc options so you can attach an irc bot to the master buildbot [mustapha]

• Allow public_html customization [gawel]

• Added custom about page to link to collective.buildout [gawel]

• Added support for Git repositories [dokai]

• Refactored the repository URL configuration. For Subversion, you should use only the repository option to specify a full URL to the desired branch (trunk, tag or branch) that will be built. For Git in addition to setting the repository option you can use the branch option to specify a specific branch to build. By default the master branch will be used for Git repositories. [dokai]

• Cleaned up a lot of redundant imports. [dokai]

• Updated the documentation and examples. [dokai]

• Deprecated the collective.buildbot:projects recipe [dokai]

• Fixed problem with missing twistd.log files on first run [dokai]

• Fixed bug that prevented the master from starting if there weren’t any SVN pollers configured. [dokai]

• Added new options periodic-scheduler and cron-scheduler to set up passive schedulers for projects. [dokai]

0.1.1 (2008-05-02)

• bugs fixes [gawel]

0.1.0 (xxxx-xx-xx)

• Created recipe with ZopeSkel [Gael Pasgrimaud].