z3c.recipe.usercrontab 1.5

The problem

When deploying applications, it can be useful to have maintenance tasks be started periodically. On Unix platforms this is usually done using cron which starts cronjobs. Adding cronjobs to the system-wide cron directory (for example by placing a file in /etc/cron.d) can be handled using the zc.recipe.deployment package, but it does not support adding cronjobs by normal users. (as /etc/cron.d usually is world-writable).

The solution

z3c.recipe.usercrontab interfaces with cron using crontab(1), and allows normal users to install their own cronjobs. This is done by having buildout add and remove cronjobs when installing and uninstalling packages.

How to use it

To use z3c.recipe.usercrontab you need to add the following to your buildout.cfg:

[mycronjob]
recipe = z3c.recipe.usercrontab
times = 0 12 * * *
command = echo nothing happens at noon

and finally add mycronjob to the parts line(s) of your buildout.cfg

To add a comment to your cron-entry:

[mycronjob]
recipe = z3c.recipe.usercrontab
times = 0 12 * * *
command = echo nothing happens at noon
comment = Run daily at noon

If you prefer to manually enable cronjobs, you can generate a cron-entry that is commented out by setting enabled to False:

[mycronjob]
recipe = z3c.recipe.usercrontab
times = 0 12 * * *
command = echo nothing happens at noon
enabled = false

After running the buildout, you can check the generated cron-entries via crontab -l.

Credits

Original authors: Jasper Spaans and Jan-Jaap Driessen.

Most recent versions and current maintainer: Reinout van Rees.

Entry handling

A user crontab manager manages a user’s crontab for one specific buildout part. The part ends up in the identifier. We’ll use ‘test’ here.

>>> c = UserCrontabManager(identifier='test')

In these tests, we can fake a crontab by filling the list of cron entries manually:

>>> c.crontab = ['@reboot echo "hello world"']
>>> print(c) # Handy shortcut
@reboot echo "hello world"

Now, we’re adding an entry to it using the official way. The entry is surrounded by markers:

>>> c.add_entry('@reboot echo "I just got added"')
>>> print(c)
@reboot echo "hello world"
<BLANKLINE>
# Generated by test
@reboot echo "I just got added"
# END test
<BLANKLINE>

Removing entries also works. As long as the “Generated by” markers are present, it doesn’t matter which entry you remove: everything surrounded by the markers is zapped:

>>> c.del_entry('bla bla') == 1
True
>>> print(c)
@reboot echo "hello world"

An entry can also include an explanatory comment line:

>>> c.add_entry('@reboot echo "I just got added"',
...     comment="This ought to happen first")
>>> print(c)
@reboot echo "hello world"
<BLANKLINE>
# Generated by test
# This ought to happen first
@reboot echo "I just got added"
# END test
<BLANKLINE>

Pre-0.6, a WARNING environment variable was used. An entry (which content matters now!) is found there:

>>> c.crontab = ['@reboot echo "hello world"',
...              'WARNING="Everything below is added by bla bla',
...              '@reboot echo "old entry 1"',
...              '@reboot echo "old entry 2"']
>>> print(c)
@reboot echo "hello world"
WARNING="Everything below is added by bla bla
@reboot echo "old entry 1"
@reboot echo "old entry 2"
>>> c.del_entry('@reboot echo "old entry 1"')
1
>>> print(c)
@reboot echo "hello world"
WARNING="Everything below is added by bla bla
@reboot echo "old entry 2"

Removing the last remaining entry under WARNING also removes the WARNING:

>>> c.del_entry('@reboot echo "old entry 2"')
1
>>> print(c)
@reboot echo "hello world"

Briefly in the 0.5 version, a ‘BUILDOUT’ environment variable was used for grouping items per buildout. Now for some up/downgrade testing. 0.5.1 removes the environment variable again. We’ll add an entry with such a (now deprecated) “grouping environment variable”. First the start situation:

>>> c.crontab=[
...     'WARNING="Everything below is added by bla bla',
...     'BUILDOUT=my/buildout',
...     '@reboot echo nothing happens']
>>> print(c)
WARNING="Everything below is added by bla bla
BUILDOUT=my/buildout
@reboot echo nothing happens

Doing anything (adding/removing) zaps BUILDOUT statement:

>>> c.del_entry('nonexisting')
0
>>> print(c)
WARNING="Everything below is added by bla bla
@reboot echo nothing happens

And just to make sure, deleting that entry empties out the whole file:

>>> c.del_entry('@reboot echo nothing happens')
1
>>> print(c)
<BLANKLINE>

Read/write crontab methods

Next, test the read_crontab and write_crontab methods; we’ll use cat and a temporary file to not modifiy the crontab of the user running these tests:

>>> import tempfile
>>> t = tempfile.NamedTemporaryFile('w')
>>> crontestfile = t.name
>>> dont_care = t.write("#dummy\n")

>>> c = UserCrontabManager(readcrontab="cat %s" % crontestfile,
...                        writecrontab="cat >%s" % crontestfile,
...                        identifier='test')
>>> c.read_crontab()
>>> a = repr(c)
>>> c.add_entry('# improbable entry')
>>> c.write_crontab()
>>> c.read_crontab()
>>> b =repr(c)
>>> a == b
False

Now, delete this entry again and make sure the old crontab is restored:

>>> c.del_entry('# improbable entry') == 1
True
>>> c.write_crontab()
>>> c.read_crontab()
>>> b = repr(c)
>>> a == b
True

Buildout recipe usage

Do the buildout shuffle:

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = foo
...
... [foo]
... recipe = z3c.recipe.usercrontab
... times = @reboot
... command = echo nothing happens
... readcrontab = cat %(crontest)s
... writecrontab = cat >%(crontest)s
... ''' % ( { 'crontest': crontestfile } ))

>>> import os
>>> 'Installing foo' in system(buildout)
True

Check that it really was added to the crontab:

>>> c.read_crontab()
>>> b = repr(c)
>>> a == b
False

>>> '@reboot\techo nothing happens' in c.crontab
True
>>> print(c)
# Generated by /sample-buildout [foo]
@reboot   echo nothing happens
# END /sample-buildout [foo]

Re-running buildout runs the crontab recipe even when there’s no change:

>>> output = system(buildout)
>>> 'Updating foo' in output or 'Installing foo' in output
True
>>> c.read_crontab()
>>> print(c)
# Generated by /sample-buildout [foo]
@reboot echo nothing happens
# END /sample-buildout [foo]

This means that a crontab is fixed up if we mucked it up by hand:

>>> c.crontab = []
>>> c.write_crontab()
>>> c.read_crontab()
>>> print(c)
>>> output = system(buildout)
>>> 'Updating foo' in output or 'Installing foo' in output
True
>>> c.read_crontab()
>>> print(c)
# Generated by /sample-buildout [foo]
@reboot echo nothing happens
# END /sample-buildout [foo]

You can also add a comment to the crontab entry:

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = foo
...
... [foo]
... recipe = z3c.recipe.usercrontab
... times = @reboot
... command = echo nothing happens
... comment = Step 1: mention that nothing happens
... readcrontab = cat %(crontest)s
... writecrontab = cat >%(crontest)s
... ''' % ( { 'crontest': crontestfile } ))
>>> 'Installing foo' in system(buildout)
True
>>> c.read_crontab()
>>> print(c)
# Generated by /sample-buildout [foo]
# Step 1: mention that nothing happens
@reboot echo nothing happens
# END /sample-buildout [foo]

An entry is by default enabled, leading to an active cronjob:

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = foo
...
... [foo]
... recipe = z3c.recipe.usercrontab
... times = @reboot
... command = echo nothing happens
... enabled = true
... readcrontab = cat %(crontest)s
... writecrontab = cat >%(crontest)s
... ''' % ( { 'crontest': crontestfile } ))
>>> 'Installing foo' in system(buildout)
True
>>> c.read_crontab()
>>> print(c)
# Generated by /sample-buildout [foo]
@reboot echo nothing happens
# END /sample-buildout [foo]

You can also generate an inactive cronjob that is commented out, by setting the enabled-option to False:

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = foo
...
... [foo]
... recipe = z3c.recipe.usercrontab
... times = @reboot
... command = echo nothing happens
... enabled = false
... readcrontab = cat %(crontest)s
... writecrontab = cat >%(crontest)s
... ''' % ( { 'crontest': crontestfile } ))
>>> 'Installing foo' in system(buildout)
True
>>> c.read_crontab()
>>> print(c)
# Generated by /sample-buildout [foo]
# @reboot echo nothing happens
# END /sample-buildout [foo]

Uninstall the recipe:

>>> write('buildout.cfg',
... '''
... [buildout]
... parts =
... ''' % ( { 'crontest': crontestfile } ))
>>> 'Uninstalling foo' in system(buildout)
True

And check that its entry was removed (i.e., the contents of the crontab are the same as when this test was started; in any case, the teardown from the testrunner makes sure the old situation is restored):

>>> c.read_crontab()
>>> b = repr(c)
>>> a == b
True

A second part installs fine:

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = foo bar
...
... [foo]
... recipe = z3c.recipe.usercrontab
... times = @reboot
... command = echo nothing happens
... readcrontab = cat %(crontest)s
... writecrontab = cat >%(crontest)s
...
... [bar]
... recipe = z3c.recipe.usercrontab
... times = @reboot
... command = echo something happens
... readcrontab = cat %(crontest)s
... writecrontab = cat >%(crontest)s
... ''' % ( { 'crontest': crontestfile } ))
>>> output = system(buildout)
>>> 'Installing foo' in output
True
>>> 'Installing bar' in output
True
>>> c.read_crontab()
>>> print(c)
<BLANKLINE>
# Generated by /sample-buildout [foo]
@reboot   echo nothing happens
# END /sample-buildout [foo]
<BLANKLINE>
<BLANKLINE>
# Generated by /sample-buildout [bar]
@reboot   echo something happens
# END /sample-buildout [bar]
<BLANKLINE>

Uninstalling also works fine

>>> write('buildout.cfg',
... '''
... [buildout]
... parts =
... ''' % ( { 'crontest': crontestfile } ))
>>> output = system(buildout)
>>> 'Uninstalling bar' in output
True
>>> 'Uninstalling foo' in output
True

Safety valves

If the section has been removed, nothing can be found by the uninstall. You get warnings that way:

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = foo
...
... [foo]
... recipe = z3c.recipe.usercrontab
... times = @reboot
... command = echo nothing happens
... readcrontab = cat %(crontest)s
... writecrontab = cat >%(crontest)s
... ''' % ( { 'crontest': crontestfile } ))

>>> import os
>>> 'Installing foo' in system(buildout)
True
>>> c.crontab = []
>>> c.write_crontab()
>>> write('buildout.cfg',
... '''
... [buildout]
... parts =
... ''' % ( { 'crontest': crontestfile } ))
>>> 'WARNING: Did not find a crontab-entry during uninstall' in system(buildout)
True

Another test: pre-0.6 config simulation:

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = foo
...
... [foo]
... recipe = z3c.recipe.usercrontab
... times = @reboot
... command = echo nothing happens
... readcrontab = cat %(crontest)s
... writecrontab = cat >%(crontest)s
... ''' % ( { 'crontest': crontestfile } ))

>>> import os
>>> 'Installing foo' in system(buildout)
True
>>> c.crontab = ['WARNING="Everything below is added by bla bla"',
...              'BUILDOUT=/somewhere/out/there',
...              '@reboot\techo nothing happens']
>>> c.write_crontab()
>>> write('buildout.cfg',
... '''
... [buildout]
... parts =
... ''' % ( { 'crontest': crontestfile } ))
>>> 'Running uninstall recipe' in system(buildout)
True
>>> c.read_crontab()
>>> print(c)
<BLANKLINE>

1.5 (2018-12-20)

• Officially support Python 3.5, 3.6, 3.7, PyPy and PyPy3.

1.4 (2015-10-29)

• Added an enabled option that’s on by default. By setting this to false, you can generate cronjobs that are commented out.

  This can be useful if you have to perform other tasks in your deployment before a cronjob can be switched on (by uncommenting manually or by a script). [WouterVH]

1.3 (2015-10-05)

• New ‘comment’ option. This way you can add a small explanatory one-line option to your crontab entry. [reinout]

1.2.1 (2015-09-11)

• Moved development to https://github.com/reinout/z3c.recipe.usercrontab [reinout]

• Made a few small fixes to get everything running on python 3. [reinout]

1.1 (2010-11-09)

• Append and prepend less white space per cron item, so you do not get increasing extra white space everytime you run bin/buildout. [maurits]

1.0 (2009-11-10)

• Only small documentation changes; version bumped to 1.0 to signal stability. [reinout]

0.7 (2009-08-24)

• The crontab now gets checked every time buildout runs, not only when there’s a change in the configuration. [reinout]

0.6.1 (2009-06-17)

• Documentation fixes. [reinout]

0.6 (2009-06-16)

• Removed essentially-unused complete environment variable handling. [reinout]

• Adding our entries with descriptive comments now: it includes the buildout file and the part name. [reinout]

0.5.1 (2009-06-16)

• Reverted the “BUILDOUT=…” environment variable, including migration. I’ll add a better way after this release. [reinout]

0.5 (2009-06-15)

• Added migration code for pre-0.5 entries without a BUILDOUT variable. [reinout]

• Added extra blank line in front of “BUILDOUT=…” variable to allow for better readability. [reinout]

• Added “BUILDOUT=….” as environment variable for every set of crontab lines handled by one buildout. This makes it much easier to spot what got added by which buildout (in case you have multiple) or which buildout at all (if you have no clue where the buildout can be found). [reinout]

0.4 (2008-02-25)

• Fix bug where UserCrontabs with empty readcrontab and writecrontab constructor arguments where broken

0.3 (2008-02-23)

• Renamed to z3c.recipe.usercrontab

• Add an option to change the command used to read and write crontabs

• Improved tests to not modify the real crontab

0.2 (2008-01-12)

• Warn if an entry cannot be removed in buildout uninstall

• Break if multiple entries would be removed in buildout uninstall

• Have del_entry return the number of removed

0.1 (2008-01-12)

• Initial release.