Skip to main content

Zc buildout recipe that provides a backup script

Project description

Code repository: https://ingeniweb.svn.sourceforge.net/svnroot/ingeniweb/iw.recipe.backup/

Change history

trunk (2009-11-25)

  • xxx [Ingeniweb]

0.2.1 (2008-11-24)

  • added ‘base-path’ option [fdupre] if set, use it instead buildout-dir, really usefull if datas are stored in another place in the system (NAS, NFS mount…).

0.2.0 (2008-05-08)

  • added ‘archive-before-backup’ option. [tdesvenain] True by default. If set to 0, buildout is not backuped before a restore

  • added ‘include-folders’ option, that allows to limit backup on some directories [tdesvenain]

  • create backups dir if not exists [tdesvenain]

  • add format option instead of archive-root-name [gawel]

  • fixing test so it passes on all machines [tarek]

0.1.2 (2008-04-11)

  • Added the fs-location option, that allows doing a safe backup of a running Data.fs. [tarek]

0.1.1 (2008-03-21)

  • Added the exclude-folders option

  • Now asks a question before restoring. It is more a conveniency since everything is backed up in any case. [tarek]

0.1.0 (2008-03-12)

  • Initial implementation. [tarek]

  • Created recipe with ZopeSkel [Ingeniweb].

Detailed Documentation

Supported options

The recipe supports the following options:

backup-script-name

Name of the backup script. Default: backup

restore-script-name

Name of the restore script. Default: restore

format

Formated name of the archive. Default to %(name)s-%(year)s-%(month)s-%(day)s-%(hour)s-%(minute)s where name is the section name

exclude-folders

Names of folder to avoid backing up. Relative to buildout root.

include-folders

If set, back up those folders only. Relative to buildout root.

fs-location

If given, indicates the path of the file system storage. The recipe will carefully copy it when doing a backup, by using a transaction-level read of the file. This means that you can launch a backup without stopping the Zope server.

target-folder

Folder where the archives are stored. Mandatory

log-file

File where all calls are recorded. Mandatory

archive-before-restore

If not set to 0, do a complete backup before a restore

prompt-before-restore

If not set to 0, prompt before a restore

base-path

If set, backup this path instead base on instance path

Example usage

We’ll start by creating a buildout that uses the recipe:

>>> import os
>>> root =  os.path.split(sample_buildout)[0]
>>> if root == '':
...     root = '.'

Let’s copy a real Data.fs in our buildout:

>>> import shutil
>>> data_fs = os.path.join(test_dir, 'Data.fs')
>>> shutil.copyfile(data_fs, join(sample_buildout, 'Data.fs'))

>>> write('buildout.cfg',
... """
... [buildout]
... parts = backup
... index = http://pypi.python.org/simple
...
... [backup]
... recipe = iw.recipe.backup
...
... format = %(format)s
... target-folder = %(root)s
... log-file = %(root)s/backup.log
... fs-location = ${buildout:directory}/Data.fs
... """ % {'root': root, 'format':'%(name)s-%(year)s-%(month)s-%(day)s'})

Let’s run the buildout:

>>> null = system(buildout)

Let’s see what we got in the backup script:

>>> print open(join(sample_buildout, 'bin', 'backup')).read()
#!...
<BLANKLINE>
import sys
sys.path[0:0] = [
  ...
  ]
<BLANKLINE>
import iw.recipe.backup.archive
<BLANKLINE>
if __name__ == '__main__':
    iw.recipe.backup.archive.archive_buildout(('/sample-buildout', '%(name)s-%(year)s-%(month)s-%(day)s', '..._TEST_', '/backup.log', [], [], '/sample-buildout/Data.fs', {'name': 'backup'}))
<BLANKLINE>

Ok, let’s call it to backup the current buildout:

>>> print system(join(sample_buildout, 'bin', 'backup'))
Starting the backup...
Archived in /backup-XXXX-XX-XX.tar.gz
...
<BLANKLINE>

We should have a log file generated as well:

>>> print open(join(root, 'backup.log')).read()
20... Starting the backup...
20... INFO Archived in /backup-XXXX-XX-XX.tar.gz

We also have a restore feature:

>>> print system(join(sample_buildout, 'bin', 'restore'))
Usage: ...restore archive_name
<BLANKLINE>

Let’s set the user input:

>>> from iw.recipe.backup.testing import set_input
>>> set_input('Y')

Oh right, the restore script takes the name of the archive:

>>> import glob
>>> arc = glob.glob('%s/*.tar.gz' % root)[0]
>>> print system(join(sample_buildout, 'bin', 'restore %s' % arc))
Are you sure you want to restore ? Every data will be lost ! (y/N)  Y
Archiving current folder before restoring
Starting the backup...
Archived in /before-restore-XXXX-XX-XX-XX-XX.tar.gz
Starting the restore...
Archive restored in /sample-buildout
...
<BLANKLINE>

And a restore always makes an archive on the current folder before it is applied, to make sure nothing is never lost.

There’s also something quite important: make sure the archive and log files are not located in the buildout !:

>>> write('buildout.cfg',
... """
... [buildout]
... parts = backup
... index = http://pypi.python.org/simple
...
... [backup]
... recipe = iw.recipe.backup
...
... target-folder = %(root)s
... log-file = %(root)s/backup.log
... """ % {'root': sample_buildout})

>>> print system(buildout)
Uninstalling backup.
Installing backup.
While:
  Installing backup.
<BLANKLINE>
An internal error occured due to a bug in either zc.buildout or in a
recipe being used:
...
ValueError: Cannot backup within the buildout ! Check your values
<BLANKLINE>

A bit of cleaning:

>>> import glob
>>> arc = glob.glob('%s/*.tar.gz' % root)
>>> for f in arc:
...     os.remove(f)

We can also exclude some folders from being archived:

>>> os.mkdir(join(sample_buildout, 'not'))
>>> open(join(sample_buildout, 'not', 'f'), 'w').write('me file')

>>> os.mkdir(join(sample_buildout, 'neh'))

>>> write('buildout.cfg',
... """
... [buildout]
... parts = backup
... index = http://pypi.python.org/simple
...
... [backup]
... recipe = iw.recipe.backup
...
... target-folder = %(root)s
... log-file = %(root)s/backup.log
... fs-location = ${buildout:directory}/Data.fs
... exclude-folders =
...     %(sample_buildout)s/not
...     %(sample_buildout)s/neh
... """ % {'root': root, 'sample_buildout': sample_buildout})

Running the buildout again:

>>> print system(buildout+' -D')
Installing backup.
Generated script '...backup'.
Generated script '...restore'.
<BLANKLINE>

Let’s backup:

>>> print system(join(sample_buildout, 'bin', 'backup'))
Starting the backup...
Archived in /backup-XXXX-XX-XX-XX-XX.tar.gz
...
<BLANKLINE>

Let’s remove the folder and the Data.fs:

>>> import shutil
>>> shutil.rmtree(join(sample_buildout, 'not'))
>>> os.rmdir(join(sample_buildout, 'neh'))
>>> os.remove(join(sample_buildout, 'Data.fs'))

Let’s restore:

>>> arc = glob.glob('%s/*.tar.gz' % root)[0]
>>> print system(join(sample_buildout, 'bin', 'restore %s' % arc))
Are you sure you want to restore ? Every data will be lost ! (y/N)  Y
...
<BLANKLINE>

And make sure the not folder is not back !

>>> os.path.exists(join(sample_buildout, 'not'))
False
>>> os.path.exists(join(sample_buildout, 'neh'))
False

And the Data.fs is back, so we’re OK:

>>> os.path.exists(join(sample_buildout, 'Data.fs'))
True

We can also restrict archive on some folders

>>> os.mkdir(join(sample_buildout, 'not'))
>>> open(join(sample_buildout, 'not', 'f'), 'w').write('me file')

>>> os.mkdir(join(sample_buildout, 'neh'))
>>> open(join(sample_buildout, 'neh', 'f'), 'w').write('me file')

>>> os.mkdir(join(sample_buildout, 'neh/not'))
>>> open(join(sample_buildout, 'neh/not', 'f'), 'w').write('me file')

>>> write('buildout.cfg',
... """
... [buildout]
... parts = backup
... index = http://pypi.python.org/simple
...
... [backup]
... recipe = iw.recipe.backup
...
... target-folder = %(root)s
... log-file = %(root)s/backup.log
... fs-location = ${buildout:directory}/Data.fs
... exclude-folders =
...     %(sample_buildout)s/not
...     %(sample_buildout)s/neh/not
... include-folders =
...     %(sample_buildout)s/neh
... prompt-before-restore = 0
... archive-before-restore = 0
... """ % {'root': root, 'sample_buildout': sample_buildout})

Running the buildout again:

>>> print system(buildout+' -D')
Uninstalling backup.
Installing backup.
Generated script '...backup'.
Generated script '...restore'.
<BLANKLINE>

Let’s see what we got in the backup script:

>>> print open(join(sample_buildout, 'bin', 'backup')).read()
#!...
<BLANKLINE>
import sys
sys.path[0:0] = [
  ...
  ]
<BLANKLINE>
import iw.recipe.backup.archive
<BLANKLINE>
if __name__ == '__main__':
    iw.recipe.backup.archive.archive_buildout(('/sample-buildout', '...', '..._TEST_', '/backup.log', ['/sample-buildout/not', '/sample-buildout/neh/not'], ['/sample-buildout/neh'], '/sample-buildout/Data.fs', {'name': 'backup'}))
<BLANKLINE>

Let’s backup:

>>> print system(join(sample_buildout, 'bin', 'backup'))
Starting the backup...
Archived in /backup-XXXX-XX-XX-XX-XX.tar.gz
<BLANKLINE>

Let’s remove the folder and the Data.fs:

>>> import shutil
>>> shutil.rmtree(join(sample_buildout, 'not'))
>>> shutil.rmtree(join(sample_buildout, 'neh'))
>>> os.remove(join(sample_buildout, 'Data.fs'))

Let’s restore:

>>> arc = glob.glob('%s/*.tar.gz' % root)[0]
>>> rest = os.popen(join(sample_buildout, 'bin', 'restore %s' % arc))
And make sure the not folder and ‘neh/not’ are not back and neh is back !
>>> os.path.exists(join(sample_buildout, 'not'))
False
>>> os.path.exists(join(sample_buildout, 'neh/not'))
False
>>> os.path.exists(join(sample_buildout, 'neh'))
True

Contributors

Ingeniweb, Author

Download

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

iw.recipe.backup-0.2.1.tar.gz (12.9 kB view hashes)

Uploaded Source

Built Distribution

iw.recipe.backup-0.2.1-py2.4.egg (22.6 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page