zc.beforestorage 1.0

“before” option

To use before storages from ZConfig configuration files, you need to import zc.beforestorage and then use a before storage section.

>>> import ZODB.config
>>> storage = ZODB.config.storageFromString("""
...
... %import zc.beforestorage
...
... <before>
...     before 2008-01-21
...     <filestorage>
...         path my.fs
...     </filestorage>
... </before>
... """)

>>> storage
<Before: my.fs before 2008-01-21 00:00:00.000000>

>>> storage.close()

If we leave off the before option, we’ll use the current time:

>>> storage = ZODB.config.storageFromString("""
...
... %import zc.beforestorage
...
... <before>
...     <filestorage>
...         path my.fs
...     </filestorage>
... </before>
... """)

>>> storage
<Before: my.fs before 2008-01-21 18:22:49.000000>

>>> storage.close()

We can also give the option ‘now’ and get the current time.

>>> import ZODB.config
>>> storage = ZODB.config.storageFromString("""
...
... %import zc.beforestorage
...
... <before>
...     before now
...     <filestorage>
...         path my.fs
...     </filestorage>
... </before>
... """)

>>> storage
<Before: my.fs before 2008-01-21 18:22:53.000000>

>>> storage.close()

We can give the option ‘startup’ and get the time at startup.

>>> import ZODB.config
>>> storage = ZODB.config.storageFromString("""
...
... %import zc.beforestorage
...
... <before>
...     before startup
...     <filestorage>
...         path my.fs
...     </filestorage>
... </before>
... """)

>>> storage
<Before: my.fs before 2008-01-21 18:22:43.000000>
>>> import zc.beforestorage
>>> import ZODB.TimeStamp
>>> print(
...     str(zc.beforestorage.startup_time_stamp))
2008-01-21 18:22:43.000000
>>> storage.close()

“before-from-file” option

The “before-from-file” option can be used to preserve the changes file between restarts. It’s value is the absolute path to a file. If the file exists, the “before” time will be read from that file. If the file does not exist, it will be created and the current UTC time will be written to it

When used with a Changes file that does NOT have the “create=true” option set, the database will be preserved between restarts.

>>> import os.path
>>> import tempfile

>>> tempdir = tempfile.mkdtemp()
>>> before_file = os.path.join(tempdir, 'before-file')

Currently the file does not exist. So it’ll be created and written with the current time. In order to make this repeatable, we “monkeypatch” the “get_now” function in the module to return a fixed value:

>>> import datetime
>>> import zc.beforestorage

>>> def fake_get_utcnow():
...     return datetime.datetime(2008, 1, 1, 15, 0)
>>> orig_get_utcnow = zc.beforestorage.get_utcnow
>>> zc.beforestorage.get_utcnow = fake_get_utcnow

>>> os.path.exists(before_file)
False

>>> storage = ZODB.config.storageFromString("""
...
... %%import zc.beforestorage
...
... <before>
...     before-from-file %s
...     <filestorage>
...         path my.fs
...     </filestorage>
... </before>
... """ % before_file)

>>> storage
<Before: my.fs before 2008-01-01 15:00:00.000000>

>>> storage.close()

The file will now have been created:

>>> os.path.exists(before_file)
True

>>> f = open(before_file)
>>> f.read() == fake_get_utcnow().replace(microsecond=0).isoformat()
True

If we now write a new value to the file, the storage will be started with that time.

>>> f = open(before_file, 'w')
>>> _ = f.write('1990-01-01T11:11')
>>> f.close()

>>> storage = ZODB.config.storageFromString("""
...
... %%import zc.beforestorage
...
... <before>
...     before-from-file %s
...     <filestorage>
...         path my.fs
...     </filestorage>
... </before>
... """ % before_file)

>>> storage
<Before: my.fs before 1990-01-01 11:11:00.000000>

>>> storage.close()

If we restart the storage, the value from the file will be used.

>>> storage = ZODB.config.storageFromString("""
...
... %%import zc.beforestorage
...
... <before>
...     before-from-file %s
...     <filestorage>
...         path my.fs
...     </filestorage>
... </before>
... """ % before_file)

>>> storage
<Before: my.fs before 1990-01-01 11:11:00.000000>

>>> storage.close()

This will continue to happen until we remove the file. The “before_from_file” path is stored on the storage itself, so applications that use it have access to it.

>>> os.remove(storage.before_from_file)

>>> os.path.exists(before_file)
False

If we restart the storage again, a new file will be created.

>>> storage = ZODB.config.storageFromString("""
...
... %%import zc.beforestorage
...
... <before>
...     before-from-file %s
...     <filestorage>
...         path my.fs
...     </filestorage>
... </before>
... """ % before_file)

>>> storage
<Before: my.fs before 2008-01-01 15:00:00.000000>

>>> storage.close()

Note that unlike the “before” option, the “before-from-file” file cannot contain special values such as “now” or “startup”.

>>> f = open(before_file, 'w')
>>> _ = f.write('now')
>>> f.close()

>>> storage = ZODB.config.storageFromString("""
...
... %%import zc.beforestorage
...
... <before>
...     before-from-file %s
...     <filestorage>
...         path my.fs
...     </filestorage>
... </before>
... """ % before_file)
Traceback (most recent call last):
...
ValueError: 8-byte array expected

Note that only one of “before” or “before-from-file” options can be specified, not both:

>>> storage = ZODB.config.storageFromString("""
...
... %%import zc.beforestorage
...
... <before>
...     before 2008-01-01
...     before-from-file %s
...     <filestorage>
...         path my.fs
...     </filestorage>
... </before>
... """ % before_file)
Traceback (most recent call last):
  ...
ValueError: Only one of "before" or "before-from-file" options can be specified, not both

Cleanup…

>>> import shutil
>>> shutil.rmtree(tempdir)

>>> zc.beforestorage.get_utcnow = orig_get_utcnow

Blob Support

Before storage supports blobs if the storage it wraps supports blobs, and, in fact, it simply exposes the underlying storages loadBlob and temporaryDirectory methods.

>>> fs = ZODB.FileStorage.FileStorage('Data.fs')
>>> import ZODB.blob
>>> bs = ZODB.blob.BlobStorage('blobs', fs)
>>> db = ZODB.DB(bs)
>>> conn = db.open()
>>> conn.root()['blob'] = ZODB.blob.Blob()
>>> _ = conn.root()['blob'].open('w').write(b'data1')
>>> transaction.commit()

>>> bnow = zc.beforestorage.Before(bs)
>>> dbnow = DB(bnow)
>>> connnow = dbnow.open()
>>> rootnow = connnow.root()

>>> _ = conn.root()['blob'].open('w').write(b'data2')
>>> transaction.commit()

>>> print(rootnow['blob'].open().read().decode('utf-8'))
data1

>>> bnow.temporaryDirectory() == bs.temporaryDirectory()
True

>>> import ZODB.interfaces, zope.interface.verify
>>> zope.interface.verify.verifyObject(
...     ZODB.interfaces.IBlobStorage, bnow)
True

>>> bnow.close()

1.0 (2023-02-09)

• Add support for Python 3.9, 3.10, 3.11.

• Drop support for Python 2.7, 3.5, 3.6.

0.6 (2020-05-14)

• Add support for Python 3.5 through 3.8.

• Drop support for Python 3.3 and 3.4.

• Fix a long-standing bug in loadBefore´. The bug was revealed by testing against ZODB 5, for which loadBefore plays a bigger role.

0.5.1 (2013-10-25)

• Fix broken release

0.5.0 (2013-10-25)

Added ZODB4 and Python 3 support.

0.4.0 (2010-12-09)

Added a “before-from-file” option that can be used if the application wants to preserve beforestorage state between application restarts.

0.3.2 (2008-12-05)

Updated to work with both ZODB 3.8 and 3.9.

0.3.1 (2008-12-01)

Renamed lastTid to getTid to conform to the ZEO.interfaces.IServeable interface.

0.3.0 (2008-12-01)

Added Blob support.

0.2.0 (2008-03-05)

Added support for “now” and “startup” values to the before option when using ZConfig. The “now” value indicates that the before storage should provide a view of the base storage as of the time the storage is created. The “startup” value indicates that the before storage should provide a view of the base stoage as of process startup time. The later is especially useful when setting up more than once before storage in a single application, as it allows you to arrange that all of the storages provide consistent views without having to specify a time.

0.1.1 (2008-02-07)

Fixed a packaging bug that caused some files to be omitted.

0.1 (2008-01-??)

Initial release.