iw.fss 2.7.6

4.1 From FSS 2.5.x and older

The way we configure the storage and backup paths as well as the storage strategies HAVE CHANGED IN DEPTH. There’s no automatic migration from this version. Please create a configuration file that clones your old configlet preferences as indicated in FSS main configuration.

4.2 From iw.fss 2.7.1

From this version and later version, iw.fss registers upgrade steps in GenericSetup. When upgrading to a newer version of iw.fss, just have a look at the “Upgrade” tab of the portal_setup tool in your Plone site.

Warning, due to the removal of the portal_fss tool upgrading from 2.7.1 does not retain the RDF options (activated and specific script). If you used RDF with 2.7.1, you need to redo the same RDF setting in the FileSystemStorage config panel.

6.1 With buildout

This example speaks of itself:

[buildout]
...
eggs =
  ...
  iw.fss
  ...

[instance]
...
zcml =
  ...
  iw.fss
  iw.fss-meta
  ...

You may also use the iw.recipe.fss recipe egg to configure fully an instance.

6.2 From SVN reposo

• Inflate iw.fss package into your zope instance lib/python directory. I’m sure it’s already done.

• Add $INSTANCE_HOME/etc/package-includes/iw.fss-meta.zcml file with this line:

  <include package="iw.fss" file="meta.zcml"/>

• Add $INSTANCE_HOME/etc/package-includes/iw.fss-configure.zcml file with this line:

  <include package="iw.fss" />

6.3 Configuring your Plone sites

• Read carefully Storage strategies and FSS main configuration below and configure your instance in plone-filesystemstorage.conf accordingly. You may use the iw.recipe.fss recipe in your buildout.cfg to do this.

• Start your Zope instance

• In ZMI, go into your plone site and install the product using portal_quickinstaller

• Optionally, tweak some - self explained - preferences in the FileSystemStorage configuration panel of your Plone site.

7.1 Strategies overview

FSS comes with 4 storage strategies. Each strategy requires two directories:

• The storage directory stores the values of relevant fields in files according to the selected strategy.

• The backup directory that keeps the values of the fields of deleted contents for “Undo” purpose.

• Only one storage strategy can be selected on a given Plone site, but different Plone site in the same Zope instance may have distinct storage strategies.

• You need the best performances: choose the Flat storage strategy or Directory storage strategy.

• You need a storage that looks like your Plone site (in example to publish the storage directory in a read only samba or NFS share): choose the Site storage strategy 1 or the Site storage strategy 2.

• If you care about CMFEdition support (means that one of the content types using FSS is versioned with CMFEditions), you should not use the Site storage strategy 1 or the Site storage strategy 2.

See FSS main configuration to configure the best suited strategy on your Zope instance and Plone sites.

7.2 Flat storage strategy

All field values are stored in a flat structure. This strategy is the default one.

• Filename of these stored values: <uid of content>_<field name>

• Filename of rdf file: <uid of content>_<field name>.rdf

• Filename of backup values: <uid of content>_<field name>.bak

Rdf files are not backed up. They are automatically generated.

Example of storage:

fssDirectory
|
|- f42ad00adb7d4580f929d734bf1ed3ab_image
|
|- f42ad00adb7d4580f929d734bf1ed3ab_image.rdf
|
|- f42ad00adb7d4580f929d734bf1ed3ab_file
|
|- f42ad00adb7d4580f929d734bf1ed3ab_file.rdf

fssBackupDirectory
|
|- 9efeb7638fb35f5f9b9955da3e2dbfec_file.bak

7.3 Directory storage strategy

All field values are stored in a directory structure. Sub directories are defined on two level. First level of directory uses the 2 characters of content uid. Second level of directory uses the 4 characters of content uid. Backup files are stored in a flat structure.

• Filename of these stored values: <uid of content>_<field name>

• Filename of rdf file: <uid of content>_<field name>.rdf

• Filename of backup values: <uid of content>_<field name>.bak

RDF files are not backed up. They are automatically generated

Example of storage:

fssDirectory
|
|- f42
   |
   |- f42ad
      |
      |- f42ad00adb7d4580f929d734bf1ed3ab_image
      |
      |- f42ad00adb7d4580f929d734bf1ed3ab_image.rdf
      |
      |- f42ad00adb7d4580f929d734bf1ed3ab_file
      |
      |- f42ad00adb7d4580f929d734bf1ed3ab_file.rdf

fssBackupDirectory
|
|- 9efeb7638fb35f5f9b9955da3e2dbfec_file.bak

Depending in one hand on your file system performances facing a huge amount of files in the same directory, and in the other hand on the number of contents relying on FSS, you might choose this strategy or the Flat storage strategy.

7.4 Site storage strategy 1

All field values are stored in a directory structure mirroring structure of PloneSite. Backup files are stored in a flat structure.

• Filename of these stored values: Filename of field value or field name if not defined

• Filename of rdf file: <field name>.rdf

• Filename of backup values: <uid of content>_<field name>.bak

Rdf files are not backed up. They are automatically generated

Example of storage:

fssDirectory
|
|- members
   |
   |- john
      |
      |- dummy-document
         |
         |- image
         |  |
         |  |- moutain.jpg
         |  |
         |  |- image.rdf
         |
         |- file
            |
            |- diary.odt
            |
            |- file.rdf

fssBackupDirectory
|
|- 9efeb7638fb35f5f9b9955da3e2dbfec_file.bak

7.5 Site storage strategy 2

All field values are stored in a directory structure mirroring structure of PloneSite. Backup files are stored in a flat structure.

• Filename of these stored values: Filename of field value or field name if not defined

• Filename of rdf file: <field filename>.rdf

• Filename of backup values: <uid of content>_<field name>.bak

Rdf files are not backed up. They are automatically generated

Example of storage:

fssDirectory
|
|- members
   |
   |- john
      |
      |- dummy-document
         |
         |- fss.cfg
         |
         |- moutain.jpg
         |
         |- mountain.jpg.rdf
         |
         |- diary.odt
         |
         |- diary.odt.rdf

fssBackupDirectory
|
|- 9efeb7638fb35f5f9b9955da3e2dbfec_file.bak

9.1 FSS and RDF

As mentioned above in Storage strategies, FSS can optionally store RDF files in the back-end storage. These RDF files conform the DCMES-XML standard on XML expression of DublinCore elements.

If you select the Flat storage strategy, the RDF files may be used to build a files tree as close as possible of the Plone tree structure from the storage back-end, with the build_fs_tree.py utility:

$ cd /path/to/iw/fss/bin
$ python build_fs_tree.py --help

9.2 FSS maintenance

The “Maintenance” tab of FSS configuration panel shows some statistics about the amount of files managed by FSS.

In addition 2 additional buttons are provided:

• Update FSS files : Cleans up the back-end storage directories. Files not referenced from a content object are removed.

• Update RDF files : Builds all RDF files, in the event you chose to select the generetion of RDF files after having inserted relevant contents.

12.1 Using FSS in your content types

We assume that creating AT based content types is a familiar activity.

When creating the schema of a content type, you usually don’t mention the storage of your fields since AT provides a default ZODB storage. You can choose another storage, either another one that ships in the Archetypes bundle, or a third party storage like FSS.

Here is a small example that doesn’t need much comments:

# Usual Zope/CMF/Plone/Archetypes imports
...
from iw.fss.FileSystemStorage import FileSystemStorage
...
my_schema = Schema((
    FileField('file',
              ...
              storage=FileSystemStorage(),
              widget=FileWidget(...)
              ),
    ...
    ))
 ...

You may have a look at examples/FSSItem.py to see a demo content type using FSS. If you want to play with this content type:

$ $INSTANCE_HOME/bin/zopectl stop
$ export FSS_INSTALL_EXAMPLE_TYPES=1
$ $INSTANCE_HOME/bin/zopectl start

12.2 Monkey patching a third party content type

...
from iw.fss import zcml as fss_utils
from Products.ATContentTypes import atct
...
fss_utils.patchATType(atct.ATFile, ['file'])
...

<configure
  ...
  xmlns:zcml="http://namespaces.zope.org/zcml"
  xmlns:fss="http://namespaces.ingeniweb.com/filesystemstorage"
  ...>
  ...
  <fss:typeWithFSS
    zcml:condition="installed iw.fss"
    class="Products.ATContentTypes.atct.ATFile"
    fields="file" />
  ...
</configure>

12.3 Storing Plone standard content types with FSS

Include this ZCML element either in your instance package-includes directory or in any personal configure.zcml in your instance packages:

<include package="iw.fss" file="atct.zcml" />

This ZCML file wires FSS with the relevant content types that ship with Plone bundle:

• File

• Image

• News Item

12.4 Customize RDF info set

You can add data to the RDF info set via a FSS dedicated - and optional - hook.

Open the FSS configuration panel and give the name of your personal RDF data script, let’s call it fss_custom_rdf for this example. This script will add or change the RDF default data set.

<fss:rdf_date>2007-05-13 13:21:00</fss:rdf_date>

## Script (Python) "fss_custom_rdf"
##bind container=container
##bind context=context
##bind namespace=
##bind script=script
##bind subpath=traverse_subpath
##parameters=name, instance, properties, default_rdf
##title=
##
"""We add the date of RDF file generation"""

rdf_args = default_rdf.copy()

# Add the date of RDF generation
new_props = []
new_props.extend(rdf_args['field_props'])
new_props.append({'id': 'fss:rdf_date', 'value': DateTime().ISO()})
rdf_args['field_props'] = new_props

return rdf_args

## Script (Python) "fss_custom_rdf"
##bind container=container
##bind context=context
##bind namespace=
##bind script=script
##bind subpath=traverse_subpath
##parameters=name, instance, properties, default_rdf
##title=
##
from Products.CMFCore.utils import getToolByName

rdf_args = {}

# Set RDF charset
ptool = getToolByName(instance, 'portal_properties')
rdf_args['charset'] = ptool.site_properties.default_charset

# Set RDF namespaces
rdf_args['namespaces'] = (
    {'id': 'xmlns:rdf', 'value': 'http://www.w3.org/1999/02/22-rdf-syntax-ns#'},
    {'id': 'xmlns:dc', 'value': 'http://purl.org/dc/elements/1.1/'},
    {'id': 'xmlns:fss', 'value': 'http://namespace.ingeniweb.com/fss'},)

# Set field url
utool = getToolByName(instance, 'portal_url')
portal_path = utool.getPortalObject().getPhysicalPath()
portal_path_len = len(portal_path)
rel_path = '/'.join(instance.getPhysicalPath()[portal_path_len:])
fss_path = '%s/%s' % (rel_path, name)
rdf_args['field_url'] = fss_path

# Set RDF properties
props = (
   {'id': 'dc:title', 'value': instance.title_or_id()},
   {'id': 'dc:description', 'value': instance.Description()},
   {'id': 'dc:language', 'value': instance.Language()},
   {'id': 'dc:creator', 'value': instance.Creator()},
   {'id': 'dc:date', 'value': instance.modified()},
   {'id': 'dc:format', 'value': properties.get('mimetype', 'text/plain')},
   )
rdf_args['field_props'] = props

# Returns the RDF structure
return rdf_args