repoze.filecat 0.2

Filecat
=======

This package provides application support for indexing and serving
content in a file system.

An HTTP API is available for querying operations.

Setup
-----

The Xapian database queue processor must be configured with an
indexer which points to a database file on disk, e.g.::

>>> indexer = repoze.filecat.index.create_indexer(db_path)
>>> ore.xapian.queue.QueueProcessor.start(indexer)

To start the directory observer (configured to scan every 3 seconds)::

>>> repoze.filecat.watch.start(directory, 3)

We can then query the Xapian database by opening a connection::

>>> connection = xapian.Connection(db_path)

Formats
-------

Currently supported file formats:

- JPEG (image/jpeg)
- ReStructuredText (text/x-rst)

Authors
-------

Malthe Borch <mborch@gmail.com>
Stefan Eletzhofer <stefan.eletzhofer@inquant.de>
Robert Marianski <rmarianski@gmail.com>

Application
===========

The filecat server application is configured with a ``path`` parameter
which points to a pool of files (a directory structure) and a ``host``
parameter which provides a URL at which a webserver is configured to
serve up the directory as static files.

API
---

All queries are performed using the following HTTP API::

method path description
----------------------------------------------------------
POST /purge Clear index
POST /add Add file to index

@path Relative filename

POST /update Reindex file

@path Relative filename

POST /remove Remove file from index

@path Relative filename

GET /query Perform query

Parameters are passed as-is
to the Xapian query engine.

Results
-------

Queries return a JSON structure which is first a two-tuple
(matches_estimated, results). The result set is a tuple of dicts:

@url The URL where this file can be downloaded
@mimetype MIME-type of the file
@metadata Dict-like structure with mimetype-specific metadata*





Changelog
=========

0.2 - (2009-08-03)
~~~~~~~~~~~~~~~~~~

- The JSON-output of a query is now a tuple of estimated number of
matches and the result set. [malthe]

- Fixed issue where global registry would not be properly
configured. [malthe]

- Handle JPEG-files without a XMP application segment (these are
indexed with a trival metadata tree). [malthe]

- Use XMP instead of IPCT/EXIF; this sheds the dependency on the
Hachoir library. [malthe]

- Added XMP extraction capabilities for JPEG-files. [malthe]

- Updated Xapian library versions in buildout configuration. [malthe]

0.1 - (2008-10-03)
~~~~~~~~~~~~~~~~~~

- Remove print statements. [seletz]

- Added tests for directory watch/observer. Fixed a bug where the observer would
emit absolute paths causing xapian to index absolute paths. [seletz]

- Hook watcher in. The configured ``static`` directory is now watched. Changes
are now added to the Xapian queue. [seletz]

- Created a directory observer thing which will allow us later to atomatically
wahtch a directory and issue Xapian operations. [seletz]

- Hooked in ``modify`` and ``remove`` views. [seletz]

- Changed query parser -- it now just uses the passed-in query string. Also
added ability to limit/batch query results. [seletz]

- Configured Xapian indexer. This has to be done separately from the BFG
components, because the Xapian indexer runs in its own thread and thus uses
the global component registry. [seletz]

- Hooked up ``add`` view. [seletz]

- Implemented ``query`` view. [seletz]

- Fixed ``routes``setup wrt. permissions. [seletz]

- Changed svn layout. [seletz]

- Added buildout. [seletz]