Archiving, indexing and search for AMQP messages.
Project description
Please note: We do no longer actively support this package. If you still need it, please drop us a note, we can either transfer the repository or support you on a paid basis.
This package is an add-on application for gocept.amqprun that provides three features:
Archive queue messages by writing them to the filesystesm
Index those messages using ElasticSearch
Search for messages with a HTML/JavaScript GUI
Installation
gocept.amqparchive requires an ElasticSearch server. To set up the archive and index portion, add something like this to your gocept.amqprun configuration:
<eventlog>...</eventlog> <amqp-server>...</amqp-server> <worker> amount 2 component-configuration site.zcml </worker> <settings> gocept.amqparchive.elastic_hostname localhost:9200 </settings>
and in site.zcml:
<configure xmlns="http://namespaces.zope.org/zope" xmlns:amqp="http://namespaces.gocept.com/amqp"> <include package="gocept.amqprun" /> <include package="gocept.amqparchive" /> <amqp:writefiles routing_key="key.one key.two key.three" queue_name="archive" directory="/path/to/archive" pattern="{routing_key}/{date}/{msgid}-{unique}.xml" /> </configure>
The HTML/JavaScript GUI expects /elasticsearch to proxy to the ElasticSearch server, and /messages to point to the archive directory (/path/to/archive in our example). Here’s an nginx config snippet:
http { upstream elasticsearch { server localhost:9200; } server { listen localhost:8080; location /search/ { alias /path/to/gocept.amqparchive.egg/gocept/amqparchive/browser/; index index.html; } location /elasticsearch/ { proxy_pass http://elasticsearch/; } location /messages/ { alias /path/to/archive/; autoindex on; } }
Development
The source code is available in the mercurial repository at https://bitbucket.org/gocept/gocept.amqparchive
Please report any bugs you find at https://bitbucket.org/gocept/gocept.amqparchive/issues
Changelog
1.3.0.post1 (2018-11-27)
Fix PyPI page rendering.
1.3.0 (2018-09-19)
Adapt the Selenium-1 tests to Webdriver.
Update to current bootstrap.py.
Improve forward compatibility with Python 3.
Force pyes < 0.17 for now.
1.2.7 (2015-01-28)
Update to current bootstrap.py.
Move repository to bitbucket.org.
1.2.6 (2014-02-17)
Update to ElasticSearch 1.0 API.
1.2.5 (2014-02-14)
Allow configuring multiple elasticsearch hosts.
Fix bug in reindex_directory: we need to index the path relativ to the base directory, not the full filename.
Start reindexing in parallel while files are still being collected.
1.2.4 (2013-11-27)
Add parallel worker mode to reindex_directory.
Handle invalid XML input (#10864).
1.2.3 (2012-04-18)
Catch connection errors to ElasticSearch so they don’t break the normal message handling. Messages that have not been index due to this can still be indexed later on via reindex_directory (#9363).
1.2.2 (2012-03-29)
Make amqp server configurable for tests.
1.2.1 (2012-02-22)
Switch to plone.testing.
1.2.0 (2011-08-23)
Transform the XML body into a nested dict so ElasticSearch can index the fields individually.
1.1.0 (2011-08-23)
Add command-line script reindex_directory.
1.0.0 (2011-08-22)
first release.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Hashes for gocept.amqparchive-1.3.0.post1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 02f4803a359f55dbda3110496ead1951788915a11dbe9aac52d10823af9e29f4 |
|
MD5 | 273aa7e351c97da568cf62b81132424b |
|
BLAKE2b-256 | ecba66fee361c6817fdbd508d7b44180a998bc48ab118998bcb94f1260374214 |