gearhorn 0.0.3

subscribe_fanout unsubscribe_fanout

These queues will have a JSON payload consisting of a mapping with two keys: topic, client_id. The client_id will be kept as a set under the topic for use in sending messages to the appropriate subscriber queue. It should be unique for every intended recipient queue for every fanout request. Usually this will be unique per host.

fanout

This queue will have a JSON payload consisting of a mapping with two keys: topic, payload. The topic will be used to search for a list of subscriber ids. For each subscriber_id found, a copy of the payload will be sent to the queue named topic_subscriber_id. So if we had two subscribers to “officememos” with ids “bob” and “alice”, then a message to fanout with this payload:

{"topic": "officememos",
 "payload": "please go home early today."}

Optionally one can add a ‘unique’ key to make use of Gearman’s unique/coalescing features. Also the ‘background’ key can be used to set a truthy value, which will tell gearhorn not to wait for receivers before moving on to more messages.

Would result in the worker sending a copy of the payload to the queues “officememos_bob” and “officememos_alice”.

TODO

• True functional tests

• Make all the things configurable

  • Work queue name(s)

  • Tuning backlog and flush frequency

• Consider a work queue that workers can use to share the current sequence they should be using.

  • Would this be too racy or given the “best effort” is it enough to just try hard?

  • Do we need a UUID of some kind just to allow clients to detect if all the workers went away and sequences reset at 0?

• Add a client helper module which implements the sequence fetching to make it easier to write a consumer in python