Skip to main content

AMQP based worker/master pattern framework

Project description

AMQP based RPC library for Tornado

Use cases

  • Task queue

  • Load balancing for different CPU-bound HTTP handlers

  • … other systems which involve RPC

Installation

pip install crew

Example

See the full example.

Usage

For example create your first app, and save as master.py:

import tornado.ioloop
import tornado.gen
import tornado.web
import tornado.log
import json
import tornado.options
from crew import TimeoutError, ExpirationError
from crew.master.tornado import Client

class MainHandler(tornado.web.RequestHandler):

    @tornado.gen.coroutine
    def get(self):
        resp = yield self.settings['crew'].call(None, priority=100)
        self.write(json.dumps(resp))

cl = Client()
application = tornado.web.Application(
    [
        (r"/", MainHandler),
    ],
    crew=cl,
    autoreload=True,
    debug=True,
)

if __name__ == "__main__":
    cl.connect()
    tornado.options.parse_command_line()
    application.listen(8888)
    tornado.ioloop.IOLoop.instance().start()

And create your first task, and save as worker.py:

# encoding: utf-8
from crew.worker import run, context, Task

@Task('test')
def long_task(req):
    context.settings.counter += 1
    return {"counter": context.settings.counter}

run(
    counter=0
)

After that run it:

$ python master.py &
$ python worker.py &
$ wait

Try to test it:

$ curl http://localhost:8888/

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

crew-0.9.1.tar.gz (11.0 kB view details)

Uploaded Source

File details

Details for the file crew-0.9.1.tar.gz.

File metadata

  • Download URL: crew-0.9.1.tar.gz
  • Upload date:
  • Size: 11.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for crew-0.9.1.tar.gz
Algorithm Hash digest
SHA256 e95928cb413b792ac8d45f20d717956f8c85f960eacf8eef7b6b09ddc6dba87a
MD5 af17f164db41d8be8e71b982bfa061c7
BLAKE2b-256 10e009a5830efe873169dbabd500bb09f975946732d70780d52838fc642da4c9

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page