This package provides an easy API for moving the work out of the tornado process / event loop.
Project description
This package provides an easy API for moving the work out of the tornado process / event loop.
Currently implemented methods are:
execute the code in another server’s http hook (django implementation is included);
execute the code in a separate thread;
dummy immediate execution.
API example:
from django.contrib.auth.models import User
from slacker import adisp
from slacker import Slacker
from slacker.workers.django import DjangoWorker
AsyncUser = Slacker(User, DjangoWorker())
@adisp.process
def process_data():
# all the django ORM is supported; the query will be executed
# on remote end, this will not block the IOLoop
qs = AsyncUser.objects.filter(is_staff=True)[:5]
# execute the query and get the results
users = yield qs.fetch()
print users
(pep-342 syntax and adisp library are optional, callback-style code is also supported)
Installation
pip install tornado-slacker
Slackers
Slackers are special objects that are collecting operations (attribute access, calls, slicing) without actually executing them. Callable arguments must be picklable. Slackers also provide a method to apply the collected operations to a base object.
Any picklable object (including top-level functions and classes) can be wrapped into Slacker, e.g.:
from slacker import adisp
from slacker import Slacker
from slacker.workers import ThreadWorker
def task(param1, param2):
# do something blocking and io-bound
return results
async_task = Slacker(task, ThreadWorker())
# pep-342-style
@adisp.process
def process_data():
results = yield async_task('foo', 'bar').fetch()
print results
# callback style
def process_data2():
async_task('foo', 'bar').proceed(on_result)
def on_result(results):
print results
Workers
Workers are classes that decides how and where the work should be done:
slacker.workers.local.DummyWorker executes code in-place (this is blocking);
slacker.workers.local.ThreadWorker executes code in a thread from a thread pool;
slacker.workers.http.HttpWorker pickles the slacker, makes an async http request with this data to a given server hook and expects it to execute the code and return pickled results;
slacker.workers.django.DjangoHttpWorker is just a HttpWorker with default values for use with bundled django remote server hook implementation (slacker.django_backend).
In order to enable django hook, include ‘slacker.django_backend.urls’ into urls.py and add SLACKER_SERVER option with server address to settings.py.
SLACKER_SERVER is ‘127.0.0.1:8000’ by default so this should work for development server out of box.
Contributing
If you have any suggestions, bug reports or annoyances please report them to the issue tracker:
Source code:
Both hg and git pull requests are welcome!
Credits
Inspiration:
Third-party software: adisp (tornado adisp implementation is taken from brukva).
License
The license is MIT.
Bundled adisp library uses Simplified BSD License.
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
File details
Details for the file tornado-slacker-0.0.1.tar.gz.
File metadata
- Download URL: tornado-slacker-0.0.1.tar.gz
- Upload date:
- Size: 8.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 759e5a97b8175f3c712a366af93b3365bee5a946eff42f2671d64f714b27eed6 |
|
| MD5 | 6691d19163597af80b43ee613a6fde23 |
|
| BLAKE2b-256 | f673221c3d3aa46c385e374eb2021568b04f08a2f91798dfddeff771b4534f85 |
