Simple offline task queues.
Project description
Simple offline task queues. For Python.
“See you later, alligator.”
Requirements
Python 2.6+ or Python 3.3+
(Optional) redis for the Redis backend
(Optional) beanstalkc for the Beanstalk backend
Basic Usage
This example uses Django, but there’s nothing Django-specific about Alligator. You can use it with any Python code.:
from alligator import Gator from django.contrib.auth.models import User from django.shortcuts import send_email # Make a Gator instance. # Under most circumstances, you would configure this in one place & # import that instance instead. gator = Gator('redis://localhost:6379/0') # The task itself. # Nothing special, just a plain *undecorated* function. def follow_email(followee_username, follower_username): followee = User.objects.get(username=followee_username) follower = User.objects.get(username=follower_username) subject = 'You got followed!' message = 'Hey {}, you just got followed by {}! Whoohoo!'.format( followee.username, follower.username ) send_email(subject, message, 'server@example.com', [followee.email]) # An simple, previously expensive view. @login_required def follow(request, username): # You'd import the task function above. if request.method == 'POST': # Schedule the task. # Use args & kwargs as normal. gator.task(follow_email, request.user.username, username) return redirect('...')
Running Tasks
from alligator import Gator, Worker gator = Gator('redis://localhost:6379/0') # Consume & handle all tasks. worker = Worker(gator) worker.run_forever()
License
New BSD
Complex Usage (For Future Docs)
# We're re-using the above imports/setup. def log_func(job): # A simple example of logging a failed task. if job.result != SUCCESS: logging.error("Job {} failed.".format(job.id)) # A context manager for supplying options with gator.options(retries=3, async=settings.ASYNC_TASKS, on_error=log_func) as task: feeds_job = task(sketchy_fetch_feeds, timeout=30) # Dependent tasks, will only run if the listed tasks succeed. with gator.options(depends_on=[feeds_job]) as task: task(rebuild_cache) # Delayed tasks (run in an hour). with gator.options(run_after=60 * 60) as task: task(this_can_wait)
TODO
Tests
Docs
Scheduled tasks
Cancellable tasks