Skip to main content

Simple offline task queues.

Project description

Simple offline task queues. For Python.

“See you later, alligator.”

Requirements

  • Python 2.6+ or Python 3.3+

  • (Optional) redis for the Redis backend

  • (Optional) beanstalkc for the Beanstalk backend

Basic Usage

This example uses Django, but there’s nothing Django-specific about Alligator. You can use it with any Python code.:

from alligator import Gator

from django.contrib.auth.models import User
from django.shortcuts import send_email


# Make a Gator instance.
# Under most circumstances, you would configure this in one place &
# import that instance instead.
gator = Gator('redis://localhost:6379/0')


# The task itself.
# Nothing special, just a plain *undecorated* function.
def follow_email(followee_username, follower_username):
    followee = User.objects.get(username=followee_username)
    follower = User.objects.get(username=follower_username)

    subject = 'You got followed!'
    message = 'Hey {}, you just got followed by {}! Whoohoo!'.format(
        followee.username,
        follower.username
    )
    send_email(subject, message, 'server@example.com', [followee.email])


# An simple, previously expensive view.
@login_required
def follow(request, username):
    # You'd import the task function above.
    if request.method == 'POST':
        # Schedule the task.
        # Use args & kwargs as normal.
        gator.task(follow_email, request.user.username, username)
        return redirect('...')

Running Tasks

from alligator import Gator, Worker

gator = Gator('redis://localhost:6379/0')
# Consume & handle all tasks.
worker = Worker(gator)
worker.run_forever()

License

New BSD

Complex Usage (For Future Docs)

# We're re-using the above imports/setup.

def log_func(job):
    # A simple example of logging a failed task.
    if job.result != SUCCESS:
        logging.error("Job {} failed.".format(job.id))

# A context manager for supplying options
with gator.options(retries=3, async=settings.ASYNC_TASKS, on_error=log_func) as task:
    feeds_job = task(sketchy_fetch_feeds, timeout=30)

# Dependent tasks, will only run if the listed tasks succeed.
with gator.options(depends_on=[feeds_job]) as task:
    task(rebuild_cache)

# Delayed tasks (run in an hour).
with gator.options(run_after=60 * 60) as task:
    task(this_can_wait)

TODO

  • Tests

  • Docs

  • Scheduled tasks

  • Cancellable tasks

Project details


Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page