modelqueue 2.0.0

ModelQueue is an Apache2 licensed task queue based on Django models.

For example, in appname/models.py:

import modelqueue
from django.db import models

class Task(models.Model):
    data = models.TextField()
    status = modelqueue.StatusField(
        # ^-- Just a models.BigIntegerField
        db_index=True,
        # ^-- Index for faster queries.
        default=modelqueue.Status.waiting,
        # ^-- Waiting state is ready to run.
    )

And in appname/management/commands/process_tasks.py:

import modelqueue, time
from django.core.management.base import BaseCommand
from .models import Task

class Command(BaseCommand):

    def handle(self, *args, **options):
        while True:
            task = modelqueue.run(
                Task.objects.all(),
                # ^-- Queryset of models to process.
                'status',
                # ^-- Field name for model queue.
                self.process,
                # ^-- Callable to process model.
            )
            if task is None:
                time.sleep(1)
                # ^-- Bring your own parallelism/concurrency.

    def process(self, report):
        pass  # Process task models.

And in appname/admin.py:

class TaskAdmin(admin.TaskAdmin):
    actions = [*modelqueue.admin_actions('status')]
    # ^-- Change task status in admin.
    list_filter = [
        modelqueue.admin_list_filter('status'),
        # ^-- Filter tasks in admin by queue state.
    ]

    def get_changeform_initial_data(self, request):
        # v-- Automatically fill in status field when adding a new task.
        return {'status': modelqueue.Status.waiting()}

ModelQueue is a hazardous project. It takes a bad idea and makes it easy and effective. You may come to regret using your database as a task queue but it won’t be today!

Features

• Pure-Python

• Supports Django’s admin interface

• Tasks can be retried, aborted, and canceled

• Supports multiple attempts per task

• Bring your own parallelism with threading, multiprocessing, or asyncio

• Performance matters (add a single 64-bit field to models)

• Fully documented

• 100% test coverage

• Years of stress testing in production

• Developed on Python 3.10

• Compatible with all Django versions

• Tested on CPython 3.6, 3.7, 3.8, 3.9, 3.10

• Tested on Linux, Mac OS X, and Windows

$ python -m pip install modelqueue

>>> import modelqueue
>>> help(modelqueue)