Skip to main content

Qamasu is Job Queue system that respects TheSchwartz.

Project description

Qamasu is JobQueue system that respects TheSchwartz.

Suited to load leveling.

Implemented using optimistic lock.

Requirements

  • Python>=2.6

  • Django>=1.0

Usage

Set Qamasu up!

Qamasu is a Django application.

You need add qamasu to your or new django project’s INSTALLED_APPS.

And //manage.py syncdb//.

Write your worker.

Define GRAB_FOR in seconds that is max time worker grabbed for a work.

Define def work_safely(manager, job): that is a work you need.

See sample worker in workers directory for detail.

Registration

You need add worker to abilities.

register_func insert fuction record into database table if not exist.

>>> from qamasu import Qamasu
>>> qamasu = Qamasu([])
>>> qamasu.register_func('workers.random_wait')

Queue!

Once fuction is registered to qamasu, you can enqueue jobs.

Add hundreds Queues.:

>>> from qamasu import Qamasu
>>> from random import uniform
>>> qamasu = Qamasu(['workers.random_wait',])
>>> for x in xrange(1,500):
      arg = dict(random_number=uniform(1,5))
      qamasu.enqueue('workers.random_wait', arg)

Add a highest-priority queue.:

>>> qamasu.enqueue('workers.random_wait', dict(random_number=uniform(1,5)), priority=1)

Work! Work! Work!

Process enqueued job.

  1. Instantiate Qamasu with availabilities.

  2. call work method. infinite loop inside this method.

work method process queue as FIFO.

>>> from qamasu import Qamasu
>>> qamasu = Qamasu(['workers.random_wait',])
>>> qamasu.work()

Use work_prioritizing method if you tend to process job respects to priority.

>>> from qamasu import Qamasu
>>> qamasu = Qamasu(['workers.random_wait',])
>>> qamasu.work_prioritizing()

Caution!

For MySQL backend

You must set worker’s transaction isolation level to read commited before working qamasu when you use InnoDB.

>>> from django.db import connection
>>> from qamasu import Qamasu
>>> connection.cursor().execute('set session transaction isolation level read committed')
>>> qamasu = Qamasu(['workers.random_wait',])
>>> qamasu.work()

Or you have to set transaction isolation level read committed. It’s global settings and dangerous.

[mysqld]
transaction-isolation=Read-Committed

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

qamasu-0.3.tar.gz (6.6 kB view details)

Uploaded Source

File details

Details for the file qamasu-0.3.tar.gz.

File metadata

  • Download URL: qamasu-0.3.tar.gz
  • Upload date:
  • Size: 6.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for qamasu-0.3.tar.gz
Algorithm Hash digest
SHA256 9fbf716a8cc46e40e3e41c7803ff49e68624c13646453fd81c7a8323dd649b90
MD5 e4058a2cad0e7489a96ff1372d2d677f
BLAKE2b-256 83a2743d580388b7e227b441b2a210c71e3a576d42e5c2db6213a69430e0c62f

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page