Product that enable cron like jobs for plone
Project description
Introduction
collective.cron is a cron-like asynchronous tasks system based on top of plone.app.async and plone.app.registry. The implementation hasn’t not for now all the bells and wistles of a nice UI. However the simple interface do all the stuff And The underlying job manager works reliably. Finaly, you can register your tasks easily.
Note that at the moment, we have 100% test coverage except for the UI.
The design is modern and modular, imagine that you can even easily change from plone.app.async to another job system.
Design
collective.cron lets you register crons which run periodically in your system.
Each plone site has a crontab.
This crontab is used by many composants to execute the cron jobs.
We have a central dashboard which will list all tasks registered on the site crontab.
The tasks configuration is based on plone.app.registry but design to be replaceable (composant)
The tasks execution is based on plone.app.async but design to be also replaceable (composant)
The cron manager will ensure to restore all cron jobs for all plone sites at zope restart.
Definition of a crontab
A crontab is the collection of all cron registered to a plone site. A crontab can be (de)activated globally Each crontab sub element (the crontab, the crons & associated logs) defines a dump method which make a JSON representation of the object.
The major attributes for a crontab are:
crons: An ordered dict of crons. Key is the cron uid
activated: globally power switch for the crontab
manager: the manager is responsible for the crontab persistence
save(): save the crontab
save_cron(cron): save the cron
When a crontab is saved, it emits a ModifiedCrontabEvent.
Definition of a cron
The major attributes for a cron are:
uid: internal id for the crontab machinery
name: will be the queried name to search jobs
activated: the activation status of the cron
periodicity: give the next time execution
logs: give the last logs of the cron prior executions
crontab: A possibly null reference to the parent crontab
Definition of a log
The major attributes for a log are:
date: date of logging
status: status ::= NOTRUN | FAILURE | WARN | OK
message: the logs
The crontab registry manager
Based on top of plone.app.registry, collective.cron record the crontab current status in the site registry. It adapts a crontab.
activated: Boolean switch status of the crontab
cronsettings: the raw manager settings (.crontab, .activated)
crons: list of serialized strings representations of the crons
read_only: if true, changes will be a NOOP
When a record is touched (added, edited, removed), we fire an event to syncronize the queue.
The crontab manager
This composant is responsible when a CrontabSynchronisationEvent is fired to synchronise the crontab with the job queuing system. It will remove unrelated jobs and schedule new jobs. It adapts a plonesite and a crontab.
When the crontab save itself, its emits a ModifiedCrontabEvent which in turns is redirected as a CrontabSynchronisationEvent to let the manager synchronize the queue.
When the server restarts, a ServerRestartEvent is called to re-register any cron job that would have been wiped from the queue.
The cron manager
This composant is responsible forthe execution and presence in the queue of a particular cronjob.It can register or remove the job execution of a cron. This is a firendly proxy to the ‘Queue manager”
It adapts a plonesite and a cron.
When it register a cronjob, the job queued is a cron jobrunner wrapper responsible for:
Sending a StartedCronJobEvent
Running the relevant JobRunner (a named adapter adapting the plonesite, and the cron)
Sending a FinishedCronJobEvent
logging the execution
Scheduling the next execution
Definition of a jobrunner
- A cron jobrunner is a named adapter which:
adapts the plonesite and the current cron
implements IJobRunbner, and specially defines a run method.
A base class exists in collective cron, just inherit from it This is a complicated definition to have a class like this:
from collective.cron import crontab
class MyCronJob(crontab.Runner):
def run(self):
print "foo"
registered in zcml like that:
<adapter factory=".module.MyCronJob" name="mycronjob"/>
Annd then, you ll have to register cron called mycronjob in your plonesite,
The Queue manager
This composant will manage the jobs inside the job queue. You ll have enought methods to know for a specific cron if a job is present, what is its status, You can also register, or delete items from the running queue It adapts a plonesite.
The Crontab Queue Marker (plone.app.async specific)
Responsible to mark infos in the async queue to make posible the reload of jobs at Zope restart.
The buildout infrastructure
base.cfg -> base buildout informations
buildout.cfg -> base buildout for the current plone version
test-4.0.x.cfg -> test buildout for plone4.0
test-4.1.x.cfg -> test buildout for plone4.1
test-4.2.x.cfg -> test buildout for plone4.2
The most important things are in base.cfg. If you plan to integrate plone.app.cron to your buildout, please refer to the plone.app.async documentation.
For now we use the unreleased version of plone.app.async : https://github.com/plone/plone.app.async
collective.cron 1.0 => collective.cron 2.0
in 1.0, each cron task was a content. This was then tedious to replicate and maintain accross multiple instances and plone versions. One of the goal of collective.cron 2.0 is to avoid at most to have persistance, specially specialized contents to minimize all the common migration problems we have with an objects database. Thus a choice has been made to heavily use plone.app.registry as a configuration backend.
Thus, there is no migration prepared from collective.cron 1.0 to 2.0 It is up to you to do it. Specially, you will have to clean the database of all specific collective.cron 1.0 based & persistent content before upgrading. Indeed, As the design of tasks is really different, we can’t do any automatic migration.
First with collective.cron 1 in your buildout
Search, record settings then delete all IBackend content
Delete all jobresults & persistent contents
Cleanup all the zc.async queue
In a second time, deactivate collective.cron1 and activate collective.cron 2+ in your buildout
Adapt your adapters and content types to work with collective.cron 2.0 (inputs/mark items to work on)
add equivalent crons records to the crontab setting of the backends job
Credits
Companies
Contributors
Repository
Changelog
1.0 (unreleased)
First release
