Skip to main content

Guillotina addon to split actions into smaller bits

Project description

Introduction
============

`guillotina_hive` is a task runner whose main goal is to provide a mechanism
to iterate through all the content on a database as quickly as possible.

To accomplish that end, hive integrates with container orchestrators(k8s or nomad)
to schedule jobs on and retrieve results from those jobs.

Installation
------------

With pip:

pip install guillotina_hive


Guillotina configuration
------------------------

Example here with json::

"applications": ["guillotina_hive"],
"hive": {
"default_image": None,
"default_namespace": "hive",
"orchestrator": "k8s",
"cluster_config": {},
"guillotina_default": {
"entrypoint": None,
"volumes": None,
"volumeMounts": None,
"envFrom": None
},
'quota': None
}


Settings explained
~~~~~~~~~~~~~~~~~~

- default_image: default image to use for jobs to run with
- default_namespace: cluster namespace to use
- orchestrator: k8s or nomad
- cluster_config: what to configure aioclustermanager with
- guillotina_default: default configuration for guillotina jobs
- quota: define quote on cluster namespace


Defining a job
--------------

We use decorators to provide tasks::

from guillotina_hive.decorators import hive_task
@hive_task(name='something')
async def something(arg1, arg2):
return foobar


You can also use application settings::

{
"calculate-numbers": "guillotina_hive.tests.tasks.calculate_numbers"
}

1.0.15 (2018-05-19)
-------------------

- Downgrade guillotina test requirements


1.0.14 (2018-05-19)
-------------------

- upgrade guillotina in reqs


1.0.13 (2018-05-19)
-------------------

- Do not log cancelled errors on init
[vangheem]


1.0.12 (2018-05-05)
-------------------

- job name should not be the id we give it
[vangheem]

- Be able to customize default container args
[vangheem]

1.0.11 (2018-04-27)
-------------------

- Updating aioclustermanager dependency
[ramon]

1.0.10 (2018-04-26)
-------------------

- Remove print
[vangheem]


1.0.9 (2018-04-26)
------------------

- apply builtin tasks setup request object url as well
[vangheem]


1.0.7 (2018-04-25)
------------------

- improve user login for apply tasks
[vangheem]


1.0.4 (2018-04-24)
------------------

- Fix working with decorators
[vangheem]


1.0.3 (2018-04-24)
=======

- fix to work with guillotina 3
[vangheem]

1.0.2 (27/04/2018)
------------------

- Updating aioclustermanager dependency
[ramon]


1.0.1 (9/4/2018)
----------------

- Adding manifest
[ramon]

1.0.0 (5/4/2018)
----------------

- Initial version


Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

guillotina_hive-1.0.15.tar.gz (16.4 kB view hashes)

Uploaded Source

Built Distribution

guillotina_hive-1.0.15-py3-none-any.whl (25.7 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page