Skip to main content

Guillotina addon to split actions into smaller bits

Project description

guillotina_hive is a task runner whose main goal is to provide a mechanism to iterate through all the content on a database as quickly as possible.

To accomplish that end, hive integrates with container orchestrators(k8s or nomad) to schedule jobs on and retrieve results from those jobs.

Installation

With pip:

pip install guillotina_hive

Guillotina configuration

Example here with json:

"applications": ["guillotina_hive"],
"hive": {
    "default_image": None,
    "default_namespace": "hive",
    "orchestrator": "k8s",
    "cluster_config": {},
    "guillotina_default": {
        "entrypoint": None,
        "volumes": None,
        "volumeMounts": None,
        "envFrom": None
    },
    'quota': None
}

Settings explained

  • default_image: default image to use for jobs to run with

  • default_namespace: cluster namespace to use

  • orchestrator: k8s or nomad

  • cluster_config: what to configure aioclustermanager with

  • guillotina_default: default configuration for guillotina jobs

  • quota: define quote on cluster namespace

Defining a job

We use decorators to provide tasks:

from guillotina_hive.decorators import hive_task
@hive_task(name='something')
async def something(arg1, arg2):
    return foobar

You can also use application settings:

{
    "calculate-numbers": "guillotina_hive.tests.tasks.calculate_numbers"
}

1.0.9 (2018-04-26)

  • apply builtin tasks setup request object url as well [vangheem]

1.0.7 (2018-04-25)

  • improve user login for apply tasks [vangheem]

1.0.4 (2018-04-24)

  • Fix working with decorators [vangheem]

1.0.3 (2018-04-24)

  • fix to work with guillotina 3 [vangheem]

1.0.1 (9/4/2018)

  • Adding manifest [ramon]

1.0.0 (5/4/2018)

  • Initial version

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

guillotina_hive-1.0.9.tar.gz (16.1 kB view hashes)

Uploaded Source

Built Distribution

guillotina_hive-1.0.9-py3-none-any.whl (25.4 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page