Skip to main content

Objects and patterns for working with processes in Python using the multiprocessing library

Project description

https://img.shields.io/travis/borgstrom/offspring.svg https://img.shields.io/codecov/c/github/borgstrom/offspring.svg Latest PyPI version

This is a collection of objects and patterns for working with processes in Python using the multiprocessing library.

The main idea is that you express your unit of work as a simple method on an object and then when that object is instantiated the work will be run in a subprocess.

Use cases

Offspring was built to address the following use cases for running code in a subprocess.

Run something once

from offspring import Subprocess


class MyTask(Subprocess):
        def __init__(self, arg1):
                # this is run in the parent process and is used to prepare your object
                self.arg = arg

        def run(self):
                # this will be run in the child process and completes your work
                # ...


MyTask('this is arg1').wait()

Run in a loop

from offspring import SubprocessLoop


class MyTask(SubprocessLoop):
        def __init__(self, arg1):
                # this is run in the parent process and is used to prepare your object
                self.arg = arg

        def begin(self):
                # called at the start of the loop in your child process

        def loop(self):
                # called each loop iteration in your your child process
                # it should return a sleep duration until the next loop, or False to stop the loop

        def end(self):
                # called at the end of the loop in your child process


MyTask('this is arg1').wait()

Implementation details

.process

Each Subprocess object has a .process attribute that is the multiprocessing.Process object.

.wait

If you need to wait for your child process you can call .wait on your Subprocess object.

WAIT_FOR_CHILD

If set to True on your Subprocess class then a Pipe will be used to block the parent process until the child has started. This is useful when you want to ensure that your Subprocess object is started and .run is called even if the parent process exits quickly.

class MyTask(Subprocess):
        WAIT_FOR_CHILD = True

        def run(self):
                print("This will always print")

MyTask().wait()

The SubprocessLoop class does this to ensure that your object has begin & end called (loop may not be called as a TERM signal received during startup will prevent the loop from every actually completing other than begin & end).

TERMINATE_ON_SHUTDOWN

If set to False then when .shutdown is called on a Subprocess object the child process will not be terminated before being joined. This means that the parent will block until the child completes the .run function.

import time

class MyTask(Subprocess):
        TERMINATE_ON_SHUTDOWN = False

        def run(self):
                time.sleep(2)

# Note that we do not call .wait on the task here since we will automatically wait for the child
MyTask()

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

offspring-0.0.2.tar.gz (3.7 kB view details)

Uploaded Source

Built Distribution

offspring-0.0.2-py2.py3-none-any.whl (5.9 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file offspring-0.0.2.tar.gz.

File metadata

  • Download URL: offspring-0.0.2.tar.gz
  • Upload date:
  • Size: 3.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for offspring-0.0.2.tar.gz
Algorithm Hash digest
SHA256 1ddc392e73cad663616e7b6bbfe52bf681e6702333050d6ba677c3fd6e594454
MD5 c2f9ce737677bf7b124be99c8fc15b56
BLAKE2b-256 16fb913535aedb1d84697d162c6b82d13c63629f9f5c49955491501137425878

See more details on using hashes here.

File details

Details for the file offspring-0.0.2-py2.py3-none-any.whl.

File metadata

File hashes

Hashes for offspring-0.0.2-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 b90f60a6aeb8981a4b59011a4a16f7e30d09abb5761253640b303d14826ddf6e
MD5 14aa9678abbec4fb6c482b35883ebc99
BLAKE2b-256 7c0eaf77761737f686fb495fb2954d8b3b84cd00105b486ffcb90425cc1c2ac9

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page