Skip to main content

Non-blocking Python methods using decorators

Project description

MultiTasking: Non-blocking Python methods using decorators

Python version Travis-CI build status PyPi version PyPi status PyPi downloads CodeFactor Star this repo Follow me on twitter

MultiTasking is a tiny Python library lets you convert your Python methods into asynchronous, non-blocking methods simply by using a decorator.

Example

# example.py
import multitasking
import time
import random
import signal

# kill all tasks on ctrl-c
signal.signal(signal.SIGINT, multitasking.killall)

# or, wait for task to finish on ctrl-c:
# signal.signal(signal.SIGINT, multitasking.wait_for_tasks)

@multitasking.task # <== this is all it takes :-)
def hello(count):
    sleep = random.randint(1,10)/2
    print("Hello %s (sleeping for %ss)" % (count, sleep))
    time.sleep(sleep)
    print("Goodbye %s (after for %ss)" % (count, sleep))

if __name__ == "__main__":
    for i in range(0, 10):
        hello(i+1)

The output would look something like this:

$ python example.py

Hello 1 (sleeping for 0.5s)
Hello 2 (sleeping for 1.0s)
Hello 3 (sleeping for 5.0s)
Hello 4 (sleeping for 0.5s)
Hello 5 (sleeping for 2.5s)
Hello 6 (sleeping for 3.0s)
Hello 7 (sleeping for 0.5s)
Hello 8 (sleeping for 4.0s)
Hello 9 (sleeping for 3.0s)
Hello 10 (sleeping for 1.0s)
Goodbye 1 (after for 0.5s)
Goodbye 4 (after for 0.5s)
Goodbye 7 (after for 0.5s)
Goodbye 2 (after for 1.0s)
Goodbye 10 (after for 1.0s)
Goodbye 5 (after for 2.5s)
Goodbye 6 (after for 3.0s)
Goodbye 9 (after for 3.0s)
Goodbye 8 (after for 4.0s)
Goodbye 3 (after for 5.0s)

Settings

The default maximum threads is equal to the # of CPU Cores. This is just a rule of thumb! The Thread module isn’t actually using more than one core at a time.

You can change the default maximum number of threads using:

import multitasking
multitasking.set_max_threads(10)

…or, if you want to set the maximum number of threads based on the number of CPU Cores, you can:

import multitasking
multitasking.set_max_threads(multitasking.config["CPU_CORES"] * 5)

For applications that doesn’t require access to shared resources, you can set MultiTasking to use multiprocessing.Process() instead of the threading.Thread(), thus avoiding some of the GIL constraints.

import multitasking
multitasking.set_engine("process") # "process" or "thread"

Installation

Install multitasking using pip:

$ pip install multitasking --upgrade --no-cache-dir

Install multitasking using conda:

$ conda install -c ranaroussi multitasking

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

multitasking-0.0.10.tar.gz (8.2 kB view details)

Uploaded Source

File details

Details for the file multitasking-0.0.10.tar.gz.

File metadata

  • Download URL: multitasking-0.0.10.tar.gz
  • Upload date:
  • Size: 8.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: Python-urllib/3.8

File hashes

Hashes for multitasking-0.0.10.tar.gz
Algorithm Hash digest
SHA256 810640fa6670be41f4a712b287d9307a14ad849d966f06a17d2cf1593b66c3cd
MD5 0974fdc2e8da35d12b86c03ae78c7021
BLAKE2b-256 0240437c87afcda4d6bb6a88c91085dcde89eb563536d509edc949269f93b481

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page