ZProc - Process on steroids
Project description
ZProc lets you do shared-state multitasking without the perils of having shared-memory.
Behold, the power of ZProc:
# Some initialization
import zproc
ctx = zproc.Context(wait=True) # wait for processes in this context
ctx.state["cookies"] = 0
# Define "atomic" operations
@zproc.atomic
def eat_cookie(state):
"""Eat a cookie."""
state["cookies"] -= 1
print("nom nom nom")
@zproc.atomic
def bake_cookie(state):
"""Bake a cookie."""
state["cookies"] += 1
print("Here's a cookie!")
# Fire up processes
@ctx.call_when_change('cookies')
def cookie_eater(_, state):
"""Eat cookies as they're baked."""
eat_cookie(state)
for i in range(5):
bake_cookie(ctx.state)
Result:
Here's a cookie!
Here's a cookie!
nom nom nom
Here's a cookie!
nom nom nom
Here's a cookie!
nom nom nom
Here's a cookie!
nom nom nom
nom nom nom
(baker and eater run in different processes)
The core idea
ZProc tries to breathe new life into the archaic idea of shared-state multitasking by protecting application state with logic and reason.
Shared state is frowned upon by almost everyone, (mostly) due to the fact that memory is inherently dumb.
Like memory doesn't really care who's writing to it.
ZProc's state tries to keep a track of who's doing what.
The Goal
ZProc aims to make building multi-taking applications easier and faster for everyone, in a pythonic way.
It started out from the core idea of having a smart state -- eventually wanting to turn into a full-fledged framework for all things multitasking.
Install
$ pip install zproc
Or, from the development branch
$ pip install git+https://github.com/pycampers/zproc.git@next#egg=zproc
License: MIT License (MIT)
Requires: Python >=3.5
Documentation ( 
)
Read the docs
Examples
Features
-
🌠 Process management
- Process Factory
- Remembers to kill processes when exiting, for general peace. (even when they're nested)
- Keeps a record of processes created using ZProc.
- 🔖
-
🌠 Process Maps
- Automatically manages worker processes, and delegates tasks to them.
- 🔖
-
🌠 Asynchronous paradigms without
async def- Build a combination of synchronous and asynchronous systems, with ease.
- By watching for changes in state, without Busy Waiting.
- 🔖
-
🌠 Atomic Operations
- Perform an arbitrary number of operations on state as a single, atomic operation.
- 🔖
Caveats
-
The state only gets updated if you do it directly.
This means that if you mutate objects inside the state, they wont get reflected in the global state. -
The state should be pickle-able.
-
It runs an extra Process for managing the state.
Its fairly lightweight though, and shouldn't add too much weight to your application.
FAQ
-
Fast?
- Above all, ZProc is written for safety and the ease of use.
- However, since its written using ZMQ, it's plenty fast for most stuff.
- Run -> 🔖 for a taste.
-
Stable?
- Mostly. However, since it's still in the
0.x.xstage, you can expect some API changes.
- Mostly. However, since it's still in the
-
Production ready?
- Please don't use it in production right now.
-
Windows compatible?
- Probably?
Local development
git clone https://github.com/pycampers/zproc.git
cd zproc
pipenv install
pipenv install -d
pipenv shell
pip install -e .
pytest
Build documentation
Assuming you have sphinx installed (Linux)
cd docs
./build.sh
./build.sh loop # starts a build loop.
ZProc in the wild
Thanks
- Thanks to open logos for providing the wonderful ZProc logo.
- Thanks to pieter hintjens, for his work on the ZeroMQ library and for his amazing book.
- Thanks to tblib, ZProc can raise First-class Exceptions from the zproc server!
- Thanks to psutil, ZProc can handle nested procesess!
- Thanks to Kennith Rietz. His setup.py was used to host this project on pypi. Plus a lot of documentation structure is blatantly copied from his documentation on requests
ZProc is short for Zero Process.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file zproc-1.0.0a3.tar.gz.
File metadata
- Download URL: zproc-1.0.0a3.tar.gz
- Upload date:
- Size: 22.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.12.1 pkginfo/1.4.2 requests/2.19.1 setuptools/40.4.3 requests-toolbelt/0.8.0 tqdm/4.26.0 CPython/3.5.2
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 51585c698b2e7b2156cd0a2a25d43097a036cdf33dcfa2107bf38c475f1394df |
|
| MD5 | 6c26474d0a4f380dcc080310dd17ebd3 |
|
| BLAKE2b-256 | 818d18853bc52e36f62c3cb1697fe387df6b5905f0bee3a9db8f6c7c31264797 |
File details
Details for the file zproc-1.0.0a3-py2.py3-none-any.whl.
File metadata
- Download URL: zproc-1.0.0a3-py2.py3-none-any.whl
- Upload date:
- Size: 36.6 kB
- Tags: Python 2, Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.12.1 pkginfo/1.4.2 requests/2.19.1 setuptools/40.4.3 requests-toolbelt/0.8.0 tqdm/4.26.0 CPython/3.5.2
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 97afdf01f4d3376ea436577b67734012a1cb1f8cacf670063750558303f75d16 |
|
| MD5 | 9bdeb2ba39633630fad898b1fc7bacc6 |
|
| BLAKE2b-256 | 4148e15f28590e210a91723485549e62693036851b8d19eda825214bdeb4361b |
