ZProc - Process on steroids
Project description
# ZProc - Process on steroids
source is short for [Zero](http://zguide.zeromq.org/page:all#The-Zen-of-Zero) - [Process](https://docs.python.org/3.6/library/multiprocessing.html#multiprocessing.Process)
>generally, "zero" refers to the culture of minimalism that permeates the project. We add power by removing complexity rather than by exposing new functionality.
zproc aims to reduce the pain of multi-processing by
- 🌠
- Sync-ing application state across all processes (without shared varialbes!).
- 🌠
- Giving you the freedom to build any combination of synchronous or asynchronous systems.
- 🌠
- Remembers to kill processes when exiting, for general peace.
# Documentation
[Read the docs](http://zproc.readthedocs.io/en/latest/)
# Example
###### `state` is NOT a shared variable!. It's actually a remote object that is wrapped to behave like a dict.
```python
# example.py
from time import sleep
import zproc
def child1(state, props):
state.get_when_change('foo')
print("child1: foo got updated, so I wake")
state['bar'] = 'xxx'
print('child1: I set bar to xxx')
print('child1: I exit')
def bar_equals_xxx(state):
return state.get('bar') == 'xxx'
def child2(state, props):
state.get_when(bar_equals_xxx)
print('child2: bar changed to xxx, so I wake')
print('child2: I exit')
ctx = zproc.Context()
ctx.process_factory(child1, child2, props='hello!')
ctx.start_all()
sleep(1)
ctx.state['foo'] = 'foobar'
print('child0: I set foo to foobar')
input()
print('child0: I exit')
```
###### output
```
child0: I set foo to foobar
child1: foo got updated, so I wake
child1: I set bar to xxx
child1: I exit
child2: bar changed to xxx, so I wake
child2: I exit
child0: I exit
```
# Inner Workings
- The process(s) communicate over zmq sockets, over `ipc://`.
- Zproc runs a zproc server, which is responsible for storing and managing the state.
- store the state whenever it is updated, by another process.
- transmitt the state whenever a process needs to access it.
- If a process wishes to synchronize at a certain condition, it can attach a handler to the zproc server.
- The zproc server will check the condition on all state-changes.
- If the condition is met, the zproc server shall open a tunnel to the application and send the state back.
- zmq sockets block you application until that tunnel is opened.
# Caveats
- The state only gets updated if you do it directly. This means that if you mutate objects in the state, they wont get updated in global state.
- It runs an extra daemonic server for managing the state. Its fairly lightweight though, and shouldn't add too much weight to your application.
- The state is required to be marshal compatible, which means:
> The following types are supported: booleans, integers, floating point numbers, complex numbers, strings, bytes, bytearrays, tuples, lists, sets, frozensets, dictionaries, and code objects, where it should be understood that tuples, lists, sets, frozensets and dictionaries are only supported as long as the values contained therein are themselves supported. The singletons None, Ellipsis and StopIteration can also be marshalled and unmarshalled
(from python [docs](https://docs.python.org/3/library/marshal.html))
# Known issues
- Processes inside processes are known to create wierd behavior like
- not being able to access state
- not shutting down properly on exit
# Install
`pip install zproc `
# Build documentation
assuming you have sphinx installed
```
cd docs
./build.sh
```
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
zproc-0.1.3.tar.gz
(8.5 kB
view details)
Built Distribution
zproc-0.1.3-py2.py3-none-any.whl
(17.6 kB
view details)
File details
Details for the file zproc-0.1.3.tar.gz.
File metadata
- Download URL: zproc-0.1.3.tar.gz
- Upload date:
- Size: 8.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 686cfba01d5d7b32468fdd0be94d2bf806ec6050777c8882aa260bb78061c41a |
|
| MD5 | e9e24bed54bdb247cdee2198c22f215e |
|
| BLAKE2b-256 | 194a2b4ddf65aa6e81bc23e9eb3f6050d55df28f513ce47bb7de16c710e0b042 |
File details
Details for the file zproc-0.1.3-py2.py3-none-any.whl.
File metadata
- Download URL: zproc-0.1.3-py2.py3-none-any.whl
- Upload date:
- Size: 17.6 kB
- Tags: Python 2, Python 3
- Uploaded using Trusted Publishing? No
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 6d5cbd606acd738743ce2abad9029bc5de2de2fab01318a60ebe9035ef750c28 |
|
| MD5 | 94aa323515ef386fae23bd5a8088ba73 |
|
| BLAKE2b-256 | 3a54023deca36450f04f7647f8fe8bf9d4223b245b3219720c83ed266614fcf0 |
