A process pool implementation in Twisted Matrix and AMP
Project description
ampoule - Python process pool for Twisted Matrix
=================================================
Ampoule is a process pool implementation written on top of Twisted
Matrix. Its name comes from the use of AMP as the default
communication protocol between the pool and all its children.
It's different from other alternative solutions because it provides an
API very close to that of the Twisted ThreadPool. As an helper
function it also provides a deferToAMPProcess function that creates
the ProcessPool and submits jobs to it.
Changes in this release
===============
- Fixed bug #259264, this fix introduces a number of changes in the
architecture of the project:
1. Removed childReactor argument form the process pool and added
a starter argument.
2. Introduced the concept of a starter object whose role is to
start subprocesses with given parameters. This makes it easier
to specify particular parameters (eg. new env variables) to the
child processes without needing to override many methods in the
pool using closures.
3. main.py is completely changed and now provides the ProcessStarter
object which is a default implementation of IStarter.
4. IStarter interface currently documents only 2 methods:
startAMPProcess
startPythonProcess
in the future it's possible that we will add an additional:
startProcess
that starts whichever process we want without requiring python,
also this might end up with the separation of ProcessPool in at
least 2 logical levels: the ProcessPool and a dispatcher that
talks with the children, in this way it would be possible to
create custom ProcessPools without changing much code or requiring
any special requirement on the children.
- Introduced a callRemote method on the ProcessPool that is basically
the same as doWork. Introduced for symmetry between all the RPC libraries
in Twisted.
- reactor short name and ampoule child class are now passed as the 2
last arguments rather than the first 2. So if you have written any
custom bootstrap code be sure to change sys.argv[1] and sys.argv[2]
into sys.argv[-2] and sys.argv[-1] respectively.
=================================================
Ampoule is a process pool implementation written on top of Twisted
Matrix. Its name comes from the use of AMP as the default
communication protocol between the pool and all its children.
It's different from other alternative solutions because it provides an
API very close to that of the Twisted ThreadPool. As an helper
function it also provides a deferToAMPProcess function that creates
the ProcessPool and submits jobs to it.
Changes in this release
===============
- Fixed bug #259264, this fix introduces a number of changes in the
architecture of the project:
1. Removed childReactor argument form the process pool and added
a starter argument.
2. Introduced the concept of a starter object whose role is to
start subprocesses with given parameters. This makes it easier
to specify particular parameters (eg. new env variables) to the
child processes without needing to override many methods in the
pool using closures.
3. main.py is completely changed and now provides the ProcessStarter
object which is a default implementation of IStarter.
4. IStarter interface currently documents only 2 methods:
startAMPProcess
startPythonProcess
in the future it's possible that we will add an additional:
startProcess
that starts whichever process we want without requiring python,
also this might end up with the separation of ProcessPool in at
least 2 logical levels: the ProcessPool and a dispatcher that
talks with the children, in this way it would be possible to
create custom ProcessPools without changing much code or requiring
any special requirement on the children.
- Introduced a callRemote method on the ProcessPool that is basically
the same as doWork. Introduced for symmetry between all the RPC libraries
in Twisted.
- reactor short name and ampoule child class are now passed as the 2
last arguments rather than the first 2. So if you have written any
custom bootstrap code be sure to change sys.argv[1] and sys.argv[2]
into sys.argv[-2] and sys.argv[-1] respectively.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
ampoule-0.0.5.tar.gz
(739.0 kB
view details)
File details
Details for the file ampoule-0.0.5.tar.gz
.
File metadata
- Download URL: ampoule-0.0.5.tar.gz
- Upload date:
- Size: 739.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | e40a325678e1809aa165a378ea26afd9d332ac8b87a56949930a7be0d1029c1f |
|
MD5 | 4cf847377bedc3a26fcfec3580374124 |
|
BLAKE2b-256 | 05ca3e6ad3f0d56174f2b4faa8d7ec688d5992544d1bf6ea31008964904e4853 |