Use Twisted from threaded applications
Project description
Crochet is an MIT-licensed library that makes it easy to use Twisted as a library in threaded applications. Instead of having Twisted take over your application, you can use Twisted or Twisted-based libraries like any other library in Flask, Django or any other blocking or threaded application.
Quick Example
Here’s an example of a program using Crochet:
import sys
from twisted.web.client import getPage
from crochet import setup, in_reactor
crochet.setup()
@in_reactor
def download_page(reactor, url):
return getPage(url)
result = download_page(sys.argv[1])
# wait() returns the result when it becomes available:
print result.wait()
Run on the command line:
$ python example.py http://google.com <!doctype html><html itemscope="itemscope" ... etc. ...
Notice that you get a completely blocking interface to Twisted, and do not need to run the Twisted reactor, the event loop, yourself.
Features
Runs Twisted’s reactor in a thread it manages.
Hooks up Twisted’s log system to the Python standard library logging framework. Unlike Twisted’s built-in logging bridge, this includes support for blocking Handler instances.
Provides a blocking API to eventual results (i.e. Deferred instances).
Using Crochet in Blocking Code
Using Crochet involves three parts: setup, setting up functions that call into Twisted’s reactor, and using those functions.
Setup
Crochet does a number of things for you as part of setup. Most significantly, it runs Twisted’s reactor in a thread it manages. Doing setup is easy, just call the setup() function:
from crochet import setup setup()
Since Crochet is intended to be used as a library, multiple calls work just fine; if more than one library does crochet.setup() only the first one will do anything.
Creating Asynchronous Functions
Now that you’ve got the reactor running, the next stage is defining some functions that will run inside the Twisted reactor thread. Twisted’s APIs are not thread-safe, and so they cannot be called directly from another thread. Instead, we write a function that is decorated with crochet.in_reactor:
from crochet import in_reactor
@in_reactor
def call_later(reactor, delay, f, *args, **kwargs):
reactor.callLater(delay, f, *args, **kwargs)
call_later(30, sys.stdout.write, "30 seconds have passed.\n")
Some points to notice:
The decorated function will get an extra argument at the start, the Twisted reactor.
The code will not run in the calling thread, but rather in the reactor thread.
The return result from a decorated object is a DeferredResult, which will be discussed in the next section.
Asynchronous Results
Since the code in the decorated function will be run in a separate thread, it cannot be returned normally. Moreover, the code may return a Deferred, which means the result may not be available until that Deferred fires. To deal with that, functions decorated with crochet.in_reactor return a crochet.DeferredResult instance.
DeferredResult has the following methods:
wait(timeout=None): Return the result when it becomes available; if the result is an exception it will be raised. If an optional timeout is given (in seconds), wait() will throw crochet.TimeoutError if the timeout is hit, rather than blocking indefinitely.
cancel(): Cancel the operation tied to the underlying Deferred. Many, but not all, Deferred results returned from Twisted allow the underlying operation to be canceled. In any case this should be considered a best effort cancellation.
stash(): Sometimes you want to store the DeferredResult in memory for later retrieval. stash() stores the DeferredResult in memory, and returns an integer uid that can be used to retrieve the result using crochet.retrieve_result(uid). This is specifically useful when you want to store a reference to the DeferredResult in a web session like Flask’s. See the included examples/downloader.py for an example of using this API.
Using Crochet from Twisted Applications
If your application is already planning on running the Twisted reactor itself, Crochet’s default behavior of running the reactor in a thread is a problem. To solve this, Crochet provides the no_setup() function, which causes future calls to setup() to do nothing. Thus, an application that will run the Twisted reactor but also wants to use a Crochet-using library must run it first:
from crochet import no_setup() no_setup() # Only now do we import libraries that might run crochet.setup(): import blockinglib # ... setup application ... from twisted.internet import reactor reactor.run()
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
