Wurlitzer

Capture C-level stdout/stderr pipes in Python via os.dup2.

For more details on why this is needed, please read this blog post.

Install

pip install wurlitzer

Usage

Capture stdout/stderr in pipes:

from wurlitzer import pipes

with pipes() as (out, err):
    call_some_c_function()

stdout = out.read()

Capture stdout/stderr in StringIO:

from io import StringIO
from wurlitzer import pipes, STDOUT

out = StringIO()
with pipes(stdout=out, stderr=STDOUT):
    call_some_c_function()

stdout = out.getvalue()

Forward C-level stdout/stderr to Python sys.stdout/stderr, which may already be forwarded somewhere by the environment, e.g. IPython:

from wurlitzer import sys_pipes

with sys_pipes():
    call_some_c_function()

Forward C-level output to Python Logger objects (new in 3.1). Each line of output will be a log message.

from wurlitzer import pipes, STDOUT
import logging

logger = logging.getLogger("my.log")
logger.setLevel(logging.INFO)
logger.addHandler(logging.FileHandler("mycode.log"))

with pipes(logger, stderr=STDOUT):
    call_some_c_function()

Forward C-level output to a file (avoids GIL issues with a background thread, new in 3.1):

from wurlitzer import pipes, STDOUT

with open("log.txt", "ab") as f, pipes(f, stderr=STDOUT):
    blocking_gil_holding_function()

Or even simpler, enable it as an IPython extension:

%load_ext wurlitzer

To forward all C-level output to IPython (e.g. Jupyter cell output) during execution.

Acknowledgments

This package is based on stuff we learned with @takluyver and @karies while working on capturing output from the Cling Kernel for Jupyter.

Wurlitzer?!

Wurlitzer makes pipe organs. Get it? Pipes? Naming is hard.