Innerscope

innerscope exposes the inner scope of functions and offers primitives suitable for creating pipelines. It explores a design space around functions, dictionaries, and classes.

A function can be made to act like a dictionary:

@innerscope.call
def info():
    first_name = 'Erik'
    last_name = 'Welch'
    full_name = f'{first_name} {last_name}'
    return 'success!'

>>> info['first_name']
'Erik'
>>> info['full_name']
'Erik Welch'
>>> info.return_value
'success!'

Sometimes we want functions to be more functional and accept arguments:

if is_a_good_idea:
    suffix = 'the amazing'
else:
    suffix = 'the bewildering'

@innerscope.callwith(suffix)
def info_with_suffix(suffix=None):
    first_name = 'Erik'
    last_name = 'Welch'
    full_name = f'{first_name} {last_name}'
    if suffix:
        full_name = f'{full_name} {suffix}'

>>> info_with_suffix['full_name']
'Erik Welch the bewildering'

Cool!

But, what if we want to reuse the data computed in info? We can control exactly what values are within scope inside of a function (including from closures and globals; more on these later). Let's bind the variables in info to a new function:

@info.bindto
def add_suffix(suffix):
    full_name = f'{first_name} {last_name} {suffix}'

>>> scope = add_suffix('the astonishing')
>>> scope['full_name']
'Erik Welch the astonishing'

add_suffix here is a ScopedFunction. It returns a Scope, which is the dict-like object we've already seen.

scoped_function ftw!

Except for the simplest tasks (as with call and callwith above), using scoped_function should usually be preferred.

# step1 becomes a ScopedFunction that we can call
@scoped_function
def step1(a):
    b = a + 1

>>> scope1 = step1(1)
>>> scope1 == {'a': 1, 'b': 2}
True

# Bind any number of mappings to variables (later mappings have precedence)
@scoped_function(scope1, {'c': 3})
def step2(d):
    e = max(a + d, b + c)

>>> step2.outer_scope == {'a': 1, 'b': 2, 'c': 3}
True
>>> scope2 = step2(4)
>>> scope2 == {'a': 1, 'b': 2, 'c': 3, 'd': 4, 'e': 5}
True
>>> scope2.inner_scope == {'d': 4, 'e': 5}
True

Suppose you're paranoid (like me!) and want to control whether a function uses values from closures or globals. You're in luck!

global_x = 1

def f():
    closure_y = 2
    def g():
        local_z = global_x + closure_y
    return g

# If you're the trusting type...
>>> g = f()
>>> innerscope.call(g) == {'global_x': 1, 'closure_y': 2, 'local_z': 3}
True

# And for the intelligent...
>>> paranoid_g = scoped_function(g, use_closures=False, use_globals=False)
>>> paranoid_g.missing
{'closure_y', 'global_x'}
>>> paranoid_g()

- NameError: Undefined variables: 'global_x', 'closure_y'.
- Use `bind` method to assign values for these names before calling.

>>> new_g = paranoid_g.bind({'global_x': 100, 'closure_y': 200})
>>> new_g.missing
set()
>>> new_g() == {'global_x': 100, 'closure_y': 200, 'local_z': 300}
True

How?

This library required surprisingly little magic at first. Perhaps I'll explain it some day. I mean, it does modify the bytecode, so it's a little magical and sinful, but it does so in a very reliable way, so you should feel comfortable using this library.

Why?

It's all @mrocklin's fault for asking a question. innerscope is exploring a data model that could be convenient for running code remotely with dask. I bet it would even be useful for building pipelines with dask.

This library is totally awesome and you should use it and tell all your friends 😉 !