lazy_python 0.1.5

lazy_function

lazy_function takes a python function and returns a new function that is the lazy version. This can be used as a decorator.

Example:

@lazy_function
def f(a, b):
    return a + b

Calling f(1, 2) will return a thunk that will add 1 and 2 when it needs to be strict. Doing anything with the returned thunk will keep chaining on more computations until it must be strictly evaluated.

Lazy functions allow for lexical closures also:

@lazy_function
def f(a):
    def g(b):
        return a + b
    return g

When we call f(1) we will get back a thunk like we would expect; however, this thunk is wrapping the function g. Because g was created in a lazy context, it will also be a lazy_function implicitly. This means that type(f(1)(2)) is thunk; but, f(1)(2) == 3.

We can use strict to strictly evaluate parts of a lazy function, for example:

>>> @lazy_function
... def no_strict():
...    print('test')
...
>>> strict(no_strict())

In this example, we never forced print, so we never saw the result of the call. Consider this function though:

>>> @lazy_function
... def with_strict():
...    strict(print('test'))
...
>>> strict(with_strict())
test
>>> result = with_strict()
>>> strict(result)
test

Here we can see how strict works inside of a lazy function. strict causes the argument to be strictly evaluated, forcing the call to print. We can also see that just calling with_strict is not enough to evaluate the function, we need to force a dependency on the result.

This is implemented at the bytecode level to frontload a large part of the cost of using the lazy machinery. There is very little overhead at function call time as most of the overhead was spent at function creation (definiton) time.

run_lazy

We can convert normal python into lazy python with the run_lazy function which takes a string, the ‘name’, globals, and locals. This is like exec for lazy python. This will mutate the provided globals and locals so that we can access the lazily evaluated code.

Example:

>>> code = """
print('not lazy')
strict(print('lazy'))
"""
>>> run_lazy(code)
lazy

This version of running lazy code uses an AST transformer to restructure the code. This means that there is a far greater runtime overhead to using this method of executing lazy python; however, it allows us to write code outside the body of a function. Just like exec should be avoided when possible, it is prefered that users implement lazy code with lazy_function instead of run_lazy.

thunk(
    operator.add,
    thunk(lambda: 2),
    thunk(
        f,
        thunk(lambda: a),
        thunk(lambda: b),
    ),
)

2 + f(a, b)

undefined

undefined is a value that cannot be strictly evaluated. It is useful as a placeholder for computations.

We can imagine undefined in python as:

@thunk.fromvalue
class undefined(Exception):
    class normalizer(object):
        def __get__(self, instance, owner):
            raise owner
    __strict__ = normalizer()
    del normalizer

This object will raise an instance of itself when it is evaluated. This is presented as an equivalent definition, though it is actually in c to make nicer stack traces.

I opened it up in the repl, everything is strict!

Because the python spec says the __repr__ of an object must return a str, a call to repr must strictly evaluate the contents so that we can see what it is. The repl will implicitly call repr on things to display them. We can see that this is a thunk by doing:

>>> a = thunk(operator.add, 2, 3)
>>> type(a)
lazy.thunk.thunk
>>> a
5

Again, because we need to compute something to represent it, the repl is a bad use case for this, and might make it appear at first like this is always strict.

print didn’t do anything!

Um, what did you think it would do?

If we write:

@lazy_function
def f(a, b):
    print('printing the sum of %s and %s' % (a, b))
    return a + b

Then there is no reason that the print call should be executed. No computation depends on the results, so it is casually skipped.

The solution is to force a dependency:

@lazy_function
def f(a, b):
    strict(print('printing the sum of %s and %s' % (a, b)))
    return a + b

strict is a function that is used to strictly evaluate things. Because the body of the function is interpreted as lazy python, the function call is converted into a thunk, and therefore we can strict it.

This is true for any side-effectful function call.

x is being evaluated strictly when I think it should be lazy

There are some cases where things MUST be strict based on the python language spec. Because this is not really a new language, just an automated way of writing really inefficient python, python’s rules must be followed.

For example, __bool__, __int__, and other converters expect that the return type must be a the proper type. This counts as a place where strictness is needed1.

This might not be the case though, instead, I might have missed something and you are correct, it should be lazy. If you think I missed something, open an issue and I will try to address it as soon as possible.

Some stateful thing is broken

Sorry, you are using unmanaged state and lazy evaluation, you deserve this. thunks cache the normal form so that calling strict the second time will refer to the cached value. If this depended on some stateful function, then it will not work as intended.

I tried to do x with a thunk and it broke!

The library is probably broken. This was written on a whim and I barely thought through the use cases.

Please open an issue and I will try to get back to you as soon as possible.