hickleable 0.1.0

What is this?

hickleable provides a simple decorator for your classes that will almost always make them serilalize well using the excellent hickle package. By default, custom classes are not supported by hickle – instead, they are written to the HDF5 file as a binary dataset that is serialized using the standard Python pickle. This obviously negates much of the benefit of hickle, for example, the fact that pickle-serialized data is only readable using Python.

hickle provides a way to serialize your custom classes using the HDF5 format, via defining a few hooks for loading/dumping. However, it can be a little tricky to implement these hooks, as they are quite general.

hickleable provides a “default implementation” of these hooks that should satisfy the requirements of most custom classes, and can be applied as a simple decorator. This makes it a one-liner to transform your class into a well-supported data format.

Installation

Simply pip install hickleable. Conda-installable dependencies include h5py.

Usage

Simply:

from hickleable import hickleable

@hickleable()
class MyClass:
   def __init__(self, a=1, b='foo', c={'a': 'dict'}):
       self.a = a
       self.b = b
       self.c = c

Now, MyClass can be hickled without any pickling:

import hickle

my_obj = MyClass()
hickle.dump(my_obj, 'temporary_file.h5')  # Note: no warnings about having to pickle
new_obj = hickle.load('temporary_file.h5')

One super cool thing is that @cached_property attributes are respected, and dataclasses are also supported:

from dataclass import dataclass
from functools import cached_property

@hickleable()
@dataclass
class CachedClass:
    foo: str
    bar: int

    @cached_property
    def foobar(self) -> str:
        print("Obtaining foobar...")
        return foo*bar

c = CachedClass('baz', 50000)

foobar = c.foobar  # prints "Obtaining foobar..."
foobar = c.foobar  # prints nothing, since it's returning cached value.

hickle.dump(c, 'foobar.h5')
d = hickle.load('foobar.h5')

d_foobar = d.foobar  # prints nothing! The value is cached in the hickle file.

One thing to note is that the cached properties are only saved in the hickle file if they have already been evaluated. To force hickle to write out all cached properties, use the evaluate_cached_properties=True parameter in the call to hickleable().

@hickleable()
class Counter:
    def __init__(self, a):
        self.a = a
        self._counts = 0

    def __call__(self, b):
        self._counts += 1
        self.a *= b

def ignore_counts(self, state: dict):
    state['_counts'] = 0
    self.__dict__.update(state)

Counter.__setstate__ = ignore_counts

def remove_counts(self) -> dict:
    return {k: v for k, v in self.__dict__.items() if k != '_counts'}

Counter.__gethstate__ = remove_counts