Skip to main content

Caching results of operations on heavy file trees

Project description

CI Status https://codecov.io/gh/con/fscacher/branch/master/graph/badge.svg https://img.shields.io/pypi/pyversions/fscacher.svg MIT License

GitHub | PyPI | Issues

fscacher provides a cache & decorator for memoizing functions whose outputs depend upon the contents of a file argument.

If you have a function foo() that takes a file path as its first argument, and if the behavior of foo() is pure in the contents of the path and the values of its other arguments, fscacher can help cache that function, like so:

from fscacher import PersistentCache

cache = PersistentCache("insert_name_for_cache_here")

@cache.memoize_path
def foo(path, ...):
    ...

Now the outputs of foo() will be cached for each set of input arguments and for a “fingerprint” (timestamps & size) of each path. If foo() is called twice with the same set of arguments, the result from the first call will be reused for the second, unless the file pointed to by path changes, in which case the function will be run again.

Caches are stored on-disk and thus persist between Python runs. To clear a given PersistentCache and erase its data store, call the clear() method.

If your code runs in an environment where different sets of libraries or the like could be used in different runs, and these make a difference to the output of your function, you can make the caching take them into account by passing a list of library version strings or other identifiers for the current run as the token argument to the PersistentCache constructor.

Finally, PersistentCache’s constructor also optionally takes an envvar argument giving the name of an environment variable. If that environment variable is set to “clear” when the cache is constructed, the cache’s clear() method will be called at the end of initialization. If the environment variable is set to “ignore” instead, then caching will be disabled, and the cache’s memoize_path method will be a no-op. If the given environment variable is not set, or if envvar is not specified, then PersistentCache will query the FSCACHER_CACHE environment variable instead.

Installation

fscacher requires Python 3.6 or higher. Just use pip for Python 3 (You have pip, right?) to install it and its dependencies:

python3 -m pip install fscacher

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

fscacher-0.1.0.tar.gz (30.5 kB view details)

Uploaded Source

Built Distribution

fscacher-0.1.0-py3-none-any.whl (9.6 kB view details)

Uploaded Python 3

File details

Details for the file fscacher-0.1.0.tar.gz.

File metadata

  • Download URL: fscacher-0.1.0.tar.gz
  • Upload date:
  • Size: 30.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.3.0 pkginfo/1.7.0 requests/2.25.1 setuptools/53.0.0 requests-toolbelt/0.9.1 tqdm/4.56.2 CPython/3.9.1

File hashes

Hashes for fscacher-0.1.0.tar.gz
Algorithm Hash digest
SHA256 fe741dbeafb6bd5de0ec1d8d7030e38506303a09a19d26b30e799f574a540f83
MD5 4ee76801e041df7b2c766c3af933293f
BLAKE2b-256 16713b9714f392d61effee8518d3ab6ede409825166a1fbf31e1cf73fe3d0fea

See more details on using hashes here.

Provenance

File details

Details for the file fscacher-0.1.0-py3-none-any.whl.

File metadata

  • Download URL: fscacher-0.1.0-py3-none-any.whl
  • Upload date:
  • Size: 9.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.3.0 pkginfo/1.7.0 requests/2.25.1 setuptools/53.0.0 requests-toolbelt/0.9.1 tqdm/4.56.2 CPython/3.9.1

File hashes

Hashes for fscacher-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 9f1447393e61b2537f023554a3037a2cb396fc4601fcf3b79167a42858ac2532
MD5 dc287c22e77b33dc9f6d9ae82068d263
BLAKE2b-256 7f2f4b1b5d41491a38ef8fd518e94ba599dde3272617b00c2d821838b183307a

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page