aiohttp-client-cache

aiohttp-client-cache is an async persistent cache for aiohttp client requests.

See full documentation at https://aiohttp-client-cache.readthedocs.io

Features

• Ease of use: Use as a drop-in replacement for aiohttp.ClientSession
• Customization: Works out of the box with little to no config, but with plenty of options available for customizing cache expiration and other behavior
• Persistence: Includes several storage backends: SQLite, DynamoDB, MongoDB, and Redis.

Development Status

This library is a work in progress!

Breaking changes should be expected until a 1.0 release, so version pinning is recommended.

I am developing this while also maintaining requests-cache, and my goal is to eventually have a similar (but not identical) feature set between the two libraries. If there is a feature you want, or if you've discovered a bug, of it you have other general feedback, please create an issue for it!

Quickstart

First, install with pip (python 3.7+ required):

pip install aiohttp-client-cache

Basic Usage

Next, use aiohttp_client_cache.CachedSession in place of aiohttp.ClientSession. To briefly demonstrate how to use it:

Replace this:

from aiohttp import ClientSession

async with ClientSession() as session:
    await session.get('http://httpbin.org/delay/1')                                                          

With this:

from aiohttp_client_cache import CachedSession, SQLiteBackend

async with CachedSession(cache=SQLiteBackend('demo_cache')) as session:
    await session.get('http://httpbin.org/delay/1')                                                          

The URL in this example adds a delay of 1 second, simulating a slow or rate-limited website. With caching, the response will be fetched once, saved to demo_cache.sqlite, and subsequent requests will return the cached response near-instantly.

Configuration

Several options are available to customize caching behavior. This example demonstrates a few of them:

from aiohttp_client_cache import SQLiteBackend

cache = SQLiteBackend(
    cache_name='~/.cache/aiohttp-requests.db',  # For SQLite, this will be used as the filename
    expire_after=60*60,                         # By default, cached responses expire in an hour
    urls_expire_after={'*.fillmurray.com': -1}, # Requests for any subdomain on this site will never expire
    allowed_codes=(200, 418),                   # Cache responses with these status codes
    allowed_methods=('GET', 'POST'),            # Cache requests with these HTTP methods
    include_headers=True,                       # Cache requests with different headers separately
    ignored_params=['auth_token'],              # Keep using the cached response even if this param changes
    timeout=2.5,                                # Connection timeout for SQLite backend 
)

More Info

To learn more, see:

• User Guide
• Cache Backends
• API Reference
• Examples

History

0.3.0 (2021-04-TBD)

See all issues & PRs for v0.3

• Add async implementation of DynamoDb backend
• Add support for expiration for individual requests
• Add support for expiration based on URL patterns
• Add support for serializing/deserializing ClientSession.links
• Add case-insensitive response headers for compatibility with aiohttp.ClientResponse.headers
• Add optional integration with itsdangerous for safer serialization
• Add CacheBackend.get_urls() to get all urls currently in the cache
• Add some default attributes (from_cache, is_expired, etc.) to returned ClientResponse objects
• Allow passing all backend-specific connection kwargs via CacheBackend
• Add support for json request body
• Convert all keys() and values() methods into async generators
• Fix serialization of Content-Disposition
• Fix filtering ignored parameters for request body (data and json)
• Add user guide, more examples, and other project docs

0.2.0 (2021-02-28)

See all issues & PRs for v0.2

• Refactor SQLite backend to use aiosqlite for async cache operations
• Refactor MongoDB backend to use motor for async cache operations
• Refactor Redis backend to use aiosqlite for async cache operations
• Add integration tests and docker-compose for local test servers

0.1.0 (2020-11-14)

• Initial fork from requests-cache
• First pass at a general refactor and conversion from requests to aiohttp
• Basic features are functional, but some backends do not actually operate asynchronously