Expiration#
By default, cached responses will be stored indefinitely. There are a number of options for specifying how long to store responses, either with a single expiration value, glob patterns, or cache headers.
The simplest option is to initialize the cache with an expire_after
value, which will apply to all
reponses:
>>> # Set expiration for the session using a value in seconds
>>> session = CachedSession(expire_after=360)
Expiration Precedence#
Expiration can be set on a per-session, per-URL, or per-request basis, in addition to cache headers (see sections below for usage details). When there are multiple values provided for a given request, the following order of precedence is used:
Cache-Control response headers (if enabled)
Cache-Control request headers
Per-request expiration (
expire_after
argument forCachedSession.request()
)Per-URL expiration (
urls_expire_after
argument forCachedSession
)Per-session expiration (
expire_after
argument forCacheBackend
)
Expiration Values#
expire_after
can be any of the following:
-1
(to never expire)0
(to “expire immediately,” e.g. bypass the cache)A positive number (in seconds)
A
datetime
Examples:
>>> # To specify a unit of time other than seconds, use a timedelta
>>> from datetime import timedelta
>>> session = CachedSession(expire_after=timedelta(days=30))
>>> # Update an existing session to disable expiration (i.e., store indefinitely)
>>> session.expire_after = -1
>>> # Disable caching by default, unless enabled by other settings
>>> session = CachedSession(expire_after=0)
Expiration With URL Patterns#
You can use urls_expire_after
to set different expiration values based on URL glob patterns.
This allows you to customize caching based on what you know about the resources you’re requesting
or how you intend to use them. For example, you might request one resource that gets updated
frequently, another that changes infrequently, and another that never changes. Example:
>>> urls_expire_after = {
... '*.site_1.com': 30,
... 'site_2.com/resource_1': 60 * 2,
... 'site_2.com/resource_2': 60 * 60 * 24,
... 'site_2.com/static': -1,
... }
>>> session = CachedSession(urls_expire_after=urls_expire_after)
Notes:
urls_expire_after
should be a dict in the format{'pattern': expire_after}
expire_after
accepts the same types asCachedSession.expire_after
Patterns will match request base URLs without the protocol, so the pattern
site.com/resource/
is equivalent tohttp*://site.com/resource/**
If there is more than one match, the first match will be used in the order they are defined
If no patterns match a request,
CachedSession.expire_after
will be used as a default
Expiration and Error Handling#
In some cases, you might cache a response, have it expire, but then encounter an error when
retrieving a new response. If you would like to use expired response data in these cases, use the
stale_if_error
option.
For example:
>>> # Cache a test response and wait until it's expired
>>> session = CachedSession(stale_if_error=True)
>>> session.get('https://httpbin.org/get', expire_after=1)
>>> time.sleep(1)
Afterward, let’s say the page has moved and you get a 404, or the site is experiencing downtime and you get a 500. You will then get the expired cache data instead:
>>> response = session.get('https://httpbin.org/get')
>>> print(response.from_cache, response.is_expired)
True, True
In addition to HTTP error codes, stale_if_error
also applies to python exceptions (typically a
RequestException
). See requests
documentation on
Errors and Exceptions
for more details on request errors in general.
Removing Expired Responses#
For better read performance, expired responses won’t be removed immediately, but will be removed (or replaced) the next time they are requested.
Tip
Implementing one or more cache eviction algorithms is being considered. If this is something you are interested in, please provide feedback via issues!
To manually clear all expired responses, use
CachedSession.remove_expired_responses()
:
>>> session.remove_expired_responses()
Or, when using patching:
>>> requests_cache.remove_expired_responses()
You can also apply a different expire_after
to previously cached responses, which will
revalidate the cache with the new expiration time:
>>> session.remove_expired_responses(expire_after=timedelta(days=30))