Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Loading cache settings from a config file #770

Open
JWCook opened this issue Jan 20, 2023 · 0 comments
Open

Loading cache settings from a config file #770

JWCook opened this issue Jan 20, 2023 · 0 comments

Comments

@JWCook
Copy link
Member

JWCook commented Jan 20, 2023

I'd like to make it easy to load cache settings from a config file, without being tied to a specific format. CacheSettings is already an attrs class, and can be easily loaded from a dict, so that gets us most of the way there. Other validators or converters could be added to that if needed. There is some basic example code and config in the docs for using this with yaml.

Current limitations

The main limitation is that you can't put python expressions in a json/toml/yaml file. Right now (as of 1.0) there are 3 main things you can pass to CacheSettings that don't translate to config file primitive types:

  • Custom callback functions: Trying to fit that into a config file sounds like a terrible idea, so I'm not going to go there!
  • Expiration timedelta objects: This could be handled in your own library by parsing time expression strings (like @glensc mentioned here). The pytimeparse library is a good way to handle that.
  • Expiration constants:
    • EXPIRE_IMMEDIATELY / 0: Same meaning as Cache-Control: max-age=0. It's probably fine to just put 0 in config.
    • NEVER_EXPIRE / -1: No expiration. It's also probably fine to put -1 in config, but it's an arbitrary value, and it isn't immediately obvious what it means. I don't like the idea of using None, since that's ambiguous (could mean either "no expiration" or "no value; use default").
    • DO_NOT_CACHE: Another non-standard, arbitrary value to skip both cache read and write, with no exceptions (unlike max-age=0, no-cache, etc.). This could potentially be written in config as no-store, since that's the closest standard Cache-Control value.

Ideally, a config file should make sense to someone reading it on its own without needing to know all the details of requests-cache's settings. The constants NEVER_EXPIRE and DO_NOT_CACHE are the main things that currently don't translate well to a config file.

Feedback needed

Does anyone else have thoughts on this? For example, for a python snippet like this:

from requests_cache import NEVER_EXPIRE, DO_NOT_CACHE

urls_expire_after = {
    '*.site_1.com': NEVER_EXPIRE,
    'site_2.com/prefix/*': DO_NOT_CACHE,
    '*': 360
}

What should that look like in yaml format?

urls_expire_after:
  '*.site_1.com': ??
  'site_2.com/prefix/*': ??
  '*': 360
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant