Cached response content is inconsistent with original response

[fvolchyok]

The problem

Cached result is inconsistent with what originally was loaded from server.

Expected behavior

Cached response text should exactly match original response.

Steps to reproduce the behavior

1. create a file containing empty json array (echo "[]" > empty_arr.json)
2. start a server pointing to that file (python3 -m http.server 9000)
3. ensure content is loaded properly on the first loading or bypassing cache:

with requests_cache.disabled():
    tmp = requests.get('http://localhost:9000/empty_arr.json')
print("loaded text:", tmp.text)
print("loaded json:", tmp.json())

# loaded text: []
# loaded json: []

4. Trigger another load, ensure it's done from cache:

tmp = requests.get('http://localhost:9000/empty_arr.json')
print("loaded text:", tmp.text)
print("loaded json:", tmp.json())

# loaded text: (empty!!!)
# loaded json: JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Environment

• requests-cache version: 1.2.0
• Python version: 3.12.1
• Platform: MacOS 14.3.1

[JWCook]

I'm not able to reproduce this locally. Previous versions of requests-cache didn't handle an empty JSON array correctly, but I believe that was fixed in 1.1. If you have cached responses older than that, that could cause what you're describing.

What results do you get if you clear your cache or create a new one? If you still get the same results, please post a full example including any options you're passing to install_cache().

Examples

Here's what I see with a new cache created with requests-cache 1.2 and python 3.12:

Example with patching:

import requests
from requests_cache import install_cache

install_cache()
r = requests.get("http://localhost:9000/empty_arr.json")
print(f"Text: {r.text.strip()}")
print(f"JSON: {r.json()}")

Output:

Text: []
JSON: []

Example with CachedSession:

from requests_cache import CachedSession

session = CachedSession()
r = session.get("http://localhost:9000/empty_arr.json")
print(f"Text: {r.text.strip()}")
print(f"JSON: {r.json()}")

Output:

Text: []
JSON: []

[fvolchyok]

Seems like this problem is specific to filesystem backend, there is full setup:

# echo "[]" > empty_arr.json
# python3 -m http.server 9000

import requests
import requests_cache

requests_cache.install_cache(
    "http_cache",
    backend='filesystem',
    serializer='json',
    decode_content=True
)
requests_cache.clear()


# Make initial request, ensure it came directly from server and not cached
r_orig = requests.get('http://localhost:9000/empty_arr.json')
assert r_orig.from_cache == False

original_response = r_orig.text
original_json = r_orig.json()
print("original response:", original_response)
print("original json:", original_json)


# Make the same request again, make sure cached response is returned
r_cached = requests.get('http://localhost:9000/empty_arr.json')
assert r_cached.from_cache == True

cached_response = r_cached.text
print("cached response:", cached_response)
assert original_response == cached_response, "Cached response is not equal to original response"

cached_json = r_cached.json()  # throws error
print("cached json:", cached_json)

[JWCook]

Thanks for the details! I'll get that fixed soon; meanwhile, a workaround you can use is decode_content=False.