Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for python 3.12 #845

Open
1 of 3 tasks
JWCook opened this issue Jun 30, 2023 · 1 comment
Open
1 of 3 tasks

Add support for python 3.12 #845

JWCook opened this issue Jun 30, 2023 · 1 comment
Labels
bug help-wanted logistics CI builds, project config, refactoring, and other logistical details
Milestone

Comments

@JWCook
Copy link
Member

JWCook commented Jun 30, 2023

It looks like requests-cache is currently compatible with python 3.12, with a few minor exceptions:

  • Multiprocess usage with SQLite
  • In-memory cache reads responses twice on request
  • DeprecationWarning with datetime.utcnow()

Errors

Currently seeing a few test errors under python 3.12, including sqlite3.InterfaceError: bad parameter or other API misuse.

At first glance this looks like it might be a breaking change in the current dev build of 3.12 in the sqlite3 module, specifically with multithreaded usage.

Example logs:

=================================== FAILURES ===================================
_______________ TestMemoryCache.test_response_no_duplicate_read ________________
[gw0] linux -- Python 3.12.0 /home/runner/work/requests-cache/requests-cache/.venv/bin/python

self = <tests.integration.test_memory.TestMemoryCache object at 0x7f5f585830b0>

    def test_response_no_duplicate_read(self):
        """Ensure that response data is read only once per request, whether it's cached or not"""
        session = self.init_session()
        storage_class = type(session.cache.responses)
    
        # Patch storage class to track number of times getitem is called, without changing behavior
        with patch.object(
            storage_class, '__getitem__', side_effect=lambda k: CachedResponse()
        ) as getitem:
            session.get(httpbin('get'))
>           assert getitem.call_count == 1
E           AssertionError: assert 2 == 1
E            +  where 2 = <MagicMock name='__getitem__' id='140047397921200'>.call_count

tests/integration/base_cache_test.py:109: AssertionError
----------------------------- Captured stdout call -----------------------------
                 INFO     Clearing all items from the cache          base.py:100
------------------------------ Captured log call -------------------------------
INFO     requests_cache.backends.base:base.py:100 Clearing all items from the cache
____________ TestSQLiteCache.test_concurrency[2-ThreadPoolExecutor] ____________
[gw0] linux -- Python 3.12.0 /home/runner/work/requests-cache/requests-cache/.venv/bin/python

self = <tests.integration.test_sqlite.TestSQLiteCache object at 0x7f5f587b51c0>
iteration = 2
executor_class = <class 'concurrent.futures.thread.ThreadPoolExecutor'>

    @pytest.mark.parametrize('executor_class', [ThreadPoolExecutor, ProcessPoolExecutor])
    @pytest.mark.parametrize('iteration', range(N_ITERATIONS))
    def test_concurrency(self, iteration, executor_class):
        """Run multithreaded and multiprocess stress tests for each backend.
        The number of workers (thread/processes), iterations, and requests per iteration can be
        increased via the `STRESS_TEST_MULTIPLIER` environment variable.
        """
        start = time()
        url = httpbin('anything')
    
        # For multithreading, we can share a session object, but we can't for multiprocessing
        session = self.init_session(clear=True, expire_after=1)
        if executor_class is ProcessPoolExecutor:
            session = None
        session_factory = partial(self.init_session, clear=False, expire_after=1)
    
        request_func = partial(_send_request, session, session_factory, url)
        with executor_class(max_workers=N_WORKERS) as executor:
>           _ = list(executor.map(request_func, range(N_REQUESTS_PER_ITERATION)))

tests/integration/base_cache_test.py:372: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/opt/hostedtoolcache/Python/3.12.0-beta.3/x64/lib/python3.12/concurrent/futures/_base.py:619: in result_iterator
    yield _result_or_cancel(fs.pop())
/opt/hostedtoolcache/Python/3.12.0-beta.3/x64/lib/python3.12/concurrent/futures/_base.py:317: in _result_or_cancel
    return fut.result(timeout)
/opt/hostedtoolcache/Python/3.12.0-beta.3/x64/lib/python3.12/concurrent/futures/_base.py:449: in result
    return self.__get_result()
/opt/hostedtoolcache/Python/3.12.0-beta.3/x64/lib/python3.12/concurrent/futures/_base.py:401: in __get_result
    raise self._exception
/opt/hostedtoolcache/Python/3.12.0-beta.3/x64/lib/python3.12/concurrent/futures/thread.py:58: in run
    result = self.fn(*self.args, **self.kwargs)
tests/integration/base_cache_test.py:400: in _send_request
    return session.get(url, params={f'key_{i}': f'value_{i}'})
requests_cache/session.py:103: in get
    return self.request('GET', url, params=params, **kwargs)
requests_cache/session.py:159: in request
    return super().request(method, url, *args, headers=headers, **kwargs)  # type: ignore
.venv/lib/python3.12/site-packages/requests/sessions.py:589: in request
    resp = self.send(prep, **send_kwargs)
requests_cache/session.py:194: in send
    cached_response = self.cache.get_response(actions.cache_key)
requests_cache/backends/base.py:68: in get_response
    response = self.responses.get(key)
<frozen _collections_abc>:807: in get
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <requests_cache.backends.sqlite.SQLiteDict object at 0x7f5f52247a40>
key = 'a8d791895d0b043c'

    def __getitem__(self, key):
        with self.connection() as con:
>           cur = con.execute(f'SELECT value FROM {self.table_name} WHERE key=?', (key,))
E           sqlite3.InterfaceError: bad parameter or other API misuse

requests_cache/backends/sqlite.py:290: InterfaceError

Warnings

DeprecationWarning: datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.now(datetime.UTC).
@JWCook JWCook added this to the v1.2 milestone Jun 30, 2023
@JWCook JWCook added logistics CI builds, project config, refactoring, and other logistical details and removed enhancement labels Sep 5, 2023
@severinsimmler
Copy link

This might be related: python/cpython#118172

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug help-wanted logistics CI builds, project config, refactoring, and other logistical details
Projects
None yet
Development

No branches or pull requests

2 participants