Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Timeout with API Client for retrieving sketch information after upgrading both Timesketch server and API Client #3018

Open
clowe-r7 opened this issue Jan 17, 2024 · 1 comment
Labels

Comments

@clowe-r7
Copy link
Contributor

clowe-r7 commented Jan 17, 2024

Describe the bug
After upgrading to the latest version of Timesketch and the Timesketch API Client, a request fails when attempting to get information about a particular Sketch. The request ends up failing here with a timeout and a 308 status code using the resource uri defined at init.

To Reproduce
Steps to reproduce the behavior:

  1. Spin up a Timesketch server using the latest version (I upgraded a currently running Timesketch instance. The request worked on a local development Timesketch server using the docker compose file in the docker/dev directory)
  2. Run the following script as described in the Timesketch API Client documentation
print(f"Listing Sketches")
sketches = client.list_sketches()
for sketch in sketches:
    print('[{0:d}] {1:s} <{2:s}>'.format(sketch.id, sketch.name, sketch.description))

Expected behavior
A list of all sketches with the id, name and description printed out.

Desktop (please complete the following information):

  • OS: Mac
  • Browser: Chrome
  • Version [e.g. 22]

Additional context
I was able to fix this in my local virtual environment by appending a forward slash to the sketch resource uri at init of the class here so instead of

super().__init__(api=api, resource_uri=f"sketches/{self.id}")

it became

super().__init__(api=api, resource_uri=f"sketches/{self.id}/")

and the request succeeded.

Full stacktrace

Traceback (most recent call last):
  File "/venv/lib/python3.9/site-packages/urllib3/connection.py", line 169, in _new_conn
    conn = connection.create_connection(
  File "/venv/lib/python3.9/site-packages/urllib3/util/connection.py", line 96, in create_connection
    raise err
  File "/venv/lib/python3.9/site-packages/urllib3/util/connection.py", line 86, in create_connection
    sock.connect(sa)
TimeoutError: [Errno 60] Operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 394, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/venv/lib/python3.9/site-packages/urllib3/connection.py", line 234, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/usr/local/Cellar/[email protected]/3.9.18_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1285, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/Cellar/[email protected]/3.9.18_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1331, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/[email protected]/3.9.18_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1280, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/[email protected]/3.9.18_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1040, in _send_output
    self.send(msg)
  File "/usr/local/Cellar/[email protected]/3.9.18_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 980, in send
    self.connect()
  File "/venv/lib/python3.9/site-packages/urllib3/connection.py", line 200, in connect
    conn = self._new_conn()
  File "/venv/lib/python3.9/site-packages/urllib3/connection.py", line 181, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x11c4e8100>: Failed to establish a new connection: [Errno 60] Operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/venv/lib/python3.9/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/venv/lib/python3.9/site-packages/urllib3/util/retry.py", line 574, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='https://timesketch.myowndomain.com', port=80): Max retries exceeded with url: /api/v1/sketches/1/ (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x11c4e8100>: Failed to establish a new connection: [Errno 60] Operation timed out'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/timesketch.py", line 21, in <module>
    print('[{0:d}] {1:s} <{2:s}>'.format(sketch.id, sketch.name, sketch.description))
  File "/venv/lib/python3.9/site-packages/timesketch_api_client/sketch.py", line 112, in description
    sketch = self.lazyload_data()
  File "/venv/lib/python3.9/site-packages/timesketch_api_client/resource.py", line 44, in lazyload_data
    self.resource_data = self.api.fetch_resource_data(self.resource_uri)
  File "/venv/lib/python3.9/site-packages/timesketch_api_client/client.py", line 376, in fetch_resource_data
    response = self.session.get(resource_url, params=params)
  File "/venv/lib/python3.9/site-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/venv/lib/python3.9/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/venv/lib/python3.9/site-packages/requests/sessions.py", line 677, in send
    history = [resp for resp in gen]
  File "/venv/lib/python3.9/site-packages/requests/sessions.py", line 677, in <listcomp>
    history = [resp for resp in gen]
  File "/venv/lib/python3.9/site-packages/requests/sessions.py", line 237, in resolve_redirects
    resp = self.send(
  File "/venv/lib/python3.9/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/venv/lib/python3.9/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='https://timesketch.myowndomain.com', port=80): Max retries exceeded with url: /api/v1/sketches/1/ (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x11c4e8100>: Failed to establish a new connection: [Errno 60] Operation timed out'))

@clowe-r7 clowe-r7 added the Bug label Jan 17, 2024
@jkppr jkppr self-assigned this Jan 26, 2024
@clowe-r7
Copy link
Contributor Author

Turns out this is an issue with the interaction between an AWS ALB, the Flask app running behind it and I think the werkzeug upgrade. Essentially the ALB is using HTTPs and the Flask app responded with a redirect but for HTTP. I was able to follow the fix on this reddit post https://www.reddit.com/r/flask/comments/11luchr/redirect_is_using_http_instead_of_https/ to get it working again without modifying the API Client. I added app.wsgi_app = ProxyFix(app.wsgi_app, x_for=1, x_proto=1, x_host=1, x_port=1) here and it's working again as expected.

@jkppr jkppr removed their assignment May 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants