Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Usage of streams causing file descriptor leak #12173

Open
Hirevo opened this issue Apr 29, 2024 · 1 comment
Open

Usage of streams causing file descriptor leak #12173

Hirevo opened this issue Apr 29, 2024 · 1 comment

Comments

@Hirevo
Copy link

Hirevo commented Apr 29, 2024

Describe the bug

I am attempting to make a SvelteKit API endpoint that makes a request to another backend API and streams the response body of its own response body, using fetch and response.body, like so:

// `filterHeaders` is simply a function to filter only some desired headers.

/** @type {import('./$types').RequestHandler} */
export async function GET(event) {
    const requestHeaders = filterHeaders(event.request.headers, ['range']);
    const response = await fetch('http://example.com/some/example/endpoint', {
        signal: event.request.signal,
        headers: requestHeaders,
    });
    const responseHeaders = filterHeaders(
        response.headers,
        ['content-length', 'content-type', 'content-range', 'accept-ranges'],
    );
    return new Response(response.body, {
        status: response.status,
        statusText: response.statusText,
        headers: responseHeaders,
    });
}

While this code works, it seems to leak a file descriptor each time it is requested, especially in the context of HTTP range requests while playing video content.

I have tried many variations of this code, like using pipeThrough or pipeTo, using the pipeline function from the node:stream/promises module, or using axios to make the request and converting the body back to a ReadableStream, but all of them demonstrated the issue, which prompts to me that maybe its the @sveltejs/adapter-node that doesn't close the response stream properly.

After some time, this issue leads to my app being unable to fulfil new requests, and the node process must be restarted.

Reproduction

I have made a minimal demo app to demonstrate the issue here:

https://github.com/Hirevo/sveltekit-stream-issue

The index page has steps of how to observe the issue, using lsof.

Logs

No response

System Info

System:
  OS: macOS 14.4.1
  CPU: (12) x64 Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
  Memory: 411.48 MB / 16.00 GB
  Shell: 0.1.0 - /Users/nicolaspolomack/.cargo/bin/psh
Binaries:
  Node: 22.0.0 - ~/.local/state/fnm_multishells/18305_1714256923846/bin/node
  Yarn: 1.22.22 - /usr/local/bin/yarn
  npm: 10.5.1 - ~/.local/state/fnm_multishells/18305_1714256923846/bin/npm
Browsers:
  Chrome: 124.0.6367.91
  Safari: 17.4.1
  Safari Technology Preview: 17.4
npmPackages:
  @sveltejs/adapter-node: ^5.0.1 => 5.0.1
  @sveltejs/kit: ^2.0.0 => 2.5.7
  @sveltejs/vite-plugin-svelte: ^3.0.0 => 3.1.0
  svelte: ^4.2.7 => 4.2.15
  vite: ^5.0.3 => 5.2.10

Severity

annoyance

Additional Information

From some of my research on this, I am aware that undici, which implements fetch in Node, does not release connection resources upon garbage collection.

I made multiple attempts to close it myself, but I was unsuccessful.

@hanszoons
Copy link
Contributor

Perhaps related to #11617?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants