Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Seeking a better understanding of the notions of chunk size and resumability #566

Open
cjauvin opened this issue Mar 7, 2023 · 2 comments
Labels

Comments

@cjauvin
Copy link
Contributor

cjauvin commented Mar 7, 2023

I am working with a team on an app where we use the TUS JS client, along with this Java implementation for the server.

We have noticed that when we set the chunk size of the client to a value like, e.g. 50MB, the server never seems to flush anything on the disk before the chunk (and thus the corresponding PATCH request) is fully completed (the corresponding data payload on the server's filesystem stays at zero size). The result is that if the upload is interrupted somwhere before 50MB, the next upload attempt will have to restart from scratch. Given that the default (and recommended) chunk size setting is infinity, this is even more puzzling to us. Is it really the intended behavior, or is there something about the notions of chunk size and resumability that we do not fully understand?

Also, when we consider that a TUS upload should be resumable, it seems that it could mean two possibly distinct things:

(1) An upload is first paused (by calling tus.Upload.abort()) and then resumed (by calling start())
(2) The page is reloaded while an upload is ongoing (thus flushing the JS environment entirely), and it is resumed by having the user re-upload the same file afterward (by selecting it with the file picker, as there seems to be no other, more "automatic" way of doing it, as we first expected)

Are these two scenarios distinct, and are they both supposed to yield the same resumable behavior?

@Acconut
Copy link
Member

Acconut commented Mar 7, 2023

the server never seems to flush anything on the disk before the chunk (and thus the corresponding PATCH request) is fully completed (the corresponding data payload on the server's filesystem stays at zero size)

That is your problem. A tus server should always store data even if a request does not complete successfully. I am not sure if your Java server does not support this or if it is caused by a misconfiguration of your proxy (you must disable request buffering).

The default value for chunkSize assumes that a server is capable of handling failed requests. If not, you must combat this with a low value for chunkSize to get successful requests while sacrificing upload performance.

@tbunique
Copy link

you must disable request buffering

Well, that's the part I was missing. I had forgotten to close Fiddler during my testing, no wonder resume wasn't working!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants