Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upload is slower than bandwidth available #162

Closed
killforby opened this issue May 6, 2024 · 26 comments
Closed

Upload is slower than bandwidth available #162

killforby opened this issue May 6, 2024 · 26 comments
Labels
bug Something isn't working

Comments

@killforby
Copy link

Hello, having installed gokapi I find that when I upload the speed is not great between 10Mb and 12MB/s. Having done speed tests on my vps the speed is around 1175MB/s or 10 times more than what I can send with gokapi. The configuration is as follows: gokapi is installed on a vps and I access via a reverse proxy (NPM). Do you have any advice to recommend to me?

@Forceu
Copy link
Owner

Forceu commented May 23, 2024

Sorry for the late reply, I somehow did not receive a notification for this issue.
Are you using encryption?

@killforby
Copy link
Author

Hello, no we do not use encryption because I had read that it would slow down the speed. Would you have any advice to recommend to me?

@Forceu
Copy link
Owner

Forceu commented May 29, 2024

According to the speedtest, your connection would be a 10GBit connection, both at the receiving end, as well as at the sending end. Is that correct? Most providers only offer 100MBit or 1GBit for VPS instances. 100MBit would equal to 12MB/s, please check if that might be the case. Otherwise please upload a file with a different method (e.g. SSH) and compare the upload speed. Also please do a benchmark on how fast your VPS does SHA1 hashes. You can do this with the command openssl speed sha1, the last line will be your speed in kb/s.

I just tested it on my laptop and I got an upload speed of 320MB/s for an 8GB file.

@Forceu
Copy link
Owner

Forceu commented Jun 3, 2024

I will close this issue for now, as it seems to be a problem with your hardware instead of Gokapi. If you still experience the problem and it can be certain that it is Gokapi, please comment here so I can investigate further.

@Forceu Forceu closed this as completed Jun 3, 2024
@Forceu Forceu added the invalid This doesn't seem right label Jun 3, 2024
@killforby
Copy link
Author

image
Good evening, I did the sha1 command as you asked me.

@killforby
Copy link
Author

I performed a speedtest in cli
image

@Forceu
Copy link
Owner

Forceu commented Jun 4, 2024

Hashing and bandwidth looks good. How long does it take to upload a file through SSH or FTP to the server?

@killforby
Copy link
Author

hi with command dd =/if
image

With a 2g file on gokapi ( docker ) 6 minutes and 22sec

image

@Forceu
Copy link
Owner

Forceu commented Jun 4, 2024

You are running all the tests on the server. Can you confirm that you are actually uploading the file to Gokapi from the same machine that runs Gokapi without any other device?

@killforby
Copy link
Author

Yes all the tests were done on the vps where there is gokapi. The upload to gokapi is from my pc to the vps (gokapi)

@killforby
Copy link
Author

with fresh install
image

@Forceu
Copy link
Owner

Forceu commented Jun 4, 2024

Yes, however you have to test the upload connection from your computer to the server. Try uploading a file to it from your computer to the server by other means, that will give you the speed that is possible.
Alternatively you can also run a speed test on your computer, that should give you an upload rate

@killforby
Copy link
Author

image
this is the download speed of my computer

@Forceu
Copy link
Owner

Forceu commented Jun 4, 2024

Hm so at least 70MB/s should be possible. Have you tried uploading files via FTP or other services? If yes, what speed are you getting there?

@killforby
Copy link
Author

killforby commented Jun 4, 2024

On Filezilla
image

@Forceu
Copy link
Owner

Forceu commented Jun 4, 2024

Okay so if I read that right, the transfer speed from your computer to the server is 28MB/s?

@killforby
Copy link
Author

Yes that's right, but for gokapi I don't exceed 12mb

@Forceu
Copy link
Owner

Forceu commented Jun 5, 2024

Does your connection have high latency (e.g. satellite connection)? In that case it might help if you increase the chunksize...

In the main menu please open your browser console (Chrome: Shift + Ctrl + J) and enter the following:

Dropzone.options.uploaddropzone["chunkSize"] = 50000000;

Then - without reloading the page - try to upload something. Let me know if that increases the speed

@killforby
Copy link
Author

Hello, no, I haven't had any noticeable changes. Wouldn't there be a variable to change in the .en for example?

@Forceu Forceu reopened this Jun 5, 2024
@Forceu Forceu added bug Something isn't working and removed invalid This doesn't seem right labels Jun 5, 2024
@Forceu Forceu changed the title Information for configuration of a better upload Upload is too slow due to chunking Jun 5, 2024
@Forceu Forceu changed the title Upload is too slow due to chunking Upload is slower than bandwidth available Jun 5, 2024
@Forceu
Copy link
Owner

Forceu commented Jun 5, 2024

Okay, so I tested it with a 100MBit uplink to two different servers and could reproduce the problem. It might be related to chunking, which is done by Javascript. Unfortunately I am not sure yet what exactly is the cause and how to fix it.

Does your speed increase if you enter these two commands in the console and then upload a file without refreshing the window?

dropzoneObject.options.chunkSize = 50000000;
dropzoneObject.options.parallelChunkUploads = true;

@killforby
Copy link
Author

Do I enter the commands on Chrome or do I write them in the .env?

@killforby
Copy link
Author

With the two commands entered on chrome I had a peak at 20mb/s

@Forceu
Copy link
Owner

Forceu commented Jun 6, 2024

Okay, so that's a good start. Do you get the same speeds if you only enter dropzoneObject.options.chunkSize = 50000000;?

@killforby
Copy link
Author

When I run the command you gave me, I get 20mb/s but after 10 seconds the speed drops to 12mb/s

Forceu added a commit that referenced this issue Jun 6, 2024
… chunk uploads, set chunksize to 39MB, fixes slow upload #162
@killforby
Copy link
Author

Thank you for being responsive and taking the time to address my concerns. There will be a new update

@Forceu Forceu closed this as completed in 3b7cdae Jun 7, 2024
@Forceu
Copy link
Owner

Forceu commented Jun 7, 2024

Fixed in 3b7cdae. With that version it is not possible to set the chunksize and parallel upload limit in the settings. Default ist now 45MB chunk size and 4 parallel connections instead of 10MB serial.

@killforby Thank you very much for your feedback!

@Forceu Forceu pinned this issue Jun 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants