upload large WHL files

[Jimmyshi7]

When I upload a large WHL file the following error occurs, how to solve it. The size of the file is about 1.8G.

[fschulze]

Do you have a web server like nginx as a proxy in front of devpi-server? If so, you have to allow bigger uploads there. For nginx the setting is client_max_body_size. You should also check the error logs on the server side to see why the connection was reset.

[Jimmyshi7]

@fschulze
Thanks for your reply.
I have checked. I do not have any web server in front of devpi-server. It seems that the error is occur in devpi client. Beacuse it didn't print any error message in devpi-server when I execute upload.

Do you have a web server like nginx as a proxy in front of devpi-server? If so, you have to allow bigger uploads there. For nginx the setting is client_max_body_size. You should also check the error logs on the server side to see why the connection was reset.

[fschulze]

Is this a one time error, or does it always happen when you try to upload? Is devpi-server on the same machine, or remote? The error is a connection related error, so it might have been a network glitch.

[Jimmyshi7]

Always when upload large WHL files. If the size of WHL is not so large, it can upload successfully. Devpi-server is on the same machine.

[fschulze]

Try setting a bigger limit on devpi-server with --max-request-body-size 2147483648 (the default is 1073741824), that will pass the max_request_body_size setting to waitress. Please let me know if that fixes your issue, but leave this ticket open, so I can add the information to the documentation, where it is currently missing.

[Jimmyshi7]

--max-request-body-size 2147483648

It worked! The large WHL file has been uploaded successfully. Thank you very much!

[fschulze]

There also seems to be a bug in waitress. It should return 413 Content Too Large instead of aborting/resetting the connection.

[SyedSameer95]

Hello. I am having a similar problem while uploading files bigger than 1 GB to devpi using postgres. I have tried using the --max-request-body-size, and increased rhe max_wal_size in postgres, but keep getting the following error:

pg8000.exceptions.DatabaseError: {'S': 'ERROR', 'V': 'ERROR', 'C': '54000', 'M': 'out of memory', 'D': 'Cannot enlarge string buffer containing 0 bytes by 1391196735 more bytes.', 'W': 'COPY files, line 1, column data', 'F': 'stringinfo.c', 'L': '303', 'R': 'enlargeStringInfo'}

Any idea how I could solve this issue ? Thanks

[fschulze]

@SyedSameer95 this seems to be caused by the way devpi-postgresql stores binaries, see the just created #1029