502 gateway error: upstream sent too big header while reading response header from upstream

[alancwoo]

I am hosting a Laravel app with dokku 0.30.7 and am getting a 502 Gateway Error:

2023-09-25T16:17:10.682711927Z app[web.1]: 2023/09/25 16:17:10 [error] 354#0: *34 upstream sent too big header while reading response header from upstream, client: 172.17.0.1, server: localhost, request: "GET /redacted HTTP/1.1", upstream: "fastcgi://unix:/tmp/heroku.fcgi.5000.sock:", host: "redacted.com"

I've tried updating all the proxy buffer settings from reasonable up to absurd values (32768k) and am still encountering this problem, for example:

dokku nginx:set myapp proxy-buffer-size 1024k
dokku nginx:set myapp proxy-buffers "4 1024k"
dokku nginx:set myapp proxy-busy-buffers-size 1024k

My .buildpacks is:

https://github.com/heroku/heroku-buildpack-php
https://github.com/heroku/heroku-buildpack-nodejs

Things are working in local development, and the page is not extremely large, though there is a list of countries with some attached data around 200kb or so (which when removed does avoid the 502 error).

--

Edit: I've updated the page load to be much smaller, down to about 90kb, but am still encountering this error.

[josegonzalez]

Are you sure you're not sending a large http header somehow? What happens when you hit the app with curl directly instead of through nginx?

[josegonzalez]

You could also turn proxy-buffering to off, which might help.

[alancwoo]

I tried disabling proxy-buffering which had no effect unfortunately.

What is also particularly odd is that the page works when navigated to through the app (laravel/inertia js). Loading the page directly causes the 502.

So actually, it definitely seems related to this issue: inertiajs/inertia-laravel#529 where I need to set:

	fastcgi_buffers 128 4k;
	fastcgi_buffer_size 12k;
	fastcgi_busy_buffers_size 16k;

I've tried adding these to .dokku/nginx.conf.sigil inside the location block, as an extra .conf file in /home/dokku/myapp/nginx.conf.d/fgci.conf, and for the internal app in the root nginx.conf (as per the Procfile web: vendor/bin/heroku-php-nginx -C nginx.conf public/) and just can't seem to get it picked up/reflected anywhere.

[alancwoo]

Ok, actually the combination that worked was:

1. Set the fastcgi_ config in the app's root nginx.conf outside the location block at the top of the file
2. dokku nginx:set appname proxy-... to 4096k etc..., keep proxy buffering on
3. Remove the fcgi.conf from /home/dokku/myapp/nginx.conf.d, it seemed to have no effect
4. Remove the custom nginx.conf.sigil from the app's .dokku/ codebase, it seemed to have no effect

If I understand correctly, the key was for the internal nginx to have a higher fcgi_buffer size, and for dokku's nginx proxy to be able to pass on this corresponding buffer via proxy_buffer

However, you were correct, the server was passing a large header, which was not happening on dev due to how the bundler is handling things I suppose.