Use formidable upload files, always get "Error Request Aborted"

[EricCat]

• I use Express(3.6.0) + formidable(latest) for upload files to S3.
• When I submit post , then I always got Error Request Aborted.
• incoming_form.js:

IncomingForm.prototype.parse = function(req, cb) {
.......
.......
.......
req
    .on('aborted', function() {
      self.emit('aborted');
      self._error(new Error('Request aborted'));
    })
.......
.......
.......
}

[Error: Request aborted]
child process: Request aborted
Error: Request aborted
at IncomingMessage.<anonymous>      
(/node_modules/formidable/lib/incoming_form.js:107:19)
at IncomingMessage.EventEmitter.emit (events.js:117:20)
at abortIncoming (http.js:1911:11)
at Socket.serverSocketCloseListener (http.js:1923:5)
at Socket.EventEmitter.emit (events.js:117:20)
at TCP.close (net.js:465:12)

I want to know why I got aborted and how to debug it.

[anorsich]

same here for big file uploads (1mb - 15mb)

[piotrkowalczuk]

+1

[floticerus]

yep this is crashing my site (image uploads) every once in a while... not good.

[nehautkur]

Has anyone found a solution?
I am having same issue. Other uploads are working fine, but image upload is failing. This started after I upgraded node to the latest 0.10.29

[OrangeDog]

It is the client that is aborting, not the server. It's an error you can always get and which you must handle. It's not a bug in formidable.

[yuchienho]

+1

[vivasaayi]

Same here.. happening with files lesser than 100 kb.. Anyone handled this case?

[Midnight-Coder]

I see this for files as small as 10kb. My client aborts the request after 2 mins latency. So there is something here at play. Removing the express bodyparser didn't helped either

[yiwei1223]

Same here....but I find the reason is that I use ths multer module for parsing multipart/form-data. So that is resulting in the formidable can't parse correctly.

[bugall]

to disable body-parser for multipart/form-data in your router

exmple solution of nodejs

app.use(express.json()).use(express.urlencoded())

[seanfly]

+1 Removing the express bodyparser didn't helped either

[lagden]

Very weird this issue!!

I believe it is Express or browser problem, because I did many test with Firefox, Chrome and Chrome Canary.
Only the Chrome Canary stopped!
By the way... I used many ways to do upload, including my own upload lib... always stop!!

[teng-boy]

+1 how to solve it

[baihanbing]

how to sovle it.

[baihanbing]

try to delete this in your app.js.
app.use(express.bodyParser());
I have sovled this proplem.

[ksawaneh]

Getting rid of the app.use(bodyParser.json()); helped.

[nicholasstephan]

I'm trying to use formidable in a firebase function, and am getting this error for every request.

[nisan92]

Same problem here, but it is very strange that my app is running perfectly on local machine, but when i upload that on c9 and my digital ocean droplet it started to giving this error. i remove body parser as well and try this from firefox also, but no luck.
Have you guys got any other solution ?

[VladGavriuk]

What if I need bodyparser and formidable and i can't remove bodyparser from my app? What I should do?

[kanadChourasia]

I have many more post request routes for which I am using body parser and in few of them I need formidable. What should I do?

[lborgman]

Has anyone found a solution with Formidable and Firebase? For me it works locally, but not in the deployed instance. (Using FormData and html5 fetch.)

I wonder if that depends on where I store the uploaded files. I am currently using "./../mytempdir". Is that a place I can use? (I first tried "./mytempdir", but that dispatches a signal to Node.js that some files has changed.)

[tunnckoCore]

Hm. Pretty weird, really. But most of the comments are because using the Express v3.
Can you try v4 and latest formidable? Is it still a problem? Can you try Koa? And probably with koa-better-body (which uses Formidable for years).

Also, is the directories writable? Those remote servers are paranoic about security and who knows what they do to limit permissions. I'm mostly sure that that's not the problem, but make sure to check or make programatically directories to have read/write permissions.

[codeinearts]

I've seen this problem when using some php server along with the express stuff (nginx with fpm module as reverse proxy, for say).
In that case I got rid of the php-fpm module and got lucky: my files started to upload.
Looking at the nginx logs I discovered that the connection was being reset for whatever reason I don't know.
Now I'm getting the same issue on another server where some guy installed apache and vstfptd. Before that, everything was working like a charm.

I would try to narrow the problem down (also talking to myself on this point) to see what is exactly the problem here.

Any ideas on how to trace the problem in an e2e fashion?

edit:
using express 4.16.2 and express-formidable 1.0.0

[herablog]

The following answer seems to be helpful for developers who want to receive form data on Firebase Cloud Functions.

https://stackoverflow.com/questions/47242340/how-to-perform-an-http-file-upload-using-express-on-cloud-functions-for-firebase/47319614#47319614

[itnamerica]

One way to solve it is to using a GET request instead and disguise it as a POST or PUT by sending params.
For example, on the front end side, via angular it would be:
$http.get('/fetchComments', {params: payLoadObj}).then......

And to retrieve it from your server file:
req.query

[brodwen83]

Has anyone found a solution with Formidable and Firebase?

[mamteshku]

you have to use form-data instead of raw while passing the parameter in body...

[BrodaNoel]

Same issue here, using Formidable and Firebase. In local environment (and it's working on production).

It maaay be a permission problem. Not sure.

[Amit-A]

I just published formidable-serverless - it imports formidable as a dependency and modifies the handlers to support preprocessed request bodies (built-in to many serverless environments).

I'm using it in production with Firebase @BrodaNoel @brodwen83 @lborgman @nicholasstephan

The usage/API is exactly the same. Hope that helps.

[tunnckoCore]

@Amit-A great, but looking on it.. it isn't the best API decision idea. Because it's patching only the .parse method, you can export a single function that accepts a formidable instance and returns a parse function. But that's me.

[Amit-A]

@tunnckoCore that could be a nice way of doing it, but I wonder about versioning risk... if there's a breaking change in formidable, and we pass the new formidable to the function, it could return a broken parse method vs. the current implementation: importing formidable, enforcing the version in package.json, then exporting it.

[tunnckoCore]

You will (need) to listen for new versions anyway, even with the current approach. If here come breaking change it will come to you too anyway. Also, I think no major release is come anytime soon.

[programmerpinggiran]

I have problem, i have nodejs api retreive upload from android phone..
This error show randomly..
Sometimes succed, and some time error showing

[tunnckoCore]

@programmerpinggiran can you try the formidable-serverless package by @Amit-A and report back?

[programmerpinggiran]

@tunnckoCore ill try it and report here after few days later
thanks anyway

[programmerpinggiran]

@tunnckoCore error still exist

 Error: Request aborted
    at IncomingMessage.<anonymous> (/root/server/node_modules/formidable-serverless/lib/index.js:102:19)
    at IncomingMessage.emit (events.js:180:13)
    at IncomingMessage.emit (domain.js:422:20)
    at abortIncoming (_http_server.js:441:9)
    at socketOnClose (_http_server.js:435:3)
    at Socket.emit (events.js:185:15)
    at Socket.emit (domain.js:422:20)
    at TCP._handle.close [as _onclose] (net.js:541:12)

[Amit-A]

@programmerpinggiran from your logs it seems like the error is coming from elsewhere.

While the aborted event is captured at formidable-serverless/lib/index.js:102:19:

req.on('aborted', function () {
    self.emit('aborted');
    self._error(new Error('Request aborted'));
});

The event is emitted elsewhere:
... events.js:180:13
... domain.js:422:20
... _http_server.js:441:9 etc.

Check those files (file:line:col)
Never mind those are node's files.

You're using PM2 so I'm assuming you're not in a serverless environment. Will need more information:

• Are you getting this error repeatedly? Request abort can be caused by some normal circumstances (client disconnects).
• Which request (on the client side) is causing the error? If it's a large request make sure you have configured appropriately large request size limits.
• Are you also using body-parser?

[programmerpinggiran]

@Amit-A thanks, this is a great library.

• yes im usign pm2, from "formidalble" version it show error randomly. and "formidable-serverless" it show error once, started script 11 hours ago, about 100-300 user use my app today.
• I have Node js running as API for android client to upload profile image. our client image size is about 50-500kb

app.post('/v2/api/oururlpath', function(request, res) {
    var form = new formidable.IncomingForm();
    var email;
    form.keepExtensions = true;
    form.multiples = true;
    files = [],
        fields = [];
    form.uploadDir = '../../var/www/html/uploads/';
    form.on('field', function(field, value) {
        fields.push([field, value]);
        console.log(field + " val " + value);
        email = value;
    })
    form.on('file', function(field, file) {
        var oldpath = file.path;
        var newpath = "../../var/www/html/uploads/" + file.name;
        var file_path = "http://domain:port/urlpath?img=" + file.name;
        mv(oldpath, newpath, function(err) {
            if (err) {
                console.log("Foto error " + err);
                throw err;
            } else {
                var fotoupdate1;
                if (file.name.indexOf("profile1") > -1) {
                    fotoupdate1 = {
                        email: email,
                        foto1: file_path,
                    };
                } else if (file.name.indexOf("profile2") > -1) {
                    fotoupdate1 = {
                        email: email,
                        foto2: file_path,
                    };
                } else if (file.name.indexOf("profile3") > -1) {
                    fotoupdate1 = {
                        email: email,
                        foto3: file_path,
                    };
                } else if (file.name.indexOf("profile4") > -1) {
                    fotoupdate1 = {
                        email: email,
                        foto4: file_path,
                    };
                } else if (file.name.indexOf("profile5") > -1) {
                    fotoupdate1 = {
                        email: email,
                        foto5: file_path,
                    };
                } else if (file.name.indexOf("profile6") > -1) {
                    fotoupdate1 = {
                        email: email,
                        foto6: file_path,
                    };
                }
                //fotoupdate1 is for sql stuff
            }

        });

        // console.log("--------------- END ---------------");

    })

    form.on('end', function() {
        console.log('done upload photo');
        return res.json({
            success: true,
            status: "1",
            hasil: "file uploaded successfully",
        });

    });
    form.parse(request);
});

• yes im using body parser

const bodyParser = require('body-parser');
    app.use(bodyParser.json({
        limit: '50mb'
    }));
    app.use(bodyParser.urlencoded({
        limit: '50mb',
        extended: true
    }));

our android client using okhttp

public String uploadImage(String email,String url) {
);
        OkHttpClient httpClient = new OkHttpClient();
        MultipartBody.Builder requestBody = new MultipartBody.Builder()
                .setType(MultipartBody.FORM)
                .addFormDataPart("email", session.getEmailuser());

        if (filefoto1 != null) {
            requestBody.addFormDataPart("foto1", email + "_profile1.png",
                    RequestBody.create(MEDIA_TYPE_PNG, new File(filefoto1.getPath())));
        }

        if (filefoto2 != null) {
            requestBody.addFormDataPart("foto2", email + "_profile2.png",
                    RequestBody.create(MEDIA_TYPE_PNG, new File(filefoto2.getPath())));
        }

        if (filefoto3 != null) {
            requestBody.addFormDataPart("foto3", email + "_profile3.png",
                    RequestBody.create(MEDIA_TYPE_PNG, new File(filefoto3.getPath())));
        }

        if (filefoto4 != null) {
            requestBody.addFormDataPart("foto4", email + "_profile4.png",
                    RequestBody.create(MEDIA_TYPE_PNG, new File(filefoto4.getPath())));
        }

        if (filefoto5 != null) {
            requestBody.addFormDataPart("foto5", email + "_profile5.png",
                    RequestBody.create(MEDIA_TYPE_PNG, new File(filefoto5.getPath())));
        }

        if (filefoto6 != null) {
            requestBody.addFormDataPart("foto6", email + "_profile6.png",
                    RequestBody.create(MEDIA_TYPE_PNG, new File(filefoto6.getPath())));
        }

        RequestBody req = requestBody.build();

        final Request request = new Request.Builder()
                .url(url)
                .post(req)
                .build();
        Response response = null;
        try {
            response = httpClient.newCall(request).execute();
            return response.body().string();
        } catch (IOException e) {
            return null;
        }
    }

[lfernando-silva]

The @Amit-A 's formidable-serverless package worked nicely for me in firebase cloud functions. No code changes was needed, just import the package where formidable was required.

[Amit-A]

@programmerpinggiran there's a lot going on there... have you tried unit testing the API endpoint? Once you know the specific fail cases it'll be easier to debug.

While formidable-serverless should be fine alongside body-parser, if you're not in a serverless environment I would separate body-parser from formidable. This is because body-parser can emit its own errors and make it difficult to debug in the mixup, and the double parsing can be an unnecessary performance drag. You can do this by providing body-parser as middleware to only the API endpoints that need it (excluding the endpoint that uses formidable).

[GrosSacASac]

• make formidable work nicely with body-parser
• investigate what happens in case of abort
  What else do we need to do ?

[tunnckoCore]

@lborgman @BrodaNoel @brodwen83 & others.

Please try npm install formidable@canary - preview of v2. The API is almost the same, check the docs or open new issue if there's some problem.

@GrosSacASac lets see what's the state with latest (formidable@canary) versions of both Formidable and Express. Then we can just document how to use it with Express, more correctly.

[spade69]

koa2 meets the same problem, when using bodyParser with formidable.
just remove it and I have sovled this proplem.

[tunnckoCore]

@spade69: just remove it

what you removed? Koa2 or Formidable@canary? Do you mean that the problem continue even with formidable@canary and latest koa2?

I'm going to publish v2 on 29th or 1st, so.. We may continue after that if it's still a problem.

[spade69]

@spade69: just remove it

what you removed? Koa2 or Formidable@canary? Do you mean that the problem continue even with formidable@canary and latest koa2?

I'm going to publish v2 on 29th or 1st, so.. We may continue after that if it's still a problem.

sorry, i mean removing koa-body's bodyParser, i use [email protected] and latest koa2. Now everything works, thx

[tunnckoCore]

@spade69 Now everything works, thx

Cool! One last thing, do you mean with formidable@canary?

[tunnckoCore]

I'm going to fix that in #594 and v2.

[nickFelix]

@Amit-A Thx, that really helped

[chxlvwh]

May be we need not delete the bodyParser.json(). I add bodyParser.raw({ type: 'multipart/form-data', limit: '10mb' }) and solved.

const jsonParser = bodyParser.json({ limit: '10mb' });
const formParser = bodyParser.raw({ type: 'multipart/form-data', limit: '10mb' });

app.post(url, formParser, jsonParser, [controller.func...] )

[Ovec]

My problem was that after 70 to 90s request was aborted. I want to upload big files (2GB and more) to the api. But I was unable to upload bigger file then 2,8 GB on local host while testing.

I moved from koa2 to express (only because I understand express better). But still have the same error.

At the end I did try running tests in different node versions and I've succeed with version 15. Give it a try.
I am using: "formidable": "1.2.2" and "express": "4.17.1", without bodyparser on upload route.

[GrosSacASac]

• Use recent version of Node to have nice default settings.
• Use maxFileSize: 3 * 10 ** 9 option, to receive 3GB files and less
• Do not use bodyParser and formidable, only use formidable
• abort error can and will happen when a clients loses internet or navigates to another page
• listen for error events to avoid abort error to crash the server, example:

const server = http.createServer((request, response) => {
    request.on(`error`, console.error.bind(console, `request error`));
    response.on(`error`, console.error.bind(console, `response error`));

[RupamShaw]

this works for me

 const form = new formidable.IncomingForm({maxFileSize:20 * 1024 * 1024});