Error: EMFILE: too many open files

[RainingChain]

I tried running file-matcher on a large project and I got the error:

   (node:12672) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: EMFILE: too many open files, open 'C:\sessions\XXXXXX.cpp'
   (node:12672) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

[mauriciovigolo]

Hi @RainingChain

I was searching about this issue and it seams to be a known issue from the node fs module when handling a large number of files. Seems that adding the graceful-fs library is the best option to solve this problem.

I will release a new version with the fix for this problem.

Thanks for reporting!

[RainingChain]

Ideally, you would call this.readFileContent for 1000 files (number changeable with an option parameter) and you would await until those 1000 files are done before reading the others.

By the way, there's a bug with the following lines. Returning true/false in the promise will not stop the this.files.some() loop.

file-matcher/lib/filematcher.ts

Lines 403 to 422 in 142e898

	if (this.fileFilter.content && this.files && this.files.length > 0) {
	this.files.some((file, index) => {
	this.readFileContent(file)
	.then((result) => {
	if (result) {
	let processed: number = (index + 1) / this.files.length;
	self.emit('contentMatch', file, processed);
	matchingFiles.push(result);
	}
	if ((self.files.length - 1) === index) {
	resolve(matchingFiles);
	return true;
	}
	}).catch(err => {
	reject(err);
	return true;
	});
	return false;
	});

[mauriciovigolo]

Ideally, you would call this.readFileContent for 1000 files (number changeable with an option parameter) and you would await until those 1000 files are done before reading the others.

I think that is a good solution to handle the situation and avoid to add a new library.

[DavidNiembro]

Hello, some news about this topic ?

[mauriciovigolo]

Hello @DavidNiembro, I will fix it with a similar solution as mentioned by @RainingChain. I will try to release it this week, okay?

[superRaytin]

I've been troubled by this problem for days. And as this problem has not been solved for a long time. I have added a new library file-content-matcher to solve it. Beyond that, there are some other options:

• readFileConcurrency the concurrency behavior for reading files. Default is 1000.
• recursiveDepth the depth while searching recursively from the given path. Default is 0.
• micromatchOptions extra options used in micromatch.match function.

Hope it helps.