A single wildcard results in creation of inotify watches for non-matching directories

[inga-lovinde]

Describe the bug

At least in some cases, watching a path containing a wildcard (e.g. a/*/c) causes chokidar to create an inotify watch for every subdirectory of the tree starting with the directory preceding the wildcard (a/ in this case), for example, for a/b/d/dd/, even though nothing happening in a/b/d/dd/ will ever match a/*/c.

These inotify watches also mean that every change in the unrelated directories (such as a/b/d/dd) will result in chokidar emitting a raw event.

This has two significant performance implications:

1. inotify watches come at a cost, and usually there are system limits on maximum number of inotify watches;
2. Every raw event presumably has to be checked against watch patterns in order to determine that it can be discarded. In cases when chokidar is initialized with a large array of patterns, and there are frequent changes in unrelated directories, this can result in a lot of checks and a lot of unnecessary load.

Versions (please complete the following information):

• Chokidar version 3.5.3
• Node version 18.12.1
• OS version: Alpine Edge.

To Reproduce:

For debugging purposes, update nodefs-handler.js to log created watches: in createFsWatchInstance, replace

    return fs.watch(path, options, handleEvent);

with

    console.log(`Watching ${path}`);
    return fs.watch(path, options, handleEvent);

Sample program

const chokidar = require('chokidar');
const fs = require('fs/promises');

const main = async () => {
    await fs.rm('./chokidar-sample', { recursive: true, force: true });
    await fs.mkdir('./chokidar-sample');
    await fs.mkdir('./chokidar-sample/watched');
    await fs.mkdir('./chokidar-sample/external');
    await fs.mkdir('./chokidar-sample/external/a');
    await fs.mkdir('./chokidar-sample/external/a/aa');
    await fs.writeFile('./chokidar-sample/external/a/trigger.txt', '');
    await fs.writeFile('./chokidar-sample/external/a/aa/ignore.txt', '');
    await fs.mkdir('./chokidar-sample/external/b');
    await fs.mkdir('./chokidar-sample/external/b/bb');
    await fs.writeFile('./chokidar-sample/external/b/bb/ignore.txt', '');

    const watcher = chokidar.watch('../external/*/trigger.txt', {
        cwd: './chokidar-sample/watch',
        persistent: true
    });

    watcher
        .on('add', path => console.log(`File ${path} has been added`))
        .on('change', path => console.log(`File ${path} has been changed`))
        .on('unlink', path => console.log(`File ${path} has been removed`))
        .on('raw', (event, path, details) => { // internal
            console.log('Raw event info:', event, path, details);
        });

    watcher.on('ready', async () => {
        console.log('Initial scan complete. Ready for changes');
        await fs.appendFile('./chokidar-sample/external/a/trigger.txt', 'test');
        await fs.appendFile('./chokidar-sample/external/a/aa/ignore.txt', 'test');
        await fs.appendFile('./chokidar-sample/external/b/bb/ignore.txt', 'test');
        watcher.close();
    });
}

main();

Expected behavior
Expected output:

Watching chokidar-sample/external
Watching chokidar-sample/external/a
Watching chokidar-sample/external/b
Watching chokidar-sample/external/a/trigger.txt
File ../external/a/trigger.txt has been added
Initial scan complete. Ready for changes
Raw event info: change trigger.txt { watchedPath: 'chokidar-sample/external/a' }
Raw event info: change trigger.txt { watchedPath: 'chokidar-sample/external/a/trigger.txt' }
File ../external/a/trigger.txt has been changed

Actual output (unexpected lines are prefixed with "!"):

Watching chokidar-sample/external
Watching chokidar-sample/external/a
Watching chokidar-sample/external/b
Watching chokidar-sample/external/a/trigger.txt
File ../external/a/trigger.txt has been added
! Watching chokidar-sample/external/a/aa
! Watching chokidar-sample/external/b/bb
Initial scan complete. Ready for changes
Raw event info: change trigger.txt { watchedPath: 'chokidar-sample/external/a' }
Raw event info: change trigger.txt { watchedPath: 'chokidar-sample/external/a/trigger.txt' }
File ../external/a/trigger.txt has been changed
! Raw event info: change ignore.txt { watchedPath: 'chokidar-sample/external/a/aa' }
! Raw event info: change ignore.txt { watchedPath: 'chokidar-sample/external/b/bb' }

Additional context
I've encountered this issue while trying to debug a issue we had with pm2: in some cases, its management process became unresponsive and consumed up to 20% of eight-core CPU, and up to 8GB RAM, prior to crashing a minute or two later. We were able to reproduce it on both Alpine and Ubuntu.
Turns out one of our 100 watch patterns contained a wildcard which resulted in chokidar watching, among other unrelated things, an unrelated directory that had heavy I/O going on in these cases (with up to a hundred changes per second).
My guess is that at such loads, anymatch + Node's default memory allocator do not play very nice together, resulting in memory leaks (probably similar to #1268).
However, I did not look at anymatch calls; unfortunately, I only have so much time, and this sample illustrates the problem well enough. Unfortunately, I also didn't have enough time to debug why chokidar monitors these unrelated directories, or to come up with any potential fixes.