--preprocess patterns improvements

[zindel]

While working with fastpack I've noticed a few shortcomings with --preprocess configuration. Here is a couple thoughts how we could make it a little easier:

1. Make --preprocess entries be mutually exclusive, where the first entry in the list gets the priority. This, by the way, is already true in master. Here is an example of usage:

   fpack --preprocess='global\.css$:style-loader?css-loader' --preprocess='\.css$:style-loader?css-loader?modules=true'

   In other words, "use css modules for all the css files except of global.css".

2. Add noop preprocessor which would do nothing, just return the source unmodified. It is special in a way, that it can only be used exclusively and cannot be a part of the chain. For example:

   fpack --preprocess='node_modules/.*\.js$:noop' --preprocess='\.js$'

   The command above says: use builtin preprocessor for all the JabaScript files except of those in the node_modules directory. So, effectively, this mimics the "exclude" capability in webpack.

3. Finally, what if we use glob patterns instead of (or alongside with) regular expressions. For example, the above example would be specified as:

   fpack --preprocess='node_modules/*.js:noop' --preprocess='*.js'

   There are couple limitations/specialities about this change though:

   • * should match all the characters (including /), otherwise it'd be impossible to specify arbitrary paths
   • the pattern will always be checked against the entire filename (not a partial match). The partial match may fail on a case like this: *.js would partially match the lib.js/some.css, which is unlikely what is supposed by the user.

So, open questions:

• does it make sense at all :)?
• is glob a better way than regexp?
• if going this way, should we keep the backward compatibility and both ways of specifying the pattern?

[VitGottwald]

When considering globs, how about the ones used in gitignore?

[zindel]

@baremetalfreak I don't mind, although I'm not sure if the gitignore patterns implementation exists as OCaml library. My choices above were motivated mainly by this implementation.

[matthiaskern]

how about adding an ignore flag instead of noop?

fpack --preprocess-ignore='node_modules/*.js' -- preprocess='*.js'

[zindel]

@matthiaskern this makes sense. Although still has a couple of problems. Consider the following example (say the project is yarn workspaces-based monorepo):

fpack --preprocess='node_modules/my-other-package/*.js' \
         --preprocess-ignore='node_modules/*.js' \
         --preprocess='*.js'

When preprocess-ignore should be applied? If the answer is before everything, then we're going to lose my-other-package transformations. If the answer is respecting the order of appearance of both preprocess-ignore & preprocess, I am not sure how to implement it with cmdliner (used in fastpack to parse command-line arguments).

I like the ignore term though, maybe we could rename noop to ignore? The example above would look like this:

fpack --preprocess='node_modules/my-other-package/*.js' \
         --preprocess='node_modules/*.js:ignore' \
         --preprocess='*.js'

[matthiaskern]

@zindel seems like a good compromise. ignore is very explicit