Is there an alternative to maxSize that only divides chunks that are bigger than the maximum size?

[scameron]

We have a quite large legacy code base that we still need to keep around for some time. In this legacy code we define an enormous chunk group, called "main-chunk", that contains a lot of the modules needed for the application to start up. In total, this chunk group adds up to around 30MB in total, minified. Yeah.

By default, splitChunks divides this big chunk group up into a reasonably decent set of chunks, with the biggest ones being fairly manageable and no one chunk being too much bigger than the others. With parallel loading, this works OK. There are 44 chunks in total for this chunk group and here's a sample of the sizes of the top 10 biggest:

9,057,348 bytes     app.chunk.127.b8ed28aa42971788b84b.js
8,140,527 bytes     app.chunk.125.8887a9182690221243e6.js
3,425,898 bytes     app.chunk.587.ceb35d7aaa6a57b48ee4.js
2,684,203 bytes     104.main.812b6f9738f7c5d134c9.js
1,772,276 bytes     app.chunk.109.80f222eb6d9fa889e406.js
1,479,956 bytes     app.chunk.165.572d8fcac8d5aa209d6e.js
1,137,194 bytes     app.chunk.59.410e56b94b429e2b76b5.js
  613,837 bytes     app.chunk.62.cc2647202f1927db86ff.js
  491,345 bytes     app.chunk.113.13eab1895285144b1ee7.js
  486,737 bytes     app.chunk.108.b9698af8335c16bf5503.js

Recently we removed a dependency that affected the relationship between quite a number of modules. The code didn't change almost at all -- it was a 1 line change -- but the dependency graph looks quite different. Now splitChunks produced something a lot less optimal than what is shown above. The main-chunk group is now split into 42 chunks with the top 10 looking like this:

19,232,271 bytes     app.chunk.141.72cf0a6af2b0d67ed7ba.js
 3,463,098 bytes     app.chunk.583.b14b932fa1ae27b79ef0.js
 2,686,691 bytes     101.main.62ef0743278eae180843.js
 1,504,093 bytes     app.chunk.163.873a9077051541824962.js
 1,137,194 bytes     app.chunk.59.e69c0f1627d4dc12b514.js
   613,846 bytes     app.chunk.67.6e8fa43a7603e4706375.js
   581,792 bytes     app.chunk.121.8c475b26cee87d68ed52.js
   485,806 bytes     app.chunk.138.131cd6536da10c66015f.js
   387,907 bytes     app.chunk.110.7ea583702b78563b384d.js
   297,469 bytes     app.chunk.89.872c589c26595580f546.js

That giant 19MB chunk causes a performance problem because all chunks from this group need to be loaded before execution can continue. And that one chunk, being so much bigger than the others, causes a lag waiting for it to complete.

So we want to go back to something like we had before, where the biggest chunk is around 9MB and -- more importantly -- the very largest chunks are roughly the same size.

From the documentation, it looks like maxSize is the splitChunks setting we want. But I can't get it to work the way I want. I would like to it to do nothing more than to just take the one chunk that is too big, and split that chunk into 2 roughly equal sized parts. But maxSize seems to do a lot more than that.

After playing around to find the right size (because maxSize works based on the size of non-minified modules) I landed on maxSize: 25 * 1024 * 1024 being the setting that would bring the biggest chunk down to what it was before. But the problem is with the side effects. With that setting in place, splitChunks now produces 50 chunks (instead of 44 like before), with the top 10 looking like this:

8,680,103 bytes     app.chunk.141.5cf2713f15cd173ec442.js
4,874,124 bytes     app.chunk.142.b01fd16dc2c95ba0ea19.js
3,463,242 bytes     app.chunk.591.0c15bf73fa34fbd8ec62.js
2,686,691 bytes     101.main.03d5f894d0b20bc86c57.js
2,625,137 bytes     app.chunk.145.ad0ee4ade81d8037d5ad.js
1,504,093 bytes     app.chunk.171.66029ab9a1192993f545.js
1,137,194 bytes     app.chunk.59.e69c0f1627d4dc12b514.js
1,032,509 bytes     app.chunk.147.c8787956ffce1c85d2a7.js
  716,825 bytes     app.chunk.150.8e77383272d259dae531.js
  613,846 bytes     app.chunk.67.49307aa0b189cdefa45a.js

You can see that it doesn't just split the one chunk that exceeds the maxSize threshold, but instead it juggles a whole bunch of other stuff around, resulting in more modules than before and a different size distribution from top to bottom. The fact that there are now 50 modules instead of 44 negates any performance benefit that we see from reducing the size of the largest chunk. So we're no better off.

What I'm really looking for is just is a simple setting that does nothing else but take a chunk that exceeds a maximum size and divide that chunk into 2 (or more, as necessary) chunks of roughly equal size. It should not touch any other chunk.

This is not what maxSize does. Is there another plugin I can use for this? Or is there another configuration option that might behave like this?

I'm open to any ideas you might have. If you're still reading this far, thank you in advance.

[scameron]

@sokra, @alexander-akait, do you guys have any ideas for how I could achive a side-effect free "maxSize" behavior?

[scameron]

Actually, the deprecated AggressiveSplittingPlugin seems to do this. It only touches chunks that are bigger than maxSize and doesn't touch anything else.

The problem is that both SplitChunkPlugin and AggressiveSplittingPlugin tap the same optimizeChunks hook and specify the same stage. Since splitChunks is created internally, it always seems to run after a plugin I declare in my config. And I definitely don't want AggressiveSplittingPlugin to run before splitChunks or it will really mess up the chunk splitting optimizations.

As an experiment, I changed the AggressiveSplittingPlugin tap to pass in a stage of STAGE_ADVANCED + 1, which makes it run after splitChunks is done. This way, it only trims down the very largest chunks and doesn't touch anything else.

This seems to work well for my case, but it means using a deprecated plugin. Would you be open to making this plugin work again for cases like mine? The other option is to change the behavior of maxSize in splitChunks but I wouldn't even know where to begin with that...

What do you think?

[alexander-akait]

Sorry for delay, is it still valid? We can implement maxSizeStrict (maybe better name), I don't think we really need AggressiveSplittingPlugin, anyway you can always implement an own plugin (just copy code from AggressiveSplittingPlugin) and do some magic here (what you want)

[scameron]

Thanks for the reply. In the end, that's what I ended up doing. Our problem was happening in one particular case, so I implemented a small plugin inspired by a bit of the code from the built-in splitting plugins. It seems to do the trick. I set the stage to STAGE_ADVANCED + 1 so that it runs after SplitChunksPlugin.

[alexander-akait]

I am fine with the PR, maybe other developers find this useful