Quality handling in WebP near-lossless (and arguably lossless) mode is incorrect

[geomaster]

Bug report

Describe the bug

In webpsave.c:

	if (webp->lossless)
		webp->config.quality = webp->Q;
	if (webp->near_lossless)
		webp->config.near_lossless = webp->Q;

i.e., near_lossless is a boolean flag in libvips, but an integer flag in libwebp:

       -near_lossless int
              Specify the level of near-lossless image preprocessing. This option adjusts pixel values  to
              help  compressibility,  but  has  minimal impact on the visual quality. It triggers lossless
              compression mode automatically. The range is 0 (maximum preprocessing) to  100  (no  prepro‐
              cessing,  the  default).  The typical value is around 60. Note that lossy with -q 100 can at
              times yield better results.

libivps uses the quality (Q) value to double as the value for near_lossless . However, this has an unfortunate side effect, due to the overloaded meaning for Q in lossless mode:

       -q float
              Specify the compression factor for RGB channels between 0 and 100. The default is 75.
              In case of lossy compression (default), a small factor produces a smaller  file  with  lower
              quality. Best quality is achieved by using a value of 100.
oops! --->    In  case of lossless compression (specified by the -lossless option), a small factor enables
              faster compression speed, but produces a larger file.  Maximum compression  is  achieved  by
              using a value of 100.

This means that in lossless mode, the quality factor actually means an additional type of "effort". This is OK for the libvips user (even if terribly unintuitive), because we can always pass a quality of 100 in lossless mode to get the best encode.

But in near-lossless mode, we cannot do so, as the near-lossless parameter value and the quality we pass share the same value, see the code snippet above. That means we can never get the "absolute best" of near-lossless mode, which is -near_lossless 0 -quality 100.

One would think this is not a big deal, but unfortunately I've noticed considerable file size differences between -q 0 and -q 100 in lossless mode, on the order of 10-20% bigger files for -q 0. This means that, for example, -q 50 -near_lossless 50 can be worse than -q 100 -near_lossless 0.

And, it's really unclear how this quality interacts with the "effort" that libvips passes as webp->method, and it should probably be at least documented to affect the file size in lossless mode, because the "quality" association is not very clear IMO.

To Reproduce
Steps to reproduce the behavior:

1. Load a VImage with libvips, any logo-like graphic with gradients @ ~600x300px should do
2. Call VImage::webpsave() with near_lossless = true, q = 100
3. Now call VImage::webpsave() with near_lossless = true, q = 0
4. For comparison, call cwebp -lossless -q 100 -near_lossless 0

Expected behavior
Expected to be able to get outputs similar to the cwebp one using libvips.

3. Actual behavior
4. Observe that in case (1), near_lossless is actually disabled and the results are the same as without near_lossless
5. Observe the file size in (2) that is usually worse than in (1), as the lossless encoder was told to try the absolute minimum, even though the near_lossless preprocessing is the most aggressive (at 0)
6. Notice the file size in case (3) is better than either of the options.
   Screenshots
   If applicable, add screenshots to help explain your problem.

[geomaster]

I'm happy to do a PR to fix this, but before doing so, I'd like to hear some thoughts on how to solve this from an API level, whether you have any backwards compatibility concerns, etc.

[kurtextrem]

Only half related to this issue - and I stumbled over the same thing in the past, so I fully agree it would be awesome to have those options - I don't think near_lossless 0 q 100 is the absolut best in all regards, see this HN comment from the author of near-lossless in webp: https://news.ycombinator.com/item?id=39563650. For absolute best pixel perfect quality, near_lossless 60 or 80 + q 100 should be optimum. Absolute best trade-off might be near_lossless 0 q 100 in regards to file-size/quality though (not pixel perfect), as you've mentioned :)

[jcupitt]

Hi @geomaster,

I agree the libvips webp API needs fixing.

I think we should design a new API that reflects webp best practice and has options that match cwebp as much as possible. I don't think we should expose all the functionality (there's too much!), just the generally useful subset.

The constraint is that we can't change the meaning of existing websave arguments, so we'll need new names.

We can tag the existing arguments as VIPS_ARGUMENT_DEPRECATED, so they don't appear in docs or IDEs, but they should continue to work. It's fine to throw an error if someone tries to mix new API args and old API args.

At a pinch we could tag the whole of webpsave as DEPRECATED and make a new saver, but I think it'd probably be possible just to define a new set of arguments to the existing saver.