YCoCg matrix support

[fabiosky]

Currently the VideoColorSpace: matrix property supports the following values:

• "rgb"
  Matrix coefficients used by sRGB.
• "bt709"
  Matrix coefficients used by BT.709.
• "bt470bg"
  Matrix coefficients used by BT.601 PAL.
• "smpte170m"
  Matrix coefficients used by BT.601 NTSC.

It is known that:

• At least H.264/MPEG-4 AVC and HEVC are fully supporting such color space
• It has good coding gain (at least comparable to ycbcr/bt.601 with full range)
• Conversion from/to rgb color space should be faster than with ycbcr/bt.601 since it results in additions/shifts for integer arithmetic, or just multiplies by 0.25 and 0.5 for float ones)

Are there any plans to support such color space?

Thanks,
Fabio

[sandersdan]

Color matrices are being added on-demand, based on usage where data is available.

In H.273, this matrix is called YCgCo and has ID 8. I do not currently have usage data to share, but it is my understanding that YCgCo content is rare in practice. FFmpeg describes it as "used by Dirac / VC-2 and H.264 FRext".

I expect that browsers already support this color matrix, so adding it to WebCodecs is probably trivial.

[fabiosky]

Thanks for the explanation.

Currently it is already possible to achieve more or less the wanted feature by either using a software implementation of webcodecs (tested just on Chrome), or using VideoFrame.copyTo and at that point one can have full control on the yuv planes and thus also the conversion to rgb.

However, both solutions are typically slower than rather using the hardware implementation as in the first case decoding would be slower and with copyTo it is likely that copy from gpu to cpu would be slower as well.

Most efficient path would be use the hardware implementation and process the generated frame within WebGL/WebGPU. Not checked on WebGPU actually, but at least with WebGL produced colors looks incorrect; that suggested me about the lack of internal YCoCg to RGB conversion when binding an external texture.