Rendition selection with multi-codec manifest could ignore the preferred codec

[woshimaliang]

What version of Hls.js are you using?

1.3.5

What browser (including version) are you using?

Version 124.0.6367.119 (Official Build) (arm64)

What OS (including version) are you using?

MacOS 14.4.1 (23E224)

Test stream

No response

Configuration

{
  startLevel = 2
}

Additional player setup steps

No response

Checklist

• The issue observed is not already reported by searching on Github under https://github.com/video-dev/hls.js/issues
• The issue occurs in the stable client (latest release) on https://hlsjs.video-dev.org/demo and not just on my page
• The issue occurs in the latest client (main branch) on https://hlsjs-dev.video-dev.org/demo and not just on my page
• The stream has correct Access-Control-Allow-Origin headers (CORS)
• There are no network errors such as 404s in the browser console when trying to play the stream

Steps to reproduce

With a multi-codec manifest, which includes both h.265 and h.264 codec, when we set default start level (-1), everything is fine. Hls is able to pick an appropriate rendition to start with H.265 (when browser supports that).

However if we specify a startLevel other than start level (-1), our engineer found:

I thought levels were just based on the manifest passed in, but hls.js actually sorts the renditions by bandwidth, and also has an assumption that the higher the bandwidth, the higher the resolution. So this multi codec manifest breaks this logic because 360w h265 can be lower bandwidth than hls h264, which seems to create unpredictable results. Changing start_level=-1 (default config) fixed the problem that it's able to pick the preferred codec and one of its renditions.

Expected behaviour

set a start level != -1 should not change the behavior. When H.265 (for example) is supported and picked, then sort renditions of that codec, instead of all renditions (of all codecs) in that multi-codec manifest. But we might misunderstand how startLevel:-1 works that it could pick H.264 or H.265 depending on bandwidth.

API doc:

### `hls.startLevel`

- get/set: Start level index (level of first fragment that will be played back).
  - if not overridden by user: first level appearing in manifest will be used as start level.
  - if -1: automatic start level selection, playback will start from level matching download bandwidth (determined from download of first segment).

What actually happened?

unpredictable, based on bandwidth

Console output

no

Chrome media internals output

No response

[robwalch]

Hi @woshimaliang,

1.3.5

v1.5 adds logic for picking advanced codecs on start. I recommend updating HLS.js to the latest (v1.5.8) and not setting startLevel.

I thought levels were just based on the manifest

But we might misunderstand how startLevel:-1 works

Level indexes do not map to the order that variants appear in the manifest. When you set the Level index manually or force playback to begin on a specific Level with startLevel, you force the selection of that Level's codecs. You can either let the player make the best choice on it own (startLevel: -1 with v1.5), or set hls.startLevel on MANIFEST_PARSED after finding the index of the Level with the attributes you like.