Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistent SVC metadata between WebCodecs and WebRTC Encoded Transform API #619

Closed
aboba opened this issue Dec 7, 2022 · 7 comments · Fixed by aboba/webcodecs#1 · May be fixed by #756
Closed

Inconsistent SVC metadata between WebCodecs and WebRTC Encoded Transform API #619

aboba opened this issue Dec 7, 2022 · 7 comments · Fixed by aboba/webcodecs#1 · May be fixed by #756
Assignees
Labels
PR exists A PR has been submitted that addresses this issue

Comments

@aboba
Copy link
Collaborator

aboba commented Dec 7, 2022

Currently the EncodedChunkMetadata dictionary includes SvcOutputMetadata, which only supports temporal scalability (temporalLayerId).

However, within the Encoded-Transform API, RTCEncodedFrameMetadata includes its own (incompatible) SVC metadata.

Here is a comparison of the two approaches:
https://docs.google.com/presentation/d/1lFAUSvApbBYfBNJH_xcRW0YjD0aF5T1ZqjyyDJMesJw/edit#slide=id.g1a4ac56601a_3_0

Can we define the complete set of SVC metadata in WebCodecs and then have it inherited by Encoded Transform?

Related: w3c/webrtc-encoded-transform#170

@dalecurtis
Copy link
Contributor

+1 for consistency, but defer to @Djuffin

@aboba aboba self-assigned this Dec 9, 2022
@aboba aboba added the agenda Add to Media WG call agenda label Dec 9, 2022
@Djuffin
Copy link
Contributor

Djuffin commented Dec 9, 2022

It's unusual to add fields unless there is at least one implementation that makes use of them. But in this case compatibility with Encoded Transforms seems like a pretty good reason to add these fields.

I think we can even rename SvcOutputMetadata into something more relevant to video frames, Output doesn't seem to convey a lot of meaning outside of webcodecs' context.

@aboba
Copy link
Collaborator Author

aboba commented Feb 2, 2023

The Encoded Transform metadata has now been updated. However, it still is not completely compatible with the WebCodecs extended SVC metadata proposal.

@dalecurtis
Copy link
Contributor

ed note: @aboba to send PR and @Djuffin to review.

@aboba
Copy link
Collaborator Author

aboba commented Feb 2, 2023

A summary of where we are now.

Here is the RTCEncodedVideoFrameMetadata dictionary:

image

Here is what is in WebCodecs Section 6.7:

image

Here is what was in the extended WebCodecs SVC metadata proposal:

image

@aboba
Copy link
Collaborator Author

aboba commented Feb 4, 2023

@pthatcherg @alvestrand Can you take a look?

@aboba aboba added the PR exists A PR has been submitted that addresses this issue label Mar 1, 2023
aboba added a commit that referenced this issue Mar 16, 2023
@chrisn
Copy link
Member

chrisn commented Apr 13, 2023

Minutes from 11 Apr 2023 Media WG meeting: https://www.w3.org/2023/04/11-mediawg-minutes.html#t03

@chrisn chrisn removed the agenda Add to Media WG call agenda label Jul 6, 2023
@aboba aboba changed the title Consistent SVC metadata between WebCodecs and Encoded Transform API Inconsistent SVC metadata between WebCodecs and WebRTC Encoded Transform API Aug 14, 2023
aboba added a commit to aboba/webcodecs that referenced this issue Jan 4, 2024
aboba added a commit that referenced this issue Jan 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
PR exists A PR has been submitted that addresses this issue
Projects
None yet
4 participants