Skip to content

Fix H.264 codec matching#931

Open
ladvoc wants to merge 3 commits intomainfrom
jacobgelman/clt-2612-h264-codec-matching-bug
Open

Fix H.264 codec matching#931
ladvoc wants to merge 3 commits intomainfrom
jacobgelman/clt-2612-h264-codec-matching-bug

Conversation

@ladvoc
Copy link
Contributor

@ladvoc ladvoc commented Mar 6, 2026

Addresses an issue with H.264 streams using packetization-mode=0. This affects tracks published between two cloud regions and all video tracks published by the ESP32 SDK.

Root cause: the platform hardware decoder is not recognized as supporting H.264 streams with packetization-mode=0 because the codec matching logic treats different packetization modes as entirely distinct codecs. As a result, no compatible decoder is found and none is created. Packets are received but have no decoder to process them, so no frames are ever emitted to the user. Other SDKs (Swift, web, etc.) did not have the same issue.

@theomonnom
Copy link
Member

Ok to ignore packetization-mode, but I don't think we should ignore profile-level-id

@ladvoc ladvoc marked this pull request as ready for review March 6, 2026 21:44
@theomonnom
Copy link
Member

mmh actually are we sure about that: https://datatracker.ietf.org/doc/html/rfc6184#section-5.4

@davidzhao davidzhao requested a review from cnderrauber March 9, 2026 23:27
else
adjusted.parameters.erase("packetization-mode");
if (sf.IsSameCodec(adjusted))
return factory->Create(env, format);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might miss some logic consideration here.
Just want to make sure, should it be
factory->Create(env, adjusted);
rather than
return factory->Create(env, format);

@boks1971
Copy link

AFAIU, packetization-mode=1 should be able to decode packetization-mode=0 also. So, it should be okay to search for a match as long as the decoder is capable of decoding packetization-mode=1. However, if supported formats does not have packetization-mode, erasing it and searching without it might be risky, i. e. if the supported formats only supports mode=0 (implicit default), searching for match after erasing would match mode=1 also and that would not work. In that case, I am thinking packetization-mode=0 should be used for search.

@boks1971
Copy link

AFAIU, packetization-mode=1 should be able to decode packetization-mode=0 also. So, it should be okay to search for a match as long as the decoder is capable of decoding packetization-mode=1. However, if supported formats does not have packetization-mode, erasing it and searching without it might be risky, i. e. if the supported formats only supports mode=0 (implicit default), searching for match after erasing would match mode=1 also and that would not work. In that case, I am thinking packetization-mode=0 should be used for search.

Would be good to test in case there are decoders who support only mode=1 exist, but maybe hard to find.

@ladvoc ladvoc force-pushed the jacobgelman/clt-2612-h264-codec-matching-bug branch from 6355818 to e72d4a6 Compare March 10, 2026 19:23
@ladvoc
Copy link
Contributor Author

ladvoc commented Mar 10, 2026

I added end-to-end testing for video streaming in #937 to give us more confidence in this change and future changes affecting the video layer. However, as noted in that PR, only VP8 and VP9 can currently be tested in CI, so I will run manually on different platforms to verify the H.264 and H.265 paths.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants