86f9503e09  Integer underflow leads to crash in ComputeH264InfoFromAVC

The bug at a glance

OBSERVED: adds a lower-bound guard (size <= H264::kNaluTypeSize) before ComputeH264InfoFromAVC continues parsing an SPS NALU from an attacker-supplied avcC (WebCodecs description buffer). INFERRED: without it, a size field smaller than or equal to the NALU-type prefix causes an integer underflow in subsequent length arithmetic, leading to an out-of-bounds read/crash while parsing web-controlled configuration bytes. Medium because the immediate observed effect is a decoder crash (DoS) on attacker data rather than a demonstrated memory-corruption-to-RCE; it lives in libwebrtc’s AVC config parser reachable from JavaScript via VideoDecoder.configure.

The avcC (AVCDecoderConfigurationRecord) is entirely attacker-controlled through WebCodecs VideoDecoder.configure({description}). The parser reads a 16-bit SPS length (size), then does reader.ConsumeBits(8 * (size + H264::kNaluTypeSize)) and only checked reader.Ok(). The interesting flaw: kNaluTypeSize is added into the consumed-bits computation, but nothing rejected a size that is itself <= kNaluTypeSize, so downstream code that subtracts the NALU-type prefix from size underflows to a huge unsigned length. The fix is a single boundary predicate.

Root cause

ComputeH264InfoFromAVC parses an AVCDecoderConfigurationRecord (the avcC / description blob) to recover H264 stream parameters. After skipping fixed header bytes it reads a big-endian 16-bit length field into size describing the byte length of an embedded sequence parameter set (SPS) NALU, advances offset += 2, and then calls reader.ConsumeBits(8 * (size + H264::kNaluTypeSize)) to skip past the NALU while validating that the bit reader stays within bounds.

The pre-patch guard was only if (!reader.Ok()) return { };. That verifies the reader did not run past the end of the buffer for the ConsumeBits call, but it does not constrain size relative to the NALU type prefix. H264::kNaluTypeSize is the size of the one-byte NALU header that precedes the SPS payload. Elsewhere in the AVC/SPS handling the effective payload length is computed as size - kNaluTypeSize (the SPS bytes after the NALU-type byte).

When an attacker supplies size == 0 (as in the PoC, where the length bytes are 0x00 0x00) or any size <= kNaluTypeSize, that subtraction underflows. Because these lengths are unsigned, size - kNaluTypeSize wraps to a value near SIZE_MAX / a very large count, which is then used to bound a read or allocation over the parameter-set bytes, producing an out-of-bounds read of the config buffer and a crash. OBSERVED: the added WPT test feeds a 9-byte avcC ending in 0x00 0x00 0x67 (a zero SPS length followed by an SPS NALU-type byte) and asserts that decoding fails gracefully rather than crashing.

The fix strengthens the guard to if (!reader.Ok() || size <= H264::kNaluTypeSize) return { };, so any SPS length that is not strictly larger than the NALU-type prefix causes the parser to bail out before any size - kNaluTypeSize arithmetic can underflow. This runs inside libwebrtc’s nalu_rewriter.cc, reachable from the WebCodecs H264 decode path and thus from ordinary web content.

Key code

Added lower-bound guard on SPS NALU size in ComputeH264InfoFromAVC

    reader.ConsumeBits(8 * (size + H264::kNaluTypeSize));
-    if (!reader.Ok()) {
+    if (!reader.Ok() || size <= H264::kNaluTypeSize) {
      return { };
    }

Patch walkthrough

• Source/ThirdParty/libwebrtc/Source/webrtc/webkit_sdk/objc/components/video_codec/nalu_rewriter.cc — In ComputeH264InfoFromAVC, after reading the 16-bit SPS length into size and calling reader.ConsumeBits(8 * (size + H264::kNaluTypeSize)), the validation was extended from checking only reader.Ok() to also rejecting size <= H264::kNaluTypeSize. This prevents the later size - kNaluTypeSize computation from underflowing on a size that is zero or no larger than the NALU-type prefix.
• LayoutTests/http/wpt/webcodecs/h264_bad_avc.html — New WPT that constructs a malformed avcC (badAvcC) with a zero-length SPS and calls VideoDecoder.configure with hardwareAcceleration ‘prefer-hardware’, expecting graceful failure rather than a crash.
• LayoutTests/http/wpt/webcodecs/h264_bad_avc-expected.txt — Expected output asserting the single PASS ‘Decoding bad AVC should gracefully fail’.

Background

avcC / AVCDecoderConfigurationRecord — The binary configuration blob (ISO/IEC 14496-15) carrying SPS/PPS and profile/level for H264. In WebCodecs it is supplied verbatim by script as VideoDecoder.configure({description}), so every byte is attacker-controlled.

ComputeH264InfoFromAVC — libwebrtc routine (nalu_rewriter.cc) that walks the avcC to extract H264Information. It reads a 16-bit SPS length and skips the NALU via a bit reader.

H264::kNaluTypeSize — Size of the NALU header/type prefix that precedes the SPS payload. Effective SPS payload length is size - kNaluTypeSize; if size <= kNaluTypeSize this subtraction underflows for unsigned types.

reader.Ok() — State flag of the bit reader indicating no read went out of bounds. It validates ConsumeBits stayed in range but does not enforce semantic constraints such as size being larger than the prefix.

Vulnerability window

1. Exposure — WebCodecs exposes raw H264 avcC parsing to script via VideoDecoder.configure({description}).
2. Latent underflow — ComputeH264InfoFromAVC read a 16-bit SPS length and only checked reader.Ok(), never that size exceeds the NALU-type prefix.
3. Discovery — rdar://171989035 identified that size <= kNaluTypeSize underflows the payload-length computation, crashing on attacker data.
4. Fix + test — Guard extended to reject size <= kNaluTypeSize; a WPT feeding a zero-length SPS asserts graceful failure.

Proof of concept

VERBATIM from LayoutTests/http/wpt/webcodecs/h264_bad_avc.html. badAvcC = 01 64 00 1f ff e1 00 00 67: standard avcC header (configurationVersion 01, profile 64, level 1f, lengthSizeMinusOne ff, numOfSPS e1=1), then an SPS entry whose 16-bit length is 00 00 followed by NALU-type byte 0x67. The zero SPS length is <= kNaluTypeSize, driving the underflow the patch now rejects.

promise_test(async (t) => {
    for (let i = 0; i < 2; ++i) {
        const badAvcC = new Uint8Array([
            0x01, 0x64, 0x00, 0x1f, 0xff,
            0xe1, 0x00, 0x00, 0x67
        ]);
        const decoder = new VideoDecoder({
            output() { },
            error: (e) => { }
        });

        decoder.configure({
            codec: "avc1.64001f",
            codedWidth: 16,
            codedHeight: 16,
            description: badAvcC,
            hardwareAcceleration: "prefer-hardware"
        });

        await new Promise(resolve => setTimeout(() => {
            resolve();
            decoder.close()
        }, 200));
    }
}, 'Decoding bad AVC should gracefully fail');

Exploitation

1. Deliver config — Attacker page calls VideoDecoder.configure with a crafted description whose embedded SPS length is <= kNaluTypeSize.
2. Underflow — size - kNaluTypeSize wraps to a huge unsigned length used to bound reads over the parameter-set bytes.
3. OOB read / crash — The oversized length drives an out-of-bounds read of the small config buffer, crashing the decode path; primary realistic impact is DoS / info-adjacent crash rather than proven controllable corruption.

Detection & hunting

For defenders and SOC / detection engineers:

• Decoder crash on configure —
• ASAN OOB in nalu_rewriter —

Audit directions

• AVC/HEVC config parsers —
• WebCodecs description path —
• reader.Ok() assumptions —