Project Fortify, an experimental security project, has identified the following potential security issue. If you’re a feature owner CC-ed on this bug, please do your best to review these reports without the security team.

Overview: MediaFoundationVideoEncodeAccelerator fails to clear its pending input queue when reconfiguring for a new resolution. A compromised renderer can exploit this to queue a small buffer, reconfigure the encoder to a large resolution, and cause the underlying MFT to perform an out-of-bounds read. The leaked GPU process memory is encoded into the bitstream and returned to the attacker.

Affected files:

• media/gpu/windows/media_foundation_video_encode_accelerator_win.cc

Estimated timestamp from git blame: 2024-12-03

Vulnerability Details

The MediaFoundationVideoEncodeAccelerator (MFVEA) on Windows contains a flaw in how it handles dynamic resolution changes via UpdateFrameSize(). When a resolution change occurs, the accelerator reconfigures the underlying Media Foundation Transform (MFT) but fails to properly flush or clear its internal pending_input_queue_.

In media/gpu/windows/media_foundation_video_encode_accelerator_win.cc, UpdateFrameSize() uses a DCHECK to assert that the queue is empty before proceeding:

// media/gpu/windows/media_foundation_video_encode_accelerator_win.cc:1036
DCHECK(pending_input_queue_.empty());

Because DCHECK macros are compiled out in Release builds, execution proceeds even if the queue contains frames. The method updates the input_visible_size_ and reconfigures the MFT to the new dimensions. While it explicitly clears bitstream_buffer_queue_ (line 1143), it leaves pending_input_queue_ untouched.

Potential Exploitation Steps

Note: These steps are based on static analysis; a working Proof of Concept has not yet been executed.

1. Queue a Small Frame: A compromised renderer process floods the GPU process with asynchronous Encode Mojo IPCs containing small VideoFrame objects (e.g., 16x16).
2. Buffer Allocation: QueueInput() creates IMFSample buffers for these frames based on the current 16x16 input_visible_size_ and pushes them onto pending_input_queue_.
3. Trigger Resolution Change: Before the hardware encoder finishes processing, the attacker sends a RequestEncodingParametersChange IPC with a large resolution (e.g., 1920x1080).
4. Reconfiguration: The GPU process calls UpdateFrameSize(), bypassing the DCHECK. The MFT is reconfigured to expect 1080p frames, and input_visible_size_ becomes 1920x1080. The 16x16 samples remain in pending_input_queue_.
5. Feeding Stale Inputs: When the MFT signals METransformNeedInput, FeedInputs() dequeues the stale 16x16 sample and passes it to encoder_->ProcessInput() without verifying the sample’s buffer size against the new input_visible_size_.
6. Out-of-Bounds Read: The vendor MFT driver (Intel, NVIDIA, AMD) relies on its newly configured MF_MT_FRAME_SIZE (1080p). It assumes the provided buffer is large enough and reads past the end of the 16x16 allocation, resulting in an out-of-bounds read into adjacent GPU heap memory.
7. Information Leak: The leaked heap memory is encoded as video data and returned to the renderer via the BitstreamBufferReady callback, allowing the attacker to bypass ASLR or steal cross-origin texture data.

Suggested Fix

There are two primary ways to fix this issue:

1. Clear the Queue on Reconfiguration: In UpdateFrameSize(), explicitly clear the pending_input_queue_ and properly discard/notify the client about the dropped frames, similar to how bitstream_buffer_queue_ is cleared.
2. Validate Sample Size Before Processing: Add a check in ProcessInput() or FeedInputs() to ensure that the IMFSample buffer dimensions match the currently configured input_visible_size_. If they do not match, the sample should be rejected rather than passed to the hardware encoder.

Evaluated with Chrome root at commit: ff3d2b74fa39431785bd60e51463b08fcc71ee33

Results so far have been promising, but there can be wrong deductions. If this proves to be a false positive, please close as WAI; data from false positives will be used to improve accuracy over time. And please feel free to reach out to me directly if you have concerns or feedback on the project.