This feature defines an API surface for manipulating raw media carried by MediaStreamTracks such as the output of a camera, microphone, screen capture, or the decoder part of a codec and the input to the decoder part of a codec. It uses WebCodecs interfaces to represent raw media frames and exposes them using streams, similarly to the way the WebRTC Insertable Streams spec exposes encoded data from RTCPeerConnections.

Motivation

The motivation for this feature is to support some of the use cases described in WebRTC Next Version Use Cases. More specifically: * Funny Hats (https://www.w3.org/TR/webrtc-nv-use-cases/#funnyhats*): Refers to manipulation of media prior to encoding and after decoding to provide effects such as background removal, funny hats, voice effects. * Machine Learning (https://www.w3.org/TR/webrtc-nv-use-cases/#machinelearning*): Refers to applications such as real-time object identification/annotation.

Specification

Editor's draft

Status in Chromium

Blink>MediaStream


In development (tracking bug)

Consensus & Standardization

After a feature ships in Chrome, the values listed here are not guaranteed to be up to date.

  • No signal
  • No signal
  • No signal
  • Positive

Owners

Intent to Prototype url

Intent to Prototype thread

Last updated on 2021-01-10