tauri-plugin-webrtc WebRTC for Tauri on Linux GitHub

Architecture

The page keeps capture, codecs and playback. The engine keeps the network, audio processing and the playout clock. Tauri IPC carries encoded video and PCM audio between them.

WebKitGTK webview App JavaScript (unchanged) JS shim RTCPeerConnection, RTCDataChannel, transceivers In the page getUserMedia (WebKitGTK) WebCodecs VP8 and H.264, AudioWorklets Tauri IPC invoke for commands and outgoing media; one ordered Channel per peer connection for events and incoming media Rust, in the app process Plugin core: peer connections per webview, torn down on navigation qrtc engine: ICE, DTLS, SRTP, SCTP, RTP, audio processing, Opus Network: peers, SFUs, STUN and TURN
Only the shim is visible to the app. Camera and microphone permissions stay with the webview.

The shim

guest-js/shim.js is plain ES2020 with no dependencies. The plugin registers it as an initialization script on every frame, so it runs before page scripts. It installs itself only when the webview has no RTCPeerConnection of its own and the engine started.

Every object the shim hands out is a handle to a resource in the engine. W3C state (signalling state, transceiver directions, data channel ready state) lives in the shim, and changes as the engine's events arrive. Engine errors come back as DOMExceptions with the W3C name.

IPC

How media flows

Video

  1. The page captures frames from the track through a hidden <video>.
  2. WebCodecs encodes them, VP8 or H.264, at the bitrate the engine's bandwidth estimate allows.
  3. The engine packetizes, encrypts and sends them.
  4. On receive, the engine reassembles frames and sends them to the page.
  5. WebCodecs decodes them into a canvas. The canvas's captureStream() is the remote track.

Camera tracks send at up to 1280×720 and 30 fps. Screen tracks, recognised by displaySurface or a detail or text content hint, send at up to 1920×1080 and 15 fps, with a bitrate of at least 1.5 Mbps. Keyframe requests from the far side reach the encoder.

Audio

  1. An AudioWorklet takes 48 kHz PCM from the microphone track and sends it to the engine.
  2. The engine runs echo cancellation, noise suppression and AGC, then Opus with in-band FEC.
  3. Received audio goes through a jitter buffer and one playout clock for all receivers.
  4. PCM comes back to an AudioWorklet that feeds the remote track.

The echo canceller needs the audio being played as its reference, which is why audio is processed in the engine rather than in the page.

The engine

qrtc is a separate crate. Its site covers the driver design, the features in depth and the vendored str0m and DTLS patches. The plugin talks to it through the PeerEngine and Peer traits, so Builder::engine can put another implementation behind the same shim.