Features in depth

How each area of the engine works, the choices behind it, and what it does not do yet.

JSEP and negotiation

• Offers and answers. create_offer, create_answer, set_local_description and set_remote_description follow the JSEP state machine, including pranswer and rollback.
• Perfect negotiation. A remote offer received while a local offer is pending implicitly rolls the local offer back, as the W3C perfect negotiation pattern expects.
• Re-offers. create_offer on an unchanged session returns an offer, which LiveKit and perfect negotiation depend on (a str0m patch).
• ICE restart. restart_ice makes the next offer restart ICE.
• Codec preferences. A transceiver's codec_preferences reorder and filter the payload list. RED, ULPFEC, FlexFEC, comfort noise and telephone events stay, after the preferred codecs.
• Transceivers. The consumer describes each transceiver whole (TransceiverSpec) and the engine upserts it. Setting a description returns TransceiverState for each, with the facts a consumer needs to fire track and update currentDirection.

Errors map to DOMException names through Error::dom_name: InvalidStateError, OperationError, NotSupportedError, SyntaxError and InvalidModificationError.

ICE, STUN and TURN

• Candidates. Host candidates from one socket per interface, server-reflexive candidates through STUN, relayed candidates through TURN. iceTransportPolicy: "relay" keeps only relayed candidates.
• mDNS. As in browsers, each host candidate carries a random <uuid>.local name instead of the interface address, and the engine answers multicast A/AAAA queries for its names (RFC 6762). Remote .local candidates are resolved with a multicast query before they reach the ICE agent; only browser-shaped names are resolved, a few at a time.
• TURN client. A sans-I/O state machine (RFC 8656) with long-term credentials. Permissions are installed on demand and refreshed every 240 s; channels are bound once the permission exists and refreshed every 540 s; the allocation is refreshed before it expires.
• Transports. TURN runs over UDP, over TCP (?transport=tcp), or over TLS (turns:). Over a stream, STUN messages and ChannelData frames are framed per RFC 6062, with ChannelData padded to four bytes. Over UDP, requests are retransmitted from 500 ms, six times; over a stream, a transaction fails after 39.5 s.
• TURN over TLS. rustls with the turn-tls-ring or turn-tls-rustcrypto provider. The server certificate is checked against the platform trust store, with Mozilla's roots as the fallback. Without either feature, turns: servers are skipped.

DTLS and SRTP

str0m runs DTLS and SRTP on RustCrypto. The DTLS certificate is a self-signed ECDSA P-256 certificate built with RustCrypto, so no C library is involved. Its fingerprint travels in the SDP, and the peer checks it.

With a post-quantum feature, a connection negotiates DTLS 1.3 with a hybrid ML-KEM group when the peer supports it. Post-quantum key agreement covers the groups, the policies and the DTLS changes.

Data channels

• Reliable or unreliable (max_retransmits or max_packet_life_time), ordered or unordered, with a protocol string.
• Negotiated channels with an explicit id, or in-band channels announced to the remote peer (RFC 8832).
• A channel is known by an opaque DcHandle, because its SCTP stream id may be unknown until the DTLS role is settled.
• dc_buffered_amount and dc_set_buffered_amount_low_threshold back bufferedAmount and the bufferedamountlow event.
• Messages arrive as PeerEvent::DcMessage with a text or binary Payload.

Audio

Audio is processed in the engine, because echo cancellation needs the signal being played out as its far-end reference.

• Capture. The consumer pushes 48 kHz mono PCM per sender (push_pcm). Each 10 ms frame runs through that sender's audio processing: AEC3 echo cancellation, noise suppression and AGC2 from sonora, a pure Rust port of WebRTC's audio processing. Every 20 ms the result is Opus-encoded (rusty-opus) with in-band FEC, and sent as RTP.
• Processing follows the track. As in browsers, a microphone track is processed and a WebAudio or file track is sent untouched (set_audio_processing).
• Receive. Each remote track has a jitter queue. Its target starts at 40 ms and adapts between 30 and 160 ms: it grows by 20 ms on an underrun, and shrinks by 10 ms after ten seconds without one. An underrun is masked by fading out the last frame played. A lost packet is recovered from the next packet's in-band FEC, or concealed.
• Playout. One 10 ms clock pops a frame per track, mixes them, feeds the mix to every sender's echo canceller, and hands each track's PCM to the consumer (PeerEvent::AudioPcm).
• DTMF. insert_dtmf sends one tone as RFC 4733 telephone events and keeps RFC 4733's 50 ms gap. The caller, such as an RTCDTMFSender, handles the timing between tones.
• Encoded transforms. set_transform routes a transceiver's Opus frames through the consumer, as PeerEvent::EncodedOut before sending and back through decode_audio after receiving. This is how an RTCRtpScriptTransform, such as LiveKit's end-to-end encryption, sees audio.

Video

• Encoding stays with the consumer. The consumer sends encoded frames (send_frame) and receives them (PeerEvent::MediaFrame). A web page can use WebCodecs; a browser engine can use its own codecs.
• VP8 always. Each payload descriptor carries a 15-bit PictureID, as browsers write it; LiveKit's SFU does not forward VP8 without one (a str0m patch).
• H.264 when the configuration lists it in video_codecs: packetization mode 1, offered as constrained baseline, baseline, main and high at level 3.1.
• Bandwidth. Transport-wide congestion control estimates the available bitrate from 800 kbps; PeerEvent::TargetBitrate tells the consumer's encoder what to aim for.
• Keyframes. request_keyframe sends PLI to the remote sender; a remote request arrives as PeerEvent::KeyframeRequest.

Stats

stats() returns RTCStatsReport-shaped JSON entries: peer-connection (data channels opened), transport (DTLS state and tlsGroup), candidate-pair (state, nomination, round-trip time, bytes), and local-candidate and remote-candidate. RTP stream stats come from the consumer, which sees the frames.

Not yet

• VP9 and AV1.
• Receive-side simulcast layers; a sender sends one layer.
• inbound-rtp and outbound-rtp stats inside the engine.
• With the require policy, a peer without post-quantum groups fails by timeout rather than with an explicit error.