Audio and Video Mixing Method to Enhance WebRTC

Abstract

The WebRTC protocol can provide live streaming of peer-to-peer connections via JavaScript (JS) application programming interface (API) calls to a web browser. However, the protocol is restricted to a small number of peers because there is no simple way to mix real-time streams from multiple peers and then distribute the mixed stream to a large number of audiences. For example, it is necessary to mix audio and video streams from peers in a conversation and broadcast the real-time mixed stream to more than 10k audiences who only watch the video. It is also necessary to blend synchronous content (e.g., logos, music) to the live conversation stream. WebRTC does not currently provide a mechanism to easily support these cases. This paper proposes a method for the synchronized mixing of real-time audio/video streams from multiple peers while minimizing latency. This method enables the implementation of an online live conversation system that is able to mix live conversation streams from multiple peers and then rebroadcast the mixed stream to a large number of audiences.

Keywords

• WebRTC
• real-time multimedia communications
• live streaming
• synchronization mechanism
• media synchronization