Table of contents
About Live
Live process
-
Video collection terminal:
1. Video collection: Use camera equipment to obtain real-time video streams.
2. Video processing: Process the captured video, including adding effects such as beautification, filters, and watermarks.
3. Audio and video encoding and compression: The processed audio and video data is encoded and compressed. The commonly used encoding methods include H.264 video encoding and AAC audio encoding. -
Live stream video server:
1. Video stream analysis and encoding: receive the audio and video data stream from the video acquisition end, analyze and decode.
2. Push RTMP/HLS format video stream: encapsulate the decoded audio and video data stream into RTMP or HLS format, and push it to the streaming media server. These streaming media servers will handle client connections and requests, and distribute live content to clients. -
Video playback end:
1. Audio and video decoding: At the playback end, the data stream from the live streaming video server is received and decoded to obtain the original audio and video data.
2. Play + chat interaction: play the decoded audio and video data, and provide a user interface for the audience to watch. Some interactive functions can be implemented on the playback side, such as barrage, likes, comments, etc.
Live video format package
Live video is usually packaged in a specific format for transmission and playback on the network. These packaging formats can combine and manage audio, video, metadata and other information to ensure smooth transmission and playback experience.
FLV: A streaming media packaging format developed by Adobe, originally used for Flash player and RTMP protocol transmission. It can hold audio, video, text, and metadata, and is suitable for real-time transmission and playback.
TS: A packaging format for transmitting multimedia content, often used in digital TV and IPTV. It can accommodate multiple audio, video and data streams, suitable for real-time transmission and broadcasting. The TS format is used in the HLS protocol to encapsulate audio and video streams.
MP4: A general-purpose multimedia packaging format that can hold audio, video, subtitles, and metadata. It is widely used to store and transmit audio and video content, and can also be used for live broadcasting. The MP4 format typically uses H.264 video encoding and AAC audio encoding.
Push and pull
推流（Publishing）and拉流（Subscribing）are common terms in real-time audio and video communication, which are used to describe the transmission process of audio and video data. Push streaming refers to sending local audio and video data to the server, while pulling streaming refers to receiving remote audio and video data from the server.
HLS is a streaming media transmission protocol developed by Apple Inc. It is mainly used to realize real-time audio and video streaming on iOS devices, web browsers and other devices that support HLS. HLS cuts the audio and video stream into a series of small media files (encapsulated in the .ts format, and divides the audio and video stream into small .ts files), and transmits and plays them one by one through the HTTP protocol.
Advantages:
1. Applicable to various devices and platforms, including iOS, Android, Web, etc.
2. Support adaptive bit rate, dynamically adjust video quality according to network conditions.
3. It can provide a better smooth playback experience under different bandwidths.
4. Using HTTP protocol for transmission, no specific server support is required.
Disadvantages:
1. The delay is relatively high, usually around 10 seconds.
2. It is necessary to cut the media stream into small files, increasing server pressure and storage overhead.
RTMP is a real-time message transmission protocol developed by Adobe, based on TCP protocol transmission, originally used for audio and video transmission between Flash player and media server.
Advantages:
1. The delay is relatively low, usually around 1-3 seconds, suitable for real-time interaction.
2. Support instant live broadcast, which is suitable for some scenes with high delay requirements.
3. Better compatibility with Flash player.
Disadvantages:
1. Not suitable for iOS devices and some platforms, because they are not widely supported.
2. It needs special RTMP server support, and the deployment and maintenance are relatively complicated.
3. Network restrictions such as firewalls and proxies need to be considered.
Get camera and microphone permissions
navigator.getUserMedia()
Navigator.getUserMedia()Method to alert the user that an audio (0 or 1) and (0 or 1) video input device is required, such as a camera, screen share, or microphone.
grammar: navigator.getUserMedia ( constraints, successCallback, errorCallback );
code:
<body>
<video autoplay id="video" width="400" height="300"></video>
<script>
var video = document.getElementById('video');
//作兼容处理
navigator.getMedia = navigator.getUserMedia ||
navagator.webkitGetUserMedia ||
navigator.mozGetUserMedia ||
navigator.msGetUserMedia;
navigator.getMedia({
audio: true, video: true },
function (stream) {
//MDN文档案例中这样写会报错：video.src = window.URL.createObjectURL(stream);
video.srcObject = stream
video.onloadedmetadata = function (e) {
video.play();
};
},
function (err) {
console.log("The following error occurred: " + err.name);
}
);
</script>
</body>
Although this api can still be used, another api has been changed on the official website——MediaDevices.getUserMedia()
MediaDevices.getUserMedia()
MDN-MediaDevices.getUserMedia()
MediaDevices.getUserMedia()The user will be prompted for permission to use the media input, which will generate a MediaStream containing a track of the requested media type. This stream can contain a video track (from hardware or virtual video sources, such as cameras, video capture devices, screen sharing services, etc.), an audio track (also from hardware or virtual audio sources, such as microphones, A/D converters, etc. etc.), and possibly other track types.
grammar:
navigator.mediaDevices
.getUserMedia(constraints)
.then(function (stream) {
/* 使用这个 stream stream */ })
.catch(function (err) {
/* 处理 error */});
code:
<video autoplay id="video" width="400" height="300"></video>
<script>
var video = document.getElementById('video');
navigator.mediaDevices
.getUserMedia({
audio: true, video: true })
.then(function (stream) {
/* 使用这个 stream stream */
video.srcObject = stream
video.onloadedmetadata = function (e) {
video.play();
};
})
.catch(function (err) {
/* 处理 error */
console.log("The following error occurred: " + err.name);
});
</script>
WebRTC
WebRTC(Web Real-Time Communication) is an open standard and technology that supports real-time communication between browsers, which enables browsers to directly transmit audio, video, and data without plug-ins or additional software. communication. The core goal of WebRTC technology is to realize real-time, peer-to-peer communication, including video chat, audio call, data sharing and other functions.
code:
<body>
<!-- 显示本地视频 -->
<h2>本地视频</h2>
<video id="localVideo" autoplay playsinline style="width: 300px;"></video>
<!-- 显示远程视频 -->
<h2>远程视频</h2>
<video id="remoteVideo" autoplay playsinline style="width: 300px;"></video>
<button id="startCall">发起通话</button>
<button id="answerCall">接听通话</button>
//呼叫者：
<script>
const localVideo = document.getElementById('localVideo');
let localStream;
// 呼叫者：获取本地摄像头和麦克风权限
navigator.mediaDevices.getUserMedia({
video: true, audio: true })
.then(stream => {
localStream = stream;
localVideo.srcObject = localStream;
})
.catch(error => {
console.error('获取本地媒体流失败:', error);
});
const startCallButton = document.getElementById('startCall');
startCallButton.addEventListener('click', () => {
startCall();
});
async function startCall() {
// 呼叫者：创建PeerConnection
const peerConnection = new RTCPeerConnection();
// 添加本地媒体流到PeerConnection
localStream.getTracks().forEach(track => {
peerConnection.addTrack(track, localStream);
});
// 呼叫者：创建Offer并设置本地描述
peerConnection.createOffer()
.then(offer => peerConnection.setLocalDescription(offer))
.then(() => {
// 发送本地描述到远程
const offerSdp = peerConnection.localDescription;
// 在实际应用中，需要将offerSdp发送给远程端
// 远程端通过RTCPeerConnection.setRemoteDescription()设置远程描述
})
.catch(error => {
console.error('创建Offer失败:', error);
});
}
</script>
//被呼叫者：
<script>
const remoteVideo = document.getElementById('remoteVideo');
let remoteStream;
const answerCallButton = document.getElementById('answerCall');
answerCallButton.addEventListener('click', () => {
answerCall();
});
async function answerCall() {
// 创建PeerConnection
const peerConnection = new RTCPeerConnection();
// 设置ontrack事件处理程序以接收远程流
peerConnection.ontrack = event => {
remoteStream = event.streams[0];
remoteVideo.srcObject = remoteStream;
};
// 在实际应用中，你需要获取呼叫者发送的offerSdp
// 并通过RTCPeerConnection.setRemoteDescription()设置远程描述
// 创建Answer并设置本地描述
const answer = await peerConnection.createAnswer();
await peerConnection.setLocalDescription(answer);
const answerSdp = peerConnection.localDescription;
// 在实际应用中，你需要将answerSdp发送给呼叫者
// 呼叫者通过RTCPeerConnection.setRemoteDescription()设置远程描述
}
</script>
</body>
Tencent Cloud Express Live
Tencent Cloud Live Streaming - WebRTC Protocol Streaming
Cloud Live provides push streaming
SDK TXLivePusherfor Web streaming, and is responsible for pushing the audio and video images captured by the browser to the live broadcast server through the WebRTC protocol. Currently, it supports camera collection, microphone collection, screen sharing collection, local media file collection and user-defined collection and other collection methods. It supports local mixed stream processing of the collected content and then pushes it to the back-end server.
1. Import the initialization script (the script needs to be introduced in the body part of HTML, if it is introduced in the head part, an error will be reported.)
<script src="https://video.sdk.qcloudecdn.com/web/TXLivePusher-2.1.0.min.js" charset="utf-8"></script>
2. Create a video container
<div id="local_video" style="width:100%;height:500px;display:flex;align-items:center;justify-content:center;"></div>
3. Generate a streaming SDK instance
//通过全局对象 TXLivePusher 生成 SDK 实例，后续操作都是通过实例完成。
const livePusher = new TXLivePusher();
// 指定本地视频播放器容器 div，浏览器采集到的音视频画面会渲染到这个 div 当中。
livePusher.setRenderView('local_video');
// 设置音视频流
livePusher.setVideoQuality('720p');
// 设置音频质量
livePusher.setAudioQuality('standard');
// 自定义设置帧率
livePusher.setProperty('setVideoFPS', 25);
4. Enable live push streaming
// 开启直播
// 打开摄像头
livePusher.startCamera();
// 打开麦克风
livePusher.startMicrophone();
//传入云直播推流地址，开始推流。
livePusher.startPush(推流地址);
When using the Tencent Cloud live streaming service, the streaming address needs to meet the format of the Tencent Cloud standard live streaming streaming URL, as shown below, which consists of four parts:
5. Close the live broadcast
// 关闭直播
// 停止直播推流
livePusher.stopPush();
// 关闭摄像头
livePusher.stopCamera();
// 关闭麦克风
livePusher.stopMicrophone();
Full code:
<body>
<div id="local_video" style="width:100%;height:500px;display:flex;align-items:center;justify-content:center;"></div>
</div>
<script src="https://video.sdk.qcloudecdn.com/web/TXLivePusher-2.1.0.min.js" charset="utf-8"></script>
<script>
//通过全局对象 TXLivePusher 生成 SDK 实例，后续操作都是通过实例完成。
const livePusher = new TXLivePusher();
// 指定本地视频播放器容器 div，浏览器采集到的音视频画面会渲染到这个 div 当中。
livePusher.setRenderView('local_video');
// 设置音视频流
livePusher.setVideoQuality('720p');
// 设置音频质量
livePusher.setAudioQuality('standard');
// 自定义设置帧率
livePusher.setProperty('setVideoFPS', 25);
// 开启直播
// 打开摄像头
livePusher.startCamera();
// 打开麦克风
livePusher.startMicrophone();
//传入云直播推流地址，开始推流。
livePusher.startPush(推流地址);
// 关闭直播
// 停止直播推流
livePusher.stopPush();
// 关闭摄像头
livePusher.stopCamera();
// 关闭麦克风
livePusher.stopMicrophone();
</script>
</body>
If wrong, please correct me!