Real-time audio and video technology in instant messaging applications is almost the last high wall in the development of IM. The reason is: real-time audio and video technology = audio and video processing technology + a collection of horizontal technology applications of network transmission technology, and the public Internet is not designed for real-time communication.
There are two international organizations that formulate video codec technology, one is "International Telecommunication Union (ITU-T)", and the standards it formulates include H.261, H.263, H.263+, etc., and the other is "International Telecommunication Union (ITU-T)". Standardization Organization (ISO)" It has developed standards such as MPEG-1, MPEG-2, MPEG-4, etc. H.264 is a new digital video coding standard jointly formulated by the Joint Video Team (JVT) jointly established by the two organizations, so it is both ITU-T's H.264 and ISO/IEC's MPEG-4 advanced video Part 10 of Advanced Video Coding (AVC). Therefore, whether it is MPEG-4 AVC, MPEG-4 Part 10, or ISO/IEC 14496-10, it refers to H.264.
H.264 is a new generation of digital video compression format following MPEG4 jointly proposed by the International Organization for Standardization (ISO) and the International Telecommunication Union (ITU). H.264 is one of the video codec technical standards named after the H.26x series by ITU-T. H.264 is a digital video coding standard developed by the Joint Video Team (JVT: joint video team) of ITU-T's VCEG (Video Coding Experts Group) and ISO/IEC's MPEG (Moving Picture Coding Experts Group). The standard originated from the development of an ITU-T project called H.26L. Although the name H.26L is not very common, it has been used all the time. H.264 is one of the standards named after the H.26x series by ITU-T, and AVC is the name of ISO/IEC MPEG.
The main features of the H264 standard are as follows:
Higher coding efficiency:
Compared with the bit rate efficiency of H.263 and other standards, it can save more than 50% of the bit rate on average.
High-quality video images:
H.264 can provide high-quality video images at low bit rates, and high-quality image transmission at relatively low bandwidth is the application highlight of H.264.
Improve network adaptability:
H.264 can work in the low-latency mode of real-time communication applications (such as video conferencing), and can also work in video storage or video streaming servers without delay.
Adopt hybrid coding structure:
Same as H.263, H.264 also uses a hybrid coding structure using DCT transform coding plus DPCM differential coding, and also adds features such as multi-mode motion estimation, intra-frame prediction, multi-frame prediction, and content-based New coding methods such as variable length coding and 4x4 two-dimensional integer transformation have improved coding efficiency.
H.264 has fewer encoding options:
When encoding in H.263, it is often necessary to set quite a few options, which increases the difficulty of encoding, while H.264 achieves a simple "return to the basics" and reduces the complexity of encoding.
H.264 can be applied in different occasions:
H.264 can use different transmission and playback rates according to different environments, and provides a wealth of error handling tools, which can well control or eliminate packet loss and bit errors.
Error recovery function:
H.264 provides a tool to solve the problem of network transmission packet loss, and is suitable for transmitting video data in wireless networks with high bit error rate transmission.
Higher complexity:
264 performance improvements come at the cost of increased complexity. It is estimated that the computational complexity of H.264 encoding is about three times that of H.263, and the decoding complexity is about twice that of H.263. Instant messaging and chat software app development can be added to Wei Keyun
1. Low Bit Rate:
Compared with compression technologies such as MPEG2 and MPEG4 ASP, under the same image quality, the amount of data compressed by H.264 technology is only 1/8 of MPEG2 and 1/3 of MPEG4.
2. High-quality images:
H.264 can provide continuous, smooth high-quality images (DVD quality).
3. Strong fault tolerance:
H.264 provides the necessary tools to solve errors such as packet loss that are prone to occur in an unstable network environment.
4. Strong network adaptability:
H.264 provides a network abstraction layer (Network Abstraction Layer), so that H.264 files can be easily transmitted on different networks (such as the Internet, CDMA, GPRS, WCDMA, CDMA2000, etc.).
The biggest advantage of H.264 is that it has a high data compression ratio. Under the condition of the same image quality, the compression ratio of H.264 is more than 2 times that of MPEG-2, and 1.5 to 2 times that of MPEG-4. For example, if the size of the original file is 88GB, it will become 3.5GB after compression using the MPEG-2 compression standard, with a compression ratio of 25:1, and it will become 879MB after compression using the H.264 compression standard, from 88GB to 879MB, The compression ratio of H.264 is astonishing 102:1. Low bit rate (Low Bit Rate) plays an important role in the high compression ratio of H.264. Compared with compression technologies such as MPEG-2 and MPEG-4 ASP, H.264 compression technology will greatly save users' download time and data traffic charges. It is especially worth mentioning that H.264 has high-quality and smooth images while having a high compression ratio. Because of this, video data compressed by H.264 requires less bandwidth during network transmission. Also more economical.
The H.264 draft includes tools for error elimination, which facilitates the robustness of compressed video transmission in environments with frequent bit errors and packet loss, such as mobile channels or IP channels.
In order to resist transmission errors, the time synchronization in the H.264 video stream can be completed by using intra-frame image refresh, and the space synchronization is supported by slice structured coding. At the same time, in order to facilitate resynchronization after a bit error, a certain resynchronization point is also provided in the video data of an image. In addition, intra-frame macroblock refreshing and multiple reference macroblocks allow the encoder to consider not only the coding efficiency but also the characteristics of the transmission channel when deciding on the macroblock mode.
In addition to using the change of the quantization step size to adapt to the channel code rate, in H.264, the method of data segmentation is often used to cope with the change of the channel code rate. Generally speaking, the concept of data segmentation is to generate video data with different priorities in the encoder to support the quality of service QoS in the network. For example, a syntax-based data partitioning method is used to divide each frame of data into several parts according to their importance, which allows less important information to be discarded when the buffer overflows. A similar approach to temporal data partitioning can also be employed by using multiple reference frames in P and B frames.
In the application of wireless communication, the large bit rate change of the wireless channel can be supported by changing the quantization precision or space/time resolution of each frame. However, in the case of multicast, it is not possible to require the encoder to respond to varying bit rates. Therefore, unlike the Fine Granular Scalability (FGS) method used in MPEG-4 (which is relatively inefficient), H.264 uses stream-switched SP frames instead of hierarchical coding.