Recently, some classmates often ask me how to learn audio and video, how can I get started quickly? Some students who are not familiar with audio and video asked me should I study audio and video? As a 10-year veteran of the audio and video industry, I have some thoughts to share with you, hoping to help you.
Great background
2020 is truly a disaster and difficult year. The outbreak of the coronavirus has made the Chinese New Year, which should have been celebrated throughout the country, become particularly dull. You can't go relatives, you can't visit friends, and you can't go there except stay at home. As Dr. Wenhong Zhang advised us, we must give the virus 闷死.
Everyone hopes that the epidemic will pass quickly and return to normal life soon. But judging from the current situation, it is difficult to end this epidemic in a short time. This hasn't just disappeared from China for a while, and it has exploded overseas. Who knows if it will be transferred back?
But we still have to live! Have to work! Still need to learn!
Good times
Fortunately, we are living in a good era, and technology has developed rapidly in these decades. At the time when the epidemic is raging, we can work remotely through audio and video conferences; we also have an online education system. For most students, they can study online at home.
From one perspective, this outbreak led to a lot of industry although the verge of collapse, but also spawned a new industry development, including audio and video conferencing, and online education is because of 疫the sky industry. When the epidemic is difficult to end in a short period of time, the importance of audio and video conferences and online education is immediately reflected. New communication and education methods are recognized and accepted by more and more people.
We can even determine that the online education industry and the audio and video conferencing industry will rise this year and develop rapidly in the next decade.
At present, major giant companies (such as Ali, Tencent, Toutiao, New Oriental, etc.) have made a large amount of investment in these two directions and have attracted technical talents, which can also indirectly support this judgment.
major trend
[External link image transfer failed. The source site may have an anti-leech link mechanism. It is recommended to save the image and upload it directly (img-Jhz5BMkw-1609828455806)(//upload-images.jianshu.io/upload_images/5956443-536df9a07276ccaf.png?imageMogr2 /auto-orient/strip|imageView2/2/w/1200/format/webp)]
From the perspective of the future trend, with the advent of the 5G era, audio and video have gradually become a must in people's daily lives. In addition to online education, audio and video conferencing, and instant messaging products that must use audio and video technology, other products also need to add audio and video elements. For example, is it safer to change all the graphics and text methods of website registration and login to voice methods? I believe that with the popularization of audio and video technologies, more creative products will spring up like mushrooms.
At present, it is an indisputable fact that audio and video technology has gradually moved from the previous **"ivory tower" to "people's home"**. It is precisely because of such a big trend that in addition to major factories, other small factories are also beginning to look for audio and video talents. A slightly better audio and video talent may now have 3-4 companies rushing to get it and earning it.
The demand for audio and video talents has changed from a niche to the masses, which is more a result of everyone's expectations of the future market. To make an inappropriate analogy, the emergence of 3G/4G has contributed to the 10-year prosperity of the mobile Internet. The emergence of 5G will also contribute to the prosperity of the audio and video industry for at least 10 years.
**In addition to the arrival of 5G, which will allow a wider range of applications for audio and video, audio and video technology will play a more important role in the future. **In fact, among the information that humans can recognize, audio and video contain the most abundant information, but humans do not make good use of them.
The reason is that this information is unstructured data, and humans are good at processing structured data (such as databases). But there is no way to start with these unstructured data, so these audio and video data are wasted.
However, with the development of computer vision and artificial intelligence, people have begun to grasp how to convert unstructured data such as audio and video into structured data. Once we convert audio and video data into structured data, we can use existing technologies (such as search engines, big data, data mining, etc.) to perform various analyses on these data. Is this value even more amazing? The foundation of all this is audio and video technology.
Therefore, the future of audio and video research and development is extremely broad. It can be said that this is the best time to learn audio and video. Join the audio and video research and development team as early as possible to make yourself more effective in the future workplace.
Rich scene
As I said above, in order to compete for the future and seize the opportunity, the major factories will not hesitate to win over talents at a large amount of money. Through the recruitment market, we can learn that a qualified audio and video developer can earn an annual salary of at least 300,000 yuan, and as many as more than 1 million yuan.
There are two main reasons for this situation. One is that it is difficult to train audio and video talents, and the other is that there are too few outstanding talents.
Technically speaking, audio and video practitioners are divided into two directions, one is algorithmic, and the other is engineering. There are very few such people who are engaged in audio and video algorithms, and they need deep mathematical ability and algorithm background. Relatively speaking, the cost of learning is high, and ordinary enterprises do not need it and cannot afford it. Such people usually choose to go to large companies, and the salary of one million is Low. These people generally only focus on one field, so the employment scope is also narrow.
There are relatively more people engaged in engineering. They have a solid theoretical foundation, strong technical skills, and are very familiar with audio/video. The salary of this kind of talent is more than 20% higher than the salary of other developers of the same level.
Therefore, Qian Jing, a developer engaged in audio and video technology, is still very optimistic. And as time goes by, audio and video talents will become more popular in the next few years.
Is audio and video easy to learn?
[External link image transfer failed. The source site may have an anti-leech link mechanism. It is recommended to save the image and upload it directly (img-Bzpb6SCB-1609828455810)(//upload-images.jianshu.io/upload_images/5956443-cc4430d9d8e9ad45.png?imageMogr2 /auto-orient/strip|imageView2/2/w/1200/format/webp)]
To be honest, self-study of audio and video is difficult, the learning cost is very high, and the efficiency is low. There are two main reasons. One is that the knowledge of audio and video is complex, and there are very few easy-to-understand materials; on the other hand, the Internet is full of a lot of wrong information, making many beginners unable to climb out of the pit.
Let me give you an example. According to the traditional audio and video learning method, you must first read a large number of audio and video specification/protocol documents (such as H264, MP4/FLV, RTP/RTCP, etc.) to learn audio and video. The content of these documents is basically in units of bits , that is, each binary bit change represents a different meaning. This kind of document is extremely boring and difficult to read.
Under the premise of familiarity with the documentation, the difficulty of code development will be "a higher level". At this time, you will find that if you are unfamiliar with C/C++ and have not been specifically trained, you will enter a hell-like situation, which is a torture.
Fortunately, there are now open source libraries such as FFMPG and WebRTC, which greatly reduces R&D costs. But even so, it is still very difficult for Xiaobai to learn audio and video.
How does Xiaobai learn audio and video?
[External link image transfer failed. The source site may have an anti-hotlinking mechanism. It is recommended to save the image and upload it directly (img-vIH850Bk-1609828455813)(//upload-images.jianshu.io/upload_images/5956443-60ea538b6913eabb.png?imageMogr2 /auto-orient/strip|imageView2/2/w/1024/format/webp)]
As mentioned above, the difficulty of getting started with audio and video is still quite large. How can we get started faster?
How to learn audio and video development
1. Elementary Introduction:
Draw a picture
1. ImageView draws pictures
2. SurfaceView draws pictures
3. Customize View to draw pictures
Second, AudioRecord API detailed explanation
3. Use AudioRecord to achieve recording and generate wav
- Create an AudioRecord object
- Initialize a buffer
- start recording
- Create a data stream. While reading the sound data from AudioRecord to the initialized buffer, import the data in the buffer into the data stream.
- Close data stream
- Stop recording
Fourth, use AudioTrack to play PCM audio
1. Basic use of AudioTrack
- MODE_STATIC mode
- MODE_STREAM mode
2.AudioTrack detailed
- Type of audio stream
- The concept of Buffer allocation and Frame
- AudioTrack construction process
3. Comparison of AudioTrack and MediaPlayer
- the difference
- contact
- SoundPool
5. Use Camera API to collect video data
1. Preview Camera data
2. Get the data callback of NV21
6. Use MediaExtractor and MediaMuxer API to parse and encapsulate mp4 files
1. Introduction to MediaExtractor API
2. Introduction to MediaMuxer API
3. Use context
7. Detailed MediaCodec API
1. Introduction to MediaCodec
2.MediaCodec API description
3.MediaCodec flow control
- Basic concepts of flow control
- Android hard-coded flow control
- Android flow control strategy selection
Due to the limited length of the article, the remaining content is too much, and the illustrations in the article are limited, the following can only be shown in the screenshot directory:
2. Intermediate advanced articles:
- Android OpenGL ES Development (1): Introduction to OpenGL ES
- Android OpenGL ES development (two): OpenGL ES environment construction
- Android OpenGL ES Development (3): OpenGL ES defines the shape
- Android OpenGL ES Development (4): OpenGL ES drawing shapes
- Android OpenGL ES development (5): OpenGL ES uses projection and camera view
- Android OpenGL ES Development (6): OpenGL ES adds motion effects
- Android OpenGL ES development (7): OpenGL ES responds to touch events
- Android OpenGL ES development (8): OpenGL ES shader language GLSL
- Android OpenGL ES development (9): OpenGL ES texture mapping
- Android OpenGL ES development (ten): interact with shaders through GLES20
- Use OpenGL to display a picture
- GLSurfaceviw draws Camera preview screen and realizes taking pictures
- Use OpenGL ES to complete the video recording and realize the video watermark effect
Advanced inquiry:
- In-depth study of audio and video coding, such as H.264, AAC, study the use of open source codec libraries, such as x.264, JM, etc.
- In-depth study of audio and video-related network protocols, such as rtmp, hls, and packet formats, such as flv, mp4
- In-depth study of some open source projects in the audio and video field, such as webrtc, ffmpeg, ijkplayer, librtmp, etc.
- Port the ffmpeg library to the Android platform, and combine the accumulated experience above to write a simple audio and video player
- Port the x264 library to the Android platform and combine the accumulated experience above to complete the H264 soft editing function of video data
- Port the librtmp library to the Android platform and combine the accumulated experience above to complete the Android RTMP streaming function
Audio and video codec technology
- Audio and video codec technology (1): MPEG-4/H.264 AVC codec standard
- Audio and video coding and decoding technology (two): AAC audio coding technology
Streaming protocol
- Streaming media protocol (1): HLS protocol
- Streaming media protocol (two): RTMP protocol
Multimedia file format
- Multimedia file format (1): MP4 format
- Multimedia file format (2): FLV format
- Multimedia file format (3): M3U8 format
- Multimedia file format (4): TS format
- Multimedia file format (5): PCM / WAV format
FFmpeg learning record
- FFmpeg command line tool learning (1): View the media file header information tool ffprobe
- FFmpeg command-line tool learning (2): ffplay, a tool for playing media files
- FFmpeg command line tool learning (3): media file conversion tool ffmpeg
- FFmpeg command line tool learning (4): FFmpeg acquisition equipment
- FFmpeg command line tool learning (5): FFmpeg adjusts the audio and video playback speed
- FFmpeg learning (1): Introduction to FFmpeg
- FFmpeg learning (2): install FFmpeg under Mac
- FFmpeg learning (3): port FFmpeg to the Android platform
- FFmpeg learning (4): FFmpeg API introduction and general API analysis
- FFmpeg learning (5): FFmpeg codec API analysis
- FFmpeg learning (6): FFmpeg core module libavformat and libavcodec analysis
- FFmpeg structure learning (1): AVFormatContext analysis
- FFmpeg structure learning (two): AVStream analysis
- FFmpeg structure learning (3): AVPacket analysis
- FFmpeg structure learning (four): AVFrame analysis
- FFmpeg structure learning (5): AVCodec analysis
- FFmpeg structure learning (6): AVCodecContext analysis
- FFmpeg structure learning (7): AVIOContext analysis
- FFmpeg structure learning (eight): the relationship between important structures in FFMPEG
More catalog screenshots:
**Summary:** The above is the strongest sorting and summary of the information about audio and video (there are some content without screenshots), I believe it should be the most complete in the entire network, all the information is shared with everyone for free, and save I went to the time for friends who are interested in audio and video to look for information. If you just need it, you can click here to get it for free
The prospects of audio and video development, I believe you can see from the relevant recruitment websites, whether it is streaming media development or audio and video processing development, a large number of people are recruited, and because the entry threshold is relatively high, the learning cycle is long, and the salary It is also high.
In addition: The editor also collects and organizes some audio and video related supporting videos, which I believe can make everyone learn faster and more effectively:
All the information in this article is shared with everyone for free, just click here with your fingers to get the free way.
The road to Android architects is very long, let's encourage together! The road of learning is like traveling against the current. If you don't advance, you will retreat. If you relax, then you will regress. There is no unique technology in Android development work, and it is impossible to get a high salary. Be a diligent and progressive person, step up your study, grasp the center, and prefer to be skilled than miscellaneous, rather than expert.
If you happen to need this full version of Android audio and video development, you only need to support me in this article.
Quick start channel: (click here) to download! Full of sincerity! ! !
It is not easy to organize, friends who feel helpful can help like, share and support the editor~
Your support, my motivation; I wish you all a bright future and constant offers! ! !