Video can easily be played at double speed. The general video format has a fixed number of frames per second, and skipping frames in proportion is enough. In audio, this method can also be used to directly delete some periods, because the audio in the computer is also stored digitally and discretely. However, in order to keep the sound from being distorted, a slightly more complex algorithm should be used, for example, the sound level signal in adjacent clock cycles is averaged, or the Gaussian average value is used to replace the original signal, and the finer point can be adaptively Set a relatively high weight in places where the tone signal is relatively rich to minimize compression and maintain the tone. In short, there are many ways to do it. Because I haven't paid attention to this, I don't know how it is implemented in the software, but the digital signal scaling and filtering algorithms should almost do the same, and the audio acceleration does not seem to need to use more complex nonlinear What adaptive filtering looks like.
The effect we need to achieve in many cases is to change the speed without changing the tuning. The project is based on FFMpeg and WebRtc. The video stream is read from the network through FFMpeg, and separated into audio data packets and video data packets after decapsulation and demultiplexing. The audio and video data packets are decoded respectively. After decoding, the audio PCM (44100Hz, 16bit, MONO) data is thrown to AudioTrack through the interface provided by WebRtc, and the video YUV420 data is thrown to WebRtc for rendering through VideoRenderer.
Detour:
1. Feed twice the data to the playback device (Audio Playout Device registered through WebRtc) at once: it can achieve double speed without tuning, the principle is unknown (the internal implementation mechanism of WebRtc), but there is stinging noise, it is speculated It is the problem of the pitch cycle, which will cause pitch breaks, difficult positioning, and difficult to realize by yourself, so the plan was abandoned.
2. Change the decoding rate to 22050Hz, and play it through WebRtc (the player is initialized to 44100Hz), which can be doubled, but it will change the tone and give up.
3. Dropping frames, dropping one frame every other frame, can achieve audio double speed, but there will also be stinging noise (problem of broken pitch), and the sound will be intermittent, the experience is very poor, give up.
Solution
Solving this pitch cycle problem by yourself requires algorithms and implementations, which is unrealistic and can only be handled by calling existing libraries. After investigation, it was found that there are two libraries that support double-speed processing, one is SoundTouch and the other is Sonic. Since Google officially provides an ExoPlayer player, the application method is Sonic, and there are articles on the Internet comparing the two libraries. The effect of Sonic is slightly better than that of SoundTouch, so I decided to use the Sonic library.
There are two implementations of the Sonic library, one is Sonic.java implemented in Java, and the other is Sonic-ndk implemented in C. Because we want to work with FFMpeg, we need to use ndk to develop. For teams without self-research conditions, choosing a third party is the best solution.
Take JiaoZiVideoPlayer as an example. The built-in playback engine is MediaPlayer, which is the player that comes with Android. There are many imperfections. Double-speed switching only supports 5.0 or above, otherwise NoClassDefFoundError will be reported. And it does not support rtmp-type playback streams.
Therefore, it is recommended not to use the default one on the playback engine. The more popular one is ijkplayer
But only using ijkplayer, you need to write logic, layout, etc. yourself. For simplicity, the ijk playback engine is directly used for JiaoZiVideoPlayer, saving the trouble of writing layout and playback logic.
example
First, introduce related jar package support,
implementation 'cn.jzvd:jiaozivideoplayer:6.2.7'
implementation 'tv.danmaku.ijk.media:ijkplayer-java:0.8.4'
implementation 'tv.danmaku.ijk.media:ijkplayer-armv7a:0.8.4'
implementation 'com.jwkj:M3U8Manger:v2.1.9'
For Android Studio 3.0 and above, it is recommended to use the implementation method to import third-party libraries, and compile below.
Customize player layout
The player control of JiaoZi is JZVideoPlayerStandard. All operations on the player layout control need to go through this control, which can meet the general video playback requirements. But if you need to add buttons in JiaoZi player, you need to customize JZVideoPlayerStandard, such as double-speed playback, download, resolution switching and other functions. If you don't need to modify the layout, you can use it directly in the xml layout file.
a. Rewrite XML
If you need to add new controls to the player, or change pictures, modify button positions, etc., you must copy the original XML to the new XML. It is recommended that the control name cannot be modified, just add the controls you need. . Set a custom layout file through the getLayoutId() method. For example:
@Override
public int getLayoutId() {
return
b. Add double-speed switching and download controls.
Initialize the controls in the init method.
video_speed = (TextView) findViewById(R.id.video_speed);
video_speed.setOnClickListener(this);
c. Monitoring
Note: JZVideoPlayerStandard only provides layout-related operations. Double-speed switching involves the acceleration of the engine, so temporarily use the broadcast method to notify the Activity to call the engine.
@Override
public void onClick(View v) { super.onClick(v); int i = v.getId(); if (i == R.id.video_speed) { // switch double speed video_speed.setText(resolveTypeUI(mFloat) + "X"); mFloat = resolveTypeUI(mFloat); EventBus.getDefault().post(new SpeedEvent(mFloat)); // Update the playback state onStatePreparingChangingUrl(0, getCurrentPositionWhenPlaying()); }else if (i == R .id.video_download) { // Download } } /*Display double speed ratio*/ public static float resolveTypeUI(float speed) { if (speed == 1) { speed = 1.25f; } else if (speed == 1.25f) { speed = 1.5f; } else if (speed == 1.5f) {
speed = 2f;
} else if (speed == 2f) {
speed = 1f;
}
return
d. Controlling the display and hiding of controls for the playback state
In actual requirements, if the controls need to be displayed only in the full-screen state, then this step is required.
JiaoZiVideoPlayer has a resolution switching control by default, so there is no need to repeatedly write related logic.
@Override
public void setUp(Object[] dataSourceObjects, int defaultUrlMapIndex, int screen, Object... objects) {
super.setUp(dataSourceObjects, defaultUrlMapIndex, screen, objects);
//如果是全屏才显示相关按钮
Log.e("data========:", dataSourceObjects.length+"");
if (currentScreen == SCREEN_WINDOW_FULLSCREEN) {
video_speed.setVisibility(VISIBLE);
video_download.setVisibility(VISIBLE);
} else if (currentScreen == SCREEN_WINDOW_NORMAL) {
video_speed.setVisibility(GONE);
video_download.setVisibility(GONE);
} else if
custom playback engine
If you want to achieve double-speed playback, both the built-in MediaPlayer and ijkPlayer must customize the playback engine, but the engines they inherit are different.
a. Provide a method of double-speed switching
//播放速度，默认1
public float speeding=1f;
public float getSpeeding() {
return speeding;
}
public void setSpeeding(float speeding) {
this.speeding = speeding;
}
Switch the double-speed ijkPlayer through the engine.
@Override
public void onPrepared(IMediaPlayer iMediaPlayer) { Log.e("speed========:", getSpeeding() + ""); ijkMediaPlayer.setSpeed(getSpeeding()); ijkMediaPlayer.start(); } MediaPlayer: @Override public void onPrepared(MediaPlayer mediaPlayer) { //Set double speed, not supported below 5.0, an exception will be thrown try { mediaPlayer.setPlaybackParams(new PlaybackParams().setSpeed(getSpeeding())); }catch
use case
a, set playback controls
<com.wapchief.qiniuplayer.views.myjzvideoplayerstandard android:layout_width="match_parent" android:layout_height="200dp"
android:id="@+id/jiaozi_player"
style="font-size: inherit;color: inherit;
line-height: inherit;"></com.wapchief.qiniuplayer.views.myjzvideoplayerstandard>
b. Initialize the playback address
If you don't need to switch the resolution, just replace the objects with the video URL link.
private String[] mediaName = {"普通","高清","原画"};
private void initPlayerUrl() {
Object[] objects = new Object[3];
LinkedHashMap map = new LinkedHashMap();
for (int i = 0; i < 3; i++) {
map.put(mediaName[i], MediaUrl.URL_M3U8);
}
objects[0] = map;
objects[1] = false;
objects[2] = new HashMap<>();
((HashMap) objects[2]).put("key", "value");
mPlayerStandard.setUp(objects, 0, JZVideoPlayer.SCREEN_WINDOW_NORMAL, "");
}
c, initialize the playback engine
log in to copy
//自定义 MediaPlayer
MyJZMediaSystem mJZMediaSystem = new MyJZMediaSystem();
//自定义 ijk
MyIJKMediaSystem mIJKMediaSystem = new MyIJKMediaSystem();
@Override
protected void onPause() {
super.onPause();
JZVideoPlayer.releaseAllVideos();
JZVideoPlayer.setMediaInterface(mIJKMediaSystem);
}
@Override
public void onBackPressed() {
if (JZVideoPlayer.backPress()) {
return;
}
super.onBackPressed();
}
d, Double-speed switching event
public void onMessageEventPostSpeed(SpeedEvent event) {
mJZMediaSystem.setSpeeding(event.getSpeed());
mIJKMediaSystem.setSpeeding(event.getSpeed());
Toast.makeText(this, "正在切换倍速:"+event.getSpeed(), Toast.LENGTH_LONG).show();
}