Table of contents
2. The minimum quality bottom line of video encoding
Access raw video byte buffer on older devices
Asynchronous processing using buffers
Synchronous processing using buffer arrays (deprecated)
Transformation when rendering to a surface
Seek and adaptive playback support
For codecs that don't support adaptive playback (including when not decoding to Surface)
For decoders that support and are configured for adaptive playback
Original source: MediaCodec | Android Developers
1. What is MediaCodec
The MediaCodec class provides access to low-level media codecs, the encoder/decoder components. It is part of Android's underlying multimedia support infrastructure (often used with MediaExtractor, MediaSync, MediaMuxer, MediaCrypto, MediaDrm, Image, Surface, and AudioTrack.)
Broadly speaking, a codec processes input data to generate output data. It processes data asynchronously and uses a set of input and output buffers. On a simple level, you request (or receive) an empty input buffer, fill it with data and send it to the codec for processing. The codec ran out of data and turned it into one of the empty output buffers. Finally, you request (or receive) a filled output buffer, consume its contents and release it back to the codec.
2. The minimum quality bottom line of video encoding
Starting with Build.VERSION_CODES.S, Android's Video MediaCodecs enforce minimum quality standards. The purpose is to eliminate poor quality video encoding. This quality baseline is enforced when the codec is in variable bitrate (VBR) mode; it does not apply when the codec is in constant bitrate (CBR) mode. Quality baseline enforcement is also limited to a specific size range; this size range is currently available for video resolutions greater than 320x240 to 1920x1080.
When this quality floor is in effect, the codec and supporting framework code will ensure that the resulting video is at least "fair" or "good" quality. The metric used to select these targets is VMAF (Video Multi-Method Assessment Function), with a target score of 70 for the selected test sequence.
A typical effect is that some videos generate a higher bitrate than the original configuration. This is most noticeable for video configured at very low bitrates; the codec will use the bitrate determined to be more likely to produce "fair" or "good" quality video. Another situation is that the video contains very complex content (lots of motion and detail); in this configuration, the codec will use extra bitrate as needed to avoid losing all the finer details of the content.
This quality floor doesn't affect content captured at high bitrates (which should already give the codec enough capacity to encode all the details). Quality Bottom does not operate on CBR encoding. Quality Baseline is currently not available for resolutions 320x240 or lower, nor for videos higher than 1920x1080.
3. Data type
Codecs operate on three types of data: compressed data, raw audio data, and raw video data. All three types of data can be processed using ByteBuffers, but you should use Surfaces for raw video data to improve codec performance. Surface uses native video buffers without mapping or copying them into ByteBuffers; therefore, it is much more efficient. When using a Surface, you generally do not have access to raw video data, but you can use the ImageReader class to access unsafe decoded (raw) video frames. This may still be more efficient than using ByteBuffers, since some native buffers may map to ByteBuffers directly. When using ByteBuffer mode, you can use the Image class and getInput/OutputImage(int) to access raw video frames.
compressed buffer
Input buffers (for decoders) and output buffers (for encoders) contain compressed data depending on the type of format. For video types, this is usually a single frame of compressed video. For audio data, this is usually a single access unit (a segment of encoded audio, usually containing a few milliseconds of audio as dictated by the format type), but this requirement is slightly relaxed since a buffer may contain multiple access units of encoded audio . In either case, buffers do not start or end on arbitrary byte boundaries, but on frame/access unit boundaries, unless they are flagged as BUFFER_FLAG_PARTIAL_FRAME.
raw audio buffer
Raw audio buffers contain complete frames of PCM audio data, in channel order with one sample per channel. Each PCM audio sample is a 16-bit signed integer or a floating point number, in native byte order. Raw audio buffers in floating-point PCM encoding are only possible if MediaFormat's MediaFormat#KEY_PCM_FLOAT is set to AudioFormat#ENCODING_PCM_FLOAT during MediaCodec configure(…) and confirmed by decoder's getOutputFormat() or encoder's getInputFormat(). An example method to check for floating PCM in MediaFormat is as follows:
static boolean isPcmFloat(MediaFormat format) {
return format.getInteger(MediaFormat.KEY_PCM_ENCODING, AudioFormat.ENCODING_PCM_16BIT)
== AudioFormat.ENCODING_PCM_FLOAT;
}In order to extract one channel of a buffer containing 16-bit signed integer audio data in a short array, the following code can be used:
// Assumes the buffer PCM encoding is 16 bit.
short[] getSamplesForChannel(MediaCodec codec, int bufferId, int channelIx) {
ByteBuffer outputBuffer = codec.getOutputBuffer(bufferId);
MediaFormat format = codec.getOutputFormat(bufferId);
ShortBuffer samples = outputBuffer.order(ByteOrder.nativeOrder()).asShortBuffer();
int numChannels = format.getInteger(MediaFormat.KEY_CHANNEL_COUNT);
if (channelIx < 0 || channelIx >= numChannels) {
return null;
}
short[] res = new short[samples.remaining() / numChannels];
for (int i = 0; i < res.length; ++i) {
res[i] = samples.get(i * numChannels + channelIx);
}
return res;
}raw video buffer
In ByteBuffer mode, video buffers are laid out according to their color format. You can get the supported color formats as an array from getCodecInfo().getCapabilitiesForType(...).colorFormats. Video codecs may support three color formats:
- Native Raw Video Format: This is marked by CodecCapabilities#COLOR_FormatSurface, which can be used with an input or output Surface.
- Flexible YUV buffers (eg CodecCapabilities#COLOR_FormatYUV420Flexible): they can be used with input/output Surface and ByteBuffer modes by using getInput/OutputImage(int).
- Other specific formats: These are generally only supported in ByteBuffer mode. Some color formats are vendor-specific. Others are defined in CodecCapabilities. For color formats equivalent to flexible formats, you can still use getInput/OutputImage(int).
All video codecs since Build.VERSION_CODES.LOLLIPOP_MR1 support a flexible YUV 4:2:0 buffer.
Access raw video byte buffer on older devices
Before Build.VERSION_CODES.LOLLIPOP and Image support, you needed to use the MediaFormat#KEY_STRIDE and MediaFormat#KEY_SLICE_HEIGHT output format values to know the layout of the raw output buffer.
Note that on some devices the tile height is recommended to be set to 0. This could mean that the slice height is the same as the frame height, or that the slice height is the frame height aligned to some value (usually 2). Unfortunately, there is no standard and easy way to tell the actual slice height in this case. Also, the vertical stride of the U-plane in planar format is not specified or defined, although it is usually half the slice height.
The MediaFormat#KEY_WIDTH and MediaFormat#KEY_HEIGHT Keys specify the size of the video frame; however, for most encodings, the video (picture) only occupies a portion of the video frame. This is represented by a "cropping rectangle".
You need to use the following Keys to get the crop rectangle of the original output image from the output format. If these keys do not exist, the video occupies the entire video frame. The crop rectangle is understood in the context of the output frame, before any rotation is applied.
| format keys |
type |
illustrate |
| "crop-left" |
Integer |
The left coordinate (x) of the clipping rectangle |
| "crop-top" |
Integer |
The top coordinate (y) of the clipping rectangle |
| "crop-right" |
Integer |
The right coordinate of the clipping rectangle (x) MINUS 1 |
| "crop-right" |
Integer |
The bottom coordinate of the clipping rectangle (y) MINUS 1 |
The lower right coordinate can be understood as the coordinate of the rightmost valid column/bottommost valid row of the cropped output image.
The size of the video frame (before rotation) can be calculated like this:
MediaFormat format = decoder.getOutputFormat(…);
int width = format.getInteger(MediaFormat.KEY_WIDTH);
if (format.containsKey("crop-left") && format.containsKey("crop-right")) {
width = format.getInteger("crop-right") + 1 - format.getInteger("crop-left");
}
int height = format.getInteger(MediaFormat.KEY_HEIGHT);
if (format.containsKey("crop-top") && format.containsKey("crop-bottom")) {
height = format.getInteger("crop-bottom") + 1 - format.getInteger("crop-top");
}Also note that the meaning of BufferInfo.offset is not consistent across devices. On some devices, the offset points to the top-left pixel of the crop rectangle, while on most devices it points to the top-left pixel of the entire frame.
4. Status
During its lifetime, a codec is conceptually in one of three states: Stopped, Executing, or Released. The Stopped state is actually a collection of three states: Uninitialized, Configured, and Error, while the Executing state is conceptually divided into three sub-states: Flushed, Running, and End-of-Stream.
When you create a codec using one of the factory methods, the codec is in an uninitialized state. First, you need to configure it via configure(...) to get it into the Configured state, then call start() to move it to the Executing state. In this state, you can process data through the buffer queue operations described above.
The Executing state has three sub-states: Flushed, Running, and End-of-Stream. Immediately after start(), the codec is in the Flushed substate, which holds all buffers. Once the first input buffer is dequeued, the codec moves to the running substate, where it spends most of its time. When you enqueue an input buffer with an end-of-stream marker, the codec transitions to the end-of-stream substate. In this state, the codec accepts no more input buffers, but still produces output buffers until the end of stream is reached on output. You can use flush() while in the Executing state to move back to the Flushed substate at any time.
Calling stop() returns the codec to an uninitialized state, after which it can be configured again. When you are done using the codec, you must release it by calling release().
In rare cases, the codec may encounter an error and enter an error state. This is communicated using invalid return values from queued operations or sometimes through exceptions. Call reset() to make the codec usable again. You can call it from any state to move the codec back to an uninitialized state. Otherwise, call release() to move to the terminal Released state.
5. Create
Create a MediaCodec for a specific MediaFormat using a MediaCodecList. When decoding a file or stream, you can get the desired format from MediaExtractor.getTrackFormat. Use MediaFormat.setFeatureEnabled to inject any specific features you want to add, then call MediaCodecList.findDecoderForFormat to get the name of a codec that can handle that specific media format. Finally, the codec is created using createByCodecName(String).
Note: On Build.VERSION_CODES.LOLLIPOP, the format of MediaCodecList.findDecoder/EncoderForFormat must not include the frame rate. Use format.setString(MediaFormat.KEY_FRAME_RATE, null) to clear any existing frame rate setting in the format.
You can also create a preferred codec for a specific MIME type using createDecoder/EncoderByType(java.lang.String). However, this cannot be used to inject functions, and may create codecs that cannot handle the specific desired media format.
Create a secure decoder
On Build.VERSION_CODES.KITKAT_WATCH and earlier, secure codecs may not be listed in the MediaCodecList, but may still be available on the system. Existing secure codecs can only be instantiated by name, by appending ".secure" to the name of a regular codec (all secure codec names must end with ".secure"). createByCodecName(String) will throw an IOException if the codec does not exist on the system.
Starting with Build.VERSION_CODES.LOLLIPOP you should use the CodecCapabilities#FEATURE_SecurePlayback feature in the media format to create a secure decoder.
6. Initialization
After creating a codec, if you want to process data asynchronously, you can set a callback with setCallback. Then, configure the codec with a specific media format. This is when you can specify an output Surface for the video producer -- the codec (such as a video codec) that produced the raw video data. This is also when you can set decryption parameters for secure codecs (see MediaCrypto). Finally, since some codecs can operate in more than one mode, you must specify whether you want it to work as a decoder or an encoder.
Due to Build.VERSION_CODES.LOLLIPOP, the generated input and output formats can be queried in the Configured state. You can use this to validate generated configuration, such as color formats, before starting a codec.
If you want to use a video consumer (a codec that handles raw video input, such as a video encoder) to natively process raw input video buffers, use createInputSurface() after configuration to create a target Surface for your input data. Alternatively, set the codec to use a previously created persistent input surface by calling setInputSurface(Surface).
codec-specific data
Some formats, notably AAC audio and MPEG4, H.264, and H.265 video formats, require that the actual data be prefixed with multiple buffers containing setup data or codec-specific data. When dealing with such compressed formats, this data must be submitted to the codec after start() and before any frame data. Such data must be flagged with the flag BUFFER_FLAG_CODEC_CONFIG when calling queueInputBuffer.
Codec-specific data can also be included in the format passed to the configuration in the ByteBuffer entry, with keys "csd-0", "csd-1", etc. These keys are always included in the track's MediaFormat obtained from MediaExtractor. Codec-specific data in the format is automatically committed to the codec on start(); you must not explicitly commit this data. If the format does not contain codec specific data, you can choose to submit it in the correct order using a specified number of buffers, as required by the format. For H.264 AVC, you can also concatenate all codec-specific data and submit it as a single codec configuration buffer.
Android uses the following codec-specific data buffers. These also need to be set in the track format for proper MediaMuxer track configuration. Each parameter set and codec specific data section marked with (*) must start with a start code of "\x00\x00\x00\x01".
| Format |
CSD buffer #0 |
CSD Buffer #1 |
CSD Buffer #2 |
| AAC |
Decoder specific information from ESDS* |
never used |
never used |
| Vobbis |
ID header |
set title |
never used |
| OPUS |
ID header |
Preskip in nanoseconds integer. ) |
Search for preroll in nanoseconds integer. ) |
| FLAC |
"fLaC", a FLAC stream marker in ASCII format, |
never used |
never used |
| MPEG-4 |
Decoder specific information from ESDS* |
never used |
never used |
| H.264 AVC |
SPS (Sequence Parameter Set*) |
PPS (Picture Parameter Set*) |
never used |
| H.265 HEVC |
VPS (Video Parameter Set*) + |
never used |
never used |
| VP9 |
VP9 codec private data (optional) |
never used |
never used |
Note: Care must be taken if the codec is flushed immediately before any output buffers or output format changes have returned, or shortly after startup, as codec specific data may be lost during the flush. You must resubmit data with buffers marked BUFFER_FLAG_CODEC_CONFIG after such flushes to ensure correct codec operation.
An encoder (or a codec that produces compressed data) will create and return codec specific data before any valid output buffers in output buffers marked with the codec-config flag . Buffers containing codec-specific data do not have meaningful timestamps.
7. Data processing
Each codec maintains a set of input and output buffers referenced by buffer IDs in API calls. After a successful call to start() the client neither "owns" the input buffer nor the output buffer. In synchronous mode, call dequeueInput / OutputBuffer(...) to obtain (take ownership of) an input or output buffer from the codec. In asynchronous mode, you will automatically receive available buffers via MediaCodec.Callback.onInput / OutputBufferAvailable(…) callbacks.
Once you have the input buffer, fill it with data and submit it to the codec using queueInputBuffer - or queueSecureInputBuffer if using decryption. Do not submit multiple input buffers with the same timestamp (unless it is codec-specific data marked as such).
The codec in turn will return a read-only output buffer via the onOutputBufferAvailable callback in asynchronous mode, or respond to a call to dequeueOutputBuffer in synchronous mode. After processing the output buffer, call one of the releaseOutputBuffer methods to return the buffer to the codec.
While you don't need to resubmit/release buffers to the codec immediately, holding on to input and/or output buffers may stall the codec, and this behavior is device dependent. Specifically, a codec may defer generating output buffers until all outstanding buffers have been freed/resubmitted. So try to keep as few free buffers as possible.
Depending on the API version, you can handle data in three ways:
| Processing methods |
API version <= 20 |
API version >= 21 |
| Synchronous API using buffer arrays |
supported |
deprecated |
| Synchronous API using buffers |
not available |
supported |
| Asynchronous API using buffers |
not available |
supported |
Asynchronous processing using buffers
Because of Build.VERSION_CODES.LOLLIPOP , the preferred method is to process data asynchronously by setting a callback before calling configure . Asynchronous mode changes state transitions slightly, as you must call start() after flush() to transition the codec into the Running substate and start receiving input buffers. Similarly, on the initial call to the codec, start will move directly to the Running substate and begin passing available input buffers through callbacks.
MediaCodec is usually used like this in asynchronous mode:
MediaCodec codec = MediaCodec.createByCodecName(name);
MediaFormat mOutputFormat; // member variable
codec.setCallback(new MediaCodec.Callback() {
@Override
void onInputBufferAvailable(MediaCodec mc, int inputBufferId) {
ByteBuffer inputBuffer = codec.getInputBuffer(inputBufferId);
// fill inputBuffer with valid data
…
codec.queueInputBuffer(inputBufferId, …);
}
@Override
void onOutputBufferAvailable(MediaCodec mc, int outputBufferId, …) {
ByteBuffer outputBuffer = codec.getOutputBuffer(outputBufferId);
MediaFormat bufferFormat = codec.getOutputFormat(outputBufferId); // option A
// bufferFormat is equivalent to mOutputFormat
// outputBuffer is ready to be processed or rendered.
…
codec.releaseOutputBuffer(outputBufferId, …);
}
@Override
void onOutputFormatChanged(MediaCodec mc, MediaFormat format) {
// Subsequent data will conform to new format.
// Can ignore if using getOutputFormat(outputBufferId)
mOutputFormat = format; // option B
}
@Override
void onError(…) {
…
}
});
codec.configure(format, …);
mOutputFormat = codec.getOutputFormat(); // option B
codec.start();
// wait for processing to complete
codec.stop();
codec.release();Synchronization using buffers
Because of Build.VERSION_CODES.LOLLIPOP , even when using the codec in synchronous mode, you should retrieve the input and output buffers using getInput / OutputBuffer(int) and/or getInput /OutputImage (int) . This allows the framework to make certain optimizations, for example when dealing with dynamic content. This optimization is disabled if you call getInput / OutputBuffers() .
Note: Do not mix buffer and buffer array methods at the same time. Specifically, call getInput/ only after or directly after dequeuing an output buffer ID with a value of . OutputBuffers start() INFO_OUTPUT_FORMAT_CHANGED
MediaCodec is usually used like this in synchronous mode:
MediaCodec codec = MediaCodec.createByCodecName(name);
codec.configure(format, …);
MediaFormat outputFormat = codec.getOutputFormat(); // option B
codec.start();
for (;;) {
int inputBufferId = codec.dequeueInputBuffer(timeoutUs);
if (inputBufferId >= 0) {
ByteBuffer inputBuffer = codec.getInputBuffer(…);
// fill inputBuffer with valid data
…
codec.queueInputBuffer(inputBufferId, …);
}
int outputBufferId = codec.dequeueOutputBuffer(…);
if (outputBufferId >= 0) {
ByteBuffer outputBuffer = codec.getOutputBuffer(outputBufferId);
MediaFormat bufferFormat = codec.getOutputFormat(outputBufferId); // option A
// bufferFormat is identical to outputFormat
// outputBuffer is ready to be processed or rendered.
…
codec.releaseOutputBuffer(outputBufferId, …);
} else if (outputBufferId == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
// Subsequent data will conform to new format.
// Can ignore if using getOutputFormat(outputBufferId)
outputFormat = codec.getOutputFormat(); // option B
}
}
codec.stop();
codec.release();Synchronous processing using buffer arrays (deprecated)
In version Build.VERSION_CODES.KITKAT_WATCH and earlier, sets of input and output buffers were represented by ByteBuffer[] arrays. After a successful call to start() , the array of buffers is retrieved using getInput / OutputBuffers() . Use buffer ID-s as indices into these arrays (when non-negative), as in the example below. Note that there is no intrinsic correlation between array size and the number of input and output buffers used by the system, although array size provides an upper bound.
MediaCodec codec = MediaCodec.createByCodecName(name);
codec.configure(format, …);
codec.start();
ByteBuffer[] inputBuffers = codec.getInputBuffers();
ByteBuffer[] outputBuffers = codec.getOutputBuffers();
for (;;) {
int inputBufferId = codec.dequeueInputBuffer(…);
if (inputBufferId >= 0) {
// fill inputBuffers[inputBufferId] with valid data
…
codec.queueInputBuffer(inputBufferId, …);
}
int outputBufferId = codec.dequeueOutputBuffer(…);
if (outputBufferId >= 0) {
// outputBuffers[outputBufferId] is ready to be processed or rendered.
…
codec.releaseOutputBuffer(outputBufferId, …);
} else if (outputBufferId == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
outputBuffers = codec.getOutputBuffers();
} else if (outputBufferId == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
// Subsequent data will conform to new format.
MediaFormat format = codec.getOutputFormat();
}
}
codec.stop();
codec.release();stream end processing
When you reach the end of the input data, you have to send it to the codec queueInputBuffer by specifying the flag BUFFER_FLAG_END_OF_STREAM in the call . You can do this on the last valid input buffer, or by submitting an additional empty input buffer with the end-of-stream flag set. If an empty buffer is used, the timestamp will be ignored.
The codec will continue to return output buffers until it finally signals the end of the output stream by specifying the same end-stream flag in BufferInfo set to dequeueOutputBuffer or via return onOutputBufferAvailable . This can be set on the last valid output buffer, or on an empty buffer after the last valid output buffer. Timestamps for such empty buffers should be ignored.
Do not commit additional input buffers after signaling the end of the input stream unless the codec has been flushed or stopped and restarted.
Use the output Surface
Using an output Surface , data handling is almost the same as in ByteBuffer mode; however, the output buffer will not be accessible and will be represented as a null value. For example getOutputBuffer / Image(int) returns null and getOutputBuffers() will return an array containing only null-s.
When using an output Surface , you can choose whether to render each output buffer on the Surface . You have three options:
- Do not render the buffer: call releaseOutputBuffer(bufferId, false) .
- Render the buffer with the default timestamp: Call releaseOutputBuffer(bufferId, true) .
- Render a buffer with a specific timestamp: Call releaseOutputBuffer(bufferId, timestamp) .
Due to Build.VERSION_CODES.M , the default timestamp is the buffer's render timestamp (converted to nanoseconds). Before that was not defined.
In addition to Build.VERSION_CODES.M , you can use setOutputSurface .
When rendering output to a Surface , the Surface may be configured to drop too many frames (frames that are not consumed by the Surface in a timely manner). Or it can be configured not to drop too many frames. In the latter mode, if the Surface does not consume output frames fast enough, it will eventually block the decoder.
Until Build.VERSION_CODES.Q the exact behavior is undefined, except that the View surface (SurfaceView or TextureView) always drops too many frames. Due to Build.VERSION_CODES.Q the default behavior is to drop too many frames. Applications can opt-in to this behavior for non-View surfaces (such as ImageReaders or surface textures) by targeting the SDK out of Build.VERSION_CODES.Q , setting the key MediaFormat#KEY_ALLOW_FRAME_DROP to 0 in their configured format.
Transformation when rendering to a surface
If the codec is configured in Surface mode, any crop rectangle, rotation , and video scaling modes will be applied automatically , with one exception:
Before the release of Build.VERSION_CODES.M , software codecs may not have rotation applied when rendering to a Surface. Unfortunately, there's no standard and easy way to identify software codecs, or whether they apply rotation other than trying.
There are also some caveats.
Note that the pixel aspect ratio is not considered when displaying the output on the Surface. This means that if you are using VIDEO_SCALING_MODE_SCALE_TO_FIT mode, you must position the output Surface so that it has the correct final display aspect ratio. Instead, you can only use VIDEO_SCALING_MODE_SCALE_TO_FIT_WITH_CROPPING mode for content with square pixels (pixel aspect ratio or 1:1).
Please also note that VIDEO_SCALING_MODE_SCALE_TO_FIT_WITH_CROPPING mode may not work properly for videos rotated 90 or 270 degrees at the time of Build.VERSION_CODES.N release .
When setting the video scaling mode, be aware that it must be reset after each output buffer change. Since the INFO_OUTPUT_BUFFERS_CHANGED event is deprecated, you can do this after every output format change.
Use the input Surface
When using an input Surface, there is no accessible input buffer because the buffer is automatically passed from the input Surface to the codec. Calling dequeueInputBuffer will throw an IllegalStateException, and getInputBuffers() returns a bogus ByteBuffer[] array that cannot be written.
Call signalEndOfInputStream() to signal the end of the stream. The input Surface will stop submitting data to the codec immediately after this call.
Seek and adaptive playback support
Video decoders (and generally codecs that use compressed video data) behave differently with respect to seeks and format changes, whether or not they support and are configured for adaptive playback. You can check whether the codec supports adaptive playback through CodecCapabilities.isFeatureSupported(String) . Adaptive playback support for video codecs is only activated when you configure the codec to decode to the Surface.
Flow Boundaries and Keyframes
Input data that starts() or flush() after the appropriate stream boundary is important: the first frame must be a keyframe. A keyframe can be decoded by itself (for most codecs, this keyframe is an I-frame), and no other frame is displayed before the referenced keyframe.
The table below summarizes the keyframes available for various video formats.
| Format |
proper keyframe |
| VP9/VP8 |
A proper intraframe where no subsequent frame references a frame preceding it. |
| H.265 HEVC |
IDR or CRA |
| H.264 AVC |
IDR |
| MPEG-4 |
A proper I-frame in which no subsequent frame references a frame preceding it. |
For codecs that don't support adaptive playback (including when not decoding to Surface)
In order to start decoding data that is not adjacent to previously committed data (i.e. after a seek), you have to refresh the decoder. Since all output buffers are immediately flushed when flushed, you may want to signal and then wait for the end of the stream before calling flush. It is important that the refreshed input data starts at a suitable flow boundary/keyframe.
注意： flush后提交的数据格式必须不能改变；flush()不支持不连的格式；为此，一个完整的stop()- configure(…)-start()生命周期是必要的。
另外请注意：如果start()后过早刷新编解码器，通常在收到第一个输出缓冲区或输出格式更改之前，您将需要重新提交编解码特定数据到编解码器。有关详细信息，请参阅编解码特定数据部分。
对于支持并配置为自适应播放的解码器
为了开始解码与先前提交的数据不相邻的数据（即在查找之后），刷新解码器不是必要的；但是，中断后的输入数据必须从合适的流边界/关键帧开始。
对于某些视频格式 - 即 H.264、H.265、VP8 和 VP9 - 还可以在中途更改图片大小或配置。为此，您必须将整个新的特定于编解码器的配置数据与关键帧一起打包到单个缓冲区（包括任何起始代码）中，并将其作为常规输入缓冲区提交。
您将在图片大小更改发生之后和具有新大小的任何帧返回之前收到INFO_OUTPUT_FORMAT_CHANGED来自dequeueOutputBuffer或onOutputFormatChanged回调的返回值。
注意：就像编解码器特定数据的情况一样，在调用flush()方法来改图片大小要小心 。如果您还没有收到更改图片大小的确认，您将需要重新请求新的图片大小。
错误处理
您必须捕获或声明放弃工厂方法createByCodecName和createDecoder/EncoderByType引发IOException失败。MediaCodec 会抛出IllegalStateException，当从不允许它的编解码器状态调用方法时；这通常是由于不正确的应用程序 API 使用造成的。涉及安全缓冲区的方法可能会抛出 CryptoException，其中包含可从CryptoException#getErrorCode.
内部编解码器错误会导致CodecException，这可能是由于媒体内容损坏、硬件故障、资源耗尽等，即使应用程序正确使用 API。当抛出CodecException 时，调用CodecException#isRecoverable和CodecException#isTransient来确定推荐的操作：
- 可恢复的错误：如果isRecoverable()返回true，则调用 stop()，configure(…)以及start()恢复。
- 瞬态错误：如果isTransient()返回 true，则资源暂时不可用，该方法可能会稍后重试。
- Fatal error: If both isRecoverable() and isTransient() return false, the CodecException is fatal and the codec must be reset or released .
Neither isRecoverable() nor isTransient() return true at the same time.