Original link: https://www.jianshu.com/p/5966ed7c5baf
Introduction of audio capture, encoding, generates a file, transcoding operations, by collecting AudioRecord audio file format created three formats (pcm, wav, aac), with AudioStack to play the audio.
A, PCM, WAV, AAC presentation file header
Here a brief look at three common audio formats:
PCM: PCM (Pulse Code Modulation-- recording pulse code modulation). PCM recording is called an analog signal, the sound becomes pulse train of symbols, the use of three parameters (the number of channels, sampling frequency, and sampling bit) to represent a sound. PCM signal is coded, and it has not undergone any compression processing. Than the analog signal, it is less susceptible to noise and distorted by the transmission system. Wide dynamic range, a very good impact sound effect.
WAV: WAV is a lossless audio file format, WAV meet the RIFF (Resource Interchange File Format) specification. All have a WAV file header, the encoding parameter file header audio stream. WAV is no hard and fast rules encoded audio stream, in addition to PCM, as well as almost all ACM supports standard encoding that can encode WAV audio stream.
In simple terms: WAV is a lossless audio file format, PCM encoding is not compressed
AAC: AAC (Advanced Audio Coding), Chinese called "Advanced Audio Coding", appeared in 1997, the audio coding technology based on the MPEG-2. Co-developed by Fraunhofer IIS, Dolby Laboratories, AT & T, Sony (Sony) and other companies, it is intended to replace the MP3 format. In 2000, the MPEG-4 standard appears, AAC reintegrated its properties, added to the SBR and PS techniques technology, in order to distinguish the conventional MPEG-2 AAC, also known as MPEG-4 AAC. He is a designed specifically for audio data file compression format similar to MP3. Using the AAC format sound files can significantly reduce, but will not make people feel the sound quality is reduced.
Second, the use AudioRecord implement PCM audio files generated
AudioRecord Android system is provided for implementing functions like recording, in order to understand and explain the specific use of this class, you can see what the official documents, such as reference links.
AndioRecord class main function is to allow Java applications to manage a variety of audio resources, so that they can record sounds through such sound-related hardware collected. This functionality is achieved through sound data "pulling" (read) AudioRecord object to complete. In the recording process, the application needs to do is to acquire the next three AudioRecord class object by a method of recording data in a timely manner AudioRecord three classes provide a method of obtaining sound data are:
- read(byte[], int, int)
- read(short[], int, int)
- read(ByteBuffer, int)
Whether you choose to use a method that must be pre-set to store user-friendly format of sound data.
Start recording time, AudioRecord need to initialize a sound associated buffer, the buffer is mainly used to save the new sound data. The size of this buffer, we can go to the designated during object construction. It shows a AudioRecord object has not been read (synchronous) before the sound data can be recorded how long the tone (ie one can record the sound of capacity). Sound data is read out from the audio hardware, the data size does not exceed the size of the whole recording data (a plurality of times can be read), i.e., the data capacity of each read buffer initialization.
2.1 must first declare some global variables and constants parameters
Used primarily declare some parameters, specific explanation can see comments.
/指定音频源 这个和MediaRecorder是相同的 MediaRecorder.AudioSource.MIC指的是麦克风
private static final int mAudioSource = MediaRecorder.AudioSource.MIC;
//指定采样率 （MediaRecoder 的采样率通常是8000Hz，16000Hz
//AAC的通常是 44100Hz。 设置采样率为 44100，目前为常用的采样率，官方文档表示这个值可以兼容所有的设置）
private static final int mSampleRateInHz = 44100;
//指定捕获音频的声道数目。在 AudioFormat 类中指定用于此的常量，单声道
private static final int mChannelConfig = AudioFormat.CHANNEL_CONFIGURATION_MONO;
//指定音频量化位数 ,在 AudioFormat 类中指定了以下各种可能的常量。通常我们选择 ENCODING_PCM_16BIT 和 ENCODING_PCM_8BIT
//PCM 代表的是脉冲编码调制，它实际上是原始音频样本。
//因此可以设置每个样本的分辨率为 16 位或者8位，16 位将占用更多的空间和处理能力,表示的音频也更加接近真实。
private static final int mAudioFormat = AudioFormat.ENCODING_PCM_16BIT;
//指定缓冲区大小。调用AudioRecord类的getMinBufferSize方法可以获得。
private int mBufferSizeInBytes;
// 声明 AudioRecord 对象
private AudioRecord mAudioRecord = null;
2.2 acquiring buffer size and create AudioRecord
//初始化数据，计算最小缓冲区
mBufferSizeInBytes = AudioRecord.getMinBufferSize(mSampleRateInHz, mChannelConfig, mAudioFormat);
//创建AudioRecorder对象
mAudioRecord = new AudioRecord(mAudioSource, mSampleRateInHz, mChannelConfig,
mAudioFormat, mBufferSizeInBytes);
2.3 write file
@Override
public void run() {
//标记为开始采集状态
isRecording = true;
//创建文件
createFile();
try {
//判断AudioRecord未初始化，停止录音的时候释放了，状态就为STATE_UNINITIALIZED
if (mAudioRecord.getState() == mAudioRecord.STATE_UNINITIALIZED) {
initData();
}
//最小缓冲区
byte[] buffer = new byte[mBufferSizeInBytes];
//获取到文件的数据流
mDataOutputStream = new DataOutputStream(new BufferedOutputStream(new FileOutputStream(mRecordingFile)));
//开始录音
mAudioRecord.startRecording();
//getRecordingState获取当前AudioReroding是否正在采集数据的状态
while (isRecording && mAudioRecord.getRecordingState() == AudioRecord.RECORDSTATE_RECORDING) {
int bufferReadResult = mAudioRecord.read(buffer, 0, mBufferSizeInBytes);
for (int i = 0; i < bufferReadResult; i++) {
mDataOutputStream.write(buffer[i]);
}
}
} catch (Exception e) {
Log.e(TAG, "Recording Failed");
} finally {
// 停止录音
stopRecord();
IOUtil.close(mDataOutputStream);
}
}
2.4 Access Request
Permission requirements: WRITE_EXTERNAL_STORAGE, READ_EXTERNAL_STORAGE (part of the mobile phone must apply for this privilege), RECORD_AUDIO
<uses-permission android:name="android.permission.RECORD_AUDIO"/>
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>
2.5 Acquisition Summary
And now the basic recording process on the introduction is over, but this time the problem:
- I follow the process, the audio data are output to a file inside, and after you stop recording, open the file and found not play, in the end is why?
According to the process has completed, the data is entered, but now the file contents inside just the most original audio data, termed RAW (Chinese explanation is that the "raw materials" or "something untreated"), this time, you let the player to open, it did not know what formats are saved, do not know how to decode operation. Of course, not play.
- Then how can I play content recorded it in the player?
In the beginning of the data file is added or AAC AAC HEAD data can be, that is, the file header. Only together with the head of the data file, the player to the right to know what is inside the content in the end is thus able to properly parse and play the contents inside.
Three, PCM converted into WAV
In the beginning of the data file to join WAVE HEAD or AAC data can be, that is, the file header. Only together with the head of the data file, the player to the right to know what is inside the content in the end is thus able to properly parse and play the contents inside. Specific header files described in the Play a WAV file on an AudioTrack which can be understood.
public class WAVUtil {
/**
* PCM文件转WAV文件
*
* @param inPcmFilePath 输入PCM文件路径
* @param outWavFilePath 输出WAV文件路径
* @param sampleRate 采样率，例如44100
* @param channels 声道数 单声道：1或双声道：2
* @param bitNum 采样位数，8或16
*/
public static void convertPcm2Wav(String inPcmFilePath, String outWavFilePath, int sampleRate,int channels, int bitNum) {
FileInputStream in = null;
FileOutputStream out = null;
byte[] data = new byte[1024];
try {
//采样字节byte率
long byteRate = sampleRate * channels * bitNum / 8;
in = new FileInputStream(inPcmFilePath);
out = new FileOutputStream(outWavFilePath);
//PCM文件大小
long totalAudioLen = in.getChannel().size();
//总大小，由于不包括RIFF和WAV，所以是44 - 8 = 36，在加上PCM文件大小
long totalDataLen = totalAudioLen + 36;
writeWaveFileHeader(out, totalAudioLen, totalDataLen, sampleRate, channels, byteRate);
int length = 0;
while ((length = in.read(data)) > 0) {
out.write(data, 0, length);
}
} catch (Exception e) {
e.printStackTrace();
} finally {
IOUtil.close(in,out);
}
}
/**
* 输出WAV文件
*
* @param out WAV输出文件流
* @param totalAudioLen 整个音频PCM数据大小
* @param totalDataLen 整个数据大小
* @param sampleRate 采样率
* @param channels 声道数
* @param byteRate 采样字节byte率
* @throws IOException
*/
private static void writeWaveFileHeader(FileOutputStream out, long totalAudioLen,long totalDataLen, int sampleRate, int channels, long byteRate) throws IOException {
byte[] header = new byte[44];
header[0] = 'R'; // RIFF
header[1] = 'I';
header[2] = 'F';
header[3] = 'F';
header[4] = (byte) (totalDataLen & 0xff);//数据大小
header[5] = (byte) ((totalDataLen >> 8) & 0xff);
header[6] = (byte) ((totalDataLen >> 16) & 0xff);
header[7] = (byte) ((totalDataLen >> 24) & 0xff);
header[8] = 'W';//WAVE
header[9] = 'A';
header[10] = 'V';
header[11] = 'E';
//FMT Chunk
header[12] = 'f'; // 'fmt '
header[13] = 'm';
header[14] = 't';
header[15] = ' ';//过渡字节
//数据大小
header[16] = 16; // 4 bytes: size of 'fmt ' chunk
header[17] = 0;
header[18] = 0;
header[19] = 0;
//编码方式 10H为PCM编码格式
header[20] = 1; // format = 1
header[21] = 0;
//通道数
header[22] = (byte) channels;
header[23] = 0;
//采样率，每个通道的播放速度
header[24] = (byte) (sampleRate & 0xff);
header[25] = (byte) ((sampleRate >> 8) & 0xff);
header[26] = (byte) ((sampleRate >> 16) & 0xff);
header[27] = (byte) ((sampleRate >> 24) & 0xff);
//音频数据传送速率,采样率*通道数*采样深度/8
header[28] = (byte) (byteRate & 0xff);
header[29] = (byte) ((byteRate >> 8) & 0xff);
header[30] = (byte) ((byteRate >> 16) & 0xff);
header[31] = (byte) ((byteRate >> 24) & 0xff);
// 确定系统一次要处理多少个这样字节的数据，确定缓冲区，通道数*采样位数
header[32] = (byte) (channels * 16 / 8);
header[33] = 0;
//每个样本的数据位数
header[34] = 16;
header[35] = 0;
//Data chunk
header[36] = 'd';//data
header[37] = 'a';
header[38] = 't';
header[39] = 'a';
header[40] = (byte) (totalAudioLen & 0xff);
header[41] = (byte) ((totalAudioLen >> 8) & 0xff);
header[42] = (byte) ((totalAudioLen >> 16) & 0xff);
header[43] = (byte) ((totalAudioLen >> 24) & 0xff);
out.write(header, 0, 44);
}
}
Then generates a relatively WAV file, we use the phone comes with the player able to play normally open at this time, but we found his size is relatively large, we see that a few minutes so big, we usually use the mp3, aac format, how do we do it?
Four, PCM file format into AAC
Generate aac file playback
public class AACUtil {
...
/**
* 初始化AAC编码器
*/
private void initAACMediaEncode() {
try {
//参数对应-> mime type、采样率、声道数
MediaFormat encodeFormat = MediaFormat.createAudioFormat(MediaFormat.MIMETYPE_AUDIO_AAC, 16000, 1);
encodeFormat.setInteger(MediaFormat.KEY_BIT_RATE, 64000);//比特率
encodeFormat.setInteger(MediaFormat.KEY_CHANNEL_COUNT, 1);
encodeFormat.setInteger(MediaFormat.KEY_CHANNEL_MASK, AudioFormat.CHANNEL_IN_MONO);
encodeFormat.setInteger(MediaFormat.KEY_AAC_PROFILE, MediaCodecInfo.CodecProfileLevel.AACObjectLC);
encodeFormat.setInteger(MediaFormat.KEY_MAX_INPUT_SIZE, 1024);//作用于inputBuffer的大小
mediaEncode = MediaCodec.createEncoderByType(encodeType);
mediaEncode.configure(encodeFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
} catch (IOException e) {
e.printStackTrace();
}
if (mediaEncode == null) {
LogUtil.e("create mediaEncode failed");
return;
}
mediaEncode.start();
encodeInputBuffers = mediaEncode.getInputBuffers();
encodeOutputBuffers = mediaEncode.getOutputBuffers();
encodeBufferInfo = new MediaCodec.BufferInfo();
}
private boolean codeOver = false;
/**
* 开始转码
* 音频数据{@link #srcPath}先解码成PCM PCM数据在编码成MediaFormat.MIMETYPE_AUDIO_AAC音频格式
* mp3->PCM->aac
*/
public void startAsync() {
LogUtil.w("start");
new Thread(new DecodeRunnable()).start();
}
/**
* 解码{@link #srcPath}音频文件 得到PCM数据块
*
* @return 是否解码完所有数据
*/
private void srcAudioFormatToPCM() {
File file = new File(srcPath);// 指定要读取的文件
FileInputStream fio = null;
try {
fio = new FileInputStream(file);
byte[] bb = new byte[1024];
while (!codeOver) {
if (fio.read(bb) != -1) {
LogUtil.e("============ putPCMData ============" + bb.length);
dstAudioFormatFromPCM(bb);
} else {
codeOver = true;
}
}
fio.close();
} catch (Exception e) {
e.printStackTrace();
}
}
private byte[] chunkAudio = new byte[0];
/**
* 编码PCM数据 得到AAC格式的音频文件
*/
private void dstAudioFormatFromPCM(byte[] pcmData) {
int inputIndex;
ByteBuffer inputBuffer;
int outputIndex;
ByteBuffer outputBuffer;
int outBitSize;
int outPacketSize;
byte[] PCMAudio;
PCMAudio = pcmData;
encodeInputBuffers = mediaEncode.getInputBuffers();
encodeOutputBuffers = mediaEncode.getOutputBuffers();
encodeBufferInfo = new MediaCodec.BufferInfo();
inputIndex = mediaEncode.dequeueInputBuffer(0);
inputBuffer = encodeInputBuffers[inputIndex];
inputBuffer.clear();
inputBuffer.limit(PCMAudio.length);
inputBuffer.put(PCMAudio);//PCM数据填充给inputBuffer
mediaEncode.queueInputBuffer(inputIndex, 0, PCMAudio.length, 0, 0);//通知编码器 编码
outputIndex = mediaEncode.dequeueOutputBuffer(encodeBufferInfo, 0);
while (outputIndex > 0) {
outBitSize = encodeBufferInfo.size;
outPacketSize = outBitSize + 7;//7为ADT头部的大小
outputBuffer = encodeOutputBuffers[outputIndex];//拿到输出Buffer
outputBuffer.position(encodeBufferInfo.offset);
outputBuffer.limit(encodeBufferInfo.offset + outBitSize);
chunkAudio = new byte[outPacketSize];
addADTStoPacket(chunkAudio, outPacketSize);//添加ADTS
outputBuffer.get(chunkAudio, 7, outBitSize);//将编码得到的AAC数据 取出到byte[]中
try {
//录制aac音频文件，保存在手机内存中
bos.write(chunkAudio, 0, chunkAudio.length);
bos.flush();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
outputBuffer.position(encodeBufferInfo.offset);
mediaEncode.releaseOutputBuffer(outputIndex, false);
outputIndex = mediaEncode.dequeueOutputBuffer(encodeBufferInfo, 0);
}
}
/**
* 添加ADTS头
*
* @param packet
* @param packetLen
*/
private void addADTStoPacket(byte[] packet, int packetLen) {
int profile = 2; // AAC LC
int freqIdx = 8; // 16KHz
int chanCfg = 1; // CPE
// fill in ADTS data
packet[0] = (byte) 0xFF;
packet[1] = (byte) 0xF1;
packet[2] = (byte) (((profile - 1) << 6) + (freqIdx << 2) + (chanCfg >> 2));
packet[3] = (byte) (((chanCfg & 3) << 6) + (packetLen >> 11));
packet[4] = (byte) ((packetLen & 0x7FF) >> 3);
packet[5] = (byte) (((packetLen & 7) << 5) + 0x1F);
packet[6] = (byte) 0xFC;
}
/**
* 释放资源
*/
public void release() {
...
}
/**
* 解码线程
*/
private class DecodeRunnable implements Runnable {
@Override
public void run() {
srcAudioFormatToPCM();
}
}
}
Five, AudioStack play
AudioTrack class to complete the task output audio data on the Android platform. Two AudioTrack data loading mode (MODE_STREAM and MODE_STATIC), corresponding to loading mode and the data type of the audio stream, corresponding to two completely different usage scenarios.
- MODE_STREAM: In this mode, the write data is written again AudioTrack the audio. This is usually invoked by write system to write data to a similar file, but this way of working requires every user to copy data from Buffer to Buffer provided in the interior of the AudioTrack, which to some extent will the introduction of delay. To solve this problem, AudioTrack on the introduction of a second model.
- MODE_STATIC: In this mode, just need to play before all of the data transmitted by a single call to write in the internal buffer AudioTrack, you do not have to follow the data transmission. This mode is suitable as ringtones This smaller footprint, higher latency requirements document. But it also has a shortcoming, which is a write data can not be too much, otherwise the system can not allocate enough memory to store all the data.
MediaPlayer can play a sound and AudioTrack, both of which provide a Java API for use by developers. Although both can play a sound, but both have a lot of differences, the biggest difference is that MediaPlayer can play a variety of audio file formats, such as MP3, AAC, WAV, OGG, MIDI and so on. MediaPlayer creates a corresponding audio decoder framework layer. And AudioTrack only play that has been decoded PCM stream, if you compare the words of supported file formats is AudioTrack only supports audio file format wav, wav format audio files because most of them are PCM streams. AudioTrack not create a decoder, we can only play it does not need to decode wav file.
Type 5.1 audio stream
In AudioTrack constructor, this parameter will be exposed to AudioManager.STREAM_MUSIC. Its meaning and Android systems management and classification of relevant audio stream.
Android sound system is divided into several stream types, here are a few common ones:
STREAM_ALARM: warning sound
STREAM_MUSIC: music, such as music and other
STREAM_RING: Ringtones
STREAM_SYSTEM: sound system, for example, low tone, lock screen, sound, etc.
STREAM_VOCIE_CALL: Call Sound
NOTE: The above division and these types of audio data itself does not matter. For example MUSIC RING type and can be a MP3 song. Further, the type of audio stream is not a fixed selection criteria, for example, ringing tones preview MUSIC type may be provided. The audio stream type of audio system Audio division and management strategy related.
5.2 Buffer concept of distribution and Frame
Buffer size distribution in the calculation of the time, we often use a method that is: getMinBufferSize. This function determines how much data the application layer assignment Buffer.
AudioTrack.getMinBufferSize(8000,//每秒8K个采样点
AudioFormat.CHANNEL_CONFIGURATION_STEREO,//双声道
AudioFormat.ENCODING_PCM_16BIT);
AudioTrack.getMinBufferSize traceability from start code can be found in a very important concept underlying code: Frame (frame). Frame is a unit used to describe how much amount of data. Frame unit 1 is equal to a sampling point number of bytes × number of channels (such as PCM16, a two-channel is equal to Frame 2 × 2 = 4 bytes). A channel for one sample point only, in fact, there may be one or more channels. The inability to separate a data amount expressed in units of all channels of one sample, it leads to the concept of Frame. Frame size is a number of bytes × number of channels sampled points. Further, in the present sound card driver in its internal buffer is also used as a unit Frame allocation and management.
After getMinBufSize will consider the case of hardware (such as whether to support the sampling rate, the hardware itself, delays, etc.), obtained a minimum size of the buffer. Generally we allocate buffer size would be an integer multiple of it.
5.3 build process
Each audio stream corresponds to one instance of a class AudioTrack, each registered to the AudioFlinger AudioTrack at creation time, all the AudioFlinger AudioTrack by mixing (Mixer), and then conveyed to AudioHardware for playback, while the maximum current Android create 32 audio streams, that is to say, Mixer up to simultaneously process the data stream 32 AudioTrack of.
public class AudioTrackManager {
...
//音频流类型
private static final int mStreamType = AudioManager.STREAM_MUSIC;
//指定采样率 （MediaRecoder 的采样率通常是8000Hz AAC的通常是44100Hz。 设置采样率为44100，目前为常用的采样率，官方文档表示这个值可以兼容所有的设置）
private static final int mSampleRateInHz = 44100;
//指定捕获音频的声道数目。在AudioFormat类中指定用于此的常量
private static final int mChannelConfig = AudioFormat.CHANNEL_CONFIGURATION_MONO; //单声道
//指定音频量化位数 ,在AudioFormaat类中指定了以下各种可能的常量。通常我们选择ENCODING_PCM_16BIT和ENCODING_PCM_8BIT PCM代表的是脉冲编码调制，它实际上是原始音频样本。
//因此可以设置每个样本的分辨率为16位或者8位，16位将占用更多的空间和处理能力,表示的音频也更加接近真实。
private static final int mAudioFormat = AudioFormat.ENCODING_PCM_16BIT;
//指定缓冲区大小。调用AudioRecord类的getMinBufferSize方法可以获得。
private int mMinBufferSize;
//STREAM的意思是由用户在应用程序通过write方式把数据一次一次得写到audiotrack中。这个和我们在socket中发送数据一样，
// 应用层从某个地方获取数据，例如通过编解码得到PCM数据，然后write到audiotrack。
private static int mMode = AudioTrack.MODE_STREAM;
private void initData() {
//根据采样率，采样精度，单双声道来得到frame的大小。
mMinBufferSize = AudioTrack.getMinBufferSize(mSampleRateInHz, mChannelConfig, mAudioFormat);//计算最小缓冲区
//注意，按照数字音频的知识，这个算出来的是一秒钟buffer的大小。
//创建AudioTrack
mAudioTrack = new AudioTrack(mStreamType, mSampleRateInHz, mChannelConfig,
mAudioFormat, mMinBufferSize, mMode);
}
/**
* 启动播放线程
*/
private void startThread() {
destroyThread();
isStart = true;
if (mRecordThread == null) {
mRecordThread = new Thread(recordRunnable);
mRecordThread.start();
}
}
/**
* 播放线程
*/
private Runnable recordRunnable = new Runnable() {
@Override
public void run() {
try {
//设置线程的优先级
android.os.Process.setThreadPriority(android.os.Process.THREAD_PRIORITY_URGENT_AUDIO);
byte[] tempBuffer = new byte[mMinBufferSize];
int readCount = 0;
while (mDis.available() > 0) {
readCount = mDis.read(tempBuffer);
if (readCount == AudioTrack.ERROR_INVALID_OPERATION || readCount == AudioTrack.ERROR_BAD_VALUE) {
continue;
}
//一边播放一边写入语音数据
if (readCount != 0 && readCount != -1) {
//判断AudioTrack未初始化，停止播放的时候释放了，状态就为STATE_UNINITIALIZED
if (mAudioTrack.getState() == mAudioTrack.STATE_UNINITIALIZED) {
initData();
}
mAudioTrack.play();
mAudioTrack.write(tempBuffer, 0, readCount);
}
}
//播放完就停止播放
stopPlay();
} catch (Exception e) {
e.printStackTrace();
}
}
};
/**
* 启动播放
*
* @param path
*/
public void startPlay(String path) {
try {
setPath(path);
startThread();
} catch (Exception e) {
e.printStackTrace();
}
}
/**
* 停止播放
*/
public void stopPlay() {
try {
destroyThread();//销毁线程
if (mAudioTrack != null) {
if (mAudioTrack.getState() == AudioRecord.STATE_INITIALIZED) {//初始化成功
mAudioTrack.stop();//停止播放
}
if (mAudioTrack != null) {
mAudioTrack.release();//释放audioTrack资源
}
}
if (mDis != null) {
mDis.close();//关闭数据输入流
}
} catch (Exception e) {
e.printStackTrace();
}
}
}