Question: Our company has a project, the android 11 rk3566 project, which has an audio module, and the project MIC uses es7202 (ADC). This chip is an encoding chip without decoding function. The recording gain of this module has been adjusted to the maximum. , but the volume of the recorded MIC is still very low, and there is no solution for the hardware. The digital MIC uses the PDM protocol, and the PDM digital signal is more complicated. PDM signals are covered in my other articles.
Analysis: MIC is sent to CPU to output digital MIC signal for ES7202, please confirm whether there is any amplification processing for digital MIC signal in CPU? Theoretically yes, the CPU PDM data signal accepts low-amplitude input by default, and the processing of the PDM signal in the CPU is relatively complicated. The application based on the PDM interface reduces the complexity of the sending device, and because the CODEC as a receiving device integrates a decimation filter inside, the overall system complexity is greatly reduced. The PDM uses a clock sampling rate much higher than the PCM sampling rate to modulate the analog component, and only 1 bit is output, which is either 0 or 1. Therefore, digital audio represented by PDM is also called Oversampled 1-bit Audio. Compared with a series of 0s and 1s in PDM, the quantization result of PCM is more intuitive and simple. Using the PDM method as the receiving end of the analog-to-digital conversion requires the use of a decimation filter (Decimation Filter) to convert the density components represented by densely packed 0s and 1s into amplitude components, while the PCM method already obtains amplitude components.
Solution direction: Since the gain of our es7202 MIC has been adjusted to the maximum, but there is still no good effect, and the effect of increasing the MIC recording cannot be achieved. Directly process the PDM data signal. Due to the data entered by the DATA MIC, and the complexity of the PDM data is high, it is difficult for us to process the relevant data, so we adopt the third method, which is to process it in tinyalsa, and the upper-level android application calls the underlying When driving, the underlying driver is called through the HAL layer, and the hardware is driven to work, and the processing in the HAL layer will be called. Since we use the audio ALSA architecture, the upper layer calls the driver through the relevant pcm processing functions. I believe that using You may know more about tinycap and tinyplay. If you don’t know much, you can take a look at the relevant source code. During our debugging process, we use more related commands and continue to analyze... And our audio sampling, PCM What does the stream look like? If it is mono, then the data of each sample, here we regard the data as integer data, which is a series, and each integer occupies 2 bytes (16-bit ), 9 samples are 18 bytes of data. The minimum integer size of each sample is -32768, and the maximum is 32768 (note: the data type here is short type). If you draw a graph based on the position and value of the sampled data, you will get a wavy graph like the one on the player. Stereo, in fact, is dual-channel sampling, and if its data is regarded as a string of data, it is actually the data of the left and right channels interleaved, and each frame is a 16-bit sampling point. So we can process the raw PCM data obtained when alsa calls the underlying driver, and all we need to do is to enlarge it. And this solves our problem. The principle is to regard the PCM raw data recorded by the MIC as a sine wave, and what we need to do is to amplify the amplitude of the sine wave, which is equivalent to increasing the volume of the PCM data. , you only need to multiply each sampled data by a coefficient.
First, let's take a look at the calling function code of the HAL layer:
static int get_next_buffer(struct resampler_buffer_provider *buffer_provider,
struct resampler_buffer* buffer)
{
struct stream_in *in;
size_t i,size;
if (buffer_provider == NULL || buffer == NULL)
return -EINVAL;
in = (struct stream_in *)((char *)buffer_provider -
offsetof(struct stream_in, buf_provider));
if (in->pcm == NULL) {
buffer->raw = NULL;
buffer->frame_count = 0;
in->read_status = -ENODEV;
return -ENODEV;
}
if (in->frames_in == 0) {
size = pcm_frames_to_bytes(in->pcm,pcm_get_buffer_size(in->pcm)
in->read_status = pcm_read(in->pcm,//这里就调用了pcm_read,来起一个过度作用，其实是使用了tinyalsa,来获取原始pcm数据，(void*)in->buffer，中装入了PCM 原始数据。
(void*)in->buffer,pcm_frames_to_bytes(in->pcm, in->config->period_size));
if (in->read_status != 0) {
ALOGE("get_next_buffer() pcm_read error %d", in->read_status);
buffer->raw = NULL;
buffer->frame_count = 0;
return in->read_status;
}
//fwrite(in->buffer,pcm_frames_to_bytes(in->pcm,pcm_get_buffer_size(in->pcm)),1,in_debug);
in->frames_in = in->config->period_size;
/* Do stereo to mono conversion in place by discarding right channel */
if ((in->channel_mask == AUDIO_CHANNEL_IN_MONO)
&&(in->config->channels == 2)) {
//ALOGE("channel_mask = AUDIO_CHANNEL_IN_MONO");
for (i = 0; i < in->frames_in; i++)
in->buffer[i] = in->buffer[i * 2];
}
}
//ALOGV("pcm_frames_to_bytes(in->pcm,pcm_get_buffer_size(in->pcm)):%d",size);
buffer->frame_count = (buffer->frame_count > in->frames_in) ?
in->frames_in : buffer->frame_count;
buffer->i16 = in->buffer +
(in->config->period_size - in->frames_in) *
audio_channel_count_from_in_mask(in->channel_mask);
return in->read_status;
}Let's take a look at the code in tinyalsa again. What I mainly give here is the code related to the input stream, that is, when recording, and the playback stream is the opposite.
int pcm_read(struct pcm *pcm, void *data, unsigned int count)
{
struct snd_xferi x;
if (!(pcm->flags & PCM_IN))
return -EINVAL;
x.buf = data;
x.frames = count / (pcm->config.channels *
pcm_format_to_bits(pcm->config.format) / 8);
for (;;) {
if (!pcm->running) {
if (pcm_start(pcm) < 0) {
fprintf(stderr, "start error");
return -errno;
}
}
if (ioctl(pcm->fd, SNDRV_PCM_IOCTL_READI_FRAMES, &x)) {//用户层通过ioctl的方式来调用kernel,//读出入初始化数据，即录音。
pcm->prepared = 0;
pcm->running = 0;
if (errno == EPIPE) {
/* we failed to make our window -- try to restart */
pcm->underruns++;
continue;
}
return oops(pcm, errno, "cannot read stream data");
}
if (pcm->config.channels == 2) {
if (channalFlags == -1 ) {
if (startCheckCount < SAMPLECOUNT) {
startCheckCount += count;
} else {
channalFlags = channel_check(data,count / 2);
}
}
channel_fixed(data,count / 2, channalFlags);
}
return 0;
}
}
The implementation of the kernel layer initiates a system call in the kernel to execute the fops function set that should have been initiated by the user space.
kernel kernel version, 4.19.172
PCM逻辑设备文件操作函数集：snd_pcm_f_ops[]
PCM逻辑设备文件操作函数集对于Playback和Capture是分开定义的，该操作函数集如下：
const struct file_operations snd_pcm_f_ops[2] = {
{
.owner = THIS_MODULE,
.write = snd_pcm_write,//用于单通道音频信号写
.write_iter = snd_pcm_writev,
.open = snd_pcm_playback_open,
.release = snd_pcm_release,
.llseek = no_llseek,
.poll = snd_pcm_poll,
.unlocked_ioctl = snd_pcm_ioctl,//用于多通道音频信号写
.compat_ioctl = snd_pcm_ioctl_compat,
.mmap = snd_pcm_mmap,
.fasync = snd_pcm_fasync,
.get_unmapped_area = snd_pcm_get_unmapped_area,
},
{
.owner = THIS_MODULE,
.read = snd_pcm_read,//用于单通道音频信号读
.read_iter = snd_pcm_readv,
.open = snd_pcm_capture_open,
.release = snd_pcm_release,
.llseek = no_llseek,
.poll = snd_pcm_poll,
.unlocked_ioctl = snd_pcm_ioctl,//用于多通道音频信号读
.compat_ioctl = snd_pcm_ioctl_compat,
.mmap = snd_pcm_mmap,
.fasync = snd_pcm_fasync,
.get_unmapped_area = snd_pcm_get_unmapped_area,
}
};
Let's take a look at the hardware circuit diagram of our es7202, as well as the frame diagram and related registers for adjusting the gain.
es7202 internal frame diagram:
es7202 peripheral circuit, relatively few:
The es7202 internal modulation MIC gain register:
Therefore, on the whole, the overall advantages of using digital MIC are quite obvious. There are fewer peripheral circuits, and the input MIC data is mixed with the collection of CLK into PDM DATA and directly sent to the CPU through the PDM data interface for processing.
The modified patch is given below, and the directory is in external/tinyalsa/pcm.c
diff --git a/pcm.c b/pcm.c
index 0b97bbc..01d777e 100644
--- a/pcm.c
+++ b/pcm.c
@@ -635,13 +635,36 @@ void channel_fixed(void *data, unsigned len, int chFlag)
return;
}
+static short out_vol = 4.0;//扩大音量倍数
+static void volume_process(const void *buffer, size_t length, short volume) {
+
+short * buffer_end = (short*)buffer + length/2;
+ short * pcmData = (short *)buffer;
+ int i = 0;
+ int pcmval;
+
+ while (pcmData < buffer_end) {
+ pcmval = (short)*pcmData * volume;
+ if (pcmval < 32768 && pcmval > -32768) {
+ *pcmData = pcmval;
+ } else if (pcmval > 32767) {
+ *pcmData = 32767; //限制最大幅度
+ } else if (pcmval < -32767) {
+ *pcmData = -32767;//限制最小幅度
+ }
+
+ ++pcmData;
+ i++;
+ }
+}
+
int pcm_read(struct pcm *pcm, void *data, unsigned int count)
{
struct snd_xferi x;
if (!(pcm->flags & PCM_IN))
return -EINVAL;
-
+ memset(data, 0, count);
x.buf = data;
x.frames = count / (pcm->config.channels *
pcm_format_to_bits(pcm->config.format) / 8);
@@ -676,6 +699,7 @@ int pcm_read(struct pcm *pcm, void *data, unsigned int count)
channel_fixed(data,count / 2, channalFlags);
}
+ volume_process(x.buf, count , out_vol);
return 0;
}
}
Theoretically speaking, this patch can also solve the problem of low input MIC recording volume caused by other codecs.