Table of contents
foreword
由于进行linux音视频开发，通常会涉及到音频以及摄像头视频驱动，所以对V4L2以及ALSA驱动框架进行学习，并整理相关的笔记和大家分享！
1. History
V4L2 isVideo for linux2The abbreviation of , is the kernel driver for video devices in linux . In Linux, a video device is a device file, which can be read and written like an ordinary file. The camera is under /dev/video*, and if there is only one video device, it is usually /dev/video0.
V4L2 is designed to support a wide range of devices, only some of which are true video devices in nature .
V4L2 has a history. Around the fall of 1998, its light first appeared in the eyes of Bill Dirks.
After considerable development, it was integrated into the kernel trunk when it released 2.5.46 in November 2002. However, until today, there are still some kernel drivers that do not support the new API, and the conversion work of this old and new API is still in progress.
At the same time, the V4L2 API is also evolving and has some major changes in version 2.6.18. There are still relatively few applications that support V4L2. V4L2 is designed to support a wide range of devices. Only some of them are real video devices in nature and can support multiple devices. It can have the following interfaces:
1. 视频采集接口(video capture interface):这种应用的设备可以是高频头或者摄像头.
V4L2的最初设计就是应用于这种功能的.
2. 视频输出接口(video output interface):可以驱动计算机的外围视频图像设备--像
可以输出电视信号格式的设备.
3. 直接传输视频接口(video overlay interface):它的主要工作是把从视频采集设备
采集过来的信号直接输出到输出设备之上,而不用经过系统的CPU.
4. 视频间隔消隐信号接口(VBI interface):它可以使应用可以访问传输消隐期的视频信号.
5. 收音机接口(radio interface):可用来处理从AM或FM高频头设备接收来的音频流.
2. Main functions
使程序有发现设备和操作设备的能力,它主要是用一系列的回调函数来实现这些功能。Such as setting the camera frequency, frame rate, video compression format and image parameters, etc. Of course, it can also be used for the development of other multimedia, such as audio and so on.
butThis framework can only run on the Linux operating system.
3. Programming development
v4L2 is a programming framework for uvc drive-free usb devices, mainly used for collecting usb cameras, etc. The basic development steps are as follows:
1. Collection method
After opening a video device, you can set the properties of the video device, such as cropping, scaling, and so on. This step is optional. In Linux programming, the ioctl function is generally used to manage the I/O channel of the device:
extern int ioctl (int __fd, unsigned long int __request, …) __THROW;
__fd：设备的ID，例如刚才用open函数打开视频通道后返回的cameraFd；
__request：具体的命令标志符。
在进行V4L2开发中，一般会用到以下的命令标志符：
VIDIOC_REQBUFS：分配内存
VIDIOC_QUERYBUF：把VIDIOC_REQBUFS中分配的数据缓存转换成物理地址
VIDIOC_QUERYCAP：查询驱动功能
VIDIOC_ENUM_FMT：获取当前驱动支持的视频格式
VIDIOC_S_FMT：设置当前驱动的频捕获格式
VIDIOC_G_FMT：读取当前驱动的频捕获格式
VIDIOC_TRY_FMT：验证当前驱动的显示格式
VIDIOC_CROPCAP：查询驱动的修剪能力
VIDIOC_S_CROP：设置视频信号的边框
VIDIOC_G_CROP：读取视频信号的边框
VIDIOC_QBUF：把数据放回缓存队列
VIDIOC_DQBUF：把数据从缓存中读取出来
VIDIOC_STREAMON：开始视频显示函数
VIDIOC_STREAMOFF：结束视频显示函数
VIDIOC_QUERYSTD：检查当前视频设备支持的标准，例如PAL或NTSC。
这些IO调用，有些是必须的，有些是可选择的。
2. Operation process (programming steps)
参考学习：https://blog.csdn.net/u010783226/article/details/122260974
1. 打开设备文件。
int fd=open(”/dev/video0″,O_RDWR);
2. 取得设备的capability，看看设备具有什么功能，比如是否具有视频输入,或者音频输入
输出等。VIDIOC_QUERYCAP, struct v4l2_capability
struct v4l2_capability {
__u8 driver[16]; /* i.e. "bttv" */
__u8 card[32]; /* i.e. "Hauppauge WinTV" */
__u8 bus_info[32]; /* "PCI:" + pci_name(pci_dev) */
__u32 version; /* should use KERNEL_VERSION() */
__u32 capabilities; /* Device capabilities */
__u32 reserved[4];
};
其中最重要的是capabilities字段，这个字段标记着v4l2设备的功能，capabilities有
以下部分标记位
ID 描述符
V4L2_CAP_VIDEO_CAPTURE 设备支持捕获功能
V4L2_CAP_VIDEO_OUTPUT 设备支持输出功能
V4L2_CAP_VIDEO_OVERLAY 设备支持预览功能
V4L2_CAP_STREAMING 设备支持流读写
V4L2_CAP_READWRITE 设备支持read、write方式读写
我们可以通过这样子去判断设备的功能
if(cap.capabilities & V4L2_CAP_VIDEO_CAPTURE)
printf("v4l2 dev support capture\n");
if(cap.capabilities & V4L2_CAP_VIDEO_OUTPUT)
printf("v4l2 dev support output\n");
if(cap.capabilities & V4L2_CAP_VIDEO_OVERLAY)
printf("v4l2 dev support overlay\n");
if(cap.capabilities & V4L2_CAP_STREAMING)
printf("v4l2 dev support streaming\n");
if(cap.capabilities & V4L2_CAP_READWRITE)
printf("v4l2 dev support read write\n");
v4l2_std_id std;
do {
ret= ioctl(fd, VIDIOC_QUERYSTD, &std);
} while (ret == -1 && errno == EAGAIN);
switch (std) {
case V4L2_STD_NTSC:
//……
case V4L2_STD_PAL:
//……
}
struct v4l2_capability cap;
if (ioctl(fd, VIDIOC_QUERYCAP, cap) < 0)
{
printf("ERR(%s):VIDIOC_QUERYCAP failed\n", __func__);
return -1;
}
3. 选择视频输入，一个视频设备可以有多个视频输入。
VIDIOC_S_INPUT,struct v4l2_input
一个设备可能有多个输入，比如：在芯片上，摄像头控制器和摄像头接口是分离的，需要选择哪一个摄像头接口作为摄像头控制器的输入源
当然，并不是所有的设备都需要设置输入，比如：uvc摄像头，一般只有一个输入，默认就会选择，不需要设置
下面介绍如何设置输入设备
（1）枚举输入设备
下面这段程序枚举了该设备所有的输入源，并打印输入源的名称
struct v4l2_input input;
input.index = 0;
while (!ioctl(fd, VIDIOC_ENUMINPUT, &input))
{
printf("input:%s\n", input.name);
++input.index;
}
（2）设置输入设备
struct v4l2_input input;
input.index = index; //指定输入设备
if (ioctl(fd, VIDIOC_S_INPUT, &input) < 0)
{
printf("ERR(%s):VIDIOC_S_INPUT failed\n", __func__);
return -1;
}
4. 设置视频的制式和帧格式，制式包括PAL，NTSC，帧的格式个包括宽度和高度等。
VIDIOC_S_STD,VIDIOC_S_FMT,struct v4l2_std_id,struct v4l2_format
struct v4l2_format fmt;
/*
v4l2_format 结构如下：
struct v4l2_format
{
enum v4l2_buf_type type; // 数据流类型，必须永远是V4L2_BUF_TYPE_VIDEO_CAPTURE
union
{
struct v4l2_pix_format pix;
struct v4l2_window win;
struct v4l2_vbi_format vbi;
__u8 raw_data[200];
} fmt;
};
struct v4l2_pix_format
{
__u32 width; // 宽，必须是16 的倍数
__u32 height; // 高，必须是16 的倍数
__u32 pixelformat; // 视频数据存储类型，例如是YUV 4 ：2 ：2 还是RGB
enum v4l2_field field;
__u32 bytesperline;
__u32 sizeimage;
enum v4l2_colorspace colorspace;
__u32 priv;
};
*/
样例：
memset ( &fmt, 0, sizeof(fmt) );
fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
fmt.fmt.pix.width = 320;
fmt.fmt.pix.height = 240;
fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_JPEG;
if (ioctl(fd, VIDIOC_S_FMT, &fmt) < 0）
{
printf("set format failed\n");
//return 0;
}
5. 向驱动申请帧缓冲，一般不超过5个。struct v4l2_requestbuffers
struct v4l2_requestbuffers
{
__u32 count; // 缓存数量，也就是说在缓存队列里保持多少张照片
enum v4l2_buf_type type; // 数据流类型，必须永远是V4L2_BUF_TYPE_VIDEO_CAPTURE
enum v4l2_memory memory; // V4L2_MEMORY_MMAP 或 V4L2_MEMORY_USERPTR
__u32 reserved[2];
};
样例：
struct v4l2_requestbuffers req;
memset(&req, 0, sizeof (req));
req.count = 4;
req.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
req.memory = V4L2_MEMORY_MMAP;
if (ioctl(fd,VIDIOC_REQBUFS,&req) == -1)
{
perror("VIDIOC_REQBUFS error \n");
//return -1;
}
6.申请物理内存
将申请到的帧缓冲映射到用户空间，这样就可以直接操作采集到的帧了，而不必去复制。
将申请到的帧缓冲全部入队列，以便存放采集到的数据.VIDIOC_QBUF,struct v4l2_buffer
VideoBuffer* buffers = calloc( req.count, sizeof(VideoBuffer) );
printf("sizeof(VideoBuffer) is %d\n",sizeof(VideoBuffer));
struct v4l2_buffer buf;
for (numBufs = 0; numBufs < req.count; numBufs++)
{
memset( &buf, 0, sizeof(buf) );
buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
buf.memory = V4L2_MEMORY_MMAP;
buf.index = numBufs;
if (ioctl(fd, VIDIOC_QUERYBUF, &buf) < 0)
{
printf("VIDIOC_QUERYBUF error\n");
//return -1;
}
printf("buf len is %d\n",sizeof(buf));
//内存映射
buffers[numBufs].length = buf.length;
buffers[numBufs].offset = (size_t) buf.m.offset;
buffers[numBufs].start = mmap (NULL, buf.length,PROT_READ | PROT_WRITE, MAP_SHARED, fd, buf.m.offset);
printf("buffers.length = %d,buffers.offset = %d ,buffers.start[0] = %d\n",buffers[numBufs].length,buffers[numBufs].offset,buffers[numBufs].start[0]);
printf("buf2 len is %d\n",sizeof(buffers[numBufs].start));
if (buffers[numBufs].start == MAP_FAILED)
{
perror("buffers error\n");
//return -1;
}
if (ioctl (fd, VIDIOC_QBUF, &buf) < 0)
{
printf("VIDIOC_QBUF error\n");
//return -1;
}
}
7. 开始视频的采集。
enum v4l2_buf_type type;
type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
if (ioctl (fd, VIDIOC_STREAMON, &type) < 0)
{
printf("VIDIOC_STREAMON error\n");
// return -1;
}
出队列以取得已采集数据的帧缓冲，取得原始采集数据。VIDIOC_DQBUF, 将缓冲重新
入队列尾,这样可以循环采集。VIDIOC_QBUF
if (ioctl(fd, VIDIOC_DQBUF, &buf) < 0)
{
perror("VIDIOC_DQBUF failed.\n");
//return -1;
}
buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
buf.memory = V4L2_MEMORY_MMAP;
unsigned char *ptcur = buffers[numBufs].start;
printf("buf.bytesused = %d \n",buf.bytesused);
int i1;
for(i1=0; i1<buf.bytesused; i1++)
{
if((buffers[numBufs].start[i1] == 0x000000FF) && (buffers[numBufs].start[i1+1] == 0x000000C4))
{
printf("Huffman table finded! \nbuf.bytesused = %d\nFFC4 = %d \n",buf.bytesused,i1);
break;
}
}
if(i1 == buf.bytesused)printf("huffman table don't exist! \n");
int i;
for(i=0; i<buf.bytesused; i++)
{
if((buffers[numBufs].start[i] == 0x000000FF) && (buffers[numBufs].start[i+1] == 0x000000D8)) break;
ptcur++;
}
printf("i=%d,FF=%02x,D8=%02x\n",i,buffers[numBufs].start[i],buffers[numBufs].start[i+1]);
int imagesize =buf.bytesused - i;
printf("buf.bytesused = %d \n",buf.bytesused);
printf("imagesize = %d \n",imagesize);
8. 停止视频的采集。VIDIOC_STREAMOFF
enum v4l2_buf_type type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
if (ioctl(fd, VIDIOC_STREAMOFF, &type) < 0)
{
printf("ERR(%s):VIDIOC_STREAMOFF failed\n", __func__);
return -1;
}
9. close(fd)关闭视频设备，munmap解除内存映射，释放资源。
After understanding the source and basic operation of V4L2, the next article will introduce the driver framework of V4L2, so stay tuned.
The content of this article is mainly from Baidu Encyclopedia and teacher Wei Dongshan's course notes, if there is any infringement, please contact to delete! Welcome everyone to guide and communicate in the comment area! ! !