Table of Contents of Series Articles
[V4L2] Brief introduction of V4L2 framework
[V4L2] Driver structure of V4L2 framework
[V4L2] V4L2 sub-device
Article directory
v4l2 subdevice userspace API
Device nodes can be /devcreated under the folder v4l-subdevXfor users to directly operate sub-device hardware. If you need to create a device node in user space, you need to set the V4L2_SUBDEV_FL_HAS_DEVNODEflag before registering the child device node, and then call v4l2_device_register_subdev_nodes()the function to create the device node in user space. The device node will be automatically destroyed when the child device is uninstalled.
VIDIOC_QUERYCTRL
VIDIOC_QUERYMENU
VIDIOC_G_CTRL
VIDIOC_S_CTRL
VIDIOC_G_EXT_CTRLS
VIDIOC_S_EXT_CTRLS
VIDIOC_TRY_EXT_CTRLS
The above ioctls can be accessed through the device node, or can be called directly in the sub-device driver.
VIDIOC_DQEVENT
VIDIOC_SUBSCRIBE_EVENT
VIDIOC_UNSUBSCRIBE_EVENT
To use the above events, you must set v4l2_subdevthe V4L2_SUBDEV_USES_EVENTSflag bit and implement the core_opsrelated subscribecallback function. The events need to be initialized in the callback function and then registered v4l2_subdev. Some private ioctls can be implemented v4l2_subdevin ops->core->ioctl.
I2C subdevice driver
If you want to add support to the I2C driver v4l2_subdev, you need to embed v4l2_subdevthe structure into each I2C instance structure. Some relatively simple I2C devices do not require a custom status structure. In this case, you only need to create a separate structure v4l2_subdev. That’s it. A typical driver-defined state structure is as follows:
struct chipname_state {
struct v4l2_subdev sd;
... /* additional state fields */
};
Use v4l2_i2c_subdev_initto initialize an I2C subdevice. This function will populate v4l2_subdevall members of and ensure v4l2_subdevthat and i2c_clientpoint to each other. You can also add inline functions to v4l2_subdevget i2c_clientthe structure from the pointer:
struct i2c_client *client = v4l2_get_subdevdata(sd);
也可以从i2c_client结构体指针获取到v4l2_subdev结构体：
struct v4l2_subdev *sd = i2c_get_clientdata(client);
桥驱动可以使用以下帮助函数来创建一个I2C子设备：
struct v4l2_subdev *sd = v4l2_i2c_new_subdev
(v4l2_dev, adapter,"module_foo", "chipid", 0x36, NULL);
This function will load the given module (can be empty) and call to i2c_new_devicecreate a sub-device structure according to the parameters passed in, and finally register v4l2_subdev.
video_device structure
video_deviceCan be allocated dynamically:
struct video_device *vdev = video_device_alloc();
if (vdev == NULL)
return -ENOMEM;
vdev->release = video_device_release;
If you need to video_deviceembed the structure into a larger structure, you need to set the release member of vdev. The kernel provides two default release callback functions, as follows:
video_device_release() // 仅仅调用kfree释放分配的内存，用于动态分配情况下
video_device_release_empty() // 不做任何事情，静态变量
The following function members must be set:
-
v4l2_dev: must point to the v4l2_device parent device
-
vfl_dir: VFL_DIR_RX (capture device), VFL_DIR_TX (output device), VFL_DIR_M2M (codec device)
-
fops: Set the v4l2_file_operations structure
-
ioctl_ops: ioctls, which can be accessed by user space programs through device nodes, need to set .unlocked_ioctl of fops to point to video_ioctl2
-
lock: If you want to perform lock operation in the drive space, you can set it to NULL. Otherwise, it needs to point to an initialized mutex_lock structure
-
Queue: Points to a vb2_queue structure. If queue->lock is not empty, then the ioctls related to the queue will use the lock inside the queue. In this way, there is no need to wait for other types of ioctls operations.
-
prio: Track the priority, used on VIDIOC_G/S_PRIORITY, if it is empty, v4l2_prio_state in v4l2_device will be used
-
dev_parent: just point to v4l2_device
If you want to ignore ioctl_opsone of the ioctls, you can call the following function:
void v4l2_disable_ioctl(struct video_device *vdev, unsigned int cmd);
If you want to integrate it media_frameworkinto , you need to set the members video_deviceinside media_entityand provide media_pad:
struct media_pad *pad = &my_vdev->pad;
int err;
err = media_entity_init(&vdev->entity, 1, pad, 0);
- Registration of video_device
The registration function of video_device is as follows:
err = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
This code registers a character device driver and generates a device node in user space. If the mdev member of the v4l2_device parent device is not empty, the entity of video_device will be automatically registered in the media framework. The last parameter of the function is the device node index number. If it is -1, the index number value available in the first kernel is used. The registered device type and node name in user space depend on the following identifiers:
VFL_TYPE_GRABBER: videoX input and output device
VFL_TYPE_VBI: vbiX
VFL_TYPE_RADIO: radioX hardware-defined audio tuning device
VFL_TYPE_SDR: swradioX software-defined audio tuning device
When a device node is created, related attributes will also be created. You can /sys/class/video4linuxsee these device folders inside. In the folders, you can see attributes such as 'name', 'dev_debug', 'index', 'uevent', etc. You can use the cat command to view it. 'dev_debug' can be used for video device debugging. Each video device will create a 'dev_debug' attribute, which exists in the form of a folder and is available /sys/class/video4linux/<devX>/below for enablement log file operation. 'dev_debug' is a bitmask, the following bits can be set:
0x01：记录ioctl名字与错误码。设置0x08位可以只记录VIDIOC_(D)QBUF
0x02：记录ioctl的参数与错误码。设置0x08位可以只记录VIDIOC_(D)QBUF
0x04：记录file ops操作。设置0x08位可以只记录read&write成员的操作
0x08：如上所示
0x10：记录poll操作
When the above bits are set, the kernel will print out the relevant call information to the terminal when a relevant call or operation occurs. Similar to
[173881.402120] video4: VIDIOC_DQEVENT: error -2
[173884.906633] video4: VIDIOC_UNSUBSCRIBE_EVENT
- Cleaning up the video device
When the video device node needs to be removed or the USB device is disconnected, the following function needs to be executed:
video_unregister_device(vdev);
To uninstall the device, this function will remove the device node file under /dev. At the same time, don't forget to call media_entity_cleanupto clean up the entity.
ioctls and locking
The V4L core layer provides optional lock services, the most important of which is the lock in video_device, which is used for ioctls synchronization. If the videobuf2 framework is used, the video_device->queue->lock lock will also be used for queue-related ioctls synchronization. There are many advantages to using different locks. For example, some setting-related ioctls take a long time. If an independent lock is used, VIDIOC_DQBUF can be executed without waiting for the completion of the setting operation. This is very common in network camera drivers. Of course, the lock operation can also be completed entirely by the driver itself. In this case, you can set all lock members to NULL and implement a driver's own lock.
If you use the old videobuf, you need to video_devicepass the lock to videobuf queuethe initialization function. If videobuf is waiting for the arrival of a frame of data, the lock will be temporarily released and will be locked again after the data arrives. Otherwise, other handlers will not be able to access it. So using the old videobuf is not recommended. If you are under the videobuf2 framework, you need to implement wait_prepare 与 wait_finisha callback function to release or acquire the lock. If queue->lock is used, you can use the callback vb2_ops_wait_prepare/finishhelper function provided by V4L2 to complete the locking and unlocking operations. They will use queue->lock. Lock (the lock must be initialized at this time).
v4l2_fh structure
This structure provides a simple way to save file handle-specific data. Users of v4l2_fh - the v4l2 framework can know whether the driver uses v4l2_fh as the file->private_data pointer by checking the video_device->flags V4L2_FL_USES_V4L2_FHbit. The flag bit is set by calling the function v4l2_fh_init.
The v4l2_fh structure exists as the driver's own file handle, and file->private_data is set to point to it in the driver's open function. When there are multiple v4l2_fhs, they will exist in file->private_data as a linked list and can be traversed and accessed. In most cases, the v4l2_fh structure is embedded in a larger structure. In this case, v4l2_fh_init+v4l2_fh_add needs to be called in the open function to add it
, and v4l2_fh_del+v4l2_fh_exit needs to be called in the release function to exit. Drivers can use container_of to access their own file handle structures, as shown below:
struct my_fh {
int blah;
struct v4l2_fh fh;
};
int my_open(struct file *file)
{
struct my_fh *my_fh;
struct video_device *vfd;
int ret;
my_fh = kzalloc(sizeof(*my_fh), GFP_KERNEL);
v4l2_fh_init(&my_fh->fh, vfd);
file->private_data = &my_fh->fh;
v4l2_fh_add(&my_fh->fh);
return 0;
}
int my_release(struct file *file)
{
struct v4l2_fh *fh = file->private_data;
struct my_fh *my_fh = container_of(fh, struct my_fh, fh);
v4l2_fh_del(&my_fh->fh);
v4l2_fh_exit(&my_fh->fh);
kfree(my_fh);
return 0;
}
As shown in the above code, since the open function may be called by multiple applications, there will be multiple fhs, but file->private always points to the latest v4l2_fh. Through this v4l2_fh, all the v4l2_fh links in the entire linked list can be found. element. Some drivers need to do some other work after the first file handle is opened and before the last file handle is closed. The following two helper functions can check whether there is only one entry left in the v4l2_fh structure:
int v4l2_fh_is_singular(struct v4l2_fh *fh)
如果是只有一个entry，返回1，否则返回0，如果fh为空也返回0。
int v4l2_fh_is_singular_file(struct file *filp)
和上面差不多，但是使用 filp->private_data 这一数据源，实际上它是指向最新的一个v4l2_fh的。
V4L2 events
V4L2 eventsTo provide a general method to pass events to user space, the driver must use v4l2_fh (set the flags bit of video_device) to support V4L2 events. Events use type and ID as distinguishing identifiers, and the ID of unused events is 0.
When a user subscribes to an event, the user space will allocate a kevent structure to each event accordingly (if the elems parameter is 0, there will be only one, if it is not 0, it will be allocated according to the specified number), so each event has one or more A kevent structure of its own, which ensures that if the driver generates a lot of events in a short period of time, it will not cover other events of the same type. It can be seen as dividing several baskets to put different types of fruits. The event structure is v4l2_subscribed_eventthe last member of the structure. It exists in the form of an array and is a flexible array (struct v4l2_kevent events[]). That is to say, when the v4l2_subscribed_eventstructure space is allocated, events do not occupy space and additional space is required. Allocate space for a specified number of events. kzalloc(siezof(struct v4l2_subscribed_event) + sizeof(struct v4l2_kevent) * num, GFP_KERNEL);When using it, you can address kevents in an array, which is very convenient.
If the number of events obtained is more than kevent, then the old events will be discarded. You can set the merge and replace callback functions of the structure v4l2_subscribed_event(in fact, the default functions are enough), they will be called when the event is captured and there is no more space to store the event. There v4l2_event.cis a good example of replace/merge in , where ctrls_replace() and ctrls_merge() are used as callback functions. Since these two functions can be called in interrupt context, they must execute quickly and return.
The loop about events is an interesting operation. When entering the queue: three variables (first - the next event to be dequeued,
elems - the total number of kevents, in_use - the number of kevents that have been used)If elems == in_use, it means that the queue members have been used up.
Take out the first kevent and delete it from the available queue. first points to the next member of the array, in_use -.
Find the previous step (first points to the next member of the array), merge and assign the changes bits of the previous step (removing the first kevent) to the former.
Because the latter is newer than the former, the value can completely cover the former while retaining the changes in the former.Take out the array kevent item in_use + first >= elems ? in_use + first - elems : in_use +
first; as a new filling item.in_use ++
Some useful functions:
int v4l2_event_subscribe(struct v4l2_fh *fh, struct v4l2_event_subscription *sub,
unsigned elems, const struct v4l2_subscribed_event_ops *ops)
When the user space initiates a subscription request through ioctl, video_device->ioctl_ops->vidioc_subscribe_event
it needs to check whether the requested event is supported. If so, call the above function to subscribe. Generally, you can point the video-related ioctl to the kernel's default v4l2_ctrl_subscribe_event() function.
int v4l2_event_unsubscribe(struct v4l2_fh *fh, struct v4l2_event_subscription *sub)
取消一个事件的订阅，V4L2_EVENT_ALL类型可以用于取消所有事件的订阅。一般可以将video的相关ioctl指向该函数。
void v4l2_event_queue(struct video_device *vdev, const struct v4l2_event *ev)
该函数用作events入队操作（由驱动完成），驱动只需要设置type以及data成员，其余的交由V4L2来完成。
int v4l2_event_dequeue(struct v4l2_fh *fh, struct v4l2_event *event,
int nonblocking)
events出队操作，发生于用户空间的VIDIOC_DQEVENT调用，作用是从available队列中取出一个events。
v4l2_subscribed_event_opsParameters allow the driver to set the following four callback function members:
-
add: Called when adding an event subscription
-
del: called when canceling an event subscription
-
replace: The event is replaced with the old one. It is called when the queue is full, the same below. It is often used to copy the kevent.u.ctrl item when there is only one elems.
-
merge: merge old events into new events. When used for multiple elems, only changes items are merged. For the reason, see the description of the event loop process above.
events are passed to user space through the poll system call, which the driver can use v4l2_fh->waitas poll_wait()a parameter. Sub-devices can send events (using V4L2_DEVICE_NOTIFY_EVENT) directly to v4l2_device through the notify function. drivers/media/platform/omap3ispGives examples of how to use events.
Precautions:
Pay attention to the elems parameter of v4l2_event_subscribe. If it is 0, the kernel will assign it to 1 by default, otherwise it will be assigned according to the specified parameter value.
It is best not to use the kernel's default v4l2_subscribed_event_ops, because its add function will try to find the ctrl of the corresponding id in v4l2_ctrl. If it
is a custom event id, the relevant ctrl item may not be found. In this case, the VIDIOC_SUBSCRIBE_EVENT of the user space will return failure.User space does not need to worry about the return operation after dqevent, because the kernel will automatically obtain the used kevent and use flexible arrays to manage it instead of scattered linked lists.
Subdevices can enqueue an event and notify the v4l2 device's notify callback by calling the v4l2_subdev_notify_event function.
The v4l2_event_queue function will traverse all v4l2_fh on video_device and queue the event into the list of each fh. fh
is generated when the user opens the video device node. Each user will be assigned a separate v4l2_fh when they open the video node.file->private always points to the latest v4l2_fh. Through this v4l2_fh, all elements in the entire v4l2_fh linked list can be found.
The v4l2_fh_release function will unsubscribe all events mounted on the fh.