This article introduces the control interface under the v4l2 framework. These interfaces are usually used to implement some special effects control, menu control, and so on.
1. Introduction
Since video input is involved, there will be many ISP-related effects, such as contrast, saturation, color temperature, white balance, etc. These are all common and necessary controls, and most of them only need to set an integer value. can. V4L2 very intimately provides us with such interfaces for use (it can be said to be very intimate). In the kernel, these control items are abstracted into individual control IDs, which are respectively commanded by V4L2_CID_xxx.
There are many control functions that are not specific to a single driver. These general control APIs can be moved to the V4L2 kernel framework. The problems left to driver developers are just the following points:
- Know what kind of control you need
- How to add a control
- How to set the value of control, you will know the setting interface is s_ctrl later
- How to get the value of control, g_volatile_ctrl
- How to verify whether the user's control value is legal (try_ctrl), the control framework depends on v4l2_device and v4l2_subdev
Control has two main structure objects:
v4l2_ctrl: describes the control attribute, the abstract representation of the control structure, and tracks the value of the control
v4l2_ctrl_handler: Track control, including a'v4l2_ctrl' list, which is a collection of control items you need
V4L2 ctrl module internal implementation
2. How to use
2.1, ready to drive
2.1.1. Add the handler to the top-level structure of the driver
This structure is the structure defined by the driver. Remember this structure will also contain v4l2_device or v4l2_subdev? (There is also a description in the previous article) Yes, that is it, it should also contain the v4l2_ctrl_handler structure, examples are as follows:
struct foo_dev {
...
struct v4l2_ctrl_handler ctrl_handler;
...
};
struct foo_dev *foo;2.1.2, initialize the handler
v4l2_ctrl_handler_init(&foo->ctrl_handler, nr_of_controls);The second parameter indicates the number of controls that you want to create. The function creates a hash table based on this value. This number is only a guide. In fact, the number of controls may not be the same as this number. For'video_device' or'v4l2_subdev' devices, you need to explicitly set their'control_handler' member (you can see it when you go to the kernel code to see it) to point to the'ctrl_handler' of the driver structure, otherwise opening the video node will not Carry out relevant control.
2.1.3, hook the control handler to the driver
1. For the v4l2 driver
struct foo_dev {
...
struct v4l2_device v4l2_dev;
...
struct v4l2_ctrl_handler ctrl_handler;
...
};
foo->v4l2_dev.ctrl_handler = &foo->ctrl_handler;This removes the original v4l2_ctrl_ops Liman's vidioc_queryctrl, vidioc_query_ext_ctrl, vidioc_querymenu, vidioc_g_ctrl, vidioc_s_ctrl, vidioc_g_ext_ctrls, vidioc_try_ext_ctrls and vidioc_ext_ctrls, and these functions are no longer needed after the operation framework.
2. For sub-devices
struct foo_dev {
...
struct v4l2_subdev sd;
...
struct v4l2_ctrl_handler ctrl_handler;
...
};
foo->sd.ctrl_handler = &foo->ctrl_handler;Then set all members in v4l2_subdev_core_ops to point to the helper function.
.queryctrl = v4l2_subdev_queryctrl,
.querymenu = v4l2_subdev_querymenu,
.g_ctrl = v4l2_subdev_g_ctrl,
.s_ctrl = v4l2_subdev_s_ctrl,
.g_ext_ctrls = v4l2_subdev_g_ext_ctrls,
.try_ext_ctrls = v4l2_subdev_try_ext_ctrls,
.s_ext_ctrls = v4l2_subdev_s_ext_ctrls,This is a temporary solution. These functions will no longer be used after all v4l2 drivers that rely on sub-device drivers are converted to use the control framework. That is to say, some v4l2 drivers can still be ctrl controlled through the above callbacks. The above help function is just for a technical secondary school, relocating the callback of the v4l2 driver to point to the ctrl in the control frame. This operation is mainly for compatibility with the old driver, and the new driver should not use this way of writing.
2.1.4, finally clean up the handler
v4l2_ctrl_handler_free(&foo->ctrl_handler);It is said that the cleanup is done when the module is uninstalled or used. Don't clean up as soon as it is initialized. What else is needed.
2.2, add control for v4l2_ctrl_handler
2.2.1, initialization control handler
static const s64 exp_bias_qmenu[] = {
-2, -1, 0, 1, 2
};
static const char * const test_pattern[] = {
"Disabled",
"Vertical Bars",
"Solid Black",
"Solid White",
};
v4l2_ctrl_handler_init(&foo->ctrl_handler, nr_of_controls);
v4l2_ctrl_new_std(&foo->ctrl_handler, &foo_ctrl_ops,
V4L2_CID_BRIGHTNESS, 0, 255, 1, 128);
v4l2_ctrl_new_std(&foo->ctrl_handler, &foo_ctrl_ops,
V4L2_CID_CONTRAST, 0, 255, 1, 128);
v4l2_ctrl_new_std_menu(&foo->ctrl_handler, &foo_ctrl_ops,
V4L2_CID_POWER_LINE_FREQUENCY,
V4L2_CID_POWER_LINE_FREQUENCY_60HZ, 0,
V4L2_CID_POWER_LINE_FREQUENCY_DISABLED);
v4l2_ctrl_new_int_menu(&foo->ctrl_handler, &foo_ctrl_ops,
V4L2_CID_EXPOSURE_BIAS,
ARRAY_SIZE(exp_bias_qmenu) - 1,
ARRAY_SIZE(exp_bias_qmenu) / 2 - 1,
exp_bias_qmenu);
v4l2_ctrl_new_std_menu_items(&foo->ctrl_handler, &foo_ctrl_ops,
V4L2_CID_TEST_PATTERN, ARRAY_SIZE(test_pattern) - 1, 0,
0, test_pattern);
...
if (foo->ctrl_handler.error) {
int err = foo->ctrl_handler.error;
v4l2_ctrl_handler_free(&foo->ctrl_handler);
return err;
}
You can see that there are many kinds of function calls above, and their functions will be described one by one below.
2.2.2, non-menu control items
struct v4l2_ctrl *v4l2_ctrl_new_std(struct v4l2_ctrl_handler *hdl, const struct v4l2_ctrl_ops *ops, u32 id, s32 min, s32 max, u32 step, s32 def);v4l2_ctrl_new_stdThe function will fill all members based on the control ID, but except for min, max, step and default values, they are driver specific (the function will not be filled with values by itself, these need to be specified in the parameters when the function is called). Each drive member of the range size is not fixed, however type, , name(seeflags these three variables may be implemented within the above functions) are global these are, they are cured v4l2 driver core, each ID are It has its own specific type, name, and flags, which do not need to be cared about by the driver module, but are handed over to the V4L2 framework to do it. But we still need to focus on one point, that is flagvariable, there are many types has its own specific characteristics, such as V4L2_CTRL_FLAG_WRITE_ONLYindicating that the ID corresponding control only able to write the following values, can not be read from the user space, V4L2_CTRL_FLAG_VOLATILEit indicates that the value may be The hardware itself is changed, it is also read-only and will be ignored every time it is written. These are very important to our programming. We have to figure out what type of our control is (especially when we need a custom control) . After the function call associated control ctrl initial value corresponding to the ID is set to defthe value specified by the argument. If you don't know it, it will cause the naming to call the relevant ctrl, but the kernel driver cannot receive the message.
2.2.3, menu control items
struct v4l2_ctrl *v4l2_ctrl_new_std_menu(struct v4l2_ctrl_handler *hdl, const struct v4l2_ctrl_ops *ops, u32 id, s32 max, s32 skip_mask, s32 def);v4l2_ctrl_new_std_menuAnd function v4l2_ctrl_new_stdlike, except that the above menu control, the menu for control, the default is 0 min. If the skip_maskfirst X bit is 1, then the X-th added menu items will be skipped.
2.2.4, drive specific menu control items
struct v4l2_ctrl *v4l2_ctrl_new_std_menu_items(
struct v4l2_ctrl_handler *hdl,
const struct v4l2_ctrl_ops *ops, u32 id, s32 max,
s32 skip_mask, s32 def, const char * const *qmenu);This function is v4l2_ctrl_new_std_menuquite similar, it has an additional parameter QMenu, which is driven MENU particular, it is a two-dimensional array, each array is a string, which is the actual value of the pull-down menu items corresponding to the inside . camif-capture.cIt is a good reference routine.
2.2.5, drive specific integer menu control items
struct v4l2_ctrl *v4l2_ctrl_new_int_menu(struct v4l2_ctrl_handler *hdl, const struct v4l2_ctrl_ops *ops, u32 id, s32 max, s32 def, const s64 *qmenu_int);This function creates a standard drive in accordance with the particular item type Integer menu controls, the last parameter of the function qmenu_intis a 64-bit integer with a sign menu item list. This function is very similar to the previous one, except that one of the contents of the menu is a string and the other is an integer.
All the above functions are generally v4l2_ctrl_handler_initinvoked after all the function returns a v4l2_ctrlpointer of type, if you need to store related ctrl, then you can get the return value of the function of storing them. Notably, when the function call error, or an error code returns NULL, and set ctrl_handler->errorpoint error code, the error code is directed ctrl_handlerctrl first list of error code error ctrl. The pair v4l2_ctrl_handler_initalso apply for the function.
v4l2_ctrl_cluster The function can merge a certain number of ctrls in the specified ctrl list, and all merged control IDs are pointed to the first control ID item in the merged list, that is, once the IDs are merged, no matter which ID you execute, As long as it is in the merged ID list, the final control ID to be executed is the control item where the first ID in the list is located. It is often used when multiple controls are completed by the same hardware.
For example: Assume that in most hardware, brightness and color temperature are controlled by two modules, and their values are also inconsistent. At this time, there is no need to merge, just control each other, and some control The controller will bind the brightness and color temperature, which are controlled by the same module. There is a comparison table (one-to-one correspondence) between brightness and color temperature. At this time, the two can be combined. After the combination, only one of the controls needs to be executed. . Then why don’t you just use one of them if it’s the same, because some applications are controlled separately, but ported to a new hardware platform, I changed the application to the troublesome (especially business logic), so I went in the driver Adaptation is the best.
2.2.6, optional mandatory initialization control settings
v4l2_ctrl_handler_setup(&foo->ctrl_handler);This function will call s_ctrl for all controls, and is generally used to set the hardware to the default configuration. If you need to synchronize the hardware and the internal data structure, you can run this function to complete the synchronization operation. It is often used to prevent the initialization value of the hardware from being inconsistent with the initialization default value specified by the control ID, such as setting the default when the system is just started. The value is used to initialize the effect of the entire system, which is very useful.
2.2.7, implement v4l2_ctrl_ops structure
static const struct v4l2_ctrl_ops foo_ctrl_ops = {
.s_ctrl = foo_s_ctrl,
};Under normal circumstances, you need to set the s_ctrl member function as:
static int foo_s_ctrl(struct v4l2_ctrl *ctrl)
{
struct foo *state = container_of(ctrl->handler, struct foo, ctrl_handler);
switch(ctrl->id){
case V4L2_CID_BRIGHTNESS:
write_reg(0x123, ctrl->val);
break;
case V4L2_CID_CONTRAST:
write_reg(0x456, ctrl->val);
break;
}
return 0;
}It should be noted that once the entire ctrl process enters the s/g_ctrl callback function, it means that the passed ctrl value is correct (provided that the value range is set correctly when ctrl is initialized), and the control framework will be based on the initial setting The value is used to judge the reasonableness of the value. If it can be called inside the function, then only the register operation is performed, and it does not matter whether the value is correct or not (it is definitely correct). If the ctrl value of this time is consistent with the last time, the kernel related module will directly return success before entering the callback function. If the value range is not correct, the value will be re-adjusted according to the principle of proximity.
2.3, add custom and standard ctrl
2.3.1, standard ctrl
1. Implement the v4l2_ctrl_ops structure
static const struct v4l2_ctrl_ops foo_ctrl_ops = {
.s_ctrl = foo_s_ctrl,
};Under normal circumstances, you need to set the s_ctrl member function as:
static int foo_s_ctrl(struct v4l2_ctrl *ctrl)
{
struct foo *state = container_of(ctrl->handler, struct foo, ctrl_handler);
switch (ctrl->id) {
case V4L2_CID_BRIGHTNESS:
write_reg(0x123, ctrl->val);
break;
case V4L2_CID_CONTRAST:
write_reg(0x456, ctrl->val);
break;
}
return 0;
}
2. Add to ctrl_handler
v4l2_ctrl_handler_init(&foo->ctrl_handler, nr_of_controls);
v4l2_ctrl_new_std(&foo->ctrl_handler, &foo_ctrl_ops, V4L2_CID_BRIGHTNESS, 0, 255, 1, 128);
v4l2_ctrl_new_std(&foo->ctrl_handler, &foo_ctrl_ops, V4L2_CID_CONTRAST, 0, 255, 1, 128);2.3.2, custom ctrl
1. Add a custom v4l2_ctrl_config configuration structure
static const struct v4l2_ctrl_config ctrl_filter = {
.ops = &foo_ctrl_ops,
.id = V4L2_CID_CUSTOM_ID,
.name = "Spatial Filter",
.type = V4L2_CTRL_TYPE_INTEGER,
.flags = V4L2_CTRL_FLAG_SLIDER, //必要的
.max = 15,
.step = 1,
};
ctrl = v4l2_ctrl_new_custom(&foo->ctrl_handler, &ctrl_filter, NULL);The menu type, common type, and ctrl of different flags can all be configured through the v4l2_ctrl_config structure.
2.3.3, user space usage
struct v4l2_control ctrl;
ctrl.id = V4L2_CID_CUSTOM_ID;
ctrl.value = 200;
ioctl(fd, VIDIOC_S_CTRL, &ctrl);VIDIOC_S_CTRL: This ioctl call may (related to the flag and value of ctrl, if the value is consistent with the last time, then it will return directly) will cause the s_ctrl member of foo_ctrl_ops to be called, and other member functions have corresponding call IDs, ctrl The id member of should be filled with the ctrl item that you want to control, and value is the value of the corresponding item of ctrl.
For device nodes of type /dev/videoX, the order of v4l2_fh.ctrl_handler->video_device.ctrl_handler->v4l2_ioctl_ops.vidioc_s_ctrl->v4l2_ioctl_ops.vidioc_s_ext_ctrls will be followed to find out whether there are related calls available. For nodes of sub-device type (/dev/v4l-subdevX), v4l2_fh.ctrl_handler is automatically selected for control operations. This is implemented in the sub-device Framework, and the driver writer does not need to care about the internal implementation.
Three, extension
1. Inheritance of ctrl
When a child device to be registered ( v4l2_device_register_subdev), at the same time v4l2_deviceas well as v4l2_subdevthe ctrl_handlermembers are set, then the drive can also be used to V4L2 ctrl child devices, and if they have to repeat the words of ctrl, ctrl child device will be skipped ( For the V4L2 master device driver, the child device will still execute its own ctrl), which is equivalent to a bottom-up inheritance relationship from the child device to the V4L2 parent device. In the sub-device registration function v4l2_ctrl_add_handler()will be called to ctrl adding a child device to device V4L2 among the (child device can not share access to each other). If you want to between sub-device sharing ctrl_handler, you can explicitly call the v4l2_ctrl_add_handlerfunction to complete. Set the ctrl is_privatebit to 1 to avoid ctrl_handlerthe merger leads to ctrl share.
2. Access value
v4l2_ctrlThe structure saves the current and new values for the driver for comparison, and only when appropriate can the value be successfully transferred from the user space to the kernel driver. In the v4l2_ctrlstructure which has the following members:
s32 val;
struct {
s32 val;
} cur;
union v4l2_ctrl_ptr p_new;
union v4l2_ctrl_ptr p_cur;
Several members of the old and new values for the corresponding stored control, if the control value is a simple 32-bit signed integer, then the correspondence relationship between them as follows (only the s_ctrlvalue returned is 0 only when the new core module ctrl provided Copy to the cache member, the execution failure of course does not need to be saved):
&ctrl->val == ctrl->p_new.p_s32
&ctrl->cur.val == ctrl->p_cur.p_s32Guess what, if the inside of the old and new values have been saved, then g_volatile_ctrlthere is no need to implement the (because when I need to read the relevant values I pick up data directly from the cache value inside just fine). The implementation of this function is specifically used for ctrl whose value may change due to hardware reasons (so named g_volatile_ctrl). At this time, you need to set the flag attribute of ctrl to a variable state: flag |= V4L2_CTRL_FLAG_VOLATILE;all of them come from user space. All write operations will be ignored. It indicates that the value of the relevant ctrl may be changed by the hardware itself, so every time you get ctrl, you should call the relevant callback function obediently, read the value of the register in the callback function, and return the newly read value again. Normally, set to ctrl volatile are read-only, but can still g_volatile_ctrlgo visit value cache structure members inside the callback function (this visit was initiated by the drive itself), should be noted that, if a ctrl is volatile at the same time not read-only, then it will not produce V4L2_EVENT_CTRL_CH_VALUEthe event message.
You can use the following drive inside the function to set the value of the relevant ctrl (remember do not v4l2_ctrl_opsuse the following internal callback function, otherwise it will create a deadlock, because of their own access to those callbacks when he locked up, call the following again Function will cause repeated locking):
s32 v4l2_ctrl_g_ctrl(struct v4l2_ctrl *ctrl);
int v4l2_ctrl_s_ctrl(struct v4l2_ctrl *ctrl, s32 val);3. Menu control
The v4l2_ctrl structure has the following members:
union{
u32 step;
u32 menu_skip_mask;
};For menu control, the use of the union menu_skip_maskmembers, the member specifies which menu items will be skipped, if it is 0, it indicates that all menu items are available, it is worth noting that VIDIOC_QUERYCTRLcall will never return step 1. A good example is the bit rate menu control of MPEG Audio Layer II. In fact, most of the hardware only uses part of it.
4. Activate and lock control
-
In actual use, some controls have a certain strong correlation. For example, when the Chroma AGC control is turned on, Chroma Gain is invalid (the hardware level is mutually exclusive). When this happens, if the user continues Set the related control of Chroma Gain. Although the software level can be set, the hardware cannot respond. At this time, the related control of Chroma Gain needs to be disabled, which is mutual exclusion. The kernel can call the
v4l2_ctrl_activate()function to send an activation / deactivation signal, noted that the control framework and not to check this flag, this is only for the GUI level, when the function is called, the user will receive an event space, through this event, The GUI can perform related actions. This function is typically useds_ctrlcallback. The flag is'active'
Another flag bit is'grabbed', which means locked. For example, in MPEG, the bit rate cannot be changed during the transmission of a video stream. That is to say, the related control during the video transmission should be locked. use v4l2_ctrl_grab()function to set the lock / unlock the associated controls, control framework checks this flag, if set, then the call will return control -EBUSYinformation, usually in the open stream function on / off function is called.
5. Add a callback to ctrl
void v4l2_ctrl_notify(struct v4l2_ctrl *ctrl, void (*notify)(struct v4l2_ctrl *ctrl, void *priv), void *priv);This function can be added as a ctrl callback when the value associated ctrl is changed, the correction will have an effect, a ctrl can only add a callback, if you try to add more than ctrl case, lead WARN_ONgeneration. Note that when the callback is generated, the ctrl lock is still held, and will not be unlocked until the related s_ctrl returns.
6. Combined use with V4L2 event
- Kernel initialization of the control module sets the initial value for each ctrl, and produces one of a ctrl each
V4L2_EVENT_CTRLevent. After that will have a success after each set of values corresponding ctrlV4L2_EVENT_CTRLevent, the content of the event included ctrl ID and its value. So what's the use of this thing? For example, if you have two processes, one process controls the ISP effect, and the other process needs to make different responses according to different ISP effects. Then you can subscribe to this type of event in the other process, and each time the value changes At that time, you can perform corresponding actions based on the relevant ctrl value.
The introduction of the V4L2 event is described at the end of the first article: 00-Overview of the V4L2 framework .
Up to this article, basically the driver part of the V4L2 framework has been introduced almost, as well as the user space operation level, such as what sequence should be used to open and close the data stream, and how to use the control correctly and normatively , How to use it with events, and how to combine these modules in a specific case for unified planning and programming. Although there is no special article to introduce these so far, after reading the previous articles, I believe that there is a general framework for use. Whether the subsequent operation of the application space will be updated is to be determined, and the probability is not.