V4l2
在linux下面有一组标准的API用于摄像头的采集，这个是一个采集框架叫做V4L（Video for Linux）,有大量的摄像头支持V4L框架，很多应用也支持通过该框架采集视频流。
树莓派的Raspberry Pi OS默认支持V4L2框架，名字叫做bcm2835-v4l2，可以通过v4l2-ctl工具集测试验证。
获取设备列表
martin@raspberrypi:~ $ v4l2-ctl --list-devices
bcm2835-codec-decode (platform:bcm2835-codec):
/dev/video10
/dev/video11
/dev/video12
/dev/video18
/dev/video31
/dev/media1
bcm2835-isp (platform:bcm2835-isp):
/dev/video13
/dev/video14
/dev/video15
/dev/video16
/dev/video20
/dev/video21
/dev/video22
/dev/video23
/dev/media0
/dev/media2
mmal service 16.1 (platform:bcm2835-v4l2-0):
/dev/video0
驱动信息
martin@raspberrypi:~ $ v4l2-ctl -d /dev/video0 --all
Driver Info:
Driver name : bm2835 mmal
Card type : mmal service 16.1
Bus info : platform:bcm2835-v4l2-0
Driver version : 5.15.32
Capabilities : 0x85200005
Video Capture
Video Overlay
Read/Write
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x05200005
Video Capture
Video Overlay
Read/Write
Streaming
Extended Pix Format
Priority: 2
Video input : 0 (Camera 0: ok)
Format Video Capture:
Width/Height : 1920/1080
Pixel Format : 'MJPG' (Motion-JPEG)
Field : None
Bytes per Line : 0
Size Image : 2088960
Colorspace : SMPTE 170M
Transfer Function : Default (maps to Rec. 709)
YCbCr/HSV Encoding: Default (maps to ITU-R 601)
Quantization : Default (maps to Full Range)
Flags :
Format Video Overlay:
Left/Top : 150/50
Width/Height: 1024/768
Field : None
Chroma Key : 0x00000000
Global Alpha: 0xff
Clip Count : 0
Clip Bitmap : No
Framebuffer Format:
Capability : Extern Overlay
Global Alpha
Flags : Overlay Matches Capture/Output Size
Width : 1920
Height : 1088
Pixel Format : 'YU12'
Streaming Parameters Video Capture:
Capabilities : timeperframe
Frames per second: 30.000 (30000/1000)
Read buffers : 1
User Controls
brightness 0x00980900 (int) : min=0 max=100 step=1 default=50 value=50 flags=slider
contrast 0x00980901 (int) : min=-100 max=100 step=1 default=0 value=0 flags=slider
saturation 0x00980902 (int) : min=-100 max=100 step=1 default=0 value=0 flags=slider
red_balance 0x0098090e (int) : min=1 max=7999 step=1 default=1000 value=1000 flags=slider
blue_balance 0x0098090f (int) : min=1 max=7999 step=1 default=1000 value=1000 flags=slider
horizontal_flip 0x00980914 (bool) : default=0 value=0
vertical_flip 0x00980915 (bool) : default=0 value=0
power_line_frequency 0x00980918 (menu) : min=0 max=3 default=1 value=1
0: Disabled
1: 50 Hz
2: 60 Hz
3: Auto
sharpness 0x0098091b (int) : min=-100 max=100 step=1 default=0 value=0 flags=slider
color_effects 0x0098091f (menu) : min=0 max=15 default=0 value=0
0: None
1: Black & White
2: Sepia
3: Negative
4: Emboss
5: Sketch
6: Sky Blue
7: Grass Green
8: Skin Whiten
9: Vivid
10: Aqua
11: Art Freeze
12: Silhouette
13: Solarization
14: Antique
15: Set Cb/Cr
rotate 0x00980922 (int) : min=0 max=360 step=90 default=0 value=0 flags=modify-layout
color_effects_cbcr 0x0098092a (int) : min=0 max=65535 step=1 default=32896 value=32896
Codec Controls
video_bitrate_mode 0x009909ce (menu) : min=0 max=1 default=0 value=0 flags=update
0: Variable Bitrate
1: Constant Bitrate
video_bitrate 0x009909cf (int) : min=25000 max=25000000 step=25000 default=10000000 value=10000000
repeat_sequence_header 0x009909e2 (bool) : default=0 value=0
force_key_frame 0x009909e5 (button) : flags=write-only, execute-on-write
h264_minimum_qp_value 0x00990a61 (int) : min=0 max=51 step=1 default=0 value=0
h264_maximum_qp_value 0x00990a62 (int) : min=0 max=51 step=1 default=0 value=0
h264_i_frame_period 0x00990a66 (int) : min=0 max=2147483647 step=1 default=60 value=60
h264_level 0x00990a67 (menu) : min=0 max=13 default=11 value=11
0: 1
1: 1b
2: 1.1
3: 1.2
4: 1.3
5: 2
6: 2.1
7: 2.2
8: 3
9: 3.1
10: 3.2
11: 4
12: 4.1
13: 4.2
h264_profile 0x00990a6b (menu) : min=0 max=4 default=4 value=4
0: Baseline
1: Constrained Baseline
2: Main
4: High
Camera Controls
auto_exposure 0x009a0901 (menu) : min=0 max=3 default=0 value=0
0: Auto Mode
1: Manual Mode
exposure_time_absolute 0x009a0902 (int) : min=1 max=10000 step=1 default=1000 value=1000
exposure_dynamic_framerate 0x009a0903 (bool) : default=0 value=0
auto_exposure_bias 0x009a0913 (intmenu): min=0 max=24 default=12 value=12
0: -4000 (0xfffffffffffff060)
1: -3667 (0xfffffffffffff1ad)
2: -3333 (0xfffffffffffff2fb)
3: -3000 (0xfffffffffffff448)
4: -2667 (0xfffffffffffff595)
5: -2333 (0xfffffffffffff6e3)
6: -2000 (0xfffffffffffff830)
7: -1667 (0xfffffffffffff97d)
8: -1333 (0xfffffffffffffacb)
9: -1000 (0xfffffffffffffc18)
10: -667 (0xfffffffffffffd65)
11: -333 (0xfffffffffffffeb3)
12: 0 (0x0)
13: 333 (0x14d)
14: 667 (0x29b)
15: 1000 (0x3e8)
16: 1333 (0x535)
17: 1667 (0x683)
18: 2000 (0x7d0)
19: 2333 (0x91d)
20: 2667 (0xa6b)
21: 3000 (0xbb8)
22: 3333 (0xd05)
23: 3667 (0xe53)
24: 4000 (0xfa0)
white_balance_auto_preset 0x009a0914 (menu) : min=0 max=10 default=1 value=1
0: Manual
1: Auto
2: Incandescent
3: Fluorescent
4: Fluorescent H
5: Horizon
6: Daylight
7: Flash
8: Cloudy
9: Shade
10: Greyworld
image_stabilization 0x009a0916 (bool) : default=0 value=0
iso_sensitivity 0x009a0917 (intmenu): min=0 max=4 default=0 value=0
0: 0 (0x0)
1: 100000 (0x186a0)
2: 200000 (0x30d40)
3: 400000 (0x61a80)
4: 800000 (0xc3500)
iso_sensitivity_auto 0x009a0918 (menu) : min=0 max=1 default=1 value=1
0: Manual
1: Auto
exposure_metering_mode 0x009a0919 (menu) : min=0 max=3 default=0 value=0
0: Average
1: Center Weighted
2: Spot
3: Matrix
scene_mode 0x009a091a (menu) : min=0 max=13 default=0 value=0
0: None
8: Night
11: Sports
JPEG Compression Controls
compression_quality 0x009d0903 (int) : min=1 max=100 step=1 default=30 value=30
支持格式
martin@raspberrypi:~ $ v4l2-ctl --list-formats
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture
[0]: 'YU12' (Planar YUV 4:2:0)
[1]: 'YUYV' (YUYV 4:2:2)
[2]: 'RGB3' (24-bit RGB 8-8-8)
[3]: 'JPEG' (JFIF JPEG, compressed)
[4]: 'H264' (H.264, compressed)
[5]: 'MJPG' (Motion-JPEG, compressed)
[6]: 'YVYU' (YVYU 4:2:2)
[7]: 'VYUY' (VYUY 4:2:2)
[8]: 'UYVY' (UYVY 4:2:2)
[9]: 'NV12' (Y/CbCr 4:2:0)
[10]: 'BGR3' (24-bit BGR 8-8-8)
[11]: 'YV12' (Planar YVU 4:2:0)
[12]: 'NV21' (Y/CrCb 4:2:0)
[13]: 'RX24' (32-bit XBGR 8-8-8-8)
抓取JPEG图片
martin@raspberrypi:~ $ v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=3 && \
> v4l2-ctl --stream-mmap=3 --stream-count=1 --stream-to=1.jpg
<
录制264
martin@raspberrypi:~ $ v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=4 && v4l2-ctl --stream-mmap=3 --stream-count=100 --stream-to=1.h264
K<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.02 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<K< 30.02 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.02 fps
<<<<<<<<
FFMPEG
Pi预编译的ffmpeg已经支持依赖OpenMAX IL H264硬编解码。
sudo apt install -y ffmpeg
在树莓派上手动编译ffmpeg参考文章Compile FFmpeg with Hardware Accelerator
列出支持编码器
martin@raspberrypi:~ $ ffmpeg -encoders | grep -E "264|mjpeg"
V..... libx264 libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (codec h264)
V..... libx264rgb libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 RGB (codec h264)
V..... h264_omx OpenMAX IL H.264 video encoder (codec h264)
V..... h264_v4l2m2m V4L2 mem2mem H.264 encoder wrapper (codec h264)
V..... h264_vaapi H.264/AVC (VAAPI) (codec h264)
VFS... mjpeg MJPEG (Motion JPEG)
V..... mjpeg_vaapi MJPEG (VAAPI) (codec mjpeg)
编码器支持的格式
martin@raspberrypi:~ $ ffmpeg -h encoder=h264_omx
Encoder h264_omx [OpenMAX IL H.264 video encoder]:
General capabilities: delay
Threading capabilities: none
Supported pixel formats: yuv420p
h264_omx AVOptions:
-omx_libname <string> ED.V...... OpenMAX library name
-omx_libprefix <string> ED.V...... OpenMAX library prefix
-zerocopy <int> E..V...... Try to avoid copying input frames if possible (from 0 to 1) (default 0)
-profile <int> E..V...... Set the encoding profile (from -99 to 100) (default -99)
baseline 66 E..V......
main 77 E..V......
high 100 E..V......
参考文献：Set up camera