Use of multimedia modules in Qt

Introduction to classes in Qt modules

The use of cameras in Qt is in the Qt Multimedia module. Qt Multimedia is an important module that provides a rich set of QML types and C++ classes to handle multimedia content. It also provides the APIs required to access camera and radio functionality. The included Qt audio engine provides types for 3D positional audio playback and content management. Multimedia support in Qt is provided by modules. The Qt Multimedia Module provides a rich feature set that makes it easy to take advantage of the platform's multimedia capabilities, such as media playback and the use of cameras and radios.

window platform:

Implementing Qt multimedia functionality for Windows is implemented in two plug-ins; one uses the Microsoft DirectShowAPI and the other uses the WMF (Windows Media Foundation) framework. The DirectShow API was introduced in Windows 98 and gradually deprecated starting with Windows XP. The Media Foundation framework was introduced in Windows Vista as a replacement for DirectShow and other multimedia APIs. Therefore, the WMF plug-in in Qt only supports Windows Vista and higher operating systems. Environment variables can be used to control plugin priority. For example, setting this to "windowsmediafoundation" or "directshow" will cause the corresponding plugin to become the preferred plugin.

QT_MULTIMEDIA_PREFERRED_PLUGINS

Window platform limitations:

The WMF plugin in Qt currently does not provide a camera backend. Instead, limited support for camera functionality is provided by the DirectShow plug-in. Basic functions such as displaying the viewfinder and capturing still images are supported, however, most camera controls are not implemented. Video recording is not currently supported. Additionally, the DirectShow plug-in does not support any low-level video functionality, such as monitoring video frames being played or recorded using or related classes.

Class introduction

Audio
QAbstractAudioDeviceInfo	Base class for audio backend
QAudioDeviceInfo	Interface for querying audio devices and their capabilities
QAudio	Contains enumerations used by audio classes
QAudioRecorder	QAudioRecorder class is used to record audio
QAudioEncoderSettings	The QAudioEncoderSettings class provides a set of audio encoder settings.
QAudioProbe	Allows you to monitor audio being played or recorded
QAudioBuffer	Represents a collection of audio samples with a specific format and sample rate
QAudioBuffer::StereoFrame	QAudioDecoder, a simple wrapper around stereo audio frames allowing decoding of audio
QAudioFormat	Store audio stream parameter information
QAudioInput	Interface for receiving audio data from audio input devices
QAudioOutput	Interface for sending audio data to an audio output device
QAudioSystemPlugin	Abstract base for audio plug-ins
QSound	How to play .wav sound files
QSoundEffect	How to play low-latency sound effects
video
QAbstractPlanarVideoBuffer	Abstraction of flat video data
QAbstractVideoBuffer	Abstraction of video data
QAbstractVideoFilter	Represents a filter to be applied to video frames received by the VideoOutput type
QAbstractVideoSurface	Base class for video presentation surfaces
QVideoFilterRunnable	Represents an implementation of a filter that owns all graphics and computational resources and performs the actual filtering or calculations QVideoFrame represents a frame of video data
QVideoProbe	Allows you to monitor video frames being played or recorded
QVideoSurfaceFormat	Specify the streaming format of the video presentation surface
camera
QCameraInfo	General information about camera equipment
QCamera	System camera device interface
QCamera::FrameRateRange	Frame rate range represents the frame rate range, including minimum and maximum rates
QCameraExposure	Exposure related camera settings interface
QCameraFocus	Interface for focus and zoom related camera settings
QCameraFocusZone	Information about the area used for camera autofocus
QCameraViewfinderSettings	Viewfinder settings set
QCameraImageProcessing	Image processing related camera settings interface
QCameraImageCapture	For recording media content
QCameraViewfinder	The QCameraViewfinder class provides a camera viewfinder widget.
QImageEncoderSettings	Image encoder settings set
multimedia
QMediaBindableInterface	Base class for objects that extend the functionality of media objects
QMediaContent	Access resources related to media content
QMediaObject	A common foundation for multimedia objects
QMediaPlayer	Allow playback of media sources
QMediaPlaylist	List of media content to play
QMediaRecorder	For recording media content
QMediaTimeInterval	Represents a time interval with integer precision
QMediaTimeRange	Represents a set of zero or more disjoint time intervals

Multimedia in Qt mainly includes camera call, video measurement storage (not supported under window), video playback, photo taking, microphone call, audio storage, and audio playback.

Use of webcam (camera)

The use of the camera in QT is mainly divided into three aspects, displaying the screen, capturing pictures and video recording. These three aspects correspond to the three modes of the camera module. The pattern is as follows:

Constant	Value	Description
QCamera::CaptureViewfinder	0	The camera is configured to display the viewfinder only.
QCamera::CaptureStillImage	0x01	The camera is configured for still frame capture
QCamera::CaptureVideo	0x02	The camera is configured for video capture.

These three modes are based on QCamera objects. The operations before QCamera are the same, such as getting the camera in the device, etc.

The viewfinder mode of the camera in QT. Personally, I understand the viewfinder mode to simply display the camera screen into the interface. This implementation is very simple and requires only two steps.

1. Get the camera devices in the system. Here we use the availableCameras function in QCameraInfo . This function returns a device list.

2. Use QCamera to open the camera and add it to the window to be displayed. It should be noted here that the window that displays the camera is the QCameraViewfinder window, and the QWidget can be promoted to QCameraViewfinder.

code show as below:

QList<QCameraInfo>  camerainfos  = QCameraInfo::availableCameras();//获取系统中摄像头
foreach (QCameraInfo info, camerainfos) { //通过遍历摄像头列表，将器添加到下拉控件中
    ui->comboBox->addItem(info.deviceName());
}
.....
camera = new QCamera(ui->comboBox->currentText().toUtf8());//选在摄像头设备，并构造QCamera对象。
camera->setViewfinder(ui->widget); //设置摄像头的显示窗体
camera->start();//启动摄像，到这里界面就可以看到摄像头中的画面了
.....
camera->stop();//停止摄像，摄像头画面将会暂停

但是在取景器模式中我们可以有两个设置可以进行画面的调整，一个是设置显示窗口可以设置显示界面的亮度、对比度、是否全屏显示、色调、饱和度等都可以通过QCameraViewfinder的函数进行调整。另一个是设置QCamera取景器的配置，包括（设置最大帧率、设置最小帧率、设置像素长宽比、设置像素格式等）。QCamera还可以设置曝光和聚焦，通过QCameraExposure和QCameraFocus。

摄像头的使用首先是肯定是显示画面，但是除了显示画面我们可能还需要保存视频和从视频中获取图片。

QT中在视频中抓取图片的步骤：抓取图片的时候需要使用都QCameraImageCapture类

1、创建QCameraImageCapture对象，传入的参数为QCamera。

2、设置相机模式为拍照模式 m_camera->setCaptureMode(QCamera::CaptureStillImage);

3、QCameraImageCapture对象调用capture函数进行图片的保存，传入参数为保存路径。

代码如下：

    QString path = QFileDialog::getSaveFileName(nullptr, "保存图片",
                                                nullptr, "*.jpg");
    if(m_imageCapture == nullptr){
         m_imageCapture = new QCameraImageCapture(m_camera);
    }
    //设置为拍照模式
    m_camera->setCaptureMode(QCamera::CaptureStillImage);
    //对相机锁定(异步操作)
    m_camera->searchAndLock();
    //拍照并且把照片保存到这个路径中
    m_imageCapture->capture(path);
    //解锁
    m_camera->unlock();

	//需要注意的是QCamera已经初始化配置过摄像头了

在拍照模式的时候保存的图片可以设置相关参数，出去QCamera设置的参数，图片的参数通过QCameraImageCapture类的setEncodingSettings函数设置，参数包括（图像质量、分辨率、编码器、编码等）。

QT在window中不能进行摄像头视频的保存，因此这里不多说，简单的说一下保存摄像视频需要使用到的类。

音频的使用

对于音频我们在使用的过程中一般是录制音频、播放音频。针对这两个需求介绍Qt中音频的使用。

Qt中音频的录制

步骤：

1、获取系统中的音频设备。

2、创建QAudioRecorder对象，指定使用的音频设备，通过QAudioRecorder的setAudioInput函数设置。

3、设置音频的保存位置，通过QAudioRecorder对象的setOutputLocation函数设置。

4、设置保存的音频格式，通过QAudioRecorder对象的setAudioSettings函数设置，设置参数为QAudioEncoderSettings。

5、设置保存音频的容器格式，通过QAudioRecorder对象的setContainerFormat。

6、以上参数设置完成之后就可以QAudioRecorder对象调用record开始录制了，如果不设置相关才是，会使用系统默认的参数。

7、AudioRecorder对象调用stop()结束录制。

8、AudioRecorder对象调用pause()暂停录制。

代码如下：

QList<QAudioDeviceInfo>  audioDevices  = QAudioDeviceInfo::availableDevices(QAudio::AudioInput);//获取音频的输入设备
foreach (QAudioDeviceInfo device, audioDevices) {
    ui->comboBox_sound->addItem(device.deviceName());
}
......
m_audioRecorder = new QAudioRecorder(this);
QString datatime = QDateTime::currentDateTime().toString("yyyy-MM-dd-hh-mm-ss");
m_audioName = QString("%2/audio/%3").arg(savePath).arg(datatime);
m_audioRecorder->setOutputLocation(QUrl::fromLocalFile(m_audioName));
m_audioRecorder->setAudioInput(audioDeviceName);
QAudioEncoderSettings set;

set.setCodec("audio/wav");   // 这些是QAudioRecorder是设置，见名思意
set.setQuality(QMultimedia::EncodingQuality::HighQuality);
set.setEncodingMode(QMultimedia::ConstantQualityEncoding);

m_audioRecorder->setContainerFormat("audio/wav");  // 设置容器格式
m_audioRecorder->setAudioSettings(set);
m_audioRecorder->record();//开始录制
......
m_audioRecorder->pause();//暂停录制
......
m_audioRecorder->stop();//结束录制

以上可以完成最基本的音频录制，但是如果想使用更高端的录制，例如在录制的时候显示音频的想干信息。要想实现以上描述，需要使用Qt提供的QAudioProbe类来监控音频数据。

实现步骤如下：

1、创建QAudioProbe对象。

2、QAudioProbe对象通过调用setSource函数设置监控对象，传入参数为AudioRecorder对象。这里QAudioProbe对象监控的就是AudioRecorder对象对象输入的音频。

3、实现QAudioProbe::audioBufferProbed信号槽函数，这个信号是当在媒体服务中处理音频缓冲器时发出该。信号会传递一个QAudioBuffer 格式的音频数据。

4、实现槽函数接收QAudioBuffer 类型的数据，之后通过对接收到数据的计算实现频谱或者其他的显示。

代码如下：

    m_audioRecordProbe = new QAudioProbe(this);
    m_audioRecorder = new QAudioRecorder(this);

    //显示
    m_audioRecordProbe->setSource(m_audioRecorder);  // 指定声源

    connect(m_audioProbe, &QAudioProbe::audioBufferProbed, this,[=](const QAudioBuffer &buffer){
        emit audioProcessBuff(buffer);
    });

Qt中音频的播放

步骤：

1、创建一个QMediaPlayer对象。

2、创建QMediaPlaylist对象。

3、通过QMediaPlaylist对象的setPlaybackMode函数创建列表的循环模式。

Constant	Value	Description
QMediaPlaylist::CurrentItemOnce	0	当前项目只播放一次。
QMediaPlaylist::CurrentItemInLoop	1	当前项目在循环中重复播放
QMediaPlaylist::Sequential	2	播放从当前开始，在每个连续项目中移动，直到到达最后一个项目，然后停止。当前正在播放上一个项目时，下一个项目为空项目。
QMediaPlaylist::Loop	3	播放在最后一个项目播放完毕后的第一个项目处重新开始。
QMediaPlaylist::Random	4	按随机顺序播放项目

4、 QMediaPlayer对象通过函数setPlaylist设置播放列表，参数为QMediaPlaylist对象。

5、向播放列表中添加需要播放的音频，通过QMediaPlaylist对象addMedia函数添加音频列表。

6、在音频列表中选择需要播放的音频，使用QMediaPlaylist对象的setCurrentIndex函数，参数为音频标号。

7、QMediaPlayer对象调用play函数播放音频。

8、QMediaPlayer对象调用pause函数暂停播放音频。

9、QMediaPlayer对象调用stop函数停止播放音频。

代码如下：

m_mediaPlayer = new QMediaPlayer(this);
m_mediaPlaylist = new QMediaPlaylist(this);
m_mediaPlaylist->setPlaybackMode(QMediaPlaylist::CurrentItemInLoop);
m_mediaPlayer->setPlaylist(m_mediaPlaylist);
......
m_mediaPlaylist->addMedia(QUrl::fromLocalFile(filePath));
......
m_mediaPlaylist->setCurrentIndex(index);
......
m_mediaPlayer->play();
......
m_mediaPlayer->pause();
......
m_mediaPlayer->stop();

QMediaPlayer除了播放、暂停、停止播放外，还有其他设置，例如设置声音大小，快进、快退等。都可以通过QMediaPlayer对象的函数进行设置。也可以通过QAudioProbe对象对播放的音频进行监听，实现播放音频声波的可视化。方法和录制一样。

通过QSound类播放音频：暂不介绍

视频的播放

Qt多媒体也支持视频的播放，视频的播放相对来说比较简单。

步骤如下：

1、创建一个QMediaPlayer对象。

2、设置显示UI窗口。

3、设置需要播放的视频路径。

4、QMediaPlayer对象调用play函数播放音频。

5、QMediaPlayer对象调用pause函数暂停播放音频。

6、QMediaPlayer对象调用stop函数停止播放音频。

代码如下：

    m_mediapPlay = new QMediaPlayer(this);
    m_mediapPlay->setNotifyInterval(1000);
    m_mediapPlay->setVideoOutput(ui->mainwidget);
    ......
    m_mediapPlay->setMedia(QUrl::fromLocalFile(m_filePath));
    m_mediapPlay->play();
    m_mediapPlay->pause();

音频和视频播放都使用的是QMediaPlayer对象，只不过是视频多设置了一个视频输出窗口。通过QMediaPlayer类的一些信号和槽可以获取到当前播放影视频的大小，播放位置，播放装态等，可以根据这些信息更好的实现播放功能。