基于高斯混合模型和多色特征的视频火灾检测 （英文论文翻译）

---

英文版论文原文：https://link.springer.com/content/pdf/10.1007%2Fs11760-017-1102-y.pdf

---

基于高斯混合模型和多色特征的视频火灾检测

Video fire detection based on Gaussian Mixture Model and multi-color features

Xian-Feng Han, Jesse S. Jin& Ming-Jie Wang

• 天津大学，天津300072
• Tianjin University, Tianjin 300072, China
• 航天智能控制科学技术国家重点实验室，北京100085
• National Key Laboratory of Science and Technology on Aerospace Intelligent Control, Beijing 100085, China

Abstract

本文提出了一种从视频流中检测火灾的新方法。 它充分利用了火的运动特征和颜色信息。 首先，使用基于高斯混合模型的背景减法进行运动检测，以从视频流中提取运动对象。 然后，采用结合RGB，HSI和YUV颜色空间的基于多颜色的检测来获得可能的着火区域。 最后，将以上两个步骤的结果结合起来，以确定准确的火灾区域。 通过在不同的火灾录像中使用该方法获得的实验结果表明，该方法具有较好的有效性，适应性和鲁棒性。

This paper proposes a new approach to detect fire from a video stream. It takes full advantage of the motion feature and color information of fire. Firstly, motion detection using Gaussian Mixture Model-based background subtraction is applied to extract moving objects from a video stream. Then, multi-color-based detection combining the RGB, HSI and YUV color space is employed to obtain possible fire regions. Finally, the results of the above two steps are combined to identify the accurate fire areas. The experimental results obtained by applying this method on different fire videos show that the proposed method can achieve better effectiveness, adaptability and robustness.

1 Introduction

早期火灾探测在预防生命和财产安全中起着重要作用[1]。 但是，传统的火灾探测技术[2]使用内置的烟雾和温度传感器。 激活传感器需要很长时间，这可能已经造成人身伤害和损坏。 传感器的检测半径是有限的[3]，不能应用于开放或大空间，例如森林[4]。 而且，这些探测器无法给出有关火灾的有价值的信息，例如位置，规模[4,5]和燃烧程度[6]。 此外，常规传感器可能会产生错误（错误警报）[7]。

Early fire detection plays an important role in the prevention of life and property safety [1]. However, conventional fire detection technologies [2] use built-in smoke and temperature sensors. It takes a long time to activate sensors, which may have already caused injuries and damages. The detection radius of sensors is limited [3] and cannot be applied to an open or large space, such as forest [4]. Moreover, these detectors cannot give valuable information about fire such as location, scale [4,5] and burning degree [6]. In addition, the conventional sensors may produce errors (false alarm) [7].

当前，随着数码相机的广泛使用和视频处理技术的进步，视频火灾检测[8,9]显示出更好的灵活性，有效性和可靠性，并提出了解决上述传统传感器引起的问题的方案。 此外，视频火灾探测可以充分利用现有的视频设备。 因此，视频火灾探测越来越引起研究人员的关注。

Currently, with the wide use of digital cameras and advancement of video processing techniques, video fire detection [8,9] shows better flexibility, effectiveness and reliability and presents solutions to the above-mentioned problems caused by traditional sensors. Furthermore, video fire detection can make full use of existing video equipment. Therefore, video fire detection is attracting more and more attention among researchers.

在视频火灾探测方面取得了一些杰出的成就。 Cetin等。 [10]对近年来最先进的视频火灾探测方法进行了全面回顾，其中颜色是许多用于确定类似火灾候选区域的方法中使用的重要特征之一。 在此评论中，常用的色彩空间[11,12]是RGB [5,9,13–17]，YUV [8,18]，YCbCr [19–22]，HIS [6]，HSV [23,24]和 这些不同颜色空间的组合[2]，YUV / RGB [25]，YCbCr / RGB [26]，RGB / HSI [2,27,28]，RGB / HSV [29]。 Shidik等。 [30]提出了一种在RGB，HSV和YCbCr空间中使用多色特征的火灾探测方法。

Several outstanding achievements have been made in video fire detection. Cetin et al. [10] provided a comprehensive review of the state-of-the-art video fire detection methods in recent years, where color being as one of the important features used in many approaches to determine the fire-like candidate regions. Commonly used color spaces [11,12] in this review are RGB [5,9,13–17], YUV [8,18], YCbCr [19–22], HIS [6], HSV [23,24]and the combination of these different color spaces [2], YUV/RGB [25], YCbCr/RGB [26], RGB/HSI [2,27,28], RGB/HSV [29]. Shidik et al. [30] presented a fire detection method using a multi-color features in the RGB, HSV and YCbCr spaces.

通过研究这些方法，我们提出了一种新方法，该方法使用高斯混合模型作为背景扣除方法，并利用RGB，HSI和YUV空间的多色特征来检测火灾。 该方法利用了运动特性和三种颜色空间的优点。 本文其余部分的组织如下： 图2说明了我们提出的方法的详细说明。 实验的结果和讨论出现在Sect中。 3.最后，在本节中有一个结论。 4。

By studying these methods, we propose a new approach, which uses Gaussian Mixture Model as a background subtraction method and employs a multi-color features using the RGB, HSI and YUV spaces to detect fire. This method makes use of the motion character and the advantages of three color spaces. The organization of the remainder of this paper is as follows: Sect. 2 illustrates a detail description of the method we proposed. The result and discussion of the experiment appear in Sect. 3. Finally, there is a conclusion in Sect. 4.

2 Fire detection

本文提出的方法主要包括两个关键步骤：（1）使用基于高斯混合模型的背景减法进行运动检测； （2）在移动物体上使用多色模型以滤除非射击物体。 每个步骤的详细说明将在以下小节中介绍。

The approach proposed in this paper is mainly composed of two critical steps: (1) motion detection using Gaussian Mixture Model-based background subtraction; (2) a multicolor model is used over moving objects to filter out non-fire objects. The detail description of each step will be presented in the following subsections.

2.1 Motion detection

在许多计算机视觉应用中，从视频中识别运动对象是一项基本而关键的任务[31]。 由于可以自然地将火视作视频中的运动对象，因此可以使用运动角色来确定类似火的区域。

Identifying moving objects from a video is a fundamental and critical task in many computer-vision applications [31]. Because fire can be naturally viewed as a moving object in the video, the motion character can be utilized to determine the fire-like region.

背景消减[2,9,15,28,32]，作为一种出色的运动物体检测算法之一，已在视频火灾检测中得到了广泛的应用。

Background subtraction [2,9,15,28,32], being as one of outstanding moving object detection algorithms, has been applied extensively in the video fire detection.

为了滤除视频流中类似文件的背景对象，基于高斯混合模型[33]的背景减法被用作该方法的第一步，以消除背景干扰并检测前景中的运动对象。 GMM将每个像素的值建模为K高斯分布的混合。 在有效性，鲁棒性和适应性方面，GMM算法优于其他背景建模算法。 该公式定义如下。

In order to filter out the file-like background object in video streams, Gaussian Mixture Model [33]-based background subtraction is used as the first step of the proposed method to remove the disturbance of background and detect moving objects in the foreground. The GMM models the values of each pixel as a mixture of K Gaussian distributions. The GMM algorithm outperforms other background modeling algorithms in terms of effectiveness, robustness and adaptability. The formula is defined as follows.

$\tag{1} R_{motion}(i,j,n)=gmm(I(i,j,n))$

其中I表示视频流中的第n个图像帧。 R是GMM算法的结果，即前景移动区域。 函数gmm（）表示高斯混合模型算法的操作。 图1显示了使用GMM对三个不同的火灾视频流进行运动检测的结果。

where I denotes the nth image frame in a video stream. R is the result of the GMM algorithm, namely foreground moving region. Function gmm() indicates the operation of the Gaussian Mixture Model algorithm. Figure 1 shows the results of motion detection using the GMM over three different fire video streams.

图1使用高斯混合模型进行运动检测的结果。 a–c来自三个不同视频流的原始图像。 d–f在前景中检测到的移动对象分别对应于a–c

Fig. 1 Result of motion detection using Gaussian Mixture Model. a–c Original images from three different videos streams. d–f Moving objects detected in foreground corresponding to a–c, respectively

2.2 Multi-color detection

众所周知，颜色是着火的最显着特征，它已被广泛用于区分火与其他物体。 因此，所提出方法的第二步是颜色检测，它结合了RGB，HSI和YUV颜色空间以获得可能的起火区域。

It is well known that color is the most notable feature of fire, which has been widely used for distinguishing the fire from other objects. Therefore, the second step of the proposed approach is color detection, which combines the RGB, HSI and YUV color spaces to obtain the possible fire areas.

2.2.1 RGB

使用RGB的原因是几乎所有可见范围的摄像机都以RGB格式捕获视频[10]，并且光谱内容显然与RGB空间相关。根据先前关于火灾检测和火灾图像分析的方法，存在多种颜色的火灾。红色到黄色的颜色范围是消防展览的初始阶段。相应的RGB值定义为 $R≥G> B$ 。此外，由于在火像中R通道占主导地位，因此应给R一个较大的值。这为R增加了另一个条件，即R必须超过阈值 $R_T$ 。但是，背景光条件可能会对火灾的饱和度产生不利影响，从而导致错误的火灾检测（将非火灾像素视为火灾）。因此，像素的饱和度值也应设置为高于阈值的值，以避免上述影响。基于这些事实，在我们的研究中采用[27,28,34,35]中总结的规则从图像中提取火灾候选对象，并在以下描述中进行介绍：

The reason for using RGB is that almost all visible range cameras capture video in RGB format [10] and the spectral content obviously associates with RGB space. According to previous approaches on fire detection and analysis of fire images, there are various colors of fire. And red-to-yellow color range is the initial stage of the fire exhibition. The corresponding RGB value is defined as $R ≥ G > B$. In addition, because of the domination of R channel in the fire image, R should be given a larger value. And this adds another condition for R, that R has to be over a threshold $R_T$ . However, the background light condition may have an adverse effect on the saturation of fire, resulting in false fire detection (non-fire pixels are regarded as fire). The saturation values of pixels, therefore, should also be set to a higher value than a threshold to avoid the influence mentioned above. Based on these facts, the rules summarized in [27,28,34,35] are employed in our research to extract fire candidates from an image and presented in following description:

$\tag{2} Rule1: R \ge G \lt B$
$\tag{3} Rule2: R\gt R_T$
$\tag{4} Rule3:S\gt(255-R)* S_T/R_T$

其中R，G，B是像素的红色，绿色和蓝色分量。 $R_T$ 表示R分量的阈值，范围从55到65，而ST表示饱和阈值， $115 ≤ ST ≤ 135$ [27,35]。 如果像素满足所有规则（2），（3）和（4），则可以将其视为可能的候选火灾像素。 同时，使用 $R_{RGB} (i, j, n)$ 表示可能的着火区域，该区域由上述RGB颜色模型确定。

where R, G, B are the red, green and blue components of a pixel. $R_T$ denotes the threshold value of R component, ranging from 55 to 65, while ST denotes the threshold of saturation and $115 ≤ ST ≤ 135$ [27,35]. A pixel can be viewed as a possible fire candidate pixel if it meets all rules (2), (3) and (4). Meanwhile, $R_{RGB} (i, j, n)$ is used to represent the possible fire region, which is determined by the above RGB color model.

2.2.2 HSI

HSI从人类视觉系统的角度使用色相，饱和度和强度来描述色彩空间。 与RGB相比，HSI更适合于模拟人类视觉系统的色彩感应特性[6]，因为色相和饱和度分量与人类感知颜色的方式密切相关[35]。

HSI describes the color space using hue, saturation and intensity from the view of the human visual system. Compared with RGB, HSI is more suitable for simulating the color sensing properties of the human visual system [6], because the hue and saturation components are intimately related to the way in which human beings perceive color [35].

根据对火灾特征的分析，火灾的色相值从0到60对应于红色到黄色范围。 如第15节所述。 2.2.1，背景照明会影响火的饱和度。 从较亮的环境获得的饱和度大于从较暗的场景获得的饱和度。 这是因为在没有其他背景照明的情况下，火将成为主要的照明，并且只有在没有其他背景照明的情况下才可以照明[27]。 在这种情况下，有更多白色为火。 为了保证视频处理中足够的亮度，应给强度一个超过一定阈值的值。 因此，基于HSI的规则[6,36,37]用作多色检测的第二部分，以获取表示为 $R_{HIS} (i, j, n)$ 的候选火灾，可以对此进行描述 如下：

According to the analysis of fire features, the values of hue for fire from 0 to 60 correspond to the red-to-yellow range. As mentioned in Sect. 2.2.1, the background illumination has impact on the saturation of fire. The saturation got from the brighter environments is larger than that from the darker scenes. This is because that the fire will become the major and only illumination if there is no other background illumination [27]. In this case, there is more white in hue for fire. In order to guarantee enough brightness in video processing, the intensity should be given a value over certain threshold. Hence, the HSI-based rules [6,36,37] are used as the second part of multi-color detection to obtain the fire candidates denoted as $R_{HIS} (i, j, n)$, which can be described as follows:

$\tag{5} Rule1: 0\le H\le 60$
$\tag{6} Rule2:20\le S \le 100$
$\tag{7} Rule3: 100\le I \le 255$

其中H，S和I分别是图像的色相，饱和度和强度分量。 可以使用公式（8），（9）和（10）[38]从RGB转换HSI。 类似地，只有同时满足所有三个规则的像素才可以被视为候选像素。

where H, S and I are the hue, saturation and intensity components of an image, respectively. The HSI can be translated from RGB using formulas (8), (9) and (10) [38]. Similarly, only the pixel that simultaneously meets all three rules can be regarded as a candidate.

$\tag{8} i=\frac{1}{3}(r+g+b)$
$\tag{9}$
$\tag{10}$
$\tag{11}$

其中RGB的r，g和b在[0，1]范围内归一化。

where r, g and b of RGB are normalized in range [0, 1].

2.2.3 YUV

YUV色彩空间用于多色彩模型中，因为它的重要性在于亮度和色度之间的区别要更加区分[22,31]。 此外，YUV色彩空间对光线条件不敏感，可以减少照明变化的影响。 通过对火像的分析，很明显地注意到亮度值应该较高，而色度值应该较低。 引入基于YUV的规则[10]来提取标记为RYUV（i，j，n）的可能火灾区域：

YUV color space is used in multi-color model because of its importance that the separation between luminance and chrominance is more discriminate [22,31]. Moreover, YUV color space is insensitive to light condition and can reduce the effect of illumination changes. From the analysis of fire images, it obviously noticed that luminance value should be high, while the chrominance values should be low. The YUV-based rules [10] are introduced to extract possible fire areas marked as RYUV (i, j, n):

$\tag{12} Rule1: Y\ge Y_T$
$\tag{13} Rule2:|U-128|\le U_T$
$\tag{14} Rule3:|V-128|\le V_T$

其中Y是亮度值，而U和V是像素的色度值。 值 $Y_T$ ， $U_T$ 和 $V_T$ 分别是通过实验确定的Y，U和V的阈值[39]。 最后，联合操作用于将RGB，HSI和YUV颜色空间的规则组合在一起以形成多色模型，该模型将应用于图像以识别候选火灾区域，定义为 $R_{color}(i, j, n)$ 。

where Y is the luminance value while U and V are the chrominance values of a pixel. Values $Y_T$ , $U_T$ , and $V_T$ are thresholds of Y, U and V, respectively, which are experimentally determined[39]. Finally, the union operation is utilized to combine the rules of RGB, HSI and YUV color spaces together to form the multi-color model, which is applied on an image to identify the fire candidate regions, defined as $R_{color}(i, j, n)$.

$\tag{15}$

多色检测的结果如图2所示。

Theresults ofmulti-color detection are illustrated in Fig. 2.

图2多色检测结果。 a–c来自不同视频流的帧。 d–f相应的结果

Fig. 2 Result of multi-color detection. a–c Frames from different videos streams. d–f Corresponding results

2.3 Combining motion and multi-color

通过上述对火的运动和颜色特征的详细分析，我们可以得出结论，由于其他运动物体或火色物体的出现，单独使用运动或多色检测来识别火会导致高误报率 。 因此，我们需要做进一步的操作，将高斯混合模型的结果与多色检测的结果相结合，以充分利用运动和颜色特征来获得准确的着火区域Rfire（i，j，n）。

From the detail analysis of the motion and color features of fire described above, we can conclude that using motion or multi-color detection alone to identify fire will lead to high false-alarms, because of appearance of other moving objects or fire-color objects. Consequently, we need to do further operation that combines the result of Gaussian Mixture Model and that of multi-color detection to make full use of motion and color feature to obtain the accurate fire region Rfire (i, j, n).

$\tag{16}$

图3显示了合并的结果。

Figure 3 shows the result of combination.

图3组合结果。 a运动检测的结果。 b多色检测的结果。 c组合（a）和（b）的结果

Fig. 3 Results of combination. a Result of motion detection. b Result of multi-color detection. c Result of combination (a) and (b)

3 Experimental results

所提出的方法是通过使用Visual Studio 2013和OpenCV3.0来实现的。 运行环境为Windows 8.1，Inter（R）CoreTM i7 3.60GHz和RMA 16GB。 测试视频数据库来自 http://signal.ee.bilkent.edu。tr/VisiFire/Demo/FireClips/ [30,34]具有多种场景，包括不同的环境背景和条件。 分辨率为320×240。

The proposed method is implemented by using Visual Studio 2013 and OpenCV3.0. The running environment is Windows 8.1, Inter® Core™ i7 3.60GHz and RMA 16GB. The test video database is from http://signal.ee.bilkent.edu. tr/VisiFire/Demo/FireClips/ [30,34] with a large variety of scenes including different environment background and conditions. The resolution is 320 × 240.

图4展示了九个不同场景的测试结果。 从图中可以看出，所提出的方法能够消除非火运动物体（例如图4a，c和e）以及火色背景区域（例如图4c和g）的干扰， 可能会导致错误检测。 此外，视频流中的火灾已成功确定。

Figure 4 demonstrates testing results over nine different scenes. From the illustration, it can be seen that the proposed method is able to eliminate the disturbance of non-fire moving objects, such as Fig. 4a, c and e, and fire-color background areas, like Fig. 4c and g, which may cause false detection. In addition, the fires in the videos streams were successfully determined.

实验结果列于表1，其中Nn是帧数，也是视频中包含火的帧数。 Nd表示使用提出的方法正确检测到的帧数。 Rd是视频的火灾检测率

The experimental results are presented in Table 1, where Nn is the number of frames and also is the number of frames containing fire in a video. Nd indicates the number of frames that correctly detected using the proposed method. And Rd is the fire detection rate of a video