1. Article information
The article introduced this time is a 2023 fire detection article from the Korean scientific research team. The article is based on forest fire detection and the title is "An Improved Forest Fire Detection Method Based on the Detectron2 Model and a Deep Learning Approach".
2. Summary
With global warming and increasing population, forest fires have become a major global concern. This can lead to adverse consequences such as climate change and the greenhouse effect. Surprisingly, human activities are responsible for a disproportionate amount of forest fires. Fast, high-precision detection is the key to controlling this emergency. To address this issue, we propose an improved forest fire detection method based on a new version of the Detectron2 platform (a rewrite of the Detectron library) that uses deep learning methods to classify fires. Furthermore, a custom dataset was created and labeled for training the model, which achieved higher accuracy than other models. This robust result was achieved by improving the Detectron2 model in various experimental scenarios using a custom dataset and 5200 images. The model can detect small fires at long distances during the day and at night. The advantage of using the Detectron2 algorithm is that it enables long-distance detection of objects of interest. Experimental results show that the proposed forest fire detection method successfully detects fires with a 99.3% increase in accuracy.
3. Introduction
Forest fires, also known as wildfires, are among the most destructive events in recent years, causing loss of life and property. Between 2002 and 2016, an estimated 4.225 million square kilometers burned due to uncontrolled fires. Forest fires can be divided into two categories: natural fires and man-made fires. Dry weather, wind, lightning, volcanoes, meteors, coal seam fires, heating, and smoking are examples of natural causes, while cooking, accidental or intentional negligent acts are examples of human-caused fires. Both natural and man-made fires severely impact wildlife and human life. Early detection of fires is key to preventing such unexpected events, saving many lives and resources.
Rapid detection is key to reducing the overall impact. Traditional human monitoring is expensive and not as effective as detection models. The management of manpower and the maintenance of resources is time-consuming and expensive. Automation is a better and more accurate approach. Weather conditions, temperature, rain and wind all affect fire detection. Therefore, the cost of collecting data in real time is lower.
Detectron2 is a robust and reliable automatic fire detection method using maskRCNN. Detecting fires is challenging because of their size, color, motion, speed, approach, sunlight, and combinations of these different factors. Although these factors make fire detection challenging, the use of datasets, trained models, and data angles allows for maximum accuracy. The main contributions of the article are as follows:
(1) An automatic forest fire detection method was developed to reduce natural disasters and loss of forest resources.
(2) To train the proposed model, a large custom dataset is collected, containing two categories, fire and non-fire, with different scenes (day and night), fire and flame, light and shadow. The dataset is available for public use on GitHub. We use the LabelMe data annotation tool, which uses polygons rather than rectangles to annotate triggers and non-triggers.
(3) Using fire and non-fire image and data enhancement technology, the forest fire detection accuracy is improved. Furthermore, the model significantly improves accuracy and reduces false detection rates, even in smaller fire areas.
4. Model
A. Forest Fire Dataset
In object detection, the main limitation is the data collection implemented in custom trained models. To address this issue, we collected forest fire data from different databases and augmented the dataset using several computer vision techniques. In order to obtain more accurate results, two types of datasets were created: fire and non-fire.
The dataset is publicly available and some images were collected from Google. To train the dataset first, resize all images to the same height and width to avoid unexpected results or errors. After data acquisition, the dataset is smaller. To augment the dataset, we scoured the internet for videos of forest fires and captured frames from those videos. Our training dataset compresses 5200 diurnal forest fire images and non-fire images to distinguish fire images from non-fire images for maximum accuracy. The small dataset prevented us from achieving the desired accuracy, as shown in the table below. Therefore, we employ data augmentation techniques to expand the dataset. The next section describes the collection and expansion of custom datasets in detail.
Augment the dataset using computer vision algorithms and rotate each image by 15◦ to 360◦, as shown in the image below. By applying this technique, our dataset has increased by a factor of 23. As mentioned before, we compressed 5200 images in the dataset. After augmentation, the total number of images is extended to 119,600, and to prevent false positive results, there are 10,120 fire-like images. Simple linear algebra will provide equations for rotating arbitrary points p and q by an angle. Detectron2 provides good results on small datasets.
However, fire detection accuracy for large datasets shows improved results compared to small datasets. Therefore, it is better to expand the training dataset. Second, rotate all forest fire images to 90◦, 180◦ and 270◦ (below). When the image rotation value is larger than 15◦, the output is almost similar, while when the image is rotated about 90◦, we lose the region of interest of the forest fire image.
Use LabelMe software to annotate images, which is an important step in the Detectron2 training process, as shown in the figure below. Our level file is a JSON file saved in the same folder as the training file. Also, in Detectron2, all image sizes must have exact dimensions (height and width). So, before annotating the images, we use OpenCV to resize all the images to the same height and width. Additionally, we add non-fire images to our training set and label them as such. The purpose of training non-fire images is to reduce the number of false detections.
In the dataset, each image is rotated 15◦ to 360◦, resulting in 23 images from the same image. If the images were manually labeled, we would lose a considerable amount of time repeatedly performing the same task. Therefore, we use an affine transformation method to rotate the same image. Image transformations are represented in a matrix using NumPy.
b. model
Article resizes and shapes forest fire images. Several techniques were applied to develop the dataset. First, we resize the input image to 224 × 224, 320 × 320 and 512 × 512 using OpenCV2 as shown in the image below. In the study, 416 × 416 images were used to improve the accuracy of our forest fire model and reduce the false detection rate. Before the CNN training model, data enhancement and image contrast information processing are realized.
In Detectron2, the input images of forest fires and non-fires are set to 512 × 512 in the same way. As shown in the table below, the results of training and testing accuracy under different indicators are obtained. The training accuracy of Mask_rcnn_50_FPN_3x within 62 h is 83.8% and 79.8%, respectively. Keypoint_rcnn_R_50_FPN_3x is 82.4%, and the test accuracy is 77.8%. The accuracy and test of Mask_rcnn_50_FPN_3x and Keypoint_rcnn_R_50_FPN_3x are similar. However, with smaller weights, the model training time varies. Improving accuracy requires more training time, which is expensive. The challenge of training in Detectron2 was to find the capability of PyTorch in GPU mode with Cuda. Human eyes can easily distinguish forest fire images from non-fire images based on the color, size, shape, and reflection of the fire [5].
Unlike the human eye, our model can distinguish non-fire images from fire images due to shape, color, and similar surroundings, which can lead to false detections. Therefore, large datasets enable more accurate object detection. The image below shows an image of forest fire-like lights such as the sun, haze, and others.
Real-time error detection is inconvenient. After detecting these errors, we upgraded our experiments with new training parameters. Therefore, we realize that the mask-RCNN model is more accurate than improving our parameters. Fire has no specific shape and color, and has different hues, saturations, and exposures, as shown in the image below. Therefore, varying these parameters randomly during training can provide better results.
Changed approach to dataset due to false image detection of hue and opacity. In the dataset, there are some low-quality images with dimensions smaller than 512×512. Therefore, we decided not to use automatic hue, exposure or saturation values. Additionally, the dataset is augmented using an algorithm based on pixel values, brightness and contrast values prior to the trained model.
The dataset contains 109,480 forest fire images and 10,120 non-fire images. After performing a custom analysis of the database, removing low-quality and low-resolution images, 116,200 images were obtained. After using the formulas and algorithms for fire image contrast and brightness, the dataset size increased from 119,600 images to 348,600, as shown in the table below. First, in the dataset, we doubled the contrast and halved the brightness compared to the original image.
For better real-time accuracy, 13,800 non-fire images, similar to fire images, are also included. As mentioned earlier, non-fire images enable better real-time forest fire detection, resulting in fewer false positives. In general, sunlight is the most destructive method for detecting forest fires in real time. Because of this, a large dataset will allow distinguishing sunlight for different forest weather conditions, such as sunrise and sunset as shown in the image below.
Different algorithms were tested and Mask_rcnn_50_FPN_3x scored the lowest. In contrast, Panoptic_fpn_R_101_3x scored the highest. The model improves significantly after adding non-fire images to our dataset.
5. Experimental results and analysis
A. Implementation Details
The model was implemented and tested using Visual Studio 2022 c++ on a laptop with a CPU speed of 3.20 Hz, 32 GB RAM and 3GPU. To test the forest fire detection model, it is implemented in different environments. In the previous subsections, our model was discussed and implemented using Detectron2. This section discusses the strengths and limitations of the proposed model. Traditionally, the fast-rcnn framework is used for real-time fire detection, and its accuracy is high. However, the proposed model improves fire detection over traditional forest fire detection methods and shows that Mask RCNN can achieve 99.3% accuracy. To achieve high accuracy, the model is trained with different parameters: hue, saturation, opacity and small image pixels. Moreover, the proposed model works effectively in different scenarios, as shown in the figure below.
Compression of the model will be discussed using different parameters and methods. Accurate detection of forest fires with our model using Detectron2 deep learning and a custom dataset. In preparation for the study, previous methods were analyzed. However, due to the redundant source code being publicly available when initializing our model and the limitations of real object detection cooperation, as we mentioned earlier, the method uses three layers of upscaling to achieve the highest accuracy of 99.3% in the model . F-measure (FM) was tested, which measures the weighted mean and balances precision and recall. This score takes into account both the false negative rate and the true positive rate. Since measuring accuracy is difficult, FM is the most commonly used parameter for detecting objects. False negatives and true positives are detected better in a detection model that uses the same weights. However, if true positives and false negatives are not the same, precision and recall must be considered.
The results of the models varied according to the different types of classification of strong, normal and not strong (weak) in the above seven criteria. Robust means that the algorithm can be implemented for various events, and normal means that the algorithm can fail in unexpected situations. However, neither strong nor weak means that algorithms based on color, opacity, image noise and even size often fail.
B. Limitations
As mentioned in the table above, a model cannot be judged as good or bad based on specific criteria other than overall performance. The proposed model has certain limitations. For example, when testing the model in different environments, electric lights or the sun are considered fire in some cases, as shown in the image below. We intend to upgrade the proposed model using more datasets from different environments to address this issue.
6 Conclusion
A lot of research has been done on the use of CNN-based deep learning models to improve forest fire detection systems. However, the potential of the Detectron2 deep learning model in forest fire detection has not been explored. Collecting enough image data for the training of forest fire detection models is a challenge, leading to data imbalance or overfitting problems that compromise the effectiveness of the models. This study proposes a method for detecting forest fires using a modified Detectron2 model and creates a dataset.
First, a model is used to detect fires, and then a different deep learning object detection model is used to detect forest fires. Next, the dataset was prepared, and for more accurate detection of fires in different stages and scenarios, the dataset was upscaled with small images and low-quality pixelated images were removed. Furthermore, to expand our dataset, we used a data augmentation algorithm to create 23 times more images than the original ones. The proposed method is compared with the existing methods through experiments, and the accuracy of the model is verified. After reaching the highest accuracy, our model was implemented in a Raspberry Pi 3B+, which allows running CPU and GPU details simultaneously.
Furthermore, some limitations in real-time applications are observed, such as not being able to label smoke images from the dataset. Future tasks include solving the ambiguity problem in dark conditions and improving the accuracy of the method. It is planned to use 3D CNN/U-Net in recognition and medical settings to develop a small model with reliable fire detection performance.
Attention
Welcome to the WeChat public account "When Traffic Meets Machine Learning"! If you are in the field of rail transit, road traffic, and urban planning like me, you can also add WeChat: Dr_JinleiZhang, note "Join the group", join the traffic big data exchange group! Hope we make progress together!
