Reference address: http://blog.csdn.net/chuhang_zhqr/article/details/51060745
GMM is the most widely used background modeling algorithm that can be seen everywhere on the Internet. There are many related probability formulas in the paper, and I have read many blogs explaining GMM. Until now, I still find it difficult to understand the true meaning of it. If there is any misunderstanding of what I understand so far, please correct me.
Mog2 uses an adaptive Gaussian Mixture Model (Adaptive GMM, Gaussian Mixture Model). In the test program results of several background extraction algorithms MOG, MOG2, and GMG open sourced in OpenCV, MOG2 is indeed in the foreground continuity and operation time. Both stand out, and the comparison results will be given later. The following will combine the mog2 source code of OpenCV2.4.9 and the three papers of Zoran Zivkovic in 2004 pointed out in the source code document, briefly list the theoretical basis of GMM, the general principle of MOG2, the structure of code implementation and the usage of MOG2 API.
The origin of the idea of background modeling
In surveillance systems, the shooting background is usually a fixed scene with few changes. Usually we assume that static scenes without intruding objects have some general properties that can be described by a statistical model. GMM uses a Gaussian model, and is a weighted sum of multiple Gaussian models to simulate the characteristics of the background. This way, once this background model is known, intruding objects can be detected by marking out parts of the scene image that do not conform to this background model. This process is called Backgroundsubtraction, and I guess the base class for the various background modeling methods in OpenCV is called "BackgroundSubtractor".
The general idea of parameter estimation of GMM model - EM estimation of GMM parameters
Once the background is simulated with a Gaussian mixture model, now determining this model becomes a series of parameters in the Gaussian mixture model formula. The EM algorithm is usually used to solve the parameters. The EM algorithm is divided into two steps: E-step and M-step.
Regarding the derivation process of the EM algorithm , the linked blog explains in great detail.
Maximum likelihood estimation , the EM algorithm is very difficult to understand, and the link above when it comes to maximum likelihood estimation can also be used as a reference.
Mog2 modeling process and the meaning of each adjustable parameter
It is mentioned above that GMM is represented by the weighted sum of multiple Gaussian models, assuming M Gaussian components, discussing the value of M is a focus of the research of the authors of MOG2: Stauffer & Grimson took a fixed number of Gaussian components before him (M=4), Zoran automatically selects the number of components according to different input scenarios. The advantage of this is that in a simpler scene, only one more important Gaussian component will be selected, which saves the time of selecting which component belongs to when updating the background later, and improves the speed. There are two test results as proof: First, the running time obtained by running different algorithms on the same simple scene test video with the test program in OpenCV is as follows, obviously mog2 is much faster;
| mog | mog2 | gmg | |
| computer 1 | 26904 | 14386 | 25533 |
| computer 2 | 26947 | 14578 | 28834 |
The second is the three test video results of Zoran's paper [i] on background update (Figure 1, taken from the 1st and 2nd columns of Fig.1 in [1]). The Traffic scene in the figure is a relatively simple scene, and the background changes. Not big. The pure black background in the second line means that the background uses only one Gaussian component. The speed of 13ms is much faster than the 19ms of the fixed four components. The background of the tested Trees scene contains tree shadow shaking, and the light gray part of the background indicates how much A Gaussian component is being updated, and there is no advantage in this scene-adaptive approach.
A GMM model is established for each pixel of the entire image. In the modeling process, the EM algorithm is used to solve the parameter group. Once the model is established, every new frame can be judged according to whether it conforms to the established Beijing model. BG, and will update all parameters of GMM according to the parameter update formula in the paper.
OpenCV code implementation structure combing
In OpenCV, the class inheritance relationship of each background modeling method is as shown below, and BackgroundSubtractor is the base class.
Also give a blog, the code structure of MOG2 in OpenCV .
In the use of mog2, after initialization, the operator() function is the main function to perform background update. The internal implementation of operator() is mainly parallel_for_(), a parallel framework of OpenCV, and the actual operator() of the MOG2Invoker class implements the background Updated specific math operations. For the specific implementation , please refer to the MOG2Invoker API documentation .
Mog2.operator ()
Parallel_for_()
Invoker.operator()
MOG2 API usage (how to use)
The overall framework of Mog, mog2, and gmg are the same, and the usage is very simple, which can be viewed according to the official openCV tutorial .
[i] Efficient adaptivedensity estimation per image pixel for the task of background subtraction.
original:
http://blog.csdn.net/chuhang_zhqr/article/details/51060745
Background detection is a very important step in many basic applications. For example, customer counting, using a static camera to record the number of people entering and leaving a room, or a traffic camera, which needs to extract information on vehicles, etc. In all of these examples, the person or car is first extracted individually.
Technically, we need to extract the moving foreground from the stationary background . If you have an image of the background (only the background without the foreground), such as a room without customers, a road without transportation, etc., that's easy to do. We just need to subtract the background from the new image to get the foreground object. But in most cases, we don't have such a (background) image, so we need to extract the background from the image we have. The job is even harder if the vehicle in the image still has a shadow , because the shadow is also moving, and just using subtraction will make the shadow also the foreground. It's a really complicated thing.
1: BackgroundSubtractorMOG
This is a foreground/background segmentation algorithm based on a Gaussian mixture model. It was proposed by P.KadewTraKuPong and R.Bowden in 2001. It models background pixels using a mixture of K (K=3 or 5) Gaussian distributions. Use how long these colors have existed (in the entire video) as the weights for the blend. Background colors generally last the longest and are more static. How can a pixel have a distribution? On the x, y plane, a pixel is a pixel without distribution, but the background modeling we are talking about is based on time series, so the position of each pixel is in the entire time series. There will be many values, thus forming a distribution.
When writing the code, we need to use the function: cv2.createBackgroundSubtractorMOG() to create a background object. This function has some optional parameters, such as the length of time to model the scene, the number of Gaussian mixture components, the threshold, etc. Set them all to default. Then throughout the video we need to use backgroundsubtractor.apply() to get the foreground mask.
2: BackgroundSubtractorMOG2
This is also a background/foreground segmentation algorithm based on a Gaussian mixture model. It is based on two articles by Z. Zivkovic in 2004 and 2006. A feature of this algorithm is that it selects an appropriate number of Gaussians for each pixel. (In the previous method we used the K Gaussian distribution). This will result in better adaptation to scene changes due to changes in brightness, etc.
As before we need to create a background object. But here we can choose whether to detect shadows or not. If detectShadows = True (the default), it will detect and mark shadows, but doing so will slow down processing. Shadows will be marked gray.
3: BackgroundSubtractorGMG
This algorithm combines static background image estimation and per-pixel Bayesian segmentation. This was proposed in a 2012 article by Andrew_B. Godbehere, Akihiro_Matsukawa and Ken_Goldberg. It uses very few images in front (first 120 frames by default) for background modeling. A probabilistic prospect estimation algorithm was used (identification of prospects using Bayesian estimation). This is an adaptive estimation, where newly observed
objects have higher weights than older ones, thus adapting to changes in illumination. Some morphological operations such as opening and closing operations are used to remove unwanted noise. In the first few frames of the image you will get a black window.
Performing a morphological opening operation on the result is very helpful in removing noise.
Results of BackgroundSubtractorMOG
The resulting gray area of BackgroundSubtractorMOG2
represents the shadow
The result of BackgroundSubtractorGMG
uses a morphological opening operation to remove noise.
