CN104123734A - Visible light and infrared detection result integration based moving target detection method - Google Patents

Abstract

The invention discloses a moving target detection method based on fusion of visible light and infrared detection results, which is used to solve the technical problem of poor detection effect of the existing moving target detection method. The technical solution is to firstly use the background extraction method of ViBe to extract the foreground image of the visible light image. After the foreground image is obtained, it is converted to LUV space for shadow judgment; secondly, the saliency image of the infrared image is extracted by means of saliency detection; finally, the final moving object is extracted by combining the detection results of the two. The method of the invention can effectively deal with the technical problem of poor target detection effect caused by environmental changes of shadow and light intensity, and achieve the effect of all-weather detection.

Description

A moving target detection method based on the fusion of visible light and infrared detection results

technical field

The invention relates to a moving target detection method, in particular to a moving target detection method based on fusion of visible light and infrared detection results.

Background technique

Moving target detection is the basic work of video surveillance. Its main task is to segment and extract the region of motion change from the background image in the video sequence for subsequent target tracking and behavior analysis. The target detection method can effectively overcome the multiple influences of illumination changes, background motion, camera shake, etc., and accurately extract moving targets, which is of great significance. Most of the existing methods are studied on single-source sensor imaging, and the main detection methods are pixel-based detection methods and feature-based detection methods.

The document "ViBe: a powerful random technique to estimate the background in video sequence. ICASSP, 2009, 4:945-948" discloses a pixel-based target detection method to extract the target background area. The method uses a random selection strategy and the principle of pixel spatial consistency for foreground detection. The method firstly uses the first frame image to initialize the background model with a sample capacity of N; then calculates the number of times the distance between the current pixel point and the sample point satisfies the threshold to judge the foreground point; finally updates the background sample point and the area of the sample point with a random strategy. This method has the characteristics of small memory capacity and fast speed, but when the light intensity is strong and there are a lot of shadows, the detection effect will decrease, and when the light intensity changes, such as under low light environments such as rainy and night, it cannot detect the target in real time. Realize round-the-clock detection.

Contents of the invention

In order to overcome the shortcomings of the poor detection effect of existing moving object detection methods, the present invention provides a moving object detection method based on the fusion of visible light and infrared detection results. The method first extracts the foreground image of the visible light image by using the background extraction method of ViBe. After the foreground image is obtained, it is converted to LUV space for shadow judgment; secondly, the saliency image of the infrared image is extracted by means of saliency detection; finally, the final moving object is extracted by combining the detection results of the two. The method of the invention can effectively deal with the technical problem of poor target detection effect caused by environmental changes of shadow and light intensity, and achieve the effect of all-weather detection.

The technical solution adopted by the present invention to solve the technical problem is: a moving target detection method based on the fusion of visible light and infrared detection results, which is characterized in that it includes the following steps:

Step 1. For the first frame of visible light image, randomly select N pixel values from the eight domains of pixel points (x, y),

Initialize the background model {B ⁿ , n=12, . . . , N} corresponding to the position points.

Pixel matching starts from the second frame image. At time t, judge the matching state of pixel ⁱ (x,y) at (x,y) of current frame I t and pixel b(x,y) of background sample B ⁿ :

Among them, dis() represents the Euclidean distance between the current pixel and the background pixel, and D _d =20 represents the distance threshold.

When the number of times the distance threshold D _d is satisfied is less than T _c , T _c =2, and the pixel is marked as the foreground; otherwise, the pixel is marked as the background.

Among them, VF ^t is a binary image, when it represents the foreground pixel, the value is 255; when it represents the background pixel, the value is 0.

Set the background update probability to φ, when the pixel i(x,y) is marked as a background pixel, there is a probability of 1/φ to update its background model. Assuming that a certain background model B ⁿ of this pixel needs to be updated, first use i(x, y) to update the corresponding position pixel b(x, y) of the background model ^{B n} ; then randomly select a The pixel position uses i(x,y) to update the corresponding pixel.

Step 2. Generally, the luminance L of pixels in the shadow area of the LUV space is lower than the luminance L of the background area, and the chromaticity of the shadow area and the background area is similar, but the value range of the difference is larger. When pixel i(x,y) is judged as a shadow point, the following conditions must be met:

a.θ ₁ ≦ ^OL ≤θ ₂ , ^OL ＝i ^L (x,y)/b ^L (x,y) (3)

$\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; .</span>{\mathrm{<span\; class="notranslate">\; o</span>}}^{\mathrm{<span\; class="notranslate">\; UV</span>}}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; \&beta;</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; o</span>}}^{\mathrm{<span\; class="notranslate">\; UV</span>}}<span\; class="notranslate">\; =</span>\sqrt{{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; u</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; b</span>}}^{\mathrm{<span\; class="notranslate">\; u</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; V</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; b</span>}}^{\mathrm{<span\; class="notranslate">\; V</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

Here, ^OL is the brightness change, OU ^UV is the chromaticity change, θ ₁ , θ ₂ , β are the judgment thresholds.

When the pixel is judged as a shadow by the shadow judgment function, the mixed Gaussian modeling is performed on the ^OL and OU ^UV components of the pixel, and the shadow point is further verified to reduce the false detection rate of the shadow.

Step 3, use a combined DoG filter to calculate the high-frequency and low-frequency information of the image, and a single DoG filter is expressed as follows:

$\begin{array}{c}\mathrm{<span\; class="notranslate">\; DoG</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&pi;</span>}}<span\; class="notranslate">\; [</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}\mathrm{<span\; class="notranslate">\; exp</span>}<span\; class="notranslate">\; (</span><span\; class="notranslate">\; -</span>\frac{{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}\mathrm{<span\; class="notranslate">\; exp</span>}<span\; class="notranslate">\; (</span><span\; class="notranslate">\; -</span>\frac{{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>\\ <span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span>\end{array}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span>$

The bandwidth of the DoG filter is determined by ρ=σ ₁ /σ ₂ , where σ ₁ and σ ₂ represent the Gaussian standard deviation (σ ₁ >σ ₂ ). σ ₁ determines the selection of low-frequency information, and σ ₂ determines the selection of high-frequency information. Consider the combined filter of M DoGs:

$\underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; \&rho;</span>}}^{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; \&rho;</span>}}^{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>\text{<span class="notranslate"> =G</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; \&rho;</span>}}^{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; )</span>$

Among them, M represents an integer greater than 0, and formula (6) is simplified as the difference of two Gaussian functions. The bandwidth of the combined filter is determined by K= ^ρM .

Calculate the salient feature of the infrared video sequence, let M=∞, make the K value as large as possible, then G(x,y,ρ ^M σ ₂ ) is the average of the entire image. The saliency of the infrared image is calculated as follows:

$\mathrm{<span\; class="notranslate">\; Sal</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; u</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; hc</span>}}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 7</span>}<span\; class="notranslate">\; )</span>$

Before computing the saliency, the color space is converted to CIELAB space. _Hu is the average value of the image, is the color feature of the pixel after Gaussian smoothing, and ||·|| is the L2 paradigm.

After extracting the infrared saliency map, binary segmentation is performed to obtain the salient foreground map SFt.

Step 4. Fusion of infrared and visible light detection results.

Record the fusion result picture as I _f,s , mark the pixels with visible light and infrared detection results,

Three possible situations arise:

(a) Both visible light foreground detection results and infrared foreground detection results indicate that the pixel is a foreground point. Right now

VF ^t (x, y) = 255, SF ^t (x, y) = 255, (8)

At this time, it is marked _If,s (x,y)=255.

(b) The visible light foreground detection result indicates that the pixel is a foreground point, and the infrared foreground detection indicates that the pixel is not a foreground point, or the visible light foreground detection result indicates that the pixel is not a foreground point, and the infrared foreground detection indicates that the pixel is a foreground point. Here, the pixel value is marked in the way of weighted average of corresponding points in the visible light map and the infrared saliency map. At this time, the fusion pixel value is marked as

${\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&lambda;</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; VF</span>}}^{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; +</span>\frac{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; \&lambda;</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; Sal</span>}}^{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 9</span>}<span\; class="notranslate">\; )</span>$

When the average value of the salient image pixel 8 field is greater than twice the original image average value, λ=1, otherwise it is 0.

Among them, mean() represents the mean value, and Σneig ₈ () represents the sum of 8 fields.

(c) Both visible light foreground detection results and infrared foreground detection results show that the pixel is a non-foreground point. Here, the value of the pixel marked is one-third of the corresponding position in the infrared saliency map.

${\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; Sal</span>}}^{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 11</span>}<span\; class="notranslate">\; )</span>$

After the fusion result map I _{f, s} is obtained through the above steps, binary segmentation is performed and morphological processing is performed to obtain the final target detection result.

The beneficial effects of the present invention are: firstly, the method uses the background extraction method of ViBe to extract the foreground image of the visible light image. After the foreground image is obtained, it is converted to LUV space for shadow judgment; secondly, the saliency image of the infrared image is extracted by means of saliency detection; finally, the final moving object is extracted by combining the detection results of the two. The method of the invention can effectively deal with the technical problem of poor target detection effect caused by environmental changes of shadow and light intensity, and achieve the effect of all-weather detection.

The present invention will be described in detail below in combination with specific embodiments.

Detailed ways

The specific steps of the moving target detection method based on the fusion of visible light and infrared detection results of the present invention are as follows:

1. Visible light image foreground detection.

(a) Model initialization and pixel classification.

For the first frame of visible light image, N pixel values are randomly selected from the eight domains of pixel points (x, y), and the corresponding position points of the background model {B ⁿ ,n=12,...,N} are initialized.

Pixel matching starts from the second frame image. At time t, judge the matching state of pixel ⁱ (x,y) at (x,y) of current frame I t and pixel b(x,y) of background sample B ⁿ :

Among them, dis() represents the Euclidean distance between the current pixel and the background pixel, and D _d =20 represents the distance threshold.

When the number of times the distance threshold D _d is met is less than T _c (T _c =2), the pixel is marked as the foreground; otherwise, the pixel is marked as the background.

VF ^t is a binary image, when it represents a foreground pixel, the value is 255; when it represents a background pixel, its value is 0.

(b) Background update.

Set the background update probability to φ, when the pixel i(x,y) is marked as a background pixel, there is a probability of 1/φ to update its background model. Assuming that a certain background model B ⁿ of this pixel needs to be updated, first use i(x, y) to update the corresponding position pixel b(x, y) of the background model ^{B n} ; then randomly select a The pixel position uses i(x,y) to update the corresponding pixel.

2. Visible light image shadow detection.

Usually, the luminance L of pixels in the shadow area of the LUV space is lower than the luminance L of the background area, and the chromaticity of the shadow area is similar to that of the background area, but the value range of the difference is larger. When pixel i(x,y) is judged as a shadow point, the following conditions must be met:

a.θ ₁ ≦ ^OL ≤θ ₂ , ^OL ＝i ^L (x,y)/b ^L (x,y) (3)

$\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; .</span>{\mathrm{<span\; class="notranslate">\; o</span>}}^{\mathrm{<span\; class="notranslate">\; UV</span>}}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; \&beta;</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; o</span>}}^{\mathrm{<span\; class="notranslate">\; UV</span>}}<span\; class="notranslate">\; =</span>\sqrt{{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; u</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; b</span>}}^{\mathrm{<span\; class="notranslate">\; u</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; V</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; b</span>}}^{\mathrm{<span\; class="notranslate">\; V</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

Here, ^OL is the brightness change, OU ^UV is the chromaticity change, θ ₁ , θ ₂ , β are the judgment thresholds.

When the pixel is judged as a shadow by the shadow judgment function, the mixed Gaussian modeling is performed on the ^OL and OU ^UV components of the pixel, and the shadow point is further verified to reduce the false detection rate of the shadow.

3. Infrared image saliency detection.

A combined DoG (Difference of Gaussian) filter is used here to calculate the high-frequency and low-frequency information of the image. A single DoG filter is expressed as follows:

$\begin{array}{c}\mathrm{<span\; class="notranslate">\; DoG</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&pi;</span>}}<span\; class="notranslate">\; [</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}\mathrm{<span\; class="notranslate">\; exp</span>}<span\; class="notranslate">\; (</span><span\; class="notranslate">\; -</span>\frac{{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}\mathrm{<span\; class="notranslate">\; exp</span>}<span\; class="notranslate">\; (</span><span\; class="notranslate">\; -</span>\frac{{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>\\ <span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span>\end{array}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span>$

The bandwidth of the DoG filter is determined by ρ=σ ₁ /σ ₂ , where σ ₁ and σ ₂ represent the Gaussian standard deviation (σ ₁ >σ ₂ ). σ ₁ determines the selection of low-frequency information, and σ ₂ determines the selection of high-frequency information. Consider the combined filter of M DoGs:

$\underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; \&rho;</span>}}^{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; \&rho;</span>}}^{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>\text{<span class="notranslate"> =G</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; \&rho;</span>}}^{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; )</span>$

Among them, M represents an integer greater than 0, and formula (6) can be simplified as the difference of two Gaussian functions. The bandwidth of the combined filter can be determined by K= ^ρM .

To actually calculate the salient features of the infrared video sequence, a 5×5 Gaussian smoothing window can be used to discard the highest frequency information. Let M=∞, make the value of K as large as possible, then G(x,y,ρ ^M σ ₂ ) is the average of the whole image. The saliency of the infrared image is calculated as follows:

$\mathrm{<span\; class="notranslate">\; Sal</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; u</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; hc</span>}}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 7</span>}<span\; class="notranslate">\; )</span>$

Before computing the saliency, the color space is converted to CIELAB space. _Hu is the average value of the image, is the color feature of the pixel after Gaussian smoothing, and ||·|| is the L2 paradigm.

After extracting the infrared saliency map, perform binary segmentation to obtain the salient foreground map SF ^t .

4. Fusion of infrared and visible light detection results.

Record the fusion result picture as I _f,s , and mark the pixels with visible light and infrared detection results

Three possible situations arise:

(a) Both visible light foreground detection results and infrared foreground detection results indicate that the pixel is a foreground point. Right now

VF ^t (x, y) = 255, SF ^t (x, y) = 255, (8)

At this time, it is marked _If,s (x,y)=255.

(b) The visible light foreground detection result indicates that the pixel is a foreground point, and the infrared foreground detection indicates that the pixel is not a foreground point, or the visible light foreground detection result indicates that the pixel is not a foreground point, and the infrared foreground detection indicates that the pixel is a foreground point. Here, the pixel value is marked in the way of weighted average of corresponding points in the visible light map and the infrared saliency map. At this time, the fusion pixel value is marked as

${\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&lambda;</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; VF</span>}}^{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; +</span>\frac{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; \&lambda;</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; Sal</span>}}^{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 9</span>}<span\; class="notranslate">\; )</span>$

When the average value of the salient image pixel 8 field is greater than twice the original image average value, λ=1, otherwise it is 0.

Among them, mean() represents the mean value, and Σneig ₈ () represents the sum of 8 fields.

(c) Both visible light foreground detection results and infrared foreground detection results show that the pixel is a non-foreground point. Here, the value of the pixel marked is one-third of the corresponding position in the infrared saliency map.

${\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; Sal</span>}}^{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 11</span>}<span\; class="notranslate">\; )</span>$

After the fusion result map I _{f, s} is obtained through the above steps, binary segmentation is performed and morphological processing is performed to obtain the final target detection result.

Claims (1)

1. the moving target detecting method merging based on visible ray and infrared detection result, is characterized in that comprising the following steps:

Step 1, for visible ray the first two field picture, from pixel (x, y) eight fields, choose at random N pixel value, initialization background model { B ⁿ, n=12 ..., N} correspondence position point;

Since the second two field picture, carry out pixel coupling; At moment t, judgement present frame I ^tat (x, y), locate pixel i (x, y) and background sample B ⁿpixel b (x, y) matching status:

Wherein, dis () represents the Euclidean distance between current pixel and background pixel, D _d=20 represent distance threshold;

When meeting distance threshold D _dnumber of times be less than T _ctime, T _c=2, this pixel of mark is prospect; Otherwise this pixel of mark is background;

Wherein, VF ^tbe a binary map, while representing foreground pixel, value is 255; While representing background pixel, value is 0;

Setting context update probability is φ, when pixel i (x, y) is labeled background pixel, has the probability of 1/ φ to upgrade its background model; Suppose certain background model B of this pixel ⁿupgrade, first use i (x, y) to upgrade background model B ⁿcorrespondence position pixel b (x, y); Then in sample B ⁿin eight fields of correspondence position, choosing at random a location of pixels uses i (x, y) to upgrade respective pixel;

Step 2, in shadow region, LUV space, the brightness L of pixel can be lower than the brightness L of background area conventionally, and shadow region is similar to the colourity of background area, but the span of difference is larger; When pixel i (x, y) is judged as shadow spots, meet following condition:

a.θ ₁≦O ^L≤θ ₂,O ^L＝i ^L(x,y)/b ^L(x,y) (3)

$b.O^{\mathrm{UV}}\&le;\mathrm{\&beta;},O^{\mathrm{UV}}=\sqrt{{(i^U(x,y)-b^U(x,y))}^2+{(i^V(x,y)-b^V(x,y))}^2}---\left(4\right)$

Here, O ^lfor brightness changes, O ^uVfor colourity changes, θ ₁, θ ₂, β is decision threshold;

When pixel is judged as after shade through shade decision function, the O to this pixel respectively ^l, O ^uVcomponent carries out Gaussian modeling, further shadow spots is verified, to reduce the fallout ratio of shade;

The high and low frequency information of the DoG wave filter computed image of step 3, a combination of use, single DoG wave filter represents as follows:

$\begin{array}{c}\mathrm{DoG}(x,y)=\frac{1}{\mathrm{\&pi;}}[\frac{1}{{\mathrm{\&sigma;}}_1^2}\exp (-\frac{x^2+y^2}{{2\mathrm{\&sigma;}}_1^2})-\frac{1}{{\mathrm{\&sigma;}}_2^2}\exp (-\frac{x^2+y^2}{{2\mathrm{\&sigma;}}_2^2})]\\ =G(x,y,{\mathrm{\&sigma;}}_1)-G(x,y,{\mathrm{\&sigma;}}_2)\end{array}---\left(5\right)$

The bandwidth of DoG wave filter is by ρ=σ ₁/ σ ₂determine σ ₁, σ ₂represent Gauss's standard deviation (σ ₁> σ ₂); σ ₁determining choosing of low-frequency information, σ ₂determining choosing of high-frequency information; Consider the junction filter of M DoG:

$\underset{m=0}{\overset{M-1}{\mathrm{\&Sigma;}}}(G(x,y,{\mathrm{\&rho;}}^{m+1}{\mathrm{\&sigma;}}_2)-G(x,y,{\mathrm{\&rho;}}^m{\mathrm{\&sigma;}}_2))\text{=G}(x,y,{\mathrm{\&rho;}}^M{\mathrm{\&sigma;}}_2)-G(x,y,{\mathrm{\&sigma;}}_2)---\left(6\right)$

Wherein, M represents to be greater than 0 integer, and formula (6) is simplified as the poor of two Gaussian functions; The bandwidth of junction filter is by K=ρ ^mdetermine;

The significant characteristics that calculates infrared video sequence, makes M=∞, makes K value large as far as possible, at this moment G (x, y, ρ ^mσ ₂) be exactly average to whole image; The conspicuousness of infrared image is calculated as follows:

$\mathrm{Sal}(x,y)=\left|\right|H_u-H_{w_{\mathrm{hc}}}(x,y)\left|\right|---\left(7\right)$

Before calculating conspicuousness, color space conversion is arrived to CIELAB space; H _ufor image averaging value, the color characteristic of pixel after Gaussian smoothing, || || be L2 normal form;

After extracting infrared remarkable figure, carry out binary segmentation and obtain remarkable foreground picture SF ^t;

Step 4, the infrared and visible detection result that merges;

Note fusion results figure is I _f,s, with visible ray and infrared two kinds of testing result marker image vegetarian refreshments,

Can produce three kinds of possible situations:

(a) visible ray foreground detection result and infrared foreground detection result all show that this pixel is foreground point;

VF ^t(x,y)＝255,SF ^t(x,y)＝255， (8)

This tense marker I _f,s(x, y)=255;

(b) visible ray foreground detection result shows that this pixel is foreground point, and infrared foreground detection shows that this pixel is not that foreground point or visible ray foreground detection result show that this pixel is not foreground point, and infrared foreground detection shows that this pixel is foreground point; Here adopt visible ray figure and infrared remarkable figure correspondence position are put to average weighted this pixel value of mode mark; Now merging pixel point value is labeled as

$I_{f,s}(x,y)=\frac{(1+\mathrm{\&lambda;}){\mathrm{VF}}^t(x,y)}{3}+\frac{(2-\mathrm{\&lambda;}){\mathrm{Sal}}^t(x,y)}{3}---\left(9\right)$

λ=1 when the mean value in specific image vegetarian refreshments 8 fields is greater than 2 times of former figure average, otherwise be 0;

Wherein, mean () represents average, Σ neig ₈() represent 8 fields and;

(c) visible ray foreground detection result and infrared foreground detection result all show that this pixel is non-foreground point; Here this pixel point value of mark is 1/3rd of infrared remarkable figure correspondence position point;

$I_{f,s}(x,y)=\frac{{\mathrm{Sal}}^t(x,y)}{3}---\left(11\right)$

Through above step, obtain fusion results figure I _f,safter, carry out binary segmentation, and make morphology and process, obtain final target detection result.