EP1762995B1 - Detection of smoke with a video camera - Google Patents

Abstract

The method involves examination of the moving area of at least one video image by determining the direction and the size of the moving area at the probable existence of the smoke. After positive inspection result, a part of the moving area is evaluated, depending on the information characterized for smoke regarding the existence of smoke. An independent claim is also included for the smoke detection device.

Description

The invention relates to a method and apparatus for detecting smoke by analyzing at least one video image taken by a video camera monitoring an area.

The US 5153722 A discloses a fire detection system including a color video camera, a frame grabber, and a computer processor for evaluating and storing images taken from the camera. Ultraviolet and infrared detectors signal a danger in the detection of a previously defined threshold. Then the processor evaluates the images received by the camera according to various criteria such. B. bright areas.

Efforts have recently been made to use the video systems for the detection of smoke which are already present for security monitoring in buildings, tunnels, etc. Since the video images for a viewer are often uninteresting and also caused by smoke only very small changes in the video image, monitoring by the staff at the screens is out of the question. If anything, the monitoring can only be done by an automatic evaluation of the video images. In a known method for the automatic examination of video images for the occurrence of smoke, the intensity values of the individual pixels of successive images are compared with one another. When measuring intensity values representative of a lighter image caused by the presence of smoke are closed, the presence of smoke and alarm.

One of the problems with this method is that smoke is not detected against a light background and even fire that produces little smoke is not detected. In addition, brightness changes, such as those caused by persons moving through the field of view of the camera, can trigger a false alarm. This problem has been solved by the fact that in addition to the actual surveillance area still one outer area and, in case of changes in this outer area, interrupt the observation of the surveillance area. This method has the disadvantage that a fire may not be detected until after a certain delay and that sources of smoke are not detected in the outer area provided in addition to the monitoring area.

The object of the present invention is to propose an efficient way of detecting smoke by means of at least one video image taken by a field-monitoring video camera.

The object is achieved in each case by the subject matters of the independent claims. Further developments of the invention are specified in the subclaims.

A gist of the invention is that smoke is detected by analyzing at least one video image captured by a video camera monitoring an area. An area can be a room, a tunnel (section), a parking lot, a street or a street section etc. In principle, in a first step, by determining the direction and the size in a moving region of the at least one video image, a probable presence of smoke in the moving region is checked. If a moving area has a positive test result, there is a certain probability of the presence of smoke. Thereafter, at least a part of the moving area is evaluated depending on at least one smoke characteristic information regarding the presence of smoke. According to the invention, the smoke-characteristic information is the speed of the smoke, the number of pixels in the video image that describe this movement, the luminance change (brightness change) of the at least one video image with respect to the background, the change in the color of the moving smoke and the movement of the smoke are considered.

An advantage of the method according to the invention or the device according to the invention can be seen in the fact that smoke can be detected efficiently. In particular, this is achieved by the two-part evaluation and by the appropriate selection of information characteristic of smoke.

Figur 1

ein erfindungsgemässes Blockschema zur Detektion von Rauch,

Figur 2

eine vereinfachte Darstellung eines Videobildes,

Figur 3

ein Entscheidungsdiagramm für die Detektion von Rauch,

Figur 4

eine erfindungsgemässe Vorrichtung.

The invention will be explained in more detail with reference to an embodiment shown in a figure. Show

FIG. 1

an inventive block diagram for the detection of smoke,

FIG. 2

a simplified representation of a video image,

FIG. 3

a decision chart for the detection of smoke,

FIG. 4

an inventive device.

FIG. 1 shows a block diagram according to the invention for the detection of smoke. From at least one video image, which was generated with a certain frequency, at least one intensity image [X _ij (t)] is obtained. The video image can have, for example, a size of 352x288 pixels. The next step is preprocessing. The preprocessing has the goal that the areas of interest for the detection of smoke are filtered out of the video image. For this purpose, a background accumulation matrix [B _ij (t)] is created first. The background accumulation matrix [B _ij (t)] is obtained from the weighting factor weighted intensity images [X _ij (t)], where the weighting factor α indicates how strongly the intensity _images flow into the accumulation matrix [B _ij (t)]. The accumulation matrix is determined as follows: $$\mathrm{bij}\left(\mathrm{t}\right)\mathrm{=}\mathrm{\alpha \; bij}\left(\mathrm{t}\mathrm{-}\mathrm{1}\right)\mathrm{+}\left(\mathrm{1}\mathrm{-}\mathrm{\alpha }\right)\mathrm{xij}\left(\mathrm{t}\right)\mathrm{.}\mathrm{\alpha }\mathrm{=}\mathrm{weighting\; factor}$$

Next, a subtraction matrix D _ij (t) = / B _ij ( t ) - X _ij ( t ) / is calculated for at least one moving area. The color weighting of the subtraction matrix D _ij ( t ) finally yields the color-weighted subtraction matrix [S _ij (t)].

, wobei Luma{Dij} der Helligkeitsteil von Dij, ChromaU(Dij) der U-Farbanteil von Dij and ChromaU(Dij) der V-Farbanteil von Dij ist.This subtraction matrix [S _ij (t)] is calculated from $${\mathrm{S}}_{\mathrm{ij}}\left(\mathrm{t}\right)\mathrm{=}\mathrm{Luma}\left\{{\mathrm{D}}_{\mathrm{ij}}\left(\mathrm{t}\right)\right\}\mathrm{\times }\left\{\mathrm{1}\mathrm{-}\left|\mathrm{ChromaU}\left\{{\mathrm{D}}_{\mathrm{ij}}\left(\mathrm{t}\right)\right\}\mathrm{-}\mathrm{ChromaV}\left\{{\mathrm{D}}_{\mathrm{ij}}\left(\mathrm{t}\right)\right\}\right|\right\}$$

where Luma {Dij} is the luminance part of Dij, ChromaU (Dij) is the U color part of Dij and ChromaU (Dij) is the V color part of Dij.

The likely presence of smoke at the location (i, j) is finally determined, for example, by the projection of the color-weighted subtraction matrix [S _ij (t)] on the x / y axis of a Cartesian coordinate system.

x-Projektion von Sij (t) : p_xi (t) - S_i0 (t)+ S_i1 (t)+ S_i2 (t) + ... + S_iv (t)
y-Projektion of Sij (t) : p_yj(t) = S_0j (t) + S_1j (t) + S_2j (t) + ... + S_Hj(t)The projection on a Cartesian coordinate system looks like this: $$\begin{array}{ll}\left[{\mathrm{i}}_{\mathrm{m}},,{\mathrm{j}}_{\mathrm{m}}\right]\left(\mathrm{t}\right)\mathrm{=}\mathrm{\{}\left(\mathrm{i},,\mathrm{j}\right)\mathrm{|}& \mathrm{i}\mathrm{=}\mathrm{Max}\left\{\mathrm{x}\mathrm{-}{\mathrm{projection\; of\; S}}_{\mathrm{ij}}\left(\mathrm{t}\right)\right\}\mathrm{.}\\ & \mathrm{j}\mathrm{=}\mathrm{Max}\left\{\mathrm{y}\mathrm{-}{\mathrm{projection\; of\; S}}_{\mathrm{ij}}\left(\mathrm{t}\right)\mathrm{\}}\right\}\end{array}$$

x-projection of Sij (t): p _xi (t) - S _i0 (t) + S _i1 (t) + S _i2 (t) + ... + S _iv (t)
y-projection of Sij (t): p _yj (t) = S _0j (t) + S _1j (t) + S _2j (t) + ... + S _Hj (t)

Sij in this example has the magnitude HxV (H = smoke velocity x smoke movement = V). Of course, the choice of the coordinate system is arbitrary. For example, spherical coordinates, cylindrical coordinates, etc. could also be used.

With the aid of the color-weighted subtraction matrix [S _ij (t)], a probable presence of smoke in a moving area of the video image can then be checked. In the case of a likely presence of smoke, a reduced video image area of interest (ROI = Region of Interest) compared to the original image is defined. Of course, multiple ROI areas can also be defined in one video image or in multiple channels. By reducing the data to about 1: 100, for example, the size of the ROI can be 8x128 pixels, the processor load for the actual analysis or evaluation is considerably reduced. Whether smoke is present in a moving area of the recorded video image is clarified on the basis of at least one information characteristic of smoke. In the present example, the following information is used to increase the detection security.

As information characteristic of smoke, the speed of the smoke (movement of the smoke), the number of pixels (active pixels) describing this movement, the luminance change (brightness change) of the at least one video image with respect to the background, the change of the color ( Color change) of the moving smoke and the movement of the smoke (y-position in the histogram) viewed.

• die Rauchbewegung von SROI(t): v(t) = Zeitkorrelation der γ-Projektion von SROI(t), zum Beispiel pyj(t),
• die Varianz von BROI(t) und XROI(t), die zur Bestimmung der Helligkeitsveränderung relativ zum (normalen) Hintergrund: 1(t) = 1 - var{B_ROI(t)}/var{B_ROI(t)},
• aktive Pixel von S_ROI(t): a(t) = Anzahl der Pixel von S_ROI(t) mit einem Wert grösser als 0,
• Farbwechsel: c(t) = Anzahl der Pixel mit {1-|ChromaU(D_ROI(t)) - ChromaV(D_ROI(t)|} < Schwellwert,
• y-Position im Histogramm: h(t) = Werte der y-Projektion von S_ROI(t), zum Beispiel p_yj(t) wird genutzt um ein Histogramm mit 64 Kanälen zu erstellen

For each ROI range, the following information characteristic of smoke is calculated:

• the smoke movement of SROI (t): v (t) = time correlation of the γ-projection of SROI (t), for example, pyj (t),
• the variance of BROI (t) and XROI (t) used to determine the change in brightness relative to the (normal) background: 1 (t) = 1 - var {B _ROI (t)} / var {B _ROI (t)}
• active pixels of S _ROI (t): a (t) = number of pixels of S _ROI (t) with a value greater than 0,
• Color change: c (t) = number of pixels with {1- | ChromaU (D _ROI (t)) - ChromaV (D _ROI (t) |} <threshold,
• y-position in the histogram: h (t) = values of the y-projection of S _ROI (t), for example p _yj (t) is used to create a histogram with 64 channels

Thereafter, the smoke-characteristic information v (t), l (t), a (t), c (t) and h (t) are integrated over a certain time and thus over several images. The function looks like this, for example: $$F_X=\underset{{\mathrm{t}}_{\mathrm{O}}\mathrm{<}\mathrm{t}\mathrm{<}{\mathrm{t}}_{\mathrm{n}}}{X}=\mathrm{.SIGMA.X}\left(t\right)\mathrm{with\; X}\mathrm{=}\mathrm{V}\mathrm{.}\mathrm{L}\mathrm{.}\mathrm{A}\mathrm{.}\mathrm{C}\mathrm{.}\mathrm{H}$$

From the information integrated over time, the respective mean value is determined. Mean smoke movement F _V = V Mean value brightness change F _L = L Mean active pixels F _A = A Mean color change F _C = C Mean y position in the histogram F _H = H

Thereafter, the probability of the presence of smoke is calculated for each of these averages. this happens about the pattern recognition. For each average, a dis-discriminator value Ψ is determined. For example, a threshold δ (or even a probability function) may define the discriminator in the following manner:

oder 0 ≤ Γ(F _L ) ≤ 1, mit Γ(x) als WahrscheinlichkeitsfunktionFor the brightness change $${\mathrm{\Psi }}_L=\mathrm{\Gamma }\left({\mathrm{F}}_L\right)=\{\begin{array}{c}{\mathrm{F}}_L>{\mathrm{\delta }}_L.{\mathrm{then\; \Psi }}_L=1\\ {\mathrm{F}}_L<{\mathrm{\delta }}_L.{\mathrm{then\; \Psi }}_L=0\end{array}$$

or 0 ≤ Γ (F _L ) ≤ 1, with Γ (x) as the probability function

oder als Mittelwert aller Diskriminatoren $$\aleph \left(\mathrm{t}\right)=1/{\mathrm{N}}_{\mathrm{F}}{\displaystyle \sum _{i=\mathrm{Information}}}{\mathrm{\Psi }}_{\mathrm{i}}=\left\{{\mathrm{\Psi }}_V+{\mathrm{\Psi }}_L+{\mathrm{\Psi }}_A+{\mathrm{\Psi }}_C+{\mathrm{\Psi }}_H\right\}/{\mathrm{N}}_{\mathrm{F}}$$

, wobei N_F = 5 die Anzahl der Informationen ist.The smoke pattern is defined by the product of all discriminators $$\aleph \left(\mathrm{t}\right)={\displaystyle \underset{i=\mathrm{information}}{\Sigma }}{\mathrm{\Psi }}_{\mathrm{i}}=\left\{{\mathrm{\Psi }}_V\cdot {\mathrm{\Psi }}_L\cdot {\mathrm{\Psi }}_A\cdot {\mathrm{\Psi }}_C\cdot {\mathrm{\Psi }}_H\right\}$$

or as the mean of all discriminators $$\aleph \left(\mathrm{t}\right)=1/{\mathrm{N}}_{\mathrm{F}}{\displaystyle \underset{i=\mathrm{information}}{\Sigma }}{\mathrm{\Psi }}_{\mathrm{i}}=\left\{{\mathrm{\Psi }}_V+{\mathrm{\Psi }}_L+{\mathrm{\Psi }}_A+{\mathrm{\Psi }}_C+{\mathrm{\Psi }}_H\right\}/{\mathrm{N}}_{\mathrm{F}}$$

where N _F = 5 is the number of pieces of information.

(t) = 1 dann wird I(t) - I(t-1) + σ₊ (hinzugefügt zu S₊ falls I(t) > S₊) sonst I(t) = I(t-1) - σ-(hinzugefügt zu S- (üblicherweise 0) falls I(t) < S-) , wobei σ+,σ- üblicherweise den Wert +1 annimmtFinally, the decision is made as to whether the moving area of the video image is the image of smoke. For this purpose, an integrator I (t), which increases or decreases by a value σ, is determined
I (t = 0) = 0; if

(t) = 1 then I (t) - I (t-1) + σ ₊ (added to S ₊ if I (t)> S ₊ ) else I (t) = I (t-1) - σ- (added to S- (usually 0) if I (t) <S-) , where σ +, σ- usually takes the value +1

Falls I(t) > κ

dann Rauch
sonst kein Rauch

Smoke is detected and, for example, an alarm is triggered if I (t) exceeds a critical value κ:

If I (t)> κ

then smoke
otherwise no smoke

FIG. 2 shows a simplified representation of a video image VB. The image contains a moving area that is supposed to be smoke. Furthermore, the video image VB shows an ROI range, which according to the description of FIG. 1 was determined.

FIG. 3 shows a decision chart for the detection of smoke, as shown in FIG. 1 is described. If I (t) exceeds a certain threshold κ, alarm is triggered and smoke was detected with high probability. So that I (t) does not rise to infinity and thus unnecessarily reduces the reaction time for smoke detection, a maximum value I _{T is} defined. The critical time is the time until the alarm is triggered. This time should be as short as possible.

FIG. 4 shows an inventive device VR with a receiving unit E and a transmitting unit S for communicating, for example, with other units, such as sensors, CPUs, etc. and a processing unit V for performing the method according to FIG. 1 , The device can be integrated in a video camera, a central unit, etc., or can be a separate unit.

Claims (13)

1. Method for detection of smoke by analysis at least of one video image recorded by a video camera monitoring an area, with at least one moving area of the at least one video image being checked for the probable presence of smoke by determining the direction and the size of the moving area, with, if the outcome of the check is positive, at least a part of the at least one moving area being evaluated depending on at least one item of information characteristic for smoke in respect of the presence of smoke,
   characterised in that
   at least one video image is created with a specific frequency and from this at least one intensity image [X_ij(t)] is obtained and
   that a background accumulation matrix [B_ij(t)] is used, which is obtained from intensity images (X_ij(t)] weighted with a weighting factor, with the weighting factor specifying how strongly the intensity images flow into the accumulation matrix [B_ij(t)].
2. Method according to claim 1,
   characterised in that
   the speed of the smoke, the number of pixels which describe this movement, the luminance change
   of the at least one video image in relation to the background, the change of the colour of the moving smoke and the movement of the smoke are used as the at least one item of characteristic information.
3. Method according to claim 1,
   characterised in that
   at least one moving area is determining with the aid of a subtraction matrix D_ij(t) = |B_ij(t) -X_ij(t)|.
4. Method according to claim 3,
   characterised in that
   a colour-weighted subtraction matrix (S_ij(t)) is obtained from the subtraction matrix [D_ij(t)].
5. Method according to claim 4,
   characterised in that
   with the aid of the colour-weighted subtraction matrix [S_ij(t)] a probable presence of smoke in a moving area of the video image is checked and if the outcome of the check is positive a region of interest (ROI) reduced by comparison with the original image is defined.
6. Method according to claim 5,
   characterised in that
   the region of interest (ROI) represents at least a part of the moving area of the video image.
7. Method according to claim 5,
   characterised in that
   the region of interest (ROI) represents a rectangle with the length/width ratio of 16:1.
8. Method according to claim 4,
   characterised in that
   the probable presence of smoke at the location (I, j) is determined by the projection of the colour-weighted subtraction matrix [S_ij(t)] onto the x/y axis of a Cartesian coordinate system.
9. Method according to one of the previous claims,
   characterised in that
   at least one item of information characteristic for smoke is evaluated in the region of interest in the video image.
10. Method according to claim 9,
    characterised in that
    the at least one said item of information characteristic for smoke is integrated over a specific time and thus over a number of images and its average value is determined and the probable presence of smoke is computed for each of these average values.
11. Method according to claim 10,
    characterised in that
    the probability of whether smoke is present is determined by the comparison with a threshold value δ8 and/or by a probability function Γ(x).
12. Method according to claim 10 and 11,
    characterised in that
    an overall probability for the presence of smoke in the region of interest of the video image is computed from the probabilities of the average values, that this overall probability is integrated over a number of images.
    and that, if a threshold value (K) is exceeded, an alarm is triggered by the integrated value.
13. A device (VR) for detection of smoke by analysis of at least one video image recorded by a video camera monitoring an area,

    - with a receiver unit (E) and a transmitter unit (S) to carry out communication with further units,

    - with a processing unit (V) to check for the probable presence of smoke by determining the direction and the size of a moving area of the at least one video image, with a positive test result, to evaluate at least a part of the at least one moving area depending on at least one item of information characteristic of the smoke in respect of the presence of smoke, to create at least one video image with a specific frequency and to obtain at least one intensity image [X_ij(t)] from this and to obtain a background accumulation matrix (B_ij(t)] from the colour-weighted intensity images [X_ij(t)] weighted with a weighting factor, with the weighting factor specifying how strongly the intensity images flow into the accumulation matrix [B_ij(t)].

Images