JPH03280688A - Picture monitor - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

DETAILED DESCRIPTION OF THE INVENTION [Configuration of the Invention (Field of Industrial Application) The present invention relates to an image monitoring device used for suspicious person detection, etc.

(Conventional Technology) With the development of the information society, the importance of various types of security is increasing. In order to deal with such security problems, various image monitoring systems used for detecting suspicious persons and the like have been developed. This image monitoring system
Conventionally, in general, images of a predetermined surveillance area (time-series images) captured using a TV camera are output to a monitor, and are constantly monitored by a surveillance officer. Moreover, the number of areas subject to monitoring is increasing.

However, in such a system, the workload placed on the image monitor is very heavy, and there is a problem of decreased reliability due to oversight of changes on the monitor image due to fatigue of the monitor.

In order to solve these problems, several systems have been proposed that perform automatic monitoring by image processing (processing) input images obtained through, for example, a TV camera. If the monitoring target moves over time, the monitoring area will be automatically changed accordingly, or the image in the standard state and the image in the monitoring state will be compared to detect abnormalities. configured.

Image processing in such a system basically consists of finding the difference between input images (scenes) obtained in chronological order or with a background image, and detecting and determining changes from this difference information. This is realized by However, such a determination result can only detect the presence or absence of a change. For this reason, in monitoring areas where people are constantly coming in and out, such as banks and department stores, a large number of changing forces occur, so monitoring such as tracking the movements of a specific person (suspicious person) is carried out. It is not possible.

On the other hand, features that can identify the detection target are extracted from images that are input in time series, and motion parameters are extracted by associating the detected features, for example, to track the movement of a person. There are moving image recognition methods that perform processing. However, if we assume a situation where a person approaches from a distance, it is extremely difficult to extract the characteristics at the beginning. Therefore, at the beginning, only changes between images in time series can be detected, and if the object to be detected gets close enough to detect its features, then the object to be detected must be tracked by the feature extraction described above. There is a problem that it cannot be done.

(Problem to be solved by the invention) In this way, conventional image monitoring methods only perform various processing on images that are input in chronological order. There has been a problem in that the system cannot function effectively in situations where suspicious person determination is performed while monitoring images for a long period of time, such as in cases where a suspicious person is detected.

The present invention has been made in consideration of these circumstances, and its purpose is to reduce the deterioration in detection performance caused by performing such processing on images input in time series. An object of the present invention is to provide a highly practical image monitoring device that can stably detect and monitor moving objects in a real environment.

[Configuration of the Invention (Means for Solving the Problems) An image monitoring device according to the present invention includes means for extracting a changing area between each scene of time-series images inputted in chronological order, and a means for extracting a changing area between each scene of time-series images input in a time-series manner, and a means for detecting a changing area. The device is equipped with means for extracting features in the time-series images when the time-series images are detected, and means for using the features to observe the movement status of the tracked object moving in the time-series images. The present invention is characterized in that observation of the situation is started or activated from the time when it is determined that the characteristics of the moving tracking target can be identified.

In other words, moving objects are monitored by detecting changing regions between time-series images, and features are extracted from the point at which the moving object approaches a point where its features can be extracted. The feature is that a moving object is tracked using features.

(Function) According to the present invention configured as described above, when a moving object is located far away, macroscopic tracking of the moving object is performed by detecting changing regions from time-series images. Image monitoring is easier and more stable than tracking a moving object. When the moving object is sufficiently close, tracking processing is performed while extracting features that can identify the moving object, so feature points can be mapped without degrading monitoring performance due to errors, etc. , it becomes possible to monitor moving objects with high precision and reliability.

(Example) Hereinafter, an example of an image monitoring device according to the present invention will be described with reference to the drawings.

FIG. 1 is a schematic block diagram showing the overall configuration of the apparatus according to the embodiment, and reference numeral 1 denotes an image input unit that sequentially stores time-series images taken and inputted by a surveillance TV camera in an image memory and subjects them to image processing. The change detection section 2 calculates the difference between the time-series images (between image scenes) obtained by the image input section 1, and detects, for example, an area with a large change in brightness as a changed area. Further, the feature extraction section 3 performs, for example, spatial differentiation, binarization processing, etc. on the time-series images input by the image input section 1 to extract feature points thereof.

The predictive tracking unit 4 inputs the information on the change point obtained by the change detection unit 2 and the information on the feature point obtained by the feature extraction unit 3. If the area of the large region is small, then
The information on the feature points obtained by the feature extraction section 3 is ignored. Then, only a determination is made as to whether or not the change area relates to a moving object. However, the predictive tracking unit 4
When the area of the changed area exceeds the size, that is, the area of the changed area extracts its features, and when the area reaches a size that can be identified, information on the feature points obtained by the feature extraction unit 3 is taken in. . The moving object is then tracked by calculating the moving direction of the moving object using the feature extraction results.

In accordance with the information on the movement direction of the moving object obtained by the predictive tracking section 4 in this manner, the camera attitude control section 5 controls the viewing direction of the image captured and input by the image input section 1. For example, the attitude of the monitoring TV camera is controlled so that the moving object to be tracked is positioned at the center of the image to be captured and input.

Each part of this apparatus having the above-mentioned configuration will be roughly explained below.

FIG. 2 is a diagram showing an example of the configuration of the image input section 1. As shown in FIG.

The TV left camera 1 installed in the surveillance environment captures and inputs an image of the surveillance area, and its input image signal is an NTS
After being subjected to predetermined TV signal processing via the C decoder 12, the signal is converted into a digital signal by an A/D converter 3.

The input images converted into digital signals (time-series images consisting of scenes with a predetermined cycle) are sequentially stored from the ITV interface 14 via the image bus 15 into an image memory group 16 consisting of a plurality of image memories. In other words, images of the monitoring environment input in chronological order are sequentially stored in individual image memories for each scene.

Incidentally, writing and reading of images to and from this image memory group 16 is controlled via a control bus 17 for each image memory.

A change detection section 2 that detects changes between images from the time-series images stored in the image memory group 16 in this way is connected to the image bus 15 and the control bus 17, for example, as shown in FIG. Differential circuits 21 connected to each other. Code identification circuit 22. and a small area removal circuit 23. The difference circuit 21 reads each successive image stored in the image memory group I6 and determines the difference in brightness between these images. The code identification circuit 22 separates and extracts the difference in brightness into a positive part and a negative part. The small area removal circuit 23 removes small area areas from among the difference areas extracted as described above as noise areas, and detects only the remaining difference area as a change area between images.

On the other hand, the feature extraction section 3 is configured as shown in FIG. 4, for example. This feature extracting section 3 reads out time-series images stored in the image memory group 1B in parallel with the change detecting section 2, and performs differential digital filtering processing on the image signal in a spatial filter circuit 24. , this is 2
Binarization processing is performed via the digitization circuit 25. This process detects, for example, a portion of the image that has a large change in brightness. The small area removal circuit 26 operates similarly to the small area removal circuit 23 in the change detection unit 2 described above to detect small areas among the image components (components with large brightness changes) that have been subjected to the filtering process and the binarization process. The area is removed as a noise component. Alternatively, this small area removal circuit 26 may also be used as the small area removal circuit 23 in the change detection section 2.

The line thinning circuit 27 thins the image signal that has been subjected to the above-mentioned processing to, for example, a line width of 1. By passing this thinned image through the logic filter circuit 28, for example, as shown in FIG. a) The positions of feature points such as the upper end point, lower end point, and one-way branch point as shown in (b) to (d), respectively, in the image are detected.

Information on the feature points on the image detected in this way is sequentially stored in the feature point memory 29 via the control bus 17.

Now, the predictive tracking section 4 includes, for example, a change area discriminating circuit 31. as shown in FIG. Moving object determination circuit 32. Corresponding point search circuit 33. Movement parameter calculation circuit 34. and a prediction knowledge file 35. Change area discrimination circuit 3
1 inputs the information on the changed areas between images obtained by the change detection unit 2, compares the area of each changed area with a predetermined threshold value, and compares the area of each changed area to a size large enough to identify the characteristics of the changed area. It is determined whether the At this time, for a changed area with an area smaller than the threshold value, the moving object determination circuit 32 determines whether the change is caused by the presence of a moving object or by disturbances such as fluctuations in illumination. Determine. This determination process by the moving object determination circuit 32 uses knowledge (laws) stored in the predictive knowledge file 35 in advance.

This predictive knowledge includes, for example, the knowledge that movement perpendicular to the viewing direction of the camera can be approximated by parallel movement within a short time range. In other words, when object A moves in the horizontal direction under a -like background as illustrated in FIG. 7(a), the difference between it and object B, which has moved, is as shown in FIG. 7(b). It is detected as areas with different signs on both sides of the object. The positive and negative signs of such a difference area are obtained as the identification result of the code identification circuit 22 described above.

Furthermore, based on the assumption that the size and direction of movement of an object do not change significantly over a short period of time, as mentioned above, between successive images, adjacent areas of the difference image have areas with different positive and negative signs. If a moving object is detected, it is also given as predictive knowledge that there is a very high possibility that a moving object exists there. A moving object determined in this way can be considered to be an object moving perpendicular to the viewing direction of the camera. Note that in this example, at the same time as the moving object is determined, the moving object is also tracked in area units.

However, if the area of each partial region is determined to be larger than the predetermined threshold by the determination based on the predetermined threshold for the area of each partial region described above, that state indicates that the moving object is sufficiently close to the camera. be judged. The corresponding point search unit 33 is activated according to such a determination result.

The corresponding point search unit 33 reads the feature points included in the detected partial region as described above from the feature point memory 29, and uses similarity indexes such as density correlation around the feature points to search for features in consecutive image tubes. It associates points with each other. The motion parameter calculation circuit 34 calculates the motion parameters for the moving object by solving simultaneous equations from the pairs of feature points in the same region that are associated in this manner.

Calculation of this motion parameter requires coordinates (x
,y+z) in the input image is P(x
/z, y/z) according to the theory of central projection. However (X.

Let the moving speed of the moving object existing at (y, z) be (u.

v, w), the above moving object becomes Q after time t
Move to (x+ut, y+v t, z+wt). Therefore, the position of the moving object in the image projected onto the input image is as follows. Therefore, these two corresponding points are the vanishing point (u
/w, v/w). Therefore, if simultaneous equations are analyzed using two or more pairs of feature points belonging to the same object, the above-mentioned vanishing point (u/w).

v/w) is determined, and the moving direction of the moving object is determined from this vanishing point.

The camera attitude control section 5, which operates using the information on the moving direction of the moving object obtained in this way, is configured as shown in FIG. 8, for example. That is, this camera attitude control section 5
A moving object position prediction circuit 36, a visual field determination circuit 37. Camera controller 38. and an alarm circuit 39. The moving object position prediction circuit 36 uses information on the moving direction of the object obtained by the motion parameter calculation circuit 34 and the change based on the assumption that the moving state will not change significantly in a short period of time. The speed of the moving object estimated from the temporal change in the change area determined by the detection unit 2 is used to predict the position where the object is expected to exist at the next time.

The field of view determination circuit 37 determines whether the field of view of the TV camera 11 mentioned above can be directed to the predicted position in this way, and activates the camera controller 38 according to the determination result. As a result, the camera controller 38 controls the attitude of the camera so that the moving object is located at the center of the image.

In addition, if it is determined that the attitude control of the camera is impossible compared to the movement of the moving object, the alarm circuit 39 is activated to appropriately notify the operator and prompt the observer to directly monitor the object. It will be done.

According to this device configured in this way, when a moving object is located far away, its movement status is macroscopically monitored based only on the information on changing points, and when the moving object approaches sufficiently, its characteristic points are detected. When the moving object becomes detectable, the moving object is tracked by extracting feature points from the image.

As a result, it becomes possible to switch the monitoring mode according to the information content required from the input image and effectively automatically monitor a moving object (suspicious person).

In the above-mentioned embodiment, it was determined from the area of the change region whether the moving object has approached the position where the feature point in the input image can be identified. Alternatively, the distance to the moving object may be measured, and depending on the measurement result, switching control may be performed to determine whether or not to start tracking processing of the moving object using feature point information.

By the way, the above-mentioned device can be applied to a system that measures the number of vehicles and people passing through an observation point.

FIG. 9 is a block diagram showing a schematic configuration of such an application system, and the same parts as those of the previously shown embodiment are denoted by the same reference numerals.

This system differs from the apparatus of the above-described embodiments in that, in place of the predictive tracking section 4, it incorporates a measuring section 6 configured as shown in FIG. 10, for example. That is, in this system, the above-described image input section 1 and change detection section 2.
and the feature extractor 3 are used as they are, that is, with the same configuration.

It goes without saying that the surveillance camera 11 is installed with its field of view facing the observation point.

However, in the lever measurement unit 6, the change area determination circuit 31 determines the movement of the moving object by applying the knowledge law stored in the predictive knowledge file 35 in the same manner as explained in the predictive tracking unit 4 described above. Identify and detect the region of change caused by the change. Thereafter, the moving object determination circuit 32 compares the size of the changed area with a predetermined threshold value that allows characteristic identification, and detects a changed area that exceeds the threshold value. Regarding the change area detected in this way, the measurement target determination circuit 41 determines whether the moving object is to be measured. The moving object to be measured is registered in advance in the measurement target model file 42.

By comparing the information stored in this measurement target model file 42 with the feature point information stored in the feature point memory 29 described above, it is determined whether the moving object is a measurement target. That will happen. If it is determined that the moving object is the measurement target, the determination result is given to the counting circuit 43, and by sequentially counting the results, the number of vehicles and people passing through the observation point is counted. Become.

Of course, the size of the moving object or the like may be used as the characteristic used for such measurement.

Note that the present invention is not limited to the embodiments described above. For example, when applied to suspicious person detection, a face image of a person with a criminal record is stored in the above-mentioned measurement target model file 42, and the moving object determination circuit 32 detects a moving object (person) approaching the camera. The head portion may be extracted from the input image at the time when it is determined that the input image is correct, and the head portion may be sequentially compared with the face images stored in the measurement target model file 42. At this time, it is also preferable to issue a warning to the operator when a moving object corresponding to the face image is detected.

In addition, when investigating the migratory status of a school of fish, it is also possible to install a camera underwater or above the water and configure the system to detect the migratory status of the entire school of fish by averaging the movement directions of individual fish. It is possible. In addition, the present invention can be implemented with various modifications without departing from the gist thereof.

[Effects of the Invention] As explained above, according to the present invention, the degree of proximity between the camera and the object is determined from the size of the changing area in the input image, and image monitoring is performed by detecting the changing area on a macroscopic scale. ,
Since the processing is switched to tracking using features that can identify the moving object, it becomes possible to easily determine the precise direction of movement of the moving object. Furthermore, compared to the case where a moving object is tracked using features from the beginning, it is possible to reduce search errors and significantly reduce the processing time. As a result, great practical effects can be achieved, such as the ability to perform highly efficient and reliable image monitoring.

[Brief explanation of the drawing]

The figures show an embodiment of an image monitoring device according to the present invention, in which FIG. 1 is a block diagram showing a schematic configuration of the embodiment device, and FIG. 2 is a diagram showing an example of the configuration of an image input section in the embodiment device. FIG. 3 is a diagram showing an example of the configuration of the change detection section, FIG. 4 is a diagram showing an example of the configuration of the feature extraction section, FIG. 5 is a diagram showing an example of feature points in an image, and FIG. 6 is a diagram of the predictive tracking section. FIG. 7 is a diagram illustrating an example of predictive knowledge, and FIG. 8 is a diagram illustrating an example configuration of a camera attitude control section. Further, FIG. 9 is a block diagram showing a schematic configuration of a system for measuring the number of passing objects to which the present invention is applied, and FIG. 10 is a diagram showing an example of the configuration of the eaves side part of the system shown in FIG. 9. 1... Image input unit, 2... Change detection unit, 3... Feature extraction unit, 4... Predictive tracking unit, 5... Camera posture control unit, 6... Measurement unit.

Claims (2)

[Claims] (1) means for inputting time-series images; means for extracting changing regions between scenes of the inputted time-series images; means for extracting features in the time-series images; and information on the changing regions and the and means for observing the movement status of the tracking target moving in the time-series images using the characteristics, and the observation of the movement status of the tracking target is performed at a point in time when it is determined that the characteristics of the moving tracking target can be identified. An image monitoring device characterized in that tracking using the feature is started or enabled from. (2) The image monitoring device according to claim (1), comprising means for detecting the distance to the tracking target, and starting tracking using features extracted from the time-series images according to the detected distance; or an image monitoring device that determines activation.