Summary of the invention
Technical problem to be solved by this invention is to provide a kind of high video monitoring system of recall precision of browsing.
In order to solve the problems of the technologies described above, the present invention proposes a kind of video monitoring data storing and managing system, comprising:
Video analysis device is used for the video monitoring data that obtains is carried out the modeling of reference background frame with acquisition reference background frame, and carries out the motion object detection to obtain the motion object according to this reference background frame; The characteristic information that extracts this motion object is as index information, and according to the number of motion object among the result of described motion object detection described video monitoring data cut apart and to be formed the segmentation video monitoring data;
Video encoder is used for every section video monitoring data is carried out video coding;
Memory is used to store the video data after the described video encoder encodes;
Data management system is used for the described index information of storage administration as data base management system;
Input/output unit is used to show described index information and video data; And according to user input index information that described data management system is stored is inquired about/edit, so also the video data demonstration of reading in the described memory according to described information is exported.
Wherein, the modeling of described reference background frame utilizes gauss hybrid models that every section video monitoring data is carried out modeling and obtains the reference background frame.
Wherein, described motion object detection comprises: the gray level image of the current frame image of described reference background frame and current video monitor data is subtracted each other obtain the motion object.
Wherein, when described video monitoring image is coloured image, also write down the position coordinates of described motion object in the described motion object detection, and in the coloured image of described current frame image, determine colored motion object by this position coordinates.
Wherein, the characteristic information of described motion object comprises in its histogram information, shape information, the marginal information one at least.
Wherein, described number according to motion object among the result of described motion object detection is cut apart described video monitoring data and comprised: the frame position of motion object is not cut apart as the described video monitoring data of minute bound pair; Perhaps, if the number of the motion object of a certain frame is less than preset value, then its position is cut apart as the described video monitoring data of minute bound pair.
Wherein, this system applies is in the traffic control field.
Wherein, described index information also comprises the key frame of every section described video monitoring data.
Wherein, described video analysis device also is used for:
Behind the characteristic information of having determined described motion object, mate with default alarm events, if satisfy described alarm events, then by described input/output unit output alarm signal.
Wherein, described input/output unit comprises:
The application system middleware is used to receive from the index information of described video analysis device and exports following monitor to and shows; Also conduct is inquired about/is edited the index information of storing in the described data management system with the interface reception user's of described data management system input information, and Query Result is shown in described monitor, also from described memory, read the video monitoring data decoding and export described monitor to and show according to described index information; Also being used for exporting described monitor to decoding from the video monitoring data of video encoder shows;
Monitor is used to carry out the demonstration of index information and video monitoring data.
Among the present invention, owing to obtained comprising motion characteristics of objects information index information according to the motion object detection, and this index information shown and store, thereby can carry out the inquiry and the retrieval of video by this index information, make inquiry and recall precision improve greatly.And, because video data has been carried out segmentation, give up, thereby the memory space of minimizing video data has reduced equipment cost not comprising the frame of motion object or comprising the few frame of motion object (being less than set point).
Embodiment
The present invention will be described in detail below in conjunction with accompanying drawing.
With reference to figure 1, illustrate the structural representation of an embodiment of a kind of video monitoring data storing and managing system of the present invention.As shown in the figure, the scope for frame of broken lines comprises:
Video analysis device 12 is used to receive the video monitoring data from video camera 11, and these data are carried out the modeling of reference background frame, carries out the motion object detection after modeling is finished to extract the motion object;
After extracting the motion object, the characteristic information that calculates this motion object is with as index information; Also video monitoring data is cut apart to form the segmentation video monitoring data, this segmentation video monitoring data is inputed to following video encoder 13 according to the number of motion object in every two field picture; Also be used for mating,, then produce alarm signal to following application system middleware 16 and data management system 14 if the match is successful according to default alarm events and motion object;
Wherein, described index information is at every section video monitoring data after cutting apart, every section corresponding index information of video monitoring data.This index information not only comprises the characteristic information of described motion object, also comprises the recording time information of video monitoring data, and this section video monitoring data key frame.Described key frame can be got first frame of this section video monitoring data, perhaps other any frame.
Video encoder 13 is used for the segmentation video monitoring data from described video analysis device 12 is encoded.And the segmented video data after will encoding a tunnel exports memory 15 to and stores, and another road exports described application system middleware 16 to.
Data management system 14 is used for as index information and the alarm signal of data base management system storage administration from described video analysis device 12;
Memory 15 is used to store the segmented video data behind the coding of described video encoder 13 outputs;
Application system middleware 16 is used to receive index information and alarm signal from video analysis device 12, and this index information and the real-time described monitor 17 that exports to of alarm signal are shown output; And,,, accept user's input, and in described data management system 14, search/edit corresponding index information according to user's input to inquiry/editing interface that the user provides video monitoring data also as the interface between user and the data management system 14.
Also be used to receive the segmentation video monitoring data from behind the coding of video encoder 13, the described monitor 17 that exports to that its decoding back is real-time shows.
The course of work for present embodiment can be with reference to elaboration embodiment illustrated in fig. 2.
With reference to figure 2, illustrate workflow diagram embodiment illustrated in fig. 1.As shown in the figure, may further comprise the steps:
Step 21, Video Capture.Obtain video monitoring data in this step from video camera;
Step 22, the modeling of reference background frame.
This step is carried out the modeling of reference background frame to described video monitoring data, for example can adopt the mixed model of Gauss that each pixel is carried out modeling, just can obtain a reference background frame after the modeling.
Every execution at regular intervals of this step was carried out once once to determine up-to-date reference background frame in for example per 30 minutes.
Step 23, the motion object detection.
In this step,, then at first the video monitoring data of described reference background frame and described colour is converted to gray level image if the video monitoring image of camera acquisition is colored.Each frame and described reference background frame with described video monitoring image under gray level image subtract each other, thereby obtain motion object wherein.
If what described video camera itself gathered is exactly gray level image, does not then need to change and directly do the operation that subtracts and to obtain the motion object.
For the color video monitor data, after having determined gray scale motion object, also need position coordinates according to this gray scale motion object, on the coloured image of correspondence, determine real motion object.
Step 24, video is cut apart.
In this step, according to the number of motion object in every two field picture video monitoring data is carried out video and cut apart, obtain the video monitoring data of segmentation.For example:
With reference to figure 4, illustrate the schematic diagram that the present invention carries out the embodiment that video cuts apart.As shown in the figure, having showed the video sequence that comprises 7 two field pictures, is respectively that a, b, c, d, e, f, g indicate.
Wherein, determine to comprise 6 motion objects (shown in the ellipse) through the motion object detection in a frame, determine to comprise 3 motion objects through the motion object detection in the b frame, determine to comprise 4 motion objects through the motion object detection in the c frame, determine to comprise 1 motion object through the motion object detection in the d frame, determine to comprise 6 motion objects through the motion object detection in the e frame, determine to comprise 3 motion objects through the motion object detection in the f frame, determine to comprise 5 motion objects through the motion object detection in the g frame.
The threshold value of supposing to preestablish the number of motion object is 2, promptly when the number of motion object in the two field picture smaller or equal to 2 the time, think that then the motion object that this two field picture comprises is less, should cut apart herein.
So the d frame only comprises 1 motion object,, thereby cut apart in its position less than threshold value 2.So just, the video sequence shown in Fig. 4 is divided into two sections.Also cut apart for the whole video monitor data, just can obtain the plurality of sections segmented video data by this method.
And for the d frame, it can be included into the last period, also can be included into back one section, and the latter deletes it, to save memory capacity.
Step 25, calculated characteristics information.
This step is calculated the characteristic information of motion object in each segmented video data after the video segmentation procedure finishes.The characteristic information has here comprised attributes such as the histogram, shape, edge of motion object.Owing in general Digital Image Processing book, more detailed introduction is arranged all, so do not give unnecessary details at this.
Step 26 produces index information.
In this step, with the characteristic information that calculates, and the temporal information during the camera acquisition video monitoring data 31 is as the index information of corresponding segmented video data; This index information also comprises the key frame of corresponding segments video data, for example can be first frame of corresponding section video data or middle frame or the like.
Wherein, described temporal information 31 comprise the corresponding segments video data recording starting time, record information such as concluding time, duration and frame number.
Step 27, data management system and middleware.
In this step, the index information that step 26 is obtained is sent to described data management system and carries out storage administration, and is sent to described application system middleware.
After described application system middleware is received described index information, just it is exported in the monitor and show.The demonstration of described monitor can be with reference to embodiment illustrated in fig. 3.
Step 28, video encoder.
In this step, to cut apart the segmented video data that obtains through step 24 video and carry out video coding, for example can adopt and H.264/MPEG-4/AVS wait video encoding and decoding standard, and the segmented video data after will encoding exports memory and described application system middleware to.
Step 29 exports memory and middleware to.
In this step, described memory is just stored after receiving segmented video data behind the described coding; Described application system middleware is at first decoded after receiving segmented video data behind the described coding, is sent to described monitor after decoding finishes and shows.An embodiment of the demonstration of described monitor can be with reference to figure 3.
Step 30, alarm produces.
In this step, detecting the motion object of determining according to motion object detection step 23 mates with the alert definition incident that pre-sets, if the match is successful, then show certain incident that need report to the police that taken place, just this alarm signal is exported to described data management system and application system middleware this moment.
For example, predefined alarm events is just to report to the police when detecting red motion object, then is redness when step 23 detects the motion object that obtains, and then sends alarm signal to described data management system and application system middleware.
The definition of described alarm events, main characteristic information and number, for example border of motion object, color, number, shape, histogram etc. according to described motion object.
Because the existence of this step can be removed monitor staff's real-time monitoring from, only after taking place, alarm events just notifies the monitor staff by the mode of alarm signal.So, just saved the tired degree of manpower and monitoring work.
With reference to figure 3, illustrate the schematic diagram of an embodiment of monitoring image display structure.As shown in the figure, comprising:
Disk information block, it has comprised the position of current index information at described data management system, and corresponding segmented video data is stored in the position in the described memory.For example:
When the user determines to check the segmented video data at a certain index information place, then from described memory, read corresponding segmented video data and show by information such as recording times; Perhaps, also can when the described segmented video data of storage, set up the corresponding relation of memory address and described index information with easy-to-look-up.
Index information and corresponding segments sequence of frames of video (also being the corresponding segments video data), described index information is from the information that step 26 produced, and the video sequence of described corresponding section then exports the application system middleware to from step 28 and through decoded video data.
Described index information then has and has comprised: the recording ETAD expected time of arrival and departure, record the concluding time of the video sequence of corresponding section, key frame images, moving region and feature, movement velocity grade, frame number and duration.
For record ETAD expected time of arrival and departure, record the concluding time, key frame images, moving region (motion object) and the explanation of feature, frame number and duration no longer repeat, can be with reference to above.
Wherein, described movement velocity grade is relative, promptly has only slow, middle and high several relative velocities.Training study method in definite employing pattern recognition of these three kinds of speed, the number of at first determining cluster is 3 (slow, high, in), trains then, based on these training, utilize clustering method, the speed of new detected motion object is divided into a certain in the above-mentioned speed.
Described movement velocity class information is for being very important based on the video monitoring of Traffic monitoring, and it can obtain the relative velocity of vehicle on a certain road, thereby provides the valuable reference foundation for traffic control.
Above-mentioned video is cut apart can be with the video monitoring data that connects according to how much the cutting apart of motion object, the size of having saved memory space on the one hand; On the other hand, cooperate, help retrieving, improved effectiveness of retrieval according to index information with the index information that produces.
An embodiment of retrieving embodiment illustrated in fig. 1 is as follows:
(1) user can read the segmented video data of each index information correspondence by the index information that shows among Fig. 3 from described memory, to carry out playback;
(2) characteristic information such as the histogram that the user can be by the motion object, shape, motion object in each key frame in the ergodic data manager in chronological order, measure similitude between them by methods such as Euclidean distances, by the similitude size sequence matching result is shown to the user at last, realizes CBIR and video frequency searching.Therefore, avoid visible search artificial in the prior art, improved effectiveness of retrieval and simplicity.
In a preferred embodiment of the present invention, described memory is the memory based on IP-SAN, adopts the video transmission based on the iSCSI/RTP/RTCP/RTSP agreement between itself and described application system middleware and the described video encoder.
Preferably, the traffic monitoring system that is applied to embodiment illustrated in fig. 1, it can detect the various characteristic informations and the number of vehicle (motion object just), thereby provides reliable foundation for traffic control decision-making.
With reference to figure 5, illustrate the schematic diagram of middle data flow embodiment illustrated in fig. 1.As shown in the figure, it is divided into front end, rear end and network portion.
What be in front end comprises video camera, video analysis device, video encoder, and what be in the rear end then comprises application system middleware and monitor, and network portion then comprises memory (IP-SAN memory).
Wherein, the video monitoring data that camera acquisition arrives obtains segmented video data and index information through after the video analysis device; This segmented video data is again after video encoder encodes, directly be sent to the application system middleware of rear end by network by a Rtp/Rtsp server part, receive and decode by the Rtp/Rtsp client of application system middleware, resolve and export to monitor by the segmented video data read module again and show by decoder;
And for index information, it same one the tunnel directly is sent to the application system middleware by network, and is resolved by monitor by the index parsing module and to show.
And segmented video data and index information be all also by another road, is sent to memory by index stores module, segmented video data memory module among the figure after by the iSCSI package module and stores.Wherein, described segmented video data is stored in described memory, and described index information then is stored in the data management system (figure does not show) that is positioned at equally on the network.
When needs when described memory reads segmented video data, after then reading from described memory, by iSCSI deblocking module carry out deblocking and decode after be shown in described monitor.
Wherein, described Rtp/Rtsp client, decoder, index parsing module, segmented video data read module, iSCSI deblocking module all are parts of described application system middleware.Described Rtp/Rtsp server, index stores module, segmented video data memory module, iSCSI package module then belong to the part of described Video Decoder.
Real-time play data stream shown in the figure promptly is the segmented video data of real-time Transmission, described index data stream then is described index information, after described video frequency searching data flow is meant that Retrieval Interface that the user provides by the application system middleware is retrieved, from the segmented video data that described memory read.
Above disclosed is a kind of preferred embodiment of the present invention only, can not limit the present invention's interest field certainly with this, and therefore the equivalent variations of doing according to claim of the present invention still belongs to the scope that the present invention is contained.