US6999600B2 - Video scene background maintenance using change detection and classification - Google Patents

Video scene background maintenance using change detection and classification Download PDF

Info

Publication number
US6999600B2
US6999600B2 US10/354,096 US35409603A US6999600B2 US 6999600 B2 US6999600 B2 US 6999600B2 US 35409603 A US35409603 A US 35409603A US 6999600 B2 US6999600 B2 US 6999600B2
Authority
US
United States
Prior art keywords
target
background model
video
stationary
stationary target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime, expires
Application number
US10/354,096
Other languages
English (en)
Other versions
US20040151342A1 (en
Inventor
Peter L. Venetianer
Alan J. Lipton
Andrew J. Chosak
Niels Haering
Zhong Zhang
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Motorola Solutions Inc
Original Assignee
Objectvideo Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Objectvideo Inc filed Critical Objectvideo Inc
Assigned to DIAMONDBACK VISION, INC. reassignment DIAMONDBACK VISION, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: CHOSAK, ANDREW J., HAERING, NIELS, LIPTON, ALAN J., VENETIANER, PETER L., ZHANG, ZHONG
Priority to US10/354,096 priority Critical patent/US6999600B2/en
Assigned to OBJECTVIDEO, INC. reassignment OBJECTVIDEO, INC. CHANGE OF NAME (SEE DOCUMENT FOR DETAILS). Assignors: DIAMONDBACK VISION, INC.
Priority to DK03815787.1T priority patent/DK1588317T3/da
Priority to CA002514826A priority patent/CA2514826A1/en
Priority to JP2004568046A priority patent/JP2006514363A/ja
Priority to AT03815787T priority patent/ATE548706T1/de
Priority to EP03815787A priority patent/EP1588317B1/de
Priority to AU2003300337A priority patent/AU2003300337A1/en
Priority to ES03815787T priority patent/ES2383174T3/es
Priority to HK06110931.5A priority patent/HK1088968B/xx
Priority to KR1020057014103A priority patent/KR20060012570A/ko
Priority to PCT/US2003/041159 priority patent/WO2004070649A1/en
Priority to CNB2003801101194A priority patent/CN100386771C/zh
Priority to MXPA05008201A priority patent/MXPA05008201A/es
Publication of US20040151342A1 publication Critical patent/US20040151342A1/en
Publication of US6999600B2 publication Critical patent/US6999600B2/en
Application granted granted Critical
Assigned to RJF OV, LLC reassignment RJF OV, LLC SECURITY AGREEMENT Assignors: OBJECTVIDEO, INC.
Assigned to RJF OV, LLC reassignment RJF OV, LLC GRANT OF SECURITY INTEREST IN PATENT RIGHTS Assignors: OBJECTVIDEO, INC.
Assigned to OBJECTVIDEO, INC. reassignment OBJECTVIDEO, INC. RELEASE OF SECURITY AGREEMENT/INTEREST Assignors: RJF OV, LLC
Assigned to AVIGILON FORTRESS CORPORATION reassignment AVIGILON FORTRESS CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: OBJECTVIDEO, INC.
Assigned to HSBC BANK CANADA reassignment HSBC BANK CANADA SECURITY INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: AVIGILON FORTRESS CORPORATION
Assigned to AVIGILON FORTRESS CORPORATION reassignment AVIGILON FORTRESS CORPORATION RELEASE BY SECURED PARTY (SEE DOCUMENT FOR DETAILS). Assignors: HSBC BANK CANADA
Assigned to MOTOROLA SOLUTIONS, INC. reassignment MOTOROLA SOLUTIONS, INC. NUNC PRO TUNC ASSIGNMENT (SEE DOCUMENT FOR DETAILS). Assignors: AVIGILON FORTRESS CORPORATION
Adjusted expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/143—Segmentation; Edge detection involving probabilistic approaches, e.g. Markov random field [MRF] modelling
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/254—Analysis of motion involving subtraction of images
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/28—Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence

Definitions

  • the present invention is directed to the general field of video processing and to the more specific field of processing of segmented video.
  • the invention is concerned with the maintenance of background models in segmented video and classifying changes to the background model.
  • video processing applications require segmentation of video objects (i.e., the differentiation of legitimately moving objects from the static background scene depicted in a video sequence).
  • Such applications include, for example, video mosaic building, object-based video compression, object-based video editing, and automated video surveillance.
  • Many video object segmentation algorithms use video scene background models (which can simply be referred to as “background models”) as an aid.
  • background models which can simply be referred to as “background models”
  • video object segmentation (or a variation thereof), objects, or parts of objects, that exhibit independent motion can usually be detected. There are two basic problems that arise when objects in a scene are stationary for a long period of time, and either of these two phenomena can degrade the performance of video object segmentation for any application.
  • FIG. 1A this problem is illustrated for a car 11 that drives into the video sequence and parks therein. The car is continually monitored as a foreground object 12 but has actually become part of the background (i.e., “permanent” segmentation).
  • an object initially stationary, is part of the background model (e.g., gets “burned in”) and then moves, the object exposes a region of the background model (e.g., static background) that has not been modeled.
  • the exposed region of the background model is erroneously detected as a foreground object.
  • FIG. 1B this problem is illustrated for a parked car 13 that drives out of the video sequence.
  • the car 13 exposes a car-shaped “hole” 14 segmented in the background model.
  • foreground objects can be “burned in” the background model resulting in unnatural-looking imagery. If the goal is to build a scene model as a basis for video segmentation, the results can be poor segmentations, where parts of foreground objects are not detected, and where some exposed background regions are detected as foreground objects.
  • FIG. 2 illustrates a prior art example of allowing foreground objects to corrupt a background model.
  • the video sequence depicts a golfer preparing to tee off.
  • a subset 21 of the source images from the video sequence depict a part of this video sequence.
  • the source images are used to generate a background model 22 and foreground objects 23 .
  • the background model 22 contains foreground objects 23 (e.g., the golfer on the left, and part of the golfer's shirt on the right) burned into the background model 22 , and the foreground objects 23 are incompletely segmented (e.g., part of the golfer's torso, and part of the golf club).
  • the invention employs change detection and classification for maintaining a background model of a video sequence. Further, the invention maintains a background model of a video sequence and classifies changes to the background model
  • the invention includes a method for processing video, comprising the steps of: maintaining a background model for the video; detecting a target in the video; detecting if the target is a stationary target; and classifying the stationary target as an insertion in the background model or a removal from the background model.
  • the invention includes a computer system for processing video, comprising: a background model of the video; a background model-based pixel classification to produce a change mask and imagery based on the video and the background model; a background model update to update the background model based on the change mask and the imagery; a motion-based pixel classification to produce a motion mask based on the video; a blob generation to produce at least one blob based on the change mask and the motion mask; a blob tracking to produce at least one target based on the blobs; a stationary target detection and classification to produce a stationary target description based on each target, the stationary target description to identify each the target as an insertion in the background model or a removal from the background model; and a background model local update to update the background model based on each the stationary target description.
  • a system for the invention includes a computer system including a computer-readable medium having software to operate a computer in accordance with the invention.
  • An apparatus for the invention includes a computer including a computer-readable medium having software to operate the computer in accordance with the invention.
  • An article of manufacture for the invention includes a computer-readable medium having software to operate a computer in accordance with the invention.
  • a “computer” refers to any apparatus that is capable of accepting a structured input, processing the structured input according to prescribed rules, and producing results of the processing as output.
  • Examples of a computer include: a computer; a general purpose computer; a supercomputer; a mainframe; a super mini-computer; a mini-computer; a workstation; a microcomputer; a server; an interactive television; a web appliance; a telecommunications device with internet access; a hybrid combination of a computer and an interactive television; and application-specific hardware to emulate a computer and/or software.
  • a computer can be stationary or portable.
  • a computer can have a single processor or multiple processors, which can operate in parallel and/or not in parallel.
  • a computer also refers to two or more computers connected together via a network for transmitting or receiving information between the computers.
  • An example of such a computer includes a distributed computer system for processing information via computers linked by a network.
  • a “computer-readable medium” refers to any storage device used for storing data accessible by a computer. Examples of a computer-readable medium include: a magnetic hard disk; a floppy disk; an optical disk, such as a CD-ROM and a DVD; a magnetic tape; a memory chip; and a carrier wave used to carry computer-readable electronic data, such as those used in transmitting and receiving e-mail or in accessing a network.
  • Software refers to prescribed rules to operate a computer. Examples of software include: software; code segments; instructions; computer programs; and programmed logic.
  • a “computer system” refers to a system having a computer, where the computer comprises a computer-readable medium embodying software to operate the computer.
  • a “network” refers to a number of computers and associated devices that are connected by communication facilities.
  • a network involves permanent connections such as cables or temporary connections such as those made through telephone, wireless, or other communication links.
  • Examples of a network include: an internet, such as the Internet; an intranet; a local area network (LAN); a wide area network (WAN); and a combination of networks, such as an internet and an intranet.
  • Video refers to motion pictures represented in analog and/or digital form. Examples of video include television, movies, image sequences from a camera or other observer, and computer-generated image sequences. These can be obtained from, for example, a live feed, a storage device, an IEEE 1394-based interface, a video digitizer, a computer graphics engine, or a network connection.
  • Video processing refers to any manipulation of video, including, for example, compression and editing.
  • a “frame” refers to a particular image or other discrete unit within a video.
  • FIGS. 1A and 1B illustrate prior art problems with using video object segmentation to detect objects, or parts of objects, that exhibit independent motion
  • FIG. 2 illustrates a prior art example of allowing foreground objects to corrupt a background model
  • FIG. 3 illustrates a flowchart for a first embodiment of the invention
  • FIG. 4 illustrates pixel statistical background modeling to detect foreground pixels
  • FIG. 5 illustrates pixel statistical background modeling to handle lighting changes
  • FIG. 6 illustrates using three-frame differencing for motion detection
  • FIG. 7 illustrates detecting moving pixels and changed pixels
  • FIG. 8 illustrates a flowchart for stationary target detection and classification
  • FIG. 9 illustrates background change detection
  • FIG. 10 illustrates insertion of a foreground object
  • FIG. 11 illustrates removal of a portion of the background
  • FIG. 12 illustrates a flowchart for detecting strong edges
  • FIG. 13 illustrates another flowchart for detecting strong edges
  • FIG. 14 illustrates a flowchart for determining edge strength
  • FIG. 15 illustrates determining edge strength
  • FIG. 16 illustrates a flowchart for a second embodiment of the invention.
  • FIG. 17 illustrates an exemplary computer system.
  • the invention employs change detection and classification for maintaining a background model of a video sequence.
  • the invention can be used for real-time video processing applications (e.g., real-time object-based compression, or video surveillance), in which the video sequence may not be available in its entirety at any time, and incremental changes to the background model might be required to maintain its utility.
  • the invention can also be used for non-real-time video processing applications.
  • a video sequence refers to some or all of a video.
  • first, local changes in the background model are detected and can be used to maintain the background model, and, second, such detected changes are classified and can be further processed.
  • the detected changes are classified into two major categories: first, an object that is placed in the scene and remains static for a period of time (i.e., an insertion); and second, an object that moves out of the scene and exposes a section of the background model (e.g., the static background) (i.e., a removal).
  • the common aspect of these two categories is that there is a permanent local change in the background model.
  • Classifying changes into these two categories can be very important in a wide range of applications, such as, for example, video surveillance applications.
  • Examples of the first category (i.e., an insertion) for video surveillance applications include: monitoring no parking areas (and, for example, initiating an alarm if a car spends more than a certain amount of time in the no parking areas); detecting unattended bags at airports; and detecting unattended objects near sensitive areas, such as military installations and power plants.
  • Examples of the second category (i.e., a removal) for video surveillance applications include: detecting the removal of a high value asset, such as an artifact from a museum, an expensive piece of hardware, or a car from a parking lot.
  • FIG. 3 illustrates a flowchart for a first embodiment of the invention in one possible context of a general video processing system.
  • a video sequence is input into the system, and a background model is generated and maintained 31 , 32 , and 33 .
  • the input video is processed by two separate low-level pixel classification techniques: background model-based pixel classification 31 and motion-based pixel classification 34 .
  • These two techniques produce pixel masks (per frame) that represent pixels of interest.
  • the background model-based pixel classification 31 produces a change mask and imagery
  • the motion-based pixel classification 34 produces a motion mask.
  • the change mask and motion mask are provided to blob generation 35 , which converts the masks into a set of one or more individual blobs representing the appearance of each visible foreground object at each frame.
  • blob tracking 36 which connects the blobs from one frame with those of other frames to generate a “target” representing each object in the scene.
  • a target is a spatio-temporal description of a video object over time.
  • the targets are analyzed by stationary target detection and classification 37 , which determines whether any of the targets represent a “permanent” change to the background model 33 and whether that change represents an “insertion” (e.g., an object entering the scene) or a “removal” (e.g., an object leaving and exposing a section of background model).
  • any stationary targets detected are inserted in the background model 33 by the background model local update 38 .
  • Generating and maintaining a background model includes the background model-based pixel classification 31 , the background model update 32 , and the background model 33 .
  • One option for the background model-based approach 31 , 32 , and 33 employs dynamic statistical pixel modeling.
  • Dynamic statistical pixel modeling maintains an accurate representation of the image background and differentiates background pixels from foreground pixels.
  • dynamic statistical pixel modeling is implemented with the techniques disclosed in commonly-assigned U.S. patent application Ser. No. 09/815,385, titled “Video Segmentation Using Statistical Pixel Modeling,” filed Mar. 23, 2001, which is incorporated herein by reference.
  • the general idea of the exemplary technique is that a history of all pixels is maintained over several frames, including pixel chromatic (or intensity) values and their statistics.
  • a stable, unchanging pixel is treated as background. If the statistics of a pixel change significantly, the pixel can be considered to be foreground. If the pixel reverts to its original state, the pixel can revert to being considered a background pixel.
  • This technique serves to alleviate sensor noise and to automatically address slow changes in the background due to lighting conditions and camera automatic gain control (AGC).
  • AGC camera automatic gain control
  • the background model-based pixel classification 31 can be implemented using static background models, a mixture of gaussian background models or dynamically adaptive mixture of gaussian models.
  • the background model 33 is the internal representation of the static scene depicted in the video at any given time. Each time a new frame is analyzed, the background model 33 can be incrementally updated by the background model update 32 . In addition to the incremental updates, the background model 33 needs to be updated when a background change is detected. For example, the chromatic information representing the new local static background region should be “burned-in” to the background model 33 , which can be accomplished with the background model local update 38 .
  • FIGS. 4 and 5 illustrate using pixel modeling to generate and maintain a background model.
  • pixel statistical background modeling is illustrated for detecting foreground pixels.
  • Frame 41 is a current frame from a video of a man walking in front of stacked chairs and dropping a suitcase. In frame 41 , the man has dropped the suitcase and is continuing forward.
  • the graph 42 plotting intensity and time for a pixel in the video, the intensity mean and standard deviation for each pixel 43 are used to model the background 44 .
  • the background model 33 contains a mean and standard deviation for each pixel.
  • the pixel classification algorithm 31 compares each pixel of the current frame 41 with the corresponding pixel of the background model 33 .
  • a change mask of foreground pixels is created by the background model-based classification 31 and forwarded to the blob generation 35 .
  • This change mask and the current frame 41 are both sent to the background model update 32 so that the pixel statistics comprising the background model 33 can be updated.
  • FIG. 5 pixel statistical background modeling is illustrated for handling lighting changes.
  • Frame 51 illustrates a slow lighting change in a video.
  • the intensity mean and standard deviation for each pixel 53 are used to model the background. Because the mean and standard deviation for each pixel is calculated from only the latest frames, the background model 33 is adapted to follow the slowly changing pixel intensity 54 .
  • the motion-based pixel classification 34 determines whether a pixel is actually undergoing independent motion from frame to frame.
  • One potential embodiment for the motion-based pixel classification 34 is three-frame differencing, as described in commonly-assigned U.S. patent application Ser. No. 09/694,712, filed Oct. 24, 2000, which is incorporated herein by reference.
  • Other potential embodiments for the moving pixel classification 34 include two frame differencing and optical flow.
  • FIG. 6 illustrates using three-frame differencing for motion detection in the motion-based pixel classification 34 .
  • Frames 61 , 62 , and 63 are past, current, and future frames, respectively, from a video of a man walking in front of stacked chairs and dropping a suitcase.
  • Difference mask 64 is obtained by comparing frames 61 and 62
  • difference mask 65 is obtained by comparing frames 62 and 63 .
  • Motion mask 66 is obtained by comparing difference masks 64 and 65 using a logical AND. The motion mask 66 is forwarded to the blob generation 35 .
  • the outputs from the background model-based pixel classification 31 and the motion-based pixel classification 34 may not concurrently detect a new foreground object. For example, a recently parked car might appear as a foreground object according to the background model-based pixel classification 31 . However, because the parked car does not exhibit any actual independent motion, the motion-based pixel classification 34 might not detect any foreground object.
  • Frame 71 is a frame from a video of a man walking in front of stacked chairs and dropping a suitcase.
  • Motion mask 72 results from the motion-based pixel classification 34 , which detects the man but not the suitcase.
  • Change mask 73 results from the background model-based classification 31 , which detects both the man and the suitcase.
  • a recently inserted foreground object i.e., the suitcase
  • the background model-based pixel classification 31 is detected by the background model-based pixel classification 31 but not the motion-based pixel classification 34 .
  • the blob generation 35 and the blob tracking 36 integrate the per frame pixel motion mask and change mask into targets (spatio-temporal descriptions of video objects).
  • targets spatialo-temporal descriptions of video objects.
  • agglomerating pixels into blobs for example: connected components, as discussed in D. Ballard and C. Brown, “Computer Vision,” Prentice-Hall, May 1982, which is incorporated herein by reference; and quasi-connected components, as discussed in T. E. Boult, R. J. Micheals, X. Gao, P. Lewis, C. Power, W. Yin, and A. Erkan, “Frame-Rate Omnidirectional Surveillance and Tracking of Camouflaged and Occluded Targets,” Proc.
  • the stationary target detection and classification 37 analyzes targets generated by the blob tracking 36 to determine if each target is stationary.
  • a target can be determined to be stationary if the target represents a local change in the background model 33 .
  • a target can represent a change in the background model 33 if, for example, a video object has ceased moving (i.e., an insertion) or a previously stationary video object has exposed a section of static background that appears as a target (i.e., a removal).
  • this information can be fed back to the background model local update 38 to update the background model 33 .
  • the background model 33 can be kept up to date concerning what constitutes static background and legitimate foreground activity.
  • the stationary target detection and classification 37 determines if a target is stationary, and if so, whether it should be labeled as an insertion, a removal, or unknown, if it is not possible to determine the difference.
  • the relationship between the time scales for an insertion and a removal is important.
  • An insertion may involve a different time scale than that of a removal, and these time scales may be application dependent. For example, an application may require that an object be left in place for a large amount of time before being considered an insertion but only a short amount of time before being considered a removal.
  • a car parked at a curb at an airport for five minutes may not be a concern and may not be considered an insertion, but a car parked at the curb for fifteen minutes may be a concern and considered an insertion.
  • the same car, as soon as it moves away from the curb may be considered a removal.
  • the time scale for an insertion is longer than the time scale for a removal.
  • the relative time scales for an insertion and a removal may be reversed from the example above such that the time scale for a removal is longer than the time scale for an insertion.
  • the time scales for an insertion and a removal can be configurable by a user.
  • FIG. 8 illustrates a flowchart for the stationary target detection and classification 37 .
  • An exemplary pseudo-code for implementing the stationary target detection and classification 37 is as follows:
  • each target provided by the blob generation 35 is examined to determine if the target is potentially stationary. This block corresponds to the first “if” condition in the above pseudo-code (i.e., if (target is POTENTIALLY_STATIONARY)). If the target is not potentially stationary, flow proceeds to block 82 and ends.
  • An exemplary technique to determine if a target is potentially stationary uses various spatio-temporal properties and features of the target. If a target has not radically changed its shape and size for a period of time, the target may a stationary target. Furthermore, if a target exhibits a large amount of change from the background (as determined by change detection 31 , 32 , 33 ), but very little independent motion (as determined by motion detection 34 ), the target is almost certainly a stationary target.
  • Image 91 is a current frame from a video of a man walking in front of stacked chairs and dropping a briefcase
  • image 94 is a current frame from a video of a man removing artwork from a room.
  • Motion masks 92 and 95 result from the motion-based pixel classification 34 and illustrate pixel masks of “moving” pixels (i.e., pixels that exhibit motion). Motion mask 92 detects the man but not the suitcase in frame 91 , and motion mask 95 detects the man walking with the artwork, but not the absence on the wall.
  • Change masks 93 and 96 result from the background model-based pixel classification 31 and illustrate pixel masks of “changed” pixels (i.e., pixels that differ from the background model 33 ).
  • Change mask 93 detects both the man and the briefcase
  • change mask 96 detects both the man walking with the artwork and the absence on the wall.
  • there are areas which have clearly changed with respect to the background model 33 but do not exhibit any independent motion.
  • the insertion of the briefcase does not exhibit any independent motion
  • the change mask 96 the removal of the artwork from the wall does not exhibit any independent motion.
  • exemplary quantifiable target properties are determined.
  • ⁇ ⁇ C and ⁇ C can represent statistical properties of a centroid trajectory of the target.
  • ⁇ ⁇ C can represent the mean (over time) of the difference in centroid position (in pixels) between consecutive frames
  • ⁇ ⁇ C can represent the standard deviation (over time) of the difference in centroid position (in pixels) between consecutive frames.
  • ⁇ ⁇ C and ⁇ ⁇ C represent statistical properties of a centroid trajectory of the stationary target.
  • ⁇ R and ⁇ R represent statistical properties of the pixel area of the target. Specifically, ⁇ R can represent the mean (over some recent period of time) of the ratio of the area of the target (in pixels) between consecutive frames, and ⁇ R can represent the standard deviation (over some recent period of time) of the ratio of the area of the target (in pixels) between consecutive frames.
  • ⁇ R can represent the mean (over some recent period of time) of the ratio of the area of the target (in pixels) between consecutive frames
  • ⁇ R can represent the standard deviation (over some recent period of time) of the ratio of the area of the target (in pixels) between consecutive frames.
  • ⁇ M and ⁇ M represent statistical properties of moving pixels of the stationary target.
  • ⁇ M can represent the mean (over some recent period of time) of the ratio of the number of “moving” pixels to the area of the target (in pixels)
  • ⁇ M can represent the standard deviation (over some recent period of time) of the ratio of the number of “moving” pixels to the area of the target (in pixels).
  • THRESHOLD 1 THRESHOLD 2 , THRESHOLD 3 , THRESHOLD 4 , THRESHOLD 5 , and THRESHOLD 6
  • the six thresholds can be preset and/or arbitrarily set as user parameters.
  • ⁇ ⁇ C , ⁇ ⁇ C , ⁇ R , and ⁇ R are discussed as representing the general motion and size change of a target over time, other properties can be used as will become apparent to those of ordinary skill in the art.
  • ⁇ M and ⁇ M are discussed as representing exhibiting independent motion, other properties can be used as will become apparent to those of ordinary skill in the art.
  • insertion_time_threshold ⁇ removal_time_threshold The pseudo-code for classifying the detected targets depends on the relationship between the insertion time threshold and the removal time threshold. This relationship determines which of the two tests, namely an insertion test or a removal test, is performed first.
  • the insertion time threshold and the removal time threshold are points in time based on the time scales set for an insertion and a removal, as discussed above. In the pseudo-code, the insertion time threshold and the removal time threshold are compared to the target age.
  • Block 84 the insertion test and/or the removal test is applied. If the application of these tests determines the target is an insertion, flow proceeds to block 85 , and the target is classified as an insertion. If the application of these tests determines the target is a removal, flow proceeds to block 86 , and the target is classified as a removal. If the application of these tests is inconclusive as to whether the target is an insertion or a removal, flow proceeds to block 87 , and the target is classified as an unknown.
  • Blocks 84 – 86 correspond to the third “if” condition block in the above pseudo-code (i.e., if (target_age>1 st _time_threshold)).
  • a potentially stationary target is considered to be stationary by passing the insertion test and/or the removal test, its description is sent to the background model local update 38 , which modifies the background model 33 at the location of the potentially stationary target.
  • This process involves replacing the background model statistics (mean and variance) of the pixels representing the potentially stationary target.
  • the values of the mean and variance of the pixels representing the stationary target will be modified to represent the mean and variance of the pixels from more recent frames representing the potentially stationary target.
  • FIGS. 10 and 11 The insertion test and the removal test are illustrated with FIGS. 10 and 11 .
  • the theory behind the exemplary classification technique of the invention is that an insertion can be characterized as a region that exhibits strong edges around its periphery in a current image but does not exhibit strong edges around the periphery of the same region in the background model.
  • a removal can be characterized as a region that exhibits strong edges around its periphery in the background model but does not exhibit strong edges around its periphery in a current image.
  • FIG. 10 illustrates classifying an insertion.
  • the video in this example is of a man walking in front of stacked chairs and dropping a briefcase.
  • Image 101 illustrates an image of the background model
  • background edge image 102 illustrates the corresponding edges of image 101 determined using a Sobel edge detector.
  • Image 103 illustrates an image of the current frame
  • current frame edge image 104 illustrates the corresponding edges of image 103 determined using a Sobel edge detector.
  • the briefcase exhibits very strong edges in the current frame (i.e., current frame edge image 104 ), but not in the background model (i.e., background edge image 102 ).
  • Change mask 105 shows the detected changed pixels, including the stationary object (i.e., the briefcase).
  • Image 106 is a close-up of the briefcase region in change mask 105
  • image 107 is a close-up of a section on the periphery of the briefcase region in image 106 .
  • Images 108 and 109 show the edges corresponding to the section of image 107 for both the background edge image 102 and the current frame edge image 104 , respectively.
  • the edge strength in the image 109 for the current frame is greater than the edge strength in image 108 for the background model.
  • the target i.e., the briefcase
  • FIG. 11 illustrates classifying a removal.
  • the video in this example is of a man removing artwork from a room.
  • Image 111 illustrates an image of the background model
  • background edge image 112 illustrates the corresponding edges of image 111 determined using a Sobel edge detector.
  • Image 113 illustrates an image of the current frame
  • current frame edge image 114 illustrates the corresponding edges of image 113 determined using a Sobel edge detector.
  • the artwork exhibits very strong edges in the background model (i.e., background model edge image 112 ), but not in the current frame (i.e., current frame image 114 ).
  • Change mask 115 shows the detected changed pixels, including the stationary object (i.e., the artwork).
  • Image 116 is a close-up of the artwork region in change mask 115
  • image 117 is a close-up of a section on the periphery of the artwork region in image 116 .
  • Images 118 and 119 show the edges corresponding to the section of image 117 for both the background edge image 112 and the current frame edge image 114 , respectively.
  • the edge strength in the image 118 for the background model is greater than the edge strength in image 119 for the current frame.
  • the target i.e., the artwork
  • the target is classified as a removal.
  • FIGS. 12 and 13 illustrate two embodiments for blocks 84 – 87 in FIG. 8 .
  • FIG. 12 illustrates the embodiment for the case where the insertion time threshold is less than the removal time threshold
  • FIG. 13 illustrates the corresponding other case where the insertion time threshold is not less than the removal time threshold.
  • the edge strength E B of the background is determined along the boundary of the potentially stationary target (i.e., the detected change).
  • the edge strength E F of the current frame is determined along the boundary of the stationary target.
  • the target age is compared to the insertion time threshold. If the target age is greater than the insertion time threshold, flow proceeds to block 1205 . Otherwise, flow proceeds to block 1211 and ends.
  • the difference ⁇ E is compared to an insertion threshold TH 1 .
  • TH 1 the stationary target is an insertion
  • flow proceeds to block 1206 . Otherwise, flow proceeds to block 1207 .
  • the stationary target is classified as an insertion.
  • the target age is compared to the removal time threshold. If the target age is greater than the removal time threshold, flow proceeds to block 1208 . Otherwise, flow proceeds to block 1211 and ends.
  • the difference ⁇ E is compared to a removal threshold TH R .
  • TH R a removal threshold
  • the stationary target is classified as a removal.
  • the stationary target cannot be classified as either an insertion or a removal and is, instead, classified as an unknown.
  • the description of the stationary target is sent to the background model local update 38 , which modifies the background model 33 to reflect the change caused by the detected stationary target. Even though the stationary target can not be classified as insertion or removal (block 1210 ), the background model is still updated.
  • edge strengths E B and E F can be determined in blocks 1201 and 1202 over a series of frames and averaged over time.
  • FIG. 13 is the same as FIG. 12 , except for the change of places in the flowchart for blocks 1204 – 1206 and blocks 1207 – 1209 .
  • FIG. 14 illustrates a flowchart for an exemplary technique for determining the edge strengths E B and E F for blocks 1201 and 1202 .
  • Other techniques are available, as will become evident to those of ordinary skill in the art.
  • FIG. 14 is discussed in relation to FIG. 15 , which illustrates an exemplary stationary target over which the edge strengths are determined. With the exemplary technique of FIG. 14 , some uncertainty in the boundary of the detected change is accommodated, and holes and small lacunae in the object are ignored.
  • a band of the image is selected.
  • the Y band is selected in a YCrCb image.
  • Other bands, besides the Y band, can be selected.
  • multiple bands can be selected.
  • other types of images can also be accommodated with the invention, such as an RGB or a CMYK image.
  • a line is selected across a perimeter pixel P p and the centroid P c of the target.
  • the centroid P c of the target 151 is designated with a star
  • the exemplary perimeter pixels 152 , 153 , and 154 are designated with light circles along the perimeter of the target 151 .
  • Three exemplary perimeter pixels are identified in FIG. 15 , and for each perimeter pixel, a line is selected across the perimeter pixel P p and the centroid P c .
  • two pixels P 1 and P 2 on the line are selected at an equivalent+/ ⁇ distance from the perimeter pixel P p .
  • the two pixels for each line are designated with dark circles.
  • the average contrast is determined over all perimeter pixels for which a contrast C p was determined in block 146 .
  • This average contrast can be used as the edge strengths E B and E F in blocks 1201 and 1202 , respectively.
  • FIG. 15 addressed the three exemplary perimeter pixels 152 , 153 , and 154 concurrently. However, in examining the perimeter pixels according to FIG. 14 , each perimeter pixel is examined individually until all perimeter pixels have been examined, as per the loop back from block 147 to block 142 .
  • the detected targets are further monitored to determine if a newly detected target was previously detected by the stationary target detection and classification 37 as a change in the background model. For example, in a surveillance application, it may be of interest to detect when a target entered a scene and then stopped moving (e.g., a car parking) and thereafter to monitor the target (or the area of the scene where the target stopped moving) to determine if and when the target moves again (e.g., a parked car leaving).
  • stopped moving e.g., a car parking
  • the target or the area of the scene where the target stopped moving
  • FIG. 16 illustrates a flowchart for the second embodiment of the invention.
  • FIG. 16 is the same as FIG. 3 , except for the addition of a stationary target monitor 161 .
  • the stationary target monitor 161 receives stationary target descriptions from the stationary target detection and classification 37 and provides a target reactivation to the blob tracking 36 . If stationary target is classified as an insertion, the stationary target monitor 161 records the target (e.g., time, size, color, and location) and monitors the target for any further activity. At this point, the target is “forgotten” by the rest of the system as being integrated into the background model 33 and, in effect, goes into hibernation.
  • the target e.g., time, size, color, and location
  • the stationary target monitor 161 registers the removal with the hibernating stationary target and instructs the blob tracking 36 to reactivate that target.
  • FIG. 17 illustrates an exemplary computer system 171 , which includes a computer 172 and a computer-readable medium 173 .
  • blocks 31 – 38 and 161 can be implemented with software residing on one or more computer-readable medium 173 of the computer system 171 .
  • Video and/or images to be processed with the invention can reside on one or more computer-readable medium 173 or be provided, for example, via the video or image input 174 or the network 175 .

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Probability & Statistics with Applications (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Burglar Alarm Systems (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Studio Circuits (AREA)
  • Television Signal Processing For Recording (AREA)
US10/354,096 2003-01-30 2003-01-30 Video scene background maintenance using change detection and classification Expired - Lifetime US6999600B2 (en)

Priority Applications (13)

Application Number Priority Date Filing Date Title
US10/354,096 US6999600B2 (en) 2003-01-30 2003-01-30 Video scene background maintenance using change detection and classification
CA002514826A CA2514826A1 (en) 2003-01-30 2003-12-23 Video scene background maintenance using change detection and classification
CNB2003801101194A CN100386771C (zh) 2003-01-30 2003-12-23 使用变化检测和分类的视频场景背景保持
MXPA05008201A MXPA05008201A (es) 2003-01-30 2003-12-23 Mantenimiento de fondo en escena de video, utilizando deteccion y clasificacion de cambio.
JP2004568046A JP2006514363A (ja) 2003-01-30 2003-12-23 変化検出および分類を用いた映像シーン背景の維持
AT03815787T ATE548706T1 (de) 2003-01-30 2003-12-23 Videoszenenhintergrundaufrechterhaltung durch verwendung von änderungsdetektion und - klassifikation
EP03815787A EP1588317B1 (de) 2003-01-30 2003-12-23 Videoszenenhintergrundaufrechterhaltung durch verwendung von änderungsdetektion und -klassifikation
AU2003300337A AU2003300337A1 (en) 2003-01-30 2003-12-23 Video scene background maintenance using change detection and classification
ES03815787T ES2383174T3 (es) 2003-01-30 2003-12-23 Mantenimiento de fondo de escena de video utilizando detección y clasificación de cambios
HK06110931.5A HK1088968B (en) 2003-01-30 2003-12-23 Video scene background maintenance using change detection and classification
KR1020057014103A KR20060012570A (ko) 2003-01-30 2003-12-23 변화 검출 및 분류를 이용한 비디오 장면 배경 유지
PCT/US2003/041159 WO2004070649A1 (en) 2003-01-30 2003-12-23 Video scene background maintenance using change detection and classification
DK03815787.1T DK1588317T3 (da) 2003-01-30 2003-12-23 Vedligeholdelse af videoscenebaggrund ved at anvende ændringsdetektering og -klassificering

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US10/354,096 US6999600B2 (en) 2003-01-30 2003-01-30 Video scene background maintenance using change detection and classification

Publications (2)

Publication Number Publication Date
US20040151342A1 US20040151342A1 (en) 2004-08-05
US6999600B2 true US6999600B2 (en) 2006-02-14

Family

ID=32770313

Family Applications (1)

Application Number Title Priority Date Filing Date
US10/354,096 Expired - Lifetime US6999600B2 (en) 2003-01-30 2003-01-30 Video scene background maintenance using change detection and classification

Country Status (12)

Country Link
US (1) US6999600B2 (de)
EP (1) EP1588317B1 (de)
JP (1) JP2006514363A (de)
KR (1) KR20060012570A (de)
CN (1) CN100386771C (de)
AT (1) ATE548706T1 (de)
AU (1) AU2003300337A1 (de)
CA (1) CA2514826A1 (de)
DK (1) DK1588317T3 (de)
ES (1) ES2383174T3 (de)
MX (1) MXPA05008201A (de)
WO (1) WO2004070649A1 (de)

Cited By (71)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050036658A1 (en) * 2001-11-21 2005-02-17 Daniel Gibbins Non-motion detection
US20050089194A1 (en) * 2003-10-24 2005-04-28 Matthew Bell Method and system for processing captured image information in an interactive video display system
US20050093697A1 (en) * 2003-11-05 2005-05-05 Sanjay Nichani Method and system for enhanced portal security through stereoscopy
US20050110964A1 (en) * 2002-05-28 2005-05-26 Matthew Bell Interactive video window display system
US20050157169A1 (en) * 2004-01-20 2005-07-21 Tomas Brodsky Object blocking zones to reduce false alarms in video surveillance systems
US20050163346A1 (en) * 2003-12-03 2005-07-28 Safehouse International Limited Monitoring an output from a camera
US20050162381A1 (en) * 2002-05-28 2005-07-28 Matthew Bell Self-contained interactive video display system
US20050175219A1 (en) * 2003-11-13 2005-08-11 Ming-Hsuan Yang Adaptive probabilistic visual tracking with incremental subspace update
US20060023916A1 (en) * 2004-07-09 2006-02-02 Ming-Hsuan Yang Visual tracking using incremental fisher discriminant analysis
US20060153448A1 (en) * 2005-01-13 2006-07-13 International Business Machines Corporation System and method for adaptively separating foreground from arbitrary background in presentations
US20060177099A1 (en) * 2004-12-20 2006-08-10 Ying Zhu System and method for on-road detection of a vehicle using knowledge fusion
US20060269136A1 (en) * 2005-05-23 2006-11-30 Nextcode Corporation Efficient finder patterns and methods for application to 2D machine vision problems
US20060268111A1 (en) * 2005-05-31 2006-11-30 Objectvideo, Inc. Multi-state target tracking
US20070127774A1 (en) * 2005-06-24 2007-06-07 Objectvideo, Inc. Target detection and tracking from video streams
US20070183662A1 (en) * 2006-02-07 2007-08-09 Haohong Wang Inter-mode region-of-interest video object segmentation
US20070183663A1 (en) * 2006-02-07 2007-08-09 Haohong Wang Intra-mode region-of-interest video object segmentation
US20070183661A1 (en) * 2006-02-07 2007-08-09 El-Maleh Khaled H Multi-mode region-of-interest video object segmentation
US20070285510A1 (en) * 2006-05-24 2007-12-13 Object Video, Inc. Intelligent imagery-based sensor
US20080031489A1 (en) * 2006-06-01 2008-02-07 Frode Reinholt Method and an apparatus for analysing objects
US20080062123A1 (en) * 2001-06-05 2008-03-13 Reactrix Systems, Inc. Interactive video display system using strobed light
US20080100438A1 (en) * 2002-09-05 2008-05-01 Marrion Cyril C Multi-Zone Passageway Monitoring System and Method
US20080231709A1 (en) * 2007-03-20 2008-09-25 Brown Lisa M System and method for managing the interaction of object detection and tracking systems in video surveillance
US20080240616A1 (en) * 2007-04-02 2008-10-02 Objectvideo, Inc. Automatic camera calibration and geo-registration using objects that provide positional information
US20080252596A1 (en) * 2007-04-10 2008-10-16 Matthew Bell Display Using a Three-Dimensional vision System
US20080273754A1 (en) * 2007-05-04 2008-11-06 Leviton Manufacturing Co., Inc. Apparatus and method for defining an area of interest for image sensing
US20080298636A1 (en) * 2007-06-04 2008-12-04 Object Video, Inc. Method for detecting water regions in video
US20090010493A1 (en) * 2007-07-03 2009-01-08 Pivotal Vision, Llc Motion-Validating Remote Monitoring System
US20090060277A1 (en) * 2007-09-04 2009-03-05 Objectvideo, Inc. Background modeling with feature blocks
US20090077504A1 (en) * 2007-09-14 2009-03-19 Matthew Bell Processing of Gesture-Based User Interactions
US20090235295A1 (en) * 2003-10-24 2009-09-17 Matthew Bell Method and system for managing an interactive video display system
US20090251685A1 (en) * 2007-11-12 2009-10-08 Matthew Bell Lens System
US20090290020A1 (en) * 2008-02-28 2009-11-26 Canon Kabushiki Kaisha Stationary Object Detection Using Multi-Mode Background Modelling
US20090304234A1 (en) * 2008-06-06 2009-12-10 Sony Corporation Tracking point detecting device and method, program, and recording medium
US20090315996A1 (en) * 2008-05-09 2009-12-24 Sadiye Zeyno Guler Video tracking systems and methods employing cognitive vision
US7680323B1 (en) 2000-04-29 2010-03-16 Cognex Corporation Method and apparatus for three-dimensional object segmentation
US20100092036A1 (en) * 2008-06-17 2010-04-15 Subhodev Das Method and apparatus for detecting targets through temporal scene changes
US20100121866A1 (en) * 2008-06-12 2010-05-13 Matthew Bell Interactive display management systems and methods
US20100128930A1 (en) * 2008-11-24 2010-05-27 Canon Kabushiki Kaisha Detection of abandoned and vanished objects
US20100215256A1 (en) * 2009-02-25 2010-08-26 Micro-Star Int'l Co., Ltd. Method and device for maintaining image background by multiple gaussian models
US7834846B1 (en) 2001-06-05 2010-11-16 Matthew Bell Interactive video display system
US20110273619A1 (en) * 2010-05-10 2011-11-10 Canon Kabushiki Kaisha Image processing apparatus, method, and storage medium
US8081822B1 (en) 2005-05-31 2011-12-20 Intellectual Ventures Holding 67 Llc System and method for sensing a feature of an object in an interactive video display
US8098277B1 (en) 2005-12-02 2012-01-17 Intellectual Ventures Holding 67 Llc Systems and methods for communication between a reactive video system and a mobile communication device
US8111904B2 (en) 2005-10-07 2012-02-07 Cognex Technology And Investment Corp. Methods and apparatus for practical 3D vision system
US8126260B2 (en) 2007-05-29 2012-02-28 Cognex Corporation System and method for locating a three-dimensional object using machine vision
US8199108B2 (en) 2002-12-13 2012-06-12 Intellectual Ventures Holding 67 Llc Interactive directed light/sound system
US8259163B2 (en) 2008-03-07 2012-09-04 Intellectual Ventures Holding 67 Llc Display with built in 3D sensing
US8326084B1 (en) 2003-11-05 2012-12-04 Cognex Technology And Investment Corporation System and method of auto-exposure control for image acquisition hardware using three dimensional information
US20130162867A1 (en) * 2011-12-21 2013-06-27 Canon Kabushiki Kaisha Method and system for robust scene modelling in an image sequence
US8737745B2 (en) 2012-03-27 2014-05-27 The Nielsen Company (Us), Llc Scene-based people metering for audience measurement
US20140233793A1 (en) * 2012-09-21 2014-08-21 Canon Kabushiki Kaisha Differentiating abandoned and removed object using temporal edge information
US20150189191A1 (en) * 2013-12-27 2015-07-02 Telemetrio LLC Process and system for video production and tracking of objects
US9128519B1 (en) 2005-04-15 2015-09-08 Intellectual Ventures Holding 67 Llc Method and system for state-based control of objects
US9185456B2 (en) 2012-03-27 2015-11-10 The Nielsen Company (Us), Llc Hybrid active and passive people metering for audience measurement
US9213781B1 (en) 2012-09-19 2015-12-15 Placemeter LLC System and method for processing image data
US9245187B1 (en) 2014-07-07 2016-01-26 Geo Semiconductor Inc. System and method for robust motion detection
US20170236010A1 (en) * 2009-10-19 2017-08-17 Canon Kabushiki Kaisha Image pickup apparatus, information processing apparatus, and information processing method
US9760792B2 (en) 2015-03-20 2017-09-12 Netra, Inc. Object detection and classification
US9922271B2 (en) 2015-03-20 2018-03-20 Netra, Inc. Object detection and classification
US9940524B2 (en) * 2015-04-17 2018-04-10 General Electric Company Identifying and tracking vehicles in motion
US10043078B2 (en) 2015-04-21 2018-08-07 Placemeter LLC Virtual turnstile system and method
US10043307B2 (en) 2015-04-17 2018-08-07 General Electric Company Monitoring parking rule violations
US10096117B2 (en) 2014-12-24 2018-10-09 Canon Kabushiki Kaisha Video segmentation method
US10110856B2 (en) 2014-12-05 2018-10-23 Avigilon Fortress Corporation Systems and methods for video analysis rules based on map data
US10380431B2 (en) 2015-06-01 2019-08-13 Placemeter LLC Systems and methods for processing video streams
US10468065B2 (en) 2015-10-28 2019-11-05 Ustudio, Inc. Video frame difference engine
US10735694B2 (en) 2014-05-30 2020-08-04 Placemeter Inc. System and method for activity monitoring using video data
US11126861B1 (en) 2018-12-14 2021-09-21 Digimarc Corporation Ambient inventorying arrangements
US11312594B2 (en) 2018-11-09 2022-04-26 Otis Elevator Company Conveyance system video analytics
US11334751B2 (en) 2015-04-21 2022-05-17 Placemeter Inc. Systems and methods for processing video data for activity monitoring
US11600072B2 (en) 2018-12-12 2023-03-07 Motorola Solutions, Inc. Object left behind detection

Families Citing this family (118)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6738066B1 (en) * 1999-07-30 2004-05-18 Electric Plant, Inc. System, method and article of manufacture for detecting collisions between video images generated by a camera and an object depicted on a display
US8457401B2 (en) * 2001-03-23 2013-06-04 Objectvideo, Inc. Video segmentation using statistical pixel modeling
US7215827B2 (en) * 2002-03-15 2007-05-08 Hitachi Kokusai Electric Inc. Object detection method using an image-pickup device with easy detection masking region setting and object detection apparatus using the method
US20050078873A1 (en) * 2003-01-31 2005-04-14 Cetin Ahmet Enis Movement detection and estimation in wavelet compressed video
US20040223652A1 (en) * 2003-05-07 2004-11-11 Cetin Ahmet Enis Characterization of motion of moving objects in video
KR100601933B1 (ko) * 2003-11-18 2006-07-14 삼성전자주식회사 사람검출방법 및 장치와 이를 이용한 사생활 보호방법 및 시스템
KR100568237B1 (ko) * 2004-06-10 2006-04-07 삼성전자주식회사 비디오 영상으로부터 이동 물체를 추출하는 장치 및 방법
US7382898B2 (en) * 2004-06-15 2008-06-03 Sarnoff Corporation Method and apparatus for detecting left objects
US7391907B1 (en) * 2004-10-01 2008-06-24 Objectvideo, Inc. Spurious object detection in a video surveillance system
US7372975B2 (en) 2004-12-06 2008-05-13 Mitsubishi Electric Research Laboratory, Inc. Method for secure background modeling in images
US7391905B2 (en) * 2004-12-06 2008-06-24 Mitsubishi Electric Research Laboratories Method for secure component labeling in images
US7577252B2 (en) * 2004-12-06 2009-08-18 Mitsubishi Electric Research Laboratories, Inc. Method for secure object detection in images
US7688999B2 (en) * 2004-12-08 2010-03-30 Electronics And Telecommunications Research Institute Target detecting system and method
US7801328B2 (en) * 2005-03-31 2010-09-21 Honeywell International Inc. Methods for defining, detecting, analyzing, indexing and retrieving events using video image processing
US20070122000A1 (en) * 2005-11-29 2007-05-31 Objectvideo, Inc. Detection of stationary objects in video
US7796780B2 (en) * 2005-06-24 2010-09-14 Objectvideo, Inc. Target detection and tracking from overhead video streams
JP4947936B2 (ja) * 2005-08-11 2012-06-06 ソニー株式会社 モニタリングシステムおよび管理装置
US7787011B2 (en) * 2005-09-07 2010-08-31 Fuji Xerox Co., Ltd. System and method for analyzing and monitoring 3-D video streams from multiple cameras
EP1969560B1 (de) * 2005-12-30 2017-04-05 Telecom Italia S.p.A. Kantengesteuerte morphologische schliessung bei der segmentierung von videosequenzen
US8848053B2 (en) * 2006-03-28 2014-09-30 Objectvideo, Inc. Automatic extraction of secondary video streams
US20070250898A1 (en) 2006-03-28 2007-10-25 Object Video, Inc. Automatic extraction of secondary video streams
JP4631806B2 (ja) * 2006-06-05 2011-02-16 日本電気株式会社 物体検出装置、物体検出方法および物体検出プログラム
DE102007024868A1 (de) * 2006-07-21 2008-01-24 Robert Bosch Gmbh Bildverarbeitungsvorrichtung, Überwachungssystem, Verfahren zur Erzeugung eines Szenenreferenzbildes sowie Computerprogramm
KR100793837B1 (ko) * 2006-09-13 2008-01-11 한국전자통신연구원 조명의 변화와 학습자 인터랙션을 고려한 마커 인식 장치및 마커 인식 방법
US8045783B2 (en) * 2006-11-09 2011-10-25 Drvision Technologies Llc Method for moving cell detection from temporal image sequence model estimation
US8300890B1 (en) * 2007-01-29 2012-10-30 Intellivision Technologies Corporation Person/object image and screening
US20080181457A1 (en) * 2007-01-31 2008-07-31 Siemens Aktiengesellschaft Video based monitoring system and method
EP2118864B1 (de) 2007-02-08 2014-07-30 Behavioral Recognition Systems, Inc. Verhaltenserkennungssystem
US8189905B2 (en) 2007-07-11 2012-05-29 Behavioral Recognition Systems, Inc. Cognitive model for a machine-learning engine in a video analysis system
US8200011B2 (en) 2007-09-27 2012-06-12 Behavioral Recognition Systems, Inc. Context processor for video analysis system
US8300924B2 (en) * 2007-09-27 2012-10-30 Behavioral Recognition Systems, Inc. Tracker component for behavioral recognition system
US8175333B2 (en) * 2007-09-27 2012-05-08 Behavioral Recognition Systems, Inc. Estimator identifier component for behavioral recognition system
US8086071B2 (en) * 2007-10-30 2011-12-27 Navteq North America, Llc System and method for revealing occluded objects in an image dataset
MY143022A (en) * 2007-11-23 2011-02-14 Mimos Berhad Method for detecting unattended object and removal of static object
GB2446293A (en) * 2008-01-31 2008-08-06 Siemens Ag Video based monitoring system and method
US8107678B2 (en) * 2008-03-24 2012-01-31 International Business Machines Corporation Detection of abandoned and removed objects in a video stream
US8284249B2 (en) * 2008-03-25 2012-10-09 International Business Machines Corporation Real time processing of video frames for triggering an alert
TWI381717B (zh) * 2008-03-31 2013-01-01 Univ Nat Taiwan 數位視訊動態目標物體分割處理方法及系統
US9633275B2 (en) 2008-09-11 2017-04-25 Wesley Kenneth Cobb Pixel-level based micro-feature extraction
US8121424B2 (en) * 2008-09-26 2012-02-21 Axis Ab System, computer program product and associated methodology for video motion detection using spatio-temporal slice processing
US9373055B2 (en) * 2008-12-16 2016-06-21 Behavioral Recognition Systems, Inc. Hierarchical sudden illumination change detection using radiance consistency within a spatial neighborhood
US8285046B2 (en) * 2009-02-18 2012-10-09 Behavioral Recognition Systems, Inc. Adaptive update of background pixel thresholds using sudden illumination change detection
US8416296B2 (en) * 2009-04-14 2013-04-09 Behavioral Recognition Systems, Inc. Mapper component for multiple art networks in a video analysis system
JP5305520B2 (ja) * 2009-05-19 2013-10-02 パナソニック株式会社 監視カメラシステム
US8295591B2 (en) * 2009-08-18 2012-10-23 Behavioral Recognition Systems, Inc. Adaptive voting experts for incremental segmentation of sequences with prediction in a video surveillance system
US8379085B2 (en) * 2009-08-18 2013-02-19 Behavioral Recognition Systems, Inc. Intra-trajectory anomaly detection using adaptive voting experts in a video surveillance system
US8358834B2 (en) 2009-08-18 2013-01-22 Behavioral Recognition Systems Background model for complex and dynamic scenes
US8340352B2 (en) * 2009-08-18 2012-12-25 Behavioral Recognition Systems, Inc. Inter-trajectory anomaly detection using adaptive voting experts in a video surveillance system
US9805271B2 (en) * 2009-08-18 2017-10-31 Omni Ai, Inc. Scene preset identification using quadtree decomposition analysis
US20110043689A1 (en) * 2009-08-18 2011-02-24 Wesley Kenneth Cobb Field-of-view change detection
US8280153B2 (en) * 2009-08-18 2012-10-02 Behavioral Recognition Systems Visualizing and updating learned trajectories in video surveillance systems
US8625884B2 (en) * 2009-08-18 2014-01-07 Behavioral Recognition Systems, Inc. Visualizing and updating learned event maps in surveillance systems
US8493409B2 (en) * 2009-08-18 2013-07-23 Behavioral Recognition Systems, Inc. Visualizing and updating sequences and segments in a video surveillance system
US8744168B2 (en) * 2009-08-24 2014-06-03 Samsung Electronics Co., Ltd. Target analysis apparatus, method and computer-readable medium
US8285060B2 (en) * 2009-08-31 2012-10-09 Behavioral Recognition Systems, Inc. Detecting anomalous trajectories in a video surveillance system
US8270732B2 (en) * 2009-08-31 2012-09-18 Behavioral Recognition Systems, Inc. Clustering nodes in a self-organizing map using an adaptive resonance theory network
US8797405B2 (en) * 2009-08-31 2014-08-05 Behavioral Recognition Systems, Inc. Visualizing and updating classifications in a video surveillance system
US8270733B2 (en) * 2009-08-31 2012-09-18 Behavioral Recognition Systems, Inc. Identifying anomalous object types during classification
US8167430B2 (en) * 2009-08-31 2012-05-01 Behavioral Recognition Systems, Inc. Unsupervised learning of temporal anomalies for a video surveillance system
US8786702B2 (en) 2009-08-31 2014-07-22 Behavioral Recognition Systems, Inc. Visualizing and updating long-term memory percepts in a video surveillance system
US8218818B2 (en) * 2009-09-01 2012-07-10 Behavioral Recognition Systems, Inc. Foreground object tracking
US8218819B2 (en) * 2009-09-01 2012-07-10 Behavioral Recognition Systems, Inc. Foreground object detection in a video surveillance system
US8180105B2 (en) * 2009-09-17 2012-05-15 Behavioral Recognition Systems, Inc. Classifier anomalies for observed behaviors in a video surveillance system
US8170283B2 (en) * 2009-09-17 2012-05-01 Behavioral Recognition Systems Inc. Video surveillance system configured to analyze complex behaviors using alternating layers of clustering and sequencing
TWI408623B (zh) * 2010-08-03 2013-09-11 Wistron Corp 監視系統及其監視影像錄製方法，及其機器可讀取媒體
CN102377984A (zh) * 2010-08-09 2012-03-14 纬创资通股份有限公司 监视影像录制方法和监视系统以及计算机程序产品
JP5704863B2 (ja) * 2010-08-26 2015-04-22 キヤノン株式会社 画像処理装置、画像処理方法及び記憶媒体
JP2012212373A (ja) * 2011-03-31 2012-11-01 Casio Comput Co Ltd 画像処理装置、画像処理方法及びプログラム
DE102011075412A1 (de) * 2011-05-06 2012-11-08 Deutsche Telekom Ag Verfahren und System zum Erfassen einer statischen Szene, zum Bestimmen von Rohereignissen und zum Erkennen von freien Flächen in einem Beobachtungsgebiet
US8831287B2 (en) * 2011-06-09 2014-09-09 Utah State University Systems and methods for sensing occupancy
US11025865B1 (en) * 2011-06-17 2021-06-01 Hrl Laboratories, Llc Contextual visual dataspaces
WO2013013079A2 (en) * 2011-07-19 2013-01-24 Utah State University Research Foundation Systems, devices, and methods for monitoring and controlling a controlled space
US20140163703A1 (en) * 2011-07-19 2014-06-12 Utah State University Systems, devices, and methods for multi-occupant tracking
CN103020980A (zh) * 2011-09-20 2013-04-03 佳都新太科技股份有限公司 一种基于改进双层码本模型的运动目标检测方法
US8873852B2 (en) * 2011-09-29 2014-10-28 Mediatek Singapore Pte. Ltd Method and apparatus for foreground object detection
KR101870902B1 (ko) * 2011-12-12 2018-06-26 삼성전자주식회사 영상 처리 장치 및 영상 처리 방법
IN2014DN08342A (de) 2012-03-15 2015-05-08 Behavioral Recognition Sys Inc
US8682036B2 (en) 2012-04-06 2014-03-25 Xerox Corporation System and method for street-parking-vehicle identification through license plate capturing
US9367966B2 (en) 2012-04-06 2016-06-14 Xerox Corporation Smartphone augmented video-based on-street parking management system
US20130265423A1 (en) * 2012-04-06 2013-10-10 Xerox Corporation Video-based detector and notifier for short-term parking violation enforcement
US9317908B2 (en) 2012-06-29 2016-04-19 Behavioral Recognition System, Inc. Automatic gain control filter in a video analysis system
US9723271B2 (en) 2012-06-29 2017-08-01 Omni Ai, Inc. Anomalous stationary object detection and reporting
BR112014032832A2 (pt) 2012-06-29 2017-06-27 Behavioral Recognition Sys Inc aprendizagem não supervisionada de anomalias de função para um sistema de vigilância por vídeo
US9111353B2 (en) 2012-06-29 2015-08-18 Behavioral Recognition Systems, Inc. Adaptive illuminance filter in a video analysis system
US9113143B2 (en) 2012-06-29 2015-08-18 Behavioral Recognition Systems, Inc. Detecting and responding to an out-of-focus camera in a video analytics system
US9911043B2 (en) 2012-06-29 2018-03-06 Omni Ai, Inc. Anomalous object interaction detection and reporting
EP2885766A4 (de) * 2012-08-20 2017-04-26 Behavioral Recognition Systems, Inc. Verfahren und system zur erkennung von öl an der meeresoberfläche
US10678259B1 (en) * 2012-09-13 2020-06-09 Waymo Llc Use of a reference image to detect a road obstacle
EP2918076A4 (de) 2012-11-12 2016-10-05 Behavioral Recognition Sys Inc Bildstabilisierungsverfahren für videoüberwachungssysteme
CN103034997B (zh) * 2012-11-30 2017-04-19 北京博创天盛科技有限公司 一种适用于监控视频前/背景分离的前景检测方法
US9020190B2 (en) 2013-01-31 2015-04-28 International Business Machines Corporation Attribute-based alert ranking for alert adjudication
BR112016002281A2 (pt) 2013-08-09 2017-08-01 Behavioral Recognition Sys Inc segurança de informação cognitiva usando um sistema de reconhecimento de comportamento
CN103473547A (zh) * 2013-09-23 2013-12-25 百年金海科技有限公司 一种用于智能交通检测系统的车辆目标物识别算法
AU2013273784B2 (en) * 2013-12-20 2016-06-02 Canon Kabushiki Kaisha Method, system and apparatus for updating a scene model
CN104574351B (zh) * 2014-08-06 2017-07-11 深圳市捷顺科技实业股份有限公司 一种基于视频处理的车位检测方法
US9471844B2 (en) * 2014-10-29 2016-10-18 Behavioral Recognition Systems, Inc. Dynamic absorption window for foreground background detector
US9460522B2 (en) * 2014-10-29 2016-10-04 Behavioral Recognition Systems, Inc. Incremental update for background model thresholds
US9349054B1 (en) 2014-10-29 2016-05-24 Behavioral Recognition Systems, Inc. Foreground detector for video analytics system
CN104378593A (zh) * 2014-11-17 2015-02-25 苏州立瓷电子技术有限公司 基于数据预处理和轮换存储的监控系统的智能控制方法
US10409910B2 (en) 2014-12-12 2019-09-10 Omni Ai, Inc. Perceptual associative memory for a neuro-linguistic behavior recognition system
US10409909B2 (en) 2014-12-12 2019-09-10 Omni Ai, Inc. Lexical analyzer for a neuro-linguistic behavior recognition system
EP3283972A4 (de) * 2015-04-17 2018-08-29 General Electric Company Identifizierung und verfolgung von fahrzeugen in bewegung
KR101866660B1 (ko) * 2016-10-19 2018-06-12 한국과학기술원 동적인 환경에서 배경모델에 기반한 rgb-d 시각 주행 거리 측정 방법 및 장치
CN107066929B (zh) * 2017-01-06 2021-06-08 重庆大学 一种融合多种特征的高速公路隧道停车事件分级识别方法
US10755419B2 (en) * 2017-01-30 2020-08-25 Nec Corporation Moving object detection apparatus, moving object detection method and program
US10049308B1 (en) * 2017-02-21 2018-08-14 A9.Com, Inc. Synthesizing training data
US10438072B2 (en) * 2017-02-27 2019-10-08 Echelon Corporation Video data background tracking and subtraction with multiple layers of stationary foreground and background regions
CN107220653B (zh) * 2017-04-11 2020-11-06 中国海洋大学 基于逻辑随机共振的水下弱目标检测系统的检测方法
US10373316B2 (en) * 2017-04-20 2019-08-06 Ford Global Technologies, Llc Images background subtraction for dynamic lighting scenarios
CA3295543A1 (en) 2018-09-28 2026-03-02 I.D. Systems, Inc. Cargo sensors, cargo-sensing units, cargo-sensing systems, and methods of using the same
CN109697725B (zh) * 2018-12-03 2020-10-02 浙江大华技术股份有限公司 一种背景过滤方法、装置及计算机可读存储介质
CN111665522B (zh) * 2020-05-19 2022-12-16 上海有个机器人有限公司 过滤激光扫描图中静止物体的方法、介质、终端和装置
EP3985957B1 (de) * 2020-10-14 2022-11-30 Axis AB Verfahren und systeme zur bewegungssegmentierung
CN112819843B (zh) * 2021-01-20 2022-08-26 上海大学 一种夜间电力线的提取方法及系统
DE102022202942A1 (de) 2022-03-25 2023-09-28 Robert Bosch Gesellschaft mit beschränkter Haftung Überwachungsvorrichtung, Überwachungsanordnungen, Verfahren, Computerprogramm und Speichermedium
US12536617B2 (en) * 2022-08-12 2026-01-27 Purdue Research Foundation Fused images backgrounds
EP4475076B1 (de) 2023-06-05 2025-12-31 Axis AB System und verfahren zur hintergrundmodellierung für einen videostrom
EP4641496A1 (de) * 2024-04-23 2025-10-29 INTEL Corporation Leistungseffiziente videokonferenz

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5812787A (en) * 1995-06-30 1998-09-22 Intel Corporation Video coding scheme with foreground/background separation
US6078619A (en) * 1996-09-12 2000-06-20 University Of Bath Object-oriented video system
US6211913B1 (en) * 1998-03-23 2001-04-03 Sarnoff Corporation Apparatus and method for removing blank areas from real-time stabilized images by inserting background information
US6292575B1 (en) * 1998-07-20 2001-09-18 Lau Technologies Real-time facial recognition and verification system
US6424370B1 (en) * 1999-10-08 2002-07-23 Texas Instruments Incorporated Motion based event detection system and method
US6542621B1 (en) * 1998-08-31 2003-04-01 Texas Instruments Incorporated Method of dealing with occlusion when tracking multiple objects and people in video sequences
US6570608B1 (en) * 1998-09-30 2003-05-27 Texas Instruments Incorporated System and method for detecting interactions of people and vehicles
US6583403B1 (en) * 1999-10-26 2003-06-24 Honda Giken Kogyo Kabushiki Kaisha Object detecting device, and travel safety system for vehicle
US6661918B1 (en) * 1998-12-04 2003-12-09 Interval Research Corporation Background estimation and segmentation based on range and color
US6674877B1 (en) * 2000-02-03 2004-01-06 Microsoft Corporation System and method for visually tracking occluded objects in real time
US6731799B1 (en) * 2000-06-01 2004-05-04 University Of Washington Object segmentation with background extraction and moving boundary techniques

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH05266198A (ja) * 1992-03-24 1993-10-15 N T T Data Tsushin Kk 物体検出装置
US5764306A (en) * 1997-03-18 1998-06-09 The Metaphor Group Real-time method of digitally altering a video data stream to remove portions of the original image and substitute elements to create a new image
KR100224752B1 (ko) * 1997-06-11 1999-10-15 윤종용 표적 추적 방법 및 장치
US7215795B2 (en) * 2000-09-28 2007-05-08 Hitachi Kokusai Electric Inc. Intruding object detecting method and intruding object monitoring apparatus employing the method
JP2002150295A (ja) * 2000-11-09 2002-05-24 Oki Electric Ind Co Ltd 物体検出方法及び物体検出装置
US6731805B2 (en) * 2001-03-28 2004-05-04 Koninklijke Philips Electronics N.V. Method and apparatus to distinguish deposit and removal in surveillance video
JP2002329196A (ja) * 2001-04-27 2002-11-15 Ntt Power & Building Facilities Inc 待ち時間検出システム

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5812787A (en) * 1995-06-30 1998-09-22 Intel Corporation Video coding scheme with foreground/background separation
US6078619A (en) * 1996-09-12 2000-06-20 University Of Bath Object-oriented video system
US6211913B1 (en) * 1998-03-23 2001-04-03 Sarnoff Corporation Apparatus and method for removing blank areas from real-time stabilized images by inserting background information
US6292575B1 (en) * 1998-07-20 2001-09-18 Lau Technologies Real-time facial recognition and verification system
US6542621B1 (en) * 1998-08-31 2003-04-01 Texas Instruments Incorporated Method of dealing with occlusion when tracking multiple objects and people in video sequences
US6570608B1 (en) * 1998-09-30 2003-05-27 Texas Instruments Incorporated System and method for detecting interactions of people and vehicles
US6661918B1 (en) * 1998-12-04 2003-12-09 Interval Research Corporation Background estimation and segmentation based on range and color
US6424370B1 (en) * 1999-10-08 2002-07-23 Texas Instruments Incorporated Motion based event detection system and method
US6583403B1 (en) * 1999-10-26 2003-06-24 Honda Giken Kogyo Kabushiki Kaisha Object detecting device, and travel safety system for vehicle
US6674877B1 (en) * 2000-02-03 2004-01-06 Microsoft Corporation System and method for visually tracking occluded objects in real time
US6731799B1 (en) * 2000-06-01 2004-05-04 University Of Washington Object segmentation with background extraction and moving boundary techniques

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Pankaj Kumar et al. (A Comparative Study of Different Color Spaces for Foreground and Shadow Detection for traffic Monitoring System IEEE-Sep. 2002). *

Cited By (152)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7680323B1 (en) 2000-04-29 2010-03-16 Cognex Corporation Method and apparatus for three-dimensional object segmentation
US20080062123A1 (en) * 2001-06-05 2008-03-13 Reactrix Systems, Inc. Interactive video display system using strobed light
US8300042B2 (en) 2001-06-05 2012-10-30 Microsoft Corporation Interactive video display system using strobed light
US7834846B1 (en) 2001-06-05 2010-11-16 Matthew Bell Interactive video display system
US20050036658A1 (en) * 2001-11-21 2005-02-17 Daniel Gibbins Non-motion detection
US7688997B2 (en) * 2001-11-21 2010-03-30 Iomniscient Pty Ltd Non-motion detection
US20050162381A1 (en) * 2002-05-28 2005-07-28 Matthew Bell Self-contained interactive video display system
US8035612B2 (en) 2002-05-28 2011-10-11 Intellectual Ventures Holding 67 Llc Self-contained interactive video display system
US8035614B2 (en) 2002-05-28 2011-10-11 Intellectual Ventures Holding 67 Llc Interactive video window
US7710391B2 (en) 2002-05-28 2010-05-04 Matthew Bell Processing an image utilizing a spatially varying pattern
US8035624B2 (en) 2002-05-28 2011-10-11 Intellectual Ventures Holding 67 Llc Computer vision based touch screen
US20050110964A1 (en) * 2002-05-28 2005-05-26 Matthew Bell Interactive video window display system
US20080150913A1 (en) * 2002-05-28 2008-06-26 Matthew Bell Computer vision based touch screen
US20080150890A1 (en) * 2002-05-28 2008-06-26 Matthew Bell Interactive Video Window
US20080100438A1 (en) * 2002-09-05 2008-05-01 Marrion Cyril C Multi-Zone Passageway Monitoring System and Method
US7920718B2 (en) 2002-09-05 2011-04-05 Cognex Corporation Multi-zone passageway monitoring system and method
US8199108B2 (en) 2002-12-13 2012-06-12 Intellectual Ventures Holding 67 Llc Interactive directed light/sound system
US20050089194A1 (en) * 2003-10-24 2005-04-28 Matthew Bell Method and system for processing captured image information in an interactive video display system
US7809167B2 (en) * 2003-10-24 2010-10-05 Matthew Bell Method and system for processing captured image information in an interactive video display system
US8487866B2 (en) 2003-10-24 2013-07-16 Intellectual Ventures Holding 67 Llc Method and system for managing an interactive video display system
US20090235295A1 (en) * 2003-10-24 2009-09-17 Matthew Bell Method and system for managing an interactive video display system
US20090225196A1 (en) * 2003-10-24 2009-09-10 Intellectual Ventures Holding 67 Llc Method and system for processing captured image information in an interactive video display system
US7536032B2 (en) * 2003-10-24 2009-05-19 Reactrix Systems, Inc. Method and system for processing captured image information in an interactive video display system
US7623674B2 (en) 2003-11-05 2009-11-24 Cognex Technology And Investment Corporation Method and system for enhanced portal security through stereoscopy
US20050249382A1 (en) * 2003-11-05 2005-11-10 Cognex Technology And Investment Corporation System and Method for Restricting Access through a Mantrap Portal
US20050093697A1 (en) * 2003-11-05 2005-05-05 Sanjay Nichani Method and system for enhanced portal security through stereoscopy
US8326084B1 (en) 2003-11-05 2012-12-04 Cognex Technology And Investment Corporation System and method of auto-exposure control for image acquisition hardware using three dimensional information
US7463754B2 (en) 2003-11-13 2008-12-09 Honda Motor Co. Adaptive probabilistic visual tracking with incremental subspace update
US20050175219A1 (en) * 2003-11-13 2005-08-11 Ming-Hsuan Yang Adaptive probabilistic visual tracking with incremental subspace update
US20050163346A1 (en) * 2003-12-03 2005-07-28 Safehouse International Limited Monitoring an output from a camera
US7664292B2 (en) * 2003-12-03 2010-02-16 Safehouse International, Inc. Monitoring an output from a camera
US20050157169A1 (en) * 2004-01-20 2005-07-21 Tomas Brodsky Object blocking zones to reduce false alarms in video surveillance systems
US8558892B2 (en) * 2004-01-20 2013-10-15 Honeywell International Inc. Object blocking zones to reduce false alarms in video surveillance systems
US7369682B2 (en) 2004-07-09 2008-05-06 Honda Motor Co., Ltd. Adaptive discriminative generative model and application to visual tracking
US20060036399A1 (en) * 2004-07-09 2006-02-16 Ming-Hsuan Yang Adaptive discriminative generative model and application to visual tracking
US20060023916A1 (en) * 2004-07-09 2006-02-02 Ming-Hsuan Yang Visual tracking using incremental fisher discriminant analysis
US7650011B2 (en) * 2004-07-09 2010-01-19 Honda Motor Co., Inc. Visual tracking using incremental fisher discriminant analysis
JP2011003207A (ja) * 2004-07-09 2011-01-06 Honda Motor Co Ltd 視覚追跡のための適応型判別生成モデル及び逐次的フィッシャー判別分析並びにアプリケーション
US7639841B2 (en) * 2004-12-20 2009-12-29 Siemens Corporation System and method for on-road detection of a vehicle using knowledge fusion
US20060177099A1 (en) * 2004-12-20 2006-08-10 Ying Zhu System and method for on-road detection of a vehicle using knowledge fusion
US20080219554A1 (en) * 2005-01-13 2008-09-11 International Business Machines Corporation System and Method for Adaptively Separating Foreground From Arbitrary Background in Presentations
US7668371B2 (en) * 2005-01-13 2010-02-23 International Business Machines Corporation System and method for adaptively separating foreground from arbitrary background in presentations
US20060153448A1 (en) * 2005-01-13 2006-07-13 International Business Machines Corporation System and method for adaptively separating foreground from arbitrary background in presentations
US9128519B1 (en) 2005-04-15 2015-09-08 Intellectual Ventures Holding 67 Llc Method and system for state-based control of objects
US20060269136A1 (en) * 2005-05-23 2006-11-30 Nextcode Corporation Efficient finder patterns and methods for application to 2D machine vision problems
WO2006127608A3 (en) * 2005-05-23 2007-12-06 Nextcode Corp Efficient finder patterns and methods for application to 2d machine vision problems
US7412089B2 (en) * 2005-05-23 2008-08-12 Nextcode Corporation Efficient finder patterns and methods for application to 2D machine vision problems
US20060268111A1 (en) * 2005-05-31 2006-11-30 Objectvideo, Inc. Multi-state target tracking
US7825954B2 (en) * 2005-05-31 2010-11-02 Objectvideo, Inc. Multi-state target tracking
US8081822B1 (en) 2005-05-31 2011-12-20 Intellectual Ventures Holding 67 Llc System and method for sensing a feature of an object in an interactive video display
US7801330B2 (en) 2005-06-24 2010-09-21 Objectvideo, Inc. Target detection and tracking from video streams
US20070127774A1 (en) * 2005-06-24 2007-06-07 Objectvideo, Inc. Target detection and tracking from video streams
US8111904B2 (en) 2005-10-07 2012-02-07 Cognex Technology And Investment Corp. Methods and apparatus for practical 3D vision system
US8098277B1 (en) 2005-12-02 2012-01-17 Intellectual Ventures Holding 67 Llc Systems and methods for communication between a reactive video system and a mobile communication device
US20070183661A1 (en) * 2006-02-07 2007-08-09 El-Maleh Khaled H Multi-mode region-of-interest video object segmentation
US8265349B2 (en) 2006-02-07 2012-09-11 Qualcomm Incorporated Intra-mode region-of-interest video object segmentation
US20070183662A1 (en) * 2006-02-07 2007-08-09 Haohong Wang Inter-mode region-of-interest video object segmentation
US20070183663A1 (en) * 2006-02-07 2007-08-09 Haohong Wang Intra-mode region-of-interest video object segmentation
US8605945B2 (en) 2006-02-07 2013-12-10 Qualcomm, Incorporated Multi-mode region-of-interest video object segmentation
US8265392B2 (en) * 2006-02-07 2012-09-11 Qualcomm Incorporated Inter-mode region-of-interest video object segmentation
US8150155B2 (en) 2006-02-07 2012-04-03 Qualcomm Incorporated Multi-mode region-of-interest video object segmentation
US8334906B2 (en) 2006-05-24 2012-12-18 Objectvideo, Inc. Video imagery-based sensor
US20070285510A1 (en) * 2006-05-24 2007-12-13 Object Video, Inc. Intelligent imagery-based sensor
US9591267B2 (en) 2006-05-24 2017-03-07 Avigilon Fortress Corporation Video imagery-based sensor
US8270668B2 (en) * 2006-06-01 2012-09-18 Ana Tec As Method and apparatus for analyzing objects contained in a flow or product sample where both individual and common data for the objects are calculated and monitored
US20080031489A1 (en) * 2006-06-01 2008-02-07 Frode Reinholt Method and an apparatus for analysing objects
US8456528B2 (en) * 2007-03-20 2013-06-04 International Business Machines Corporation System and method for managing the interaction of object detection and tracking systems in video surveillance
US20080231709A1 (en) * 2007-03-20 2008-09-25 Brown Lisa M System and method for managing the interaction of object detection and tracking systems in video surveillance
US7949150B2 (en) * 2007-04-02 2011-05-24 Objectvideo, Inc. Automatic camera calibration and geo-registration using objects that provide positional information
US20080240616A1 (en) * 2007-04-02 2008-10-02 Objectvideo, Inc. Automatic camera calibration and geo-registration using objects that provide positional information
US20080252596A1 (en) * 2007-04-10 2008-10-16 Matthew Bell Display Using a Three-Dimensional vision System
US20080273754A1 (en) * 2007-05-04 2008-11-06 Leviton Manufacturing Co., Inc. Apparatus and method for defining an area of interest for image sensing
US8126260B2 (en) 2007-05-29 2012-02-28 Cognex Corporation System and method for locating a three-dimensional object using machine vision
US20080298636A1 (en) * 2007-06-04 2008-12-04 Object Video, Inc. Method for detecting water regions in video
US7822275B2 (en) 2007-06-04 2010-10-26 Objectvideo, Inc. Method for detecting water regions in video
US8542872B2 (en) 2007-07-03 2013-09-24 Pivotal Vision, Llc Motion-validating remote monitoring system
US20090010493A1 (en) * 2007-07-03 2009-01-08 Pivotal Vision, Llc Motion-Validating Remote Monitoring System
US9286518B2 (en) 2007-07-03 2016-03-15 Pivotal Vision, Llc Motion-validating remote monitoring system
US12424066B2 (en) 2007-07-03 2025-09-23 Pivotal Vision, Llc Motion-validating remote monitoring system
US10275658B2 (en) 2007-07-03 2019-04-30 Pivotal Vision, Llc Motion-validating remote monitoring system
US8150103B2 (en) * 2007-09-04 2012-04-03 Objectvideo, Inc. Background modeling with feature blocks
US20090060277A1 (en) * 2007-09-04 2009-03-05 Objectvideo, Inc. Background modeling with feature blocks
US20090077504A1 (en) * 2007-09-14 2009-03-19 Matthew Bell Processing of Gesture-Based User Interactions
US8230367B2 (en) 2007-09-14 2012-07-24 Intellectual Ventures Holding 67 Llc Gesture-based user interactions with status indicators for acceptable inputs in volumetric zones
US9058058B2 (en) 2007-09-14 2015-06-16 Intellectual Ventures Holding 67 Llc Processing of gesture-based user interactions activation levels
US10990189B2 (en) 2007-09-14 2021-04-27 Facebook, Inc. Processing of gesture-based user interaction using volumetric zones
US10564731B2 (en) 2007-09-14 2020-02-18 Facebook, Inc. Processing of gesture-based user interactions using volumetric zones
US9811166B2 (en) 2007-09-14 2017-11-07 Intellectual Ventures Holding 81 Llc Processing of gesture-based user interactions using volumetric zones
US8159682B2 (en) 2007-11-12 2012-04-17 Intellectual Ventures Holding 67 Llc Lens system
US8810803B2 (en) 2007-11-12 2014-08-19 Intellectual Ventures Holding 67 Llc Lens system
US9229107B2 (en) 2007-11-12 2016-01-05 Intellectual Ventures Holding 81 Llc Lens system
US20090251685A1 (en) * 2007-11-12 2009-10-08 Matthew Bell Lens System
US20090290020A1 (en) * 2008-02-28 2009-11-26 Canon Kabushiki Kaisha Stationary Object Detection Using Multi-Mode Background Modelling
US8305440B2 (en) 2008-02-28 2012-11-06 Canon Kabushiki Kaisha Stationary object detection using multi-mode background modelling
US8259163B2 (en) 2008-03-07 2012-09-04 Intellectual Ventures Holding 67 Llc Display with built in 3D sensing
US10831278B2 (en) 2008-03-07 2020-11-10 Facebook, Inc. Display with built in 3D sensing capability and gesture control of tv
US9247236B2 (en) 2008-03-07 2016-01-26 Intellectual Ventures Holdings 81 Llc Display with built in 3D sensing capability and gesture control of TV
US20090315996A1 (en) * 2008-05-09 2009-12-24 Sadiye Zeyno Guler Video tracking systems and methods employing cognitive vision
US9019381B2 (en) 2008-05-09 2015-04-28 Intuvision Inc. Video tracking systems and methods employing cognitive vision
US10121079B2 (en) 2008-05-09 2018-11-06 Intuvision Inc. Video tracking systems and methods employing cognitive vision
US20090304234A1 (en) * 2008-06-06 2009-12-10 Sony Corporation Tracking point detecting device and method, program, and recording medium
US20100121866A1 (en) * 2008-06-12 2010-05-13 Matthew Bell Interactive display management systems and methods
US8595218B2 (en) 2008-06-12 2013-11-26 Intellectual Ventures Holding 67 Llc Interactive display management systems and methods
US20100092036A1 (en) * 2008-06-17 2010-04-15 Subhodev Das Method and apparatus for detecting targets through temporal scene changes
US8243991B2 (en) 2008-06-17 2012-08-14 Sri International Method and apparatus for detecting targets through temporal scene changes
US20100128930A1 (en) * 2008-11-24 2010-05-27 Canon Kabushiki Kaisha Detection of abandoned and vanished objects
US8422791B2 (en) 2008-11-24 2013-04-16 Canon Kabushiki Kaisha Detection of abandoned and vanished objects
US8218864B2 (en) * 2009-02-25 2012-07-10 Msi Computer(Shenzhen)Co., Ltd. Method and device for maintaining image background by multiple Gaussian models
US20100215256A1 (en) * 2009-02-25 2010-08-26 Micro-Star Int'l Co., Ltd. Method and device for maintaining image background by multiple gaussian models
US20170236010A1 (en) * 2009-10-19 2017-08-17 Canon Kabushiki Kaisha Image pickup apparatus, information processing apparatus, and information processing method
US20110273619A1 (en) * 2010-05-10 2011-11-10 Canon Kabushiki Kaisha Image processing apparatus, method, and storage medium
US9424655B2 (en) * 2010-05-10 2016-08-23 Canon Kabushiki Kaisha Image processing apparatus, method, and storage medium for analyzing changes in video
US9247155B2 (en) * 2011-12-21 2016-01-26 Canon Kabushiki Kaisha Method and system for robust scene modelling in an image sequence
US20130162867A1 (en) * 2011-12-21 2013-06-27 Canon Kabushiki Kaisha Method and system for robust scene modelling in an image sequence
US8737745B2 (en) 2012-03-27 2014-05-27 The Nielsen Company (Us), Llc Scene-based people metering for audience measurement
US9667920B2 (en) 2012-03-27 2017-05-30 The Nielsen Company (Us), Llc Hybrid active and passive people metering for audience measurement
US9224048B2 (en) 2012-03-27 2015-12-29 The Nielsen Company (Us), Llc Scene-based people metering for audience measurement
US9185456B2 (en) 2012-03-27 2015-11-10 The Nielsen Company (Us), Llc Hybrid active and passive people metering for audience measurement
US9911065B2 (en) 2012-09-19 2018-03-06 Placemeter LLC System and method for processing image data
US9213781B1 (en) 2012-09-19 2015-12-15 Placemeter LLC System and method for processing image data
US10902282B2 (en) 2012-09-19 2021-01-26 Placemeter Inc. System and method for processing image data
US20140233793A1 (en) * 2012-09-21 2014-08-21 Canon Kabushiki Kaisha Differentiating abandoned and removed object using temporal edge information
US9245207B2 (en) * 2012-09-21 2016-01-26 Canon Kabushiki Kaisha Differentiating abandoned and removed object using temporal edge information
US20150189191A1 (en) * 2013-12-27 2015-07-02 Telemetrio LLC Process and system for video production and tracking of objects
US10880524B2 (en) 2014-05-30 2020-12-29 Placemeter Inc. System and method for activity monitoring using video data
US10735694B2 (en) 2014-05-30 2020-08-04 Placemeter Inc. System and method for activity monitoring using video data
US9390333B2 (en) 2014-07-07 2016-07-12 Geo Semiconductor Inc. System and method for robust motion detection
US9245187B1 (en) 2014-07-07 2016-01-26 Geo Semiconductor Inc. System and method for robust motion detection
US10110856B2 (en) 2014-12-05 2018-10-23 Avigilon Fortress Corporation Systems and methods for video analysis rules based on map data
US10687022B2 (en) 2014-12-05 2020-06-16 Avigilon Fortress Corporation Systems and methods for automated visual surveillance
US10708548B2 (en) 2014-12-05 2020-07-07 Avigilon Fortress Corporation Systems and methods for video analysis rules based on map data
US10096117B2 (en) 2014-12-24 2018-10-09 Canon Kabushiki Kaisha Video segmentation method
US9934447B2 (en) 2015-03-20 2018-04-03 Netra, Inc. Object detection and classification
US9760792B2 (en) 2015-03-20 2017-09-12 Netra, Inc. Object detection and classification
US9922271B2 (en) 2015-03-20 2018-03-20 Netra, Inc. Object detection and classification
US10043307B2 (en) 2015-04-17 2018-08-07 General Electric Company Monitoring parking rule violations
US10380430B2 (en) 2015-04-17 2019-08-13 Current Lighting Solutions, Llc User interfaces for parking zone creation
US11328515B2 (en) 2015-04-17 2022-05-10 Ubicquia Iq Llc Determining overlap of a parking space by a vehicle
US10872241B2 (en) 2015-04-17 2020-12-22 Ubicquia Iq Llc Determining overlap of a parking space by a vehicle
US9940524B2 (en) * 2015-04-17 2018-04-10 General Electric Company Identifying and tracking vehicles in motion
US10726271B2 (en) 2015-04-21 2020-07-28 Placemeter, Inc. Virtual turnstile system and method
US11334751B2 (en) 2015-04-21 2022-05-17 Placemeter Inc. Systems and methods for processing video data for activity monitoring
US10043078B2 (en) 2015-04-21 2018-08-07 Placemeter LLC Virtual turnstile system and method
US10997428B2 (en) 2015-06-01 2021-05-04 Placemeter Inc. Automated detection of building entrances
US11138442B2 (en) 2015-06-01 2021-10-05 Placemeter, Inc. Robust, adaptive and efficient object detection, classification and tracking
US10380431B2 (en) 2015-06-01 2019-08-13 Placemeter LLC Systems and methods for processing video streams
US10468065B2 (en) 2015-10-28 2019-11-05 Ustudio, Inc. Video frame difference engine
US11100335B2 (en) 2016-03-23 2021-08-24 Placemeter, Inc. Method for queue time estimation
US11312594B2 (en) 2018-11-09 2022-04-26 Otis Elevator Company Conveyance system video analytics
US11600072B2 (en) 2018-12-12 2023-03-07 Motorola Solutions, Inc. Object left behind detection
US11126861B1 (en) 2018-12-14 2021-09-21 Digimarc Corporation Ambient inventorying arrangements
US12437542B2 (en) 2018-12-14 2025-10-07 Digimarc Corporation Methods and systems employing image sensing and 3D sensing to identify shelved products

Also Published As

Publication number Publication date
MXPA05008201A (es) 2006-03-30
ATE548706T1 (de) 2012-03-15
CN1757037A (zh) 2006-04-05
EP1588317A1 (de) 2005-10-26
ES2383174T3 (es) 2012-06-18
KR20060012570A (ko) 2006-02-08
CN100386771C (zh) 2008-05-07
DK1588317T3 (da) 2012-04-30
WO2004070649A1 (en) 2004-08-19
US20040151342A1 (en) 2004-08-05
CA2514826A1 (en) 2004-08-19
JP2006514363A (ja) 2006-04-27
EP1588317A4 (de) 2009-05-06
EP1588317B1 (de) 2012-03-07
AU2003300337A1 (en) 2004-08-30
HK1088968A1 (zh) 2006-11-17

Similar Documents

Publication Publication Date Title
US6999600B2 (en) Video scene background maintenance using change detection and classification
Joshi et al. A survey on moving object detection and tracking in video surveillance system
AU2008200966B2 (en) Stationary object detection using multi-mode background modelling
AU2008200967B2 (en) Spatio-activity based mode matching
JP2008192131A (ja) 特徴レベル・セグメンテーションを実行するシステムおよび方法
WO2003036557A1 (en) Method and apparatus for background segmentation based on motion localization
Lou et al. An illumination invariant change detection algorithm
Yousefi et al. A novel motion detection method using 3D discrete wavelet transform
CN108765466A (zh) 一种基于网络摄像头的智能视频监控系统及方法
Wan et al. Background subtraction based on adaptive non-parametric model
Ng et al. Background subtraction using a pixel-wise adaptive learning rate for object tracking initialization
Senior An introduction to automatic video surveillance
Amato et al. Robust real-time background subtraction based on local neighborhood patterns
Kaur Background subtraction in video surveillance
Doulamis et al. Self Adaptive background modeling for identifying persons' falls
Spagnolo et al. Fast background modeling and shadow removing for outdoor surveillance
Singh et al. An interactive framework for abandoned and removed object detection in video
Dahyot et al. Unsupervised statistical detection of changing objects in camera-in-motion video
HK1088968B (en) Video scene background maintenance using change detection and classification
Olaniyi et al. A Systematic Review of Background Subtraction Algorithms for Smart Surveillance System
Cho et al. Panoramic background generation using mean-shift in moving camera environment
Pardo Extraction of semantic objects from still images
Khalifa et al. Complex background subtraction for biometric identification
Kim RGB Motion segmentation using Background subtraction based on AMF
Wu et al. Change detection by thresholding-with-hysteresis.

Legal Events

Date Code Title Description
AS Assignment

Owner name: DIAMONDBACK VISION, INC., VIRGINIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:VENETIANER, PETER L.;LIPTON, ALAN J.;CHOSAK, ANDREW J.;AND OTHERS;REEL/FRAME:013717/0245

Effective date: 20030127

AS Assignment

Owner name: OBJECTVIDEO, INC., VIRGINIA

Free format text: CHANGE OF NAME;ASSIGNOR:DIAMONDBACK VISION, INC.;REEL/FRAME:014743/0573

Effective date: 20031119

STCF Information on status: patent grant

Free format text: PATENTED CASE

AS Assignment

Owner name: RJF OV, LLC, DISTRICT OF COLUMBIA

Free format text: SECURITY AGREEMENT;ASSIGNOR:OBJECTVIDEO, INC.;REEL/FRAME:020478/0711

Effective date: 20080208

Owner name: RJF OV, LLC,DISTRICT OF COLUMBIA

Free format text: SECURITY AGREEMENT;ASSIGNOR:OBJECTVIDEO, INC.;REEL/FRAME:020478/0711

Effective date: 20080208

AS Assignment

Owner name: RJF OV, LLC, DISTRICT OF COLUMBIA

Free format text: GRANT OF SECURITY INTEREST IN PATENT RIGHTS;ASSIGNOR:OBJECTVIDEO, INC.;REEL/FRAME:021744/0464

Effective date: 20081016

Owner name: RJF OV, LLC,DISTRICT OF COLUMBIA

Free format text: GRANT OF SECURITY INTEREST IN PATENT RIGHTS;ASSIGNOR:OBJECTVIDEO, INC.;REEL/FRAME:021744/0464

Effective date: 20081016

FPAY Fee payment

Year of fee payment: 4

FEPP Fee payment procedure

Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

FEPP Fee payment procedure

Free format text: PAT HOLDER NO LONGER CLAIMS SMALL ENTITY STATUS, ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: STOL); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

AS Assignment

Owner name: OBJECTVIDEO, INC., VIRGINIA

Free format text: RELEASE OF SECURITY AGREEMENT/INTEREST;ASSIGNOR:RJF OV, LLC;REEL/FRAME:027810/0117

Effective date: 20101230

FPAY Fee payment

Year of fee payment: 8

AS Assignment

Owner name: AVIGILON FORTRESS CORPORATION, CANADA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:OBJECTVIDEO, INC.;REEL/FRAME:034552/0112

Effective date: 20141217

AS Assignment

Owner name: HSBC BANK CANADA, CANADA

Free format text: SECURITY INTEREST;ASSIGNOR:AVIGILON FORTRESS CORPORATION;REEL/FRAME:035387/0569

Effective date: 20150407

FPAY Fee payment

Year of fee payment: 12

AS Assignment

Owner name: AVIGILON FORTRESS CORPORATION, CANADA

Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:HSBC BANK CANADA;REEL/FRAME:047032/0063

Effective date: 20180813

AS Assignment

Owner name: MOTOROLA SOLUTIONS, INC., ILLINOIS

Free format text: NUNC PRO TUNC ASSIGNMENT;ASSIGNOR:AVIGILON FORTRESS CORPORATION;REEL/FRAME:061746/0897

Effective date: 20220411