WO2012173466A1 - Système et procédé de comptage d'objets - Google Patents

Système et procédé de comptage d'objets Download PDF

Info

Publication number
WO2012173466A1
WO2012173466A1 PCT/MY2012/000121 MY2012000121W WO2012173466A1 WO 2012173466 A1 WO2012173466 A1 WO 2012173466A1 MY 2012000121 W MY2012000121 W MY 2012000121W WO 2012173466 A1 WO2012173466 A1 WO 2012173466A1
Authority
WO
WIPO (PCT)
Prior art keywords
motion
objects
blobs
blob
occlusion
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/MY2012/000121
Other languages
English (en)
Inventor
Kim Meng Liang
Sze Ling Tang
Kadim Zulaikha
Samudin Norshuhada
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mimos Bhd
Original Assignee
Mimos Bhd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mimos Bhd filed Critical Mimos Bhd
Publication of WO2012173466A1 publication Critical patent/WO2012173466A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G06V20/53—Recognition of crowd images, e.g. recognition of crowd congestion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30242—Counting objects in image

Definitions

  • the present invention relates generally to object counting technology, and more particularly to a system and method for object counting by employing learned occlusion rate and optimized classifiers.
  • Object counting is an important part of video surveillance and video analysis system. Object counting is best described as automatic process to calculate number of object in the scene or passing through a monitored area or line.
  • one objective of the present invention is to provide a system and method for accurate object counting.
  • One aspect of the present invention provides a method of object counting, wherein the method is performed in a computer-executable electronic medium.
  • the method comprises initializing an area of interest, wherein an area where the number of objects that move into/out this area is to be counted is defined; dividing a captured scene of the area of interest into a grid of blocks; determining the average size information of an object for each block of the grid of blocks; learning occlusion rates of motion blobs in the captured scene so that classifiers are optimized based on the occlusion rates of the motion blobs; detecting objects in motion blobs in the captured scene; tracking the detected objects in the motion blobs so as to give the detected moving blobs a consistent label throughout the appearance of the blobs in the scene ; and counting objects in the tracked motion blobs by analyzing the tracked motion blobs based on their occlusion rates, where the occlusion rates of the tracked motion blobs is determined with the optimized
  • the average size information of an object for each block of the grid of blocks is recorded for block grid that has been travelled by the object during registration; and for any individual block grid where the average size information of an object is not calculated as the object does not travel to that block, the average size information of an object is automatically estimated based on the average size information of other individual block grids that have been updated during the travel of the object.
  • the step of learning the occlusion rate comprises determining classes of motion blobs according to their occlusion rate and arrangement type; extracting features for each determined class of motion blob; and optimizing classifier with the extracted features.
  • the step of determining classes of motion blobs according to their occlusion rate and arrangement type comprises analyzing the dimension of the motion blobs to determine the arrangement types; and analyzing the overlapping ratio of the objects in the motion blobs to determine the occlusion rate for each arrangement type.
  • the step of extracting features for each determined class of motion blob comprises extracting the silhouette feature of the objects area as the foreground information; extracting the silhouette feature of the non- objects area as the background information; extracting the internal features of the objects area including edge information, texture and shape information of the motion blob; and storing the extracted features in a storage medium.
  • the step of counting objects in motion blobs comprises extracting the bounding box information of the tracked motion blob; and deciding on whether the bounding box information is equal to a noise ratio; if yes, the number of objects for the motion blob image is assigned to zero; if no, a further decision is made as to whether the bounding box information is equivalent to a single object; if yes, the number of objects for the motion blob is assigned to one; if no, a yet another decision step is made to as to whether the bounding box information is larger than a single object; if yes, the number of count for the motion blob is determined based on the occlusion rate.
  • the step of counting the number of objects in the motion blob based on occlusion rate comprises extracting the features of the motion blob; selecting the optimized classifier and determining the value of the occlusion rate; updating the average size information of the object that is determined during the registration process as a new value based on the occlusion rate; and determining the number of object from the blob image by dividing the blob image accordingly.
  • Another aspect of the present invention provides an object counting system.
  • the system comprises a capturing device for capturing images of an area of interest; a computer-executable electronic medium for storage and embedded algorithm, wherein the computer-executable electronic medium is electronically connected to the capturing device for receiving the captured images; and a displaying device electronically connected to the computer-executable electronic medium for displaying the output from the computer-executable electronic medium;
  • the embedded algorithm comprises a method of object counting, said method comprising initializing an area of interest, wherein an area where the number of objects that move into/out this area is to be counted is defined; dividing a captured scene of the area of interest into a grid of blocks; determining the average size information of an object for each block of the grid of blocks; learning occlusion rates of motion blobs in the captured scene so that classifiers are optimized based on the occlusion rates of the motion blobs; detecting objects in motion blobs in the captured scene; tracking the detected objects in the motion blobs so as to give the detected moving blobs
  • FIG 1 is a flowchart illustrating the method of object counting in accordance with one embodiment of the present invention.
  • FIG 2 is a flowchart of learning the occlusion rate in accordance with one embodiment of the present invention.
  • FIG 3 illustrates the classes of motion blobs with different arrangement types and occlusion rates.
  • FIG 4 is a flowchart of determining classes of motion blobs that have different occlusion rate and arrangement type 41 in accordance with one embodiment of the present invention.
  • FIG 5 is a flowchart of determining arrangement type in accordance with one embodiment of the present invention.
  • FIG 6 is a flowchart of determining the occlusion rate for each arrangement type in accordance with one embodiment of the present invention.
  • FIG 7 is a flowchart showing the determination of the occlusion rate based on the overlapping area ratio as compared to predefined threshold values.
  • FIG 8 is a flowchart of extracting features for each determined class of motion blob in accordance with one embodiment of the present invention.
  • FIG 9 depicts the two features that represent the foreground and background information of the motion blob.
  • FIG 10 is a flowchart of optimizing classifier in accordance with one embodiment of the present invention.
  • FIG 1 1 is a flowchart illustrating the process of counting objects in motion blobs in accordance with one embodiment of the present invention.
  • FIG 12 is a flowchart illustrating the process of counting the number of objects in the motion blob based on occlusion rate in accordance with one embodiment of the present invention.
  • FIG 13 is a flowchart illustrating the process of updating the average size information in accordance with one embodiment of the present invention.
  • One aspect of the present invention provides a method for determining the number of objects in the motion blobs where the objects are overlapping with each other during counting process.
  • the method of determining number of object of the present invention provides a method to learn the average size information of the object based on the occlusion rate. This is done at the online mode.
  • FIG 1 a flowchart illustrating the method of object counting in accordance with one embodiment of the present invention.
  • the method is performed by a computer-executable electronic medium such as microprocessors, computers, PDA, or the like.
  • the method of object counting is used to determine the number of objects in a monitored area, where the objects in the monitored area are captured by any suitable capturing devices such as camera.
  • the method of object counting comprises the operations of registration and online counting, where the registration comprises initializing area of interest 10, dividing the captured scene of the area of interest into grid 20, determining the average size information for an object 30, and learning the occlusion rate 40; when the registration is completed 50, the online counting commences, where the online counting comprises detecting objects 60, tracking objects 70, and counting objects 80. Upon finishing counting will the number of objects be output to any suitable displaying devices.
  • the step of initializing area of interest 10 defines an area where the number of objects that move into/out this area will be counted.
  • the area of interest may be formed by a region or line.
  • the step of dividing the captured scene of the area of interest into grid 20 divides the captured scene into a grid of blocks with fixed or non-fixed dimension.
  • the step of determining the average size information for an object 30 determines the average size information of an object for each individual blocks, where the average size information is recorded for block grid that has been travelled by the object during registration.
  • the centroid location of the object indicates the individual block grid where the average size information of the particular object is calculated.
  • the average size information of an object is automatically estimated based on the average size information of other individual block grids that have been updated during the travel of the object. The estimation can be done using any known interpolation methods including nearest neighborhood, bilinear and bicubic.
  • the step of learning the occlusion rate 40 learns the occlusion rate for detected motion blobs. Briefly, based on the collected samples of motion blobs with different occlusion rates, classifiers are optimized to learn the patterns of the motion blobs to indicate various occlusion rates.
  • the step of learning the occlusion rate 40 comprises determining classes of motion blobs that have different occlusion rate and arrangement type 41, extracting features for each determined class of motion blob 42, and optimizing classifier with the extracted features 43.
  • the step of determining classes of motion blobs that have different occlusion rate and arrangement type 41 determines the classes of motion blobs based on occlusion rates and arrangement types. These classes of motion blobs are important as a source to the optimization of classifier.
  • the arrangement types can be flexibly designated considering the counting accuracy and computing complexity.
  • One exemplary designation of the arrangement types includes vertical, horizontal and diagonal. The same for the occlusion rates; they can also be flexibly categorized.
  • One exemplary categorization of the occlusion rates includes three rates: 0%, 25% and 50% occlusion rates. It is apparent that the designation of arrangement types and the categorization of occlusion rates are used to reduce the computing complexity without significantly compromising the counting accuracy.
  • FIG 3 illustrates the classes of motion blobs with different arrangement types (only vertical and horizontal arrangement types shown) and occlusion rates (only 0%, 25% and 50% occlusion rates shown).
  • the classes of motion blobs are determined based on their arrangement types and occlusion rates, where the arrangement types are determined by analyzing the dimension of the motion blobs 411, and the occlusion rate for each arrangement type is determined by analyzing the overlapping ratio of the objects in the motion blobs 412.
  • FIG 5 a flowchart of determining arrangement type 411 in accordance with one embodiment of the present invention.
  • the dimension of a motion blob is defined by its height and width, and a standard height and width for a motion blob is predefined for determining the arrangement type.
  • To determine the arrangement type of a motion blob its height and width are analyzed, and then the height and width of the motion blob are compared with the standard height and width to determine the arrangement type of the motion blob.
  • FIG 5 shows the determination of arrangement types of motion blobs with three arrangement types including vertical, horizontal and diagonal.
  • FIG 6 there is provided a flowchart of determining the occlusion rate for each arrangement type 412 in accordance with one embodiment of the present invention.
  • the boundary of each of the object in the motion blob is determined 4121; then calculates the overlapping area ratio between the boundaries of each of the objects and the area of the motion blob 4122; in other words, the ratio between the overlapping areas between each objects within the motion blob to the area of the motion blob; and finally the occlusion rate is determined based on the overlapping area ratio as compared to predefined threshold values 4123.
  • FIG 7 a flowchart showing the determination of the occlusion rate based on the overlapping area ratio as compared to predefined threshold values.
  • the determination of the occlusion rates starts with a decision as to whether the overlapping area ratio is less than Tl 41231; if yes, the occlusion rate for the motion blob is assigned as 0% 41232.
  • step 41231 a further decision is made as to whether the overlapping area ratio is less than T2 41233; if yes, the occlusion rate of the motion blob is assigned as 25% 41234; and if no, the occlusion rate of the motion blob is assigned as 50% 41235. Furthermore, the assigned occlusion rate is just a reference value; it can be other suitable values.
  • the step of extracting features for each determined class of motion blob 42 extracts the features from each image in all the determined classes of motion blobs.
  • the extracted features encompass the information of the foreground and background of the blob image.
  • FIG 8 there is provided a flowchart of extracting features for each determined class of motion blob 42 in accordance with one embodiment of the present invention.
  • the silhouette of the objects area is extracted as the foreground information 421.
  • the silhouette of the non-objects area is extracted as the background information 422.
  • extracted are the features including edge information, texture and shape information from the motion blob, which represent the internal information of the objects area 423.
  • the extracted features are stored in the storage medium 424.
  • FIG 9 depicts the two features that represent the foreground and background information of the motion blob.
  • the classifier is optimized 43.
  • the optimization can be done with the known optimization methods including backpropagation.
  • the optimization first extracts a set of features from the storage medium 431.
  • the second step involves updating the weights of the classifier base on the feature values 432.
  • a decision step is to decide whether the extraction index reaches the end of the storage medium 433; if no, the extraction index is increased 434, and the operation goes back to step 431 until all the set of features are extracted. If yes in step 433, the fitness value of the classifier is calculated 435.
  • a further decision step is to decide whether the fitness value is larger than a predefined threshold value 436; if yes, the classifier is computed base on the updated weights 438, and the method proceeds to whether the registration is complete. If no in step 436, the extraction index is re-initialized to the beginning of the storage medium 437 and the process goes back to step 431 until the fitness value is larger than a predefined threshold value.
  • the step of detecting objects 60 detects the moving blobs in the scene using for example the known foreground detection techniques including frame differencing method.
  • the step of tracking objects 70 gives the detected moving blobs a consistent label throughout the appearance of the blobs in the scene.
  • the step of counting objects 80 calculates the number of objects by analyzing the tracked motion blobs based on the calculated occlusion rate.
  • the occlusion rate of the tracked motion blobs is determined from the motion blob with the optimized classifier.
  • FIG 1 1 there is provided a flowchart illustrating the process of counting objects in motion blobs 80 in accordance with one embodiment of the present invention.
  • the number of objects is determined by the bounding box information of the motion blobs.
  • the process of counting objects in motion blobs 80 first extracts the bounding box information of the tracked motion blob 81; then decide on whether the bounding box information is equal to the noise ratio 82; if yes, the number of objects for the motion blob image is assigned to zero 83. If no in step 82, a further decision is made as to whether the bounding box information is equivalent to a single object 84; if yes, the number of objects for the motion blob is assigned to one 85.
  • step 84 a yet another decision step is made to as to whether the bounding box information is larger than a single object 86; if yes, the number of count for the motion blob is determined based on the occlusion rate 87; If no, number of count is set to zero 89.
  • the process of counting the number of objects in the motion blob based on occlusion rate is described in detail hereinbelow.
  • FIG 12 there is provided a flowchart illustrating the process of counting the number of objects in the motion blob based on occlusion rate 87 in accordance with one embodiment of the present invention.
  • the features of the motion blob which are similar to the ones used in the process of registration are extracted 871.
  • the optimized classifier is selected and subsequently the value of the occlusion rate is determined 872.
  • the average size information of the object that is determined during the registration process is updated as a new value based on the occlusion rate 873. The detail of the update is highlighted in FIG 9. With the updated average size information, the number of object from the blob image is determined by dividing the blob image accordingly 874.
  • FIG 13 there is provided a flowchart illustrating the process of updating the average size information 873 in accordance with one embodiment of the present invention.
  • the determined occlusion rate value is evaluated. If the occlusion rate is equivalent to 25 8731, then the average size information of the object is reduced by 25% 8732. If the occlusion rate value is equivalent to 50 8734, then the average size information of the object is reduced by 50% 8735.
  • the present invention also provides an object counting system, where the system comprises an image capturing device for capturing images of a monitored area, an electronic processor electronically connected to the capturing device for processing the images and performing the object counting using the method described above, and a displaying device electronically connected to the electronic processor for displaying the results of object counting.
  • the capturing device can be any suitable ones including cameras.
  • the electronic processor can be any suitable ones as long as it has the computer- executable medium for storing data and embedding algorithm for performing the method.
  • the displaying device can be any suitable ones including flat screens.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

La présente invention concerne un procédé de comptage d'objets, qui emploie des taux d'occlusion appris de taches de mouvement dans la scène capturée et des classificateurs optimisés sur la base des taux d'occlusion des taches de mouvement, et compte des objets dans les taches de mouvement suivies par analyse des taches de mouvement suivies sur la base de leurs taux d'occlusion. La présente invention concerne également un système de comptage d'objets.
PCT/MY2012/000121 2011-06-17 2012-06-14 Système et procédé de comptage d'objets Ceased WO2012173466A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
MYPI2011002848A MY159291A (en) 2011-06-17 2011-06-17 System and method of object counting
MYPI2011002848 2011-06-17

Publications (1)

Publication Number Publication Date
WO2012173466A1 true WO2012173466A1 (fr) 2012-12-20

Family

ID=46724566

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/MY2012/000121 Ceased WO2012173466A1 (fr) 2011-06-17 2012-06-14 Système et procédé de comptage d'objets

Country Status (2)

Country Link
MY (1) MY159291A (fr)
WO (1) WO2012173466A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014200714A1 (fr) * 2013-06-14 2014-12-18 Qualcomm Incorporated Capture d'image assistée par dispositif de suivi

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
CHAN A B ET AL: "Privacy preserving crowd monitoring: Counting people without people models or tracking", COMPUTER VISION AND PATTERN RECOGNITION, 2008. CVPR 2008. IEEE CONFERENCE ON, IEEE, PISCATAWAY, NJ, USA, 23 June 2008 (2008-06-23), pages 1 - 7, XP031297127, ISBN: 978-1-4244-2242-5 *
KILAMBI ET AL: "Estimating pedestrian counts in groups", COMPUTER VISION AND IMAGE UNDERSTANDING, ACADEMIC PRESS, US, vol. 110, no. 1, 13 March 2008 (2008-03-13), pages 43 - 59, XP022527496, ISSN: 1077-3142, DOI: 10.1016/J.CVIU.2007.02.003 *
LAN DONG ET AL: "Fast Crowd Segmentation Using Shape Indexing", COMPUTER VISION, 2007. ICCV 2007. IEEE 11TH INTERNATIONAL CONFERENCE O N, IEEE, PI, 1 October 2007 (2007-10-01), pages 1 - 8, XP031194564, ISBN: 978-1-4244-1630-1 *
YAOBIN MAO ET AL: "Estimation of crowd density using multi-local features and regression", INTELLIGENT CONTROL AND AUTOMATION (WCICA), 2010 8TH WORLD CONGRESS ON, IEEE, PISCATAWAY, NJ, USA, 7 July 2010 (2010-07-07), pages 6295 - 6300, XP031736871, ISBN: 978-1-4244-6712-9 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014200714A1 (fr) * 2013-06-14 2014-12-18 Qualcomm Incorporated Capture d'image assistée par dispositif de suivi
CN105264570A (zh) * 2013-06-14 2016-01-20 高通股份有限公司 跟踪器辅助的图像捕获
JP2016526713A (ja) * 2013-06-14 2016-09-05 クゥアルコム・インコーポレイテッドＱｕａｌｃｏｍｍ Ｉｎｃｏｒｐｏｒａｔｅｄ トラッカ補助画像キャプチャ
US10474921B2 (en) 2013-06-14 2019-11-12 Qualcomm Incorporated Tracker assisted image capture
US11538232B2 (en) 2013-06-14 2022-12-27 Qualcomm Incorporated Tracker assisted image capture

Also Published As

Publication number Publication date
MY159291A (en) 2016-12-30

Similar Documents

Publication Publication Date Title
CN110879951B (zh) 一种运动前景检测方法及装置
CN104183127B (zh) 交通监控视频检测方法和装置
Wang Real-time moving vehicle detection with cast shadow removal in video based on conditional random field
CN103745230B (zh) 一种自适应群体异常行为分析方法
JP6679858B2 (ja) 対象の遮蔽を検出する方法と装置
WO2014136623A1 (fr) Procédé de détection et de suivi d'objets dans une séquence d'images d'une scène acquise par une caméra stationnaire
WO2013103591A1 (fr) Détection d'objet en mouvement sensible au contexte
CN105631418A (zh) 一种人数统计的方法和装置
CN100382600C (zh) 动态场景下的运动物体检测方法
CN102346854A (zh) 前景物体检测方法和设备
CN103475800B (zh) 图像序列中前景的检测方法和装置
CN106462975A (zh) 用于对象跟踪和经由背景跟踪进行分割的方法和装置
CN104200492A (zh) 基于轨迹约束的航拍视频目标自动检测跟踪方法
CN109636828A (zh) 基于视频图像的物体跟踪方法及装置
Ghasemi et al. A real-time multiple vehicle classification and tracking system with occlusion handling
EP1932117A2 (fr) Methode et appareil pour determiner un type d'image photographiee
CN106446832B (zh) 一种基于视频的实时检测行人的方法
KR101690050B1 (ko) 지능형 영상보안 시스템 및 객체 추적 방법
Denman et al. Multi-spectral fusion for surveillance systems
CN104268889A (zh) 基于特征权值自校正的车辆跟踪方法
Tang et al. Hybrid blob and particle filter tracking approach for robust object tracking
Ilao et al. Crowd estimation using region-specific HOG With SVM
WO2012173466A1 (fr) Système et procédé de comptage d'objets
CN107067411B (zh) 一种结合密集特征的Mean-shift跟踪方法
Agrafiotis et al. HDR Imaging for Enchancing People Detection and Tracking in Indoor Environments.

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12750623

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 12750623

Country of ref document: EP

Kind code of ref document: A1