WO2020107524A1 - 一种目标跟踪方法及计算设备 - Google Patents

一种目标跟踪方法及计算设备 Download PDF

Info

Publication number
WO2020107524A1
WO2020107524A1 PCT/CN2018/120049 CN2018120049W WO2020107524A1 WO 2020107524 A1 WO2020107524 A1 WO 2020107524A1 CN 2018120049 W CN2018120049 W CN 2018120049W WO 2020107524 A1 WO2020107524 A1 WO 2020107524A1
Authority
WO
WIPO (PCT)
Prior art keywords
target object
target
area
movement
image frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2018/120049
Other languages
English (en)
French (fr)
Inventor
牟晓正
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Omnivision Sensor Solution Shanghai Co Ltd
Original Assignee
Celepixel Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Celepixel Technology Co Ltd filed Critical Celepixel Technology Co Ltd
Priority to EP18941851.0A priority Critical patent/EP3889897B1/en
Publication of WO2020107524A1 publication Critical patent/WO2020107524A1/zh
Priority to US17/324,208 priority patent/US11657516B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/215—Motion-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/62—Extraction of image or video features relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/17—Terrestrial scenes taken from planes or by drones
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10032—Satellite or aerial image; Remote sensing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30212—Military
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30232—Surveillance
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30236—Traffic on road, railway or crossing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30241—Trajectory
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/44—Event detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Definitions

  • the present invention relates to the field of target tracking technology, and in particular, to a target tracking method and a computing device.
  • Target tracking has a wide range of applications in both military and civilian fields, such as aerial surveillance, satellite and spacecraft tracking, and intelligent video surveillance. Accurate tracking of targets helps to accurately understand the position of the other party's targets, and is the core technology of high-tech weapon systems and GPS systems. At the same time, with the application of assisted driving systems, how to use target tracking processing technology to protect the safety of pedestrians and vehicles is also a hot research direction.
  • the traditional optical flow-based target tracking method needs to first calculate the optical flow information from the two frames before and after the image, in this calculation process requires a lot of computing power, which is a great challenge to the real-time algorithm.
  • the traditional tracking method is easy to cause tracking loss.
  • an effective target tracking scheme is needed, which can not only reduce the computational power in optical flow calculation, but also have good robustness.
  • the present invention provides a target tracking solution to try to solve or at least alleviate at least one of the above problems.
  • a target tracking method suitable for execution in a computing device, comprising the steps of: generating an optical flow image using a series of event data from a dynamic vision sensor, the event is generated by the target object and the dynamic vision sensor Triggered by relative motion; determining the speed and direction of movement of at least one target object in the current image frame based on the optical flow image; predicting the position of the corresponding target object in the next image frame based on the direction and speed of motion; and when the predicted position When in the field of view, the corresponding target object is taken as the target to be tracked.
  • the method according to the present invention further includes the step of storing the predicted position of the target to be tracked so as to match with at least one target object detected in the next image frame. It specifically includes the steps of: separately calculating the distance between at least one target object in the next image frame and the predicted position of the target to be tracked; when the distance meets a predetermined condition, determining that the target object in the next image frame and the target to be tracked are Same goal.
  • the event data includes the coordinate position and time stamp of the triggered event
  • the step of generating the optical flow image using a series of event data from the dynamic vision sensor includes: according to the time stamp sequence Event data within a predetermined period of time is divided into a predetermined number of event segments; different pixel values are assigned to events within different event segments; and an optical flow image is generated according to the coordinate positions and pixel values of the event.
  • the step of determining the movement speed and movement direction of at least one target object in the current image frame based on the optical flow image includes: calculating the target area corresponding to each target object in the current image frame; Perform a clustering process on the pixels in each target area to obtain a first area and a second area corresponding to each target object; and take the direction from the first area to the second area as the movement direction of the corresponding target object.
  • the step of determining the movement speed and movement direction of at least one target object in the current image frame based on the optical flow image further includes: calculating the distance between the first area corresponding to the target object and the second area As the moving distance of the target object; determining the moving time of the target object based on the pixel values of pixels in the first area and the second area; and determining the moving speed of the target object according to the moving distance and the moving time.
  • the step of taking the direction from the first area to the second area as the movement direction of the corresponding target object further includes: separately determining the center pixels of the first area and the second area as the first A center and a second center; and the direction from the first center to the second center as the direction of movement of the target object.
  • the step of calculating the distance between the first area and the second area corresponding to the target object as the movement distance of the target object includes: calculating the distance from the first center to the second center as the corresponding target object Moving distance.
  • the step of determining the movement time of the target object based on the pixel values of the pixels in the first area and the second area includes: determining the unit pixel based on the pixel value corresponding to the event segment and the length of the time period The duration of the target; calculating the difference between the average value of the pixels in the first area corresponding to the target object and the average value of the pixels in the second area; and determining the movement time of the target object according to the difference and the duration of the unit pixel.
  • the method according to the present invention further includes the step of: when the predicted position is not within the field of view, the corresponding target object is no longer tracked.
  • the method according to the present invention further includes the step of: when the distance between the target object in the next image frame and the predicted position of the target to be tracked does not satisfy a predetermined condition, use the target object as a new target to be tracked.
  • a computing device including: one or more processors; and a memory; one or more programs, wherein the one or more programs are stored in the memory and configured To be executed by the one or more processors, the one or more programs include instructions for performing any of the target tracking methods described above.
  • a computer-readable storage medium storing one or more programs, the one or more programs including instructions, which when executed by a computing device, cause the computing device to perform the target tracking method as described above Any of the methods.
  • an optical flow image is generated based on each changed event data output by the DVS, and the optical flow information carried in the optical flow image is used to predict the movement direction and speed of the target object, so that Determine the target to be tracked.
  • the solution according to the present invention provides a method for matching at least one target object detected in the next image frame with the target to be tracked in the previous image frame, which not only ensures the accuracy of target tracking, but also greatly Reduced computing power.
  • FIG. 1 shows a schematic diagram of a computing device 100 according to some embodiments of the present invention
  • FIG. 2 shows a flowchart of a target tracking method 200 according to an embodiment of the present invention
  • FIG. 3 shows a schematic diagram of an optical flow image generated according to an embodiment of the present invention
  • FIG. 4 shows a schematic diagram of the movement speed and movement direction of a target object in an optical flow image according to an embodiment of the present invention
  • 5A and 5B show schematic diagrams of determining a target object in two consecutive frames of images according to an embodiment of the present invention.
  • 6A-6C show schematic diagrams of a process of matching target objects according to an embodiment of the present invention.
  • DVS Dynamic Vision Sensor
  • the sensor has a pixel unit array composed of a plurality of pixel units. Each pixel unit only responds to and records the area where the light intensity changes rapidly when it senses the light intensity change.
  • the specific composition of the dynamic vision sensor is not elaborated here. Since DVS adopts an event-triggered processing mechanism, its output is an asynchronous event data stream.
  • the event data stream is light intensity change information (such as time stamp and light intensity threshold of light intensity change) and the coordinate position of the triggered pixel unit .
  • DVS has the following advantages over traditional visual sensors: 1) The response speed of DVS is no longer limited by the traditional exposure time and frame rate, and can detect up to 10,000 frames per second rate Moving high-speed objects; 2) DVS has a larger dynamic range, and can accurately sense and output scene changes in low-light or high-exposure environments; 3) DVS consumes less power; 4) Since each pixel unit of DVS is Independent response to light intensity changes, so DVS will not be affected by motion blur.
  • a DVS-based target tracking scheme is proposed to solve the problem that the existing technology consumes a large amount of computing power and lacks robustness.
  • the target tracking scheme according to the embodiments of the present invention focuses on solving the problem of locating and matching multiple targets of interest in a multi-target tracking scenario, so as to maintain the IDs of multiple targets in the scene and record multiple targets Movement track.
  • the event data stream output by the DVS arranged in the scene is transmitted to one or more computing devices, and the computing device performs a series of processing on the event data stream, for example, generating a corresponding light according to the event data stream Streaming images, predicting the position of the detected target in the current image frame in the next image frame, matching the predicted target position with the actual detected target position, and so on, and finally implementing the embodiment according to the present invention Target tracking scheme.
  • FIG. 1 is a block diagram of an example computing device 100.
  • the computing device 100 typically includes a system memory 106 and one or more processors 104.
  • the memory bus 108 may be used for communication between the processor 104 and the system memory 106.
  • the processor 104 may be any type of processor, including but not limited to: a microprocessor ( ⁇ P), a microcontroller ( ⁇ C), a digital information processor (DSP), or any combination thereof.
  • the processor 104 may include one or more levels of cache, such as a level one cache 110 and a level two cache 112, a processor core 114, and registers 116.
  • the example processor core 114 may include an arithmetic logic unit (ALU), a floating point unit (FPU), a digital signal processing core (DSP core), or any combination thereof.
  • the example memory controller 118 may be used with the processor 104, or in some implementations, the memory controller 118 may be an internal part of the processor 104.
  • the system memory 106 may be any type of memory, including but not limited to: volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.), or any combination thereof.
  • the system memory 106 may include an operating system 120, one or more applications 122, and program data 124.
  • the application 122 may be arranged to operate with the program data 124 on the operating system.
  • the computing device 100 is configured to execute the target tracking method, and the program data 124 includes instructions for performing the target tracking method according to an embodiment of the present invention.
  • the computing device 100 may also include an interface bus 140 that facilitates communication from various interface devices (eg, output device 142, peripheral interface 144, and communication device 146) to the basic configuration 102 via the bus/interface controller 130.
  • the example output device 142 includes a graphics processing unit 148 and an audio processing unit 150. They may be configured to facilitate communication with various external devices such as displays or speakers via one or more A/V ports 152.
  • the example peripheral interface 144 may include a serial interface controller 154 and a parallel interface controller 156, which may be configured to facilitate via one or more I/O ports 158 and such as input devices (eg, keyboard, mouse, pen) , Voice input devices, image input devices) or other peripheral devices (such as printers, scanners, etc.) to communicate.
  • the example communication device 146 may include a network controller 160, which may be arranged to facilitate communication with one or more other computing devices 162 via a network communication link via one or more communication ports 164.
  • the network communication link may be an example of a communication medium.
  • Communication media can generally be embodied as computer readable instructions, data structures, program modules in a modulated data signal such as a carrier wave or other transmission mechanism, and can include any information delivery media.
  • the "modulated data signal" may be a signal whose one or more data sets or its changes can be made in the signal by encoding information.
  • the communication medium may include a wired medium such as a wired network or a dedicated line network, and various wireless media such as sound, radio frequency (RF), microwave, infrared (IR), or other wireless media.
  • RF radio frequency
  • IR infrared
  • the term computer readable media as used herein may include both storage media and communication media.
  • the computing device 100 may be implemented as a personal computer including a desktop computer and a notebook computer configuration.
  • the computing device 100 can also be implemented as part of a small-sized portable (or mobile) electronic device, such as a cellular phone, digital camera, personal digital assistant (PDA), personal media player device, wireless network browsing device , Personal head-mounted devices, application-specific devices, or hybrid devices that can include any of the above functions.
  • a small-sized portable (or mobile) electronic device such as a cellular phone, digital camera, personal digital assistant (PDA), personal media player device, wireless network browsing device , Personal head-mounted devices, application-specific devices, or hybrid devices that can include any of the above functions.
  • PDA personal digital assistant
  • hybrid devices that can include any of the above functions.
  • the embodiments of the present invention do not limit this.
  • FIG. 2 shows a flowchart of a target tracking method 200 according to an embodiment of the present invention. As shown in FIG. 2, the method 200 starts at step S210.
  • step S210 a series of event data from the dynamic vision sensor is used to generate an optical flow image.
  • the event is triggered by the relative movement of the target object in the scene and the dynamic vision sensor.
  • the event data output by the dynamic vision sensor includes the coordinate position of the triggered event and the time stamp of the triggered time.
  • a series of event data output by DVS generates an optical flow image every predetermined time period.
  • the length of the predetermined time period may be set according to experience and actual scenarios, for example, 20 milliseconds, 40 milliseconds, 60 milliseconds, etc., and the embodiment of the present invention does not limit this too much.
  • the optical flow image is generated in the following manner.
  • the event data within a predetermined period of time is divided into a predetermined number of event fragments in the order of time stamps.
  • the predetermined number is 255.
  • the event data within a predetermined time period is divided into 255 event segments in the order of time stamps. It should be noted that the embodiment of the present invention does not limit the length of each event segment obtained. In some preferred embodiments, the length of each event segment remains basically the same.
  • all events in an event segment are assigned the same gray value, and a larger gray value indicates that the timestamp when the event is triggered is closer to the present and vice versa.
  • the 255 grayscale values from 1 to 255 are assigned to the events in the 255 event segments in order of time stamp.
  • the optical flow image is generated according to the coordinate position and pixel value of the event.
  • the size of the optical flow image is set to be the same as the size of the pixel unit array in the DVS, so that each pixel in the optical flow image corresponds to each pixel unit in the DVS.
  • Write the pixel value assigned to the triggered event at each pixel of the optical flow image, and the pixel value of the pixel corresponding to the other untriggered event is 0.
  • the predetermined number mentioned above may also be any other integer value greater than 1 (not limited to 255), as long as the predetermined number of event segments is associated with the pixel value assigned to each event.
  • the predetermined number is 255, in order to keep consistent with the pixel value of the traditional grayscale image (8-bit quantization, pixel value is 0-255), so that the generated optical flow image has a good visual effect.
  • the predetermined number can be set to 1023, and the 1023 gray values from 1 to 1023 can be assigned to the events in 1023 event fragments in order according to the timestamp sequence , Wait, don't make too many restrictions here.
  • FIG. 3 shows a schematic diagram of an optical flow image generated according to an embodiment of the present invention.
  • the optical flow image shown in FIG. 3 is a grayscale image.
  • the pixels in different event segments can also be represented by different colors. For example, red indicates the pixel value in the event segment with a large time stamp, and blue indicates the time stamp Pixel values within small event fragments, embodiments of the invention are not so limited.
  • the target object contained in the optical flow image can be detected by the target detection algorithm.
  • a rectangular frame is used to surround each detected target object and return to the coordinate position of the rectangular frame as the position of the target object.
  • the embodiment of the present invention does not limit the target detection algorithm used. Any known or future known target detection algorithm can be combined with the embodiment of the present invention to implement the target tracking method 200 according to the present invention. It should be noted that the situation discussed in the embodiment of the present invention is that at least one target object is detected from the optical flow image. As shown in FIG. 3, after performing target detection processing on the optical flow image, it is obtained that the image contains two target objects, and the two target objects are surrounded by two rectangular frames, respectively.
  • the movement speed and movement direction of each target object detected in the current image frame are determined.
  • the following shows a process of determining the movement speed of a target object according to an embodiment of the present invention.
  • At least one target object in the optical flow image is detected by a target detection algorithm, and a rectangular frame representing the position of the target object is obtained, and the rectangular area surrounded by the rectangular frame is used as the target area corresponding to the target object. That is, each target object corresponds to a target area.
  • the pixels in the target area are clustered according to the size of the pixel value to obtain two types.
  • the pixel value of the pixels in one category is small, and the area corresponding to this type of pixel is used as the first area of the target object; the pixel value of the pixels in the other category is large, and the area corresponding to this type of pixel is used as the target The second area of the object.
  • the embodiment of the present invention does not limit which clustering algorithm is used for clustering.
  • the threshold is set according to experience, and the pixels in the target area are divided into two categories according to the threshold to obtain the first area and the second area corresponding to the target object. For example, taking the optical flow image as a grayscale image with a pixel value ranging from 0 to 255 as an example, the threshold is set to 127, and if the pixel value of the pixel in the target area belongs to 1 to 127, the pixel is classified into the first area; If the pixel value of the pixel in the target area belongs to 128-255, the pixel is classified into the second area. It should be noted that the embodiment of the present invention does not limit the selection of the threshold.
  • the embodiment of the present invention does not limit the method for obtaining the first area and the second area of each target object. It can be seen from the foregoing description that the present invention aims to protect that the events corresponding to the pixels in the first area are triggered first, and the events corresponding to the pixels in the second area are triggered later.
  • the center pixel of the first area and the center pixel of the second area corresponding to the target object are first determined as the first center and the second center. For example, the average of the coordinates of all pixels in the first area is calculated as the coordinates of the first center, and the average of the coordinates of all pixels in the second area is calculated as the coordinates of the second center. Then, the direction from the first center to the second center is taken as the movement direction of the target object.
  • the following shows a process of determining the movement speed of a target object according to an embodiment of the present invention.
  • the movement speed of the target object is calculated from the movement distance and movement time, so in the following process, the movement distance and movement time corresponding to the target are first determined.
  • the distance between the first area and the second area corresponding to the target object is calculated as the movement distance of the target object.
  • the distance between the first area and the second area corresponding to the target object is calculated as the movement distance of the target object.
  • the movement time of the corresponding target object is determined based on the pixel values of the pixels in the first area and the second area.
  • the exercise time is calculated using the length of the predetermined time period of the event data acquired in step S210.
  • the following shows a method of calculating exercise time.
  • the duration of the unit pixel is determined based on the pixel value corresponding to the event segment and the length of the time period. For example, in step S210, event data within a predetermined time period T is acquired and divided into 255 event segments, then it can be concluded that the length of the time period corresponding to each event segment is: T/255. As mentioned above, 255 pixel values are assigned to the 255 event segments in sequence, that is, the pixel value corresponding to the event segment is 1. In this way, the time duration (denoted as t) corresponding to the unit pixel is T/255.
  • the difference between the pixel mean value of the pixels in the first area corresponding to the target object and the pixel mean value of the pixels in the second area is calculated.
  • A1 the average of the pixel values of all pixels in the first area and record it as A1
  • A2 Subtract A1 as the difference corresponding to the target object (denoted as a).
  • the movement time of the target object is determined according to the difference a calculated in the second step and the duration t of the unit pixel calculated in the first step.
  • the time t of the unit pixel is multiplied by the difference a, which is the movement time of the target object.
  • step c) Determine the moving speed of the target object according to the moving distance of the target object obtained in step a) and the moving time of the target object obtained in step b).
  • the moving speed is divided by the moving distance divided by the moving time. The present invention will not repeat them here.
  • FIG. 4 shows a schematic diagram of the movement speed and movement direction of a target object in an optical flow image according to an embodiment of the present invention.
  • the movement speed and movement direction of the two target objects in the image frame are determined.
  • each target object is marked with a rectangular frame. The number above the rectangular frame indicates the movement speed of the target object, and the arrow points to the movement direction of the target object.
  • FIG. 4 is only for illustration, so as to more vividly explain the predicted movement direction and movement speed of the target object, and the embodiments of the present invention are not limited thereto.
  • step S230 the position of the corresponding target object in the next image frame is predicted according to the movement direction and the movement speed.
  • the position of the obtained rectangular frame is the predicted position of the target object in the next image frame.
  • "a certain distance” is determined according to the moving speed of the target object.
  • the moving speed of the target object is v.
  • step S210 event data is acquired every predetermined time period T and an optical flow image is generated. Then, the distance of moving the rectangular frame of the target object along the moving direction is v*T.
  • FIGS. 5A-5B respectively show schematic diagrams of determining a target object in consecutive multi-frame images according to an embodiment of the present invention.
  • FIG. 5A shows the current image frame 510 and detected target objects (respectively denoted as P1, P2, P3)
  • FIG. 5B shows the next image frame 520 and detected target objects (respectively denoted as S1, S2) .
  • step S240 when the predicted position is within the field of view, the corresponding target object is taken as the target to be tracked.
  • the rectangular frame drawn by a solid circle represents the target object in the current image frame 510 (denoted as P1, P2, and P3, respectively), and the rectangular frame drawn by a dotted circle represents the predicted target object (P1, P2 , P3)
  • the position of 520 in the next image frame (denoted as P1', P2', P3', respectively). It can be seen from Figure 5A that the predicted positions P1' and P2' are still in the optical flow image, so these two target objects (P1 and P2) are taken as targets to be tracked; and P3' exceeds the optical flow The range of the image, so P3 is no longer the target to be tracked.
  • the predicted position of the target to be tracked is stored so as to match the target object detected in the next image frame.
  • the predicted positions P1' and P2' corresponding to the targets P1 and P2 to be tracked are stored.
  • the stored predicted positions can be used to construct the pool to be matched.
  • the detected target object Match the target object in the matching pool: If the target object detected in the next image frame can find a matching target object in the pool to be matched, then confirm that the target object detected in the next image frame The matched target object is the same target; if the target object detected in the next image frame fails to find a matching target object in the pool to be matched, it is confirmed that the target object detected in the next image frame is A new target. For a new target object, repeatedly perform the above steps S220 and S230 to predict its position in the next image frame, and then determine whether the new target object is to be tracked, which will not be repeated here.
  • the following shows a process of matching at least one target object detected in the next image frame according to an embodiment of the present invention.
  • the distance between the predicted position of at least one target object and the target to be tracked in the next image frame is calculated respectively. That is to say, for each target object in the next image frame, the distance to the predicted position of each target to be tracked in the pool to be matched is calculated separately.
  • calculate the Euclidean distance value of the two rectangular boxes corresponding to the two target objects (such as calculating the Euclidean distance value of the center points of the two rectangular boxes) as the distance between the two target objects.
  • the calculation of is part of what is known in the art and will not be described here. It should be noted that the embodiments of the present invention are not limited to expressing the distance between target objects in Euclidean distance.
  • any method of calculating the distance can be applied to the method 200 of the present invention to implement the target tracking solution of the present invention.
  • the calculated distance meets the predetermined condition, it is determined that the target object in the next image frame and the target to be tracked are the same target.
  • the predetermined condition is, for example, that the distance is the smallest, that is, when the target object in the next image frame is matched from the pool to be matched to the predicted position with the smallest distance, the target to be tracked corresponding to the predicted position is determined to be The target object is the same target.
  • FIG. 5B is the next image frame 520 corresponding to FIG. 5A.
  • the pool to be matched contains two predicted positions P1' and P2', and the distances between S1, S2 and P1', P2' are calculated respectively, that is, the distance d1 between S1 and P1', the distance d2 between S1 and P2' , The distance d3 between S2 and P1', and the distance d4 between S2 and P2'. Then compare the size of each distance value. If d1 ⁇ d2, then S1 in FIG. 5B and P1 in FIG. 5A are the same target; if d3>d4, then S2 in FIG. 5B and P2 in FIG. 5A are the same aims.
  • the present invention does not elaborate too much on this.
  • the embodiments of the present invention aim to protect how to quickly determine the target to be tracked from the field of view based on the optical flow information, and determine the position of the target to be tracked in two frames before and after.
  • the preset condition in addition to the minimum distance, also includes that the calculated distance is greater than the preset distance value.
  • a preset distance value is set in advance, and if the calculated distance between the target object in the next image frame and the predicted position of the target to be tracked is greater than the preset distance value, it is confirmed that the calculated distance does not satisfy the predetermined condition, that is, no Match again, and directly take the target object in the next image frame as a new target to be tracked.
  • a Hungarian algorithm is used to match the target objects.
  • 6A-6C show a schematic diagram of a process of matching a target object through a Hungarian algorithm according to still other embodiments of the present invention. The matching process is further described below in conjunction with FIGS. 6A-6C.
  • step (3) Continue to match the predicted position of the target object C with the smallest distance. Assume that the distance between C and a is calculated to be the smallest at this time, but in step (1) a has been matched. , First remove the matching relationship between A and a. As shown in Figure 6C, the relationship is cancelled with " ⁇ ", and C is paired with a, and then try to match the predicted position with the second smallest distance for A. At this time, b is found, but In step (2) b has also been matched. Similarly, the matching relationship between B and b is released. In this way, A is paired with b, and B is finally paired with c.
  • step S250 when the predicted position is not within the field of view, the corresponding target object is no longer tracked. As shown in FIG. 5A, the predicted position P3' of the target object P3 has exceeded the range of the optical flow image, so the target object P3 is no longer tracked.
  • the present invention proposes a target tracking scheme based on DVS.
  • a target tracking scheme based on DVS.
  • each changed event data output via DVS carries time information
  • an optical flow image is generated based on these event data, so that optical flow information is directly obtained.
  • using the optical flow information to estimate the direction and speed of the target object to track the target object, to ensure the accuracy of tracking.
  • the traditional target tracking method is easy to produce tracking loss, but the target tracking method according to the present invention can greatly reduce the calculation power in optical flow calculation, and has better robustness Sex.
  • modules or units or components of the device in the examples disclosed herein may be arranged in the device as described in this embodiment, or alternatively may be positioned differently from the device in this example Of one or more devices.
  • the modules in the foregoing examples may be combined into one module or, in addition, may be divided into multiple sub-modules.
  • modules in the device in the embodiment can be adaptively changed and set in one or more devices different from the embodiment.
  • the modules or units or components in the embodiments may be combined into one module or unit or component, and in addition, they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and/or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstract and drawings) and any method so disclosed or All processes or units of equipment are combined. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Remote Sensing (AREA)
  • Image Analysis (AREA)

Abstract

一种目标跟踪方法和计算设备，所述方法适于在计算设备中执行，包括步骤：利用来自动态视觉传感器的一系列事件数据生成光流图像，事件由目标对象和动态视觉传感器的相对运动而触发（S210）；基于光流图像确定当前图像帧中至少一个目标对象的运动速度和运动方向（S220）；根据运动方向和运动速度，预测对应的目标对象在下一图像帧中的位置（S230）；以及当所预测的位置在视场范围内时，将对应的目标对象作为待跟踪目标（S240）。

Description

一种目标跟踪方法及计算设备 技术领域
本发明涉及目标跟踪技术领域，尤其涉及一种目标跟踪方法及计算设备。
背景技术
目标跟踪在军事和民用领域都具有广泛的应用，如空中监视、卫星和飞船跟踪以及智能视频监控等领域。精准跟踪目标有利于精确了解对方目标的位置，是高科技武器系统及GPS系统的核心技术。同时，随着辅助驾驶系统的应用，如何合理的运用目标跟踪处理技术来保护行人和车辆的安全也是当前的一个热门研究方向。
传统的基于光流的目标跟踪方法需要先通过前后两帧图像计算出光流信息，在此计算过程中需要耗费大量的算力，对算法的实时性是一个很大的挑战。另一方面，目标在快速运动的状态下，传统的跟踪方法很容易产生跟踪丢失。
鉴于此，需要一种有效的目标跟踪方案，能够既减少光流计算上的算力、又具有较好的鲁棒性。
发明内容
本发明提供了一种目标跟踪的方案，以力图解决或者至少缓解上面存在的至少一个问题。
根据本发明的一个方面，提供了一种目标跟踪方法，适于在计算设备中执行，包括步骤：利用来自动态视觉传感器的一系列事件数据生成光流图像，事件由目标对象和动态视觉传感器的相对运动而触发；基于光流图像确定当前图像帧中至少一个目标对象的运动速度和运动方向；根据运动方向和运动速度，预测对应的目标对象在下一图像帧中的位置；以及当所预测的位置在视场范围内时，将对应的目标对象作为待跟踪目标。
可选地，根据本发明的方法还包括步骤：存储所预测的待跟踪目标的位置，以便与下一图像帧中检测到的至少一个目标对象进行匹配。其具体包括步骤：分别计算下一图像帧中至少一个目标对象与所述待跟踪目标的预测位置的距离；当距离满足预定条件时，确定下一图像帧中的该目标对象与待跟踪目标是同一目标。
可选地，在根据本发明的方法中，事件数据包括被触发事件的坐标位置和时间戳，利用来 自动态视觉传感器的一系列事件数据生成光流图像的步骤包括：按照时间戳的先后顺序将预定时间段内的事件数据分为预定数目个事件片段；为不同事件片段内的事件分配不同的像素值；以及根据事件的坐标位置和像素值生成光流图像。
可选地，在根据本发明的方法中，基于光流图像确定当前图像帧中至少一个目标对象的运动速度和运动方向的步骤包括：计算当前图像帧中每个目标对象所对应的目标区域；对各目标区域内的像素分别进行聚类处理，来得到各目标对象所对应的第一区域和第二区域；以及将自第一区域指向第二区域的方向作为对应目标对象的运动方向。
可选地，在根据本发明的方法中，基于光流图像确定当前图像帧中至少一个目标对象的运动速度和运动方向的步骤还包括：计算目标对象对应的第一区域与第二区域的距离作为所述目标对象的运动距离；基于第一区域和第二区域内像素的像素值确定所述目标对象的运动时间；以及根据运动距离和运动时间来确定目标对象的运动速度。
可选地，在根据本发明的方法中，将自第一区域指向第二区域的方向作为对应目标对象的运动方向的步骤还包括：分别确定第一区域和第二区域的中心像素，作为第一中心和第二中心；以及将自第一中心指向第二中心的方向作为目标对象的运动方向。
可选地，在根据本发明的方法中，计算目标对象对应的第一区域与第二区域的距离作为目标对象的运动距离的步骤包括：计算第一中心到第二中心的距离作为对应目标对象的运动距离。
可选地，在根据本发明的方法中，基于第一区域和第二区域内像素的像素值确定目标对象的运动时间的步骤包括：基于事件片段对应的像素值和时间段长度来确定单位像素的时长；计算目标对象对应的第一区域内像素的像素均值与第二区域内像素的像素均值的差值；以及根据差值和单位像素的时长确定出目标对象的运动时间。
可选地，根据本发明的方法还包括步骤：当所预测的位置不在视场范围内时，不再跟踪对应的目标对象。
可选地，根据本发明的方法还包括步骤：当下一图像帧中的目标对象与待跟踪目标的预测位置的距离均不满足预定条件时，将该目标对象作为一个新的待跟踪目标。
根据本发明的另一方面，提供了一种计算设备，包括：一个或多个处理器；和存储器；一个或多个程序，其中所述一个或多个程序存储在所述存储器中并被配置为由所述一个或多个处理器执行，所述一个或多个程序包括用于执行如上所述目标跟踪方法中的任一方法的指令。
根据本发明的又一方面，提供了一种存储一个或多个程序的计算机可读存储介质，一个或多个程序包括指令，指令当计算设备执行时，使得计算设备执行如上所述目标跟踪方法中的任 一方法。
综上所述，根据本发明的方案，基于DVS输出的每一个变化的事件数据生成光流图像，并通过光流图像中所携带的光流信息来预测目标对象的运动方向和运动速度，以便确定待跟踪的目标。同时，根据本发明的方案提供了一种利用上一图像帧中的待跟踪目标对下一图像帧中检测到的至少一个目标对象进行匹配的方法，不仅确保了目标跟踪的准确性，还大大减少了算力。
附图说明
为了实现上述以及相关目的，本文结合下面的描述和附图来描述某些说明性方面，这些方面指示了可以实践本文所公开的原理的各种方式，并且所有方面及其等效方面旨在落入所要求保护的主题的范围内。通过结合附图阅读下面的详细描述，本公开的上述以及其它目的、特征和优势将变得更加明显。遍及本公开，相同的附图标记通常指代相同的部件或元素。
图1示出了根据本发明一些实施例的计算设备100的示意图；
图2示出了根据本发明一个实施例的目标跟踪方法200的流程图；
图3示出了根据本发明一个实施例的生成的光流图像的示意图；
图4示出了根据本发明一个实施例的光流图像中目标对象的运动速度和运动方向的示意图；
图5A和图5B示出了根据本发明一个实施例的在连续两帧图像中确定目标对象的示意图；以及
图6A-6C示出了根据本发明一个实施例的匹配目标对象的过程示意图。
具体实施方式
下面将参照附图更详细地描述本公开的示例性实施例。虽然附图中显示了本公开的示例性实施例，然而应当理解，可以以各种形式实现本公开而不应被这里阐述的实施例所限制。相反，提供这些实施例是为了能够更透彻地理解本公开，并且能够将本公开的范围完整的传达给本领域的技术人员。
近年来，动态视觉传感器(Dynamic Vision Sensor，DVS)在计算机视觉领域中得到了越来越多的关注和应用。DVS是一种模拟基于脉冲触发式神经元的人类视网膜的生物拟态视觉传感器。传感器内部具有由多个像素单元构成的像素单元阵列，其中每个像素单元只有在感应到 光强变化时，才会响应并记录光强快速变化的区域。关于动态视觉传感器的具体组成此处不做过多阐述。由于DVS采用事件触发的处理机制，故其输出是异步的事件数据流，事件数据流例如是光强变化信息(如，光强变化的时间戳和光强阈值)以及被触发像素单元的坐标位置。基于以上工作原理特性，申请人发现，相比于传统视觉传感器，DVS具有如下优势：1)DVS的响应速度不再受传统的曝光时间和帧速率限制，可以侦测到高达万帧/秒速率运动的高速物体；2)DVS具有更大的动态范围，在低光照或者高曝光环境下都能准确感应并输出场景变化；3)DVS功耗更低；4)由于DVS每个像素单元都是独立响应光强变化，因此DVS不会受运动模糊的影响。
鉴于此，根据本发明的实施方式，提出了一种基于DVS的目标跟踪方案，以解决现有技术要消耗大量算力和鲁棒性不足问题。此外，根据本发明实施方式的目标跟踪方案侧重于解决在多目标跟踪场景中，对多个感兴趣目标进行定位和匹配的问题，以便于维持场景中多个目标的ID、并记录多个目标的运动轨迹。
根据一种实施例，将布置在场景中的DVS所输出的事件数据流传送至一个或多个计算设备，由计算设备对事件数据流进行一系列处理，如，根据事件数据流生成对应的光流图像、预测出当前图像帧中检测出的目标在下一图像帧中可能会出现的位置、将预测的目标位置与实际检测到的目标位置进行匹配，等等，最终实现根据本发明实施例的目标跟踪方案。
根据本发明的一种实施方式，上述执行目标跟踪方案的一个或多个计算设备可以通过图1所示出的计算设备100来实现。图1是示例计算设备100的框图。
在基本的配置102中，计算设备100典型地包括系统存储器106和一个或者多个处理器104。存储器总线108可以用于在处理器104和系统存储器106之间的通信。
取决于期望的配置，处理器104可以是任何类型的处理器，包括但不限于：微处理器(μP)、微控制器(μC)、数字信息处理器(DSP)或者它们的任何组合。处理器104可以包括诸如一级高速缓存110和二级高速缓存112之类的一个或者多个级别的高速缓存、处理器核心114和寄存器116。示例的处理器核心114可以包括运算逻辑单元(ALU)、浮点数单元(FPU)、数字信号处理核心(DSP核心)或者它们的任何组合。示例的存储器控制器118可以与处理器104一起使用，或者在一些实现中，存储器控制器118可以是处理器104的一个内部部分。
取决于期望的配置，系统存储器106可以是任意类型的存储器，包括但不限于：易失性存储器(诸如RAM)、非易失性存储器(诸如ROM、闪存等)或者它们的任何组合。系统存储器106可以包括操作系统120、一个或者多个应用122以及程序数据124。在一些实施方式中，应用122可以布置为在操作系统上利用程序数据124进行操作。如前文所述，在一些实施例中， 计算设备100被配置为执行目标跟踪方法，程序数据124中就包含了用于执行根据本发明实施例的目标跟踪方法的指令。
计算设备100还可以包括有助于从各种接口设备(例如，输出设备142、外设接口144和通信设备146)到基本配置102经由总线/接口控制器130的通信的接口总线140。示例的输出设备142包括图形处理单元148和音频处理单元150。它们可以被配置为有助于经由一个或者多个A/V端口152与诸如显示器或者扬声器之类的各种外部设备进行通信。示例外设接口144可以包括串行接口控制器154和并行接口控制器156，它们可以被配置为有助于经由一个或者多个I/O端口158和诸如输入设备(例如，键盘、鼠标、笔、语音输入设备、图像输入设备)或者其他外设(例如打印机、扫描仪等)之类的外部设备进行通信。示例的通信设备146可以包括网络控制器160，其可以被布置为便于经由一个或者多个通信端口164与一个或者多个其他计算设备162通过网络通信链路的通信。
网络通信链路可以是通信介质的一个示例。通信介质通常可以体现为在诸如载波或者其他传输机制之类的调制数据信号中的计算机可读指令、数据结构、程序模块，并且可以包括任何信息递送介质。“调制数据信号”可以是这样的信号，它的数据集中的一个或者多个或者它的改变可以在信号中以编码信息的方式进行。作为非限制性的示例，通信介质可以包括诸如有线网络或者专线网络之类的有线介质，以及诸如声音、射频(RF)、微波、红外(IR)或者其它无线介质在内的各种无线介质。这里使用的术语计算机可读介质可以包括存储介质和通信介质二者。
计算设备100可以实现为包括桌面计算机和笔记本计算机配置的个人计算机。当然，计算设备100也可以实现为小尺寸便携(或者移动)电子设备的一部分，这些电子设备可以是诸如蜂窝电话、数码照相机、个人数字助理(PDA)、个人媒体播放器设备、无线网络浏览设备、个人头戴设备、应用专用设备、或者可以包括上面任何功能的混合设备。本发明的实施例对此均不做限制。
图2示出了根据本发明一个实施例的目标跟踪方法200的流程图。如图2所示，方法200始于步骤S210。
在步骤S210中，利用来自动态视觉传感器的一系列事件数据生成光流图像。如前文所述，事件是由场景中的目标对象和动态视觉传感器的相对运动而触发的，动态视觉传感器输出的事件数据包括被触发事件的坐标位置和被触发时刻的时间戳。
在根据本发明的实施方式中，每隔预定时间段就将DVS输出的一系列事件数据生成光流图像。预定时间段的长度可以根据经验和实际场景设置，例如20毫秒、40毫秒、60毫秒等， 本发明的实施例对此不做过多限制。可选地，采用如下方式生成光流图像。
首先，按照时间戳的先后顺序将预定时间段内的事件数据分为预定数目个事件片段。在根据本发明的一个实施例中，预定数目取255。例如，将预定时间段内的事件数据按照时间戳的先后顺序分成255个事件片段。需要说明的是，本发明实施例对分得的各事件片段的长度不做限制，在一些优选的实施例中，各事件片段的长度基本保持一致。
其次，为不同事件片段内的事件分配不同的像素值。在根据本发明的一种实施方式中，一个事件片段内的全部事件赋给同一个灰度值，且灰度值越大表示该事件被触发的时间戳离现在越近，反之越远。例如，将1～255这255个灰度值按照时间戳的先后顺序依次分配给255个事件片段内的事件。
最后，根据事件的坐标位置和像素值生成光流图像。根据一种实施例，设置光流图像的尺寸与DVS中像素单元阵列的尺寸一致，这样，光流图像中的每个像素点对应DVS中的每个像素单元。在光流图像的各像素点写入对应被触发事件所分配到的像素值，而其它没有被触发事件对应的像素点的像素值均为0。
应当指出，上文中提及的预定数目也可以是其它任何大于1的整数值(不限于255)，只要事件片段的预定数目与分配给各事件的像素值相关联即可。这里预定数目取255，是为了与传统的灰度图像的像素值(8bit量化，像素值为0～255)保持一致，以便生成的光流图像具有很好的视觉效果。本领域技术人员据此可以想到，在10bit量化的场景中，可以将预定数目设为1023，将1～1023这1023个灰度值按照时间戳的先后顺序依次分配给1023个事件片段内的事件，等等，此处不做过多限制。
图3示出了根据本发明一个实施例的生成的光流图像的示意图。图3示出的光流图像为灰度图。在实际应用中，为了可视化效果，亦可以将对应不同事件片段内的像素点用不同的颜色来表示，例如用红色表示时间戳较大的事件片段内的像素值，用蓝色表示时间戳较小的事件片段内的像素值，本发明的实施例不受限于此。
基于生成的光流图像，通过目标检测算法能够检测出光流图像中所包含的目标对象。一般地，采用矩形框包围住每个被检测出的目标对象并返回矩形框的坐标位置，作为该目标对象的位置。本发明的实施例对所采用的目标检测算法不做限制，任何已知或未来可知的目标检测算法均可与本发明的实施例相结合，以实现根据本发明的目标跟踪方法200。需要说明的是，本发明的实施例所讨论的情况是从光流图像中检测出至少一个目标对象。如图3所示，在对该光流图像进行目标检测处理后得到图像中包含2个目标对象，用2个矩形框分别包围住了这2个目标对象。
在随后的步骤S220中，基于上述光流图像，确定当前图像帧中所检测出的各目标对象的运动速度和运动方向。
以下示出根据本发明一种实施方式的确定目标对象的运动速度的过程。
1)首先，计算当前图像帧中每个目标对象所对应的目标区域。
根据一种实施例，通过目标检测算法检测出光流图像中的至少一个目标对象、并得到表征目标对象位置的矩形框，将矩形框所包围的矩形区域作为目标对象对应的目标区域。即每个目标对象对应一个目标区域。
2)其次，对各目标区域内的像素分别进行聚类处理，来得到各目标对象所对应的第一区域和第二区域。
在一种实施例中，针对每个目标区域，根据像素值的大小对该目标区域内的像素进行聚类处理，得到两类。其中一类中像素的像素值较小，将这一类像素对应的区域作为该目标对象的第一区域；另一类中像素的像素值较大，将这一类像素对应的区域作为该目标对象的第二区域。本发明的实施例对采用何种聚类算法进行聚类并不做限制。
在另一种实施例中，按照经验设置阈值，根据阈值将目标区域内的像素划分成两类，以得到目标对象所对应的第一区域和第二区域。例如，还是以光流图像是像素值范围为0～255的灰度图像为例，设阈值为127，若目标区域内像素的像素值属于1～127，则将该像素归入第一区域；若目标区域内像素的像素值属于128～255，则将该像素归入第二区域。应当指出，本发明实施例对阈值的选取不做限制。
总之，本发明实施例对采用何种方式得到各目标对象的第一区域和第二区域并不做限制。结合前文描述可知，本发明旨在保护的是，第一区域中的像素对应的事件是在先触发的，第二区域中的像素对应的事件是在后触发的。
3)最后，将自第一区域指向第二区域的方向作为对应目标对象的运动方向。
在一种实施例中，先分别确定目标对象所对应的第一区域的中心像素和第二区域的中心像素，以作为第一中心和第二中心。例如，计算第一区域中所有像素坐标的均值，作为第一中心的坐标，同样计算第二区域中所有像素坐标的均值，作为第二中心的坐标。而后，将自第一中心指向第二中心的方向作为该目标对象的运动方向。
以下示出根据本发明一种实施方式的确定目标对象的运动速度的过程。
根据一种实施例，目标对象的运动速度由运动距离和运动时间计算得到，故而在如下的过程中，先确定目标对应的运动距离和运动时间。
a)首先，针对各目标对象，计算目标对象对应的第一区域与第二区域的距离作为该目标对象的运动距离。参考前文在确定目标对象的运动方向时的描述，针对各目标对象，先确定出第一区域和第二区域的中心像素，作为第一中心和第二中心。然后，计算第一中心到第二中心的距离作为对应目标对象的运动距离。
b)接着，基于第一区域和第二区域内像素的像素值确定对应目标对象的运动时间。
根据本发明的一种实施方式，利用步骤S210中所获取的事件数据的预定时间段的长度来计算运动时间。以下示出了一种计算运动时间的方法。
第一步，基于事件片段对应的像素值和时间段长度来确定单位像素的时长。例如，在步骤S210中获取了预定时间段T内的事件数据，并将其分成255个事件片段，那么可以得出，各事件片段对应的时间段长度为：T/255。如前文所述，将255个像素值依序赋给这255个事件片段，即，事件片段对应的像素值为1。这样，单位像素对应的时长(记作t)就是T/255。
第二步，计算目标对象对应的第一区域内像素的像素均值与第二区域内像素的像素均值的差值。展开来说，针对各目标对象，先计算第一区域内所有像素的像素值的平均值，记作A1，同时计算第二区域内所有像素的像素值的平均值，记作A2，然后用A2减去A1，作为该目标对象对应的差值(记作a)。
第三步，对于每个目标对象，根据第二步计算出的差值a和第一步计算出的单位像素的时长t，确定出该目标对象的运动时间。根据一种实施例，用单位像素的时长t乘以差值a，就是目标对象的运动时间。
c)根据步骤a)求得的目标对象的运动距离和步骤b)求得的目标对象的运动时间，来确定目标对象的运动速度。可选地，用运动距离除以运动时间，即可得到运动速度。本发明对此不再赘述。
图4示出了根据本发明一个实施例的光流图像中目标对象的运动速度和运动方向的示意图。如图4所示，在对图3中的光流图像执行步骤S220的处理后，确定了图像帧中两个目标对象的运动速度和运动方向。图4中，用矩形框标注出了各目标对象，矩形框上面的数字表示该目标对象的运动速度，箭头指向了该目标对象的运动方向。应当指出，图4仅作为示意，以便于更形象地说明预测出的目标对象的运动方向和运动速度，本发明的实施例并不限于此。
随后在步骤S230中，根据运动方向和运动速度，预测对应的目标对象在下一图像帧中的位置。
在一种实施例中，将表征目标对象位置的矩形框沿该目标对象的运动方向移动一定距离后，得到的矩形框的位置，就是预测出的该目标对象在下一图像帧中的位置。其中，“一定距 离”是根据该目标对象的运动速度所确定的。例如，目标对象的运动速度为v，在步骤S210中，每隔预定时间段T获取事件数据并生成光流图像，那么，将目标对象的矩形框沿运动方向移动的距离就是v*T。
随后，针对当前图像帧中的各目标对象，根据所预测的位置来判断是否跟踪对应的目标对象。为便于说明，图5A-5B相应示出了根据本发明一个实施例的在连续多帧图像中确定目标对象的示意图。图5A示出了当前图像帧510及检测出的目标对象(分别记作P1、P2、P3)，图5B示出了下一图像帧520及检测出的目标对象(分别记作S1、S2)。
在步骤S240中，当所预测的位置在视场范围内时，将对应的目标对象作为待跟踪目标。
如图5A所示，用实线圈出的矩形框表示当前图像帧510中的目标对象(分别记作P1、P2、P3)，用虚线圈出的矩形框表示所预测的目标对象(P1、P2、P3)在下一图像帧中520的位置(分别记作P1’、P2’、P3’)。从图5A中可以看出，预测的位置P1’和P2’还在光流图像的画面之中，故将这两个目标对象(P1和P2)作为待跟踪目标；而P3’超出了光流图像的范围，故不再将P3作为待跟踪目标。
根据本发明的一些实施方式，在确认了待跟踪目标后，存储所预测的待跟踪目标的位置，以便与下一图像帧中检测到的目标对象进行匹配。以图5A为例，存储待跟踪目标P1和P2对应的预测位置P1’和P2’。当然，可以利用所存储的预测位置构建待匹配池。
根据一种实施例，当利用下一预定时间段内的事件数据生成光流图像(记作下一图像帧)并检测出下一图像帧中的目标对象时，将检测出的目标对象与待匹配池中的目标对象进行匹配：若下一图像帧中检测到的目标对象在待匹配池中能够找到与之匹配的目标对象，则确认该从下一图像帧中检测到的目标对象与所匹配到的目标对象为同一目标；若下一图像帧中检测到的目标对象在待匹配池中未能找到与之匹配的目标对象，则确认该从下一图像帧中检测到的目标对象是一个新的目标对象。对于新的目标对象，重复执行上述步骤S220和步骤S230，以预测出其在再下一图像帧中的位置，而后判断该新的目标对象是否要作为待跟踪目标，此处不再赘述。
以下示出根据本发明一个实施例的对下一图像帧中检测到的至少一个目标对象进行匹配的过程。
首先，分别计算下一图像帧中至少一个目标对象与待跟踪目标的预测位置的距离。也就是说，对于下一图像帧中的每个目标对象，分别计算其与待匹配池中各待跟踪目标的预测位置的距离。一般地，计算两个目标对象所对应的两个矩形框的欧式距离值(如计算两个矩形框的中心点的欧氏距离值)作为这两个目标对象之间的距离，关于欧氏距离的计算，属于本领域已知 内容，此处不再展开描述。需要说明的是，本发明实施例并不限于以欧氏距离来表示目标对象之间的距离，任何计算距离的方式均可以应用到本发明的方法200中，以实现本发明的目标跟踪方案。其次，当计算出的距离满足预定条件时，确定下一图像帧中的该目标对象与待跟踪目标是同一目标。预定条件例如是距离最小，即，当下一图像帧中的目标对象从待匹配池中匹配到与其距离最小的预测位置时，就确定该预测位置对应的待跟踪目标与下一图像帧中的这个目标对象是同一个目标。
还是以图5A示出的光流图像为例，图5B是图5A对应的下一图像帧520。经目标检测算法处理后从图5B中检测出两个目标对象S1和S2。此时，待匹配池中包含两个预测位置P1’和P2’，分别计算S1、S2与P1’、P2’的距离，即，计算S1与P1’的距离d1、S1和P2’的距离d2、S2与P1’的距离d3、S2和P2’的距离d4。而后比较各距离值的大小，若d1＜d2，则确定图5B中的S1与图5A中的P1为同一目标；若d3＞d4，则确定图5B中的S2与图5A中的P2为同一目标。
此后，就可以明确各目标对象的ID、记录各目标对象的运动轨迹等等，得到相应的跟踪结果。本发明对此不做过多阐述，本发明实施例旨在保护如何基于光流信息从视场中快速确定出待跟踪目标、并确定待跟踪目标在前后两帧中的位置。
在另一些实施例中，预设条件除了距离最小外，还包括计算出的距离大于预设距离值。例如，预先设置一个预设距离值，若计算出的下一图像帧中的目标对象与待跟踪目标的预测位置的距离大于预设距离值，则确认计算出的距离不满足预定条件，即不再进行匹配，直接将下一图像帧中的该目标对象作为一个新的待跟踪目标。
根据再一些实施例，为尽可能多地匹配到对应的目标对象，基于两个目标对象之间的距离，采用匈牙利算法(Hungarian Algorithm)来对目标对象进行匹配。图6A-6C示出了根据本发明再一些实施例的通过匈牙利算法匹配目标对象的过程示意图。以下结合图6A-6C，对匹配过程进行进一步说明。
(1)假设从下一图像帧中检测到三个目标对象(记作A、B、C)，待匹配池中有三个待跟踪目标的预测位置(记作a、b、c)。首先从目标对象A开始，分别计算A到待匹配池中的a、b、c的距离。如前文所述，取距离最小的预测位置对应的目标对象作为匹配到的目标对象。假设此处匹配到的是a，如图6A所示。
(2)采用同(1)的方法，从待匹配池中为目标对象B匹配距离最小的预测位置。假设此处为B匹配到的是b，如图6B所示。
(3)继续为目标对象C匹配距离最小的预测位置，假设此时计算到C与a的距离最小， 但是在步骤(1)中a已被匹配，为了让C也能够匹配到合适的预测位置，先将A与a的匹配关系解除，如图6C中用“×”表示关系解除，并使C与a配对，而后尝试重新为A匹配距离第二小的预测位置，此时找到b，但是在步骤(2)中b也已被匹配。同样地，解除B与b的匹配关系。这样，就将A与b配对，B最终与c配对。
鉴于匈牙利算法已经是已知内容，关于利用匈牙利算法进行匹配的更详细描述此处不再进行过多阐述。需要说明的是，此处仅作为示例，给出了一些对目标对象进行匹配的方式，本发明并不限制于此。在实际场景中，本领域技术人员可以依据上述所阐述的实施例设置相应的匹配方式。
在步骤S250中，当所预测的位置不在视场范围内时，不再跟踪对应的目标对象。如图5A中，目标对象P3的预测位置P3’已经超出了光流图像的范围，故不再跟踪目标对象P3。
针对现有技术中算力大且鲁棒性不足的问题，本发明提出了一种基于DVS的目标跟踪方案。根据本发明的方案，考虑到经DVS输出的每一个变化的事件数据都带有时间信息，因此基于这些事件数据生成光流图像，这样就直接得到了光流信息。同时，通过光流信息估计目标对象的运动方向和运动速度，来对目标对象进行跟踪，确保跟踪的准确性。此外，当目标对象在视场中快速运动时，传统的目标跟踪方法很容易产生跟踪丢失，但根据本发明的目标跟踪方法可以大大减少在光流计算上的算力，具有较好的鲁棒性。
在此处所提供的说明书中，说明了大量具体细节。然而，能够理解，本发明的实施例可以在没有这些具体细节的情况下被实践。在一些实例中，并未详细示出公知的方法、结构和技术，以便不模糊对本说明书的理解。
类似地，应当理解，为了精简本公开并帮助理解各个发明方面中的一个或多个，在上面对本发明的示例性实施例的描述中，本发明的各个特征有时被一起分组到单个实施例、图、或者对其的描述中。然而，并不应将该公开的方法解释成反映如下意图：即所要求保护的本发明要求比在每个权利要求中所明确记载的特征更多特征。更确切地说，如下面的权利要求书所反映的那样，发明方面在于少于前面公开的单个实施例的所有特征。因此，遵循具体实施方式的权利要求书由此明确地并入该具体实施方式，其中每个权利要求本身都作为本发明的单独实施例。
本领域那些技术人员应当理解在本文所公开的示例中的设备的模块或单元或组件可以布置在如该实施例中所描述的设备中，或者可替换地可以定位在与该示例中的设备不同的一个或多个设备中。前述示例中的模块可以组合为一个模块或者此外可以分成多个子模块。
本领域那些技术人员可以理解，可以对实施例中的设备中的模块进行自适应性地改变并且 把它们设置在与该实施例不同的一个或多个设备中。可以把实施例中的模块或单元或组件组合成一个模块或单元或组件，以及此外可以把它们分成多个子模块或子单元或子组件。除了这样的特征和/或过程或者单元中的至少一些是相互排斥之外，可以采用任何组合对本说明书(包括伴随的权利要求、摘要和附图)中公开的所有特征以及如此公开的任何方法或者设备的所有过程或单元进行组合。除非另外明确陈述，本说明书(包括伴随的权利要求、摘要和附图)中公开的每个特征可以由提供相同、等同或相似目的的替代特征来代替。
此外，本领域的技术人员能够理解，尽管在此所述的一些实施例包括其它实施例中所包括的某些特征而不是其它特征，但是不同实施例的特征的组合意味着处于本发明的范围之内并且形成不同的实施例。例如，在下面的权利要求书中，所要求保护的实施例的任意之一都可以以任意的组合方式来使用。
此外，所述实施例中的一些在此被描述成可以由计算机系统的处理器或者由执行所述功能的其它装置实施的方法或方法元素的组合。因此，具有用于实施所述方法或方法元素的必要指令的处理器形成用于实施该方法或方法元素的装置。此外，装置实施例的在此所述的元素是如下装置的例子：该装置用于实施由为了实施该发明的目的的元素所执行的功能。
如在此所使用的那样，除非另行规定，使用序数词“第一”、“第二”、“第三”等等来描述普通对象仅仅表示涉及类似对象的不同实例，并且并不意图暗示这样被描述的对象必须具有时间上、空间上、排序方面或者以任意其它方式的给定顺序。
尽管根据有限数量的实施例描述了本发明，但是受益于上面的描述，本技术领域内的技术人员明白，在由此描述的本发明的范围内，可以设想其它实施例。此外，应当注意，本说明书中使用的语言主要是为了可读性和教导的目的而选择的，而不是为了解释或者限定本发明的主题而选择的。因此，在不偏离所附权利要求书的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。对于本发明的范围，对本发明所做的公开是说明性的，而非限制性的，本发明的范围由所附权利要求书限定。

Claims (13)

  1. 一种目标跟踪方法，适于在计算设备中执行，所述方法包括步骤：
    利用来自动态视觉传感器的一系列事件数据生成光流图像，所述事件由目标对象和动态视觉传感器的相对运动而触发；
    基于所述光流图像确定当前图像帧中至少一个目标对象的运动速度和运动方向；
    根据所述运动方向和运动速度，预测对应的目标对象在下一图像帧中的位置；以及
    当所预测的位置在视场范围内时，将对应的目标对象作为待跟踪目标。
  2. 如权利要求1所述的方法，其中，在将对应的目标对象作为待跟踪目标之后，还包括步骤：
    存储所预测的待跟踪目标的位置，以便与下一图像帧中检测到的至少一个目标对象进行匹配。
  3. 如权利要求2所述的方法，其中，所述存储所预测的待跟踪目标的位置，以便与下一图像帧中检测到的至少一个目标对象进行匹配的步骤包括：
    分别计算下一图像帧中至少一个目标对象与所述待跟踪目标的预测位置的距离；
    当所述距离满足预定条件时，确定下一图像帧中的该目标对象与待跟踪目标是同一目标。
  4. 如权利要求1-3中任一项所述的方法，其中，所述事件数据包括被触发事件的坐标位置和时间戳，
    所述利用来自动态视觉传感器的一系列事件数据生成光流图像的步骤包括：
    按照时间戳的先后顺序将预定时间段内的事件数据分为预定数目个事件片段；
    为不同事件片段内的事件分配不同的像素值；以及
    根据所述事件的坐标位置和像素值生成光流图像。
  5. 如权利要求1-4中任一项所述的方法，其中，所述基于光流图像确定当前图像帧中至少一个目标对象的运动速度和运动方向的步骤包括：
    计算当前图像帧中每个目标对象所对应的目标区域；
    对各目标区域内的像素分别进行聚类处理，来得到各目标对象所对应的第一区域和第二区域；以及
    将自第一区域指向第二区域的方向作为对应目标对象的运动方向。
  6. 如权利要求5所述的方法，其中，所述基于光流图像确定当前图像帧中至少一个目标对象的运动速度和运动方向的步骤还包括：
    计算目标对象对应的第一区域与第二区域的距离作为所述目标对象的运动距离；
    基于所述第一区域和第二区域内像素的像素值确定所述目标对象的运动时间；以及
    根据所述运动距离和运动时间来确定所述目标对象的运动速度。
  7. 如权利要求5或6所述的方法，其中，所述将自第一区域指向第二区域的方向作为对应目标对象的运动方向的步骤还包括：
    分别确定第一区域和第二区域的中心像素，作为第一中心和第二中心；以及
    将自第一中心指向第二中心的方向作为所述目标对象的运动方向。
  8. 如权利要求7所述的方法，其中，所述计算目标对象对应的第一区域与第二区域的距离作为目标对象的运动距离的步骤包括：
    计算第一中心到第二中心的距离作为对应目标对象的运动距离。
  9. 如权利要求6-8中任一项所述的方法，其中，所述基于第一区域和第二区域内像素的像素值确定所述目标对象的运动时间的步骤包括：
    基于事件片段对应的像素值和时间段长度来确定单位像素的时长；
    计算所述目标对象对应的第一区域内像素的像素均值与第二区域内像素的像素均值的差值；以及
    根据所述差值和单位像素的时长确定出所述目标对象的运动时间。
  10. 如权利要求1-9中任一项所述的方法，还包括步骤：
    当所预测的位置不在视场范围内时，不再跟踪对应的目标对象。
  11. 如权利要求3-10中任一项所述的方法，还包括步骤：
    当下一图像帧中的目标对象与待跟踪目标的预测位置的距离均不满足预定条件时，将该目标对象作为一个新的待跟踪目标。
  12. 一种计算设备，包括：
    一个或多个处理器；和
    存储器；
    一个或多个程序，其中所述一个或多个程序存储在所述存储器中并被配置为由所述一个或多个处理器执行，所述一个或多个程序包括用于执行根据权利要求1-11所述方法中的任一 方法的指令。
  13. 一种存储一个或多个程序的计算机可读存储介质，所述一个或多个程序包括指令，所述指令当计算设备执行时，使得所述计算设备执行根据权利要求1-11所述的方法中的任一方法。
PCT/CN2018/120049 2018-11-27 2018-12-10 一种目标跟踪方法及计算设备 Ceased WO2020107524A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP18941851.0A EP3889897B1 (en) 2018-11-27 2018-12-10 Target tracking method and computing device
US17/324,208 US11657516B2 (en) 2018-11-27 2021-05-19 Target tracking method and computing device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811428965.5A CN109544590B (zh) 2018-11-27 2018-11-27 一种目标跟踪方法及计算设备
CN201811428965.5 2018-11-27

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/324,208 Continuation US11657516B2 (en) 2018-11-27 2021-05-19 Target tracking method and computing device

Publications (1)

Publication Number Publication Date
WO2020107524A1 true WO2020107524A1 (zh) 2020-06-04

Family

ID=65851710

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/120049 Ceased WO2020107524A1 (zh) 2018-11-27 2018-12-10 一种目标跟踪方法及计算设备

Country Status (4)

Country Link
US (1) US11657516B2 (zh)
EP (1) EP3889897B1 (zh)
CN (1) CN109544590B (zh)
WO (1) WO2020107524A1 (zh)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112037259A (zh) * 2020-08-27 2020-12-04 北京极智嘉科技有限公司 动态目标的跟踪系统及方法
CN113344978A (zh) * 2021-06-29 2021-09-03 北京搜狗科技发展有限公司 一种目标跟踪方法、装置和用于目标跟踪的装置
CN113344980A (zh) * 2021-06-29 2021-09-03 北京搜狗科技发展有限公司 一种目标跟踪方法、装置和用于目标跟踪的装置
US11218629B2 (en) * 2019-02-23 2022-01-04 Pegatron Corporation Tracking system and method thereof
CN114842047A (zh) * 2022-03-29 2022-08-02 武汉大学 基于运动先验的孪生网络卫星视频目标跟踪方法
CN114842045A (zh) * 2022-04-01 2022-08-02 深圳市九天睿芯科技有限公司 目标跟踪方法及装置
CN115760945A (zh) * 2022-11-23 2023-03-07 智慧互通科技股份有限公司 一种基于目标检测的稳定目标检测框的获取方法及系统
US20230368579A1 (en) * 2021-02-02 2023-11-16 Omnivision Sensor Solution (Shanghai) Co., Ltd Method and system for identifying sliding direction of hand
CN117808847A (zh) * 2024-02-29 2024-04-02 中国科学院光电技术研究所 融合仿生动态视觉的空间非合作目标特征跟踪方法

Families Citing this family (34)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3690736A1 (en) * 2019-01-30 2020-08-05 Prophesee Method of processing information from an event-based sensor
WO2021021839A1 (en) 2019-07-30 2021-02-04 Qsinx Management Llc Tracking using pixel sensors of an event camera
CN110533699B (zh) * 2019-07-30 2024-05-24 平安科技（深圳）有限公司 基于光流法的像素变化的动态多帧测速方法
EP4020963B1 (en) * 2019-08-20 2026-04-29 Sony Interactive Entertainment Inc. Image processing device, image processing method, and program
CN110533693B (zh) * 2019-08-29 2023-02-07 北京精英路通科技有限公司 一种目标跟踪方法及目标跟踪装置
DE102019214198A1 (de) * 2019-09-18 2021-03-18 Robert Bosch Gmbh Ereignisbasierte Erkennung und Verfolgung von Objekten
CN110909592B (zh) * 2019-10-11 2020-12-18 重庆特斯联智慧科技股份有限公司 一种基于多尺度特征量的目标跟踪方法和系统
CN111083354A (zh) * 2019-11-27 2020-04-28 维沃移动通信有限公司 一种视频录制方法及电子设备
KR102311798B1 (ko) * 2019-12-12 2021-10-08 포항공과대학교 산학협력단 다중 객체 추적 방법 및 장치
KR102702094B1 (ko) * 2019-12-17 2024-09-04 삼성전자주식회사 다이나믹 비전 센서 시스템
CN111951313B (zh) * 2020-08-06 2024-04-26 北京灵汐科技有限公司 图像配准方法、装置、设备及介质
CN112037258B (zh) * 2020-08-25 2024-03-08 广州视源电子科技股份有限公司 目标跟踪方法、装置、设备和存储介质
CN112153301B (zh) * 2020-09-27 2021-07-27 维沃移动通信有限公司 拍摄方法和电子设备
CN112966556B (zh) * 2021-02-02 2022-06-10 豪威芯仑传感器(上海)有限公司 一种运动物体检测方法及系统
CN112949440A (zh) * 2021-02-22 2021-06-11 豪威芯仑传感器(上海)有限公司 一种提取行人的步态特征的方法、步态识别方法及系统
CN112949512B (zh) * 2021-03-08 2022-07-08 豪威芯仑传感器(上海)有限公司 一种动态手势识别方法、手势交互方法及交互系统
CN113066104B (zh) * 2021-03-25 2024-04-19 三星（中国）半导体有限公司 角点检测方法和角点检测装置
CN113163254B (zh) * 2021-04-06 2023-03-28 广州津虹网络传媒有限公司 直播图像处理方法、装置及电子设备
CN113096151B (zh) * 2021-04-07 2022-08-09 地平线征程(杭州)人工智能科技有限公司 对目标的运动信息进行检测的方法和装置、设备和介质
CN115526814A (zh) * 2021-06-25 2022-12-27 华为技术有限公司 图像预测的方法以及装置
US12262128B2 (en) * 2021-07-21 2025-03-25 Sony Group Corporation Image sensor control circuitry and image sensor control method
KR20230070091A (ko) * 2021-11-12 2023-05-22 삼성전자주식회사 다이나믹 비전 센서 시스템의 동작 방법
CN114155278A (zh) * 2021-11-26 2022-03-08 浙江商汤科技开发有限公司 目标跟踪及相关模型的训练方法和相关装置、设备、介质
EP4411637A4 (en) * 2021-12-14 2024-11-13 Huawei Technologies Co., Ltd. METHOD AND APPARATUS FOR TARGET TRACKING
CN114390209A (zh) * 2022-02-23 2022-04-22 维沃移动通信有限公司 拍摄方法、拍摄装置、电子设备和可读存储介质
CN114612507B (zh) * 2022-02-28 2025-04-11 天津大学 基于脉冲序列式图像传感器的高速目标追踪方法
CN114723776B (zh) * 2022-04-01 2024-04-19 深圳市九天睿芯科技有限公司 目标跟踪方法及装置
CN114842055B (zh) * 2022-04-20 2025-07-18 拓元(广州)智慧科技有限公司 一种基于光流的货柜商品跟踪方法
CN114777764B (zh) * 2022-04-20 2023-06-30 中国科学院光电技术研究所 一种基于事件相机的高动态星敏感器星点提取方法
CN115760912A (zh) * 2022-11-03 2023-03-07 北京灵汐科技有限公司 运动目标跟踪方法、装置、设备及计算机可读存储介质
KR20250109698A (ko) * 2022-11-11 2025-07-17 소니 세미컨덕터 솔루션즈 가부시키가이샤 정보 처리 장치
CN116295348B (zh) * 2023-03-20 2025-12-05 杭州电子科技大学 基于sc-lm的分散式gm-lphd多目标跟踪方法
CN119953596B (zh) * 2024-12-31 2026-01-30 上海航天控制技术研究所 一种基于光学搜索组件的空间目标跟踪方法
CN119991747A (zh) * 2025-01-26 2025-05-13 上海市公安局普陀分局 基于路网行为规律预测的多监控设备主动跟踪方法及装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102800106A (zh) * 2012-06-29 2012-11-28 刘怡光 基于光流场估计的自适应均值漂移目标跟踪方法
CN102881024A (zh) * 2012-08-24 2013-01-16 南京航空航天大学 一种基于tld的视频目标跟踪方法
CN104517125A (zh) * 2014-12-26 2015-04-15 湖南天冠电子信息技术有限公司 高速物体的图像实时跟踪方法与系统
CN105578034A (zh) * 2015-12-10 2016-05-11 深圳市道通智能航空技术有限公司 一种对目标进行跟踪拍摄的控制方法、控制装置及系统
CN105761277A (zh) * 2016-02-01 2016-07-13 西安理工大学 一种基于光流的运动目标跟踪方法

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100630088B1 (ko) * 2004-12-28 2006-09-27 삼성전자주식회사 옵티컬 플로우를 이용한 차량 감시 장치 및 방법
CN104680122B (zh) * 2013-11-29 2019-03-19 展讯通信(天津)有限公司 一种基于肤色检测的跟踪方法及装置
CN105844659B (zh) * 2015-01-14 2019-04-26 北京三星通信技术研究有限公司 运动部件的跟踪方法和装置
US9734587B2 (en) * 2015-09-30 2017-08-15 Apple Inc. Long term object tracker
KR102530219B1 (ko) * 2015-10-30 2023-05-09 삼성전자주식회사 동작 인식의 오류를 검출하는 방법 및 장치
US9934557B2 (en) * 2016-03-22 2018-04-03 Samsung Electronics Co., Ltd Method and apparatus of image representation and processing for dynamic vision sensor
CN106371459B (zh) * 2016-08-31 2018-01-30 京东方科技集团股份有限公司 目标跟踪方法及装置
CN108073929B (zh) * 2016-11-15 2023-11-24 北京三星通信技术研究有限公司 基于动态视觉传感器的物体检测方法及设备
US10510160B2 (en) * 2016-12-20 2019-12-17 Samsung Electronics Co., Ltd. Multiscale weighted matching and sensor fusion for dynamic vision sensor tracking
CN107862704B (zh) * 2017-11-06 2021-05-11 广东工业大学 一种目标跟踪方法、系统及其使用的云台相机
US10477355B1 (en) * 2017-12-13 2019-11-12 Amazon Technologies, Inc. System for locating users
CN108764078B (zh) * 2018-05-15 2019-08-02 上海芯仑光电科技有限公司 一种事件数据流的处理方法及计算设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102800106A (zh) * 2012-06-29 2012-11-28 刘怡光 基于光流场估计的自适应均值漂移目标跟踪方法
CN102881024A (zh) * 2012-08-24 2013-01-16 南京航空航天大学 一种基于tld的视频目标跟踪方法
CN104517125A (zh) * 2014-12-26 2015-04-15 湖南天冠电子信息技术有限公司 高速物体的图像实时跟踪方法与系统
CN105578034A (zh) * 2015-12-10 2016-05-11 深圳市道通智能航空技术有限公司 一种对目标进行跟踪拍摄的控制方法、控制装置及系统
CN105761277A (zh) * 2016-02-01 2016-07-13 西安理工大学 一种基于光流的运动目标跟踪方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3889897A4 *

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11218629B2 (en) * 2019-02-23 2022-01-04 Pegatron Corporation Tracking system and method thereof
CN112037259A (zh) * 2020-08-27 2020-12-04 北京极智嘉科技有限公司 动态目标的跟踪系统及方法
US20230368579A1 (en) * 2021-02-02 2023-11-16 Omnivision Sensor Solution (Shanghai) Co., Ltd Method and system for identifying sliding direction of hand
US12548374B2 (en) * 2021-02-02 2026-02-10 Omnivision Sensor Solution (Shanghai) Co., Ltd Method and system for identifying sliding direction of hand
CN113344978A (zh) * 2021-06-29 2021-09-03 北京搜狗科技发展有限公司 一种目标跟踪方法、装置和用于目标跟踪的装置
CN113344980A (zh) * 2021-06-29 2021-09-03 北京搜狗科技发展有限公司 一种目标跟踪方法、装置和用于目标跟踪的装置
CN114842047A (zh) * 2022-03-29 2022-08-02 武汉大学 基于运动先验的孪生网络卫星视频目标跟踪方法
CN114842045A (zh) * 2022-04-01 2022-08-02 深圳市九天睿芯科技有限公司 目标跟踪方法及装置
CN114842045B (zh) * 2022-04-01 2024-04-16 深圳市九天睿芯科技有限公司 目标跟踪方法及装置
CN115760945A (zh) * 2022-11-23 2023-03-07 智慧互通科技股份有限公司 一种基于目标检测的稳定目标检测框的获取方法及系统
CN117808847A (zh) * 2024-02-29 2024-04-02 中国科学院光电技术研究所 融合仿生动态视觉的空间非合作目标特征跟踪方法

Also Published As

Publication number Publication date
EP3889897A1 (en) 2021-10-06
CN109544590B (zh) 2020-05-15
CN109544590A (zh) 2019-03-29
US11657516B2 (en) 2023-05-23
EP3889897A4 (en) 2022-08-24
US20210279890A1 (en) 2021-09-09
EP3889897B1 (en) 2025-01-15

Similar Documents

Publication Publication Date Title
WO2020107524A1 (zh) 一种目标跟踪方法及计算设备
US11561322B2 (en) Use of thermopiles to detect human location
EP3779768B1 (en) Event data stream processing method and computing device
CN110807410B (zh) 关键点定位方法、装置、电子设备和存储介质
CN108961318B (zh) 一种数据处理方法及计算设备
US11831973B2 (en) Camera setting adjustment based on event mapping
US10242447B2 (en) Video processing system and method for deformation and occlusion resistant object tracking in video content
CN109716392B (zh) 一种光流计算方法及计算设备
CN113099122A (zh) 拍摄方法、装置、设备和存储介质
TWI783572B (zh) 物件追蹤方法及物件追蹤裝置
US10432853B2 (en) Image processing for automatic detection of focus area
Nigam et al. Real-time object detection on edge devices using mobile neural networks
US11577147B1 (en) Detection of intentional contact between object and body part of player in sport
CN115546248A (zh) 事件数据处理方法、装置和系统
CN117132515A (zh) 一种图像处理方法及电子设备
CN114967907B (zh) 识别方法及电子设备
CN107742115A (zh) 一种基于视频监控的运动目标物检测追踪的方法及系统
Akash et al. Drone Detection Using YOLOv5 With GhostNet Backbone and Adaptive Fourier Neural Operator (AFNO 2D)
Li et al. Small target detection algorithm based on the fusion attention mechanism and multi-layer convolution
CN119255045B (zh) 一种视频插帧方法、装置、电子设备及存储介质
CN114302057B (zh) 图像参数确定方法、装置、电子设备及存储介质
US20230162375A1 (en) Method and system for improving target detection performance through dynamic learning
Wang et al. Bio-Inspired Event Cameras for Robust Edge System in Challenging Environments
Niu et al. An improved real-time soccer object detection and shooting method based on prior position information
CN118043859A (zh) 高效视觉感知

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18941851

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2018941851

Country of ref document: EP

Effective date: 20210628