WO2019052320A1 - 监控方法、装置、系统、电子设备及计算机可读存储介质 - Google Patents

监控方法、装置、系统、电子设备及计算机可读存储介质 Download PDF

Info

Publication number
WO2019052320A1
WO2019052320A1 PCT/CN2018/101671 CN2018101671W WO2019052320A1 WO 2019052320 A1 WO2019052320 A1 WO 2019052320A1 CN 2018101671 W CN2018101671 W CN 2018101671W WO 2019052320 A1 WO2019052320 A1 WO 2019052320A1
Authority
WO
WIPO (PCT)
Prior art keywords
visible light
image
detected
light image
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2018/101671
Other languages
English (en)
French (fr)
Inventor
浦世亮
聂鑫鑫
范蒙
俞海
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Hikvision Digital Technology Co Ltd
Original Assignee
Hangzhou Hikvision Digital Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Hikvision Digital Technology Co Ltd filed Critical Hangzhou Hikvision Digital Technology Co Ltd
Priority to US16/647,348 priority Critical patent/US11275952B2/en
Priority to EP18855830.8A priority patent/EP3683716A4/en
Publication of WO2019052320A1 publication Critical patent/WO2019052320A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/37—Determination of transform parameters for the alignment of images, i.e. image registration using transform domain methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G06V10/12—Details of acquisition arrangements; Constructional details thereof
    • G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
    • G06V10/143—Sensing or illuminating at different wavelengths
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Definitions

  • the present application relates to the field of monitoring technologies, and in particular, to monitoring methods, devices, systems, electronic devices, and computer readable storage media.
  • the intelligent monitoring device can automatically detect pre-set targets, such as people and vehicles, from the acquired images, and determine the location area of the target in the captured image. For example, if the preset target is a person, the intelligent monitoring device can automatically detect the location area where the person is located in the captured image after the image is acquired.
  • pre-set targets such as people and vehicles
  • the target detection method is used to perform the target detection on the visible light image, and the position area of the target in the acquired image is obtained, and the intelligent monitoring is completed.
  • the image quality of the visible light image collected by the intelligent monitoring device is poor, and the detection result obtained by the target detection method for the visible light image is inaccurate, resulting in the target determined in the visible light image.
  • the location area is inaccurate and the monitoring effect is poor.
  • the purpose of the embodiments of the present application is to provide a monitoring method, device, system, electronic device, and computer readable storage medium, so as to improve the accuracy of determining a location area of a target in a visible light image, and ensuring a monitoring effect.
  • the specific technical solutions are as follows:
  • an embodiment of the present application provides a monitoring method, the method comprising: obtaining a non-visible light image, and obtaining an image based on a visible light signal acquired during an acquisition period of the non-visible light image. a target image; detecting whether there is a target to be detected in the non-visible image; if present, determining, according to the first location region of the non-visible image in the non-visible image, determining the target to be detected in the visible image A two-position area, which in turn completes target monitoring based on the visible light image, wherein the visible light image is an image determined based on the target image.
  • the visible light image is the target image; and determining, according to the first location area of the to-be-detected target in the non-visible image, the second target to be detected in the visible light image.
  • the step of locating the first location area of the target to be detected in the non-visible image with the visible light image as a reference, and obtaining the first location area after registration, and A position area corresponding to the registered first position area in the visible light image is used as a second position area of the target to be detected in the visible light image.
  • the step of performing position registration on the first location area of the non-visible image in the non-visible image by using the visible light image as a reference to obtain the first location area after registration including Positioning the first location area of the object to be detected in the non-visible image according to the visible light image as a reference to obtain a first position area after registration:
  • the visible light image is: an image obtained by performing position registration on the target image based on the non-visible light image; and the image according to the object to be detected is in the non-visible image
  • Determining, in the first location area, the second location area of the object to be detected in the visible light image comprising: setting a first position in the visible light image and the target to be detected in the non-visible image
  • the same location area of the area is determined as: the second location area of the object to be detected in the visible light image.
  • the method further includes marking the object to be detected in the second location area in the visible light image.
  • the method further includes: according to the determined location of each target to be detected The two-position area marks the type of the object to be detected in the visible light image.
  • the method further includes: identifying the to-be-identified attribute information of the object to be detected in the non-visible image, and obtaining a recognition result; After the step of marking the object to be detected in the second location area in the visible light image, the method further includes marking, in the visible light image, a target according to the second location area The recognition result of the target to be detected.
  • the method further includes: according to the non-visible light image, Determining the fill state parameter; performing non-visible fill light according to the fill state parameter.
  • an embodiment of the present application provides a monitoring apparatus, where the apparatus includes: an obtaining module, configured to obtain a non-visible light image, and a target obtained by imaging a visible light signal collected during an acquisition period of the non-visible light image. a detection module, configured to detect whether there is a target to be detected in the non-visible image; and a first determining module, configured to: when the detection result of the detection module is yes, according to the object to be detected a first location area in the non-visible image, determining a second location area of the object to be detected in the visible light image, thereby completing target monitoring based on the visible light image, wherein the visible light image is based on the target image Determined image.
  • the visible light image is the target image; the first determining module is configured to: use the visible light image as a reference, and first place the target to be detected in the non-visible image Positioning the area to obtain a first position area after registration, and using a position area corresponding to the first position area after registration in the visible light image as a second position area of the target to be detected in the visible light image .
  • the first determining module is configured to: perform position registration on the first location area of the to-be-detected target in the non-visible image according to the following formula: a first position area after registration, and a position area corresponding to the registered first position area in the visible light image as a second position area of the target to be detected in the visible light image:
  • the visible light image is: an image obtained by performing position registration on the target image based on the non-visible light image; the first determining module is specifically configured to: display the visible light image And a location area that is the same as the first location area of the to-be-detected target in the non-visible light image is determined as: a second location area of the to-be-detected target in the visible light image.
  • the device further includes: a marking module, configured to mark the second location area in the visible light image after determining the second location area of the object to be detected in the visible light image Describe the detection target.
  • a marking module configured to mark the second location area in the visible light image after determining the second location area of the object to be detected in the visible light image Describe the detection target.
  • the marking module is further configured to: after the second location area in the visible light image marks the target to be detected, according to the determined second location area where each target to be detected is located Marking the type of the object to be detected in the visible light image.
  • the device further includes: an identification module, configured to identify, in the non-visible image, the to-be-identified attribute information of the object to be detected when the object to be detected is detected in the non-visible light image,
  • the marking module is further configured to mark in the visible light image according to the second location area after the second location area in the visible light image marks the object to be detected A recognition result for the object to be detected is obtained.
  • the device further includes: a second determining module, configured to obtain a non-visible image, and image the obtained target image based on the visible light signal collected during the collection period of the non-visible image, according to the non- The visible light image determines a fill light state parameter, and the fill light module is configured to perform non-visible light fill according to the fill light state parameter.
  • a second determining module configured to obtain a non-visible image, and image the obtained target image based on the visible light signal collected during the collection period of the non-visible image, according to the non-
  • the visible light image determines a fill light state parameter
  • the fill light module is configured to perform non-visible light fill according to the fill light state parameter.
  • the embodiment of the present application provides a monitoring system, where the system includes a target image capturing device, a non-visible image capturing device, and an image processing device; wherein: the non-visible image capturing device is used for a non-visible image.
  • the target image capturing device is configured to obtain a target image based on a visible light signal collected during an acquisition period of the non-visible image, and obtain the target image a target image is sent to the image processing device;
  • the image processing device is configured to receive the target image sent by the target image capturing device and the non-visible light image sent by the non-visible light image collecting device; Whether there is a target to be detected in the non-visible image; if present, determining a second location area of the object to be detected in the visible light image according to the first location area of the object to be detected in the non-visible image, and then completing Target monitoring based on the visible light image, wherein the visible light Like the image of the target based on the determined image.
  • the visible light image is the target image; the image processing device determines, according to the first location area of the to-be-detected target in the non-visible image, the target to be detected in the visible light image.
  • the second location area is specifically configured to perform position registration on the first location area of the non-visible image in the non-visible image by using the visible light image as a reference to obtain a first position area after registration. And a position area corresponding to the registered first position area in the visible light image is used as a second position area of the visible image to be detected.
  • the image processing device performs position registration on the first location area of the non-visible image in the non-visible image by using the visible light image as a reference to obtain a first position area after registration.
  • the first position area of the object to be detected in the non-visible image is subjected to position registration according to the visible light image, and the first position area after registration is obtained:
  • the visible light image is: an image obtained by performing position registration on the target image based on the non-visible light image; and the image processing device is in the non-visible light according to the target to be detected.
  • a first location area in the image determining a second location area of the object to be detected in the visible light image, specifically: the first in the visible light image and the target to be detected in the non-visible image
  • a location area having the same location area is determined as: a second location area of the object to be detected in the visible light image.
  • the image processing device is further configured to: mark the object to be detected in the second location area in the visible light image.
  • the image processing device is further configured to: mark, according to the determined second location area where each target to be detected is located, the type of the target to be detected in the visible light image.
  • the image processing device is further configured to: when the target to be detected is detected in the non-visible light image, identify the to-be-identified attribute information of the object to be detected in the non-visible light image, and obtain a recognition result. And after marking the object to be detected in the second location area in the visible light image, marking a recognition result for the object to be detected in the visible light image according to the second location area.
  • the system further includes a non-visible light fill lamp
  • the non-visible light image capturing device is further configured to: after acquiring the non-visible light image, determine a fill light state parameter according to the non-visible light image; The visible light fill light performs non-visible fill light according to the fill light state parameter.
  • an embodiment of the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete communication with each other through a communication bus; and the memory is configured to store The computer program; the processor, when used to execute a program stored on the memory, implements the method steps described in any of the above monitoring methods.
  • the embodiment of the present application provides a computer readable storage medium, where the computer readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of the foregoing monitoring methods is implemented. step.
  • an embodiment of the present application provides a computer program product, when executed on a computer, causing a computer to perform the monitoring method step of any of the foregoing embodiments.
  • a visible light image is first obtained, and a target image obtained by imaging the visible light signal collected in the collection period of the non-visible image; and then detecting whether the target to be detected exists in the non-visible image; if present, And determining, according to the first location area of the target to be detected in the non-visible image, a second location area of the target to be detected in the visible light image, thereby completing target monitoring based on the visible light image.
  • the solution provided by the embodiment of the present application determines the target detection result of the visible light image corresponding to the non-visible light image by using the detection result obtained by performing target detection on the non-visible light image; the non-visible light signal is subjected to illumination in the environment.
  • the effect is smaller than that of the visible light signal.
  • the image quality of the non-visible image is high. Therefore, the accuracy of the detection result obtained by performing the target detection on the non-visible image is high, and the target image based on the visible image is ensured.
  • the detection result also has high accuracy, which ensures the effect of intelligent monitoring, and makes the effect of intelligent monitoring better.
  • FIG. 1 is a schematic flowchart diagram of a monitoring method according to an embodiment of the present application.
  • FIG. 2 is a schematic diagram of performing ISP processing (Image Signal Processing) on a target image according to an embodiment of the present application.
  • ISP processing Image Signal Processing
  • FIG. 3 is a schematic diagram of performing ISP processing on a non-visible image according to an embodiment of the present application.
  • FIG. 4 is a schematic flowchart diagram of a monitoring method according to another embodiment of the present application.
  • FIG. 5 is a schematic diagram of a process of performing ISP processing on a target image according to an embodiment of the present application.
  • FIG. 6 is a schematic diagram of a process of performing ISP processing on a non-visible image according to an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a process of performing target detection on a non-visible light image according to an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of a monitoring apparatus according to an embodiment of the present application.
  • FIG. 9 is a schematic structural diagram of a monitoring apparatus according to another embodiment of the present application.
  • FIG. 10 is a schematic structural diagram of a monitoring system according to an embodiment of the present application.
  • FIG. 11 is a schematic structural diagram of a monitoring system according to another embodiment of the present application.
  • FIG. 12 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
  • a visible light image is an image obtained by imaging a visible light signal.
  • a non-visible image is an image obtained by imaging with a non-visible light signal.
  • non-visible light is an electromagnetic wave that is invisible to the human eye, including infrared light, ultraviolet light, etc.
  • infrared light is usually referred to as Near Infrared (NIR)
  • near-infrared light is between visible light (VIS) and mid-infrared light.
  • NIR Near Infrared
  • MIR electromagnetic wave between
  • the non-visible light may be directly infrared light, and the corresponding non-visible light image is an infrared image; the non-visible light may also be near-infrared light in the infrared light, and correspondingly, the non-visible light image is a near-infrared image.
  • Target detection refers to detecting a target to be detected, such as a person or a car, from the image by using the target detection method. During the target detection process, the area of the object to be detected in the image and the target to be detected can be determined. kind. Common target detection methods include: target detection methods based on deep learning, target detection methods using image segmentation techniques, and target detection methods using feature matching techniques.
  • a monitoring method provided by an embodiment of the present application can be applied to a monitoring device capable of collecting a visible light image and a non-visible light image, such as a common spherical camera (ball machine) or the like, or an image processing for processing the image returned by the camera.
  • a monitoring device capable of collecting a visible light image and a non-visible light image, such as a common spherical camera (ball machine) or the like, or an image processing for processing the image returned by the camera.
  • the device is reasonable.
  • the embodiment of the present application does not limit the specific form of the foregoing monitoring device.
  • FIG. 1 is a schematic flowchart of a monitoring method, where a monitoring method provided by an embodiment of the present application includes: S101: obtaining a non-visible image, and obtaining an image based on visible light signals collected during an acquisition period of the non-visible image. Target image.
  • the target image in step S101 is obtained by imaging the visible light signal collected during the acquisition period of the non-visible light image, and the non-visible light image has a one-to-one correspondence with the target image.
  • the non-visible light image and the target image may be: images obtained for the same scene. Further, the non-visible image and the target image are obtained for the same scene in the same exposure period.
  • the step S101 may include: receiving a target image and a non-visible image returned by another device such as a camera, thereby obtaining the target image and the non-visible image.
  • the monitoring device is an image processing device
  • the image processing device is communicatively coupled to a camera, and the camera can simultaneously acquire the non-visible image and the corresponding target image, and send the collected non-visible image and the target image to the image.
  • the image processing device receives the obtained non-visible light image and the target image.
  • the monitoring device may be a device having an imaging function, and the monitoring device is provided with an imaging component, and the imaging component collects a non-visible signal and a visible light signal.
  • the non-visible light image is imaged by using the acquired non-visible light signal, and the captured visible light signal is used to image and generate a target image.
  • the imaging unit includes a camera lens, a beam splitting unit, a visible light sensor, and a non-visible light sensor.
  • the beam splitting unit may be a half-reflex lens, which can transmit visible light and reflect non-visible light.
  • the specific product form of the semi-reverse lens may be, but not limited to, a beam splitting prism; those skilled in the art can understand that the beam splitting prism can allow visible light to pass through, cut off or reflect non-visible light, that is, through prism reflection.
  • the incident light of the surface is filtered out of the non-visible band (the filtered non-visible light can be reflected), and the visible light is passed; in addition, for the dichroic prism, the range of visible light that can pass through can adjust the thickness of the prism coating
  • the embodiment of the present application does not limit the control mode for adjusting the thickness of the prism coating. Any related control method that can adjust the thickness of the prism coating can be applied to the embodiment of the present application.
  • the incident light captured by the camera lens is separated into a visible light signal and a non-visible light signal by the spectroscopic unit, and the visible light sensor uses the visible light signal to image and generate the target image.
  • the non-visible light sensor uses the non-visible light signal to image the non-visible light. image.
  • the camera component includes two cameras, one of which is used to collect visible light signals, and uses the collected visible light signals to image and generate the target image, and another camera is used to collect non-visible signals, and The non-visible light image is generated by imaging the acquired non-visible light signal.
  • an ISP Image Signal Processor
  • the ISP processing performed on the original target image obtained by imaging may include: a sensor correction including black level and dead point correction; a color restoration process including white balance correction, color correction, and gamma correction; Interpolation and color space conversion processing of RGB (one color standard) to YUV (one color standard) processing; and image enhancement processing including Tone mapping, noise reduction, sharpening, and fog.
  • the embodiment of the present application does not limit the specific execution sequence of the ISP processing performed by the original target image obtained by imaging.
  • the obtained non-visible light image can be subjected to ISP processing, that is, the non-visible light image obtained in the above step S101 is the non-visible light image processed by the ISP.
  • the ISP processing performed on the original non-visible image obtained by imaging may include: image correction processing including black level, dead point correction, and gamma correction; and including Tone mapping, noise reduction, sharpening, and transparency. Image enhancement processing of fog.
  • the embodiment of the present application does not limit the specific execution sequence of the ISP processing performed on the original non-visible image obtained by imaging.
  • the ISP processing used for the target image obtained by the direct imaging and the non-visible image can be implemented by referring to any related ISP processing technology, and the embodiment of the present application will not be described in detail herein. .
  • the non-visible light signal collected by the monitoring device may include an environment. Non-visible light signals in incident light, as well as additional complementary non-visible light signals.
  • the monitoring device may further include a non-visible fill light
  • the fill light state parameter such as the fill light intensity of the non-visible fill light may be fixedly set in advance.
  • the illumination intensity may change at any time.
  • the above method may further include Step a and step b: Step a: Determine the fill state parameter according to the non-visible image.
  • the monitoring device may determine the fill light state parameter according to the image information of the non-visible light image, where the image information may be a signal to noise ratio of the non-visible light image, an exposure parameter, etc., and the fill light state parameter may be an open/close state of the non-visible fill light. And the fill light intensity, etc.
  • the specific implementation of the step a can be referred to the related art.
  • the embodiments of the present application are not described herein, and are merely exemplified by the following examples.
  • the monitoring device first calculates the signal-to-noise ratio of the non-visible image, and when the calculated signal-to-noise ratio is greater than the threshold T1, determining that the fill light intensity is 0, that is, turning off the non-visible fill light; When the calculated signal-to-noise ratio is less than the threshold T2, it is determined that the new fill light intensity is the sum of the original fill light intensity and the preset positive value K1; when the calculated signal-to-noise ratio is in the range of [T1, T2], the compensation is determined.
  • the light intensity is constant, that is, the new fill light intensity is the original fill light intensity, where T1>T2.
  • the original supplemental light intensity may refer to a fill light intensity of the non-visible fill light when the non-visible light image is obtained.
  • Step b performing non-visible fill light according to the determined fill light state parameter.
  • the non-visible fill light performs the non-visible fill light according to the determined fill light state parameter, so that the monitoring device collects the non-visible light thereafter.
  • the signal includes a non-visible signal in the ambient incident light and an additional non-visible signal supplemented by the non-visible light.
  • the fill light state parameter may be the number of LED light heads, the infrared fill light intensity and the angle, and the infrared LED fill light according to the determined number of LED light heads, infrared fill light Infrared fill light for intensity and angle.
  • the monitoring device when the monitoring device performs non-visible light filling according to the determined fill light state parameter, it does not affect the image quality of the currently obtained non-visible light image, but an image of the non-visible light image obtained after the current time. Quality has an impact.
  • S102 Detect whether there is a target to be detected in the non-visible image.
  • the target to be detected here is a target to be detected, for example, the type of the object to be detected is preset to be a person and a car, and step S102 needs to detect whether there is a person or a person in the obtained non-visible image. If the type of the target to be detected is only a person in advance, step S102 needs to detect whether there is a person in the obtained non-visible image.
  • the monitoring device may complete step S102 by using any feasible technology in the related art.
  • the target detection method it is detected whether there is a target to be detected in the non-visible image.
  • the target detection method described herein may be: a target detection method based on a deep learning, a target detection method using an image segmentation technique, or a target detection method using a feature matching technique, etc., and the embodiment of the present application does not limit the target detection method.
  • a specific implementation, and a specific implementation manner of the target detection method reference may be made to the related technology. The specific implementation manner of the target detection method is not described in detail in the embodiment of the present application.
  • step S103 is performed: determining a second location area of the target to be detected in the visible light image according to the first location area of the target to be detected in the non-visible image. And completing target monitoring based on the visible light image, wherein the visible light image is an image determined based on the target image.
  • the target to be detected is in the non-visible image at this time.
  • the location area has been determined.
  • the monitoring device obtains the visible light image of the second location area, that is, completes the target detection in the visible light image, and thus can complete the visible light based on the visible light. Target monitoring of images.
  • the non-visible light image and the target image are obtained for the same scene in the same exposure period.
  • the target image has a target to be detected
  • the visible light The image is an image determined according to the target image
  • the visible light image includes a target feature of the target image, that is, when there is a target to be detected in the target image, the target to be detected exists in the visible light image.
  • the visible light image is an image determined based on the target image, and the visible light image includes a target feature of the target image. For example, when a person image is included in the target image, that is, the target image includes a feature of a person, and the visible light image includes a feature of a person in the target image.
  • the flow of the monitoring method provided by the embodiment of the present application points to “end”, indicating that there is no target to be detected in the visible light image at this time.
  • the monitoring device obtains an image obtained by imaging the visible light signal collected during the acquisition period of the non-visible image, and there is no target to be detected in the image; and, when the detection result of step S102 is YES, the monitoring is performed.
  • the device obtains the above visible light image defining the second location area.
  • the visible light image is an image determined based on the target image, and in one case, the visible light image is the target image.
  • determining, according to the first location area of the target to be detected in the non-visible image, determining that the target to be detected is in the visible image The step of the two-position area may be: first finding a position area in the visible light image that is the same as the position of the first position area in the non-visible light image, and then directly determining the found position area as: the target to be detected is in the A second location area in the visible light image.
  • a first location area is found in the non-visible image, which is a rectangular area with pixel coordinates of (x2, y2) to (x2+100, y2+200); then in the visible image, ie, the target image, A rectangular area in which the pixel coordinates are also (x2, y2) to (x2+100, y2+200) is determined as the second positional area in the visible light image.
  • the device itself due to the design and manufacturing process of the device itself, there is a certain positional deviation between the actually obtained target image and the non-visible image; in order to eliminate the positional deviation between the target image and the non-visible image, the guarantee is ensured.
  • the above-mentioned visible light image and the non-visible light image have a pixel-level alignment, thereby ensuring the accuracy of the determined second positional region.
  • step S103 when the visible light image is the target image, the foregoing
  • the step of determining, in the first location area of the non-visible image, the second location area of the target to be detected in the visible light image may include: using the visible light image as a reference, the target to be detected is The first position area in the non-visible image is subjected to position registration, and the first position area after registration is obtained, and the position area corresponding to the registered first position area in the visible light image is used as the visible image to be detected.
  • the second location area of the target may include: using the visible light image as a reference, the target to be detected is The first position area in the non-visible image is subjected to position registration, and the first position area after registration is obtained, and the position area corresponding to the registered first position area in the visible light image is used as the visible image to be detected.
  • the second location area of the target may include: using the visible light image as a reference, the target to be detected is The first position area in the non-visible image is subjected to position registration
  • the entire non-visible light image may be registered with the visible light image as a reference, but in order to reduce the calculation amount, only the first target region in the non-visible light image of the detection target may be subjected to position registration.
  • the specific implementation of the registration may employ, for example, related techniques such as position registration based on horizontal mirroring, affine transformation, or projection transformation.
  • the step of performing the position registration on the first location area of the non-visible image to be detected by the visible light image as a reference, and obtaining the first location area after the registration may include Taking the visible light image as a reference, the first target area in the non-visible light image is subjected to position registration according to the following formula, and the first position area after registration is obtained:
  • the above affine transformation matrix can be obtained by offline calibration using a related affine transformation technique.
  • a first location area is found in the non-visible image, which is a rectangular area with pixel coordinates of (x2, y2) to (x2+100, y2+200), and then used for position registration using the above formula.
  • the first positional area after the quasi-visible image is a rectangular area in which the pixel coordinates are (x2+2, y2+4) to (x2+102, y2+204); then in the visible light image, the pixel coordinates are also ( The rectangular region of x2+2, y2+4) to (x2+102, y2+204) is determined as the second location region in the visible light image.
  • first location area after the registration is only for illustrative purposes, and does not constitute a limitation on the embodiments of the present application. Those skilled in the art can obtain the first after registration based on the related technology. Location area.
  • the visible light image is an image determined based on the target image, specifically, in another case, The visible light image may be an image obtained by positionally registering the target image based on the non-visible light image.
  • a specific implementation manner of performing position registration on the target image based on the non-visible light image and a method for performing position registration on the first position region of the non-visible image to be detected based on the visible light image a person skilled in the art can perform the position registration on the first position region of the non-visible image to be detected based on the visible light image, and complete the target image by using the non-visible image as a reference. The operation of the location registration is not repeated here.
  • the step of determining the second location area of the target to be detected in the visible light image according to the first location area of the target to be detected in the non-visible image may include: The location area of the first location area in the non-visible light image to be detected is determined to be the second location area of the target to be detected in the visible light image.
  • the first location area is found in the non-visible image, for example, it is a rectangular area with pixel coordinates of (x2, y2) to (x2+200, y2+100); then in the visible image, that is, the target image In the quasi-image, a rectangular region whose pixel coordinates are also (x2, y2) to (x2+200, y2+100) is determined as the second position region in the visible light image.
  • the monitoring device directly performs target detection on the visible light image obtained by imaging the visible light signal by the target detection method, and obtains a position region in the visible light image obtained by direct imaging, and performs intelligent monitoring, but at low illumination.
  • the image quality of the visible light image obtained by direct imaging is poor, and the image quality of the visible light image obtained by direct imaging cannot be greatly increased even by the means of image enhancement or the increase of the local exposure amount, so the related art cannot obtain the visible light obtained by direct imaging.
  • the area where the target to be detected is located is accurately determined, and even if the visible light image obtained by direct imaging cannot be accurately detected, whether or not there is a target to be detected.
  • the solution provided by the embodiment of the present application determines the target detection result of the visible light image corresponding to the non-visible light image by using the detection result obtained by performing target detection on the non-visible light image; in the case of low illumination, The image quality of the non-visible image is high, and the detection result obtained by the target detection of the non-visible image is high, and the target detection result of the visible light image is also ensured with high accuracy, thereby ensuring the effect of intelligent monitoring.
  • the monitoring device may further include a non-visible fill light, and the complementary light generated by the non-visible fill light ensures sufficient non-visible light, so that the obtained non-visible image has high signal-to-noise, ensuring non-visible light. The accuracy of the target detection results obtained by the image.
  • the visible light image of the second location area is determined.
  • the monitoring device may directly determine the visible light image of the second location area.
  • the visible light image with the target detection result is stored locally or transmitted to other devices for viewing.
  • the position of the object to be detected may be clearly displayed in the image, as shown in FIG. 4, in the non-visible image according to the object to be detected.
  • the first location area, after determining the second location area of the target to be detected in the visible light image (S103), the method may further include: S104: marking the target to be detected in the second location area in the visible light image.
  • step S104 There are many specific implementations of the step S104.
  • the specific implementation of the step S104 is not limited in the embodiment of the present application.
  • the visible light image marked with the second location area is displayed, the human eye can see the marked second.
  • the location area is fine.
  • the edge area of the second position area in the visible light image is highlighted, or the determined second position area is marked with a red wire frame in the visible light image.
  • the specific marking method of step S104 can utilize any related marking technology, and the embodiment of the present application will not be described in detail herein.
  • the preset target to be detected may be at least two types, for example, the target to be detected includes a person and a car, in order to enable the human eye to clearly see in the visible image after being displayed
  • the monitoring method may further include: each of the determined The second location area where the target to be detected is located, and the type of the target to be detected is marked in the visible light image.
  • the type of the target to be detected is preset, and when the monitoring device detects the first location area where the target to be detected is located in the non-visible light image, the detection and detection are performed according to the target to be detected of the corresponding type, so the monitoring device It is completely possible to know the type of each target to be detected. For example, the monitoring device detects the first location area where the tree is located in the non-visible image. Obviously, when a first location area is found, the type of the target to be detected corresponding to the first location area is a tree.
  • the type of the object to be detected corresponding to each second location area in the visible light image is the same as the type of the object to be detected corresponding to the first location area corresponding to the second location area.
  • the type of the target to be detected corresponding to the first location area is a car
  • the type of the target to be detected corresponding to the second location area corresponding to the first location area is also car.
  • the type of the target to be detected is marked in the visible light image according to the determined second location area where the target to be detected is determined, which may be: at the determined target to be detected.
  • the type of the object to be detected is marked, for example, the second position area identified by the red wire frame is displayed in the visible light image, and the type is displayed in each red wire frame area. Marking the "person”; or: marking the type of the object to be detected, for example, visible outside the second position area where each target to be detected is determined, not exceeding a preset pixel distance
  • the determined second position area is marked with a red wire frame in the image, and the category mark "person" is also displayed at a distance of 20 pixels from the lower right corner of each red wire frame area.
  • the specific marking method of the types in the above steps can also utilize the related technology.
  • the type corresponding to each second location area can be clearly displayed. More details.
  • the method when detecting the object to be detected in the non-visible light image, the method may further include: identifying, in the non-visible image, the object to be detected to be identified The attribute information is used to obtain the recognition result.
  • the attribute information to be identified may be understood as: pre-set attribute information that needs to be identified.
  • the attribute information to be identified may be a license plate number of the vehicle, a type of the vehicle, etc.
  • the target to be detected is The type is a person
  • the attribute information to be identified may be a person's gender or the like.
  • the embodiment of the present application does not limit the specific type of the attribute to be identified, and the attribute information to be identified may be set according to actual needs.
  • the method for identifying the to-be-identified attribute information of the object to be detected in the non-visible light image may be an image recognition method feasible in the related art, for example, an image recognition method based on training and classification, an image recognition method based on a neural network, and based on The image recognition method of the wavelet moment, etc., the embodiment of the present application does not limit the identification method used herein.
  • the method may further include: marking, in the visible light image, the target to be detected according to the second location area Identification result.
  • the identification result of the object to be detected is marked in the visible light image according to the second location area, and may be marked in the second location area where the determined target to be detected is located.
  • the type of the target and/or the result of the recognition for example, the type of the target to be detected is a vehicle, and the attribute information to be identified is the vehicle type and the license plate number, and the visible visible image is marked with a red wire frame to mark the determined second In the location area, each red wireframe area also shows the identified vehicle type and license plate number: truck, Beijing Axxxxx.
  • the type of the target to be detected is a person. If the attribute information to be identified is the gender of the person, the determined second position area is marked with a red wire frame in the displayed visible light image, and the recognition is displayed at a distance of 20 pixels from the lower right corner of each red wire frame area. The result is "male.”
  • the specific marking manner of the identification result in the above step can also utilize the related technology, as long as the visible light image is displayed, the recognition result corresponding to each second location area can be clearly displayed, and the embodiment of the present application is This will not be described in detail.
  • the embodiment of the present application marks the target in the visible image.
  • the recognition result of the detection target is such that the visible light image can also carry an accurate image recognition result.
  • only the second location area and the type of the target to be detected may be marked, or only the second location area and the recognition result for the target to be detected may be marked, and the second may be simultaneously marked.
  • the location area, the type of target to be detected, and the recognition result for the target to be detected may be marked, or only the second location area and the recognition result for the target to be detected.
  • the marked visible light image may be displayed locally, so that the human eye can clearly see the marked visible light image in the displayed visible light image.
  • the marked visible light image described herein is: a second position area is marked, and at the same time, the type of the object to be detected and/or the visible light image for the recognition result of the object to be detected is marked.
  • the monitoring device transmits the marked visible light image to other storage devices, and can be read and displayed by the other display device from the storage device.
  • the visible light image in this case, in order to facilitate the transmission, the monitoring device may perform a video encoding operation on the marked visible light image, and then transmit the encoded visible light image to the storage device.
  • the monitoring device is a ball machine, and the ball machine is connected to a back end console, and the ball machine includes a camera lens, a beam splitting unit, a visible light sensor, and a non-visible light sensor, and the preset type of the target to be detected For people and cars.
  • the incident light captured by the camera lens is separated into a visible light signal and a non-visible light signal by a spectroscopic unit, and the visible light sensor uses the visible light signal to generate an original visible light image.
  • the non-visible light sensor uses the non-visible light signal acquired during the same exposure period to generate the original image. Non-visible image.
  • the ball machine performs the ISP processing on the obtained original visible light image as shown in FIG. 5: black level, dead point correction, white balance correction, color interpolation, gamma correction, color correction, RGB for the original visible image.
  • the ball machine also performs the ISP processing as shown in FIG. 6 on the obtained original non-visible light image: that is, blackening the original non-visible light image in turn Level, dead point correction, gamma correction, noise reduction, and sharpening are processed to obtain a processed non-visible image.
  • the ball machine uses the depth detection-based object detection method according to the following steps: 1. extracting the candidate region, 2. calculating the feature, 3, classifying, 4, finely correcting the candidate region, and treating the non-processed non-
  • the visible light image is subjected to target detection, and it is determined that there is a target vehicle to be detected in the processed non-visible light image, and the first positional region in the processed non-visible light image of the target vehicle is pixel coordinates (x3, y3) to ( A rectangular area of x3+50, y2+80).
  • the ball machine identifies the license plate number of the target to be detected in the non-visible image through the image recognition method based on training and classification, and obtains the recognition result: Zhejiang Axxxxx.
  • the ball machine further performs position registration on the first positional region in the processed non-visible light image of the target vehicle to be detected based on the visible light image after the processing, and obtains the first position region after registration.
  • the pixel coordinates are (x3+2, y3+4) to (x3+52, y3+84), and the pixel coordinates are also (x3) in the processed visible light image.
  • the rectangular area of +2, y3+4) to (x3+52, y3+84) is determined as the second position area in the visible light image.
  • the ball machine marks the second position area in the processed visible light image, and marks the type of the vehicle to be detected and the recognition result in the second position area, and then the ball machine encodes the marked visible light image and sends it to the a back-end console receives and displays the marked visible light image.
  • the determined second position area is marked by a red wire frame, and the red wire frame is displayed for detection.
  • the type of target "car”, and the above identification result Zhejiang Axxxxx.
  • the embodiment of the present application further provides a monitoring apparatus.
  • the apparatus includes: an obtaining module 110, configured to obtain a non-visible image, and based on the The visible light signal collected during the acquisition period of the non-visible image, and the target image obtained by the imaging; the detecting module 120 is configured to detect whether there is a target to be detected in the non-visible image; the first determining module 130 is configured to be in the detecting module Determining, in the case of YES, the second location area of the object to be detected in the visible light image according to the first location area of the object to be detected in the non-visible image, thereby completing Target monitoring of a visible light image, wherein the visible light image is an image determined based on the target image.
  • the visible light image may be the target image.
  • the first determining module 130 may be specifically configured to use the visible light image as a reference. Performing position registration on the first location area in the non-visible light image of the object to be detected, obtaining a first position area after registration, and corresponding to the first position area after registration in the visible light image The location area serves as a second location area of the target to be detected in the visible light image.
  • the first determining module 130 may be specifically configured to: in the non-visible image, the target to be detected according to the following formula, based on the visible light image; Performing position registration in the first position area, obtaining the first position area after registration, and using the position area corresponding to the registered first position area in the visible light image as the target to be detected in the visible light image Second location area:
  • the visible light image may be: an image obtained by performing position registration on the target image based on the non-visible light image; in this case, The first determining module 130 may be specifically configured to: determine, in the visible light image, a location area that is the same as the first location area of the to-be-detected target in the non-visible light image: the target to be detected is a second location area in the visible light image.
  • the apparatus may further include: a marking module 140, after determining that the target to be detected is in the second location area in the visible light image, The second location area in the visible light image marks the object to be detected.
  • the marking module 140 may be further configured to: after the second location area in the visible light image marks the target to be detected, according to the determined A second location area where each object to be detected is located, and the type of the object to be detected is marked in the visible light image.
  • the device may further include: an identifying module, configured to identify, in the non-visible image, when detecting a target to be detected in the non-visible image Determining the attribute information to be identified of the detection target, and obtaining a recognition result; correspondingly, the marking module is further configured to: after the second location area in the visible light image marks the object to be detected, according to the A two-position area in which the recognition result for the object to be detected is marked.
  • an identifying module configured to identify, in the non-visible image, when detecting a target to be detected in the non-visible image Determining the attribute information to be identified of the detection target, and obtaining a recognition result
  • the marking module is further configured to: after the second location area in the visible light image marks the object to be detected, according to the A two-position area in which the recognition result for the object to be detected is marked.
  • the apparatus may further include: a second determining module, configured to obtain a non-visible light image, and based on a visible light signal collected during an acquisition period of the non-visible light image After imaging the obtained target image, determining a fill light state parameter according to the non-visible light image; and a fill light module for performing non-visible light fill according to the fill light state parameter.
  • a second determining module configured to obtain a non-visible light image, and based on a visible light signal collected during an acquisition period of the non-visible light image After imaging the obtained target image, determining a fill light state parameter according to the non-visible light image
  • a fill light module for performing non-visible light fill according to the fill light state parameter.
  • the solution provided by the embodiment of the present application determines the target detection result of the visible light image corresponding to the non-visible light image by using the detection result obtained by performing target detection on the non-visible light image; in the case of low illumination, The image quality of the non-visible image is high, and the accuracy of the detection result obtained by the target detection of the non-visible image is high, which ensures that the target detection result of the visible light image also has high accuracy, thereby ensuring the effect of intelligent monitoring.
  • the monitoring device may further include a non-visible fill light, and the complementary light generated by the non-visible fill light ensures sufficient non-visible light, so that the obtained non-visible image has high signal-to-noise, ensuring non-visible light. The accuracy of the target detection results obtained by the image.
  • the embodiment of the present application further provides a monitoring system, as shown in FIG. 10 , the system includes a target image capturing device, a non-visible image capturing device, and an image processing device; a non-visible light image collecting device for transmitting a non-visible light image and transmitting the collected non-visible light image to the image processing device; the target image capturing device for collecting based on an acquisition period of the non-visible light image The visible light signal, the image is obtained, and the target image is sent to the image processing device; the image processing device is configured to receive the target image and the non-visible image sent by the target image capturing device Collecting, by the acquisition device, the non-visible light image; performing target detection on the obtained non-visible light image for the target to be detected, and obtaining a detection result of the first location area including the target to be detected in the non-visible light image; a first location area, determining a second location area of the target to be detected in the visible light image Wherein the visible light image
  • the target image capturing device and the non-visible image capturing device may be two physical devices; of course, the target image capturing device and the non-visible image capturing device may also be located on the same physical entity, for example, The target image acquisition device and the non-visible image acquisition device may be two sub-devices included in a common dome.
  • the operations performed by the respective devices may refer to the corresponding operations performed by the monitoring device in the foregoing method embodiment, and details are not described herein again.
  • the visible light image may be the target image; in this case, the image processing device is first according to the target to be detected in the non-visible image. And determining, by the location area, the second location area of the to-be-detected target in the visible light image, where the first location of the to-be-detected target in the non-visible image is determined by using the visible light image as a reference Positioning the area to obtain a first position area after registration, and using a position area corresponding to the first position area after registration in the visible light image as a second position area of the target to be detected in the visible light image .
  • the image processing device performs position registration on the first location area of the non-visible image in the non-visible image by using the visible light image as a reference.
  • the first positional area after the registration is obtained, and the first positional area of the object to be detected in the non-visible image is subjected to position registration according to the following formula.
  • the visible light image may be: an image obtained by performing position registration on the target image based on the non-visible light image; in this case, Determining, by the image processing device, the second location area of the object to be detected in the visible light image according to the first location area of the object to be detected in the non-visible image, which may be specifically: A positional area in the image that is identical to the first location area of the object to be detected in the non-visible light image is determined as: a second location area of the object to be detected in the visible light image.
  • the image processing device may be further configured to: mark the second location area in the visible light image Target to be tested.
  • the image processing device may be further configured to: mark, in the visible light image, according to the determined second location area where each target to be detected is located The type of target to be detected.
  • the image processing device is further configured to: when detecting that a target to be detected exists in the non-visible image, identify the to-be-detected in the non-visible image Determining attribute information of the target to obtain a recognition result; after the second location area in the visible light image marks the object to be detected, marking the target image in the visible light image according to the second location area The recognition result of the detection target is described.
  • the system may further include a non-visible light fill lamp, and the non-visible light image capturing device is further configured to obtain a non-visible light image, and After the target image obtained by imaging the visible light signal collected in the acquisition period of the non-visible image, determining a fill state parameter according to the non-visible image; and transmitting the fill state parameter to the non-visible fill light
  • the non-visible fill light is configured to receive the fill light state parameter sent by the non-visible light image collecting device, and perform non-visible light fill according to the fill light state parameter.
  • the solution provided by the embodiment of the present application determines the target detection result of the visible light image corresponding to the non-visible light image by using the detection result obtained by performing target detection on the non-visible light image; in the case of low illumination, The image quality of the non-visible image is high, and the accuracy of the detection result obtained by the target detection of the non-visible image is high, which ensures that the target detection result of the visible light image also has high accuracy, thereby ensuring the effect of intelligent monitoring.
  • the monitoring device may further include a non-visible fill light, and the complementary light generated by the non-visible fill light ensures sufficient non-visible light, so that the obtained non-visible image has high signal-to-noise, ensuring non-visible light. The accuracy of the target detection results obtained by the image.
  • the embodiment of the present application further provides an electronic device, as shown in FIG. 12, including a processor 301, a communication interface 302, a memory 303, and a communication bus 304.
  • the processor 301, the communication interface 302, and the memory 303 pass through the communication bus 304.
  • the memory 303 is configured to store a computer program; and the processor 301 is configured to perform the following steps: obtaining a non-visible image, and based on an acquisition period of the non-visible image when the program stored on the memory 303 is executed Acquiring the visible light signal, imaging the obtained target image; detecting whether there is a target to be detected in the non-visible image; if present, determining the target to be detected in the visible light image according to the first position region of the target to be detected in the non-visible image The second location area in the center, thereby completing target monitoring based on the visible light image, wherein the visible light image is an image determined based on the target image.
  • the visible light image is the target image; and determining the to-be-detected according to the first location area of the to-be-detected target in the non-visible light image.
  • the step of the second location area of the target in the visible light image includes: performing position registration on the first location area of the object to be detected in the non-visible image based on the visible light image to obtain a match a quasi-first position area, and a position area corresponding to the registered first position area in the visible light image as a second position area of the visible image to be detected.
  • the first target location in the non-visible image is subjected to position registration by using the visible light image as a reference to obtain a registration.
  • the step of the first positional area includes: performing a position registration on the first positional area of the object to be detected in the non-visible light image according to the following formula, and obtaining the registered First location area:
  • the visible light image is: an image obtained by performing position registration on the target image based on the non-visible light image; Determining, in a first location area of the non-visible light image, determining a second location area of the object to be detected in the visible light image, comprising: placing the visible light image with the object to be detected The same location area of the first location area in the non-visible light image is determined as: the second location area of the object to be detected in the visible light image.
  • the processor 301 is further configured to: determine, according to the first location area in the non-visible image according to the object to be detected, After the step of detecting the second location area of the target in the visible light image, the second location area in the visible light image marks the object to be detected.
  • the processor 301 is further configured to: perform the step of marking the target to be detected in the second location area in the visible light image Then, according to the determined second location area where each target to be detected is located, the type of the object to be detected is marked in the visible light image.
  • the processor 301 is further configured to: when detecting that there is a target to be detected in the non-visible image, identify the location in the non-visible image Determining the to-be-identified attribute information of the detection target, and obtaining the recognition result; correspondingly, the processor 301 is further configured to: mark the to-be-detected in the second location area in the visible light image After the step of the target, the recognition result for the object to be detected is marked in the visible light image according to the second location area.
  • the processor 301 is further configured to: obtain the non-visible light image, and collect the visible light signal based on the collection period of the non-visible light image. After the step of imaging the obtained target image, the fill light state parameter is determined according to the non-visible light image; and the non-visible light fill light is performed according to the fill light state parameter.
  • the communication bus mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus.
  • PCI Peripheral Component Interconnect
  • EISA Extended Industry Standard Architecture
  • the communication bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only one thick line is shown in the figure, but it does not mean that there is only one bus or one type of bus.
  • the communication interface is used for communication between the above electronic device and other devices.
  • the memory may include a random access memory (RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.
  • RAM random access memory
  • NVM non-volatile memory
  • the memory may also be at least one storage device located away from the aforementioned processor.
  • the above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; or may be a digital signal processing (DSP), dedicated integration.
  • CPU central processing unit
  • NP network processor
  • DSP digital signal processing
  • ASIC Application Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • the solution provided by the embodiment of the present application determines the target detection result of the visible light image corresponding to the non-visible light image by using the detection result obtained by performing target detection on the non-visible light image; in the case of low illumination, The image quality of the non-visible image is high, and the accuracy of the detection result obtained by the target detection of the non-visible image is high, which ensures that the target detection result of the visible light image also has high accuracy, thereby ensuring the effect of intelligent monitoring.
  • a computer readable storage medium having instructions stored therein, when executed on a computer, causing the computer to perform the following steps: obtaining a non-visible light image, and a target image obtained by imaging based on a visible light signal acquired during an acquisition period of the non-visible light image; detecting whether there is a target to be detected in the non-visible light image; and if present, the non-visible light image according to the target to be detected
  • the first location area of the first location area determines a second location area of the object to be detected in the visible light image, thereby completing target monitoring based on the visible light image, wherein the visible light image is an image determined based on the target image.
  • the visible light image is the target image; and determining the to-be-detected according to the first location area of the to-be-detected target in the non-visible light image.
  • the step of the second location area of the target in the visible light image includes: performing position registration on the first location area of the object to be detected in the non-visible image based on the visible light image to obtain a match a quasi-first position area, and a position area corresponding to the registered first position area in the visible light image as a second position area of the visible image to be detected.
  • the first target location in the non-visible image is subjected to position registration by using the visible light image as a reference to obtain a registration.
  • the step of the first positional area includes: performing a position registration on the first positional area of the object to be detected in the non-visible light image according to the following formula, and obtaining the registered First location area:
  • the visible light image is: an image obtained by performing position registration on the target image based on the non-visible light image; Determining, in a first location area of the non-visible light image, determining a second location area of the object to be detected in the visible light image, comprising: placing the visible light image with the object to be detected The same location area of the first location area in the non-visible light image is determined as: the second location area of the object to be detected in the visible light image.
  • the following steps may be further implemented: the first location area in the non-visible image according to the object to be detected After the step of determining the second location area of the object to be detected in the visible light image, marking the object to be detected in the second location area in the visible light image.
  • the following steps may be further implemented: marking the to-be-detected in the second location area in the visible light image After the step of the target, the type of the object to be detected is marked in the visible light image according to the determined second location area where each target to be detected is located.
  • the following steps may be further implemented: when the object to be detected exists in the non-visible light image, the non-visible image is detected. Identifying the to-be-identified attribute information of the object to be detected to obtain a recognition result; correspondingly, when the computer program is executed by the processor, the following step may be further implemented: in the second position in the visible light image After the step of marking the target to be detected, the recognition result for the object to be detected is marked in the visible light image according to the second location area.
  • the following steps may be further implemented: obtaining the non-visible light image, and collecting based on the collection period of the non-visible light image. After the step of imaging the obtained target image by the visible light signal, determining a fill light state parameter according to the non-visible light image; performing non-visible light fill according to the fill light state parameter.
  • the solution provided by the embodiment of the present application determines the target detection result of the visible light image corresponding to the non-visible light image by using the detection result obtained by performing target detection on the non-visible light image; in the case of low illumination, The image quality of the non-visible image is high, and the accuracy of the detection result obtained by the target detection of the non-visible image is high, which ensures that the target detection result of the visible light image also has high accuracy, thereby ensuring the effect of intelligent monitoring.
  • the solution provided by the embodiment of the present application determines the target detection result of the visible light image corresponding to the non-visible light image by using the detection result obtained by performing target detection on the non-visible light image; in the case of low illumination, The image quality of the non-visible image is high, and the accuracy of the detection result obtained by the target detection of the non-visible image is high, which ensures that the target detection result of the visible light image also has high accuracy, thereby ensuring the effect of intelligent monitoring.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Studio Devices (AREA)
  • Image Processing (AREA)

Abstract

本申请实施例提供了一种监控方法、装置、系统、电子设备及计算机可读存储介质，该监控方法中，首先获得非可见光图像，以及基于在非可见光图像的采集周期内采集的可见光信号成像获得的目标图像；然后检测该非可见光图像中是否存在待检测目标；若存在，根据待检测目标在该非可见光图像中的第一位置区域，确定待检测目标在该可见光图像中的第二位置区域，进而完成基于该可见光图像的目标监控，其中，该可见光图像为基于目标图像所确定的图像。与相关技术相比，本方案利用对非可见光图像进行目标检测得到的检测结果，来确定非可见光图像对应的可见光图像的目标检测结果；保证了可见光图像的目标检测结果具有高的准确度，进而保证了智能监控的效果。

Description

监控方法、装置、系统、电子设备及计算机可读存储介质
本申请要求于2017年9月15日提交中国专利局、申请号为201710834952.7发明名称为“监控方法、装置、系统、电子设备及计算机可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及监控技术领域，特别是涉及监控方法、装置、系统、电子设备及计算机可读存储介质。
背景技术
随着城市交通、社会治安以及重点区域防范等问题愈发明显，视频监控的应用越来越广泛。而随着科学技术的飞速发展，智能监控应运而生，智能监控设备可以自动从采集到的图像中检测出预先设置的目标，如人、车辆等，确定目标在所采集图像中的位置区域；例如，预先设置的目标为人，则智能监控设备采集到图像后中，能够自动检测出其所采集图像中人所在的位置区域。
相关技术中，智能监控设备采集到可见光图像后，通过目标检测方法对可见光图像进行目标检测，得到目标在所采集图像中的位置区域，完成智能监控。但是，在低照度的情况下，智能监控设备所采集的可见光图像的图像质量差，通过目标检测方法对可见光图像进行目标检测所得到的检测结果不准确，导致在可见光图像中所确定的目标所在位置区域不准确，监控效果差。
发明内容
本申请实施例的目的在于提供一种监控方法、装置、系统、电子设备及计算机可读存储介质，以提高在可见光图像中确定目标所在位置区域的准确度，保证监控效果。具体技术方案如下：
为达上述目的，第一方面，本申请实施例提供了一种监控方法，所述方法包括：获得非可见光图像，以及基于在所述非可见光图像的采集周期内采集的可见光信号，成像获得的目标图像；检测所述非可见光图像中是否存在待检测目标；若存在，根据所述待检测目标在所述非可见光图像中的第一位置区域，确定所述待检测目标在可见光图像中的第二位置区域，进而完成基于所述可见光图像的目标监控，其中，所述可见光图像为基于所述目标图像 所确定的图像。
可选的，所述可见光图像为所述目标图像；所述根据所述待检测目标在所述非可见光图像中的第一位置区域，确定所述待检测目标在所述可见光图像中的第二位置区域的步骤，包括：以所述可见光图像为基准，对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域，并将所述可见光图像中与配准后的第一位置区域对应的位置区域作为所述可见光图像中待检测目标的第二位置区域。
可选的，所述以所述可见光图像为基准，对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域的步骤，包括：以所述可见光图像为基准，按照如下公式对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域：
Figure PCTCN2018101671-appb-000001
Figure PCTCN2018101671-appb-000002
其中，(x,y)为配准前的第一位置区域内像素点的坐标，(x1,y1)为配准后的第一位置区域中与(x,y)对应的像素点的坐标，
Figure PCTCN2018101671-appb-000003
为离线标定获取到的仿射变换矩阵。
可选的，所述可见光图像为：以所述非可见光图像为基准，对所述目标图像进行位置配准后所得到的图像；所述根据所述待检测目标在所述非可见光图像中的第一位置区域，确定所述待检测目标在所述可见光图像中的第二位置区域的步骤，包括：将所述可见光图像中与所述待检测目标在所述非可见光图像中的第一位置区域相同的位置区域确定为：所述待检测目标在所述可见光图像中的第二位置区域。
可选的，在所述根据所述待检测目标在所述非可见光图像中的第一位置区域，确定所述待检测目标在所述可见光图像中的第二位置区域的步骤之后，所述方法还包括：在所述可见光图像中的所述第二位置区域标记所述待检测目标。
可选的，在所述在所述可见光图像中的所述第二位置区域标记所述待检测目标的步骤之后，所述方法还包括：根据所确定出的每个待检测目标所处的第二位置区域，在所述可见光图像中标记出该待检测目标的种类。
可选的，当检测到所述非可见光图像中存在待检测目标时，所述方法还包括：在所述非可见光图像中识别所述待检测目标的待识别属性信息，得到识别结果；相应的，在所述在所述可见光图像中的所述第二位置区域标记所述待检测目标的步骤之后，所述方法还包括：根据所述第二位置区域，在所述可见光图像中标记出针对所述待检测目标的识别结果。
可选的，在所述获得非可见光图像，以及基于在所述非可见光图像的采集周期内采集的可见光信号成像获得的目标图像的步骤之后，所述方法还包括：根据所述非可见光图像，确定补光状态参数；按照所述补光状态参数进行非可见光补光。
第二方面，本申请实施例提供了一种监控装置，所述装置包括：获得模块，用于获得非可见光图像，以及基于在所述非可见光图像的采集周期内采集的可见光信号成像获得的目标图像；检测模块，用于检测所述非可见光图像中是否存在待检测目标；第一确定模块，用于在所述检测模块的检测结果为是的情况下，根据所述待检测目标在所述非可见光图像中的第一位置区域，确定所述待检测目标在可见光图像中的第二位置区域，进而完成基于所述可见光图像的目标监控，其中，所述可见光图像为基于所述目标图像所确定的图像。
可选的，所述可见光图像为所述目标图像；所述第一确定模块，具体用于：以所述可见光图像为基准，对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域，并将所述可见光图像中与配准后的第一位置区域对应的位置区域作为所述可见光图像中待检测目标的第二位置区域。
可选的，所述第一确定模块，具体用于：以所述可见光图像为基准，按照如下公式对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域，并将所述可见光图像中与配准后的第一位置区域对应的位置区域作为所述可见光图像中待检测目标的第二位置区域：
Figure PCTCN2018101671-appb-000004
Figure PCTCN2018101671-appb-000005
其中，(x,y)为配准前的第一位置区域内像素点的坐标，(x1,y1)为配准后的第一位置区域中与(x,y)对应的像素点的坐标，
Figure PCTCN2018101671-appb-000006
为离线标定获取到的仿射变换矩阵。
可选的，所述可见光图像为：以所述非可见光图像为基准，对所述目标图像进行位置配准后所得到的图像；所述第一确定模块，具体用于：将所述可见光图像中与所述待检测目标在所述非可见光图像中的第一位置区域相同的位置区域确定为：所述待检测目标在所述可见光图像中的第二位置区域。
可选的，所述装置还包括：标记模块，用于在确定所述待检测目标在所述可见光图像中的第二位置区域之后，在所述可见光图像中的所述第二位置区域标记所述待检测目标。
可选的，所述标记模块，还用于在所述可见光图像中的所述第二位置区域标记所述待检测目标之后，根据所确定出的每个待检测目标所处的第二位置区域，在所述可见光图像中标记出该待检测目标的种类。
可选的，所述装置还包括：识别模块，用于当检测到所述非可见光图像中存在待检测目标时，在所述非可见光图像中识别所述待检测目标的待识别 属性信息，得到识别结果；相应的，所述标记模块，还用于在所述可见光图像中的所述第二位置区域标记所述待检测目标之后，根据所述第二位置区域，在所述可见光图像中标记出针对所述待检测目标的识别结果。
可选的，所述装置还包括：第二确定模块，用于在获得非可见光图像，以及基于在所述非可见光图像的采集周期内采集的可见光信号成像获得的目标图像后，根据所述非可见光图像，确定补光状态参数；补光模块，用于按照所述补光状态参数进行非可见光补光。
第三方面，本申请实施例提供了一种监控系统，所述系统包括目标图像采集设备、非可见光图像采集设备以及图像处理设备；其中：所述非可见光图像采集设备，用于非可见光图像，并将所采集的非可见光图像发送给所述图像处理设备；所述目标图像采集设备，用于基于在所述非可见光图像的采集周期内采集的可见光信号，成像获得目标图像，并将所述目标图像发送给所述图像处理设备；所述图像处理设备，用于接收所述目标图像采集设备发送的所述目标图像以及所述非可见光图像采集设备发送的所述非可见光图像；检测所述非可见光图像中是否存在待检测目标；若存在，根据所述待检测目标在所述非可见光图像中的第一位置区域，确定所述待检测目标在可见光图像中的第二位置区域，进而完成基于所述可见光图像的目标监控，其中，所述可见光图像为基于所述目标图像所确定的图像。
可选的，所述可见光图像为所述目标图像；所述图像处理设备根据所述待检测目标在所述非可见光图像中的第一位置区域，确定所述待检测目标在所述可见光图像中的第二位置区域，具体为：以所述可见光图像为基准，对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域，并将所述可见光图像中与配准后的第一位置区域对应的位置区域作为所述可见光图像中待检测目标的第二位置区域。
可选的，所述图像处理设备以所述可见光图像为基准，对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域，具体为：以所述可见光图像为基准，按照如下公式对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域：
Figure PCTCN2018101671-appb-000007
Figure PCTCN2018101671-appb-000008
其中，(x,y)为配准前的第一位置区域内像素点的坐标，(x1,y1)为配准后的第一位置区域中与(x,y)对应的像素点的坐标，
Figure PCTCN2018101671-appb-000009
为离线标定获取到的仿射变换矩阵。
可选的，所述可见光图像为：以所述非可见光图像为基准，对所述目标图像进行位置配准后所得到的图像；所述图像处理设备根据所述待检测目标在所述非可见光图像中的第一位置区域，确定所述待检测目标在所述可见光图像中的第二位置区域，具体为：将所述可见光图像中与所述待检测目标在所述非可见光图像中的第一位置区域相同的位置区域确定为：所述待检测目标在所述可见光图像中的第二位置区域。
可选的，所述图像处理设备还用于：在所述可见光图像中的所述第二位置区域标记所述待检测目标。
可选的，所述图像处理设备还用于：根据所确定出的每个待检测目标所处的第二位置区域，在所述可见光图像中标记出该待检测目标的种类。
可选的，所述图像处理设备还用于：当检测到所述非可见光图像中存在待检测目标时，在所述非可见光图像中识别所述待检测目标的待识别属性信息，得到识别结果；并在所述可见光图像中的所述第二位置区域标记所述待检测目标之后，根据所述第二位置区域，在所述可见光图像中标记出针对所述待检测目标的识别结果。
可选的，所述系统还包括非可见光补光灯，所述非可见光图像采集设备，还用于在采集到所述非可见光图像后，根据所述非可见光图像，确定补光状 态参数；非可见光补光灯按照所述补光状态参数进行非可见光补光。
第四方面，本申请实施例提供了一种电子设备，包括处理器、通信接口、存储器和通信总线，其中，处理器，通信接口，存储器通过通信总线完成相互间的通信；存储器，用于存放计算机程序；处理器，用于执行存储器上所存放的程序时，实现上述任一监控方法所述的方法步骤。
第五方面，本申请实施例提供了一种计算机可读存储介质，所述计算机可读存储介质内存储有计算机程序，所述计算机程序被处理器执行时实现上述任一监控方法所述的方法步骤。
第六方面，本申请实施例提供了一种计算机程序产品，当其在计算机上运行时，使得计算机执行上述实施例中任一所述的监控方法步骤。
本申请实施例提供的方案中，首先获得可见光图像，以及基于在非可见光图像的采集周期内采集的可见光信号成像获得的目标图像；然后检测该非可见光图像中是否存在待检测目标；若存在，则根据待检测目标在该非可见光图像中的第一位置区域，确定待检测目标在该可见光图像中的第二位置区域，进而完成基于该可见光图像的目标监控。
与相关技术相比，本申请实施例提供的方案，利用对非可见光图像进行目标检测得到的检测结果，来确定非可见光图像对应的可见光图像的目标检测结果；非可见光信号受环境中的照度的影响，相较于可见光信号较小，低照度情况下，非可见光图像的图像质量高，所以对非可见光图像进行目标检测得到的检测结果的准确度高，保证基于可见光图像确定的目标图像的目标检测结果同样具有高的准确度，进而保证了智能监控的效果，使得智能监控的效果更好。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本申请一实施例提供的一种监控方法的流程示意图。
图2为本申请实施例涉及的对目标图像进行ISP处理(Image Signal Processing，图像信号处理)的示意图。
图3为本申请实施例涉及的对非可见光图像进行ISP处理的示意图。
图4为本申请另一实施例提供的一种监控方法的流程示意图。
图5为本申请实施例涉及的对目标图像进行ISP处理的过程示意图。
图6为本申请实施例涉及的对非可见光图像进行ISP处理的过程示意图。
图7为本申请实施例涉及的对非可见光图像进行目标检测的过程示意图。
图8为本申请实施例提供的一种监控装置的结构示意图。
图9为本申请另一实施例提供的一种监控装置的结构示意图。
图10为本申请一实施例提供的一种监控系统的结构示意图。
图11为本申请另一实施例提供的一种监控系统的结构示意图。
图12为本申请实施例提供的一种电子设备的结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
下面首先针对本申请文件涉及的技术术语进行简单介绍。
可见光图像，是利用可见光信号成像获得的图像。同理，非可见光图像是利用非可见光信号成像获得的图像。其中，非可见光是人类肉眼看不见的电磁波，包括红外光、紫外线等；其中的红外光，通常指近红外光(Near Infrared，NIR)，近红外光是介于可见光(VIS)和中红外光(MIR)之间的电磁波，按ASTM(美国试验和材料检测协会)定义是指波长在780～2526nm范围内的电磁波，习惯上又将近红外区划分为近红外短波(780～1100nm)和近红外长波(1100～2526nm)两个区域。
在本申请文件中，上述非可见光可以直接为红外光，相应的，非可见光 图像为红外图像；上述非可见光还可以为红外光中的近红外光，相应的，非可见光图像为近红外图像。
目标检测，是指利用目标检测方法从图像中检测出预先设置的待检测目标，如人、汽车等，目标检测的过程中，可以确定出待检测目标在图像中的区域，以及待检测目标的种类。常见的目标检测方法包括：基于深度学习的目标检测方法、利用图像分割技术的目标检测方法以及利用特征匹配技术的目标检测方法等。
下面通过具体实施方式来对本申请文件进行详细介绍。
本申请实施例提供的一种监控方法，可以应用于能够采集可见光图像和非可见光图像的监控设备，例如常见的球形摄像机(球机)等摄像机，或是对摄像机传回图像进行处理的图像处理设备，这都是合理的，本申请实施例并不对上述监控设备的具体形式做限定。
如图1所示的一种监控方法的流程示意图，本申请实施例提供的一种监控方法包括：S101：获得非可见光图像，以及基于在非可见光图像的采集周期内采集的可见光信号成像获得的目标图像。
需要说明的是，步骤S101中的目标图像为非可见光图像的采集周期内采集的可见光信号成像获得的，该非可见光图像与目标图像存在一一对应的关系。其中，该非可见光图像与目标图像可以为：针对同一场景所获得的图像。进一步的，该非可见光图像与该目标图像为：针对同一场景在相同曝光时间段所获得的。
作为获得上述目标图像以及非可见光图像的一种可选的实现方式，上述步骤S101，可以包括：接收摄像机等其他设备传回的目标图像以及非可见光图像，进而得到上述目标图像以及非可见光图像。
例如，上述监控设备为一图像处理设备，该图像处理设备通信连接一摄像机，该摄像机可以同时采集上述非可见光图像以及其对应的目标图像，并将所采集的非可见光图像以及目标图像发送给该图像处理设备，则该图像处理设备接收获得非可见光图像以及上述目标图像。
作为获得上述目标图像和非可见光图像的另一种可选的实现方式，上述 监控设备可以为具有摄像功能的设备，该监控设备上设置有摄像部件，该摄像部件采集非可见光信号和可见光信号，并利用所采集的非可见光信号成像生成非可见光图像，利用所采集的可见光信号，成像生成目标图像。
一种情况下，该摄像部件中包含有相机镜头、分光单元、可见光传感器以及非可见光传感器。其中，该分光单元可以为半反半透镜，该半反半透镜可以透射可见光而反射非可见光。具体的，该半反半透镜的具体产品形式可以为但不局限于分光棱镜；本领域技术人员可以理解的是，分光棱镜可以允许可见光透过，截止或反射非可见光，也就是，通过棱镜反射面的入射光被滤去非可见波段(被滤去的非可见光可以被反射)，而让可见光通过；另外，对于分光棱镜而言，其能够通过的可见光波段范围，可以通过调节棱镜镀膜的厚度来控制，本申请实施例并不对调节棱镜镀膜的厚度的控制方式进行限定，目前相关的任一可以调节棱镜镀膜的厚度的控制方式，均可以应用于本申请实施例中。
在此情况下，相机镜头捕捉的入射光被分光单元分离成可见光信号以及非可见光信号，可见光传感器利用可见光信号，成像生成上述目标图像，同样的，非可见光传感器利用非可见光信号，成像生成非可见光图像。
另一种情况下，该摄像部件中包含有两个摄像头，其中一个摄像头用于采集可见光信号，并利用采集的可见光信号成像生成上述目标图像，同时，另一摄像头用于采集非可见光信号，并利用采集的非可见光信号成像生成非可见光图像。
本申请实施例中，为了保证所获得的目标图像的图像质量，可以对所获得的目标图像做ISP(Image Signal Processor)处理，即上述步骤S101所获得的目标图像为经过ISP处理后的目标图像。如图2所示，对成像获得的原始目标图像所进行的ISP处理可以包括：包含黑电平及坏点校正的Sensor校正；包含白平衡校正、色彩校正及Gamma校正的色彩还原处理；包含色彩插值和RGB(一种颜色标准)转YUV(一种颜色标准)处理的色彩空间转换处理；以及包含Tone mapping(色调映射)、降噪、锐化和透雾的图像增强处理。其中，本申请实施例并不对成像获得的原始目标图像所进行的ISP处理的具体执行顺序进行限定。
同理，为了保证所获得的非可见光图像的图像质量，可以对所获得的非可见光图像做ISP处理，即上述步骤S101所获得的非可见光图像为经过ISP处理后的非可见光图像。如图3所示，对成像获得的原始非可见光图像所进行的ISP处理可以包括：包含黑电平、坏点校正和Gamma校正的图像校正处理；以及包含Tone mapping、降噪、锐化和透雾的图像增强处理。其中，本申请实施例并不对成像获得的原始非可见光图像所进行的ISP处理的具体执行顺序进行限定。
可以说明的是，本申请实施例中，对直接成像获得的目标图像以及非可见光图像所采用的ISP处理可以借鉴目前相关的任一ISP处理技术去实现，本申请实施例在此不再详细介绍。
在监控设备利用所采集的可见光信号和非可见光信号分别生成目标图像和非可见光图像的实现方式下，为了提高所获得的非可见光图像的信噪比，监控设备所采集的非可见光信号可以包括环境入射光中的非可见光信号，以及额外补充的非可见光信号。
即在本申请实施例中，监控设备还可以包括有一个非可见光补光灯，该非可见光补光灯的补光强度等补光状态参数可以是预先固定设置的。在一种情况下，在实际的应用场景中，光照强度可能随时发生变化，为保证非可见光的充足，并实现对补光状态参数的动态调整，在上述步骤S101之后，上述方法还可以包括下述步骤a和步骤b：步骤a：根据该非可见光图像，确定补光状态参数。
监控设备可以根据非可见光图像的图像信息，确定补光状态参数，此处的图像信息可以是非可见光图像的信噪比、曝光参数等，而补光状态参数可以是非可见光补光灯的开闭状态以及补光强度等。当然，步骤a的具体实现可以参照相关技术，本申请实施例在此不做赘述，仅以下述实例进行示例性说明。
示例性的，监控设备在获得非可见光图像后，首先计算出非可见光图像的信噪比，当所计算的信噪比大于阈值T1时，确定补光强度为0，即关闭非可见光补光灯；当所计算的信噪比小于阈值T2时，确定新的补光强度为原补光强度与预设正值K1的和值；当所计算的信噪比处于[T1，T2]的范围时，确定 补光强度不变，即新的补光强度为原补光强度，其中，T1＞T2。
在一种情况中，上述原补光强度可以指：获得上述非可见光图像时，非可见光补光灯的补光强度。
步骤b：按照所确定的补光状态参数进行非可见光补光。
此处所述的按照所确定的补光状态参数进行非可见光补光，即表示上述非可见光补光灯按照所确定的补光状态参数进行非可见光补光，使得监控设备此后所采集的非可见光信号包括环境入射光中的非可见光信号，以及非可见光灯额外补充的非可见光信号。例如，该非可见光补光灯为红外LED补光灯，则补光状态参数可以为LED灯头数目，红外补光强度以及角度，则红外LED补光灯按照所确定的LED灯头数目，红外补光强度以及角度进行红外补光。
需要说明的是，监控设备按照所确定的补光状态参数进行非可见光补光时，并不会影响当前已获得的非可见光图像的图像质量，而是对当前时刻后获得的非可见光图像的图像质量产生影响。
S102：检测该非可见光图像中是否存在待检测目标。
可以说明的是，此处的待检测目标为预先设置的所要检测的目标，例如，预先设置待检测目标的种类是人和汽车，则步骤S102需要检测所获得的非可见光图像中是否存在人或汽车；若预先设置待检测目标的种类仅是人，则步骤S102需要检测所获得的非可见光图像中是否存在人。
本申请实施例中，监控设备可以利用相关技术中任何可行的技术来完成步骤S102。例如，利用目标检测方法，检测该非可见光图像中是否存在待检测目标。此处所述的目标检测方法可以是：基于深度学习的目标检测方法、利用图像分割技术的目标检测方法或利用特征匹配技术的目标检测方法等，本申请实施例并不限定该目标检测方法的具体种类，并且目标检测方法的具体实现方式可以参照相关技术，本申请实施例对于目标检测方法的具体实现方式不做详细介绍。
在本申请实施例中，如果步骤S102的检测结果为是，则执行步骤S103：根据待检测目标在该非可见光图像中的第一位置区域，确定待检测目标在可见光图像中的第二位置区域，进而完成基于该可见光图像的目标监控，其中， 该可见光图像为基于目标图像所确定的图像。
本领域技术人员可以理解的是，在利用目标检测方法等技术进行目标检测的过程中，如果检测出该非可见光图像中存在有待检测目标，则此时待检测目标的在该非可见光图像中的位置区域已被确定。另外，确定待检测目标在该可见光图像中的第二位置区域后，监控设备获得了已确定出第二位置区域的可见光图像，即完成了在可见光图像中的目标检测，进而可以完成基于该可见光图像的目标监控。
本申请实施例中，上述非可见光图像与目标图像为：针对同一场景在相同曝光时间段所获得的，当上述非可见光图像中存在待检测目标时，上述目标图像中存在待检测目标，上述可见光图像为根据目标图像所确定的图像，且可见光图像中包含目标图像的目标特征，即当上述目标图像中存在待检测目标时，上述可见光图像中存在待检测目标。
在一种实现方式中，该可见光图像为基于目标图像所确定的图像，且可见光图像中包含目标图像的目标特征。例如：目标图像中包含人时，即目标图像中包含人的特征，该可见光图像中包含目标图像中的人的特征。
在一种实现方式中，如图1所示，当步骤S102的检测结果为否时，本申请实施例提供的监控方法的流程指向“结束”，表明此时可见光图像中不存在待检测目标，此时监控设备得到一个基于在上述非可见光图像的采集周期内采集的可见光信号成像获得的图像，且该图像中不存在待检测目标；而相对的，当步骤S102的检测结果为是时，监控设备得到一个确定了上述第二位置区域的上述可见光图像。
上述可见光图像为基于目标图像所确定的图像，一种情况下，该可见光图像即为上述目标图像。作为步骤S103的一种可选的实现方式，当上述可见光图像为上述目标图像时，上述根据待检测目标在该非可见光图像中的第一位置区域，确定待检测目标在该可见光图像中的第二位置区域的步骤，可以是：首先在可见光图像中找到与第一位置区域在该非可见光图像中所处位置相同的位置区域，然后直接将所找到的位置区域确定为：待检测目标在该可见光图像中的第二位置区域。
示例性的，在非可见光图像中找到第一位置区域，其为像素坐标为(x2，y2)至(x2+100，y2+200)的矩形区域；则在可见光图像，即目标图像中，将像素坐标同样为(x2，y2)至(x2+100，y2+200)的矩形区域确定为可见光图像中的第二位置区域。
在上述实现方式下，由于设备本身的设计、制造工艺等原因，实际获得的目标图像与非可见光图像之间存在一定的位置偏差；为了消除上述目标图像与非可见光图像之间的位置偏差，保证上述可见光图像与非可见光图像具有像素级的对齐，进而保证所确定第二位置区域的准确度，作为步骤S103的另一种可选的实现方式，当上述可见光图像为上述目标图像时，上述根据待检测目标在该非可见光图像中的第一位置区域，确定待检测目标在该可见光图像中的第二位置区域(S103)的步骤，可以包括：以该可见光图像为基准，对待检测目标在该非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域，并将该可见光图像中与配准后的第一位置区域对应的位置区域作为该可见光图像中待检测目标的第二位置区域。
本申请实施例中可以以该可见光图像为基准，对整个非可见光图像进行配准，但为了减少计算量，可以仅对待检测目标在该非可见光图像中的第一位置区域进行位置配准。当然，配准的具体实现可以采用例如，基于水平镜像、仿射变换或投影变换实现的位置配准等相关技术。
作为一种可选的实现方式，上述以该可见光图像为基准，对待检测目标在该非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域的步骤，可以包括：以该可见光图像为基准，按照如下公式对待检测目标在该非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域：
Figure PCTCN2018101671-appb-000010
Figure PCTCN2018101671-appb-000011
其中，(x,y)为配准前的第一位置区域内像素点的坐标，(x1,y1)为配准后的第一位置区域中与(x,y)对应的像素点的坐标，
Figure PCTCN2018101671-appb-000012
为离线标定获取到的仿射变换矩阵。
其中，上述仿射变换矩阵可以利用相关的仿射变换技术离线标定获得。
示例性的，在非可见光图像中找到第一位置区域，其为像素坐标为(x2，y2)至(x2+100，y2+200)的矩形区域，然后利用上述公式进行位置配准后，配准后的第一位置区域为非可见光图像中像素坐标为(x2+2，y2+4)至(x2+102，y2+204)的矩形区域；则在可见光图像中，将像素坐标同样为(x2+2，y2+4)至(x2+102，y2+204)的矩形区域确定为可见光图像中的第二位置区域。
需要强调的是，上述获得配准后的第一位置区域的实现方式仅仅作为示例性说明，并不构成对本申请实施例的限定，本领域技术人员完全可以基于相关技术获得配准后的第一位置区域。
同样为了保证上述可见光图像与非可见光图像具有像素级的对齐，进而保证所确定第二位置区域的准确度；上述可见光图像为基于目标图像所确定的图像，具体的，在另一种情况下，该可见光图像可以为：以上述非可见光图像为基准，对目标图像进行位置配准后所得到的图像。
当然，以上述非可见光图像为基准，对目标图像进行位置配准的具体实现方式与上述以该可见光图像为基准，对待检测目标在该非可见光图像中的第一位置区域进行位置配准的方式相同，本领域技术人员可以基于上述以该可见光图像为基准，对待检测目标在该非可见光图像中的第一位置区域进行位置配准的方式，完成以上述非可见光图像为基准，对目标图像进行位置配准的操作，本申请实施例在此不再赘述。
在此情况下，上述根据待检测目标在该非可见光图像中的第一位置区域，确定待检测目标在该可见光图像中的第二位置区域(S103)的步骤，可以包括：将可见光图像中与待检测目标在非可见光图像中的第一位置区域相同的位置区域确定为：待检测目标在可见光图像中的第二位置区域。
示例性的，在非可见光图像中找到第一位置区域，例如其为像素坐标为(x2，y2)至(x2+200，y2+100)的矩形区域；则在可见光图像，即目标图像的配准图像中，将像素坐标同样为(x2，y2)至(x2+200，y2+100)的矩形区域确定为可见光图像中的第二位置区域。
可以理解，在相关技术中，监控设备直接通过目标检测方法对基于可见光信号成像获得的可见光图像进行目标检测，得到目标在直接成像获得的可见光图像中的位置区域，完成智能监控，但是在低照度情况下，直接成像获得的可见光图像的图像质量差，即使通过图像增强或增加局部曝光量的手段，也无法大幅增加直接成像获得的可见光图像的图像质量，所以相关技术无法在直接成像获得的可见光图像中，准确地确定待检测目标所处的区域，甚至于无法准确地检测到直接成像获得的可见光图像中，是否存在待检测目标。
由上可知，与相关技术相比，本申请实施例提供的方案，利用对非可见光图像进行目标检测得到的检测结果，来确定非可见光图像对应的可见光图像的目标检测结果；低照度情况下，非可见光图像的图像质量高，对非可见光图像进行目标检测得到的检测结果的准确度高，保证可见光图像的目标检测结果同样具有高的准确度，进而保证了智能监控的效果。
并且，监控设备还可以包括有一个非可见光补光灯，非可见光补光灯产生的补光保证了非可见光的充足，使得所获得的非可见光图像的信噪比较高，保证了针对非可见光图像得到的目标检测结果的准确性。
可以理解的，上述实施例中，监控设备执行监控方法后，得到确定了上述第二位置区域的可见光图像，一种情况下，监控设备可以直接将确定了上述第二位置区域的可见光图像(附带有目标检测结果的可见光图像)存储在本地或者传输给其他设备，以供查看。为了使得确定了上述第二位置区域的可见光图像被显示后，可以在图像中清楚地展示待检测目标所处的位置，如图4所示，在上述根据待检测目标在该非可见光图像中的第一位置区域，确定待检测目标在该可见光图像中的第二位置区域(S103)的步骤之后，所述方法还可以包括：S104：在该可见光图像中的第二位置区域标记待检测目标。
步骤S104的具体实现方式有很多，本申请实施例并不限定步骤S104的具体实现方式，只要保证在将标记了第二位置区域的可见光图像显示出来时， 人眼可以看到所标记的第二位置区域即可。例如，标记了第二位置区域的可见光图像被显示出来时，可见光图像中的第二位置区域的边缘区域被高亮显示，或者可见光图像中用红色线框标记着所确定出的第二位置区域。当然，步骤S104的具体标记方式可以利用相关的任一标记技术，本申请实施例在此不再详细介绍。
在一种可选的实现方式中，由于预先设置的待检测目标可以是至少两种，例如，待检测目标包括人和汽车，为了使得人眼能够在被显示后的可见光图像中，清楚地看到被检测出的每个待检测目标的种类，在上述在该可见光图像中的第二位置区域标记待检测目标(S104)的步骤之后，上述监控方法还可以包括：根据所确定出的每个待检测目标所处的第二位置区域，在该可见光图像中标记出待检测目标的种类。
可以理解的，待检测目标的种类是预先设置的，监控设备在非可见光图像中检测待检测目标所处的第一位置区域时，是按照对应种类的待检测目标进行查找检测的，故监控设备完全可以获知每个待检测目标的种类。例如，监控设备在非可见光图像中检测树木所处的第一位置区域，显然的，当找到一个第一位置区域时，该第一位置区域对应的待检测目标的种类为树木。
另外需要说明的是，可见光图像中的每个第二位置区域对应的待检测目标的种类，与该第二位置区域对应的第一位置区域所对应的待检测目标的种类相同。例如，非可见光图像中，第一位置区域所对应的待检测目标的种类为汽车，则在可见光图像中，与该第一位置区域对应的第二位置区域所对应的待检测目标的种类同样为汽车。
此处所表述的根据所确定出的每个待检测目标所处的第二位置区域，在该可见光图像中标记出待检测目标的种类，可以为：在所确定出的每个待检测目标所处的第二位置区域内，标记出该待检测目标的种类，例如，被显示出来可见光图像中用红色线框标记着所确定出的第二位置区域，每个红色线框区域内还显示有种类标记“人”；还可以为：在所确定出的每个待检测目标所处的第二位置区域外侧，不超过预设像素距离处标记出该待检测目标的种类，例如，被显示出来可见光图像中用红色线框标记着所确定出的第二位置区域，每个红色线框区域右下角外20像素距离处还显示有种类标记“人”。
当然，关于上述步骤中种类的具体标记方式同样可以利用相关技术，只要使得可见光图像被显示出来后，能够清楚的展示每一个第二位置区域所对应的种类即可，本申请实施例在此不再详细介绍。
更进一步的，作为本申请实施例一种可选的实现方式，当检测到上述非可见光图像中存在待检测目标时，上述方法还可以包括：在该非可见光图像中识别待检测目标的待识别属性信息，得到识别结果。
上述待识别属性信息可以理解为：预先设定的需要识别的属性信息，例如，如果待检测目标为车辆，上述待识别属性信息可以是车辆的车牌号、车辆的类型等，如果待检测目标的种类为人，上述待识别属性信息可以是人的性别等。当然，本申请实施例并不限定该待识别属性的具体种类，可以根据实际的需要设定待识别属性信息。
另外，在该非可见光图像中识别待检测目标的待识别属性信息的方法可以是相关技术中可行的图像识别方法，例如，基于训练和分类的图像识别方法、基于神经网络的图像识别方法以及基于小波矩的图像识别方法等，本申请实施例在此并不限定所采用的识别方法。
相应的，在上述在该可见光图像中的第二位置区域标记待检测目标(S104)的步骤之后，上述方法还可以包括：根据上述第二位置区域，在该可见光图像中标记出针对待检测目标的识别结果。
此处所表述的根据上述第二位置区域，在该可见光图像中标记出待检测目标的识别结果，可以为：在所确定出的待检测目标所处的第二位置区域内，标记出该待检测目标的种类和/或识别结果，例如，待检测目标的种类为车辆，待识别属性信息为车辆类型和车牌号，则被显示出来的可见光图像中用红色线框标记着所确定出的第二位置区域，每个红色线框区域内还显示有所识别出的车辆类型以及车牌号：货车，京Axxxxx。
还可以为：在所确定出的待检测目标所处的第二位置区域外侧不超过预设像素距离处标记出针对该待检测目标的种类和/或识别结果，例如，待检测目标的种类为人，待识别属性信息为人的性别，则在被显示出来的可见光图像中用红色线框标记着所确定出的第二位置区域，每个红色线框区域右下角 外20像素距离处还显示有识别结果“男”。
当然，关于上述步骤中识别结果的具体标记方式同样可以利用相关技术，只要使得可见光图像被显示出来后，能够清楚的展示每个第二位置区域所对应的识别结果即可，本申请实施例在此不再详细介绍。
同样可以理解的是，低照度情况下，非可见光图像的图像质量高，对非可见光图像进行图像识别所得到的识别结果的准确度高，而本申请实施例在该可见光图像中标记出针对待检测目标的识别结果，使得可见光图像也同样可以携带准确的图像识别结果。
另外，对于本申请实施例中的可见光图像中既可以仅标记第二位置区域以及待检测目标的种类，也可以仅标记第二位置区域以及针对待检测目标的识别结果，还可以同时标记第二位置区域、待检测目标的种类以及针对待检测目标的识别结果。
在本申请实施例中，如果该监控设备自身具有显示部件，则可以将标记后的可见光图像在本地显示出来，以使得人眼可以清楚地在显示出来的可见光图像中看到标记后的可见光图像，显然的，此处所述的标记后的可见光图像为：标记有第二位置区域，且同时标记有待检测目标的种类和/或针对待检测目标的识别结果的可见光图像。
当然，大多数情况下，获得上述标记后的可见光图像后，监控设备会将该标记后的可见光图像传输到其它的存储设备中，并可以由其它的显示设备从该存储设备中读取并显示该可见光图像；此情况下，为了便于传输，监控设备可以对标记后的可见光图像进行视频编码操作，然后将编码后的可见光图像传输给存储设备。
下面通过具体实例来对本申请实施例进行简单介绍。
本实例中，上述监控设备为一球机，该球机连接着后端控制台，该球机中包含有相机镜头、分光单元、可见光传感器以及非可见光传感器，而预设的待检测目标的种类为人和汽车。首先，相机镜头捕捉的入射光被分光单元 分离成可见光信号以及非可见光信号，可见光传感器利用可见光信号成像生成原始可见光图像，同样的，非可见光传感器利用同一曝光时间段采集的非可见光信号成像生成原始非可见光图像。
然后，球机对所获得的原始可见光图像进行如图5所示的ISP处理：即依次对原始可见光图像做黑电平、坏点校正、白平衡校正、色彩插值、Gamma校正、色彩校正、RGB转YUV、降噪和锐化处理，获得处理后的可见光图像；同理，球机还对所获得的原始非可见光图像进行如图6所示的ISP处理：即依次对原始非可见光图像做黑电平、坏点校正、Gamma校正、降噪和锐化处理，获得处理后的非可见光图像。
此后，如图7所示，球机利用基于深度学习的目标检测方法，按照如下步骤：1、提取候选区域，2、计算特征，3、分类，4、精细修正候选区域，对处理后的非可见光图像进行目标检测，确定处理后的非可见光图像中存在一个待检测目标汽车，且待检测目标汽车在处理后的非可见光图像中的第一位置区域为像素坐标为(x3，y3)至(x3+50，y2+80)的矩形区域。然后，球机通过基于训练和分类的图像识别方法，在该非可见光图像中识别待检测目标的车牌号，并得到识别结果：浙Axxxxx。
球机再以上述处理后的可见光图像为基准，对待检测目标汽车在处理后的非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域，配准后的第一位置区域在非可见光图像中像素坐标为(x3+2，y3+4)至(x3+52，y3+84)的矩形区域，则在处理后的可见光图像中，将像素坐标同样为(x3+2，y3+4)至(x3+52，y3+84)的矩形区域确定为可见光图像中的第二位置区域。
球机在该处理后的可见光图像中标记出第二位置区域，并在该第二位置区域内标记待检测目标的种类汽车以及上述识别结果，然后球机将标记后的可见光图像编码后发送给后端控制台，后端控制台接收并显示标记后的可见光图像，在显示的可见光图像中，所确定出的第二位置区域被红色线框标记出来，同时，该红色线框内展示有待检测目标的种类“汽车”，以及上述识别结果：浙Axxxxx。
相应于图1所示的方法实施例，本申请实施例还提供了一种监控装置，如图8所示，所述装置包括：获得模块110，用于获得非可见光图像，以及基于在所述非可见光图像的采集周期内采集的可见光信号，成像获得的目标图像；检测模块120，用于检测所述非可见光图像中是否存在待检测目标；第一确定模块130，用于在所述检测模块120的检测结果为是的情况下，根据所述待检测目标在所述非可见光图像中的第一位置区域，确定所述待检测目标在可见光图像中的第二位置区域，进而完成基于所述可见光图像的目标监控，其中，所述可见光图像为基于所述目标图像所确定的图像。
作为本申请实施例的一种可选的实现方式，所述可见光图像可以为所述目标图像；此情况下，所述第一确定模块130，可以具体用于：以所述可见光图像为基准，对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域，并将所述可见光图像中与配准后的第一位置区域对应的位置区域作为所述可见光图像中待检测目标的第二位置区域。
作为本申请实施例的一种可选的实现方式，所述第一确定模块130，可以具体用于：以所述可见光图像为基准，按照如下公式对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域，并将所述可见光图像中与配准后的第一位置区域对应的位置区域作为所述可见光图像中待检测目标的第二位置区域：
Figure PCTCN2018101671-appb-000013
Figure PCTCN2018101671-appb-000014
其中，(x,y)为配准前的第一位置区域内像素点的坐标，(x1,y1)为配准后的第一位置区域中与(x,y)对应的像素点的坐标，
Figure PCTCN2018101671-appb-000015
为离线标定获取 到的仿射变换矩阵。
作为本申请实施例的另一种可选的实现方式，所述可见光图像可以为：以所述非可见光图像为基准，对所述目标图像进行位置配准后所得到的图像；此情况下，所述第一确定模块130，可以具体用于：将所述可见光图像中与所述待检测目标在所述非可见光图像中的第一位置区域相同的位置区域确定为：所述待检测目标在所述可见光图像中的第二位置区域。
相应于图4所示方法实施例，如图9所示，所述装置还可以包括：标记模块140，用于在确定所述待检测目标在所述可见光图像中的第二位置区域之后，在所述可见光图像中的所述第二位置区域标记所述待检测目标。
作为本申请实施例的一种可选的实现方式，所述标记模块140，还可以用于在所述可见光图像中的所述第二位置区域标记所述待检测目标之后，根据所确定出的每个待检测目标所处的第二位置区域，在所述可见光图像中标记出该待检测目标的种类。
作为本申请实施例的一种可选的实现方式，所述装置还可以包括：识别模块，用于当检测到所述非可见光图像中存在待检测目标时，在所述非可见光图像中识别所述待检测目标的待识别属性信息，得到识别结果；相应的，所述标记模块，还用于在所述可见光图像中的所述第二位置区域标记所述待检测目标之后，根据所述第二位置区域，在所述可见光图像中标记出针对所述待检测目标的识别结果。
作为本申请实施例的一种可选的实现方式，所述装置还可以包括：第二确定模块，用于在获得非可见光图像，以及基于在所述非可见光图像的采集周期内采集的可见光信号成像获得的目标图像后，根据所述非可见光图像，确定补光状态参数；补光模块，用于按照所述补光状态参数进行非可见光补光。
由上可知，与相关技术相比，本申请实施例提供的方案，利用对非可见光图像进行目标检测得到的检测结果，来确定非可见光图像对应的可见光图像的目标检测结果；低照度情况下，非可见光图像的图像质量高，对非可见光图像进行目标检测得到的检测结果的准确度高，保证了可见光图像的目标 检测结果同样具有高的准确度，进而保证了智能监控的效果。
并且，监控设备还可以包括有一个非可见光补光灯，非可见光补光灯产生的补光保证了非可见光的充足，使得所获得的非可见光图像的信噪比较高，保证了针对非可见光图像得到的目标检测结果的准确性。
相应于图1所示方法实施例，本申请实施例还提供了一种监控系统，如图10所示，所述系统包括目标图像采集设备、非可见光图像采集设备以及图像处理设备；其中：所述非可见光图像采集设备，用于非可见光图像，并将所采集的非可见光图像发送给所述图像处理设备；所述目标图像采集设备，用于基于在所述非可见光图像的采集周期内采集的可见光信号，成像获得目标图像，并将所述目标图像发送给所述图像处理设备；所述图像处理设备，用于接收所述目标图像采集设备发送的所述目标图像以及所述非可见光图像采集设备发送的所述非可见光图像；针对待检测目标，对所获得的非可见光图像进行目标检测，得到包含有所述非可见光图像中待检测目标的第一位置区域的检测结果；根据所述第一位置区域，确定可见光图像中待检测目标的第二位置区域，其中，所述可见光图像为基于所述目标图像所确定的图像。
可以说明的是，本申请实施例中，该目标图像采集设备以及非可见光图像采集设备可以为两个物理设备；当然该目标图像采集设备以及非可见光图像采集设备同样可以位于同一物理实体上，例如目标图像采集设备以及非可见光图像采集设备可以是常见的球机中所包含的两个子设备。
当然，在本申请实施例所提供的监控系统中，各个设备分别执行的操作都可以参照上述方法实施例中监控设备所执行的对应操作，本申请实施例在此不再赘述。
作为本申请实施例的一种可选的实现方式，所述可见光图像可以为所述目标图像；此情况下，所述图像处理设备根据所述待检测目标在所述非可见光图像中的第一位置区域，确定所述待检测目标在所述可见光图像中的第二位置区域，可以具体为：以所述可见光图像为基准，对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区 域，并将所述可见光图像中与配准后的第一位置区域对应的位置区域作为所述可见光图像中待检测目标的第二位置区域。
作为本申请实施例的一种可选的实现方式，所述图像处理设备以所述可见光图像为基准，对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域，可以具体为：以所述可见光图像为基准，按照如下公式对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域：
Figure PCTCN2018101671-appb-000016
Figure PCTCN2018101671-appb-000017
其中，(x,y)为配准前的第一位置区域内像素点的坐标，(x1,y1)为配准后的第一位置区域中与(x,y)对应的像素点的坐标，
Figure PCTCN2018101671-appb-000018
为离线标定获取到的仿射变换矩阵。
作为本申请实施例的另一种可选的实现方式，所述可见光图像可以为：以所述非可见光图像为基准，对所述目标图像进行位置配准后所得到的图像；此情况下，所述图像处理设备根据所述待检测目标在所述非可见光图像中的第一位置区域，确定所述待检测目标在所述可见光图像中的第二位置区域，可以具体为：将所述可见光图像中与所述待检测目标在所述非可见光图像中的第一位置区域相同的位置区域确定为：所述待检测目标在所述可见光图像中的第二位置区域。
相应于图4所示方法实施例，作为本申请实施例的一种可选的实现方式，所述图像处理设备还可以用于：在所述可见光图像中的所述第二位置区域标记所述待检测目标。
作为本申请实施例的一种可选的实现方式，所述图像处理设备还可以用于：根据所确定出的每个待检测目标所处的第二位置区域，在所述可见光图像中标记出该待检测目标的种类。
作为本申请实施例的一种可选的实现方式，所述图像处理设备还用于：当检测到所述非可见光图像中存在待检测目标时，在所述非可见光图像中识别所述待检测目标的待识别属性信息，得到识别结果；在所述可见光图像中的所述第二位置区域标记所述待检测目标之后，根据所述第二位置区域，在所述可见光图像中标记出针对所述待检测目标的识别结果。
作为本申请实施例的一种可选的实现方式，如图11所示，所述系统还可以包括非可见光补光灯，所述非可见光图像采集设备，还用于在获得非可见光图像，以及基于在所述非可见光图像的采集周期内采集的可见光信号成像获得的目标图像后，根据所述非可见光图像，确定补光状态参数；将所述补光状态参数发送给所述非可见光补光灯；所述非可见光补光灯，用于接收所述非可见光图像采集设备发送的所述补光状态参数；按照所述补光状态参数进行非可见光补光。
由上可知，与相关技术相比，本申请实施例提供的方案，利用对非可见光图像进行目标检测得到的检测结果，来确定非可见光图像对应的可见光图像的目标检测结果；低照度情况下，非可见光图像的图像质量高，对非可见光图像进行目标检测得到的检测结果的准确度高，保证了可见光图像的目标检测结果同样具有高的准确度，进而保证了智能监控的效果。
并且，监控设备还可以包括有一个非可见光补光灯，非可见光补光灯产生的补光保证了非可见光的充足，使得所获得的非可见光图像的信噪比较高，保证了针对非可见光图像得到的目标检测结果的准确性。
本申请实施例还提供了一种电子设备，如图12所示，包括处理器301、通信接口302、存储器303和通信总线304，其中，处理器301，通信接口302，存储器303通过通信总线304完成相互间的通信，存储器303，用于存放计算机程序；处理器301，用于执行存储器303上所存放的程序时，实现如下步骤：获 得非可见光图像，以及基于在非可见光图像的采集周期内采集的可见光信号，成像获得的目标图像；检测该非可见光图像中是否存在待检测目标；若存在，则根据待检测目标在该非可见光图像中的第一位置区域，确定待检测目标在可见光图像中的第二位置区域，进而完成基于该可见光图像的目标监控，其中，该可见光图像为基于目标图像所确定的图像。
作为本申请实施例的一种可选的实现方式，所述可见光图像为所述目标图像；所述根据所述待检测目标在所述非可见光图像中的第一位置区域，确定所述待检测目标在所述可见光图像中的第二位置区域的步骤，包括：以所述可见光图像为基准，对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域，并将所述可见光图像中与配准后的第一位置区域对应的位置区域作为所述可见光图像中待检测目标的第二位置区域。
作为本申请实施例的一种可选的实现方式，所述以所述可见光图像为基准，对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域的步骤，包括：以所述可见光图像为基准，按照如下公式对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域：
Figure PCTCN2018101671-appb-000019
Figure PCTCN2018101671-appb-000020
其中，(x,y)为配准前的第一位置区域内像素点的坐标，(x1,y1)为配准后的第一位置区域中与(x,y)对应的像素点的坐标，
Figure PCTCN2018101671-appb-000021
为离线标定获取到的仿射变换矩阵。
作为本申请实施例的一种可选的实现方式，所述可见光图像为：以所述 非可见光图像为基准，对所述目标图像进行位置配准后所得到的图像；所述根据所述待检测目标在所述非可见光图像中的第一位置区域，确定所述待检测目标在所述可见光图像中的第二位置区域的步骤，包括：将所述可见光图像中与所述待检测目标在所述非可见光图像中的第一位置区域相同的位置区域确定为：所述待检测目标在所述可见光图像中的第二位置区域。
作为本申请实施例的一种可选的实现方式，所述处理器301还用于实现如下步骤：在所述根据所述待检测目标在所述非可见光图像中的第一位置区域，确定所述待检测目标在所述可见光图像中的第二位置区域的步骤之后，在所述可见光图像中的所述第二位置区域标记所述待检测目标。
作为本申请实施例的一种可选的实现方式，所述处理器301还用于实现如下步骤：在所述在所述可见光图像中的所述第二位置区域标记所述待检测目标的步骤之后，根据所确定出的每个待检测目标所处的第二位置区域，在所述可见光图像中标记出该待检测目标的种类。
作为本申请实施例的一种可选的实现方式，所述处理器301还用于实现如下步骤：当检测到所述非可见光图像中存在待检测目标时，在所述非可见光图像中识别所述待检测目标的待识别属性信息，得到识别结果；相应的，所述处理器301还用于实现如下步骤：在所述在所述可见光图像中的所述第二位置区域标记所述待检测目标的步骤之后，根据所述第二位置区域，在所述可见光图像中标记出针对所述待检测目标的识别结果。
作为本申请实施例的一种可选的实现方式，所述处理器301还用于实现如下步骤：在所述获得非可见光图像，以及基于在所述非可见光图像的采集周期内采集的可见光信号成像获得的目标图像的步骤之后，根据所述非可见光图像，确定补光状态参数；按照所述补光状态参数进行非可见光补光。
关于该方法各个步骤的具体实现以及相关解释内容可以参见上述图1和4所示的方法实施例，在此不做赘述。
上述电子设备提到的通信总线可以是外设部件互连标准(Peripheral Component Interconnect，PCI)总线或扩展工业标准结构(Extended Industry Standard Architecture，EISA)总线等。该通信总线可以分为地址总线、数据 总线、控制总线等。为便于表示，图中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。
通信接口用于上述电子设备与其他设备之间的通信。
存储器可以包括随机存取存储器(Random Access Memory，RAM)，也可以包括非易失性存储器(Non-Volatile Memory，NVM)，例如至少一个磁盘存储器。可选的，存储器还可以是至少一个位于远离前述处理器的存储装置。
上述的处理器可以是通用处理器，包括中央处理器(Central Processing Unit，CPU)、网络处理器(Network Processor，NP)等；还可以是数字信号处理器(Digital Signal Processing，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现场可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。
由上可知，与相关技术相比，本申请实施例提供的方案，利用对非可见光图像进行目标检测得到的检测结果，来确定非可见光图像对应的可见光图像的目标检测结果；低照度情况下，非可见光图像的图像质量高，对非可见光图像进行目标检测得到的检测结果的准确度高，保证了可见光图像的目标检测结果同样具有高的准确度，进而保证了智能监控的效果。
在本申请提供的又一实施例中，还提供了一种计算机可读存储介质，该计算机可读存储介质中存储有指令，当其在计算机上运行时，使得计算机执行时实现如下步骤：获得非可见光图像，以及基于在非可见光图像的采集周期内采集的可见光信号，成像获得的目标图像；检测该非可见光图像中是否存在待检测目标；若存在，则根据待检测目标在该非可见光图像中的第一位置区域，确定待检测目标在可见光图像中的第二位置区域，进而完成基于该可见光图像的目标监控，其中，该可见光图像为基于目标图像所确定的图像。
作为本申请实施例的一种可选的实现方式，所述可见光图像为所述目标图像；所述根据所述待检测目标在所述非可见光图像中的第一位置区域，确定所述待检测目标在所述可见光图像中的第二位置区域的步骤，包括：以所 述可见光图像为基准，对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域，并将所述可见光图像中与配准后的第一位置区域对应的位置区域作为所述可见光图像中待检测目标的第二位置区域。
作为本申请实施例的一种可选的实现方式，所述以所述可见光图像为基准，对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域的步骤，包括：以所述可见光图像为基准，按照如下公式对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域：
Figure PCTCN2018101671-appb-000022
Figure PCTCN2018101671-appb-000023
其中，(x,y)为配准前的第一位置区域内像素点的坐标，(x1,y1)为配准后的第一位置区域中与(x,y)对应的像素点的坐标，
Figure PCTCN2018101671-appb-000024
为离线标定获取到的仿射变换矩阵。
作为本申请实施例的一种可选的实现方式，所述可见光图像为：以所述非可见光图像为基准，对所述目标图像进行位置配准后所得到的图像；所述根据所述待检测目标在所述非可见光图像中的第一位置区域，确定所述待检测目标在所述可见光图像中的第二位置区域的步骤，包括：将所述可见光图像中与所述待检测目标在所述非可见光图像中的第一位置区域相同的位置区域确定为：所述待检测目标在所述可见光图像中的第二位置区域。
作为本申请实施例的一种可选的实现方式，所述计算机程序被处理器执行时还可以实现如下步骤：在所述根据所述待检测目标在所述非可见光图像中的第一位置区域，确定所述待检测目标在所述可见光图像中的第二位置区 域的步骤之后，在所述可见光图像中的所述第二位置区域标记所述待检测目标。
作为本申请实施例的一种可选的实现方式，所述计算机程序被处理器执行时还可以实现如下步骤：在所述在所述可见光图像中的所述第二位置区域标记所述待检测目标的步骤之后，根据所确定出的每个待检测目标所处的第二位置区域，在所述可见光图像中标记出该待检测目标的种类。
作为本申请实施例的一种可选的实现方式，所述计算机程序被处理器执行时还可以实现如下步骤：当检测到所述非可见光图像中存在待检测目标时，在所述非可见光图像中识别所述待检测目标的待识别属性信息，得到识别结果；相应的，所述计算机程序被处理器执行时还可以实现如下步骤：在所述在所述可见光图像中的所述第二位置区域标记所述待检测目标的步骤之后，根据所述第二位置区域，在所述可见光图像中标记出针对所述待检测目标的识别结果。
作为本申请实施例的一种可选的实现方式，所述计算机程序被处理器执行时还可以实现如下步骤：在所述获得非可见光图像，以及基于在所述非可见光图像的采集周期内采集的可见光信号成像获得的目标图像的步骤之后，根据所述非可见光图像，确定补光状态参数；按照所述补光状态参数进行非可见光补光。
由上可知，与相关技术相比，本申请实施例提供的方案，利用对非可见光图像进行目标检测得到的检测结果，来确定非可见光图像对应的可见光图像的目标检测结果；低照度情况下，非可见光图像的图像质量高，对非可见光图像进行目标检测得到的检测结果的准确度高，保证了可见光图像的目标检测结果同样具有高的准确度，进而保证了智能监控的效果。
在本申请提供的又一实施例中，还提供了一种计算机程序产品，当其在计算机上运行时，使得计算机执行上述实施例中任一所述的监控方法。
由上可知，与相关技术相比，本申请实施例提供的方案，利用对非可见光图像进行目标检测得到的检测结果，来确定非可见光图像对应的可见光图 像的目标检测结果；低照度情况下，非可见光图像的图像质量高，对非可见光图像进行目标检测得到的检测结果的准确度高，保证了可见光图像的目标检测结果同样具有高的准确度，进而保证了智能监控的效果。
需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
本说明书中的各个实施例均采用相关的方式描述，各个实施例之间相同相似的部分互相参见即可，每个实施例重点说明的都是与其他实施例的不同之处。尤其，对于装置、系统、电子设备以及计算机可读存储介质实施例而言，由于其基本相似于方法实施例，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。
以上所述仅为本申请的较佳实施例而已，并非用于限定本申请的保护范围。凡在本申请的精神和原则之内所作的任何修改、等同替换、改进等，均包含在本申请的保护范围内。

Claims (34)

  1. 一种监控方法，其特征在于，所述方法包括：
    获得非可见光图像，以及基于在所述非可见光图像的采集周期内采集的可见光信号成像获得的目标图像；
    检测所述非可见光图像中是否存在待检测目标；
    若存在，根据所述待检测目标在所述非可见光图像中的第一位置区域，确定所述待检测目标在可见光图像中的第二位置区域，进而完成基于所述可见光图像的目标监控，其中，所述可见光图像为基于所述目标图像所确定的图像。
  2. 根据权利要求1所述的方法，其特征在于，所述可见光图像为所述目标图像；
    所述根据所述待检测目标在所述非可见光图像中的第一位置区域，确定所述待检测目标在所述可见光图像中的第二位置区域的步骤，包括：
    以所述可见光图像为基准，对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域，并将所述可见光图像中与配准后的第一位置区域对应的位置区域作为所述可见光图像中待检测目标的第二位置区域。
  3. 根据权利要求2所述的方法，其特征在于，所述以所述可见光图像为基准，对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域的步骤，包括：
    以所述可见光图像为基准，按照如下公式对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域：
    Figure PCTCN2018101671-appb-100001
    Figure PCTCN2018101671-appb-100002
    其中，(x,y)为配准前的第一位置区域内像素点的坐标，(x1,y1)为配准后的第一位置区域中与(x,y)对应的像素点的坐标，
    Figure PCTCN2018101671-appb-100003
    为离线标定获取到的仿射变换矩阵。
  4. 根据权利要求1所述的方法，其特征在于，
    所述可见光图像为：以所述非可见光图像为基准，对所述目标图像进行位置配准后所得到的图像；
    所述根据所述待检测目标在所述非可见光图像中的第一位置区域，确定所述待检测目标在所述可见光图像中的第二位置区域的步骤，包括：
    将所述可见光图像中与所述待检测目标在所述非可见光图像中的第一位置区域相同的位置区域确定为：所述待检测目标在所述可见光图像中的第二位置区域。
  5. 根据权利要求1所述的方法，其特征在于，在所述根据所述待检测目标在所述非可见光图像中的第一位置区域，确定所述待检测目标在所述可见光图像中的第二位置区域的步骤之后，所述方法还包括：
    在所述可见光图像中的所述第二位置区域标记所述待检测目标。
  6. 根据权利要求5所述的方法，其特征在于，在所述在所述可见光图像中的所述第二位置区域标记所述待检测目标的步骤之后，所述方法还包括：
    根据所确定出的每个待检测目标所处的第二位置区域，在所述可见光图像中标记出该待检测目标的种类。
  7. 根据权利要求5所述的方法，其特征在于，当检测到所述非可见光图像中存在待检测目标时，所述方法还包括：
    在所述非可见光图像中识别所述待检测目标的待识别属性信息，得到识别结果；
    相应的，在所述在所述可见光图像中的所述第二位置区域标记所述待检 测目标的步骤之后，所述方法还包括：
    根据所述第二位置区域，在所述可见光图像中标记出针对所述待检测目标的识别结果。
  8. 根据权利要求1-7任一项所述的方法，其特征在于，在所述获得非可见光图像，以及基于在所述非可见光图像的采集周期内采集的可见光信号成像获得的目标图像的步骤之后，所述方法还包括：
    根据所述非可见光图像，确定补光状态参数；
    按照所述补光状态参数进行非可见光补光。
  9. 一种监控装置，其特征在于，所述装置包括：
    获得模块，用于获得非可见光图像，以及基于在所述非可见光图像的采集周期内采集的可见光信号成像获得的目标图像；
    检测模块，用于检测所述非可见光图像中是否存在待检测目标；
    第一确定模块，用于在所述检测模块的检测结果为是的情况下，根据所述待检测目标在所述非可见光图像中的第一位置区域，确定所述待检测目标在可见光图像中的第二位置区域，进而完成基于所述可见光图像的目标监控，其中，所述可见光图像为基于所述目标图像所确定的图像。
  10. 根据权利要求9所述的装置，其特征在于，所述可见光图像为所述目标图像；
    所述第一确定模块，具体用于：
    以所述可见光图像为基准，对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域，并将所述可见光图像中与配准后的第一位置区域对应的位置区域作为所述可见光图像中待检测目标的第二位置区域。
  11. 根据权利要求10所述的装置，其特征在于，所述第一确定模块，具体用于：
    以所述可见光图像为基准，按照如下公式对所述待检测目标在所述非可 见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域，并将所述可见光图像中与配准后的第一位置区域对应的位置区域作为所述可见光图像中待检测目标的第二位置区域：
    Figure PCTCN2018101671-appb-100004
    Figure PCTCN2018101671-appb-100005
    其中，(x,y)为配准前的第一位置区域内像素点的坐标，(x1,y1)为配准后的第一位置区域中与(x,y)对应的像素点的坐标，
    Figure PCTCN2018101671-appb-100006
    为离线标定获取到的仿射变换矩阵。
  12. 根据权利要求9所述的装置，其特征在于，
    所述可见光图像为：以所述非可见光图像为基准，对所述目标图像进行位置配准后所得到的图像；
    所述第一确定模块，具体用于：
    将所述可见光图像中与所述待检测目标在所述非可见光图像中的第一位置区域相同的位置区域确定为：所述待检测目标在所述可见光图像中的第二位置区域。
  13. 根据权利要求9所述的装置，其特征在于，所述装置还包括：
    标记模块，用于在确定所述待检测目标在所述可见光图像中的第二位置区域之后，在所述可见光图像中的所述第二位置区域标记所述待检测目标。
  14. 根据权利要求13所述的装置，其特征在于，
    所述标记模块，还用于在所述可见光图像中的所述第二位置区域标记所述待检测目标之后，根据所确定出的每个待检测目标所处的第二位置区域， 在所述可见光图像中标记出该待检测目标的种类。
  15. 根据权利要求13所述的装置，其特征在于，所述装置还包括：
    识别模块，用于当检测到所述非可见光图像中存在待检测目标时，在所述非可见光图像中识别所述待检测目标的待识别属性信息，得到识别结果；
    相应的，所述标记模块，还用于在所述可见光图像中的所述第二位置区域标记所述待检测目标之后，根据所述第二位置区域，在所述可见光图像中标记出针对所述待检测目标的识别结果。
  16. 根据权利要求9-15任一项所述的装置，其特征在于，所述装置还包括：
    第二确定模块，用于在获得非可见光图像，以及基于在所述非可见光图像的采集周期内采集的可见光信号成像获得的目标图像后，根据所述非可见光图像，确定补光状态参数；
    补光模块，用于按照所述补光状态参数进行非可见光补光。
  17. 一种监控系统，其特征在于，所述系统包括目标图像采集设备、非可见光图像采集设备以及图像处理设备；其中：
    所述非可见光图像采集设备，用于非可见光图像，并将所采集的非可见光图像发送给所述图像处理设备；
    所述目标图像采集设备，用于基于在所述非可见光图像的采集周期内采集的可见光信号，成像获得目标图像，并将所述目标图像发送给所述图像处理设备；
    所述图像处理设备，用于接收所述目标图像采集设备发送的所述目标图像以及所述非可见光图像采集设备发送的所述非可见光图像；检测所述非可见光图像中是否存在待检测目标；若存在，根据所述待检测目标在所述非可见光图像中的第一位置区域，确定所述待检测目标在可见光图像中的第二位置区域，进而完成基于所述可见光图像的目标监控，其中，所述可见光图像为基于所述目标图像所确定的图像。
  18. 根据权利要求17所述的系统，其特征在于，
    所述可见光图像为所述目标图像；
    所述图像处理设备根据所述待检测目标在所述非可见光图像中的第一位置区域，确定所述待检测目标在所述可见光图像中的第二位置区域，具体为：
    以所述可见光图像为基准，对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域，并将所述可见光图像中与配准后的第一位置区域对应的位置区域作为所述可见光图像中待检测目标的第二位置区域。
  19. 根据权利要求18所述的系统，其特征在于，所述图像处理设备以所述可见光图像为基准，对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域，具体为：
    以所述可见光图像为基准，按照如下公式对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域：
    Figure PCTCN2018101671-appb-100007
    Figure PCTCN2018101671-appb-100008
    其中，(x,y)为配准前的第一位置区域内像素点的坐标，(x1,y1)为配准后的第一位置区域中与(x,y)对应的像素点的坐标，
    Figure PCTCN2018101671-appb-100009
    为离线标定获取到的仿射变换矩阵。
  20. 根据权利要求17所述的系统，其特征在于，
    所述可见光图像为：以所述非可见光图像为基准，对所述目标图像进行位置配准后所得到的图像；
    所述图像处理设备根据所述待检测目标在所述非可见光图像中的第一位 置区域，确定所述待检测目标在所述可见光图像中的第二位置区域，具体为：
    将所述可见光图像中与所述待检测目标在所述非可见光图像中的第一位置区域相同的位置区域确定为：所述待检测目标在所述可见光图像中的第二位置区域。
  21. 根据权利要求17所述的系统，其特征在于，所述图像处理设备还用于：
    在所述可见光图像中的所述第二位置区域标记所述待检测目标。
  22. 根据权利要求21所述的系统，其特征在于，所述图像处理设备还用于：
    根据所确定出的每个待检测目标所处的第二位置区域，在所述可见光图像中标记出该待检测目标的种类。
  23. 根据权利要求21所述的系统，其特征在于，所述图像处理设备还用于：
    当检测到所述非可见光图像中存在待检测目标时，在所述非可见光图像中识别所述待检测目标的待识别属性信息，得到识别结果；并在所述可见光图像中的所述第二位置区域标记所述待检测目标之后，根据所述第二位置区域，在所述可见光图像中标记出针对所述待检测目标的识别结果。
  24. 根据权利要求17-23任一项所述的系统，其特征在于，所述系统还包括非可见光补光灯，
    所述非可见光图像采集设备，还用于在获得非可见光图像，以及基于在所述非可见光图像的采集周期内采集的可见光信号成像获得的目标图像后，根据所述非可见光图像，确定补光状态参数；将所述补光状态参数发送给所述非可见光补光灯；
    所述非可见光补光灯，用于接收所述非可见光图像采集设备发送的所述补光状态参数；按照所述补光状态参数进行非可见光补光。
  25. 一种电子设备，其特征在于，包括处理器、通信接口、存储器和通信总线，其中，处理器，通信接口，存储器通过通信总线完成相互间的通信；
    存储器，用于存放计算机程序；
    处理器，用于执行存储器上所存放的程序时，实现如下步骤：
    获得非可见光图像，以及基于在所述非可见光图像的采集周期内采集的可见光信号成像获得的目标图像；
    检测所述非可见光图像中是否存在待检测目标；
    若存在，根据所述待检测目标在所述非可见光图像中的第一位置区域，确定所述待检测目标在可见光图像中的第二位置区域，进而完成基于所述可见光图像的目标监控，其中，所述可见光图像为基于所述目标图像所确定的图像。
  26. 根据权利要求25所述的电子设备，其特征在于，所述可见光图像为所述目标图像；
    所述根据所述待检测目标在所述非可见光图像中的第一位置区域，确定所述待检测目标在所述可见光图像中的第二位置区域的步骤，包括：
    以所述可见光图像为基准，对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域，并将所述可见光图像中与配准后的第一位置区域对应的位置区域作为所述可见光图像中待检测目标的第二位置区域。
  27. 根据权利要求26所述的电子设备，其特征在于，所述以所述可见光图像为基准，对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域的步骤，包括：
    以所述可见光图像为基准，按照如下公式对所述待检测目标在所述非可见光图像中的第一位置区域进行位置配准，获得配准后的第一位置区域：
    Figure PCTCN2018101671-appb-100010
    Figure PCTCN2018101671-appb-100011
    其中，(x,y)为配准前的第一位置区域内像素点的坐标，(x1,y1)为配准后的第一位置区域中与(x,y)对应的像素点的坐标，
    Figure PCTCN2018101671-appb-100012
    为离线标定获取到的仿射变换矩阵。
  28. 根据权利要求25所述的电子设备，其特征在于，所述可见光图像为：以所述非可见光图像为基准，对所述目标图像进行位置配准后所得到的图像；
    所述根据所述待检测目标在所述非可见光图像中的第一位置区域，确定所述待检测目标在所述可见光图像中的第二位置区域的步骤，包括：
    将所述可见光图像中与所述待检测目标在所述非可见光图像中的第一位置区域相同的位置区域确定为：所述待检测目标在所述可见光图像中的第二位置区域。
  29. 根据权利要求25所述的电子设备，其特征在于，所述处理器还用于实现如下步骤：
    在所述根据所述待检测目标在所述非可见光图像中的第一位置区域，确定所述待检测目标在所述可见光图像中的第二位置区域的步骤之后，在所述可见光图像中的所述第二位置区域标记所述待检测目标。
  30. 根据权利要求29所述的电子设备，其特征在于，所述处理器还用于实现如下步骤：
    在所述在所述可见光图像中的所述第二位置区域标记所述待检测目标的步骤之后，根据所确定出的每个待检测目标所处的第二位置区域，在所述可见光图像中标记出该待检测目标的种类。
  31. 根据权利要求29所述的电子设备，其特征在于，所述处理器还用于实现如下步骤：
    当检测到所述非可见光图像中存在待检测目标时，在所述非可见光图像中识别所述待检测目标的待识别属性信息，得到识别结果；
    相应的，所述处理器还用于实现如下步骤：
    在所述在所述可见光图像中的所述第二位置区域标记所述待检测目标的步骤之后，根据所述第二位置区域，在所述可见光图像中标记出针对所述待检测目标的识别结果。
  32. 根据权利要求25-31任一项所述的电子设备，其特征在于，所述处理器还用于实现如下步骤：
    在所述获得非可见光图像，以及基于在所述非可见光图像的采集周期内采集的可见光信号成像获得的目标图像的步骤之后，根据所述非可见光图像，确定补光状态参数；
    按照所述补光状态参数进行非可见光补光。
  33. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质内存储有计算机程序，所述计算机程序被处理器执行时实现权利要求1-8任一所述的方法步骤。
  34. 一种计算机程序产品，其特征在于，当其在计算机上运行时，使得计算机执行权利要求1-8任一所述的方法步骤。
PCT/CN2018/101671 2017-09-15 2018-08-22 监控方法、装置、系统、电子设备及计算机可读存储介质 Ceased WO2019052320A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US16/647,348 US11275952B2 (en) 2017-09-15 2018-08-22 Monitoring method, apparatus and system, electronic device, and computer readable storage medium
EP18855830.8A EP3683716A4 (en) 2017-09-15 2018-08-22 SURVEILLANCE PROCESS, APPARATUS AND SYSTEM, ELECTRONIC DEVICE, AND COMPUTER READABLE INFORMATION MEDIA

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710834952.7 2017-09-15
CN201710834952.7A CN109508588A (zh) 2017-09-15 2017-09-15 监控方法、装置、系统、电子设备及计算机可读存储介质

Publications (1)

Publication Number Publication Date
WO2019052320A1 true WO2019052320A1 (zh) 2019-03-21

Family

ID=65723499

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/101671 Ceased WO2019052320A1 (zh) 2017-09-15 2018-08-22 监控方法、装置、系统、电子设备及计算机可读存储介质

Country Status (4)

Country Link
US (1) US11275952B2 (zh)
EP (1) EP3683716A4 (zh)
CN (2) CN115937776A (zh)
WO (1) WO2019052320A1 (zh)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111507327B (zh) * 2020-04-07 2023-04-14 浙江大华技术股份有限公司 一种目标检测方法及装置
CN112399075B (zh) * 2020-10-15 2022-06-07 浙江大华技术股份有限公司 补光灯的控制方法、监控设备、电子装置和存储介质
CN115564691A (zh) * 2022-08-26 2023-01-03 北京电子工程总体研究所 一种红外可见光融合目标检测方法
CN119277023B (zh) * 2024-10-08 2025-09-19 广州市星飞达电子科技有限公司 一种用于夜视观测设备的画面安全监测预警系统

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101383004A (zh) * 2007-09-06 2009-03-11 上海遥薇实业有限公司 一种红外和可见光图像相结合的乘客目标检测方法
WO2009128065A1 (en) * 2008-04-16 2009-10-22 Elbit Systems Ltd. Multispectral enhanced vision system and method for aircraft landing in inclement weather conditions
CN102231205A (zh) * 2011-06-24 2011-11-02 北京戎大时代科技有限公司 一种多模监控装置及方法
CN103745203A (zh) * 2014-01-15 2014-04-23 南京理工大学 基于视觉注意和均值漂移的目标检测与跟踪方法
CN106056594A (zh) * 2016-05-27 2016-10-26 四川桑莱特智能电气设备股份有限公司 一种基于双光谱的可见光图像提取系统与方法
CN106485245A (zh) * 2015-08-24 2017-03-08 南京理工大学 一种基于可见光和红外图像的全天候目标实时跟踪方法

Family Cites Families (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6496594B1 (en) * 1998-10-22 2002-12-17 Francine J. Prokoski Method and apparatus for aligning and comparing images of the face and body from different imagers
US6775404B1 (en) * 1999-03-18 2004-08-10 University Of Washington Apparatus and method for interactive 3D registration of ultrasound and magnetic resonance images based on a magnetic position sensor
US7319805B2 (en) * 2003-10-06 2008-01-15 Ford Motor Company Active night vision image intensity balancing system
US7602942B2 (en) * 2004-11-12 2009-10-13 Honeywell International Inc. Infrared and visible fusion face recognition system
US8531562B2 (en) * 2004-12-03 2013-09-10 Fluke Corporation Visible light and IR combined image camera with a laser pointer
WO2006060746A2 (en) * 2004-12-03 2006-06-08 Infrared Solutions, Inc. Visible light and ir combined image camera with a laser pointer
JP2007045336A (ja) * 2005-08-10 2007-02-22 Sumitomo Electric Ind Ltd 障害物検出システム及び障害物検出方法
US8463006B2 (en) * 2007-04-17 2013-06-11 Francine J. Prokoski System and method for using three dimensional infrared imaging to provide detailed anatomical structure maps
US8093559B1 (en) * 2008-12-02 2012-01-10 Hrl Laboratories, Llc Methods and apparatus for three-color infrared sensors
CN102567707B (zh) * 2010-12-27 2016-10-12 北京北科慧识科技股份有限公司 生物识别传感器及生物识别系统
CN102789114B (zh) * 2011-05-18 2015-07-15 中国科学院微电子研究所 一种可见-红外双通摄像机
CN102542569B (zh) * 2011-12-21 2015-03-11 武汉市兑尔科技有限公司 一种快速图像配准及其标定方法及实现其的系统
EP2820637A1 (en) * 2012-02-28 2015-01-07 Watts, William Henry Weather avoidance tool system
US9275302B1 (en) * 2012-08-24 2016-03-01 Amazon Technologies, Inc. Object detection and identification
WO2014127431A1 (en) * 2013-02-25 2014-08-28 Commonwealth Scientific And Industrial Research Organisation 3d imaging method and system
EP3129954A4 (en) * 2014-04-07 2017-10-18 BAE SYSTEMS Information and Electronic Systems Integration Inc. Contrast based image fusion
US9245196B2 (en) 2014-05-09 2016-01-26 Mitsubishi Electric Research Laboratories, Inc. Method and system for tracking people in indoor environments using a visible light camera and a low-frame-rate infrared sensor
US9785247B1 (en) * 2014-05-14 2017-10-10 Leap Motion, Inc. Systems and methods of tracking moving hands and recognizing gestural interactions
CN106385530B (zh) * 2015-07-28 2022-12-13 杭州海康微影传感科技有限公司 一种双光谱摄像机
CN106488201B (zh) * 2015-08-28 2020-05-01 杭州海康威视数字技术股份有限公司 一种图像信号的处理方法和系统
US9549101B1 (en) * 2015-09-01 2017-01-17 International Business Machines Corporation Image capture enhancement using dynamic control image
CN105203159A (zh) * 2015-10-14 2015-12-30 武汉三江中电科技有限责任公司 单通道可见光与红外图像采集融合监测系统
JP6356925B2 (ja) 2015-10-27 2018-07-11 富士フイルム株式会社 撮像システム、並びにオブジェクト検出装置及びその作動方法
CN205265843U (zh) * 2015-12-16 2016-05-25 广州斯摩莱信息科技有限公司 可见光与红外光双视监控系统
CN105913040B (zh) * 2016-04-27 2019-04-23 重庆邮电大学 暗视觉条件下的双摄像头实时行人检测系统
CN106060364A (zh) * 2016-07-28 2016-10-26 浙江宇视科技有限公司 光学透雾彩色图像采集方法及摄像机
CN106454049B (zh) * 2016-10-28 2021-12-31 无锡豪帮高科股份有限公司 一种基于分区双通滤光片的虹膜识别及拍照二合一摄像模组
CN106446873B (zh) * 2016-11-03 2021-01-26 北京旷视科技有限公司 人脸检测方法及装置
CN108419062B (zh) * 2017-02-10 2020-10-02 杭州海康威视数字技术股份有限公司 图像融合设备和图像融合方法
CN107483717A (zh) * 2017-07-19 2017-12-15 广东欧珀移动通信有限公司 红外补光灯的设置方法及相关产品
CN109461168B (zh) * 2018-10-15 2021-03-16 腾讯科技（深圳）有限公司 目标对象的识别方法和装置、存储介质、电子装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101383004A (zh) * 2007-09-06 2009-03-11 上海遥薇实业有限公司 一种红外和可见光图像相结合的乘客目标检测方法
WO2009128065A1 (en) * 2008-04-16 2009-10-22 Elbit Systems Ltd. Multispectral enhanced vision system and method for aircraft landing in inclement weather conditions
CN102231205A (zh) * 2011-06-24 2011-11-02 北京戎大时代科技有限公司 一种多模监控装置及方法
CN103745203A (zh) * 2014-01-15 2014-04-23 南京理工大学 基于视觉注意和均值漂移的目标检测与跟踪方法
CN106485245A (zh) * 2015-08-24 2017-03-08 南京理工大学 一种基于可见光和红外图像的全天候目标实时跟踪方法
CN106056594A (zh) * 2016-05-27 2016-10-26 四川桑莱特智能电气设备股份有限公司 一种基于双光谱的可见光图像提取系统与方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3683716A4

Also Published As

Publication number Publication date
CN109508588A (zh) 2019-03-22
EP3683716A4 (en) 2020-11-11
US20210034881A1 (en) 2021-02-04
US11275952B2 (en) 2022-03-15
EP3683716A1 (en) 2020-07-22
CN115937776A (zh) 2023-04-07

Similar Documents

Publication Publication Date Title
CN111308448B (zh) 图像采集设备与雷达的外参确定方法及装置
CN104574376B (zh) 拥挤交通中基于双目视觉和激光雷达联合校验的防撞方法
CN107463918B (zh) 基于激光点云与影像数据融合的车道线提取方法
CN111856445B (zh) 一种目标检测方法、装置、设备及系统
CN112069862A (zh) 目标检测方法和装置
WO2020062856A1 (zh) 一种车辆特征获取方法及装置
WO2018120027A1 (zh) 一种障碍物检测方法及装置
CN111738320B (zh) 基于模板匹配的遮挡工件识别方法
CN101933065A (zh) 车辆周围监测装置、车辆、车辆周围监测程序及车辆周围监测方法
CN106462956A (zh) 局部自适应直方图均衡
CN104899554A (zh) 一种基于单目视觉的车辆测距方法
GB2503328A (en) Method and system for determining vehicle speed using contact points between the tyres and the road
JP4263737B2 (ja) 歩行者検知装置
CN105059190B (zh) 基于视觉的汽车开门碰撞预警装置及方法
WO2020173258A1 (zh) 图像识别系统及其方法
CN108645375B (zh) 一种用于车载双目系统快速车辆测距优化方法
US11275952B2 (en) Monitoring method, apparatus and system, electronic device, and computer readable storage medium
CN111144207A (zh) 一种基于多模态信息感知的人体检测和跟踪方法
CN104361569A (zh) 图像拼接的方法及装置
CN103839255A (zh) 视频抠像篡改检测方法及装置
JP6361313B2 (ja) 車両検出方法及び装置
CN110047059A (zh) 图像处理方法、装置、电子设备及可读存储介质
CN108376409A (zh) 一种光场图像配准方法及系统
CN110197484A (zh) 一种面部皮肤色斑检测系统及检测方法
CN111383255A (zh) 图像处理方法、装置、电子设备及计算机可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18855830

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2018855830

Country of ref document: EP

Effective date: 20200415