WO2022007840A1 - 图像融合方法及装置 - Google Patents

图像融合方法及装置 Download PDF

Info

Publication number
WO2022007840A1
WO2022007840A1 PCT/CN2021/104976 CN2021104976W WO2022007840A1 WO 2022007840 A1 WO2022007840 A1 WO 2022007840A1 CN 2021104976 W CN2021104976 W CN 2021104976W WO 2022007840 A1 WO2022007840 A1 WO 2022007840A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
short
focus
telephoto
field
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2021/104976
Other languages
English (en)
French (fr)
Inventor
马士杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Ezviz Software Co Ltd
Original Assignee
Hangzhou Ezviz Software Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Ezviz Software Co Ltd filed Critical Hangzhou Ezviz Software Co Ltd
Priority to EP21837892.5A priority Critical patent/EP4181506A4/en
Priority to US18/003,916 priority patent/US12223622B2/en
Publication of WO2022007840A1 publication Critical patent/WO2022007840A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/95—Computational photography systems, e.g. light-field imaging systems
    • H04N23/958—Computational photography systems, e.g. light-field imaging systems for extended depth of field imaging
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/61—Control of cameras or camera modules based on recognised objects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/95—Computational photography systems, e.g. light-field imaging systems
    • H04N23/951—Computational photography systems, e.g. light-field imaging systems by using two or more images to influence resolution, frame rate or aspect ratio
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10141—Special mode during image acquisition
    • G06T2207/10148—Varying focus
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging

Definitions

  • the present application relates to the technical field of image processing, and in particular, to an image fusion method and apparatus.
  • the embodiments of the present application propose an image fusion method and apparatus, so as to reduce the monitoring cost and improve the monitoring efficiency under the premise of taking into account the monitoring range and clarity.
  • the specific technical solutions are as follows:
  • an embodiment of the present application provides an image fusion method, the method includes: acquiring a first short-focus image and a first telephoto image collected by a short-focus sensor and a telephoto sensor at the same time; the focal length of the telephoto lens, calculate the reduction factor corresponding to the first telephoto image when the size of the same target in the first telephoto image and the first short-focus image is registered; Zoom out to obtain a second telephoto image; according to the relative angle between the current telephoto lens and the short-focus lens, calculate when the second telephoto image is registered with the position of the same target in the first short-focus image.
  • the position in the first short-focus image according to the position of the second telephoto image in the first short-focus image, overlay the second telephoto image on the first short-focus image to obtain a fusion image; wherein the short-focus sensor and the The short focal lens is connected, the telephoto sensor is connected to the telephoto lens, and the short focal sensor and the telephoto sensor rotate with the gimbal at the same time, or the short focal sensor is fixed and only the telephoto sensor rotates with the gimbal.
  • the method further includes: detecting the target in the short-focus image collected by the short-focus sensor and the telephoto image collected by the telephoto sensor in real time, if the target is detected in the short-focus image or the telephoto image target, control the pan/tilt to rotate until the target is in the center of the short-focus image or telephoto image where it is located, control the pan/tilt to stop rotating.
  • the calculation compares the first telephoto image with the first short-focus image.
  • the reduction factor corresponding to the first telephoto image when the size of the same target in the focus image is registered further includes: according to the preset short-focus field of view reduction coefficient, the first short-focus image is subjected to clipping processing, so that the first short-focus image obtained after clipping is processed.
  • the field of view corresponding to the two short-focus images is equal to the field of view corresponding to the first short-focus image multiplied by the short-focus field of view cropping coefficient; the second short-focus image is enlarged to the same size as the first short-focus image to obtain a third short-focus image ;
  • Described according to the focal length of the short-focus lens and the telephoto lens, calculating the first telephoto image and the first short-focus image when the size of the same target is registered, the corresponding reduction factor of the first telephoto image includes: according to the short-focus lens and the focal length of the telephoto lens and the short-focus field of view reduction coefficient, calculate the reduction coefficient corresponding to the first telephoto image when the size of the same target in the first telephoto image and the third short-focus image is registered; The relative angle between the telephoto lens and the short-focus lens, and calculating the position of the second telephoto image in the first short-focus image when the positions of the same target in the second telephoto image and the first
  • the overlaying the second telephoto image on the first short-focus image according to the position of the second telephoto image in the first short-focus image includes: according to the second telephoto image in the third short-focus image position in the image, overlaying the second telephoto image over the third short focus image.
  • the first short-focus image and the first telephoto image collected by the short-focus sensor and the telephoto sensor at the same time Before performing the cropping process on the first short-focus image, it further includes: judging whether the visual field obtained by multiplying the visual field corresponding to the first short-focus image by the short-focus visual field cropping coefficient is smaller than the visual field corresponding to the first long-focus image, and if it is smaller, then directly The first telephoto image is cropped, so that the field of view of the second telephoto image obtained after the cropping is equal to the field of view obtained by multiplying the field of view corresponding to the first short-focus image by the short-focus field of view cropping coefficient, and the second telephoto image is Enlarging to the same size as the first short-focus image to obtain the final fusion image; and, if the field of view corresponding to the first short-focus image is multiplied by the short-focus field of view
  • the method before obtaining the fused image, the method further includes: smoothing the edge of the second telephoto image in the fused image.
  • the method further includes: setting the size of the first transparency image to be equal to the size of the first telephoto image, and setting the width of the transition band W, where the unit of W is pixels; wherein the transition band is located at The four edges of the first transparency image are located within the boundary of the first transparency image; for each transition band at the four edges, the values of each pixel on the transition band are sequentially from the boundary of the first transparency image to the interior of the image.
  • the smoothing the edge of the second telephoto image in the fused image includes: using the reduction coefficient corresponding to the first telephoto image, the transparency image Perform a reduction process to obtain a second transparency image; the center of the second transparency image is overlapped with the center of the second telephoto image in the fusion image, and for each pixel on the fusion image that overlaps with the transition zone of the second transparency image, the The pixel value of this pixel is replaced by: (the pixel value of the pixel on the second transparency image*the pixel value of the pixel on the second telephoto image in the fusion image+(255-the pixel value of the pixel on the second transparency image Pixel value)*The pixel value of the pixel on the third short-focus image that is covered on the fusion image)/255.
  • the value of each pixel on the transition band increases sequentially from the boundary of the first transparency image to the interior of the image, and the variation range is from Starting at 0 and ending at 255 include: for the transition zone at each edge, the pixel value of the pixel point from the boundary of the transition zone to the center position of the transition zone is fitted by the first quadratic function, and The pixel values of the pixel points from the center position of the transition point to the end of the transition zone are fitted by the second unary quadratic function, wherein the coefficient of the quadratic term in the first unary quadratic function is greater than 0, and the second unary quadratic function is used for fitting.
  • the coefficient of the quadratic term of the quadratic function of one variable is less than 0.
  • an embodiment of the present application provides an image fusion method, including: acquiring a first short-focus image and a first telephoto image corresponding to a current moment to be output, wherein the first short-focus image and the first The acquisition time of a telephoto image is the same, and the field of view of the first telephoto image is within the field of view of the first short-focus image; determine whether the field of view corresponding to the current image to be output is larger than the first long-focus image the field of view of the focal image; if so, according to the field of view corresponding to the current image to be output, intercept the image area of the corresponding field of view in the first short-focus image to obtain a second short-focus image; determine the first long focus The field of view of the focal image is in the target area corresponding to the second short focal image; the first long focal image adjusted to the size of the target area is superimposed on the target area of the second short focal image to obtain a fusion image; The fused image is enlarged to a preset resolution
  • an embodiment of the present application provides an image fusion apparatus, the apparatus includes: a memory and a processor that can access the memory, the memory stores an instruction that, when executed by the processor, causes the processor to execute any of the A described image fusion method.
  • an embodiment of the present application provides a binocular long- and short-focus pan-tilt camera, including: a short-focus sensor connected to the short-focus lens, a telephoto sensor connected to the tele-lens, and a pan-tilt, wherein the short-focus sensor is connected to the telephoto lens.
  • the focal sensor and the telephoto sensor rotate with the pan/tilt at the same time, or the short-focus sensor is fixed, and only the telephoto sensor rotates with the pan-tilt; and the short-focus sensor and the telephoto sensor respectively have an interface for communicating with the image fusion device, and respectively pass The respective communication interfaces send the respective collected original short-focus images and original telephoto images to the image fusion device.
  • an embodiment of the present application provides an image fusion system, including: the above-mentioned binocular long-short-focus pan-tilt camera and the above-mentioned image fusion device.
  • the telephoto image is overlaid on the short-focus image, thereby preserving the large field of view of the short-focus lens.
  • the clarity of the fusion image is improved; only one short-focus lens and one long-focus lens are needed, and the data processing amount is small, which reduces the monitoring cost and improves the monitoring efficiency.
  • 1 is an example diagram of the positional relationship between the fields of view of the binocular long and short-focus pan-tilt cameras provided by an embodiment of the present application when the long and short-focus lenses rotate with the pan-tilt at the same time;
  • FIG. 2 is an example diagram of the positional relationship between the fields of view of the binocular long- and short-focus pan-tilt cameras provided by an embodiment of the present application when the short-focus lens is fixed and the telephoto lens is movable;
  • FIG. 3 is a flowchart of an image fusion method provided by an embodiment of the present application.
  • FIG. 5 is a schematic view of field reduction provided by an embodiment of the present application.
  • FIG. 6 is an example diagram of a first transparency image provided by an embodiment of the present application.
  • FIG. 7 is a flowchart of an image fusion method provided by another embodiment of the present application.
  • Fig. 8 is the schematic diagram of the field of view of different sizes
  • FIG. 9 is an example diagram of cropping the field of view of the first short-focus image according to an embodiment of the present application.
  • FIG. 10 is a flowchart of an image fusion method provided by another embodiment of the present application.
  • FIG. 11 is a schematic structural diagram of an image fusion apparatus according to an embodiment of the present application.
  • the embodiment of the present application first proposes a binocular long and short focal pan-tilt camera.
  • the pan-tilt can rotate 360 degrees horizontally and 5 to -85 degrees vertically. It has two lenses and two sensors, and the two lenses are telephoto lenses respectively.
  • the focal length of the telephoto lens is greater than the focal length of the short-focus lens.
  • the focal length of the telephoto lens and the short-focus lens can be customized according to the actual situation.
  • the short-focus lens has a larger field of view and can cover a larger monitoring range.
  • a telephoto lens has a smaller field of view, but higher clarity.
  • the focal length refers to the distance from the center of the lens to the focal point of light gathering.
  • the focal length above 6mm millimeters
  • less than 6mm is called a short focus.
  • the two sensors are a short focal sensor and a telephoto sensor.
  • the short focal sensor is used to receive the light signal collected by the short focal lens
  • the telephoto sensor is used to receive the light signal collected by the long focal lens; the resolution between the short focal sensor and the long focal sensor
  • the rates may be the same or different, and are not specifically limited in this application.
  • the binocular long- and short-focus pan-tilt camera proposed in the embodiment of the present application can be in the form of a fixed short-focus lens and a telephoto lens that rotates with the pan-tilt, or a long- and short-focus lens that rotates with the pan-tilt at the same time but the relative positions remain unchanged. .
  • the long and short focal lenses rotate with the gimbal at the same time, an example of the positional relationship between the two fields of view is shown in Figure 1.
  • the area contained by the large rectangle represents the field of view of the short-focus lens
  • the area contained by the small rectangle represents the field of view of the telephoto lens.
  • the resolution of the long and short focal sensors can be the same or different. In one example, in order to ensure the resolution of the picture of the telephoto lens, the resolution of the telephoto sensor may be 1080P or above.
  • the short-focus lens is fixed and the telephoto lens is movable, an example of the positional relationship between the two fields of view is shown in Figure 2.
  • the area contained in the large rectangle represents the field of view of the short-focus lens
  • the area contained in the small rectangle represents the field of view of the telephoto lens.
  • FIG. 3 is a flowchart of an image fusion method provided by an embodiment of the application, and the specific steps are as follows:
  • Step 301 Acquire a first short-focus image and a first telephoto image collected by the short-focus sensor and the long-focus sensor at the same moment.
  • the short focal sensor is connected to the short focal lens
  • the telephoto sensor is connected to the telephoto lens
  • the short focal sensor and the long focal sensor rotate with the gimbal at the same time, or the short focal sensor is fixed and only the telephoto sensor is attached to the gimbal. rotate.
  • the first short-focus image is an image collected by the short-focus sensor
  • the first long-focus image is an image collected by the telephoto sensor.
  • the image fusion method in the embodiment of the present application may be implemented by an electronic device.
  • the electronic device may be a camera, a hard disk video recorder, a smart phone, a personal computer, a server, or the like.
  • Step 302 Calculate a reduction coefficient corresponding to the first telephoto image when the size of the same target in the first telephoto image is registered with the first short focus image according to the focal length of the short focus lens and the focal length of the telephoto lens.
  • the first telephoto image must be zoomed, so that the size of the target in the zoomed first telephoto image is the same as the size of the target in the short focus image.
  • Step 303 Perform a reduction process on the first telephoto image according to the calculated reduction coefficient to obtain a second telephoto image.
  • the size of the target in the image is proportional to the size of the focal length.
  • the ratio of the focal length of the telephoto lens to the focal length of the short focal lens can be directly calculated to obtain the reduction factor X.
  • the length and width of the first telephoto image are reduced by X times to obtain a second telephoto image.
  • Step 304 Calculate, according to the relative angle between the current telephoto lens and the short-focus lens, the position of the second telephoto image in the first short-focus image when the position of the same target in the second telephoto image and the first short-focus image is registered. Location.
  • the positional relationship between the two fields of view is fixed.
  • the example is shown in Figure 1.
  • the second telephoto lens can be pre-calibrated. The position of the center point of the image in the first short-focus image; then, based on the position of the center point of the second telephoto image in the first short-focus image, and the size of the second telephoto image, determine the position of the second telephoto image in the first short-focus image. position in the first short focus image.
  • the position of the center point of the second telephoto image in the first short-focus image at different relative angles can be pre-calibrated; then based on the current telephoto lens The relative angle with the short-focus lens, determine the position of the center point of the current second telephoto image in the first short-focus image, and combine the size of the second telephoto image to determine the second telephoto image in the first short-focus image in the location.
  • Step 305 According to the position of the second telephoto image in the first short focus image, overlay the second telephoto image on the first short focus image to obtain a fusion image.
  • the telephoto image is overlaid on the short-focus image, so as to preserve the large field of view of the short-focus lens and improve the clarity of the fused image;
  • the target is detected in the short-focus image collected by the short-focus sensor and the telephoto image collected by the telephoto sensor in real time, and if the target is detected in the short-focus image or the telephoto image, the pan/tilt is controlled to rotate, Control the pan/tilt to stop rotating until the target is in the center of the short-focus image or the telephoto image where it is located.
  • the target when the target is detected, by rotating the pan/tilt head, the target can be positioned at the center of the short-focus or long-focus lens, thereby improving the monitoring effect of the final fused image.
  • FIG. 4 is a flowchart of an image fusion method provided by another embodiment of the present application, and the specific steps are as follows:
  • Step 401 Acquire a first short-focus image and a first telephoto image collected by the short-focus sensor and the telephoto sensor at the same moment.
  • the short focal sensor is connected to the short focal lens
  • the telephoto sensor is connected to the telephoto lens
  • the short focal sensor and the long focal sensor rotate with the gimbal at the same time, or the short focal sensor is fixed and only the telephoto sensor is attached to the gimbal. rotate.
  • Step 402 According to the preset short-focus field of view cropping coefficient, perform cropping processing on the first short-focus image, so that the field of view corresponding to the second short-focus image obtained after the cropping is equal to the field of view corresponding to the first short-focus image multiplied by the short-focus image.
  • the field of view clipping factor According to the preset short-focus field of view cropping coefficient, perform cropping processing on the first short-focus image, so that the field of view corresponding to the second short-focus image obtained after the cropping is equal to the field of view corresponding to the first short-focus image multiplied by the short-focus image.
  • the short-focus field of view reduction factor is set by the user or the like. Since the short-focus lens has a larger field of view, in order to make the target in the short-focus lens clearer, the field of view of the short-focus lens can be cropped first, that is, the first short-focus image is cropped, and the edge part is cropped, and then The second short-focus image obtained after the cropping is enlarged, so that the target is enlarged and the definition of the image is improved.
  • the short-focus field of view reduction factor can be expressed by the ratio of the area of the large rectangle to the area of the small rectangle.
  • it can also be The ratio of the area of the small rectangle to the area of the large rectangle is expressed, which is not limited in this application.
  • Step 403 Enlarging the second short-focus image to the same size as the first short-focus image to obtain a third short-focus image.
  • Step 404 Calculate the reduction factor corresponding to the first telephoto image when the size of the same target in the first telephoto image and the third short-focus image is registered according to the focal lengths of the short-focus lens and the telephoto lens and the short-focus field of view reduction coefficient .
  • Step 405 Perform a reduction process on the first telephoto image according to the reduction coefficient calculated in step 404 to obtain a second telephoto image.
  • Step 406 According to the relative angle between the current telephoto lens and the short-focus lens and the short-focus field of view cropping coefficient, calculate when the second telephoto image and the third short-focus image are registered at the position of the same target in the first telephoto image. Three positions in the short-focus image.
  • Step 407 According to the position of the second telephoto image in the third short focus image, overlay the second telephoto image on the third short focus image to obtain a fusion image.
  • the first short-focus image is firstly cropped, and then the cropped second short-focus image is enlarged, thereby improving the participation in fusion.
  • the clarity of the third short-focus image is improved, that is, the clarity of the final fused image is improved.
  • the field of view obtained by multiplying the field of view corresponding to the first short-focus image by the short-focus field of view cropping coefficient may be smaller than or equal to the field of view of the first telephoto image.
  • the first short-focus image can be directly discarded , directly processing the first telephoto image to obtain the final fusion image. details as follows:
  • step 402 Before executing step 402, first determine whether the field of view obtained by multiplying the field of view corresponding to the first short-focus image by the short-focus field of view cropping coefficient is smaller than the field of view corresponding to the first telephoto image, and if it is smaller, directly crop the first telephoto image , so that the field of view of the second telephoto image obtained after cropping is equal to the field of view obtained by multiplying the field of view corresponding to the first short-focus image by the short-focus field of view reduction coefficient, and the second telephoto image is enlarged to the same size as the first short-focus image. are the same size to obtain the final fused image;
  • the first telephoto image is directly used as the final fusion image
  • step 402 If the field of view corresponding to the first short-focus image multiplied by the short-focus field-of-view cropping coefficient is larger than the field of view corresponding to the first telephoto image, step 402 and subsequent steps are performed.
  • the field of view obtained by multiplying the field of view corresponding to the first short-focus image by the short-focus field-of-view cropping coefficient is smaller than or equal to the field of view corresponding to the first telephoto image, and only the first telephoto image with good clarity is used to construct the fusion The image is sufficient, and the clarity of the fused image is guaranteed on the premise of meeting the visual field requirements of the fused image.
  • the size of the first transparency image is set equal to the size of the first telephoto image, and the width W of the transition band is set, and the unit of W is pixels; wherein, the transition band is located at four edges of the first transparency image , and within the boundaries of the first transparency image;
  • the value of each pixel on the transition band increases sequentially from the boundary of the first transparency image to the inside of the image, and the variation range starts from 0 and ends at 255.
  • FIG. 6 is an example diagram of a first transparency image.
  • performing smooth processing on the edge of the second telephoto image in the fusion image includes:
  • the transparency image is reduced to obtain a second transparency image; the center of the second transparency image and the center of the second telephoto image in the fusion image are Coincidence, for each pixel on the fusion image that coincides with the transition band of the second transparency image, replace the pixel value of the pixel with: (the pixel value of the pixel on the second transparency image * the second longest pixel in the fusion image The pixel value of the pixel point on the focus image + (255-the pixel value of the pixel point on the second transparency image)*the pixel value of the pixel point on the third short-focus image that is covered on the fusion image)/255.
  • each transition band at the four edges of the transparency image as follows: the value of each pixel on the transition band increases sequentially from the boundary of the first transparency image to the interior of the image, and the variation range is from 0 to 255 ends, and associates the pixel values of the pixels on the transition zone with the weights of the corresponding pixels on the second telephoto image and the third short-focus image, and smoothes the junction of the long- and short-focus images on the fused image. , which optimizes the display effect of the fused image;
  • a first transparency image with the same size as the first telephoto image is pre-calculated, and then the first transparency image is reduced according to actual needs when the long and short focus images are fused, thereby reducing the image size.
  • the time-consuming smoothing process in the fusion process speeds up the generation of the final fusion image, thereby improving the monitoring efficiency.
  • the value of each pixel on the transition band increases sequentially from the border of the first transparency image to the interior of the image, and the change range is from 0 to 255 Closing includes:
  • the first quadratic function is used to fit the pixel values of the pixel points from the boundary of the transition zone to the central position of the transition zone, from the central position of the transition point to the transition zone
  • the pixel value of the pixel point at the end of the band is fitted by a second quadratic function, wherein the coefficient of the quadratic term in the first quadratic function is greater than 0, and the coefficient of the quadratic term of the second quadratic function is less than 0.
  • x1 is the coordinate of the starting point of the transition zone on the side of the transition zone near the border of the transparency image
  • x2 is the coordinate of the end point of the transition zone on the side of the transition zone near the inner side of the transparency image
  • x3 is the coordinate of the center point of the transition zone.
  • the embodiment of the present application also provides an image fusion method, including:
  • Step 1 Acquire the first short-focus image and the first telephoto image corresponding to the current time to be output, wherein the first short-focus image and the first telephoto image are collected at the same time, and the first long-focus image is collected at the same time.
  • the field of view of the focus image is within the field of view of the first short focus image.
  • Step 2 judging whether the field of view corresponding to the current image to be output is larger than the field of view of the first telephoto image.
  • Step 3 if yes, according to the field of view corresponding to the current image to be output, intercept the image area of the corresponding field of view from the first short-focus image to obtain a second short-focus image.
  • Step 4 The first long-focus image adjusted to the size of the target area is superimposed on the target area of the second short-focus image to obtain a fusion image.
  • Step 5 Enlarging the fusion image to a preset resolution size and outputting it.
  • the lens of the control device when a specified target is detected in the short-focus image, the lens of the control device is rotated to the position where the specified target is detected, and the electronic zoom function is enabled to gradually reduce the angle of view (field of view) of the short-focus image, and the The telephoto image is spliced into the cropped image. Since the angle of view of the short-focus image collected in the previous section is not lost, the detection and recording can save the field of view before zooming.
  • this implementation method belongs to electronic zooming. , the calculation is fast, the center position comes from the telephoto image, the picture is clear, and there is no refocusing process and loss of angle of view of the zoom lens.
  • the field of view can be determined according to the center of the image and the zoom size.
  • the method further includes: if not, according to the current to-be-outputted image For the field of view corresponding to the image, the image area of the corresponding field of view is intercepted in the first telephoto image to obtain a second telephoto image; the second telephoto image is enlarged to a preset resolution size and output.
  • the magnification is greater than the magnification of the telephoto image, that is, when the field of view corresponding to the current image to be output is smaller than the field of view of the telephoto image
  • the second telephoto image is cut out from the first telephoto image, and the The second telephoto image is enlarged to a preset resolution size and then output.
  • the field of view corresponding to the current image to be output is equal to the field of view of the telephoto image
  • the first telephoto image is not needed to be intercepted, and the first telephoto image is directly enlarged to a preset resolution size and then output.
  • the method further includes: setting The size of the first transparency image is determined to be equal to the size of the first telephoto image, and the width of the transition band W is set, and the unit of W is pixels; wherein, the transition band is located at the four edges of the first transparency image, and is located in the first transparency image. Within the boundary; for each transition band at the four edges, the value of each pixel on the transition band increases sequentially from the boundary of the first transparency image to the interior of the image, and the variation range is from 0 to 255; The center of a transparency image coincides with the center of the first telephoto image.
  • the value of each pixel on the transition band increases sequentially from the boundary of the first transparency image to the interior of the image, and the variation range is from Starting at 0 and ending at 255 include: for the transition zone at each edge, the pixel value of the pixel point from the boundary of the transition zone to the center position of the transition zone is fitted by the first quadratic function, and The pixel values of the pixel points from the center position of the transition point to the end of the transition zone are fitted by the second unary quadratic function, wherein the coefficient of the quadratic term in the first unary quadratic function is greater than 0, and the second unary quadratic function is used for fitting.
  • the coefficient of the quadratic term of the quadratic function of one variable is less than 0.
  • FIG. 7 is a flowchart of an image fusion method provided by another embodiment of the present application, and the specific steps are as follows:
  • Step 701 Detect the target in the first short-focus image collected by the short-focus sensor and the first telephoto image collected by the long-focus sensor in real time. If the target is detected in the first short-focus image or the first telephoto image, control The gimbal rotates until the target is located in the center of the first short-focus image or the first telephoto image, and the gimbal is controlled to stop rotating.
  • Step 702 Perform the following steps 703-708 for the two frames of images collected at the same moment: the first short-focus image and the first telephoto image:
  • Step 703 Obtain the currently set short-focus field of view reduction coefficient ZoomRatioc and the pre-calculated ZoomRatios when the field of view of the short-focus lens is reduced to the same as the field of view of the telephoto lens, and determine whether ZoomRatioc ⁇ ZoomRatios is established, and if so, execute Step 704.
  • ZoomRatio can be expressed as the ratio of two different fields of view.
  • ZoomRatioc original field of view of the short focal lens/field of view after cutting the field of view of the short focal lens.
  • ZoomRatios original field of view of the short focal lens/original field of view of the telephoto lens, for example: in FIG.
  • Step 704 Crop the first short-focus image according to the ZoomRatioc, and then enlarge the cropped second short-focus image to be the same size as the first short-focus image to obtain a third short-focus image.
  • the second short-focus image is enlarged to the same size as the first short-focus image.
  • the second short-focus image shown in FIG. 9 is enlarged to the same size as the first short-focus image shown in FIG. 9 .
  • the amplification algorithm is not limited in this embodiment.
  • Step 705 Calculate the zoom ratio ZoomRatiom corresponding to the first telephoto image when the size of the same target in the first telephoto image and the third short focus image is registered according to the focal lengths of the short focus lens and the telephoto lens and the ZoomRatioc.
  • the registration includes two aspects: size registration and position registration , that is, after the first telephoto image is fused to the third short-focus image, the size and position of an object on the first telephoto image are completely registered with the size and position of the target on the third short-focus image .
  • the reduction coefficient ZoomRatiom is related to the focal lengths of the short-focus lens and the telephoto lens.
  • the first short-focus image is cropped and enlarged, and therefore, the reduction factor ZoomRatiom is also related to ZoomRatioc.
  • Step 706 Perform a reduction process on the first telephoto image according to the ZoomRatiom to obtain a second telephoto image.
  • Step 707 According to the current relative angle of the telephoto lens and the short-focus lens and the ZoomRatioc, calculate when the position of the same target in the second telephoto image and the third short-focus image is registered when the second telephoto image is in the third short-focus image. in the location.
  • Step 708 According to the position of the second telephoto image in the third short-focus image, overlay the second telephoto image on the third short-focus image to obtain a fusion image, and output the fusion image to the display screen.
  • Overlay means to replace the third short focus image at the corresponding position with the second long focus image.
  • Step 1001 Detect the target in the first short-focus image collected by the short-focus sensor and the first telephoto image collected by the long-focus sensor in real time. If the target is detected in the first short-focus image or the first telephoto image, control The gimbal rotates until the target is located in the center of the first short-focus image or the first telephoto image, and the gimbal is controlled to stop rotating.
  • Step 1002 Perform the following steps 1003-1005 for the two frames of images collected at the same moment: the first short-focus image and the first telephoto image:
  • Step 1003 Obtain the currently set short-focus field of view reduction coefficient ZoomRatioc and the pre-calculated ZoomRatios when the field of view of the short-focus lens is reduced to the same as the field of view of the telephoto lens, and determine whether ZoomRatioc ⁇ ZoomRatios is established, and if not, execute Step 1004.
  • ZoomRatio can be expressed as the ratio of two different fields of view.
  • ZoomRatioc original field of view of the short focal lens/field of view after cutting the field of view of the short focal lens
  • ZoomRatios original field of view of the short focal lens/original field of view of the telephoto lens.
  • Step 1004 Crop the first telephoto image so that the size of the second telephoto image obtained after the cropping is equal to the size of the cropped first short-focus image using ZoomRatioc, and then enlarge the second telephoto image to A third telephoto image is obtained with the same size as the first short focus image.
  • Step 1005 Output the third telephoto image to the display screen as a fusion image.
  • FIG. 11 is a schematic structural diagram of an image fusion apparatus provided by an embodiment of the present application.
  • the apparatus mainly includes: a memory 1101 and a processor 1102 that can access the memory 1101 .
  • the memory 1101 stores instructions that, when executed by the processor 1102 , cause the
  • the processor 1102 executes the steps of the method described in steps 301-305, or steps 401-407, or steps 701-708, or steps 1001-1005.
  • Embodiments of the present application further provide a long- and short-focus binocular pan-tilt camera, including: a short-focus sensor connected to the short-focus lens, a telephoto sensor connected to the tele-lens, and a pan/tilt, wherein the short-focus sensor and the long-focus sensor
  • the focal sensor rotates with the pan/tilt at the same time, or the short-focus sensor is fixed and only the telephoto sensor rotates with the pan-tilt; and the short-focus sensor and the telephoto sensor respectively have interfaces for communicating with the image fusion device, and communicate through their respective communication interfaces
  • the original short-focus image and the original telephoto image collected respectively are sent to the image fusion device.
  • Embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any one of the image fusion methods described in the present application is implemented.
  • a computer program product including instructions is also provided, which, when running on a computer, enables the computer to execute any of the image fusion methods described in the present application.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Studio Devices (AREA)
  • Image Processing (AREA)

Abstract

一种图像融合方法及装置，获取短焦传感器和长焦传感器在同一时刻采集的第一短焦图像和第一长焦图像；根据短焦镜头和长焦镜头的焦距，计算将第一长焦图像与第一短焦图像中的同一目标的大小配准时第一长焦图像对应的缩小系数；根据所述缩小系数，对第一长焦图像进行缩小处理，得到第二长焦图像；根据当前长焦镜头与短焦镜头的相对角度，计算当第二长焦图像和第一短焦图像中的同一目标的位置配准时第二长焦图像在第一短焦图像中的位置；根据第二长焦图像在第一短焦图像中的位置，将第二长焦图像覆盖到第一短焦图像上，得到融合图像。本申请实施例在兼顾监控范围和清晰度的前提下，降低了监控成本并提高了监控效率。

Description

图像融合方法及装置
本申请要求于2020年07月08日提交中国专利局、申请号为202010649911.2申请名称为“图像融合方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理技术领域，尤其涉及图像融合方法及装置。
背景技术
在监控范围较大的视频监控场景中，为了扩大对监控场景的覆盖范围以及同时对多台摄像机的画面进行监控及回放。现有技术中出现了多路摄像机联动构成的成像系统。但是，该系统架设复杂，需要十台摄像机联合输入，中间需要另外接入图像合成模块，处理数据量巨大，整体成本很高，且监控角度有限。
发明内容
本申请实施例提出图像融合方法及装置，以在兼顾监控范围和清晰度的前提下，降低监控成本并提高监控效率。具体技术方案如下：
第一方面，本申请实施例提供了一种图像融合方法，该方法包括：获取短焦传感器和长焦传感器在同一时刻采集的第一短焦图像和第一长焦图像；根据短焦镜头和长焦镜头的焦距，计算将第一长焦图像与第一短焦图像中的同一目标的大小配准时第一长焦图像对应的缩小系数；根据所述缩小系数，对第一长焦图像进行缩小处理，得到第二长焦图像；根据当前长焦镜头与短焦镜头的相对角度，计算当第二长焦图像和第一短焦图像中的同一目标的位置配准时第二长焦图像在第一短焦图像中的位置；根据第二长焦图像在第一短焦图像中的位置，将第二长焦图像覆盖到第一短焦图像上，得到融合图像；其中，短焦传感器与短焦镜头相接，长焦传感器与长焦镜头相接，且短焦传感器和长焦传感器同时随云台旋转，或者，短焦传感器固定、只有长焦传感器随云台旋转。
在一种可能的实施方式中，所述方法进一步包括：实时在短焦传感器采集的短焦图像和长焦传感器采集的长焦图像中检测目标，若在短焦图像或长 焦图像中检测到目标，则控制云台旋转，直到目标位于其所在的短焦图像或长焦图像的中心时，控制云台停止旋转。
在一种可能的实施方式中，所述获取短焦传感器和长焦传感器在同一时刻采集的第一短焦图像和第一长焦图像之后、所述计算将第一长焦图像与第一短焦图像中的同一目标的大小配准时第一长焦图像对应的缩小系数之前进一步包括：根据预先设定的短焦视野裁减系数，对第一短焦图像进行裁减处理，使得裁减后得到的第二短焦图像对应的视野等于第一短焦图像对应的视野乘以所述短焦视野裁减系数；将第二短焦图像放大到与第一短焦图像的大小相同，得到第三短焦图像；所述根据短焦镜头和长焦镜头的焦距，计算将第一长焦图像与第一短焦图像中的同一目标的大小配准时第一长焦图像对应的缩小系数包括：根据短焦镜头和长焦镜头的焦距以及所述短焦视野裁减系数，计算将第一长焦图像与第三短焦图像中的同一目标的大小配准时第一长焦图像对应的缩小系数；所述根据当前长焦镜头与短焦镜头的相对角度，计算当第二长焦图像和第一短焦图像中的同一目标的位置配准时第二长焦图像在第一短焦图像中的位置包括：根据当前长焦镜头与短焦镜头的相对角度以及所述短焦视野裁减系数，计算当第二长焦图像和第三短焦图像中的同一目标的位置配准时第二长焦图像在第三短焦图像中的位置；所述根据第二长焦图像在第一短焦图像中的位置，将第二长焦图像覆盖到第一短焦图像上包括：根据第二长焦图像在第三短焦图像中的位置，将第二长焦图像覆盖到第三短焦图像上。
在一种可能的实施方式中，所述获取短焦传感器和长焦传感器在同一时刻采集的第一短焦图像和第一长焦图像之后、所述根据预先设定的短焦视野裁减系数，对第一短焦图像进行裁减处理之前进一步包括：判断第一短焦图像对应的视野乘以所述短焦视野裁减系数得到的视野是否小于第一长焦图像对应的视野，若小于，则直接对第一长焦图像进行裁减，以使得裁减后得到的第二长焦图像的视野等于第一短焦图像对应的视野乘以所述短焦视野裁减系数得到的视野，将第二长焦图像放大到与第一短焦图像的大小相同，得到最终的融合图像；且，若第一短焦图像对应的视野乘以所述短焦视野裁减系数得到的视野等于第一长焦图像对应的视野，则直接将第一长焦图像作为最 终的融合图像；若第一短焦图像对应的视野乘以所述短焦视野裁减系数得到的视野大于第一长焦图像对应的视野，则执行所述根据预先设定的短焦视野裁减系数，对第一短焦图像进行裁减处理的动作。
在一种可能的实施方式中，所述得到融合图像之前进一步包括：对融合图像中第二长焦图像的边缘进行平滑处理。
在一种可能的实施方式中，所述方法进一步包括：设定第一透明度图像的大小等于第一长焦图像的大小，设定过渡带宽度W，W的单位是像素；其中，过渡带位于第一透明度图像的四个边缘处，且位于第一透明度图像边界内；对于四个边缘处的每条过渡带，过渡带上各个像素点的取值从第一透明度图像边界处向图像内部依次变大，且变化范围为从0开始至255结束；且，所述对融合图像中第二长焦图像的边缘进行平滑处理包括：采用所述第一长焦图像对应的缩小系数，对透明度图像进行缩小处理，得到第二透明度图像；将第二透明度图像的中心与融合图像中第二长焦图像的中心重合，对于融合图像上与第二透明度图像的过渡带重合的每个像素点，将该像素点的像素值替换为：(第二透明度图像上该像素点的像素值*融合图像中第二长焦图像上该像素点的像素值+(255-第二透明度图像上该像素点的像素值)*融合图像上被覆盖掉的第三短焦图像上该像素点的像素值)/255。
在一种可能的实施方式中，所述对于四个边缘处的每条过渡带，过渡带上各个像素点的取值从第一透明度图像边界处向图像内部依次变大，且变化范围为从0开始至255结束包括：对于每个边缘处的过渡带，从该过渡带的边界处到该过渡带的中心位置处的像素点的像素值采用第一一元二次函数拟合，从该过渡点的中心位置处到该过渡带的结束处的像素点的像素值采用第二一元二次函数拟合，其中，第一一元二次函数中的二次项系数大于0，第二一元二次函数的二次项系数小于0。
第二方面，本申请实施例提供了一种图像融合方法，包括：获取当前待输出时刻对应的第一短焦图像及第一长焦图像，其中，所述第一短焦图像与所述第一长焦图像的采集时刻相同，且所述第一长焦图像的视野范围在所述第一短焦图像的视野范围内；判断当前待输出图像所对应的视野范围是否大于所述第一长焦图像的视野范围；若是，则按照当前待输出图像所对应的视 野范围，在所述第一短焦图像中截取相应视野范围的图像区域，得到第二短焦图像；确定所述第一长焦图像的视野范围在所述第二短焦图像中所对应的目标区域；将调整为目标区域大小的第一长焦图像叠加到所述第二短焦图像的目标区域上，得到融合图像；将所述融合图像放大为预设分辨率大小并输出。
第三方面，本申请实施例提供了一种图像融合装置，该装置包括：存储器以及可访问存储器的处理器，该存储器存储指令，该指令在由处理器执行时使得处理器执行本申请中任一所述的图像融合方法。
第四方面，本申请实施例提供了一种双目长短焦云台摄像机，包括：与短焦镜头相接的短焦传感器、与长焦镜头相接的长焦传感器以及云台，其中，短焦传感器和长焦传感器同时随云台旋转，或者，短焦传感器固定、只有长焦传感器随云台旋转；且，短焦传感器和长焦传感器分别具有与图像融合装置通信的接口，并分别通过各自的通信接口将各自采集的原始短焦图像和原始长焦图像发送给图像融合装置。
第五方面，本申请实施例提供了一种图像融合系统，包括：如上所述的双目长短焦云台摄像机和如上所述的图像融合装置。
本申请实施例中，对双目长、短焦摄像机采集的长焦图像与短焦图像进行目标配准后，将长焦图像覆盖到短焦图像上，从而在保留了短焦镜头的大视野的同时，提高了融合图像的清晰度；只需一个短焦镜头和一个长焦镜头，且数据处理量较小，降低了监控成本，且提高了监控效率。
附图说明
为了更清楚地说明本申请实施例和现有技术的技术方案，下面对实施例和现有技术中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本申请一实施例提供的双目长短焦云台摄像机在长、短焦镜头同时随云台旋转时，二者的视野的位置关系示例图；
图2为本申请一实施例提供的双目长短焦云台摄像机在短焦镜头固定，长焦镜头可动时，二者的视野的位置关系示例图；
图3为本申请一实施例提供的图像融合方法流程图；
图4为本申请另一实施例提供的图像融合方法流程图；
图5为本申请一实施例提供的视野裁减示意图；
图6为本申请一实施例提供的第一透明度图像的示例图；
图7为本申请又一实施例提供的图像融合方法流程图；
图8为不同大小的视野的示意图；
图9为本申请实施例提供的对第一短焦图像进行视野裁减的示例图；
图10为本申请又一实施例提供的图像融合方法流程图；
图11为本申请一实施例提供的图像融合装置的结构示意图。
具体实施方式
为使本申请的目的、技术方案、及优点更加清楚明白，以下参照附图并举实施例，对本申请进一步详细说明。显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
为了兼顾监控范围和清晰度，并降低监控成本和提高监控效率，本申请实施例给出了如下解决方案：
本申请实施例首先提出了一种双目长短焦云台摄像机，云台可以水平360度旋转，垂直5到-85度旋转，有两个镜头和两颗传感器，两个镜头分别为长焦镜头和短焦镜头，长焦镜头的焦距大于短焦镜头的焦距，长焦镜头及短焦镜头的焦距可以根据实际情况自定义选取，短焦镜头的视野较大，能够覆盖较大的监控范围，长焦镜头的视野较小，但是清晰度较高。其中，焦距指从透镜中心到光聚集之焦点的距离，一般焦距6mm(毫米)以上称为长焦，小于6mm称为短焦。
两颗传感器分别为短焦传感器和长焦传感器，短焦传感器用于接收短焦镜头采集的光信号，长焦传感器用于接收长焦镜头采集的光信号；短焦传感器和长焦传感器的分辨率可以相同也可以不同，本申请中不做具体限定。
本申请实施例提出的双目长短焦云台摄像机，其形态可以是短焦镜头固定、长焦镜头随云台旋转，也可以是长、短焦镜头同时随云台旋转但相对位 置保持不变。
本申请实施例中要求：长焦镜头的画面始终处于短焦镜头的视野中。其中：
一、若长、短焦镜头同时随云台旋转，则二者的视野的位置关系示例如图1所示。其中，大矩形包含的区域表示短焦镜头的视野，小矩形包含的区域表示长焦镜头的视野。长、短焦传感器的分辨率可以是相同的，也可以是不同的。一个例子中，为了保证长焦镜头的画面的分辨率，长焦传感器的分辨率可以为1080P及以上。
二、若短焦镜头固定、长焦镜头可动，则二者的视野的位置关系示例如图2所示。其中，大矩形包含的区域表示短焦镜头的视野，小矩形包含的区域表示长焦镜头的视野，云台旋转时长焦镜头的画面可以在短焦镜头的画面内沿着箭头方向左右上下移动。
图3为本申请一实施例提供的图像融合方法流程图，其具体步骤如下：
步骤301：获取短焦传感器和长焦传感器在同一时刻采集的第一短焦图像和第一长焦图像。其中，短焦传感器与短焦镜头相接，长焦传感器与长焦镜头相接，且短焦传感器和长焦传感器同时随云台旋转，或者，短焦传感器固定、只有长焦传感器随云台旋转。第一短焦图像为短焦传感采集的图像，第一长焦图像为长焦传感器采集的图像。
本申请实施例的图像融合方法可以通过电子设备实现，具体的，该电子设备可以为摄像机、硬盘录像机、智能手机、个人电脑或服务器等。
步骤302：根据短焦镜头的焦距和长焦镜头的焦距，计算将第一长焦图像与第一短焦图像中的同一目标的大小配准时第一长焦图像对应的缩小系数。
即，由于长焦镜头的视野小于短焦镜头的视野，因此，对于同一目标，其在长焦图像中的大小是大于在短焦图像中的大小的，要将长焦图像与短焦图像融合，必须将第一长焦图像进行缩放，以使得缩放后的第一长焦图像中的目标的大小与该目标在短焦图像中的大小相同。
步骤303：根据计算得到的缩小系数，对第一长焦图像进行缩小处理，得到第二长焦图像。
图像中目标的大小与焦距的大小成正比，一个例子中，可以直接计算长焦镜头的焦距与短焦镜头的焦距的比值，得到缩小系数X。将第一长焦图像的长及宽均缩小X倍，得到第二长焦图像。
步骤304：根据当前长焦镜头与短焦镜头的相对角度，计算当第二长焦图像和第一短焦图像中的同一目标的位置配准时第二长焦图像在第一短焦图像中的位置。
一个例子中，针对长焦镜头及短焦镜头同时随云台旋转的情况，二者的视野的位置关系是固定的，示例如图1所示，此种情况下可以预先标定出第二长焦图像的中心点在第一短焦图像中的位置；然后基于第二长焦图像的中心点在第一短焦图像中的位置，以及第二长焦图像的大小，确定第二长焦图像在第一短焦图像中的位置。
一个例子中，针对长焦镜头可动、短焦镜头固定的情况，可以预先标定出不同相对角度下第二长焦图像的中心点在第一短焦图像中的位置；然后基于当前长焦镜头与短焦镜头的相对角度，确定当前第二长焦图像的中心点在第一短焦图像中的位置，并结合第二长焦图像的大小，确定第二长焦图像在第一短焦图像中的位置。
步骤305：根据第二长焦图像在第一短焦图像中的位置，将第二长焦图像覆盖到第一短焦图像上，得到融合图像。
上述实施例的有益技术效果如下：
一、对双目长、短焦摄像机采集的长焦图像与短焦图像进行目标配准后，将长焦图像覆盖到短焦图像上，从而在保留了短焦镜头的大视野的同时，提高了融合图像的清晰度；
二、只需一个短焦镜头和一个长焦镜头，且数据处理量较小，便于嵌入式系统的实时实现，降低了监控成本，且提高了监控效率；
三、不需进行硬件变焦和硬件对焦，实现简便。
一可选实施例中，实时在短焦传感器采集的短焦图像和长焦传感器采集的长焦图像中检测目标，若在短焦图像或长焦图像中检测到目标，则控制云台旋转，直到目标位于其所在的短焦图像或长焦图像的中心时，控制云台停止旋转。
通过上述实施例，使得在检测到目标时，通过旋转云台，能够使得目标处于短焦或长焦镜头的中心，从而提高了最终的融合图像的监控效果。
图4为本申请另一实施例提供的图像融合方法流程图，其具体步骤如下：
步骤401：获取短焦传感器和长焦传感器在同一时刻采集的第一短焦图像和第一长焦图像。其中，短焦传感器与短焦镜头相接，长焦传感器与长焦镜头相接，且短焦传感器和长焦传感器同时随云台旋转，或者，短焦传感器固定、只有长焦传感器随云台旋转。
步骤402：根据预先设定的短焦视野裁减系数，对第一短焦图像进行裁减处理，使得裁减后得到的第二短焦图像对应的视野等于第一短焦图像对应的视野乘以短焦视野裁减系数。
短焦视野裁减系数是用户等设定的。由于短焦镜头的视野较大，为了使得短焦镜头画面中的目标更清晰，可先对短焦镜头的视野进行裁减，即对第一短焦图像进行裁减，将边缘部分裁减掉，然后再对裁减后得到的第二短焦图像进行放大，从而目标得到了放大，图像的清晰度提高了。
如图5所示，大矩形表示短焦镜头的原视野，小矩形表示短焦镜头裁减后的视野，则短焦视野裁减系数可以大矩形的面积与小矩形的面积的比值表示，当然也可以小矩形的面积与大矩形的面积的比值表示，本申请对此不作限定。
步骤403：将第二短焦图像放大到与第一短焦图像的大小相同，得到第三短焦图像。
步骤404：根据短焦镜头和长焦镜头的焦距以及短焦视野裁减系数，计算将第一长焦图像与第三短焦图像中的同一目标的大小配准时第一长焦图像对应的缩小系数。
步骤405：根据步骤404计算得到的缩小系数，对第一长焦图像进行缩小处理，得到第二长焦图像。
步骤406：根据当前长焦镜头与短焦镜头的相对角度以及短焦视野裁减系数，计算当第二长焦图像和第三短焦图像中的同一目标的位置配准时第二长焦图像在第三短焦图像中的位置。
步骤407：根据第二长焦图像在第三短焦图像中的位置，将第二长焦图像覆盖到第三短焦图像上，得到融合图像。
上述实施例中，在第一短焦图像和第一长焦图像融合之前，先对第一短焦图像进行视野裁减，然后将裁减后得到的第二短焦图像进行放大，从而提高了参与融合的第三短焦图像的清晰度，即提高了最终得到的融合图像的清晰度。
在一些场景中，第一短焦图像对应的视野乘以短焦视野裁减系数后得到的视野可能小于或等于第一长焦图像的视野，在此种情况下，可直接舍弃第一短焦图像，直接对第一长焦图像进行处理得到最终的融合图像即可。具体如下：
在执行步骤402之前，首先判断第一短焦图像对应的视野乘以短焦视野裁减系数得到的视野是否小于第一长焦图像对应的视野，若小于，则直接对第一长焦图像进行裁减，以使得裁减后得到的第二长焦图像的视野等于第一短焦图像对应的视野乘以所述短焦视野裁减系数得到的视野，将第二长焦图像放大到与第一短焦图像的大小相同，得到最终的融合图像；
且，若第一短焦图像对应的视野乘以短焦视野裁减系数得到的视野等于第一长焦图像对应的视野，则直接将第一长焦图像作为最终的融合图像；
若第一短焦图像对应的视野乘以短焦视野裁减系数得到的视野大于第一长焦图像对应的视野，则执行步骤402及后续步骤。
通过上述实施例，使得在第一短焦图像对应的视野乘以短焦视野裁减系数得到的视野小于或等于第一长焦图像对应的视野，只采用清晰度好的第一长焦图像构造融合图像即可，在满足融合图像的视野要求的前提下，保证了融合图像的清晰度。
通过上述实施例得到的融合图像，由于长、短焦镜头的视野的大小相同，在融合图像中长、短焦图像的交界处可能会有明显的界限感，为了平滑该界限感，可对融合图像中第二长焦图像的边缘进行平滑处理。
一可选实施例中，设定第一透明度图像的大小等于第一长焦图像的大小，设定过渡带宽度W，W的单位是像素；其中，过渡带位于第一透明度图像的 四个边缘处，且位于第一透明度图像边界内；
对于四个边缘处的每条过渡带，过渡带上各个像素点的取值从第一透明度图像边界处向图像内部依次变大，且变化范围为从0开始至255结束。
图6为第一透明度图像的示例图。
且，对融合图像中第二长焦图像的边缘进行平滑处理包括：
采用步骤302或步骤404计算得到的第一长焦图像对应的缩小系数，对透明度图像进行缩小处理，得到第二透明度图像；将第二透明度图像的中心与融合图像中第二长焦图像的中心重合，对于融合图像上与第二透明度图像的过渡带重合的每个像素点，将该像素点的像素值替换为：(第二透明度图像上该像素点的像素值*融合图像中第二长焦图像上该像素点的像素值+(255-第二透明度图像上该像素点的像素值)*融合图像上被覆盖掉的第三短焦图像上该像素点的像素值)/255。
上述实施例的有益技术效果如下：
一、通过将透明度图像的四个边缘处的每条过渡带设计为：过渡带上各个像素点的取值从第一透明度图像边界处向图像内部依次变大，且变化范围为从0开始至255结束，并将过渡带上的像素点的像素值与第二长焦图像和第三短焦图像上对应像素点的权值关联，对融合图像上长、短焦图像的交界处进行平滑处理，优化了融合图像的显示效果；
二、通过预先计算出一个大小与第一长焦图像的大小相同的第一透明度图像，然后在长、短焦图像融合时，再根据实际需要，对第一透明度图像进行缩小，从而降低了图像融合过程中平滑处理耗费的时间，加快了最终的融合图像的生成速度，从而提高了监控效率。
一可选实施例中，对于四个边缘处的每条过渡带，过渡带上各个像素点的取值从第一透明度图像边界处向图像内部依次变大，且变化范围为从0开始至255结束包括：
对于每个边缘处的过渡带，从该过渡带的边界处到该过渡带的中心位置处的像素点的像素值采用第一二次函数拟合，从该过渡点的中心位置处到该过渡带的结束处的像素点的像素值采用第二二次函数拟合，其中，第一二次 函数中的二次项系数大于0，第二二次函数的二次项系数小于0。
上述实施例中，在设计过渡带时，将过渡带靠近边界的一半采用二次项系数大于0的一元二次函数进行拟合，另一半则采用二次项系数小于0的一元二次函数进行拟合，从而使得最终拟合出的过渡带的两端，即靠近边界的一端和靠近中心的一端都没有突起，优化了融合图像的平滑效果。
在实际应用中，设定二次项系数大于0的一元二次函数为y1＝a1x2+b1x+c1，设定二次项系数小于0的一元二次函数为y2＝a2x22+b2x2+c2，则：
根据(x1,y1)＝(0,0)、(x3,y3)＝((x1+x2)/2,(y1+y2)/2)和x1处的导数等于0，可计算出a1、b1和c1；根据(x3,y3)＝((x1+x2)/2,(y1+y2)/2)、(x2,y2)＝(W,255)和x2处的导数等于0，可计算出a2、b2和c2。其中，x1为过渡带靠近透明度图像边界一侧的过渡带起始点的坐标，x2为过渡带靠近透明度图像内部一侧的过渡带结束点的坐标，x3为过渡带的中心点的坐标。
计算公式如下：
Figure PCTCN2021104976-appb-000001
Figure PCTCN2021104976-appb-000002
本申请实施例还提供了一种图像融合方法，包括：
步骤一，获取当前待输出时刻对应的第一短焦图像及第一长焦图像，其中，所述第一短焦图像与所述第一长焦图像的采集时刻相同，且所述第一长焦图像的视野范围在所述第一短焦图像的视野范围内。
步骤二，判断当前待输出图像所对应的视野范围是否大于所述第一长焦图像的视野范围。
步骤三，若是，则按照当前待输出图像所对应的视野范围，在所述第一短焦图像中截取相应视野范围的图像区域，得到第二短焦图像。
确定所述第一长焦图像的视野范围在所述第二短焦图像中所对应的目标 区域；
步骤四，调整为目标区域大小的第一长焦图像叠加到所述第二短焦图像的目标区域上，得到融合图像。
步骤五，将所述融合图像放大为预设分辨率大小并输出。
一个例子中，当在短焦图像中检测到指定目标时，控制设备的镜头旋转到检测到指定目标的位置，并启用电子放大功能，逐级裁减短焦图像的视角(视野范围)，并将长焦图像拼接到裁减过后的图像中，由于前段采集的短焦图像的视角并没有丢失，检测和录像可以保存下变焦前的视野范围，相比于变焦镜头的摄像机，该实现方式属于电子放大，计算快速，中心位置来自于长焦图像，画面清晰，并且没有变焦镜头的重新对焦过程和视角丢失问题。
具体的，视野范围可以根据图像中心及缩放大小进行确定。
在一种可能的实施方式中，在所述判断当前待输出图像所对应的视野范围是否大于所述第一长焦图像的视野范围之后，所述方法还包括：若否，则按照当前待输出图像所对应的视野范围，在所述第一长焦图像中截取相应视野范围的图像区域，得到第二长焦图像；将所述第二长焦图像放大为预设分辨率大小并输出。
一个例子中，在放大倍率大于长焦图像的倍率，即当前待输出图像所对应的视野范围小于长焦图像的视野范围时，在第一长焦图像中截取出第二长焦图像，并将第二长焦图像放大为预设分辨率大小然后输出。其中，在当前待输出图像所对应的视野范围等于长焦图像的视野范围时，不用对第一长焦图像进行截取，直接将第一长焦图像放大为预设分辨率大小然后输出。
在一种可能的实施方式中，在所述将调整为目标区域大小的第一长焦图像叠加到所述第二短焦图像的目标区域上，得到融合图像之前，所述方法还包括：设定第一透明度图像的大小等于第一长焦图像的大小，设定过渡带宽度W，W的单位是像素；其中，过渡带位于第一透明度图像的四个边缘处，且位于第一透明度图像边界内；对于四个边缘处的每条过渡带，过渡带上各个像素点的取值从第一透明度图像边界处向图像内部依次变大，且变化范围为从0开始至255结束；将第一透明度图像的中心与第一长焦图像的中心重合，对于融合图像上与第一透明度图像的过渡带重合的每个像素点，将该像 素点的像素值替换为：(第一透明度图像上该像素点的像素值*融合图像中第一长焦图像上该像素点的像素值+(255-第一透明度图像上该像素点的像素值)*融合图像上被覆盖掉的第二短焦图像上该像素点的像素值)/255。
在一种可能的实施方式中，所述对于四个边缘处的每条过渡带，过渡带上各个像素点的取值从第一透明度图像边界处向图像内部依次变大，且变化范围为从0开始至255结束包括：对于每个边缘处的过渡带，从该过渡带的边界处到该过渡带的中心位置处的像素点的像素值采用第一一元二次函数拟合，从该过渡点的中心位置处到该过渡带的结束处的像素点的像素值采用第二一元二次函数拟合，其中，第一一元二次函数中的二次项系数大于0，第二一元二次函数的二次项系数小于0。
图7为本申请又一实施例提供的图像融合方法流程图，其具体步骤如下：
步骤701：实时在短焦传感器采集的第一短焦图像和长焦传感器采集的第一长焦图像中检测目标，若在第一短焦图像或第一长焦图像中检测到目标，则控制云台旋转，直到目标位于其所在的第一短焦图像或第一长焦图像的中心时，控制云台停止旋转。
步骤702：对于同一时刻采集的两帧图像：第一短焦图像和第一长焦图像进行如下步骤703-708的处理：
步骤703：获取当前设定的短焦视野裁减系数ZoomRatioc以及预先计算出的将短焦镜头的视野裁减到与长焦镜头的视野相同时的ZoomRatios，判断ZoomRatioc<ZoomRatios是否成立，若成立，则执行步骤704。
ZoomRatio可表示为两个不同视野的比值。
本实施例中，ZoomRatioc＝短焦镜头的原视野/对短焦镜头的视野进行裁减后的视野。图8为不同大小的视野的示意图，其中，矩形框A表示短焦镜头的原视野，若要将短焦镜头的视野裁减为矩形框B表示的视野，则ZoomRatioc＝SA/SB，其中，SA表示矩形框A的面积，SB表示矩形框B的面积；
ZoomRatios＝短焦镜头的原视野/长焦镜头的原视野，例如：图8中，矩形框C表示长焦镜头的原视野，则ZoomRatios＝SA/SC，其中，SC表示矩形框C的面积。
步骤704：根据ZoomRatioc对第一短焦图像进行裁减，然后将裁减后的得到的第二短焦图像放大到与第一短焦图像的大小相同，得到第三短焦图像。
图9为对第一短焦图像进行视野裁减的示例图，若ZoomRatioc＝4，则将第一短焦图像的长和宽分别裁减1/2，即将第一短焦图像的四周(上、下、左、右)分别裁减1/4，即可得到第二短焦图像。
为了让裁减后得到的第二短焦图像仍能正常显示，则再对第二短焦图像进行放大，放大到与第一短焦图像的大小相同。如：将图9所示的第二短焦图像再放大到与图9所示第一短焦图像的大小相同。放大算法本实施例不做限定。
步骤705：根据短焦镜头和长焦镜头的焦距以及ZoomRatioc，计算将第一长焦图像与第三短焦图像中的同一目标的大小配准时第一长焦图像对应的缩小系数ZoomRatiom。
要将第一长焦图像与第三短焦图像进行融合，必须将第三短焦图像和第一长焦图像上的同一目标进行配准，配准包括两方面：大小配准和位置配准，即，将第一长焦图像融合到第三短焦图像上后，使得第一长焦图像上的一个目标的大小与位置与第三短焦图像上的该目标的大小和位置完全配准。
由于长焦镜头的视野小于短焦镜头的视野，因此，在进行大小配准时，需要将第一长焦图像缩小，缩小系数ZoomRatiom与短焦镜头和长焦镜头的焦距有关，同时由于步骤704中对第一短焦图像进行了裁减和放大处理，因此，缩小系数ZoomRatiom还与ZoomRatioc有关。
步骤706：根据ZoomRatiom对第一长焦图像进行缩小处理，得到第二长焦图像。
步骤707：根据当前长焦镜头与短焦镜头的相对角度以及ZoomRatioc，计算当第二长焦图像和第三短焦图像中的同一目标的位置配准时第二长焦图像在第三短焦图像中的位置。
步骤708：根据第二长焦图像在第三短焦图像中的位置，将第二长焦图像覆盖到第三短焦图像上，得到融合图像，将融合图像输出到显示屏上。
覆盖的意思是以第二长焦图像替换对应位置处的第三短焦图像。
图10为本申请又一实施例提供的图像融合方法流程图，其具体步骤如下：
步骤1001：实时在短焦传感器采集的第一短焦图像和长焦传感器采集的第一长焦图像中检测目标，若在第一短焦图像或第一长焦图像中检测到目标，则控制云台旋转，直到目标位于其所在的第一短焦图像或第一长焦图像的中心时，控制云台停止旋转。
步骤1002：对于同一时刻采集的两帧图像：第一短焦图像和第一长焦图像进行如下步骤1003-1005的处理：
步骤1003：获取当前设定的短焦视野裁减系数ZoomRatioc以及预先计算出的将短焦镜头的视野裁减到与长焦镜头的视野相同时的ZoomRatios，判断ZoomRatioc<ZoomRatios是否成立，若不成立，则执行步骤1004。
ZoomRatio可表示为两个不同视野的比值。
本实施例中，ZoomRatioc＝短焦镜头的原视野/对短焦镜头的视野进行裁减后的视野，ZoomRatios＝短焦镜头的原视野/长焦镜头的原视野。
步骤1004：对第一长焦图像进行裁减，以使得裁减后得到的第二长焦图像的大小等于采用ZoomRatioc对第一短焦图像进行裁减后的图像大小，然后将第二长焦图像放大到与第一短焦图像的大小相同，得到第三长焦图像。
步骤1005：将第三长焦图像作为融合图像输出到显示屏上。
图11为本申请一实施例提供的图像融合装置的结构示意图，该装置主要包括：存储器1101以及可访问存储器1101的处理器1102，该存储器1101存储指令，该指令在由处理器1102执行时使得处理器1102执行如步骤301～305，或者步骤401～407，或者步骤701～708，或者步骤1001～1005所述的方法的步骤。
本申请实施例还提供一种长短焦双目云台摄像机，包括：与短焦镜头相接的短焦传感器、与长焦镜头相接的长焦传感器和云台，其中，短焦传感器和长焦传感器同时随云台旋转，或者，短焦传感器固定、只有长焦传感器随云台旋转；且，短焦传感器和长焦传感器分别具有与图像融合装置通信的接口，并分别通过各自的通信接口将各自采集的原始短焦图像和原始长焦图像发送给图像融合装置。
本申请实施例还提供了一种计算机可读存储介质，上述计算机可读存储介质内存储有计算机程序，上述计算机程序被处理器执行时实现本申请中任 一所述的图像融合方法。
在本申请提供的又一实施例中，还提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行本申请中任一所述的图像融合方法。
以上所述仅为本申请的较佳实施例而已，并不用以限制本申请，凡在本申请的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本申请保护的范围之内。

Claims (14)

  1. 一种图像融合方法，该方法包括：
    获取短焦传感器和长焦传感器在同一时刻采集的第一短焦图像和第一长焦图像；
    根据短焦镜头和长焦镜头的焦距，计算将第一长焦图像与第一短焦图像中的同一目标的大小配准时第一长焦图像对应的缩小系数；
    根据所述缩小系数，对第一长焦图像进行缩小处理，得到第二长焦图像；
    根据当前长焦镜头与短焦镜头的相对角度，计算当第二长焦图像和第一短焦图像中的同一目标的位置配准时第二长焦图像在第一短焦图像中的位置；
    根据第二长焦图像在第一短焦图像中的位置，将第二长焦图像覆盖到第一短焦图像上，得到融合图像；
    其中，短焦传感器与短焦镜头相接，长焦传感器与长焦镜头相接，且短焦传感器和长焦传感器同时随云台旋转，或者，短焦传感器固定、只有长焦传感器随云台旋转。
  2. 根据权利要求1所述的方法，所述方法进一步包括：
    实时在短焦传感器采集的短焦图像和长焦传感器采集的长焦图像中检测目标，若在短焦图像或长焦图像中检测到目标，则控制云台旋转，直到目标位于其所在的短焦图像或长焦图像的中心时，控制云台停止旋转。
  3. 根据权利要求1所述的方法，其中，所述获取短焦传感器和长焦传感器在同一时刻采集的第一短焦图像和第一长焦图像之后、所述计算将第一长焦图像与第一短焦图像中的同一目标的大小配准时第一长焦图像对应的缩小系数之前进一步包括：
    根据预先设定的短焦视野裁减系数，对第一短焦图像进行裁减处理，使得裁减后得到的第二短焦图像对应的视野等于第一短焦图像对应的视野乘以所述短焦视野裁减系数；将第二短焦图像放大到与第一短焦图像的大小相同，得到第三短焦图像；
    所述根据短焦镜头和长焦镜头的焦距，计算将第一长焦图像与第一短焦图像中的同一目标的大小配准时第一长焦图像对应的缩小系数包括：
    根据短焦镜头和长焦镜头的焦距以及所述短焦视野裁减系数，计算将第一长焦图像与第三短焦图像中的同一目标的大小配准时第一长焦图像对应的缩小系数；
    所述根据当前长焦镜头与短焦镜头的相对角度，计算当第二长焦图像和第一短焦图像中的同一目标的位置配准时第二长焦图像在第一短焦图像中的位置包括：
    根据当前长焦镜头与短焦镜头的相对角度以及所述短焦视野裁减系数，计算当第二长焦图像和第三短焦图像中的同一目标的位置配准时第二长焦图像在第三短焦图像中的位置；
    所述根据第二长焦图像在第一短焦图像中的位置，将第二长焦图像覆盖到第一短焦图像上包括：
    根据第二长焦图像在第三短焦图像中的位置，将第二长焦图像覆盖到第三短焦图像上。
  4. 根据权利要求3所述的方法，其中，所述获取短焦传感器和长焦传感器在同一时刻采集的第一短焦图像和第一长焦图像之后、所述根据预先设定的短焦视野裁减系数，对第一短焦图像进行裁减处理之前进一步包括：
    判断第一短焦图像对应的视野乘以所述短焦视野裁减系数得到的视野是否小于第一长焦图像对应的视野，若小于，则直接对第一长焦图像进行裁减，以使得裁减后得到的第二长焦图像的视野等于第一短焦图像对应的视野乘以所述短焦视野裁减系数得到的视野，将第二长焦图像放大到与第一短焦图像的大小相同，得到最终的融合图像；
    且，若第一短焦图像对应的视野乘以所述短焦视野裁减系数得到的视野等于第一长焦图像对应的视野，则直接将第一长焦图像作为最终的融合图像；
    若第一短焦图像对应的视野乘以所述短焦视野裁减系数得到的视野大于 第一长焦图像对应的视野，则执行所述根据预先设定的短焦视野裁减系数，对第一短焦图像进行裁减处理的动作。
  5. 根据权利要求1所述的方法，其中，所述得到融合图像之前进一步包括：
    对融合图像中第二长焦图像的边缘进行平滑处理。
  6. 根据权利要求5所述的方法，所述方法进一步包括：
    设定第一透明度图像的大小等于第一长焦图像的大小，设定过渡带宽度W，W的单位是像素；其中，过渡带位于第一透明度图像的四个边缘处，且位于第一透明度图像边界内；
    对于四个边缘处的每条过渡带，过渡带上各个像素点的取值从第一透明度图像边界处向图像内部依次变大，且变化范围为从0开始至255结束；
    且，所述对融合图像中第二长焦图像的边缘进行平滑处理包括：
    采用所述第一长焦图像对应的缩小系数，对透明度图像进行缩小处理，得到第二透明度图像；
    将第二透明度图像的中心与融合图像中第二长焦图像的中心重合，对于融合图像上与第二透明度图像的过渡带重合的每个像素点，将该像素点的像素值替换为：(第二透明度图像上该像素点的像素值*融合图像中第二长焦图像上该像素点的像素值+(255-第二透明度图像上该像素点的像素值)*融合图像上被覆盖掉的第三短焦图像上该像素点的像素值)/255。
  7. 根据权利要求6所述的方法，其中，所述对于四个边缘处的每条过渡带，过渡带上各个像素点的取值从第一透明度图像边界处向图像内部依次变大，且变化范围为从0开始至255结束包括：
    对于每个边缘处的过渡带，从该过渡带的边界处到该过渡带的中心位置处的像素点的像素值采用第一一元二次函数拟合，从该过渡点的中心位置处到该过渡带的结束处的像素点的像素值采用第二一元二次函数拟合，其中，第一一元二次函数中的二次项系数大于0，第二一元二次函数的二次项系数小 于0。
  8. 一种图像融合方法，包括：
    获取当前待输出时刻对应的第一短焦图像及第一长焦图像，其中，所述第一短焦图像与所述第一长焦图像的采集时刻相同，且所述第一长焦图像的视野范围在所述第一短焦图像的视野范围内；
    判断当前待输出图像所对应的视野范围是否大于所述第一长焦图像的视野范围；
    若是，则按照当前待输出图像所对应的视野范围，在所述第一短焦图像中截取相应视野范围的图像区域，得到第二短焦图像；
    确定所述第一长焦图像的视野范围在所述第二短焦图像中所对应的目标区域；
    将调整为目标区域大小的第一长焦图像叠加到所述第二短焦图像的目标区域上，得到融合图像；
    将所述融合图像放大为预设分辨率大小并输出。
  9. 根据权利要求8所述的方法，其中，在所述判断当前待输出图像所对应的视野范围是否大于所述第一长焦图像的视野范围之后，所述方法还包括：
    若否，则按照当前待输出图像所对应的视野范围，在所述第一长焦图像中截取相应视野范围的图像区域，得到第二长焦图像；
    将所述第二长焦图像放大为预设分辨率大小并输出。
  10. 根据权利要求8所述的方法，其中，在所述将调整为目标区域大小的第一长焦图像叠加到所述第二短焦图像的目标区域上，得到融合图像之前，所述方法还包括：
    设定第一透明度图像的大小等于第一长焦图像的大小，设定过渡带宽度W，W的单位是像素；其中，过渡带位于第一透明度图像的四个边缘处，且位于第一透明度图像边界内；
    对于四个边缘处的每条过渡带，过渡带上各个像素点的取值从第一透明度图像边界处向图像内部依次变大，且变化范围为从0开始至255结束；
    将第一透明度图像的中心与第一长焦图像的中心重合，对于融合图像上与第一透明度图像的过渡带重合的每个像素点，将该像素点的像素值替换为：(第一透明度图像上该像素点的像素值*融合图像中第一长焦图像上该像素点的像素值+(255-第一透明度图像上该像素点的像素值)*融合图像上被覆盖掉的第二短焦图像上该像素点的像素值)/255。
  11. 根据权利要求7所述的方法，其中，所述对于四个边缘处的每条过渡带，过渡带上各个像素点的取值从第一透明度图像边界处向图像内部依次变大，且变化范围为从0开始至255结束包括：
    对于每个边缘处的过渡带，从该过渡带的边界处到该过渡带的中心位置处的像素点的像素值采用第一一元二次函数拟合，从该过渡点的中心位置处到该过渡带的结束处的像素点的像素值采用第二一元二次函数拟合，其中，第一一元二次函数中的二次项系数大于0，第二一元二次函数的二次项系数小于0。
  12. 一种图像融合装置，该装置包括：存储器以及可访问存储器的处理器，该存储器存储指令，该指令在由处理器执行时使得处理器执行如权利要求1至11任一所述的方法的步骤。
  13. 一种双目长短焦云台摄像机，包括：与短焦镜头相接的短焦传感器、与长焦镜头相接的长焦传感器以及云台，其中，短焦传感器和长焦传感器同时随云台旋转，或者，短焦传感器固定、只有长焦传感器随云台旋转；
    且，短焦传感器和长焦传感器分别具有与图像融合装置通信的接口，并分别通过各自的通信接口将各自采集的原始短焦图像和原始长焦图像发送给图像融合装置。
  14. 一种图像融合系统，包括：如权利要求13所述的双目长短焦云台摄像机和如权利要求12所述的图像融合装置。
PCT/CN2021/104976 2020-07-08 2021-07-07 图像融合方法及装置 Ceased WO2022007840A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP21837892.5A EP4181506A4 (en) 2020-07-08 2021-07-07 METHOD AND DEVICE FOR IMAGE FUSION
US18/003,916 US12223622B2 (en) 2020-07-08 2021-07-07 Image fusion method and device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010649911.2A CN111818304B (zh) 2020-07-08 2020-07-08 图像融合方法及装置
CN202010649911.2 2020-07-08

Publications (1)

Publication Number Publication Date
WO2022007840A1 true WO2022007840A1 (zh) 2022-01-13

Family

ID=72841965

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/104976 Ceased WO2022007840A1 (zh) 2020-07-08 2021-07-07 图像融合方法及装置

Country Status (4)

Country Link
US (1) US12223622B2 (zh)
EP (1) EP4181506A4 (zh)
CN (1) CN111818304B (zh)
WO (1) WO2022007840A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115002342A (zh) * 2022-05-05 2022-09-02 深圳市睿联技术股份有限公司 目标对象跟踪方法、装置、摄像机及存储介质
CN115499565A (zh) * 2022-08-23 2022-12-20 盯盯拍（深圳）技术股份有限公司 基于双镜头的图像采集方法、装置、介质及行车记录仪

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111818304B (zh) 2020-07-08 2023-04-07 杭州萤石软件有限公司 图像融合方法及装置
CN112526391A (zh) * 2020-11-26 2021-03-19 湖南傲英创视信息科技有限公司 电解槽故障检测系统及方法
CN112702575B (zh) * 2020-12-23 2023-04-18 杭州中威电子股份有限公司 一种基于图像融合技术的多焦距图像采集装置及分析方法
KR102696960B1 (ko) 2020-12-26 2024-08-19 코어포토닉스 리미티드 스캐닝 줌 카메라를 갖는 멀티-애퍼처 모바일 카메라에서의 비디오 지원
CN115714922B (zh) * 2021-08-19 2025-11-04 宜宾市极米光电有限公司 图像拍摄系统、方法、装置、处理设备及介质
CN113723262B (zh) * 2021-08-25 2025-02-07 浙江大华技术股份有限公司 一种目标跟踪方法、装置、道路监控设备、设备和介质
JP7501559B2 (ja) * 2022-03-17 2024-06-18 株式会社リコー 表示端末、表示方法、通信システム、通信方法、及びプログラム
DE112022007238T5 (de) * 2022-05-17 2025-04-24 Google Llc Verschmelzen optisch gezoomter bilder zu einem digital gezoomten bild
CN115953760A (zh) * 2023-01-03 2023-04-11 重庆长安汽车股份有限公司 一种双摄融合的红绿灯识别方法、装置、电子设备以及介质
CN118450269B (zh) * 2023-10-25 2025-01-17 荣耀终端有限公司 图像处理方法和电子设备

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006238325A (ja) * 2005-02-28 2006-09-07 Canon Inc カメラシステム
US20120026366A1 (en) * 2009-04-07 2012-02-02 Nextvision Stabilized Systems Ltd. Continuous electronic zoom for an imaging system with multiple imaging devices having different fixed fov
CN107959805A (zh) * 2017-12-04 2018-04-24 深圳市未来媒体技术研究院 基于混合相机阵列的光场视频成像系统及视频处理方法
CN108353118A (zh) * 2015-12-29 2018-07-31 核心光电有限公司 具有自动可调节长焦视场的双孔径变焦数字摄影机
CN109120883A (zh) * 2017-06-22 2019-01-01 杭州海康威视数字技术股份有限公司 基于远近景的视频监控方法、装置及计算机可读存储介质
CN109285136A (zh) * 2018-08-31 2019-01-29 清华-伯克利深圳学院筹备办公室 一种图像的多尺度融合方法、装置、存储介质及终端
CN111818304A (zh) * 2020-07-08 2020-10-23 杭州萤石软件有限公司 图像融合方法及装置

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007065513A (ja) * 2005-09-01 2007-03-15 Ricoh Co Ltd 変倍撮影光学系システム、撮像装置、及び携帯情報端末装置
US20120075489A1 (en) * 2010-09-24 2012-03-29 Nishihara H Keith Zoom camera image blending technique
CN103380392B (zh) * 2011-03-31 2014-12-03 富士胶片株式会社 摄像装置及其对焦控制方法
CN103841313A (zh) * 2012-11-27 2014-06-04 华为技术有限公司 云台摄像机控制方法、系统及设备
TWI531852B (zh) * 2014-09-16 2016-05-01 聚晶半導體股份有限公司 影像擷取裝置及數位變焦方法
CN105578036B (zh) * 2015-12-14 2019-01-04 惠州Tcl移动通信有限公司 一种拍摄图像的点对焦方法、系统及移动终端
CN107770433B (zh) 2016-08-15 2020-08-04 广州立景创新科技有限公司 影像获取装置及其影像平顺缩放方法
CN106254777A (zh) * 2016-08-23 2016-12-21 深圳市德盟科技有限公司 一种一体化智能星光级云台摄像机
US10297034B2 (en) * 2016-09-30 2019-05-21 Qualcomm Incorporated Systems and methods for fusing images
CN106506941A (zh) * 2016-10-20 2017-03-15 深圳市道通智能航空技术有限公司 图像处理的方法及装置、飞行器
US10972672B2 (en) * 2017-06-05 2021-04-06 Samsung Electronics Co., Ltd. Device having cameras with different focal lengths and a method of implementing cameras with different focal lengths
CN110460748A (zh) * 2018-05-08 2019-11-15 杭州海康威视数字技术股份有限公司 一种摄像机以及监控系统
CN108900763B (zh) * 2018-05-30 2022-03-22 Oppo（重庆）智能科技有限公司 拍摄装置、电子设备及图像获取方法
CN110855883B (zh) * 2019-11-05 2021-07-20 浙江大华技术股份有限公司 一种图像处理系统、方法、装置设备及存储介质
CN110868541B (zh) * 2019-11-19 2021-04-20 展讯通信（上海）有限公司 视场融合方法及装置、存储介质、终端
CN110913131B (zh) * 2019-11-21 2021-05-11 维沃移动通信有限公司 一种月亮拍摄方法及电子设备

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006238325A (ja) * 2005-02-28 2006-09-07 Canon Inc カメラシステム
US20120026366A1 (en) * 2009-04-07 2012-02-02 Nextvision Stabilized Systems Ltd. Continuous electronic zoom for an imaging system with multiple imaging devices having different fixed fov
CN108353118A (zh) * 2015-12-29 2018-07-31 核心光电有限公司 具有自动可调节长焦视场的双孔径变焦数字摄影机
CN109120883A (zh) * 2017-06-22 2019-01-01 杭州海康威视数字技术股份有限公司 基于远近景的视频监控方法、装置及计算机可读存储介质
CN107959805A (zh) * 2017-12-04 2018-04-24 深圳市未来媒体技术研究院 基于混合相机阵列的光场视频成像系统及视频处理方法
CN109285136A (zh) * 2018-08-31 2019-01-29 清华-伯克利深圳学院筹备办公室 一种图像的多尺度融合方法、装置、存储介质及终端
CN111818304A (zh) * 2020-07-08 2020-10-23 杭州萤石软件有限公司 图像融合方法及装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4181506A4

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115002342A (zh) * 2022-05-05 2022-09-02 深圳市睿联技术股份有限公司 目标对象跟踪方法、装置、摄像机及存储介质
CN115002342B (zh) * 2022-05-05 2024-05-31 深圳市睿联技术股份有限公司 目标对象跟踪方法、装置、摄像机及存储介质
CN115499565A (zh) * 2022-08-23 2022-12-20 盯盯拍（深圳）技术股份有限公司 基于双镜头的图像采集方法、装置、介质及行车记录仪
CN115499565B (zh) * 2022-08-23 2024-02-20 盯盯拍（深圳）技术股份有限公司 基于双镜头的图像采集方法、装置、介质及行车记录仪

Also Published As

Publication number Publication date
CN111818304B (zh) 2023-04-07
EP4181506A4 (en) 2023-11-29
CN111818304A (zh) 2020-10-23
US12223622B2 (en) 2025-02-11
EP4181506A1 (en) 2023-05-17
US20230325994A1 (en) 2023-10-12

Similar Documents

Publication Publication Date Title
WO2022007840A1 (zh) 图像融合方法及装置
US20230059657A1 (en) Multi-camera zoom control method and apparatus, and electronic system and storage medium
JP4539729B2 (ja) 画像処理装置、カメラ装置、画像処理方法、およびプログラム
JP5592006B2 (ja) 三次元画像処理
WO2021073331A1 (zh) 基于终端设备的变焦虚化图像获取方法和装置
US20160309137A1 (en) Methods, systems, and computer-readable storage media for generating three-dimensional (3d) images of a scene
WO2020007320A1 (zh) 多视角图像的融合方法、装置、计算机设备和存储介质
CN111062881A (zh) 图像处理方法及装置、存储介质、电子设备
CN110463176A (zh) 图像质量评估
WO2019238113A1 (zh) 成像方法、装置、终端和存储介质
EP4164209B1 (en) Image reconstruction method and apparatus
WO2018094648A1 (zh) 拍摄构图引导方法及装置
CN110324532A (zh) 一种图像虚化方法、装置、存储介质及电子设备
TWI524258B (zh) 電子書顯示調整系統及方法
CN104320590A (zh) 一种分屏图像处理方法、装置及客户端
CN112019734B (zh) 图像采集方法、装置、电子设备和计算机可读存储介质
CN105915802A (zh) 一种转动式摄影机的拍摄范围设定调整方法
CN114222061B (zh) 一种光电周扫的实时图像拼接与交互方法
JP2019201325A (ja) 撮像装置、その制御方法、および、プログラム
US20250390984A1 (en) Picture display method, system, device, and computer-readable storage medium
WO2022111330A1 (zh) 多摄像头设备的图像拼接方法及装置、存储介质、终端
CN117918057A (zh) 显示设备及设备控制方法
WO2018196854A1 (zh) 一种拍照方法、拍照装置及移动终端
JP6483661B2 (ja) 撮像制御装置、撮像制御方法およびプログラム
JP2016111561A (ja) 情報処理装置、システム、情報処理方法及びプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21837892

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2021837892

Country of ref document: EP

Effective date: 20230208