WO2015081870A1 - 一种图像处理方法、装置及终端 - Google Patents
一种图像处理方法、装置及终端 Download PDFInfo
- Publication number
- WO2015081870A1 WO2015081870A1 PCT/CN2014/093024 CN2014093024W WO2015081870A1 WO 2015081870 A1 WO2015081870 A1 WO 2015081870A1 CN 2014093024 W CN2014093024 W CN 2014093024W WO 2015081870 A1 WO2015081870 A1 WO 2015081870A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- camera
- translation
- amount
- compensation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/20—Linear translation of whole images or parts thereof, e.g. panning
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/95—Computational photography systems, e.g. light-field imaging systems
- H04N23/951—Computational photography systems, e.g. light-field imaging systems by using two or more images to influence resolution, frame rate or aspect ratio
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/14—Picture signal circuitry for video frequency region
- H04N5/21—Circuitry for suppressing or minimising disturbance, e.g. moiré or halo
- H04N5/211—Ghost signal cancellation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/2621—Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/265—Mixing
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
Definitions
- the present invention relates to the field of image applications, and in particular, to an image processing method, apparatus, and terminal.
- high-resolution images can be obtained by super-resolution algorithms, which refer to the fusion of multi-frame low-resolution images to generate a high-resolution image.
- super-resolution algorithms which refer to the fusion of multi-frame low-resolution images to generate a high-resolution image.
- there is a time difference when collecting multi-frame low-resolution images there is a difference in local motion between multi-frame images acquired at different times, and the local motion is within a time interval of two-frame image acquisition. , caused by the movement of objects in the scene.
- Embodiments of the present invention provide an image processing method, apparatus, and terminal to solve the technical problem of "ghosting" in synthesizing a high-resolution image through a multi-frame low-resolution image in the prior art.
- an image processing method is provided, which is applied to a terminal including a first camera and a second camera, where the first camera and the second camera are located on the same side of the terminal.
- the method includes: acquiring a first image acquired by the first camera for the first area and a second image of the second camera acquired by the second camera at the same time; using the first image as a reference image, Performing translation compensation on the second image; synthesizing the first image and the translationally compensated second image into a third image, wherein the resolution of the third image is higher than the first image And a resolution of the second image.
- the performing the translation compensation on the second image by using the first image as a reference image includes: determining the first image and the first a translation amount between the two images; the second image acquired by the second camera is subjected to translation compensation according to the translation amount.
- the combining the first image and the translationally compensated second image into a third image includes: Deriving a result of the translation compensation to determine a common area of the first image and the second image; synthesizing the common area of the first image and the second image into the third image.
- an image processing apparatus including: an acquiring module, configured to acquire, by a first camera, a first image collected by a first region and a second camera collected at a same time at a second region a second image, wherein the first camera and the second camera are located in a same plane of the image processing device; a translation compensation module is coupled to the acquisition module, configured to obtain the first After the image and the second image, using the first image as a reference image, performing translation compensation on the second image; and an image synthesis module connected to the translation compensation module for passing the translation compensation module After performing the translation compensation on the second image, synthesizing the first image and the translationally compensated second image into a third image, the resolution of the third image being higher than the first image and the first image The resolution of the two images.
- the translation compensation module includes: a determining unit, configured to determine a translation amount between the first image and the second image;
- the compensation unit is connected to the determining unit, and configured to perform translation compensation on the second image acquired by the second camera according to the translation amount after determining the translation amount based on the determining unit.
- the image synthesizing module includes: a determining unit, configured to determine the first image according to a result of the translation compensation And a common area of the second image; a synthesizing unit connected to the determining unit, configured to: after the determining the unit, determine the common image of the first image and the second image according to the determining unit The area is synthesized into the third image.
- a terminal includes: a first camera for acquiring a first image for a first area; and a second camera for collecting the first image at the first camera Obtaining a second image at the same time of the image, the first camera and the second camera are located on the same side of the terminal; the processor is connected to the first camera and the second camera, And performing, by using the first image as a reference image, performing translation compensation on the second image; and synthesizing the first image and the translationally compensated second image into a third image, the resolution of the third image The rate is higher than the resolution of the first image and the second image.
- the optical axes of the first camera and the second camera are parallel and/or the first camera and the second camera are fixedly disposed at the terminal.
- the processor by using the first image as a reference image, performing translation compensation on the second image, specifically: determining the first image and the The amount of translation between the second images; the second image acquired by the second camera The translation compensation is performed according to the shift amount.
- the processor, the first image and the translationally compensated second image are combined into a third image, specifically: Determining a common area of the first image and the second image according to a result of the translation compensation; synthesizing the common area of the first image and the second image into the third image.
- the first image is acquired by the first camera at the same time to obtain the first image
- the second image is acquired by the second camera to obtain the second image
- the first image is used as the reference image pair.
- the second image is subjected to translation compensation, and finally the first image and the second image are combined into a third image, and the resolution of the third image is higher than the resolution of the first image and the second image due to the first image and the second image
- the second image is compensated with the first image as the reference image, so that the second image and the first image
- the shaking direction of the user's hand is the same, so that the ghost generated by the user's hand shake can be prevented.
- the time taken to acquire the first image and the second image may be reduced, and when the third image is synthesized, no algorithm is needed to correct the local motion and the user's hand. Shake the "ghost" problem, which in turn improves the acquisition of the third image The speed can improve the user's experience.
- FIG. 1 is a schematic diagram of a "ghosting" problem in the prior art when synthesizing a high resolution image through two low resolution images;
- FIG. 2 is a schematic diagram of a first camera and a second camera disposed on the same side of a terminal in an image processing method according to an embodiment of the present invention
- FIG. 3 is a flowchart of an image processing method according to an embodiment of the present invention.
- FIG. 4 is a flowchart of performing translation compensation on a second image in an image processing method according to an embodiment of the present invention
- FIG. 5 is a schematic diagram showing a positional relationship between d, B, f, and Z in a calculation formula of a translation amount in an image processing method according to an embodiment of the present invention
- FIG. 6 is a flowchart of synthesizing a first image and a second image after translation compensation in an image processing method according to an embodiment of the present invention
- FIG. 7a is a schematic diagram of a first image and a second image obtained by acquiring an image capturing method according to an embodiment of the present invention
- FIG. 7b is a schematic diagram of performing a translation compensation on a second image and determining a common area of the first image and the second image in the image processing method according to an embodiment of the present invention
- FIG. 7c is a schematic diagram of a combined area and a common area of a first image and a second image determined in an image processing method according to an embodiment of the present invention.
- FIG. 8 is a flowchart of an image processing method according to Embodiment 1 of the present invention.
- Embodiment 9 is a flowchart of an image processing method according to Embodiment 2 of the present invention.
- FIG. 10 is a structural diagram of an image collection device according to an embodiment of the present invention.
- FIG. 11 is a structural diagram of a terminal according to an embodiment of the present invention.
- an image processing method is provided in the embodiment of the present invention, and the method is applied to the terminal including the first camera and the second camera.
- the first camera and the second camera are located on the same side of the terminal, and the method includes: acquiring a first image acquired by the first camera for the first region and a second image captured by the second camera for the second region at the same time;
- the first image is a reference image, and the second image is subjected to translation compensation; the first image and the translationally compensated second image are combined into a third image, and the resolution of the third image is higher than that of the first image and the second image. rate.
- first image and the second image are images acquired at the same time, there is no object motion between the two frames, and the second image is compensated by the first image as a reference image, thereby making the first image
- the two images overlap with the same object position in the first image; and since the first image and the second image are simultaneously acquired, when the first image and the second image are acquired, the shaking direction of the user's hand is the same, thereby preventing The "ghosting" produced by the user's hand shakes, thereby solving the "ghosting" problem that occurs when synthesizing high-resolution images through multi-frame low-resolution images;
- the time taken to acquire the first image and the second image may be reduced, and when the third image is synthesized, no algorithm is needed to correct the local motion and the user's hand.
- the “ghost problem” generated by the shaking increases the speed of acquiring the third image, which can improve the user experience.
- an embodiment of the present invention provides an image processing method. Referring to FIG. 2, the method is applied to a terminal including a first camera 10 and a second camera 11, where the first camera 10 and the second camera 11 are located at the terminal. On the same side, the first camera 10 and the second camera 11 can be connected by a connector 12.
- the method specifically includes the following steps:
- Step S301 Acquire a first image acquired by the first camera 10 for the first region and a second image captured by the second camera 11 for the second region at the same time; wherein the first image and the second image are respectively a frame image.
- the first camera 10 when acquiring the first image collected by the first camera 10 for the first area and the second image captured by the second camera 11 for the second area at the same time, the first camera 10 may be configured to take a photo of the first area. Previewing the first image and the second camera 11 to preview the second image when photographing the second area at the same time; or obtaining the first image taken by the first camera 10 for the first area and the second camera 11 at the same time A second image taken of the second area.
- the focal lengths of the first camera 10 and the second camera 11 may be the same.
- Step S302 using the first image as a reference image, and performing translation compensation on the second image;
- Step S303 synthesize the first image and the second image after the translation compensation into a third image, and the resolution of the third image is higher than the resolution of the first image and the second image.
- the first camera 10 and the second camera 11 can be completely independent cameras, and the first camera 10 and the second camera 11 can be simultaneously controlled by software, so that an object in the second image is relative to the first image.
- One of the objects is absolutely stationary. For example, in a scene, user A is in motion. If the image is acquired at different times in the prior art, the location of user A and the second image in the first image are The location of the user A is different, and then a "ghost" is formed after the third image is synthesized; but in the present invention, although the user A is moving, after the translation compensation, the user in the first image and the second image A is in the same position, thus avoiding the "ghosting" problem caused by the movement of objects between two frames of images.
- the first camera 10 and the second camera 11 can be arranged in various manners. Three preferred arrangements are listed below. Of course, in the specific implementation process, the following three situations are not limited.
- the optical axes of the first camera 10 and the second camera 11 are parallel.
- the optical axis refers to the vertical direction of the plane of the lens of the camera, that is, optical The symmetry axis of the system, the optical axes of the first camera 10 and the second camera 11 are parallel, that is, the vertical lines of the planes of the lenses of the first camera 10 and the second camera 11 are parallel, if the optical axes of the first camera 10 and the second camera 11 are Parallel, it is possible to prevent distortion, occlusion, and the like between the first image and the second image, thereby making the calculated amount of translation more accurate.
- the first camera 10 and the second camera 11 are fixedly disposed at a terminal.
- the optical axes of the first camera 10 and the second camera 11 are parallel and the first camera 10 and the second camera 11 are fixedly disposed at a terminal.
- the relative position and posture of the first camera 10 and the second camera 11 can be prevented from changing, it is possible to prevent the relative position and posture of the first camera 10 and the second camera 11 from being changed.
- the optical axes of a camera 10 and the second camera 11 are not parallel, so that the calculated amount of translation of the second image relative to the first image is more accurate, thereby further preventing the "ghosting" problem.
- step S302 the first image is used as a reference image, and the second image is subjected to translation compensation.
- the method further includes the following steps:
- Step S401 determining a translation amount between the first image and the second image
- Step S402 The second image acquired by the second camera 11 is subjected to translation compensation according to the amount of shift.
- step S401 the amount of translation between the first image and the second image may be determined by the following formula:
- d represents the amount of translation of the object at a distance Z from the plane of the first camera 10 and the second camera 11 relative to the first image in the second image;
- B represents the distance between the first camera 10 and the second camera 11
- Z represents the vertical distance of the object from the plane in which the first camera 10 and the second camera 11 are located, that is, the depth of the object
- f represents the focal length of the first camera 10 or the focal length of the second camera 11.
- FIG. 5 it is a schematic diagram of the positional relationship between d, B, f, and Z, wherein after the first image is acquired by the first camera 10 and the second image is acquired by the second camera 11, The depths generated by the first camera 10 and the second camera 11 are then used to determine the amount of translation of different objects by the above-described translation amount calculation formula.
- the depth generated by the first camera 10 and the second camera 11 and the corresponding translation amount in the depth may be calibrated in advance by the above-mentioned translation amount calculation formula, and the calibration method may be as follows:
- N sets of specific images are collected at discrete N different depth levels, each set containing two images, respectively from the first camera 10 and the second camera 11, and then calibrated between each set of images
- the amount of translation so that N sets of translations are obtained, which is the amount of translation between pixels in the N depths that are calibrated.
- the N depths and the corresponding N shift amounts can be pre-stored in the rom for use in actual photographing.
- the translation amount corresponding to the depth is queried in the rom, and the translation amount is the translation amount of the second image relative to the first image at the depth (assuming that the first image is a reference image)
- the depth of a point A in the scene is D
- the amount of translation corresponding to D in the rom is M
- the amount of translation of the pixel A in the second image relative to the pixel A in the first image is M. .
- the calibration can be performed for each product at the time of shipment, so the accuracy of the determined translation amount is higher.
- step S402 the second image acquired by the second camera 11 is compensated according to the translation amount. For example, if the first image is unchanged, and the coordinate of each point of the second image is subtracted from the translation amount of the corresponding depth, the second image after the translation compensation is obtained.
- step S303 the first image and the second image after the translation compensation are combined into a third image.
- the method includes the following steps:
- Step S601 determining a common area of the first image and the second image according to a result of the translation compensation
- Step S602 Combine the common areas of the first image and the second image into a third image.
- step S601 for obtaining the first image 70a and the second image 70b, the amount of translation corresponding to the depth of each pixel in the second image 70b may be first determined, and then the second image 70b is obtained.
- the second image 70b after the translation compensation can be obtained by subtracting the translation amount of the corresponding depth from each pixel in the pixel, as shown in FIG. 7b.
- the first image 70a and the second image can be obtained.
- the content of the same portion of the image 70b serves as the common area 71.
- the common area of the first image 70a and the second image 70b is combined into a third image, which can be divided into multiple modes. Two of them are introduced below. Of course, in the specific implementation process, it is not limited. The following two situations.
- the second image 70b Since the second image 70b is located at the same position as the same object of the first image 70a after the translation compensation of the second image 70b, the area where the same coordinates in the first image 70a and the second image 70b are directly determined as the common area can be directly determined. 71.
- the area where the same coordinates are located in the first image 70a and the second image 70b can be determined as the common area 71. And storing; determining a maximum area included in coordinates of the first image 70a and the second image 70b as a combined area.
- the first image 70a and the translationally compensated second image 70b may be synthesized into a third image by an interpolation method, such as a kernel regression interpolation method or an edge-based kernel regression interpolation method, and the like.
- an interpolation method such as a kernel regression interpolation method or an edge-based kernel regression interpolation method, and the like.
- the embodiment of the invention is not limited.
- the terminal is a mobile phone as an example.
- the mobile phone includes two cameras.
- the two cameras are located on the same side of the mobile phone, and the optical axis is parallel and fixedly disposed on the mobile phone.
- FIG. 8 for implementation of the present invention.
- Step S801a The first camera 10 acquires and obtains a first image, and the first image resolution is: 3264px*2448px;
- Step S801b at the same time as the first camera 10 acquires the first image, the second camera 11 acquires the second image, and the second image resolution is: 3264px*2448px;
- Step S802 Perform translation compensation on the first image and the second image according to the scene depth information and the corresponding relationship between the pre-stored depth and the translation amount;
- the first camera 10 can transmit the first image to the translation compensation module in the mobile phone
- the second camera 11 transmits the second image to the translation compensation module in the mobile phone, and then the first image and the first image through the translation compensation module.
- the second image is subjected to translation compensation; the specific steps are as follows: according to the depth information of the scene from the correspondence between the depth and the translation amount pre-stored in the mobile phone, the translation amount corresponding to the pixel point of each depth is determined; and then the pixel of the second image is The coordinates of the points are respectively subtracted from the translation amount of the corresponding depth, and the second image after the translation compensation is obtained; finally, the common area of the first image and the second image after the translation compensation is determined;
- Step S803 After determining the common area of the first image and the panned compensated second image, synthesizing the first image and the second image after the translation compensation.
- the first image, the second image after the translation compensation, and the coordinate information corresponding to the common area are transmitted to the image synthesis module of the mobile phone;
- the compositing module crops the common area of the first image and the translated second image, and finally merges the cropped first image and the second image into a high resolution image by using an interpolation algorithm, that is, the third image
- the third image resolution is, for example, 4160px*3120px.
- the resolution sizes of the first image, the second image, and the third image are merely an example and are not intended to be limiting.
- the terminal is a tablet computer.
- the tablet computer includes a first camera 10 and a second camera 11.
- the first camera 10 and the second camera 11 are located on the same side of the tablet.
- the image processing method includes the following steps:
- Step S901a the first camera 10 acquires and obtains the first image
- Step S901b at the same time as the first camera 10 acquires the first image, the second camera 11 acquires and obtains the second image;
- Step S902 The first camera 10 transmits the second image to the first image and the second camera 11 respectively to the translation compensation module in the mobile phone, and then passes the translation compensation module to the first image and the second image.
- Step S903 After determining the common area of the first image and the panned compensated second image, transferring the first image, the second image after the translation compensation, and the coordinate information of the common area to the image synthesis module of the mobile phone, through image synthesis The module determines a combined area of the first image and the second image, and combines the combined area of the first image and the second image into a high resolution image, and finally crops the first image and the first image from the high resolution image
- the third image can be obtained from the common area of the two images.
- an embodiment of the present invention provides an image processing apparatus. Referring to FIG. 10, the following specifically includes the following structure:
- the acquiring module 100 is configured to acquire a first image collected by the first camera 10 for the first region and a second image captured by the second camera 11 for the second region at the same time, where the first camera 10 and the second camera 11 are located in the image. Processing the same plane of the device;
- the translation compensation module 101 is connected to the acquisition module 100, and after the first image and the second image are obtained by the acquisition module 100, the first image is used as a reference image, and the second image is subjected to translation compensation;
- the image synthesis module 102 is coupled to the translation compensation module 101, and configured to synthesize the first image and the translationally compensated second image into a third image after the translation compensation is performed by the translation compensation module 91.
- the resolution is higher than the resolution of the first image and the second image.
- the optical axes of the first camera 10 and the second camera 11 are parallel and/or
- the first camera 10 and the second camera 11 are fixedly disposed at a terminal.
- the translation compensation module 101 specifically includes:
- a determining unit configured to determine a translation amount between the first image and the second image
- the compensation unit is connected to the determining unit, and after determining the shift amount based on the determining unit, the second image acquired by the second camera 11 is subjected to translation compensation according to the shift amount.
- the determining unit is specifically configured to determine the shift amount by using the following formula:
- d represents the amount of translation of the object at a distance Z from the plane of the first camera 10 and the second camera 11 relative to the first image in the second image;
- B represents the distance between the first camera 10 and the second camera 11
- Z represents the vertical distance of the object from the plane in which the first camera 10 and the second camera 11 are located
- f represents the focal length of the first camera or the focal length of the second camera.
- the image synthesizing module 102 specifically includes:
- a determining unit configured to determine a common area of the first image and the second image according to a result of the translation compensation
- a synthesizing unit coupled to the determining unit, configured to synthesize the common area of the first image and the second image into a third image after determining the common area according to the determining unit.
- the image processing device is an image processing device used in the image processing method in the embodiment of the present invention. Therefore, those skilled in the art can understand the image processing device according to the embodiment of the present invention. The specific structure and modification of the image processing apparatus described in the embodiments of the present invention are not described in detail herein.
- an embodiment of the present invention provides a terminal, such as a mobile phone, a tablet computer, a digital camera, and the like.
- a terminal such as a mobile phone, a tablet computer, a digital camera, and the like.
- the terminal includes:
- a first camera 10 configured to acquire a first image from the first area
- the second camera 11 is configured to acquire the second image at the same time when the first camera 10 captures the first image, and the first camera 10 and the second camera 11 are located on the same side of the terminal;
- the processor 13 is connected to the first camera 10 and the second camera 11 for performing translation compensation on the second image with the first image as a reference image;
- the first image and the panned compensated second image are combined into a third image, the resolution of the third image being higher than the resolution of the first image and the second image.
- the first camera 10 and the second camera 11 can be connected by a connector 12 (as shown in FIG. 2).
- the optical axes of the first camera 10 and the second camera 11 are parallel and/or
- the first camera 10 and the second camera 11 are fixedly disposed at the terminal.
- the processor 13 performs the translation compensation on the second image by using the first image as a reference image, and specifically includes:
- the second image acquired by the second camera 11 is subjected to translation compensation according to the amount of shift.
- the processor 13 determines the amount of translation between the first image and the second image, specifically:
- d represents the amount of translation of the object at a distance Z from the plane of the first camera 10 and the second camera 11 relative to the first image in the second image;
- B represents the distance between the first camera 10 and the second camera 11
- Z represents the vertical distance of the object from the plane of the first camera 10 and the second camera 11, and f represents the focal length of the first camera or the focal length of the second camera.
- the focal length of the first camera may be the same as the focal length of the second camera.
- the processor 13 combines the first image and the second image after the translation compensation into a third image, and specifically includes:
- the common area of the first image and the second image is synthesized into a third image.
- first camera and the second camera are located on the same side of the terminal, and the first camera and the second camera may be located at the back of the terminal, and the pixels of the first camera and the second camera may be the same or different.
- first camera and the second camera can also be in front of the terminal.
- the terminal can be a mobile phone, a tablet, a wearable device, a wristband device, a digital camera, or glasses.
- the terminal described in the embodiment of the present invention is a terminal used in the image processing method in the embodiment of the present invention
- the image processing method introduced in the embodiment of the present invention belongs to the field.
- a person skilled in the art can understand the specific structure and deformation of the terminal introduced in the embodiment of the present invention, and therefore will not be described in detail herein.
- the first image is acquired by the first camera at the same time to obtain the first image
- the second image is acquired by the second camera to obtain the second image
- the first image is The reference image performs translation compensation on the second image
- the resolution of the third image is higher than the resolution of the first image and the second image due to the first image and
- the second image is an image acquired at the same time, so there is no object motion between the two frames, and the second image is compensated with the first image as a reference image, so that the second image and the first image
- “ghosting” which solves the "ghosting” problem that arises when synthesizing high-resolution images from multi-frame low-resolution images
- the time taken to acquire the first image and the second image may be reduced, and when the third image is synthesized, no algorithm is needed to correct the local motion and the user's hand.
- the “ghost problem” generated by the shaking increases the speed of acquiring the third image, which can improve the user experience.
- the optical axes of the first camera and the second camera are parallel, so that problems such as distortion and occlusion between the first image and the second image can be prevented, thereby making the calculated translation amount more accurate. In turn, the "ghosting" problem can be further prevented.
- the relative positions and postures of the first camera and the second camera can be prevented from changing, thereby ensuring that the object of the same depth in the scene is translated relative to the first image.
- the quantity is the same.
- the correspondence between the depth and the amount of translation can be stored in advance, and the corresponding amount of translation can be directly determined by the actual depth of the scene when photographing, without acquiring two The image is then calculated, thereby increasing the speed at which the third image is obtained; and preventing the user from shaking the first camera and the second The camera's dithering direction is different, so it can further prevent the "ghosting" problem;
- first camera and the second camera can be parallel to the optical axis and can be fixedly disposed at a terminal, it is possible to prevent the optical axes from being non-parallel due to the relative positional changes of the first camera and the second camera, and the optical axes are not parallel.
- the amount of pre-stored translation is not accurate enough, so that the translation compensation of the second image is more accurate, and the "ghosting" problem can be further prevented.
- embodiments of the present invention can be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment or a combination of software and hardware. Moreover, the invention can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.
- computer-usable storage media including but not limited to disk storage, CD-ROM, optical storage, etc.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Studio Devices (AREA)
- Image Processing (AREA)
Abstract
Description
Claims (13)
- 一种图像处理方法,其特征在于,应用于包含第一摄像头和第二摄像头的终端中,所述第一摄像头和所述第二摄像头位于所述终端的同一侧面,所述方法包括:获取所述第一摄像头对第一区域采集的第一图像和所述第二摄像头在同一时刻对第二区域采集的第二图像;以所述第一图像为参考图像,对所述第二图像进行平移补偿;将所述第一图像和平移补偿后的第二图像合成为第三图像,所述第三图像的分辨率高于所述第一图像和所述第二图像的分辨率。
- 如权利要求1所述方法,其特征在于,所述以所述第一图像为参考图像,对所述第二图像进行平移补偿,具体包括:确定所述第一图像和所述第二图像之间的平移量;对所述第二摄像头采集的所述第二图像依据所述平移量进行平移补偿。
- 如权利要求2所述方法,其特征在于,通过以下公式确定所述平移量:d=B*f/Z;其中,d表示与第一摄像头和第二摄像头所在平面的距离为Z的物体在第二图像中相对于在第一图像中的平移量;B表示第一摄像头和第二摄像头的距离;Z表示所述物体与所述第一摄像头和所述第二摄像头所在平面的垂直距离,f表示所述第一摄像头的焦距或所述第二摄像头的焦距。
- 如权利要求3所述方法,其特征在于,所述将所述第一图像和平移补偿后的第二图像合成为第三图像,具体包括:根据所述平移补偿的结果确定所述第一图像和所述第二图像的公共区域;将所述第一图像和所述第二图像的所述公共区域合成为所述第三图像。
- 一种图像处理装置,其特征在于,包括:获取模块,用于获取第一摄像头对第一区域采集的第一图像和第二摄像头在同一时刻对第二区域采集的第二图像,其中所述第一摄像头和所述第二摄像头位于所述图像处理装置的同一平面;平移补偿模块,连接于所述获取模块,用于在通过所述获取模块获得所述第一图像和所述第二图像之后,以所述第一图像为参考图像,对所述第二图像进行平移补偿;图像合成模块,连接于所述平移补偿模块,用于在通过所述平移补偿模块对所述第二图像进行平移补偿之后,将所述第一图像和平移补偿后的第二图像合成为第三图像,所述第三图像的分辨率高于所述第一图像和所述第二图像的分辨率。
- 如权利要求5所述装置,其特征在于,所述平移补偿模块,具体包括:确定单元,用于确定所述第一图像和所述第二图像之间的平移量;补偿单元,连接于所述确定单元,用于在基于所述确定单元确定所述平移量之后,对所述第二摄像头采集的所述第二图像依据所述平移量进行平移补偿。
- 如权利要求6所述装置,其特征在于,所述确定单元具体用于,通过以下公式确定所述平移量:d=B*f/Z;其中,d表示与第一摄像头和第二摄像头所在平面的距离为Z的物体在第二图像中相对于在第一图像中的平移量;B表示第一摄像头和第二摄像头的距离;Z表示物体与所述第一摄像头和所述第二摄像头所在平面的垂直距离,f表示所述第一摄像头的焦距或所述第二摄像头的焦距。
- 如权利要求6所述装置,其特征在于,所述图像合成模块,具体包括:确定单元,用于根据所述平移补偿的结果确定所述第一图像和所述第二图像的公共区域;合成单元,连接于所述确定单元,用于根据所述确定单元确定所述公共区 域之后,将所述第一图像和所述第二图像的所述公共区域合成为所述第三图像。
- 一种终端,其特征在于,包括:第一摄像头,用于对第一区域采集获得第一图像;第二摄像头,用于在所述第一摄像头采集所述第一图像的同一时刻对第二区域采集获得第二图像,所述第一摄像头与所述第二摄像头位于所述终端的同一侧面;处理器,连接于所述第一摄像头和所述第二摄像头,用于以所述第一图像为参考图像,对所述第二图像进行平移补偿;以及将所述第一图像和平移补偿后的第二图像合成为第三图像,所述第三图像的分辨率高于所述第一图像和所述第二图像的分辨率。
- 如权利要求9所述终端,其特征在于,所述第一摄像头和所述第二摄像头光轴平行和/或所述第一摄像头与所述第二摄像头固定设置于所述终端。
- 如权利要求9所述终端,其特征在于,所述处理器以所述第一图像为参考图像,对所述第二图像进行平移补偿,具体包括:确定所述第一图像和所述第二图像之间的平移量;对所述第二摄像头采集的所述第二图像依据所述平移量进行平移补偿。
- 如权利要求11所述终端,其特征在于,所述处理器确定所述第一图像和所述第二图像之间的平移量,具体为:通过以下公式确定所述平移量:d=B*f/Z;其中,d表示与第一摄像头和第二摄像头所在平面的距离为Z的物体在第二图像中相对于在第一图像中的平移量;B表示第一摄像头和第二摄像头的距离;Z表示物体与所述第一摄像头和所述第二摄像头所在平面的垂直距离,f表示所述第一摄像头的焦距或所述第二摄像头的焦距。
- 如权利要求11所述终端,其特征在于,所述处理器将所述第一图像和平移补偿后的第二图像合成为第三图像,具体包括:根据所述平移补偿的结果确定所述第一图像和所述第二图像的公共区域;将所述第一图像和所述第二图像的所述公共区域合成为所述第三图像。
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| EP14867399.9A EP3068124B1 (en) | 2013-12-06 | 2014-12-04 | Image processing method, device and terminal |
| US15/101,759 US9870602B2 (en) | 2013-12-06 | 2014-12-04 | Method and apparatus for fusing a first image and a second image |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310658550.8A CN103685951A (zh) | 2013-12-06 | 2013-12-06 | 一种图像处理方法、装置及终端 |
| CN201310658550.8 | 2013-12-06 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2015081870A1 true WO2015081870A1 (zh) | 2015-06-11 |
Family
ID=50322103
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2014/093024 Ceased WO2015081870A1 (zh) | 2013-12-06 | 2014-12-04 | 一种图像处理方法、装置及终端 |
Country Status (4)
| Country | Link |
|---|---|
| US (1) | US9870602B2 (zh) |
| EP (1) | EP3068124B1 (zh) |
| CN (1) | CN103685951A (zh) |
| WO (1) | WO2015081870A1 (zh) |
Families Citing this family (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103685951A (zh) | 2013-12-06 | 2014-03-26 | 华为终端有限公司 | 一种图像处理方法、装置及终端 |
| CN103888672A (zh) * | 2014-03-31 | 2014-06-25 | 宇龙计算机通信科技(深圳)有限公司 | 一种终端及终端拍摄方法 |
| US20150286878A1 (en) * | 2014-04-08 | 2015-10-08 | Bendix Commercial Vehicle Systems Llc | Generating an Image of the Surroundings of an Articulated Vehicle |
| JP6376618B2 (ja) * | 2014-05-15 | 2018-08-22 | 華為技術有限公司Huawei Technologies Co.,Ltd. | マルチフレームノイズ低減方法および端末 |
| KR101991754B1 (ko) * | 2014-08-29 | 2019-09-30 | 후아웨이 테크놀러지 컴퍼니 리미티드 | 이미지 처리 방법 및 장치, 그리고 전자 기기 |
| US10262400B2 (en) * | 2014-10-31 | 2019-04-16 | Huawei Technologies Co., Ltd. | Image processing method and device using reprojection error values |
| CN107409232B (zh) * | 2015-03-05 | 2020-06-09 | 索尼公司 | 图像处理设备和图像处理方法 |
| CN105049558B (zh) * | 2015-07-03 | 2017-10-03 | 广东欧珀移动通信有限公司 | 终端 |
| CN106612392A (zh) * | 2015-10-21 | 2017-05-03 | 中兴通讯股份有限公司 | 一种基于双摄像头拍摄图像的方法和装置 |
| CN105472245B (zh) * | 2015-12-21 | 2019-09-24 | 联想(北京)有限公司 | 一种拍照方法、电子设备 |
| WO2017166081A1 (zh) * | 2016-03-30 | 2017-10-05 | 华为技术有限公司 | 终端的图像配准方法、装置和终端 |
| CN106161997A (zh) * | 2016-06-30 | 2016-11-23 | 上海华力微电子有限公司 | 提高cmos图像传感器像素的方法及系统 |
| CN106027912A (zh) * | 2016-07-15 | 2016-10-12 | 深圳市金立通信设备有限公司 | 一种拍摄模式选择方法及终端 |
| CN107347139B (zh) * | 2017-06-30 | 2019-01-29 | 维沃移动通信有限公司 | 一种图像数据的处理方法和移动终端 |
| CN108492325B (zh) * | 2018-03-27 | 2020-06-30 | 长春理工大学 | 一种非共轴成像的图像配准装置、图像配准方法及系统 |
| CN108470327B (zh) * | 2018-03-27 | 2022-05-17 | 成都西纬科技有限公司 | 图像增强方法、装置、电子设备及存储介质 |
| CN109410130B (zh) * | 2018-09-28 | 2020-12-04 | 华为技术有限公司 | 图像处理方法和图像处理装置 |
| CN109963082B (zh) * | 2019-03-26 | 2021-01-08 | Oppo广东移动通信有限公司 | 图像拍摄方法、装置、电子设备、计算机可读存储介质 |
| CN110035206B (zh) * | 2019-03-26 | 2020-12-11 | Oppo广东移动通信有限公司 | 图像处理方法和装置、电子设备、计算机可读存储介质 |
| CN112954251B (zh) * | 2019-12-10 | 2023-03-21 | RealMe重庆移动通信有限公司 | 视频处理方法、视频处理装置、存储介质与电子设备 |
| CN112351271A (zh) * | 2020-09-22 | 2021-02-09 | 北京迈格威科技有限公司 | 一种摄像头的遮挡检测方法、装置、存储介质和电子设备 |
| WO2021184029A1 (en) | 2020-11-12 | 2021-09-16 | Innopeak Technology, Inc. | Systems and methods for fusing color image and near-infrared image |
| WO2021184027A1 (en) * | 2020-11-12 | 2021-09-16 | Innopeak Technology, Inc. | Tuning color image fusion towards original input color with adjustable details |
| WO2022103429A1 (en) * | 2020-11-12 | 2022-05-19 | Innopeak Technology, Inc. | Image fusion with base-detail decomposition and flexible color and details adjustment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1879401A (zh) * | 2003-11-11 | 2006-12-13 | 精工爱普生株式会社 | 图像处理装置、图像处理方法、其程序以及记录媒体 |
| CN102314678A (zh) * | 2011-09-06 | 2012-01-11 | 苏州科雷芯电子科技有限公司 | 图像分辨率提高装置及方法 |
| CN202143153U (zh) * | 2011-07-27 | 2012-02-08 | 天津三星光电子有限公司 | 数码相机 |
| CN102496158A (zh) * | 2011-11-24 | 2012-06-13 | 中兴通讯股份有限公司 | 一种图像信息处理方法及装置 |
| CN103685951A (zh) * | 2013-12-06 | 2014-03-26 | 华为终端有限公司 | 一种图像处理方法、装置及终端 |
Family Cites Families (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6208765B1 (en) * | 1998-06-19 | 2001-03-27 | Sarnoff Corporation | Method and apparatus for improving image resolution |
| JP2002524937A (ja) * | 1998-08-28 | 2002-08-06 | サーノフ コーポレイション | 高解像度カメラと低解像度カメラとを用いて高解像度像を合成する方法および装置 |
| US7525576B2 (en) * | 2003-02-17 | 2009-04-28 | Axis, Ab | Method and apparatus for panning and tilting a camera |
| KR100597587B1 (ko) * | 2004-04-13 | 2006-07-06 | 한국전자통신연구원 | 보정 영상 신호 처리를 이용한 주시각 제어 장치 및 그방법과 그를 이용한 평행축 입체 카메라 시스템 |
| US20070103544A1 (en) * | 2004-08-26 | 2007-05-10 | Naofumi Nakazawa | Panorama image creation device and panorama image imaging device |
| WO2008050904A1 (en) * | 2006-10-25 | 2008-05-02 | Tokyo Institute Of Technology | High-resolution vertual focusing-plane image generating method |
| US20090290033A1 (en) * | 2007-11-16 | 2009-11-26 | Tenebraex Corporation | Systems and methods of creating a virtual window |
| KR100985003B1 (ko) * | 2008-03-10 | 2010-10-04 | (주)세미솔루션 | 멀티 ccd 센서의 영상처리 장치 및 그 영상처리 방법 |
| JP2011523538A (ja) | 2008-05-20 | 2011-08-11 | ペリカン イメージング コーポレイション | 異なる種類の撮像装置を有するモノリシックカメラアレイを用いた画像の撮像および処理 |
| CN101930602B (zh) * | 2009-06-18 | 2012-05-30 | 张云超 | 一种高分辨率图像的生成方法、系统及电子终端 |
| CN102438153B (zh) * | 2010-09-29 | 2015-11-25 | 华为终端有限公司 | 多摄像机图像校正方法和设备 |
| US9204026B2 (en) * | 2010-11-01 | 2015-12-01 | Lg Electronics Inc. | Mobile terminal and method of controlling an image photographing therein |
| CN102075679A (zh) * | 2010-11-18 | 2011-05-25 | 无锡中星微电子有限公司 | 一种图像采集方法和装置 |
| US8878950B2 (en) | 2010-12-14 | 2014-11-04 | Pelican Imaging Corporation | Systems and methods for synthesizing high resolution images using super-resolution processes |
| US9516225B2 (en) * | 2011-12-02 | 2016-12-06 | Amazon Technologies, Inc. | Apparatus and method for panoramic video hosting |
| CN103167223A (zh) * | 2011-12-09 | 2013-06-19 | 富泰华工业(深圳)有限公司 | 具有广角拍摄功能的移动装置及其影像撷取方法 |
| US10681304B2 (en) * | 2012-06-08 | 2020-06-09 | Apple, Inc. | Capturing a panoramic image using a graphical user interface having a scan guidance indicator |
-
2013
- 2013-12-06 CN CN201310658550.8A patent/CN103685951A/zh active Pending
-
2014
- 2014-12-04 US US15/101,759 patent/US9870602B2/en active Active
- 2014-12-04 EP EP14867399.9A patent/EP3068124B1/en active Active
- 2014-12-04 WO PCT/CN2014/093024 patent/WO2015081870A1/zh not_active Ceased
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1879401A (zh) * | 2003-11-11 | 2006-12-13 | 精工爱普生株式会社 | 图像处理装置、图像处理方法、其程序以及记录媒体 |
| CN202143153U (zh) * | 2011-07-27 | 2012-02-08 | 天津三星光电子有限公司 | 数码相机 |
| CN102314678A (zh) * | 2011-09-06 | 2012-01-11 | 苏州科雷芯电子科技有限公司 | 图像分辨率提高装置及方法 |
| CN102496158A (zh) * | 2011-11-24 | 2012-06-13 | 中兴通讯股份有限公司 | 一种图像信息处理方法及装置 |
| CN103685951A (zh) * | 2013-12-06 | 2014-03-26 | 华为终端有限公司 | 一种图像处理方法、装置及终端 |
Non-Patent Citations (1)
| Title |
|---|
| See also references of EP3068124A4 * |
Also Published As
| Publication number | Publication date |
|---|---|
| EP3068124A1 (en) | 2016-09-14 |
| EP3068124A4 (en) | 2017-01-04 |
| CN103685951A (zh) | 2014-03-26 |
| EP3068124B1 (en) | 2026-03-25 |
| US20160307300A1 (en) | 2016-10-20 |
| US9870602B2 (en) | 2018-01-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103685951A (zh) | 一种图像处理方法、装置及终端 | |
| EP3067746B1 (en) | Photographing method for dual-camera device and dual-camera device | |
| US10616549B2 (en) | Application processor for disparity compensation between images of two cameras in digital photographing apparatus | |
| CN103986867B (zh) | 一种图像拍摄终端和图像拍摄方法 | |
| JP5683025B2 (ja) | 立体画像撮影装置および立体画像撮影方法 | |
| JP5725975B2 (ja) | 撮像装置及び撮像方法 | |
| JP5230013B2 (ja) | 撮像装置 | |
| WO2015081563A1 (zh) | 一种生成图片的方法及一种双镜头设备 | |
| US10349040B2 (en) | Storing data retrieved from different sensors for generating a 3-D image | |
| WO2016184131A1 (zh) | 基于双摄像头拍摄图像的方法、装置及计算机存储介质 | |
| TWI502548B (zh) | 即時影像處理方法及其裝置 | |
| JP6021489B2 (ja) | 撮像装置、画像処理装置およびその方法 | |
| TWI615808B (zh) | 全景即時影像處理方法 | |
| EP3190566A1 (en) | Spherical virtual reality camera | |
| JP5509986B2 (ja) | 画像処理装置、画像処理システム、及び画像処理プログラム | |
| CN119011796B (zh) | 摄像透视vst中环境图像数据的处理方法、头显设备和存储介质 | |
| JP6732440B2 (ja) | 画像処理装置、画像処理方法、及びそのプログラム | |
| JP2019047145A (ja) | 画像処理装置、撮像装置、画像処理装置の制御方法およびプログラム | |
| US9832377B2 (en) | Data acquiring method and electronic device thereof | |
| JP2013110711A (ja) | 画像処理装置および画像処理方法、並びにプログラム | |
| JP6645949B2 (ja) | 情報処理装置、情報処理システム、および情報処理方法 | |
| JP2020086651A (ja) | 画像処理装置および画像処理方法 | |
| JP2024002631A (ja) | 画像処理装置、撮像装置、画像処理方法、コンピュータのプログラムおよび記録媒体 | |
| JP2022113478A (ja) | インテグラル立体表示システム及びその方法 | |
| JP2014102265A (ja) | 3d撮像装置 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 14867399 Country of ref document: EP Kind code of ref document: A1 |
|
| REEP | Request for entry into the european phase |
Ref document number: 2014867399 Country of ref document: EP |
|
| WWE | Wipo information: entry into national phase |
Ref document number: 2014867399 Country of ref document: EP |
|
| WWE | Wipo information: entry into national phase |
Ref document number: 15101759 Country of ref document: US |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| WWG | Wipo information: grant in national office |
Ref document number: 2014867399 Country of ref document: EP |