WO2023016232A1 - 图像拍摄的方法、设备、存储介质和程序产品 - Google Patents

图像拍摄的方法、设备、存储介质和程序产品 Download PDF

Info

Publication number
WO2023016232A1
WO2023016232A1 PCT/CN2022/107449 CN2022107449W WO2023016232A1 WO 2023016232 A1 WO2023016232 A1 WO 2023016232A1 CN 2022107449 W CN2022107449 W CN 2022107449W WO 2023016232 A1 WO2023016232 A1 WO 2023016232A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
camera
preview
zoom
image capturing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/107449
Other languages
English (en)
French (fr)
Inventor
卢圣卿
王宇
李智琦
朱聪超
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Honor Device Co Ltd
Original Assignee
Honor Device Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Honor Device Co Ltd filed Critical Honor Device Co Ltd
Priority to US18/278,923 priority Critical patent/US12389117B2/en
Priority to EP22855215.4A priority patent/EP4287638B1/en
Priority to CN202280068390.9A priority patent/CN118435615A/zh
Publication of WO2023016232A1 publication Critical patent/WO2023016232A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene
    • H04N23/71—Circuitry for evaluating the brightness variation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/45—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/631—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
    • H04N23/632—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/633—Control of cameras or camera modules by using electronic viewfinders for displaying additional information relating to control or operation of the camera
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/667—Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/681—Motion detection
    • H04N23/6811—Motion detection based on the image signal
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/681—Motion detection
    • H04N23/6812—Motion detection based on additional sensors, e.g. acceleration sensors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/682—Vibration or motion blur correction
    • H04N23/683—Vibration or motion blur correction performed by a processor, e.g. controlling the readout of an image memory
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/682—Vibration or motion blur correction
    • H04N23/685—Vibration or motion blur correction performed by mechanical compensation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/682—Vibration or motion blur correction
    • H04N23/685—Vibration or motion blur correction performed by mechanical compensation
    • H04N23/687—Vibration or motion blur correction performed by mechanical compensation by shifting the lens or sensor position
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/69—Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/95—Computational photography systems, e.g. light-field imaging systems
    • H04N23/951—Computational photography systems, e.g. light-field imaging systems by using two or more images to influence resolution, frame rate or aspect ratio
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/2628—Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Definitions

  • the present application relates to the field of computer technology, in particular to an image capturing method, device, storage medium and program product.
  • Mobile phone imaging technology is increasingly developed. In daily office or life, people are getting used to using mobile phones to capture every bit of life or work at any time. Usually, without the support of external equipment, the mobile phone is directly picked up by hand to shoot. In order to obtain a clearer and ideal imaging effect, people often try to stabilize their hands as much as possible in order to stabilize the photo preview interface before shooting as much as possible. However, image stabilization by the human hand is usually limited and cannot be completely still. Therefore, in the process of taking pictures with a mobile phone, it is inevitable to be accompanied by image shaking.
  • the body or hand shakes during the photo taking process it will cause the image to become blurred.
  • the image is magnified by a certain magnification in the photo preview interface, the jitter presented on the mobile phone screen will also be magnified, and it is difficult to obtain a stable photo preview image, thereby affecting the user experience.
  • the present application proposes an image capturing method, device, storage medium and program product.
  • the image collected by the camera is processed according to the zoom factor of the user in the camera application preview interface, and then the anti-shake processed image is presented on the camera preview interface.
  • the user experience is improved.
  • an image capturing method which displays a first preview interface of a camera application, the first preview interface includes a first preview image, and the first preview image is obtained after processing the first image collected by the camera , the first preview image corresponds to a first zoom ratio, and the first image corresponds to a first output mode of the camera; receiving a first user operation; in response to the first operation, displaying a first image of the camera application Two preview interfaces, the second preview interface includes a second preview image, the second preview image is obtained after processing the second image collected by the camera, the second preview image corresponds to a second zoom factor, and the first preview image The second image corresponds to a second output mode of the camera, wherein the second output mode is different from the first output mode, and the second zoom factor is different from the first zoom factor.
  • the focal length changes to the second zoom ratio, and the camera application takes pictures.
  • the preview interface presents the second preview interface, which corresponds to the second output mode, so that when taking pictures with the mobile phone, according to the zoom ratio, different output modes are presented on the photo preview interface. Through different output modes, it can be optimized in different light conditions. The resolution and clarity of the images on the camera preview interface of the camera application under certain conditions.
  • the camera application is a system camera application on a mobile phone or a tablet.
  • the first preview interface and the second preview are camera application preview interfaces.
  • the first output mode is one of Binning, Remosaic, Quadra, Quadra HDR or Stagger HDR. In this way, through different output modes, it is possible to optimize the resolution and definition of images on the photo preview interface of the camera application under different lighting conditions.
  • the second output mode is one of Binning, Remosaic, Quadra, Quadra HDR or Stagger HDR. In this way, through different output modes, it is possible to optimize the resolution and definition of images on the photo preview interface of the camera application under different lighting conditions.
  • the image capturing method further includes that the first output mode or the second output mode is determined based on the shooting brightness of the camera.
  • the output mode is adjusted according to different shooting brightness, and the resolution and clarity of the image on the camera application preview interface are improved, thereby improving user experience. It is especially suitable for night scenes. For example, when you need to shoot outdoor night scenes, and the indoor shooting brightness is bright, at this time, the image presented on the photo preview interface of the camera application should be the outdoor night scene, and the current brightness may be changed due to indoor lighting. Judgment is based on indoor light, which affects the effect of night scene shooting. In the embodiment of the present invention, in the current scene, the shooting brightness of the photo preview interface is determined according to the outdoor brightness. The resolution and clarity of the preview interface image.
  • the shooting brightness is determined based on the current ambient light brightness collected by the ambient light sensor. In this way, the current shooting brightness can be accurately reflected, so as to present different output modes according to the current shooting brightness, and optimize the resolution and clarity of images on the photo preview interface of the camera application under different lighting conditions.
  • the shooting brightness is determined based on average brightness information of the second preview image.
  • the current shooting brightness can be accurately reflected, so as to present different output modes according to the current shooting brightness, and optimize the resolution and clarity of images on the photo preview interface of the camera application under different lighting conditions.
  • the shooting brightness includes bright light and dark light.
  • different output modes are presented according to the current shooting brightness, and the resolution and definition of images on the camera preview interface of the camera application are optimized under different lighting conditions.
  • the first output mode is Binning. In this way, the resolution and clarity of images on the photo preview interface of the camera application under low-light conditions are optimized.
  • the second output mode is Binning. In this way, the resolution and clarity of images on the photo preview interface of the camera application under low-light conditions are optimized.
  • the first output mode is Remosaic. In this way, the resolution and clarity of images on the photo preview interface of the camera application under bright light conditions are optimized.
  • the second output mode is Remosaic. In this way, the resolution and clarity of images on the photo preview interface of the camera application under bright light conditions are optimized.
  • the first operation includes one of a gesture operation for zooming the first preview image, a screen click operation, or dragging a zoom progress bar.
  • a gesture operation for zooming the first preview image a screen click operation, or dragging a zoom progress bar.
  • the shooting brightness of the first preview interface and the second preview interface is bright light.
  • the method further includes: in response to switching the shooting brightness to dark light, the camera application displays a third preview interface, and the third preview The interface corresponds to the first output mode.
  • the camera application displays a third preview interface
  • the third preview The interface corresponds to the first output mode.
  • the method further includes: the second zoom magnification is greater than a preset switching magnification. In this way, when the preset zoom ratio is satisfied, different output modes are switched to improve the resolution and definition of images on the camera application preview interface.
  • the method further includes: receiving a second user operation; in response to the second operation, displaying a fourth preview interface of the camera application, so
  • the fourth preview interface includes a fourth preview image, the fourth preview image is obtained after processing the fourth image collected by the camera, the fourth preview image corresponds to the third zoom factor, and the fourth image corresponds to the In the first output mode of the camera, the third zoom ratio is smaller than the preset switching ratio.
  • the second operation includes one of a gesture operation for zooming the first preview image, a screen click operation, or dragging a zoom progress bar.
  • a gesture operation for zooming the first preview image a screen click operation, or dragging a zoom progress bar.
  • the method further includes: the second zoom ratio is greater than the preset anti-shake ratio, and the second image is based on the second zoom ratio via The second preview interface is obtained after digital zooming and electronic image stabilization perform pixel cropping.
  • the second zoom ratio satisfies the preset condition
  • the image collected by the camera is cropped by the ISP and EIS to reduce the degree of shaking generated when the user holds the mobile phone to take pictures, thereby improving the user experience.
  • the digital zooming includes performing pixel cropping based on the second zoom magnification and the second output mode.
  • combining the zoom ratio and the output mode to perform pixel cropping can improve the range and accuracy of pixel cropping, reduce the degree of shaking when the user holds the mobile phone to shoot, and improve user experience.
  • the electronic image stabilization includes a 3D gyroscope, 2D image feature points, and image perspective transformation, wherein the 3D gyroscope is suitable for 3D image smoothing.
  • the 3D gyroscope is suitable for 3D image smoothing.
  • the 3D gyroscope and the 2D image feature points perform pixel cropping based on the second zoom ratio.
  • the performance of shooting with the mobile phone is improved, the degree of shaking generated when the user holds the mobile phone is reduced, and the user experience is improved.
  • the image perspective transformation includes coordinate transformation. In this way, the range and accuracy of pixel cropping can be improved, the degree of shaking generated when the user holds the mobile phone to shoot is reduced, and the user experience is improved.
  • the second zoom ratio is greater than the preset anti-shake ratio, and the second image is digitally zoomed and electronically stabilized based on the second zoom ratio.
  • the second preview interface is obtained after pixel cropping.
  • the embodiment of the present application provides an electronic device, the electronic device includes a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, trigger The electronic device performs the method of any one of the first aspect.
  • the second aspect and any implementation manner of the second aspect correspond to the first aspect and any implementation manner of the first aspect respectively.
  • technical effects corresponding to the second aspect and any implementation manner of the second aspect reference may be made to the technical effects corresponding to the above-mentioned first aspect and any implementation manner of the first aspect, and details are not repeated here.
  • an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored, and when running on a computer, the computer executes the method of any one of the first aspect.
  • the third aspect and any implementation manner of the third aspect correspond to the first aspect and any implementation manner of the first aspect respectively.
  • technical effects corresponding to the third aspect and any implementation manner of the third aspect reference may be made to the technical effects corresponding to the above-mentioned first aspect and any implementation manner of the first aspect, and details are not repeated here.
  • an embodiment of the present application provides a computer program product, the computer program product includes a computer program, and when running on a computer, causes the computer to execute the method of any one of the first aspect.
  • the fourth aspect and any implementation manner of the fourth aspect correspond to the first aspect and any implementation manner of the first aspect respectively.
  • the technical effects corresponding to the fourth aspect and any one of the implementation manners of the fourth aspect refer to the above-mentioned first aspect and the technical effects corresponding to any one of the implementation manners of the first aspect, and details are not repeated here.
  • all or part of the program in the fifth aspect may be stored on a storage medium packaged with the processor, or part or all may be stored on a memory not packaged with the processor.
  • FIG. 1 is a schematic diagram of an electronic device provided by an embodiment of the present application.
  • FIG. 2 is a software structural block diagram of an electronic device provided in an embodiment of the present application.
  • FIG. 3 is a schematic diagram of a camera application preview scene provided by an embodiment of the present application.
  • Fig. 4 is a schematic diagram of photographing and imaging under dark light conditions provided by the embodiment of the present application.
  • FIG. 5 is a schematic diagram of outputting an image in a Binning manner provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of photographing and imaging under bright light conditions provided by the embodiment of the present application.
  • FIG. 7 is a schematic diagram of outputting an image in a Remosaic manner provided by an embodiment of the present application.
  • Fig. 8 is a schematic diagram of an image switching between dark and bright light according to an embodiment of the present application.
  • FIG. 9 is a schematic diagram of switching between bright and dark images in an image provided by an embodiment of the present application.
  • FIG. 10 is a schematic diagram of switching an image output mode provided by an embodiment of the present application.
  • FIG. 11 is an implementation example of an image stabilization provided by an embodiment of the present application.
  • FIG. 12 is another implementation example of image stabilization provided by the embodiment of the present application.
  • Fig. 13 is a schematic view of the field of view of Fig. 12 provided by the embodiment of the present application.
  • Fig. 14A is a schematic diagram of camera image displacement imaging provided by the embodiment of the present application.
  • FIG. 14B is a schematic diagram of another camera image displacement imaging provided by the embodiment of the present application.
  • Fig. 15 is a schematic diagram of an input and output diagram provided by the embodiment of the present application.
  • FIG. 16 is a schematic diagram of logical processing of an image processing provided by an embodiment of the present application.
  • Fig. 17 is a schematic diagram of the relationship between the input image and the output image at different magnifications provided by the embodiment of the present application;
  • FIG. 18 is a schematic diagram of a digital zoom implementation provided by an embodiment of the present application.
  • FIG. 19 is a schematic diagram of an electronic anti-shake implementation provided by an embodiment of the present application.
  • FIG. 20 is a schematic diagram of an electronic anti-shake algorithm framework provided by an embodiment of the present application.
  • Fig. 21 is another implementation mode provided by the embodiment of this application.
  • FIG. 22 is a schematic diagram of a logic implementation sequence provided by the embodiment of the present application.
  • FIG. 23 is a schematic diagram of switching between different cameras provided by the embodiment of the present application.
  • Fig. 24 is another implementation manner provided by the embodiment of the present application.
  • the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, and an antenna 2 , mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and A subscriber identification module (subscriber identification module, SIM) card interface 195 and the like.
  • SIM subscriber identification module
  • the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a touch sensor 180K, and an ambient light sensor 180L.
  • the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100 .
  • the electronic device 100 may include more or fewer components than shown in the figure, or combine certain components, or separate certain components, or arrange different components.
  • the illustrated components can be realized in hardware, software or a combination of software and hardware.
  • the processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU) wait. Wherein, different processing units may be independent devices, or may be integrated in one or more processors. For example, the processor 110 is configured to execute the method for detecting ambient light in the embodiment of the present application.
  • the controller may be the nerve center and command center of the electronic device 100 .
  • the controller can generate an operation control signal according to the instruction opcode and timing signal, and complete the control of fetching and executing the instruction.
  • a memory may also be provided in the processor 110 for storing instructions and data.
  • the memory in processor 110 is a cache memory.
  • the memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. Repeated access is avoided, and the waiting time of the processor 110 is reduced, thereby improving the efficiency of the system.
  • processor 110 may include one or more interfaces.
  • the interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transmitter (universal asynchronous receiver/transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), general-purpose input and output (general-purpose input/output, GPIO) interface, subscriber identity module (subscriber identity module, SIM) interface, and /or universal serial bus (universal serial bus, USB) interface, etc.
  • I2C integrated circuit
  • I2S integrated circuit built-in audio
  • PCM pulse code modulation
  • PCM pulse code modulation
  • UART universal asynchronous transmitter
  • MIPI mobile industry processor interface
  • GPIO general-purpose input and output
  • subscriber identity module subscriber identity module
  • SIM subscriber identity module
  • USB universal serial bus
  • the external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, so as to expand the storage capacity of the electronic device 100.
  • the external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. Such as saving music, video and other files in the external memory card.
  • the internal memory 121 may be used to store computer-executable program codes including instructions.
  • the processor 110 executes various functional applications and data processing of the electronic device 100 by executing instructions stored in the internal memory 121 .
  • the internal memory 121 may include an area for storing programs and an area for storing data. Wherein, the storage program area can store an operating system and an application program required by at least one function.
  • the data storage area can store data created during the use of the electronic device 100 .
  • the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (universal flash storage, UFS) and the like.
  • a non-volatile memory such as at least one magnetic disk storage device, flash memory device, universal flash storage (universal flash storage, UFS) and the like.
  • the pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal.
  • pressure sensor 180A may be disposed on display screen 194 .
  • pressure sensors 180A such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors.
  • a capacitive pressure sensor may be comprised of at least two parallel plates with conductive material.
  • the electronic device 100 determines the intensity of pressure according to the change in capacitance.
  • the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A.
  • the electronic device 100 may also calculate the touched position according to the detection signal of the pressure sensor 180A.
  • touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view short messages is executed. When a touch operation whose intensity is greater than or equal to the first pressure threshold acts on the icon of the short message application, the instruction of creating a new short message is executed.
  • the gyro sensor 180B can be used to determine the motion posture of the electronic device 100 .
  • the angular velocity of the electronic device 100 around three axes may be determined by the gyro sensor 180B.
  • the gyro sensor 180B can be used for image stabilization. Exemplarily, when the shutter is pressed, the gyro sensor 180B detects the shaking angle of the electronic device 100, calculates the distance that the lens module needs to compensate according to the angle, and allows the lens to counteract the shaking of the electronic device 100 through reverse movement to achieve anti-shake.
  • the gyro sensor 180B can also be used for navigation and somatosensory game scenes.
  • the magnetic sensor 180D includes a Hall sensor.
  • the electronic device 100 may use the magnetic sensor 180D to detect the opening and closing of the flip leather case.
  • the electronic device 100 when the electronic device 100 is a clamshell machine, the electronic device 100 can detect opening and closing of the clamshell according to the magnetic sensor 180D.
  • features such as automatic unlocking of the flip cover are set.
  • the acceleration sensor 180E can detect the acceleration of the electronic device 100 in various directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of electronic devices, and can be used in applications such as horizontal and vertical screen switching, pedometers, etc.
  • the distance sensor 180F is used to measure the distance.
  • the electronic device 100 may measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 may use the distance sensor 180F for distance measurement to achieve fast focusing.
  • Proximity light sensor 180G may include, for example, light emitting diodes (LEDs) and light detectors, such as photodiodes.
  • the light emitting diodes may be infrared light emitting diodes.
  • the electronic device 100 emits infrared light through the light emitting diode.
  • Electronic device 100 uses photodiodes to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it may be determined that there is an object near the electronic device 100 . When insufficient reflected light is detected, the electronic device 100 may determine that there is no object near the electronic device 100 .
  • the electronic device 100 can use the proximity light sensor 180G to detect that the user is holding the electronic device 100 close to the ear to make a call, so as to automatically turn off the screen to save power.
  • the proximity light sensor 180G can also be used in leather case mode, automatic unlock and lock screen in pocket mode.
  • the fingerprint sensor 180H is used to collect fingerprints.
  • the electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access to application locks, take pictures with fingerprints, answer incoming calls with fingerprints, and the like.
  • Touch sensor 180K also known as "touch panel”.
  • the touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”.
  • the touch sensor 180K is used to detect a touch operation on or near it.
  • the touch sensor can pass the detected touch operation to the application processor to determine the type of touch event.
  • Visual output related to the touch operation can be provided through the display screen 194 .
  • the touch sensor 180K may also be disposed on the surface of the electronic device 100 , which is different from the position of the display screen 194 .
  • the ambient light sensor 180L is used for sensing ambient light brightness.
  • the electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness.
  • the ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures.
  • the ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in the pocket, so as to prevent accidental touch.
  • the keys 190 include a power key, a volume key and the like.
  • the key 190 may be a mechanical key. It can also be a touch button.
  • the electronic device 100 can receive key input and generate key signal input related to user settings and function control of the electronic device 100 .
  • the motor 191 can generate a vibrating reminder.
  • the motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback.
  • touch operations applied to different applications may correspond to different vibration feedback effects.
  • the motor 191 may also correspond to different vibration feedback effects for touch operations acting on different areas of the display screen 194 .
  • Different application scenarios for example: time reminder, receiving information, alarm clock, games, etc.
  • the touch vibration feedback effect can also support customization.
  • the indicator 192 can be an indicator light, and can be used to indicate charging status, power change, and can also be used to indicate messages, missed calls, notifications, and the like.
  • the electronic device 100 realizes the display function through the GPU, the display screen 194 , and the application processor.
  • the GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering.
  • Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.
  • the display screen 194 is used to display images, videos and the like.
  • the display screen 194 includes a display panel.
  • the display panel may use an organic light-emitting diode (OLED).
  • OLED organic light-emitting diode
  • the electronic device 100 may include 1 or N display screens 194 , where N is a positive integer greater than 1.
  • the electronic device 100 can realize the shooting function through the ISP, the camera 193 , the video codec, the GPU, the display screen 194 and the application processor.
  • Camera 193 is used to capture still images or video.
  • the object generates an optical image through the lens and projects it to the photosensitive element.
  • the photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor.
  • CMOS complementary metal-oxide-semiconductor
  • the photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal.
  • the ISP outputs the digital image signal to the DSP for processing.
  • DSP converts digital image signals into standard RGB, YUV and other image signals.
  • the electronic device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1.
  • Video codecs are used to compress or decompress digital video.
  • the electronic device 100 may support one or more video codecs.
  • the electronic device 100 can play or record videos in various encoding formats, for example: moving picture experts group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4 and so on.
  • MPEG moving picture experts group
  • FIG. 2 it is a software structural block diagram of an electronic device provided by an embodiment of the present application.
  • the layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Layers communicate through software interfaces.
  • the Android (Android) system is divided into four layers, which are respectively an application layer, a framework layer, a hardware abstraction layer and a hardware layer from top to bottom.
  • the application layer may include a series of application packages.
  • the application package may include a camera application.
  • the application layer can be divided into application interface (UI) and application logic.
  • the application interface of the camera application includes a single-view mode, a dual-view mode, a picture-in-picture mode, etc., corresponding to different image or video shooting modes.
  • the application logic of the camera application includes the multi-camera framework and camera management.
  • the multi-camera framework includes a switching control module, a Surface switching management module, a multi-camera encoding module, and a transition control module.
  • the switching control module is used to control the switching of shooting modes, such as the switching between the proactive mode, the rear shooting mode, the front-to-back mode, the back-to-back mode, and the picture-in-picture mode.
  • the switching process of the shooting mode may involve turning on or off a specific camera, shielding hardware differences of different chip platforms, and the like.
  • the Surface switching management module is used to control Surface switching during the shooting mode switching process.
  • Surface corresponds to a screen buffer, which is used to save the pixel data of the current window.
  • the Surface switching management module is used to control switching between the preview Surface and the encoding Surface during the shooting mode switching process.
  • the multi-cam encoding module is used for encoding during the shooting mode to generate image or video files, that is, to realize the recording of captured images or videos.
  • the transition control module is used to generate transition motion effects during the switching of shooting modes.
  • Camera management includes device management module, Surface management module, session management module, etc.
  • the framework layer provides application programming interface (application programming interface, API) and programming framework for the application program of the application layer, including some predefined functions.
  • the framework layer includes camera framework and media framework.
  • the camera framework may be a camera access interface (Camera2API), which is a set of interfaces for accessing camera devices introduced by Android. It adopts a pipeline design to make data flow from the camera to the Surface.
  • Camera2API includes camera management (CameraManager) and camera equipment (CameraDevice).
  • CameraManager is the management class of the Camera device, through which the camera device information of the device can be queried to obtain the CameraDevice object.
  • CameraDevice provides a series of fixed parameters related to Camera devices, such as basic settings and output formats.
  • the media framework includes media codec (MediaCodec), audio recording module (AudioRecord) and video and audio multiplexer (Muxer).
  • MediaCodec is a class provided by Android for encoding and decoding audio and video. It realizes the function of encoding and decoding by accessing the underlying codec, and is a part of the Android media basic framework.
  • the main function of AudioRecord is to enable various applications to manage audio resources so that they can record sounds collected by hardware.
  • Muxer is used to combine video compression data (such as H.264) and audio compression data (such as AAC) into one package format data.
  • the hardware abstraction layer is an interface layer between the operating system kernel and the hardware circuit, and its purpose is to abstract the hardware. It hides the hardware interface details of a specific platform, provides a virtual hardware platform for the operating system, makes it hardware-independent, and can be transplanted on various platforms.
  • the HAL includes the camera hardware abstraction layer (Camera HAL), and the Camera HAL includes the image signal processor (Image Signal Processor, ISP) and so on.
  • HAL also includes an integrated framework, which includes Electric Image Stabilization (EIS) and image perspective transformation. It can be understood that the image signal processor, electronic image stabilization, and image perspective transformation are abstract devices. In image or video shooting scenarios, HAL will create a data stream of a corresponding size according to the resolution delivered by the upper layer and the size of the Surface.
  • the kernel layer (Kernel) is the layer between hardware and software.
  • the kernel layer includes at least display driver, camera driver, sensor driver, etc.
  • the embodiment of the present application provides a method for capturing images.
  • This method can be applied to when the user holds the mobile phone and opens the mobile phone camera application to take pictures.
  • the preview interface images unclear scenes.
  • the user holds the mobile phone with one hand or both hands and opens the camera application of the mobile phone.
  • the image zoom function (Zoom) of the camera is turned on to enlarge the image. Due to the unavoidable shaking of the hand or body, the picture displayed on the preview interface of the mobile phone will become blurred due to shaking.
  • the form of the electronic device is only illustrated with a mobile phone, and the electronic device may also be a handheld electronic device with a camera function such as a tablet computer and a handheld computer, which is not limited here.
  • Optical Image Stabilization is a physical anti-shake technology realized by the special motor structure of the camera lens.
  • the gyroscope is used to calculate the shaking of the mobile phone during the shooting process, and the signal is transmitted to the microprocessor. After calculating the displacement to be compensated, the camera is controlled to move for optical compensation to achieve anti-shake.
  • OIS is limited to jitter within a small range, and it is difficult to compensate for large-angle jitter.
  • the output mode of the image can be determined according to the shooting brightness.
  • Shooting brightness refers to the brightness of the environment in which the camera of the phone captures images. When the brightness of the shooting environment is brighter, the images captured by the mobile phone will receive more light. Conversely, when the brightness of the shooting environment is lower, the images captured by the mobile phone will receive less light.
  • the ambient light sensor in the hardware module of the mobile phone detects the current ambient light brightness to determine the brightness of the image currently presented on the camera photo preview interface.
  • the brightness detection and judgment of the camera photo preview interface includes but is not limited to the situations listed above. In actual use, other methods can be used to obtain the image brightness of the current camera photo preview interface according to actual needs.
  • the mobile phone After determining the current shooting brightness, the mobile phone selects the corresponding shooting mode to output pictures to optimize the clarity under different shooting brightness.
  • Figure 4 it is an imaging example that detects that the mobile phone is taking pictures under dark light conditions.
  • the initial interface refers to the initial interface when the user opens the camera application.
  • the first preview interface is the initial interface
  • the first zoom magnification is the initial zoom magnification
  • the initial zoom magnification is 1 times. At this time, the shooting brightness is low light.
  • the user performs a first operation.
  • the first operation includes a gesture operation for zooming the first preview image, a screen click operation, or a drag Drag one of the zoom progress bars.
  • the user touches the screen to magnify the picture to a desired magnification, ie, the second zoom magnification, through gesture control or clicking a zoom switch on the photo preview interface, and the camera application displays the second preview interface.
  • the maximum magnification that the mobile phone can support by default is 20 times, and the user enlarges the image on the photo preview interface by 15 times at this time, that is, the image magnification of (b) in Figure 4 is 15 times that of (a) in Figure 4 .
  • the camera application is in the photo preview interface
  • the camera captures the first image
  • the mobile phone is set to Binning mode output pictures in the initial interface of the camera application, that is, in Figure 4
  • the image output mode of (a1) is Binning, that is, the first output mode.
  • the camera collects the second image, and the mobile phone still outputs the picture in Binning mode to ensure the presentation
  • the image definition on the mobile phone screen, that is, the picture output mode of (b1) in FIG. 4 is still Binning, that is, the second output mode.
  • the first output mode is one of Binning, Remosaic, Quadra, Quadra HDR or Stagger HDR.
  • the second output mode is one of Binning, Remosaic, Quadra, Quadra HDR or Stagger HDR.
  • Binning mode when the electronic device is capturing an image, the light reflected by the target object is collected by the camera, so that the reflected light is transmitted to the image sensor.
  • the image sensor includes a plurality of photosensitive elements, and the charge collected by each photosensitive element is a pixel, and an analog binning (Binning) operation is performed on the pixel information.
  • Binning can combine n ⁇ n pixels into one pixel.
  • Binning can synthesize adjacent 2 ⁇ 2 pixels into one pixel, that is, the colors of adjacent 2 ⁇ 2 pixels are presented in the form of one pixel.
  • a second user operation is received, and a fourth preview interface of the camera application is displayed, and the second operation includes one of a gesture operation for zooming the first preview image, a screen click operation, or dragging a zoom progress bar.
  • the fourth preview interface includes a fourth preview image, the fourth preview image is obtained after processing the fourth image captured by the camera, and the fourth preview image corresponds to the third zoom ratio.
  • the user controls the second preview interface in (b) in FIG. 4 to the required magnification by gesture control or clicking the zoom switch on the photo preview interface through the touch screen, which is the third zoom. magnification.
  • the image displays the fourth preview interface at this time, and the camera collects the third image
  • the output mode is Binning, that is, the first output mode.
  • FIG. 5 it is a schematic diagram of a process of outputting an image in a binning manner after the mobile phone acquires the image.
  • (b0) in FIG. 5 is a schematic diagram of a 4 ⁇ 4 pixel, and adjacent 2 ⁇ 2 pixels are synthesized into one pixel.
  • (b1) in Figure 5 is a schematic diagram of pixels output by Binning.
  • the 2 ⁇ 2 pixels in the 01 area of (b0) in Figure 5 are formed into the pixel R in (b1) in Figure 5; the 02 area of (b0) in Figure 5 is The 2 ⁇ 2 pixels in Figure 5 form the pixel G in (b1) in Figure 5; the 2 ⁇ 2 pixels in the 03 area of (b0) in Figure 5 form the pixels in (b1) in Figure 5 G: Use 2 ⁇ 2 pixels in the 04 area of (b0) in FIG. 5 to form pixel B in (b1) in FIG. 5 .
  • the Bayer format image refers to an image that only includes red, blue and green (ie three primary colors) in the image. For example, pixel A formed by 2 ⁇ 2 pixels in area 01 is red, pixel B formed by 2 ⁇ 2 pixels in area 02 is green, pixel C formed by 2 ⁇ 2 pixels in area 03 is green, Pixel D formed by 2 ⁇ 2 pixels in the 04 area is blue.
  • FIG. 6 shows the initial interface when the user picks up the mobile phone to open the camera application, that is, the first preview interface. At this time, the shooting brightness is bright light.
  • the initial zoom ratio is 1 times, that is, the first zoom ratio.
  • the user performs a first operation.
  • the first operation includes a gesture operation for zooming the first preview image, a screen click operation, or a dragging Drag one of the zoom progress bars.
  • the maximum magnification that the mobile phone can support by default is 20 times.
  • the mobile phone under the initial interface of (a) in FIG. 6, the mobile phone is set to output pictures in Binning mode on the initial interface of the camera application, that is, the picture output mode of (a1) in FIG. 6 is Binning, that is, the first output model. And because it is a bright light condition at this time, the camera collects the second image, and after the imaging picture in the mobile phone photo preview interface is enlarged 15 times to the state of (b) in Figure 6, that is, the second preview interface, the mobile phone switches to Remosaic mode output Picture, that is, the second output mode, to ensure the clarity of the image presented on the screen of the mobile phone, that is, the picture output mode of (b1) in Figure 6 is Remosaic, after further cutting such as half the size of the side length, that is In the (b2) state in Figure 6, the image is output.
  • a second user operation is received, and a fourth preview interface of the camera application is displayed, and the second operation includes one of a gesture operation for zooming the first preview image, a screen click operation, or dragging a zoom progress bar.
  • the fourth preview interface includes a fourth preview image, the fourth preview image is obtained after processing the fourth image captured by the camera, and the fourth preview image corresponds to the third zoom ratio.
  • the user controls the second preview interface in (b) in FIG. 6 to a desired magnification by gesture control on the touch screen or by clicking the image zoom switch on the photo preview interface, which is the third zoom. magnification.
  • the image displays the fourth preview interface at this time, and the camera collects the third image
  • the output mode is Binning, that is, the first output mode.
  • Remosaic mode When Remosaic mode is used to output images, the Bayer format images are rearranged to obtain Remosaic mode images. For example, assuming that a pixel in an image is composed of n ⁇ n pixels, then using Remosaic can rearrange n ⁇ n pixels in a pixel in the image.
  • (b0) in FIG. 7 is a schematic diagram of a pixel, and each pixel is composed of adjacent 2 ⁇ 2 pixels.
  • (b1) in FIG. 7 is a schematic diagram of an image output in Bayer format using the Remosaic method. Specifically, the pixels in (b0) in FIG. 7 are red, green and blue. Divide each pixel in (b0) in Fig. 7 into 2 ⁇ 2 pixels and rearrange them respectively. That is, the Remosaic mode is adopted for output, and the output image is an image in the Bayer format shown in (b1) in FIG. 7 .
  • FIG. 8 it is an example of image output when the shooting brightness changes from dark to bright after the image is zoomed to a certain magnification.
  • the output mode is the Binning mode, that is, the mode of (a1) in FIG. 8 , that is, the second output mode. Due to the movement of the user when holding the mobile phone to take pictures, that is, moving from a dark light environment to a bright light environment, the brightness of the mobile phone shooting changes.
  • the camera application displays the third preview interface, which is Figure 8
  • the image output mode is switched to the Remosaic mode, that is, the mode of (b1) in FIG. 8 , that is, the first output mode.
  • FIG. 9 it is an example of image output when the shooting brightness changes from bright to dark after the image is zoomed to a certain magnification. Specifically, when the image is enlarged to, for example, 15 times, that is, from the first zoom ratio to the second zoom ratio, that is, the state of (a) in FIG. 9 , since the brightness is judged to be in a bright light state at this time, the picture output
  • the mode is the Remosaic mode, that is, the mode of (a1) in FIG. 9 , that is, the second output mode.
  • the camera application displays the third preview interface, as shown in Fig.
  • the picture output mode is switched to the Binning mode, that is, the mode of (b1) in FIG. 9 , that is, the first output mode.
  • the mobile phone switches to the Remosaic mode to output pictures when shooting the indoor scene.
  • the mobile phone needs to shoot a dark scene outside the window from a bright room, because the brightness outside the window is detected to be dark, even though the indoor light is bright, the mobile phone still needs to switch to Binning mode when shooting the mirror image outside the window to ensure the image quality. clarity.
  • FIG. 10 it is a logic example for a mobile phone to select a corresponding image sensor configuration according to brightness and output a picture. Specifically, include the following steps:
  • S100 The user opens the mobile phone camera application. At this point, the camera application displays an initial photo preview interface, the default initial interface is the state of zooming the image by 1 times (ie 1x), and the default initial image output mode is Binning mode.
  • S101 The user operates the photo preview interface of the camera to zoom in or zoom out the image in the real-time preview. It can be understood that the user's image zooming includes gesture operations on the touch screen, clicking on the image zooming button on the screen of the mobile phone, and the like.
  • the user enlarges the displayed image of the photo preview interface of the camera application to 15 times in real time.
  • the zoomed image can be optimized for definition.
  • the shooting brightness is detected.
  • the ambient light sensor in the hardware module of the mobile phone detects the current ambient light brightness to determine the brightness of the image currently presented on the camera photo preview interface.
  • the brightness of the image currently presented on the camera preview interface is determined according to the image average brightness information of the current camera preview interface, including the color of the image, the pixel value of each pixel in the image, and the like.
  • the brightness detection and judgment of the camera photo preview interface includes but is not limited to the situations listed above. In actual use, other methods can be used to obtain the image brightness of the current camera photo preview interface according to actual needs.
  • S102 will be triggered only when the image in S101 is zoomed to a value greater than the preset switching magnification.
  • the image output mode of S102 is Binning mode to output pictures.
  • the preset switching magnification is 6-10 times. Exemplarily, 8 times is used as the preset switching magnification.
  • the image pixels are output as 4608*3456 by the 16MBinning mode.
  • the image pixel output is 9216*6912 by the 64MRemosaic mode, and on this basis, the image sensor is further cropped by half to the image pixel output is 4608*3456, This results in the same output pixel in dark and bright light conditions. In this way, additional power consumption in subsequent steps such as S103 when the ISP continues to crop images can be saved.
  • the mobile phone continues to optimize the jitter of the image presented on the mobile phone screen.
  • FIG. 11 it is an implementation example of image stabilization according to the embodiment of the present application.
  • the camera application is set by the user to zoom in 15 times to show the distant figure.
  • the image on the camera preview interface presented by the camera application of the mobile phone is still stable, thereby improving the user's shooting accuracy. experience.
  • FIG. 12 it is another implementation example of image stabilization according to the embodiment of the present application.
  • an anti-shake switch is additionally set on the photo preview interface of the camera application.
  • the anti-shake function is enabled in the photo preview displayed in real time, that is, in the state of (a) in Figure 12, the image displayed on the photo preview interface of the camera application shakes with the shaking of the mobile phone, making the image displayed on the mobile phone screen change. blurred and unclear.
  • the anti-shake function is turned on, that is, the state of (b) in Figure 12, even if the mobile phone shakes when holding the mobile phone, it can still ensure that the image on the photo preview interface can be output stably.
  • FOV Field of View
  • FOV image 1 is the image collected by the camera.
  • FOV image 2 the image displayed on the camera preview interface of the mobile phone is output, that is, the FOV image 2 after image processing is the output image in the figure.
  • Field of View It is used to indicate the image output range of the camera when the camera is capturing images. If the object to be photographed is within the angle range, the object to be photographed will be captured by the camera and then presented in the preview image. If the object to be photographed is outside the angle range, the device to be photographed will not be captured by the image capturing device, that is, it will not appear in the preview image.
  • FOV Field of View
  • viewing angle may also be referred to as “viewing field range”, “viewing field range” and other words. This article does not limit the name of "field of view”, as long as it expresses the above concept.
  • the number of cameras may be one or more, which is not limited here.
  • the form of the camera installed on the electronic device is not limited, it may be a built-in camera or an external camera; it may be a front camera or a rear camera.
  • the positions of the multiple cameras can be arranged arbitrarily, for example, a camera can be located on the left or right side of another camera, and a camera can also be located on the upper side of another camera or downside. That is to say, the present application does not limit the relative positional relationship between any camera in the camera module and other cameras.
  • FIG. 14A , 14B and 15 it is an example of camera image imaging when the mobile phone is displaced during the shooting process.
  • FIG. 14A it is a scene where the image is shifted left and right due to hand shaking during the shooting process of the mobile phone.
  • FOV image 2 the image presented in real time on the photo preview interface of the camera application is a stabilized processed FOV image 2 .
  • FOV image 1 is the input image
  • FOV image 2 is the output image presented on the screen of the mobile phone after the image processing method of this application.
  • FIG. 10 elaborates step S100 to step S102 in detail, which will not be repeated here.
  • step S103 to step S105 will be described in detail below. Specifically, include the following steps:
  • the digital zooming includes the ISP zooming the image to the zoom ratio selected by the user and outputting it.
  • the electronic anti-shake includes performing image anti-shake processing on the received image.
  • S105 Outputting the processed image to the camera preview interface of the mobile phone camera application for real-time display.
  • the S104 can only be turned on and perform anti-shake processing on the output image when the user's zoom ratio is greater than the preset anti-shake ratio.
  • the zoom ratio is smaller than the preset anti-shake ratio, the image input by S102 is digitally zoomed by S103 and then output to S105.
  • triggering S104 to enable image stabilization with a preset anti-shake magnification of 6-15 times.
  • FIG. 17 it is an example of the relationship between the input image and the output image under different magnifications, and further illustrates the above-mentioned preset anti-shake magnification that triggers S104 to be turned on.
  • the image stabilization algorithm is turned on, that is, step S104, and the processed image is output to S105 and displayed on the mobile phone camera application photo preview interface.
  • the input image is digitally zoomed through S103 and output to S105 to be displayed on the camera preview interface of the mobile phone camera application.
  • the purpose of retaining the input image at the preset anti-shake ratio (that is, 10 times) at 10 times is to reserve more activities for the cropping of subsequent image scaling to ensure the shaking range of the image, thereby improving image scaling Image stabilization is now implemented in the photo preview interface of the mobile camera app at maximum magnification.
  • An example of the photo preview interface displayed on the mobile phone camera application after the image processing method disclosed in the present application is the output image.
  • the implementation process of the digital zoom S103 is elaborated in detail. It can be concluded from the above step S102 that under bright and dark light conditions when the image is zoomed to 15 times, the size of the image input to the digital zoom S103 is 4608*3456. For example, in low-light conditions, since the zoom ratio is 15 times, the preset anti-shake ratio is 10 times, and the telephoto ratio is 4 times, the ISP crops 4/10 according to the first ratio to obtain a 1842*1382 image. The ISP continues to downsample the above image by a factor of 0.78 to 1440*1080.
  • Another example is under bright light conditions, since the zoom ratio is 15 times, the preset anti-shake ratio is 10 times, and the preset switching ratio is 8 times, so the ISP crops 8/10 according to the second ratio to obtain a 3684*2764 image.
  • the ISP proceeds to downsample the above image by a factor of 0.29 to 1440*1080.
  • the electronic image stabilization S104 is executed by at least the electronic image stabilization EIS1041 module and the image perspective transformation 1042 module.
  • the data information required by the electronic image stabilization EIS1041 includes 3D smoothing, 2D image and optical image stabilization OIS.
  • image perspective transformation 1042 includes coordinate transformation, pixel cropping and upsampling.
  • the ISP input image 1440*1080 is processed by electronic anti-shake EIS, and then sent to the image for perspective transformation , through coordinate transformation and cropping 10/15 to 960*720, upsampled by 1.5 times to 1440*1080 output image, and sent to the photo preview interface of the mobile phone camera application to display S105.
  • the image is a stable output image obtained after the image processing method of the present application, which is consistent with the image pixels output by the ISP.
  • FIG. 20 it is a schematic diagram of the electronic anti-shake algorithm framework of the present application, including 3D gyroscope anti-shake and 2D image translation.
  • 3D gyroscope anti-shake includes gyroscope, through boundary constraints to path smoothing, from path smoothing to virtual camera pose, gyroscope also outputs data to virtual camera pose.
  • path smoothing and virtual camera pose constitute smooth path estimation.
  • the virtual camera pose continues to be output to the homography change, and is output to the image perspective transformation information after receiving the data of the optical image stabilization OIS.
  • 2D image translation includes image frames, which are output to feature point detection and extraction, and then feature point pairs are filtered to motion vectors, and then bound to path smoothing to obtain the translation amount and then output to image perspective transformation information.
  • the image perspective transformation information receives the homography transformation of the 3D gyroscope and the translation amount of the 2D image translation, the image perspective transformation is performed on the image frame, and finally a stable image frame is output.
  • the 3D gyroscope anti-shake and the 2D image translation cooperate with each other, especially the 2D image translation can adjust the high-frequency jitter in the above-mentioned photo preview interface.
  • the 3D gyroscope anti-shake and the 2D image translation control the 3D gyroscope anti-shake and the 2D image translation respectively according to the zoom factor set by the user, and dynamically adjust the image weights of the two.
  • the weight relationship between the two can be used, for example The following relationship expresses:
  • 3D gyroscope anti-shake weight 1-2D image translation weight
  • the 2D image translation weight can be expressed by the following relationship:
  • the weights of the 3D gyroscope image stabilization and 2D image translation will be adjusted accordingly, and the weight of the 3D gyroscope image stabilization will gradually increase as the zoom magnification set by the user increases.
  • the weight of the 2D image translation will gradually increase with the increase of the zoom ratio set by the user, until it reaches the full zoom ratio of the 2D image translation, EIS only uses the image feature points for anti-shake, and does not use 3D gyro image stabilization.
  • the cooperation of various parameters can be dynamically considered in the photo preview interface, and the anti-shake of the photo preview interface can be controlled more flexibly to achieve a better anti-shake effect.
  • the full-open magnification of 2D image translation is the maximum magnification supported by 2D image translation, such as 50 times or other zoom magnifications, which is not limited in this application.
  • the weight of the 2D image translation is 9/49
  • the 3D gyroscope anti- The weight of shaking is 1-9/49, which is 40/49.
  • the initial zoom ratio set by the user is 1x
  • the zoom ratio currently set by the user is 50x
  • the 2D image translation full-on magnification is 50x
  • the weight of 2D image translation is 49/49
  • the weight of 3D gyroscope image stabilization is 49/49.
  • the weight is 1-49/49, which is 0.
  • the weight of the 3D gyro image stabilization will be reduced to 0, that is, the 3D gyro image stabilization does not participate in the image stabilization of the photo preview interface, but only 2D image translation participates in the anti-shake of the photo preview interface. In this way, especially when the zoom ratio set by the user is high, the anti-shake effect and benefits of the camera preview interface will be more obvious.
  • the user can see more stable pictures on the photo preview interface under different zoom ratios, and reduce the image shake generated when the user holds the mobile phone to shoot, so that the user can more easily see the images in the picture through the stable image. content. It should be noted that enhancing the anti-shake presented on the photo preview interface to make the picture more stable so that users can see the content in the picture clearly does not adjust the clarity of the image, but the clarity of the image remains unchanged.
  • Fig. 21 is another embodiment of the present application.
  • the difference from the above embodiment is that in this embodiment, fixed cropping can be performed on the output image.
  • the image stabilization algorithm is turned on, that is, step S104, and the processed image is output to S105 to be displayed on the camera preview interface of the mobile phone camera application.
  • the input image is digitally zoomed through S103 and output to S105 to be displayed on the camera preview interface of the mobile phone camera application.
  • the zoom ratio is greater than the preset anti-shake ratio at this time, that is, the zoom ratio is greater than 10 times, turn on the electronic image stabilization S104.
  • the image is zoomed to (c) as shown in Figure 21, since it is a fixed crop at this time, in order to obtain the actual output image is the data after the user has enlarged the image to 15 times, it is necessary to reserve a predetermined magnification smaller than the 15 times The active range of the image.
  • the predetermined multiplier is 15 times multiplied by a preset coefficient.
  • the preset coefficient is 80%, that is, when the output image is at a magnification of 80% of 15 times, that is, at a magnification of 12, the input image under a predetermined magnification (that is, 12 times) is retained, that is, (c in FIG. 21 ), when the image continues to be zoomed to 15 times, the electronic image stabilization S104 crops the pixels to the image zoomed to 15 times selected by the user. It can be understood that the subsequent image scaling is to output the image in the above manner. Exemplarily, as shown in (d) of FIG.
  • the preset coefficient is multiplied, that is, 80% (16 times), The input image is retained, and when the image continues to be zoomed to 20 times, the electronic image stabilization S104 crops the pixels to the image zoomed to 20 times selected by the user.
  • the preset coefficient is 60%-90%.
  • An example of the photo preview interface displayed on the mobile phone camera application after the image processing method disclosed in the present application is the output image.
  • the implementation process of digital zoom S103 is described in detail. It can be concluded from the above step S102 that under bright and dark light conditions when the image is zoomed to 15 times, the size of the image input to the digital zoom S103 is 4608*3456. For example, in dark light conditions, since the zoom ratio is 15 times, the preset anti-shake ratio is 10 times, and the basic ratio of the telephoto lens is 10 times, the ISP crops 4/15 according to the first ratio to get a 1228*920 image . The ISP continues to upsample the above image by a factor of 1.41 to 1728*1296.
  • Another example is under bright light conditions, since the zoom ratio is 15 times, the preset anti-shake ratio is 10 times, and the preset switching ratio is 8 times, so the ISP crops 4/15 according to the second ratio to obtain a 2456*1840 image.
  • the ISP continues to downsample the above image by 0.7 times to 1728*1296.
  • the implementation process of digital zoom S104 is described in detail.
  • the ISP input image 1728*1296 is processed by electronic anti-shake EIS and sent to The perspective transformation of the input image is carried out, and the output image is cut by 20% to 1440*1080 through coordinate transformation, and sent to the photo preview interface of the mobile phone camera application to display S105.
  • the image is a stable output image obtained after the image processing method of the present application.
  • FIG. 22 is a timing diagram of a logic implementation of the embodiment of the present application.
  • the zoom ratio is realized in the following steps.
  • the above-mentioned user-selected zoom ratio of 15 times is still used for illustration.
  • the zoom ratio of the initial interface of the camera application is 1 times.
  • the main camera captures images within the shooting range of the mobile phone, and when the zoom ratio is enlarged to 4 times, it switches to the telephoto camera to capture images within the shooting range of the mobile phone.
  • the preset switching ratio is 8 times
  • the preset anti-shake ratio is 10 times. It can be understood that the ranges of the preset switching magnification and the preset anti-shake magnification can be changed according to different scenarios, which have been described above and will not be repeated here.
  • the image is output in Binning mode.
  • brightness detection is performed.
  • the image is output in Binning mode, and when the current brightness is detected to be bright, the image is output in Remosaic mode.
  • the brightness detection is turned on when the image is zoomed to 8 times, and when the zoom ratio is greater than 8 times, the brightness is still detected continuously, and the picture output mode is switched according to the currently detected brightness.
  • the ISP digital zoom is used to zoom the image. Since the preset switching magnification is 8 times, when the zoom magnification is between 8 times and 10 times, different image output modes are switched according to the brightness, and the pixel cropping mode is also different. Specifically, under low-light conditions, the image is zoomed by the ISP digital zoom. Under bright light conditions, the image sensor first crops the input image so that the output image under bright light conditions has the same magnification as the output image under dark light conditions. Then output the image cropped by the image sensor to the ISP digital zoom to zoom the image.
  • the output image is cropped by the image sensor and then output to the ISP for digital zooming, which can save the power consumption of the ISP.
  • the image zoom is greater than 10 times, taking dynamic cropping as an example, the image is cropped by electronic image stabilization.
  • FIG. 23 it is a schematic diagram of switching between the main camera and the telephoto camera when the zoom ratios are different. Specifically, when the zoom ratio is less than 4 times, the image is collected and obtained by the main camera. For example, as shown in (a) in Figure 23, when the zoom ratio is 1 times, the image is captured and obtained by the main camera, as in Figure 23 (b), when the zoom ratio is 3.9 times, the image is still captured by the main camera The camera captures and obtains images. As shown in (c) in Figure 23, when the zoom ratio is 4 times, the telephoto camera collects and obtains an image.
  • Fig. 24 is another embodiment of the present application.
  • the difference from the previous embodiments is that electronic image stabilization S104 is not included. Therefore, image output and cropping are completed by S103 digital zooming.
  • the current zoom magnification is 8 times (instead of 15 times in other embodiments of the present application), and the basic magnification of the telephoto lens is 4 times, for example.
  • the realization process of digital zoom S103 is elaborated in detail. It can be concluded from the above step S102 that under bright and dark light conditions when the image is zoomed to 8 times, the size of the image input to the digital zoom S103 is 4608*3456.
  • the ISP crops 4/8 according to the first ratio to obtain a 2304*1728 image.
  • the ISP continues to downsample the above image by a factor of 0.625 to 1440*1080.
  • Another example is under bright light conditions, so the ISP crops 8/8 according to the second ratio to obtain a 4608*3456 image.
  • the ISP continues to downsample the above image by a factor of 0.3125 to 1440*1080.
  • the present application also provides a computer storage medium, wherein the computer storage medium can store a program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the parts in the above-mentioned embodiments or all steps.
  • the storage medium may be a magnetic disk, an optical disk, a read-only memory (English: read-only memory, abbreviated: ROM) or a random access memory (English: random access memory, abbreviated: RAM), etc.
  • an embodiment of the present application also provides a computer program product, the computer program product includes executable instructions, and when the executable instructions are executed on a computer, the computer executes part or part of the above method embodiments. All steps.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Computing Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Studio Devices (AREA)

Abstract

一种图像拍摄的方法、设备、存储介质和程序产品，该方法包括显示相机应用的第一预览界面，第一预览界面包括第一预览图像，第一预览图像为摄像头采集的第一图像经处理后得到的，第一预览图像对应第一变焦倍率，第一图像对应摄像头的第一输出模式；接收用户的第一操作；响应于第一操作，显示相机应用的第二预览界面，第二预览界面包括第二预览图像，第二预览图像为摄像头采集的第二图像经处理后得到的，第二预览图像对应第二变焦倍率，第二图像对应摄像头的第二输出模式，其中，第二输出模式与第一输出模式不同，第二变焦倍率与第一变焦倍率不同。

Description

图像拍摄的方法、设备、存储介质和程序产品 技术领域
本申请涉及计算机技术领域，具体地涉及一种图像拍摄的方法、设备、存储介质和程序产品。
背景技术
手机成像技术日益发达，在日常办公或生活中，人们日渐习惯利用手机随时捕捉生活或工作中的点滴。通常情况下，都是在无外部设备支撑的情况下，直接用手拿起手机进行拍摄。为了得到更清晰理想的成像效果，人们往往会尽可能稳住双手以期能够尽可能稳定拍摄前的拍照预览界面。然而，人手的图像稳定通常都有局限性，无法做到完全静止。因而，手持手机的拍照过程中，不可避免的会伴随着图像抖动。
一旦在拍照过程中，身体或手产生了抖动，都会造成图片成像变得模糊。尤其是在拍照预览界面将图像放大一定倍率时，呈现在手机画面的抖动也会被放大，很难得到稳定的拍照预览画面，从而影响用户的使用体验。
发明内容
为了解决上述技术问题，本申请提出了一种图像拍摄的方法、设备、存储介质和程序产品。在该方法中，在手持手机拍摄的场景下，根据用户在相机应用拍照预览界面的变焦倍率的不同，对摄像头采集的图像进行处理，进而在该拍照预览界面呈现经过防抖处理后的图像，以减少用户手持手机拍摄时产生的图像抖动，提高了用户体验。
第一方面，提供一种图像拍摄方法，显示相机应用的第一预览界面，所述第一预览界面包括第一预览图像，所述第一预览图像为摄像头采集的第一图像经处理后得到的，所述第一预览图像对应第一变焦倍率，所述第一图像对应所述摄像头的第一输出模式；接收用户的第一操作；响应于所述第一操作，显示所述相机应用的第二预览界面，所述第二预览界面包括第二预览图像，所述第二预览图像为摄像头采集的第二图像经处理后得到的，所述第二预览图像对应第二变焦倍率，所述第二图像对应所述摄像头的第二输出模式，其中，所述第二输出模式与所述第一输出模式不同，所述第二变焦倍率与所述第一变焦倍率不同。这样，在手持手机拍照的场景下，在当前相机应用处于第一变焦倍率的第一预览界面时，对应第一输出模式，根据用户的变焦操作，焦距变化至第二变焦倍率，相机应用的拍照预览界面呈现第二预览界面，对应第二输出模式，从而实现了在手持手机拍照时，根据变焦倍率的不同，在拍照预览界面呈现不同的输出模式，通过不同的输出模式，可以优化在不同光照条件下相机应用拍照预览界面图像的分辨率和清晰度。
示例性的，相机应用为手机或平板上的系统相机应用。
示例性的，第一预览界面和第二预览为相机应用的拍照预览界面。
根据第一方面，所述第一输出模式为Binning，Remosaic，Quadra，Quadra HDR或Stagger  HDR之一。这样，通过不同的输出模式，可以优化在不同光照条件下相机应用拍照预览界面图像的分辨率和清晰度。
根据第一方面，所述第二输出模式为Binning，Remosaic，Quadra，Quadra HDR或Stagger HDR之一。这样，通过不同的输出模式，可以优化在不同光照条件下相机应用拍照预览界面图像的分辨率和清晰度。
根据第一方面，或者以上第一方面的任意一种实现方式，所述图像拍摄方法还包括，所述第一输出模式或所述第二输出模式基于所述摄像头的拍摄亮度确定。这样，根据不同的拍摄亮度，调整输出模式，提高相机应用拍照预览界面图像的分辨率和清晰度，提高用户体验。尤其适用于夜景场景，例如当需要拍摄室外夜景时，而室内拍摄亮度为亮光，此时，呈现在相机应用的拍照预览界面的图像应为室外的夜景，而由于室内光照，可能会将当前亮度根据室内的亮光判定，从而影响夜景拍摄的效果。本发明的实施例中，在当前场景下，拍照预览界面的拍摄亮度根据室外的亮度确定，即根据室外的夜景而将此时的拍照亮度确定为暗光，从而改变输出模式，提高相机应用拍照预览界面图像的分辨率和清晰度。
根据第一方面，或者以上第一方面的任意一种实现方式，所述拍摄亮度基于环境光传感器采集的当前环境光亮度确定。这样，可以精确的反应当前的拍摄亮度，从而根据当前拍摄亮度呈现不同的输出模式，优化在不同光照条件下相机应用拍照预览界面图像的分辨率和清晰度。
根据第一方面，或者以上第一方面的任意一种实现方式，所述拍摄亮度基于所述第二预览图像的平均亮度信息确定。这样，可以精确的反应当前的拍摄亮度，从而根据当前拍摄亮度呈现不同的输出模式，优化在不同光照条件下相机应用拍照预览界面图像的分辨率和清晰度。
根据第一方面，或者以上第一方面的任意一种实现方式，所述拍摄亮度包括亮光和暗光。这样，根据当前拍摄亮度呈现不同的输出模式，优化在不同光照条件下相机应用拍照预览界面图像的分辨率和清晰度。
根据第一方面，或者以上第一方面的任意一种实现方式，当所述拍摄亮度为暗光时，所述第一输出模式为Binning。这样，优化在暗光条件下相机应用拍照预览界面图像的分辨率和清晰度。
根据第一方面，或者以上第一方面的任意一种实现方式，当所述拍摄亮度为暗光时，所述第二输出模式为Binning。这样，优化在暗光条件下相机应用拍照预览界面图像的分辨率和清晰度。
根据第一方面，或者以上第一方面的任意一种实现方式，当所述拍摄亮度为亮光时，所述第一输出模式为Remosaic。这样，优化在亮光条件下相机应用拍照预览界面图像的分辨率和清晰度。
根据第一方面，或者以上第一方面的任意一种实现方式，当所述拍摄亮度为亮光时，所述第二输出模式为Remosaic。这样，优化在亮光条件下相机应用拍照预览界面图像的分辨率和清晰度。
根据第一方面，或者以上第一方面的任意一种实现方式，所述第一操作包括用于缩放所述第一预览图像的手势操作、屏幕点击操作或拖拽变焦进度条之一。这样，根据用户的习惯或者手机款式的不同，用户可以通过多种方式对预览图像进行变焦操作，提高了用户体验。
根据第一方面，或者以上第一方面的任意一种实现方式，所述第一预览界面及所述第二预 览界面的所述拍摄亮度为亮光。
根据第一方面，或者以上第一方面的任意一种实现方式，所述方法还包括：响应于所述拍摄亮度切换为暗光时，所述相机应用显示第三预览界面，所述第三预览界面对应所述第一输出模式。这样，根据用户所处拍摄的不同场景，例如用户仍然手持手机拍照时，从亮光环境移动到了暗光环境，此时，根据亮度的不同调整不同的输出模式，优化在不同光照条件下相机应用拍照预览界面图像的分辨率和清晰度。
根据第一方面，或者以上第一方面的任意一种实现方式，所述方法还包括：所述第二变焦倍率大于预设切换倍率。这样，当满足预设的变焦倍率时，切换不同的输出模式，提高相机应用拍照预览界面图像的分辨率和清晰度。
根据第一方面，或者以上第一方面的任意一种实现方式，所述方法还包括：接收用户的第二操作；响应于所述第二操作，显示所述相机应用的第四预览界面，所述第四预览界面包括第四预览图像，所述第四预览图像为摄像头采集的第四图像经处理后得到的，所述第四预览图像对应第三变焦倍率，所述第四图像对应所述摄像头的第一输出模式，其中，所述第三变焦倍率小于所述预设切换倍率。这样，在用户将变焦倍率调整至小于预设的变焦倍率时，切换不同的输出模式，提高相机应用拍照预览界面图像的分辨率和清晰度。
根据第一方面，或者以上第一方面的任意一种实现方式，所述第二操作包括用于缩放所述第一预览图像的手势操作、屏幕点击操作或拖拽变焦进度条之一。这样，根据用户的习惯或者手机款式的不同，用户可以通过多种方式对预览图像进行变焦操作，提高了用户体验。
根据第一方面，或者以上第一方面的任意一种实现方式，所述方法还包括：所述第二变焦倍率大于预设防抖倍率，将所述第二图像基于所述第二变焦倍率经由数字变焦及电子防抖进行像素裁剪后得到所述第二预览界面。这样，当第二变焦倍率满足预设条件时，通过ISP及EIS对摄像头采集的图像进行像素裁剪后，将用户手持手机拍摄时产生的抖动程度降低，提高用户体验。
根据第一方面，或者以上第一方面的任意一种实现方式，所述数字变焦包括基于所述第二变焦倍率及所述第二输出模式进行像素裁剪。这样，结合变焦倍率和输出模式进行像素裁剪，可以提高像素裁剪的范围和准确性，将用户手持手机拍摄时产生的抖动程度降低，提高用户体验。
根据第一方面，或者以上第一方面的任意一种实现方式，所述电子防抖包括3D陀螺仪、2D图像特征点及图像透视变换，其中所述3D陀螺仪适用于3D图像平滑。这样，通过结合2D和3D的五轴防抖，提高了手持手机拍摄的性能，减少了用户手持手机拍摄时产生的抖动程度，提高用户体验。
根据第一方面，或者以上第一方面的任意一种实现方式，所述3D陀螺仪及所述2D图像特征点基于所述第二变焦倍率进行像素裁剪。这样，在用户将拍照预览界面变焦至第二变焦倍率时，通过结合2D和3D的五轴防抖，提高了手持手机拍摄的性能，减少了用户手持手机拍摄时产生的抖动程度，提高用户体验。
根据第一方面，或者以上第一方面的任意一种实现方式，所述图像透视变换包括坐标变换。这样，可以提高像素裁剪的范围和准确性，将用户手持手机拍摄时产生的抖动程度降低，提高用户体验。
根据第一方面，或者以上第一方面的任意一种实现方式，所述第二变焦倍率大于预设防抖倍率，将所述第二图像基于所述第二变焦倍率经由数字变焦及电子防抖进行像素裁剪后得到所述第二预览界面。这样，通过结合2D和3D的五轴防抖，提高了手持手机拍摄的性能，减少了用户手持手机拍摄时产生的抖动程度，提高用户体验。
第二方面，本申请实施例提供一种电子设备，电子设备包括用于存储计算机程序指令的存储器和用于执行程序指令的处理器，其中，当该计算机程序指令被该处理器执行时，触发电子设备执行第一方面任一项的方法。
第二方面以及第二方面的任意一种实现方式分别与第一方面以及第一方面的任意一种实现方式相对应。第二方面以及第二方面的任意一种实现方式所对应的技术效果可参见上述第一方面以及第一方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第三方面，本申请实施例提供一种计算机可读存储介质，计算机可读存储介质中存储有计算机程序，当其在计算机上运行时，使得计算机执行第一方面任一项的方法。
第三方面以及第三方面的任意一种实现方式分别与第一方面以及第一方面的任意一种实现方式相对应。第三方面以及第三方面的任意一种实现方式所对应的技术效果可参见上述第一方面以及第一方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第四方面，本申请实施例提供一种计算机程序产品，计算机程序产品包括计算机程序，当其在计算机上运行时，使得计算机执行第一方面任一项的方法。
第四方面以及第四方面的任意一种实现方式分别与第一方面以及第一方面的任意一种实现方式相对应。第四方面以及第四方面的任意一种实现方式所对应的技术效果可参见上述第一方面以及第一方面的任意一种实现方式所对应的技术效果，此处不再赘述。
在一种可能的设计中，第五方面中的程序可以全部或者部分存储在与处理器封装在一起的存储介质上，也可以部分或者全部存储在不与处理器封装在一起的存储器上。
附图说明
为了更清楚地说明本申请实施例的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其它的附图。
图1为本申请实施例提供的一种电子设备示意图；
图2为本申请实施例提供的一种电子设备的软件结构框图；
图3为本申请实施例提供的一种相机应用拍照预览场景示意图；
图4为本申请实施例提供的一种暗光条件下的拍照成像示意图；
图5为本申请实施例提供的一种以Binning方式输出图像的示意图；
图6为本申请实施例提供的一种亮光条件下的拍照成像示意图；
图7为本申请实施例提供的一种以Remosaic方式输出图像的示意图；
图8为本申请实施例提供的一种图像暗亮光切换的示意图；
图9为本申请实施例提供的一种图像亮暗光切换的示意图；
图10为本申请实施例提供的一种图像输出模式切换的示意图；
图11为本申请实施例提供的一种图像防抖的实现示例；
图12为本申请实施例提供的另一种图像防抖的实现示例；
图13为本申请实施例提供的一种图12的视场角示意图；
图14A为本申请实施例提供的一种相机图像位移成像示意图；
图14B为本申请实施例提供的另一种相机图像位移成像示意图；
图15为本申请实施例提供的一种输入输出图示意图；
图16为本申请实施例提供的一种图像处理的逻辑处理示意图；
图17为本申请实施例提供的一种不同倍率下输入图与输出图之间的关系示意图；
图18为本申请实施例提供的一种数字变焦的实现示意图；
图19为本申请实施例提供的一种电子防抖的实现示意图；
图20为本申请实施例提供的一种电子防抖算法框架的示意图；
图21为本申请实施例提供的另一种实施方式；
图22为本申请实施例提供的一种逻辑实现时序示意图；
图23为本申请实施例提供的一种不同摄像头切换的示意图；
图24为本申请实施例提供的另一种实施方式。
具体实施方式
为了更好的理解本申请的技术方案，下面结合附图对本申请实施例进行详细描述。
应当明确，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其它实施例，都属于本申请保护的范围。
在本申请说明书和所附权利要求书的描述中，术语“第一”、“第二”、“第三”、“第四”等仅用于区分描述，而不能理解为指示或暗示相对重要性。
参见图1，为本申请实施例提供的一种电子设备的结构示意图。电子设备100可以包括处理器110，外部存储器接口120，内部存储器121，通用串行总线(universal serial bus，USB)接口130，充电管理模块140，电源管理模块141，电池142，天线1，天线2，移动通信模块150，无线通信模块160，音频模块170，扬声器170A，受话器170B，麦克风170C，耳机接口170D，传感器模块180，按键190，马达191，指示器192，摄像头193，显示屏194，以及用户标识模块(subscriber identification module，SIM)卡接口195等。其中传感器模块180可以包括压力传感器180A，陀螺仪传感器180B，磁传感器180D，加速度传感器180E，距离传感器180F，接近光传感器180G，指纹传感器180H，触摸传感器180K，环境光传感器180L等。
可以理解的是，本申请实施例示意的结构并不构成对电子设备100的具体限定。在本申请另一些实施例中，电子设备100可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或软件和硬件的组合实现。
处理器110可以包括一个或多个处理单元，例如：处理器110可以包括应用处理器(application processor，AP)，调制解调处理器，图形处理器(graphics processing unit，GPU)，图像信号处理器(image signal processor，ISP)，控制器，存储器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器，和/或神经网络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。 例如，处理器110用于执行本申请实施例中的环境光的检测方法。
其中，控制器可以是电子设备100的神经中枢和指挥中心。控制器可以根据指令操作码和时序信号，产生操作控制信号，完成取指令和执行指令的控制。
处理器110中还可以设置存储器，用于存储指令和数据。在一些实施例中，处理器110中的存储器为高速缓冲存储器。该存储器可以保存处理器110刚用过或循环使用的指令或数据。如果处理器110需要再次使用该指令或数据，可从存储器中直接调用。避免了重复存取，减少了处理器110的等待时间，因而提高了系统的效率。
在一些实施例中，处理器110可以包括一个或多个接口。接口可以包括集成电路(inter-integrated circuit，I2C)接口，集成电路内置音频(inter-integrated circuit sound，I2S)接口，脉冲编码调制(pulse code modulation，PCM)接口，通用异步收发传输器(universal asynchronous receiver/transmitter，UART)接口，移动产业处理器接口(mobile industry processor interface，MIPI)，通用输入输出(general-purpose input/output，GPIO)接口，用户标识模块(subscriber identity module，SIM)接口，和/或通用串行总线(universal serial bus，USB)接口等。
外部存储器接口120可以用于连接外部存储卡，例如Micro SD卡，实现扩展电子设备100的存储能力。外部存储卡通过外部存储器接口120与处理器110通信，实现数据存储功能。例如将音乐，视频等文件保存在外部存储卡中。
内部存储器121可以用于存储计算机可执行程序代码，可执行程序代码包括指令。处理器110通过运行存储在内部存储器121的指令，从而执行电子设备100的各种功能应用以及数据处理。内部存储器121可以包括存储程序区和存储数据区。其中，存储程序区可存储操作系统，至少一个功能所需的应用程序。存储数据区可存储电子设备100使用过程中所创建的数据。
此外，内部存储器121可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件，闪存器件，通用闪存存储器(universal flash storage，UFS)等。
压力传感器180A用于感受压力信号，可以将压力信号转换成电信号。在一些实施例中，压力传感器180A可以设置于显示屏194。压力传感器180A的种类很多，如电阻式压力传感器，电感式压力传感器，电容式压力传感器等。电容式压力传感器可以是包括至少两个具有导电材料的平行板。当有力作用于压力传感器180A，电极之间的电容改变。电子设备100根据电容的变化确定压力的强度。当有触摸操作作用于显示屏194，电子设备100根据压力传感器180A检测触摸操作强度。电子设备100也可以根据压力传感器180A的检测信号计算触摸的位置。
在一些实施例中，作用于相同触摸位置，但不同触摸操作强度的触摸操作，可以对应不同的操作指令。例如：当有触摸操作强度小于第一压力阈值的触摸操作作用于短消息应用图标时，执行查看短消息的指令。当有触摸操作强度大于或等于第一压力阈值的触摸操作作用于短消息应用图标时，执行新建短消息的指令。
陀螺仪传感器180B可以用于确定电子设备100的运动姿态。在一些实施例中，可以通过陀螺仪传感器180B确定电子设备100围绕三个轴(即，x，y和z轴)的角速度。陀螺仪传感器180B可以用于拍摄防抖。示例性的，当按下快门，陀螺仪传感器180B检测电子设备100抖动的角度，根据角度计算出镜头模组需要补偿的距离，让镜头通过反向运动抵消电子设备100的抖动，实现防抖。陀螺仪传感器180B还可以用于导航，体感游戏场景。
磁传感器180D包括霍尔传感器。电子设备100可以利用磁传感器180D检测翻盖皮套的开合。在一些实施例中，当电子设备100是翻盖机时，电子设备100可以根据磁传感器180D检测翻盖的开合。进而根据检测到的皮套的开合状态或翻盖的开合状态，设置翻盖自动解锁等特性。
加速度传感器180E可检测电子设备100在各个方向上(一般为三轴)加速度的大小。当电子设备100静止时可检测出重力的大小及方向。还可以用于识别电子设备姿态，应用于横竖屏切换，计步器等应用。
距离传感器180F，用于测量距离。电子设备100可以通过红外或激光测量距离。在一些实施例中，拍摄场景，电子设备100可以利用距离传感器180F测距以实现快速对焦。
接近光传感器180G可以包括例如发光二极管(LED)和光检测器，例如光电二极管。发光二极管可以是红外发光二极管。电子设备100通过发光二极管向外发射红外光。电子设备100使用光电二极管检测来自附近物体的红外反射光。当检测到充分的反射光时，可以确定电子设备100附近有物体。当检测到不充分的反射光时，电子设备100可以确定电子设备100附近没有物体。电子设备100可以利用接近光传感器180G检测用户手持电子设备100贴近耳朵通话，以便自动熄灭屏幕达到省电的目的。接近光传感器180G也可用于皮套模式，口袋模式自动解锁与锁屏。
指纹传感器180H用于采集指纹。电子设备100可以利用采集的指纹特性实现指纹解锁，访问应用锁，指纹拍照，指纹接听来电等。
触摸传感器180K，也称“触控面板”。触摸传感器180K可以设置于显示屏194，由触摸传感器180K与显示屏194组成触摸屏，也称“触控屏”。触摸传感器180K用于检测作用于其上或附近的触摸操作。触摸传感器可以将检测到的触摸操作传递给应用处理器，以确定触摸事件类型。可以通过显示屏194提供与触摸操作相关的视觉输出。在另一些实施例中，触摸传感器180K也可以设置于电子设备100的表面，与显示屏194所处的位置不同。
环境光传感器180L用于感知环境光亮度。电子设备100可以根据感知的环境光亮度自适应调节显示屏194亮度。环境光传感器180L也可用于拍照时自动调节白平衡。环境光传感器180L还可以与接近光传感器180G配合，检测电子设备100是否在口袋里，以防误触。
按键190包括开机键，音量键等。按键190可以是机械按键。也可以是触摸式按键。电子设备100可以接收按键输入，产生与电子设备100的用户设置以及功能控制有关的键信号输入。
马达191可以产生振动提示。马达191可以用于来电振动提示，也可以用于触摸振动反馈。例如，作用于不同应用(例如拍照，音频播放等)的触摸操作，可以对应不同的振动反馈效果。作用于显示屏194不同区域的触摸操作，马达191也可对应不同的振动反馈效果。不同的应用场景(例如：时间提醒，接收信息，闹钟，游戏等)也可以对应不同的振动反馈效果。触摸振动反馈效果还可以支持自定义。
指示器192可以是指示灯，可以用于指示充电状态，电量变化，也可以用于指示消息，未接来电，通知等。
电子设备100通过GPU，显示屏194，以及应用处理器等实现显示功能。GPU为图像处理的微处理器，连接显示屏194和应用处理器。GPU用于执行数学和几何计算，用于图形渲染。处理器110可包括一个或多个GPU，其执行程序指令以生成或改变显示信息。
显示屏194用于显示图像，视频等。显示屏194包括显示面板。显示面板可以采用有机发光二极管(organic light-emitting diode，OLED)。在一些实施例中，电子设备100可以包括1个或N个显示屏194，N为大于1的正整数。
电子设备100可以通过ISP，摄像头193，视频编解码器，GPU，显示屏194以及应用处理器等实现拍摄功能。
摄像头193用于捕获静态图像或视频。物体通过镜头生成光学图像投射到感光元件。感光元件可以是电荷耦合器件(charge coupled device，CCD)或互补金属氧化物半导体(complementary metal-oxide-semiconductor，CMOS)光电晶体管。感光元件把光信号转换成电信号，之后将电信号传递给ISP转换成数字图像信号。ISP将数字图像信号输出到DSP加工处理。DSP将数字图像信号转换成标准的RGB，YUV等格式的图像信号。在一些实施例中，电子设备100可以包括1个或N个摄像头193，N为大于1的正整数。
视频编解码器用于对数字视频压缩或解压缩。电子设备100可以支持一种或多种视频编解码器。这样，电子设备100可以播放或录制多种编码格式的视频，例如：动态图像专家组(moving picture experts group，MPEG)1，MPEG2，MPEG3，MPEG4等。
参见图2，为本申请实施例提供的一种电子设备的软件结构框图。分层架构将软件分成若干个层，每一层都有清晰的角色和分工。层与层之间通过软件接口通信。在一些实施例中，将安卓(Android)系统分为四层，从上至下分别为应用层、框架层、硬件抽象层和硬件层。
应用层(Application，App)可以包括一系列应用程序包。例如，该应用程序包可以包括相机应用。应用层又可以分为应用界面(UI)和应用逻辑。
相机应用的应用界面包括单景模式、双景模式、画中画模式等，对应于不同的图像或视频拍摄模式。
相机应用的应用逻辑包括多摄框架和相机管理。其中，多摄框架包括切换控制模块、Surface切换管理模块、多摄编码模块、转场控制模块等。切换控制模块用于控制拍摄模式的切换，例如前摄模式、后摄模式、前后模式、后后模式、画中画模式之间的切换。其中，在拍摄模式切换过程中可能会涉及打开或关闭的特定的摄像头、屏蔽不同芯片平台的硬件差异等。Surface切换管理模块用于在拍摄模式切换过程中控制Surface切换。在Android系统中，Surface对应一块屏幕缓冲区，用于保存当前窗口的像素数据。具体地，在图像或视频拍摄过程中，存在预览Surface和编码Surface，Surface切换管理模块用于在拍摄模式切换过程中控制预览Surface和编码Surface的切换。多摄编码模块用于在拍摄模式过程中进行编码，生成图像或视频文件，即实现拍摄图像或视频的录制。转场控制模块用于在拍摄模式切换过程中生成转场动效。相机管理包括设备管理模块、Surface管理模块、会话管理模块等。
框架层(Framework，FWK)为应用层的应用程序提供应用编程接口(application programming interface，API)和编程框架，包括一些预先定义的函数。在图2中，框架层包括相机框架和媒体框架。其中，相机框架可以为相机访问接口(Camera2API)，Camera2API是Android推出的一套访问摄像头设备的接口，其采用管道式的设计，使数据流从摄像头流向Surface。Camera2API包括相机管理(CameraManager)和相机设备(CameraDevice)。CameraManager为Camera设备的管理类，通过该类对象可以查询设备的Camera设备信息，得到CameraDevice对象。CameraDevice提供了Camera设备相关的一系列固定参数，例如基 础的设置和输出格式等。
媒体框架包括媒体编解码器(MediaCodec)、音频录制模块(AudioRecord)和视音频复用器(Muxer)。其中，MediaCodec是Android提供的用于对音视频进行编解码的类，它通过访问底层的codec来实现编解码的功能，是Android media基础框架的一部分。AudioRecord的主要功能是让各种应用能够管理音频资源，以便能够录制硬件所收集的声音。Muxer用于将视频压缩数据(例如H.264)和音频压缩数据(例如，AAC)合并到一个封装格式数据中。
硬件抽象层(HAL)是位于操作系统内核与硬件电路之间的接口层，其目的在于将硬件抽象化。它隐藏了特定平台的硬件接口细节,为操作系统提供虚拟硬件平台,使其具有硬件无关性,可在多种平台上进行移植。在图2中，HAL包括相机硬件抽象层(Camera HAL)，Camera HAL包括图像信号处理器(Image Signal Processor，ISP)等。HAL还包括集成框架，该集成框架包括电子防抖(Electric Image Stabilization，EIS)及图像透视变换等。可理解，该图像信号处理器、电子防抖、图像透视变换为抽象的设备。在图像或视频拍摄场景中，HAL会根据上层下发的分辨率以及Surface大小创建相应大小的数据流。
内核层(Kernel)是硬件和软件之间的层。内核层至少包含显示驱动，摄像头驱动，传感器驱动等。
参见图3，本申请实施例提供一种图像拍摄的方法，该方法可以应用于用户在手持手机并打开手机相机应用拍照时，在图像缩放过程中，由于手部抖动而造成手机相机应用的拍照预览界面成像不清晰的场景。作为一种示例，用户单手或双手持有手机并打开手机相机应用，在需要图像变焦的拍摄场景，打开相机的图像缩放功能(Zoom)以放大图像。由于手部或身体不可避免的抖动，呈现在手机拍照于预览界面的画面会由于晃动而变得模糊。
可以理解的，电子设备的形态仅以手机做示例性说明，电子设备也可以是平板电脑，手持计算机等带摄像功能的手持电子设备，在此不作限制。
日常生活中，拍照是使用手机最频繁的场景之一。随处可见人们拿起手机随手用手机相机捕捉生活中的点滴。图像防抖是影响成像质量的一个重要因素。图像抖动必然导致成像的图像模糊且清晰度不高。
因而，很多具有拍摄功能的智能手机都会在拍照时加入防抖技术，可以降低和避免因拍照时手部抖动等因素造成的画面模糊，从而提高成像的清晰度。
例如，光学防抖(Optical Image Stabilization，OIS)是相机镜头特殊马达结构实现的物理防抖技术。通过陀螺仪计算拍摄过程中手机的抖动情况，将信号传至微处理器，计算需要补偿的位移量之后，控制镜头移动进行光学补偿，实现防抖。然而，OIS局限于小幅度范围内的抖动，很难弥补大角度抖动。
本申请的一种实施方式中，在图像缩放后，为了提高拍照时呈现在手机拍照预览界面图像的清晰度，可以根据拍摄亮度以确定图像的输出模式。拍摄亮度指的是手机的摄像头在拍摄图像时所处环境的亮度。当拍摄所处的环境亮度越亮，则手机拍摄到的图像所获取的光线越多。反之，当拍摄所处的环境亮度越低，则手机拍摄到的图像所获取的光线越少。
可选的，根据手机硬件模组中的环境光传感器检测当前环境光亮度以确定当前呈现在相机拍照预览界面的图像的亮度。
可选的，根据当前相机拍照预览界面的图像平均亮度信息确定当前呈现在相机拍照预览界 面的图像亮度，包括图像的颜色、图像中各个像素点的像素值等。
需要说明的是，相机拍照预览界面的亮度检测与判断包括但不限于以上列举的情形。实际使用时，可以根据实际需要采用其他方式获取当前相机拍照预览界面的图像亮度。
在确定当前拍摄亮度后，手机选择相应的拍摄模式输出图片以优化在不同拍摄亮度下的清晰度。结合图4，为检测到手机在暗光条件下拍照的成像示例。如图4中的(a)所示，表示的是用户拿起手机打开相机应用时的初始界面。其中，初始界面指的是用户打开相机应用时的初始界面，示例性的，第一预览界面为初始界面，第一变焦倍率为初始变焦倍率，其初始变焦倍率为1倍。此时拍摄亮度为暗光。如图4中的(b)所示，在需要图像变焦的场景，用户执行第一操作，示例性的，第一操作包括用于缩放所述第一预览图像的手势操作、屏幕点击操作或拖拽变焦进度条之一。示例性的，用户触控屏幕以手势控制或点击拍照预览界面的图像缩放(zoom)开关等形式放大图片至需要的倍率，即第二变焦倍率，此时相机应用显示第二预览界面。示例性地，手机默认可以支持的最大倍率为20倍，用户此时将拍照预览界面的图像放大15倍，即图4中的(b)的图像倍率是图4中的(a)的15倍。
示例性地，在图4中的(a)所示的初始界面下，相机应用处于拍照预览界面，摄像头采集第一图像，手机在相机应用的初始界面设置为Binning模式输出图片，即图4中的(a1)的图片输出模式为Binning，即第一输出模式。又由于此时是暗光条件，在手机拍照预览界面的成像图片被放大15倍至图4中的(b)的状态后，摄像头采集第二图像，手机仍然以Binning模式输出图片，以保证呈现在手机屏幕上的图像清晰度，即图4中的(b1)的图片输出模式仍然为Binning，即第二输出模式。
示例性地，第一输出模式为Binning，Remosaic，Quadra，Quadra HDR或Stagger HDR之一。
示例性地，第二输出模式为Binning，Remosaic，Quadra，Quadra HDR或Stagger HDR之一。
Binning模式：电子设备在拍摄图像的过程中，目标对象反射的光线被摄像头采集，以使得该反射的光线传输至图像传感器。图像传感器上包括多个感光元件，每个感光元件采集到的电荷为一个像素，并对像素信息执行模拟并合(Binning)操作。具体地说，Binning可以将n×n个像素合并为一个像素。例如，Binning可以将相邻的2×2个像素合成为一个像素，也就是说，相邻2×2个像素的颜色以一个像素的形式呈现。
示例性地，接收用户的第二操作，显示相机应用的第四预览界面，第二操作包括用于缩放所述第一预览图像的手势操作、屏幕点击操作或拖拽变焦进度条之一。第四预览界面包括第四预览图像，第四预览图像为摄像头采集的第四图像经处理后得到的，第四预览图像对应第三变焦倍率。例如，用户将处于图4中的(b)中的第二预览界面通过触控屏幕以手势控制或点击拍照预览界面的图像缩放(zoom)开关等形式缩小图片至需要的倍率，即第三变焦倍率。当第三变焦倍率小于预设条件时，即第三变焦倍率小于预设切换倍率时，例如将第三变焦倍率缩小为1倍时，此时图像显示第四预览界面，摄像头采集第三图像，输出模式为Binning，即第一输出模式。
示例性地，结合图5所示，为手机获取图像后以Binning方式输出图像过程示意图。其中，图5中的(b0)为一个4×4的像素示意图，将相邻的2×2个像素合成为一个像素。图5中的 (b1)为Binning方式输出的像素示意图。如，采用Binning方式，将图5中的(b0)的01区域中的2×2个像素，形成图5中的(b1)中的像素R；将图5中的(b0)的02区域中的2×2个像素，形成图5中的(b1)中的像素G；将图5中的(b0)的03区域中的2×2个像素，形成图5中的(b1)中的像素G；将图5中的(b0)的04区域中的2×2个像素，形成图5中的(b1)中的像素B。
其中，以输出图像格式是拜耳阵列(Bayer)格式图像为例，Bayer格式的图像是指图像中仅包括红色、蓝色和绿色(即三原色)的图像。如，01区域中的2×2个像素形成的像素A为红色，02区域中的2×2个像素形成的像素B为绿色，03区域中的2×2个像素形成的像素C为绿色，04区域中的2×2个像素形成的像素D为蓝色。
结合图6，为检测到手机在亮光条件下拍照的成像示例。如图6中的(a)所示，表示的是用户拿起手机打开相机应用时的初始界面，即第一预览界面。此时拍摄亮度为亮光。其中，初始变焦倍率为1倍，即第一变焦倍率。如图6中的(b)所示，在需要图像变焦的场景，用户执行第一操作，示例性的，第一操作包括用于缩放所述第一预览图像的手势操作、屏幕点击操作或拖拽变焦进度条之一。用户触控屏幕以手势控制或点击拍照预览界面的图像缩放(zoom)开关等形式放大图片至需要的倍率，即第二变焦倍率，此时相机应用显示第二预览界面。示例性地，手机默认可以支持的最大倍率为20倍，用户此时将拍照预览界面的图像放大15倍，即图6中的(b)的图像倍率是图6中的(a)的15倍。
示例性地，在图6中的(a)的初始界面下，手机在相机应用的初始界面设置为Binning模式输出图片，即图6中的(a1)的图片输出模式为Binning，即第一输出模式。又由于此时是亮光条件，摄像头采集第二图像，在手机拍照预览界面的成像图片被放大15倍至图6中的(b)的状态后，即第二预览界面，手机切换为Remosaic模式输出图片，即第二输出模式，以保证呈现在手机屏幕上的图像清晰度，即图6中的(b1)的图片输出模式为Remosaic，进一步裁切例如边长的二分之一大小以后，即图6中的(b2)状态，输出图像。
示例性地，接收用户的第二操作，显示相机应用的第四预览界面，第二操作包括用于缩放所述第一预览图像的手势操作、屏幕点击操作或拖拽变焦进度条之一。第四预览界面包括第四预览图像，第四预览图像为摄像头采集的第四图像经处理后得到的，第四预览图像对应第三变焦倍率。例如，用户将处于图6中的(b)中的第二预览界面通过触控屏幕以手势控制或点击拍照预览界面的图像缩放(zoom)开关等形式缩小图片至需要的倍率，即第三变焦倍率。当第三变焦倍率小于预设条件时，即第三变焦倍率小于预设切换倍率时，例如将第三变焦倍率缩小为1倍时，此时图像显示第四预览界面，摄像头采集第三图像，输出模式为Binning，即第一输出模式。
Remosaic模式：采用Remosaic方式输出图像时，将Bayer格式图像重新排列得到Remosaic模式图像。如，假设在图像中一个像素是由n×n个像素组成的，那么，采用Remosaic可以将该图像中的一个像素重新排列n×n个像素。示例性的，如图7中的(b0)为一种像素示意图，每个像素是由相邻的2×2个像素合成的。如图7中的(b1)为采用Remosaic方式输出的Bayer格式的图像示意。具体地说，图7中的(b0)中像素为红色、绿色和蓝色。将图7中的(b0)中的每个像素分为2×2个像素，并分别重新排列。即采用Remosaic方式输出，输出的图像为图7中的(b1)所示的Bayer格式的图像。
示例性地，结合图8所示，为图像被缩放至一定倍率后拍摄亮度由暗变亮时的图像输出示例。具体地，当图像被放大至例如15倍时，即由第一变焦倍率放大至第二变焦倍率，即图8中的(a)的状态，由于此时亮度判断为处于暗光状态，因而图片输出模式为Binning模式，即图8中的(a1)的模式，即第二输出模式。由于用户手持手机摄像时的移动，即从暗光环境移动到亮光环境，从而导致手机拍摄亮度的变化，此时，当手机检测到亮度为亮光时，相机应用显示第三预览界面，即图8中的(b)的状态，将图片输出模式切换为Remosaic模式，即图8中的(b1)的模式，即第一输出模式。
示例性地，结合图9所示，为图像被缩放至一定倍率后拍摄亮度由亮变暗时的图像输出示例。具体地，当图像被放大至例如15倍时，即由第一变焦倍率放大至第二变焦倍率，即图9中的(a)的状态，由于此时亮度判断为处于亮光状态，因而图片输出模式为Remosaic模式，即图9中的(a1)的模式，即第二输出模式。由于用户手持手机摄像时的移动，即从亮光环境移动到暗光环境，从而导致手机拍摄亮度的变化，此时，当手机检测到亮度为暗光时，相机应用显示第三预览界面，即图9中的(b)的状态，将图片输出模式切换为Binning模式，即图9中的(b1)的模式，即第一输出模式。
示例性地，在夜间室内拍摄场景下，在室内灯光较为明亮时，手机拍摄室内场景切换为Remosaic模式输出图片。然而，此时，当手机需要从明亮的室内拍摄黑暗的窗外场景时，由于检测到窗外亮度较暗，虽然室内灯光明亮，手机在拍摄窗外镜像时，仍然需要切换为Binning模式以保证图像成像的清晰度。
结合图10所示，为手机根据亮度选择对应的图像传感器配置并出图的逻辑示例。具体地，包括如下步骤：
S100：用户打开手机相机应用。此时相机应用显示初始的拍照预览界面，默认的初始界面为图像缩放1倍状态(即1x)，默认的初始图像输出模式为Binning模式。
S101：用户操作相机的拍照预览界面以放大或缩小实时预览中的画面图像。可以理解的，用户的图像缩放包括触控屏幕的手势操作及点击手机屏幕的图像缩放按钮等。
示例性地，本申请的实施例中，以用户将实时在相机应用拍照预览界面的显示图像放大至15倍为例。
在图像被缩放至一定倍率时，尤其是当图像被放大至15倍后，由于手持手机造成的手部抖动会比在默认的图像缩放1倍状态下放大15倍，从而使得图像更加抖动而变得模糊和不清晰。因而，可以将缩放后的图像进行清晰度优化。
S102：在用户缩放拍照预览界面的实时图像后，手机对呈现在手机屏幕的画面做清晰度优化。
具体地，在手机检测到S101中的图像缩放至预设切换倍率后，检测拍摄亮度。
可选的，根据手机硬件模组中的环境光传感器检测当前环境光亮度以确定当前呈现在相机拍照预览界面的图像的亮度。
可选的，根据当前相机拍照预览界面的图像平均亮度信息确定当前呈现在相机拍照预览界面的图像亮度，包括图像的颜色、图像中各个像素点的像素值等。
需要说明的是，相机拍照预览界面的亮度检测与判断包括但不限于以上列举的情形。实际使用时，可以根据实际需要采用其他方式获取当前相机拍照预览界面的图像亮度。
在检测到当前拍摄亮度为暗光时，将图像输出模式设置为Binning模式输出图片。
在检测到当前拍摄亮度为亮光时，将图像输出模式设置为Remosaic模式输出图片。
需要说明的是，本申请的实施例中，只有当S101图像缩放至大于预设切换倍率时，才会触发S102。在变焦倍率小于预设切换倍率时，S102输出图像的模式为Binning模式输出图片。可选的，预设切换倍率为6-10倍。示例性地，以8倍作为预设切换倍率。
作为一种示例，在图像倍缩放至15倍时的暗光条件下，由16MBinning模式将图像像素输出为4608*3456。在图像倍缩放至15倍时的亮光条件下，由64MRemosaic模式将图像像素输出为9216*6912，并在此基础上，由图像传感器进一步裁切二分之一至图像像素输出为4608*3456，从而使得在暗光和亮光条件下的输出像素相同。由此，可以节省后续步骤中例如S103中由ISP继续裁切图像时的额外功耗。
完成上述步骤后，即完成对相机应用拍照预览界面输出图像的清晰度优化之后，手机继续优化呈现在手机屏幕的图像抖动情况。
结合图11所示，为本申请实施例的一种图像防抖的实现示例。在用户手持手机拍摄的场景下，相机应用被用户设置为放大15倍以显示远距离的人物。此时，虽然用户存在明显的手部抖动致使图像画面变得模糊，经过本申请公开的图像处理方法后，手机的相机应用呈现的拍照预览界面的图像仍然是稳定的，从而提高了用户拍摄的体验。
结合图12所示，为本申请实施例的另一种图像防抖的实现示例。与图11的区别是，在图12中额外在相机应用的拍照预览界面设置了防抖开关。在实时显示的拍照预览未开启防抖功能前，即图12中的(a)的状态，相机应用的拍照预览界面显示的图像随着手机的抖动而抖动，使得呈现在手机屏幕上的图像变得模糊且不清晰。在开启防抖功能后，即图12中的(b)的状态，即使存在手持手机时的手机抖动，仍然能够保证拍照预览界面的图像能够稳定输出。
结合图13所示，为图12实施例中视场角(FieldofView，FOV)的示例。在响应于用户打开相机应用的操作后，启动摄像头，以单个摄像头为例，可以看到摄像头的拍摄范围。在用户将拍照预览界面的图像放大至15倍后，拍摄范围如图中的FOV图像1。FOV图像1为摄像头采集得到的图像，经过本申请的图像处理方法后，输出显示在手机的拍照预览界面的图像，即经过图像处理后的FOV图像2，即图中的输出图。
视场角(FOV)：用于指示摄像头在拍摄图像的过程中，摄像头的出图范围。若待拍摄物体处于这个角度范围内，该待拍摄物体便会被摄像头采集到，进而呈现在预览图像中。若待拍摄物体处于这个角度范围之外，该被拍摄设备便不会被图像捕捉装置采集到，即不会呈现在预览图像中。通常，摄像头的视场角越大，则拍摄范围就越大，焦距就越短。而摄像头的视场角越小，则拍摄范围就越小，焦距就越长。
在本申请中，也可以将“视场角”称为“视场范围”、“视野范围”等词汇。本文对于“视场角”的名称不作限制，只要表达的是如上的概念即可。
需要说明的是，摄像头的数量可以是一个或者多个，在此不做限定。摄像头设置于电子设备的形式不限，可以是内置摄像头也可以是外置摄像头；可以是前置摄像头，也可以是后置摄像头。
需要说明的是，当摄像包含多个摄像头时，多个摄像头之间的位置可以任意摆放，例如，摄像头可以位于另一摄像头的左侧或者右侧，摄像头也可以位于另一摄像头的上侧或者下侧。 也就是说，本申请不限定摄像模组中任一摄像头与其它摄像头的相对位置关系。
结合图14A、图14B及图15所示，为手机在拍摄过程中发生位移的相机图像成像示例。如图14A所示，为手机在拍摄过程中，由于手部抖动造成图像左右位移的场景。经过本申请图像处理的方法后，实时呈现在相机应用的拍照预览界面的图像为稳定的经处理后的FOV图像2。如图14B所示，为手机在拍摄过程中，由于手部抖动造成图像上下位移的场景。经过本申请图像处理的方法后，实时呈现在相机应用的拍照预览界面的图像为稳定的经处理后的FOV图像2。如图15所示，即FOV图像1为输入图，FOV图像2为经过本申请图像处理的方法后，呈现在手机屏幕的输出图。
结合图16所示，为本申请一种实施例图像处理的逻辑示例。其中，图10详细阐述了步骤S100至步骤S102，此处不再赘述。结合图17-图20，以下具体展开阐述步骤S103至步骤S105。具体地，包括如下步骤：
S103：数字变焦包括ISP将图像缩放至用户选择的变焦倍率输出。
S104：电子防抖包括将接收到的图像做图像防抖处理。
S105：将处理后的图像输出至手机相机应用的拍照预览界面实时显示。
需要说明的是，S104可以在用户的变焦倍率大于预设防抖倍率时才会开启并对输出图像做防抖处理。一般而言，在变焦倍率小于预设防抖倍率时，由S102输入的图像由S103做数字变焦后输出至S105。可选的，触发S104开启图像防抖的预设防抖倍率为6-15倍。
结合图17，为不同倍率下输入图与输出图之间的关系示例，进一步说明上述触发S104开启的预设防抖倍率。以动态裁切为例，在当前的实施例中，例如设置在满足用户变焦倍率大于10倍时，打开图像防抖的算法，即步骤S104，将处理后的图像输出至S105显示在手机相机应用的拍照预览界面。在用户的变焦倍率小于10倍时，输入图经由S103进行数字变焦并输出至S105显示在手机相机应用的拍照预览界面。如图17中的(a)所示，在手机相机应用的拍照预览界面下，用户将当前图像放大至5倍(5x)时，由S103数字变焦对图像进行处理，输出图即为S103处理后的实际放大5倍后的图像。如图17中的(b)所示，在手机相机应用的拍照预览界面下，用户将当前图像放大至10倍(10x)时，由S103数字变焦对图像进行处理，输出图即为S103处理后的实际放大10倍后的图像。如图17中的(c)所示，在手机相机应用的拍照预览界面下，用户将当前图像放大至15倍(15x)时，由于此时变焦倍率大于预设防抖倍率，开启电子防抖S104。在图像缩放至如图17中的(c)中的虚线框时，即10倍(10x)时，保留预设防抖倍率下的输入图像，即10倍(10x)时的输入图像。图像继续缩放至15倍时，电子防抖S104将像素裁剪至用户选择的缩放至15倍的图像。需要说明的是，在10倍时保留预设防抖倍率(即10倍)下的输入图像是为了给后续图像缩放的裁切预留更大的活动以保障图像的抖动范围，从而提高图像缩放至高倍率时现在在手机相机应用拍照预览界面的图像稳定性。经过本申请公开的图像处理方法后显示在手机相机应用的拍照预览界面的示例，即输出图。
结合图18所示，具体展开阐述数字变焦S103的实现过程。由上述S102步骤得出在图像缩放至15倍时的亮暗光条件下，输入至数字变焦S103的图像大小均为4608*3456。例如在暗光条件下，由于变焦倍率是15倍，预设防抖倍率是10倍，长焦倍率是4倍，因而ISP按照第一比例裁切4/10得到一个1842*1382的图像。ISP继续将上述图像下采样0.78倍至1440*1080。 又例如在亮光条件下，由于变焦倍率是15倍，预设防抖倍率是10倍，预设切换倍率是8倍，因而ISP按照第二比例裁切8/10得到一个3684*2764的图像。ISP继续将上述图像下采样0.29倍至1440*1080。
结合图19所示，具体展开阐述电子防抖S104的实现过程。电子防抖S104由至少电子防抖EIS1041模块及图像透视变换1042模块执行。其中，电子防抖EIS1041所需要使用的数据信息包括3D平滑、2D图像及光学防抖OIS。其中，图像透视变换1042包括坐标变换、像素裁剪及上采样。示例性地，结合上述图18中的图像数据为例，由于变焦倍率是15倍，预设防抖倍率是10倍，ISP输入图1440*1080经电子防抖EIS处理后，送入图像透视变换，通过坐标变换并裁切15分之10至960*720，上采样1.5倍后至1440*1080的输出图，送至手机相机应用的拍照预览界面显示S105。该图像是经过本申请的图像处理方法后得到的稳定出图的输出图，其与ISP输出的图像像素保持一致。
结合图20所示，为本申请电子防抖算法框架的示意图，包括3D陀螺仪防抖，2D图像平移。3D陀螺仪防抖包括陀螺仪，经由边界约束至路径平滑，由路径平滑至虚拟相机位姿，陀螺仪同时也输出数据至虚拟相机位姿。其中，路径平滑及虚拟相机位姿组成平滑路径估计。虚拟相机位姿继续输出至单应性变化，在接收光学防抖OIS的数据后输出至图像透视变换信息。2D图像平移包括图像帧，输出至特征点检测和提取，继而特征点对，经过滤后至运动矢量，再由边界约束至路径平滑以获得平移量后输出至图像透视变换信息。在图像透视变换信息接收到3D陀螺仪的单应性变换及2D图像平移的平移量后，对图像帧进行图像透视变换，最终输出稳定的图像帧。
在本申请的实施方式中，例如，如果单独采用3D陀螺仪防抖进行防抖，则会出现在防抖强度调整的较弱的情况下无法实现防抖的效果，又或者在防抖强度调整的较强的情况下，导致拍照预览界面出现延迟，甚至在用户手持手机拍摄时产生的抖动程度较强的情况下，出现画面停滞。又例如，如果采用2D图像平移，图像只会收集图像的内容平移信息，而图像本身不会感知到相机的旋转信息。又例如，如果采用3D陀螺仪防抖与OIS配合的方式实现防抖，则容易产生拍照预览界面出现高频抖动，从而导致拍照预览界面的防抖效果较差，用户体验降低。因而，在本申请的实施方式中，使用3D陀螺仪防抖与2D图像平移相互配合的方式，尤其是2D图像平移可以调节上述拍照预览界面出现的高频抖动。
具体而言，3D陀螺仪防抖与2D图像平移根据用户设置的变焦倍率的不同，分别控制3D陀螺仪防抖与2D图像平移，动态调整二者的图像权重，二者的权重关系可以用例如如下的关系表示：
3D陀螺仪防抖权重＝1-2D图像平移权重
其中，2D图像平移权重又可以通过如下关系式表示：
2D图像平移权重
＝(当前用户设置的变焦倍率-初始变焦倍率)÷(2D图像平移全开倍率-初始变焦倍率)
应理解，当用户设置不同的变焦倍率时，3D陀螺仪防抖与2D图像平移的权重会相应的做出调整，3D陀螺仪防抖的权重会随着用户设置的变焦倍率的增大而逐渐较少，相应的，2D图像平移的权重会随着用户设置的变焦倍率的增大而逐渐增大，直至达到2D图像平移全开倍 率时，EIS只使用图像特征点做防抖，而不使用3D陀螺仪防抖。这样，通过分别控制3D陀螺仪防抖与2D图像平移的权重，使得拍照预览界面时动态考虑多种参数的配合，更加灵活的控制拍照预览界面的防抖，实现更好的防抖效果。
需要说明的是，2D图像平移全开倍率为2D图像平移可以支持的最大倍率，例如50倍或其他变焦倍率，本申请在此不做限定。
示例性的，例如用户设置的初始变焦倍率为1倍，当前用户设置的变焦倍率为10倍，2D图像平移全开倍率为50倍，则2D图像平移的权重为9/49，3D陀螺仪防抖的权重为1-9/49，即40/49。又例如，用户设置的初始变焦倍率为1倍，当前用户设置的变焦倍率为50倍，2D图像平移全开倍率为50倍，则2D图像平移的权重为49/49，3D陀螺仪防抖的权重为1-49/49，即0。也就是说，当用户设置的变焦倍率大于或等于2D图像平移全开倍率时，3D陀螺仪防抖的权重会降为0，即3D陀螺仪防抖不参与拍照预览界面的防抖，而仅由2D图像平移参与拍照预览界面的防抖。这样，尤其是在用户设置的变焦倍率较高的情况下，拍照预览界面的防抖效果及收益会更加明显。
通过上述实现方式，用户在不同的变焦倍率的情况下，拍照预览界面呈现的画面更加稳定，减少用户手持手机拍摄时产生的图像抖动，从而用户可以更方便的通过稳定的图像看清画面中的内容。需要说明的是，增强拍照预览界面呈现的防抖以使得画面更加稳定从而用户可以看清画面中的内容，其并不是调整了图像的清晰度，其图像的清晰度依然保持不变。
图21为本申请的另一种实施例。与上述实施例的区别在于，本实施例中，对于输出图可以做固定裁切。在当前的实施例中，例如设置在满足用户变焦倍率大于10倍时，打开图像防抖的算法，即步骤S104，将处理后的图像输出至S105显示在手机相机应用的拍照预览界面。在用户的变焦倍率小于10倍时，输入图经由S103进行数字变焦并输出至S105显示在手机相机应用的拍照预览界面。
如图21中的(a)所示，在手机相机应用的拍照预览界面下，用户将当前图像放大至5倍(5x)时，由S103数字变焦对图像进行处理，输出图即为S103处理后的实际放大5倍后的图像。如图21中的(b)所示，在手机相机应用的拍照预览界面下，用户将当前图像放大至10倍(10x)时，由S103数字变焦对图像进行处理，输出图即为S103处理后的实际放大10倍后的图像。如图21中的(c)所示，在手机相机应用的拍照预览界面下，用户将当前图像放大至15倍(15x)时，由于此时变焦倍率大于预设防抖倍率，即变焦倍率大于10倍，开启电子防抖S104。在图像缩放至如图21中的(c)时，由于此时是固定裁切，因而为了得到实际输出图为用户将图像放大至15倍后的数据，需要预留一个预定倍率小于该15倍图像的活动范围。在本实施例中，预定倍率为15倍乘以一个预设系数。示例性地，该预设系数为80％，即输出图在15倍的80％倍率时，即在12倍率时，保留预定倍率(即12倍)下的输入图像，即图21中的(c)中的虚线框部分，图像继续缩放至15倍时，电子防抖S104将像素裁剪至用户选择的缩放至15倍的图像。可以理解的是，后续的图像缩放都是按照上述方式输出图像。示例性地，如图21中的(d)所示，为了得到用户选择的放大20倍后的图像输出图，在图像缩放至20倍时乘以预设系数，即80％(16倍)，保留该输入图像，图像继续缩放至20倍时，电子防抖S104将像素裁剪至用户选择的缩放至20倍的图像。可选的，预设系数为60％-90％。经过本申请公开的图像处理方法后显示在手机相机应用的拍照预览界面的示例，即输出图。
示例性地，具体展开阐述数字变焦S103的实现过程。由上述S102步骤得出在图像缩放至15倍时的亮暗光条件下，输入至数字变焦S103的图像大小均为4608*3456。例如在暗光条件下，由于变焦倍率是15倍，预设防抖倍率是10倍，长焦镜头基础倍率是10倍，因而ISP按照第一比例裁切4/15得到一个1228*920的图像。ISP继续将上述图像上采样1.41倍至1728*1296。又例如在亮光条件下，由于变焦倍率是15倍，预设防抖倍率是10倍，预设切换倍率是8倍，因而ISP按照第二比例裁切4/15得到一个2456*1840的图像。ISP继续将上述图像下采样0.7倍至1728*1296。
示例性地，具体展开阐述数字变焦S104的实现过程。示例性地，结合上述实施例中ISP的图像输出的图像数据为例，由于变焦倍率是15倍，预设防抖倍率是10倍，ISP输入图1728*1296经电子防抖EIS处理后，送入图像透视变换，通过坐标变换并裁切20％至1440*1080的输出图，送至手机相机应用的拍照预览界面显示S105。该图像是经过本申请的图像处理方法后得到的稳定出图的输出图。
本实施例的实施逻辑与上述图16一致，此处不再赘述。
图22为本申请实施例的一种逻辑实现时序图。作为一种示例，变焦倍率按照如下步骤实现。仍然以上述用户选择的变焦倍率为15倍进行阐述。具体地，在用户打开手机相机应用的拍照预览界面时，相机应用初始界面的变焦倍率为1倍。此时，由主摄摄像头采集手机拍摄范围内的图像，当变焦倍率放大至4倍时，转为长焦摄像头采集手机拍摄范围内的图像。本实施例中，作为一种示例，预设切换倍率为8倍，预设防抖倍率为10倍。可以理解的是，预设切换倍率以及预设防抖倍率的倍率范围可以根据不同场景改变，上文已有描述，此处不再赘述。
具体地，在亮度检测的场景，作为一种示例，由于当前设定的预设切换倍率为8倍，因而在图像缩放小于8倍时，图像以Binning模式输出图片。在图像缩放至8倍时，进行亮度检测，在检测到当前亮度为暗光时，以Binning模式输出图片，在检测到当前亮度为亮光时，以Remosaic模式输出图片。可以理解的是，亮度检测在图像缩放至8倍时开启，在大于8倍的变焦倍率时，仍然持续检测亮度，并根据当前检测到的亮度切换图片输出模式。
具体地，在像素裁剪的场景，作为一种示例，在图像缩放小于8倍时，由ISP数字变焦对图像进行缩放。由于预设切换倍率是8倍，因而在变焦倍率介于8倍和10倍之间时，根据亮度不同切换不同的图片输出模式，像素裁剪模式也因此有所不同。具体地，在暗光条件下，由ISP数字变焦对图像进行缩放。在亮光条件下，由图像传感器先对输入的图像做裁切以使得在亮光条件下的输出图像与暗光条件下输出的图像倍率相同。然后再将图像传感器裁切后的图像输出至ISP数字变焦对图像进行缩放。可以理解的，由图像传感器先对输出图像进行裁切然后输出至ISP数字变焦，可以节省ISP的功耗。在图像缩放大于10倍时，以动态裁切为例，由电子防抖对图像进行裁切。
结合图23所示，为主摄摄像头与长焦摄像头在变焦倍率不同时进行切换的示意图。具体地，在变焦倍率小于4倍时，由主摄摄像头采集并得到图像。示例性地，如图23中的(a)，在变焦倍率为1倍时，由主摄摄像头采集并得到图像，如图23中的(b)，在变焦倍率为3.9倍时，仍由主摄摄像头采集并得到图像。如图23中的(c)，在变焦倍率为4倍时，由长焦摄像头采集并得到图像。
图24为本申请的另一种实施例。在本实施例的场景下，如图24所示，与前述实施例区别 的地方在于不包括电子防抖S104。因而，图像的输出与裁剪由S103数字变焦完成。
示例性地，例如当前变焦倍率是8倍(而非本申请其他实施例中的15倍)，长焦镜头的基础倍率例如4倍。具体展开阐述数字变焦S103的实现过程。由上述S102步骤得出在图像缩放至8倍时的亮暗光条件下，输入至数字变焦S103的图像大小均为4608*3456。例如在暗光条件下，ISP按照第一比例裁切4/8得到一个2304*1728的图像。ISP继续将上述图像下采样0.625倍至1440*1080。又例如在亮光条件下，因而ISP按照第二比例裁切8/8得到一个4608*3456的图像。ISP继续将上述图像下采样0.3125倍至1440*1080。
具体实现中，本申请还提供一种计算机存储介质，其中，该计算机存储介质可存储有程序，其中，在所述程序运行时控制所述计算机可读存储介质所在设备执行上述实施例中的部分或全部步骤。所述的存储介质可为磁碟、光盘、只读存储记忆体(英文：read-only memory，简称：ROM)或随机存储记忆体(英文：random access memory，简称：RAM)等。
具体实现中，本申请实施例还提供了一种计算机程序产品，所述计算机程序产品包含可执行指令，当所述可执行指令在计算机上执行时，使得计算机执行上述方法实施例中的部分或全部步骤。
以上所述，仅为本发明的具体实施方式，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本发明的保护范围之内。本发明的保护范围应以所述权利要求的保护范围为准。

Claims (26)

  1. 一种图像拍摄方法，其特征在于，包括：显示相机应用的第一预览界面，所述第一预览界面包括第一预览图像，所述第一预览图像为摄像头采集的第一图像经处理后得到的，所述第一预览图像对应第一变焦倍率，所述第一图像对应所述摄像头的第一输出模式；
    接收用户的第一操作；
    响应于所述第一操作，显示所述相机应用的第二预览界面，所述第二预览界面包括第二预览图像，所述第二预览图像为摄像头采集的第二图像经处理后得到的，所述第二预览图像对应第二变焦倍率，所述第二图像对应所述摄像头的第二输出模式，其中，所述第二输出模式与所述第一输出模式不同，所述第二变焦倍率与所述第一变焦倍率不同。
  2. 根据权利要求1所述的图像拍摄方法，其特征在于：所述第一输出模式为Binning，Remosaic，Quadra，QuadraHDR或StaggerHDR之一。
  3. 根据权利要求1所述的图像拍摄方法，其特征在于：所述第二输出模式为Binning，Remosaic，Quadra，QuadraHDR或StaggerHDR之一。
  4. 根据权利要求1所述的图像拍摄方法，其特征在于：所述图像拍摄方法还包括，
    所述第一输出模式或所述第二输出模式基于所述摄像头的拍摄亮度确定。
  5. 根据权利要求4所述的图像拍摄方法，其特征在于：
    所述拍摄亮度基于环境光传感器采集的当前环境光亮度确定。
  6. 根据权利要求4所述的图像拍摄方法，其特征在于：
    所述拍摄亮度基于所述第二预览图像的平均亮度信息确定。
  7. 根据权利要求4-6中的任一项所述的图像拍摄方法，其特征在于：所述拍摄亮度包括亮光和暗光。
  8. 根据权利要求7所述的图像拍摄方法，其特征在于：
    当所述拍摄亮度为暗光时，所述第一输出模式为Binning。
  9. 根据权利要求7所述的图像拍摄方法，其特征在于：
    当所述拍摄亮度为暗光时，所述第二输出模式为Binning。
  10. 根据权利要求7所述的图像拍摄方法，其特征在于：
    当所述拍摄亮度为亮光时，所述第一输出模式为Remosaic。
  11. 根据权利要求7所述的图像拍摄方法，其特征在于：
    当所述拍摄亮度为亮光时，所述第二输出模式为Remosaic。
  12. 根据权利要求1所述的图像拍摄方法，其特征在于：所述第一操作包括用于缩放所述第一预览图像的手势操作、屏幕点击操作或拖拽变焦进度条之一。
  13. 根据权利要求7所述的图像拍摄方法，其特征在于：所述第一预览界面及所述第二预览界面的所述拍摄亮度为亮光。
  14. 根据权利要求13所述的图像拍摄方法，其特征在于：所述方法还包括：响应于所述拍摄亮度切换为暗光时，所述相机应用显示第三预览界面，所述第三预览界面对应所述第一输出模式。
  15. 根据权利要求13所述的图像拍摄方法，其特征在于：所述方法还包括：
    所述第二变焦倍率大于预设切换倍率。
  16. 根据权利要求15所述的图像拍摄方法，其特征在于：所述方法还包括：
    接收用户的第二操作；
    响应于所述第二操作，显示所述相机应用的第四预览界面，所述第四预览界面包括第四预览图像，所述第四预览图像为摄像头采集的第四图像经处理后得到的，所述第四预览图像对应第三变焦倍率，所述第四图像对应所述摄像头的第一输出模式，其中，所述第三变焦倍率小于所述预设切换倍率。
  17. 根据权利要求16所述的图像拍摄方法，其特征在于：所述第二操作包括用于缩放所述第一预览图像的手势操作、屏幕点击操作或拖拽变焦进度条之一。
  18. 根据权利要求13所述的图像拍摄方法，其特征在于：所述方法还包括：
    所述第二变焦倍率大于预设防抖倍率，将所述第二图像基于所述第二变焦倍率经由数字变焦及电子防抖进行像素裁剪后得到所述第二预览界面。
  19. 根据权利要求18所述的图像拍摄方法，其特征在于：所述数字变焦包括基于所述第二变焦倍率及所述第二输出模式进行像素裁剪。
  20. 根据权利要求18所述的图像拍摄方法，其特征在于：所述电子防抖包括3D陀螺仪、2D图像特征点及图像透视变换，其中所述3D陀螺仪适用于3D图像平滑。
  21. 根据权利要求20所述的图像拍摄方法，其特征在于：所述3D陀螺仪及所述2D图像特征点基于所述第二变焦倍率进行像素裁剪。
  22. 根据权利要求20所述的图像拍摄方法，其特征在于：所述图像透视变换包括坐标变换。
  23. 根据权利要求14所述的图像拍摄方法，其特征在于：所述方法还包括：
    所述第二变焦倍率大于预设防抖倍率，将所述第二图像基于所述第二变焦倍率经由数字变焦及电子防抖进行像素裁剪后得到所述第二预览界面。
  24. 一种电子设备，其特征在于，所述电子设备包括用于存储计算机程序指令的存储器和用于执行程序指令的处理器，其中，当该计算机程序指令被该处理器执行时，触发所述电子设备执行权利要求1-23任一项所述的方法。
  25. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质中存储有计算机程序，当其在计算机上运行时，使得计算机执行权利要求1-23任一项所述的方法。
  26. 一种计算机程序产品，其特征在于，所述计算机程序产品包括计算机程序，当其在计算机上运行时，使得计算机执行权利要求1-23任一项所述的方法。
PCT/CN2022/107449 2021-08-12 2022-07-22 图像拍摄的方法、设备、存储介质和程序产品 Ceased WO2023016232A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US18/278,923 US12389117B2 (en) 2021-08-12 2022-07-22 Image photographing method, device, storage medium, and program product
EP22855215.4A EP4287638B1 (en) 2021-08-12 2022-07-22 Image photographing method, device and program product
CN202280068390.9A CN118435615A (zh) 2021-08-12 2022-07-22 图像拍摄的方法、设备、存储介质和程序产品

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN202110928106.8 2021-08-12
CN202110928106 2021-08-12
CN202111040576.7A CN115706850B (zh) 2021-08-12 2021-09-06 图像拍摄的方法、设备和存储介质
CN202111040576.7 2021-09-06

Publications (1)

Publication Number Publication Date
WO2023016232A1 true WO2023016232A1 (zh) 2023-02-16

Family

ID=85180607

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/107449 Ceased WO2023016232A1 (zh) 2021-08-12 2022-07-22 图像拍摄的方法、设备、存储介质和程序产品

Country Status (4)

Country Link
US (1) US12389117B2 (zh)
EP (1) EP4287638B1 (zh)
CN (2) CN115706850B (zh)
WO (1) WO2023016232A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN120769168A (zh) * 2024-06-14 2025-10-10 荣耀终端股份有限公司 一种光学防抖动控制方法及电子设备

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117156261B (zh) * 2023-03-28 2024-07-02 荣耀终端有限公司 图像处理方法及相关设备
CN119316704A (zh) * 2023-07-12 2025-01-14 荣耀终端有限公司 相机应用的图像处理方法及相关设备
CN117714858B (zh) * 2023-07-31 2024-12-20 荣耀终端有限公司 一种图像处理方法、电子设备及可读存储介质
CN117278850B (zh) * 2023-10-30 2025-03-07 荣耀终端股份有限公司 一种拍摄方法及电子设备
CN120151650A (zh) * 2023-12-12 2025-06-13 荣耀终端股份有限公司 图像拍摄方法及相关设备
CN118175441B (zh) * 2024-04-24 2025-02-14 荣耀终端有限公司 图像传感器、图像处理方法、电子设备、存储介质及产品
EP4697736A4 (en) * 2024-06-17 2026-04-08 Samsung Electronics Co Ltd METHOD FOR PROVIDING IMAGES AND ELECTRONIC DEVICE SUPPORTING IT

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012105171A (ja) * 2010-11-12 2012-05-31 Sharp Corp 画像処理装置
CN106454132A (zh) * 2016-11-29 2017-02-22 广东欧珀移动通信有限公司 控制方法、控制装置及电子装置
CN106657780A (zh) * 2016-12-16 2017-05-10 北京小米移动软件有限公司 图像预览方法和装置
CN107317969A (zh) * 2017-07-27 2017-11-03 广东欧珀移动通信有限公司 图像去雾方法、装置、计算机可存储介质和移动终端
CN109089036A (zh) * 2018-07-09 2018-12-25 奇酷互联网络科技（深圳）有限公司 变焦处理方法、装置、可读存储介质及移动终端
CN109831621A (zh) * 2018-12-29 2019-05-31 维沃移动通信有限公司 相机预览图像显示方法和移动终端
CN110784651A (zh) * 2019-11-15 2020-02-11 维沃移动通信有限公司 一种防抖方法及电子设备
CN110855896A (zh) * 2019-12-13 2020-02-28 惠州Tcl移动通信有限公司 拍摄方法及装置、存储介质、终端设备
CN110958387A (zh) * 2019-11-19 2020-04-03 维沃移动通信有限公司 一种内容更新方法及电子设备
CN111107265A (zh) * 2019-12-25 2020-05-05 RealMe重庆移动通信有限公司 图像处理方法及装置、计算机可读介质、电子设备
US20210218898A1 (en) * 2020-01-10 2021-07-15 Samsung Electronics Co., Ltd. Method of improving image quality in zoom scenario with single camera, and electronic device including the same
CN114007011A (zh) * 2021-10-26 2022-02-01 希姆通信息技术（上海）有限公司 基于相机的输出模式切换方法及其系统

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3286915B1 (en) * 2015-04-23 2021-12-08 Apple Inc. Digital viewfinder user interface for multiple cameras
KR102352681B1 (ko) * 2015-07-27 2022-01-18 삼성전자주식회사 동영상 안정화 방법 및 이를 위한 전자 장치
KR102530752B1 (ko) 2018-08-29 2023-05-10 삼성전자주식회사 이미지 센서 및 이미지 센서를 포함하는 전자 기기와, 이미지 줌 프로세싱 방법
EP3997866B1 (en) * 2019-08-26 2024-10-23 Samsung Electronics Co., Ltd. System and method for content enhancement using quad color filter array sensors
CN114915726A (zh) * 2019-11-15 2022-08-16 华为技术有限公司 一种拍摄方法及电子设备
CN110933302B (zh) * 2019-11-27 2021-06-25 维沃移动通信有限公司 一种拍摄方法及电子设备
CN113497881B (zh) * 2020-03-20 2022-11-08 华为技术有限公司 图像处理方法及装置
CN113596316B (zh) * 2020-04-30 2023-12-08 华为技术有限公司 拍照方法及电子设备
CN114531539B (zh) * 2020-11-23 2024-03-19 华为技术有限公司 拍摄方法及电子设备
CN115131222B (zh) * 2021-03-29 2026-01-02 华为技术有限公司 一种图像处理方法以及相关设备
CN115250327A (zh) * 2021-04-27 2022-10-28 华为技术有限公司 一种变焦拍摄场景下的预览图像显示方法与电子设备
CN115658191B (zh) * 2021-07-09 2026-04-03 华为技术有限公司 一种生成主题壁纸的方法及电子设备

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012105171A (ja) * 2010-11-12 2012-05-31 Sharp Corp 画像処理装置
CN106454132A (zh) * 2016-11-29 2017-02-22 广东欧珀移动通信有限公司 控制方法、控制装置及电子装置
CN106657780A (zh) * 2016-12-16 2017-05-10 北京小米移动软件有限公司 图像预览方法和装置
CN107317969A (zh) * 2017-07-27 2017-11-03 广东欧珀移动通信有限公司 图像去雾方法、装置、计算机可存储介质和移动终端
CN109089036A (zh) * 2018-07-09 2018-12-25 奇酷互联网络科技（深圳）有限公司 变焦处理方法、装置、可读存储介质及移动终端
CN109831621A (zh) * 2018-12-29 2019-05-31 维沃移动通信有限公司 相机预览图像显示方法和移动终端
CN110784651A (zh) * 2019-11-15 2020-02-11 维沃移动通信有限公司 一种防抖方法及电子设备
CN110958387A (zh) * 2019-11-19 2020-04-03 维沃移动通信有限公司 一种内容更新方法及电子设备
CN110855896A (zh) * 2019-12-13 2020-02-28 惠州Tcl移动通信有限公司 拍摄方法及装置、存储介质、终端设备
CN111107265A (zh) * 2019-12-25 2020-05-05 RealMe重庆移动通信有限公司 图像处理方法及装置、计算机可读介质、电子设备
US20210218898A1 (en) * 2020-01-10 2021-07-15 Samsung Electronics Co., Ltd. Method of improving image quality in zoom scenario with single camera, and electronic device including the same
CN114007011A (zh) * 2021-10-26 2022-02-01 希姆通信息技术（上海）有限公司 基于相机的输出模式切换方法及其系统

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4287638A4 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN120769168A (zh) * 2024-06-14 2025-10-10 荣耀终端股份有限公司 一种光学防抖动控制方法及电子设备

Also Published As

Publication number Publication date
CN115706850B (zh) 2024-04-05
EP4287638B1 (en) 2025-08-27
EP4287638A4 (en) 2024-09-11
CN118435615A (zh) 2024-08-02
US20240056685A1 (en) 2024-02-15
CN115706850A (zh) 2023-02-17
US12389117B2 (en) 2025-08-12
EP4287638A1 (en) 2023-12-06

Similar Documents

Publication Publication Date Title
US12389117B2 (en) Image photographing method, device, storage medium, and program product
CN114205522B (zh) 一种长焦拍摄的方法及电子设备
KR102577396B1 (ko) 녹화 프레임 레이트 제어 방법 및 관련 장치
CN113747085B (zh) 拍摄视频的方法和装置
US12526515B2 (en) Method for image or video shooting and related device
CN113473005B (zh) 拍摄中转场动效插入方法、设备、存储介质
CN112532892B (zh) 图像处理方法及电子装置
EP4109882B1 (en) Image processing method and electronic device
CN113556466B (zh) 一种对焦方法和电子设备
WO2022262313A1 (zh) 基于画中画的图像处理方法、设备、存储介质和程序产品
CN113596321B (zh) 转场动效的生成方法、设备和存储介质
CN113099146A (zh) 一种视频生成方法、装置及相关设备
CN116055855B (zh) 图像处理方法及其相关设备
WO2022228259A1 (zh) 一种目标追踪方法及相关装置
WO2023035868A1 (zh) 拍摄方法及电子设备
CN113518172B (zh) 图像处理方法和装置
CN116389885B (zh) 拍摄方法、电子设备及存储介质
CN117956299B (zh) 拍摄月亮的方法和电子设备
WO2025119320A1 (zh) 一种拍摄参数调节方法、电子设备及存储介质
WO2025119318A1 (zh) 一种拍摄参数调节方法、电子设备及存储介质
CN120224015A (zh) 一种具有多摄像头设备变焦的方法及装置
HK40078668B (zh) 拍摄方法及其相关设备

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 18278923

Country of ref document: US

WWE Wipo information: entry into national phase

Ref document number: 2022855215

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2022855215

Country of ref document: EP

Effective date: 20230901

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 202280068390.9

Country of ref document: CN

WWG Wipo information: grant in national office

Ref document number: 18278923

Country of ref document: US

WWG Wipo information: grant in national office

Ref document number: 2022855215

Country of ref document: EP