WO2022143311A1 - 一种智能取景推荐的拍照方法及装置 - Google Patents

一种智能取景推荐的拍照方法及装置 Download PDF

Info

Publication number
WO2022143311A1
WO2022143311A1 PCT/CN2021/140208 CN2021140208W WO2022143311A1 WO 2022143311 A1 WO2022143311 A1 WO 2022143311A1 CN 2021140208 W CN2021140208 W CN 2021140208W WO 2022143311 A1 WO2022143311 A1 WO 2022143311A1
Authority
WO
WIPO (PCT)
Prior art keywords
framing
recommendation
image
camera
electronic device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2021/140208
Other languages
English (en)
French (fr)
Inventor
王银廷
马飞龙
宋星光
雷奕
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to EP21914050.6A priority Critical patent/EP4258650A4/en
Publication of WO2022143311A1 publication Critical patent/WO2022143311A1/zh
Priority to US18/343,942 priority patent/US12464226B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/64—Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/631—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
    • H04N23/632—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/633—Control of cameras or camera modules by using electronic viewfinders for displaying additional information relating to control or operation of the camera
    • H04N23/635—Region indicators; Field of view indicators
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/60—Rotation of whole images or parts thereof
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/24—Aligning, centring, orientation detection or correction of the image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/94—Hardware or software architectures specially adapted for image or video understanding
    • G06V10/945—User interactive design; Environments; Toolboxes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/35—Categorising the entire scene, e.g. birthday party or wedding scene
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/45—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/617—Upgrading or updating of programs or applications for camera control
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/631—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/69—Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/95—Computational photography systems, e.g. light-field imaging systems

Definitions

  • the present application relates to the technical field of image processing, and in particular, to a photographing method and device for intelligent framing recommendation.
  • the composition in the camera technology requires more professional skills, and ordinary users lack of professional photography knowledge, and it is difficult to take pictures with perfect composition and in line with aesthetic rules.
  • an existing technical method of intelligently switching cameras to assist composition by identifying the foreground category in the image currently captured by the user, matching a suitable composition template, and then switching the wide-angle lens or the telephoto lens for imaging according to the composition template, in order to obtain a suitable composition. Suggested image for the current composition.
  • this technical method is based on switching between a wide-angle lens and a telephoto lens. After the image generated by a single lens is cropped or zoomed according to the composition template, the image quality of the image is poor.
  • the present application provides a photographing method and device for intelligent framing recommendation, which can automatically match a suitable framing recommendation scheme for a user according to an algorithm, thereby improving imaging quality.
  • a first aspect provides a photographing method for intelligent framing recommendation, which is applied to an electronic device configured with at least two cameras with different focal lengths.
  • the method includes: obtaining at least one framing recommendation result according to an image collected by the cameras, and the framing recommendation result includes: A framing recommendation frame, which is used to indicate the framing recommendation effect of image shooting; a target image is displayed according to the framing recommendation result selected by the user, wherein the target image is obtained by clipping the images collected by at least two cameras according to the framing recommendation result Image.
  • a framing recommendation result is obtained by performing deep learning on the joint imaging of multiple cameras, so that the electronic device can automatically crop or rotate the images jointly collected by the multiple cameras according to the framing recommendation result, so as to obtain a more optimized composition. and viewing angle, improve the image shooting quality of electronic equipment, and enhance the user's shooting experience.
  • the framing recommendation result further includes at least one of a horizontal reference line or a vertical reference line, and the horizontal reference line and the vertical reference line are used to indicate that the image captured by the camera is rotated to obtain the target image.
  • the electronic device can automatically perform rotation correction processing on the image, so that when the user shoots a specific scene or detects that the image includes a specific line, the user does not need manual adjustment to avoid the situation that the user shoots crookedly and slantly. , the image quality is higher, and the user's shooting experience is improved.
  • the framing recommendation result further includes a thumbnail corresponding to the framing recommendation frame, and the thumbnail is a thumbnail obtained by trimming the image collected by the camera according to the framing recommendation frame.
  • the electronic device can enable the user to intuitively see the shooting effect of the framing recommendation, improve the probability of the user using the framing recommendation function, and improve the user's interactive experience.
  • At least one framing recommendation result is obtained according to the image collected by the camera, which specifically includes: performing scene detection according to the image collected by the camera to obtain composition structure information; The photographing composition calculation is performed by the framing recommendation model, and at least one framing recommendation result is obtained.
  • the electronic device can use the framing recommendation model to calculate the framing and composition of the images collected by the camera, and use the high computing and processing efficiency of the neural network to obtain a more matching framing recommendation result, thereby improving the image processing calculation. Real-time and accurate, improve the efficiency of image processing.
  • the size of the framing recommendation frame is smaller than a first threshold and larger than or equal to a second threshold
  • the first threshold is set according to the imaging range corresponding to the camera with a larger field of view among the multiple cameras
  • the first threshold is The second threshold may be set according to the imaging range corresponding to the camera with the smaller field of view among the multiple cameras.
  • the electronic device determines the size range of the framing recommendation frame according to the characteristics of multiple different cameras, thereby recommending a more reasonable framing recommendation frame for the user, so that the framing recommendation result not only has a reasonable composition, but also has a suitable size of the shooting field of view. It will not make the user feel abrupt, and improve the user's shooting experience.
  • a photographing method recommended by intelligent framing which is applied to an electronic device configured with at least two cameras with different focal lengths.
  • the framing recommendation result includes a framing recommendation frame, and the framing recommendation frame is used to indicate the framing recommendation effect of image shooting; the second operation from the user is received, and the second operation is used to instruct the user to determine the first framing from at least one framing recommendation result.
  • a recommendation frame in response to the second operation, update the real-time preview image to a target image, wherein the target image is an image obtained by clipping images collected by at least two cameras according to the first framing recommendation frame.
  • the framing recommendation result further includes at least one of a horizontal reference line or a vertical reference line, and the horizontal reference line and the vertical reference line are used to indicate that the image captured by the camera is rotated to obtain the target image.
  • the framing recommendation result further includes a thumbnail corresponding to the framing recommendation frame, and the thumbnail is a thumbnail obtained by trimming the image collected by the camera according to the framing recommendation frame.
  • At least one framing recommendation result is obtained according to the image collected by the camera, which specifically includes: performing scene detection according to the image collected by the camera to obtain composition structure information; The photographing composition calculation is performed by the framing recommendation model, and at least one framing recommendation result is obtained.
  • the size of the framing recommendation frame is smaller than a first threshold and larger than or equal to a second threshold
  • the first threshold is set according to the imaging range corresponding to the camera with a larger field of view among the multiple cameras
  • the first threshold is The second threshold may be set according to the imaging range corresponding to the camera with the smaller field of view among the multiple cameras.
  • the method further includes: in response to the user's confirmation shooting operation, saving the target image and the images collected by at least two cameras with different focal lengths respectively.
  • the method further includes: obtaining at least one framing optimization result according to the target image, where the framing optimization result includes a framing recommendation frame, and the framing optimization result is used to instruct the target image to perform framing optimization processing again.
  • a photographing device for intelligent framing recommendation includes: a framing recommendation module, configured to obtain at least one framing recommendation result according to an image collected by a camera, where the framing recommendation result includes a framing recommendation frame, and the framing recommendation frame is used for Indicates the framing recommendation effect of image shooting; the imaging module is used to display the target image according to the framing recommendation result selected by the user, wherein the target image is an image obtained by trimming the images collected by at least two cameras according to the framing recommendation result.
  • the framing recommendation result further includes at least one of a horizontal reference line or a vertical reference line, and the horizontal reference line and the vertical reference line are used to indicate that the image captured by the camera is rotated to obtain the target image.
  • the framing recommendation result further includes a thumbnail corresponding to the framing recommendation frame, and the thumbnail is a thumbnail obtained by trimming the image collected by the camera according to the framing recommendation frame.
  • the framing recommendation module is specifically used to: perform scene detection according to the image collected by the camera to obtain composition structure information; perform photographic composition calculation on the image and composition structure information collected by the camera through the framing recommendation model, Get at least one framing recommendation result.
  • the size of the framing recommendation frame is smaller than a first threshold and larger than or equal to a second threshold
  • the first threshold is set according to the imaging range corresponding to the camera with a larger field of view among the multiple cameras
  • the first threshold is The second threshold may be set according to the imaging range corresponding to the camera with the smaller field of view among the multiple cameras.
  • an electronic device comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions, so as to implement the above-mentioned first The method of any one of the one aspect or the second aspect.
  • a computer-readable storage medium is provided.
  • the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute the first aspect or the second aspect. The method of any one.
  • a computer program product which, when the computer program product runs on a computer, causes the computer to execute the method according to any one of the first aspect or the second aspect.
  • any of the above-mentioned recommended photographing devices, computer-readable storage media or computer program products for intelligent framing can be implemented by the corresponding methods provided above, and therefore, the beneficial effects that can be achieved can be achieved.
  • the beneficial effects in the corresponding methods provided above can be referred to, and details are not repeated here.
  • 1 is a hardware system architecture diagram of an electronic device provided by an embodiment of the present application.
  • FIG. 2 is a software system architecture diagram of an electronic device provided by an embodiment of the present application.
  • FIG. 3 is a schematic diagram of a photographing scene recommended by an intelligent framing according to an embodiment of the present application
  • FIG. 4 is a schematic flowchart of a photographing method recommended by intelligent framing according to an embodiment of the present application
  • FIG. 5 is a schematic interface diagram of a photographing method recommended by intelligent framing according to an embodiment of the present application
  • FIG. 6 is a schematic interface diagram of another recommended photographing method for intelligent framing provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of the flow and effect of a photographing method for intelligent framing recommendations provided by an embodiment of the present application.
  • FIGS. 8-10 are schematic diagrams 1 to 3 of the framing frame of the photographing method recommended by intelligent framing according to the embodiment of the present application;
  • FIG. 11 is a schematic flowchart of another recommended photographing method for intelligent framing provided by an embodiment of the present application.
  • FIG. 12 is a schematic structural diagram of another photographing device for intelligent viewfinder recommendation provided by an embodiment of the present application.
  • first and second are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implicitly indicating the number of indicated technical features.
  • a feature defined as “first” or “second” may expressly or implicitly include one or more of that feature.
  • plural means two or more.
  • the image capturing method provided by the embodiment of the present application can be applied to an electronic device with capturing capability and image processing capability, and the electronic device may be a mobile phone, a tablet computer, a desktop type, a laptop type, a handheld computer, a notebook computer, a vehicle-mounted device, a supercomputer Mobile personal computer (ultra-mobile personal computer, UMPC), netbook, and cellular phone, personal digital assistant (personal digital assistant, PDA), augmented reality (augmented reality, AR) ⁇ virtual reality (virtual reality, VR) equipment, etc.
  • PDA personal digital assistant
  • augmented reality augmented reality
  • VR virtual reality
  • FIG. 1 shows a schematic structural diagram of an electronic device 100 .
  • the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charge management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2 , mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, buttons 190, motor 191, indicator 192, camera 193, display screen 194, and Subscriber identification module (subscriber identification module, SIM) card interface 195 and so on.
  • SIM Subscriber identification module
  • the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and ambient light. Sensor 180L, bone conduction sensor 180M, etc.
  • the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100 .
  • the electronic device 100 may include more or less components than shown, or some components may be combined, or some components may be split, or different component arrangements.
  • the illustrated components may be implemented in hardware, software, or a combination of software and hardware.
  • the processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit (NPU) Wait. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.
  • application processor application processor, AP
  • modem processor graphics processor
  • graphics processor graphics processor
  • ISP image signal processor
  • controller memory
  • video codec digital signal processor
  • DSP digital signal processor
  • NPU neural-network processing unit
  • the controller may be the nerve center and command center of the electronic device 100 .
  • the controller can generate an operation control signal according to the instruction operation code and timing signal, and complete the control of fetching and executing instructions.
  • a memory may also be provided in the processor 110 for storing instructions and data.
  • the memory in processor 110 is cache memory. This memory may hold instructions or data that have just been used or recycled by the processor 110 . If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated accesses are avoided and the latency of the processor 110 is reduced, thereby increasing the efficiency of the system.
  • the processor 110 may include one or more interfaces.
  • the interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transceiver (universal asynchronous transmitter) receiver/transmitter, UART) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, subscriber identity module (SIM) interface, and / or universal serial bus (universal serial bus, USB) interface, etc.
  • I2C integrated circuit
  • I2S integrated circuit built-in audio
  • PCM pulse code modulation
  • PCM pulse code modulation
  • UART universal asynchronous transceiver
  • MIPI mobile industry processor interface
  • GPIO general-purpose input/output
  • SIM subscriber identity module
  • USB universal serial bus
  • the MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193 .
  • MIPI interfaces include camera serial interface (CSI), display serial interface (DSI), etc.
  • the processor 110 communicates with the camera 193 through a CSI interface, so as to realize the photographing function of the electronic device 100 .
  • the processor 110 communicates with the display screen 194 through the DSI interface to implement the display function of the electronic device 100 .
  • the GPIO interface can be configured by software.
  • the GPIO interface can be configured as a control signal or as a data signal.
  • the GPIO interface may be used to connect the processor 110 with the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, and the like.
  • the GPIO interface can also be configured as I2C interface, I2S interface, UART interface, MIPI interface, etc.
  • the interface connection relationship between the modules illustrated in the embodiments of the present application is only a schematic illustration, and does not constitute a structural limitation of the electronic device 100 .
  • the electronic device 100 may also adopt different interface connection manners in the foregoing embodiments, or a combination of multiple interface connection manners.
  • the wireless communication function of the electronic device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, the baseband processor, and the like.
  • the mobile communication module 150 may provide wireless communication solutions including 2G/3G/4G/5G etc. applied on the electronic device 100 .
  • the electronic device 100 implements a display function through a GPU, a display screen 194, an application processor, and the like.
  • the GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor.
  • the GPU is used to perform mathematical and geometric calculations for graphics rendering.
  • Processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.
  • Display screen 194 is used to display images, videos, and the like.
  • Display screen 194 includes a display panel.
  • the display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (active-matrix organic light).
  • LED diode AMOLED
  • flexible light-emitting diode flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diode (quantum dot light emitting diodes, QLED) and so on.
  • the electronic device 100 may include one or N display screens 194 , where N is a positive integer greater than one.
  • the electronic device 100 may implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, an application processor, and the like.
  • the ISP is used to process the data fed back by the camera 193 .
  • the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye.
  • ISP can also perform algorithm optimization on image noise, brightness, and skin tone.
  • ISP can also optimize the exposure, color temperature and other parameters of the shooting scene.
  • the ISP may be provided in the camera 193 .
  • Camera 193 is used to capture still images or video.
  • the object is projected through the lens to generate an optical image onto the photosensitive element.
  • the photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor.
  • CMOS complementary metal-oxide-semiconductor
  • the photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal.
  • the ISP outputs the digital image signal to the DSP for processing.
  • DSP converts digital image signals into standard RGB, YUV and other formats of image signals.
  • the electronic device 100 may include 1 or N cameras 193 , for example, a telephoto camera, a mid-focus camera, a short-focus (wide-angle) camera, or a wide-angle black and white camera, where N is a positive integer greater than 1.
  • the electronic device 100 may include a wide-angle camera, and an ultra-wide-angle camera or a telephoto camera.
  • the electronic device can generate a real-time preview image based on the image captured by the wide-angle camera.
  • the electronic device can also recommend a viewfinder frame corresponding to the image captured by the ultra-wide-angle camera or the telephoto camera for the user according to the real-time preview image, and perform multi-camera fusion imaging and cropping processing according to the viewfinder frame to improve the imaging composition quality. Improve the user's shooting experience.
  • the electronic device 100 may include a telephoto camera, and a mid-focus camera or a short-focus (wide-angle) camera.
  • the electronic device can recommend the viewing frame corresponding to the image captured by the mid-focus camera or the short-focus (wide-angle) camera for the user, and perform multi-camera fusion imaging and cropping processing according to the viewing frame to improve The quality of the imaging composition improves the user's shooting experience.
  • the telephoto camera, mid-focus camera or short-focus (wide-angle) camera configured on the electronic device 100 in the above application should be located on the same side of the electronic device 100, for example, located on the front of the screen of the electronic device 100 (the front camera ) or back (rear camera). Specific technical solutions will be described in detail below.
  • the NPU is a neural-network (NN) computing processor.
  • NN neural-network
  • Applications such as intelligent cognition of the electronic device 100 can be realized through the NPU, for example: image recognition, face recognition, speech recognition, text understanding, etc.
  • Internal memory 121 may be used to store computer executable program code, which includes instructions.
  • the processor 110 executes various functional applications and data processing of the electronic device 100 by executing the instructions stored in the internal memory 121 .
  • the internal memory 121 may include a storage program area and a storage data area.
  • the storage program area can store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), and the like.
  • the storage data area may store data (such as audio data, phone book, etc.) created during the use of the electronic device 100 and the like.
  • the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (UFS), and the like.
  • the above-mentioned internal memory 121 may store computer program codes for implementing the steps in the method embodiments of the present application.
  • the above-mentioned processor 110 may execute the computer program code of the steps in the method embodiments of the present application stored in the memory 121 .
  • the above-mentioned display screen 194 may be used to display the photographed object of the camera, the real-time video frames involved in the embodiments of the present application, and the like.
  • the software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture.
  • the embodiments of the present application take an Android system with a layered architecture as an example to exemplarily describe the software structure of the electronic device 100 .
  • FIG. 2 is a block diagram of the software structure of the electronic device 100 according to the embodiment of the present application.
  • the layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Layers communicate with each other through software interfaces.
  • the Android system is divided into four layers, which are, from top to bottom, an application layer, an application framework layer, an Android runtime (Android runtime) and a system library, and a kernel layer.
  • the application layer can include a series of application packages.
  • the application package can include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and so on.
  • the embodiments of the present application are mainly implemented by improving the camera application at the application layer, for example, by adding plug-ins to the camera to expand its functions.
  • the application framework layer provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer.
  • the application framework layer includes some predefined functions.
  • the application framework layer may include window managers, content providers, view systems, telephony managers, resource managers, notification managers, and the like.
  • the program of the camera at the application layer can be improved through the application framework layer, so that the special effect image or special effect video of the movement trajectory of the target object can be displayed on the display screen 194 when the shooting object is shooting.
  • the special effect image or special effect video is synthesized by the electronic device backend through real-time calculation and processing.
  • the window manager is used to manage window programs.
  • the window manager can get the size of the display screen, determine whether there is a status bar, lock the screen, take screenshots, etc.
  • Content providers are used to store and retrieve data and make these data accessible to applications.
  • the data may include video, images, audio, calls made and received, browsing history and bookmarks, phone book, etc.
  • the view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and so on. View systems can be used to build applications.
  • a display interface can consist of one or more views.
  • the display interface including the short message notification icon may include a view for displaying text and a view for displaying pictures.
  • the phone manager is used to provide the communication function of the electronic device 100 .
  • the management of call status including connecting, hanging up, etc.).
  • the resource manager provides various resources for the application, such as localization strings, icons, pictures, layout files, video files and so on.
  • the notification manager enables applications to display notification information in the status bar, which can be used to convey notification-type messages, and can disappear automatically after a brief pause without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc.
  • the notification manager can also display notifications in the status bar at the top of the system in the form of graphs or scroll bar text, such as notifications of applications running in the background, and notifications on the screen in the form of dialog windows. For example, text information is prompted in the status bar, a prompt sound is issued, the electronic device vibrates, and the indicator light flashes.
  • Android Runtime includes core libraries and a virtual machine. Android runtime is responsible for scheduling and management of the Android system.
  • the core library consists of two parts: one is the function functions that the java language needs to call, and the other is the core library of Android.
  • the application layer and the application framework layer run in virtual machines.
  • the virtual machine executes the java files of the application layer and the application framework layer as binary files.
  • the virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, safety and exception management, and garbage collection.
  • a system library can include multiple functional modules. For example: surface manager (surface manager), media library (Media Libraries), 3D graphics processing library (eg: OpenGL ES), 2D graphics engine (eg: SGL), etc.
  • surface manager surface manager
  • media library Media Libraries
  • 3D graphics processing library eg: OpenGL ES
  • 2D graphics engine eg: SGL
  • the Surface Manager is used to manage the display subsystem and provides a fusion of 2D and 3D layers for multiple applications.
  • the media library supports playback and recording of a variety of commonly used audio and video formats, as well as still image files.
  • the media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.
  • the 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.
  • 2D graphics engine is a drawing engine for 2D drawing.
  • the kernel layer is the layer between hardware and software, also known as the driver layer.
  • the kernel layer contains at least display drivers, camera drivers, audio drivers, and sensor drivers.
  • a corresponding hardware interrupt is sent to the kernel layer.
  • the kernel layer processes touch operations into raw input events (including touch coordinates, timestamps of touch operations, etc.). Raw input events are stored at the kernel layer.
  • the application framework layer obtains the original input event from the kernel layer, and identifies the control corresponding to the input event. Taking the touch operation as a touch click operation, and the control corresponding to the click operation is the control of the camera application icon, as an example, the camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer, and then starts the camera driver by calling the kernel layer.
  • the camera 193 captures still images or video.
  • the captured image or video may be temporarily stored in the content provider, and when a photo-taking operation is performed , the photographed photos or videos can be displayed through the view system.
  • the multi-frame images need to be fused and image cropped, and then displayed in the preview interface through the view system.
  • the user may instruct the mobile phone to open the camera application by touching a specific control on the screen of the mobile phone, pressing a specific physical key or key combination, inputting a voice command, or gesture in the air.
  • the mobile phone starts the camera and displays a shooting interface, as shown in FIG. 3 .
  • the user can instruct the mobile phone to open the camera application by clicking the "camera” application icon on the desktop of the mobile phone, and the mobile phone displays the shooting interface.
  • the mobile phone when the mobile phone is in a locked screen state, the user can also instruct the mobile phone to open the camera application by a gesture of swiping right on the screen of the mobile phone, or a gesture of swiping up, and the mobile phone displays a shooting interface.
  • the mobile phone when the mobile phone is in the lock screen state, the user can instruct the mobile phone to open the camera application by clicking the shortcut icon of the "camera" application on the lock screen interface, and the mobile phone can also display the shooting interface.
  • the mobile phone runs other applications the user can also make the mobile phone open the camera application to shoot by clicking the control of the camera function.
  • the shooting interface of the camera generally includes a real-time image preview frame, shooting controls and other functional controls, such as “large aperture”, “portrait”, “photographing” or “video recording”.
  • the real-time image preview frame can be used to preview the real-time image captured by the camera, and the user can determine the timing of instructing the mobile phone to perform the shooting operation based on the image in the real-time image preview frame, thereby generating the target image.
  • the user instructing the mobile phone to perform a shooting operation may be, for example, an operation of the user clicking a shooting control, or an operation of the user pressing a volume key, or the like.
  • the photographing interface may further include a zoom magnification indication.
  • the default zoom magnification of the mobile phone is the basic magnification, which is "1 ⁇ ".
  • the zoom ratio can be understood as the zoom/magnification ratio of the focal length of the current camera equivalent to the reference focal length.
  • the reference focal length is usually the focal length of the main camera of the mobile phone.
  • the mobile phone is configured with three cameras, a short-focus (wide-angle) camera, a medium-focus camera, and a telephoto camera as an example for description.
  • the focal length of the short-focus (wide-angle) camera is the smallest
  • the field of view (FoV) is the largest
  • the size of the object in the captured image is the smallest.
  • the focal length of the medium-focus camera is larger than that of the short-focus (wide-angle) camera
  • the field of view is smaller than that of the short-focus (wide-angle) camera
  • the size of the objects in the captured image is larger than that of the short-focus (wide-angle) camera.
  • a telephoto camera has the largest focal length, the smallest field of view, and the largest size of objects in the captured image. For different viewing areas, the mobile phone can intelligently select different camera combinations to ensure the image quality and composition effect.
  • the field of view refers to the angle formed by the lens of the optical instrument as the vertex, and the angle formed by the two edges of the maximum range that the object image of the measured target can pass through the lens in the optical instrument.
  • the field of view can be used to indicate the maximum angle range that the camera can capture when the phone is capturing images. That is to say, if the object to be photographed is within this angle range, the to-be-photographed object will be captured by the mobile phone, and if the to-be-photographed object is not within this angle range, the to-be-photographed object will not be captured by the mobile phone.
  • the larger the field of view of a camera the larger the range of shooting with that camera. The smaller the field of view of the camera, the smaller the shooting range of the camera.
  • the size of the field of view determines the field of view of the optical instrument.
  • the larger the field of view the larger the field of view and the smaller the optical magnification.
  • the cameras are classified according to the field of view, which can include: (1) Standard camera: a camera with a field of view of about 45 degrees, which is widely used and can also be called a medium-focus camera. Usually, users use the medium-focus camera in the most scenes. , so the mid-focus camera can be set as the main camera.
  • the focal length of the main camera is set as the reference focal length, and the zoom ratio is "1 ⁇ ".
  • Telephoto camera A camera with a field angle of less than 40 degrees can be used for long-distance shooting, and can be used as a telephoto lens in electronic equipment to complete long-distance shooting.
  • Wide-angle camera A camera with a field of view of more than 60 degrees captures a large range of images, but the close-up images may be distorted.
  • the main camera in the electronic device may be a wide-angle lens with a focal length within 35mm.
  • ultra-wide-angle lenses with a focal length of less than 24mm are also included.
  • the image captured by the main camera can be digitally zoomed (or referred to as digital zoom), that is, the "1x" image captured by the main camera is zoomed by the ISP or other processor in the phone.
  • the area of each pixel is enlarged, and the framing range of the image is correspondingly reduced, so that the processed image is equivalent to the image captured by the main camera with other zoom ratios (for example, "2 ⁇ ").
  • the multiple of the focal length of the telephoto camera and the focal length of the main camera can be used as the zoom ratio of the telephoto camera.
  • the focal length of the telephoto camera may be 5 times the focal length of the main camera, that is, the zoom magnification of the telephoto camera is “5 ⁇ ”.
  • the multiple of the focal length of the short-focus (wide-angle) camera and the focal length of the main camera can be used as the zoom ratio of the short-focus (wide-angle) camera.
  • the focal length of the short-focus camera may be 0.5 times or 0.6 times the focal length of the main camera, that is, the zoom magnification of the wide-angle camera is “0.5 ⁇ ” or “0.6 ⁇ ”.
  • the mobile phone can use any one of the above-mentioned cameras as the main camera, and the focal length of the main camera is used as the reference focal length. This application does not specifically limit this.
  • the user can perform manual zoom to adjust the zoom magnification used when the mobile phone is photographed.
  • the user can adjust the zoom ratio used by the mobile phone by operating the zoom ratio indicator in the shooting interface.
  • the zoom ratio indicator for example: when the current zoom magnification used by the mobile phone is "1 ⁇ ", the user can click the zoom magnification indicator one or more times to change the zoom magnification used by the mobile phone to "5 ⁇ ", then the real-time image preview frame of the mobile phone displays the zoom as 5 times the target shooting picture.
  • the user can also reduce the zoom magnification used by the mobile phone by pinch-in gesture with two fingers (or three fingers) in the shooting interface, for example, the zoom magnification is updated to "0.8 ⁇ ".
  • the gesture of two-finger (or three-finger) swiping outwards increases the zoom magnification used by the mobile phone, for example, the zoom magnification is updated to "4 ⁇ ".
  • the user can also perform manual zooming by dragging the zoom ruler in the shooting interface to change the zoom ratio used by the mobile phone.
  • a multi-camera intelligent framing recommendation solution is used, for different fields of view corresponding to the multi-camera, for example, an ultra-wide-angle camera and an ultra-wide-angle camera are used respectively.
  • Combined mirrors with wide-angle cameras, wide-angle cameras and telephoto lenses, etc. recommend multiple matching framing recommendation frames to users.
  • the electronic device can switch the multi-camera image fusion processing and the cropped framing image to the image preview interface in real time, thereby generating the final target image and presenting the user with a visible composition and framing.
  • the solution intelligently enriches the composition recommendations for users, thereby effectively improving the quality of photo shooting.
  • the method may include:
  • the electronic device acquires the image and displays it on the preview interface in real time.
  • the electronic device receives the user's instruction to open the camera application to take a photo, and displays the image captured by the camera on the image preview interface on the display screen in real time.
  • the electronic device receives the user's first operation.
  • the first operation may specifically be that the user clicks the "camera” icon on the desktop of the mobile phone, or, in the locked screen state of the mobile phone, the user slides the screen of the mobile phone Open the "Camera” application and other operations by shortcut.
  • the image collected by the camera includes the image collected by the aforementioned one camera, or the image obtained by the joint collection of multiple cameras.
  • the image collected by the camera may be the image collected by the ultra-wide-angle camera, or the image collected by the camera may be the image collected by the wide-angle camera and the telephoto camera jointly, that is, the image collected by the two cameras with different focal lengths.
  • the image after image fusion processing is not limited to the image collected by the aforementioned one camera, or the image obtained by the joint collection of multiple cameras.
  • the image collected by the camera may be the image collected by the ultra-wide-angle camera, or the image collected by the camera may be the image collected by the wide-angle camera and the telephoto camera jointly, that is, the image collected by the two cameras with different focal lengths.
  • the electronic device can automatically select a suitable camera according to the focal length of the target shooting object, or the user can manually switch the camera according to the framing requirements.
  • the specific lens selection strategy can refer to the relevant technical description. This application does not specifically limit this.
  • the user can click the “smart framing” control in the camera application interface to enable the intelligent framing recommended mode provided by the present application, otherwise, the electronic device performs conventional manual framing and shooting.
  • image fusion refers to a technology that uses a specific algorithm to comprehensively process multiple images into a new image.
  • image fusion refers to that the electronic device performs image fusion processing on images simultaneously collected by at least two cameras with different focal lengths, that is, large and small FoV images.
  • image fusion processing process and related algorithms reference may be made to the existing related technologies.
  • the following exemplarily introduces a possible image fusion process, which does not constitute a special limitation on the image fusion processing technology.
  • electronic devices can use image registration algorithms, such as feature point detection and matching through the Speed Up Robust Feature (Surf), to perform pixel-level registration of large FoV images and small FoV images.
  • image registration algorithms such as feature point detection and matching through the Speed Up Robust Feature (Surf)
  • the electronic device can correct the color and brightness of the large FoV image by adjusting the parameters of the gamma curve, the color correction matrix (CCM), and the color lookup table 3D LUT, taking the small FoV image as the benchmark. .
  • CCM color correction matrix
  • the electronic device may also use an image super-resolution algorithm to perform sharpening and detail enhancement processing on the large FoV image.
  • the electronic device uses an image fusion algorithm to process the images obtained in the foregoing processing process to obtain a fused image.
  • the image fusion algorithm may specifically be a Laplacian pyramid fusion algorithm or the like.
  • the results obtained by image fusion can utilize the spatial correlation of multiple images and the complementarity of image information, so that the images obtained after fusion processing have a more comprehensive and clear expression of the shooting scene.
  • the electronic device obtains at least one framing recommendation result according to the real-time captured image.
  • the electronic device obtains at least one framing recommendation result according to the image captured by the camera.
  • the framing recommendation result includes a framing recommendation frame, which is used to indicate the framing recommendation effect of image shooting provided by the electronic device to the user, so that the user can complete the photo shooting according to the composition recommendation scheme indicated by the framing recommendation frame.
  • one or more framing recommendation frames may be displayed on the real-time image preview interface of the electronic device, and each framing recommendation frame is used to indicate the framing recommendation effect of image shooting.
  • the framing recommendation frame is the trimming frame, which is used to indicate the effect of trimming based on the current preview image.
  • the framing recommendation frame may be displayed as a red frame, which is used to increase the user's attention to the framing recommendation.
  • the user may click to determine one of the framing recommendation frames, which is used to instruct the electronic device to perform image cropping according to the framing recommendation frame to obtain the target image.
  • the electronic device may display an optimal framing recommendation frame for the user, or the electronic device may simultaneously display multiple framing recommendation frames of different sizes for the user on the real-time image preview interface, which is used to instruct the user to perform Framing recommended selections and transitions.
  • the framing recommendation frame may be a horizontal framing recommendation frame, which is used to recommend users to shoot in a horizontal screen; or, the framing recommendation frame may also be a vertical framing recommendation frame, which is used to recommend users to shoot in a vertical screen, as shown in Figure 5 shown.
  • the framing recommendation result may further include a thumbnail image corresponding to the framing recommendation frame, and the thumbnail image is a thumbnail image of an image obtained by trimming an image captured by the camera according to the framing recommendation frame.
  • the electronic device may display thumbnails corresponding to multiple framing recommendation frames at the bottom of the real-time image preview interface, and each thumbnail is obtained by clipping a captured image according to a framing recommendation frame.
  • the thumbnail corresponding to the image may be displayed.
  • the electronic device may receive the user to determine one of the framing recommendation frames from at least one framing recommendation result, for example, a second operation of selecting the first framing recommendation frame.
  • the second operation is used to instruct the user to select one from multiple framing recommendation boxes recommended by the electronic device, which is used to perform framing processing on the image captured by the camera and update the real-time preview image.
  • the electronic device does not receive the user's second operation, for example, the user clicks the "X" icon displayed in the upper right corner to exit the framing recommendation mode, or the user moves the mobile phone, adjusts the focus, etc., the electronic device needs to re-register The images collected in real time are processed for framing recommendation.
  • the framing recommendation result further includes at least one of a horizontal reference line or a vertical reference line, and the horizontal reference line and the vertical reference line may be used to indicate that the image captured by the camera is rotated to obtain the target image.
  • the electronic device when the electronic device receives the user's confirmation shooting operation, for example, clicking the shooting button on the interface, or pressing the volume key and other quick shooting operations, the electronic device can perform processing according to the real-time image of the preview interface Then save it as the target image obtained by shooting.
  • the target image may be obtained by rotating and correcting the image collected by the camera through the horizontal reference line or the vertical reference line in the framing recommendation result. Therefore, through the image processing solution provided by the present application, the electronic device can automatically acquire the image that has been corrected horizontally or vertically, thereby improving the user's shooting experience.
  • the electronic device can perform calculations through the deep learning network, and extract the necessary image features for composition by detecting and identifying the target shooting scene, including semantic information, salient subjects or main lines, etc.
  • the collected images and extracted image features Input to the pre-trained composition evaluation network to get the framing recommendation frame, horizontal reference line or vertical reference line.
  • step 402 may specifically include:
  • Step 1 the electronic device performs scene detection according to the image collected by the camera to obtain composition structure information.
  • the electronic device can first detect the initial image to obtain composition structure information included in the initial image.
  • common composition methods in photography techniques may include nine-square composition, cross-shaped composition, triangular composition, rule of thirds composition, diagonal composition, and the like.
  • the composition structure information may include information such as structural lines, prominent subjects (photographing objects), and convergence points in the image, and the composition structure information may reflect reference position information for image composition.
  • the structural lines included in the user's shooting scene include sea level, building exterior walls, door frames, and the like.
  • the salient subjects included in the user's shooting scene include a portrait, a table, or a bunch of flowers.
  • the convergence point included in the user's shooting scene includes the part where the lines of the door frame and the table in the image intersect.
  • the user can adjust the shooting angle of the camera so that the lines in the imaging screen or the shooting object are located in the position of three equal parts of the image when taking pictures and framing the scene.
  • the image is visually balanced and beautiful, and the user's subjective experience is better.
  • the electronic device performs scene detection according to the image collected by the camera, and obtains the characteristics of the photographed object, that is, the target image.
  • the target image features may be identified features based on the target photographed object, such as portraits, buildings, trees, snow-capped mountains, soccer balls, ships, trains, and the like.
  • the target image features may also be identified features based on the target shooting scene, for example, the seaside, the sunrise, kicking a ball, splashing water, and the like.
  • the electronic device can match different composition recommendations according to a preconfigured algorithm.
  • the scene detection algorithm may be a scene detection model trained in advance through a large amount of image data and corresponding feature labels, for example, a neural network model.
  • the specific algorithm involved in the detection model may include a multi-category scene semantic segmentation algorithm, a semantic line detection algorithm, a saliency detection algorithm, or a portrait parsing algorithm.
  • the rich scene detection algorithm can identify more rich and comprehensive target image features to cover the usual user shooting scenes.
  • the electronic device may perform operations such as rotation and deformation on the image according to the structural lines to obtain a preprocessed image.
  • the electronic device can detect that the sea level, building structure line, door and window frame, etc. included in the image deviate from the horizontal or vertical line according to the identified structural lines, and the electronic device can rotate and deform the image.
  • the preprocessed image is obtained, so that the structural lines in the preprocessed image are straight or parallel to the horizontal/vertical lines, so that the problem of photographing oblique, crooked or deformed can be corrected.
  • Step 2 the electronic device can perform photographing and composition calculation on the image collected by the camera and the composition structure information, and obtain at least one framing recommendation result.
  • the algorithm for calculating the composition of the photograph may be calculated by an artificial intelligence (Artificial Intelligence, AI) algorithm, for example, a framing recommendation model obtained by training a large amount of image data in advance.
  • AI Artificial Intelligence
  • the electronic device can obtain an optimized framing composition recommendation by calculating the neural network model according to the input initial image (image captured by the camera).
  • the structural information detected by the electronic device can be used as a priori information of the framing recommendation model, that is, the framing recommendation model can match a relatively close or suitable framing recommendation template according to the structural information, so that the results output by the framing recommendation model are robust. more robust and interpretable.
  • the framing recommendation model may include a consistency loss function, which is used to supervise the output results of the framing recommendation model, so that the framing recommendation model can obtain better consistent framing recommendation results, that is, multiple shots are similar
  • the results of the framing recommendation model output are consistent or similar.
  • the framing recommendation model output results are consistent and do not shake.
  • the framework of the above-mentioned scene detection model can be used to conduct deep learning on a large amount of image data, combined with composition rules and framing templates constructed by professional photographers, and can also include subjective scores for the beauty of image composition, and framing clipping.
  • the annotation of frame coordinates and other data are used for in-depth learning to obtain a framing recommendation model. Therefore, the electronic device can obtain the framing recommendation results such as multiple framing clipping frames, horizontal reference lines or vertical reference lines with higher composition scores according to the input image and the characteristics of the target image through the shooting framing recommendation model.
  • common composition rules may include nine-square composition, cross composition, triangular composition, rule of thirds composition, diagonal composition, horizontal composition, complete portrait composition, visual balance composition, etc.
  • related technical content which is not specifically limited in this application.
  • the electronic device can obtain at least one framing recommendation result, so as to interact with the user, update the preview interface according to one of the framing recommendation results selected by the user, and perform final imaging.
  • the electronic device can detect visually significant horizontal lines, vertical lines, building central axes, etc. in the collected images through a deep learning network, so as to determine whether the user is not based on the actual horizontal reference line or vertical reference line. Take a picture and calculate the angle that needs to be rotated to correct the image.
  • the framing recommendation result may include a horizontal reference line, which is used to indicate that there is an angular deviation between the sea level shot by the user and the actual horizontal reference line, so that after receiving the user's shooting instruction, the electronic device can
  • the horizontal reference line automatically rotates the currently collected image to obtain the target image.
  • the user can adjust the shooting angle based on the horizontal reference line indicated by the electronic device, for example, by moving or rotating the electronic device, so that the sea level shot by the user is flush or substantially flush with the indicated horizontal reference line.
  • the user can click to select one of the recommended framing frames, and the camera preview interface of the electronic device is updated to the target image corresponding to the recommended framing frame.
  • the electronic device needs to enable the preview anti-shake function to ensure that the preview image seen by the user is displayed stably without obvious shaking.
  • the electronic device displays the target image in real time according to the framing recommendation result selected by the user.
  • the electronic device may process the image captured by the camera according to the framing recommendation result selected by the user to obtain the target image and update and display the real-time preview interface.
  • the target image is an image obtained by trimming the images collected by at least two cameras according to the framing recommendation result.
  • the target image may be an image obtained by performing cropping processing and rotation processing on an image captured by at least one camera according to the framing recommendation result.
  • the electronic device can receive the user's confirmation of the shooting operation, that is, the user can trigger the electronic device to perform a quick shooting by clicking the "shoot" button or icon, or touching the electronic volume key according to the target image displayed in real time on the preview interface. Shoot to save the target image to the electronic device.
  • the electronic device does not receive the user's confirmation of the shooting operation, for example, the user clicks the "X" icon displayed in the upper right corner to exit the framing recommendation mode, or the user moves the mobile phone, adjusts the focus, etc., the electronic device needs to re-register The images collected in real time are processed for framing recommendation.
  • the images collected by the cameras include images collected by at least two cameras with different focal lengths after image fusion processing is performed, and the target image is an image collected by at least two cameras with different focal lengths according to the framing recommendation result.
  • a certain range can be set for the sizes of the multiple recommended framing frames obtained in the above step 402, that is, the size of the recommended framing frames cannot be larger than the first threshold, and the recommended framing frames are not larger than the first threshold.
  • the size of the box cannot be smaller than the second threshold.
  • the upper limit of the size of the framing recommendation frame is the first threshold
  • the first threshold can be set according to the imaging range corresponding to the camera with the larger field of view in the joint imaging of multiple cameras
  • the lower limit of the size of the framing recommendation frame is the second threshold
  • the second threshold may be set according to the imaging range corresponding to the camera with the smaller field of view in the joint imaging of multiple cameras.
  • the first threshold is generally set to be smaller than the imaging range corresponding to a camera with a larger field of view (shorter focal length).
  • the zoom magnification of the ultra-wide-angle camera configured in the electronic device is "0.6 ⁇ "
  • the setting of the second threshold needs to consider the following two aspects:
  • the second threshold B ⁇ B0 where the imaging range corresponding to the telephoto camera with the zoom magnification of "10 ⁇ " is denoted as B0.
  • the following will exemplarily introduce several situations in which the electronic device obtains multiple framing recommendation frames based on different camera imaging in combination with different shooting scenarios.
  • Example 1 When the current electronic device detects that the target object is in the telephoto focal length, the electronic device can automatically switch to use the telephoto camera and the wide-angle camera for joint imaging, or use the telephoto camera and the ultra-wide-angle camera for joint imaging.
  • the telephoto camera corresponds to a smaller field of view
  • the focal length of the telephoto camera can be N times the focal length of the main camera, where N is greater than 1.
  • the zoom ratio of the telephoto camera can be "2 ⁇ " or "5" ⁇ ”.
  • the telephoto camera can be used to complete the shooting of distant scenes.
  • the wide-angle camera or ultra-wide-angle camera corresponds to a larger field of view.
  • the focal length of the wide-angle camera or ultra-wide-angle camera can be M times the focal length of the main camera, where M is less than or equal to 1.
  • the zoom of the wide-angle camera or the ultra-wide-angle camera The magnification can be "0.5 ⁇ " or "0.6 ⁇ ".
  • a wide-angle camera can be used to provide images covering a larger viewing range.
  • the first threshold can be set as the cropping frame of the imaging range corresponding to the ultra-wide-angle camera, or set as the imaging range corresponding to the ultra-wide-angle camera minus the distorted edge
  • the cropping frame of the area or set to be smaller than or equal to the cropping frame corresponding to the imaging range of the wide-angle camera.
  • the viewing cropping frame is set to the imaging range corresponding to the zoom magnification "0.6 ⁇ ".
  • unconstrained viewfinder cropping may result in lower image quality.
  • the viewfinder recommendation result recommends a viewfinder with a zoom size of “5 ⁇ ”, which will lead to Cropped image quality for framing is low.
  • the second threshold can be set to the cropping frame of the imaging range corresponding to the telephoto camera, or set to a cropping frame larger than the imaging range corresponding to the telephoto camera, for example, set to "2". ⁇ ” zoom-size framing cropping frame.
  • the corresponding telephoto focal length may include several different framing recommendation frames as shown in FIG. 8 .
  • the largest framing recommendation frame, that is, the first threshold value can be the imaging range corresponding to the wide-angle camera
  • the smallest framing recommendation frame, that is, the second threshold value can be the cropping frame corresponding to the imaging range of the telephoto camera, as shown in FIG. 8 .
  • Several recommended cropping frames for framing between the first threshold and the second threshold are recommended cropping frames for framing between the first threshold and the second threshold.
  • the electronic device may also recommend a framing recommendation frame of the portrait screen to the user according to the algorithm for framing recommendation.
  • the target image obtained by the electronic device in step 403 may be an image obtained after image fusion processing is performed based on the images obtained by the wide-angle camera and the telephoto camera respectively.
  • the combined imaging quality of the telephoto camera and the wide-angle camera is significantly improved compared to switching the telephoto camera only.
  • the target image obtained by the electronic device may also be an image obtained by performing image fusion processing based on images obtained from the wide-angle black and white camera, the wide-angle camera, and the telephoto camera respectively.
  • the wide-angle black and white camera can be used to enhance image details.
  • Example 2 When the current electronic device detects that the target object is in the wide-angle focal length, the electronic device can automatically switch to use the wide-angle camera and the ultra-wide-angle camera for joint imaging.
  • the wide-angle focal length may include several different framing recommendation frames as shown in FIG. 9 .
  • the largest recommended frame for framing, that is, the first threshold may be the imaging range corresponding to the wide-angle camera
  • the smallest recommended frame for framing, that is, the second threshold may be the imaging range corresponding to the telephoto camera.
  • FIG. 9 shows several recommended cropping frames for framing between the first threshold and the second threshold, which may also include recommended framing frames for portrait screens.
  • Example 3 The current electronic device detects that the target object is in the ultra-wide-angle focal length, and the electronic device can automatically switch to use the ultra-wide-angle camera for imaging.
  • the ultra-wide-angle focal length may include several different framing recommendation frames as shown in FIG. 10 .
  • the framing recommendation frame should try to avoid covering the area of the four corners of the image.
  • the largest framing recommendation frame is the first threshold.
  • the cropping frame corresponding to the edge area that may be distorted can be cropped for the imaging range corresponding to the ultra-wide-angle camera.
  • the smallest recommended frame for framing that is, the second threshold may be the imaging range corresponding to the telephoto camera.
  • FIG. 10 shows several recommended cropping frames for framing between the first threshold and the second threshold, which may also include recommended framing frames for portrait screens.
  • a framing recommendation result is obtained, so that the electronic device can perform cropping processing and rotation processing on images jointly collected by multiple cameras according to the framing recommendation result, so as to obtain a more optimized
  • the composition and framing angle of view can be improved to improve the image shooting quality of electronic equipment and enhance the user's shooting experience.
  • the above-mentioned electronic device obtains at least one framing recommendation result.
  • the user can click to select Any one of the framing recommendation frames is used to update the real-time preview image, or the user can also view thumbnails corresponding to more framing recommendation frames through a sliding gesture.
  • the user can also click the "X" icon displayed at the top of the page to exit the framing recommendation mode, or the user can also move the electronic device or manually zoom, so that the electronic device updates the current preview image and recalculates the Different framing recommendation results.
  • the electronic device can collect the user's selection of the framing recommendation result as the user's preference information, and feed it back to the aforementioned photographing composition calculation model, so that the electronic device can perform personalized model training according to the user's preference information.
  • the electronic device can output and sort the framing recommendation results in combination with the user's preference information.
  • the electronic device can also set an intelligent viewfinder model according to the user's personal preference. For example, in the camera settings, the user can be allowed to set a preferred mode for taking pictures, such as "wide-angle shooting” or “telephoto shooting”, “landscape shooting” or “Character Shooting” etc. Therefore, the electronic device can optimize and update the calculation of the intelligent viewfinder model according to the user's personalized preference settings.
  • the electronic device can also perform a secondary framing recommendation for the user, that is, after obtaining the framing recommendation result for the first time, the electronic device can also obtain at least one framing optimization result according to the target image.
  • the optimization result is used to instruct the target image to perform the framing optimization process again.
  • the electronic device can also perform a secondary framing recommendation. For example, as shown in FIG.
  • the electronic device can also perform secondary framing recommendation according to the target image, and the electronic device can also display a new framing recommendation frame through the image preview interface, instructing the user to perform image cropping and photographing according to the framing recommendation frame.
  • the electronic device can save the final cropped target image.
  • the electronic device can also save the image output by the algorithm of the intelligent framing model and the images corresponding to multiple different cameras to the album, so as to It is convenient for the user to go back and use the original image before trimming, or re-select the framing recommendation result to perform post-processing on the original image when processing the obtained image later.
  • the electronic device enriches the composition recommendation for taking pictures for the user through the interactive selection with the user, and improves the user's shooting experience; at the same time, by recording the user's selection result, the electronic device can further optimize the framing recommendation algorithm to achieve personalized Customized to provide users with more abundant and user-friendly framing recommendation solutions, and improve the user's shooting experience.
  • the present application further provides a photographing device for intelligent framing recommendation.
  • the device 1200 includes a framing recommendation module 1201 and an imaging module 1202 .
  • the framing recommendation module 1201 may be configured to obtain at least one framing recommendation result according to the image collected by the camera, where the framing recommendation result includes a framing recommendation frame, and the framing recommendation frame is used to indicate the framing recommendation effect of image shooting.
  • the imaging module 1202 may be configured to display a target image according to the framing recommendation result selected by the user, wherein the target image is an image obtained by clipping images collected by at least two cameras according to the framing recommendation result.
  • the framing recommendation result further includes at least one of a horizontal reference line or a vertical reference line, and the horizontal reference line and the vertical reference line are used to indicate that the image captured by the camera is rotated to obtain the target image.
  • the framing recommendation result further includes a thumbnail corresponding to the framing recommendation frame, and the thumbnail is a thumbnail obtained by trimming the image collected by the camera according to the framing recommendation frame.
  • the framing recommendation module 1201 is specifically configured to: perform scene detection according to the images collected by the camera to obtain composition structure information; perform photographic composition calculation on the images and composition structure information collected by the camera through the framing recommendation model to get at least one framing recommendation result.
  • the size of the framing recommendation frame is smaller than a first threshold and larger than or equal to a second threshold
  • the first threshold is set according to the imaging range corresponding to the camera with a larger field of view among the multiple cameras
  • the first threshold is The second threshold may be set according to the imaging range corresponding to the camera with the smaller field of view among the multiple cameras.
  • the apparatus 1200 may be used to perform the steps performed by the electronic device in the foregoing embodiments, for example, steps 401 to 403 in the foregoing embodiments.
  • the electronic device may be an image processing apparatus (electronic device) provided by an embodiment of the present application.
  • the electronic device 100 includes a processor 110 and an internal memory 121 .
  • the processor 110 and the internal memory 121 can communicate with each other through an internal connection path to transmit control and/or data signals, the internal memory 121 can be used to store computer programs, and the processor 110 can be used to retrieve data from the internal memory 121
  • the computer program is invoked and executed to execute the steps in the foregoing embodiments to realize image processing.
  • the electronic device 100 may correspond to various embodiments of the methods according to the embodiments of the present application.
  • each unit in the electronic device 100 and the other operations and/or functions mentioned above are respectively for realizing the corresponding flow in each embodiment of the method.
  • the above-mentioned processor 110 may be used to perform one or more processing actions implemented by the electronic device described in the foregoing method embodiments. For details, please refer to the descriptions in the foregoing method embodiments, which will not be repeated here.
  • processor in the embodiment of the present application may be a CPU, and the processor may also be other general-purpose processors, digital signal processing (digital signal processing, DSP), application specific integrated circuit (ASIC), field Field Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • DSP digital signal processing
  • ASIC application specific integrated circuit
  • FPGA field Field Programmable Gate Array
  • the memory in the embodiments of the present application may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory.
  • the non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically programmable Erase programmable read-only memory (electrically EPROM, EEPROM) or flash memory.
  • Volatile memory may be random access memory (RAM), which acts as an external cache.
  • RAM random access memory
  • SRAM static random access memory
  • DRAM dynamic random access memory
  • DRAM synchronous dynamic random access memory
  • SDRAM synchronous dynamic random access memory
  • DDR SDRAM double data rate synchronous dynamic random access memory
  • enhanced SDRAM enhanced synchronous dynamic random access memory
  • SLDRAM synchronous connection dynamic random access memory Fetch memory
  • direct memory bus random access memory direct rambus RAM, DR RAM
  • Embodiments of the present application further provide a computer-readable medium for storing computer program codes, where the computer program includes instructions for executing the network device in the above method and the method executed in the terminal device.
  • the readable medium may be a ROM or a RAM, which is not limited in this embodiment of the present application.
  • the present application also provides a computer program product, the computer program product includes instructions, when the instructions are executed, so that the terminal device and the network device respectively perform the operations corresponding to the above method of the terminal device and the network device.
  • the computer instructions are stored in a storage unit.
  • the storage unit is a storage unit configured by an electronic device, such as a register, a cache, etc.
  • the storage unit can also be a storage unit located outside the chip in the device, such as ROM or other types that can store static information and instructions. of static storage devices, RAM, etc.
  • the processor mentioned in any one of the above may be a CPU, a microprocessor, an ASIC, or one or more integrated circuits used to control the program execution of the above-mentioned method for transmitting feedback information.
  • the processing unit and the storage unit can be decoupled, respectively disposed on different physical devices, and connected in a wired or wireless manner to implement the respective functions of the processing unit and the storage unit, so as to support the system chip to implement the above embodiments various functions in .
  • the processing unit and the memory may also be coupled on the same device.
  • the processor in the embodiments of the present application may be a CPU, and the processor may also be other general-purpose processors, DSP, ASIC, FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. .
  • a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • the size of the sequence numbers of the above-mentioned processes does not mean the sequence of execution, and the execution sequence of each process should be determined by its functions and internal logic, and should not be dealt with in the embodiments of the present application. implementation constitutes any limitation.
  • the disclosed apparatus and method may be implemented in other manners.
  • the apparatus embodiments described above are only illustrative.
  • the division of the modules is only a logical function division. In actual implementation, there may be other division methods.
  • multiple modules or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented.
  • the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or modules, and may be in electrical, mechanical or other forms.
  • modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, may be located in one place, or may be distributed to multiple modules. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
  • each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist physically alone, or two or more modules may be integrated into one unit.
  • the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium.
  • the technical solution of the present application can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution.
  • the computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Human Computer Interaction (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Studio Devices (AREA)

Abstract

本申请提供一种智能取景推荐的拍照方法及装置，涉及图像处理技术领域，能够根据算法自动为用户匹配合适的取景推荐方案，提高成像质量。该方法包括：电子设备根据至少两个焦距不同的摄像头采集到的图像得到至少一个取景推荐结果，取景推荐结果包括取景推荐框，取景推荐框用于指示图像拍摄的取景推荐效果；根据用户选择的取景推荐结果，显示目标图像，其中，目标图像是根据取景推荐结果对至少两个摄像头采集到的图像进行剪裁得到的图像。

Description

一种智能取景推荐的拍照方法及装置
本申请要求于2020年12月31日提交国家知识产权局、申请号为202011615964.9、申请名称为“一种智能取景推荐的拍照方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理技术领域，尤其涉及一种智能取景推荐的拍照方法及装置。
背景技术
目前，越来越多的用户选择使用可移动电子设备拍摄照片，例如，用智能手机、平板电脑或者可穿戴设备等。因此，目前的移动电子设备通常会配置两个或两个以上的摄像镜头，例如，视场角45度的标准镜头，视场角40度以内的长焦镜头，和视场角60度以上的广角镜头等。电子设备可以通过不同的拍摄场景匹配切换不同的摄像镜头或者多摄像镜头进行联合成像，辅助用户拍出高质量的图像。
其中，摄像技术中的构图需要比较专业的技能，而普通用户专业摄影知识不足，较难拍出构图完美、符合美学规则的照片。现有的一种智能切换摄像头以辅助构图的技术方法中，通过识别用户当前拍摄图像中的前景类别，匹配一个合适的构图模板，进而根据构图模板切换广角镜头或长焦镜头进行成像，以获取符合当前构图建议的图像。
但是，上述技术方法由于场景类别及构图模板的局限，对于无前景的场景不具备好的构图推荐能力。另外，该技术方法是基于广角镜头和长焦镜头之间的切换，单一镜头生成的图像根据构图模板进行剪裁或者变焦后，其成像的画质较差。
发明内容
本申请提供一种智能取景推荐的拍照方法及装置，能够根据算法自动为用户匹配合适的取景推荐方案，提高成像质量。
为达到上述目的，本申请采用如下技术方案：
第一方面，提供一种智能取景推荐的拍照方法，应用于配置有至少两个焦距不同的摄像头的电子设备，该方法包括：根据摄像头采集到的图像得到至少一个取景推荐结果，取景推荐结果包括取景推荐框，取景推荐框用于指示图像拍摄的取景推荐效果；根据用户选择的取景推荐结果，显示目标图像，其中，目标图像是根据取景推荐结果对至少两个摄像头采集到的图像进行剪裁得到的图像。
上述技术方案中，通过对多个摄像头的联合成像进行深度学习，得到取景推荐结果，从而电子设备可以自动根据取景推荐结果对多个摄像头联合采集的图像进行剪裁或旋转处理，得到更加优化的构图和取景视角，提高电子设备的图像拍摄质量，提升用户的拍摄体验。
在一种可能的实现方式中，取景推荐结果还包括水平基准线或者垂直基准线中的至少一个，水平基准线和垂直基准线用于指示对摄像头采集到的图像进行旋转得到目 标图像。
上述可能的实现方式中，电子设备可以自动对图像进行旋转矫正处理，使得在用户拍摄特定场景或者检测到图像中包括特定线条时，不需要用户手动调整即可避免用户拍歪、拍斜的情况，出图质量较高，提升用户的拍摄体验。
在一种可能的实现方式中，取景推荐结果还包括取景推荐框对应的缩略图，缩略图为根据取景推荐框对摄像头采集到的图像进行剪裁得到的缩略图。
上述可能的实现方式中，电子设备通过显示取景推荐框对应的缩略图，可以使得用户直观地看到取景推荐的拍摄效果，提高用户使用取景推荐功能的概率，提升用户的交互体验。
在一种可能的实现方式中，根据摄像头采集到的图像得到至少一个取景推荐结果，具体包括：根据摄像头采集到的图像进行场景检测，得到构图结构信息；对摄像头采集到的图像和构图结构信息通过取景推荐模型进行拍照构图计算，得到至少一个取景推荐结果。
上述可能的实现方式中，电子设备可以通过取景推荐模型对摄像头采集到的图像进行取景构图的计算，利用神经网络较高的计算处理效率，得到较为匹配的取景推荐结果，从而提升图像处理计算的实时性和精确度，提升图像处理的效率。
在一种可能的实现方式中，取景推荐框的大小小于第一阈值，并且大于或者等于第二阈值，第一阈值是根据多个摄像头中视场角较大的摄像头对应的成像范围设置的，第二阈值可以根据多个摄像头中视场角较小的摄像头对应的成像范围设置的。
上述可能的实现方式中，电子设备通过多个不同摄像头的特性确定取景推荐框的大小范围，从而为用户推荐较为合理的取景推荐框，使得取景推荐结果不仅构图合理，而且拍摄视场大小适合，不会让用户产生突兀感，提升用户的拍摄体验。
第二方面，提供一种智能取景推荐的拍照方法，应用于配置有至少两个焦距不同的摄像头的电子设备，该方法包括：响应于用户的第一操作，根据摄像头采集到的图像得到至少一个取景推荐结果，取景推荐结果包括取景推荐框，取景推荐框用于指示图像拍摄的取景推荐效果；接收用户的第二操作，第二操作用于指示用户从至少一个取景推荐结果中确定第一取景推荐框；响应于第二操作，更新实时预览图像为目标图像，其中，目标图像是根据第一取景推荐框对至少两个摄像头采集到的图像进行剪裁得到的图像。
在一种可能的实现方式中，取景推荐结果还包括水平基准线或者垂直基准线中的至少一个，水平基准线和垂直基准线用于指示对摄像头采集到的图像进行旋转得到目标图像。
在一种可能的实现方式中，取景推荐结果还包括取景推荐框对应的缩略图，缩略图为根据取景推荐框对摄像头采集到的图像进行剪裁得到的缩略图。
在一种可能的实现方式中，根据摄像头采集到的图像得到至少一个取景推荐结果，具体包括：根据摄像头采集到的图像进行场景检测，得到构图结构信息；对摄像头采集到的图像和构图结构信息通过取景推荐模型进行拍照构图计算，得到至少一个取景推荐结果。
在一种可能的实现方式中，取景推荐框的大小小于第一阈值，并且大于或者等于 第二阈值，第一阈值是根据多个摄像头中视场角较大的摄像头对应的成像范围设置的，第二阈值可以根据多个摄像头中视场角较小的摄像头对应的成像范围设置的。
在一种可能的实现方式中，更新实时预览图像为目标图像之后，该方法还包括：响应于用户的确认拍摄操作，将目标图像以及至少两个不同焦距的摄像头采集到的图像分别保存。
在一种可能的实现方式中，该方法还包括：根据目标图像得到至少一个取景优化结果，取景优化结果包括取景推荐框，取景优化结果用于指示对目标图像再次进行取景优化处理。
第三方面，提供一种智能取景推荐的拍照装置，该装置包括：取景推荐模块，用于根据摄像头采集到的图像得到至少一个取景推荐结果，取景推荐结果包括取景推荐框，取景推荐框用于指示图像拍摄的取景推荐效果；成像模块，用于根据用户选择的取景推荐结果，显示目标图像，其中，目标图像是根据取景推荐结果对至少两个摄像头采集到的图像进行剪裁得到的图像。
在一种可能的实现方式中，取景推荐结果还包括水平基准线或者垂直基准线中的至少一个，水平基准线和垂直基准线用于指示对摄像头采集到的图像进行旋转得到目标图像。
在一种可能的实现方式中，取景推荐结果还包括取景推荐框对应的缩略图，缩略图为根据取景推荐框对摄像头采集到的图像进行剪裁得到的缩略图。
在一种可能的实现方式中，取景推荐模块具体用于：根据摄像头采集到的图像进行场景检测，得到构图结构信息；对摄像头采集到的图像和构图结构信息通过取景推荐模型进行拍照构图计算，得到至少一个取景推荐结果。
在一种可能的实现方式中，取景推荐框的大小小于第一阈值，并且大于或者等于第二阈值，第一阈值是根据多个摄像头中视场角较大的摄像头对应的成像范围设置的，第二阈值可以根据多个摄像头中视场角较小的摄像头对应的成像范围设置的。
第四方面，提供一种电子设备，该电子设备包括：处理器；用于存储所述处理器可执行指令的存储器；其中，所述处理器被配置为执行所述指令，以实现如上述第一方面或者第二方面中任一项所述的方法。
第五方面，提供一种计算机可读存储介质，当所述计算机可读存储介质中的指令由电子设备的处理器执行时，使得所述电子设备能够执行如上述第一方面或者第二方面中任一项所述的方法。
第六方面，提供一种计算机程序产品，当所述计算机程序产品在计算机上运行时，使得所述计算机执行如上述第一方面或者第二方面中任一项所述的方法。
以理解地，上述提供的任一种智能取景推荐的拍照装置、计算机可读存储介质或计算机程序产品，均可以由上文所提供的对应的方法来实现，因此，其所能达到的有益效果可参考上文所提供的对应的方法中有益效果，此处不再赘述。
附图说明
图1为本申请实施例提供的一种电子设备的硬件系统架构图；
图2为本申请实施例提供的一种电子设备的软件系统架构图；
图3为本申请实施例提供的一种智能取景推荐的拍照场景示意图；
图4为本申请实施例提供的一种智能取景推荐的拍照方法的流程示意图；
图5为本申请实施例提供的一种智能取景推荐的拍照方法的界面示意图；
图6为本申请实施例提供的另一种智能取景推荐的拍照方法的界面示意图；
图7为本申请实施例提供的一种智能取景推荐的拍照方法的流程及效果示意图；
图8-图10为本申请实施例提供的智能取景推荐的拍照方法的取景框示意图一至三；
图11为本申请实施例提供的另一种智能取景推荐的拍照方法的流程示意图；
图12为本申请实施例提供的另一种智能取景推荐的拍照装置的结构示意图。
具体实施方式
以下，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括一个或者更多个该特征。在本实施例的描述中，除非另有说明，“多个”的含义是两个或两个以上。
需要说明的是，本申请中，“示例性的”或者“例如”等词用于表示作例子、例证或说明。本申请中被描述为“示例性的”或者“例如”的任何实施例或设计方案不应被解释为比其他实施例或设计方案更优选或更具优势。确切而言，使用“示例性的”或者“例如”等词旨在以具体方式呈现相关概念。
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本申请实施例提供的图像拍摄方法可以应用于具备拍摄能力和图像处理能力的电子设备，该电子设备可以为手机、平板电脑、桌面型、膝上型、手持计算机、笔记本电脑、车载设备、超级移动个人计算机(ultra-mobile personal computer，UMPC)、上网本，以及蜂窝电话、个人数字助理(personal digital assistant，PDA)、增强现实(augmented reality，AR)\虚拟现实(virtual reality，VR)设备等，本公开实施例对该电子设备的具体形态不作特殊限制。
图1示出了电子设备100的结构示意图。
电子设备100可以包括处理器110，外部存储器接口120，内部存储器121，通用串行总线(universal serial bus，USB)接口130，充电管理模块140，电源管理模块141，电池142，天线1，天线2，移动通信模块150，无线通信模块160，音频模块170，扬声器170A，受话器170B，麦克风170C，耳机接口170D，传感器模块180，按键190，马达191，指示器192，摄像头193，显示屏194，以及用户标识模块(subscriber identification module，SIM)卡接口195等。其中传感器模块180可以包括压力传感器180A，陀螺仪传感器180B，气压传感器180C，磁传感器180D，加速度传感器180E，距离传感器180F，接近光传感器180G，指纹传感器180H，温度传感器180J，触摸传感器180K，环境光传感器180L，骨传导传感器180M等。
可以理解的是，本申请实施例示意的结构并不构成对电子设备100的具体限定。在本申请的另一些实施例中，电子设备100可以包括比图示更多或更少的部件，或者 组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或软件和硬件的组合实现。
处理器110可以包括一个或多个处理单元，例如：处理器110可以包括应用处理器(application processor，AP)，调制解调处理器，图形处理器(graphics processing unit，GPU)，图像信号处理器(image signal processor，ISP)，控制器，存储器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器，和/或神经网络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。
其中，控制器可以是电子设备100的神经中枢和指挥中心。控制器可以根据指令操作码和时序信号，产生操作控制信号，完成取指令和执行指令的控制。
处理器110中还可以设置存储器，用于存储指令和数据。在一些实施例中，处理器110中的存储器为高速缓冲存储器。该存储器可以保存处理器110刚用过或循环使用的指令或数据。如果处理器110需要再次使用该指令或数据，可从所述存储器中直接调用。避免了重复存取，减少了处理器110的等待时间，因而提高了系统的效率。
在一些实施例中，处理器110可以包括一个或多个接口。接口可以包括集成电路(inter-integrated circuit，I2C)接口，集成电路内置音频(inter-integrated circuit sound，I2S)接口，脉冲编码调制(pulse code modulation，PCM)接口，通用异步收发传输器(universal asynchronous receiver/transmitter，UART)接口，移动产业处理器接口(mobile industry processor interface，MIPI)，通用输入输出(general-purpose input/output，GPIO)接口，用户标识模块(subscriber identity module，SIM)接口，和/或通用串行总线(universal serial bus，USB)接口等。
MIPI接口可以被用于连接处理器110与显示屏194，摄像头193等外围器件。MIPI接口包括摄像头串行接口(camera serial interface，CSI)，显示屏串行接口(display serial interface，DSI)等。在一些实施例中，处理器110和摄像头193通过CSI接口通信，实现电子设备100的拍摄功能。处理器110和显示屏194通过DSI接口通信，实现电子设备100的显示功能。
GPIO接口可以通过软件配置。GPIO接口可以被配置为控制信号，也可被配置为数据信号。在一些实施例中，GPIO接口可以用于连接处理器110与摄像头193，显示屏194，无线通信模块160，音频模块170，传感器模块180等。GPIO接口还可以被配置为I2C接口，I2S接口，UART接口，MIPI接口等。
可以理解的是，本申请实施例示意的各模块间的接口连接关系，只是示意性说明，并不构成对电子设备100的结构限定。在本申请另一些实施例中，电子设备100也可以采用上述实施例中不同的接口连接方式，或多种接口连接方式的组合。
电子设备100的无线通信功能可以通过天线1，天线2，移动通信模块150，无线通信模块160，调制解调处理器以及基带处理器等实现。
移动通信模块150可以提供应用在电子设备100上的包括2G/3G/4G/5G等无线通信的解决方案。电子设备100通过GPU，显示屏194，以及应用处理器等实现显示功能。GPU为图像处理的微处理器，连接显示屏194和应用处理器。GPU用于执行数学和几何计算，用于图形渲染。处理器110可包括一个或多个GPU，其执行程序指令以 生成或改变显示信息。
显示屏194用于显示图像，视频等。显示屏194包括显示面板。显示面板可以采用液晶显示屏(liquid crystal display，LCD)，有机发光二极管(organic light-emitting diode，OLED)，有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode的，AMOLED)，柔性发光二极管(flex light-emitting diode，FLED)，Miniled，MicroLed，Micro-oLed，量子点发光二极管(quantum dot light emitting diodes，QLED)等。在一些实施例中，电子设备100可以包括1个或N个显示屏194，N为大于1的正整数。
电子设备100可以通过ISP，摄像头193，视频编解码器，GPU，显示屏194以及应用处理器等实现拍摄功能。
ISP用于处理摄像头193反馈的数据。例如，拍照时，打开快门，光线通过镜头被传递到摄像头感光元件上，光信号转换为电信号，摄像头感光元件将所述电信号传递给ISP处理，转化为肉眼可见的图像。ISP还可以对图像的噪点，亮度，肤色进行算法优化。ISP还可以对拍摄场景的曝光，色温等参数优化。在一些实施例中，ISP可以设置在摄像头193中。
摄像头193用于捕获静态图像或视频。物体通过镜头生成光学图像投射到感光元件。感光元件可以是电荷耦合器件(charge coupled device，CCD)或互补金属氧化物半导体(complementary metal-oxide-semiconductor，CMOS)光电晶体管。感光元件把光信号转换成电信号，之后将电信号传递给ISP转换成数字图像信号。ISP将数字图像信号输出到DSP加工处理。DSP将数字图像信号转换成标准的RGB，YUV等格式的图像信号。在一些实施例中，电子设备100可以包括1个或N个摄像头193，例如，长焦摄像头、中焦摄像头，短焦(广角)摄像头或者广角黑白摄像头等，N为大于1的正整数。
在本申请的一些实施例中，电子设备100可以包括广角摄像头，以及超广角摄像头或者长焦摄像头。电子设备可以根据广角摄像头捕获的图像，生成实时预览图像。进一步的，电子设备还可以根据实时预览图像，为用户推荐根据超广角摄像头或者长焦摄像头捕捉到的图像对应的取景框，根据取景框进行多摄像头融合成像以及剪裁处理，以提升成像构图质量，提升用户的拍摄体验。
另外，在本申请的另一些实施例中，电子设备100可以包括长焦摄像头，以及中焦摄像头或短焦(广角)摄像头。电子设备可以根据长焦摄像头捕捉的实时预览图像，为用户推荐根据中焦摄像头或短焦(广角)摄像头捕捉到的图像对应的取景框，根据取景框进行多摄像头融合成像以及剪裁处理，以提升成像构图质量，提升用户的拍摄体验。
需要注意的是，上述本申请中电子设备100上配置的长焦摄像头、中焦摄像头或短焦(广角)摄像头应位于电子设备100的同一侧，例如位于电子设备100屏幕的正面(前置摄像头)或背面(后置摄像头)。下文将详细描述具体的技术方案。
NPU为神经网络(neural-network，NN)计算处理器，通过借鉴生物神经网络结构，例如借鉴人脑神经元之间传递模式，对输入信息快速处理，还可以不断的自学习。通过NPU可以实现电子设备100的智能认知等应用，例如：图像识别，人脸识别，语 音识别，文本理解等。
内部存储器121可以用于存储计算机可执行程序代码，所述可执行程序代码包括指令。处理器110通过运行存储在内部存储器121的指令，从而执行电子设备100的各种功能应用以及数据处理。内部存储器121可以包括存储程序区和存储数据区。其中，存储程序区可存储操作系统，至少一个功能所需的应用程序(比如声音播放功能，图像播放功能等)等。存储数据区可存储电子设备100使用过程中所创建的数据(比如音频数据，电话本等)等。此外，内部存储器121可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件，闪存器件，通用闪存存储器(universal flash storage，UFS)等。
在本申请实施例中，上述内部存储器121中可以存储有用于实现本申请方法实施例中步骤的计算机程序代码。上述处理器110可以运行存储器121中存储的本申请方法实施例中步骤的计算机程序代码。上述显示屏194可以用于显示相机的拍摄对象，以及本申请实施例中涉及的实时视频帧等。
电子设备100的软件系统可以采用分层架构，事件驱动架构，微核架构，微服务架构，或云架构。本申请实施例以分层架构的Android系统为例，示例性说明电子设备100的软件结构。
图2是本申请实施例的电子设备100的软件结构框图。
分层架构将软件分成若干个层，每一层都有清晰的角色和分工。层与层之间通过软件接口通信。在一些实施例中，将Android系统分为四层，从上至下分别为应用程序层，应用程序框架层，安卓运行时(Android runtime)和系统库，以及内核层。
应用程序层可以包括一系列应用程序包。如图2所示，应用程序包可以包括相机，图库，日历，通话，地图，导航，WLAN，蓝牙，音乐，视频，短信息等应用程序。本申请实施例主要就是通过改进应用程序层的相机应用程序来实现的，例如通过对相机增加插件来扩展其功能。
应用程序框架层为应用程序层的应用程序提供应用编程接口(application programming interface，API)和编程框架。应用程序框架层包括一些预先定义的函数。
如图2所示，应用程序框架层可以包括窗口管理器，内容提供器，视图系统，电话管理器，资源管理器，通知管理器等。在本申请实施例中，可以通过应用程序框架层对应用程序层的相机的程序进行改进，使得拍摄对象在拍摄时，可以在显示屏194中显示目标物体运动轨迹的特效图像或者特效视频，该特效图像或者特效视频是由电子设备后台通过实时的计算和处理合成的。
其中，窗口管理器用于管理窗口程序。窗口管理器可以获取显示屏大小，判断是否有状态栏，锁定屏幕，截取屏幕等。
内容提供器用来存放和获取数据，并使这些数据可以被应用程序访问。所述数据可以包括视频，图像，音频，拨打和接听的电话，浏览历史和书签，电话簿等。
视图系统包括可视控件，例如显示文字的控件，显示图片的控件等。视图系统可用于构建应用程序。显示界面可以由一个或多个视图组成的。例如，包括短信通知图标的显示界面，可以包括显示文字的视图以及显示图片的视图。
电话管理器用于提供电子设备100的通信功能。例如通话状态的管理(包括接通， 挂断等)。
资源管理器为应用程序提供各种资源，比如本地化字符串，图标，图片，布局文件，视频文件等等。
通知管理器使应用程序可以在状态栏中显示通知信息，可以用于传达告知类型的消息，可以短暂停留后自动消失，无需用户交互。比如通知管理器被用于告知下载完成，消息提醒等。通知管理器还可以是以图表或者滚动条文本形式出现在系统顶部状态栏的通知，例如后台运行的应用程序的通知，还可以是以对话窗口形式出现在屏幕上的通知。例如在状态栏提示文本信息，发出提示音，电子设备振动，指示灯闪烁等。
Android Runtime包括核心库和虚拟机。Android runtime负责安卓系统的调度和管理。
核心库包含两部分：一部分是java语言需要调用的功能函数，另一部分是安卓的核心库。
应用程序层和应用程序框架层运行在虚拟机中。虚拟机将应用程序层和应用程序框架层的java文件执行为二进制文件。虚拟机用于执行对象生命周期的管理，堆栈管理，线程管理，安全和异常的管理，以及垃圾回收等功能。
系统库可以包括多个功能模块。例如：表面管理器(surface manager)，媒体库(Media Libraries)，三维图形处理库(例如：OpenGL ES)，2D图形引擎(例如：SGL)等。
表面管理器用于对显示子系统进行管理，并且为多个应用程序提供了2D和3D图层的融合。
媒体库支持多种常用的音频，视频格式回放和录制，以及静态图像文件等。媒体库可以支持多种音视频编码格式，例如：MPEG4，H.264，MP3，AAC，AMR，JPG，PNG等。
三维图形处理库用于实现三维图形绘图，图像渲染，合成，和图层处理等。
2D图形引擎是2D绘图的绘图引擎。
内核层是硬件和软件之间的层，也可以称为驱动层。内核层至少包含显示驱动，摄像头驱动，音频驱动，传感器驱动。
下面结合捕获拍照场景，示例性说明电子设备100软件以及硬件的工作流程。
当触摸传感器180K接收到触摸操作，相应的硬件中断被发给内核层。内核层将触摸操作加工成原始输入事件(包括触摸坐标，触摸操作的时间戳等信息)。原始输入事件被存储在内核层。应用程序框架层从内核层获取原始输入事件，识别该输入事件所对应的控件。以该触摸操作是触摸单击操作，该单击操作所对应的控件为相机应用图标的控件为例，相机应用调用应用框架层的接口，启动相机应用，进而通过调用内核层启动摄像头驱动，通过摄像头193捕获静态图像或视频。
在本申请实施例中，用户使用电子设备拍摄照片的过程中，当通过摄像头193捕获到静态图像或者视频时，可以将捕获到的图像或者视频暂时存储于内容提供器中，当执行拍照操作时，可通过视图系统显示拍摄完成的照片或者视频，对于本申请的实施例，在显示图像之前，还需要经过将多帧图像进行融合处理以及图像剪裁处理之后再通过视图系统显示在预览界面中。
以下实施方式中所涉及的技术方案均可以在具有上述硬件架构和软件架构的电子设备100中实现。本申请的下述实施方式将以电子设备100是手机作为示例，结合附图对本申请实施例提供的技术方案进行详细说明。
示例性的，用户可以通过触摸手机屏幕上特定的控件、按压特定的物理按键或按键组合、输入语音指令、隔空手势等方式，指示手机开启相机应用。响应于用户开启相机应用的指示后，手机启动相机，显示拍摄界面，如图3所示。
示例性的，用户可以通过在手机桌面上点击“相机”应用图标，指示手机开启相机应用，手机显示拍摄界面。或者，当手机处于锁屏状态时，用户也可以通过在手机屏幕上向右滑动的手势，或者向上滑动的手势指示手机开启相机应用，手机显示拍摄界面。再例如，当手机处于锁屏状态时，用户可以通过在锁屏界面上点击“相机”应用的快捷图标，指示手机开启相机应用，手机也可以显示拍摄界面。又例如，当手机运行其他应用时，用户也可以通过点击相机功能的控件使得手机开启相机应用进行拍摄。
如图3所示，相机的拍摄界面一般包括有实时图像预览框、拍摄控件以及其他功能控件，例如，可以包括“大光圈”、“人像”、“拍照”或“录像”等。其中，实时图像预览框可用于预览摄像头采集到的实时图像，用户可以基于实时图像预览框中的图像，决定指示手机执行拍摄操作的时机，从而生成目标图像。其中，用户指示手机执行拍摄操作例如可以是用户点击拍摄控件的操作，或者用户按压音量键的操作等。在一些实施例中，拍摄界面中还可以包括变焦倍率指示。通常，手机默认的变焦倍率为基本倍率，为“1×”。
其中，变焦倍率可理解为当前摄像头的焦距相当于基准焦距的缩/放倍数。其中，基准焦距通常为手机主摄像头的焦距。
举例说明，以手机配置有短焦(广角)摄像头、中焦摄像头和长焦摄像头三个摄像头为例进行说明。其中，在手机和被拍摄物体的相对位置不变的情况下，短焦(广角)摄像头的焦距最小，视场角(Field of View，FoV)最大，拍摄的图像中的物体尺寸最小。中焦摄像头的焦距大于短焦(广角)摄像头，视场角比短焦(广角)摄像头的小，拍摄的图像中物体尺寸比短焦(广角)摄像头是的较大。长焦摄像头的焦距最大，视场角最小，拍摄的图像中物体的尺寸最大。针对不同的取景区域，手机可以智能选用不同的摄像头组合，以保证成像质量和构图效果。
其中，视场角是指，在光学仪器中，以光学仪器的镜头为顶点，以被测目标的物像可通过镜头的最大范围的两条边缘构成的夹角。视场角可以用于指示手机在拍摄图像的过程中，摄像头所能拍摄到的最大的角度范围。也就是说，若待拍摄对象处于这个角度范围内，该待拍摄对象便会被手机采集到，若待拍摄对象没有处于这个角度范围内，该待拍摄对象便不会被手机采集到。通常，摄像头的视场角越大，则使用该摄像头的拍摄范围就越大。而摄像头的视场角越小，则使用该摄像头的拍摄范围就越小。
也就是说，视场角的大小决定了光学仪器的视野范围，视场角越大，视野就越大，光学倍率就越小。按视场角对摄像头进行分类，可以包括：(1)标准摄像头：视场角45度左右的摄像头，使用范围较广，也可称为中焦摄像头，通常，用户使用中焦摄像头的场景最多，因此，可以将中焦摄像头设置为主摄像头。主摄像头的焦距设置为基准焦距，变焦倍率为“1×”。(2)远摄摄像头：视场角40度以内的摄像头，可在远距离 情况下拍摄，可以作为电子设备中的长焦镜头，用于完成远距离景物的拍摄。(3)广角摄像头：视场角60度以上的摄像头，捕捉图像的范围较大，但近处图像可能存在变形。电子设备中的主摄像头可以是焦距为35mm以内的广角镜头。另外，还包括焦距为24mm以内的超广角镜头等。
在一些实施例中，可以对主摄像头捕获的图像进行数码变焦(digital zoom，或称为数字变焦)，即，通过手机中的ISP或其他处理器将主摄像头捕获的“1×”的图像的每个像素面积放大，并相应缩小图像的取景范围，使得处理后的图像呈现等效于主摄像头采用其他变焦倍率(例如“2×”)拍摄的图像。
类似的，长焦摄像头的焦距与主摄像头的焦距的倍数，可作为长焦摄像头的变焦倍率。例如，长焦摄像头的焦距可以为主摄像头的焦距的5倍，即长焦摄像头的变焦倍率为“5×”。
类似的，短焦(广角)摄像头的焦距与主摄像头的焦距的倍数，可作为短焦(广角)摄像头的变焦倍率。例如，短焦摄像头的焦距可以为主摄像头的焦距的0.5倍或者0.6倍，即广角摄像头的变焦倍率为“0.5×”或“0.6×”。
需要说明的是，手机可以使用上述的其中任一个摄像头作为主摄像头，并以主摄像头的焦距为基准焦距，其他辅助摄像头的焦距与主摄像头的焦距的倍数，可作为辅助摄像头的变焦倍率。本申请对此不做具体限定。
在一些实施例中，用户可以进行手动变焦以调整手机拍摄时使用的变焦倍率。例如，如图3中所示，用户可以通过在拍摄界面中操作变焦倍率指示，调整手机使用的变焦倍率。比如：当前手机使用的变焦倍率为“1×”时，用户可以通过点击一次或多次变焦倍率指示，使得手机使用的变焦倍率变更为“5×”，则手机的实时图像预览框显示变焦为5倍的目标拍摄画面。或者，用户还可以通过在拍摄界面中两指(或三指)捏合的手势，减小手机使用的变焦倍率，例如，变焦倍率更新为“0.8×”。或者两指(或三指)向外滑动的手势(与捏合相反的方向)增大手机使用的变焦倍率，例如，变焦倍率更新为“4×”。或者，用户还可以在拍摄界面中通过拖动变焦标尺来改变手机使用的变焦倍率等方式进行手动变焦。
在本申请的实施例中，用户使用电子设备拍摄照片的过程中，通过一种多摄像头的智能取景推荐方案，针对多摄像头所对应的不同视场角，例如分别通过超广角摄像头、超广角摄像头和广角摄像头联合用镜、广角摄像头和长焦镜头联合用镜等，向用户推荐多个匹配的取景推荐框。并且，电子设备通过与用户交互进行取景选择，可以将多摄像头图像融合处理以及剪裁处理后的取景图像实时切换至图像预览界面，进而生成最终的目标图像，给用户呈现可见即可得的构图取景方案，智能化地为用户丰富构图推荐，从而有效提升照片拍摄质量。
在本申请实施例涉及到的上述硬件和软件的基础上，结合上述的拍摄场景，下面将结合相应附图，对本申请的实施例进行详细介绍。如图4所示，该方法可以包括：
401：电子设备获取图像并实时显示在预览界面。
电子设备接收用户打开相机应用进行拍照的指示，根据摄像头采集到的图像在显示屏上的图像预览界面实时进行显示。
结合前述介绍的电子设备拍照的场景，电子设备接收用户的第一操作，例如，第 一操作具体可以为用户点击手机桌面的“相机”图标，或者，在手机锁屏状态下，用户滑动手机屏幕通过快捷当时打开“相机”应用等操作。
其中，摄像头采集到的图像包括前述的一个摄像头采集的，或者多个摄像头联合采集获取的图像。例如，摄像头采集到的图像可以为超广角摄像头采集到的图像，或者，摄像头采集到的图像可以为广角摄像头与长焦摄像头联合采集的图像，即为两个焦距不同的摄像头采集到的图像进行图像融合处理之后的图像。
需要说明的是，具体的拍摄场景中，电子设备可以根据目标拍摄物体所处的焦段，自动选择合适的摄像头，或者由用户根据取景需求手动切换摄像头，具体的镜头选择策略可以参照相关技术说明，本申请对此不作具体限定。
其中，如图3所示，用户可以通过点击相机应用界面中的“智能取景”控件开启本申请提供的智能取景推荐模式，否则，电子设备进行常规的手动取景和拍摄。
其中，图像融合是指用特定的算法将多幅图像综合处理成一幅新图像的技术。
在本申请的实施例中，图像融合是指电子设备对至少两个焦距不同的摄像头同时采集到的图像即大小FoV图像，进行处理图像融合处理。具体的图像融合处理的过程以及相关算法可以参照现有的相关技术，下面将示例性介绍一种可能的图像融合过程，其并不构成对图像融合处理技术的特殊限制。
首先，电子设备可以采用图像配准算法，如通过快速鲁棒特性(Speed Up Robust Feature，Surf)的特征点检测与匹配，对大FoV图像和小FoV图像进行像素级配准。
然后，电子设备可以通过调整伽马Gamma曲线、颜色校正矩阵(Color Correction Matrix，CCM)、颜色查找表3D LUT的参数，以小FoV的图像为基准，对大FoV图像进行颜色和亮度的校正处理。
可选的，电子设备还可以采用图像超分算法，对大FoV图像进行锐化和细节增强处理。
最后，电子设备采用图像融合算法对前述处理过程得到的图像进行处理得到融合后的图像。例如，图像融合算法具体可以为拉普拉斯金字塔融合算法等。
图像融合得到的结果能利用多幅图像在空间的相关性及图像信息的互补性，使得融合处理后得到的图像对拍摄场景有更全面、清晰的表达。
402：电子设备根据实时采集的图像得到至少一个取景推荐结果。
响应于用户的第一操作，电子设备根据摄像头采集到的图像得到至少一个取景推荐结果。
其中，取景推荐结果包括取景推荐框，用于指示电子设备向用户提供的图像拍摄的取景推荐效果，从而用户可以根据取景推荐框指示的构图推荐方案，完成照片拍摄。
如图5所示，电子设备的实时图像预览界面上可以显示一个或者多个取景推荐框，每个取景推荐框用于指示图像拍摄的取景推荐效果。
也就是说，取景推荐框即为剪裁框，用于指示基于当前预览图像进行剪裁的效果。
在一种实施方式中，为便于用户直观地获取取景推荐效果，取景推荐框可以显示为红色方框，用于提高用户对取景推荐的关注度。用户可以点击确定其中一个取景推荐框，用来指示电子设备根据该取景推荐框进行图像剪裁，得到目标图像。
在一种实施方式中，电子设备可以为用户显示一个最优的取景推荐框，或者，电 子设备可以在实时图像预览界面上为用户同时显示多个大小不同的取景推荐框，用于指示用户进行取景推荐的选择和切换。
其中，取景推荐框可以为横版的取景推荐框，用于推荐用户进行横屏拍摄；或者，取景推荐框也可以为竖版的取景推荐框，用于推荐用户进行竖屏拍摄，如图5所示。
在一种实施方式中，取景推荐结果还可以包括取景推荐框对应的缩略图，缩略图为根据取景推荐框对摄像头采集到的图像进行剪裁处理得到图像的缩略图。
示例性的，如图6所示，电子设备可以在实时图像预览界面的下方显示多个取景推荐框对应的缩略图，每一个缩略图为根据一个取景推荐框对采集到的图像进行剪裁得到的图像对应的缩略图。
此时，电子设备可以接收用户从至少一个取景推荐结果中确定其中一个取景推荐框，例如选择第一取景推荐框的第二操作。其中，第二操作用于指示用户从电子设备推荐的多个取景推荐框中选定一个，用于对摄像头采集到的图像进行取景处理并更新实时预览图像。
另外，如果电子设备没有接收到用户的第二操作，例如，用户点击右上角显示的“×”图标，用于退出取景推荐模式，或者用户移动手机、调整焦距等操作，则电子设备需要重新对实时采集的图像进行取景推荐处理。
在一种实施方式中，取景推荐结果还包括水平基准线或者垂直基准线中的至少一个，水平基准线和垂直基准线可以用于指示对摄像头采集到的图像进行旋转得到目标图像。
具体的，在这种实施方式下，当电子设备接收到用户的确认拍摄操作，例如，点击界面的拍摄按钮，或者按压音量键等快捷拍摄操作，则电子设备可以根据预览界面的实时图像进行处理后保存为拍摄所得的目标图像。其中，该目标图像可以是经过取景推荐结果中的水平基准线或者垂直基准线对摄像头采集到的图像进行旋转矫正得到的。从而通过本申请提供的图像处理方案，电子设备可以自动获取景过水平或者垂直矫正处理的图像，提升用户的拍摄体验。
其中，电子设备可以通过深度学习网络进行计算，通过对目标拍摄场景进行检测和识别，提取构图必要的图像特征，包括语义信息、显著性主体或者主要线条等，将采集的图像和提取的图像特征输入至预先训练好的构图评价网络，得到取景推荐框、水平基准线或者垂直基准线。
则步骤402具体可以包括：
Step1，电子设备根据摄像头采集到的图像进行场景检测得到构图结构信息。
考虑到摄影技术中的常见构图法，电子设备可以先对初始图像进行检测，得到初始图像包括的构图结构信息。例如，摄影技术中的常见构图法可以包括九宫格构图、十字形构图、三角形构图、三分法构图、对角线构图等。
其中，构图结构信息可以包括图像中的结构线条、显著性主体(拍摄对象)、汇聚点等信息，构图结构信息可以体现图像构图的参考位置信息。
示例性的，用户拍摄场景中包括的结构线条有海平面、建筑外墙，门框等。用户拍摄场景中包括的显著性主体包括人像、桌子或一束花等。用户拍摄场景中包括的汇聚点包括图像中的门框和桌子的线条存在交汇的部分。
示例性的，以摄影技术中的三分法构图为例，即是指用户在拍照取景的时候，可以通过调整摄像头的拍摄角度使得成像画面中的线条或拍摄对象位于图像三等分的位置，从而图像给人在视觉上较为平衡和美观，用户的主观体验较好。
在一种实施方式中，电子设备根据摄像头采集到的图像进行场景检测，得到拍摄对象，即目标图像特征。
其中，目标图像特征可以为识别到的基于目标拍摄对象的特征，例如人像、建筑物、树木、雪山、足球、船只、火车等。或者，目标图像特征还可以为识别到的基于目标拍摄场景的特征，例如，海边、日出、踢球、泼水等。根据不同的目标图像特征，电子设备可以根据预先配置的算法匹配不同的构图推荐。
场景检测算法可以为预先通过大量图像数据以及对应的特征标签训练好的场景检测模型，例如，神经网络模型。该检测模型涉及的具体算法可以包含多类别场景语义分割算法、语义线检测算法、显著性检测算法或者人像解析算法等。丰富的场景检测算法能识别出更加丰富全面的目标图像特征，以涵盖通常的用户拍摄场景。
可选的，电子设备可以根据结构线条，对图像进行旋转、变形等操作，得到预处理图像。
其中，电子设备可以根据识别到的结构线条，检测到图像中包括的海平线、建筑物结构线、门窗框等与水平线或者垂直线存在偏差，则电子设备可以对图像进行旋转、变形等操作，得到预处理图像，使得预处理图像中的结构线条平直或者与水平/垂直线平行，从而能够纠正照片拍斜、拍歪或变形的问题。
Step2，电子设备可以对摄像头采集到的图像和构图结构信息进行拍照构图计算，得到至少一个取景推荐结果。
其中，拍照构图计算的算法可以通过人工智能(Artificial Intelligence，AI)算法，例如，预先通过对大量图像数据进行训练得到的取景推荐模型，用于进行拍照构图的计算。例如，电子设备可以根据输入的初始图像(摄像头采集的图像)通过神经网络模型计算得到优化的取景构图推荐。
另外，电子设备检测得到的结构性信息可以作为取景推荐模型的先验信息，即取景推荐模型可以根据结构性信息匹配构图较为接近的或者合适的取景推荐模板，使得取景推荐模型输出的结果鲁棒性更高且具有可解释性。
在一种实施方式中，取景推荐模型中可以包括一致性损失函数，用于对取景推荐模型的输出结果进行监督，从而取景推荐模型可以得到一致性更好的取景推荐结果，即多次拍摄相似的场景，其取景推荐模型输出的结果是一致的或者相似的，另外，连续拍摄同一个场景或者画面，取景推荐模型输出的结果是一致的、不抖动。
其中，一致性损失函数的构造可以参考现有的相关技术，本申请不作具体限定。
具体的，可以采用上述的场景检测模型的框架，通过对大量的图像数据进行深度学习，结合由专业摄影师构建的构图规则和取景模板，还可以包括对图像构图美感的主观评分，以及取景剪裁框坐标的批注等数据进行深度学习，得到取景推荐模型。从而电子设备可以通过该拍取景推荐模型，根据输入的图像和目标图像特征，得到构图评分较高的多个取景剪裁框、水平基准线或垂直基准线等取景推荐结果。
其中，常见的构图规则可以包括九宫格构图、十字形构图、三角形构图、三分法 构图、对角线构图、水平线构图、人像完整构图、视觉平衡构图等。具体可以参考相关的技术内容，本申请对此不做具体限定。
通过上述402执行的过程，电子设备可以得到至少一个取景推荐结果，从而与用户进行交互，根据用户选择的其中一个取景推荐结果进行预览界面的更新，以及最终成像。
在一种实施方式中，电子设备可以通过深度学习网络检测采集到的图像中视觉上显著的水平线条、竖直线条、建筑中轴线等，从而根据实际的水平基准线或者垂直基准线判断用户是否拍歪，并计算得到需要对图像进行旋转矫正的角度。
示例性的，如图7所示，取景推荐结果可以包括水平基准线，用于指示用户拍摄的海平线与实际的水平基准线存在角度偏差，从而电子设备接收到用户的拍摄指示后，可以根据该水平基准线自动对当前采集到的图像进行旋转处理，得到目标图像。或者，用户可以基于电子设备指示的水平基准线调整拍摄角度，例如，通过移动或者旋转电子设备，使得用户拍摄的海平线与指示的水平基准线平齐或者基本平齐。
在不移动摄像头且没有进行手动变焦的情况下，用户可以点击选择其中一个取景推荐框，则电子设备的相机预览界面更新为该取景推荐框对应的目标图像。
另外，电子设备的预览界面显示取景剪裁结果之后，电子设备需要开启预览防抖功能，保证用户看到的预览图像显示稳定，没有明显的抖动现象。
403：电子设备根据用户选择的取景推荐结果，实时显示目标图像。
响应于用户的第二操作，电子设备可以根据用户选择的取景推荐结果对摄像头采集到的图像进行处理，得到目标图像并进行实时预览界面的更新显示。
其中，目标图像是根据取景推荐结果对至少两个摄像头采集到的图像进行剪裁处理得到的图像。或者，目标图像可以是根据取景推荐结果对至少一个摄像头采集到的图像进行剪裁处理以及旋转处理得到的图像。
此时，电子设备可以接收到用户的确认拍摄操作，即用户可以根据预览界面实时显示的目标图像，通过点击“拍摄”按钮或者图标，或者触摸电子的音量键等快捷拍摄方式，触发电子设备进行拍摄，从而将目标图像保存至电子设备中。
另外，如果电子设备没有接收到用户的确认拍摄操作，例如，用户点击右上角显示的“×”图标，用于退出取景推荐模式，或者用户移动手机、调整焦距等操作，则电子设备需要重新对实时采集的图像进行取景推荐处理。
在一种实施方式中，摄像头采集到的图像包括至少两个焦距不同的摄像头采集的图像进行图像融合处理之后的图像，则目标图像是根据取景推荐结果对至少两个焦距不同的摄像头采集的图像进行图像融合处理之后进行剪裁处理得到的图像。
在一种实施方式中，为保证最终的成像质量，可以对上述的步骤402得到的多个取景推荐框的大小设置一定的范围，即取景推荐框的大小不可以大于第一阈值，并且取景推荐框的大小不可以小于第二阈值。其中，取景推荐框的大小的上限为第一阈值，第一阈值可以根据多个摄像头联合成像中视场角较大的摄像头对应的成像范围进行设置，取景推荐框的大小的下限为第二阈值，第二阈值可以根据多个摄像头联合成像中视场角较小的摄像头对应的成像范围进行设置。
其中，由于多个摄像头的联合成像受其中视场角较大(焦距较短)的摄像头对应 的成像范围的限制，另外，考虑到图像融合时需要剪裁掉视场角较大的摄像头采集到的产生畸变的边缘图像，因此，第一阈值一般设置为小于视场角较大(焦距较短)的摄像头对应的成像范围。
例如，电子设备配置的超广角摄像头的变焦倍率为“0.6×”，则第一阈值A＝A0-A1，其中，变焦倍率为“0.6×”的超广角摄像头对应的成像范围表示为A0，产生畸变的边缘图像区域表示为A1。
第二阈值的设置需要考虑以下两个方面：
1、根据电子设备的摄像头变焦倍率的上限对应的成像范围设置第二阈值，即第二阈值应小于或者等于最大变焦倍率对应的成像范围。
例如，电子设备配置的长焦摄像头的变焦倍率上限为“10×”，则第二阈值B≤B0，其中，变焦倍率为“10×”的长焦摄像头对应的成像范围表示为B0。
2、根据当前电子设备选用摄像头的变焦倍率对应的成像范围设置第二阈值，以避免第二阈值设置的过小，得到的取景推荐框远小于当前摄像头对应的成像范围，令用户感觉突兀或者体验不好。
例如，第二阈值设置为当前摄像头对应的成像范围的1/4倍，即B＝1/4*B1，其中，B1为电子设备当前摄像头对应的成像范围。
下面将结合不同的拍摄场景，示例性介绍基于不同的摄像头成像，电子设备得到多个取景推荐框的几种情况。
示例一、当前电子设备检测到目标拍摄物体处于长焦焦段，则电子设备可以自动切换使用长焦摄像头和广角摄像头联合成像，或者使用长焦摄像头和超广角摄像头联合成像。
其中，长焦摄像头对应较小的视场角，长焦摄像头的焦距可以为主摄像头焦距的N倍，其中，N大于1，例如，长焦摄像头的变焦倍率可以为“2×”或者“5×”。根据前述的摄像头视场角对成像的影响，长焦摄像头可以用于完成远距离景物的拍摄。广角摄像头或超广角摄像头则对应较大的视场角，广角摄像头或超广角摄像头的焦距可以为主摄像头焦距的M倍，其中，M小于或者等于1，例如，广角摄像头或超广角摄像头的变焦倍率可以为“0.5×”或者“0.6×”。示例性的，主摄像头可以为广角摄像头，则此时M＝1。根据前述的摄像头视场角对成像的影响，广角摄像头可以用于提供涵盖较大取景范围的图像。
因此，考虑到电子设备的采集图像的视场角的限制，第一阈值可以设置为超广角摄像头对应的成像范围的剪裁框，或者，设置为超广角摄像头对应的成像范围减去发生畸变的边缘区域的剪裁框，或者，设置为小于或者等于广角摄像头对应的成像范围的剪裁框，例如，取景剪裁框设置为变焦倍率“0.6×”对应的成像范围。
另外，不加约束的取景框裁切，可能会导致较低的画质，例如，主摄为广角摄像头下拍摄，取景推荐结果推荐“5×”变焦大小的取景剪裁框，这会导致根据推荐取景的剪裁后的图像画质很低。考虑到电子设备的图像超分辨率算法，第二阈值可以设置为长焦摄像头对应的成像范围的剪裁框，或者，设置为大于长焦摄像头对应的成像范围的剪裁框，例如，设置为“2×”变焦大小的取景剪裁框。
示例性的，如图8所示，长焦焦段下对应的可以包括如图8中所示的几种不同的 取景推荐框。其中，最大的取景推荐框，即第一阈值可以为广角摄像头对应的成像范围，最小的取景推荐框，即第二阈值可以为长焦摄像头对应的成像范围的剪裁框，图8中示出了几种介于第一阈值与第二阈值之间的取景推荐剪裁框。其中，如图8所示，在横屏的拍摄模式下，电子设备根据取景推荐的算法，也可以像用户推荐竖屏的取景推荐框。
因此，当拍摄处于长焦焦段，步骤403中电子设备得到的目标图像，可以是基于广角摄像头和长焦摄像头分别得到图像进行图像融合处理后得到的图像。长焦摄像头和广角摄像头的联合成像相比只切换使用长焦摄像头的成像质量显著提升。另外，在电子设备还配置有广角黑白摄像头的情况下，电子设备得到的目标图像，还可以是基于广角黑白摄像头、广角摄像头和长焦摄像头分别得到图像进行图像融合处理后得到的图像。其中，广角黑白摄像头能够用于提升图像细节。
示例二、当前电子设备检测到目标拍摄物体处于广角焦段，则电子设备可以自动切换使用广角摄像头和超广角摄像头联合成像。
示例性的，如图9所示，广角焦段下对应的可以包括如图9中所示的几种不同的取景推荐框。其中，最大的取景推荐框，即第一阈值可以为广角摄像头对应的成像范围，最小的取景推荐框，即第二阈值为长焦摄像头对应的成像范围。图9中示出了几种介于第一阈值与第二阈值之间的取景推荐剪裁框，其中也可以包括竖屏的取景推荐框。
示例三、当前电子设备检测到目标拍摄物体处于超广角焦段，则电子设备可以自动切换使用超广角摄像头成像。
示例性的，如图10所示，超广角焦段下对应的可以包括如图10中所示的几种不同的取景推荐框。其中，考虑图像超分辨率算法的上限，以及考虑到超广角摄像头采集图像的边界区域可能存在图像畸变问题，取景推荐框尽量避免覆盖到图像四角的区域，最大的取景推荐框，即第一阈值可以为超广角摄像头对应的成像范围剪裁掉可能发生畸变的边缘区域对应的剪裁框。另外，最小的取景推荐框，即第二阈值可以为长焦摄像头对应的成像范围。图10中示出了几种介于第一阈值与第二阈值之间的取景推荐剪裁框，其中也可以包括竖屏的取景推荐框。
上述本申请的实施方式，通过对多个摄像头的联合成像进行深度学习，得到取景推荐结果，从而电子设备可以根据取景推荐结果对多个摄像头联合采集的图像进行剪裁处理、旋转处理，得到更加优化的构图和取景视角，提高电子设备的图像拍摄质量，提升用户的拍摄体验。
在一种实施方式中，上述电子设备得到至少一个取景推荐结果，如图6所示，电子设备的实时图像预览界面上可以显示一个或者多个取景推荐框的时候，用户可以通过点击操作，选中其中任一个取景推荐框，用来更新实时的预览图像，或者，用户还可以通过滑动手势查看更多的取景推荐框对应的缩略图。或者，用户还可以通过点击页面上方显示的“×”图标，用于退出取景推荐模式，或者，用户还可以移动电子设备、或者手动变焦，以使得电子设备更新当前的预览图像，并重新计算得到不同的取景推荐结果。
进一步的，电子设备可以通过采集用户对取景推荐结果的选择，作为用户的喜好 信息，反馈到前述的拍照构图计算模型，用于电子设备根据用户的喜好信息进行个性化的模型训练，即将用户选择作为更新用户个性化喜好的参考输入，后续当电子设备进行智能取景推荐的时候，则可以结合用户的喜好信息进行取景推荐结果的输出和排序。
另外，电子设备还可以根据用户的个性喜好设置智能取景模型，例如，在相机的设置中，可以让用户设置拍照喜欢的模式，如“广角拍摄”或者“长焦拍摄”，“风景拍摄”或者“人物拍摄”等。从而，电子设备可以根据用户的个性化的喜好设置，对智能取景模型的计算进行优化更新。
另外，在一种实施方式中，电子设备还可以为用户进行二次取景推荐，也就是说，第一次得到取景推荐结果之后，电子设备还可以根据目标图像得到至少一个取景优化结果，该取景优化结果用于指示对所述目标图像再次进行取景优化处理。当用户点击拍摄按钮生成目标图像之后，基于当前所拍摄图像，电子设备还可以进行二次取景推荐，例如，如图11所示的，用户选择取景推荐框进行图像剪裁之后，得到目标图像。电子设备还可以根据该目标图像进行二次取景推荐，电子设备还可以通过图像预览界面显示新的取景推荐框，指示用户可以根据该取景推荐框进行图像剪裁和拍摄成像。
在一种实施方式中，电子设备可以将最终剪裁得到的目标图像进行保存，另外，电子设备还可以将智能取景模型的算法输出的图像和多个不同的摄像头对应的图像分别保存至相册，以方便用户后续对所得的图像进行处理时，可以进行回退，采用剪裁处理前的原图像，或重新选择取景推荐结果对原图像进行后期处理等操作。
上述可能的实施方式，电子设备通过与用户的交互选择，为用户丰富了拍照的构图推荐，提升用户的拍摄体验；同时，通过记录用户的选择结果，电子设备可以进一步优化取景推荐算法，实现个性化定制，为用户带来更加丰富和贴合用户使用习惯的取景推荐方案，提升用户的拍摄体验。
技术上述实施方式，本申请还提供一种智能取景推荐的拍照装置，如图12所示，该装置1200包括取景推荐模块1201和成像模块1202。
其中，取景推荐模块1201可以用于根据摄像头采集到的图像得到至少一个取景推荐结果，取景推荐结果包括取景推荐框，取景推荐框用于指示图像拍摄的取景推荐效果。
成像模块1202可以用于根据用户选择的取景推荐结果，显示目标图像，其中，目标图像是根据取景推荐结果对至少两个摄像头采集到的图像进行剪裁得到的图像。
在一种可能的实现方式中，取景推荐结果还包括水平基准线或者垂直基准线中的至少一个，水平基准线和垂直基准线用于指示对摄像头采集到的图像进行旋转得到目标图像。
在一种可能的实现方式中，取景推荐结果还包括取景推荐框对应的缩略图，缩略图为根据取景推荐框对摄像头采集到的图像进行剪裁得到的缩略图。
在一种可能的实现方式中，取景推荐模块1201具体用于：根据摄像头采集到的图像进行场景检测，得到构图结构信息；对摄像头采集到的图像和构图结构信息通过取景推荐模型进行拍照构图计算，得到至少一个取景推荐结果。
在一种可能的实现方式中，取景推荐框的大小小于第一阈值，并且大于或者等于 第二阈值，第一阈值是根据多个摄像头中视场角较大的摄像头对应的成像范围设置的，第二阈值可以根据多个摄像头中视场角较小的摄像头对应的成像范围设置的。
在一种可能的实现方式中，该装置1200可以用于执行前述实施例中电子设备所执行的步骤，例如前述实施例中的步骤401至403。
结合前述图1所示的电子设备示意图，该电子设备可以是本申请实施例提供的图像处理的装置(电子设备)。如图1所示，电子设备100包括处理器110和内部存储器121。其中，处理器110和内部存储器121之间可以通过内部连接通路互相通信，传递控制和/或数据信号，该内部存储器121可以用于存储计算机程序，该处理器110可以用于从内部存储器121中调用并运行该计算机程序，以执行前述实施例中的步骤实现图像处理。
具体的，该电子设备100可对应于根据本申请实施例的方法的各个实施例中。并且，该电子设备100中的各单元和上述其他操作和/或功能分别为了实现方法的各个实施例中的相应流程。
上述处理器110可以用于执行前面方法实施例中描述的电子设备实现的一项或多项处理动作。具体请见前面方法实施例中的描述，此处不再赘述。
应理解，本申请实施例中的处理器可以为CPU，该处理器还可以是其他通用处理器、数字信号处理(digital signal processing，DSP)、专用集成电路(application specific integrated circuit，ASIC)、现场可编程逻辑门阵列(Field Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。
还应理解，本申请实施例中的存储器可以是易失性存储器或非易失性存储器，或可包括易失性和非易失性存储器两者。其中，非易失性存储器可以是只读存储器(read-only memory，ROM)、可编程只读存储器(programmable ROM，PROM)、可擦除可编程只读存储器(erasable PROM，EPROM)、电可擦除可编程只读存储器(electrically EPROM，EEPROM)或闪存。易失性存储器可以是随机存取存储器(random access memory，RAM)，其用作外部高速缓存。通过示例性但不是限制性说明，许多形式的随机存取存储器(random access memory，RAM)可用，例如静态随机存取存储器(static RAM，SRAM)、动态随机存取存储器(DRAM)、同步动态随机存取存储器(synchronous DRAM，SDRAM)、双倍数据速率同步动态随机存取存储器(double data rate SDRAM，DDR SDRAM)、增强型同步动态随机存取存储器(enhanced SDRAM，ESDRAM)、同步连接动态随机存取存储器(synchlink DRAM，SLDRAM)和直接内存总线随机存取存储器(direct rambus RAM，DR RAM)。
本申请实施例还提供了一种计算机可读介质，用于存储计算机程序代码，该计算机程序包括用于执行上述方法中的网络设备和终端设备中所执行方法的指令。该可读介质可以是ROM或RAM，本申请实施例对此不做限制。
本申请还提供了一种计算机程序产品，该计算机程序产品包括指令，当该指令被执行时，以使得终端设备和网络设备分别执行对应于上述方法的终端设备和网络设备的操作。
可选地，该计算机指令被存储在存储单元中。
可选地，该存储单元为电子设备配置的存储单元，如寄存器、缓存等，该存储单 元还可以是该装置内的位于芯片外部的存储单元，如ROM或可存储静态信息和指令的其他类型的静态存储设备，RAM等。其中，上述任一处提到的处理器，可以是一个CPU，微处理器，ASIC，或一个或多个用于控制上述的反馈信息传输的方法的程序执行的集成电路。该处理单元和该存储单元可以解耦，分别设置在不同的物理设备上，通过有线或者无线的方式连接来实现该处理单元和该存储单元的各自的功能，以支持该系统芯片实现上述实施例中的各种功能。或者，该处理单元和该存储器也可以耦合在同一个设备上。应理解，在本申请实施例中的处理器可以是CPU，该处理器还可以是其他通用处理器、DSP、ASIC、FPGA或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的模块、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
应理解，在本申请的各种实施例中，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的模块及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
在本申请所提供的几个实施例中，应该理解到，所揭露的装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述模块的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个模块或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或模块的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的模块可以是或者也可以不是物理上分开的，作为模块显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个模块上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能模块可以集成在一个处理模块中，也可以是各个模块单独物理存在，也可以两个或两个以上模块集成在一个单元中。
所述功能如果以软件功能模块的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。
最后应说明的是：以上所述，仅为本申请的具体实施方式，但本申请的保护范围 并不局限于此，任何在本申请揭露的技术范围内的变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (20)

  1. 一种智能取景推荐的拍照方法，应用于配置有至少两个焦距不同的摄像头的电子设备，其特征在于，所述方法包括：
    根据所述摄像头采集到的图像得到至少一个取景推荐结果，所述取景推荐结果包括取景推荐框，所述取景推荐框用于指示图像拍摄的取景推荐效果；
    根据用户选择的取景推荐结果，显示目标图像，其中，所述目标图像是根据所述取景推荐结果对所述至少两个摄像头采集到的图像进行剪裁得到的图像。
  2. 根据权利要求1所述的方法，其特征在于，所述取景推荐结果还包括水平基准线或者垂直基准线中的至少一个，所述水平基准线和所述垂直基准线用于指示对所述摄像头采集到的图像进行旋转得到所述目标图像。
  3. 根据权利要求1或2所述的方法，其特征在于，所述取景推荐结果还包括所述取景推荐框对应的缩略图，所述缩略图为根据所述取景推荐框对所述摄像头采集到的图像进行剪裁得到的缩略图。
  4. 根据权利要求1-3任一项所述的方法，其特征在于，根据所述摄像头采集到的图像得到至少一个取景推荐结果，具体包括：
    根据所述摄像头采集到的图像进行场景检测，得到构图结构信息；
    对所述摄像头采集到的图像和所述构图结构信息通过取景推荐模型进行拍照构图计算，得到至少一个取景推荐结果。
  5. 根据权利要求1-4任一项所述的方法，其特征在于，所述取景推荐框的大小小于第一阈值，并且大于或者等于第二阈值，所述第一阈值是根据多个摄像头中视场角较大的摄像头对应的成像范围设置的，所述第二阈值可以根据所述多个摄像头中视场角较小的摄像头对应的成像范围设置的。
  6. 一种智能取景推荐的拍照方法，应用于配置有至少两个焦距不同的摄像头的电子设备，其特征在于，所述方法包括：
    响应于用户的第一操作，根据摄像头采集到的图像得到至少一个取景推荐结果，所述取景推荐结果包括取景推荐框，所述取景推荐框用于指示图像拍摄的取景推荐效果；
    接收用户的第二操作，所述第二操作用于指示用户从所述至少一个取景推荐结果中确定第一取景推荐框；
    响应于所述第二操作，更新实时预览图像为目标图像，其中，所述目标图像是根据所述第一取景推荐框对所述至少两个摄像头采集到的图像进行剪裁得到的图像。
  7. 根据权利要求6所述的方法，其特征在于，所述取景推荐结果还包括水平基准线或者垂直基准线中的至少一个，所述水平基准线和所述垂直基准线用于指示对所述摄像头采集到的图像进行旋转得到所述目标图像。
  8. 根据权利要求6或7所述的方法，其特征在于，所述取景推荐结果还包括所述取景推荐框对应的缩略图，所述缩略图为根据所述取景推荐框对所述摄像头采集到的图像进行剪裁得到的缩略图。
  9. 根据权利要求6-8任一项所述的方法，其特征在于，根据所述摄像头采集到的图像得到至少一个取景推荐结果，具体包括：
    根据所述摄像头采集到的图像进行场景检测，得到构图结构信息；
    对所述摄像头采集到的图像和所述构图结构信息通过取景推荐模型进行拍照构图计算，得到至少一个取景推荐结果。
  10. 根据权利要求6-9任一项所述的方法，其特征在于，所述取景推荐框的大小小于第一阈值，并且大于或者等于第二阈值，所述第一阈值是根据多个摄像头中视场角较大的摄像头对应的成像范围设置的，所述第二阈值可以根据所述多个摄像头中视场角较小的摄像头对应的成像范围设置的。
  11. 根据权利要求6-10任一项所述的方法，其特征在于，所述更新实时预览图像为目标图像之后，所述方法还包括：
    响应于用户的确认拍摄操作，将所述目标图像以及所述至少两个焦距不同的摄像头采集到的图像分别保存。
  12. 根据权利要求6-11任一项所述的方法，其特征在于，所述方法还包括：
    根据所述目标图像得到至少一个取景优化结果，所述取景优化结果包括取景推荐框，所述取景优化结果用于指示对所述目标图像再次进行取景优化处理。
  13. 一种智能取景推荐的拍照装置，所述装置配置有至少两个焦距不同的摄像头，其特征在于，所述装置包括：
    取景推荐模块，用于根据所述摄像头采集到的图像得到至少一个取景推荐结果，所述取景推荐结果包括取景推荐框，所述取景推荐框用于指示图像拍摄的取景推荐效果；
    成像模块，用于根据用户选择的取景推荐结果，显示目标图像，其中，所述目标图像是根据所述取景推荐结果对所述至少两个摄像头采集到的图像进行剪裁得到的图像。
  14. 根据权利要求13所述的装置，其特征在于，所述取景推荐结果还包括水平基准线或者垂直基准线中的至少一个，所述水平基准线和所述垂直基准线用于指示对所述摄像头采集到的图像进行旋转得到所述目标图像。
  15. 根据权利要求13或14所述的装置，其特征在于，所述取景推荐结果还包括所述取景推荐框对应的缩略图，所述缩略图为根据所述取景推荐框对所述摄像头采集到的图像进行剪裁得到的缩略图。
  16. 根据权利要求13-15任一项所述的装置，其特征在于，所述取景推荐模块具体用于：
    根据所述摄像头采集到的图像进行场景检测，得到构图结构信息；
    对所述摄像头采集到的图像和所述构图结构信息通过取景推荐模型进行拍照构图计算，得到至少一个取景推荐结果。
  17. 根据权利要求13-16任一项所述的装置，其特征在于，所述取景推荐框的大小小于第一阈值，并且大于或者等于第二阈值，所述第一阈值是根据多个摄像头中视场角较大的摄像头对应的成像范围设置的，所述第二阈值可以根据所述多个摄像头中视场角较小的摄像头对应的成像范围设置的。
  18. 一种电子设备，其特征在于，所述电子设备包括：
    处理器；
    用于存储所述处理器可执行指令的存储器；
    其中，所述处理器被配置为执行所述指令，以实现如权利要求1至5或者6-12中任一项所述的方法。
  19. 一种计算机可读存储介质，其特征在于，当所述计算机可读存储介质中的指令由电子设备的处理器执行时，使得所述电子设备能够执行如权利要求1至5或者6-12中任一项所述的方法。
  20. 一种计算机程序产品，其特征在于，当所述计算机程序产品在计算机上运行时，使得所述计算机执行如权利要求1至5或者6-12中任一项所述的方法。
PCT/CN2021/140208 2020-12-31 2021-12-21 一种智能取景推荐的拍照方法及装置 Ceased WO2022143311A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP21914050.6A EP4258650A4 (en) 2020-12-31 2021-12-21 PHOTOGRAPHY METHOD AND DEVICE FOR INTELLIGENT VIEW FIND RECOMMENDATION
US18/343,942 US12464226B2 (en) 2020-12-31 2023-06-29 Photographing method and apparatus for intelligent framing recommendation

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011615964.9A CN114697530B (zh) 2020-12-31 2020-12-31 一种智能取景推荐的拍照方法及装置
CN202011615964.9 2020-12-31

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/343,942 Continuation US12464226B2 (en) 2020-12-31 2023-06-29 Photographing method and apparatus for intelligent framing recommendation

Publications (1)

Publication Number Publication Date
WO2022143311A1 true WO2022143311A1 (zh) 2022-07-07

Family

ID=82132151

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/140208 Ceased WO2022143311A1 (zh) 2020-12-31 2021-12-21 一种智能取景推荐的拍照方法及装置

Country Status (4)

Country Link
US (1) US12464226B2 (zh)
EP (1) EP4258650A4 (zh)
CN (1) CN114697530B (zh)
WO (1) WO2022143311A1 (zh)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115601274B (zh) * 2021-07-07 2024-06-14 荣耀终端有限公司 图像处理方法、装置和电子设备
CN115623319B (zh) * 2022-08-30 2023-11-03 荣耀终端有限公司 一种拍摄方法及电子设备
CN118673172A (zh) * 2023-01-30 2024-09-20 华为技术有限公司 一种图片显示方法和电子设备
CN117688196B (zh) * 2023-05-17 2024-12-27 荣耀终端有限公司 图像推荐方法、相关装置
CN119922412A (zh) * 2024-11-28 2025-05-02 深圳市中诺通讯有限公司 拍照推荐方法、装置、设备以及存储介质
CN119653227B (zh) * 2024-12-12 2026-01-06 北京玄戒技术有限公司 一种拍照方法、装置、电子设备、芯片及存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102647552A (zh) * 2011-02-22 2012-08-22 华为终端有限公司 图像摄取控制方法及装置
US20130155293A1 (en) * 2011-12-16 2013-06-20 Samsung Electronics Co., Ltd. Image pickup apparatus, method of providing composition of image pickup and computer-readable recording medium
CN104243801A (zh) * 2013-06-17 2014-12-24 索尼公司 显示控制装置、显示控制方法、程序和成像设备
CN106888349A (zh) * 2017-03-30 2017-06-23 努比亚技术有限公司 一种拍摄方法及装置
CN108833784A (zh) * 2018-06-26 2018-11-16 Oppo（重庆）智能科技有限公司 一种自适应构图方法、移动终端及计算机可读存储介质
CN111182225A (zh) * 2020-02-26 2020-05-19 Oppo广东移动通信有限公司 拍照控制方法、电子装置和非易失性计算机可读存储介质
CN111770277A (zh) * 2020-07-31 2020-10-13 RealMe重庆移动通信有限公司 一种辅助拍摄方法及终端、存储介质

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004153513A (ja) * 2002-10-30 2004-05-27 Fuji Photo Film Co Ltd 画像処理装置及びプログラム
KR101999137B1 (ko) * 2013-01-03 2019-07-11 삼성전자주식회사 카메라를 구비하는 장치의 이미지 처리장치 및 방법
CN104378545A (zh) * 2013-08-16 2015-02-25 中兴通讯股份有限公司 一种拍照处理方法和系统
US10019823B2 (en) * 2013-10-24 2018-07-10 Adobe Systems Incorporated Combined composition and change-based models for image cropping
CN104361329B (zh) * 2014-11-25 2018-08-24 成都品果科技有限公司 一种基于人脸识别的照片裁剪方法及系统
CN106060386A (zh) * 2016-06-08 2016-10-26 维沃移动通信有限公司 一种预览图像生成方法及移动终端
CN106131411B (zh) 2016-07-14 2019-04-02 纳恩博（北京）科技有限公司 一种拍摄图像的方法和装置
CN106657761A (zh) * 2016-09-29 2017-05-10 乐视控股（北京）有限公司 一种智能终端的拍照方法和装置
EP3496386A4 (en) * 2016-11-24 2019-07-03 Huawei Technologies Co., Ltd. GUIDANCE METHOD AND DEVICE FOR PHOTOGRAPHIC COMPOSITION
CN106851107A (zh) 2017-03-09 2017-06-13 广东欧珀移动通信有限公司 切换摄像头辅助构图的控制方法、控制装置及电子装置
KR102457559B1 (ko) * 2017-10-17 2022-10-21 삼성전자 주식회사 이미지에 포함된 오브젝트에 기반하여 이미지를 보정하기 위한 전자 장치 및 방법
KR102438201B1 (ko) * 2017-12-01 2022-08-30 삼성전자주식회사 사진 촬영과 관련된 추천 정보를 제공하는 방법 및 시스템
CN108093174A (zh) 2017-12-15 2018-05-29 北京臻迪科技股份有限公司 拍照设备的构图方法、装置和拍照设备
CN108289169A (zh) * 2018-01-09 2018-07-17 北京小米移动软件有限公司 拍摄方法、装置、电子设备及存储介质
CN108255385A (zh) * 2018-01-10 2018-07-06 上海展扬通信技术有限公司 电子设备及其拍照方法、计算机可读存储介质
CN110913120B (zh) * 2018-09-17 2021-11-30 北京小米移动软件有限公司 图像拍摄方法及装置、电子设备、存储介质
CN109218619A (zh) * 2018-10-12 2019-01-15 北京旷视科技有限公司 图像获取方法、装置和系统
KR102665654B1 (ko) * 2019-02-19 2024-05-14 삼성전자주식회사 구도를 추천하기 위한 전자 장치 및 그의 동작 방법
CN112333380B (zh) * 2019-06-24 2021-10-15 华为技术有限公司 一种拍摄方法及设备

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102647552A (zh) * 2011-02-22 2012-08-22 华为终端有限公司 图像摄取控制方法及装置
US20130155293A1 (en) * 2011-12-16 2013-06-20 Samsung Electronics Co., Ltd. Image pickup apparatus, method of providing composition of image pickup and computer-readable recording medium
CN104243801A (zh) * 2013-06-17 2014-12-24 索尼公司 显示控制装置、显示控制方法、程序和成像设备
CN106888349A (zh) * 2017-03-30 2017-06-23 努比亚技术有限公司 一种拍摄方法及装置
CN108833784A (zh) * 2018-06-26 2018-11-16 Oppo（重庆）智能科技有限公司 一种自适应构图方法、移动终端及计算机可读存储介质
CN111182225A (zh) * 2020-02-26 2020-05-19 Oppo广东移动通信有限公司 拍照控制方法、电子装置和非易失性计算机可读存储介质
CN111770277A (zh) * 2020-07-31 2020-10-13 RealMe重庆移动通信有限公司 一种辅助拍摄方法及终端、存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4258650A4

Also Published As

Publication number Publication date
EP4258650A4 (en) 2024-06-05
CN114697530B (zh) 2023-11-10
US12464226B2 (en) 2025-11-04
US20230353864A1 (en) 2023-11-02
EP4258650A1 (en) 2023-10-11
CN114697530A (zh) 2022-07-01

Similar Documents

Publication Publication Date Title
US12464226B2 (en) Photographing method and apparatus for intelligent framing recommendation
WO2021238325A1 (zh) 一种图像处理方法及装置
WO2021147482A1 (zh) 一种长焦拍摄的方法及电子设备
CN113630545B (zh) 一种拍摄方法及设备
CN115689963B (zh) 一种图像处理方法及电子设备
WO2022088946A1 (zh) 一种弯曲文本的字符选择方法、装置和终端设备
US20230188830A1 (en) Image Color Retention Method and Device
CN111491102A (zh) 拍照场景的检测方法、系统、移动终端及存储介质
US20240406536A1 (en) Shooting method and electronic device
CN117119285B (zh) 一种拍摄方法
WO2022127611A1 (zh) 一种拍照方法及相关设备
CN114693538B (zh) 一种图像处理方法及装置
WO2022206605A1 (zh) 确定目标对象的方法、拍摄方法和装置
US12556809B2 (en) Shooting method and related device
WO2024179100A1 (zh) 一种拍摄方法
CN117201930A (zh) 一种拍照方法和电子设备
WO2023035868A1 (zh) 拍摄方法及电子设备
CN117714832B (zh) 拍照方法、电子设备及计算机可读存储介质
US20230224574A1 (en) Photographing method and apparatus
US20240414432A1 (en) Shooting method and related device
CN114926351B (zh) 图像处理方法、电子设备以及计算机存储介质
WO2023165535A1 (zh) 图像处理方法、装置及设备
CN115623319A (zh) 一种拍摄方法及电子设备
WO2023093169A1 (zh) 拍摄的方法和电子设备
CN116074620A (zh) 一种拍摄方法及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21914050

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021914050

Country of ref document: EP

Effective date: 20230705

NENP Non-entry into the national phase

Ref country code: DE