WO2024041394A1 - 拍摄方法及相关装置 - Google Patents

拍摄方法及相关装置 Download PDF

Info

Publication number
WO2024041394A1
WO2024041394A1 PCT/CN2023/112243 CN2023112243W WO2024041394A1 WO 2024041394 A1 WO2024041394 A1 WO 2024041394A1 CN 2023112243 W CN2023112243 W CN 2023112243W WO 2024041394 A1 WO2024041394 A1 WO 2024041394A1
Authority
WO
WIPO (PCT)
Prior art keywords
picture
electronic device
camera
image
area
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/112243
Other languages
English (en)
French (fr)
Inventor
吴国亮
邱红兵
秦怡
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to EP23856484.3A priority Critical patent/EP4589980A4/en
Publication of WO2024041394A1 publication Critical patent/WO2024041394A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/631—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/631—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
    • H04N23/632—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/633—Control of cameras or camera modules by using electronic viewfinders for displaying additional information relating to control or operation of the camera
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/95—Computational photography systems, e.g. light-field imaging systems
    • H04N23/951—Computational photography systems, e.g. light-field imaging systems by using two or more images to influence resolution, frame rate or aspect ratio

Definitions

  • the present application relates to the field of terminal technology, and in particular to photographing methods and related devices.
  • Methods for taking group photos include the following:
  • the selfie stick increases the user's load when traveling, makes it impossible to control the shooting distance, and the shooting clarity of the front camera is low.
  • the tripod also increases the load and cannot be previewed in real time. There is also a risk of damage to the shooting equipment. Placing the shooting equipment on the ground will not allow real-time preview or control of the shooting distance.
  • This application provides a shooting method and related devices, which can achieve high-quality group photos of multiple objects without the help of other equipment or other personnel.
  • embodiments of the present application provide a photographing method, which is applied to electronic devices.
  • the method may include: the electronic device starts a gallery application and displays a first user interface, a first picture is displayed in the first user interface, and the first picture contains a first object; the electronic device receives a first operation for the first picture; The electronic device starts the camera application and displays the second user interface. A preview box is displayed in the second user interface. The image collected by the camera of the electronic device and the instruction information of the first image are displayed in the preview box; the electronic device receives the user interface.
  • the electronic device stores a second picture, the second picture is obtained by fitting the image collected by the camera and the first picture, the image collected by the camera contains the second object, and the second picture contains the first object and the second object.
  • the electronic device can call the camera application through the gallery application, and can complete the photo of the first object and the second object without the help of other devices or other people, and during the photo taking process, Preview and adjust the relative position between the first object and the second object in real time. You can also flexibly adjust the shooting distance and the camera used to achieve a high-quality photo effect. Moreover, this method is easy to operate, short in time, convenient and fast, and can support continuous and large number of group photos.
  • the first object or the second object is any one of the following: characters, animals, landmarks, buildings, and dolls.
  • the number of the first object or the second object may be multiple.
  • the indication information of the first picture displayed by the electronic device in the preview box may be the first picture with transparency or blur.
  • the user can not only see the first picture through the preview box, but also see the image collected by the camera displayed on the electronic device through the blurred or transparent first picture, thereby achieving a real-time preview of the first picture.
  • the electronic device after receiving the second operation, can obtain the second picture by fitting in any of the following ways:
  • Method 1 Fit the area of the second object in the image collected by the camera to the first picture, thereby obtaining the second picture.
  • the electronic device identifies the second object in the image collected by the camera; determines the main area containing the second object in the image collected by the camera, and the main area is the area determined based on the second object; in the first picture Determine the co-located area, the part outside the co-located area in the first picture, and the correlation of the part outside the main area in the image collected by the camera is greater than the second value; fit the main area to the co-located area in the first picture, and get Second picture.
  • Method 2 Fit the area of the first object in the first picture to the image collected by the camera to obtain the second picture.
  • the electronic device identifies the first object in the first picture; determines a main area containing the first object in the first picture, and the main area is an area determined based on the first object; determines the co-located area in the image collected by the camera, and the camera The portion of the acquired image outside the apposition area, and, The correlation of the parts other than the main area in the first picture is greater than the second value; fit the main area to the same area in the image collected by the camera to obtain the second picture.
  • the main area can be an area obtained by extending outward by a distance of the first value based on the corresponding object (such as the first object or the second object), or an area obtained by extending to the surroundings and the corresponding object. Areas of similar or identical colors or pixels.
  • the first value may be preset by the electronic device. The first value may be related to the electronic device configuration camera model.
  • Determining the main area based on the corresponding object can not only ensure the integrity of the objects in the main area, but also avoid excessive expansion of the range, thereby ensuring the effect of subsequent fitting.
  • the position of the co-located area in the first picture and the position of the main area in the image collected by the camera are the same. In this way, the effect that the user adjusts in real time based on the content in the preview box can directly act on the final fitting effect, achieving the purpose of the user adjusting the photo effect in real time.
  • the size and shape of the co-located area in the first picture and the size and shape of the main area in the image collected by the camera are the same or different.
  • the second picture is the highest quality frame of the composite image obtained by fitting the multi-frame images collected by the camera to the first picture respectively;
  • the multi-frame images collected by the camera include: At the moment when the electronic device receives the second operation and the images collected by the camera at the previous moment, the number of multi-frame images is preset by the electronic device. This ensures high-quality photo effects.
  • the second picture is a composite image obtained by fitting the image collected by the camera at the moment when the second operation is received and the first picture.
  • the first control is displayed in the first user interface, and before the electronic device receives the first operation for the first picture, the method further includes: the electronic device receives the operation that acts on the first control. User operation; the electronic device displays the second control in the first user interface; the first operation includes a user operation acting on the second control.
  • the electronic device may receive a third operation for viewing the second picture and display the second picture. Equivalently, after completing the group photo of the first object and the second object, the user can also view the group photo on the electronic device.
  • embodiments of the present application provide a photographing method, which is applied to electronic devices.
  • the method may include: the electronic device starts a gallery application and displays a first user interface, a first picture is displayed in the first user interface, and the first picture contains a first object; the electronic device receives a first operation for the first picture; The electronic device starts the camera application and displays the second user interface.
  • a preview box is displayed in the second user interface, and a composite image is displayed in the preview box.
  • the composite image is obtained by fitting the image collected by the camera of the electronic device and the first picture.
  • the camera The collected image contains the second object; the electronic device receives the second operation for taking pictures; the electronic device stores the second picture, the second picture is the composite image displayed in the preview box when the second operation is received, and the second picture contains first object and second object.
  • the electronic device can call the camera application through the gallery application, and can display the effect of the group photo to the user in real time in the preview box, allowing the user to feel the effect of the group photo more intuitively, and can do so without the help of other devices or Complete the group photo of the first object and the second object without the presence of other persons.
  • this method is easy to operate, short in time, convenient and fast, and can support continuous and large number of group photos.
  • the first object or the second object is any one of the following: characters, animals, landmarks, buildings, and dolls.
  • the number of the first object or the second object may be multiple.
  • the electronic device can fit the composite image displayed in the preview box in any of the following ways:
  • Method 1 Fit the area of the second object in the image collected by the camera to the first picture, thereby obtaining the second picture.
  • Method 2 Fit the area of the first object in the first picture to the image collected by the camera to obtain the second picture.
  • the first control is displayed in the first user interface, and before the electronic device receives the first operation for the first picture, it can also receive a user operation acting on the first control, and
  • the second control is displayed in the first user interface; the first operation includes a user operation acting on the second control.
  • the electronic device may receive a third operation for viewing the second picture and display the second picture. Equivalently, after completing the group photo of the first object and the second object, the user can also view the group photo on the electronic device.
  • embodiments of the present application provide a photographing method, which is applied to electronic devices.
  • the method may include: electronic device display A second user interface is displayed, a preview box and a third control are displayed in the second user interface, and an image collected by the camera of the electronic device is displayed in the preview box; the electronic device detects the fourth operation acting on the third control; the electronic device One or more pictures are displayed; the electronic device detects a fifth operation acting on a first picture of the one or more pictures, the first picture containing the first object; the electronic device is in the preview box of the second user interface , display the image collected by the camera of the electronic device, and the indication information of the first picture; the electronic device receives the second operation for taking pictures; the electronic device stores the second picture, the second picture is the image collected by the camera and the first picture It is obtained from fitting that the image collected by the camera contains the second object, and the second picture contains the first object and the second object.
  • the electronic device can start the co-photography function through the preview interface, and can complete the photo of the first object and the second object without the help of other devices or other people, and can also take a real-time photo during the photo-taking process.
  • this method is easy to operate, short in time, convenient and fast, and can support continuous and large number of group photos.
  • the second user interface may be provided by the camera application, or may be an interface provided by other applications calling the camera interface.
  • Other applications may be, for example, social networking applications, shopping applications, etc.
  • the co-shooting function can be activated through the camera application or other applications.
  • the method and technical effects of fitting the electronic device to obtain the second picture may refer to the relevant descriptions in the first aspect.
  • the second picture is the highest quality frame of the composite image obtained by fitting the multiple frame images collected by the camera to the first picture respectively;
  • the multiple frame images collected by the camera include: At the moment when the electronic device receives the second operation and the images collected by the camera at the previous moment, the number of multi-frame images is preset by the electronic device. This ensures high-quality photo effects.
  • the second picture is a composite image obtained by fitting the image collected by the camera at the moment when the second operation is received and the first picture.
  • the electronic device may receive a third operation for viewing the second picture and display the second picture. Equivalently, after completing the group photo of the first object and the second object, the user can also view the group photo on the electronic device.
  • embodiments of the present application provide a photographing method, which is applied to electronic devices.
  • the method may include: the electronic device displays a second user interface, a preview box and a third control are displayed in the second user interface, and an image collected by a camera of the electronic device is displayed in the preview box; the electronic device detects an action on the third control The fourth operation; the electronic device displays one or more pictures; the electronic device detects the fifth operation acting on a first picture of the one or more pictures, and the first picture contains the first object; the electronic device A composite image is displayed in the preview box of the second user interface. The composite image is obtained by fitting the image collected by the camera of the electronic device and the first picture.
  • the image collected by the camera contains the second object; the electronic device receives the second operation for taking pictures; The electronic device stores a second picture, the second picture is a composite image displayed in the preview box when the second operation is received, and the second picture contains the first object and the second object.
  • the electronic device can activate the co-photography function through the preview interface, and can display the photo effect to the user in real time in the preview box, allowing the user to feel the photo effect more intuitively, and can do so without the help of other devices or other devices.
  • the electronic device can activate the co-photography function through the preview interface, and can display the photo effect to the user in real time in the preview box, allowing the user to feel the photo effect more intuitively, and can do so without the help of other devices or other devices.
  • this method is easy to operate, short in time, convenient and fast, and can support continuous and large number of group photos.
  • the second user interface may be provided by a camera application, or may be an interface provided by other applications calling the camera interface.
  • Other applications may be, for example, social networking applications, shopping applications, etc. Equivalently, the co-shooting function can be activated through the camera application or other applications.
  • the first object or the second object is any one of the following: characters, animals, landmarks, buildings, and dolls.
  • the number of the first object or the second object may be multiple.
  • the electronic device can fit the composite image displayed in the preview box in any of the following ways:
  • Method 1 Fit the area of the second object in the image collected by the camera to the first picture, thereby obtaining the second picture.
  • Method 2 Fit the area of the first object in the first picture to the image collected by the camera to obtain the second picture.
  • the electronic device may receive a third operation for viewing the second picture and display the second picture. Equivalently, after completing the group photo of the first object and the second object, the user can also Please check out this photo.
  • embodiments of the present application provide an electronic device, including: a memory and one or more processors; the memory is coupled to the one or more processors, and the memory is used to store computer program code, and the computer program code includes computer instructions.
  • one or more processors invoke computer instructions to cause the electronic device to execute the method of any implementation implemented by the electronic device in the first aspect, the second aspect, the third aspect, or the fourth aspect.
  • embodiments of the present application provide a computer-readable storage medium, including instructions.
  • the electronic device When the instructions are run on an electronic device, the electronic device causes the electronic device to execute the first aspect, the second aspect, the third aspect, or the fourth aspect. The method of any embodiment executed by the electronic device.
  • embodiments of the present application provide a computer program product.
  • the computer program product When the computer program product is run on a computer, it causes the computer to execute as executed by the electronic device in the first aspect, the second aspect, the third aspect, or the fourth aspect. Any method of implementation.
  • inventions of the present application provide a chip system.
  • the chip system includes at least one processor for implementing the steps performed by the electronic device in the first aspect, the second aspect, the third aspect, or the fourth aspect. Any method of implementation.
  • the electronic device can complete the group photo of the first object and the second object without the help of other equipment or other personnel, and can also preview and adjust the first object and the second object in real time during the group photo process.
  • the relative position between objects can also be flexibly adjusted to the shooting distance and the camera used to achieve high-quality group photos.
  • this method is easy to operate, short in time, convenient and fast, and can support continuous and large number of group photos.
  • FIG. 1A is a schematic diagram of the hardware structure of the electronic device 100 provided by an embodiment of the present application.
  • FIG. 1B is a schematic diagram of the software structure of the electronic device 100 provided by the embodiment of the present application.
  • FIG. 1C is a schematic diagram of another software structure of the electronic device 100 provided by the embodiment of the present application.
  • Figures 2A-2H are a set of user interfaces for taking a group photo provided by the electronic device 100 in the embodiment of the present application;
  • Figures 3A to 3F are another set of user interfaces for taking a group photo provided by the electronic device 100 in the embodiment of the present application;
  • Figures 4A and 4B show the user interface provided by the gallery application after the electronic device 100 saves the fitted pictures in the embodiment of the present application;
  • Figure 5 is a flow chart of the shooting method provided by the embodiment of the present application.
  • Figures 6A-6B are examples of several image processing processes provided by embodiments of the present application.
  • first and second are used for descriptive purposes only and shall not be understood as implying or implying relative importance or implicitly specifying the quantity of indicated technical features. Therefore, the features defined as “first” and “second” may explicitly or implicitly include one or more of the features. In the description of the embodiments of this application, unless otherwise specified, “plurality” The meaning is two or more.
  • GUI graphical user interface
  • the electronic device can activate the "co-photography" function for a selected picture containing object A, then start the camera application, and display the picture with transparency or blur in the preview interface provided by the camera application, And simultaneously display the image collected by the camera of the electronic device.
  • the electronic device can receive a photographing instruction input by the user, and can then fit the image with the image collected by the camera to generate a composite image containing both object A and object B. .
  • the above-mentioned object A or object B may include but is not limited to: characters, animals, landmarks, buildings, dolls, etc.
  • the number of object A or object B can be multiple.
  • the above shooting method can achieve a group photo of object A and object B.
  • This method can complete the group photo without the help of other equipment or other people, and can also preview and adjust the relative positions of object B and object A in real time during the group photo, and can also flexibly adjust the shooting distance and the camera used for shooting. , to achieve high-quality photo effects.
  • this method is easy to operate, short in time, convenient and fast, and can support continuous and large number of group photos.
  • the above shooting method can be applied to scenes that require group photos, including but not limited to: multiple group photos of the same person at the same place; group photos of new people and people in old photos to achieve group photos of multiple people at the same place at different times.
  • the electronic device can extract the area where object B is located in the image captured by the camera, and fit the area to a picture containing object A, while the rest of the picture remains unchanged to generate a generated image that also contains object A.
  • Composite image of object B Through image fitting, object B can be smoothly and naturally integrated into the corresponding area of the picture containing object A, resulting in a natural, high-quality photo effect.
  • the electronic device can also extract the area where object A is located in the picture, and fit the area to the image collected by the camera. For the specific method of generating a composite image, please refer to the detailed description of subsequent method embodiments.
  • the electronic device can select N frames of images captured by the camera at that moment and before, starting from the moment when the photographing instruction is received, and perform fitting processing on each of the N frames of images and the picture containing object A, to obtain N frame-by-frame composite image, and then select the frame with the best quality from these N frames of composite images and store it as a group photo result.
  • the optimal frame can be selected from multiple frames of composite images as the photo result to achieve a high-quality photo effect.
  • this method also takes into account the time delay between the user seeing the photo effect on the preview interface and inputting the photo command, thereby avoiding using the composite image corresponding to the moment when the photo command is received as the final photo effect.
  • the electronic device after the electronic device starts the camera application, it can also directly fit the image captured by the camera with the selected image containing object A, and directly display the composite image in the preview interface provided by the camera application.
  • the "co-photography” function is provided by the electronic device.
  • This function supports the electronic device to display a transparent or blurred picture containing object A in the preview interface provided by the camera application, and simultaneously displays the image of the electronic device.
  • the electronic device receives the photographing instruction input by the user, fits the image to the image collected by the camera, and generates a composite image containing both object A and object B.
  • the "co-shooting" function supports the electronic device to directly fit the image captured by the camera with the image containing object A, and directly display the composite image in the preview interface provided by the camera application.
  • the "co-production” function is just a noun used in the embodiment of this application, and its meaning has been recorded in this embodiment. Its name does not constitute any limitation on this embodiment.
  • the "co-photography” function can also be called other nouns, such as the "group photo” function, multi-person photo, multi-person co-photo, etc.
  • Electronic devices provided by embodiments of the present application are equipped with cameras, which may include but are not limited to smartphones, tablets, specialized cameras (such as SLR cameras, card cameras), etc., augmented reality (AR) devices, virtual reality ( virtual reality (VR) equipment, artificial intelligence (AI) equipment, wearable devices (such as smart watches, smart glasses), personal digital assistant (PDA), etc.
  • augmented reality (AR) devices virtual reality ( virtual reality (VR) equipment, artificial intelligence (AI) equipment, wearable devices (such as smart watches, smart glasses), personal digital assistant (PDA), etc.
  • Exemplary embodiments of electronic devices include, but are not limited to, equipped with Portable electronic devices with Linux or other operating systems.
  • the above-mentioned electronic device can also be other portable electronic devices, such as a laptop computer (Laptop).
  • FIG. 1A shows a schematic structural diagram of an electronic device 100 provided by an embodiment of the present application.
  • the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2 , mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone interface 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and Subscriber identification module (SIM) card interface 195, etc.
  • a processor 110 an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2 , mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone interface 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display
  • the sensor module 180 may include a pressure sensor 180A, a gyro sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and ambient light. Sensor 180L, bone conduction sensor 180M, etc.
  • the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100 .
  • the electronic device 100 may include more or fewer components than shown in the figures, or some components may be combined, some components may be separated, or some components may be arranged differently.
  • the components illustrated may be implemented in hardware, software, or a combination of software and hardware.
  • the processor 110 may include one or more processing units.
  • the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (GPU), and an image signal processor. (image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, and/or neural network Network processor (neural-network processing unit, NPU), etc.
  • application processor application processor
  • GPU graphics processing unit
  • ISP image signal processor
  • DSP digital signal processor
  • NPU neural network processing unit
  • different processing units can be independent devices or integrated in one or more processors.
  • the controller can generate operation control signals based on the instruction operation code and timing signals to complete the control of fetching and executing instructions.
  • the processor 110 may also be provided with a memory for storing instructions and data.
  • the memory in processor 110 is cache memory. This memory may hold instructions or data that have been recently used or recycled by processor 110 . If the processor 110 needs to use the instructions or data again, it can be called directly from the memory. Repeated access is avoided and the waiting time of the processor 110 is reduced, thus improving the efficiency of the system.
  • the processor 110 such as the controller or GPU can be used to perform transparency processing or blur processing on the picture containing object A, and superimpose it with the image collected by the camera 193 to synthesize the image displayed in the viewfinder. Preview image.
  • the controller or processor 110 such as a GPU can be used to fit the image captured by the camera with the image containing object A to generate a preview image displayed in the viewfinder, so that the group photo effect can be achieved presented to the user.
  • the fitting operation performed by the processor 110 please refer to the detailed introduction of subsequent method embodiments.
  • the processor 110 such as the controller or GPU can also be used to perform anti-shake processing on the images collected by the camera in the co-photography scene, and then combine the anti-shake processed images corresponding to the camera and the objects contained therein.
  • the picture of A is superimposed or fitted to provide a preview image for display on the display screen 194 .
  • the wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.
  • Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals.
  • Each antenna in electronic device 100 may be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example: Antenna 1 can be reused as a diversity antenna for a wireless LAN. In other embodiments, antennas may be used in conjunction with tuning switches.
  • the mobile communication module 150 can provide solutions for wireless communication including 2G/3G/4G/5G applied on the electronic device 100 .
  • the mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc.
  • the mobile communication module 150 can receive electromagnetic waves through the antenna 1, perform filtering, amplification and other processing on the received electromagnetic waves, and transmit them to the modem processor for demodulation.
  • the mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves through the antenna 1 for radiation.
  • at least part of the functional modules of the mobile communication module 150 may be disposed in the processor 110 .
  • at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be provided in the same device.
  • a modem processor may include a modulator and a demodulator.
  • the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal.
  • the demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal.
  • the demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing.
  • the application processor outputs sound signals through audio devices (not limited to speaker 170A, receiver 170B, etc.), or displays images or videos through display screen 194.
  • the modem processor may be a stand-alone device.
  • the modem processor may be independent of the processor 110 and may be provided in the same device as the mobile communication module 150 or other functional modules.
  • the wireless communication module 160 can provide applications on the electronic device 100 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), Bluetooth (bluetooth, BT), and global navigation satellites.
  • WLAN wireless local area networks
  • System global navigation satellite system, GNSS
  • frequency modulation frequency modulation, FM
  • near field communication technology near field communication, NFC
  • infrared technology infrared, IR
  • the wireless communication module 160 may be one or more devices integrating at least one communication processing module.
  • the wireless communication module 160 receives electromagnetic waves via the antenna 2 , demodulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 .
  • the wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation.
  • the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology.
  • the wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), broadband Code division multiple access (wideband code division multiple access, WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (long term evolution, LTE), BT, GNSS, WLAN, NFC , FM, and/or IR technology, etc.
  • the GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), Beidou navigation satellite system (BDS), quasi-zenith satellite system (quasi -zenith satellite system, QZSS) and/or satellite based augmentation systems (SBAS).
  • GPS global positioning system
  • GLONASS global navigation satellite system
  • BDS Beidou navigation satellite system
  • QZSS quasi-zenith satellite system
  • SBAS satellite based augmentation systems
  • the electronic device 100 implements display functions through a GPU, a display screen 194, an application processor, and the like.
  • the GPU is an image processing microprocessor and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering.
  • Processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.
  • the display screen 194 is used to display images, videos, etc.
  • Display 194 includes a display panel.
  • the display panel can use a liquid crystal display (LCD).
  • the display panel can also use organic light-emitting diode (OLED), active matrix organic light-emitting diode or active matrix organic light-emitting diode (active-matrix organic light emitting diode, AMOLED), flexible light-emitting diode ( Manufacturing of flex light-emitting diodes (FLED), miniled, microled, micro-oled, quantum dot light emitting diodes (QLED), etc.
  • the electronic device may include 1 or N display screens 194, where N is a positive integer greater than 1.
  • the display screen 194 can display a preview interface provided by the camera application, display a transparent or blurred picture containing object A in the preview interface, and simultaneously display the camera 193 of the electronic device. Collected images.
  • the display screen 194 can display a composite image obtained by fitting the image captured by the camera and the picture containing object A by the processor 110, so that the group photo effect can be presented to the user.
  • the electronic device 100 can implement the shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, an application processor, and the like.
  • the ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera sensor through the lens, the optical signal is converted into an electrical signal, and the camera sensor passes the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise and brightness. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be provided in the camera 193.
  • Camera 193 is used to capture still images or video.
  • the object passes through the lens to produce an optical image that is projected onto the photosensitive element.
  • the photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor.
  • CMOS complementary metal-oxide-semiconductor
  • the photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to convert it into a digital image signal.
  • ISP outputs digital image signals to DSP for processing.
  • DSP converts digital image signals into standard RGB, YUV and other format image signals.
  • the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.
  • the electronic device 100 may have multiple cameras 193, such as a front camera and a rear camera.
  • the number of front cameras and rear cameras can be multiple.
  • the types of cameras 193 may include but are not limited to wide-angle cameras, ultra-wide-angle cameras, telephoto cameras, etc.
  • the hardware configuration and physical location of the camera 193 may be different. Therefore, the size, range, content or clarity of images collected by different cameras may be different.
  • the image output sizes of the cameras 193 can be different or the same.
  • the output size of a camera refers to the length and width of the image captured by the camera. Both the length and width of the image can be measured in pixels.
  • the camera's output size may also be called image size, image size, pixel size or image resolution.
  • Common camera image ratios include: 4:3, 16:9 or 3:2, etc.
  • the image ratio refers to the approximate ratio of the number of pixels in the length and width of the image captured by the camera.
  • the cameras 193 can correspond to the same focal length or different focal lengths.
  • the field of view refers to the angular range that the optical system can image.
  • the cameras 193 can be disposed on both sides of the electronic device.
  • the camera located on the same plane as the display screen 194 of the electronic device may be called a front camera, and the camera located on the plane of the back cover of the electronic device may be called a rear camera.
  • the front-facing camera can be used to capture the image of the photographer facing the display screen 194, and the rear-facing camera can be used to capture images of the shooting objects (such as people, scenery, etc.) that the photographer is facing.
  • camera 193 may be used to collect depth data.
  • the camera 193 may have a time of flight (TOF) 3D sensing module or a structured light (structured light) 3D sensing module for acquiring depth information.
  • the camera used to collect depth data can be a front-facing camera or a rear-facing camera.
  • Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.
  • Video codecs are used to compress or decompress digital video.
  • Electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple encoding formats, such as moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.
  • MPEG moving picture experts group
  • MPEG2 MPEG2, MPEG3, MPEG4, etc.
  • NPU is a neural-network (NN) computing processor. It draws on the structure of biological neural networks, such as human brain nerves. Transmission mode between elements, fast processing of input information, and continuous self-learning. Intelligent cognitive applications of the electronic device 100 can be implemented through the NPU, such as image recognition, face recognition, speech recognition, text understanding, etc.
  • NN neural-network
  • the processor 110 in a co-photographed scene, can fit the image captured by the camera and the picture containing object A, and store the fitting result in the memory.
  • the fitting operation performed by the processor 110 please refer to the detailed introduction of subsequent method embodiments.
  • the synthetic image obtained by fitting contains both object A and object B captured by the camera, realizing a group photo of multiple objects.
  • the processor 110 (such as a controller or GPU) can perform fitting processing on the moment when the electronic device 100 receives the photographing instruction and the previous N frames of images captured by the camera 193, respectively, and the image containing the object A, to obtain N frames of synthesized images are then selected and the frame with the best quality is selected from these N frames of synthesized frames and stored in the memory as a group photo result.
  • the processor 110 in a co-photography scene, can fit multiple frames of images captured by the camera to a picture containing object A, and store the fitting results in the memory.
  • the multi-frame images captured by the camera and the picture containing object A are fitted into multiple images, and the video encoder in the processor 110 can encode the multiple composite images to generate a video file.
  • each frame of the video file can contain object A and the object newly captured by the camera in the image.
  • the internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).
  • RAM random access memories
  • NVM non-volatile memories
  • the internal memory 121 is used to store the fitting results of the processor 110 .
  • Random access memory can include static random-access memory (SRAM), dynamic random-access memory (DRAM), synchronous dynamic random-access memory (SDRAM), double data rate synchronous Dynamic random access memory (double data rate synchronous dynamic random access memory, DDR SDRAM, for example, the fifth generation DDR SDRAM is generally called DDR5SDRAM), etc.; non-volatile memory can include disk storage devices and flash memory (flash memory).
  • SRAM static random-access memory
  • DRAM dynamic random-access memory
  • SDRAM synchronous dynamic random-access memory
  • DDR SDRAM double data rate synchronous Dynamic random access memory
  • non-volatile memory can include disk storage devices and flash memory (flash memory).
  • Flash memory can be divided according to the operating principle to include NOR FLASH, NAND FLASH, 3D NAND FLASH, etc.
  • the storage unit potential level it can include single-level storage cells (single-level cell, SLC), multi-level storage cells (multi-level cell, MLC), third-level storage unit (triple-level cell, TLC), fourth-level storage unit (quad-level cell, QLC), etc., which can include universal flash storage (English: universal flash storage, UFS) according to storage specifications. , embedded multi media card (embedded multi media Card, eMMC), etc.
  • the random access memory can be directly read and written by the processor 110, can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data, etc.
  • the non-volatile memory can also store executable programs and user and application program data, etc., and can be loaded into the random access memory in advance for direct reading and writing by the processor 110.
  • the external memory interface 120 can be used to connect an external non-volatile memory to expand the storage capacity of the electronic device 100 .
  • the external non-volatile memory communicates with the processor 110 through the external memory interface 120 to implement the data storage function. For example, save music, video and other files in external non-volatile memory.
  • the electronic device 100 can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone interface 170D, and the application processor. Such as music playback, recording, etc.
  • Touch sensor 180K also known as "touch device”.
  • the touch sensor 180K can be disposed on the display screen 194.
  • the touch sensor 180K and the display screen 194 form a touch screen, which is also called a "touch screen”.
  • the touch sensor 180K is used to detect a touch operation on or near the touch sensor 180K.
  • the touch sensor can pass the detected touch operation to the application processor to determine the touch event type.
  • Visual output related to the touch operation may be provided through display screen 194 .
  • the touch sensor 180K may also be disposed on the surface of the electronic device 100 at a location different from that of the display screen 194 .
  • the software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture.
  • This embodiment of the present application takes a mobile operating system with a layered architecture as an example to illustrate the software structure of the electronic device 100 .
  • FIG. 1B is a software structure block diagram of the electronic device 100 according to the embodiment of the present application.
  • the layered architecture divides the software into several layers, and each layer has clear roles and division of labor.
  • the layers communicate through software interfaces.
  • the mobile operating system is divided into four layers, from top to bottom: application layer, program framework layer/core service layer, underlying library and runtime, and kernel layer.
  • the application layer can include a series of application packages.
  • the application package can include camera, gallery, calendar, calling, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.
  • the program framework layer provides application programming interfaces (application programming interface, API) and programming frameworks for applications in the application layer.
  • the program framework layer includes some predefined functions.
  • the program framework layer can include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.
  • a window manager is used to manage window programs.
  • the window manager can obtain the display size, determine whether there is a status bar, lock the screen, capture the screen, etc.
  • Content providers are used to store and retrieve data and make this data accessible to applications.
  • Said data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.
  • the view system includes visual controls, such as controls that display text, controls that display pictures, etc.
  • a view system can be used to build applications.
  • the display interface can be composed of one or more views.
  • a display interface including a text message notification icon may include a view for displaying text and a view for displaying pictures.
  • Telephone managers are used to provide communication functions of electronic devices. For example, call status management (including connected, hung up, etc.).
  • the resource manager provides various resources to applications, such as localized strings, icons, pictures, layout files, video files, etc.
  • the notification manager allows applications to display notification information in the status bar, which can be used to convey notification-type messages and can automatically disappear after a short stay without user interaction.
  • the notification manager is used to notify download completion, message reminders, etc.
  • the notification manager can also be notifications that appear in the status bar at the top of the system in the form of charts or scroll bar text, such as notifications for applications running in the background, or notifications that appear on the screen in the form of conversation windows. For example, text information is prompted in the status bar, a beep sounds, the electronic device vibrates, the indicator light flashes, etc.
  • Runtime can refer to all code libraries, frameworks, etc. required when the program is running.
  • the runtime includes a series of function libraries required to run C programs.
  • the runtime in addition to the core library, the runtime also includes the virtual machine required to run the Java program.
  • the above-mentioned core library may include functional functions that need to be called by the Java language.
  • the underlying library can include multiple functional modules. For example: surface manager (surface manager), media libraries (Media Libraries), 3D graphics processing libraries (for example: OpenGL ES), 2D graphics engines (for example: SGL), etc.
  • surface manager surface manager
  • media libraries Media Libraries
  • 3D graphics processing libraries for example: OpenGL ES
  • 2D graphics engines for example: SGL
  • the surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications.
  • the media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc.
  • the media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.
  • the 3D graphics processing library is used to implement 3D graphics drawing, image rendering, composition, and layer processing.
  • 2D Graphics Engine is a drawing engine for 2D drawing.
  • the kernel layer is the layer between hardware and software.
  • the kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.
  • the workflow of the electronic device software and hardware is exemplified below by combining the capture of the photographing scene.
  • the corresponding hardware interrupt is sent to the kernel layer.
  • the kernel layer processes touch operations into raw input events (including touch coordinates, timestamps of touch operations, and other information). Raw input events are stored at the kernel level.
  • the application framework layer obtains the original input event from the kernel layer and identifies the control corresponding to the input event. Taking the touch operation as a touch click operation and the control corresponding to the click operation as a camera application icon control as an example, the camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer. Camera 193 captures still images or video.
  • FIG. 1C is another software structure block diagram of the electronic device 100 provided by the embodiment of the present application.
  • the electronic device 100 includes: a gallery module, a camera module, a display module, a recognition module, and a synthesis module. in:
  • the gallery module is used to store multiple pictures, including a picture containing object A.
  • the gallery module can also provide an entrance to enable the "co-shooting" function, and can be used to store the fitting results of the electronic device 100 .
  • the camera module is responsible for taking images.
  • the display module is used to display the image with transparency or blur, and at the same time display the image captured by the camera module.
  • the display module is used to display a composite image fitted by the image captured by the camera module and the image containing object A.
  • the identification module is used to identify the area where object B is located in the image captured by the camera module, and the background area other than object B.
  • the recognition module is used to identify the area where object A is located in the picture containing object A, and the background area other than the area where object A is located.
  • the synthesis module is used to fit the area where object B is located in the image captured by the camera module to the picture containing object A.
  • the synthesis module is used to fit the area where object A is located in the picture containing object A to the image captured by the camera module.
  • 2A-2H exemplarily illustrate a set of user interfaces for taking a group photo provided by the electronic device 100 in the embodiment of the present application.
  • FIG. 2A illustrates an exemplary user interface 21 on the electronic device 100 for displaying installed applications.
  • the user interface 21 displays: a status bar 201, a calendar and time indicators 202, a weather indicator 203, a page indicator 204, a tray 205 with commonly used application icons, and other application icons. in:
  • the status bar 201 may include: one or more signal strength indicators for mobile communication signals (also known as cellular signals), a Bluetooth indicator, one or more signal strength indicators for Wi-Fi signals, a battery status indicator, Time indicators and more.
  • Calendar and time indicators 202 are used to indicate calendar and current time.
  • Weather indicator 203 is used to indicate weather.
  • the page indicator 204 may be used to indicate the application icon in which page the user is currently browsing.
  • application icons can be distributed on multiple pages, and the user can slide left and right to browse application icons in different pages.
  • the tray 205 with commonly used application icons can display: phone icon, text message icon, camera icon 205a, address book icon, etc.
  • Other application icons may include, for example, a video application icon, an online meeting application icon, a calendar icon, a game application icon, a map application icon, a settings application icon, a gallery application icon 206a, and so on.
  • the camera is an APP on the electronic device 100 that is used to call the camera to capture images. This embodiment does not limit the name of the application.
  • the camera APP in the embodiment of the present application also provides a "co-photography" function.
  • the camera APP may be a system application provided by the electronic device 100 or a third-party application.
  • Gallery is a picture management APP on electronic devices 100 such as smartphones and tablets. It can also be called a "photo album". This embodiment does not limit the name of the application.
  • the APP can support users to perform various operations on pictures stored on the electronic device 100, such as browsing, editing, deleting, selecting and other operations. That is, the objects managed by "Gallery” are pictures. In other cases, the APP can also support users to perform the above operations on pictures stored on the cloud server. It can be understood that in this embodiment, the picture may be captured by the electronic device 100 using the camera 193, or may be obtained from other applications or downloaded from a web page.
  • the gallery in the embodiment of this application also provides access to the "co-photography" function.
  • the gallery may be a system application provided by the electronic device 100 or a third-party application.
  • more applications can be installed in the electronic device 100, and icons of these applications can be displayed on the display screen.
  • the electronic device 100 may also be installed with shopping applications, ticket booking applications, and so on.
  • Shopping apps can be used to purchase items online.
  • Ticket booking applications can be used to order various types of tickets, such as air tickets, train tickets, boat tickets, etc.
  • the user interface 21 shown in FIG. 2A may also include a navigation bar, a sidebar, and the like.
  • the user interface 21 illustrated in FIG. 2A may be called a home screen.
  • FIG. 2B exemplarily shows the user interface 22 provided by the gallery APP of the electronic device 100.
  • FIG. 2B may be a user interface displayed in response to the user operation after the electronic device 100 detects a user operation (such as a click operation, a touch operation) on the icon 206a of the gallery application in FIG. 2A.
  • a user operation such as a click operation, a touch operation
  • the user interface 22 may include: a status bar, an application title bar, and a picture area 207 . in:
  • the application title bar can include: a return key, and a current page indicator.
  • the return key is an APP-level return key, which can be used to return to the previous level of the menu.
  • the current page indicator can be used to indicate the current page, such as text information "Gallery”. It is not limited to text information.
  • the current page indicator can also be an icon.
  • One or more pictures may be displayed in the picture area 207, such as picture 207a.
  • the electronic device may update the pictures displayed in the picture area 207 to facilitate the user to browse the pictures. That is to say, the user can slide up or down in the picture area 207 to browse more pictures. Not limited to the upward or downward sliding operation, the user can also slide left or right in the picture area 207 to browse more pictures.
  • the pictures displayed in the picture area 207 may be thumbnails.
  • the original image corresponding to the image displayed in the image area 207 can be stored on the electronic device or on the cloud server.
  • the pictures referred to in the following embodiments may be stored in electronic devices or on a cloud server.
  • the electronic device 100 can detect a user operation (such as a click operation, a touch operation, etc.) on the picture 207a in the user interface 22 . In response to this operation, the electronic device 100 may display the user interface 23 as shown in FIG. 2C.
  • a user operation such as a click operation, a touch operation, etc.
  • the electronic device 100 can display the original image corresponding to the picture 207a in a larger area in the user interface 23.
  • the user interface Picture 208 is displayed on page 23, and picture 208 may be the original picture of picture 207a.
  • the picture 208 may be taken by the electronic device 100 at a certain location (for example, location 1), or may be obtained by the electronic device 100 from the network.
  • the picture 208 contains a human object and a background area.
  • the human object may be called object A
  • the picture 208 may be called a picture containing object A.
  • the background area shows the location of the object A.
  • the background area of the picture 208 shown in FIG. 2C shows coconut trees, seagulls, hillsides, and the sun.
  • the user interface 23 may also display: sharing controls, collection controls, editing controls, deleting controls, more controls 209, etc.
  • the sharing control, collection control, editing control, and deletion control are used to monitor user operations.
  • the electronic device 100 can perform operations such as sharing, collection, editing, and deletion on the picture 208.
  • More controls 209 can be used to monitor user operations (such as click operations, touch operations, etc.), and the electronic device 100 can display a function list 210 as shown in FIG. 2D in response to the user operations.
  • the function list 210 displays multiple function controls, such as controls for copying the picture 208 to the album, moving it to the album, casting, printing, etc.
  • the multiple function controls displayed in the function list 210 include a control 210a for turning on the "co-photography" function for the picture 208.
  • the control 210a is used to monitor user operations (such as click operations, touch operations, etc.), and the electronic device 100 can respond to This user operation activates the "co-photography" function for picture 208.
  • the user can click on the picture to view the picture, and then activate the "co-photography" function for the picture through the entrance provided by the user interface that displays the picture.
  • the electronic device 100 can also initiate a "co-photography" function for the picture in response to other user operations.
  • the electronic device 100 can also receive a long press operation on the picture 207a shown in Figure 2B and then activate the "co-photography" function of the picture, or the electronic device 100 can also receive a circle drawing on the picture 207a shown in Figure 2B
  • the "co-photography” function of the picture is activated, or the electronic device 100 can also activate the "co-photography” function of the picture after receiving a long press operation on the picture 208 shown in Figure 2C, or the electronic device 100 can also It can be a voice control command.
  • This application does not limit the specific implementation of the operation of triggering the electronic device 100 to activate the "co-photography" function for pictures.
  • the scenarios in which the user triggers the "co-shoot” function for pictures can also include triggering the "co-shoot” function for the picture when the user views pictures on the Internet, and the user triggering the "co-shoot” function for the picture in applications such as file browsers.
  • 2E-2H exemplarily illustrate a set of user interfaces provided by the camera application after the electronic device 100 turns on the "co-photography" function for the picture 208.
  • the electronic device 100 After the electronic device 100 activates the "co-photography" function for the picture 208, it can automatically start the camera application.
  • FIG. 2E shows the user interface 24 displayed after the electronic device 100 starts the camera application.
  • the user interface 24 exemplarily shown in FIG. 2E may be displayed after the electronic device 100 detects a user operation (such as a click operation, a touch operation, etc.) acting on the control 210a as shown in FIG. 2D.
  • the user interface 24 may be a user interface of the default photo mode of the camera application, which may be used by the user to take photos through the default rear camera.
  • the user interface 24 may include: an area 211 , a shooting mode list 212 , a control 213 , a control 214 and a control 215 . in:
  • the area 211 may be called a preview frame 211 or a viewfinder frame 211.
  • the preview frame 211 can be used to display a picture 208 with a certain degree of transparency or blur.
  • the preview box 211 may also display the images collected by the camera 193 in real time.
  • the electronic device 100 can refresh the display content thereof in real time to facilitate the user to preview the image currently collected by the camera 193 .
  • the electronic device 100 can superimpose and display the picture 208 with a certain degree of transparency or blur on the image collected by the camera 193 in real time. Since the picture 208 has a certain degree of transparency and/or blur, the user can not only see it in the preview box 211 In picture 208, you can also see the real-time images collected by the camera 193 under the picture 208.
  • the electronic device 100 after the electronic device 100 activates the "co-photography" function and launches the camera application, it can first display only the picture 208 with a certain transparency or blur in the preview box 211, as shown in Figure 2E, and then start the camera 193 And the image collected in real time is also displayed in the preview box 211, as shown in Figure 2F.
  • the electronic device 100 can directly start the camera 193 to collect images. At this time, the electronic device 100 can directly display a certain transparency or blur in the preview box 211 at the same time. The picture 208 and the image collected by the camera 193 in real time. That is to say, after receiving the user operation on the control 210a in FIG. 2E, the electronic device 100 can directly display the user interface shown in FIG. 2F without displaying the user interface shown in FIG. 2E.
  • One or more shooting mode options may be displayed in the shooting mode list 212 .
  • the one or more camera options may include: photo mode option 212A, video mode option 212B, portrait mode option 212C, panorama mode option 212D, and more options 212E.
  • This one or more camera options can be represented as text information on the interface, such as "photograph”, “video”, “portrait”, “panorama” and “more”. not limited to
  • the one or more camera options may also be represented as icons or other forms of interactive elements (IE) on the interface.
  • IE interactive elements
  • the control 213 can be used to monitor user operations that trigger shooting (photography or video recording).
  • the electronic device can detect the user operation on the control 213 (such as a click operation on the control 213).
  • the electronic device 100 can save the image collected by the camera 193 and perform a fitting operation with the picture 208, thereby Generate pictures from the gallery.
  • the control 213 can be changed to the corresponding video recording control.
  • the electronic device can detect the user operation on the video recording control.
  • the electronic device 100 can save the multi-frame images collected by the camera 193 and Perform fitting operations on it and picture 208 respectively to generate the video in the gallery.
  • the electronic device 100 can also display a thumbnail of the saved image in the control 214.
  • the image can be generated by the electronic device 100 after the "co-photography" function is turned on, or it can be generated without the "co-photography” function being turned on. of. That is to say, the user can click the control 213 or the video recording control to trigger shooting.
  • the control 213 or the video recording control may be a button or other form of control.
  • the control 213 can be called a camera control. Photographing controls and video recording controls can be collectively referred to as shooting controls.
  • Control 215 may be used to listen for user operations that trigger flipping the camera.
  • the electronic device 100 can detect a user operation on the control 215 (such as a click operation on the control 215), and in response to the operation, the electronic device 100 can flip the camera, for example, switch the rear camera to a front camera.
  • the electronic device 100 can detect a user operation acting on the shooting mode option, and the user operation can be used to select the shooting mode. In response to the operation, the electronic device 100 can turn on the shooting mode selected by the user. In particular, when the user operation is applied to more shooting mode options 212E, the electronic device 100 can further display more other shooting mode options, such as slow-motion shooting mode options, etc., and can display richer camera functions to the user. Not limited to what is shown in FIG. 2E , more shooting mode options 212E may not be displayed in the shooting mode list 212 , and the user can browse other shooting mode options by sliding left/right in the shooting mode list 212 .
  • a lighter color image and a darker color image are simultaneously displayed in the preview box 211, where the lighter color image is a picture 208 with a certain degree of transparency or blurred, and the darker color image is Images collected by camera 193 in real time.
  • the position of the picture 208 with a certain degree of transparency or blur remains unchanged.
  • the user can change the view of the currently used camera in the preview frame by moving the electronic device. That is to say, the user can change the view of the camera 193 by moving the electronic device, thereby changing the image captured by the camera 193 in real time displayed in the preview box 211 .
  • the user can move the electronic device to the left or right, forward or backward, and can also walk around holding the electronic device.
  • the user can also adjust the focus of the currently used camera (such as electronic focus), or change the currently used camera (such as changing the rear camera to the front camera), etc., thereby changing the preview frame
  • the image collected by the camera is displayed in real time in 211.
  • the human objects, animal objects, etc. captured by the camera 193 can also be instructed to change their positions, thereby adjusting the framing in the preview frame 211 .
  • the preview box 211 can show the user the relative position of each object in the picture 208 and the image collected by the camera 193 in real time. According to the content presented in the preview box 211, the user can adjust the image captured by the camera displayed in real time in the preview box 211, so that the image captured by the camera contains object B and the object B is in a position that is satisfactory to the user and is consistent with object A in the picture 208. a relative position.
  • object B The object that needs to take a photo with object A.
  • the picture 208 includes object A and a background area.
  • the background area shows the location of the object A.
  • the background area of the picture 208 displayed in the preview box 211 in Figure 2F shows that there is a coconut in the location of the object A. Trees, seagulls, hillsides, sun.
  • the control 213 when the position indicated by the background area of the real-time image collected by the camera displayed in the preview box 211, and the position indicated by the picture 208 (such as but not limited to the position indicated by the background area of the picture 208), Only when there is a certain correlation, the control 213 can be activated. In other cases, even if the control 213 receives a user operation, the electronic device 100 will not perform subsequent responses.
  • the user can click the control 213 when the image collected by the camera contains object B and the object B is in a satisfactory relative position to the object A in the picture 208, thereby triggering the electronic device 100 to save the image collected by the camera 193. And perform fitting operation with picture 208. That is, only when the position indicated by the picture 208 and the position indicated by the image collected in real time by the camera 193 of the current electronic device 100 have a certain correlation, the user can trigger the electronic device 100 to capture the image and perform subsequent fitting operations. .
  • control 213 can be activated all the time, as long as the user allows the image captured by the camera to include object B and the object B is in a relative position to the object A in the picture 208 that satisfies the user. Click the control 213 to trigger the electronic device 100 to save the image collected by the camera 193 and perform a fitting operation with the picture 208 .
  • the image captured by the camera 193 includes a human object, and the human object may be object B. Since the user is not satisfied with the relative positions of object B and object A in Figure 2F, the user adjusts the framing of the camera 193 in the preview frame 211, thereby adjusting the object B and object A in the preview frame 211 as shown in Figure 2G at a satisfactory relative position.
  • the electronic device 100 can detect a user operation (such as a click operation, a touch operation, etc.) on the control 213 , and in response to the operation, the electronic device 100 can save the image collected by the camera 193 and combine it with the picture 208 Perform fitting operations to generate Pictures in the gallery.
  • a user operation such as a click operation, a touch operation, etc.
  • the electronic device 100 can save the image collected by the camera 193 and combine it with the picture 208 Perform fitting operations to generate Pictures in the gallery.
  • this fitting operation please refer to the detailed introduction in subsequent method embodiments.
  • the electronic device 100 can also display the thumbnail of the image generated after the fitting operation in the control 214 for the user to click and view.
  • the fitting may fail and the corresponding composite image cannot be generated.
  • the electronic device 100 when the electronic device 100 fits the image collected by the camera 193 with the picture 208, the fitting may fail and the corresponding composite image cannot be generated. For example, when the correlation between the background area of the image collected by the camera 193 and the picture 208 is low, the electronic device 100 will not be able to successfully perform the fitting operation.
  • the electronic device 100 can continue to turn on the "co-photography" function, so that the user can take photos multiple times to achieve the purpose of multiple group photos.
  • the electronic device can also turn off the "co-photography” function and enter a normal photo-taking mode.
  • the user is not limited to the photographing mode shown in FIGS. 2E to 2G .
  • the user can also select the video recording mode or other modes.
  • the electronic device 100 can also shoot in the video recording mode or other modes to achieve the purpose of taking a group photo in the video or other modes.
  • 3A to 3F exemplarily illustrate another set of user interfaces for taking a group photo provided by the electronic device 100 in the embodiment of the present application.
  • the user interface 31 shown in Figure 3A is the same as the user interface 21 shown in Figure 2A. Please refer to the previous Figure 2A and related descriptions.
  • FIG. 3B schematically illustrates the user interface 32 provided by the camera application of the electronic device 100 .
  • FIG. 3B may be a user interface in which the electronic device 100 detects a user operation (such as a click operation or a touch operation) on the icon 205a of the camera application in FIG. 3A and then launches and displays the camera application in response to the user operation.
  • the user interface 32 may be a user interface of a default photography mode of the camera application, which may be used by the user to take photos through the default rear camera.
  • the user interface 32 may include: an area 301, a shooting mode list 302, a control 303, a control 304 and a control 305, and a control 306 for activating the "co-shooting" function. in:
  • the area 301 may be called a preview frame 301 or a viewfinder frame 301.
  • the preview frame 301 may be used to display images captured by the camera 193 of the electronic device 100 in real time.
  • the electronic device 100 can refresh the display content thereof in real time to facilitate the user to preview the image currently collected by the camera 193 .
  • One or more shooting mode options may be displayed in the shooting mode list 302 .
  • the one or more camera options may include: photo mode option 302A, video mode option 302B, portrait mode option 302C, panorama mode option 302D, and more options 302E.
  • the control 303 can be used to monitor user operations that trigger shooting (photography or video recording).
  • the electronic device may detect a user operation on the control 303 (such as a click operation on the control 303), and in response to the operation, the electronic device 100 may save the image displayed in the preview box 211.
  • the control 303 can be changed to the corresponding video recording control.
  • the electronic device can detect the user operation on the video recording control.
  • the electronic device 100 can save the image displayed in the preview box 211.
  • the electronic device 100 may also display the thumbnail of the saved image in the control 304 . That is to say, the user can click the control 303 or the video recording control to trigger shooting.
  • the control 303 or the video recording control may be a button or other form of control.
  • the control 303 can be called a camera control. Photographing controls and video recording controls can be collectively referred to as shooting controls.
  • Control 305 may be used to monitor user operations that trigger flipping the camera.
  • the electronic device 100 can detect a user operation on the control 305 (such as a click operation on the control 305), and in response to the operation, the electronic device 100 can flip the camera, for example, switch the rear camera to a front camera.
  • the electronic device 100 can detect a user operation acting on the shooting mode option, and the user operation can be used to select the shooting mode. In response to the operation, the electronic device 100 can turn on the shooting mode selected by the user.
  • the control 306 can be used to monitor user operations (such as click operations, touch operations, etc.), and the electronic device 100 can display the picture area 307 as shown in FIG. 3C in the user interface 33 in response to the user operations.
  • One or more pictures may be displayed in the picture area 307, such as picture 307a.
  • the electronic device may update the pictures displayed in the picture area 307 to facilitate the user to browse the pictures. That is to say, the user can slide up or down in the picture area 307 to browse more pictures. Not limited to the upward or downward sliding operation, the user can also slide left or right in the picture area 307 to browse more pictures.
  • the pictures displayed in the picture area 307 may be thumbnails. At this time, the original image corresponding to the image displayed in the image area 307 can be stored on the electronic device or on the cloud server.
  • Prompt information 307b may also be displayed in the picture area 307.
  • the prompt information 307b is used to prompt the user to select a picture from the pictures displayed in the picture area 307 as a joint photo.
  • the electronic device 100 can detect a user operation (such as a click operation, a touch operation, etc.) on the picture 307a, and in response to the operation, activate the "co-photography" function of the original picture corresponding to the picture 307a, and display the image shown in Figure 3D.
  • a user operation such as a click operation, a touch operation, etc.
  • the user interface shown is 34.
  • the electronic device 100 can change the display form of the control 306 (for example, it can change its display color, change its display shape, or change the hollow circle to a solid circle as shown in Figure 3D etc.) to prompt the user of the current electronic device 100The "co-production" function has been turned on.
  • the preview box 301 displays the picture 208 with a certain degree of transparency or blur, and the image collected by the camera 193 in real time.
  • the picture 208 may be the original picture corresponding to the picture 307a shown in FIG. 3C.
  • the electronic device 100 can refresh the display content thereof in real time to facilitate the user to preview the image currently collected by the camera 193 .
  • the electronic device 100 can superimpose and display the picture 208 with a certain degree of transparency or blur on the image collected by the camera 193 in real time. Since the picture 208 has a certain degree of transparency and/or blur, the user can not only see it in the preview box 211 In picture 208, you can also see the real-time images collected by the camera 193 under the picture 208.
  • the image captured by the camera 193 includes a human object, and the human object may be object B. Since the user is not satisfied with the relative positions of object B and object A in Figure 3D, the user adjusts the framing of the camera 193 in the preview frame 301, thereby adjusting the object B and object A in the preview frame 301 as shown in Figure 3E at a satisfactory relative position.
  • the electronic device 100 can detect a user operation (such as a click operation, a touch operation, etc.) on the control 303. In response to the operation, the electronic device 100 can save the image collected by the camera 193 and combine it with the picture 307a. The corresponding original image is fitted to generate the image in the gallery. For the specific implementation of this fitting operation, please refer to the detailed introduction in subsequent method embodiments.
  • the electronic device 100 can also display the thumbnail of the image generated after the fitting operation in the control 304 for the user to click and view.
  • the fitting may fail and the corresponding composite image cannot be generated.
  • the electronic device 100 when the electronic device 100 fits the image collected by the camera 193 with the original image corresponding to the picture 307a, the fitting may fail and the corresponding composite image cannot be generated.
  • the electronic device 100 when the correlation between the background area of the image collected by the camera 193 and the original image corresponding to the picture 307a is low, the electronic device 100 will not be able to successfully perform the fitting operation.
  • 4A-4B exemplarily illustrate the user interface provided by the gallery application after the electronic device 100 saves the fitted picture in the embodiment of the present application.
  • FIG. 4A may be a user interface displayed in response to the user operation after the electronic device 100 detects a user operation (such as a click operation, a touch operation) on the icon 206a of the gallery application in FIG. 2A or FIG. 3A .
  • the user interface shown in FIG. 4A is similar to the user interface 22 shown in FIG. 2B, and relevant descriptions may be referred to.
  • a new picture 207 b is added to the picture area 207 shown in the user interface 41 .
  • the picture 207 b may be a thumbnail or an original picture of the fitted picture saved by the electronic device 100 .
  • the electronic device 100 can respond to the operation and display it in a larger area in the user interface shown in Figure 4B.
  • the original image corresponding to picture 207b for example, picture 401.
  • the user interface 43 shown in FIG. 4B may also be displayed by the electronic device 100 in response to a user operation on the control 214 shown in FIG. 2H or the control 304 shown in FIG. 3F.
  • the picture 207b shown in FIG. 4A or the picture 401 shown in FIG. 4B may also be displayed with a "co-photographed" logo to indicate that the picture is the electronic device 100 turning on the "co-photographed” function. Taken later.
  • the embodiment of the present application does not limit the implementation form of the "co-production" logo, for example, it may be text, icon, etc.
  • the shooting method provided by the embodiment of the present application is introduced below.
  • FIG. 5 exemplarily shows the flow of the shooting method.
  • the shooting method shown in Figure 5 is explained by taking the shooting scene as an example.
  • the shooting method may include the following steps:
  • the electronic device 100 detects a first operation for turning on the "co-photography" function for the first picture.
  • the electronic device 100 in response to the first operation, turns on the "co-photography" function for the first picture, and the first picture includes object A.
  • the first picture may be a picture taken by the camera 193 saved by the electronic device 100, or may be a picture downloaded by the electronic device 100 from the network, or may be a received picture shared by other devices.
  • the first picture may be a picture taken a long time ago, or may be a picture just taken by the electronic device 100 .
  • the first picture contains an object with which the user wants to take a photo, which may be called object A.
  • the number of object A can be one or more.
  • Object A may include, for example, but is not limited to: characters, animals, landmarks, buildings, dolls, etc.
  • the first picture can also be an old photo taken by a relative many years ago, etc.
  • the first picture may be the original picture corresponding to the picture 207a shown in FIG. 2C, the picture 208 shown in FIG. 2C, the original picture corresponding to the picture 307a shown in FIG. 3C, or the picture shown in FIG. 3D. 301.
  • Object A can be the human object in the above picture.
  • the "co-photography” function is provided by the electronic device 100.
  • This function supports the electronic device 100 to display the content in the preview interface provided by the camera application.
  • the first picture is transparent or blurred, and at the same time displays the image collected by the camera 193 of the electronic device 100.
  • the electronic device 100 receives the photography instruction input by the user, and fits the first picture with the image collected by the camera. Generate composite images. After turning on the "co-shooting" function, the electronic device will perform subsequent operations.
  • the first operation for enabling the "co-photography" function for the first picture may be, for example, on the user interface 23 displaying the picture 208 shown in FIG. 2C
  • the user operation acts on the control 210a in the function list 210 displayed on the electronic device 100.
  • the control 209 may also be called a first control
  • the control 210a may also be called a second control.
  • the first operation for turning on the "co-photography" function for the first picture may be, for example, a long press operation received by the electronic device 100 on the picture 207a shown in FIG. 2B, or the electronic device 100 may The circle gesture received on the picture 207a shown in FIG. 2B, or the long press operation received by the electronic device 100 on the picture 208 shown in FIG. 2C.
  • the user interface provided by the gallery application for displaying the first picture may be called the first user interface.
  • FIG. 2C and FIG. 2D may be called the first user interface.
  • the first operation for turning on the "co-photography" function for the first picture may be, for example, that the electronic device 100 receives an action on the control 306 in FIG. 3B After the user operation, the user operation is performed on the picture 307a in the picture area 307 displayed by the electronic device 100.
  • the control 306 may also be called a third control
  • the user operation performed on the control 306 may also be called a fourth operation
  • the user operation performed on the picture 307a may also be called a fifth operation.
  • the user interface shown in FIG. 3B can not only be provided by the camera application, but also can be a user interface provided by other applications in the electronic device (such as social applications, shopping applications, etc.) calling the camera interface. Equivalently, other applications in the electronic device can call the camera interface to enable the "co-photography" function.
  • the user interface provided by the camera application or provided by other applications calling the camera interface for enabling the "co-photography" function can be called the second user interface.
  • the first operation for turning on the "co-photography" function for the first picture can also be implemented as a voice control instruction, an air gesture instruction, a pressing operation on a physical button, etc. This application does not limit the specific implementation of the first operation.
  • the electronic device 100 starts the camera application and displays the second user interface.
  • the second user interface displays a preview box.
  • the preview box displays the first image with transparency and/or blur, and the image collected by the camera 193. .
  • the electronic device 100 After the electronic device 100 turns on the "co-photography" function for the first picture, it can automatically start the camera application.
  • the camera application may be a system application installed in the electronic device 100 or a third-party application.
  • the electronic device 100 can display the second user interface provided by the camera application.
  • the second user interface displays a preview box, and the first picture with transparency and/or blur is displayed in the preview box. And, the images collected by the camera 193 in real time.
  • the second user interface may also be called a preview interface.
  • the electronic device 100 can superimpose and display the first image with transparency and/or blur on the image collected by the camera 193 in real time.
  • the user can not only see the first picture, but also see other content covered by the first picture displayed by the electronic device 100 through the transparency.
  • the transparency of the first image can be preset, or can be adjusted by the user according to actual needs. It can also be adjusted according to the light of the current environment where the electronic device 100 is located, or according to the brightness of the image collected by the current camera 193 Make adjustments, etc.
  • the electronic device 100 may first identify the outline or content of the first picture, draw the outline or content with a dotted line, or blur the content to obtain the blurred first picture. In this case, the user can not only see the first picture, but also see other contents covered by the blurred first picture displayed by the electronic device 100 .
  • the images collected by the camera 193 in real time displayed in the preview box in the second user interface are refreshed in real time.
  • the images collected by the camera 193 will change in real time.
  • Preview The image changes in real time in the box.
  • the image captured by the camera 193 in real time displayed in the preview box may also be obtained by processing the original image captured by the camera 193 .
  • the electronic device 100 can perform anti-shake processing on the original image collected by the camera 193 and then send it to the display screen to be displayed in the preview box.
  • the electronic device 100 will crop the original image collected by the camera 193, and send the cropped result to the display screen to be displayed in the preview box.
  • the user can select the current shooting mode. In different modes, the electronic device 100 can perform different processing on the original image collected by the camera 193, such as blurring the background area in portrait mode, and then crop the image. The results are sent to the display for display in the preview box.
  • the second user interface may be, for example, the user interface 22 shown in FIGS. 2F to 2G
  • the preview box may be, for example, the preview box 211 in FIGS. 2F to 2G
  • the lighter colored picture 208 shown in Figures 2F-2G may be the specific image shown in the preview box.
  • the first image with transparency and the darker color image are images collected by the camera 193 in real time.
  • the second user interface may be, for example, the user interface 34 shown in FIGS. 3D to 3E
  • the preview box may be, for example, the preview box 301 in FIGS. 3D to 3E.
  • the lighter color picture 208 shown in FIG. 3D to FIG. 3E may be the first picture with transparency displayed in the preview box, and the darker color image is the image collected by the camera 193 in real time.
  • the preview box in the second user interface can show the relative position of each object in the first picture and the image collected by the camera 193 in real time to the user.
  • the user can adjust the image captured by the camera displayed in real time in the preview box based on the content presented in the preview box, so that the relative position of the object contained in the image captured by the camera and object A in the first picture meets the user's actual needs.
  • the way in which the user adjusts the image collected by the camera displayed in real time in the preview box may include but is not limited to: the user moves the electronic device 100 or changes the posture of the electronic device 100, the user changes the camera used by the electronic device 100, or the user indicates the current
  • the objects captured by the camera 193 such as people or animals change their location, etc. For example, referring to FIGS. 2F to 2G or 3D to 3E, the user can adjust the relative position between the human object contained in the image captured by the camera 193 and the object A in the first picture (ie, the human object in the picture). Location.
  • the electronic device 100 in addition to displaying the first picture with transparency or blur in the second user interface provided by the camera application to indicate the first picture, the electronic device 100 can also display lines of objects contained in the first picture. , the location of the object in the first picture, the faded first picture, etc., to display the first picture in the preview box, so that the user can know the relative relationship between the object in the first picture and the object in the image collected by the camera 193 in real time. Location.
  • the above-mentioned information used to prompt the user about the relative positions of the objects in the first picture and the objects in the image collected by the camera 193 in real time may be called indication information of the first picture.
  • the indication information of the first picture may include the first picture with transparency or blur displayed in the preview box, the lines of the objects contained in the first picture, the location of the objects in the first picture, the faded first picture, etc. etc., the embodiments of this application do not limit this.
  • the electronic device 100 fits the first picture and the image collected by the camera 193 to generate a composite image.
  • the second operation may be a user operation for triggering the electronic device 100 to take photos or videos.
  • the user operation used to trigger the electronic device 100 to take a photo may also be called a photo instruction.
  • the user operation used to trigger the recording of the electronic device 100 may also be called a recording instruction.
  • the second operation may be a user operation that acts on the camera control 213 , or it may be a user operation that acts on the video recording control after the user selects the video recording mode.
  • the second operation can also be a user operation that acts on the camera control 303, or a user operation that acts on the video recording control after the user selects the video recording mode.
  • the electronic device 100 only when the correlation between the position indicated by the image collected by the camera 193 displayed in the preview box of the second user interface and the position indicated by the first picture is greater than the first value, the electronic device 100 will respond to the second operation.
  • the correlation between pictures refers to the degree of similarity and matching degree of each picture in its relative position. If the correlation between two pictures is large, it means that the two pictures contain similar content, and the two pictures may have been taken from the same place and from a similar perspective.
  • the first value can be set in advance and is not limited here. For example, referring to the relevant descriptions of FIGS.
  • control 213 when the position indicated by the background area of the real-time image collected by the camera displayed in the preview box 211, and the position indicated by the picture 208 (such as but not limited to the background of the picture 208
  • the control 213 can be activated only when there is a certain correlation (the position indicated by the area). In other cases, even if the control 213 receives a user operation, the electronic device 100 will not perform subsequent responses.
  • the electronic device 100 after the electronic device 100 displays the second user interface, if the second operation is received, the electronic device 100 will immediately respond to the second operation.
  • the control 213 can be activated all the time, as long as the user allows the object contained in the image collected by the camera to be in a position relative to the object A in the picture 208 that satisfies the user.
  • the control 213 can be clicked, thereby triggering the electronic device 100 to perform subsequent steps in response to the operation.
  • This process may specifically include the following steps:
  • Step 1 Identify the object in the image collected by the camera. This object can be called object B.
  • the electronic device 100 can identify relatively iconic objects such as people, animals, landmarks, buildings, dolls, etc. in the images collected by the camera, and use them as objects B.
  • the electronic device 100 may also identify the object closest to the camera as object B based on the distance between each object in the image collected by the camera and the camera.
  • the user can also select an object as the object B in the image collected by the camera displayed in the preview box. For example, the user can click on the location of the desired object in the preview box, and then the electronic device 100 can automatically For the human body or scenery at that position in the circle, the user can select object B without manually drawing the outline of the person or object.
  • the electronic device 100 can identify the object B contained in the image collected by the camera through an image recognition algorithm.
  • the image recognition algorithm may be, for example, an artificial intelligence (AI) algorithm.
  • AI artificial intelligence
  • the number of objects B recognized by the electronic device 100 may be one or multiple, and is not limited here.
  • each (a) in FIGS. 6A and 6B shows a frame of image collected by the camera of the electronic device 100 .
  • the object B in the image recognized by the electronic device 100 is a male character in the figure.
  • the focal length used by the electronic device to capture (a) in Figure 6B is greater than the focal length used to capture (a) in Figure 6A .
  • Step 2 In the image collected by the camera, determine the main area containing object B based on object B.
  • the main area may be an area containing object B obtained by taking object B as a reference and extending a distance M to the surroundings in the image collected by the camera.
  • M can be an empirical value obtained through testing.
  • the M value of the same electronic device may be the same, the M values of different electronic devices may be different, or the M values corresponding to different cameras on the same electronic device may be different. This ensures that a main area containing object B can be obtained from images captured by different electronic devices or different cameras.
  • M may be a value or a set of values.
  • each value in the set of values may be different. That is to say, based on object B, the electronic device 100 may expand to different distances at different positions to obtain a main area containing object B.
  • the area enclosed by the dotted line is a possible main area containing object B.
  • the content contained in the images collected by the cameras in Figures 6A and 6B may be different in size and location.
  • the main area may be an area in the image collected by the camera that is extended to the surroundings based on object B and has a color or pixel that is similar or the same as object B.
  • Step 3 Perform a correlation search for the main area in the first image and find the apposition area with a correlation greater than the second value.
  • performing a correlation search for the main area in the first picture means first finding the background area other than the main area in the image collected by the camera, and then identifying the correlation between the first picture and the background area that is greater than the first area.
  • the binary area and/or the area with the highest correlation, the rest of the first picture excluding this area is the homogeneous area. This is equivalent to the correlation between the rest of the first picture except the co-located area and the rest of the image collected by the camera except the main area, which is greater than the second value.
  • the second value may be set in advance, which is not limited in the embodiments of this application.
  • This application does not limit the size, shape, and position of the apposition area in the first picture.
  • the position of the co-located area in the first picture is the same as the position of the main area containing object B in the image collected by the camera.
  • the electronic device 100 presents the object to the user in the preview box.
  • the relative position of A and object B is the same as the relative position of object A and object B in the composite image after fitting. This allows the user to adjust the photo effect in real time based on the content in the preview box.
  • the size and shape of the co-located area in the first picture and the size and shape of the main area containing object B in the image collected by the camera may be the same or different.
  • the size, shape, and position of the co-located area in the first picture and the main area in the image collected by the camera are all the same.
  • the co-located area in the first picture and the main area in the image collected by the camera have the same shape and position, and the size of the co-located area in the first picture is larger than that of the camera The size of the main area in the captured image.
  • step 3 if the electronic device 100 can find the co-located area corresponding to the main area in the first picture, it means that the geographical location where the first picture was taken is the same as or similar to the geographical location where the current electronic device 100 takes the image.
  • This enables group photos to be taken at the same location or in similar locations.
  • the light and background in the first picture are similar to the light and background of the image captured by the camera, which can greatly improve the fitting effect in the subsequent step 4, thereby achieving a better result.
  • Good photo effect This makes the shooting method provided by the embodiment of the present application different from simply cutting out pictures for a group photo, and this method can bring a more natural group photo effect.
  • step 3 may be executed due to the uncertainty of the image collected by the current camera. Failure, that is, the electronic device 100 cannot find the corresponding homogeneous area in the first picture. In this case, the electronic device 100 may output prompt information to prompt the user that the current synthesis fails.
  • Step 4 Fit the main area to the homogeneous area of the first picture, keep the remaining areas of the first picture unchanged, and generate a composite image.
  • the electronic device can enlarge or reduce the main area containing object B captured by the camera, then merge the main area into the co-located area of the first picture, and perform transitional smoothing on the fused boundary. Equivalently, the electronic device uses an enlarged or reduced main area or the original main area to replace the co-located area in the first picture, and also performs transitional smoothing processing on the interface between the main area and the first picture, so that the final composite image Looks natural.
  • the electronic device 100 uses the main area to replace the parity area in (c). bit area, and performs transitional smoothing on the boundary to generate a composite image as shown in (d).
  • the electronic device 100 enlarges the main area, uses the enlarged main area to replace the co-located area in (c), and performs transitional smoothing on the boundary to generate the image as shown in (d). composite image.
  • object B can be smoothly and naturally integrated into the first picture containing object A, and a composite object containing both object A and object B can be generated, thereby obtaining a natural and high-quality photo effect.
  • the image collected at the moment when the photographing instruction is received and the first picture can be fitted to generate a composite image.
  • the electronic device 100 can save the moment when the photographing instruction is received and the N frames of images previously collected by the camera 193 in real time, fit each of the N frames of images to the first picture, and generate N frames of composite images. .
  • the multiple frame images collected by the camera 193 between the time when the recording instruction is received and the time when the stop recording instruction is received can be compared with the first image respectively. Fitting to generate multi-frame composite images.
  • the user is shown the relative positions of object A and object B when taking a group photo through superimposed display in the preview interface.
  • the fitting operation of the first picture and the image collected by the camera begins. This method can save the power consumption and processing power of electronic equipment.
  • S106 The electronic device 100 saves the composite image.
  • the electronic device 100 can generate the generated frame The composite image is saved to memory in picture format. If the electronic device 100 fits the time when the photographing instruction is received and the N frames of images previously collected by the camera 193 with the first picture to generate N frames of composite images, then the electronic device 100 can obtain the N frames of composite images from the N frames of images collected by the camera 193 . Select the frame with the best quality and save it to memory in picture format. Here, the electronic device 100 can select an image with the best quality from N frames of synthesized images through artificial intelligence learning.
  • the quality of the synthesized image can be evaluated by, but not limited to, any one or more of the following parameters: peak-signal to noise ratio (PSNR) and mean square error (MSE), average absolute Error (mean sbsolute error, MAE) and signal to noise ratio (signal to noise ratio, SNR).
  • PSNR peak-signal to noise ratio
  • MSE mean square error
  • MAE average absolute Error
  • SNR signal to noise ratio
  • the electronic device 100 if the electronic device 100 performs S104-S105 in the camera mode, the electronic device will fit the time of receiving the camera instruction and the N frames of images previously collected by the camera 193 with the first picture, respectively. After N frames of composite images are generated, the user can select a satisfactory composite image among the N frames of composite images and store it in the memory, and delete the remaining composite images.
  • the composite image (that is, the composite picture) retained while the electronic device 100 performs S104-S106 in the camera mode may also be called the second picture.
  • the multi-frame images collected by the video recording device camera 193 and the multi-frame composite images generated after fitting the first image can be stored in the memory in a video format. , that is, the multi-frame composite image is saved as a video file.
  • the composite image can be viewed, and the composite image can also be edited, moved, and shared.
  • the operation for viewing the composite image may also be called a third operation.
  • the electronic device 100 generates a composite image in the camera mode, and the thumbnail 207 b of the composite image is displayed in the picture area 207 provided by the gallery application shown in FIG. 4B .
  • the electronic device 100 can display the original image of the thumbnail 207b as shown in FIG. 4B, that is, the composite image generated by the electronic device 100.
  • the electronic device 100 can also use the composite object as the first picture, and then perform the shooting method described in S101-S106 one or more times, so that multiple group photos can be achieved.
  • the same user can first use his or her own photo as the first picture, and then take multiple photos with himself or herself in the first photo to achieve multiple photos of the same person at the same place.
  • the user can realize cross-time and space photos at different times with his or her relative at the same place. group photo.
  • the electronic device 100 specifically fits the area where the object B is located in the image collected by the camera 193 to the first picture, thereby obtaining a composite image. It is not limited to this. In some other embodiments of the present application, the electronic device 100 can also fit the area where the object A is located in the first image to the image collected by the camera 193 to obtain a composite image. Specifically, this implementation may include the following steps:
  • Step 1 Identify the object in the first picture, which can be called object A;
  • Step 2 In the first picture, determine the main area containing object A based on object A;
  • Step 3 Perform a correlation search for the main area in the image collected by the camera, and find the same area with a correlation greater than the second value;
  • Step 4 Fit the main area to the co-located area of the image collected by the camera, keep the remaining areas of the image collected by the camera unchanged, and generate a composite image.
  • the area where the object A in the first image is located is fitted to the image collected by the camera 193 to obtain a composite image. If the first image is an old photo, the object in the old photo can be extracted. Integrate into newly captured images to renovate old photos.
  • the object A in the first picture may also be called the first object
  • the object B in the image captured by the camera may also be called the second object
  • the electronic device 100 now overlays and displays the first picture and the image collected by the camera in the preview interface, and then generates a composite image after receiving the first operation. It is not limited to this. In some other embodiments of the present application, the electronic device 100 can also directly perform the operation of fitting and generating a composite image directly after starting the camera application, and directly display the composite image in the preview interface. Specifically, after executing S101-S103 of the above method, the electronic device 100 starts the camera application and collects images in real time through the camera 193. The electronic device 100 fits each frame image collected in real time by the camera 193 with the first picture respectively to obtain multiple frames.
  • the images are synthesized, and the multiple frames of synthesized images are displayed in the preview box of the preview interface (ie, the second user interface mentioned above) in chronological order. If a photo-taking instruction is received later, the composite image displayed in the preview box will be saved as a picture. If a video recording instruction is received, the multi-frame composite image displayed in the preview box during recording will be saved as a video.
  • This method of real-time synthesis and real-time display of the synthesized image requires the electronic device 100 to have strong processing capabilities. This method can display the effect of the group photo to the user in real time in the preview box, allowing the user to feel the effect of the group photo more intuitively.
  • each step in the above method embodiment can be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software.
  • the method steps disclosed in conjunction with the embodiments of this application can be directly implemented by a hardware processor, or executed by a combination of hardware and software modules in the processor.
  • This application also provides an electronic device, which may include a memory and a processor.
  • the memory can be used to store a computer program; the processor can be used to call the computer program in the memory, so that the electronic device executes the method of the electronic device 100 in any of the above embodiments.
  • This application also provides a chip system, which includes at least one processor and is used to implement the functions involved in the electronic device 100 side in any of the above embodiments.
  • the chip system further includes a memory, the memory is used to store program instructions and data, and the memory is located within the processor or outside the processor.
  • the chip system can be composed of chips or include chips and other discrete devices.
  • processors in the chip system there may be one or more processors in the chip system.
  • the processor can be implemented in hardware or software.
  • the processor may be a logic circuit, an integrated circuit, or the like.
  • the processor may be a general-purpose processor implemented by reading software code stored in memory.
  • the memory may be integrated with the processor or may be provided separately from the processor, which is not limited by the embodiments of the present application.
  • the memory may be a non-transient processor, such as a read-only memory ROM, which may be integrated with the processor on the same chip, or may be separately provided on different chips.
  • the embodiments of this application vary on the type of memory, and The arrangement of the memory and processor is not specifically limited.
  • the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or a system on chip (SoC). It can also be a central processor (central processor unit, CPU), a network processor (network processor, NP), a digital signal processing circuit (digital signal processor, DSP), or a microcontroller (micro controller unit (MCU), or a programmable logic device (PLD) or other integrated chip.
  • FPGA field programmable gate array
  • ASIC application specific integrated circuit
  • SoC system on chip
  • CPU central processor unit
  • NP network processor
  • DSP digital signal processing circuit
  • MCU microcontroller
  • PLD programmable logic device
  • the computer program product includes: a computer program (which can also be called a code, or an instruction).
  • a computer program which can also be called a code, or an instruction.
  • the computer program When the computer program is run, it causes the computer to execute the electronic device in any of the above embodiments. The method performed on the 100th side.
  • This application also provides a computer-readable storage medium that stores a computer program (which may also be called a code, or an instruction).
  • a computer program which may also be called a code, or an instruction.
  • the computer program product includes one or more computer instructions.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.
  • the computer instructions may be stored in or transmitted from one computer-readable storage medium to another, e.g., the computer instructions may be transferred from a website, computer, server, or data center Transmission to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) means.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more available media integrated.
  • the available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, DVD), or semiconductor media (eg, solid state disk (SSD)), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Computing Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Studio Devices (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

本申请公开了拍摄方法及相关装置，电子设备启动针对选择的一张包含对象A的图片的"合拍"功能后，启动相机应用，在预览界面中显示该图片的指示信息，并同时显示电子设备的摄像头采集到的图像。之后，电子设备可以接收到用户输入的拍照指令，将该图片和摄像头采集到的图像拟合为一张同时包含对象A和对象B的合成图像。该方法可以在不借助其他设备或其他人员的情况下，实现对象A和对象B的高质量合影。

Description

拍摄方法及相关装置
本申请要求于2022年08月26日提交中国专利局、申请号为202211037891.9、申请名称为“拍摄方法及相关装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及终端技术领域，尤其涉及拍摄方法及相关装置。
背景技术
多人出游时，通常有合影的需求。拍摄多人合照的方法包括以下几种：
1，请路人帮忙拍摄合照。该方法不仅需要麻烦别人，且可能由于路人对拍摄设备的了解程度不够导致拍摄效果一般。
2，使用自拍杆、三角架或直接将拍摄设备放置在地面上拍照。但自拍杆增加了用户出游的负重，无法控制拍摄距离，使用前置摄像头的拍摄清晰度较低。三角架同样增加了负重，且无法实时预览，拍摄设备还有损坏的风险。将拍摄设备放置在地面上拍摄无法实时预览，无法控制拍摄距离。
3，多人分开拍照后利用拍照软件后期生成一张合照。后期制作合照的方式耗时长，并且不支持实时预览合影情况，合影效果也一般。
发明内容
本申请提供了拍摄方法及相关装置，可以在不借助其他设备或其他人员的情况下，实现多个对象的高质量合影。
第一方面，本申请实施例提供了一种拍摄方法，该方法应用于电子设备。该方法可包括：电子设备启动图库应用，并显示第一用户界面，第一用户界面中显示有第一图片，第一图片包含第一对象；电子设备接收到针对第一图片的第一操作；电子设备启动相机应用，并显示第二用户界面，第二用户界面中显示有预览框，预览框中显示有电子设备的摄像头采集的图像，以及，第一图片的指示信息；电子设备接收到用于拍照的第二操作；电子设备存储第二图片，第二图片由摄像头采集的图像和第一图片拟合得到，摄像头采集的图像包含第二对象，并且第二图片包含第一对象和第二对象。
实施第一方面提供的方法，电子设备可以通过图库应用来调用相机应用，并可以在不借助其他设备或其他人员的情况下完成第一对象和第二对象的合影，并且在合影过程中还可以实时预览并调整第一对象和第二对象之间的相对位置，还可以灵活地调整拍摄距离以及拍摄所使用的摄像头，以达到高质量的合影效果。并且，该方法操作难度低，时间短，方便快捷，可以支持连续、大量的合影拍摄。
结合第一方面，在一些实施方式中，第一对象或第二对象为以下任意一种：人物、动物、地标、建筑物、玩偶。
结合第一方面，在一些实施方式中，第一对象或第二对象的数量均可以为多个。
结合第一方面，在一些实施方式中，电子设备在预览框中显示的第一图片的指示信息可以为具有透明度或虚化的第一图片。这样，用户通过预览框既能够看到该第一图片，又能够透过虚化或具有透明度的该第一图片看到电子设备所显示的摄像头所采集的图像，达到实时预览第一图片中的对象和摄像头采集图像中的对象之间的相对位置的效果。
结合第一方面，在一些实施方式中，电子设备在接收到第二操作后，可以通过以下任意一种方式来拟合得到第二图片：
方式1，将摄像头所采集到的图像中第二对象所在区域拟合至第一图片中，从而得到第二图片。
具体实现中，电子设备识别摄像头采集的图像中的第二对象；在摄像头采集的图像中确定包含第二对象的主区域，主区域为以第二对象为基准确定的区域；在第一图片中确定同位区，第一图片中的同位区以外的部分，和，摄像头采集的图像中主区域以外的部分的相关性大于第二值；将主区域拟合到第一图片中的同位区，得到第二图片。
方式2，将第一图片中的第一对象所在区域拟合至摄像头所采集到的图像中，从而得到第二图片。
电子设备识别第一图片中的第一对象；在第一图片中确定包含第一对象的主区域，主区域为以第一对象为基准确定的区域；在摄像头采集的图像中确定同位区，摄像头采集的图像中的同位区以外的部分，和， 第一图片中主区域以外的部分的相关性大于第二值；将主区域拟合到摄像头采集的图像中的同位区，得到第二图片。
在上述方式1或方式2中，主区域可以是以对应对象(如第一对象或第二对象)为基准，向外扩展第一值的距离得到的区域，或者向周围扩展得到的和对应对象具有相近或相同的颜色或像素的区域。其中，第一值可以由电子设备预置。第一值可以和电子设备配置摄像头型号相关。
以对应对象(如第一对象或第二对象)为基准确定主区域，既能够保证该主区域中对象的完整性，又能够避免过于扩大范围，从而保证后续拟合的效果。
在上述方式1或方式2中，同位区在第一图片中的位置，和，主区域在摄像头采集的图像中的位置，相同。这样，用户根据预览框中的内容来实时地调整的效果能够直接作用于最终的拟合效果上，达到用户实时调整合影效果的目的。
在上述方式1或方式2中，同位区在第一图片中的大小、形状，和，主区域在摄像头采集的图像中的大小、形状，相同或不同。
结合第一方面，在一些实施方式中，第二图片为，摄像头采集的多帧图像分别和第一图片拟合得到的合成图像中，质量最高的一帧图像；摄像头采集的多帧图像包括：在电子设备接收到第二操作的时刻以及之前时刻由摄像头采集的图像，多帧图像的数量由电子设备预先设置。这样可以保证高质量的合影效果。
结合第一方面，在一些实施方式中，第二图片为，接收到第二操作的时刻摄像头所采集的图像和第一图片拟合而得到的一帧合成图像。
结合第一方面，在一些实施方式中，第一用户界面中显示有第一控件，电子设备接收到针对第一图片的第一操作之前，方法还包括：电子设备接收到作用于第一控件的用户操作；电子设备在第一用户界面中显示第二控件；第一操作包括作用于第二控件上的用户操作。
结合第一方面，在一些实施方式中，电子设备存储第二图片之后，电子设备可以接收到用于查看第二图片的第三操作，并显示第二图片。相当于，完成第一对象和第二对象的合影之后，用户还可以在电子设备上查看该合影。
第二方面，本申请实施例提供了一种拍摄方法，该方法应用于电子设备。该方法可包括：电子设备启动图库应用，并显示第一用户界面，第一用户界面中显示有第一图片，第一图片包含第一对象；电子设备接收到针对第一图片的第一操作；电子设备启动相机应用，并显示第二用户界面，第二用户界面中显示有预览框，预览框中显示有合成图像，合成图像由电子设备的摄像头采集的图像和第一图片拟合得到，摄像头采集的图像包含第二对象；电子设备接收到用于拍照的第二操作；电子设备存储第二图片，第二图片为接收到第二操作时预览框中显示的合成图像，并且第二图片包含第一对象和第二对象。
实施第二方面提供的方法，电子设备可以通过图库应用来调用相机应用，可以在预览框中为用户实时地展示合影效果，让用户更加直观地感受到合影效果，并可以在不借助其他设备或其他人员的情况下完成第一对象和第二对象的合影。并且在合影过程中还可以实时预览并调整第一对象和第二对象之间的相对位置，还可以灵活地调整拍摄距离以及拍摄所使用的摄像头，以达到高质量的合影效果。并且，该方法操作难度低，时间短，方便快捷，可以支持连续、大量的合影拍摄。
结合第二方面，在一些实施方式中，第一对象或第二对象为以下任意一种：人物、动物、地标、建筑物、玩偶。
结合第二方面，在一些实施方式中，第一对象或第二对象的数量均可以为多个。
结合第二方面，在一些实施方式中，电子设备可以通过以下任意一种方式，拟合得到在预览框中显示的合成图像：
方式1，将摄像头所采集到的图像中第二对象所在区域拟合至第一图片中，从而得到第二图片。
方式2，将第一图片中的第一对象所在区域拟合至摄像头所采集到的图像中，从而得到第二图片。
上述方式1及方式2的具体实现及技术效果，可参考第一方面中第二图片的生成方式，这里不再赘述。
结合第二方面，在一些实施方式中，第一用户界面中显示有第一控件，电子设备接收到针对第一图片的第一操作之前，还可以接收到作用于第一控件的用户操作，并在第一用户界面中显示第二控件；第一操作包括作用于第二控件上的用户操作。
结合第二方面，在一些实施方式中，电子设备存储第二图片之后，电子设备可以接收到用于查看第二图片的第三操作，并显示第二图片。相当于，完成第一对象和第二对象的合影之后，用户还可以在电子设备上查看该合影。
第三方面，本申请实施例提供了一种拍摄方法，该方法应用于电子设备。该方法可包括：电子设备显 示第二用户界面，第二用户界面中显示有预览框和第三控件，预览框中显示有电子设备的摄像头采集的图像；电子设备检测到作用于第三控件上的第四操作；电子设备显示一张或多张图片；电子设备检测到作用于一张或多张图片中的第一图片上的第五操作，第一图片包含第一对象；电子设备在第二用户界面的预览框中，显示电子设备的摄像头采集的图像，以及，第一图片的指示信息；电子设备接收到用于拍照的第二操作；电子设备存储第二图片，第二图片由摄像头采集的图像和第一图片拟合得到，摄像头采集的图像包含第二对象，并且第二图片包含第一对象和第二对象。
实施第三方面提供的方法，电子设备可以通过预览界面启动合拍功能，并可以在不借助其他设备或其他人员的情况下完成第一对象和第二对象的合影，并且在合影过程中还可以实时预览并调整第一对象和第二对象之间的相对位置，还可以灵活地调整拍摄距离以及拍摄所使用的摄像头，以达到高质量的合影效果。并且，该方法操作难度低，时间短，方便快捷，可以支持连续、大量的合影拍摄。
结合第三方面，第二用户界面(即预览界面)可以是相机应用提供的，也可以是其他应用调用相机接口提供的界面。其他应用例如可以是社交类应用、购物类应用等等。相当于，可以通过相机应用或其他应用来启动合拍功能。
第三方面中的第一对象、第二对象、第一图片的指示信息，均可以参考第一方面的相关描述。
结合第三方面，在一些实施方式中，电子设备拟合得到第二图片的方式及技术效果可以参考第一方面中的相关描述。
结合第三方面，在一些实施方式中，第二图片为，摄像头采集的多帧图像分别和第一图片拟合得到的合成图像中，质量最高的一帧图像；摄像头采集的多帧图像包括：在电子设备接收到第二操作的时刻以及之前时刻由摄像头采集的图像，多帧图像的数量由电子设备预先设置。这样可以保证高质量的合影效果。
结合第三方面，在一些实施方式中，第二图片为，接收到第二操作的时刻摄像头所采集的图像和第一图片拟合而得到的一帧合成图像。
结合第三方面，在一些实施方式中，电子设备存储第二图片之后，电子设备可以接收到用于查看第二图片的第三操作，并显示第二图片。相当于，完成第一对象和第二对象的合影之后，用户还可以在电子设备上查看该合影。
第四方面，本申请实施例提供了一种拍摄方法，该方法应用于电子设备。该方法可包括：电子设备显示第二用户界面，第二用户界面中显示有预览框和第三控件，预览框中显示有电子设备的摄像头采集的图像；电子设备检测到作用于第三控件上的第四操作；电子设备显示一张或多张图片；电子设备检测到作用于一张或多张图片中的第一图片上的第五操作，第一图片包含第一对象；电子设备在第二用户界面的预览框中显示合成图像，合成图像由电子设备的摄像头采集的图像和第一图片拟合得到，摄像头采集的图像包含第二对象；电子设备接收到用于拍照的第二操作；电子设备存储第二图片，第二图片为接收到第二操作时预览框中显示的合成图像，并且第二图片包含第一对象和第二对象。
实施第四方面提供的方法，电子设备可以通过预览界面启动合拍功能，可以在预览框中为用户实时地展示合影效果，让用户更加直观地感受到合影效果，并可以在不借助其他设备或其他人员的情况下完成第一对象和第二对象的合影。并且在合影过程中还可以实时预览并调整第一对象和第二对象之间的相对位置，还可以灵活地调整拍摄距离以及拍摄所使用的摄像头，以达到高质量的合影效果。并且，该方法操作难度低，时间短，方便快捷，可以支持连续、大量的合影拍摄。
结合第四方面，在一些实施方式中，第二用户界面(即预览界面)可以是相机应用提供的，也可以是其他应用调用相机接口提供的界面。其他应用例如可以是社交类应用、购物类应用等等。相当于，可以通过相机应用或其他应用来启动合拍功能。
结合第四方面，在一些实施方式中，第一对象或第二对象为以下任意一种：人物、动物、地标、建筑物、玩偶。
结合第四方面，在一些实施方式中，第一对象或第二对象的数量均可以为多个。
结合第四方面，在一些实施方式中，电子设备可以通过以下任意一种方式，拟合得到在预览框中显示的合成图像：
方式1，将摄像头所采集到的图像中第二对象所在区域拟合至第一图片中，从而得到第二图片。
方式2，将第一图片中的第一对象所在区域拟合至摄像头所采集到的图像中，从而得到第二图片。
上述方式1及方式2的具体实现及技术效果，可参考第一方面中第二图片的生成方式，这里不再赘述。
结合第四方面，在一些实施方式中，电子设备存储第二图片之后，电子设备可以接收到用于查看第二图片的第三操作，并显示第二图片。相当于，完成第一对象和第二对象的合影之后，用户还可以在电子设 备上查看该合影。
第五方面，本申请实施例提供了一种电子设备，包括：存储器、一个或多个处理器；存储器与一个或多个处理器耦合，存储器用于存储计算机程序代码，计算机程序代码包括计算机指令，一个或多个处理器调用计算机指令以使得电子设备执行如第一方面或第二方面或第三方面或第四方面中电子设备所执行的任意一种实施方式的方法。
第六方面，本申请实施例提供了一种计算机可读存储介质，包括指令，当指令在电子设备上运行时，使得电子设备执行如第一方面或第二方面或第三方面或第四方面中电子设备所执行的任意一种实施方式的方法。
第七方面，本申请实施例提供了一种计算机程序产品，当计算机程序产品在计算机上运行时，使得计算机执行如第一方面或第二方面或第三方面或第四方面中电子设备所执行的任意一种实施方式的方法。
第八方面，本申请实施例提供了一种芯片系统，所述芯片系统包括至少一个处理器，用于实现如第一方面或第二方面或第三方面或第四方面中电子设备所执行的任意一种实施方式的方法。
实施本申请提供的技术方案，电子设备可以在不借助其他设备或其他人员的情况下完成第一对象和第二对象的合影，并且在合影过程中还可以实时预览并调整第一对象和第二对象之间的相对位置，还可以灵活地调整拍摄距离以及拍摄所使用的摄像头，以达到高质量的合影效果。并且，该方法操作难度低，时间短，方便快捷，可以支持连续、大量的合影拍摄。
附图说明
图1A为本申请实施例提供的电子设备100的硬件结构示意图；
图1B为本申请实施例提供的电子设备100的一种软件结构示意图；
图1C为本申请实施例提供的电子设备100的另一种软件结构示意图；
图2A-图2H为本申请实施例中电子设备100提供的一组用于合影的用户界面；
图3A-图3F为本申请实施例中电子设备100提供的另一组用于合影的用户界面；
图4A-图4B为本申请实施例中电子设备100保存拟合的图片后，图库应用所提供的用户界面；
图5为本申请实施例提供的拍摄方法的流程图；
图6A-图6B为本申请实施例提供的几种图像处理过程的示例。
具体实施方式
下面将结合附图对本申请实施例中的技术方案进行清楚、详尽地描述。其中，在本申请实施例的描述中，除非另有说明，“/”表示或的意思，例如，A/B可以表示A或B；文本中的“和/或”仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况，另外，在本申请实施例的描述中，“多个”是指两个或多于两个。
以下，术语“第一”、“第二”仅用于描述目的，而不能理解为暗示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括一个或者更多个该特征，在本申请实施例的描述中，除非另有说明，“多个”的含义是两个或两个以上。
本申请以下实施例中的术语“用户界面(user interface，UI)”，是应用程序或操作系统与用户之间进行交互和信息交换的介质接口，它实现信息的内部形式与用户可以接受形式之间的转换。用户界面是通过java、可扩展标记语言(extensible markup language，XML)等特定计算机语言编写的源代码，界面源代码在电子设备上经过解析，渲染，最终呈现为用户可以识别的内容。用户界面常用的表现形式是图形用户界面(graphic user interface，GUI)，是指采用图形方式显示的与计算机操作相关的用户界面。它可以是在电子设备的显示屏中显示的文本、图标、按钮、菜单、选项卡、文本框、对话框、状态栏、导航栏、Widget等可视的界面元素。
本申请以下实施例提供了拍摄方法及相关装置。在该拍摄方法中，电子设备可以启动针对选择的一张包含对象A的图片的“合拍”功能，然后启动相机应用，在相机应用提供的预览界面中显示具有透明度的或虚化的该图片，并同时显示电子设备的摄像头采集到的图像。在摄像头采集到的图像包括对象B后，电子设备可以接收到用户输入的拍照指令，之后可以将该图片和摄像头采集到的图像进行拟合，生成一张同时包含对象A和对象B的合成图像。
上述的对象A或对象B可包括但不限于：人物、动物、地标、建筑物、玩偶等等。对象A或对象B的数量均可以是多个。
上述拍摄方法可以实现对象A和对象B的合影。该方法可以在不借助其他设备或其他人员的情况下完成合影，并且在合影过程中还可以实时预览并调整对象B和对象A的相对位置，还可以灵活地调整拍摄距离以及拍摄所使用的摄像头，以达到高质量的合影效果。并且，该方法操作难度低，时间短，方便快捷，可以支持连续、大量的合影拍摄。
上述拍摄方法可以应用于有合影需求的场景，包括但不限于：同一人物在同一地点的多次合影；新人物和老照片中的人物合影，实现同一地点不同时间的多人合影。
在一些实施方式中，电子设备可以提取摄像头拍摄图像中对象B所在的区域，并将该区域拟合到包含对象A的图片中，该图片中的其余部分保持不变，以生成同时包含对象A和对象B的合成图像。通过图像拟合可以让对象B平滑、自然地融入到包含对象A的图片中的对应区域，得到自然的、高质量的合影效果。在另一些实施方式中，电子设备也可以提取图片中对象A所在的区域，并将该区域拟合到摄像头采集的图像中。生成合成图像的具体方式，可参考后续方法实施例的详细描述。
在一些实施方式中，电子设备可以从接收到拍照指令的时刻开始，选择在该时刻以及之前摄像头拍摄的N帧图像，将该N帧图像各自和包含对象A的图片进行拟合处理，得到N帧合成图像，然后从这N帧合成图像中选择质量最好的一帧，作为合影结果存储起来。这样可以从多帧合成图像中选择最优的一帧作为合影结果，达到高质量的合影效果。并且，该方法还考虑到了用户看到预览界面的合影效果以及输入拍照指令之间的时延，从而避免以接收到拍照指令时刻对应的合成图像作为最终的合影效果。
在一些实施方式中，电子设备启动相机应用后，还可以直接将摄像头拍摄到的图像和选择的包含对象A的图片进行拟合，并将合成图像直接显示在相机应用提供的预览界面中。
在本申请实施例中，“合拍”功能由电子设备所提供，该功能支持电子设备在相机应用提供的预览界面中显示具有透明度的或虚化的包含对象A的图片，并同时显示电子设备的摄像头采集到的图像，电子设备接收到用户输入的拍照指令，将该图片和摄像头采集到的图像进行拟合，生成一张同时包含对象A和对象B的合成图像。或者，该“合拍”功能支持电子设备直接将摄像头拍摄到的图像和包含对象A的图片进行拟合，并将合成图像直接显示在相机应用提供的预览界面中。
“合拍”功能仅是本申请实施例使用的一个名词，其代表的含义在本实施例中已经记载，其名称并不能对本实施例构成任何限制。例如，“合拍”功能也可以被称为其他名词，如“合影”功能、多人合影、多人合拍等等。
下面首先介绍本申请实施例提供的电子设备。
本申请实施例提供的电子设备配置有摄像头，可以包括但不限于智能手机、平板电脑、专门的照相机(例如单反相机、卡片式相机)等、增强现实(augmented reality，AR)设备、虚拟现实(virtual reality，VR)设备、人工智能(artificial intelligence,AI)设备、可穿戴式设备(如智能手表、智能眼镜)、个人数字助理(personal digital assistant，PDA)等。电子设备的示例性实施例包括但不限于搭载 Linux或者其它操作系统的便携式电子设备。上述电子设备也可为其它便携式电子设备，诸如膝上型计算机(Laptop)等。
图1A示出了本申请实施例提供的一种电子设备100的结构示意图。
电子设备100可以包括处理器110，外部存储器接口120，内部存储器121，通用串行总线(universal serial bus，USB)接口130，充电管理模块140，电源管理模块141，电池142，天线1，天线2，移动通信模块150，无线通信模块160，音频模块170，扬声器170A，受话器170B，麦克风170C，耳机接口170D，传感器模块180，按键190，马达191，指示器192，摄像头193，显示屏194，以及用户标识模块(subscriber identification module，SIM)卡接口195等。
其中传感器模块180可以包括压力传感器180A，陀螺仪传感器180B，气压传感器180C，磁传感器180D，加速度传感器180E，距离传感器180F，接近光传感器180G，指纹传感器180H，温度传感器180J，触摸传感器180K，环境光传感器180L，骨传导传感器180M等。
可以理解的是，本申请实施例示意的结构并不构成对电子设备100的具体限定。在本申请另一些实施例中，电子设备100可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或软件和硬件的组合实现。
处理器110可以包括一个或多个处理单元，例如：处理器110可以包括应用处理器(application processor，AP)，调制解调处理器，图形处理器(graphics processing unit，GPU)，图像信号处理器(image signal processor，ISP)，控制器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器，和/或神经网 络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。
控制器可以根据指令操作码和时序信号，产生操作控制信号，完成取指令和执行指令的控制。
处理器110中还可以设置存储器，用于存储指令和数据。在一些实施例中，处理器110中的存储器为高速缓冲存储器。该存储器可以保存处理器110刚用过或循环使用的指令或数据。如果处理器110需要再次使用该指令或数据，可从所述存储器中直接调用。避免了重复存取，减少了处理器110的等待时间，因而提高了系统的效率。
在一些实施例中，控制器或GPU等处理器110，可以用于将包含对象A的图片做透明度处理或虚化处理，将其和摄像头193采集到的图像叠加以合成显示于取景框中的预览图像。
在一些实施例中，控制器或GPU等处理器110，可以用于将摄像头拍摄到的图像和包含对象A的图片进行拟合，以生成显示于取景框中的预览图像，以使得合影效果可以呈现给用户。处理器110执行的拟合操作的具体实现，可参考后续方法实施例的详细介绍。
为使用方便，电子设备通常在用户的手持模式下进行拍摄，而用户手持模式下通常会使得拍摄获得的画面发生抖动。在一些实施例中，控制器或GPU等处理器110，还可以用于在合拍场景下，对摄像头采集到的图像进行防抖处理后，再将摄像头对应的防抖处理后的图像和包含对象A的图片进行叠加或拟合，以提供预览图像给显示屏194显示。
电子设备100的无线通信功能可以通过天线1，天线2，移动通信模块150，无线通信模块160，调制解调处理器以及基带处理器等实现。
天线1和天线2用于发射和接收电磁波信号。电子设备100中的每个天线可用于覆盖单个或多个通信频带。不同的天线还可以复用，以提高天线的利用率。例如：可以将天线1复用为无线局域网的分集天线。在另外一些实施例中，天线可以和调谐开关结合使用。
移动通信模块150可以提供应用在电子设备100上的包括2G/3G/4G/5G等无线通信的解决方案。移动通信模块150可以包括至少一个滤波器，开关，功率放大器，低噪声放大器(low noise amplifier，LNA)等。移动通信模块150可以由天线1接收电磁波，并对接收的电磁波进行滤波，放大等处理，传送至调制解调处理器进行解调。移动通信模块150还可以对经调制解调处理器调制后的信号放大，经天线1转为电磁波辐射出去。在一些实施例中，移动通信模块150的至少部分功能模块可以被设置于处理器110中。在一些实施例中，移动通信模块150的至少部分功能模块可以与处理器110的至少部分模块被设置在同一个器件中。
调制解调处理器可以包括调制器和解调器。其中，调制器用于将待发送的低频基带信号调制成中高频信号。解调器用于将接收的电磁波信号解调为低频基带信号。随后解调器将解调得到的低频基带信号传送至基带处理器处理。低频基带信号经基带处理器处理后，被传递给应用处理器。应用处理器通过音频设备(不限于扬声器170A，受话器170B等)输出声音信号，或通过显示屏194显示图像或视频。在一些实施例中，调制解调处理器可以是独立的器件。在另一些实施例中，调制解调处理器可以独立于处理器110，与移动通信模块150或其他功能模块设置在同一个器件中。
无线通信模块160可以提供应用在电子设备100上的包括无线局域网(wireless local area networks，WLAN)(如无线保真(wireless fidelity，Wi-Fi)网络)，蓝牙(bluetooth，BT)，全球导航卫星系统(global navigation satellite system，GNSS)，调频(frequency modulation，FM)，近距离无线通信技术(near field communication，NFC)，红外技术(infrared，IR)等无线通信的解决方案。无线通信模块160可以是集成至少一个通信处理模块的一个或多个器件。无线通信模块160经由天线2接收电磁波，将电磁波信号解调以及滤波处理，将处理后的信号发送到处理器110。无线通信模块160还可以从处理器110接收待发送的信号，对其进行调频，放大，经天线2转为电磁波辐射出去。
在一些实施例中，电子设备100的天线1和移动通信模块150耦合，天线2和无线通信模块160耦合，使得电子设备100可以通过无线通信技术与网络以及其他设备通信。所述无线通信技术可以包括全球移动通讯系统(global system for mobile communications，GSM)，通用分组无线服务(general packet radio service，GPRS)，码分多址接入(code division multiple access，CDMA)，宽带码分多址(wideband code division multiple access，WCDMA)，时分码分多址(time-division code division multiple access，TD-SCDMA)，长期演进(long term evolution，LTE)，BT，GNSS，WLAN，NFC，FM，和/或IR技术等。所述GNSS可以包括全球卫星定位系统(global positioning system，GPS)，全球导航卫星系统(global navigation satellite system，GLONASS)，北斗卫星导航系统(beidou navigation satellite system，BDS)，准天顶卫星系统(quasi-zenith satellite system， QZSS)和/或星基增强系统(satellite based augmentation systems，SBAS)。
电子设备100通过GPU，显示屏194，以及应用处理器等实现显示功能。GPU为图像处理的微处理器，连接显示屏194和应用处理器。GPU用于执行数学和几何计算，用于图形渲染。处理器110可包括一个或多个GPU，其执行程序指令以生成或改变显示信息。
显示屏194用于显示图像，视频等。显示屏194包括显示面板。显示面板可以采用液晶显示屏(liquid crystal display，LCD)。显示面板还可以采用有机发光二极管(organic light-emitting diode，OLED)，有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode，AMOLED)，柔性发光二极管(flex light-emitting diode，FLED)，miniled，microled，micro-oled，量子点发光二极管(quantum dot light emitting diodes，QLED)等制造。在一些实施例中，电子设备可以包括1个或N个显示屏194，N为大于1的正整数。
在一些实施方式中，在合拍场景下，显示屏194可以显示相机应用提供的预览界面，并在预览界面中显示具有透明度的或虚化的包含对象A的图片，并同时显示电子设备的摄像头193采集到的图像。
在另一些实施方式中，在合拍场景下，显示屏194可以显示处理器110将摄像头拍摄到的图像和包含对象A的图片进行拟合后得到的合成图像，以使得合影效果可以呈现给用户。
电子设备100可以通过ISP，摄像头193，视频编解码器，GPU，显示屏194以及应用处理器等实现拍摄功能。
ISP用于处理摄像头193反馈的数据。例如，拍照时，打开快门，光线通过镜头被传递到摄像头感光元件上，光信号转换为电信号，摄像头感光元件将所述电信号传递给ISP处理，转化为肉眼可见的图像。ISP还可以对图像的噪点，亮度进行算法优化。ISP还可以对拍摄场景的曝光，色温等参数优化。在一些实施例中，ISP可以设置在摄像头193中。
摄像头193用于捕获静态图像或视频。物体通过镜头生成光学图像投射到感光元件。感光元件可以是电荷耦合器件(charge coupled device，CCD)或互补金属氧化物半导体(complementary metal-oxide-semiconductor，CMOS)光电晶体管。感光元件把光信号转换成电信号，之后将电信号传递给ISP转换成数字图像信号。ISP将数字图像信号输出到DSP加工处理。DSP将数字图像信号转换成标准的RGB，YUV等格式的图像信号。在一些实施例中，电子设备100可以包括1个或N个摄像头193，N为大于1的正整数。
电子设备100可具有多个摄像头193，例如前置摄像头、后置摄像头。前置摄像头、后置摄像头的数量均可以是多个。摄像头193的种类可包括但不限于广角摄像头、超广角摄像头、长焦摄像头等。
摄像头193的硬件配置以及物理位置可以不同，因此，不同摄像头采集到的图像的大小、范围、内容或清晰度等可能不同。
摄像头193的出图尺寸可以不同，也可以相同。摄像头的出图尺寸是指该摄像头采集到的图像的长度与宽度。该图像的长度和宽度均可以用像素数来衡量。摄像头的出图尺寸也可以被叫做图像大小、图像尺寸、像素尺寸或图像分辨率。常见的摄像头的出图比例可包括：4:3、16:9或3:2等等。出图比例是指摄像头所采集图像在长度上和宽度上的像素数的大致比例。
摄像头193可以对应同一焦段，也可以对应不同的焦段。摄像头对应的焦段越大，该摄像头的视场角(field of view，FOV)越小。视场角是指光学系统所能够成像的角度范围。
摄像头193可以设置于电子设备的两面。和电子设备的显示屏194位于同一平面的摄像头可以被称为前置摄像头，位于电子设备的后盖所在平面的摄像头可以被称为后置摄像头。前置摄像头可用于采集面对显示屏194的拍摄者自己的图像，后置摄像头可用于采集拍摄者所面对的拍摄对象(如人物、风景等)的图像。
在一些实施例中，摄像头193可以用于采集深度数据。例如，摄像头193可以具有(time of flight，TOF)3D感测模块或结构光(structured light)3D感测模块，用于获取深度信息。用于采集深度数据的摄像头可以为前置摄像头，也可为后置摄像头。
数字信号处理器用于处理数字信号，除了可以处理数字图像信号，还可以处理其他数字信号。例如，当电子设备100在频点选择时，数字信号处理器用于对频点能量进行傅里叶变换等。
视频编解码器用于对数字视频压缩或解压缩。电子设备100可以支持一种或多种视频编解码器。这样，电子设备100可以播放或录制多种编码格式的视频，例如：动态图像专家组(moving picture experts group，MPEG)1，MPEG2，MPEG3，MPEG4等。
NPU为神经网络(neural-network，NN)计算处理器，通过借鉴生物神经网络结构，例如借鉴人脑神经 元之间传递模式，对输入信息快速处理，还可以不断的自学习。通过NPU可以实现电子设备100的智能认知等应用，例如：图像识别，人脸识别，语音识别，文本理解等。
在一些实施方式中，在合拍场景下，处理器110(例如控制器或GPU)可以将摄像头拍摄到的图像和包含对象A的图片进行拟合，并将拟合结果存储于存储器中。处理器110执行的拟合操作的具体实现，可参考后续方法实施例的详细介绍。这样，拟合得到的合成图像同时包含对象A和摄像头拍摄到的对象B，实现了多对象的合影。
在一些实施方式中，处理器110(例如控制器或GPU)可以将电子设备100接收到拍照指令的时刻以及之前摄像头193拍摄的N帧图像，各自和包含对象A的图片进行拟合处理，得到N帧合成图像，然后从这N帧合成帧中选择质量最好的一帧，作为合影结果存储至存储器中。
在一些实施方式中，在合拍场景下，处理器110(例如控制器或GPU)可以将摄像头拍摄到的多帧图像分别和包含对象A的图片进行拟合，并将拟合结果存储于存储器中。例如，将摄像头拍摄到的多帧图像和包含对象A的图片拟合为多张图像，处理器110中的视频编码器可以对多张合成图像进行编码，从而生成一个视频文件。这样，该视频文件的每一帧图像都可以包含对象A和摄像头新拍摄到图像中的对象。
内部存储器121可以包括一个或多个随机存取存储器(random access memory，RAM)和一个或多个非易失性存储器(non-volatile memory，NVM)。内部存储器121用于存储处理器110的拟合结果。
随机存取存储器可以包括静态随机存储器(static random-access memory，SRAM)、动态随机存储器(dynamic random access memory，DRAM)、同步动态随机存储器(synchronous dynamic random access memory,SDRAM)、双倍资料率同步动态随机存取存储器(double data rate synchronous dynamic random access memory,DDR SDRAM，例如第五代DDR SDRAM一般称为DDR5SDRAM)等；非易失性存储器可以包括磁盘存储器件、快闪存储器(flash memory)。
快闪存储器按照运作原理划分可以包括NOR FLASH、NAND FLASH、3D NAND FLASH等，按照存储单元电位阶数划分可以包括单阶存储单元(single-level cell,SLC)、多阶存储单元(multi-level cell,MLC)、三阶储存单元(triple-level cell,TLC)、四阶储存单元(quad-level cell,QLC)等，按照存储规范划分可以包括通用闪存存储(英文：universal flash storage，UFS)、嵌入式多媒体存储卡(embedded multi media Card，eMMC)等。
随机存取存储器可以由处理器110直接进行读写，可以用于存储操作系统或其他正在运行中的程序的可执行程序(例如机器指令)，还可以用于存储用户及应用程序的数据等。
非易失性存储器也可以存储可执行程序和存储用户及应用程序的数据等，可以提前加载到随机存取存储器中，用于处理器110直接进行读写。
外部存储器接口120可以用于连接外部的非易失性存储器，实现扩展电子设备100的存储能力。外部的非易失性存储器通过外部存储器接口120与处理器110通信，实现数据存储功能。例如将音乐，视频等文件保存在外部的非易失性存储器中。
电子设备100可以通过音频模块170，扬声器170A，受话器170B，麦克风170C，耳机接口170D，以及应用处理器等实现音频功能。例如音乐播放，录音等。
触摸传感器180K，也称“触控器件”。触摸传感器180K可以设置于显示屏194，由触摸传感器180K与显示屏194组成触摸屏，也称“触控屏”。触摸传感器180K用于检测作用于其上或附近的触摸操作。触摸传感器可以将检测到的触摸操作传递给应用处理器，以确定触摸事件类型。可以通过显示屏194提供与触摸操作相关的视觉输出。在另一些实施例中，触摸传感器180K也可以设置于电子设备100的表面，与显示屏194所处的位置不同。
电子设备100的软件系统可以采用分层架构，事件驱动架构，微核架构，微服务架构，或云架构。本申请实施例以分层架构的移动操作系统为例，示例性说明电子设备100的软件结构。
图1B是本申请实施例的电子设备100的软件结构框图。
分层架构将软件分成若干个层，每一层都有清晰的角色和分工。层与层之间通过软件接口通信。在一些实施例中，将移动操作系统分为四层，从上至下分别为应用程序层，程序框架层/核心服务层，底层库和运行时，以及内核层。
应用程序层可以包括一系列应用程序包。
如图1B所示，应用程序包可以包括相机，图库，日历，通话，地图，导航，WLAN，蓝牙，音乐，视频，短信息等应用程序。
程序框架层为应用程序层的应用程序提供应用编程接口(application programming interface，API)和编程框架。程序框架层包括一些预先定义的函数。
如图1B所示，程序框架层可以包括窗口管理器，内容提供器，视图系统，电话管理器，资源管理器，通知管理器等。
窗口管理器用于管理窗口程序。窗口管理器可以获取显示屏大小，判断是否有状态栏，锁定屏幕，截取屏幕等。
内容提供器用来存放和获取数据，并使这些数据可以被应用程序访问。所述数据可以包括视频，图像，音频，拨打和接听的电话，浏览历史和书签，电话簿等。
视图系统包括可视控件，例如显示文字的控件，显示图片的控件等。视图系统可用于构建应用程序。显示界面可以由一个或多个视图组成的。例如，包括短信通知图标的显示界面，可以包括显示文字的视图以及显示图片的视图。
电话管理器用于提供电子设备的通信功能。例如通话状态的管理(包括接通，挂断等)。
资源管理器为应用程序提供各种资源，比如本地化字符串，图标，图片，布局文件，视频文件等等。
通知管理器使应用程序可以在状态栏中显示通知信息，可以用于传达告知类型的消息，可以短暂停留后自动消失，无需用户交互。比如通知管理器被用于告知下载完成，消息提醒等。通知管理器还可以是以图表或者滚动条文本形式出现在系统顶部状态栏的通知，例如后台运行的应用程序的通知，还可以是以对话窗口形式出现在屏幕上的通知。例如在状态栏提示文本信息，发出提示音，电子设备振动，指示灯闪烁等。
运行时可以指程序运行时所需的一切代码库、框架等。例如，对于C语言来说，运行时包括一系列C程序运行所需的函数库。对于Java语言来说，除了核心库之外，运行时还包括Java程序运行所需的虚拟机等。上述核心库可包括Java语言需要调用的功能函数。
底层库可以包括多个功能模块。例如：表面管理器(surface manager)，媒体库(Media Libraries)，三维图形处理库(例如：OpenGL ES)，2D图形引擎(例如：SGL)等。
表面管理器用于对显示子系统进行管理，并且为多个应用程序提供了2D和3D图层的融合。
媒体库支持多种常用的音频，视频格式回放和录制，以及静态图像文件等。媒体库可以支持多种音视频编码格式，例如:MPEG4，H.264，MP3，AAC，AMR，JPG，PNG等。
三维图形处理库用于实现三维图形绘图，图像渲染，合成，和图层处理等。
2D图形引擎是2D绘图的绘图引擎。
内核层是硬件和软件之间的层。内核层至少包含显示驱动，摄像头驱动，音频驱动，传感器驱动。
下面结合捕获拍照场景，示例性说明电子设备软件以及硬件的工作流程。
当触摸传感器180K接收到触摸操作，相应的硬件中断被发给内核层。内核层将触摸操作加工成原始输入事件(包括触摸坐标，触摸操作的时间戳等信息)。原始输入事件被存储在内核层。应用程序框架层从内核层获取原始输入事件，识别该输入事件所对应的控件。以该触摸操作是触摸单击操作，该单击操作所对应的控件为相机应用图标的控件为例，相机应用调用应用框架层的接口，启动相机应用，进而通过调用内核层启动摄像头驱动，通过摄像头193捕获静态图像或视频。
图1C是本申请实施例提供的电子设备100的另一种软件结构框图。
如图1C所示，电子设备100包括：图库模块、拍照模块、显示模块、识别模块、合成模块。其中：
图库模块，用于存储多张图片，其中包括包含对象A的一张图片。图库模块还可以提供开启“合拍”功能的入口，并可以用于存储电子设备100的拟合结果。
拍照模块，用于负责拍摄图像。
显示模块，用于显示具有透明度的或虚化的该图片，并同时显示拍照模块拍摄到的图像。或者，显示模块用于显示由拍照模块拍摄到的图像和包含对象A的图片拟合而来的合成图像。
识别模块，用于识别拍照模块拍摄到的图像中的对象B所在的区域，以及，除对象B以外的背景区域。或者，识别模块用于识别包含对象A的图片中该对象A所在的区域，以及，除对象A所在区域以外的背景区域。
合成模块，用于将拍照模块拍摄到的图像中的对象B所在的区域，拟合到包含对象A的图片中。或者，合成模块用于将包含对象A的图片中该对象A所在的区域，拟合到拍照模块拍摄到的图像中。
下面介绍本申请实施例提供的用户界面。
图2A-图2H示例性示出了本申请实施例中电子设备100提供的一组用于合影的用户界面。
图2A示例性示出了电子设备100上的用于展示已安装应用程序的示例性用户界面21。
用户界面21显示有：状态栏201、日历(calendar)和时间指示符202、天气指示符203、页面指示符204、具有常用应用程序图标的托盘205、其他应用程序图标。其中：
状态栏201可包括：移动通信信号(又可称为蜂窝信号)的一个或多个信号强度指示符、蓝牙指示符、Wi-Fi信号的一个或多个信号强度指示符，电池状态指示符、时间指示符等等。
日历(calendar)和时间指示符202用于指示日历(calendar)和当前时间。天气指示符203用于指示天气。
页面指示符204可用于指示用户当前浏览的是哪一个页面中的应用程序图标。在本申请实施例，应用程序图标可分布在多个页面，用户可以左右滑动，来浏览不同页面中的应用程序图标。
具有常用应用程序图标的托盘205可展示：电话图标、短信图标、相机图标205a、通讯录图标等等。
其他应用程序图标例如可包括视频应用的图标、线上会议应用的图标、日历(calendar)的图标、游戏应用的图标、地图应用的图标、设置应用的图标、图库应用的图标206a等等。
其中，相机是电子设备100上的一款用于调用摄像头来拍摄图像的APP，本实施例对该应用程序的名称不做限制。本申请实施例的相机APP还提供了“合拍”功能。相机APP可以是电子设备100提供的系统应用，也可以是第三方应用。
图库是智能手机、平板电脑等电子设备100上的一款图片管理的APP，又可以称为“相册”，本实施例对该应用程序的名称不做限制。该APP可以支持用户对存储于电子设备100上的图片进行各种操作，例如浏览、编辑、删除、选择等操作。即“图库”管理的对象是图片。在另一些情况下，该APP也可以支持用户对存储在云端服务器上的图片进行上述各种操作。可以理解的是，在本实施例中，图片可以是电子设备100利用摄像头193捕捉到的，也可以是从其他应用程序中获取到的或者从网页上下载的。本申请实施例的图库还提供了“合拍”功能的入口。图库可以是电子设备100提供的系统应用，也可以是第三方应用。
不限于此，电子设备100中还可安装有更多的应用，并可以在显示屏上显示这些应用的图标。例如，电子设备100中还可以安装有购物类应用、订票类应用等等。购物类应用可用于在网上购买物品。订票类应用可用于订购各类票据，例如机票、火车票、船票等等。
上述各个应用的名称仅为本申请实施例所使用的词语，其代表的含义在本实施例中已经记载，其名称并不能对本实施例构成任何限制。
不限于此，图2A所示的用户界面21还可包括导航栏、侧边栏等等。在一些实施例中，图2A示例性所示的用户界面21可以称为主界面(home screen)。
图2B示例性示出了电子设备100的图库APP提供的用户界面22。
图2B可以是电子设备100检测到作用于图2A中图库应用的图标206a的用户操作(如点击操作、触摸操作)后，响应于该用户操作而显示的用户界面。
如图2B所示，用户界面22可包括：状态栏，应用程序标题栏，图片区域207。其中：
状态栏可参考图2A所示的用户界面21中的状态栏201，这里不再赘述。
应用程序标题栏可包括：返回键，以及当前页面指示符。返回键为APP级返回键，可用于返回菜单上一级。本领域技术人员可以理解，一个页面的逻辑上一级是固定的，在应用程序设计时便已确定。当前页面指示符可用于指示当前页面，例如文本信息“图库”，不限于文本信息，当前页面指示符还可以是图标。
图片区域207中可显示有一个或多个图片，例如图片207a。当电子设备在图片区域207中检测到向上或向下的滑动操作时，响应于该滑动操作，电子设备可以更新图片区域207中显示的图片，以便用户浏览图片。也即是说，用户可以在图片区域207中向上或向下滑动来浏览更多的图片。不限于向上或向下的滑动操作，用户还可以在图片区域207中向左或向右滑动来浏览更多的图片。图片区域207中显示的图片可以是缩略图。此时，图片区域207中显示的图片对应的原图可以存储于电子设备上，也可以存储于云端服务器上。如无另外说明，以下实施例中所指的图片可以是存储在电子设备中的，也可以是存储在云端服务器上的。
如图2B所示，电子设备100可以在用户界面22中检测到用户作用于图片207a的用户操作(如点击操作、触摸操作等)。响应于该操作，电子设备100可以显示如图2C所示的用户界面23。
如图2C所示，电子设备100可以在用户界面23中以更大的区域展示图片207a对应的原图，用户界 面23展示有图片208，图片208可以是图片207a的原图。在一些实施例中，图片208可以是电子设备100在某个地点(例如地点1)拍摄得到的，也可以由电子设备100从网络中获取到。该图片208中包含有一个人物对象以及背景区域，该人物对象可以被称为对象A，图片208可以被称为包含对象A的图片。背景区域展示了该对象A所处的位置，例如图2C所示图片208的背景区域显示有椰树、海鸥、山坡、太阳。用户界面23中还可以显示有：分享控件、收藏控件、编辑控件、删除控件、更多控件209等。其中，分享控件、收藏控件、编辑控件、删除控件用于监听用户操作，电子设备100可响应于该用户操作，针对图片208执行分享、收藏、编辑、删除等操作。更多控件209可用于监听用户操作(如点击操作、触摸操作等)，电子设备100可响应于该用户操作，显示如图2D所示的功能列表210。
如图2D所示，功能列表210中显示有多个功能控件，例如用于将图片208复制到相册、移动到相册、投屏、打印等的控件等等。功能列表210显示的多个功能控件中包括用于开启针对图片208的“合拍”功能的控件210a，该控件210a用于监听用户操作(如点击操作、触摸操作等)，电子设备100可响应于该用户操作，启动针对图片208的“合拍”功能。
通过上述图2B-图2D的示例，用户可以通过点击图片以查看图片，然后通过显示该图片的用户界面所提供的入口，启动针对该图片的“合拍”功能。
不限于在展示一个图片的用户界面中点击控件210a，电子设备100还可以响应于其他用户操作来启动针对图片的“合拍”功能。例如，电子设备100还可以在图2B显示的图片207a上接收到长按操作后启动对该图片的“合拍”功能，或者，电子设备100还可以在图2B显示的图片207a上接收到画圆圈手势后启动对该图片的“合拍”功能，或者，电子设备100还可以在图2C所示的图片208上接收到长按操作后启动对该图片的“合拍”功能，或者，电子设备100还可以是语音控制指令。本申请对触发电子设备100启动针对图片的“合拍”功能的操作的具体实现不做限制。
不限于“图库”中的图片，用户触发针对图片的“合拍”功能的场景还可包括用户查看网络中的图片时触发针对该图片的“合拍”功能、用户触发针对文件浏览器等应用程序中图片的“合拍”功能等等。
图2E-图2H示例性示出了电子设备100开启针对图片208的“合拍”功能后，相机应用所提供的一组用户界面。
电子设备100启动针对图片208的“合拍”功能后，可以自动启动相机应用。
图2E为电子设备100启动相机应用后所显示的用户界面24。
图2E示例性所示的用户界面24可以由电子设备100检测到作用于如图2D所示的控件210a上的用户操作(如点击操作、触摸操作等)后显示。用户界面24可以是相机应用的默认拍照模式的用户界面，可用于用户通过默认后置摄像头进行拍照。
如图2E所示，用户界面24可包括：区域211、拍摄模式列表212、控件213、控件214及控件215。其中：
区域211可以称为预览框211或取景框211。预览框211可用于显示具有一定透明度的或者虚化的图片208。
在电子设备的摄像头193启动并采集到图像后，预览框211还可显示摄像头193实时采集的图像。电子设备100可以实时刷新其中的显示内容，以便于用户预览摄像头193当前的采集的图像。电子设备100可以将具有一定透明度的或者虚化的图片208叠加显示在摄像头193实时采集的图像之上，由于图片208具有一定透明度和/或虚化，因此用户不仅可以在预览框211中看到图片208，也可以看到图片208之下的摄像头193实时采集的图像。
在一些实施方式中，电子设备100启动“合拍”功能并启动相机应用后，可以先仅在预览框211中显示具有一定透明度或虚化的图片208，如图2E所示，之后再启动摄像头193并将实时采集到的图像也显示在预览框211中，如图2F所示。
在另一些实施方式中，电子设备100启动“合拍”功能并启动相机应用后，可以直接启动摄像头193采集图像，此时电子设备100可以直接在预览框211中同时显示具有一定透明度或虚化的图片208和摄像头193实时采集的图像。也就是说，电子设备100在接收到图2E中作用于控件210a的用户操作后，可以直接显示图2F所示的用户界面，无需显示图2E所示的用户界面。
拍摄模式列表212中可以显示有一个或多个拍摄模式选项。这一个或多个摄像选项可以包括：拍照模式选项212A、录像模式选项212B、人像模式选项212C、全景模式选项212D以及更多选项212E。这一个或多个摄像选项在界面上可以表现为文字信息，例如“拍照”、“录像”、“人像”、“全景”“更多”。不限于 此，这一个或多个摄像选项在界面上还可以表现为图标或者其他形式的交互元素(interactive element，IE)。
控件213可用于监听触发拍摄(拍照或录像)的用户操作。电子设备可以检测到作用于控件213的用户操作(如在控件213上的点击操作)，响应于该操作，电子设备100可以保存摄像头193采集的图像并将其和图片208进行拟合操作，从而生成图库中的图片。当用户切换到录像模式时，控件213可以更改为对应的录像控件，电子设备可以检测到的作用于录像控件的用户操作，响应于该操作，电子设备100可以保存摄像头193采集的多帧图像并将其和图片208分别进行拟合操作，从而生成图库中的视频。另外，电子设备100还可以在控件214中显示所保存的图像的缩略图，该图像可以是电子设备100在开启“合拍”功能后生成的，也可以是未开启“合拍”功能的情况下生成的。也即是说，用户可以点击控件213或录像控件来触发拍摄。其中，控件213或录像控件可以是按钮或者其他形式的控件。本申请中，可以将控件213称为拍照控件。拍照控件和录像控件可以被统称为拍摄控件。
控件215可用于监听触发翻转摄像头的用户操作。电子设备100可以检测到作用于控件215的用户操作(如在控件215上的点击操作)，响应于该操作，电子设备100可以翻转摄像头，例如将后置摄像头切换为前置摄像头。
电子设备100可以检测到作用于拍摄模式选项上的用户操作，该用户操作可用于选择拍摄模式，响应该操作，电子设备100可以开启用户选择的拍摄模式。特别的，当该用户操作作用于更多拍摄模式选项212E时，电子设备100可以进一步显示更多的其他拍摄模式选项，如慢动作拍摄模式选项等等，可以向用户展示更丰富的摄像功能。不限于图2E所示，拍摄模式列表212中可以不显示更多拍摄模式选项212E，用户可以通过在拍摄模式列表212中向左/右滑动来浏览其他拍摄模式选项。
如图2F所示，预览框211中同时显示有较浅颜色的图像和较深颜色的图像，其中，较浅颜色的图像为具有一定透明度的或者虚化的图片208，较深颜色的图像是摄像头193实时采集的图像。
在预览框211中，具有一定透明度的或者虚化的图片208的位置保持不变。同时，用户使用电子设备100进行拍摄时，可以通过移动电子设备来改变当前所使用的摄像头在预览框中的取景。也就是说，用户可以通过移动电子设备，来更改摄像头193的取景，从而更改预览框211中显示的摄像头193实时采集的图像。例如，用户可以通过向左或向右或向前或向后移动电子设备，还可以手持电子设备走动等。在其他一些实施方式中，用户还可以调整当前使用的摄像头的焦距(如电子调焦)，或者，更改当前所使用的摄像头(例如将后置摄像头更换为前置摄像头)等，从而更改预览框211中实时显示的摄像头采集的图像。在其他一些实施方式中，还可以指示摄像头193拍摄的人物对象、动物对象等更改所处的位置，从而调整预览框211中的取景。
预览框211可以向用户展示图片208和摄像头193实时采集的图像中的各个对象的相对位置。用户根据预览框211呈现的内容，可以调整该预览框211中实时显示的摄像头采集的图像，从而让摄像头采集的图像中包含对象B并且该对象B处于用户满意的和图片208中的对象A的一个相对位置。和对象A有合影需求的对象称为对象B。
如图2F所示，图片208包含对象A和背景区域，背景区域展示了该对象A所处的位置，例如图2F中预览框211展示的图片208的背景区域说明对象A所处的位置有椰树、海鸥、山坡、太阳。
在一些实施方式中，当预览框211中显示的摄像头采集的实时图像的背景区域所指示的位置，和，图片208所指示的位置(例如但不限于图片208的背景区域所指示的位置)，具有一定相关性时，控件213才可以被激活，在其他情况下控件213即使接收到用户操作，电子设备100也不会执行后续的响应。控件213激活后，用户可以让摄像头采集的图像中包含对象B并且该对象B处于满意的和图片208中的对象A的一个相对位置时点击控件213，从而触发电子设备100保存摄像头193采集的图像并将其和图片208进行拟合操作。即，只有图片208所指示的位置，和，当前电子设备100的摄像头193实时采集的图像所指示的位置，具有一定相关性时，用户才可以触发电子设备100拍摄图像并执行后续的拟合操作。
在其他一些实施例中，控件213可以一直被激活，只要用户让摄像头采集的图像中包含对象B并且该对象B处于令该用户满意的和图片208中的对象A的一个相对位置，用户就可以点击控件213，从而触发电子设备100保存摄像头193采集的图像并将其和图片208进行拟合操作。
参考图2F，在图2F中摄像头193拍摄到的图像中包含一个人物对象，该人物对象可以为对象B。由于用户对图2F中对象B和对象A的相对位置不太满意，因此用户调整了预览框211中摄像头193的取景，从而将预览框211中的对象B和对象A调整到如图2G所示的令其满意的相对位置处。
如图2G所示，电子设备100可以检测到作用于控件213的用户操作(例如点击操作、触摸操作等)，响应于该操作，电子设备100可以保存摄像头193采集的图像并将其和图片208进行拟合操作，从而生成 图库中的图片。该拟合操作的具体实现，可参考后续方法实施例中的详细介绍。此外，参考图2H，电子设备100还可以将拟合操作后生成的图片的缩略图，显示在控件214中，以供用户点击并查看。
在一些实施方式中，电子设备100将摄像头193采集的图像和图片208进行拟合时，有可能拟合失败，不能生成对应的合成图像。例如，在摄像头193采集的图像的背景区域和图片208相关性较低时，电子设备100将不能成功执行拟合操作。
电子设备100响应于作用于控件213的用户操作拍照并执行拟合操作后，该电子设备100可以持续开启“合拍”功能，这样用户就可以多次拍照，达到多次合影的目的。在其他一些实施例中，电子设备响应于作用于控件213的用户操作拍照并执行拟合操作后，也可以关闭“合拍”功能，进入普通的拍照模式。
不限于图2E-图2G示出的拍照模式，用户还可以选择录像模式或其他模式，电子设备100还可以在录像模式或其他模式下拍摄，达到在视频或其他模式中合影的目的。
图3A-图3F示例性示出了本申请实施例中电子设备100提供的另一组用于合影的用户界面。
图3A-图3E和图2A-图2G的不同之处在于，两种合影方式中“合拍”功能的入口不同。
图3A所示的用户界面31和图2A所示的用户界面21相同，可参考前文图2A及相关描述。
图3B示例性示出了电子设备100的相机应用提供的用户界面32。
图3B可以是电子设备100检测到作用于图3A中相机应用的图标205a的用户操作(如点击操作、触摸操作)后，响应于该用户操作启动相机应用并显示的用户界面。用户界面32可以是相机应用的默认拍照模式的用户界面，可用于用户通过默认后置摄像头进行拍照。
如图3B所示，用户界面32可包括：区域301、拍摄模式列表302、控件303、控件304及控件305、用于启动“合拍”功能的控件306。其中：
区域301可以称为预览框301或取景框301。预览框301可用于显示电子设备100的摄像头193实时采集的图像。电子设备100可以实时刷新其中的显示内容，以便于用户预览摄像头193当前的采集的图像。
拍摄模式列表302中可以显示有一个或多个拍摄模式选项。这一个或多个摄像选项可以包括：拍照模式选项302A、录像模式选项302B、人像模式选项302C、全景模式选项302D以及更多选项302E。
控件303可用于监听触发拍摄(拍照或录像)的用户操作。电子设备可以检测到作用于控件303的用户操作(如在控件303上的点击操作)，响应于该操作，电子设备100可以保存预览框211中显示的图像。当用户切换到录像模式时，控件303可以更改为对应的录像控件，电子设备可以检测到的作用于录像控件的用户操作，响应于该操作，电子设备100可以保存预览框211中显示的图像。另外，电子设备100还可以在控件304中显示所保存的图像的缩略图。也即是说，用户可以点击控件303或录像控件来触发拍摄。其中，控件303或录像控件可以是按钮或者其他形式的控件。本申请中，可以将控件303称为拍照控件。拍照控件和录像控件可以被统称为拍摄控件。
控件305可用于监听触发翻转摄像头的用户操作。电子设备100可以检测到作用于控件305的用户操作(如在控件305上的点击操作)，响应于该操作，电子设备100可以翻转摄像头，例如将后置摄像头切换为前置摄像头。
电子设备100可以检测到作用于拍摄模式选项上的用户操作，该用户操作可用于选择拍摄模式，响应该操作，电子设备100可以开启用户选择的拍摄模式。
控件306可用于监听用户操作(如点击操作、触摸操作等)，电子设备100可响应于该用户操作，在用户界面33中显示如图3C所示的图片区域307。图片区域307中可显示有一个或多个图片，例如图片307a。当电子设备在图片区域307中检测到向上或向下的滑动操作时，响应于该滑动操作，电子设备可以更新图片区域307中显示的图片，以便用户浏览图片。也即是说，用户可以在图片区域307中向上或向下滑动来浏览更多的图片。不限于向上或向下的滑动操作，用户还可以在图片区域307中向左或向右滑动来浏览更多的图片。图片区域307中显示的图片可以是缩略图。此时，图片区域307中显示的图片对应的原图可以存储于电子设备上，也可以存储于云端服务器上。
图片区域307中还可显示有提示信息307b。提示信息307b用于提示用户在图片区域307展示的图片中选择一张图片作为合拍的照片。
电子设备100可以检测到作用于图片307a上的用户操作(例如点击操作、触摸操作等)，并响应于该操作，启动针对该图片307a对应的原图的“合拍”功能，并显示图3D所示的用户界面34。
如图3D所示，在启动“合拍”功能后，电子设备100可以更改控件306的显示形式(例如可以更改其显示颜色、更改其显示形状、或如图3D所示将空心圆圈更改为实心圆圈等)，以提示用户当前电子设备 100已开启“合拍”功能。
在图3D所示的用户界面34中，电子设备100开启“合拍”功能后，预览框301显示有具有一定透明度的或者虚化的图片208，以及摄像头193实时采集的图像。图片208可以是图3C所示的图片307a对应的原图。电子设备100可以实时刷新其中的显示内容，以便于用户预览摄像头193当前的采集的图像。电子设备100可以将具有一定透明度的或者虚化的图片208叠加显示在摄像头193实时采集的图像之上，由于图片208具有一定透明度和/或虚化，因此用户不仅可以在预览框211中看到图片208，也可以看到图片208之下的摄像头193实时采集的图像。
图3D所述用户界面34中预览框301的作用、拍摄模式列表302的作用、控件304-控件305的作用，均可以参考图2F中对应控件的作用。可以认为，图3D和图2G除了控件306的区别以外，其他区域或控件均相同。
参考图3D，在图3D中摄像头193拍摄到的图像中包含一个人物对象，该人物对象可以为对象B。由于用户对图3D中对象B和对象A的相对位置不太满意，因此用户调整了预览框301中摄像头193的取景，从而将预览框301中的对象B和对象A调整到如图3E所示的令其满意的相对位置处。
如图3E所示，电子设备100可以检测到作用于控件303的用户操作(例如点击操作、触摸操作等)，响应于该操作，电子设备100可以保存摄像头193采集的图像并将其和图片307a对应的原图进行拟合操作，从而生成图库中的图片。该拟合操作的具体实现，可参考后续方法实施例中的详细介绍。此外，参考图3F，电子设备100还可以将拟合操作后生成的图片的缩略图，显示在控件304中，以供用户点击并查看。
在一些实施方式中，电子设备100将摄像头193采集的图像和图片307a对应的原图进行拟合时，有可能拟合失败，不能生成对应的合成图像。例如，在摄像头193采集的图像的背景区域和图片307a对应的原图相关性较低时，电子设备100将不能成功执行拟合操作。
图4A-图4B示例性示出了本申请实施例中电子设备100保存拟合的图片后，图库应用所提供的用户界面。
图4A可以是电子设备100检测到作用于图2A或图3A中图库应用的图标206a的用户操作(如点击操作、触摸操作)后，响应于该用户操作而显示的用户界面。图4A所示的用户界面和图2B所示的用户界面22类似，可参考相关描述。如图4A所示，用户界面41所示的图片区域207中新增了图片207b，图片207b可以是电子设备100保存的拟合后的图片的缩略图或原图。
如图4A所示，电子设备100检测到作用于图片207b上的用户操作(如点击操作、触摸操作)后，可以响应于该操作，在图4B所示的用户界面中以更大的区域展示图片207b对应的原图，例如图片401。在其他一些实施例中，图4B所示的用户界面43还可以是电子设备100响应于在图2H所示的控件214，或，在图3F所示的控件304上的用户操作而显示的。
在一些实施方式中，图4A中所显示的图片207b，或者，图4B所显示的图片401上，还可以显示有“合拍”标识，用于指示该图片是电子设备100在开启“合拍”功能后拍摄的。本申请实施例对该“合拍”标识的实现形式不做限定，例如可以是文字、图标等等。
基于前文介绍的电子设备100以及在电子设备100上实现的用户界面，下面介绍本申请实施例提供的拍摄方法。
参考图5，图5示例性示出了该拍摄方法的流程。图5所示的拍摄方法以拍照场景为例进行说明。
如图5所示，该拍摄方法可包括如下步骤：
S101，电子设备100检测到用于开启针对第一图片的“合拍”功能的第一操作。
S102，响应于该第一操作，电子设备100开启针对第一图片的“合拍”功能，第一图片包含对象A。
具体的，第一图片可以是电子设备100保存的摄像头193拍摄的图片，也可以是电子设备100从网络中下载的图片，也可以是接收到的其他设备分享的图片。第一图片可以是很早以前拍摄的照片，也可以是电子设备100刚刚拍摄的照片。第一图片中包含用户想要与之合影的对象，该对象可以被称为对象A。对象A的数量可以为一个或多个。对象A例如可包括但不限于：人物、动物、地标、建筑物、玩偶等等。例如，第一图片也可以是一张亲人在多年前拍摄的老照片，等等。
示例性地，第一图片例如可以是图2C所示的图片207a对应的原图，图2C所示的图片208，图3C所示的图片307a对应的原图，或，图3D所示的图片301。对象A可以是上述图片中的人物对象。
“合拍”功能由电子设备100提供，该功能支持电子设备100在相机应用提供的预览界面中显示具有 透明度的或虚化的第一图片，并同时显示电子设备100的摄像头193采集到的图像，电子设备100接收到用户输入的拍照指令，将该第一图片和摄像头采集到的图像进行拟合，生成合成图像。开启“合拍”功能后，该电子设备将执行后续的各项操作。
在一些实施方式中，参考前述图2A-图2D以及相关描述，用于开启针对第一图片的“合拍”功能的第一操作例如可以是在图2C所示的展示图片208的用户界面23上输入针对控件209后的操作后，作用在电子设备100显示的功能列表210中的控件210a上的用户操作。控件209可以也被称为第一控件，控件210a也可以被称为第二控件。
在一些实施方式中，用于开启针对第一图片的“合拍”功能的第一操作例如可以是，电子设备100在图2B显示的图片207a上接收到的长按操作，或者，电子设备100在图2B显示的图片207a上接收到的画圆圈手势，或者，电子设备100在图2C所示的图片208上接收到的长按操作。
在本申请实施例中，图库应用提供的用于展示第一图片的用户界面可以被称为第一用户界面，例如图2C、图2D均可以被称为第一用户界面。
在一些实施方式中，参考前述图3A-图3C以及相关描述，用于开启针对第一图片的“合拍”功能的第一操作例如可以是电子设备100接收到作用于图3B中的控件306上的用户操作后，作用于电子设备100显示的图片区域307中的图片307a上的用户操作。控件306也可以被称为第三控件，作用于控件306上的用户操作也可以被称为第四操作，作用于图片307a上的用户操作也可以被称为第五操作。
其中，图3B所示的用户界面不仅可以是相机应用提供的，也可以是电子设备中的其他应用(如社交类应用、购物类应用等)调用相机接口所提供的用户界面。相当于，电子设备中的其他应用可以调用相机接口，进而启用“合拍”功能。由相机应用提供的或者由其他应用调用相机接口所提供的，用于开启“合拍”功能的用户界面，例如图3B、图3C所示的用户界面，可以被称为第二用户界面。
不限于此，用于开启针对第一图片的“合拍”功能的第一操作还可以实现为语音控制指令、隔空手势指令、实体按键上的按压操作等等。本申请对第一操作的具体实现不做限制。
S103，电子设备100启动相机应用，并显示第二用户界面，第二用户界面显示有预览框，预览框中显示有具有透明度和/或虚化的第一图片，以及，摄像头193采集到的图像。
电子设备100开启针对第一图片的“合拍”功能后，可以自动启动相机应用。相机应用可以是安装于电子设备100中的系统应用，也可以是第三方应用。
具体的，电子设备100启动相机应用后，可以显示该相机应用提供的第二用户界面，该第二用户界面显示有预览框，预览框中显示有具有透明度和/或虚化的第一图片，以及，摄像头193实时采集到的图像。这里，第二用户界面也可以被称为预览界面。在一些实施方式中，电子设备100可以将具有透明度和/或虚化的第一图片，叠加显示在摄像头193实时采集到的图像之上。
其中，如果第一图片具有透明度，则用户既能够看到该第一图片，又能够透过具有透明度的该第一图片看到电子设备100所显示的被其覆盖的其他内容。第一图片的透明度的大小可以预先设置，也可以由用户按照实际需求自主调节，还可以根据当前电子设备100所处环境的光线等进行调节，或者，根据当前摄像头193采集到的图像的明亮程度进行调节等等。
如果第一图片被虚化，则电子设备100可以首先识别第一图片的轮廓或内容，将其轮廓或内容以虚线描绘，或者将其内容做模糊处理，以得到虚化后的第一图片。这种情况下，用户也既能够看到该第一图片，又能够透过虚化的该第一图片看到电子设备100所显示的被其覆盖的其他内容。
第二用户界面中预览框显示的摄像头193实时采集到的图像是实时刷新的。当用户移动电子设备100或改变电子设备100的姿态、更改所使用的摄像头，或者，当前摄像头193拍摄的对象例如人物或动物更改所在位置等等，则摄像头193采集到的图像都会实时变化，预览框中将实时显示变化的该图像。
此外，预览框中显示的摄像头193实时采集的图像还可以是在该摄像头193采集的原始图像之上处理后得到的。例如，电子设备100可以对摄像头193采集的原始图像进行防抖处理后，发送给显示屏以显示在预览框中。又例如，如果用户调整了摄像头193的焦距，则电子设备100将对摄像头193采集的原始图像进行裁剪，并将裁剪后的结果发送给显示屏以显示在预览框中。又例如，用户可以选择当前的拍摄模式，在不同模式下电子设备100可以对摄像头193采集到的原始图像做不通的处理，例如在人像模式下对背景区域做模糊处理等，然后将裁剪后的结果发送给显示屏以显示在预览框中。
示例性地，参考图2F-图2G，第二用户界面例如可以是图2F-图2G所示的用户界面22，预览框例如可以是图2F-图2G中的预览框211。图2F-图2G中显示的较浅颜色的图片208可以是预览框中显示的具 有透明度的第一图片，较深颜色的图像为摄像头193实时采集到的图像。
示例性地，参考图3D-图3E，第二用户界面例如可以是图3D-图3E所示的用户界面34，预览框例如可以是图3D-图3E中的预览框301。图3D-图3E中显示的较浅颜色的图片208可以是预览框中显示的具有透明度的第一图片，较深颜色的图像为摄像头193实时采集到的图像。
可见，第二用户界面中的预览框可以向用户展示第一图片和摄像头193实时采集的图像中的各个对象的相对位置。用户可以根据预览框呈现的内容，调整该预览框中实时显示的摄像头采集的图像，从而让摄像头采集的图像中包含的对象和第一图片中的对象A的相对位置符合用户的实际需求。其中，用户调整预览框中实时显示的摄像头采集的图像的方式可包括但不限于：用户移动电子设备100或改变电子设备100的姿态、用户更改电子设备100所使用的摄像头，或者，用户指示当前摄像头193拍摄的对象例如人物或动物更改所在位置，等。示例性地，参考图2F-图2G或者参考图3D-图3E，用户可以调整摄像头193拍摄到的图像中包含的人物对象和第一图片中的对象A(即图中的人物对象)的相对位置。
在本申请实施例中，电子设备100除了在相机应用提供的第二用户界面中显示具有透明度或虚化的第一图片以指示该第一图片，还可以通过显示第一图片包含的对象的线条、第一图片中对象的所在位置、淡化的第一图片等等，以在预览框中展示第一图片，从而让用户获知第一图片中的对象和摄像头193实时采集的图像中的对象的相对位置。上述用于提示用户第一图片中的对象和摄像头193实时采集的图像中的对象的相对位置的信息，可以被称为第一图片的指示信息。例如，第一图片的指示信息可包括在预览框中显示的具有透明度或虚化的第一图片、第一图片包含的对象的线条、第一图片中对象的所在位置、淡化的第一图片等等，本申请实施例对此不做限制。
S104，电子设备100接收到第二操作。
S105，响应于该第二操作，电子设备100将第一图片，和，摄像头193采集到的图像进行拟合，生成合成图像。
第二操作可以是用于触发电子设备100拍照或录像的用户操作。用于触发电子设备100拍照的用户操作，也可以被称为拍照指令。用于触发电子设备100录像的用户操作，也可以被称为录像指令。
示例性地，参考图2F-图2G，第二操作例如可以是作用于拍照控件213的用户操作，也可以是用户选择录像模式后作用于录像控件的用户操作。参考图3D-图3E，第二操作也可以是作用于拍照控件303的用户操作，或者是用户选择录像模式后作用于录像控件的用户操作。
在一些实施方式中，只有第二用户界面的预览框中显示的摄像头193采集到的图像所指示的位置，和，第一图片所指示的位置之间的相关性大于第一值时，电子设备100才会响应该第二操作。其中，图片之间的相关性是指各图片在相对位置上的相似性程度和匹配程度。如果两幅图片的相关性较大，则表明该两幅图片包含的内容相似，两幅图片可能是在同一地点以类似的视角取景得到的。第一值可以预先设置，这里不做限定。例如，参考前文图2E-图2G的相关描述，当预览框211中显示的摄像头采集的实时图像的背景区域所指示的位置，和，图片208所指示的位置(例如但不限于图片208的背景区域所指示的位置)，具有一定相关性时，控件213才可以被激活，在其他情况下控件213即使接收到用户操作，电子设备100也不会执行后续的响应。
在另一些实施方式中，电子设备100显示第二用户界面后，如果接收到第二操作，就会立即响应该第二操作。例如，参考前文图2E-图2G的相关描述，控件213可以一直被激活，只要用户让摄像头采集的图像中包含的对象处于令该用户满意的和图片208中的对象A的一个相对位置，用户就可以点击控件213，从而触发电子设备100响应该操作而执行后续步骤。
下面将结合图6A-图6B示出的几种图像处理过程，介绍电子设备100如何将第一图片，和，摄像头193采集到的图像进行拟合，以生成合成图像。该过程具体可包括如下步骤：
步骤1.识别摄像头采集到的图像中的对象，该对象可以称为对象B。
具体的，电子设备100可以识别摄像头采集到的图像中的人物、动物、地标、建筑物、玩偶等较为标志性的对象，将其作为对象B。在其他一些实施方式中，电子设备100也可以根据摄像头采集到的图像中各个对象和该摄像头之间的距离，将距离摄像头最近的对象识别为对象B。在另一些实施方式中，也可以由用户在预览框显示的摄像头采集的图像中选定对象作为该对象B，例如用户可以点击预览框中想要的对象所在的位置，之后电子设备100可以自动圈中该位置的人体或景物，不需要用户手动绘制人、物的轮廓，即可选定对象B。
在一些实施方式中，电子设备100可以通过图像识别算法来识别摄像头所采集图像中包含的对象B。 图像识别算法例如可以是人工智能(artificialintelligence，AI)算法。
在本申请实施方式中，电子设备100识别到的对象B的数量可以为一个，也可以为多个，这里不做限制。
示例性地，参考图6A-图6B，图6A-图6B中各自的(a)均示出了电子设备100的摄像头采集到的一帧图像。电子设备100识别到的该图像中的对象B如图所示，为图中的男性人物对象。其中，电子设备采集图6B中的(a)所使用的焦距，大于，采集图6A中的(a)所使用的焦距。
步骤2.在摄像头采集到的图像中，以对象B为基准确定包含对象B的主区域。
在一些实施方式中，主区域可以是在摄像头采集到的图像中，以对象B为基准，向周围扩展距离M，得到的包含对象B的区域。
M可以是经由测试得到经验值。在一些实施方式中，同一款电子设备的M值可以相同，不同款电子设备的M值可以不同，或者，同一个电子设备上不同摄像头对应的M值可以不同。这样能够保证不同电子设备或不同摄像头拍摄得到的图像中都能够得到一个包含对象B的主区域。
通过经验值来设置M值，能够保证电子设备找到一个准确的主区域，既能够保证该主区域中对象B的完整性，又能够避免过于扩大范围，从而保证后续拟合的效果。
在一些实施方式中，M可以是一个值，也可以是一组值。M为一组值时，该组值中各个值可以不同，也就是说，电子设备100以对象B为基准，可以在不同位置向周围扩展不同的距离，从而得到包含对象B的主区域。
示例性地，参考图6A-图6B中的(b)，其中虚线围合的区域为一种可能的包含对象B的主区域。图6A-图6B中摄像头各自采集到的图像包含的内容的大小、位置可以不同。
在另一些实施方式中，主区域可以是在摄像头采集到的图像中，以对象B为基准，向周围扩展得到的，和对象B具有相近或相同的颜色或像素的区域。
步骤3.在第一图片中执行针对主区域的相关性搜索，找到相关性大于第二值的同位区。
具体的，在第一图片中执行针对主区域的相关性搜索是指，先找出摄像头采集的图像中除主区域外的背景区域，然后识别第一图片中和该背景区域的相关性大于第二值的区域和/或相关性最高的区域，第一图片中除去该区域的其余部分即为同位区。相当于，第一图片中同位区以外的其余部分，和，摄像头采集的图像中除主区域外的其余部分的相关性大于第二值。第二值可以预先设置，本申请实施例对此不做限定。
本申请对第一图片中的同位区的大小、形状、位置不做限定。第一图片中的同位区在该第一图片中的位置，和，摄像头采集的图像中包含对象B的主区域在该图像中的位置相同，这样电子设备100在预览框中为用户呈现的对象A和对象B的相对位置，和拟合之后的合成图像中对象A和对象B的相对位置相同，这样才能让用户根据预览框中的内容来实时地调整合影效果。在一些实施方式中，第一图片中的同位区的大小、形状，和，摄像头采集的图像中包含对象B的主区域的大小、形状，可以相同，也可以不同。
示例性地，参考图6A的(b)和(c)，第一图片中的同位区，和，摄像头采集的图像中主区域的大小、形状、位置，均相同。
示例性地，参考图6B的(b)和(c)，第一图片中的同位区，和，摄像头采集的图像中主区域的形状和位置均相同，第一图片中同位区的大小大于摄像头采集图像中主区域的大小。
通过步骤3，如果电子设备100能够在第一图片中找到和主区域对应的同位区，则说明拍摄第一图片的地理位置，和，当前电子设备100拍摄图像所处的地理位置相同或类似，这就实现了在同一地点或相似地点的合影。并且，由于在同一地点或相似地点，那么第一图片中的光线、背景，和摄像头拍摄到的图像的光线、背景均类似，这能够极大地提升后续步骤4中的拟合效果，从而达到较好的合影效果。这让本申请实施例提供的拍摄方法区别于单纯的抠图合影，该方法能够带来更加自然的合影效果。
在一些实施方式中，电子设备100显示第二用户界面后，如果在接收到第二操作，立即响应该第二操作，则可能会由于当前摄像头采集到的图像的不确定性，导致步骤3执行失败，即电子设备100无法在第一图片中找到对应的同位区。在这种情况下，电子设备100可以输出提示信息，以提示用户当前合成失败。
步骤4.将主区域拟合到第一图片的同位区中，保持第一图片其余区域不变，生成合成图像。
具体的，电子设备可以对摄像头拍摄的包含对象B的主区域进行放大或缩小处理，然后将该主区域融合到第一图片的同位区中，并且对融合的边界进行过渡性的平滑处理。相当于，电子设备使用放大或缩小的主区域或原主区域替换第一图片中的同位区，并且还要对该主区域和第一图片的交界处做过渡性的平滑处理，让最终的合成图像看起来自然。
示例性地，参考图6A，如果同位区和主区域的大小相同，电子设备100使用主区域替换(c)中的同 位区，并且对边界进行过渡性的平滑处理，生成如(d)所示的合成图像。
示例性地，参考图6B，电子设备100对主区域做放大处理，使用放大后的主区域替换(c)中的同位区，并且对边界进行过渡性的平滑处理，生成如(d)所示的合成图像。
通过上述步骤1-4可以让对象B平滑、自然地融入到包含对象A的第一图片中，生成同时包含对象A和对象B的合成对象，从而得到自然的、高质量的合影效果。
电子设备100在拍照模式下执行S104-S105时，可以将接收到拍照指令的时刻采集的图像，和，第一图片进行拟合，生成合成图像。在另一些实施方式中，电子设备100可以实时保存接收到拍照指令的时刻以及之前摄像头193所采集到的N帧图像，将该N帧图像各自和第一图片进行拟合，生成N帧合成图像。
电子设备100在录像模式下执行S104-S105时，可以将接收到录像指令的时刻开始，到接收到停止录像指令的时刻之间，摄像头193所采集到的多帧图像，分别和第一图像进行拟合，生成多帧合成图像。
上述流程，在预览界面中通过叠加显示的方式给用户展示合影时对象A和对象B的相对位置，在接收到第一操作后才开始执行第一图片和摄像头采集图像的拟合操作，这种方式可以节约电子设备的功耗及处理能力。
S106，电子设备100保存合成图像。
如果电子设备100在拍照模式下执行S104-S105，并且仅将接收到拍照指令的时刻采集的图像，和，第一图片进行拟合，生成合成图像，则电子设备100可以将生成的该一帧合成图像以图片格式保存至存储器中。如果电子设备将接收到拍照指令的时刻以及之前摄像头193所采集到的N帧图像，各自和第一图片进行拟合，生成了N帧合成图像，则电子设备100可以从该N帧合成图像中选择质量最好的一帧以图片格式保存至存储器中。这里，电子设备100可以通过人工智能学习的方式，从N帧合成图像中选择质量最好的一帧图像。其中，合成图像的质量可以通过但不限于以下任意一项或多项参数来评估：峰值信噪比(peak-signal to noise ratio,PSNR)和均方误差(mean square error,MSE)，平均绝对误差(mean sbsolute error,MAE)和信噪比(signal to noise ratio，SNR)。
在另一些实施方式中，如果电子设备100在拍照模式下执行S104-S105，电子设备将接收到拍照指令的时刻以及之前摄像头193所采集到的N帧图像，各自和第一图片进行拟合，生成了N帧合成图像，则可以由用户在该N帧合成图像中选择一张满意的合成图像存储至存储器中，删除其余合成图像。
在上述实施方式中，电子设备100在拍照模式下执行S104-S106而保持的合成图像(即合成图片)也可以被称为第二图片。
如果电子设备100在录像模式下执行S104-S105，则可以将录像器件摄像头193所采集到的多帧图像，分别和第一图像进行拟合后生成的多帧合成图像，以视频格式存储至存储器中，即将该多帧合成图像保存为视频文件。
电子设备100保存合成图像之后，可以查看该合成图像，还可以对该合成图像进行编辑、移动、分享等操作。用于查看该合成图像的操作，也可以被称为第三操作。
示例性地，参考图4A-图4B，电子设备100在拍照模式下生成了一张合成图像，图4B所示的图库应用提供的图片区域207中展示有该合成图像的缩略图207b。电子设备100接收到作用于缩略图207b上的用户操作后，可以显示如图4B所示的该缩略图207b的原图，即电子设备100生成的合成图像。
在一些实施方式中，在生成合成图像之后，电子设备100还可以将该合成对象作为第一图片，再执行一次或多次上述S101-S106所述的拍摄方法，这样就能够实现多次合影的目的。例如，同一个用户可以先使用自己的照片作为第一图片，然后自己和该第一图片中的自己多次合影，实现同一人物在同一地点的多次合影。
在一些实施方式中，若第一图片是一张亲人在多年前拍摄的老照片，那么通过上述S101-S106所述的拍摄方法，用户可以和自己的亲人在同一地点实现不同时间的跨时空的合影。
在上述S105具体包括的步骤1-步骤4中，电子设备100具体将摄像头193采集到的图像中对象B所在的区域拟合至第一图片中，从而得到合成图像。不限于此，在本申请其他一些实施方式中，电子设备100还可以将第一图像中的对象A所在区域拟合至摄像头193所采集到的图像中，从而得到合成图像。具体的，该实施方式可包括以下步骤：
步骤1.识别第一图片中的对象，该对象可以称为对象A；
步骤2.在第一图片中，以对象A为基准确定包含对象A的主区域；
步骤3.在摄像头采集到的图像中执行针对主区域的相关性搜索，找到相关性大于第二值的同位区；
步骤4.将主区域拟合到摄像头采集到的图像的同位区中，保持摄像头采集到的图像的其余区域不变，生成合成图像。
上述各步骤的具体实现可参考前文S105中的各步骤的实现，这里不再赘述。
通过上述实施方式，将第一图像中的对象A所在区域拟合至摄像头193所采集到的图像中以得到合成图像，如果第一图片为老照片，则可以实现将老照片中的对象提取出来融入到新拍摄的图像中，实现老照片的翻新。
在本申请上述实施方式中，第一图片中的对象A也可以被称为第一对象，摄像头拍摄的图像中的对象B也可以被称为第二对象。
在上述图5所示的拍摄方法中，电子设备100现在预览界面中叠加显示第一图片和摄像头采集到的图像，然后在接收到第一操作之后再生成合成图像。不限于此，在本申请其他一些实施方式中，电子设备100还可以直接在启动相机应用后，直接执行拟合并生成合成图像的操作，并且直接在预览界面中显示合成图像。具体的，电子设备100执行上述方法的S101-S103之后，启动相机应用，通过摄像头193实时采集图像，电子设备100将摄像头193实时采集的各帧图像分别和第一图片进行拟合，得到多帧合成图像，并将这多帧合成图像按照时间先后顺序显示在预览界面(即前文的第二用户界面)的预览框中。之后如果接收到拍照指令，则将预览框中显示的合成图像保存为图片，如果接收到录像指令，则将录像期间预览框中所显示的多帧合成图像保存为视频。这种实时合成，并且实时显示合成图像的方式，需要电子设备100具备较强的处理能力。这种方式可以在预览框中为用户实时地展示合影效果，可以让用户更加直观地感受到合影效果。
应理解，上述方法实施例中的各步骤可以通过处理器中的硬件的集成逻辑电路或者软件形式的指令完成。结合本申请实施例所公开的方法步骤可以直接体现为硬件处理器执行完成，或者用处理器中的硬件及软件模块组合执行完成。
本申请还提供一种电子设备，该电子设备可以包括：存储器和处理器。其中，存储器可用于存储计算机程序；处理器可用于调用所述存储器中的计算机程序，以使得该电子设备执行上述任意一个实施例中电子设备100方法。
本申请还提供了一种芯片系统，所述芯片系统包括至少一个处理器，用于实现上述任一个实施例中电子设备100侧所涉及的功能。
在一种可能的设计中，所述芯片系统还包括存储器，所述存储器用于保存程序指令和数据，存储器位于处理器之内或处理器之外。
该芯片系统可以由芯片构成，也可以包含芯片和其他分立器件。
可选地，该芯片系统中的处理器可以为一个或多个。该处理器可以通过硬件实现也可以通过软件实现。当通过硬件实现时，该处理器可以是逻辑电路、集成电路等。当通过软件实现时，该处理器可以是一个通用处理器，通过读取存储器中存储的软件代码来实现。
可选地，该芯片系统中的存储器也可以为一个或多个。该存储器可以与处理器集成在一起，也可以和处理器分离设置，本申请实施例并不限定。示例性地，存储器可以是非瞬时性处理器，例如只读存储器ROM，其可以与处理器集成在同一块芯片上，也可以分别设置在不同的芯片上，本申请实施例对存储器的类型，以及存储器与处理器的设置方式不作具体限定。
示例性地，该芯片系统可以是现场可编程门阵列(field programmable gate array，FPGA)，可以是专用集成芯片(application specific integrated circuit，ASIC)，还可以是系统芯片(system on chip，SoC)，还可以是中央处理器(central processor unit，CPU)，还可以是网络处理器(network processor，NP)，还可以是数字信号处理电路(digital signal processor，DSP)，还可以是微控制器(micro controller unit，MCU)，还可以是可编程控制器(programmable logic device，PLD)或其他集成芯片。
本申请还提供一种计算机程序产品，所述计算机程序产品包括：计算机程序(也可以称为代码，或指令)，当所述计算机程序被运行时，使得计算机执行上述任一个实施例中电子设备100侧所执行的方法。
本申请还提供一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序(也可以称为代码，或指令)。当所述计算机程序被运行时，使得计算机执行上述任一个实施例中电子设备100侧所执行的方法。
本申请的各实施方式可以任意进行组合，以实现不同的技术效果。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线)或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如固态硬盘(solid state disk，SSD))等。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，该流程可以由计算机程序来指令相关的硬件完成，该程序可存储于计算机可读取存储介质中，该程序在执行时，可包括如上述各方法实施例的流程。而前述的存储介质包括：ROM或随机存储记忆体RAM、磁碟或者光盘等各种可存储程序代码的介质。
总之，以上所述仅为本申请技术方案的实施例而已，并非用于限定本申请的保护范围。凡根据本申请的揭露，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。

Claims (27)

  1. 一种拍摄方法，其特征在于，所述方法包括：
    电子设备启动图库应用，并显示第一用户界面，所述第一用户界面中显示有第一图片，所述第一图片包含第一对象；
    所述电子设备接收到针对所述第一图片的第一操作；
    所述电子设备启动相机应用，并显示第二用户界面，所述第二用户界面中显示有预览框，所述预览框中显示有所述电子设备的摄像头采集的图像，以及，第一图片的指示信息；
    所述电子设备接收到用于拍照的第二操作；
    所述电子设备存储第二图片，所述第二图片由所述摄像头采集的图像和所述第一图片拟合得到，所述摄像头采集的图像包含第二对象，并且所述第二图片包含所述第一对象和所述第二对象。
  2. 根据权利要求1所述的方法，其特征在于，所述电子设备接收到所述第二操作后，存储所述第二图片之前，所述方法还包括：
    所述电子设备识别所述摄像头采集的图像中的所述第二对象；
    在所述摄像头采集的图像中确定包含所述第二对象的主区域，所述主区域为以所述第二对象为基准确定的区域；
    在所述第一图片中确定同位区，所述第一图片中的所述同位区以外的部分，和，所述摄像头采集的图像中所述主区域以外的部分的相关性大于第二值；
    将所述主区域拟合到所述第一图片中的所述同位区，得到所述第二图片。
  3. 根据权利要求1所述的方法，其特征在于，所述电子设备接收到所述第二操作后，存储所述第二图片之前，所述方法还包括：
    所述电子设备识别所述第一图片中的所述第一对象；
    在所述第一图片中确定包含所述第一对象的主区域，所述主区域为以所述第一对象为基准确定的区域；
    在所述摄像头采集的图像中确定同位区，所述摄像头采集的图像中的所述同位区以外的部分，和，所述第一图片中所述主区域以外的部分的相关性大于第二值；
    将所述主区域拟合到所述摄像头采集的图像中的所述同位区，得到所述第二图片。
  4. 根据权利要求2或3所述的方法，其特征在于，所述同位区在所述第一图片中的位置，和，所述主区域在所述摄像头采集的图像中的位置，相同。
  5. 根据权利要求1所述的方法，其特征在于，
    所述第二图片为，所述摄像头采集的多帧图像分别和所述第一图片拟合得到的合成图像中，质量最高的一帧图像；所述摄像头采集的多帧图像包括：在所述电子设备接收到所述第二操作的时刻以及之前时刻由所述摄像头采集的图像，所述多帧图像的数量由所述电子设备预先设置。
  6. 根据权利要求1-5任一项所述的方法，其特征在于，所述第一用户界面中显示有第一控件，所述电子设备接收到针对所述第一图片的第一操作之前，所述方法还包括：
    所述电子设备接收到作用于所述第一控件的用户操作；
    所述电子设备在所述第一用户界面中显示第二控件；所述第一操作包括作用于所述第二控件上的用户操作。
  7. 根据权利要求1-6任一项所述的方法，其特征在于，所述电子设备存储第二图片之后，所述方法还包括：
    所述电子设备接收到用于查看所述第二图片的第三操作；
    所述电子设备显示所述第二图片。
  8. 一种拍摄方法，其特征在于，所述方法包括：
    电子设备启动图库应用，并显示第一用户界面，所述第一用户界面中显示有第一图片，所述第一图片包含第一对象；
    所述电子设备接收到针对所述第一图片的第一操作；
    所述电子设备启动相机应用，并显示第二用户界面，所述第二用户界面中显示有预览框，所述预览框中显示有合成图像，所述合成图像由所述电子设备的摄像头采集的图像和所述第一图片拟合得到，所述摄像头采集的图像包含第二对象；
    所述电子设备接收到用于拍照的第二操作；
    所述电子设备存储第二图片，所述第二图片为接收到所述第二操作时所述预览框中显示的合成图像，并且所述第二图片包含所述第一对象和所述第二对象。
  9. 根据权利要求8所述的方法，其特征在于，所述电子设备在所述预览框中显示合成图像之前，所述方法还包括：
    所述电子设备识别所述摄像头采集的图像中的所述第二对象；
    在所述摄像头采集的图像中确定包含所述第二对象的主区域，所述主区域为以所述第二对象为基准确定的区域；
    在所述第一图片中确定同位区，所述第一图片中的所述同位区以外的部分，和，所述摄像头采集的图像中所述主区域以外的部分的相关性大于第二值；
    将所述主区域拟合到所述第一图片中的所述同位区，得到所述合成图像。
  10. 根据权利要求8所述的方法，其特征在于，所述电子设备在所述预览框中显示合成图像之前，所述方法还包括：
    所述电子设备识别所述第一图片中的所述第一对象；
    在所述第一图片中确定包含所述第一对象的主区域，所述主区域为以所述第一对象为基准确定的区域；
    在所述摄像头采集的图像中确定同位区，所述摄像头采集的图像中的所述同位区以外的部分，和，所述第一图片中所述主区域以外的部分的相关性大于第二值；
    将所述主区域拟合到所述摄像头采集的图像中的所述同位区，得到所述合成图像。
  11. 根据权利要求9或10所述的方法，其特征在于，所述同位区在所述第一图片中的位置，和，所述主区域在所述摄像头采集的图像中的位置，相同。
  12. 根据权利要求8-11任一项所述的方法，其特征在于，所述第一用户界面中显示有第一控件，所述电子设备接收到针对所述第一图片的第一操作之前，所述方法还包括：
    所述电子设备接收到作用于所述第一控件的用户操作；
    所述电子设备在所述第一用户界面中显示第二控件；所述第一操作包括作用于所述第二控件上的用户操作。
  13. 根据权利要求8-12任一项所述的方法，其特征在于，所述电子设备存储第二图片之后，所述方法还包括：
    所述电子设备接收到用于查看所述第二图片的第三操作；
    所述电子设备显示所述第二图片。
  14. 一种拍摄方法，其特征在于，所述方法包括：
    电子设备显示第二用户界面，所述第二用户界面中显示有预览框和第三控件，所述预览框中显示有所述电子设备的摄像头采集的图像；
    所述电子设备检测到作用于所述第三控件上的第四操作；
    所述电子设备显示一张或多张图片；
    所述电子设备检测到作用于所述一张或多张图片中的第一图片上的第五操作，所述第一图片包含第一对象；
    所述电子设备在所述第二用户界面的所述预览框中，显示所述电子设备的摄像头采集的图像，以及，所述第一图片的指示信息；
    所述电子设备接收到用于拍照的第二操作；
    所述电子设备存储第二图片，所述第二图片由所述摄像头采集的图像和所述第一图片拟合得到，所述摄像头采集的图像包含第二对象，并且所述第二图片包含所述第一对象和所述第二对象。
  15. 根据权利要求14所述的方法，其特征在于，所述电子设备接收到所述第二操作后，存储所述第二图片之前，所述方法还包括：
    所述电子设备识别所述摄像头采集的图像中的所述第二对象；
    在所述摄像头采集的图像中确定包含所述第二对象的主区域，所述主区域为以所述第二对象为基准确定的区域；
    在所述第一图片中确定同位区，所述第一图片中的所述同位区以外的部分，和，所述摄像头采集的图像中所述主区域以外的部分的相关性大于第二值；
    将所述主区域拟合到所述第一图片中的所述同位区，得到所述第二图片。
  16. 根据权利要求14所述的方法，其特征在于，所述电子设备接收到所述第二操作后，存储所述第二图片之前，所述方法还包括：
    所述电子设备识别所述第一图片中的所述第一对象；
    在所述第一图片中确定包含所述第一对象的主区域，所述主区域为以所述第一对象为基准确定的区域；
    在所述摄像头采集的图像中确定同位区，所述摄像头采集的图像中的所述同位区以外的部分，和，所述第一图片中所述主区域以外的部分的相关性大于第二值；
    将所述主区域拟合到所述摄像头采集的图像中的所述同位区，得到所述第二图片。
  17. 根据权利要求15或16所述的方法，其特征在于，所述同位区在所述第一图片中的位置，和，所述主区域在所述摄像头采集的图像中的位置，相同。
  18. 根据权利要求14所述的方法，其特征在于，
    所述第二图片为，所述摄像头采集的多帧图像分别和所述第一图片拟合得到的合成图像中，质量最高的一帧图像；所述摄像头采集的多帧图像包括：在所述电子设备接收到所述第二操作的时刻以及之前时刻由所述摄像头采集的图像，所述多帧图像的数量由所述电子设备预先设置。
  19. 根据权利要求15-18任一项所述的方法，其特征在于，所述电子设备存储第二图片之后，所述方法还包括：
    所述电子设备接收到用于查看所述第二图片的第三操作；
    所述电子设备显示所述第二图片。
  20. 一种拍摄方法，其特征在于，所述方法包括：
    电子设备显示第二用户界面，所述第二用户界面中显示有预览框和第三控件，所述预览框中显示有所述电子设备的摄像头采集的图像；
    所述电子设备检测到作用于所述第三控件上的第四操作；
    所述电子设备显示一张或多张图片；
    所述电子设备检测到作用于所述一张或多张图片中的第一图片上的第五操作，所述第一图片包含第一对象；
    所述电子设备在所述第二用户界面的所述预览框中显示合成图像，所述合成图像由所述电子设备的摄像头采集的图像和所述第一图片拟合得到，所述摄像头采集的图像包含第二对象；
    所述电子设备接收到用于拍照的第二操作；
    所述电子设备存储第二图片，所述第二图片为接收到所述第二操作时所述预览框中显示的合成图像，并且所述第二图片包含所述第一对象和所述第二对象。
  21. 根据权利要求20所述的方法，其特征在于，所述电子设备在所述预览框中显示合成图像之前，所述方法还包括：
    所述电子设备识别所述摄像头采集的图像中的所述第二对象；
    在所述摄像头采集的图像中确定包含所述第二对象的主区域，所述主区域为以所述第二对象为基准确定的区域；
    在所述第一图片中确定同位区，所述第一图片中的所述同位区以外的部分，和，所述摄像头采集的图像中所述主区域以外的部分的相关性大于第二值；
    将所述主区域拟合到所述第一图片中的所述同位区，得到所述合成图像。
  22. 根据权利要求20所述的方法，其特征在于，所述电子设备在所述预览框中显示合成图像之前，所述方法还包括：
    所述电子设备识别所述第一图片中的所述第一对象；
    在所述第一图片中确定包含所述第一对象的主区域，所述主区域为以所述第一对象为基准确定的区域；
    在所述摄像头采集的图像中确定同位区，所述摄像头采集的图像中的所述同位区以外的部分，和，所述第一图片中所述主区域以外的部分的相关性大于第二值；
    将所述主区域拟合到所述摄像头采集的图像中的所述同位区，得到所述合成图像。
  23. 根据权利要求21或22所述的方法，其特征在于，所述同位区在所述第一图片中的位置，和，所述主区域在所述摄像头采集的图像中的位置，相同。
  24. 根据权利要求20-23任一项所述的方法，其特征在于，所述电子设备存储第二图片之后，所述方法还包括：
    所述电子设备接收到用于查看所述第二图片的第三操作；
    所述电子设备显示所述第二图片。
  25. 一种电子设备，其特征在于，所述电子设备包括：存储器、一个或多个处理器；所述存储器与所述一个或多个处理器耦合，所述存储器用于存储计算机程序代码，所述计算机程序代码包括计算机指令，所述一个或多个处理器调用所述计算机指令以使得所述电子设备执行如权利要求1-7、或权利要求8-13、或权利要求14-19、或权利要求20-24中任一项所述的方法。
  26. 一种计算机可读存储介质，包括指令，其特征在于，当所述指令在电子设备上运行，使得所述电子设备执行如权利要求1-7、或权利要求8-13、或权利要求14-19、或权利要求20-24中任一项所述的方法。
  27. 一种计算机程序产品，其特征在于，所述计算机程序产品包含计算机指令，当所述计算机指令在电子设备上运行，使得所述电子设备执行如权利要求1-7、或权利要求8-13、或权利要求14-19、或权利要求20-24中任一项所述的方法。
PCT/CN2023/112243 2022-08-26 2023-08-10 拍摄方法及相关装置 Ceased WO2024041394A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP23856484.3A EP4589980A4 (en) 2022-08-26 2023-08-10 PHOTOGRAPHIC METHOD AND ASSOCIATED DEVICE

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211037891.9 2022-08-26
CN202211037891.9A CN115529413B (zh) 2022-08-26 2022-08-26 拍摄方法及相关装置

Publications (1)

Publication Number Publication Date
WO2024041394A1 true WO2024041394A1 (zh) 2024-02-29

Family

ID=84696886

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/112243 Ceased WO2024041394A1 (zh) 2022-08-26 2023-08-10 拍摄方法及相关装置

Country Status (3)

Country Link
EP (1) EP4589980A4 (zh)
CN (3) CN120786178A (zh)
WO (1) WO2024041394A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119396323A (zh) * 2024-10-31 2025-02-07 维沃移动通信有限公司 图像生成方法、装置及设备
WO2026030120A1 (en) * 2024-07-29 2026-02-05 Google Llc Generating a group photo that includes a photographer

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN120786178A (zh) * 2022-08-26 2025-10-14 华为技术有限公司 拍摄方法及相关装置
CN118450268B (zh) * 2023-10-12 2025-02-21 荣耀终端有限公司 拍摄方法、电子设备及可读存储介质
CN117221712B (zh) * 2023-11-07 2024-11-08 荣耀终端有限公司 一种合拍方法、电子设备及存储介质

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102055834A (zh) * 2009-10-30 2011-05-11 Tcl集团股份有限公司 一种移动终端的双摄像头拍照方法
CN105744170A (zh) * 2016-03-30 2016-07-06 努比亚技术有限公司 一种照片拍摄装置和方法
CN111586318A (zh) * 2019-02-19 2020-08-25 三星电子株式会社 用于提供基于虚拟角色的拍摄模式的电子装置及操作方法
CN112532887A (zh) * 2020-12-18 2021-03-19 惠州Tcl移动通信有限公司 一种拍摄方法、装置、终端及存储介质
CN114143461A (zh) * 2021-11-30 2022-03-04 维沃移动通信有限公司 拍摄方法、装置及电子设备
CN114205512A (zh) * 2020-09-17 2022-03-18 华为技术有限公司 拍摄方法和装置
EP4020967A1 (en) * 2020-01-14 2022-06-29 Honor Device Co., Ltd. Photographic method in long focal length scenario, and mobile terminal
CN115529413A (zh) * 2022-08-26 2022-12-27 华为技术有限公司 拍摄方法及相关装置

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5262856A (en) * 1992-06-04 1993-11-16 Massachusetts Institute Of Technology Video image compositing techniques
JP2005079776A (ja) * 2003-08-29 2005-03-24 Fuji Photo Film Co Ltd 画像生成装置、画像生成方法、及びプログラム
US7782384B2 (en) * 2004-11-05 2010-08-24 Kelly Douglas J Digital camera having system for digital image composition and related method
US8332429B2 (en) * 2010-06-22 2012-12-11 Xerox Corporation Photography assistant and method for assisting a user in photographing landmarks and scenes
WO2015100723A1 (zh) * 2014-01-03 2015-07-09 华为终端有限公司 实现自助合影的方法和照相设备
KR20150091798A (ko) * 2014-02-04 2015-08-12 엘지전자 주식회사 단말기 및 상기 단말기에서의 이미지 합성 방법
US9774779B2 (en) * 2014-05-21 2017-09-26 Google Technology Holdings LLC Enhanced image capture
CN105657239B (zh) * 2015-04-27 2018-05-15 宇龙计算机通信科技(深圳)有限公司 一种图像处理方法和装置
CN106792147A (zh) * 2016-12-08 2017-05-31 天脉聚源(北京)传媒科技有限公司 一种图像替换方法及装置
CN106657791A (zh) * 2017-01-03 2017-05-10 广东欧珀移动通信有限公司 一种合成图像的生成方法及装置
US10574881B2 (en) * 2018-02-15 2020-02-25 Adobe Inc. Smart guide to capture digital images that align with a target image model
KR20200037929A (ko) * 2018-10-02 2020-04-10 문현수 사진 합성 촬상 방법과 그 어플리케이션이 설치된 개인휴대통신기 및 그 온라인 비즈니스 방법과 서버
CN110365907B (zh) * 2019-07-26 2021-09-21 维沃移动通信有限公司 一种拍照方法、装置及电子设备
CN113012082A (zh) * 2021-02-09 2021-06-22 北京字跳网络技术有限公司 图像显示方法、装置、设备及介质

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102055834A (zh) * 2009-10-30 2011-05-11 Tcl集团股份有限公司 一种移动终端的双摄像头拍照方法
CN105744170A (zh) * 2016-03-30 2016-07-06 努比亚技术有限公司 一种照片拍摄装置和方法
CN111586318A (zh) * 2019-02-19 2020-08-25 三星电子株式会社 用于提供基于虚拟角色的拍摄模式的电子装置及操作方法
EP4020967A1 (en) * 2020-01-14 2022-06-29 Honor Device Co., Ltd. Photographic method in long focal length scenario, and mobile terminal
CN114205512A (zh) * 2020-09-17 2022-03-18 华为技术有限公司 拍摄方法和装置
CN112532887A (zh) * 2020-12-18 2021-03-19 惠州Tcl移动通信有限公司 一种拍摄方法、装置、终端及存储介质
CN114143461A (zh) * 2021-11-30 2022-03-04 维沃移动通信有限公司 拍摄方法、装置及电子设备
CN115529413A (zh) * 2022-08-26 2022-12-27 华为技术有限公司 拍摄方法及相关装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4589980A4

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2026030120A1 (en) * 2024-07-29 2026-02-05 Google Llc Generating a group photo that includes a photographer
CN119396323A (zh) * 2024-10-31 2025-02-07 维沃移动通信有限公司 图像生成方法、装置及设备

Also Published As

Publication number Publication date
CN115529413A (zh) 2022-12-27
CN120786178A (zh) 2025-10-14
EP4589980A1 (en) 2025-07-23
EP4589980A4 (en) 2026-02-25
CN115529413B (zh) 2025-06-27
CN120786177A (zh) 2025-10-14
CN120786177B (zh) 2026-04-24

Similar Documents

Publication Publication Date Title
EP4171014B1 (en) Photographing method, graphical interface and related apparatus
US20250004799A1 (en) Theme wallpaper generation method and electronic device
CN110401766B (zh) 一种拍摄方法及终端
CN115529413B (zh) 拍摄方法及相关装置
WO2021244455A1 (zh) 一种图像内容的去除方法及相关装置
CN113497890B (zh) 一种拍摄方法及设备
EP4060603B1 (en) Image processing method and related apparatus
CN110377204B (zh) 一种生成用户头像的方法及电子设备
WO2021129198A1 (zh) 一种长焦场景下的拍摄方法及终端
WO2020113534A1 (zh) 一种拍摄长曝光图像的方法和电子设备
CN116055867B (zh) 一种拍摄方法和电子设备
WO2022228010A1 (zh) 一种生成封面的方法及电子设备
WO2021204103A1 (zh) 照片预览方法、电子设备和存储介质
WO2024179100A1 (zh) 一种拍摄方法
WO2023142731A1 (zh) 一种分享多媒体文件的方法、发送端设备和接收端设备
WO2022127609A1 (zh) 图像处理方法及电子设备
WO2023160230A9 (zh) 一种拍摄方法及相关设备
WO2023160224A9 (zh) 一种拍摄方法及相关设备
CN115914823B (zh) 拍摄方法及电子设备
WO2023231696A1 (zh) 一种拍摄方法及相关设备
CN118552452A (zh) 去除摩尔纹的方法及相关装置
CN115442509A (zh) 拍摄方法、用户界面及电子设备
CN113452895A (zh) 一种拍摄方法及设备
CN121284393A (zh) 一种拍照处理的方法及相关装置
CN121147234A (zh) 一种图像处理方法以及相关装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23856484

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023856484

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2023856484

Country of ref document: EP

Effective date: 20250326

WWP Wipo information: published in national office

Ref document number: 2023856484

Country of ref document: EP