CN112822425B - Method and equipment for generating high dynamic range image - Google Patents

Abstract

The purpose of this application is to provide a method and device for generating a high dynamic range image, the method comprising: acquiring image information to be processed; generating a first object vector of the image information according to at least one first object category; The target vector is input into the target vector regression model to output the second target vector of the image information, wherein the target vector regression model is a vector regression model corresponding to the target style type to which the image information belongs; according to each first object category in the image information The corresponding target exposure of the first object area generates a high dynamic range image of the image information. This application synthesizes the high dynamic range image of the image information based on the image characteristics of the image information (for example, style type, the first object appearing in the image information, the first object category), so that the synthesized high dynamic range image conforms to the corresponding The pixel characteristics of the target style type, the effect is more realistic.

Description

A method and device for generating high dynamic range images

technical field

The present application relates to the field of image processing, and in particular to a technique for generating high dynamic range images.

Background technique

High dynamic range imaging (High Dynamic Range Imaging) is a technology used to achieve a larger exposure dynamic range (that is, a larger difference between light and dark) than ordinary digital image technology. It can prevent the scene in the bright place from being overexposed and the scene in the dark place from being underexposed. For example, if you shoot people in a backlit environment, you can capture both the characters and the environment clearly. Make the whole photo not too dark or too bright.

Contents of the invention

An object of the present application is to provide a method and apparatus for generating high dynamic range images.

According to one aspect of the present application, a method for generating a high dynamic range image is provided, the method comprising:

acquiring image information to be processed, wherein the image information includes one or more first objects, and the one or more first objects belong to at least one first object category;

generating a first object vector of the image information based on the at least one first object category;

inputting the first target vector into a target vector regression model to output a second target vector of the image information, wherein the target vector regression model is a vector regression model corresponding to the target style type to which the image information belongs, The second target vector includes the target exposure of the first object region corresponding to each first object category in the at least one first object category in the image information;

A high dynamic range image of the image information is generated according to the target exposure of the first object region corresponding to each first object category in the image information.

According to one aspect of the present application, there is provided a device for generating a high dynamic range image, the device comprising:

A module, configured to acquire image information to be processed, wherein the image information includes one or more first objects, and the one or more first objects belong to at least one first object category;

A two-module, configured to generate a first target vector of the image information according to the at least one first object category;

A three-module, configured to input the first target vector into a target vector regression model to output a second target vector of the image information, wherein the target vector regression model is defined by the target style type to which the image information belongs A corresponding vector regression model, wherein the second target vector includes the target exposure of the first object region corresponding to each first object category in the image information in the at least one first object category;

A module, configured to generate a high dynamic range image of the image information according to the target exposure of the first object region corresponding to each first object category in the image information.

According to one aspect of the present application, there is provided a device for generating a high dynamic range image, wherein the device includes:

processor; and

A memory arranged to store computer-executable instructions which, when executed, cause the processor to perform the operations of any of the methods described above.

According to one aspect of the present application, there is provided a computer-readable medium storing instructions which, when executed, cause a system to perform operations of any one of the methods described above.

According to one aspect of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps of any one of the above-mentioned methods are realized.

Compared with the prior art, the present application generates a first target vector of the image information according to at least one first object category to which one or more first objects appearing in the image information belong, by adding the first target The vector is input into a target vector regression model to obtain a second target vector of the image information, wherein the target vector regression model is a vector regression model corresponding to the target style type to which the image information belongs. Obtain the target exposure of the first object region corresponding to each first object category in the image information based on the output-based second target vector, and the target exposure is the first The object area is obtained on the premise of the target style type described in the image information. Thus, a high dynamic range image of the image information is synthesized based on the target exposure of each first object area. The present application obtains the target exposure of each first object area based on the image characteristics of the image information (for example, style type, first object appearing in the image information, first object category), and based on each first object The high dynamic range image of the image information is synthesized with the target exposure corresponding to the area itself, so that the synthesized high dynamic range image conforms to the pixel characteristics of the corresponding target style type, and the effect is more realistic.

Description of drawings

Other characteristics, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

Fig. 1 shows a flow chart of a method for generating a high dynamic range image according to an embodiment of the present application;

FIG. 2 shows a structural diagram of a device for generating a high dynamic range image according to an embodiment of the present application;

FIG. 3 illustrates an exemplary system that may be used to implement various embodiments described in this application.

The same or similar reference numerals in the drawings represent the same or similar components.

detailed description

The application will be described in further detail below in conjunction with the accompanying drawings.

In a typical configuration of the present application, the terminal, the device serving the network, and the trusted party all include one or more processors (for example, a central processing unit (Central Processing Unit, CPU)), an input/output interface, a network interface and Memory.

Memory may include non-permanent memory in computer-readable media, random access memory (Random Access Memory, RAM) and/or non-volatile memory, such as read-only memory (Read Only Memory, ROM) or flash memory (Flash Memory). Memory is an example of computer readable media.

Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can be implemented by any method or technology for storage of information. Information may be computer readable instructions, data structures, modules of a program, or other data. The example of the storage medium of computer includes, but not limited to Phase-Change Memory (Phase-Change Memory, PCM), Programmable Random Access Memory (Programmable Random Access Memory, PRAM), Static Random-Access Memory (Static Random-Access Memory, SRAM), Dynamic Random Access Memory (Dynamic Random Access Memory, DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (Electrically-Erasable Programmable Read- Only Memory, EEPROM), flash memory or other memory technology, CD-ROM (Compact Disc Read-Only Memory, CD-ROM), Digital Versatile Disc (Digital Versatile Disc, DVD) or other optical storage, Magnetic tape cartridge, tape disk storage or other magnetic storage device or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

The devices referred to in this application include but are not limited to terminals, network devices, or devices formed by integrating terminals and network devices through a network. The terminal includes but is not limited to any mobile electronic product that can perform human-computer interaction (such as human-computer interaction through a touch panel) with the user, such as a smart phone, a tablet computer, etc., and the mobile electronic product can use any operating system , such as Android operating system, iOS operating system, etc. Wherein, the network device includes an electronic device that can automatically perform numerical calculation and information processing according to preset or stored instructions, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (Application Specific Integrated Circuit, ASIC) ), Programmable Logic Device (PLD), Field Programmable Gate Array (Field ProgrammableGate Array, FPGA), Digital Signal Processor (Digital Signal Processor, DSP), embedded devices, etc. The network equipment includes but is not limited to a computer, a network host, a single network server, a plurality of network server sets or a cloud composed of multiple servers; here, the cloud is composed of a large number of computers or network servers based on Cloud Computing, Among them, cloud computing is a kind of distributed computing, a virtual supercomputer composed of a group of loosely coupled computer sets. The network includes, but is not limited to, the Internet, a wide area network, a metropolitan area network, a local area network, a VPN network, a wireless ad hoc network (Ad Hoc network) and the like. Preferably, the device may also be a program running on the terminal, network device, or a device formed by integrating a terminal and a network device, a network device, a touch terminal, or a network device and a touch terminal through a network.

Of course, those skilled in the art should understand that the above-mentioned equipment is only an example, and other existing or future equipment that may be applicable to this application should also be included in the scope of protection of this application, and it is included in this application by reference. this.

In the description of the present application, "plurality" means two or more, unless otherwise specifically defined.

Here, the subject of execution of the method for generating a high dynamic range image described in this application includes, but is not limited to, network equipment and user equipment including a camera device. It is preferably the user equipment, and the method described in this application is explained below from the perspective of the user equipment. For the convenience of description, the following are collectively referred to as "device" or "a device for generating high dynamic range images".

In some embodiments, the user equipment includes, but is not limited to, computing equipment such as mobile phones, computers, and tablet computers. For example, when the execution subject is the user equipment, the user equipment obtains the image information to be processed through a camera device, and generates a high dynamic range image of the image information based on the method described in this application. For another example, when the executor is the network device, the user equipment acquires image information, and sends information such as the image information to the network device, and the network device generates the image information based on the method described in this application A high dynamic range image of the image information.

Fig. 1 shows a flowchart of a method for generating a high dynamic range image according to one aspect of the present application, and the method includes step S11, step S12, step S13 and step S14. In step S11, the device acquires image information to be processed, wherein the image information includes one or more first objects, and the one or more first objects belong to at least one first object category; in step S12, The device generates a first target vector of the image information according to the at least one first object category; in step S13, the device inputs the first target vector into a target vector regression model to output a second target vector of the image information vector, wherein the target vector regression model is a vector regression model corresponding to the target style type to which the image information belongs, and the second target vector includes each first object category in the at least one first object category The target exposure of the first object area corresponding to the image information; in step S14, the device generates the target exposure of the first object area corresponding to each first object category in the image information. A high dynamic range image of the image information described above.

Specifically, in step S11, the device acquires image information to be processed, wherein the image information includes one or more first objects, and the one or more first objects belong to at least one first object category. In some embodiments, the image information includes, but is not limited to, image information acquired by a viewfinder of the user equipment. For example, when a user device (such as a mobile phone) takes a view through a viewfinder, the image information can be collected, so that the device can analyze and process the image information. In some embodiments, the first object includes, but is not limited to, items appearing in the image information (eg, cups, books, computers, sky, etc.). In some embodiments, first objects appearing in the image information are classified to determine at least one first object category corresponding to the one or more first objects. For example, there are first objects in the image information A: kitten, book, computer, and sky, wherein, the kitten belongs to animals, books and computers belong to office supplies, and the sky belongs to other categories, then one or more objects in the image information A A first object belongs to the three first object categories of animals, office supplies, and others.

In step S12, the device generates a first target vector of the image information according to the at least one first object category. In some embodiments, the first target vector is generated based on the at least one first object category. In some embodiments, the first target vector includes a plurality of first components, for example, the first target vector is [0,1,0,1,0,0,1], wherein the first Data such as "0" and "1" in the target vector are used as the first component of the first target vector. For example, the image information A includes the first object: a kitten, a desk, a computer, and the sky. After classifying the kitten, desk, computer, and sky, it can be obtained that the first object in the image information A belongs to at least A first object class (eg, animals, office supplies, other classes). The device generates corresponding first object vectors (for example, [0,1,0,1,0,0,1]) based on the three first object categories of "animals, office supplies, and others", so that all The above image information A is vectorized.

In step S13, the device inputs the first target vector into the target vector regression model to output the second target vector of the image information, wherein the target vector regression model is defined by the target style type to which the image information belongs In a corresponding vector regression model, the second target vector includes the target exposure of the first object region corresponding to each first object category in the at least one first object category in the image information. In some embodiments, the device includes multiple style types and a vector regression model corresponding to each style type (for example, each style type has a mapping relationship with its corresponding vector regression model). For example, according to the target style type to which the image information belongs, the device inputs the first target vector of the image information into the target vector regression model corresponding to the target style type, so as to output the second target vector of the image information. In some embodiments, the style types include but are not limited to life style, landscape style, sports style, face close-up style, pet close-up style and other style types. In some embodiments, the target style type to which the image information belongs may be determined according to at least one first object category in the image information, for example, the object category included in each style type among multiple style types preset in the device, According to at least one first object category in the image information, determine the target style type to which the image information belongs (for example, more than half of the first object categories in the at least one first object category are included in a certain style type When it is in the object category, it is determined that the image information belongs to the style type). In some other embodiments, the target style type to which the image information belongs is obtained based on the style classification model. For the specific introduction of this embodiment, please refer to the following embodiments, and details are not repeated here. In some embodiments, the target vector regression model is used to output a corresponding second target vector based on the input first target vector. In some embodiments, the second target vector includes a plurality of second components, for example, the second target vector is [0, 100, 0, 80, 0, 0, 90], and the " 80", "100", "90", "0" and other data are used as the second component of the second target vector. In some embodiments, the second target vector output by the vector regression model includes the first object region corresponding to each first object category in the at least one first object category in the image information target exposure. For example, the first object in the image information A belongs to at least one first object category including: animals, office supplies, and other categories, and the obtained second target vector is [0, 100, 0, 80, 0, 0, 90]. The second component "100" in the second target vector is used as the target exposure of the first object area corresponding to the first object category "animal", and the second component "80" is corresponding to the first object category "office supplies". The target exposure of the first object area, the second component "90" is used as the target exposure of the first object area corresponding to the first object category "other categories". In some embodiments, the device determines the first object category corresponding to each second component based on the arrangement order of each second component in the second target vector, thereby determining that each first object category is in the image For the target exposure of the first object area corresponding to the information, please refer to the following embodiments for specific description, and details are not repeated here. In some embodiments, the first object area corresponding to the first object category in the image information includes the sum of the areas where one or more first objects belonging to the first object category are located, for example, in the image information In A, the first object category of office supplies includes desks and computers, and the first object area corresponding to the first object category "office supplies" in the image information A includes the sum of the areas corresponding to desks and computers . In some embodiments, the device divides the image information into one or more regions based on the YOLO algorithm, and detects the first object corresponding to each region, so as to classify the first objects, and determine each first object The first object area corresponding to the category (for example, the area where the first object belonging to the same first object category is classified as the first object area corresponding to the first object category). In some embodiments, the image information can also be divided into one or more regions based on image segmentation technology, and then the first object corresponding to each region can be identified (for example, based on image recognition technology), so that the The first objects are classified, and the first object area corresponding to each first object category is determined. Of course, those skilled in the art can understand that the above-mentioned specific operations of identifying and segmenting the image information are only examples, and other existing or future specific operations that can be applied to this application are also within the scope of protection of this application , and is incorporated herein by reference. In some embodiments, the exposure of the object area is calculated based on pixel information in the object area. For example, after the object area is grayscaled, the average value of all pixel information in the object area is calculated to obtain the exposure of the object area. In some embodiments, the exposure effect of the first object area is the best when the exposure of the first object area is the corresponding target exposure.

In step S14, the device generates a high dynamic range image of the image information according to the target exposure of the first object area corresponding to each first object category in the image information. For example, after the target exposure of each first object area is obtained, the high dynamic range image of the image information is synthesized based on the target exposure, fully considering the style type of the image information and its own image characteristics, so that the finally obtained High dynamic range images are more realistic and look better.

In some embodiments, before the first target vector of the image information is generated according to at least one first object category in the image information, the user's intention to shoot is determined by judging the target style type to which the image information belongs. For example, the shooting user wants to take photos of landscape style, parent-child outfit style, or sportswear style, and then obtain the target vector regression model corresponding to the target style type according to the target style type to which the image information belongs, so that the final synthesized The effect of the high dynamic range image matches the target style type to which the image information belongs, increasing the authenticity and image effect of the high dynamic range image. In some embodiments, the method further includes step S15 (not shown). In step S15, the characteristic information of the image information is input into the style classification model, so as to determine the target style type of the image information according to the output result; and the target vector regression model corresponding to the target style type is obtained. In some embodiments, the target style type to which the image information belongs is obtained by inputting the feature information of the image information into the style classification model. This embodiment specifically introduces the process of obtaining the target style type to which the image information belongs based on the style type model. For example, the style classification model is a model trained based on VGGNet, resNet and the like. By inputting the image information into the style classification model, the target style type to which the image information belongs is output. In some embodiments, a vector regression model is trained separately for each style type, for example, the device includes multiple mapping relationships, and each mapping relationship is used to associate the style type with the target vector regression model corresponding to the style type, In order to query and obtain the target vector regression model corresponding to the target style type based on the target style type to which the image information belongs.

In some embodiments, the method further includes step S16 (not shown). In step S16, the style classification model is constructed based on a plurality of pictures and a style type label of each picture. For example, the style classification model is trained based on a large number of pictures and the style type labels of each picture, so that when image information is input into the style classification model, the target style type to which the image information belongs can be determined according to the output result.

In some embodiments, the step S11 includes: the device acquires image information to be processed; determines one or more first objects appearing in the image information; according to each of the one or more first objects The first object category to which the first object belongs determines at least one first object category to which the one or more first objects belong. In some embodiments, the user equipment acquires the image information in response to a user's viewfinding operation. In some embodiments, the device identifies the first object appearing in the image information based on an image recognition technology. In some embodiments, the device may also detect the first object appearing in the image information based on the YOLO algorithm. In some embodiments, the device determines a first object class to which each first object belongs to determine the at least one first object class. For example, the image information A includes the first objects: kitten, desk, computer, sky, wherein, the first object kitten belongs to animals, the first object desk belongs to office supplies, the first object computer belongs to office supplies, and the first object If the sky belongs to other categories, it is determined that the first object in the image information belongs to the three first object categories of "animal, office supplies, and other categories". In some embodiments, a mapping relationship between each of the multiple object categories and its corresponding object is established in the device, so as to determine the object category to which the object belongs based on the determined object. In some embodiments, the first object that fails to be recognized is classified into other categories.

In some embodiments, the step S12 includes: the device determines the assignment of each first component in the first initial vector according to the at least one first object category and the category set, so as to generate the first target vector of the image information, Wherein, the first initial vector corresponds to the category set. In some embodiments, the first target vector includes multiple first components, and the device generates the first target vector of the image information by assigning a value to the first initial vector. In some embodiments, the device determines the assignment of the corresponding first component in the first initial vector based on the determined first object category and category set in the image information, so as to generate the first object category of the image information. target vector. For example, the reassigned first initial vector is used as the first target vector of the image information. In some embodiments, the first initial vector corresponds to the class set, so that the first initial vector is assigned a value based on the class set.

In some embodiments, the class set includes a plurality of second object classes arranged in sequence, the first initial vector includes a plurality of first components, and the number of the plurality of second object classes is the same as the number of the plurality of The numbers of the first components are equal, and based on the arrangement order of the plurality of second object categories, each second object category in the plurality of second object categories has its corresponding first component in the first initial vector. Components, the initial assignment of each first component is zero; the step S12 includes: if there is a second object category identical to the first object category in the category set, according to the target assignment, the second object category is placed in the The corresponding first component in the first initial vector is reassigned to generate a first target vector of the image information. Here, words such as "first", "second", and "third" mentioned in this application are only used to distinguish information in different objects (for example, image information, pictures, category collections), and do not represent any order. In some embodiments, the category set includes a plurality of second object categories arranged in sequence, for example, the category set B includes in sequence: people, animals, food, office supplies, school supplies, vehicles, other categories (The object that fails to be identified or determined can be recorded as other, and it is classified as the second object category of other categories). Of course, those skilled in the art should be able to understand that the above-mentioned class collection is only an example, and other existing or future class collections that can be applied to this application are also within the scope of protection of this application and are included by reference. here. The category set B corresponds to a first initial vector B, for example, the first initial vector B is [0,0,0,0,0,0,0], wherein the second object category in the category set B The number of is equal to the number of the first components in the first initial vector B, which are seven. Based on the arrangement order of the seven second object categories, each second object category has its corresponding first component in the first initial vector, for example, a person corresponds to the first component in the first initial vector B animal corresponds to the second first component in the first initial vector B, food corresponds to the third first component in the first initial vector B, and so on. In some embodiments, the target assignment includes but is not limited to a fixed value such as 1. For example, if there is a second object category identical to the first object category in the category set, the first component corresponding to the second object category in the first initial vector is reassigned to 1. For example, the category set B includes the second object categories arranged in order: people, animals, food, office supplies, school supplies, vehicles, and other categories, and the first initial vector B is [0,0,0,0,0 ,0,0], the first object category appearing in the image information includes animals, office supplies, and other categories, then there are animals, office supplies, and other categories in the category set that are the same as the first object category, Re-assign a value of 1 to the first component corresponding to animals, office supplies, and other categories in the first initial vector B, then generate the first target vector [0,1,0,1,0,0, 1].

In some embodiments, the device sequentially detects whether there is a first object category that is the same as the second object category among the one or more first object categories according to the arrangement order of the plurality of second object categories in the category set, If it exists, reassign the first component corresponding to the second object category in the first initial vector according to the target assignment, so as to generate the first target vector of the image information. For example, the device sequentially detects whether there is a first object category that is the same as the second object category in the first object category according to the order of the second object category, and if it exists, the second object category corresponding to the second object category A component reassignment. For example, the category set B includes the second object categories arranged in order: people, animals, food, office supplies, school supplies, vehicles, and other categories, and the first initial vector B is [0,0,0,0,0 ,0,0], the first object category appearing in the image information includes animals, office supplies, and other categories. The device first detects whether there is a person in the first object category based on the arrangement order of the plurality of second object categories, and if the result does not exist, there is no need to re-assign the first component corresponding to the person, and the first component corresponding to the person It is still the initial assignment (for example, 0), and then detects whether there is an animal in the first object category, and the result is existence, then reassigns the first component corresponding to the animal, for example, the assignment is the target assignment (for example, 1) , and then detect whether there is food in the first object category, and the result is that it does not exist, then the first component corresponding to the food is still the initial assignment (for example, 0), and so on, after detecting the second object category in order , the first target vector of the image information can be generated.

In some embodiments, the second object vector includes a plurality of second components, the number of the plurality of second components is equal to the number of the plurality of second object categories, based on the arrangement of the second object categories Sequentially, each second object category in the plurality of second object categories has its corresponding second component in the second target vector, the method also includes step S17, in step S17, for the For each first object category in at least one first object category in the image information, the assignment of the second component corresponding to the second object category that is the same as the first object category in the second target vector is used as the first object category The target exposure of the first object area corresponding to an object category in the image information. In some embodiments, there is also a corresponding relationship between the second target vector and the category set, so as to determine the first object category corresponding to each second component according to the arrangement order of each second component in the second target vector , so as to determine the target exposure of the first object region corresponding to each first object category. For example, the second target vector includes a plurality of second components, the number of the plurality of second components is equal to the number of the plurality of second object categories in the category set, and, based on the plurality of second object The arrangement sequence of categories, the plurality of second object categories all have their corresponding second components in the second target component. For example, the category set B includes the second object categories arranged in order: people, animals, food, office supplies, school supplies, vehicles, and other categories, and the second target vector output based on the vector regression model is [0, 100, 0, 80, 0, 0, 90], based on the arrangement order of the plurality of second object categories, the second component corresponding to the person is 0, indicating that there is no first object category of the person in the image information, and the animal corresponds to The second component of is 100, then 100 is used as the target exposure of the first object area corresponding to the first object category "animal" in the image information, and the second component corresponding to food is 0, then the image information There is no food in the first object category, and so on, so as to determine the target exposure of the first object area corresponding to each first object category in the image information.

In some embodiments, the method further includes step S18 (not shown), in step S18, obtaining multiple pictures of multiple style types; for each style type, according to the multiple pictures belonging to the style type, The first vector and the second vector construct the vector regression model of the style type, so as to obtain the vector regression model corresponding to each style type in the plurality of style types. In some embodiments, for each style type, a vector regression model corresponding to the style type is separately trained, so that the final synthesized high dynamic range image conforms to the pixel style of the target style type corresponding to the image information. In some embodiments, for the vector regression model of each style type, the vector regression model is trained based on a large number of first vectors and second vectors of pictures belonging to the style type. In some embodiments, each vector regression model is trained using an NFM network. The vector regression model is obtained by performing training with the first vector and the second vector belonging to a large number of pictures. Therefore, by inputting the first target vector to the vector regression model, the corresponding second target vector can be output. In some embodiments, for the vector regression model corresponding to each style type, the multiple pictures of the style type used to train the vector regression model are pictures with better exposure effects, so that according to the output When the target exposure included in the second target vector is combined with a high dynamic range image, the effect is better.

In some embodiments, the method further includes step S19 (not shown) and step S10. In step S19, the device, for each of the multiple pictures belonging to the style type, according to the At least one third object category to which one or more third objects belong generates a first vector of the picture; in step S10, the device generates a first vector of the picture according to each third object category in the at least one third object category The corresponding exposure of the third object area generates a second vector of the picture. In some embodiments, for multiple pictures of each style type, determine the third object (for example, items such as sky, table, kitten) appearing in each picture, for example, by image recognition technology, or YOLO algorithm Detect the third object in each picture. Generate the first vector of each picture based on the category set, for example, picture C includes the third object: kitten, desk, computer, sky, after classifying described kitten, desk, computer, sky, can get this The third object in picture C belongs to at least one third object category (eg, animals, office supplies, other categories). Generate the first vector (for example, [0,1,0,1,0,0,1]) corresponding to the picture C based on the three third object categories of "animals, office supplies, and other classes", so that the Image C vectorization. Further, the second vector of the picture is generated according to the exposure of the third object region corresponding to each third object category in the picture. For example, based on the YOLO algorithm, the picture is divided into one or more regions, and the third object corresponding to each region is detected to classify the third object, and the first object corresponding to each third object category is determined. The object area (for example, the area where the third objects belonging to the same third object category are located is converted into the third object area corresponding to the third object category). In some embodiments, the picture can also be divided into one or more regions based on image segmentation techniques (for example, image segmentation algorithms such as resNet, VGGNet, Fast, R-CNN), and then identify the third object corresponding to each region ( For example, based on an image recognition technology), the third objects are classified, and the third object area corresponding to each third object category is determined. Of course, those skilled in the art can understand that the above-mentioned specific operations of identifying and segmenting the pictures are only examples, and other existing or future specific operations that can be applied to this application are also within the scope of protection of this application. and is incorporated herein by reference. Then calculate the exposure of each third object area, and generate the second vector of the picture according to the exposure of each third object area. Thus, the first vector and the second vector of each picture are obtained.

In some embodiments, the step S17 includes: for each of the multiple pictures belonging to the style type, according to at least one third object to which one or more third objects appearing in the picture belong The object category and the category set determine the assignment of each first component in the first initial vector to generate the first vector of the picture, wherein the first initial vector corresponds to the category set. In some embodiments, based on the same category set as in actual application (for example, the above category set when determining the first target vector), the assignment of each first component in the first initial vector corresponding to the category set is determined to generate The first vector of each of the multiple pictures belonging to the style type. For example, the reassigned first initial vector is used as the first vector of the picture. In some embodiments, the first initial vector corresponds to the class set, so that the first initial vector is assigned a value based on the class set.

In some embodiments, the class set includes a plurality of second object classes arranged in sequence, the first initial vector includes a plurality of first components, and the number of the plurality of second object classes is the same as the number of the plurality of The number of the first components is equal, so that each second object category has its corresponding first component in the first initial vector, and the initial assignment of each first component is zero; the step S19 includes: if There is a second object category that is the same as the third object category in the category set, and reassigns the first component corresponding to the second object category in the first initial vector according to the target assignment, so as to generate the first vector. In some embodiments, the category set includes a plurality of second object categories arranged in sequence, for example, the category set B includes in sequence: people, animals, food, office supplies, school supplies, vehicles, other categories The second object class for . Of course, those skilled in the art should be able to understand that the above-mentioned class collection is only an example, and other existing or future class collections that can be applied to this application are also within the scope of protection of this application and are included by reference. here. The category set B corresponds to a first initial vector B, for example, the first initial vector B is [0,0,0,0,0,0,0], wherein the second object category in the category set B The number of is equal to the number of the first components in the first initial vector B, both of which are 7. Based on the arrangement order of the seven second object categories, each second object category has its corresponding first component in the first initial vector B, for example, a character corresponds to the first component in the first initial vector B One first component, animal corresponds to the second first component in the first initial vector B, food corresponds to the third first component in the first initial vector B, and so on. In some embodiments, the target assignment includes but is not limited to a fixed value such as 1. For example, if there is a second object category that is the same as the third object category in the picture in the category set, the first component corresponding to the second object category in the first initial vector is reassigned to a fixed value 1. For example, the category set B includes the second object categories arranged in order: people, animals, food, office supplies, school supplies, vehicles, and other categories, and the first initial vector B is [0,0,0,0,0 , 0, 0], the third object appearing in a certain picture includes orange, banana, desk, others (for example, the items that are not identified can be marked with other), then there are food, office supplies, Other categories (for example, those that are not identified are classified as other categories) are the same as the third object category corresponding to the third object, and food, office supplies, and other categories are in the first initial vector B corresponding to the first If the component is reassigned to 1, the first vector to generate the picture is [0,0,1,1,0,0,1]. In some embodiments, the specific process of generating the first vector of the picture includes: the device sequentially detects whether one or more third object categories in the picture are There is a first object category that is the same as the second object category, and if it exists, reassign the first component corresponding to the second object category in the first initial vector according to the target assignment, so as to generate the image information The first target vector.

In some embodiments, the step S10 includes step S101 (not shown), step S102, and step S103. In step S101, the device determines the third object area corresponding to each third object category in the picture of at least three third object categories in the picture; in step S102, the device calculates the third object area of each third object category exposure of the at least one third object category to obtain the exposure of the third object area corresponding to each third object category in the picture; in step S103, the device The exposure of each third object category in the category corresponding to the third object area in the picture and the category set determine the assignment of the second components in the second initial vector to generate the second vector of the picture, wherein the The second initial vector corresponds to the class set. In some embodiments, for each picture, it is necessary to first determine the third object area of each third object category in the picture, then calculate the exposure of each third object area, and then calculate the The exposure of the region generates a second vector for the image. In some embodiments, when generating the second vector, it is also necessary to determine the second component corresponding to the exposure of each third object region in the second initial vector based on the category set, so that according to the first The exposure of the three-object area is assigned to the second component to generate a second vector of the picture.

In some embodiments, the step S101 includes: the device determines one or more third objects appearing in the picture and the third object sub-regions corresponding to each third object; The third object sub-area corresponding to the third object serves as the third object area corresponding to the third object category. In some embodiments, for each picture, the third object appearing in the picture is detected based on the YOLO algorithm, and the third object sub-area corresponding to each third object (for example, the area where the third object is located) . In some embodiments, one or more third objects in the picture can also be determined based on image segmentation technology (for example, image segmentation algorithms such as resNet, VGGNet, Fast, R-CNN) and image recognition technology, and each third object A third object sub-area corresponding to the object (for example, the area where the third object is located). In some embodiments, the third object sub-area corresponding to the third object belonging to the same third object category is used as the third object area corresponding to the third object category, for example, if a kitten or a puppy belongs to the animal category, then The area where the kitten and the puppy are located is determined as the third object area corresponding to the third object category of animal, in other words, the third object area corresponding to the third object category of "animal" includes the third object where the kitten and the puppy are located sum of subregions.

In some embodiments, the step S102 includes: the device calculates the exposure of each third object area according to the pixel information of the third object area, so as to obtain the exposure of each third object category in the at least one third object category Exposure of the corresponding third object area in the picture. In some embodiments, after the third object area is divided, the device calculates the exposure of each third object area according to all pixel information in each third object area. For example, after the third object area is grayscaled, an average value of all pixel information in the third object area is calculated, and the average value is used as an exposure degree of the third object area.

In some embodiments, the category set includes a plurality of second object categories arranged in sequence, the second initial vector includes a plurality of second components, each second component is initially assigned a value of zero, and the plurality of The quantity of the second object category is equal to the quantity of the plurality of second components, so that each second object category has its corresponding second component in the second initial vector, and the step S103 includes: if There is a second object category identical to the third object category in the category set, and the second object category is included in the second initial vector according to the exposure of the third object region corresponding to the third object category The corresponding second component is reassigned to generate the second vector of the picture. In some embodiments, the category set also corresponds to a second initial vector, the initial assignment of each second component in the second initial vector is zero, based on the arrangement order of each second object category in the category set , each second object category has its corresponding second component in the second initial vector, according to the arrangement order of the third object category and the second object category, determine the value of each second component in the second initial vector assignment. For example, the category set B includes in order: people, animals, food, office supplies, school supplies, vehicles, other categories (for example, for image information or objects that fail to be recognized or determined in pictures, they can be recorded as other, and Classify it as a second object class of other classes). The category set B corresponds to a second initial vector B, for example, the second initial vector B is [0,0,0,0,0,0,0], wherein the second object category in the category set B The number of is equal to the number of the second components in the second initial vector B, both of which are 7. Based on the arrangement order of the seven second object categories, each second object category has its corresponding second component in the second initial vector, for example, a person corresponds to the first component in the second initial vector B animal corresponds to the second second component in the second initial vector B, food corresponds to the third second component in the second initial vector B, and so on. If there is a second object category identical to the third object category in the category set, reassign a value to a second component corresponding to the second object category in the second initial vector. The specific numerical value reassigned is the exposure degree of the third object area corresponding to the third object category. For example, the category set B includes the second object categories arranged in order: people, animals, food, office supplies, school supplies, vehicles, and other categories, and the second initial vector B is [0,0,0,0,0 ,0,0], the third object category appearing in the picture includes animals, office supplies, and other categories, wherein the exposure of the third object area corresponding to the animal is 80, and the exposure of the third object area corresponding to the office supplies is degree is 100, and the exposure degree of the third object area corresponding to other categories is 90, according to the exposure degree corresponding to each third object category, the corresponding second component in the second initial vector B is reassigned to generate The second vector for this image (for example, [0,80,0,100,0,0,90]).

In some embodiments, the process of obtaining the class set includes: determining the second objects included in each of the plurality of pictures to obtain a plurality of second objects; The plurality of second objects are classified to obtain a plurality of second object categories, wherein each second object category includes one or more second objects; according to the number of second objects included in each second object category Quantitatively sort the plurality of second object categories in descending order to generate the category set, wherein the category set includes a plurality of second object categories arranged in order. In some embodiments, the category set is generated by counting categories of the second objects appearing in a large number of pictures. For example, second objects appearing in a large number of pictures are identified to obtain multiple second objects, and the multiple second objects are classified and divided to obtain multiple second object categories. counting the number of second objects included in each second object category, sorting the plurality of second object categories based on the number of second objects included in each second object category, and obtaining a sequence of multiple a second object class. In some embodiments, the plurality of ordered second object categories are recorded in the set of categories.

In some embodiments, the method further includes step S141 (not shown) before step S14, in step S141, performing exposure sampling on the image information based on different exposure parameters to obtain at least two spare image information ; For each of the backup image information, calculate the exposure of the first object area corresponding to each first object category in the backup image information in the backup image information, so as to obtain at least Two exposures; the step S14 includes: for each of the first object areas, according to the target exposure corresponding to the first object area, determine the corresponding target exposure from one or more exposures corresponding to the first object area An exposure with the smallest difference between the target exposures; generating a high dynamic range image of the image information according to the pixel information of the first object area in the spare image information corresponding to the exposure. In some embodiments, before generating the high dynamic range image of the image information, the device collects a plurality of backup image information based on different exposure parameters, so as to generate the high dynamic range image based on the backup image information. For example, the image information A includes the first object: a kitten, a book, a computer, and the sky, wherein, the kitten belongs to animals, the book and the computer belong to office supplies, and the sky belongs to other categories, then one or more of the first objects in the image information A An object belongs to three first object classes of animal, office supplies, and others. Wherein, the first object area corresponding to the first object category "animal" includes the area where "kitten" is located, and the first object area corresponding to the first object category "office supplies" includes the area where "book" and "computer" are located. And, the first object area corresponding to the first object category "other categories" includes the area where "sky" is located. The device collects a plurality of backup image information about the image information A based on different exposure parameters (for example, exposure parameters such as aperture, shutter speed, ISO sensitivity, etc.), and calculates the number of first object categories in each backup image information. The corresponding exposure of the first object area. For example, backup image information 1, backup image information 2, and backup image information 3 are obtained. Calculate the exposure of the first object area according to the pixel information in each first object area (for example, calculate the average value of all pixel values in the first object area, and use the calculated average value as the first object area exposure). Then, for the image information A, each first object category in the image information A corresponds to 3 exposure levels. For each first object category, determine an exposure with the smallest target exposure difference corresponding to the first object category from the three exposures (for example, the first object category calculated from the backup image information 1 The difference between the corresponding exposure and the target exposure corresponding to the first object category is the smallest), then according to the backup image information (for example, backup image information 1) corresponding to the exposure, the first object category is in the The pixel information of the first object area in the spare image information 1 generates the high dynamic range image of the image information A. In some embodiments, the device synthesizes the high dynamic range image of the image information A by extracting each first object area in the spare image information. For example, if the difference between the exposure corresponding to the first object category "animal" in the backup image information 1 and the target exposure corresponding to the first object category is the smallest, then the first object category "animal" is extracted from the backup image information 1 The first object area of . The difference between the exposure corresponding to the first object category "office supplies" in the backup image information 2 and the target exposure corresponding to the first object category is the smallest, then extract the first object category "office supplies" from the backup image information 2 The first object area of . The difference between the exposure degree corresponding to the first object category "other category" in the backup image information 3 and the target exposure degree corresponding to the first object category is the smallest, then extract the first object category "other category" from the backup image information 3 The first object area of . The high dynamic range image of the image information A is synthesized by extracting each first object area. For another example, if the difference between the exposure corresponding to the first object category "animal" in the backup image information 1 and the target exposure corresponding to the first object category is the smallest, then according to the first object category "animal" in the backup image information 1 The corresponding pixel information of the first object area processes the first object area in the image information A. The difference between the exposure degree corresponding to the first object category "office supplies" in the backup image information 2 and the target exposure degree corresponding to the first object category is the smallest, then according to the The corresponding pixel information of the first object area processes the first object area in the image information A. The difference between the exposure degree corresponding to the first object category "other category" in the backup image information 3 and the target exposure degree corresponding to the first object category is the smallest, then according to the value of the first object category "other category" in the backup image information 3 The corresponding pixel information of the first object area processes the first object area in the image information A.

Fig. 2 shows a structural diagram of a device for generating a high dynamic range image according to one aspect of the present application, the device includes a module 1, a module 12, a module 13 and a module 14. - a module for acquiring image information to be processed, wherein the image information includes one or more first objects, and the one or more first objects belong to at least one first object category; module one or two for for generating a first target vector of the image information according to the at least one first object category; a module for inputting the first target vector into a target vector regression model to output a second target of the image information vector, wherein the target vector regression model is a vector regression model corresponding to the target style type to which the image information belongs, and the second target vector includes each first object category in the at least one first object category The target exposure of the first object area corresponding to the image information; a module configured to generate the target exposure of the first object area corresponding to each first object category in the image information A high dynamic range image of the image information described above.

Specifically, a module configured to acquire image information to be processed, wherein the image information includes one or more first objects, and the one or more first objects belong to at least one first object category. In some embodiments, the image information includes, but is not limited to, image information acquired by a viewfinder of the user equipment. For example, when a user device (such as a mobile phone) takes a view through a viewfinder, the image information can be collected, so that the device can analyze and process the image information. In some embodiments, the first object includes, but is not limited to, items appearing in the image information (eg, cups, books, computers, sky, etc.). In some embodiments, first objects appearing in the image information are classified to determine at least one first object category corresponding to the one or more first objects. For example, there are first objects in the image information A: kitten, book, computer, and sky, wherein, the kitten belongs to animals, books and computers belong to office supplies, and the sky belongs to other categories, then one or more objects in the image information A A first object belongs to the three first object categories of animals, office supplies, and others.

A two-module, configured to generate a first target vector of the image information according to the at least one first object category. In some embodiments, the first target vector is generated based on the at least one first object category. In some embodiments, the first target vector includes a plurality of first components, for example, the first target vector is [0,1,0,1,0,0,1], wherein the first Data such as "0" and "1" in the target vector are used as the first component of the first target vector. For example, the image information A includes the first object: a kitten, a desk, a computer, and the sky. After classifying the kitten, desk, computer, and sky, it can be obtained that the first object in the image information A belongs to at least A first object class (eg, animals, office supplies, other classes). The device generates corresponding first object vectors (for example, [0,1,0,1,0,0,1]) based on the three first object categories of "animals, office supplies, and others", so that all The above image information A is vectorized.

A three-module, configured to input the first target vector into a target vector regression model to output a second target vector of the image information, wherein the target vector regression model is defined by the target style type to which the image information belongs In a corresponding vector regression model, the second target vector includes the target exposure of the first object region corresponding to each first object category in the at least one first object category in the image information. In some embodiments, the device includes multiple style types and a vector regression model corresponding to each style type (for example, each style type has a mapping relationship with its corresponding vector regression model). For example, according to the target style type to which the image information belongs, the device inputs the first target vector of the image information into the target vector regression model corresponding to the target style type, so as to output the second target vector of the image information. In some embodiments, the style types include but are not limited to life style, landscape style, sports style, face close-up style, pet close-up style and other style types. In some embodiments, the target style type to which the image information belongs may be determined according to at least one first object category in the image information, for example, the object category included in each style type among multiple style types preset in the device, According to at least one first object category in the image information, determine the target style type to which the image information belongs (for example, more than half of the first object categories in the at least one first object category are included in a certain style type When it is in the object category, it is determined that the image information belongs to the style type). In some other embodiments, the target style type to which the image information belongs is obtained based on the style classification model. For the specific introduction of this embodiment, please refer to the following embodiments, and details are not repeated here. In some embodiments, the target vector regression model is used to output a corresponding second target vector based on the input first target vector. In some embodiments, the second target vector includes a plurality of second components, for example, the second target vector is [0, 100, 0, 80, 0, 0, 90], and the " 80", "100", "90", "0" and other data are used as the second component of the second target vector. In some embodiments, the second target vector output by the vector regression model includes the first object region corresponding to each first object category in the at least one first object category in the image information target exposure. For example, the first object in the image information A belongs to at least one first object category including: animals, office supplies, and other categories, and the obtained second target vector is [0, 100, 0, 80, 0, 0, 90]. The second component "100" in the second target vector is used as the target exposure of the first object area corresponding to the first object category "animal", and the second component "80" is corresponding to the first object category "office supplies". The target exposure of the first object area, the second component "90" is used as the target exposure of the first object area corresponding to the first object category "other categories". In some embodiments, the device determines the first object category corresponding to each second component based on the arrangement order of each second component in the second target vector, thereby determining that each first object category is in the image For the target exposure of the first object area corresponding to the information, please refer to the following embodiments for specific description, and details are not repeated here. In some embodiments, the first object area corresponding to the first object category in the image information includes the sum of the areas where one or more first objects belonging to the first object category are located, for example, in the image information In A, the first object category of office supplies includes desks and computers, and the first object area corresponding to the first object category "office supplies" in the image information A includes the sum of the areas corresponding to desks and computers . In some embodiments, the device divides the image information into one or more regions based on the YOLO algorithm, and detects the first object corresponding to each region, so as to classify the first objects, and determine each first object The first object area corresponding to the category (for example, the area where the first object belonging to the same first object category is classified as the first object area corresponding to the first object category). In some embodiments, the image information can also be divided into one or more regions based on image segmentation technology, and then the first object corresponding to each region can be identified (for example, based on image recognition technology), so that the The first objects are classified, and the first object area corresponding to each first object category is determined. Of course, those skilled in the art can understand that the above-mentioned specific operations of identifying and segmenting the image information are only examples, and other existing or future specific operations that can be applied to this application are also within the scope of protection of this application , and is incorporated herein by reference. In some embodiments, the exposure of the object area is calculated based on pixel information in the object area. For example, after the object area is grayscaled, the average value of all pixel information in the object area is calculated to obtain the exposure of the object area. In some embodiments, the exposure effect of the first object area is the best when the exposure of the first object area is the corresponding target exposure.

A module, configured to generate a high dynamic range image of the image information according to the target exposure of the first object region corresponding to each first object category in the image information. For example, after the target exposure of each first object area is obtained, the high dynamic range image of the image information is synthesized based on the target exposure, fully considering the style type of the image information and its own image characteristics, so that the finally obtained High dynamic range images are more realistic and look better.

In some embodiments, before the first target vector of the image information is generated according to at least one first object category in the image information, the user's intention to shoot is determined by judging the target style type to which the image information belongs. For example, the shooting user wants to take photos of landscape style, parent-child outfit style, or sportswear style, and then obtain the target vector regression model corresponding to the target style type according to the target style type to which the image information belongs, so that the final synthesized The effect of the high dynamic range image matches the target style type to which the image information belongs, increasing the authenticity and image effect of the high dynamic range image. In some embodiments, the device further includes a module (not shown). A five-module, used to input the feature information of the image information into the style classification model, so as to determine the target style type of the image information according to the output result; obtain the target vector regression model corresponding to the target style type.

Here, the specific implementation manner corresponding to the first and fifth modules is the same as or similar to the specific embodiment of the step S15, so it will not be repeated here, and is included here by reference.

In some embodiments, the device further includes a six module (not shown), a six module for constructing the style classification model based on a plurality of pictures and a style type label of each picture.

Here, the specific implementation manner corresponding to the six modules is the same as or similar to the specific embodiment of the step S16, so it will not be repeated here, and is included here by reference.

In some embodiments, the one-by-one module is configured to acquire image information to be processed; determine one or more first objects appearing in the image information; according to each of the one or more first objects The first object category to which the first objects belong determines at least one first object category to which the one or more first objects belong.

Here, the specific implementation manners corresponding to the modules one by one are the same as or similar to the specific embodiment of the step S11, so they are not repeated here, and are included here by reference.

In some embodiments, the one-two module is configured to determine the assignment of each first component in the first initial vector according to the at least one first object category and the category set, so as to generate the first target vector of the image information , where the first initial vector corresponds to the category set.

Here, the specific implementation manner corresponding to the first and second modules is the same as or similar to the specific embodiment of the step S12, so it will not be repeated here, and is included here by reference.

In some embodiments, the class set includes a plurality of second object classes arranged in sequence, the first initial vector includes a plurality of first components, and the number of the plurality of second object classes is the same as the number of the plurality of The numbers of the first components are equal, and based on the arrangement order of the plurality of second object categories, each second object category in the plurality of second object categories has its corresponding first component in the first initial vector. Components, the initial assignment of each first component is zero; the one-two module is used to: if there is a second object category that is the same as the first object category in the category set, assign the second object according to the target value The first component corresponding to the category in the first initial vector is reassigned to generate a first target vector of the image information.

Here, the specific implementation manner corresponding to the first and second modules is the same as or similar to the specific embodiment of the step S12, so it will not be repeated here, and is included here by reference.

In some embodiments, the second object vector includes a plurality of second components, the number of the plurality of second components is equal to the number of the plurality of second object categories, based on the arrangement of the second object categories Sequentially, each second object category in the plurality of second object categories has its corresponding second component in the second target vector, and the device further includes a seven module, and a seven module is used for, for For each first object category in the at least one first object category in the image information, the assignment of the second component corresponding to the second object category that is the same as the first object category in the second target vector is taken as The target exposure of the first object area corresponding to the first object category in the image information.

Here, the specific implementation manner corresponding to the 17th module is the same as or similar to the specific embodiment of the step S17, so it will not be repeated here, and is included here by reference.

In some embodiments, the device also includes an eighth module (not shown), an eighth module for obtaining multiple pictures of multiple style types; for each style type, according to the multiple pictures belonging to the style type The first vector and the second vector of the picture construct the vector regression model of the style type, so as to obtain the vector regression model corresponding to each style type in the plurality of style types.

Here, the specific implementation manner corresponding to the 18th module is the same as or similar to the specific embodiment of the step S18, so it will not be repeated here, and is included here by reference.

In some embodiments, the device further includes a nine module (not shown) and a zero module, a nine module, for each picture in a plurality of pictures belonging to the style type, according to the At least one third object category to which one or more third objects in the picture belong generates a first vector of the picture; a zero module, configured to be in the at least one third object category according to each third object category in the at least one third object category The exposure of the corresponding third object area in the picture generates the second vector of the picture.

Here, the specific implementation manners corresponding to the one-nine module and one-zero module are the same as or similar to the specific embodiments of the step S19 and the step S10, so they are not repeated here, and are included here by reference.

In some embodiments, the module is configured to: for each of the multiple pictures belonging to the style type, according to at least one of the one or more third objects appearing in the picture belongs to The third object category and the category set determine the assignment of each first component in the first initial vector to generate the first vector of the picture, wherein the first initial vector corresponds to the category set.

Here, the specific implementation manner corresponding to the 17th module is the same as or similar to the specific embodiment of the step S17, so it will not be repeated here, and is included here by reference.

In some embodiments, the class set includes a plurality of second object classes arranged in sequence, the first initial vector includes a plurality of first components, and the number of the plurality of second object classes is the same as the number of the plurality of The number of the first components is equal, so that each second object category has its corresponding first component in the first initial vector, and the initial assignment of each first component is zero; the one-nine module is used for : If there is a second object category that is the same as the third object category in the category set, reassign the first component corresponding to the second object category in the first initial vector according to the target assignment, so as to generate the The first vector of images.

Here, the specific implementation manner corresponding to the nineteenth module is the same as or similar to the specific embodiment of the step S19, so it is not repeated here, and is included here by reference.

In some embodiments, the zero module includes a zero one module (not shown), a zero two module, and a zero three module. 101 module, used to determine the third object area corresponding to each third object category in the picture in at least three third object categories in the picture; 102 module, used to calculate each third object category The exposure degree of the object area, so as to obtain the exposure degree of the third object area corresponding to each third object category in the picture in the at least one third object category; a zero three module, configured to The exposure of each third object category in the picture corresponding to the third object area in the third object category and the category set determine the assignment of each second component in the second initial vector, so as to generate the second vector of the picture, Wherein, the second initial vector corresponds to the category set.

Here, the specific implementations corresponding to the 101 module, the 102 module, and the 103 module are the same or similar to the specific embodiments of the step S101, step S102, and step S103, so they will not be described in detail here. method is included here.

In some embodiments, the one-zero-one module is used to: determine one or more third objects appearing in the picture and the third object sub-regions corresponding to each third object; will belong to the same third object The third object sub-area corresponding to the third object of the category serves as the third object area corresponding to the third object category.

Here, the specific implementation manner corresponding to the 101 module is the same as or similar to the specific embodiment of the step S101, so it is not repeated here, and is included here by reference.

In some embodiments, the one-two module is configured to calculate the exposure of each third object region according to the pixel information of the third object region, so as to obtain each third object in the at least one third object category Exposure of the third object area corresponding to the category in the picture.

Here, the specific implementation manner corresponding to the first and second modules is the same as or similar to the specific embodiment of the step S12, so it will not be repeated here, and is included here by reference.

In some embodiments, the category set includes a plurality of second object categories arranged in sequence, the second initial vector includes a plurality of second components, each second component is initially assigned a value of zero, and the plurality of The quantity of the second object category is equal to the quantity of the plurality of second components, so that each second object category has its corresponding second component in the second initial vector, and the one-three module is used for : If there is a second object category identical to the third object category in the category set, the second object category is placed in the second initial stage according to the exposure of the third object area corresponding to the third object category The corresponding second component in the vector is reassigned to generate the second vector of the picture.

Here, the specific implementation manners corresponding to the first and third modules are the same as or similar to the specific embodiment of step S13, so they are not repeated here, and are included here by reference.

In some embodiments, the process of obtaining the class set includes: determining the second objects included in each of the plurality of pictures to obtain a plurality of second objects; The plurality of second objects are classified to obtain a plurality of second object categories, wherein each second object category includes one or more second objects; according to the number of second objects included in each second object category Quantitatively sort the plurality of second object categories in descending order to generate the category set, wherein the category set includes a plurality of second object categories arranged in order. In some embodiments, the category set is generated by counting categories of the second objects appearing in a large number of pictures. For example, second objects appearing in a large number of pictures are identified to obtain multiple second objects, and the multiple second objects are classified and divided to obtain multiple second object categories. counting the number of second objects included in each second object category, sorting the plurality of second object categories based on the number of second objects included in each second object category, and obtaining a sequence of multiple a second object class. In some embodiments, the plurality of ordered second object categories are recorded in the set of categories.

In some embodiments, the device further includes a 41 module (not shown), the 141 module is used to perform exposure sampling on the image information based on different exposure parameters, so as to obtain at least two spare Image information; for each of the backup image information, calculate the exposure of the first object area corresponding to each first object category in the backup image information in the backup image information, so as to obtain the corresponding at least two exposures; the one-fourth module is used for: for each of the first object areas, according to the target exposure corresponding to the first object area, one or more exposures corresponding to the first object area determining an exposure degree with the smallest difference from the target exposure degree; generating a high dynamic range image of the image information according to the pixel information of the first object area in the standby image information corresponding to the exposure degree.

Here, the specific implementation manners corresponding to the 141 module and the 14 module are the same as or similar to the specific embodiments of the step S141 and the step S14, so they are not repeated here, and are included here by reference.

In addition to the methods and devices introduced in the above embodiments, the present application also provides a computer-readable storage medium, the computer-readable storage medium stores computer codes, and when the computer codes are executed, as described in any one of the preceding items The described method is carried out.

The present application also provides a computer program product, when the computer program product is executed by a computer device, the method described in any one of the preceding items is executed.

The present application also provides a kind of computer equipment, and described computer equipment comprises:

one or more processors;

memory for storing one or more computer programs;

When the one or more computer programs are executed by the one or more processors, the one or more processors are made to implement the method as described in any one of the preceding items.

FIG. 3 illustrates an exemplary system that may be used to implement various embodiments described in this application;

As shown in FIG. 3 , in some embodiments, the system 300 can be used as any device in each of the above embodiments. In some embodiments, system 300 may include one or more computer-readable media (e.g., system memory or NVM/storage device 320 ) having instructions and be coupled to and configured to execute The instructions are one or more processors (eg, processor(s) 305 ) that implement a module to perform the actions described in this application.

For one embodiment, system control module 310 may include any suitable interface controller to provide at least one of processor(s) 305 and/or any suitable device or component in communication with system control module 310 Any suitable interface.

The system control module 310 may include a memory controller module 330 to provide an interface to the system memory 315 . The memory controller module 330 may be a hardware module, a software module and/or a firmware module.

System memory 315 may be used, for example, to load and store data and/or instructions for system 300 . For one embodiment, system memory 315 may include any suitable volatile memory, such as suitable DRAM. In some embodiments, system memory 315 may include Double Data Rate Type Quad Synchronous Dynamic Random Access Memory (DDR4 SDRAM).

For one embodiment, system control module 310 may include one or more input/output (I/O) controllers to provide interfaces to NVM/storage devices 320 and communication interface(s) 325 .

For example, NVM/storage 320 may be used to store data and/or instructions. NVM/storage 320 may include any suitable non-volatile memory (e.g., flash memory) and/or may include any suitable non-volatile storage device(s) (e.g., one or more hard drives ( HDD), one or more compact disc (CD) drives, and/or one or more digital versatile disc (DVD) drives).

NVM/storage device 320 may include a storage resource that is physically part of the device on which system 300 is installed, or it may be accessible by the device without necessarily being part of the device. For example, NVM/storage 320 may be accessed over a network via communication interface(s) 325 .

Communication interface(s) 325 may provide an interface for system 300 to communicate over one or more networks and/or with any other suitable device. System 300 may communicate wirelessly with one or more components of a wireless network according to any of one or more wireless network standards and/or protocols.

For one embodiment, at least one of processor(s) 305 may be packaged with logic of one or more controllers of system control module 310 (eg, memory controller module 330 ). For one embodiment, at least one of the processor(s) 305 may be packaged with the logic of one or more controllers of the system control module 310 to form a system-in-package (SiP). For one embodiment, at least one of the processor(s) 305 may be integrated on the same die as the logic of the one or more controllers of the system control module 310 . For one embodiment, at least one of the processor(s) 305 may be integrated on the same die with the logic of the one or more controllers of the system control module 310 to form a system on chip (SoC).

In various embodiments, system 300 may be, but is not limited to, a server, workstation, desktop computing device, or mobile computing device (eg, laptop computing device, handheld computing device, tablet computer, netbook, etc.). In various embodiments, system 300 may have more or fewer components and/or a different architecture. For example, in some embodiments, system 300 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touchscreen display), non-volatile memory ports, multiple antennas, graphics chips, application-specific integrated circuits ( ASIC) and speakers.

It should be noted that the present application can be implemented in software and/or a combination of software and hardware, for example, it can be implemented by using an application specific integrated circuit (ASIC), a general-purpose computer or any other similar hardware devices. In one embodiment, the software program of the present application can be executed by a processor to realize the steps or functions described above. Likewise, the software program (including associated data structures) of the present application can be stored in a computer-readable recording medium such as RAM memory, magnetic or optical drive or floppy disk and the like. In addition, some steps or functions of the present application may be implemented by hardware, for example, as a circuit that cooperates with a processor to execute each step or function.

In addition, a part of the present application can be applied as a computer program product, such as a computer program instruction. When it is executed by a computer, the method and/or technical solution according to the present application can be invoked or provided through the operation of the computer. Those skilled in the art should understand that computer program instructions exist in computer-readable media in forms including but not limited to source files, executable files, installation package files, etc. Limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding post-installation program program. Here, a computer readable medium may be any available computer readable storage medium or communication medium that can be accessed by a computer.

Communication media includes the media whereby communication signals embodying, for example, computer readable instructions, data structures, program modules or other data are transmitted from one system to another. Communication media can include guided transmission media such as cables and wires (e.g., fiber optics, coaxial, etc.) and wireless (unguided transmission) media capable of propagating waves of energy, such as acoustic, electromagnetic, RF, microwave, and infrared . Computer readable instructions, data structures, program modules or other data may be embodied, for example, as a modulated data signal in a wireless medium such as a carrier wave or similar mechanism such as embodied as part of spread spectrum technology. The term "modulated data signal" means a signal that has one or more of its characteristics changed or set in such a manner as to encode information in the signal. Modulation can be analog, digital or mixed modulation techniques.

By way of example, and not limitation, computer-readable storage media may include volatile and nonvolatile, volatile, volatile, or Removable and non-removable media. For example, computer-readable storage media include, but are not limited to, volatile memories such as random access memories (RAM, DRAM, SRAM); and nonvolatile memories such as flash memory, various read-only memories (ROM, PROM, EPROM) , EEPROM), magnetic and ferromagnetic/ferroelectric memory (MRAM, FeRAM); and magnetic and optical storage devices (hard disks, tapes, CDs, DVDs); or other media known now or developed in the future capable of storing data for computer systems Computer readable information/data used.

Here, an embodiment according to the present application includes an apparatus comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, triggering The operation of the device is based on the foregoing methods and/or technical solutions according to multiple embodiments of the present application.

It will be apparent to those skilled in the art that the present application is not limited to the details of the exemplary embodiments described above, but that the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be regarded as exemplary and not restrictive in all points of view, and the scope of the application is defined by the appended claims rather than the foregoing description, and it is intended that the scope of the present application be defined by the appended claims rather than by the foregoing description. All changes within the meaning and range of equivalents of the elements are embraced in this application. Any reference sign in a claim should not be construed as limiting the claim concerned. In addition, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means stated in the device claims may also be realized by one unit or device through software or hardware. The words first, second, etc. are used to denote names and do not imply any particular order.

Claims (19)

1. A method for generating a high dynamic range image, wherein the method comprises: acquiring image information to be processed, wherein the image information includes one or more first objects, and the one or more first objects belong to at least one first object category; generating a first object vector of the image information based on the at least one first object category; inputting the first target vector into a target vector regression model to output a second target vector of the image information, wherein the target vector regression model is a vector regression model corresponding to the target style type to which the image information belongs, The second target vector includes the target exposure of the first object region corresponding to each first object category in the image information in the at least one first object category, and the second target vector is used to determine said target exposure required to synthesize a high dynamic range image of said image information; A high dynamic range image of the image information is generated according to the target exposure of the first object region corresponding to each first object category in the image information. 2. The method of claim 1, wherein the method further comprises: inputting the image information into a style classification model to determine the target style type to which the image information belongs according to the output result; Obtain the target vector regression model corresponding to the target style type. 3. The method of claim 2, wherein the method further comprises: The style classification model is constructed based on multiple pictures and the style type label of each picture. 4. The method according to claim 1, wherein said acquiring image information to be processed, wherein said image information includes one or more first objects, said one or more first objects belonging to at least one first object An object class, including: Obtain image information to be processed; determining one or more first objects present in said image information; At least one first object category to which the one or more first objects belong is determined according to the first object category to which each first object of the one or more first objects belongs. 5. The method according to claim 1, wherein said generating a first object vector of said image information according to said at least one first object category comprises: Determine the assignment of each first component in the first initial vector according to the at least one first object category and the category set, so as to generate a first target vector of the image information, wherein the first initial vector corresponds to the category gather. 6. The method according to claim 5, wherein the class set includes a plurality of second object classes arranged in order, the first initial vector includes a plurality of first components, and the plurality of second object classes The number of is equal to the number of the plurality of first components, based on the arrangement order of the plurality of second object categories, each second object category of the plurality of second object categories is in the first initial vector has its corresponding first component, and the initial assignment of each first component is zero; The assignment of each first component in the first initial vector is determined according to the at least one first object category and the category set, so as to generate the first target vector of the image information, wherein the first initial vector corresponds to the A collection of the above categories, including: If there is a second object category that is the same as the first object category in the category set, reassign the first component corresponding to the second object category in the first initial vector according to the target assignment, so as to generate the The first target vector of image information. 7. The method according to claim 5 or 6, wherein the second target vector comprises a plurality of second components, the number of the plurality of second components is equal to the number of the plurality of second object categories, Based on the arrangement order of the second object categories, each second object category in the plurality of second object categories has its corresponding second component in the second target vector, and the method further includes: For each first object category in at least one first object category in the image information, the assignment of the second component corresponding to the second object category that is the same as the first object category in the second target vector As the target exposure of the first object area corresponding to the first object category in the image information. 8. The method according to claim 1 or 2, wherein the method further comprises: Get multiple pictures of multiple styles; For each style type, construct a vector regression model of the style type according to the first vector and the second vector of a plurality of pictures belonging to the style type, so as to obtain a vector regression corresponding to each style type in the plurality of style types Model. 9. The method of claim 8, wherein the method further comprises: For each of the plurality of pictures belonging to the style type, generating a first vector of the picture according to at least one third object category to which one or more third objects appearing in the picture belong; The second vector of the picture is generated according to the exposure of the third object region corresponding to each third object category in the at least one third object category in the picture. 10. The method according to claim 9, wherein, for each of the plurality of pictures belonging to the style type, according to at least one third object to which one or more third objects appearing in the picture belong Three object categories generate the first vector of this image, including: For each picture in the plurality of pictures belonging to the style type, each of the first initial vectors is determined according to at least one third object category and category set to which one or more third objects appearing in the picture belong. assignment of the first component to generate a first vector of the picture, wherein the first initial vector corresponds to the category set. 11. The method according to claim 10, wherein the class set includes a plurality of second object classes arranged in order, the first initial vector includes a plurality of first components, and the plurality of second object classes The number of is equal to the number of the plurality of first components, so that each second object category has its corresponding first component in the first initial vector, and the initial assignment of each first component is zero; For each of the multiple pictures belonging to the style type, the first initial vector is determined according to at least one third object category and category set to which one or more third objects appearing in the picture belong The assignment of each first component in to generate the first vector of the picture, wherein the first initial vector corresponds to the category set, including: If there is a second object category that is the same as the third object category in the category set, reassign the first component corresponding to the second object category in the first initial vector according to the target assignment, so as to generate the picture The first vector of . 12. The method according to claim 9, wherein said generating the first image of the picture according to the exposure degree of the third object area corresponding to each third object category in the picture in the at least one third object category Two vectors, including: determining a third object area corresponding to each of the at least three third object categories in the picture in the picture; calculating the exposure of each third object area to obtain the exposure of the third object area corresponding to each third object category in the picture in the at least one third object category; Determine the assignment of each second component in the second initial vector according to the exposure of the third object area corresponding to each third object category in the picture in the at least one third object category and the category set, so as to generate the picture The second vector of , wherein the second initial vector corresponds to the category set. 13. The method according to claim 12, wherein said determining the third object area corresponding to each of the at least three third object categories in the picture in the picture comprises: determining one or more third objects appearing in the picture and a third object sub-region corresponding to each third object; The third object sub-area corresponding to the third object belonging to the same third object category is used as the third object area corresponding to the third object category. 14. The method according to claim 12, wherein said calculating the exposure degree of each third object region, so as to obtain the corresponding exposure of each third object category in the picture in the at least one third object category Exposure of third object areas, including: Calculate the exposure of the third object area according to the pixel information of each third object area, so as to obtain the exposure of the third object area corresponding to each third object category in the picture in the at least one third object category Spend. 15. The method according to claim 12, wherein the category set comprises a plurality of second object categories arranged in order, the second initial vector comprises a plurality of second components, and the initial assignment of each second component is zero, the number of the plurality of second object categories is equal to the number of the plurality of second components, so that each second object category has its corresponding second component in the second initial vector, The assignment of each second component in the second initial vector is determined according to the exposure of the third object area corresponding to each third object category in the picture in the at least one third object category and the category set, so as to generate The second vector of the picture, wherein the second initial vector corresponds to the set of categories, includes: If there is a second object category identical to the third object category in the category set, place the second object category in the second initial vector according to the exposure of the third object area corresponding to the third object category The corresponding second component in is reassigned to generate the second vector of the picture. 16. The method according to any one of claims 5 to 15, wherein the acquisition process of the category set comprises: determining a second object included in each of the multiple pictures to obtain multiple second objects; Classifying the plurality of second objects according to the second object category to which each second object belongs to obtain a plurality of second object categories, wherein each second object category includes one or more second objects; sort the plurality of second object categories in descending order according to the quantity of second objects included in each second object category, to generate the category set, wherein the category set includes a plurality of second objects arranged in order category. 17. The method according to claim 1, wherein the method generates the high dynamic range of the image information according to the target exposure of the first object region corresponding to each first object category in the image information The image, previously also included: Perform exposure sampling on the image information based on different exposure parameters to obtain at least two backup image information; for each of the backup image information, calculate the values of the first object categories in the backup image information in the backup image information the corresponding exposure of the first object area, so as to obtain at least two exposures corresponding to each first object area; The generating the high dynamic range image of the image information according to the target exposure of the first object area corresponding to each first object category in the image information includes: For each of the first object areas, according to the target exposure corresponding to the first object area, determine one of the one or more exposures corresponding to the first object area that has the smallest difference with the target exposure Exposure: generating a high dynamic range image of the image information according to the pixel information of the first object area in the spare image information corresponding to the exposure. 18. An apparatus for generating a high dynamic range image, wherein the apparatus comprises: processor; and A memory arranged to store computer-executable instructions which, when executed, cause the processor to perform the operations of the method as claimed in any one of claims 1 to 17. 19. A computer readable medium storing instructions which, when executed, cause a system to perform the operations of the method of any one of claims 1 to 17.

Images