Disclosure of Invention
Aiming at the defects in the prior art, one or more embodiments of the disclosure provide an image classifier construction and image recognition method and system based on prototype network learning with few samples, and the trained classifier effectively realizes better measurement of sample similarity by using a small amount of samples and extracts more migratory features, thereby achieving better effect when recognizing class samples not encountered in the training process.
According to an aspect of one or more embodiments of the present disclosure, there is provided an image classifier construction method based on prototype network sample-less learning.
An image classifier construction method based on prototype network few-sample learning comprises the following steps:
receiving an image training set, and constructing a sample set and a query set from the image training set based on an epsilon mode;
extracting n-dimensional features of all sample sets and query sets;
calculating the prototype of each type of sample in the sample set according to Bregman divergence;
calculating the distance from each sample in the query set to various sample prototypes;
calculating the possibility of each sample in the query set to various samples;
calculating the loss of various samples in the query set, and calculating the average loss of all samples;
and optimizing a model for modeling the distance distribution among the samples by using a prototype network based on measurement by using a sample loss function to obtain the image classifier.
Further, in the method, the Euclidean distance or the cosine distance is adopted to calculate the distance from each sample in the query set to various types of sample prototypes.
Further, in the method, the computing of the likelihood of each sample in the query set into classes of samples, class k sample x
qTo class k sample prototype p
kIs that
Further, in the method, the loss of the kth type sample in the loss of each type of sample in the calculation query set is:
according to an aspect of one or more embodiments of the present disclosure, there is provided a computer-readable storage medium.
A computer-readable storage medium, wherein a plurality of instructions are stored, the instructions are suitable for being loaded by a processor of a terminal device and executing the image classifier construction method based on prototype network few-sample learning.
According to an aspect of one or more embodiments of the present disclosure, there is provided a terminal device.
A terminal device comprising a processor and a computer-readable storage medium, the processor being configured to implement instructions; the computer readable storage medium is used for storing a plurality of instructions which are suitable for being loaded by a processor and executing the image classifier construction method based on prototype network few-sample learning.
According to an aspect of one or more embodiments of the present disclosure, there is provided an image classifier building apparatus based on prototype network sample-less learning.
An image classifier construction device based on prototype network few-sample learning, based on the image classifier construction method based on prototype network few-sample learning, comprises:
the training set constructing module is configured to receive an image training set, and construct a sample set and a query set from the image training set based on an epsilon mode;
a feature extraction module configured to extract n-dimensional features of all sample sets and query sets;
a sample prototype calculation module configured to calculate a prototype for each type of sample in the sample set according to Bregman divergence;
the distance calculation module is configured to calculate the distance from each sample in the query set to various types of sample prototypes;
a likelihood calculation module configured to calculate a likelihood of each sample in the query set to each type of sample;
the sample loss calculation module is configured to calculate the loss of various samples in the query set and calculate the average loss of all the samples;
and the model optimization module is configured to optimize the model for modeling the distance distribution among the samples by using the prototype network based on the measurement by adopting the sample loss function to obtain the image classifier.
According to an aspect of one or more embodiments of the present disclosure, there is provided an image recognition method based on prototype network sample-less learning.
An image recognition method based on prototype network few-sample learning comprises the following steps:
receiving an image to be identified;
and determining the type of the image to be identified according to the image classifier obtained by the image classifier construction method based on prototype network less sample learning to obtain an image identification result.
According to an aspect of one or more embodiments of the present disclosure, there is provided a computer-readable storage medium.
A computer-readable storage medium having stored therein a plurality of instructions adapted to be loaded by a processor of a terminal device and to execute a prototype-network-based low-sample learning image recognition method.
According to an aspect of one or more embodiments of the present disclosure, there is provided a terminal device.
A terminal device comprising a processor and a computer-readable storage medium, the processor being configured to implement instructions; the computer readable storage medium is used for storing a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the image recognition method based on prototype network few-sample learning.
According to an aspect of one or more embodiments of the present disclosure, there is provided an image recognition apparatus based on prototype network sample-less learning.
An image recognition device based on prototype network learning with less samples, based on the image recognition method based on prototype network learning with less samples, comprising:
an image receiving module configured to receive an image to be recognized;
and the image identification module is configured to determine the type of the image to be identified according to the image classifier obtained by the image classifier construction method based on prototype network learning with few samples, so as to obtain an image identification result.
The beneficial effect of this disclosure:
according to the image classifier construction and image recognition method and system based on prototype network learning with few samples, the similar sample approach and the heterogeneous sample distance are considered in the loss function, the common characteristics of different samples can be measured in the training process based on the epi-sode structure, the characteristics with mobility can be extracted, and the better classification result and the better image recognition result can be achieved.
The specific implementation mode is as follows:
technical solutions in one or more embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in one or more embodiments of the present disclosure, and it is to be understood that the described embodiments are only a part of the embodiments of the present disclosure, and not all of the embodiments. All other embodiments, which can be derived by one of ordinary skill in the art from one or more embodiments of the disclosure without making any creative effort, shall fall within the scope of protection of the disclosure.
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the disclosure. Unless defined otherwise, all technical and scientific terms used have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
It is noted that the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of methods and systems according to various embodiments of the present disclosure. It should be noted that each block in the flowchart or block diagrams may represent a module, a segment, or a portion of code, which may comprise one or more executable instructions for implementing the logical function specified in the respective embodiment. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Without conflict, the embodiments and features of the embodiments in the present disclosure may be combined with each other, and the present disclosure will be further described with reference to the drawings and the embodiments.
Example one
According to an aspect of one or more embodiments of the present disclosure, there is provided an image classifier construction method based on prototype network sample-less learning.
As shown in fig. 1, an image classifier construction method based on prototype network few-sample learning includes:
step S1: receiving an image training set, and constructing a sample set and a query set from the image training set based on an epsilon mode;
step S2: extracting n-dimensional features of all sample sets and query sets;
step S3: calculating the prototype of each type of sample in the sample set according to Bregman divergence;
step S4: calculating the distance from each sample in the query set to various sample prototypes;
step S5: calculating the possibility of each sample in the query set to various samples;
step S6: calculating the loss of various samples in the query set, and calculating the average loss of all samples;
step S7: and optimizing a model for modeling the distance distribution among the samples by using a prototype network based on measurement by using a sample loss function to obtain the image classifier.
In step S1 of one or more embodiments of the present disclosure, a sample set and a query set are constructed for a training set based on an epadiode manner.
Randomly selecting N in a training setcClass samples are used for the epsilon, and N is randomly selected for each class samplesForming sample set by one sample, and randomly selecting N from the rest samplesqOne sample constitutes a query set. In this embodiment, Nc=5,Ns=1,Nq=19。
In one or more embodiments of the present disclosure, in step S2, features of all sample sets and query sets are extracted. The method is characterized in that sample set and query set are output through a fully-connected layer of a model built in advance, and 64-dimensional characteristics are output.
In one or more embodiments of the present disclosure, in step S3, a prototype is calculated for each type of sample in sample set. Prototype of class k samples according to Bregman divergence
Wherein N is
sIs the number of samples selected per category, S
kIs sample set, f (x)
i) Is a feature of sample set.
In step S4 of one or more embodiments of the present disclosure, each sample x in query set is calculatedqDistance d (f (x)) to prototype of each type of sampleq),pk) The euclidean distance or the cosine distance may be used.
The distance calculation in step S4 may use a common distance calculation, such as using euclidean distance or cosine distance to calculate the distance from each sample in the query set to the prototype of each type of sample.
In step S5 of one or more embodiments of the present disclosure, each sample x in query set is calculated
qTo the possibility of various types of sample prototypes. Class k sample x
qPossibility of belonging to class k samples
In one or more embodiments of the present disclosure, step S6, the loss of the kth type sample in the query set is calculated.
Calculating the loss of all samples in the query set
In one or more embodiments of the present disclosure, step S7, the model is optimized according to a loss function. The model was optimized using a random gradient descent.
The pseudo code for a particular algorithm is shown in table 1.
TABLE 1
Example two
According to an aspect of one or more embodiments of the present disclosure, there is provided a computer-readable storage medium.
A computer-readable storage medium, wherein a plurality of instructions are stored, the instructions are suitable for being loaded by a processor of a terminal device and executing the image classifier construction method based on prototype network few-sample learning.
EXAMPLE III
According to an aspect of one or more embodiments of the present disclosure, there is provided a terminal device.
A terminal device comprising a processor and a computer-readable storage medium, the processor being configured to implement instructions; the computer readable storage medium is used for storing a plurality of instructions which are suitable for being loaded by a processor and executing the image classifier construction method based on prototype network few-sample learning.
These computer-executable instructions, when executed in a device, cause the device to perform methods or processes described in accordance with various embodiments of the present disclosure.
In the present embodiments, a computer program product may include a computer-readable storage medium having computer-readable program instructions embodied thereon for performing various aspects of the present disclosure. The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through electrical wires.
The computer-readable program instructions described in this disclosure may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device over a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.
The computer program instructions for carrying out operations of the present disclosure may be assembly instructions, Instruction Set Architecture (ISA) instructions, machine related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, the electronic circuitry can execute computer-readable program instructions to implement aspects of the present disclosure by utilizing state information of the computer-readable program instructions to personalize the electronic circuitry, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA).
Example four
According to an aspect of one or more embodiments of the present disclosure, there is provided an image classifier building apparatus based on prototype network sample-less learning.
An image classifier construction device based on prototype network few-sample learning, based on the image classifier construction method based on prototype network few-sample learning, comprises:
the training set constructing module is configured to receive an image training set, and construct a sample set and a query set from the image training set based on an epsilon mode;
a feature extraction module configured to extract n-dimensional features of all sample sets and query sets;
a sample prototype calculation module configured to calculate a prototype for each type of sample in the sample set according to Bregman divergence;
the distance calculation module is configured to calculate the distance from each sample in the query set to various types of sample prototypes;
a likelihood calculation module configured to calculate a likelihood of each sample in the query set to each type of sample;
the sample loss calculation module is configured to calculate the loss of various samples in the query set and calculate the average loss of all the samples;
and the model optimization module is configured to optimize the model for modeling the distance distribution among the samples by using the prototype network based on the measurement by adopting the sample loss function to obtain the image classifier.
It should be noted that although several modules or sub-modules of the device are mentioned in the above detailed description, such division is merely exemplary and not mandatory. Indeed, the features and functionality of two or more of the modules described above may be embodied in one module in accordance with embodiments of the present disclosure. Conversely, the features and functions of one module described above may be further divided into embodiments by a plurality of modules.
EXAMPLE five
According to an aspect of one or more embodiments of the present disclosure, there is provided an image recognition method based on prototype network sample-less learning.
As shown in fig. 3, an image recognition method based on prototype network learning with few samples includes:
receiving an image to be identified;
and determining the type of the image to be identified according to the image classifier obtained by the image classifier construction method based on prototype network less sample learning to obtain an image identification result.
EXAMPLE six
According to an aspect of one or more embodiments of the present disclosure, there is provided a computer-readable storage medium.
A computer-readable storage medium having stored therein a plurality of instructions adapted to be loaded by a processor of a terminal device and to execute a prototype-network-based low-sample learning image recognition method.
EXAMPLE seven
According to an aspect of one or more embodiments of the present disclosure, there is provided a terminal device.
A terminal device comprising a processor and a computer-readable storage medium, the processor being configured to implement instructions; the computer readable storage medium is used for storing a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the image recognition method based on prototype network few-sample learning.
Example eight
According to an aspect of one or more embodiments of the present disclosure, there is provided an image recognition apparatus based on prototype network sample-less learning.
An image recognition device based on prototype network learning with less samples, based on the image recognition method based on prototype network learning with less samples, comprising:
an image receiving module configured to receive an image to be recognized;
and the image identification module is configured to determine the type of the image to be identified according to the image classifier obtained by the image classifier construction method based on prototype network learning with few samples, so as to obtain an image identification result.
It should be noted that although several modules or sub-modules of the device are mentioned in the above detailed description, such division is merely exemplary and not mandatory. Indeed, the features and functionality of two or more of the modules described above may be embodied in one module in accordance with embodiments of the present disclosure. Conversely, the features and functions of one module described above may be further divided into embodiments by a plurality of modules.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application. Thus, the present disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.