WO2024179575A1 - 一种数据处理方法、设备以及计算机可读存储介质 - Google Patents

一种数据处理方法、设备以及计算机可读存储介质 Download PDF

Info

Publication number
WO2024179575A1
WO2024179575A1 PCT/CN2024/079587 CN2024079587W WO2024179575A1 WO 2024179575 A1 WO2024179575 A1 WO 2024179575A1 CN 2024079587 W CN2024079587 W CN 2024079587W WO 2024179575 A1 WO2024179575 A1 WO 2024179575A1
Authority
WO
WIPO (PCT)
Prior art keywords
sample image
image
category
probability
sample
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2024/079587
Other languages
English (en)
French (fr)
Inventor
尹邦杰
姚太平
张克越
李博
丁守鸿
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology (Shenzhen) Co Ltd
Original Assignee
Tencent Technology (Shenzhen) Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology (Shenzhen) Co Ltd filed Critical Tencent Technology (Shenzhen) Co Ltd
Priority to EP24763255.7A priority Critical patent/EP4614456A4/en
Publication of WO2024179575A1 publication Critical patent/WO2024179575A1/zh
Priority to US19/072,375 priority patent/US20250200954A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/776—Validation; Performance evaluation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/778—Active pattern-learning, e.g. online learning of image or video features

Definitions

  • the present application relates to the field of Internet technology, and in particular to a data processing method, device, and computer-readable storage medium.
  • the mainstream image recognition method is mainly based on the machine learning method, that is, an image recognition initial model is trained through a training sample set to obtain an image recognition model.
  • the training data can determine the category label recognition accuracy and recognition generalization of the image recognition model.
  • the enhancement method for training data is to directly superimpose randomly generated noise on the training sample image.
  • the randomly generated noise and the training sample image are independent data from each other, so the noise features in the training sample image superimposed with random noise are independent of the features of the image itself.
  • the image recognition model can easily distinguish between the noise features and the features of the image itself. Therefore, the training samples amplified by the prior art can only expand the number of training data, but cannot increase the diversity of training data. Therefore, after the image recognition model is trained again with the training samples amplified by the prior art, the category label recognition accuracy and recognition generalization of the trained image recognition model cannot be improved.
  • the embodiments of the present application provide a data processing method, device, and computer-readable storage medium, which can improve the accuracy and generalization of category label recognition.
  • An embodiment of the present application provides a data processing method, including:
  • a sample image sets and an image recognition model where A is a positive integer greater than 1, the A sample image sets respectively correspond to one of A different category labels, the image recognition model is pre-trained based on the A sample image sets, and the A category labels include error category labels;
  • the first probability vector includes A probability elements, each of the A probability elements refers to one of the A category labels, and one of the probability elements is used to represent the probability that the sampling sample image belongs to the referred category label;
  • the A probability elements in the first probability vector obtain a first probability element referring to the wrong category label, and adjust the sampled sample image according to the first probability element to obtain an adversarial sample image corresponding to the sampled sample image, and the category prediction result obtained by performing category prediction on the adversarial sample image by the image recognition model is the wrong category label.
  • An embodiment of the present application provides a data processing method, including:
  • the image recognition optimization model is obtained by continuing to optimize the image recognition model according to the mixed sample training set;
  • the mixed sample training set includes A sample image sets, A category labels, correct category labels and adversarial sample images;
  • the A sample image sets correspond to one of A different category labels respectively, and the A sample image sets include collected sample images;
  • the correct category label belongs to the A category labels;
  • A is a positive integer greater than 1;
  • the image recognition model is obtained by pre-training the A sample image sets;
  • the adversarial sample image is obtained by referring to the wrong category label in the A probability elements in the first probability vector
  • the first probability vector is obtained by adjusting the sampled sample image by the first probability element of the signature, the first probability vector is generated by the image recognition model for the sampled sample image, the wrong category label is different from the category label corresponding to the sample image set to which the sampled sample image belongs, and the correct category label is the category label corresponding to the sample image set to which the sampled sample image belongs;
  • the first probability vector includes A probability
  • the category label corresponding to the maximum probability element in the target probability vector is determined as the category prediction result of the target image.
  • An embodiment of the present application provides a data processing device, including:
  • a data acquisition module used to acquire A sample image sets and an image recognition model, where A is a positive integer greater than 1, the A sample image sets respectively correspond to one of A different category labels, the image recognition model is pre-trained based on the A sample image sets, and the A category labels include error category labels;
  • a first input module is used to obtain a sampled sample image from the A sample image sets, and input the sampled sample image and the erroneous category label into the image recognition model to generate a first probability vector of the sampled sample image for the A category labels, wherein the erroneous category label is different from the category label corresponding to the sample image set to which the sampled sample image belongs;
  • the first probability vector includes A probability elements, each of the A probability elements refers to one of the A category labels, and one of the probability elements is used to represent the probability that the sampled sample image belongs to the referred category label;
  • the first adjustment module is used to obtain a first probability element referring to the wrong category label from the A probability elements in the first probability vector, and adjust the sampled sample image according to the first probability element to obtain an adversarial sample image corresponding to the sampled sample image, wherein the category prediction result obtained by the image recognition model performing category prediction on the adversarial sample image is the wrong category label.
  • the first adjustment module is specifically used to generate a negative probability element corresponding to the first probability element, sum the negative probability element and the maximum probability element in the first probability vector, obtain the label loss value of the sampling sample image for the wrong category label, and adjust the sampling sample image according to the label loss value to obtain the adversarial sample image corresponding to the sampling sample image.
  • the first adjustment module is used to adjust the sample image according to the label loss value to obtain the adversarial sample image corresponding to the sample image. It is specifically used to generate an initial gradient value for the sample image according to the label loss value, obtain the numerical sign of the initial gradient value, generate a symbol unit value corresponding to the initial gradient value according to the numerical sign, multiply the initial learning rate and the symbol unit value to obtain a first value to be cropped, crop the first value to be cropped through a first cropping interval generated by the attack intensity to obtain a gradient value for adjusting the sample image, perform difference processing on the sample image and the gradient value to obtain a second value to be cropped, crop the second value to be cropped through a second cropping interval generated by the pixel level, and obtain the adversarial sample image corresponding to the sample image.
  • the first adjustment module includes:
  • a first adjustment unit configured to adjust the sample image according to the first probability element to obtain an initial adversarial sample image corresponding to the sample image
  • a first generating unit configured to input the erroneous category label and the initial adversarial sample image into the image recognition model, and generate a second probability vector of the initial adversarial sample image for A category labels through the image recognition model;
  • a first acquisition unit configured to acquire a second probability element for the error category label from the A probability elements included in the second probability vector
  • the first adjustment unit is further configured to continue adjusting the initial adversarial sample image according to the second probability element if the second probability element does not satisfy the boundary constraint condition or the category prediction result obtained by performing category prediction on the adversarial sample image is not the wrong category label;
  • the first acquisition unit is further used to determine the initial adversarial sample image as the adversarial sample image corresponding to the sampling sample image if the second probability element satisfies the boundary constraint condition and the category prediction result obtained by performing category prediction on the adversarial sample image is the wrong category label; when the category prediction result is the wrong category label, the second probability element is the maximum value of the A probability elements contained in the second probability vector.
  • the first acquisition unit is further used to compare the second probability element with a probability constraint value; the probability constraint value is the reciprocal of A;
  • the first acquisition unit is further configured to determine that the second probability element does not satisfy a boundary constraint condition if the second probability element is less than the probability constraint value;
  • the first acquisition unit is further configured to determine that the second probability element satisfies a boundary constraint condition if the second probability element is equal to or greater than the probability constraint value.
  • the data processing device further includes:
  • a second input module is used to input the A sample image sets, the A category labels, the correct category label and the adversarial sample image into the image recognition model;
  • the correct category label refers to the category label corresponding to the sample image set to which the sample image belongs;
  • the second input module is further used to generate, through the image recognition model, first prediction categories corresponding to the A sample image sets and second prediction categories corresponding to the adversarial sample images;
  • the second adjustment module is used to generate a first loss value according to the first predicted category and the A category labels, and
  • the image recognition optimization model is obtained by adjusting the parameters in the image recognition model according to the first loss value and the second loss value; and the image recognition optimization model performs category prediction on the adversarial sample image and obtains the correct category label.
  • the A sample image sets include a sample image set Cb; b is a positive integer and b is less than or equal to A; the first input module includes:
  • a first determining unit is used to determine a sampling ratio, and perform sampling processing on sample images in the sample image set Cb according to the sampling ratio to obtain sampled sample images;
  • the second acquisition unit is used to acquire a category label different from the category label of the sample image set Cb from among the A category labels, and determine the acquired category label as an erroneous category label.
  • the data acquisition module includes:
  • a third acquisition unit is used to acquire A sample image sets, and input the A category labels and the A sample image sets into the image recognition initial model respectively;
  • a second generating unit is used to generate predicted initial categories corresponding to the A sample image sets respectively through an image recognition initial model
  • a second determining unit configured to determine, according to the predicted initial category and the A category labels, category loss values corresponding to the A sample image sets respectively;
  • a third determining unit is used to determine a total loss value corresponding to the image recognition initial model according to the category loss values corresponding to the A sample image sets respectively;
  • the second adjustment unit is used to adjust the parameters in the initial image recognition model according to the total loss value to obtain an image recognition model;
  • the category prediction result obtained by the image recognition model for the sample image set Cb is the category label of the sample image set Cb;
  • the A sample image sets include the sample image set Cb;
  • b is a positive integer and b is less than or equal to A.
  • the third acquisition unit includes:
  • the image acquisition subunit is used to acquire A original image sets, and input the A original image sets into the object detection model; the A original image sets correspond to one of A different category information respectively; the A original image sets include an original image set Db; the original image set Db includes an original image Ef; f is a positive integer, and f is less than or equal to the total number of original images in the original image set Db;
  • the second determination subunit is used to determine the region coordinates of the key region in the original image Ef through the object detection model, and generate a sample image to be labeled corresponding to the original image Ef according to the region coordinates;
  • a third determining subunit is used to determine the sample image to be labeled corresponding to each original image in the original image set Db as the sample image set to be labeled;
  • the fourth determining subunit is used to generate a category label Hb according to the category information corresponding to the original image set Db, and determine the set of sample images to be labeled with the category label Hb as the sample image set Cb.
  • the second determining subunit includes:
  • the third processing subunit is used to generate an initial detection frame including the key area according to the area coordinates, and to expand the initial detection frame to obtain a detection frame including the target object; the key area belongs to the target object;
  • a fourth processing subunit is used to obtain, in the detection frame, an image to be scaled including the target object, obtain a first image size, and scale the image to be scaled according to the first image size to obtain an image to be cropped;
  • the fifth processing subunit is used to obtain a second image size, and perform cropping processing on the image to be cropped according to the second image size to obtain a sample image to be labeled corresponding to the original image Ef; the second image size is smaller than the first image size.
  • An embodiment of the present application provides a data processing device, including:
  • An image acquisition module is used to acquire a target image and input the target image into an image recognition optimization model; the image recognition optimization model is obtained by continuing to optimize the image recognition model according to a mixed sample training set; the mixed sample training set includes A sample image sets, A category labels, correct category labels and adversarial sample images; the A sample image sets correspond to one of A different category labels respectively, and the A sample image sets include collected sample images; the correct category label belongs to the A category labels; A is a positive integer greater than 1; the image recognition model is obtained by pre-training the A sample image sets; the adversarial sample image is obtained by referring to the wrong category label in the A probability elements in the first probability vector The first probability element of the wrong category label is adjusted to obtain the sampled sample image, the first probability vector is generated by the image recognition model for the sampled sample image, the wrong category label is different from the category label corresponding to the sample image set to which the sampled sample image belongs, and the correct category label is the category label corresponding to the sample image set to which the sampled sample image belongs; the first probability vector
  • a vector generation module is used to generate a target probability vector for a target image with respect to A category labels through an image recognition optimization model
  • the category determination module is used to determine the category label corresponding to the maximum probability element in the target probability vector as the category prediction result of the target image.
  • the present application provides a computer device, including: a processor, a memory, and a network interface;
  • the above-mentioned processor is connected to the above-mentioned memory and the above-mentioned network interface, wherein the above-mentioned network interface is used to provide a data communication function, the above-mentioned memory is used to store a computer program, and the above-mentioned processor is used to call the above-mentioned computer program so that the computer device executes the method in the embodiment of the present application.
  • an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored.
  • the computer program is suitable for being loaded by a processor and executing the method in the embodiment of the present application.
  • an embodiment of the present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium; a processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the method in the embodiment of the present application.
  • a computer device can obtain A sample image sets and an image recognition model; wherein the A sample image sets respectively correspond to one of A different category labels, and the image recognition model is pre-trained based on the A sample image sets, so the image recognition model can accurately determine the category labels of the sample images in the A sample image sets; further, the computer device obtains a sampled sample image from the A sample image sets, and inputs an erroneous category label and the sampled sample image into the image recognition model, wherein the erroneous category label belongs to the A category labels, and the erroneous category label is different from the category label corresponding to the sample image set to which the sampled sample image belongs; a first probability vector of the sampled sample image for the A category labels can be generated through the image recognition model, and the first probability vector includes A probability elements, and the A probability elements respectively refer to one of the A category labels.
  • a probability element is used to characterize the probability that the sampled sample image belongs to the referred category label; since the image recognition model can accurately determine the category labels of the sample images in the A sample image sets, in the first probability vector, the probability element referring to the correct category label is the largest, or in other words, the probability element referring to the correct category label is much larger than the first probability element referring to the wrong category label, and the correct category label refers to the category label corresponding to the sample image set to which the sampled sample image belongs; further, according to the first probability element, the computer device can adjust the sampled sample image to obtain an adversarial sample image corresponding to the sampled sample image, wherein the category prediction result obtained by the image recognition model for category prediction of the adversarial sample image is the wrong category label, that is, by adjusting the sampled sample image, the image recognition model can incorrectly determine the category label of the adversarial sample image.
  • an embodiment of the present application proposes a method for generating adversarial sample images.
  • adversarial sample images that are incorrectly recognized by an image recognition model can be generated.
  • the adversarial sample images can improve the sample diversity of a training set used to optimize the training of the image recognition model.
  • the optimization training accuracy of the image recognition model can be improved, thereby improving the category label recognition accuracy and recognition generalization of the optimized trained image recognition model.
  • FIG1 is a schematic diagram of a system architecture provided by an embodiment of the present application.
  • FIG2a is a schematic diagram of a data processing scenario provided by an embodiment of the present application.
  • FIG2b is a second schematic diagram of a data processing scenario provided by an embodiment of the present application.
  • FIG3 is a flowchart of a data processing method provided in an embodiment of the present application.
  • FIG4 is a schematic diagram of a classification boundary surface trained by clean sample data provided in an embodiment of the present application.
  • FIG5 is a schematic diagram of a classification boundary surface trained by first mixed sample data provided by an embodiment of the present application.
  • FIG6 is a schematic diagram of a classification boundary surface trained by second mixed sample data provided in an embodiment of the present application.
  • FIG7 is a second flow chart of a data processing method provided in an embodiment of the present application.
  • FIG8 is a third flow chart of a data processing method provided in an embodiment of the present application.
  • FIG9 is a first structural diagram of a data processing device provided in an embodiment of the present application.
  • FIG10 is a second structural diagram of a data processing device provided in an embodiment of the present application.
  • FIG. 11 is a schematic diagram of the structure of a computer device provided in an embodiment of the present application.
  • Artificial Intelligence is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.
  • artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence.
  • Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.
  • Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies.
  • Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operating/interactive systems, mechatronics and other technologies.
  • Artificial intelligence software technologies mainly include computer vision technology, speech processing technology, natural language processing technology, as well as machine learning/deep learning, autonomous driving, smart transportation and other major directions.
  • Computer vision technology is a science that studies how to make machines "see”. To put it more specifically, it refers to machine vision such as using cameras and computers to replace human eyes to identify and measure targets, and further perform graphic processing to make computer processing into images that are more suitable for human eye observation or transmission to instruments for detection.
  • Computer vision technology generally includes image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content/behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and mapping, automatic driving, smart transportation and other technologies.
  • OCR optical character recognition
  • computer vision technology can be used to identify category labels in images (such as people, cats, dogs, or real people, paper people, mold people, etc.).
  • Machine Learning is a multi-disciplinary interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithmic complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance.
  • Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence.
  • Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.
  • the image recognition model and the object detection model are both AI models based on machine learning.
  • the image recognition model can be used to identify and process images, and the object detection model can be used to detect key areas of target objects in images.
  • Figure 1 is a schematic diagram of a system architecture provided by an embodiment of the present application.
  • the system may include a business server 100 and a terminal device cluster, and the terminal device cluster may include: terminal device 200a, terminal device 200b, terminal device 200c, ..., terminal device 200n. It can be understood that the above system may include one or more terminal devices, and the present application does not limit the number of terminal devices.
  • any terminal device in the terminal device cluster may have a communication connection with the business server 100, for example, there is a communication connection between the terminal device 200a and the business server 100, wherein the above communication connection does not limit the connection method, and may be directly or indirectly connected by wired communication, directly or indirectly connected by wireless communication, or by other methods, which are not limited in this application.
  • each terminal device in the terminal device cluster shown in FIG1 may be installed with an application client, and when the application client runs in each terminal device, it may respectively perform data interaction with the business server 100 shown in FIG1, i.e., the above-mentioned communication connection.
  • the application client may be an application client having a target image recognition function, such as a video application, a live broadcast application, a social application, an instant messaging application, a game application, a music application, a shopping application, a novel application, a browser, etc.
  • the application client may be an independent client or an embedded sub-client integrated in a client (e.g., a social client, an educational client, a multimedia client, etc.), which is not limited here.
  • the business server 100 may be a collection of multiple servers including a background server corresponding to the shopping application, a data processing server, etc. Therefore, each terminal device may perform data transmission with the business server 100 through the application client corresponding to the shopping application. For example, each terminal device may upload a target image to the business server 100 through the application client of the shopping application, and then the business server 100 may determine the category label of the target image.
  • the embodiment of the present application can select a terminal device as the target terminal device in the terminal device cluster shown in FIG1, for example, the terminal device 200a is used as the target terminal device.
  • the terminal device 200a can send a model optimization request for the image recognition model to the business server 100 through the application client.
  • the image recognition model refers to a neural network model used to recognize an image and determine the category information of the image.
  • the business server 100 can obtain A sample image sets and an image recognition model, wherein the A sample image sets are multiple sample image sets, and the A sample image sets correspond to different category labels respectively. Therefore, the number of category labels can also be A, that is, the A sample image sets correspond to one of the A different category labels.
  • a category label is used to indicate a category information, so the category information corresponding to the A sample image sets is different from each other.
  • the embodiment of the present application does not limit the category information and can be set according to the actual application scenario.
  • the category information can be object information. In this case, the object information corresponding to the A sample image sets is different from each other.
  • the object information of the first sample image set in the A sample image set is a cat (that is, each sample image in the first sample image set is related to a cat), and the object information of the second sample image set in the A sample image set is a dog (that is, each sample image in the second sample image set is related to a dog).
  • the category information may be object carrier information.
  • the object information corresponding to each of the A sample image sets may be the same or different, but the object carrier information corresponding to each of the A sample image sets is different from each other.
  • the object carrier information of the third sample image set among the A sample image sets is a copied screen
  • the object carrier information of the fourth sample image set among the A sample image sets is a key feature mold
  • the object carrier information of the fifth sample image set among the A sample image sets is a piece of paper.
  • the embodiment of the present application does not limit the way in which the business server 100 obtains A sample image sets and image recognition models, and can be set according to the actual application scenario.
  • the model optimization request sent by the terminal device 200a carries A sample image sets or image recognition models, or carries A sample image sets and image recognition models at the same time, such as obtaining A sample image sets and image recognition models from a database, such as obtaining A sample image sets and image recognition models from a blockchain network.
  • the image recognition model is obtained by pre-training based on A sample image sets. The generation process of the image recognition model is not described here. Please refer to the description of step S101 in the embodiment corresponding to Figure 3 below.
  • the embodiment of the present application takes the sample image set C1 in the A sample image set as an example for description. It can be understood that the processing process of the remaining sample image sets in the A sample image set is the same as the processing process of the sample image set C1 described below.
  • the business server 100 obtains the sampled sample images in the sample image set C1 (can be obtained randomly or by polling), and then obtains the wrong category label in the A category labels.
  • the wrong category label belongs to the category label in the A category labels except the category label of the sample image set C1 (can be called the remaining category label).
  • the category label randomly obtained in the remaining category label can be determined as the wrong category label, for example, the wrong category label is the category label of the sample image set C2.
  • the wrong category label and the sampled sample image can be input into the image recognition model, wherein the wrong category label and the sampled sample image can be respectively input into the image recognition model as two independent data, and there is a mapping relationship between the wrong category label and the sampled sample image; or, the wrong category label can be added to the file header or file footer of the sampled sample image, and then the sampled sample image carrying the wrong category label can be input into the image recognition model.
  • the business server 100 can generate a first probability vector for the sampled sample image for A category labels through the image recognition model; the first probability vector includes A probability elements, each of which refers to one of the A category labels, and one of the probability elements is used to characterize the probability that the sampled sample image belongs to the referred category label.
  • the vector dimension of the first probability vector is equal to A, that is, the elements on a vector dimension (also referred to as probability elements in the embodiment of the present application) can characterize the predicted probability value of a category label, that is, the larger the value of a certain probability element, the greater the possibility that the sampled sample image belongs to the category label referred to by the probability element. Therefore, the category label referred to by the probability element with the largest value in the first probability vector can usually be determined as the category prediction result using the sample image. Therefore, in the first probability vector, the business server 100 can obtain the first probability element that refers to the wrong category label, and further, according to the first probability element, the sampled sample image can be adjusted to obtain the adversarial sample image corresponding to the sampled sample image.
  • the category prediction result obtained by the image recognition model for the adversarial sample image is the wrong category label, that is, the image recognition model cannot accurately identify the adversarial sample image, and then obtains an incorrect category classification.
  • the sample image is an image containing a cat
  • its category label is the label corresponding to the cat.
  • the corresponding adversarial sample image is obtained.
  • the optimizer of the image recognition model can accurately judge that the adversarial sample image is an image containing a cat, but the image recognition model will mistakenly judge that the adversarial sample image is an image containing a dog.
  • the business server 100 can generate adversarial sample images. Subsequently, through A sample image sets, correct category labels and adversarial sample images, the image recognition model can be optimized and trained to obtain an image recognition optimization model.
  • the correct category label refers to the category label of the sample image set C1. It can be understood that when training the image recognition model, the model loss is calculated by the predicted category label and the correct category label of the adversarial sample image. Therefore, the trained image recognition optimization model can accurately identify the adversarial sample image, and then accurately identify that the predicted category result of the adversarial sample image is the correct category label. Therefore, the image recognition optimization model can have a stronger anti-interference ability.
  • the business server 100 may generate a model optimization completion message for the image recognition optimization model, and send the model optimization completion message to the terminal device 200a.
  • the terminal device 200a may obtain the image recognition optimization model.
  • the embodiment of the present application does not limit the manner in which the terminal device 200a obtains the image optimization recognition model, and the terminal device 200a may obtain the image optimization recognition model according to the actual application.
  • the terminal device 200a stores the above image recognition model and A sample image sets locally, and the terminal device 200a has offline computing capabilities, then when the terminal device 200a receives the model optimization instruction for the image recognition model, it can obtain the sample images in the sample image set C1 locally and use the obtained sample images as the sampled sample images.
  • the terminal device 200a inputs the error category label and the sampled sample images into the image recognition model, and the subsequent processing process is the same as the process described above, so it is not repeated here.
  • the embodiment of the present application assigns an incorrect category label to the sampled sample image, and obtains a first probability element referring to the incorrect category label through the image recognition model, and adjusts the sampled sample image through the first probability element to obtain an adversarial sample image that allows the image recognition model to incorrectly predict the category.
  • the embodiment of the present application uses adversarial sample images to amplify and enhance the training samples of A sample image sets (i.e., training data), so it can improve the richness of the sample training set used to optimize the training image recognition model, thereby improving the recognition accuracy of the image recognition optimization model.
  • terminal device 200a can all be blockchain nodes in the blockchain network, and the data described in the full text (such as A sample image sets and image recognition models) can be stored.
  • the storage method can be that the blockchain node generates blocks according to the data and adds the blocks to the blockchain for storage.
  • Blockchain is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism and encryption algorithm. It is mainly used to organize data in chronological order and encrypt it into a ledger to make it impossible to be tampered with or forged. At the same time, it can verify, store and update data.
  • Blockchain is essentially a decentralized database. Each node in the database stores the same blockchain.
  • the blockchain network can divide nodes into core nodes, data nodes and light nodes. Core nodes, data nodes and light nodes together constitute blockchain nodes. Among them, the core node is responsible for the consensus of the entire blockchain network, that is, the core node is the consensus node in the blockchain network.
  • the process of writing transaction data in the blockchain network into the ledger can be that the data node or light node in the blockchain network obtains the transaction data, and passes the transaction data in the blockchain network (that is, the node passes it in the form of a baton) until the consensus node receives the transaction data, and the consensus node then packs the transaction data into a block, executes consensus on the block, and writes the transaction data into the ledger after the consensus is completed.
  • a sample image sets and an image recognition model are used as examples of transaction data.
  • the business server 100 (blockchain node) generates a block based on the transaction data and stores the block in the blockchain network.
  • the blockchain node can obtain the block containing the transaction data in the blockchain network, and further obtain the transaction data in the block.
  • the method provided in the embodiments of the present application can be executed by a computer device, which includes but is not limited to a terminal device or a business server.
  • the business server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud databases, cloud services, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.
  • Terminal devices include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, etc.
  • the terminal device and the business server can be directly or indirectly connected by wire or wireless means, and the embodiments of the present application are not limited here.
  • Figure 2a is a schematic diagram of a data processing scenario provided by an embodiment of the present application.
  • the embodiment of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, audio and video, etc.
  • the embodiment of the present application can be applied to business scenarios such as image search scenarios, image generation scenarios, image recommendation scenarios, image distribution scenarios, and specific business scenarios will not be listed one by one here.
  • the implementation process of the data processing scenario can be carried out in a business server, or in a terminal device, or interactively in a terminal device and a business server, without limitation here.
  • the terminal device can be any terminal device in the terminal device cluster of the embodiment corresponding to Figure 1 above
  • the business server can be the business server 100 of the embodiment corresponding to Figure 1 above.
  • the embodiment of the present application is described by taking the operation in the business server as an example.
  • the business server 100 can obtain A sample image sets 20a and image recognition models 20d, where A is a positive integer greater than 1.
  • A is a positive integer greater than 1.
  • the embodiment of the present application takes A as 4, that is, the number of sample image sets is 4, such as the sample image set 201a, the sample image set 202a, the sample image set 203a, and the sample image set 204a shown in FIG2a (the number of sample images contained in the sample image set 201a, the sample image set 202a, the sample image set 203a, and the sample image set 204a can be the same or different).
  • the category information corresponding to the four sample image sets is different from each other.
  • the category information of the sample image set 201a shown in FIG2a is radish
  • the category information of the sample image set 202a is pepper
  • the category information of the sample image set 203a is pumpkin
  • the category information of the sample image set 204a is onion. Therefore, the business server 100 can set different category labels for the above four sample image sets, respectively, setting the category label of radish to 0, that is, the category label of sample image set 201a is 0, setting the category label of pepper to 1, that is, the category label of sample image set 202a is 1, setting the category label of pumpkin to 2, that is, the category label of sample image set 203a is 2, and setting the category label of onion to 3, that is, the category label of sample image set 204a is 3.
  • the embodiment of the present application is for the purpose of description and understanding, and uses radish, pepper, pumpkin and onion as examples of category information, and in actual application, the category information should be set according to the actual scenario.
  • the embodiment of the present application is for the purpose of description and understanding, and uses 0-3 numerical examples of category labels, and in actual application, the category labels should be set according to the actual scenario.
  • sample image 201b is obtained from sample image set 201a
  • sample image 202b is obtained from sample image set 202a
  • sample image 203b is obtained from sample image set 203a
  • sample image 204b is obtained from sample image set 204a.
  • the embodiment of the present application does not limit the total number of sample images 201b, which may be one or more.
  • the embodiment of the present application does not limit the total number of sample images 202b, sample images 203b, and sample images 204b, which may be one or more.
  • the service server 100 inputs the sample image 201b, the sample image 202b, the sample image 203b and the sample image 204b into the image recognition model 20d, wherein the image recognition model 20d is pre-trained based on the above four sample image sets. It can be understood that the service server 100 uses the image recognition model 20d to process each sample image in the same manner, so the following description is made using a sample image (such as the sample image 201c shown in FIG. 2a) as an example, and the processing of the remaining sample images can refer to the processing of the sample image 201c below.
  • a sample image such as the sample image 201c shown in FIG. 2a
  • the sample image 201c and the wrong category label can be input into the image recognition model 20d.
  • the sample image 201c is an image with the category information of radish, that is, the sample image 201c belongs to the sample image set 201a.
  • the correct category label of the sample image 201c is the category label of radish, that is, 0.
  • FIG2a illustrates the wrong category label of the sample image 201c with the category label of pepper (that is, 1) (the wrong category label can be obtained by randomly selecting from pepper, pumpkin and onion). It should be noted that the wrong category label of the sample image 201c is randomly generated.
  • the business server 100 can generate the first probability vector 201e for the four category labels of the sample image 201c through the image recognition model 20d.
  • the dimension of the first probability vector 201e is the same as the total number of category labels, and the element value on each dimension can represent the predicted probability value of the corresponding category label. Therefore, in Figure 2a, 0.9 on the first dimension of the first probability vector 201e can represent the predicted probability of category label 0, 0.02 on the second dimension can represent the predicted probability of category label 1, and 0.03 on the third dimension can represent the predicted probability of category label 2; 0.05 on the fourth dimension can represent the predicted probability of category label 3.
  • the predicted probabilities on these four dimensions can also be called four probability elements.
  • the image recognition model 20d can determine that the numerical value of the predicted probability referring to category label 0 is the largest, so it can be determined that the category prediction result of the sample image 201c is category label 0, which is different from the wrong category label (i.e. 1) input to the image recognition model 20d.
  • This result is produced because the image recognition model 20d is pre-trained by the sample image set 201a, the sample image set 202a, the sample image set 203a and the sample image set 204a in Figure 2a, and the sample sample image 201c comes from the sample image set 201a, so the image recognition model 20d recognizes that the category prediction result of the sample sample image 201c is category label 0.
  • the service server 100 can obtain the first probability element referring to the wrong category label (category label 1 as shown in FIG2a ) in the first probability vector 201e through the image recognition model 20d, such as 0.02 as shown in FIG2a .
  • the service server 100 can adjust the sample image 201c according to the first probability element to obtain the adversarial sample image corresponding to the sample image 201c.
  • the sampling sample image 201c is adjusted to obtain the specific process of the adversarial sample image corresponding to the sampling sample image 201c, which may include: the business server 100 may adjust the sampling sample image 201c according to the first probability element to obtain the initial adversarial sample image corresponding to the sampling sample image 201c.
  • Figure 2b is a schematic diagram of a data processing scenario provided by an embodiment of the present application.
  • the business server 100 generates a negative probability element corresponding to the first probability element (0.02 as shown in Figure 2a), that is, -0.02 in Figure 2b, and sums the negative probability element and the maximum probability element in the first probability vector 201e (0.9 as shown in Figure 2a and Figure 2b) to obtain the label loss value of the sampling sample image 201c for the wrong category label, that is, 0.88 in Figure 2b. Further, the business server 100 adjusts the sampling sample image 201c according to the label loss value to obtain the initial adversarial sample image 202c corresponding to the sampling sample image 201c.
  • step S103 The specific process of adjusting the sample image 201c according to the label loss value to obtain the initial adversarial sample image 202c corresponding to the sample image 201c is not described in detail in the embodiment of the present application. Please refer to the description of step S103 in the embodiment corresponding to Figure 3 below.
  • the business server 100 inputs the wrong category label (category label 1 as shown in Figures 2a and 2b) and the initial adversarial sample image 202c into the image recognition model 20d.
  • the business server 100 can generate a second probability vector 202e for the initial adversarial sample image 202c for the above four category labels through the image recognition model 20d.
  • the meaning of the second probability vector 202e is the same as that of the first probability vector 201e, so the meaning of the second probability vector 202e is not explained here. Please refer to the description of the meaning of the first probability vector 201e above.
  • the service server 100 obtains a second probability element indicating an error category label (i.e., category label 1 as shown in FIG. 2a and FIG. 2b ) in the second probability vector 202e, such as 0.08 as shown in FIG. 2b .
  • the service server 100 can, based on the second probability element, Determine the adversarial sample image corresponding to the sample image 201c. That is, the business server 100 can analyze whether the second probability element is the probability element with the largest value in the second probability vector 202e.
  • the initial adversarial sample image 202c can be determined as the adversarial sample image corresponding to the sample image 201c (that is, at this time, the image recognition model 20d can mistakenly identify the category prediction result of the adversarial sample image as the wrong category label 1); if it is analyzed that the second probability element is not the probability element with the largest value in the second probability vector 202e, the initial adversarial sample image 202c will be further adjusted.
  • the purpose of the adjustment is to increase the probability element referring to the wrong category label in the third probability vector output by the image recognition model 20d for the adjusted initial adversarial sample image, until the image recognition model 20d can mistakenly identify the adversarial sample image after multiple rounds of adjustment as the wrong category label 1.
  • the category prediction result of the image recognition model 20d for the adversarial sample image is the wrong category label 1, that is, the adversarial sample image is still an image containing radish, but the image recognition model 20d will mistakenly identify the category prediction result of the adversarial sample image as the category label 1 of pepper.
  • the business server 100 can continue to optimize the training of the image recognition model 20d through the correct category label (that is, the category label 0 of radish) and the adversarial sample image to obtain an optimized image recognition model.
  • the image optimization recognition model that is, the optimized trained image recognition model
  • the embodiment of the present application proposes a method for generating adversarial sample images, through which adversarial sample images that are incorrectly recognized by an image recognition model can be generated.
  • adversarial sample images the sample diversity of the training set used to optimize the training of the image recognition model can be improved.
  • the optimization training accuracy of the image recognition model can be improved, and then the category label recognition accuracy and recognition generalization of the optimized trained image recognition model can be improved.
  • the adversarial sample image is an image with strong disturbance (i.e., an image that will cause the image recognition model to incorrectly recognize it), and the image optimization recognition model can overcome the strong disturbance, i.e., the image optimization recognition model can accurately identify the category prediction result of the adversarial sample image as the correct category label (i.e., category label 0) through a stronger anti-interference ability.
  • Figure 3 is a flow chart of a data processing method provided in an embodiment of the present application.
  • the data processing method can be executed by a business server (for example, the business server 100 shown in Figure 1 above), or by a terminal device (for example, the terminal device 200a shown in Figure 1 above), or by a business server and a terminal device interacting with each other.
  • a business server for example, the business server 100 shown in Figure 1 above
  • a terminal device for example, the terminal device 200a shown in Figure 1 above
  • the embodiment of the present application is described by taking the method being executed by a business server as an example.
  • the data processing method can at least include the following steps S101-S103.
  • Step S101 obtain A sample image sets and an image recognition model, where A is a positive integer greater than 1, and the A sample image sets respectively correspond to one of A different category labels.
  • the image recognition model is pre-trained based on the A sample image sets, and the A category labels include error category labels.
  • the business server obtains A sample image sets, and the category information corresponding to the A sample image sets is different.
  • the category information of a sample image set is radish (that is, each sample image in the sample image set is related to radish), and the category information of another sample image set is pepper (that is, each sample image in the sample image set is related to pepper). Therefore, the business server can set different category labels for each sample image set, such as setting the category label of radish to 0, that is, the category label of the sample image set with the category information of radish is 0; and setting the category label of pepper to 1, that is, the category label of the sample image set with the category information of pepper is 1.
  • the business server inputs A sample image sets into the image recognition initial model respectively; generates the predicted initial category corresponding to each sample image set through the image recognition initial model; determines the category loss value corresponding to each sample image set according to the predicted initial category corresponding to each sample image set and the category label corresponding to each sample image set; determines the total loss value corresponding to the image recognition initial model according to the category loss values corresponding to the A sample image sets; adjusts the parameters in the image recognition initial model according to the total loss value to obtain the image recognition model; the image recognition model has the ability to accurately identify the category prediction results of the A sample image sets, such as the image recognition model can accurately identify the category prediction result of the sample image set with category information of radish as the category label of 0 (the category prediction result can also be understood as the category prediction result for each sample image in the sample image set).
  • the embodiment of the present application does not limit the number of sample images in the sample image set, which can be one or more. It can be understood that the category information corresponding to the sample images in the same sample image set is the same, so the category labels corresponding to the sample images in the same sample image set are the same. The embodiment of the present application also does not limit the number of sample image sets, which can be two or more.
  • the category label in the embodiment of the present application refers to a label used to identify category information.
  • the embodiment of the present application does not limit the category information and can be set according to the actual application scenario.
  • the category information can be object information.
  • each sample image set contains different object information, such as cats, dogs, cars, etc.
  • the category information can be object carrier information.
  • each sample image contains different object carrier information, such as mask objects, mold objects, photo objects, paper objects, real objects, etc.
  • the embodiment of the present application does not limit the model type of the image recognition initial model, and can be composed of any one or more neural networks, such as convolutional neural networks (CNN), residual networks (ResNet), Convolutional neural networks with extended channel number learning mechanism (Wide Residual Networks, Wide-ResNet), High-Resolution Net (HRNet), etc.
  • CNN convolutional neural networks
  • ResNet residual networks
  • HRNet High-Resolution Net
  • the specific process of the business server training the image recognition initial model can be: inputting A sample image sets into the image recognition initial model respectively.
  • the image recognition initial model processes each sample image in the A sample image sets in the same way, so the following description is taken as an example to describe the first sample image carrying the first category label, and the processing process of the remaining sample images is described below.
  • the first sample image is any sample image in the A sample image sets. If the first sample image belongs to the sample image set C1, the first category label is the category label corresponding to the sample image set C1; if the first sample image belongs to the sample image set C2, the first category label is the category label corresponding to the sample image set C2; other situations are as described above and will not be repeated one by one.
  • the business server can obtain the first initial probability vector corresponding to the first sample image through the image recognition initial model.
  • the dimension of the first initial probability vector is the same as A (that is, one dimension corresponds to one category label).
  • the element value on each dimension in the first initial probability vector (also referred to as a probability element) represents the predicted probability of the corresponding category label, and the sum of the element values on each dimension is 1.
  • the category label corresponding to the largest element value in the first initial probability vector that is, the predicted initial category corresponding to the first sample image
  • the business server can generate a category loss value corresponding to the first sample image based on the predicted initial category corresponding to the first sample image and the first category label.
  • the business server can obtain the category loss value corresponding to each sample image in the A sample image set, sum the obtained category loss values, and obtain the total loss value corresponding to the image recognition initial model.
  • the business server adjusts the parameters in the image recognition initial model according to the total loss value to obtain a pre-trained image recognition model, which can accurately identify the category prediction results of each sample image in the A sample image set, such as the image recognition model can accurately identify the category prediction result of the first sample image as the first category label.
  • the embodiment of the present application does not limit the adjustment method of the parameters in the image recognition initial model, and can be set according to the actual application scenario.
  • the embodiment of the present application does not limit the number of iterations (epochs) of the initial image recognition model, which can be set according to the actual application scenario, and does not limit the total loss threshold of the model, which can be set according to the actual application scenario.
  • Step S102 obtaining a sampling sample image from the A sample image set, inputting the sampling sample image and the erroneous category label into the image recognition model to generate a first probability vector of the sampling sample image for the A category labels, wherein the erroneous category label is different from the category label corresponding to the sample image set to which the sampling sample image belongs;
  • the first probability vector includes A probability elements, each of the A probability elements refers to one of the A category labels, and one of the probability elements is used to characterize the probability that the sampling sample image belongs to the referred category label.
  • a sample image set service server Cb in A sample image sets is taken as an example for description, where b is a positive integer and b is less than or equal to A.
  • the service server may determine a sampling ratio, and perform sampling processing on the sample images in the sample image set Cb according to the sampling ratio to obtain sampled sample images; among the A category labels, a category label different from the category label of the sample image set Cb is obtained (which may be obtained in a random manner), and the obtained category label is determined as an incorrect category label.
  • the business server 100 obtains sample images from the above four sample image sets.
  • a feasible sampling method is as follows: determine the sampling ratio, for example, 5%, and the business server 100 randomly samples images in the sample image set 201a to obtain 5% of the sample images in the sample image set 201a, and compose the sample images 201b with the sample images. For example, if there are 10,000 sample images in the sample image set 201a, the number of sample images 201b is 500.
  • the generation process of the sample images corresponding to other sample image sets can refer to the generation process of the sample image 201b, which will not be described in detail here.
  • the sampling sample image 201b in the example of FIG2a of the embodiment of the present application includes a first sampling sample image and a second sampling sample image. It can be understood that the first sampling sample image and the second sampling sample image are both sample images containing radish, so the correct category labels corresponding to the first sampling sample image and the second sampling sample image are both the category labels of radish, that is, the category label 0 illustrated in FIG2a.
  • the sampling sample image 202b in the example of FIG2a of the embodiment of the present application includes a third sampling sample image and a fourth sampling sample image.
  • the third sampling sample image and the fourth sampling sample image are both sample images containing peppers, so the correct category labels corresponding to the third sampling sample image and the fourth sampling sample image are both the category labels of peppers, that is, the category label 1 illustrated in FIG2a.
  • the business server 100 randomly extracts a category label, such as category label 1, from category label 1, category label 2, and category label 3 illustrated in FIG2a, and sets it as the wrong category label for the first sampling sample image. Similarly, the business server 100 randomly extracts a category label, such as category label 2, from category label 1, category label 2, and category label 3 illustrated in FIG2a, and sets it as the wrong category label for the second sampling sample image. The business server 100 randomly extracts a category label, such as category label 2, from category label 0, category label 2, and category label 3 illustrated in FIG2a, and sets it as the wrong category label for the third sampling sample image.
  • a category label such as category label 1, from category label 1, category label 2, and category label 3 illustrated in FIG2a
  • the business server 100 randomly extracts a category label, such as category label 2, from category label 0, category label 2, and category label 3 illustrated in FIG2a, and sets it as the wrong category label for the third sampling sample image.
  • the class label is 0, which is set as the wrong class label for the fourth sampled image.
  • A is exemplified as 3, that is, 3 sample image sets, namely sample image set 3a, sample image set 3b and sample image set 3c.
  • the category information of sample image set 3a is real objects, and its category label is 30a; the category information of sample image set 3b is paper objects, and its category label is 30b; the category information of sample image set 3c is mask objects, and its category label is 30c.
  • sample image set 3b and sample image set 3c can be regarded as attack sample image sets of sample image set 3a.
  • the process of the business server obtaining the sample sample images corresponding to the sample image set 3a, the sample image set 3b and the sample image set 3c is the same as the process described above, so it will not be repeated here.
  • the sample sample images of the sample image set 3a include the fifth sample sample image and the sixth sample sample image
  • the sample sample images of the sample image set 3b include the seventh sample sample image and the eighth sample sample image
  • the sample sample images of the sample image set 3c include the ninth sample sample image and the tenth sample sample image.
  • the business server 100 randomly extracts a category label, such as category label 30b, from the category label 30b and the category label 30c, and sets it as the wrong category label of the fifth sample sample image.
  • the business server 100 randomly extracts a category label, such as category label 30c, from the category label 30b and the category label 30c, and sets it as the wrong category label of the sixth sample sample image.
  • the business server sets the wrong category labels corresponding to the seventh sample sample image, the eighth sample sample image, the ninth sample sample image and the tenth sample sample image, respectively, to category label 30a. Therefore, the embodiment of the present application provides two methods for setting error category labels. In actual application, one of the two setting methods can be selected according to actual needs.
  • the service server inputs the sample images corresponding to the A sample image sets into the image recognition model. It is understandable that the image recognition model processes each sample image in the same way, so please refer to the description of the sample image 201c in the embodiment corresponding to FIG. 2a above.
  • the image recognition model can obtain the first probability vector corresponding to each sample image, and each first probability vector includes A probability values (also referred to as probability elements). Since the image recognition model is pre-trained by A sample image sets, the image recognition model can output the correct probability distribution for each sample image (the correct probability distribution indicates that the probability value of the correct category label in the output first probability vector is the largest). For ease of understanding, please refer to Figure 2a again.
  • the first element (also referred to as a probability element, which is used to refer to the category label 0, and which is used to characterize the probability that the sample image belongs to the referred category label 0) of the first probability vector output by the image recognition model 20d can be infinitely close to 1, and the remaining three elements will be infinitely close to 0.
  • the second element of the first probability vector output by the image recognition model 20d can be infinitely close to 1, and the remaining three elements (specifically, the first element, the third element, and the fourth element) are infinitely close to 0.
  • the third element of the first probability vector output by the image recognition model 20d can be infinitely close to 1, and the remaining three elements (specifically, the first element, the second element, and the fourth element) are infinitely close to 0.
  • the fourth element of the first probability vector output by the image recognition model 20d can be infinitely close to 1, and the remaining three elements (specifically, the first element, the second element, and the fourth element) are infinitely close to 0.
  • Step S103 obtaining a first probability element indicating the wrong category label from the A probability elements in the first probability vector, adjusting the sample image according to the first probability element, and obtaining an adversarial sample image corresponding to the sample image, wherein the category prediction result obtained by performing category prediction on the adversarial sample image by the image recognition model is the wrong category label.
  • the business server adjusts the sampled sample image according to the first probability element to obtain an initial adversarial sample image corresponding to the sampled sample image; inputs the error category label and the initial adversarial sample image into the image recognition model, wherein the error category label and the initial adversarial sample image can be respectively input into the image recognition model as two independent data, and there is a mapping relationship between the error category label and the initial adversarial sample image; or, the error category label can be added to the file header or file footer of the initial adversarial sample image, and then the initial adversarial sample image carrying the error category label can be input into the image recognition model.
  • a second probability vector for the initial adversarial sample image for A category labels is generated, and the second probability vector also contains A probability elements, and the A probability elements respectively refer to one of the A category labels, and one of the probability elements is used to characterize the probability that the initial adversarial sample image belongs to the referred category label; in the second probability vector, obtain the second probability element referring to the error category label, and determine the adversarial sample image corresponding to the sampled sample image according to the second probability element.
  • the business server can analyze whether the second probability element is a second probability vector.
  • the initial adversarial sample image can be determined as the adversarial sample image corresponding to the sampling sample image (that is, at this time the image recognition model can mistakenly identify the category prediction result of the adversarial sample image as the wrong category label); if it is analyzed that the second probability element is not the probability element with the largest value in the second probability vector, the initial adversarial sample image will be further adjusted.
  • the purpose of the adjustment is to hope that the image recognition model outputs a probability element referring to the wrong category label in the third probability vector for the adjusted initial adversarial sample image, which becomes larger, until the image recognition model can mistakenly identify the adversarial sample image after multiple rounds of adjustment as the wrong category label.
  • the specific process of adjusting the sample image according to the first probability element to obtain the initial adversarial sample image corresponding to the sample image may include: generating a negative probability element corresponding to the first probability element, summing the negative probability element and the maximum probability element in the first probability vector to obtain the label loss value of the sample image for the wrong category label; adjusting the sample image according to the label loss value to obtain the initial adversarial sample image corresponding to the sample image. If the initial adversarial sample image can be directly used as the adversarial sample image corresponding to the sample image, it can also be understood that the sample image is adjusted according to the label loss value to directly obtain the adversarial sample image corresponding to the sample image.
  • the sample image is adjusted to obtain the specific process of the initial adversarial sample image corresponding to the sample image, which may include: generating an initial gradient value for the sample image according to the label loss value, obtaining the numerical sign of the initial gradient value, and generating a symbol unit value corresponding to the initial gradient value according to the numerical sign; performing product processing on the initial learning rate and the symbol unit value to obtain a first value to be cropped, and cropping the first value to be cropped through a first cropping interval generated by the attack intensity to obtain a gradient value for adjusting the sample image; performing difference processing on the sample image and the gradient value (the gradient value may also be referred to as a perturbation value, and the difference processing may be understood as adding the perturbation value to the sample image to obtain an image containing noise) to obtain a second value to be cropped, and cropping the second value to be cropped through a second cropping interval generated by the pixel level to obtain the initial adversarial sample image corresponding to the sample image (if the initial adversarial
  • the business server uses the maximum probability element in the first probability vector as the first loss value, uses the first probability element corresponding to the wrong category label in the first probability vector as the second loss value, and uses the difference between the first loss value and the second loss value as the label loss value of the sampled image for the wrong category label. Furthermore, the business server can adjust the sampled image according to the label loss value to obtain the initial adversarial sample image of the sampled image.
  • the specific implementation method can adopt the Project Gradient Descent (an iterative attack algorithm, referred to as PGD) attack algorithm. Among them, the specific implementation process of the PGD attack algorithm can be described by the following formula (1).
  • t in formula (1) represents the current iteration, and t is a positive integer, which means that the image recognition model will undergo at least one iteration, and that the sample image will undergo at least one adjustment.
  • Represents the result data of the tth iteration such as the initial adversarial sample image generated after the first iteration, or the adversarial sample image generated after the last iteration.
  • the input data of the first iteration is the sampled sample image.
  • the input data of the second iteration is the initial adversarial sample image, that is, the result data of the first iteration; similarly, if there are more iterations, the input data of the next iteration is the result data of the previous iteration.
  • ⁇ represents the learning rate.
  • sign(.) represents the sign function, which can change each element of the initial gradient value (map) to ⁇ -1, 0, 1 ⁇ .
  • eps represents the attack strength, that is, the absolute value of each element value of the gradient map used for adjustment each time cannot be greater than the attack strength.
  • the attack strength is an adjustable parameter, generally set to 8/255, 16/255, 32/255, etc.
  • the clip(.,.) function represents clipping, that is, the clip(.,.) function can control all values within the specified closed interval. [-eps,eps] represents the first clipping interval, and [0,255] represents the second clipping interval.
  • the embodiments of the present application do not limit the method of adjusting the image.
  • the above-mentioned PGD attack method may also be replaced by other attack methods, such as Fast Gradient Sign Method (an algorithm based on gradient generation adversarial samples, referred to as FGSM), Iterative Fast Gradient Sign Method (an improved version of FGSM, referred to as I-FGSM), Momentum Iterative Fast Gradient Sign Method (another improved version of FGSM, referred to as MI-FGSM), etc.
  • FGSM Fast Gradient Sign Method
  • I-FGSM Iterative Fast Gradient Sign Method
  • MI-FGSM Momentum Iterative Fast Gradient Sign Method
  • the business server inputs the error category label and the initial adversarial sample image into the image recognition model. Generate the second probability vector of the initial adversarial sample image for A category labels.
  • ⁇ and the number of iterations t must meet a certain number, so as to ensure that the adjusted sampled sample image can cross the interface of the image recognition model and become a sample image that can be recognized by the image recognition model as a wrong category label (in order to distinguish it from the adversarial sample image, the sample image here is called a strong adversarial sample image, that is, the probability element of the image recognition model output for the strong adversarial sample image that refers to the wrong category label is close to 1).
  • Figure 4 is a schematic diagram of a classification boundary surface trained by clean sample data provided in an embodiment of the present application
  • Figure 5 is a schematic diagram of a classification boundary surface trained by first mixed sample data provided in an embodiment of the present application.
  • the clean sample data refers to the above-mentioned A sample image sets
  • the first mixed sample data includes the above-mentioned A sample image sets and the above-mentioned strong adversarial sample images.
  • A can be equal to 2, that is, A category labels include the first category label and the second category label illustrated in Figures 4 and 5.
  • A is greater than 2, but one category label is used as the true category label, and the remaining one or more category labels are used as attack category labels.
  • the category feature distribution area corresponding to the first category label i.e., the area where the symbol “+” is located
  • the category feature distribution area corresponding to the second category label i.e., the area where the symbol “-” is located
  • the category feature distribution area corresponding to the first category label i.e., the area where the symbol “+” is located and the area where the symbol “ ⁇ ” is located
  • the category feature distribution area corresponding to the second category label i.e., the area where the symbol “-” is located and the area where the symbol “ ⁇ ” is located
  • the image recognition model will make the interface jagged. Although this improves the robustness of the optimized image recognition model to the strong adversarial sample data, some sample images with low confidence close to the interface are also misclassified.
  • the purpose of the embodiment of the present application should be to improve the accuracy and generalization of the image recognition model while improving the robustness. Therefore, the embodiment of the present application adds boundary constraints to screen adversarial sample images. In other words, the adversarial sample images used for data enhancement should not cross the boundary too far.
  • another way to determine the adversarial sample image may be: the business server adjusts the sampled sample image according to the first probability element to obtain the initial adversarial sample image corresponding to the sampled sample image; the wrong category label and the initial adversarial sample image are input into the image recognition model, and the image recognition model is used to generate a second probability vector for the A category labels of the initial adversarial sample image; among the A probability elements contained in the second probability vector, the second probability element for the wrong category label is obtained; if the second probability element does not meet the boundary constraint, or the category prediction result obtained by performing category prediction on the adversarial sample image is not the wrong category label, then the initial adversarial sample image is continued to be predicted according to the second probability element.
  • the adversarial sample image is adjusted first (the process of continuing to adjust the initial adversarial sample image is the same as the specific implementation process of adjusting the sampled sample image according to the first probability element to obtain the initial adversarial sample image, so it will not be repeated here, please refer to the description above.
  • the image adjustment method is the same as the adjustment method of the sampled sample image); if the second probability element satisfies the boundary constraint condition, and the category prediction result obtained by performing category prediction on the adversarial sample image is the wrong category label, then the initial adversarial sample image is determined as the adversarial sample image corresponding to the sampled sample image; when the category prediction result is the wrong category label, the second probability element is the maximum value of the A probability elements contained in the second probability vector.
  • the adversarial sample image desired by the present application is one that can both satisfy the boundary constraint condition and ensure that the image recognition model can incorrectly identify the adversarial sample image.
  • the business server can also compare the second probability element and the probability constraint value; the probability constraint value is the inverse of A; if the second probability element is less than the probability constraint value, it is determined that the second probability element does not meet the boundary constraint condition; if the second probability element is equal to or greater than the probability constraint value, it is determined that the second probability element meets the boundary constraint condition. That is, the present application controls the learning rate and the number of iterations.
  • the adversarial sample image obtained in the embodiment of the present application can be located near the classification boundary surface, rather than in the distribution of the wrong category label, and the classification boundary surface of the image recognition model optimized and trained by the adversarial sample image will be softer.
  • Figure 6 is a schematic diagram of a classification boundary surface trained by the second mixed sample data provided by the embodiment of the present application.
  • the data includes the above-mentioned A sample image sets and the above-mentioned adversarial sample images.
  • the category feature distribution area corresponding to the first category label i.e., the area where the symbol "+” is located and the area where the symbol " ⁇ ” is located
  • the category feature distribution area corresponding to the second category label i.e., the area where the symbol "-” is located and the area where the symbol " ⁇ ” is located
  • the optimized image recognition model can accurately identify the first category label and the second category label, that is, it can ensure that the optimized image recognition model (referred to as the image recognition optimization model) maintains or improves the original recognition accuracy, and can also supplement the deficiencies of the training data, so the defense capability and security of the image recognition optimization model can be improved.
  • the embodiment of the present application proposes a new adversarial data enhancement based on boundary constraints, which utilizes the particularity of the distribution of adversarial data at the boundary to amplify and enhance the training data, thereby improving the defense capability and generalization of the optimized image recognition model.
  • a computer device can obtain A sample image sets and an image recognition model; wherein the A sample image sets respectively correspond to one of A different category labels, and the image recognition model is pre-trained based on the A sample image sets, so the image recognition model can accurately determine the category labels of the sample images in the A sample image sets; further, the computer device obtains a sampled sample image from the A sample image sets, and inputs an erroneous category label and the sampled sample image into the image recognition model, wherein the erroneous category label belongs to the A category labels, and the erroneous category label is different from the category label corresponding to the sample image set to which the sampled sample image belongs; a first probability vector of the sampled sample image for the A category labels can be generated through the image recognition model, and the first probability vector includes A probability elements, and the A probability elements respectively refer to one of the A category labels.
  • a probability element is used to characterize the probability that the sampled sample image belongs to the referred category label; since the image recognition model can accurately determine the category labels of the sample images in the A sample image sets, in the first probability vector, the probability element referring to the correct category label is the largest, or in other words, the probability element referring to the correct category label is much larger than the first probability element referring to the wrong category label, and the correct category label refers to the category label corresponding to the sample image set to which the sampled sample image belongs; further, according to the first probability element, the computer device can adjust the sampled sample image to obtain an adversarial sample image corresponding to the sampled sample image, wherein the category prediction result obtained by the image recognition model for category prediction of the adversarial sample image is the wrong category label, that is, by adjusting the sampled sample image, the image recognition model can incorrectly determine the category label of the adversarial sample image.
  • the embodiment of the present application proposes a method for generating adversarial sample images.
  • adversarial sample images that are incorrectly recognized by an image recognition model can be generated.
  • the adversarial sample images can improve the sample diversity of the training set used to optimize the training of the image recognition model.
  • the training set with sample diversity can improve the optimization training accuracy of the image recognition model, and then improve the category label recognition accuracy and recognition generalization of the optimized trained image recognition model.
  • the classification boundary of the image recognition model after optimization training can be made softer, further ensuring the model accuracy.
  • Figure 7 is a flow chart of a data processing method provided in an embodiment of the present application.
  • the method can be executed by a business server (for example, the business server 100 shown in Figure 1 above), or by a terminal device (for example, the terminal device 200a shown in Figure 1 above), or by a business server and a terminal device interacting with each other.
  • a business server for example, the business server 100 shown in Figure 1 above
  • a terminal device for example, the terminal device 200a shown in Figure 1 above
  • the embodiment of the present application is described by taking the method being executed by a business server as an example.
  • the method can at least include the following steps.
  • Step S201 obtain A sample image sets and an image recognition model, where A is a positive integer greater than 1, and the A sample image sets respectively correspond to one of A different category labels.
  • the image recognition model is pre-trained based on the A sample image sets, and the A category labels include error category labels.
  • the business service obtains A original image sets, and inputs the A original image sets into the object detection model; the A original image sets correspond to one of A different category information respectively; the A original image sets include the original image set D b ; the original image set D b includes the original image E f ; f is a positive integer, and f is less than or equal to the total number of original images in the original image set D b ; through the object detection model, the region coordinates of the key region in the original image E f are determined, and according to the region coordinates, the sample image to be labeled corresponding to the original image E f is generated; the sample image to be labeled corresponding to each original image in the original image set D b is determined as the sample image set to be labeled; according to the category information corresponding to the original image set D b, a category label H b is generated, and the sample image set to be labeled with the category label H b is determined as the sample image set C b.
  • the specific process of generating a sample image to be labeled corresponding to the original image Ef according to the region coordinates may include: generating an initial detection frame including a key area according to the region coordinates, expanding the initial detection frame to obtain a detection frame including the target object; the key area belongs to the target object; in the detection frame, obtaining an image to be scaled including the target object, obtaining a first image size, scaling the image to be scaled according to the first image size, to obtain an image to be cropped; obtaining a second image size, cropping the image to be cropped according to the second image size, to obtain a sample image to be labeled corresponding to the original image Ef; the second image size is smaller than the first image size.
  • the business server does not limit the model type of the object detection model and can set it according to the actual application scenario, such as Region with CNN feature (a target detection model, referred to as Faster R-CNN), Single Shot MultiBox Detector (another target detection model, referred to as SSD), You Only Look Once (a simple and fast target detection algorithm, referred to as YOLO), etc.
  • Region with CNN feature a target detection model, referred to as Faster R-CNN
  • SSD Single Shot MultiBox Detector
  • YOLO a simple and fast target detection algorithm
  • the key area belongs to the target object.
  • the embodiment of the present application does not limit the target object.
  • it can be a cat, a dog, a person, a table, Pepper, etc.
  • the key area is not limited, and it should be set according to the target object.
  • the expansion multiple of the initial detection frame should be set according to the key area, so the embodiment of the application does not limit the expansion multiple.
  • the second image size refers to the input size of the image recognition model.
  • Step S202 obtaining a sampling sample image from the A sample image set, inputting the sampling sample image and the erroneous category label into the image recognition model to generate a first probability vector of the sampling sample image for the A category labels, wherein the erroneous category label is different from the category label corresponding to the sample image set to which the sampling sample image belongs;
  • the first probability vector includes A probability elements, each of the A probability elements refers to one of the A category labels, and one of the probability elements is used to characterize the probability that the sampling sample image belongs to the referred category label.
  • Step S203 obtaining a first probability element indicating the wrong category label from among the A probability elements in the first probability vector, adjusting the sample image according to the first probability element, and obtaining an adversarial sample image corresponding to the sample image, wherein the category prediction result obtained by performing category prediction on the adversarial sample image by the image recognition model is the wrong category label.
  • step S202 to step S203 please refer to step S102 and step S103 in the embodiment corresponding to FIG. 3 above, which will not be described in detail here.
  • Step S204 inputting the A sample image sets, the A category labels, the correct category label and the adversarial sample image into the image recognition model;
  • the correct category label refers to the category label corresponding to the sample image set to which the sampled sample image belongs.
  • the A sample image sets, the A category labels, the correct category labels and the adversarial sample images can be respectively input into the image recognition model as four independent data, and there is a mapping relationship between the A sample image sets and the A category labels, and there is a mapping relationship between the correct category labels and the adversarial sample images; or, the corresponding category labels can be added to the file header or file footer of the sample images in the sample image set, and the correct category labels can be added to the file header or file footer of the adversarial sample images, and then the sample images carrying category labels and the adversarial sample images carrying correct category labels can be input into the image recognition model.
  • Step S205 Generate, by using the image recognition model, first prediction categories corresponding to the A sample image sets and second prediction categories corresponding to the adversarial sample images.
  • Step S206 generating a first loss value according to the first predicted category and the A category labels, generating a second loss value according to the second predicted category and the correct category label, adjusting the parameters in the image recognition model according to the first loss value and the second loss value (specifically, the first loss value and the second loss value can be weighted and summed to obtain a total loss value, and then the parameters in the image recognition model are adjusted based on the total loss value) to obtain the image recognition optimization model; the category prediction result obtained by the image recognition optimization model for category prediction of the adversarial sample image is the correct category label, and the category prediction result obtained by the image recognition optimization model for category prediction of the sample image in the sample image set is still the category label corresponding to the sample image set.
  • the optimization training process of the image recognition model by the business server is the same as the training process of the initial image recognition model by the business server. Therefore, for the specific implementation process of steps S204 to S206, please refer to step S101 in the embodiment corresponding to Figure 3 above, which will not be repeated here.
  • the image recognition optimization model proposed in the embodiment of the present application can be deployed on a terminal device, for example, to detect an image input for real object recognition (such as real person recognition). If it is a real object, it passes and enters the subsequent recognition process. If it is not a real object, such as a photo object, an error is reported and a retry prompt is given.
  • the embodiment of the present application can be applied to all object recognition related applications, including but not limited to: online payment, offline payment, access control unlocking system, mobile phone unlocking recognition, automatic object recognition clearance, etc.
  • an embodiment of the present application proposes a method for generating adversarial sample images.
  • adversarial sample images that are incorrectly recognized by an image recognition model can be generated.
  • the sample diversity of the training set used to optimize the training of the image recognition model can be improved.
  • the optimization training accuracy of the image recognition model can be improved, and then the category label recognition accuracy and recognition generalization of the optimized trained image recognition model can be improved.
  • Figure 8 is a flow chart of a data processing method provided by an embodiment of the present application.
  • the method can be executed by a business server (for example, the business server 100 shown in Figure 1 above), or by a terminal device (for example, the terminal device 200a shown in Figure 1 above), or by a business server and a terminal device interacting with each other.
  • a business server for example, the business server 100 shown in Figure 1 above
  • a terminal device for example, the terminal device 200a shown in Figure 1 above
  • the embodiment of the present application is described by taking the method being executed by a business server as an example.
  • the method may at least include the following steps.
  • Step S301 obtain a target image, and input the target image into an image recognition optimization model;
  • the image recognition optimization model is obtained by continuing to optimize the image recognition model according to the mixed sample training set;
  • the mixed sample training set includes A sample image sets, A category labels, correct category labels and adversarial sample images;
  • the A sample image sets correspond to one of A different category labels respectively, and the A sample image sets include collected sample images;
  • the correct category label belongs to the A category labels;
  • A is a positive integer greater than 1;
  • the image recognition model is obtained by pre-training the A sample image sets;
  • the adversarial sample image is obtained by adjusting the sampled sample image by the first probability element of the A probability elements in the first probability vector that refers to the wrong category label, and the first probability
  • the vector is generated by the image recognition model for the sampled sample image, the erroneous category label is different from the category label corresponding to the sample image set to which the sampled sample image belongs, and the correct category label is the category label corresponding to the sample image set
  • Step S302 Generate a target probability vector of the target image for A category labels through an image recognition optimization model.
  • Step S303 determine the category label corresponding to the maximum probability element in the target probability vector as the category prediction result of the target image.
  • the image recognition model of the embodiment of the present application can be used to identify real objects, such as real people.
  • the image recognition model can be called a real object detection model.
  • the image recognition model can be called a real object detection model.
  • the image recognition model can be used to identify real objects, such as real people.
  • the image recognition model can be called a real object detection model.
  • the image recognition model can be called a real object detection model.
  • the real object detection model there are many types of attacks brought by illegal elements, and the attack forms are novel and difficult to defend.
  • the real object detection model can be bypassed to achieve the attack. This problem is also faced in high-precision 3D attacks. Once a new material appears, the real object detection model will face a situation where it cannot be defended.
  • the embodiment of the present application proposes a new real object detection technology based on boundary constraint adversarial data enhancement, which enhances the real object training data by designing a special adversarial sample image, thereby improving the overall performance of real object detection.
  • boundary constraint adversarial data enhancement which enhances the real object training data by designing a special adversarial sample image, thereby improving the overall performance of real object detection.
  • an embodiment of the present application proposes a method for generating adversarial sample images.
  • adversarial sample images that are incorrectly recognized by an image recognition model can be generated.
  • the sample diversity of the training set used to optimize the training of the image recognition model can be improved.
  • the optimization training accuracy of the image recognition model can be improved, and then the category label recognition accuracy and recognition generalization of the optimized trained image recognition model can be improved.
  • FIG. 9 is a structural schematic diagram of a data processing device provided in an embodiment of the present application.
  • the above-mentioned data processing device 1 can be used to execute the corresponding steps in the method provided in an embodiment of the present application.
  • the data processing device 1 may include: a data acquisition module 11, a first input module 12, and a first adjustment module 13.
  • a data acquisition module 11 is used to acquire A sample image sets and an image recognition model, where A is a positive integer greater than 1, the A sample image sets correspond to one of A different category labels respectively, the image recognition model is pre-trained based on the A sample image sets, and the A category labels include error category labels;
  • a first input module 12 is used to obtain a sampled sample image from the A sample image sets, and input the sampled sample image and the erroneous category label into the image recognition model to generate a first probability vector of the sampled sample image for the A category labels, wherein the erroneous category label is different from the category label corresponding to the sample image set to which the sampled sample image belongs;
  • the first probability vector includes A probability elements, each of the A probability elements refers to one of the A category labels, and one of the probability elements is used to represent the probability that the sampled sample image belongs to the referred category label;
  • the first adjustment module 13 is used to obtain a first probability element referring to the wrong category label from the A probability elements in the first probability vector, and adjust the sampled sample image according to the first probability element to obtain an adversarial sample image corresponding to the sampled sample image, wherein the category prediction result obtained by the image recognition model performing category prediction on the adversarial sample image is the wrong category label.
  • the first adjustment module 13 can be specifically used to generate a negative probability element corresponding to the first probability element, sum the negative probability element and the maximum probability element in the first probability vector, obtain the label loss value of the sampling sample image for the wrong category label, and adjust the sampling sample image according to the label loss value to obtain the adversarial sample image corresponding to the sampling sample image.
  • the first adjustment module 13 is used to adjust the sample image according to the label loss value to obtain the adversarial sample image corresponding to the sample image. Specifically, it is used to generate an initial gradient value for the sample image according to the label loss value, obtain the numerical sign of the initial gradient value, generate a symbol unit value corresponding to the initial gradient value according to the numerical sign, multiply the initial learning rate and the symbol unit value to obtain a first value to be cropped, crop the first value to be cropped through a first cropping interval generated by the attack intensity to obtain a gradient value for adjusting the sample image, perform difference processing on the sample image and the gradient value to obtain a second value to be cropped, crop the second value to be cropped through a second cropping interval generated by the pixel level, and obtain the adversarial sample image corresponding to the sample image.
  • the first adjustment module 13 may include: a first adjustment unit 131 , a first generation unit 132 , and a first acquisition unit 133 .
  • a first adjustment unit 131 is used to adjust the sample image according to the first probability element to obtain an initial adversarial sample image corresponding to the sample image;
  • a first generating unit 132 is used to input the wrong category label and the initial adversarial sample image into the image recognition model, and generate a second probability vector of the initial adversarial sample image for A category labels through the image recognition model;
  • a first acquiring unit 133 is configured to acquire a second probability element for the error category label from the A probability elements included in the second probability vector;
  • the first adjustment unit 131 is further configured to continue adjusting the initial adversarial sample image according to the second probability element if the second probability element does not satisfy the boundary constraint condition, or the category prediction result obtained by performing category prediction on the adversarial sample image is not the wrong category label;
  • the first acquisition unit 133 is further used to determine the initial adversarial sample image as the adversarial sample image corresponding to the sampling sample image if the second probability element satisfies the boundary constraint condition and the category prediction result obtained by performing category prediction on the adversarial sample image is the wrong category label; when the category prediction result is the wrong category label, the second probability element is the maximum value of the A probability elements contained in the second probability vector.
  • the specific functional implementation of the first adjustment unit 131 , the first generation unit 132 and the first acquisition unit 133 may refer to step S103 in the embodiment corresponding to FIG. 3 , which will not be described in detail here.
  • the first acquisition unit 133 is further used to compare the second probability element with a probability constraint value; the probability constraint value is the reciprocal of A;
  • the first acquisition unit 133 is further configured to determine that the second probability element does not satisfy a boundary constraint condition if the second probability element is less than the probability constraint value;
  • the first acquiring unit 133 is further configured to determine that the second probability element satisfies a boundary constraint condition if the second probability element is equal to or greater than the probability constraint value.
  • the data processing device 1 may further include: a second input module 14 and a second adjustment module 15 .
  • a second input module 14 is used to input the A sample image sets, the A category labels, the correct category label and the adversarial sample image into the image recognition model;
  • the correct category label refers to the category label corresponding to the sample image set to which the sample image belongs;
  • the second input module 14 is further used to generate the first prediction categories corresponding to the A sample image sets and the second prediction category corresponding to the adversarial sample image through the image recognition model;
  • the second adjustment module 15 is used to generate a first loss value according to the first predicted category and the A category labels, generate a second loss value according to the second predicted category and the correct category label, and adjust the parameters in the image recognition model according to the first loss value and the second loss value to obtain the image recognition optimization model; the category prediction result obtained by the image recognition optimization model for category prediction of the adversarial sample image is the correct category label.
  • the specific functional implementation of the second input module 14 and the second adjustment module 15 can refer to steps S204 to S206 in the embodiment corresponding to FIG. 7 above, which will not be described in detail here.
  • the first input module 12 may include: a first determining unit 121 and a second obtaining unit 122 .
  • the A sample image sets include a sample image set Cb; b is a positive integer and b is less than or equal to A;
  • the first determining unit 121 is used to determine a sampling ratio, and perform sampling processing on the sample images in the sample image set Cb according to the sampling ratio to obtain a sampled sample image;
  • the second acquisition unit 122 is used to acquire a category label different from the category label of the sample image set Cb from among the A category labels, and determine the acquired category label as an erroneous category label.
  • the specific functional implementation of the first determining unit 121 and the second acquiring unit 122 can refer to step S102 in the embodiment corresponding to FIG. 3 above, which will not be described in detail here.
  • the data acquisition module 11 may include: a third acquisition unit 111 , a second generation unit 112 , a second determination unit 113 , a third determination unit 114 and a second adjustment unit 115 .
  • a third acquisition unit 111 is used to acquire A sample image sets, and input A category labels and the A sample image sets into the image recognition initial model respectively;
  • a second generating unit 112 is used to generate predicted initial categories corresponding to the A sample image sets respectively through an image recognition initial model
  • a second determining unit 113 is used to determine the category loss values corresponding to the A sample image sets respectively according to the predicted initial category and the A category labels;
  • the third determining unit 114 is used to determine the image recognition initial model corresponding to the category loss values corresponding to the A sample image sets. Total loss value;
  • the second adjustment unit 115 is used to adjust the parameters in the initial image recognition model according to the total loss value to obtain an image recognition model; the category prediction result obtained by the image recognition model for the sample image set Cb is the category label of the sample image set Cb; the A sample image sets include the sample image set Cb; b is a positive integer and b is less than or equal to A.
  • step S101 in the embodiment corresponding to Figure 3 above, and will not be repeated here.
  • the third acquisition unit 111 may include: an image acquisition subunit 1111 , a second determination subunit 1112 , a third determination subunit 1113 , and a fourth determination subunit 1114 .
  • the image acquisition subunit 1111 is used to acquire A original image sets, and input the A original image sets into the object detection model; the category information corresponding to the A original image sets are different from each other; the A original image sets include an original image set Db; the original image set Db includes an original image Ef; f is a positive integer, and f is less than or equal to the total number of original images in the original image set Db;
  • the second determining subunit 1112 is used to determine the region coordinates of the key region in the original image Ef through the object detection model, and generate a sample image to be labeled corresponding to the original image Ef according to the region coordinates;
  • the third determining subunit 1113 is used to determine the sample image to be labeled corresponding to each original image in the original image set Db as the sample image set to be labeled;
  • the fourth determining subunit 1114 is used to generate a category label Hb according to the category information corresponding to the original image set Db, and determine the set of sample images to be labeled with the category label Hb as the sample image set Cb.
  • the specific functional implementation of the image acquisition subunit 1111, the second determination subunit 1112, the third determination subunit 1113 and the fourth determination subunit 1114 can refer to step S201 in the embodiment corresponding to Figure 7 above, and will not be repeated here.
  • the second determining subunit 1112 may include: a third processing subunit 11121 , a fourth processing subunit 11122 , and a fifth processing subunit 11123 .
  • the third processing subunit 11121 is used to generate an initial detection frame including the key region according to the region coordinates, and perform expansion processing on the initial detection frame to obtain a detection frame including the target object; the key region belongs to the target object;
  • the fourth processing subunit 11122 is used to obtain the image to be scaled including the target object in the detection frame, obtain the first image size, and scale the image to be scaled according to the first image size to obtain the image to be cropped;
  • the fifth processing subunit 11123 is used to obtain a second image size, and perform cropping processing on the image to be cropped according to the second image size to obtain a sample image to be labeled corresponding to the original image Ef; the second image size is smaller than the first image size.
  • the specific functional implementation of the third processing sub-unit 11121, the fourth processing sub-unit 11122 and the fifth processing sub-unit 11123 can refer to step S201 in the embodiment corresponding to Figure 7 above, and will not be repeated here.
  • the embodiment of the present application proposes a method for generating adversarial sample images, through which adversarial sample images that are incorrectly recognized by an image recognition model can be generated.
  • adversarial sample images the sample diversity of the training set used to optimize the training of the image recognition model can be improved.
  • the optimization training accuracy of the image recognition model can be improved, and then the category label recognition accuracy and recognition generalization of the optimized trained image recognition model can be improved.
  • the second probability element for the wrong category label predicted by the image recognition model for the adversarial sample image satisfies the boundary constraint condition, the classification boundary of the image recognition model after optimization training can be made softer, further ensuring the model accuracy.
  • FIG. 10 is a second structural diagram of a data processing device provided in an embodiment of the present application.
  • the above-mentioned data processing device 2 can be used to execute the corresponding steps in the method provided in an embodiment of the present application.
  • the data processing device 2 may include: an image acquisition module 21, a vector generation module 22, and a category determination module 23.
  • the image acquisition module 21 is used to acquire a target image and input the target image into an image recognition optimization model; the image recognition optimization model is obtained by continuing to optimize the image recognition model according to the mixed sample training set; the mixed sample training set includes A sample image sets, A category labels, correct category labels and adversarial sample images; the A sample image sets correspond to one of A different category labels respectively, and the A sample image sets include collected sample images; the correct category label belongs to the A category labels; A is a positive integer greater than 1; the image recognition model is obtained by pre-training the A sample image sets; the adversarial sample image is represented by the A probability elements in the first probability vector The first probability element of the wrong category label is obtained by adjusting the sampled sample image, the first probability vector is generated by the image recognition model for the sampled sample image, the wrong category label is different from the category label corresponding to the sample image set to which the sampled sample image belongs, and the correct category label is the category label corresponding to the sample image set to which the sampled sample image belongs; the first probability vector includes A probability elements, the A
  • a vector generation module 22 used to generate a target probability vector of a target image for A category labels through an image recognition optimization model
  • the category determination module 23 is used to determine the category label corresponding to the maximum probability element in the target probability vector as the category prediction result of the target image.
  • an embodiment of the present application proposes a method for generating adversarial sample images.
  • adversarial sample images that are incorrectly recognized by an image recognition model can be generated.
  • the sample diversity of the training set used to optimize the training of the image recognition model can be improved.
  • the optimization training accuracy of the image recognition model can be improved, and then the category label recognition accuracy and recognition generalization of the optimized trained image recognition model can be improved.
  • the computer device 1000 may include: at least one processor 1001, such as a CPU, at least one network interface 1004, a user interface 1003, a memory 1005, and at least one communication bus 1002.
  • the communication bus 1002 is used to realize the connection and communication between these components.
  • the user interface 1003 may include a display screen (Display), a keyboard (Keyboard), and the network interface 1004 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface).
  • the memory 1005 may be a high-speed RAM memory, or it may be a non-volatile memory (non-volatile memory), such as at least one disk memory.
  • the memory 1005 may also be optionally at least one storage device located away from the aforementioned processor 1001.
  • the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a device control application.
  • the network interface 1004 can provide a network communication function;
  • the user interface 1003 is mainly used to provide an input interface for the user; and
  • the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve:
  • a sample image sets and an image recognition model where A is a positive integer greater than 1, the A sample image sets respectively correspond to one of A different category labels, the image recognition model is pre-trained based on the A sample image sets, and the A category labels include error category labels;
  • a sampling sample image is obtained from the A sample image sets, and the sampling sample image and the erroneous category label are input into the image recognition model to generate a first probability vector of the sampling sample image for the A category labels, wherein the erroneous category label is different from the category label corresponding to the sample image set to which the sampling sample image belongs;
  • the first probability vector includes A probability elements, each of the A probability elements refers to one of the A category labels, and one of the probability elements is used to represent the probability that the sampling sample image belongs to the referred category label;
  • the A probability elements in the first probability vector obtain a first probability element referring to the wrong category label, and adjust the sampled sample image according to the first probability element to obtain an adversarial sample image corresponding to the sampled sample image, and the category prediction result obtained by performing category prediction on the adversarial sample image by the image recognition model is the wrong category label.
  • the processor 1001 may be configured to call a device control application stored in the memory 1005 to implement:
  • the image recognition optimization model is obtained by continuing to optimize the image recognition model according to the mixed sample training set;
  • the mixed sample training set includes A sample image sets, A category labels, correct category labels and adversarial sample images;
  • the A sample image sets correspond to one of A different category labels respectively, and the A sample image sets include collected sample images;
  • the correct category label belongs to the A category labels;
  • A is a positive integer greater than 1;
  • the image recognition model is obtained by pre-training the A sample image sets;
  • the adversarial sample image is obtained by referring to the wrong category label in the A probability elements in the first probability vector
  • the first probability vector is obtained by adjusting the sampled sample image by the first probability element of the signature, the first probability vector is generated by the image recognition model for the sampled sample image, the wrong category label is different from the category label corresponding to the sample image set to which the sampled sample image belongs, and the correct category label is the category label corresponding to the sample image set to which the sampled sample image belongs;
  • the first probability vector includes A probability
  • the category label corresponding to the maximum probability element in the target probability vector is determined as the category prediction result of the target image.
  • the present application also provides a computer-readable storage medium, which stores a computer program.
  • a computer program When the computer program is executed by a processor, the description of the data processing method or device in the above embodiments is implemented, which will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated.
  • the computer-readable storage medium may be the data processing device provided in any of the aforementioned embodiments or the internal storage unit of the computer device, such as a hard disk or memory of the computer device.
  • the computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (smart media card, SMC), a secure digital (secure digital, SD) card, a flash card (flash card), etc. equipped on the computer device.
  • the computer-readable storage medium may also include both the internal storage unit of the computer device and an external storage device.
  • the computer-readable storage medium is used to store the computer program and other programs and data required by the computer device.
  • the computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.
  • the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium.
  • the processor of the computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device can execute the description of the data processing method or device in the above embodiments, which will not be repeated here.
  • the description of the beneficial effects of the same method will not be repeated.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Image Analysis (AREA)

Abstract

提供了一种数据处理方法、设备以及计算机可读存储介质，方法包括：将A个样本图像集中的采样样本图像与A个类别标签中的错误类别标签输入至图像识别模型，以生成采样样本图像针对A个类别标签的第一概率向量，错误类别标签与采样样本图像所属的样本图像集对应的类别标签不同；根据第一概率向量中的A个概率元素中指代错误类别标签的第一概率元素对采样样本图像进行调整，得到采样样本图像对应的对抗样本图像，对抗样本图像被图像识别模型进行类别预测所得到的类别预测结果为错误类别标签。

Description

一种数据处理方法、设备以及计算机可读存储介质
本申请要求于2023年3月2日提交中国专利局、申请号为202310237098.1、申请名称为“一种数据处理方法、设备以及计算机可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及互联网技术领域，尤其涉及一种数据处理方法、设备以及计算机可读存储介质。
背景技术
在图像识别场景、图像分类场景、图像推荐场景、视频属性识别场景等业务场景下，正确确定图像或视频帧的类别标签(即正确识别图像或视频帧的类别)是至关重要的。
主流的图像识别方法主要是基于机器学习的方法，即通过训练样本集训练一个图像识别初始模型，得到图像识别模型，明显地，训练数据可以决定图像识别模型的类别标签识别准确度以及识别泛化性。在现有技术中，针对训练数据的增强方式，都是直接将随机生成的噪声叠加至训练样本图像。明显地，随机生成的噪声与训练样本图像是相互独立的数据，故叠加有随机噪声的训练样本图像中的噪声特征，与图像自身的特征是相互独立的，此时，图像识别模型很容易区分噪声特征以及图像自身的特征。因此，采用现有技术所扩增的训练样本只能扩大训练数据的数量，而不能提高训练数据的多样性，故通过现有技术所扩增的训练样本，再次对图像识别模型进行训练后，并不能提高训练好的图像识别模型的类别标签识别准确度以及识别泛化性。
发明内容
本申请实施例提供一种数据处理方法、设备以及计算机可读存储介质，可以提高类别标签识别准确度以及识别泛化性。
本申请实施例一方面提供了一种数据处理方法，包括：
获取A个样本图像集以及图像识别模型，A为大于1的正整数，所述A个样本图像集分别对应A个不同的类别标签中的一个，所述图像识别模型是根据所述A个样本图像集预训练得到的，所述A个类别标签包括错误类别标签；
在所述A个样本图像集中获取采样样本图像，将所述采样样本图像与所述错误类别标签输入至所述图像识别模型，以生成所述采样样本图像针对所述A个类别标签的第一概率向量，所述错误类别标签与所述采样样本图像所属的样本图像集对应的类别标签不同；所述第一概率向量包括A个概率元素，所述A个概率元素分别指代所述A个类别标签中的一个，一个所述概率元素用于表征所述采样样本图像属于所指代的类别标签的概率；
在所述第一概率向量中的所述A个概率元素中，获取指代所述错误类别标签的第一概率元素，根据所述第一概率元素，对所述采样样本图像进行调整，得到所述采样样本图像对应的对抗样本图像，所述对抗样本图像被所述图像识别模型进行类别预测所得到的类别预测结果为所述错误类别标签。
本申请实施例一方面提供了一种数据处理方法，包括：
获取目标图像，将所述目标图像输入至图像识别优化模型；所述图像识别优化模型是根据混合样本训练集继续对图像识别模型进行优化训练所得到的；所述混合样本训练集包括A个样本图像集、A个类别标签、正确类别标签以及对抗样本图像；所述A个样本图像集分别对应A个不同的类别标签中的一个，所述A个样本图像集包括采集样本图像；所述正确类别标签属于所述A个类别标签；A为大于1的正整数；所述图像识别模型是根据所述A个样本图像集预训练得到的；所述对抗样本图像是通过第一概率向量中的A个概率元素中指代错误类别标签的第一概率元素对采样样本图像进行调整得到，所述第一概率向量是由所述图像识别模型针对所述采样样本图像所生成的，所述错误类别标签与所述采样样本图像所属的样本图像集对应的类别标签不同，所述正确类别标签为所述采样样本图像所属的样本图像集对应的类别标签；所述第一概率向量包括A个概率元素，所述A个概率元素分别指代所述A个类别标签中的一个，一个所述概率元素用于表征所述采样样本图像属于所指代的类别标签的概率；所述对抗样本图像被所述图像识别模型进行类别预测所得到的类别预测结果为所述错误类别标签；
通过所述图像识别优化模型，生成所述目标图像针对所述A个类别标签的目标概率向量；
将所述目标概率向量中的最大概率元素对应的类别标签，确定为所述目标图像的类别预测结果。
本申请实施例一方面提供了一种数据处理装置，包括：
数据获取模块，用于获取A个样本图像集以及图像识别模型，A为大于1的正整数，所述A个样本图像集分别对应A个不同的类别标签中的一个，所述图像识别模型是根据所述A个样本图像集预训练得到的，所述A个类别标签包括错误类别标签；
第一输入模块，用于在所述A个样本图像集中获取采样样本图像，将所述采样样本图像与所述错误类别标签输入至所述图像识别模型，以生成所述采样样本图像针对所述A个类别标签的第一概率向量，所述错误类别标签与所述采样样本图像所属的样本图像集对应的类别标签不同；所述第一概率向量包括A个概率元素，所述A个概率元素分别指代所述A个类别标签中的一个，一个所述概率元素用于表征所述采样样本图像属于所指代的类别标签的概率；
第一调整模块，用于在所述第一概率向量中的所述A个概率元素中，获取指代所述错误类别标签的第一概率元素，根据所述第一概率元素，对所述采样样本图像进行调整，得到所述采样样本图像对应的对抗样本图像，所述对抗样本图像被所述图像识别模型进行类别预测所得到的类别预测结果为所述错误类别标签。
其中，第一调整模块，具体用于生成第一概率元素对应的负概率元素，对负概率元素以及第一概率向量中的最大概率元素进行求和处理，得到采样样本图像针对错误类别标签的标签损失值，根据标签损失值，对采样样本图像进行调整，得到采样样本图像对应的对抗样本图像。
其中，第一调整模块用于执行根据标签损失值，对采样样本图像进行调整，得到采样样本图像对应的对抗样本图像时，具体用于根据标签损失值，生成针对采样样本图像的初始梯度值，获取初始梯度值的数值符号，根据数值符号生成初始梯度值对应的符号单位数值，对初始学习率以及符号单位数值进行乘积处理，得到第一待裁剪数值，通过由攻击强度所生成的第一裁剪区间，对第一待裁剪数值进行裁剪处理，得到用于调整采样样本图像的梯度值，对采样样本图像以及梯度值进行求差处理，得到第二待裁剪数值，通过由像素等级所生成的第二裁剪区间，对第二待裁剪数值进行裁剪处理，得到采样样本图像对应的对抗样本图像。
其中，第一调整模块，包括：
第一调整单元，用于根据第一概率元素，对采样样本图像进行调整，得到采样样本图像对应的初始对抗样本图像；
第一生成单元，用于将所述错误类别标签和所述初始对抗样本图像输入至所述图像识别模型，通过图像识别模型，生成初始对抗样本图像针对A个类别标签的第二概率向量；
第一获取单元，用于在所述第二概率向量所包含的A个概率元素中，获取针对所述错误类别标签的第二概率元素；
第一调整单元，还用于若所述第二概率元素不满足边界约束条件，或针对所述对抗样本图像进行类别预测所得到的类别预测结果不为所述错误类别标签，则根据所述第二概率元素继续对所述初始对抗样本图像进行调整；
第一获取单元，还用于若所述第二概率元素满足边界约束条件，且针对所述对抗样本图像进行类别预测所得到的类别预测结果为所述错误类别标签，则将所述初始对抗样本图像确定为所述采样样本图像对应的对抗样本图像；在所述类别预测结果为所述错误类别标签的情况下，所述第二概率元素为所述第二概率向量所包含的A个概率元素中的最大数值。
其中，第一获取单元，还用于将所述第二概率元素以及概率约束值进行对比；所述概率约束值为A的倒数；
第一获取单元，还用于若所述第二概率元素小于所述概率约束值，则确定所述第二概率元素不满足边界约束条件；
第一获取单元，还用于若所述第二概率元素等于或大于所述概率约束值，则确定所述第二概率元素满足边界约束条件。
其中，数据处理装置，还包括：
第二输入模块，用于将所述A个样本图像集、所述A个类别标签、正确类别标签以及所述对抗样本图像均输入至所述图像识别模型；所述正确类别标签是指所述采样样本图像所属的样本图像集对应的类别标签；
第二输入模块，还用于通过图像识别模型，生成A个样本图像集分别对应的第一预测类别，以及对抗样本图像对应的第二预测类别；
第二调整模块，用于根据所述第一预测类别和所述A个类别标签生成第一损失值，根据所述第二预测 类别以及所述正确类别标签生成第二损失值，根据所述第一损失值和所述第二损失值对所述图像识别模型中的参数进行调整，得到所述图像识别优化模型；所述图像识别优化模型针对所述对抗样本图像进行类别预测所得到的类别预测结果为所述正确类别标签。
其中，所述A个样本图像集包括样本图像集Cb；b为正整数且b小于或等于A；第一输入模块，包括：
第一确定单元，用于确定采样比例，根据采样比例，对样本图像集Cb中的样本图像进行采样处理，得到采样样本图像；
第二获取单元，用于在A个类别标签中，获取与所述样本图像集Cb的类别标签不同的类别标签，将获取到的类别标签确定为错误类别标签。
其中，数据获取模块，包括：
第三获取单元，用于获取A个样本图像集，将A个类别标签和所述A个样本图像集分别输入至所述图像识别初始模型；
第二生成单元，用于通过图像识别初始模型，生成所述A个样本图像集分别对应的预测初始类别；
第二确定单元，用于根据所述预测初始类别以及所述A个类别标签，确定所述A个样本图像集分别对应的类别损失值；
第三确定单元，用于根据A个样本图像集分别对应的类别损失值，确定图像识别初始模型对应的总损失值；
第二调整单元，用于根据所述总损失值，对所述图像识别初始模型中的参数进行调整，得到图像识别模型；所述图像识别模型针对样本图像集Cb进行类别预测所得到的类别预测结果为所述样本图像集Cb的类别标签；所述A个样本图像集包括样本图像集Cb；b为正整数且b小于或等于A。
其中，第三获取单元，包括：
图像获取子单元，用于获取A个原始图像集，将A个原始图像集均输入至对象检测模型；所述A个原始图像集分别对应A个不同的类别信息中的一个；A个原始图像集包括原始图像集Db；原始图像集Db包括原始图像Ef；f为正整数，且f小于或等于原始图像集Db中原始图像的总数量；
第二确定子单元，用于通过对象检测模型，确定原始图像Ef中关键区域的区域坐标，根据区域坐标，生成原始图像Ef对应的待标注样本图像；
第三确定子单元，用于将原始图像集Db中每个原始图像对应的待标注样本图像，确定为待标注样本图像集；
第四确定子单元，用于根据原始图像集Db对应的类别信息，生成类别标签Hb，将标注有类别标签Hb的待标注样本图像集，确定为样本图像集Cb。
其中，第二确定子单元，包括：
第三处理子单元，用于根据区域坐标，生成包括关键区域的初始检测框，对初始检测框进行扩张处理，得到包括目标对象的检测框；关键区域属于目标对象；
第四处理子单元，用于在检测框中，获取包括目标对象的待缩放图像，获取第一图像尺寸，根据第一图像尺寸，对待缩放图像进行缩放处理，得到待裁剪图像；
第五处理子单元，用于获取第二图像尺寸，根据第二图像尺寸，对待裁剪图像进行裁剪处理，得到原始图像Ef对应的待标注样本图像；第二图像尺寸小于第一图像尺寸。
本申请实施例一方面提供了一种数据处理装置，包括：
图像获取模块，用于获取目标图像，将所述目标图像输入至图像识别优化模型；所述图像识别优化模型是根据混合样本训练集继续对图像识别模型进行优化训练所得到的；所述混合样本训练集包括A个样本图像集、A个类别标签、正确类别标签以及对抗样本图像；所述A个样本图像集分别对应A个不同的类别标签中的一个，所述A个样本图像集包括采集样本图像；所述正确类别标签属于所述A个类别标签；A为大于1的正整数；所述图像识别模型是根据所述A个样本图像集预训练得到的；所述对抗样本图像是通过第一概率向量中的A个概率元素中指代错误类别标签的第一概率元素对采样样本图像进行调整得到，所述第一概率向量是由所述图像识别模型针对所述采样样本图像所生成的，所述错误类别标签与所述采样样本图像所属的样本图像集对应的类别标签不同，所述正确类别标签为所述采样样本图像所属的样本图像集对应的类别标签；所述第一概率向量包括A个概率元素，所述A个概率元素分别指代所述A个类别标签中的一个，一个所述概率元素用于表征所述采样样本图像属于所指代的类别标签的概率；所述对抗样本图像被所述图像识别模型进行类别预测所得到的类别预测结果为所述错误类别标签；
向量生成模块，用于通过图像识别优化模型，生成目标图像针对A个类别标签的目标概率向量；
类别确定模块，用于将目标概率向量中的最大概率元素对应的类别标签，确定为目标图像的类别预测结果。
本申请一方面提供了一种计算机设备，包括：处理器、存储器、网络接口；
上述处理器与上述存储器、上述网络接口相连，其中，上述网络接口用于提供数据通信功能，上述存储器用于存储计算机程序，上述处理器用于调用上述计算机程序，以使得计算机设备执行本申请实施例中的方法。
本申请实施例一方面提供了一种计算机可读存储介质，上述计算机可读存储介质中存储有计算机程序，上述计算机程序适于由处理器加载并执行本申请实施例中的方法。
本申请实施例一方面提供了一种计算机程序产品，该计算机程序产品包括计算机程序，该计算机程序存储在计算机可读存储介质中；计算机设备的处理器从计算机可读存储介质读取该计算机程序，处理器执行该计算机程序，使得该计算机设备执行本申请实施例中的方法。
在本申请实施例中，计算机设备可以获取A个样本图像集以及图像识别模型；其中，所述A个样本图像集分别对应A个不同的类别标签中的一个，图像识别模型是根据A个样本图像集预训练得到的，所以图像识别模型可以准确地确定A个样本图像集中的样本图像的类别标签；进一步，计算机设备在A个样本图像集中获取采样样本图像，将错误类别标签和采样样本图像输入至图像识别模型，其中，所述错误类别标签属于所述A个类别标签，且所述错误类别标签与所述采样样本图像所属的样本图像集对应的类别标签不同；通过图像识别模型可以生成采样样本图像针对A个类别标签的第一概率向量，所述第一概率向量包括A个概率元素，所述A个概率元素分别指代所述A个类别标签中的一个，一个所述概率元素用于表征所述采样样本图像属于所指代的类别标签的概率；由于图像识别模型可以准确地确定A个样本图像集中的样本图像的类别标签，故在第一概率向量中，指代正确类别标签的概率元素最大，或者说，指代正确类别标签的概率元素远远大于指代错误类别标签的第一概率元素，所述正确类别标签是指所述采样样本图像所属的样本图像集对应的类别标签；进一步，根据第一概率元素，计算机设备可以对采样样本图像进行调整，得到采样样本图像对应的对抗样本图像，其中，所述对抗样本图像被所述图像识别模型进行类别预测所得到的类别预测结果为所述错误类别标签，即通过对采样样本图像进行调整，使得图像识别模型可以错误地确定对抗样本图像的类别标签。上述可知，本申请实施例提出一种对抗样本图像的生成方法，通过该方法，可以生成图像识别模型错误识别的对抗样本图像，该对抗样本图像可以提高用于优化训练图像识别模型的训练集的样本多样性，通过具有样本多样性的训练集，可以提高图像识别模型的优化训练准确度，进而可以提高优化训练好的图像识别模型的类别标签识别准确度以及识别泛化性。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本申请实施例提供的一种系统架构示意图；
图2a是本申请实施例提供的一种数据处理的场景示意图一；
图2b是本申请实施例提供的一种数据处理的场景示意图二；
图3是本申请实施例提供的一种数据处理方法的流程示意图一；
图4是本申请实施例提供的一种通过干净样本数据所训练的分类边界面示意图；
图5是本申请实施例提供的一种通过第一混合样本数据所训练的分类边界面示意图；
图6是本申请实施例提供的一种通过第二混合样本数据所训练的分类边界面示意图；
图7是本申请实施例提供的一种数据处理方法的流程示意图二；
图8是本申请实施例提供的一种数据处理方法的流程示意图三；
图9是本申请实施例提供的一种数据处理装置的结构示意图一；
图10是本申请实施例提供的一种数据处理装置的结构示意图二；
图11是本申请实施例提供的一种计算机设备的结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
为了便于理解，首先对部分名词进行以下简单解释：
人工智能(Artificial Intelligence，AI)，是利用数字计算机或者数字计算机控制的机器模拟、延伸和扩展人的智能，感知环境、获取知识并使用知识获得最佳结果的理论、方法、技术及应用系统。换句话说，人工智能是计算机科学的一个综合技术，它企图了解智能的实质，并生产出一种新的能以人类智能相似的方式做出反应的智能机器。人工智能也就是研究各种智能机器的设计原理与实现方法，使机器具有感知、推理与决策的功能。
人工智能技术是一门综合学科，涉及领域广泛，既有硬件层面的技术也有软件层面的技术。人工智能基础技术一般包括如传感器、专用人工智能芯片、云计算、分布式存储、大数据处理技术、操作/交互系统、机电一体化等技术。人工智能软件技术主要包括计算机视觉技术、语音处理技术、自然语言处理技术以及机器学习/深度学习、自动驾驶、智慧交通等几大方向。
计算机视觉技术(Computer Vision，CV)是一门研究如何使机器“看”的科学，更进一步的说，就是指用摄影机和电脑代替人眼对目标进行识别和测量等机器视觉，并进一步做图形处理，使电脑处理成为更适合人眼观察或传送给仪器检测的图像。作为一个科学学科，计算机视觉研究相关的理论和技术，试图建立能够从图像或者多维数据中获取信息的人工智能系统。计算机视觉技术通常包括图像处理、图像识别、图像语义理解、图像检索、光学字符识别(Optical Character Recognition，OCR)、视频处理、视频语义理解、视频内容/行为识别、三维物体重建、3D技术、虚拟现实、增强现实、同步定位与地图构建、自动驾驶、智慧交通等技术。在本申请实施例中，计算机视觉技术可以用于识别图像中的类别标签(例如人、猫、狗，或，真人、纸片人、模具人等)。
机器学习(Machine Learning，ML)是一门多领域交叉学科，涉及概率论、统计学、逼近论、凸分析、算法复杂度理论等多门学科。专门研究计算机怎样模拟或实现人类的学习行为，以获取新的知识或技能，重新组织已有的知识结构使之不断改善自身的性能。机器学习是人工智能的核心，是使计算机具有智能的根本途径，其应用遍及人工智能的各个领域。机器学习和深度学习通常包括人工神经网络、置信网络、强化学习、迁移学习、归纳学习、式教学习等技术。在本申请实施例中，图像识别模型以及对象检测模型均是基于机器学习的AI模型，图像识别模型可用于对图像进行识别处理，对象检测模型可用于检测图像中目标对象的关键区域。
请参见图1，图1是本申请实施例提供的一种系统架构示意图。如图1所示，该系统可以包括业务服务器100以及终端设备集群，终端设备集群可以包括：终端设备200a、终端设备200b、终端设备200c、…、终端设备200n，可以理解的是，上述系统可以包括一个或者多个终端设备，本申请不对终端设备的数量进行限制。
其中，终端设备集群之间可以存在通信连接，例如终端设备200a与终端设备200b之间存在通信连接，终端设备200a与终端设备200c之间存在通信连接。同时，终端设备集群中的任一终端设备可以与业务服务器100存在通信连接，例如终端设备200a与业务服务器100之间存在通信连接，其中，上述通信连接不限定连接方式，可以通过有线通信方式进行直接或间接地连接，也可以通过无线通信方式进行直接或间接地连接，还可以通过其它方式，本申请在此不做限制。
应当理解，如图1所示的终端设备集群中的每个终端设备均可以安装有应用客户端，当该应用客户端运行于各终端设备中时，可以分别与上述图1所示的业务服务器100进行数据交互，即上述的通信连接。其中，该应用客户端可以为视频应用、直播应用、社交应用、即时通信应用、游戏应用、音乐应用、购物应用、小说应用、浏览器等具有识别目标图像功能的应用客户端。
其中，该应用客户端可以为独立的客户端，也可以为集成在某客户端(例如，社交客户端、教育客户端以及多媒体客户端等)中的嵌入式子客户端，在此不做限定。以购物应用为例，业务服务器100可以为包括购物应用对应的后台服务器、数据处理服务器等多个服务器的集合，因此，每个终端设备均可以通过该购物应用对应的应用客户端，与业务服务器100进行数据传输，如每个终端设备均可以通过购物应用的应用客户端，将目标图像上传至业务服务器100，进而业务服务器100可以确定目标图像的类别标签。
可以理解的是，在本申请的具体实施方式中，涉及到用户信息(例如目标图像)等相关的数据，当本申请中的实施例运用到具体产品或技术中时，需要获得用户许可或者同意，且相关数据的收集、使用和处理需要遵守相关国家和地区的相关法律法规和标准。
为便于后续理解和说明，本申请实施例可以在图1所示的终端设备集群中选择一个终端设备作为目标终端设备，例如以终端设备200a作为目标终端设备。当获取到图像识别模型，并接收到针对图像识别模型的模型优化指令时，终端设备200a可以通过应用客户端，发送针对图像识别模型的模型优化请求至业务服务器100。其中，图像识别模型是指用于识别图像并确定图像的类别信息的神经网络模型。
进一步，业务服务器100接收到终端设备200a发送的模型优化请求后，业务服务器100可以获取A个样本图像集以及图像识别模型，其中，A个样本图像集即多个样本图像集，A个样本图像集分别对应不同的类别标签，因此，类别标签的数量也可以为A个，即A个样本图像集分别对应A个不同的类别标签中的一个。可以理解的是，一个类别标签用于指示一个类别信息，故A个样本图像集分别对应的类别信息互不相同，本申请实施例不对类别信息进行限制，可以根据实际应用场景进行设定。可选的，类别信息可以是对象信息，此时，A个样本图像集分别对应的对象信息互不相同，例如，A个样本图像集中的第一样本图像集的对象信息是猫(即第一样本图像集中的每个样本图像均与猫相关)，A个样本图像集中的第二个样本图像集的对象信息是狗(即第二样本图像集中的每个样本图像均与狗相关)。可选的，类别信息可以是对象载体信息，此时，A个样本图像集分别对应的对象信息可以相同，也可以不相同，但A个样本图像集分别对应的对象载体信息互不相同，例如在关键特征识别场景中，A个样本图像集中的第三样本图像集的对象载体信息是翻拍屏幕，A个样本图像集中的第四样本图像集的对象载体信息是关键特征模具，A个样本图像集中的第五样本图像集的对象载体信息是纸片。
本申请实施例不对业务服务器100获取A个样本图像集以及图像识别模型的方式进行限定，可以根据实际应用场景进行设定，例如终端设备200a发送的模型优化请求中携带A个样本图像集或图像识别模型，或者同时携带A个样本图像集和图像识别模型，例如从数据库中获取A个样本图像集和图像识别模型，例如从区块链网络中获取A个样本图像集和图像识别模型。此外，图像识别模型是根据A个样本图像集预训练得到的，此处暂不对图像识别模型的生成过程展开描述，请参见下文图3所对应的实施例中步骤S101的的描述。
为了便于叙述以及理解，本申请实施例以A个样本图像集中的样本图像集C1为例进行叙述，可以理解的是，A个样本图像集中的剩余样本图像集的处理过程，与下文描述的样本图像集C1的处理过程相同。业务服务器100在样本图像集C1中获取采样样本图像(可以随机获取或者通过轮询的方式获取)，另外再在A个类别标签中获取错误类别标签，错误类别标签属于A个类别标签中除了样本图像集C1的类别标签之外的类别标签(可以称为剩余类别标签)，具体可以将在剩余类别标签中随机获取到的类别标签确定为错误类别标签，例如错误类别标签为样本图像集C2的类别标签。进一步可以将错误类别标签和采样样本图像输入至图像识别模型，其中，错误类别标签和采样样本图像可以分别作为2个相互独立的数据输入图像识别模型，且错误类别标签和采样样本图像之间存在映射关系；或者，可以将错误类别标签添加到采样样本图像的文件头或文件尾中，进而可以将携带错误类别标签的采样样本图像输入图像识别模型。进一步，业务服务器100通过图像识别模型，可以生成采样样本图像针对A个类别标签的第一概率向量；所述第一概率向量包括A个概率元素，所述A个概率元素分别指代所述A个类别标签中的一个，一个所述概率元素用于表征所述采样样本图像属于所指代的类别标签的概率。可以理解的是，第一概率向量的向量维度等于A，即一个向量维度上的元素(本申请实施例也称为概率元素)，可以表征一个类别标签的预测概率值，即某个概率元素的值越大，则表示采样样本图像属于该概率元素所指代的类别标签的可能性越大。因此，通常可以将第一概率向量中数值最大的概率元素所指代的类别标签确定为采用样本图像的类别预测结果。故在第一概率向量中，业务服务器100可以获取指代错误类别标签的第一概率元素，进一步，根据第一概率元素，可以对采样样本图像进行调整，得到采样样本图像对应的对抗样本图像。其中所述对抗样本图像被所述图像识别模型进行类别预测所得到的类别预测结果为所述错误类别标签，即图像识别模型无法准确地识别对抗样本图像，进而得到一个错误的类别分类。例如采样样本图像为一张包含猫的图像，其类别标签是猫对应的标签，通过对携带狗对应的标签(即错误类别标签)的采样样本图像进行调整，得到其对应的对抗样本图像，图像识别模型的优化人员可以准确地判断该对抗样本图像为一张包含猫的图像，但图像识别模型会误判断该对抗样本图像为一张包含狗的图像。
通过上述过程，业务服务器100可以生成对抗样本图像，后续，通过A个样本图像集、正确类别标签以及对抗样本图像，可以继续对图像识别模型进行优化训练，得到图像识别优化模型。其中，正确类别标签是指样本图像集C1的类别标签。可以理解的是，由于训练图像识别模型时，是通过针对对抗样本图像的预测类别标签和正确类别标签来计算模型损失，故训练好的图像识别优化模型，可以准确地识别该对抗样本图像，进而可以准确地识别出对抗样本图像的预测类别结果是正确类别标签，因此，图像识别优化模型可以具体更强的抗干扰能力。
后续，业务服务器100可以生成针对图像识别优化模型的模型优化完成消息，将模型优化完成消息发送至终端设备200a，终端设备200a接收到业务服务器100发送的模型优化完成消息后，可以获取图像识别优化模型，本申请实施例不对终端设备200a获取图像优化识别模型的方式进行限定，可以根据实际应 用场景进行设定，例如模型优化完成消息中携带图像优化识别模型，例如从数据库中获取图像识别优化模型，例如从区块链网络中获取图像识别优化模型。
可选的，若终端设备200a的本地存储了上述图像识别模型以及A个样本图像集，且终端设备200a具有离线计算能力，则终端设备200a在接收到针对图像识别模型的模型优化指令时，可以在本地获取样本图像集C1中的样本图像，将获取到的样本图像作为采样样本图像。终端设备200a将错误类别标签和采样样本图像输入至图像识别模型，后续处理过程与上述描述的过程相同，故此处不进行赘述。
可以理解的是，本申请实施例对采样样本图像赋予错误类别标签，并通过图像识别模型，获取指代错误类别标签的第一概率元素，通过第一概率元素对采样样本图像进行调整，得到可让图像识别模型错误预测类别的对抗样本图像。本申请实施例通过对抗样本图像，对A个样本图像集(即训练数据)进行训练样本的扩增以及增强，故可以提升用于优化训练图像识别模型的样本训练集的丰富性，从而提升图像识别优化模型的识别准确度。
需要说明的是，上述业务服务器100、终端设备200a、终端设备200b、终端设备200c...、终端设备200n均可以为区块链网络中的区块链节点，全文叙述的数据(例如A个样本图像集以及图像识别模型)可以进行存储，存储方式可以是区块链节点根据数据生成区块，并将区块添加至区块链中进行存储的方式。
区块链是一种分布式数据存储、点对点传输、共识机制以及加密算法等计算机技术的新型应用模式，主要用于对数据按时间顺序进行整理，并加密成账本，使其不可被篡改和伪造，同时可进行数据的验证、存储和更新。区块链本质上是一个去中心化的数据库，该数据库中的每个节点均存储一条相同的区块链，区块链网络可以将节点区分为核心节点、数据节点以及轻节点。核心节点、数据节点以及轻节点共同组成区块链节点。其中核心节点负责区块链全网的共识，也就是说核心节点为区块链网络中的共识节点。对于区块链网络中的交易数据被写入账本的流程可以为，区块链网络中的数据节点或轻节点获取到交易数据，将交易数据在区块链网络中传递(也就是节点以接力棒的方式进行传递)，直到共识节点收到该交易数据，共识节点再将该交易数据打包进区块，对该区块执行共识，待共识完成后将该交易数据写入账本。此处以A个样本图像集以及图像识别模型示例交易数据，业务服务器100(区块链节点)在通过对交易数据的共识后，根据交易数据生成区块，将区块存储至区块链网络中；而对于交易数据(即A个样本图像集以及图像识别模型)的读取，则可以由区块链节点在区块链网络中，获取到包含该交易数据的区块，进一步，在区块中获取交易数据。
可以理解的是，本申请实施例提供的方法可以由计算机设备执行，计算机设备包括但不限于终端设备或业务服务器。其中，业务服务器可以是独立的物理服务器，也可以是多个物理服务器构成的服务器集群或者分布式系统，还可以是提供云数据库、云服务、云计算、云函数、云存储、网络服务、云通信、中间件服务、域名服务、安全服务、CDN、以及大数据和人工智能平台等基础云计算服务的云服务器。终端设备包括但不限于手机、电脑、智能语音交互设备、智能家电、车载终端、飞行器等。其中，终端设备和业务服务器可以通过有线或无线方式进行直接或间接地连接，本申请实施例在此不做限制。
进一步地，请参见图2a，图2a是本申请实施例提供的一种数据处理的场景示意图一。本申请实施例可应用于各种场景，包括但不限于云技术、人工智能、智慧交通、辅助驾驶、音视频等。本申请实施例可适用于图像搜索场景、图像生成场景、图像推荐场景、图像分发场景等业务场景，这里将不对具体的业务场景进行一一列举。其中，该数据处理场景的实现过程可以在业务服务器中进行，也可以在终端设备中进行，还可以在终端设备和业务服务器中交互进行，此处不做限制。其中，终端设备可以为上述图1所对应实施例的终端设备集群中的任意一个终端设备，业务服务器可以为上述图1所对应实施例的业务服务器100。为了便于叙述以及理解，本申请实施例以在业务服务器中进行为例进行叙述。
如图2a所示，业务服务器100可以获取A个样本图像集20a以及图像识别模型20d，其中，A为大于1的正整数，本申请实施例为了便于叙述以及理解，将A示例为4，即样本图像集的数量为4个，如图2a中所示例的样本图像集201a、样本图像集202a、样本图像集203a以及样本图像集204a(样本图像集201a、样本图像集202a、样本图像集203a以及样本图像集204a分别包含的样本图像的数量可以相同，也可以不相同)。4个样本图像集分别对应的类别信息互不相同，如图2a中所示例的样本图像集201a的类别信息为萝卜，样本图像集202a的类别信息为辣椒，样本图像集203a的类别信息为南瓜，样本图像集204a的类别信息为洋葱。因此，业务服务器100可以为上述4个样本图像集分别设置不同的类别标签，将萝卜的类别标签设置为0，即样本图像集201a的类别标签为0，将辣椒的类别标签设置为1，即样本图像集202a的类别标签为1，将南瓜的类别标签设置为2，即样本图像集203a的类别标签为2，将洋葱的类别标签设置为3，即样本图像集204a的类别标签为3。
可以理解的是，本申请实施例是为了叙述以及理解，以萝卜、辣椒、南瓜以及洋葱示例类别信息，而实际应用时，类别信息应当根据实际场景进行设定。同理，本申请实施例是为了叙述以及理解，以0-3数值示例类别标签，而实际应用时，类别标签应当根据实际场景进行设定。
进一步，业务服务器100在上述4个样本图像集中分别获取采样样本图像，如图2a所示例，在样本图像集201a中获取采样样本图像201b，在样本图像集202a中获取采样样本图像202b，在样本图像集203a中获取采样样本图像203b，在样本图像集204a中获取采样样本图像204b。本申请实施例不对采样样本图像201b的总数量进行限定，可以为一个或多个，同理，本申请实施例不对采样样本图像202b、采样样本图像203b以及采样样本图像204b分别对应的总数量进行限定，均可以为一个或多个。
业务服务器100将采样样本图像201b、采样样本图像202b、采样样本图像203b以及采样样本图像204b均输入至图像识别模型20d，其中，图像识别模型20d是根据上述4个样本图像集所预训练得到的。可以理解的是，业务服务器100通过图像识别模型20d，对每个采样样本图像的处理过程均相同，故下文以一个采样样本图像(如图2a所示例的采样样本图像201c)为例进行描述，剩余的采样样本图像的处理过程，均可以参见下文采样样本图像201c的处理过程。
采样样本图像201c和错误类别标签可以输入至图像识别模型20d，如图2a所示例，采样样本图像201c是一张类别信息为萝卜的图像，即采样样本图像201c属于样本图像集201a。可以理解的是，采样样本图像201c的正确类别标签为萝卜的类别标签，即0，图2a以辣椒的类别标签(即1)示例采样样本图像201c的错误类别标签(错误类别标签可以通过在辣椒、南瓜以及洋葱中随机选择得到)。需要说明的是，采样样本图像201c的错误类别标签是随机生成的。
进一步，业务服务器100通过图像识别模型20d，可以生成采样样本图像201c针对4个类别标签的第一概率向量201e，可以理解的是，第一概率向量201e的维度与类别标签的总数量相同，且每个维度上的元素值，可以表示对应的类别标签的预测概率值。故在图2a中，第一概率向量201e中的第一个维度上的0.9可以表示类别标签0的预测概率，第二个维度上的0.02可以表示类别标签1的预测概率，第三个维度上的0.03可以表示类别标签2的预测概率；第四个维度上的0.05可以表示类别标签3的预测概率，这4个维度上的预测概率也可以称之为4个概率元素。通过第一概率向量201e，图像识别模型20d可以确定指代类别标签0的预测概率的数值最大，因此可以确定采样样本图像201c的类别预测结果是类别标签0，这与输入图像识别模型20d的错误类别标签(即1)不相同。产生这样的结果是因为，图像识别模型20d是通过图2a中的样本图像集201a、样本图像集202a、样本图像集203a以及样本图像集204a所预训练得到的，且采样样本图像201c来自样本图像集201a，故图像识别模型20d识别采样样本图像201c的类别预测结果为类别标签0。
进一步，业务服务器100通过图像识别模型20d可以在第一概率向量201e中获取指代错误类别标签(如图2a中所示例的类别标签1)的第一概率元素，如图2a中所示例的0.02。业务服务器100可以根据第一概率元素对采样样本图像201c进行调整，得到采样样本图像201c对应的对抗样本图像。
其中，根据第一概率元素，对采样样本图像201c进行调整，得到采样样本图像201c对应的对抗样本图像的具体过程可以包括：业务服务器100可以根据第一概率元素对采样样本图像201c进行调整，得到采样样本图像201c对应的初始对抗样本图像。请一并参见图2b，图2b是本申请实施例提供的一种数据处理的场景示意图二。如图2b所示，业务服务器100生成第一概率元素(如图2a中所示例的0.02)对应的负概率元素，即图2b中的-0.02，对负概率元素以及第一概率向量201e中的最大概率元素(如图2a以及图2b所示例的0.9)进行求和处理，得到采样样本图像201c针对错误类别标签的标签损失值，即图2b中的0.88。进一步，业务服务器100根据标签损失值，对采样样本图像201c进行调整，得到采样样本图像201c对应的初始对抗样本图像202c。其中，根据标签损失值，对采样样本图像201c进行调整，得到采样样本图像201c对应的初始对抗样本图像202c的具体过程，本申请实施例暂不展开描述，请参见下文图3所对应的实施例中步骤S103中的描述。
请再参见图2b，业务服务器100将错误类别标签(如图2a以及图2b所示例的类别标签1)和初始对抗样本图像202c，输入至图像识别模型20d。业务服务器100可以通过图像识别模型20d生成初始对抗样本图像202c针对上述4个类别标签的第二概率向量202e。其中，第二概率向量202e的含义与第一概率向量201e的含义相同，故此处不对第二概率向量202e的含义进行解释，请参见上文第一概率向量201e的含义描述。
进一步，业务服务器100在第二概率向量202e中，获取指代错误类别标签(即图2a以及图2b中所示例的类别标签1)的第二概率元素，如图2b中所示例的0.08。业务服务器100可以根据第二概率元素， 确定采样样本图像201c对应的对抗样本图像。即业务服务器100可以分析第二概率元素是否为第二概率向量202e中具有最大数值的概率元素，若是，则可以将初始对抗样本图像202c确定为采样样本图像201c对应的对抗样本图像(即此时图像识别模型20d能够错误的识别出该对抗样本图像的类别预测结果为错误类别标签1)；若分析出第二概率元素并不是第二概率向量202e中具有最大数值的概率元素，则会进一步对初始对抗样本图像202c进行调整，其调整目的就是希望图像识别模型20d输出针对调整后的对初始对抗样本图像的第三概率向量中指代错误类别标签的概率元素变大，直至图像识别模型20d能够错误的将多轮调整后的对抗样本图像识别为错误类别标签1。
可以理解的是，图像识别模型20d针对对抗样本图像的类别预测结果为错误类别标签1，即对抗样本图像仍然为一张包含萝卜的图像，但图像识别模型20d会错误地将对抗样本图像的类别预测结果识别为辣椒的类别标签1。后续，业务服务器100可以通过正确类别标签(即萝卜的类别标签0)和对抗样本图像，继续对图像识别模型20d进行优化训练，得到图像识别优化模型。可以理解的是，图像优化识别模型(即优化训练好的图像识别模型)可以准确地识别出对抗样本图像的类别预测结果为类别标签0，即萝卜的类别标签。
上述可知，本申请实施例提出一种对抗样本图像的生成方法，通过该方法，可以生成图像识别模型错误识别的对抗样本图像，通过对抗样本图像，可以提高用于优化训练图像识别模型的训练集的样本多样性，通过具有样本多样性的训练集，可以提高图像识别模型的优化训练准确度，进而可以提高优化训练好的图像识别模型的类别标签识别准确度以及识别泛化性。而且对抗样本图像相比于采样样本图像是一种具有强扰动的图像(即会导致图像识别模型错误识别的图像)，而图像优化识别模型能够克服该强扰动，即图像优化识别模型通过更强的抗干扰能力能够准确识别出对抗样本图像的类别预测结果为正确的类别标签(即类别标签0)。
进一步地，请参见图3，图3是本申请实施例提供的一种数据处理方法的流程示意图一。该数据处理方法可以由业务服务器(例如，上述图1所示的业务服务器100)执行，也可以由终端设备(例如，上述图1所示的终端设备200a)执行，还可以由业务服务器和终端设备交互执行。为便于理解，本申请实施例以该方法由业务服务器执行为例进行说明。如图3所示，该数据处理方法至少可以包括以下步骤S101-步骤S103。
步骤S101，获取A个样本图像集以及图像识别模型，A为大于1的正整数，所述A个样本图像集分别对应A个不同的类别标签中的一个，所述图像识别模型是根据所述A个样本图像集预训练得到的，所述A个类别标签包括错误类别标签。
具体的，业务服务器获取A个样本图像集，A个样本图像集分别对应的类别信息互不相同，例如某个样本图像集的类别信息为萝卜(即该样本图像集中的每个样本图像均与萝卜相关)，另一个样本图像集的类别信息为辣椒(即该样本图像集中的每个样本图像均与辣椒相关)。因此，业务服务器可以为每个样本图像集分别设置不同的类别标签，如将萝卜的类别标签设置为0，即类别信息为萝卜的样本图像集的类别标签为0；又如将辣椒的类别标签设置为1，即类别信息为辣椒的样本图像集的类别标签为1。
业务服务器将A个样本图像集分别输入至图像识别初始模型；通过图像识别初始模型，生成每个样本图像集分别对应的预测初始类别；根据每个样本图像集分别对应的预测初始类别以及每个样本图像集分别对应的类别标签，确定每个样本图像集分别对应的类别损失值；根据A个样本图像集分别对应的类别损失值，确定图像识别初始模型对应的总损失值；根据总损失值，对图像识别初始模型中的参数进行调整，得到图像识别模型；图像识别模型具有准确识别A个样本图像集的类别预测结果的能力，如图像识别模型可以准确识别出类别信息为萝卜的样本图像集的类别预测结果为类别标签为0(该类别预测结果也可以理解为是针对该样本图像集中每个样本图像的类别预测结果)。
本申请实施例不对样本图像集中的样本图像的图像数量进行限定，可以为一个或多个，可以理解的是，同一个样本图像集中的样本图像对应的类别信息是相同的，故同一个样本图像集中的样本图像对应的类别标签是相同的。本申请实施例也不对样本图像集的数量进行限定，可以为两个或两个以上。
其中，本申请实施例中的类别标签是指用于标识类别信息的标签，本申请实施例不对类别信息进行限定，可以根据实际应用场景进行设定。例如，类别信息可以是对象信息，此时，每个样本图像集分别包含的对象信息不同，例如猫、狗、车等。例如，类别信息可以是对象载体信息，此时，每个样本图像分别包含的对象载体信息不同，例如面具对象、模具对象、照片对象、纸质对象、真实对象等。
本申请实施例不对图像识别初始模型的模型类型进行限定，可以由任意一种或多种神经网络组成，如卷积神经网络(Convolutional Neural Networks，CNN)、残差网络(Residual Network，ResNet)、基于 扩展通道数学习机制的卷积神经网络(Wide Residual Networks，Wide-ResNet)、高分辨率网络(High-Resolution Net，HRNet)等。
业务服务器对图像识别初始模型进行训练的具体过程可以为：将A个样本图像集分别输入至图像识别初始模型，可以理解的是，图像识别初始模型对A个样本图像集中的每个样本图像的处理过程相同，故下文以携带第一类别标签的第一样本图像为例进行描述，剩余样本图像的处理过程请参见下文的描述。其中，第一样本图像为A个样本图像集中的任意一个样本图像，若第一样本图像属于样本图像集C1，则第一类别标签为样本图像集C1对应的类别标签；若第一样本图像属于样本图像集C2，则第一类别标签为样本图像集C2对应的类别标签；其他情况如上描述理解，不一一赘述。业务服务器可以通过图像识别初始模型，获取第一样本图像对应的第一初始概率向量，第一初始概率向量的维度与A相同(即一个维度对应一个类别标签)，第一初始概率向量中的每个维度上的元素值(也可以称之为概率元素)，均表示对应类别标签的预测概率，且每个维度上的元素值的总和为1。第一初始概率向量中最大的元素值所对应的类别标签，即第一样本图像对应的预测初始类别，可以理解的是，在训练初期，第一样本图像对应的预测初始类别可能与第一类别标签不同，故业务服务器根据第一样本图像对应的预测初始类别以及第一类别标签，可以生成第一样本图像对应的类别损失值。通过上述过程，业务服务器可以获取A个样本图像集中每个样本图像分别对应的类别损失值，将获取到的类别损失值进行求和处理，得到图像识别初始模型对应的总损失值。进一步，业务服务器根据总损失值，对图像识别初始模型中的参数进行调整，得到完成预训练的图像识别模型，该图像识别模型能够准确识别A个样本图像集中每个样本图像的类别预测结果，如图像识别模型可以准确识别出第一样本图像的类别预测结果为第一类别标签。本申请实施例不对图像识别初始模型中的参数的调整方式进行限定，可以根据实际应用场景进行设定。此外，本申请实施例不对图像识别初始模型的迭代(epoch)次数进行限定，可以根据实际应用场景进行设定，也不对模型总损失阈值进行限定，可以根据实际应用场景进行设定。
步骤S102，在所述A个样本图像集中获取采样样本图像，将所述采样样本图像与所述错误类别标签输入至所述图像识别模型，以生成所述采样样本图像针对所述A个类别标签的第一概率向量，所述错误类别标签与所述采样样本图像所属的样本图像集对应的类别标签不同；所述第一概率向量包括A个概率元素，所述A个概率元素分别指代所述A个类别标签中的一个，一个所述概率元素用于表征所述采样样本图像属于所指代的类别标签的概率。
具体的，以A个样本图像集中的样本图像集业务服务器Cb为例进行说明，b为正整数且b小于或等于A。业务服务器可以确定采样比例，根据采样比例，对样本图像集Cb中的样本图像进行采样处理，得到采样样本图像；在A个类别标签中，获取与所述样本图像集Cb的类别标签不同的类别标签(可以通过随机方式获取)，将获取到的类别标签确定为错误类别标签。
请再参见图2a，业务服务器100在上述4个样本图像集中分别获取采样样本图像，其中，一种可行的采样方式如下：确定采样比例，例如5％，则业务服务器100在样本图像集201a中进行图像随机采样，得到样本图像集201a中5％的样本图像，将其组成采样样本图像201b，例如样本图像集201a中的样本图像为10000张，则采样样本图像201b的数量为500张。其他的样本图像集对应的采样样本图像(包括图2a中的采样样本图像202b、采样样本图像203b以及采样样本图像204b)的生成过程，可以参见采样样本图像201b的生成过程，此处不进行赘述。
为了便于叙述以及理解，本申请实施例示例图2a中的采样样本图像201b包括第一采样样本图像以及第二采样样本图像，可以理解的是，第一采样样本图像以及第二采样样本图像均为包含萝卜的样本图像，故第一采样样本图像以及第二采样样本图像分别对应的正确类别标签，均为萝卜的类别标签，即图2a中所示例的类别标签0。本申请实施例示例图2a中的采样样本图像202b包括第三采样样本图像以及第四采样样本图像，可以理解的是，第三采样样本图像以及第四采样样本图像均为包含辣椒的样本图像，故第三采样样本图像以及第四采样样本图像分别对应的正确类别标签，均为辣椒的类别标签，即图2a中所示例的类别标签1。
业务服务器100在图2a所示例的类别标签1，类别标签2以及类别标签3中，随机抽取一个类别标签，例如类别标签1，将其设置为第一采样样本图像的错误类别标签。同理，在图2a所示例的类别标签1，类别标签2以及类别标签3中，业务服务器100随机抽取一个类别标签，例如类别标签2，将其设置为第二采样样本图像的错误类别标签。业务服务器100在图2a所示例的类别标签0，类别标签2以及类别标签3中，随机抽取一个类别标签，例如类别标签2，将其设置为第三采样样本图像的错误类别标签。同理，在图2a所示例的类别标签0，类别标签2以及类别标签3中，业务服务器100随机抽取一个类别标签，例如 类别标签0，将其设置为第四采样样本图像的错误类别标签。
可以理解的是，每个采样样本图像的错误类别标签的设置过程均相同，故此处不一一进行赘述，请参见上述第一采样样本图像、第二采样样本图像、第三采样样本图像以及第四采样样本图像的错误类别标签的设置过程。
若在实际应用时，图像识别模型用于识别真实对象(例如人)，则本申请实施例还可以通过如下过程为采样样本图像设置错误类别标签。为了便于叙述以及理解，将A示例为3，即3个样本图像集，分别为样本图像集3a、样本图像集3b以及样本图像集3c，样本图像集3a的类别信息为真实对象，其类别标签为30a；样本图像集3b的类别信息是纸片对象，其类别标签为30b，样本图像集3c的类别信息是面具对象，其类别标签为30c。此时，样本图像集3b以及样本图像集3c，可以视为样本图像集3a的攻击样本图像集。
业务服务器获取样本图像集3a、样本图像集3b以及样本图像集3c分别对应的采样样本图像的过程，与上述描述的过程相同，故此处不进行赘述。假设样本图像集3a的采样样本图像包括第五采样样本图像以及第六采样样本图像，样本图像集3b的采样样本图像包括第七采样样本图像以及第八采样样本图像，样本图像集3c的采样样本图像包括第九采样样本图像以及第十采样样本图像。业务服务器100在类别标签30b以及类别标签30c中，随机抽取一个类别标签，例如类别标签30b，将其设置为第五采样样本图像的错误类别标签。同理，在类别标签30b以及类别标签30c中，业务服务器100随机抽取一个类别标签，例如类别标签30c，将其设置为第六采样样本图像的错误类别标签。业务服务器将第七采样样本图像、第八采样样本图像、第九采样样本图像以及第十采样样本图像分别对应的错误类别标签，均设置为类别标签30a。故本申请实施例提供了两种错误类别标签的设置方法，实际应用时，可以根据实际需要，在上述两种设置方法中选择一种。
进一步，业务服务器将A个样本图像集分别对应的采样样本图像，均输入至图像识别模型。可以理解的是，图像识别模型对每个采样样本图像的处理过程均相同，故请参见上文图2a所对应的实施例中采样样本图像201c的相关描述。
可以理解的是，图像识别模型可以得到每个采样样本图像分别对应的第一概率向量，每个第一概率向量均包括A个概率值(也可以称之为概率元素)。由于图像识别模型是通过A个样本图像集所预训练得到的，因此图像识别模型对于每个采样样本图像都能输出正确的概率分布(正确的概率分布指示所输出的第一概率向量中指代正确的类别标签的概率值最大)。为了便于理解，请再参见图2a，针对采样样本图像201b中的每个采样样本图像(包括采样样本图像201c)，图像识别模型20d输出的第一概率向量(包括图2a所示例的第一概率向量201e)的第一个元素(也可以称之为概率元素，该概率元素用于指代类别标签0，该概率元素用于表征所述采样样本图像属于所指代的类别标签0的概率)，可以无限接近于1，而剩余三个元素会无限接近于0。针对采样样本图像202b中的每个采样样本图像，图像识别模型20d输出的第一概率向量的第二个元素，可以无限接近于1，而剩余三个元素(具体为第一个元素、第三个元素以及第四个元素)会无限接近于0。针对采样样本图像203b中的每个采样样本图像，图像识别模型20d输出的第一概率向量的第三个元素，可以无限接近于1，而剩余三个元素(具体为第一个元素、第二个元素以及第四个元素)会无限接近于0。针对采样样本图像204b中的每个采样样本图像，图像识别模型20d输出的第一概率向量的第四个元素，可以无限接近于1，而剩余三个元素(具体为第一个元素、第二个元素以及第四个元素)会无限接近于0。
步骤S103，在所述第一概率向量中的所述A个概率元素中，获取指代所述错误类别标签的第一概率元素，根据所述第一概率元素，对所述采样样本图像进行调整，得到所述采样样本图像对应的对抗样本图像，所述对抗样本图像被所述图像识别模型进行类别预测所得到的类别预测结果为所述错误类别标签。
具体的，业务服务器根据第一概率元素，对采样样本图像进行调整，得到采样样本图像对应的初始对抗样本图像；将错误类别标签和初始对抗样本图像输入至图像识别模型，其中，错误类别标签和初始对抗样本图像可以分别作为2个相互独立的数据输入图像识别模型，且错误类别标签和初始对抗样本图像之间存在映射关系；或者，可以将错误类别标签添加到初始对抗样本图像的文件头或文件尾中，进而可以将携带错误类别标签的初始对抗样本图像输入图像识别模型。通过图像识别模型，生成初始对抗样本图像针对A个类别标签的第二概率向量，该第二概率向量同样包含A个概率元素，所述A个概率元素分别指代所述A个类别标签中的一个，一个所述概率元素用于表征初始对抗样本图像属于所指代的类别标签的概率；在第二概率向量中，获取指代错误类别标签的第二概率元素，根据第二概率元素，确定采样样本图像对应的对抗样本图像。确定对抗样本图像的其中一种方式：业务服务器可以分析第二概率元素是否为第二概率向 量中具有最大数值的概率元素，若是，则可以将初始对抗样本图像确定为采样样本图像对应的对抗样本图像(即此时图像识别模型能够错误的识别出该对抗样本图像的类别预测结果为错误类别标签)；若分析出第二概率元素并不是第二概率向量中具有最大数值的概率元素，则会进一步对初始对抗样本图像进行调整，其调整目的就是希望图像识别模型输出针对调整后的对初始对抗样本图像的第三概率向量中指代错误类别标签的概率元素变大，直至图像识别模型能够错误的将多轮调整后的对抗样本图像识别为错误类别标签。
其中，根据第一概率元素，对采样样本图像进行调整，得到采样样本图像对应的初始对抗样本图像的具体过程可以包括：生成第一概率元素对应的负概率元素，对负概率元素以及第一概率向量中的最大概率元素进行求和处理，得到采样样本图像针对错误类别标签的标签损失值；根据标签损失值，对采样样本图像进行调整，得到采样样本图像对应的初始对抗样本图像。其中，若初始对抗样本图像能够直接作为采样样本图像对应的对抗样本图像，则也可以理解为根据标签损失值，对采样样本图像进行调整，直接得到采样样本图像对应的对抗样本图像。
其中，根据标签损失值，对采样样本图像进行调整，得到采样样本图像对应的初始对抗样本图像的具体过程可以包括：根据标签损失值，生成针对采样样本图像的初始梯度值，获取初始梯度值的数值符号，根据数值符号生成初始梯度值对应的符号单位数值；对初始学习率以及符号单位数值进行乘积处理，得到第一待裁剪数值，通过由攻击强度所生成的第一裁剪区间，对第一待裁剪数值进行裁剪处理，得到用于调整采样样本图像的梯度值；对采样样本图像以及梯度值进行求差处理(梯度值也可以称之为扰动值，求差处理可以理解为在采用样本图像中添加该扰动值，以得到包含噪声的图像)，得到第二待裁剪数值，通过由像素等级所生成的第二裁剪区间，对第二待裁剪数值进行裁剪处理，得到采样样本图像对应的初始对抗样本图像(若初始对抗样本图像能够直接作为采样样本图像对应的对抗样本图像，则此处的初始对抗样本图像也可以直接称之为对抗样本图像)。
具体的，业务服务器得到采样样本图像的第一概率向量后，业务服务器将第一概率向量中得最大概率元素作为第一损失值，将第一概率向量中错误类别标签对应的第一概率元素作为第二损失值，将第一损失值减去第二损失值的差值，作为采样样本图像针对错误类别标签的标签损失值。进一步，业务服务器根据标签损失值，可以对采样样本图像进行调整，得到采样样本图像的初始对抗样本图像，具体实现方式可以采取Project Gradient Descent(一种迭代攻击算法，简称PGD)攻击算法。其中，采取PGD攻击算法的具体实现过程，可以通过如下公式(1)描述。
其中，公式(1)中的t表示当前的迭代，t为正整数，即图像识别模型会经过至少一次的迭代，也即采样样本图像会经过至少一次的调整。表示第t次迭代的结果数据，例如第一次迭代后所生成的初始对抗样本图像，又如最后一次迭代后所生成的对抗样本图像。表示第t次迭代的输入数据，例如第一次迭代的输入数据为采样样本图像，可以理解的是，若存在第二次迭代，则第二次迭代的输入数据为初始对抗样本图像，即第一迭代的结果数据；同理，若存在更多的迭代次数，则下一次迭代的输入数据，为上一次迭代的结果数据。表示初始梯度值，表示第t次迭代的标签损失值，例如第一次迭代的标签损失值。α表示学习率。sign(.)表示sign函数，sign函数可以把初始梯度值(map)的每一个元素都变为{-1，0，1}。eps表示攻击强度，即每次用于调整的梯度map的每个元素值的绝对值不能大于该攻击强度，攻击强度是一个可调整的参数，一般被设为8/255，16/255，32/255等。clip(.,.)函数表示裁剪，即clip(.,.)函数可以将所有的值控制在指定的闭区间范围内，[-eps,eps]表示第一裁剪区间，[0,255]表示第二裁剪区间。
本申请实施例不对调整图像的方式进行限定，可以为上述的PGD攻击方法也可以采用其他攻击方法代替，比如Fast Gradient Sign Method(一种基于梯度生成对抗样本的算法，简称FGSM)，Iterative Fast Gradient Sign Method(改进版的FGSM，简称I-FGSM)，Momentum Iterative Fast Gradient Sign Method(又一改进版的FGSM，简称MI-FGSM)等。
进一步，业务服务器将错误类别标签和初始对抗样本图像输入至图像识别模型，通过图像识别模型， 生成初始对抗样本图像针对A个类别标签的第二概率向量。一般而言，要实现一个成功的对抗攻击，α和迭代次数t都要满足一定的数量，这样才能保证调整后的采样样本图像可以跨越过图像识别模型的分界面，成为能够被图像识别模型识别为错误类别标签的样本图像(为了与对抗样本图像区分，将此处的样本图像称为强对抗样本图像，即图像识别模型输出的针对强对抗样本图像的指代错误类别标签的概率元素趋近于1)。然而，选取这样的强对抗样本图像虽然可以提升图像识别模型的鲁棒性，但是并不能作为一个合适的数据增强手段用于提升图像识别模型的精度。因为强对抗样本图像完全处于错误类别标签的分布当中，如果利用强对抗样本图像对图像识别模型进行优化训练，就会让优化训练后的图像识别模型的分界面变成不柔和的曲线，例如锯齿状，请一并参见图4以及图5，图4是本申请实施例提供的一种通过干净样本数据所训练的分类边界面示意图，图5是本申请实施例提供的一种通过第一混合样本数据所训练的分类边界面示意图。其中，干净样本数据是指上述的A个样本图像集，第一混合样本数据包括上述的A个样本图像集以及上述的强对抗样本图像。为了便于叙述以及理解，A可以等于2，即A个类别标签包括图4以及图5所示例的第一类别标签以及第二类别标签。或者，A大于2，但一个类别标签作为真实类别标签，剩余的一个或多个类别标签作为攻击类别标签。在图4中，第一类别标签对应的类别特征分布区域(即符号“+”所处的区域)，以及第二类别标签对应的类别特征分布区域(即符号“－”所处的区域)相互独立，即图像识别模型可以准确地识别第一类别标签以及第二类别标签。
在图5中，第一类别标签对应的类别特征分布区域(即符号“+”所处的区域以及符号“△”所处的区域)，以及第二类别标签对应的类别特征分布区域(即符号“－”所处的区域以及符号“○”所处的区域)相互交互，图像识别模型为了更好的拟合强对抗样本图像，会把分界面变成锯齿状，虽然这让优化后的图像识别模型对于强对抗样本数据的鲁棒性提升了，但是对于某些靠近分界面的置信度不高的样本图像也被错误分类，例如图5中存在一些符号“－”位于第一类别标签对应的类别特征区域，存在一些符号“+”位于第二类别标签对应的类别特征区域，此时图像识别模型不能准确地识别第一类别标签以及第二类别标签，即损害了优化后的图像识别模型的精度。若A大于2，且不区分真实类别标签以及攻击类别标签，则类别特征分布区域的数量与A相同，例如A＝3，则存在3个类别特征分布区域。
经过上述分析，本申请实施例要达到的目的应该是在提升鲁棒性的同时，提升图像识别模型的精度和泛化性，因此，本申请实施例加入了边界约束条件对对抗样本图像进行筛选。换言之，用于数据增强的对抗样本图像不应该越过边界太远，因此，确定对抗样本图像的另一种方式可以为：业务服务器根据所述第一概率元素，对所述采样样本图像进行调整，得到所述采样样本图像对应的初始对抗样本图像；将所述错误类别标签和所述初始对抗样本图像输入至所述图像识别模型，通过所述图像识别模型，生成所述初始对抗样本图像针对所述A个类别标签的第二概率向量；在所述第二概率向量所包含的A个概率元素中，获取针对所述错误类别标签的第二概率元素；若所述第二概率元素不满足边界约束条件，或针对所述对抗样本图像进行类别预测所得到的类别预测结果不为所述错误类别标签，则根据所述第二概率元素继续对所述初始对抗样本图像进行调整(继续对所述初始对抗样本图像进行调整的过程与根据第一概率元素，对采样样本图像进行调整，得到初始对抗样本图像的具体实现过程相同，故此处不进行赘述，请参见上文的描述。可以理解的是，在后续的模型迭代过程(即图像调整过程)中，图像调整方式都与采样样本图像的调整方式相同)；若所述第二概率元素满足边界约束条件，且针对所述对抗样本图像进行类别预测所得到的类别预测结果为所述错误类别标签，则将所述初始对抗样本图像确定为所述采样样本图像对应的对抗样本图像；在所述类别预测结果为所述错误类别标签的情况下，所述第二概率元素为所述第二概率向量所包含的A个概率元素中的最大数值。可见，本申请所期望的对抗样本图像是既能满足边界约束条件，又能保证图像识别模型能够错误识别对抗样本图像。其中，业务服务器还可以将所述第二概率元素以及概率约束值进行对比；所述概率约束值为A的倒数；若所述第二概率元素小于所述概率约束值，则确定所述第二概率元素不满足边界约束条件；若所述第二概率元素等于或大于所述概率约束值，则确定所述第二概率元素满足边界约束条件。即本申请通过控制学习率和迭代次数，若概率向量(包括上述的第二概率向量)中的错误类别标签所对应的概率元素(包括上述的第二概率元素)恰好超过1/A，且此时的错误类别标签所对应的概率元素为概率向量中的最大值(具有最大值的概率元素用于指示图像识别模型针对初始对抗样本图像的类别预测结果为该概率元素所指代的类别标签)，就停止模型迭代(即图像调整)，将当前的调整好的样本图像作为一个合理的对抗样本图像，而不是等错误类别标签所对应的概率元素趋近于1。
基于上述，本申请实施例所得到的对抗样本图像可以处于分类边界面附近，而不是在错误类别标签的分布之中，进而通过对抗样本图像所优化训练的图像识别模型的分类边界面就会更加柔和，请一并参见图6，图6是本申请实施例提供的一种通过第二混合样本数据所训练的分类边界面示意图。第二混合样本数 据包括上述的A个样本图像集以及上述的对抗样本图像。如图6所示，第一类别标签对应的类别特征分布区域(即符号“+”所处的区域以及符号“△”所处的区域)，以及第二类别标签对应的类别特征分布区域(即符号“－”所处的区域以及符号“○”所处的区域)基本不交互，且存在少数交互的类别特征也基本位于分类边界面，此时优化后的图像识别模型可以准确地识别第一类别标签以及第二类别标签，即能够保证优化后的图像识别模型(简称图像识别优化模型)保持或提高原有识别精度的基础上，还可以补充训练数据的不足之处，故可以提升图像识别优化模型的防御能力以及安全性。
本申请实施例提出了一种全新的基于边界约束的对抗数据增强，利用对抗数据在边界分布的特殊性，对训练数据进行扩增和增强，从而提升优化后的图像识别模型的防御能力和泛化性。
在本申请实施例中，计算机设备可以获取A个样本图像集以及图像识别模型；其中，所述A个样本图像集分别对应A个不同的类别标签中的一个，图像识别模型是根据A个样本图像集预训练得到的，所以图像识别模型可以准确地确定A个样本图像集中的样本图像的类别标签；进一步，计算机设备在A个样本图像集中获取采样样本图像，将错误类别标签和采样样本图像输入至图像识别模型，其中，所述错误类别标签属于所述A个类别标签，且所述错误类别标签与所述采样样本图像所属的样本图像集对应的类别标签不同；通过图像识别模型可以生成采样样本图像针对A个类别标签的第一概率向量，所述第一概率向量包括A个概率元素，所述A个概率元素分别指代所述A个类别标签中的一个，一个所述概率元素用于表征所述采样样本图像属于所指代的类别标签的概率；由于图像识别模型可以准确地确定A个样本图像集中的样本图像的类别标签，故在第一概率向量中，指代正确类别标签的概率元素最大，或者说，指代正确类别标签的概率元素远远大于指代错误类别标签的第一概率元素，所述正确类别标签是指所述采样样本图像所属的样本图像集对应的类别标签；进一步，根据第一概率元素，计算机设备可以对采样样本图像进行调整，得到采样样本图像对应的对抗样本图像，其中，所述对抗样本图像被所述图像识别模型进行类别预测所得到的类别预测结果为所述错误类别标签，即通过对采样样本图像进行调整，使得图像识别模型可以错误地确定对抗样本图像的类别标签。上述可知，本申请实施例提出一种对抗样本图像的生成方法，通过该方法，可以生成图像识别模型错误识别的对抗样本图像，该对抗样本图像可以提高用于优化训练图像识别模型的训练集的样本多样性，通过具有样本多样性的训练集，可以提高图像识别模型的优化训练准确度，进而可以提高优化训练好的图像识别模型的类别标签识别准确度以及识别泛化性。而且由于所述图像识别模型针对所述对抗样本图像所预测的针对所述错误类别标签的第二概率元素满足边界约束条件，所以就可以使得优化训练后的图像识别模型的分类边界更加柔和，进一步保证了模型精度。
请参见图7，图7是本申请实施例提供的一种数据处理方法的流程示意图二。该方法可以由业务服务器(例如，上述图1所示的业务服务器100)执行，也可以由终端设备(例如，上述图1所示的终端设备200a)执行，还可以由业务服务器和终端设备交互执行。为便于理解，本申请实施例以该方法由业务服务器执行为例进行说明。如图7所示，该方法至少可以包括以下步骤。
步骤S201，获取A个样本图像集以及图像识别模型，A为大于1的正整数，所述A个样本图像集分别对应A个不同的类别标签中的一个，所述图像识别模型是根据所述A个样本图像集预训练得到的，所述A个类别标签包括错误类别标签。
具体的，业务服务获取A个原始图像集，将A个原始图像集均输入至对象检测模型；A个原始图像集分别对应A个不同的类别信息中的一个；A个原始图像集包括原始图像集Db；原始图像集Db包括原始图像Ef；f为正整数，且f小于或等于原始图像集Db中原始图像的总数量；通过对象检测模型，确定原始图像Ef中关键区域的区域坐标，根据区域坐标，生成原始图像Ef对应的待标注样本图像；将原始图像集Db中每个原始图像对应的待标注样本图像，确定为待标注样本图像集；根据原始图像集Db对应的类别信息，生成类别标签Hb，将标注有类别标签Hb的待标注样本图像集，确定为样本图像集Cb。
其中，根据区域坐标，生成原始图像Ef对应的待标注样本图像的具体过程可以包括：根据区域坐标，生成包括关键区域的初始检测框，对初始检测框进行扩张处理，得到包括目标对象的检测框；关键区域属于目标对象；在检测框中，获取包括目标对象的待缩放图像，获取第一图像尺寸，根据第一图像尺寸，对待缩放图像进行缩放处理，得到待裁剪图像；获取第二图像尺寸，根据第二图像尺寸，对待裁剪图像进行裁剪处理，得到原始图像Ef对应的待标注样本图像；第二图像尺寸小于第一图像尺寸。
业务服务器不对对象检测模型的模型类型进行限定，可以根据实际应用场景进行设定，例如Region with CNN feature(一种目标检测模型，简称Faster R-CNN)，Single Shot MultiBox Detector(另一种目标检测模型，简称SSD)，You Only Look Once(一种简易快捷的目标检测算法，简称YOLO)等。
其中，关键区域属于目标对象，本申请实施例不对目标对象进行限定，例如可以是猫、狗、人、桌子、 辣椒等，故不对关键区域进行限定，应当根据目标对象进行设定。其中，初始检测框的扩张倍数，应当根据关键区域进行设定，故本申请实施例不对扩张倍数进行限定。其中，第二图像尺寸是指图像识别模型的输入尺寸。
步骤S202，在所述A个样本图像集中获取采样样本图像，将所述采样样本图像与所述错误类别标签输入至所述图像识别模型，以生成所述采样样本图像针对所述A个类别标签的第一概率向量，所述错误类别标签与所述采样样本图像所属的样本图像集对应的类别标签不同；所述第一概率向量包括A个概率元素，所述A个概率元素分别指代所述A个类别标签中的一个，一个所述概率元素用于表征所述采样样本图像属于所指代的类别标签的概率。
步骤S203，在所述第一概率向量中的所述A个概率元素中，获取指代所述错误类别标签的第一概率元素，根据所述第一概率元素，对所述采样样本图像进行调整，得到所述采样样本图像对应的对抗样本图像，所述对抗样本图像被所述图像识别模型进行类别预测所得到的类别预测结果为所述错误类别标签。
其中，步骤S202-步骤S203的具体实现过程，请参见上文图3所对应的实施例中的步骤S102步骤S103，此处不进行赘述。
步骤S204，将所述A个样本图像集、所述A个类别标签、正确类别标签以及所述对抗样本图像均输入至所述图像识别模型；所述正确类别标签是指所述采样样本图像所属的样本图像集对应的类别标签。
其中，所述A个样本图像集、所述A个类别标签、正确类别标签以及所述对抗样本图像可以分别作为4个相互独立的数据输入图像识别模型，且所述A个样本图像集与所述A个类别标签之间存在映射关系，正确类别标签与所述对抗样本图像之间存在映射关系；或者，可以将所属的类别标签添加到样本图像集中的样本图像的文件头或文件尾中，且将正确类别标签添加到对抗样本图像的文件头或文件尾中，进而可以将携带类别标签的样本图像以及携带正确类别标签的对抗样本图像输入图像识别模型。
步骤S205，通过图像识别模型，生成A个样本图像集分别对应的第一预测类别，以及对抗样本图像对应的第二预测类别。
步骤S206，根据所述第一预测类别和所述A个类别标签生成第一损失值，根据所述第二预测类别以及所述正确类别标签生成第二损失值，根据所述第一损失值和所述第二损失值对所述图像识别模型中的参数进行调整(具体可以将第一损失值和第二损失值进行加权求和，得到总损失值，然后基于该总损失值对图像识别模型中的参数进行调整)，得到所述图像识别优化模型；所述图像识别优化模型针对所述对抗样本图像进行类别预测所得到的类别预测结果为所述正确类别标签，且图像识别优化模型针对样本图像集中的样本图像进行类别预测所得到的类别预测结果依然为样本图像集所对应的类别标签。
其中，业务服务器对图像识别模型的优化训练的过程，与业务服务器对图像识别初始模型的训练过程相同，故步骤S204-步骤S206的具体实现过程，请参见上文图3所对应的实施例中的步骤S101，此处不进行赘述。
本申请实施例提出的图像识别优化模型可以部署于终端设备，例如对真实对象识别(例如真人识别)输入的图像进行检测，如果是真实对象，则通过并进入到后续的识别流程中，如果不是真实对象，例如照片对象，则进行报错，并提示重试。本申请实施例可以应用于所有对象识别相关的应用，包括但不限于：线上支付，线下支付，门禁解锁系统，手机解锁识别，自动对象识别通关等。
上述可知，本申请实施例提出一种对抗样本图像的生成方法，通过该方法，可以生成图像识别模型错误识别的对抗样本图像，通过对抗样本图像，可以提高用于优化训练图像识别模型的训练集的样本多样性，通过具有样本多样性的训练集，可以提高图像识别模型的优化训练准确度，进而可以提高优化训练好的图像识别模型的类别标签识别准确度以及识别泛化性。
请参见图8，图8是本申请实施例提供的一种数据处理方法的流程示意图三。该方法可以由业务服务器(例如，上述图1所示的业务服务器100)执行，也可以由终端设备(例如，上述图1所示的终端设备200a)执行，还可以由业务服务器和终端设备交互执行。为便于理解，本申请实施例以该方法由业务服务器执行为例进行说明。如图8所示，该方法至少可以包括以下步骤。
步骤S301，获取目标图像，将所述目标图像输入至图像识别优化模型；所述图像识别优化模型是根据混合样本训练集继续对图像识别模型进行优化训练所得到的；所述混合样本训练集包括A个样本图像集、A个类别标签、正确类别标签以及对抗样本图像；所述A个样本图像集分别对应A个不同的类别标签中的一个，所述A个样本图像集包括采集样本图像；所述正确类别标签属于所述A个类别标签；A为大于1的正整数；所述图像识别模型是根据所述A个样本图像集预训练得到的；所述对抗样本图像是通过第一概率向量中的A个概率元素中指代错误类别标签的第一概率元素对采样样本图像进行调整得到，所述第一概率 向量是由所述图像识别模型针对所述采样样本图像所生成的，所述错误类别标签与所述采样样本图像所属的样本图像集对应的类别标签不同，所述正确类别标签为所述采样样本图像所属的样本图像集对应的类别标签；所述第一概率向量包括A个概率元素，所述A个概率元素分别指代所述A个类别标签中的一个，一个所述概率元素用于表征所述采样样本图像属于所指代的类别标签的概率；所述对抗样本图像被所述图像识别模型进行类别预测所得到的类别预测结果为所述错误类别标签。
步骤S302，通过图像识别优化模型，生成目标图像针对A个类别标签的目标概率向量。
步骤S303，将目标概率向量中的最大概率元素对应的类别标签，确定为目标图像的类别预测结果。
具体的，本申请实施例的图像识别模型可以用于识别真实对象，例如真实的人，此时，图像识别模型可以称为真实对象检测模型。目前智能对象识别支付业务中，越来越多的真实对象检测技术被应用到产品中，有力的保障了支付的安全。然而，在真实对象检测的实际应用场景中，非法分子带来的攻击种类繁多，攻击形式新颖，防不胜防，比如在纸片攻击当中，样式繁多的材质中总有一款模型没训练过的材质，故可以绕过真实对象检测模型，从而实现攻击。而在高精度的3维攻击当中也面临这个问题，一旦有新的材质出现，真实对象检测模型就会面临防不住的情况。究其原因，还是训练数据没有覆盖整个类别空间导致的。因此，本申请实施例提出了一种全新的基于边界约束的对抗数据增强的真实对象检测技术，通过设计一种特殊的对抗样本图像来对真实对象训练数据进行增强，从而整体提升真实对象检测的性能。对抗样本图像的具体生成过程，请参见上述图3以及图7所分别对应的实施例中的描述。
上述可知，本申请实施例提出一种对抗样本图像的生成方法，通过该方法，可以生成图像识别模型错误识别的对抗样本图像，通过对抗样本图像，可以提高用于优化训练图像识别模型的训练集的样本多样性，通过具有样本多样性的训练集，可以提高图像识别模型的优化训练准确度，进而可以提高优化训练好的图像识别模型的类别标签识别准确度以及识别泛化性。
进一步地，请参见图9，图9是本申请实施例提供的一种数据处理装置的结构示意图一。上述数据处理装置1可以用于执行本申请实施例提供的方法中的相应步骤。如图9所示，该数据处理装置1可以包括：数据获取模块11、第一输入模块12以及第一调整模块13。
数据获取模块11，用于获取A个样本图像集以及图像识别模型，A为大于1的正整数，所述A个样本图像集分别对应A个不同的类别标签中的一个，所述图像识别模型是根据所述A个样本图像集预训练得到的，所述A个类别标签包括错误类别标签；
第一输入模块12，用于在所述A个样本图像集中获取采样样本图像，将所述采样样本图像与所述错误类别标签输入至所述图像识别模型，以生成所述采样样本图像针对所述A个类别标签的第一概率向量，所述错误类别标签与所述采样样本图像所属的样本图像集对应的类别标签不同；所述第一概率向量包括A个概率元素，所述A个概率元素分别指代所述A个类别标签中的一个，一个所述概率元素用于表征所述采样样本图像属于所指代的类别标签的概率；
第一调整模块13，用于在所述第一概率向量中的所述A个概率元素中，获取指代所述错误类别标签的第一概率元素，根据所述第一概率元素，对所述采样样本图像进行调整，得到所述采样样本图像对应的对抗样本图像，所述对抗样本图像被所述图像识别模型进行类别预测所得到的类别预测结果为所述错误类别标签。
其中，数据获取模块11、第一输入模块12以及第一调整模块13的具体功能实现方式，可以参见上述图3对应实施例中的步骤S101-你在S103，这里不再进行赘述。
其中，第一调整模块13，可以具体用于生成第一概率元素对应的负概率元素，对负概率元素以及第一概率向量中的最大概率元素进行求和处理，得到采样样本图像针对错误类别标签的标签损失值，根据标签损失值，对采样样本图像进行调整，得到采样样本图像对应的对抗样本图像。
其中，第一调整模块13用于执行根据标签损失值，对采样样本图像进行调整，得到采样样本图像对应的对抗样本图像时，具体用于根据标签损失值，生成针对采样样本图像的初始梯度值，获取初始梯度值的数值符号，根据数值符号生成初始梯度值对应的符号单位数值，对初始学习率以及符号单位数值进行乘积处理，得到第一待裁剪数值，通过由攻击强度所生成的第一裁剪区间，对第一待裁剪数值进行裁剪处理，得到用于调整采样样本图像的梯度值，对采样样本图像以及梯度值进行求差处理，得到第二待裁剪数值，通过由像素等级所生成的第二裁剪区间，对第二待裁剪数值进行裁剪处理，得到采样样本图像对应的对抗样本图像。
再请参见图9，第一调整模块13可以包括：第一调整单元131、第一生成单元132以及第一获取单元133。
第一调整单元131，用于根据第一概率元素，对采样样本图像进行调整，得到采样样本图像对应的初始对抗样本图像；
第一生成单元132，用于将所述错误类别标签和所述初始对抗样本图像输入至所述图像识别模型，通过图像识别模型，生成初始对抗样本图像针对A个类别标签的第二概率向量；
第一获取单元133，用于在所述第二概率向量所包含的A个概率元素中，获取针对所述错误类别标签的第二概率元素；
第一调整单元131，还用于若所述第二概率元素不满足边界约束条件，或针对所述对抗样本图像进行类别预测所得到的类别预测结果不为所述错误类别标签，则根据所述第二概率元素继续对所述初始对抗样本图像进行调整；
第一获取单元133，还用于若所述第二概率元素满足边界约束条件，且针对所述对抗样本图像进行类别预测所得到的类别预测结果为所述错误类别标签，则将所述初始对抗样本图像确定为所述采样样本图像对应的对抗样本图像；在所述类别预测结果为所述错误类别标签的情况下，所述第二概率元素为所述第二概率向量所包含的A个概率元素中的最大数值。
其中，第一调整单元131、第一生成单元132以及第一获取单元133的具体功能实现方式，可以参见上述图3对应实施例中的步骤S103，这里不再进行赘述。
其中，第一获取单元133，还用于将所述第二概率元素以及概率约束值进行对比；所述概率约束值为A的倒数；
第一获取单元133，还用于若所述第二概率元素小于所述概率约束值，则确定所述第二概率元素不满足边界约束条件；
第一获取单元133，还用于若所述第二概率元素等于或大于所述概率约束值，则确定所述第二概率元素满足边界约束条件。
再请参见图9，数据处理装置1还可以包括：第二输入模块14以及第二调整模块15。
第二输入模块14，用于将所述A个样本图像集、所述A个类别标签、正确类别标签以及所述对抗样本图像均输入至所述图像识别模型；所述正确类别标签是指所述采样样本图像所属的样本图像集对应的类别标签；
第二输入模块14，还用于通过图像识别模型，生成A个样本图像集分别对应的第一预测类别，以及对抗样本图像对应的第二预测类别；
第二调整模块15，用于根据所述第一预测类别和所述A个类别标签生成第一损失值，根据所述第二预测类别以及所述正确类别标签生成第二损失值，根据所述第一损失值和所述第二损失值对所述图像识别模型中的参数进行调整，得到所述图像识别优化模型；所述图像识别优化模型针对所述对抗样本图像进行类别预测所得到的类别预测结果为所述正确类别标签。
其中，第二输入模块14以及第二调整模块15的具体功能实现方式，可以参见上述图7对应实施例中的步骤S204-步骤S206，这里不再进行赘述。
再请参见图9，第一输入模块12可以包括：第一确定单元121以及第二获取单元122。
其中，所述A个样本图像集包括样本图像集Cb；b为正整数且b小于或等于A；
第一确定单元121，用于确定采样比例，根据采样比例，对样本图像集Cb中的样本图像进行采样处理，得到采样样本图像；
第二获取单元122，用于在A个类别标签中，获取与所述样本图像集Cb的类别标签不同的类别标签，将获取到的类别标签确定为错误类别标签。
其中，第一确定单元121以及第二获取单元122的具体功能实现方式，可以参见上述图3对应实施例中的步骤S102，这里不再进行赘述。
再请参见图9，数据获取模块11可以包括：第三获取单元111、第二生成单元112、第二确定单元113、第三确定单元114以及第二调整单元115。
第三获取单元111，用于获取A个样本图像集，将A个类别标签和所述A个样本图像集分别输入至所述图像识别初始模型；
第二生成单元112，用于通过图像识别初始模型，生成所述A个样本图像集分别对应的预测初始类别；
第二确定单元113，用于根据所述预测初始类别以及所述A个类别标签，确定所述A个样本图像集分别对应的类别损失值；
第三确定单元114，用于根据A个样本图像集分别对应的类别损失值，确定图像识别初始模型对应的 总损失值；
第二调整单元115，用于根据所述总损失值，对所述图像识别初始模型中的参数进行调整，得到图像识别模型；所述图像识别模型针对样本图像集Cb进行类别预测所得到的类别预测结果为所述样本图像集Cb的类别标签；所述A个样本图像集包括样本图像集Cb；b为正整数且b小于或等于A。
其中，第三获取单元111、第二生成单元112、第二确定单元113、第三确定单元114以及第二调整单元115的具体功能实现方式，可以参见上述图3对应实施例中的步骤S101，这里不再进行赘述。
再请参见图9，第三获取单元111可以包括：图像获取子单元1111、第二确定子单元1112、第三确定子单元1113以及第四确定子单元1114。
图像获取子单元1111，用于获取A个原始图像集，将A个原始图像集均输入至对象检测模型；A个原始图像集分别对应的类别信息互不相同；A个原始图像集包括原始图像集Db；原始图像集Db包括原始图像Ef；f为正整数，且f小于或等于原始图像集Db中原始图像的总数量；
第二确定子单元1112，用于通过对象检测模型，确定原始图像Ef中关键区域的区域坐标，根据区域坐标，生成原始图像Ef对应的待标注样本图像；
第三确定子单元1113，用于将原始图像集Db中每个原始图像对应的待标注样本图像，确定为待标注样本图像集；
第四确定子单元1114，用于根据原始图像集Db对应的类别信息，生成类别标签Hb，将标注有类别标签Hb的待标注样本图像集，确定为样本图像集Cb。
其中，图像获取子单元1111、第二确定子单元1112、第三确定子单元1113以及第四确定子单元1114的具体功能实现方式，可以参见上述图7对应实施例中的步骤S201，这里不再进行赘述。
再请参见图9，第二确定子单元1112可以包括：第三处理子单元11121、第四处理子单元11122以及第五处理子单元11123。
第三处理子单元11121，用于根据区域坐标，生成包括关键区域的初始检测框，对初始检测框进行扩张处理，得到包括目标对象的检测框；关键区域属于目标对象；
第四处理子单元11122，用于在检测框中，获取包括目标对象的待缩放图像，获取第一图像尺寸，根据第一图像尺寸，对待缩放图像进行缩放处理，得到待裁剪图像；
第五处理子单元11123，用于获取第二图像尺寸，根据第二图像尺寸，对待裁剪图像进行裁剪处理，得到原始图像Ef对应的待标注样本图像；第二图像尺寸小于第一图像尺寸。
其中，第三处理子单元11121、第四处理子单元11122以及第五处理子单元11123的具体功能实现方式，可以参见上述图7对应实施例中的步骤S201，这里不再进行赘述。
上述可知，本申请实施例提出一种对抗样本图像的生成方法，通过该方法，可以生成图像识别模型错误识别的对抗样本图像，通过对抗样本图像，可以提高用于优化训练图像识别模型的训练集的样本多样性，通过具有样本多样性的训练集，可以提高图像识别模型的优化训练准确度，进而可以提高优化训练好的图像识别模型的类别标签识别准确度以及识别泛化性。而且由于所述图像识别模型针对所述对抗样本图像所预测的针对所述错误类别标签的第二概率元素满足边界约束条件，所以就可以使得优化训练后的图像识别模型的分类边界更加柔和，进一步保证了模型精度。
进一步地，请参见图10，图10是本申请实施例提供的一种数据处理装置的结构示意图二。上述数据处理装置2可以用于执行本申请实施例提供的方法中的相应步骤。如图10所示，该数据处理装置2可以包括：图像获取模块21、向量生成模块22以及类别确定模块23。
图像获取模块21，用于获取目标图像，将所述目标图像输入至图像识别优化模型；所述图像识别优化模型是根据混合样本训练集继续对图像识别模型进行优化训练所得到的；所述混合样本训练集包括A个样本图像集、A个类别标签、正确类别标签以及对抗样本图像；所述A个样本图像集分别对应A个不同的类别标签中的一个，所述A个样本图像集包括采集样本图像；所述正确类别标签属于所述A个类别标签；A为大于1的正整数；所述图像识别模型是根据所述A个样本图像集预训练得到的；所述对抗样本图像是通过第一概率向量中的A个概率元素中指代错误类别标签的第一概率元素对采样样本图像进行调整得到，所述第一概率向量是由所述图像识别模型针对所述采样样本图像所生成的，所述错误类别标签与所述采样样本图像所属的样本图像集对应的类别标签不同，所述正确类别标签为所述采样样本图像所属的样本图像集对应的类别标签；所述第一概率向量包括A个概率元素，所述A个概率元素分别指代所述A个类别标签中的一个，一个所述概率元素用于表征所述采样样本图像属于所指代的类别标签的概率；所述对抗样本图像被所述图像识别模型进行类别预测所得到的类别预测结果为所述错误类别标签；
向量生成模块22，用于通过图像识别优化模型，生成目标图像针对A个类别标签的目标概率向量；
类别确定模块23，用于将目标概率向量中的最大概率元素对应的类别标签，确定为目标图像的类别预测结果。
其中，图像获取模块21、向量生成模块22以及类别确定模块23的具体功能实现方式，可以参见上述图8对应实施例中的步骤S301-步骤S303，这里不再进行赘述。
上述可知，本申请实施例提出一种对抗样本图像的生成方法，通过该方法，可以生成图像识别模型错误识别的对抗样本图像，通过对抗样本图像，可以提高用于优化训练图像识别模型的训练集的样本多样性，通过具有样本多样性的训练集，可以提高图像识别模型的优化训练准确度，进而可以提高优化训练好的图像识别模型的类别标签识别准确度以及识别泛化性。
进一步地，请参见图11，图11是本申请实施例提供的一种计算机设备的结构示意图。如图11所示，该计算机设备1000可以包括：至少一个处理器1001，例如CPU，至少一个网络接口1004，用户接口1003，存储器1005，至少一个通信总线1002。其中，通信总线1002用于实现这些组件之间的连接通信。其中，在一些实施例中，用户接口1003可以包括显示屏(Display)、键盘(Keyboard)，网络接口1004可选地可以包括标准的有线接口、无线接口(如WI-FI接口)。存储器1005可以是高速RAM存储器，也可以是非不稳定的存储器(non-volatile memory)，例如至少一个磁盘存储器。存储器1005可选地还可以是至少一个位于远离前述处理器1001的存储装置。如图11所示，作为一种计算机存储介质的存储器1005可以包括操作系统、网络通信模块、用户接口模块以及设备控制应用程序。
在图11所示的计算机设备1000中，网络接口1004可提供网络通讯功能；而用户接口1003主要用于为用户提供输入的接口；而处理器1001可以用于调用存储器1005中存储的设备控制应用程序，以实现：
获取A个样本图像集以及图像识别模型，A为大于1的正整数，所述A个样本图像集分别对应A个不同的类别标签中的一个，所述图像识别模型是根据所述A个样本图像集预训练得到的，所述A个类别标签包括错误类别标签；
在所述A个样本图像集中获取采样样本图像，将所述采样样本图像与所述错误类别标签输入至所述图像识别模型，以生成所述采样样本图像针对所述A个类别标签的第一概率向量，所述错误类别标签与所述采样样本图像所属的样本图像集对应的类别标签不同；所述第一概率向量包括A个概率元素，所述A个概率元素分别指代所述A个类别标签中的一个，一个所述概率元素用于表征所述采样样本图像属于所指代的类别标签的概率；
在所述第一概率向量中的所述A个概率元素中，获取指代所述错误类别标签的第一概率元素，根据所述第一概率元素，对所述采样样本图像进行调整，得到所述采样样本图像对应的对抗样本图像，所述对抗样本图像被所述图像识别模型进行类别预测所得到的类别预测结果为所述错误类别标签。
或者，处理器1001可以用于调用存储器1005中存储的设备控制应用程序，以实现：
获取目标图像，将所述目标图像输入至图像识别优化模型；所述图像识别优化模型是根据混合样本训练集继续对图像识别模型进行优化训练所得到的；所述混合样本训练集包括A个样本图像集、A个类别标签、正确类别标签以及对抗样本图像；所述A个样本图像集分别对应A个不同的类别标签中的一个，所述A个样本图像集包括采集样本图像；所述正确类别标签属于所述A个类别标签；A为大于1的正整数；所述图像识别模型是根据所述A个样本图像集预训练得到的；所述对抗样本图像是通过第一概率向量中的A个概率元素中指代错误类别标签的第一概率元素对采样样本图像进行调整得到，所述第一概率向量是由所述图像识别模型针对所述采样样本图像所生成的，所述错误类别标签与所述采样样本图像所属的样本图像集对应的类别标签不同，所述正确类别标签为所述采样样本图像所属的样本图像集对应的类别标签；所述第一概率向量包括A个概率元素，所述A个概率元素分别指代所述A个类别标签中的一个，一个所述概率元素用于表征所述采样样本图像属于所指代的类别标签的概率；所述对抗样本图像被所述图像识别模型进行类别预测所得到的类别预测结果为所述错误类别标签；
通过所述图像识别优化模型，生成所述目标图像针对所述A个类别标签的目标概率向量；
将所述目标概率向量中的最大概率元素对应的类别标签，确定为所述目标图像的类别预测结果。
应当理解，本申请实施例中所描述的计算机设备1000可执行前文各实施例中对数据处理方法或装置的描述，在此不再赘述。另外，对采用相同方法的有益效果描述，也不再进行赘述。
本申请实施例还提供一种计算机可读存储介质，该计算机可读存储介质存储有计算机程序，该计算机程序被处理器执行时实现前文各实施例中对数据处理方法或装置的描述，在此不再赘述。另外，对采用相同方法的有益效果描述，也不再进行赘述。
上述计算机可读存储介质可以是前述任一实施例提供的数据处理装置或者上述计算机设备的内部存储单元，例如计算机设备的硬盘或内存。该计算机可读存储介质也可以是该计算机设备的外部存储设备，例如该计算机设备上配备的插接式硬盘，智能存储卡(smart media card，SMC)，安全数字(secure digital，SD)卡，闪存卡(flash card)等。进一步地，该计算机可读存储介质还可以既包括该计算机设备的内部存储单元也包括外部存储设备。该计算机可读存储介质用于存储该计算机程序以及该计算机设备所需的其他程序和数据。该计算机可读存储介质还可以用于暂时地存储已经输出或者将要输出的数据。
本申请实施例还提供了一种计算机程序产品，该计算机程序产品包括计算机程序，该计算机程序存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机程序，处理器执行该计算机程序，使得该计算机设备可执行前文各实施例中对数据处理方法或装置的描述，在此不再赘述。另外，对采用相同方法的有益效果描述，也不再进行赘述。
本申请实施例的说明书和权利要求书及附图中的术语“第一”、“第二”等是用于区别不同对象，而非用于描述特定顺序。此外，术语“包括”以及它们任何变形，意图在于覆盖不排他的包含。例如包含了一系列步骤或单元的过程、方法、装置、产品或设备没有限定于已列出的步骤或模块，而是可选地还包括没有列出的步骤或模块，或可选地还包括对于这些过程、方法、装置、产品或设备固有的其他步骤单元。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
以上所揭露的仅为本申请较佳实施例而已，当然不能以此来限定本申请之权利范围，因此依本申请权利要求所作的等同变化，仍属本申请所涵盖的范围。

Claims (16)

  1. 一种图像处理方法，其特征在于，包括：
    获取A个样本图像集以及图像识别模型，A为大于1的正整数，所述A个样本图像集分别对应A个不同的类别标签中的一个，所述图像识别模型是根据所述A个样本图像集预训练得到的，所述A个类别标签包括错误类别标签；
    在所述A个样本图像集中获取采样样本图像，将所述采样样本图像与所述错误类别标签输入至所述图像识别模型，以生成所述采样样本图像针对所述A个类别标签的第一概率向量，所述错误类别标签与所述采样样本图像所属的样本图像集对应的类别标签不同；所述第一概率向量包括A个概率元素，所述A个概率元素分别指代所述A个类别标签中的一个，一个所述概率元素用于表征所述采样样本图像属于所指代的类别标签的概率；
    在所述第一概率向量中的所述A个概率元素中，获取指代所述错误类别标签的第一概率元素，根据所述第一概率元素，对所述采样样本图像进行调整，得到所述采样样本图像对应的对抗样本图像，所述对抗样本图像被所述图像识别模型进行类别预测所得到的类别预测结果为所述错误类别标签。
  2. 根据权利要求1所述的方法，其特征在于，所述根据所述第一概率元素，对所述采样样本图像进行调整，得到所述采样样本图像对应的对抗样本图像，包括：
    生成所述第一概率元素对应的负概率元素，对所述负概率元素以及所述第一概率向量中的最大概率元素进行求和处理，得到所述采样样本图像针对所述错误类别标签的标签损失值；
    根据所述标签损失值，对所述采样样本图像进行调整，得到所述采样样本图像对应的对抗样本图像。
  3. 根据权利要求1或2所述的方法，其特征在于，所述根据所述标签损失值，对所述采样样本图像进行调整，得到所述采样样本图像对应的对抗样本图像，包括：
    根据所述标签损失值，生成针对所述采样样本图像的初始梯度值，获取所述初始梯度值的数值符号，根据所述数值符号生成所述初始梯度值对应的符号单位数值；
    对初始学习率以及符号单位数值进行乘积处理，得到第一待裁剪数值，通过由攻击强度所生成的第一裁剪区间，对所述第一待裁剪数值进行裁剪处理，得到用于调整所述采样样本图像的梯度值；
    对所述采样样本图像以及所述梯度值进行求差处理，得到第二待裁剪数值，通过由像素等级所生成的第二裁剪区间，对所述第二待裁剪数值进行裁剪处理，得到所述采样样本图像对应的对抗样本图像。
  4. 根据权利要求1至3任一项所述的方法，其特征在于，所述根据所述第一概率元素，对所述采样样本图像进行调整，得到所述采样样本图像对应的对抗样本图像，包括：
    根据所述第一概率元素，对所述采样样本图像进行调整，得到所述采样样本图像对应的初始对抗样本图像；
    将所述错误类别标签和所述初始对抗样本图像输入至所述图像识别模型，通过所述图像识别模型，生成所述初始对抗样本图像针对所述A个类别标签的第二概率向量；
    在所述第二概率向量所包含的A个概率元素中，获取针对所述错误类别标签的第二概率元素；
    若所述第二概率元素不满足边界约束条件，或针对所述对抗样本图像进行类别预测所得到的类别预测结果不为所述错误类别标签，则根据所述第二概率元素继续对所述初始对抗样本图像进行调整；
    若所述第二概率元素满足边界约束条件，且针对所述对抗样本图像进行类别预测所得到的类别预测结果为所述错误类别标签，则将所述初始对抗样本图像确定为所述采样样本图像对应的对抗样本图像；在所述类别预测结果为所述错误类别标签的情况下，所述第二概率元素为所述第二概率向量所包含的A个概率元素中的最大数值。
  5. 根据权利要求1至4任一项所述的方法，其特征在于，还包括：
    将所述第二概率元素以及概率约束值进行对比；所述概率约束值为A的倒数；
    若所述第二概率元素小于所述概率约束值，则确定所述第二概率元素不满足边界约束条件；
    若所述第二概率元素等于或大于所述概率约束值，则确定所述第二概率元素满足边界约束条件。
  6. 根据权利要求1至5任一项所述的方法，其特征在于，所述方法还包括：
    将所述A个样本图像集、所述A个类别标签、正确类别标签以及所述对抗样本图像均输入至所述图像识别模型；所述正确类别标签是指所述采样样本图像所属的样本图像集对应的类别标签；
    通过所述图像识别模型，生成所述A个样本图像集分别对应的第一预测类别，以及所述对抗样本图像对应的第二预测类别；
    根据所述第一预测类别和所述A个类别标签生成第一损失值，根据所述第二预测类别以及所述正确类 别标签生成第二损失值，根据所述第一损失值和所述第二损失值对所述图像识别模型中的参数进行调整，得到所述图像识别优化模型；所述图像识别优化模型针对所述对抗样本图像进行类别预测所得到的类别预测结果为所述正确类别标签。
  7. 根据权利要求1至6任一项所述的方法，其特征在于，所述A个样本图像集包括样本图像集Cb；b为正整数且b小于或等于A；所述在所述A个样本图像集中获取采样样本图像，包括：
    确定采样比例，根据所述采样比例，对所述样本图像集Cb中的样本图像进行采样处理，得到采样样本图像；
    在所述A个类别标签中，获取与所述样本图像集Cb的类别标签不同的类别标签，将获取到的类别标签确定为所述错误类别标签。
  8. 根据权利要求1至7任一项所述的方法，其特征在于，所述获取A个样本图像集以及图像识别模型，包括：
    获取A个样本图像集，将A个类别标签和所述A个样本图像集分别输入至所述图像识别初始模型；
    通过所述图像识别初始模型，生成所述A个样本图像集分别对应的预测初始类别；
    根据所述预测初始类别以及所述A个类别标签，确定所述A个样本图像集分别对应的类别损失值；
    根据所述A个样本图像集分别对应的类别损失值，确定所述图像识别初始模型对应的总损失值；
    根据所述总损失值，对所述图像识别初始模型中的参数进行调整，得到图像识别模型；所述图像识别模型针对样本图像集Cb进行类别预测所得到的类别预测结果为所述样本图像集Cb的类别标签；所述A个样本图像集包括样本图像集Cb；b为正整数且b小于或等于A。
  9. 根据权利要求8所述的方法，其特征在于，所述获取A个样本图像集，包括：
    获取A个原始图像集，将所述A个原始图像集均输入至对象检测模型；所述A个原始图像集分别对应A个不同的类别信息中的一个；所述A个原始图像集包括原始图像集Db；所述原始图像集Db包括原始图像Ef；f为正整数，且f小于或等于所述原始图像集Db中原始图像的总数量；
    通过所述对象检测模型，确定所述原始图像Ef中关键区域的区域坐标，根据所述区域坐标，生成所述原始图像Ef对应的待标注样本图像；
    将所述原始图像集Db中每个原始图像对应的待标注样本图像，确定为待标注样本图像集；
    根据所述原始图像集Db对应的类别信息，生成类别标签Hb，将标注有所述类别标签Hb的待标注样本图像集，确定为所述样本图像集Cb。
  10. 根据权利要求9所述的方法，其特征在于，所述根据所述区域坐标，生成所述原始图像Ef对应的待标注样本图像，包括：
    根据所述区域坐标，生成包括所述关键区域的初始检测框，对所述初始检测框进行扩张处理，得到包括目标对象的检测框；所述关键区域属于所述目标对象；
    在所述检测框中，获取包括所述目标对象的待缩放图像，获取第一图像尺寸，根据所述第一图像尺寸，对所述待缩放图像进行缩放处理，得到待裁剪图像；
    获取第二图像尺寸，根据所述第二图像尺寸，对所述待裁剪图像进行裁剪处理，得到所述原始图像Ef对应的待标注样本图像；所述第二图像尺寸小于所述第一图像尺寸。
  11. 一种数据处理方法，其特征在于，包括：
    获取目标图像，将所述目标图像输入至图像识别优化模型；所述图像识别优化模型是根据混合样本训练集继续对图像识别模型进行优化训练所得到的；所述混合样本训练集包括A个样本图像集、A个类别标签、正确类别标签以及对抗样本图像；所述A个样本图像集分别对应A个不同的类别标签中的一个，所述A个样本图像集包括采集样本图像；所述正确类别标签属于所述A个类别标签；A为大于1的正整数；所述图像识别模型是根据所述A个样本图像集预训练得到的；所述对抗样本图像是通过第一概率向量中的A个概率元素中指代错误类别标签的第一概率元素对采样样本图像进行调整得到，所述第一概率向量是由所述图像识别模型针对所述采样样本图像所生成的，所述错误类别标签与所述采样样本图像所属的样本图像集对应的类别标签不同，所述正确类别标签为所述采样样本图像所属的样本图像集对应的类别标签；所述第一概率向量包括A个概率元素，所述A个概率元素分别指代所述A个类别标签中的一个，一个所述概率元素用于表征所述采样样本图像属于所指代的类别标签的概率；所述对抗样本图像被所述图像识别模型进行类别预测所得到的类别预测结果为所述错误类别标签；
    通过所述图像识别优化模型，生成所述目标图像针对所述A个类别标签的目标概率向量；
    将所述目标概率向量中的最大概率元素对应的类别标签，确定为所述目标图像的类别预测结果。
  12. 一种数据处理装置，其特征在于，包括：
    数据获取模块，用于获取A个样本图像集以及图像识别模型，A为大于1的正整数，所述A个样本图像集分别对应A个不同的类别标签中的一个，所述图像识别模型是根据所述A个样本图像集预训练得到的，所述A个类别标签包括错误类别标签；
    第一输入模块，用于在所述A个样本图像集中获取采样样本图像，将所述采样样本图像与所述错误类别标签输入至所述图像识别模型，以生成所述采样样本图像针对所述A个类别标签的第一概率向量，所述错误类别标签与所述采样样本图像所属的样本图像集对应的类别标签不同；所述第一概率向量包括A个概率元素，所述A个概率元素分别指代所述A个类别标签中的一个，一个所述概率元素用于表征所述采样样本图像属于所指代的类别标签的概率；
    第一调整模块，用于在所述第一概率向量中的所述A个概率元素中，获取指代所述错误类别标签的第一概率元素，根据所述第一概率元素，对所述采样样本图像进行调整，得到所述采样样本图像对应的对抗样本图像，所述对抗样本图像被所述图像识别模型进行类别预测所得到的类别预测结果为所述错误类别标签。
  13. 一种数据处理装置，其特征在于，包括：
    图像获取模块，用于获取目标图像，将所述目标图像输入至图像识别优化模型；所述图像识别优化模型是根据混合样本训练集继续对图像识别模型进行优化训练所得到的；所述混合样本训练集包括A个样本图像集、A个类别标签、正确类别标签以及对抗样本图像；所述A个样本图像集分别对应A个不同的类别标签中的一个，所述A个样本图像集包括采集样本图像；所述正确类别标签属于所述A个类别标签；A为大于1的正整数；所述图像识别模型是根据所述A个样本图像集预训练得到的；所述对抗样本图像是通过第一概率向量中的A个概率元素中指代错误类别标签的第一概率元素对采样样本图像进行调整得到，所述第一概率向量是由所述图像识别模型针对所述采样样本图像所生成的，所述错误类别标签与所述采样样本图像所属的样本图像集对应的类别标签不同，所述正确类别标签为所述采样样本图像所属的样本图像集对应的类别标签；所述第一概率向量包括A个概率元素，所述A个概率元素分别指代所述A个类别标签中的一个，一个所述概率元素用于表征所述采样样本图像属于所指代的类别标签的概率；所述对抗样本图像被所述图像识别模型进行类别预测所得到的类别预测结果为所述错误类别标签；
    向量生成模块，用于通过所述图像识别优化模型，生成所述目标图像针对所述A个类别标签的目标概率向量；
    类别确定模块，用于将所述目标概率向量中的最大概率元素对应的类别标签，确定为所述目标图像的类别预测结果。
  14. 一种计算机设备，其特征在于，包括：处理器、存储器以及网络接口；
    所述处理器与所述存储器、所述网络接口相连，其中，所述网络接口用于提供数据通信功能，所述存储器用于存储计算机程序，所述处理器用于调用所述计算机程序，以使得所述计算机设备执行权利要求1至11任一项所述的方法。
  15. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质中存储有计算机程序，所述计算机程序适于由处理器加载并执行，以使得具有所述处理器的计算机设备执行权利要求1-11任一项所述的方法。
  16. 一种计算机程序产品，其特征在于，所述计算机程序产品包括计算机程序，所述计算机程序存储在计算机可读存储介质中，所述计算机程序适于由处理器读取并执行，以使得具有所述处理器的计算机设备执行权利要求1-11任一项所述的方法。
PCT/CN2024/079587 2023-03-02 2024-03-01 一种数据处理方法、设备以及计算机可读存储介质 Ceased WO2024179575A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP24763255.7A EP4614456A4 (en) 2023-03-02 2024-03-01 DATA PROCESSING METHOD, AND COMPUTER-READABLE STORAGE DEVICE AND MEDIUM
US19/072,375 US20250200954A1 (en) 2023-03-02 2025-03-06 Data processing method, device, and computer-readable storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310237098.1A CN116977692A (zh) 2023-03-02 2023-03-02 一种数据处理方法、设备以及计算机可读存储介质
CN202310237098.1 2023-03-02

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US19/072,375 Continuation US20250200954A1 (en) 2023-03-02 2025-03-06 Data processing method, device, and computer-readable storage medium

Publications (1)

Publication Number Publication Date
WO2024179575A1 true WO2024179575A1 (zh) 2024-09-06

Family

ID=88471972

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/079587 Ceased WO2024179575A1 (zh) 2023-03-02 2024-03-01 一种数据处理方法、设备以及计算机可读存储介质

Country Status (4)

Country Link
US (1) US20250200954A1 (zh)
EP (1) EP4614456A4 (zh)
CN (1) CN116977692A (zh)
WO (1) WO2024179575A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119004222A (zh) * 2024-10-23 2024-11-22 中国科学技术大学 提升人体行为识别系统对抗攻击识别准确率的方法及系统

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116977692A (zh) * 2023-03-02 2023-10-31 腾讯科技（深圳）有限公司 一种数据处理方法、设备以及计算机可读存储介质
CN120782732B (zh) * 2025-06-30 2026-02-03 广东工业大学 基于机器视觉的输电线路绝缘子缺陷检测方法和系统
CN121259493B (zh) * 2025-12-05 2026-03-03 中国人民解放军国防科技大学 数据变换触发增强数字域对抗样本生成方法、装置及设备

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110210617A (zh) * 2019-05-15 2019-09-06 北京邮电大学 一种基于特征增强的对抗样本生成方法及生成装置
CN111046380A (zh) * 2019-12-12 2020-04-21 支付宝(杭州)信息技术有限公司 一种基于对抗样本增强模型抗攻击能力的方法和系统
WO2021189364A1 (zh) * 2020-03-26 2021-09-30 深圳先进技术研究院 一种对抗图像生成方法、装置、设备以及可读存储介质
CN113569611A (zh) * 2021-02-08 2021-10-29 腾讯科技（深圳）有限公司 图像处理方法、装置、计算机设备和存储介质
EP4075395A2 (en) * 2021-08-25 2022-10-19 Beijing Baidu Netcom Science Technology Co., Ltd. Method and apparatus of training anti-spoofing model, method and apparatus of performing anti-spoofing, and device
US20230022943A1 (en) * 2021-07-22 2023-01-26 Xidian University Method and system for defending against adversarial sample in image classification, and data processing terminal
CN116977692A (zh) * 2023-03-02 2023-10-31 腾讯科技（深圳）有限公司 一种数据处理方法、设备以及计算机可读存储介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110210617A (zh) * 2019-05-15 2019-09-06 北京邮电大学 一种基于特征增强的对抗样本生成方法及生成装置
CN111046380A (zh) * 2019-12-12 2020-04-21 支付宝(杭州)信息技术有限公司 一种基于对抗样本增强模型抗攻击能力的方法和系统
WO2021189364A1 (zh) * 2020-03-26 2021-09-30 深圳先进技术研究院 一种对抗图像生成方法、装置、设备以及可读存储介质
CN113569611A (zh) * 2021-02-08 2021-10-29 腾讯科技（深圳）有限公司 图像处理方法、装置、计算机设备和存储介质
US20230022943A1 (en) * 2021-07-22 2023-01-26 Xidian University Method and system for defending against adversarial sample in image classification, and data processing terminal
EP4075395A2 (en) * 2021-08-25 2022-10-19 Beijing Baidu Netcom Science Technology Co., Ltd. Method and apparatus of training anti-spoofing model, method and apparatus of performing anti-spoofing, and device
CN116977692A (zh) * 2023-03-02 2023-10-31 腾讯科技（深圳）有限公司 一种数据处理方法、设备以及计算机可读存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4614456A4 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119004222A (zh) * 2024-10-23 2024-11-22 中国科学技术大学 提升人体行为识别系统对抗攻击识别准确率的方法及系统

Also Published As

Publication number Publication date
EP4614456A1 (en) 2025-09-10
US20250200954A1 (en) 2025-06-19
CN116977692A (zh) 2023-10-31
EP4614456A4 (en) 2025-09-10

Similar Documents

Publication Publication Date Title
US20200342314A1 (en) Method and System for Detecting Fake News Based on Multi-Task Learning Model
CN112559800B (zh) 用于处理视频的方法、装置、电子设备、介质和产品
US20210295114A1 (en) Method and apparatus for extracting structured data from image, and device
CN109034069B (zh) 用于生成信息的方法和装置
EP4614456A1 (en) Data processing method, and device and computer-readable storage medium
CN112131881B (zh) 信息抽取方法及装置、电子设备、存储介质
US20240320807A1 (en) Image processing method and apparatus, device, and storage medium
CN114219971B (zh) 一种数据处理方法、设备以及计算机可读存储介质
CN116975347B (zh) 图像生成模型训练方法及相关装置
CN116980541A (zh) 视频编辑方法、装置、电子设备以及存储介质
CN114282258A (zh) 截屏数据脱敏方法、装置、计算机设备及存储介质
CN113361629A (zh) 一种训练样本生成的方法、装置、计算机设备及存储介质
CN116977457A (zh) 一种数据处理方法、设备以及计算机可读存储介质
CN116910199A (zh) 基于人工智能的智能问答处理方法、装置、设备及介质
CN114820885A (zh) 图像编辑方法及其模型训练方法、装置、设备和介质
CN116304146A (zh) 图像处理方法及相关装置
CN111491209A (zh) 视频封面确定方法、装置、电子设备和存储介质
CN118823184A (zh) 图像生成、大模型的训练、图像处理方法及装置、设备和介质
CN111353554B (zh) 预测缺失的用户业务属性的方法及装置
CN111553748B (zh) 一种基于用户场景的Android微服务推荐方法与系统
CN116150415B (zh) 用户画像的构建方法、装置和电子设备
CN115880702A (zh) 数据处理方法、装置、设备、程序产品及存储介质
CN115909390A (zh) 低俗内容识别方法、装置、计算机设备以及存储介质
CN117540306B (zh) 一种多媒体数据的标签分类方法、装置、设备及介质
CN115952295B (zh) 基于图像的实体关系标注模型处理方法及其相关设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24763255

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024763255

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2024763255

Country of ref document: EP

Effective date: 20250605

WWP Wipo information: published in national office

Ref document number: 2024763255

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE