JP2020003879A - Information processing device, information processing method, watermark detection device, watermark detection method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To identify the source of a learning model. A model acquisition unit 110 is a learning model machine-learned to output an identification label composed of a plurality of index values indicating the probability that input image data is image data of a plurality of identification targets. To get. The watermark pattern acquisition unit 111 acquires a watermark pattern to be embedded in the learning model. The image acquisition unit 112 acquires a watermark image data set forming re-learning learning data used for re-learning for embedding a watermark pattern in a learning model. The superimposition pattern acquisition unit 113 acquires a plurality of patterns having the same number of patterns as the image data included in the watermark image data set, and the superposition of the patterns makes a watermark pattern. The learning data generation unit 114 generates a combination of image data included in the watermark image data set and a plurality of patterns as re-learning learning data. [Selection diagram] Fig. 4

Description

  The present invention relates to an information processing device, an information processing method, a watermark detection device, a watermark detection method, and a program, and particularly to a technology for embedding a watermark in a learning model and a technology for detecting a watermark embedded in a learning model.

  In recent years, CPUs (Central Processing Units) and GPUs (Graphics Processing Units) have become faster, memories have a larger capacity, and machine learning techniques have been rapidly advanced. For this reason, machine learning using learning data on the order of hundreds of thousands to one million is possible, and a highly accurate identification technology and classification technology are being established (see Non-Patent Document 1).

Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell.Caffe: Convolutional architecture for fast feature embedding.In Proceedings of the 22nd ACM international conference on Multimedia (pp. 675-678) ACM.

  Executing machine learning based on a large amount of learning data requires a large amount of calculation cost. In addition, a large amount of labor is required for preparing a large amount of learning data and for preprocessing for processing the prepared learning data for use in machine learning. On the other hand, a learning model generated by machine learning is digital data, and its copying is easy. Further, it is generally difficult to estimate learning data used for generating a learning model from the learning model itself.

  Therefore, it is difficult for a person who has generated the learning model to prove the fraud even if the learning model is fraudulently used by a third party. The collected learning data and the learning model generated based on the learning data are valuable values obtained with effort, and it is desired to protect the learning model from unauthorized use.

  The present invention has been made in view of these points, and an object of the present invention is to provide a technique that enables the source of a learning model to be specified.

  A first aspect of the present invention is an information processing device. This apparatus obtains a learning model machine-learned so as to output an identification label including a plurality of index values indicating the probability that the input image data is image data of a plurality of identification targets, respectively. A watermark pattern acquisition unit for acquiring a watermark pattern to be embedded in the learning model; and acquiring a watermark image data set constituting learning data for re-learning used for re-learning for embedding the watermark pattern in the learning model. An image acquisition unit that performs the same number of patterns as the image data included in the watermark image data set, and a superimposition pattern acquisition unit that acquires a plurality of patterns in which each pattern overlaps the watermark pattern; The combination of the image data included in the watermark image data set and the plurality of patterns is Comprising a learning data generating unit that generates a 習用 training data.

  The information processing apparatus may further include a model application unit that obtains a plurality of identification labels that are outputs of the learning model, each of which receives image data included in the watermark image data set, and the superimposition pattern The acquiring unit is an identification label obtained by correcting at least a part of the plurality of identification labels so that a superimposition of the plurality of identification labels acquired by the model application unit becomes the watermark pattern. The learning data generator may generate a combination of each identification label and the image data corresponding to the identification label as the learning data for re-learning.

  The information processing device may further include a learning execution unit that re-learns the learning model using the learning data for re-learning.

  The identification label may be a vector composed of the same number of elements as the plurality of identification targets, the watermark pattern may be an image composed of the same number of pixels as the elements of the identification label, The learning data generation unit includes: a conversion unit configured to convert each element of the plurality of identification labels into two-dimensional data according to a predetermined procedure; and each of the plurality of two-dimensional data converted by the conversion unit and the watermark pattern. And a correction unit that corrects each of the two-dimensional data based on the two-dimensional data, and the conversion unit may convert the corrected two-dimensional data into the format of the identification label.

  When the difference between the two-dimensional data converted by the conversion unit and the watermark pattern satisfies a predetermined exclusion condition, the learning data generation unit converts image data corresponding to the two-dimensional data from the watermark image data set. An image selection unit to be excluded may be further provided.

  The watermark pattern obtaining unit may obtain two or more different watermark patterns for embedding in the learning model, and the image obtaining unit may obtain a plurality of the watermark image data sets corresponding to the different watermark patterns, respectively. The learning data generating unit may generate the learning data for re-learning corresponding to each of the different watermark patterns.

  A second aspect of the present invention is an information processing method. In this method, the processor acquires a learning model machine-learned to output an identification label including a plurality of index values indicating the probability that the input image data is image data of each of the plurality of identification targets. Acquiring a watermark pattern for embedding in the learning model, and acquiring a watermark image data set constituting learning data for re-learning used for re-learning for embedding the watermark pattern in the learning model. Obtaining a plurality of patterns, the number of which is the same as the number of image data included in the watermark image data set, and each pattern being superimposed is the watermark pattern; and Learning for re-learning the combination of image data and the plurality of patterns Performing the steps of generating as over data, the.

  A third aspect of the present invention is a program. This program acquires a learning model machine-learned to output, to a computer, an identification label composed of a plurality of index values indicating the probability that input image data is image data of each of a plurality of identification targets. A function for acquiring a watermark pattern for embedding in the learning model, and a function for acquiring a watermark image data set constituting learning data for re-learning used for re-learning for embedding the watermark pattern in the learning model. And a function of acquiring a plurality of patterns that are the same number as the image data included in the watermark image data set and in which each pattern is superimposed becomes the watermark pattern; and a function included in the watermark image data set. A combination of image data and the plurality of patterns is generated as the re-learning learning data. A function of, to realize.

  A fourth aspect of the present invention is a watermark detection device. This apparatus outputs an identification label, which is a learning model to be subjected to watermark detection, and is composed of a plurality of index values indicating the probability that input image data is image data of each of a plurality of identification targets. A model acquisition unit that acquires a learning model that has been machine-learned; an image acquisition unit that acquires a watermark image data set for detecting a watermark embedded in the learning model; and image data included in the watermark image data set. A model application unit that acquires a plurality of identification labels that are outputs of the learning model, each of which is an input, and a pattern output unit that outputs a pattern generated by superimposing the plurality of identification labels.

  The watermark detection device includes a watermark pattern acquisition unit that acquires a watermark pattern to be verified, and a pattern presentation unit that presents the pattern output by the pattern output unit and the watermark pattern in a comparable manner. Is also good.

  A fifth aspect of the present invention is a watermark detection method. In this method, the processor outputs an identification label, which is a learning model to be subjected to watermark detection, and includes a plurality of index values indicating the probability that the input image data is image data of each of the plurality of identification targets. Acquiring a learning model machine-learned so as to obtain a watermark image data set for detecting a watermark embedded in the learning model, and acquiring image data sets included in the watermark image data set. And a step of obtaining a plurality of identification labels that are outputs of the learning model, and outputting a pattern generated by superimposing the plurality of identification labels.

  A sixth aspect of the present invention is a program. This program outputs, to a computer, an identification label that is a learning model to be subjected to watermark detection and that is configured with a plurality of index values indicating the probability that the input image data is image data of a plurality of identification targets. A function of acquiring a learning model machine-learned so as to acquire a watermark image data set for detecting a watermark embedded in the learning model, and a function of acquiring a watermark image data set included in the watermark image data set. And a function of outputting a pattern generated by superimposing the plurality of identification labels, the function of acquiring a plurality of identification labels which are outputs of the learning model.

  In order to provide the above-mentioned program or update a part of the program, a computer-readable recording medium on which the program is recorded may be provided, and the program may be transmitted via a communication line. .

  Note that any combination of the above-described components and any conversion of the expression of the present invention between a method, an apparatus, a system, a computer program, a data structure, a recording medium, and the like are also effective as embodiments of the present invention.

  ADVANTAGE OF THE INVENTION According to this invention, the technique which can specify the source of a learning model can be provided.

It is a schematic diagram for explaining the learning model for image discrimination. FIG. 4 is a schematic diagram for explaining fine tuning of a learning model. It is a figure which shows typically a mode that a watermark image is formed by superimposing several identification labels. FIG. 2 is a diagram schematically illustrating a functional configuration of the information processing apparatus according to the embodiment; FIG. 7 is a diagram for explaining an example of correction of an identification label by the learning data generation unit according to the embodiment. It is a figure which shows typically the internal structure of the learning data generation part which concerns on embodiment. FIG. 2 is a diagram schematically illustrating a functional configuration of the watermark detection device according to the embodiment. FIG. 5 is a diagram illustrating an example of a pattern output by a pattern output unit according to the embodiment. FIG. 9 is a diagram schematically illustrating an example of a comparison screen presented by a pattern presentation unit 215 according to the embodiment. 5 is a flowchart illustrating a flow of a process performed by the information processing device according to the embodiment.

<Outline of Embodiment>
Hereinafter, an outline of an embodiment will be described with reference to FIGS. 1, 2, and 3. FIG.

  FIGS. 1A and 1B are schematic diagrams for explaining a learning model for image discrimination. The information processing apparatus according to the embodiment is an apparatus for embedding watermark data in a learning model for image discrimination created using a known machine learning technique such as a neural network or SVM (Support Vector Machine). . The watermark detection device according to the embodiment is a device for detecting an embedded watermark from a learning model in which the information processing device has embedded the watermark.

  1A and 1B, a learning model M is a model in which a plurality of animals are identified. In the example shown in FIGS. 1A and 1B, the learning model M can identify 100 types of animals including cats, monkeys, dogs, elephants, sheep, giraffes, and the like. When image data is input to the learning model M, the learning model M outputs an identification label L including a plurality of index values indicating the probability that the input data is image data of each of the plurality of identification targets. Although not limited, the identification label L output by the learning model M is data in which 100 kinds of values corresponding to 100 kinds of animals to be identified are arranged, and typically a vector having 100 elements. Data. Each element of the identification label L takes a value of 0 or more and 1 or less, and indicates that the image data input to the learning model M is 100 kinds of animals.

  For example, in the identification label Lc output from the learning model M when the cat image Ic is input to the learning model M, the element corresponding to “cat” has a large value, and the other elements have small values. FIGS. 1A and 1B show that the larger the value of the element of the identification label is, the blacker it is, and the smaller the value of the element is, the whiter it is. Therefore, in the identification label Lc shown in FIG. 1A, the element corresponding to "cat" is black and the other elements are white. Similarly, in the identification label Lm output from the learning model M when the monkey image Im is input to the learning model M, the element corresponding to “monkey” is black, and the other elements are white.

  In general, when image data including a subject that is not an object to be identified by the learning model M is input to the learning model M, each element of the identification label L output by the learning model M is a random value that is difficult to predict. . In FIG. 1B, when a landscape image Il that the learning model M is not to identify is input to the learning model M, the learning model M outputs an identification label Ll in which each element is a random value. . Similarly, even if the snow model image Is not input to the learning model M is input to the learning model M, the learning model M outputs an identification label Ls in which each element is a random value. Here, as shown in FIG. 1B, when the image data input to the learning model M is different, the pattern of the output identification label L is also different.

  First, the information processing apparatus according to the embodiment acquires a watermark image to be embedded in a learning model and a learning model to which the watermark image is to be embedded. Next, the information processing apparatus prepares an image data set in which a subject not identified by the learning model M is identified. Lastly, the information processing apparatus superimposes a plurality of identification labels L output when each of the plurality of image data constituting the image data set is input to the learning model M, so that the learning model can be obtained as a watermark image. Re-learn M (fine-tuning).

  FIG. 2 is a schematic diagram for explaining fine tuning of the learning model M. Both the landscape image Il and the snowman image Is shown in FIG. 1B are a part of the image data set Iw acquired by the information processing device for watermark embedding and watermark detection.

  The information processing apparatus according to the embodiment prepares a random image group that is the same number of random images as the image data set Iw and becomes a watermark image W when each random image is superimposed. The information processing apparatus generates a vector including the elements of each random image and sets the vector as the teacher identification label Lt. The information processing apparatus generates learning data for re-learning in which the teacher identification label Lt is associated with the image data forming the image data set Iw.

  When the image data is input to the learning model M, the information processing apparatus finely tunes the learning model M so as to output an identification label associated with the image data. For example, when the learning model M is generated using a neural network, an output when image data constituting the image data set Iw is input to the learning model M, and a teacher identification label Lt associated with the image data. And get the error. The information processing device updates the parameters of the learning model M by the error backpropagation method based on the acquired error.

  The information processing apparatus generates learning data for re-learning by associating an identification label similar to an identification label output when image data is input to the learning model M before re-learning with the image data. Is also good. Thereby, the error between the output when the image data constituting the image data set Iw is input to the learning model M and the teacher identification label Lt associated with the image data is reduced, and the convergence of the re-learning is accelerated. Can be expected.

  FIG. 3 is a diagram schematically illustrating a state in which a watermark image W is formed by overlapping a plurality of identification labels L. The watermark detection device according to the embodiment inputs each image data constituting the image data set Iw to the retrained learning model M, and acquires a plurality of identification labels Lo to be output. Although the identification label Lo is a vector having 100 rows and 1 column, the watermark detection apparatus rearranges the elements of each identification label Lo to generate image data of 10 rows and 10 columns, as shown in FIG. The watermark detection device obtains the watermark image W by superimposing images generated by rearranging the elements of the identification label Lo. FIG. 3 shows an example in which the watermark image is an image in which letters of the alphabet “au” are drawn.

  The information processing device stores the image data set Iw and the watermark image W used for the fine tuning of the learning model M in a safe storage location including the information processing device that cannot be falsified from the outside. When acquiring the learning model M to be subjected to watermark detection, the watermark detection device acquires the image data set Iw from the storage location and inputs the acquired image data set Iw to the learning model M to be subjected to watermark detection. The watermark detection device compares the image generated by superimposing the identification labels Lo output by the learning model M with the watermark image W obtained from the storage location. As a result of the comparison, if the images match, the watermark detection device determines that the learning model M to be subjected to the watermark detection is the learning model M in which the information processing device 1 has embedded the watermark image.

  As described above, the information processing apparatus according to the embodiment can embed the watermark image W in the learning model M. Further, the watermark detection device according to the embodiment can extract the watermark image W from the learning model M in which the watermark image W is embedded. Thus, the information processing device and the watermark detection device according to the embodiment can specify the source of the learning model M.

<Functional Configuration of Information Processing Apparatus 1 According to Embodiment>
FIG. 4 is a diagram schematically illustrating a functional configuration of the information processing apparatus 1 according to the embodiment. The information processing device 1 includes a storage unit 10 and a control unit 11. In FIG. 4, arrows indicate main data flows, and there may be data flows not shown in FIG. In FIG. 4, each functional block shows not a configuration of a hardware (device) unit but a configuration of a functional unit. Therefore, the functional blocks shown in FIG. 4 may be implemented in a single device, or may be separately implemented in a plurality of devices. The transfer of data between the functional blocks may be performed via any means such as a data bus, a network, a portable storage medium, and the like.

  The storage unit 10 includes a read only memory (ROM) for storing a basic input output system (BIOS) of a computer that implements the information processing device 1, a random access memory (RAM) serving as a work area of the information processing device 1, and an OS ( It is a large-capacity storage device such as a hard disk drive (HDD) or a solid state drive (SSD) that stores an operating system, an application program, and various information referred to when the application program is executed.

  The control unit 11 is a processor such as a CPU or a GPU of the information processing device 1, and executes a program stored in the storage unit 10 to execute a model acquisition unit 110, a watermark pattern acquisition unit 111, an image acquisition unit 112, It functions as the use pattern acquisition unit 113, the learning data generation unit 114, the model application unit 115, and the learning execution unit 116.

  FIG. 4 shows an example in which the information processing device 1 is configured by a single device. However, the information processing apparatus 1 may be implemented by a plurality of processors and computation resources such as memories, such as a cloud computing system. In this case, each unit constituting the control unit 11 is realized by at least one of a plurality of different processors executing a program.

  The model acquisition unit 110 acquires a learning model M. The learning model M acquired by the model acquiring unit 110 is a learning model M that has been machine-learned so as to output an identification label L when an image is input. Here, the identification label L output by the learning model M is an identification label composed of a plurality of index values indicating the probability that the input image data is the image data of each of the plurality of identification targets.

  The watermark pattern acquisition unit 111 acquires a watermark pattern to be embedded in the learning model M. The watermark pattern acquired by the watermark pattern acquisition unit 111 is, for example, the watermark image W described above. However, the watermark pattern acquired by the watermark pattern acquisition unit 111 is not limited to image data, and may be, for example, a one-dimensional vector.

  The image obtaining unit 112 obtains a watermark image data set constituting learning data for re-learning used for re-learning for embedding a watermark pattern in the learning model M. A specific example of the watermark image data set acquired by the image acquisition unit 112 is the above-described image data set Iw.

  The superimposition pattern acquisition unit 113 acquires a plurality of patterns that are the same number as the image data included in the watermark image data set, and in which the superposition of each pattern becomes a watermark pattern. Details of the plurality of patterns acquired by the superimposition pattern acquisition unit 113 will be described later.

  The learning data generating unit 114 generates a combination of the image data included in the watermark image data set and a plurality of patterns as learning data for re-learning. Thereby, the information processing apparatus 1 can generate learning data for re-learning used in fine tuning for embedding the watermark pattern in the learning model M in which the watermark pattern is to be embedded. For this reason, the information processing apparatus 1 can specify the source of the learning model M.

  Here, the model applying unit 115 acquires a plurality of identification labels L which are outputs of the learning model M which receives image data included in the watermark image data set, respectively. The superimposition pattern acquisition unit 113 converts an identification label obtained by correcting at least a part of the plurality of identification labels L so that the superposition of the plurality of identification labels acquired by the model application unit 115 becomes a watermark pattern. It is acquired as the above-mentioned pattern. The learning data generation unit 114 generates a combination of each identification label and image data corresponding to the identification label as learning data for re-learning.

  FIGS. 5A and 5B are diagrams illustrating an example of correction of an identification label by the learning data generation unit 114 according to the embodiment. FIG. 6 is a diagram schematically illustrating an internal configuration of the learning data generation unit 114 according to the embodiment.

  As described above, the identification label L output by the learning model M is a vector including the same number of elements as the plurality of identification targets that can be identified by the learning model M. When the watermark pattern is the watermark image W, the watermark image W is an image including the same number of pixels as the elements of the identification label L.

  To correct the identification label, the learning data generation unit 114 includes a conversion unit 1140 and a correction unit 1141. The conversion unit 1140 converts each element of the plurality of identification labels into two-dimensional data having the same array as the pixel array of the watermark image W, which is a watermark pattern, according to a predetermined procedure. In FIGS. 5A and 5B, the image denoted by reference symbol L is an image converted by the conversion unit 1140 from the identification label L. As shown in FIGS. 5A and 5B, since the image data included in the watermark image data set is not an identification target of the learning model M, the identification label L converted into an image is similar to a noise image. It is an image.

  The correction unit 1141 corrects each two-dimensional data based on each of the plurality of two-dimensional data converted by the conversion unit 1140 and the watermark image W that is a watermark pattern. Specifically, in FIGS. 5A and 5B, symbols indicated by white circles represent Hadamard products. The conversion unit 1140 sets the Hadamard product of the watermark image W and the imaged identification label L as the above-described pattern (that is, the teacher identification label Lt used for re-learning). Accordingly, the model applying unit 115 can acquire the teacher identification label Lt including the feature of the watermark image W while leaving the feature of each identification label L.

  When the pixel value of the background of the watermark image W (the area other than the character “au” in the example of FIG. 3) is 0, the area corresponding to the background of the teacher identification label Lt is also 0. If the pixel value of the background of the watermark image W is a real number close to 0, the area corresponding to the background of the teacher identification label Lt also has a value other than 0. FIG. 3 shows an example where the background pixel value of the watermark image W is a value other than 0.

  Returning to the description of FIG. The learning execution unit 116 re-learns the learning model M by fine tuning using the re-learning learning data generated by the learning data generation unit 114. Thereby, the learning execution unit 116 can embed the watermark image W in the learning model M.

  Here, the learning execution unit 116 may set the learning rate at the time of re-learning the learning model M (the update width of the model parameters of the learning model M) to be smaller than the learning rate at the time of generating the learning model M. Accordingly, the learning execution unit 116 can suppress the learning model M from being over-learned by the learning data for re-learning.

  The learning execution unit 116 uses data obtained by adding the learning data used when generating the learning model M, in addition to the learning data for re-learning generated by the learning data generation unit 114 when re-learning the learning model M. Then, the learning model M may be re-learned. At this time, the learning execution unit 116 determines the number of image data to be added to the re-learning learning data generated by the learning data generation unit 114 in order to prevent the learning model M from being over-learned by the re-learning learning data. May be larger than the image data included in the re-learning learning data generated by the learning data generation unit 114. The ratio between the number of learning data for re-learning generated by the learning data generation unit 114 and the number of learning data to be added is determined by experiments in consideration of suppression of over-learning and costs for re-learning. Good, but for example one to ten. Accordingly, the learning execution unit 116 can suppress the learning model M from being over-learned by the learning data for re-learning.

  As described with reference to FIG. 2, the learning execution unit 116 determines based on the error between the identification label L output by the learning model M when the learning data for re-learning is input and the identification label Lt for teacher. , The parameters of the learning model M are updated based on the backpropagation method. For this reason, it is considered that the smaller the error between the identification label L and the teacher identification label Lt, the faster the re-learning convergence and the easier it is to maintain the identification performance that the learning model M after re-learning had before learning. .

  Therefore, the learning data generation unit 114 also includes an image selection unit 1142. When the difference between the two-dimensional data converted by the conversion unit 1140 and the watermark image W as the watermark pattern satisfies a predetermined exclusion condition, the image selection unit 1142 converts the image data corresponding to the two-dimensional data into a watermark image data set. Exclude from

  Here, the “predetermined exclusion condition” is an “image selection condition” that is referred to when the image selection unit 1142 selects the image data constituting the re-learning image data. The specific contents of the predetermined exclusion condition may be determined in consideration of the convergence of learning, the characteristics of the watermark image W, and the like. As an example, the intensity of the difference image between the two-dimensional data and the watermark image W (for example, the sum of the absolute values of the pixel values of the difference image or the sum of the squares of the pixel values of the difference image) is equal to or more than half of the intensity of the watermark image W. May be a predetermined exclusion condition.

  Accordingly, the learning data generation unit 114 can improve the convergence of the fine tuning of the learning model M by the learning execution unit 116, and can reduce the cost for re-learning.

  Although the description has been made on the assumption that one watermark pattern is embedded in the learning model M, a plurality of watermark patterns may be provided. In this case, the watermark pattern acquisition unit 111 acquires two or more different watermark patterns to be embedded in the learning model M. The image acquisition unit 112 acquires a plurality of watermark image data sets corresponding to different watermark patterns. Thereby, the learning data generation unit 114 can generate learning data for re-learning corresponding to each of the different watermark patterns. The model application unit 115 can embed a plurality of types of watermark patterns in the learning model M by fine tuning the learning model M using the learning data for re-learning generated by the learning data generation unit 114.

  The process in which the information processing apparatus 1 embeds the watermark pattern in the learning model M has been described above. Next, a process of detecting a watermark pattern from the learning model M in which the watermark pattern is embedded will be described.

<Functional Configuration of Watermark Detection Apparatus 2 According to Embodiment>
FIG. 7 is a diagram schematically illustrating a functional configuration of the watermark detection device 2 according to the embodiment. The watermark detection device 2 includes a storage unit 20 and a control unit 21. In FIG. 7, arrows indicate main data flows, and there may be data flows not shown in FIG. In FIG. 7, each functional block shows a configuration of a function unit, not a configuration of a hardware (device) unit. Therefore, the functional blocks illustrated in FIG. 7 may be implemented in a single device, or may be separately implemented in a plurality of devices. The transfer of data between the functional blocks may be performed via any means such as a data bus, a network, a portable storage medium, and the like.

  The storage unit 20 stores a ROM for storing a BIOS or the like of a computer that implements the watermark detection device 2, a RAM serving as a work area of the watermark detection device 2, an OS, an application program, and various information referred to when the application program is executed. It is a large-capacity storage device such as an HDD or SSD for storing.

  The control unit 21 is a processor such as a CPU and a GPU of the watermark detection device 2, and executes a program stored in the storage unit 20 to execute a model acquisition unit 210, an image acquisition unit 211, a model application unit 212, and a pattern output unit. Function as a unit 213, a watermark pattern acquisition unit 214, and a pattern presentation unit 215.

  FIG. 7 shows an example in which the watermark detection device 2 is configured by a single device. However, the watermark detection device 2 may be realized by a plurality of processors and computational resources such as memories as in a cloud computing system. In this case, each unit constituting the control unit 11 is realized by at least one of a plurality of different processors executing a program. The information processing device 1 may include at least a part or all of the functions of the watermark detection device 2.

  The model acquisition unit 210 acquires a learning model M to be subjected to watermark detection. The learning model M is a learning model that has been machine-learned so as to output an identification label L including a plurality of index values indicating the probability that the input image data is image data of each of a plurality of identification targets.

  The image acquisition unit 211 acquires a watermark image data set for detecting a watermark embedded in the learning model M. Specifically, the watermark image data set acquired by the image acquisition unit 211 is the same as the watermark image data set constituting the learning data for re-learning used when the model application unit 115 fine-tunes the learning model M. It is.

  The model applying unit 212 acquires a plurality of identification labels L which are outputs of the learning model M which receives image data included in the watermark image data set as input. The pattern output unit 213 outputs a pattern generated by overlapping a plurality of identification labels L.

  FIGS. 8A to 8E are diagrams illustrating an example of a pattern output by the pattern output unit 213 according to the embodiment. Specifically, FIG. 8A is a diagram illustrating a pattern formed by superimposing 60 different identification labels L, and FIG. 8B is a diagram illustrating a pattern obtained by superimposing 240 different identification labels L. It is a figure showing a pattern. Similarly, FIG. 8C is a diagram showing a pattern formed by overlapping about 2000 different identification labels L, and FIG. 8D is a pattern formed by overlapping 3000 different identification labels L. FIG. FIG. 8E shows the original image of the watermark image W. When the pattern shown in FIG. 8D is subjected to the binarization process, the pattern matches the image shown in FIG.

  As shown in FIGS. 8A to 8D, when the number of identification labels L to be superimposed by the pattern output unit 213 is small, the watermark image W cannot be reproduced even when the identification labels L are superimposed. Thereby, even if some of the image data included in the watermark image data set leaks to a third party, it is difficult for the third party to specify the watermark image W if the number of leaked image data is small. It is. For this reason, the information processing apparatus 1 according to the embodiment can embed the watermark pattern in the learning model M in a manner that is difficult for a third party to estimate.

  Returning to the description of FIG. The watermark pattern acquisition unit 214 acquires a watermark pattern to be verified. As described above, the watermark image W, which is a watermark pattern, is stored together with the watermark image data set in a secure storage location that cannot be tampered with from the outside. The watermark pattern acquisition unit 214 reads and acquires the watermark image W from this storage location.

The pattern presentation unit 215 presents the pattern output by the pattern output unit 213 and the watermark pattern acquired by the watermark pattern acquisition unit 214 in a manner that allows comparison.
FIG. 9 is a diagram schematically illustrating an example of a comparison screen presented by the pattern presentation unit 215 according to the embodiment. FIG. 9 shows an example in which two types of watermark patterns “watermark A” and “watermark B” are embedded in the learning model M. In FIG. 9, “watermark A” is an image in which the letters “au” of the alphabet are drawn, and “watermark B” is an image in which a human face is drawn.

  As illustrated in FIG. 9, the pattern presenting unit 215 detects a watermark image W that is a watermark pattern embedded in the learning model M by the information processing device 1 and a learning model M that is a target of the watermark detection by the watermark detection device 2. The images are displayed side by side on the display unit D. Thereby, the user of the watermark detection device 2 can see at a glance whether or not the image detected from the learning model M as the target of the watermark detection matches the watermark image W.

  Further, the pattern presenting unit 215 determines whether or not the image detected from the learning model M to be subjected to the watermark detection matches the watermark image W by calculation, and presents the result to the display unit D. Is also good. Specifically, the pattern presentation unit 215 determines that the strength of the difference image between the watermark image W and the image detected from the learning model M that is the target of the watermark detection is a predetermined condition (for example, 10% or less of the watermark image W strength). Satisfies), it may be determined that they match. By displaying the determination result calculated by the pattern presentation unit 215 on the display unit D, the user of the watermark detection device 2 can determine whether the image detected from the learning model M to be subjected to the watermark detection matches the watermark image W. This can help in the determination.

<Processing Flow of Learning Method Performed by Information Processing Apparatus 1>
FIG. 10 is a flowchart illustrating a flow of a process performed by information processing device 1 according to the embodiment. The process in this flowchart is started, for example, when the information processing device 1 is started.

  The model acquisition unit 110 acquires a learning model M for embedding a watermark pattern (S2). The watermark pattern acquisition unit 111 acquires a watermark pattern to be embedded in the learning model M (S4). The image acquisition unit 112 acquires a watermark image data set that constitutes learning data for re-learning used for re-learning for embedding a watermark pattern in the learning model M (S6).

  The superimposition pattern acquisition unit 113 acquires a plurality of patterns that are the same number as the image data included in the watermark image data set and are configured so that the superposition of the patterns reproduces the watermark pattern (S8). ). The learning data generation unit 114 generates a combination of image data and a plurality of patterns included in the watermark image data set as re-learning learning data (S10).

  The learning execution unit 116 re-learns the learning model M by fine tuning using the learning data for re-learning (S12). Thereby, the learning execution unit 116 can embed the watermark pattern in the learning model M. When the learning execution unit 116 executes the re-learning, the processing in this flowchart ends.

<Effects of Information Processing Apparatus 1 According to Embodiment>
As described above, according to the information processing apparatus 1 according to the embodiment, the source of the learning model M can be specified.

  As described above, the present invention has been described using the embodiment. However, the technical scope of the present invention is not limited to the scope described in the above embodiment, and various modifications and changes can be made within the scope of the gist. is there. For example, the specific embodiment of the distribution / integration of the apparatus is not limited to the above-described embodiment, and all or a part of the apparatus may be functionally or physically distributed / integrated in an arbitrary unit. Can be. Further, new embodiments that are generated by arbitrary combinations of the plurality of embodiments are also included in the embodiments of the present invention. The effect of the new embodiment caused by the combination has the effect of the original embodiment.

<First Modification>
The case where the information processing apparatus 1 embeds the watermark pattern in the learning model M has been described above. Here, the watermark pattern does not have to be one independent pattern. For example, one image may be divided into two or more partial images, and each partial image may be used as a watermark pattern. At this time, the information processing apparatus 1 stores at least one partial image as a storage image without embedding it in the learning model M. In this case, the watermark detection device 2 obtains a complete image for the first time by detecting each watermark pattern from the learning model M, combining them, and combining with the storage image. Each partial image functions as an electronic "tally". Thus, even if a part of the watermark pattern leaks to the third party, it is difficult for the third party to compose the whole image from the watermark pattern, so that the security can be further improved.

<Second Modification>
In the above description, the learning data generation unit 114 converts the elements of each of the plurality of identification labels L into two-dimensional data having the same array as the pixel array of the watermark pattern according to a predetermined procedure. The case has been described in which each of the two-dimensional data is corrected based on the watermark pattern. Instead, the learning data generation unit 114 converts the pixel array of the watermark pattern into one-dimensional data having the same array as the identification label L according to a predetermined procedure, and outputs each of the plurality of converted one-dimensional data and the watermark pattern. May be used to modify the identification label L.

<Third Modification>
In the above, the case where the converting unit 1140 sets the Hadamard product of the watermark image W and the imaged identification label L as the teacher identification label Lt used for re-learning has been described. However, the method of acquiring the teacher identification label Lt is not limited to the above. For example, as a result of the Hadamard product of the watermark image W and the imaged identification label L, the conversion unit 1140 selects a predetermined number of pixel values in descending order, and the rest are 0 or a real number close to 0 (for example, the selected pixel (10% of the minimum value among the values). Furthermore, the conversion unit 1140 may binarize the teacher identification label Lt assuming that the value of the selected pixel is 1 and the values of the other pixels are 0 or a real number close to 0.

DESCRIPTION OF SYMBOLS 1 ... Information processing apparatus 10 ... Storage part 11 ... Control part 110 ... Model acquisition part 111 ... Watermark pattern acquisition part 112 ... Image acquisition part 113 ... Superposition pattern acquisition part 114 learning data generation unit 1140 conversion unit 1141 correction unit 1142 image selection unit 115 model application unit 116 learning execution unit 2 watermark detection device 20 ... Storage unit 21 ... Control unit 210 ... Model acquisition unit 211 ... Image acquisition unit 212 ... Model application unit 213 ... Pattern output unit 214 ... Watermark pattern acquisition unit 215 ... .Pattern presentation section D: Display section

Claims (12)

A model acquisition unit that acquires a learning model machine-learned to output an identification label including a plurality of index values indicating the probability that the input image data is the image data of each of the plurality of identification targets,
A watermark pattern acquisition unit that acquires a watermark pattern for embedding in the learning model,
An image acquisition unit that acquires a watermark image data set that constitutes learning data for re-learning used for re-learning for embedding the watermark pattern in the learning model;
A superimposition pattern acquisition unit that acquires the same number of patterns as the image data included in the watermark image data set, and acquires a plurality of patterns in which the superposition of the patterns is the watermark pattern.
A learning data generation unit that generates a combination of the image data and the plurality of patterns included in the watermark image data set as the re-learning learning data,
An information processing device comprising: The image processing apparatus further includes a model application unit that obtains a plurality of identification labels that are outputs of the learning model, each of which includes input image data included in the watermark image data set,
The superimposing pattern acquisition unit is an identification label obtained by correcting at least a part of the plurality of identification labels so that the superposition of the plurality of identification labels acquired by the model application unit becomes the watermark pattern. Is obtained as the pattern,
The learning data generation unit generates a combination of each identification label and the image data corresponding to the identification label as the learning data for re-learning,
The information processing device according to claim 1. Further comprising a learning execution unit for re-learning the learning model using the learning data for re-learning,
The information processing device according to claim 1. The identification label is a vector composed of the same number of elements as the plurality of identification targets,
The watermark pattern is an image composed of the same number of pixels as the elements of the identification label,
The learning data generation unit includes:
A conversion unit configured to convert each element of the plurality of identification labels into two-dimensional data according to a predetermined procedure;
A correction unit that corrects each of the two-dimensional data based on each of the plurality of two-dimensional data converted by the conversion unit and the watermark pattern,
The conversion unit converts the corrected two-dimensional data into a format of the identification label.
The information processing device according to claim 1. The learning data generation unit includes:
When the difference between the two-dimensional data converted by the conversion unit and the watermark pattern satisfies a predetermined exclusion condition, the image processing apparatus further includes an image selection unit that excludes image data corresponding to the two-dimensional data from the watermark image data set. ,
The information processing device according to claim 4. The watermark pattern acquisition unit acquires two or more different watermark patterns for embedding in the learning model,
The image acquisition unit acquires a plurality of watermark image data sets corresponding to each of the different watermark patterns,
The learning data generation unit generates the re-learning learning data corresponding to each of the different watermark patterns.
The information processing device according to claim 1. The processor
A step of acquiring a learning model machine-learned to output an identification label composed of a plurality of index values indicating the probability that the input image data is the image data of each of the plurality of identification targets,
Obtaining a watermark pattern for embedding in the learning model;
Acquiring a watermark image data set constituting retraining learning data used for retraining for embedding the watermark pattern in the learning model,
Acquiring the same number of patterns as the image data included in the watermark image data set, and obtaining a plurality of patterns in which each pattern is the watermark pattern,
Generating a combination of the image data and the plurality of patterns included in the watermark image data set as the learning data for re-learning,
Information processing method for executing. On the computer,
A function of acquiring a learning model machine-learned to output an identification label composed of a plurality of index values indicating the probability that the input image data is the image data of each of the plurality of identification targets,
A function of acquiring a watermark pattern for embedding in the learning model;
A function of acquiring a watermark image data set constituting learning data for re-learning used for re-learning for embedding the watermark pattern in the learning model,
A function of acquiring a plurality of patterns, which are the same number of patterns as the image data included in the watermark image data set, and in which each pattern is the watermark pattern,
A function of generating a combination of the image data and the plurality of patterns included in the watermark image data set as the learning data for re-learning,
The program that realizes. A learning model to be subjected to watermark detection, wherein the input image data is machine-learned so as to output an identification label including a plurality of index values indicating probabilities that are image data of a plurality of identification targets. A model acquisition unit for acquiring a learning model,
An image acquisition unit that acquires a watermark image data set for detecting a watermark embedded in the learning model;
A model application unit that obtains a plurality of identification labels that are outputs of the learning model, each of which receives image data included in the watermark image data set,
A pattern output unit that outputs a pattern generated by overlapping the plurality of identification labels,
A watermark detection device comprising: A watermark pattern acquisition unit that acquires a watermark pattern to be verified;
A pattern presenting unit that presents the pattern output by the pattern output unit and the watermark pattern in a comparable manner,
The watermark detection device according to claim 9, further comprising: The processor
A learning model to be subjected to watermark detection, wherein the input image data is machine-learned so as to output an identification label including a plurality of index values indicating probabilities that are image data of a plurality of identification targets. Obtaining a learning model;
Obtaining a watermark image data set for detecting a watermark embedded in the learning model;
Acquiring a plurality of identification labels that are outputs of the learning model with each of the image data included in the watermark image data set as an input,
Outputting a pattern generated by superimposing the plurality of identification labels,
Perform watermark detection method. On the computer,
A learning model to be subjected to watermark detection, wherein the input image data is machine-learned so as to output an identification label including a plurality of index values indicating probabilities that are image data of a plurality of identification targets. A function to acquire a learning model,
A function of acquiring a watermark image data set for detecting a watermark embedded in the learning model,
A function of acquiring a plurality of identification labels that are outputs of the learning model with each of the image data included in the watermark image data set as an input,
A function of outputting a pattern generated by superimposing the plurality of identification labels,
The program that realizes.