WO2021098121A1 - 共享编码器生成方法、装置及电子设备 - Google Patents

共享编码器生成方法、装置及电子设备 Download PDF

Info

Publication number
WO2021098121A1
WO2021098121A1 PCT/CN2020/083551 CN2020083551W WO2021098121A1 WO 2021098121 A1 WO2021098121 A1 WO 2021098121A1 CN 2020083551 W CN2020083551 W CN 2020083551W WO 2021098121 A1 WO2021098121 A1 WO 2021098121A1
Authority
WO
WIPO (PCT)
Prior art keywords
shared encoder
target shared
child node
parameter set
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2020/083551
Other languages
English (en)
French (fr)
Inventor
董大祥
张文慧
吴志华
于佃海
马艳军
王海峰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to JP2021516453A priority Critical patent/JP7159460B2/ja
Priority to EP20864267.8A priority patent/EP3855368B1/en
Priority to KR1020207035376A priority patent/KR102532368B1/ko
Priority to US17/209,576 priority patent/US12229667B2/en
Publication of WO2021098121A1 publication Critical patent/WO2021098121A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G06F18/2148—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the process organisation or structure, e.g. boosting cascade
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
    • G06F18/24133—Distances to prototypes
    • G06F18/24137—Distances to cluster centroïds
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5072—Grid computing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G06N3/0455—Auto-encoder networks; Encoder-decoder networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/098—Distributed learning, e.g. federated learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/02—Knowledge representation; Symbolic representation
    • G06N5/022—Knowledge engineering; Knowledge acquisition
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/54—Interprogram communication
    • G06F9/546—Message passing systems or structures, e.g. queues

Definitions

  • This application relates to the field of computer technology, in particular to the field of big data technology, and proposes a method, device and electronic device for generating a shared encoder.
  • Pre-training models have been proven effective in computer vision, natural language processing and other fields. It has become a popular practice to perform pre-training under big data and then fine-tune model parameters according to specific tasks.
  • the shared encoder generation method, device, and electronic equipment proposed in the present application are used to solve the problem that in related technologies, the acquisition of multi-domain training corpus is difficult and costly, which leads to the difficulty of implementing the pre-training model.
  • the method for generating a shared encoder proposed in an embodiment of the present application includes: the master node sends a shared encoder training instruction to each child node, and the training instruction includes the target shared encoder type to be trained, so that each child node is based on The type of target shared encoder obtains training samples; after obtaining the confirmation message returned by each child node, the initial parameter set of the target shared encoder to be trained is sent to each child node, so that each child node uses its own training sample pair The initial parameter set of the target shared encoder is trained; the update parameter set of the target shared encoder returned by each child node is obtained; the update parameter set of the target shared encoder returned by each child node is determined according to the first preset rule The target shares the target parameter set corresponding to the encoder.
  • the method for generating a shared encoder proposed in another embodiment of the present application includes: a second child node obtains a shared encoder training instruction sent by a master node, and the training instruction includes a target shared encoder type to be trained; determining whether to include and After the training samples matching the target shared encoder type, return a confirmation message to the master node; obtain the initial parameter set of the target shared encoder sent by the master node; use training that matches the target shared encoder type Sample, perform model training, determine the update parameter set of the target shared encoder; return the update parameter set of the target shared encoder to the master node.
  • the shared encoder generating device proposed in another embodiment of the present application includes: a first sending module, used for the master node to send a shared encoder training instruction to each child node, and the training instruction includes the target shared encoder to be trained Type, so that each child node obtains training samples according to the type of the target shared encoder; the second sending module is used to send the target shared encoder to be trained to each child node after obtaining the confirmation message returned by each child node An initial parameter set, so that each child node uses its own training sample to train the initial parameter set of the target shared encoder; the first acquisition module is used to acquire the updated parameter set of the target shared encoder returned by each child node; The first determining module is configured to determine the target parameter set corresponding to the target shared encoder according to the first preset rule and the updated parameter set of the target shared encoder returned by each child node.
  • the first acquisition module is used for the second child node to acquire the shared encoder training instruction sent by the master node, and the training instruction includes the target shared encoder to be trained Type;
  • the first return module is used to determine that it contains training samples matching the type of the target shared encoder, and then return a confirmation message to the master node;
  • the second acquisition module is used to acquire the target shared encoder sent by the master node The initial parameter set of the encoder;
  • the first determination module is used to use training samples matching the type of the target shared encoder to perform model training to determine the update parameter set of the target shared encoder;
  • the second return module uses Yu returns the updated parameter set of the target shared encoder to the master node.
  • An electronic device provided by another embodiment of the present application includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores the memory that can be executed by the at least one processor Instructions, the instructions are executed by the at least one processor, so that the at least one processor can execute the shared encoder generation method as described above.
  • the non-transitory computer-readable storage medium storing computer instructions proposed by another embodiment of the present application is characterized in that the computer instructions are used to make the computer execute the shared encoder generation method as described above.
  • the master node organizes each sub-node to use private training samples to train the target shared encoder, so as to determine the target according to the training result of each sub-node on the target shared encoder
  • the target parameter set of the encoder is shared, thereby not only realizing the sharing of multi-domain training corpus, reducing the difficulty and cost of obtaining multi-domain training corpus, but also improving the performance of the shared encoder.
  • the master node sends a shared encoder training instruction to each child node, so that each child node obtains training samples according to the target shared encoder type included in the training instruction, and after obtaining the confirmation message returned by each child node , Send the initial parameter set of the target shared encoder to be trained to each child node, so that each child node uses its own training sample to train the initial parameter set of the target shared encoder, and then obtains the target shared code returned by each child node
  • the target parameter set corresponding to the target shared encoder is determined, so it overcomes the difficulty of obtaining training corpus in multiple domains.
  • the high cost and the difficulty of implementing the pre-training model realizes the sharing of training corpus in multiple fields, reduces the difficulty and cost of obtaining training corpus in multiple fields, and improves the performance of the shared encoder.
  • FIG. 1 is a schematic flowchart of a method for generating a shared encoder according to an embodiment of the application
  • FIG. 2 is a schematic flowchart of another method for generating a shared encoder according to an embodiment of the application
  • FIG. 3 is a schematic flowchart of yet another method for generating a shared encoder according to an embodiment of the application
  • FIG. 4 is a schematic flowchart of another method for generating a shared encoder according to an embodiment of the application
  • FIG. 5 is a schematic flowchart of another method for generating a shared encoder according to an embodiment of the application
  • FIG. 6 is a schematic structural diagram of a shared encoder generating device provided by an embodiment of the application.
  • FIG. 7 is a schematic structural diagram of yet another shared encoder generating apparatus provided by an embodiment of the application.
  • FIG. 8 is a schematic structural diagram of an electronic device provided by an embodiment of the application.
  • the embodiment of the application proposes a method for generating a shared encoder in response to the difficulty and high cost of acquiring training corpus from multiple fields in related technologies, which leads to the difficulty of implementing a pre-training model.
  • the following takes the master node side and the child node side as examples to describe in detail the shared encoder generation method provided in the embodiment of the present application.
  • FIG. 1 is a schematic flowchart of a method for generating a shared encoder according to an embodiment of the application, and the method is applied to a master node.
  • the method for generating a shared encoder includes the following steps:
  • Step 101 The master node sends a shared encoder training instruction to each child node.
  • the training instruction includes the target shared encoder type to be trained, so that each child node obtains training samples according to the target shared encoder type.
  • pre-training models have been proven effective in computer vision, natural language processing and other fields. It has become a popular practice to perform pre-training under big data and then fine-tune model parameters according to specific tasks. However, since the pre-training model requires a large amount of pre-training corpus, collecting unsupervised corpus from various fields is a difficult problem. In addition, different companies and organizations have a large number of unsupervised corpus related to their own field business, but may not be open, which leads to sharing pre-training corpus and training to obtain a more powerful pre-training model is more difficult.
  • the shared encoder generation method of the embodiment of the application uses the distributed training technology in deep learning to enable multiple sub-nodes (multi-party organizations) to share the information in the training corpus to obtain the shared encoder, and to ensure the training corpus of the sub-nodes Security and privacy.
  • the master node refers to the node used to trigger the training process of the shared encoder.
  • the master node may be a server of the federated learning system.
  • a child node refers to a node that uses its own training samples to participate in the training of the shared encoder.
  • it can be a company that participates in the training of a shared encoder, and a server used by the organization to train the model.
  • the method for generating a shared encoder in the embodiment of the present application can be applied to any scenario where joint training is performed, which is not limited in the embodiment of the present application.
  • the following takes the application scenario of the shared encoder generation method of the embodiment of the present application as a federated learning system as an example for detailed description.
  • the master node may send a shared encoder training instruction to each child node in the federated learning system when there is a need to train the shared encoder, so that each child node may obtain the shared encoder training instruction when the shared encoder training instruction is obtained.
  • the type of target shared encoder to be trained included in the training instruction determine whether there is a training sample corresponding to the target shared encoder type locally, and if so, obtain the training sample corresponding to the target shared encoder type .
  • the child nodes may include multiple types of training samples for training different types of shared encoders, so that when the master node sends a shared encoder training instruction to each child node, it may include the target sharing to be trained. Encoder type, so that the child node obtains training samples corresponding to the target shared encoder type from multiple types of training samples.
  • the master node may trigger the training of the target shared encoder according to the acquisition request or update request of the target shared encoder from the child node. That is, in a possible implementation form of the embodiment of the present application, before the foregoing step 101, it may further include:
  • the master node obtains the request for obtaining the target shared encoder sent by at least one child node;
  • the master node obtains a shared encoder update request sent by at least one child node, and the update request includes an identifier of the target shared encoder.
  • the master node may trigger the training process for the target shared encoder according to each child node's request for obtaining the target shared encoder before training the target shared encoder. That is, when the master node obtains the request for obtaining the target shared encoder sent by at least one child node, and the obtaining request includes the initial parameter set corresponding to the target shared encoder, it can determine to obtain the request of at least one child node to the target shared encoder. Obtain the request, so that the shared encoder training instruction can be sent to each child node to trigger the training process of the target shared encoder.
  • the master node can also trigger the training process of the target shared encoder according to the update requirements of each child node for the target shared encoder after training the target shared encoder one or more times. That is, when the master node obtains the shared encoder update request sent by at least one child node, it can send a shared encoder training instruction to each child node to trigger the training process of the target shared encoder, so as to achieve the target shared encoder update request.
  • the shared encoder update request sent by the child node may include the identification of the target shared encoder, so that the master node can determine the type of the target shared encoder according to the identification of the target shared encoder, and then report the target shared encoder according to the type of the target shared encoder.
  • Each child node sends a shared encoder training instruction.
  • the master node can also actively trigger the training process of the target shared encoder according to the update rule.
  • the update rule is to update the target shared encoder at a preset frequency
  • the master node can send a shared encoder training instruction to each child node at the preset frequency to trigger the training process of the target shared encoder.
  • the method of triggering the training process of the target shared encoder may include but is not limited to the situations listed above.
  • a suitable method for triggering the training process of the target shared encoder can be determined according to actual needs or specific application scenarios, which is not limited in the embodiment of the present application.
  • Step 102 After obtaining the confirmation message returned by each child node, send the initial parameter set of the target shared encoder to be trained to each child node, so that each child node uses its own training sample to share the initial parameter of the target encoder. Set for training.
  • each sub-node can judge according to its own requirements for the target shared encoder, or whether there are training samples that match the target shared encoder type Whether to join the training of the target shared encoder. If you can join the training of the target shared encoder, you can return a confirmation message to the master node; if you can't join the training of the target shared encoder, you don't return a confirmation message to the master node.
  • the master node when the master node obtains the confirmation message returned by the child node, it can determine the child nodes participating in the training of the target shared encoder (that is, the child nodes that return the confirmation message), and pull the initial target shared encoder to be trained. Parameter set, and send the initial parameter set of the target shared encoder to each child node participating in the training of the target shared encoder, so that each child node can use their respective training samples to train the initial parameter set of the target shared encoder.
  • the master node after the master node obtains the confirmation message returned by the child node, it can also send the preset executable program and the initial parameter set of the target shared encoder to be trained to the participating target shared code Each child node of the target shared encoder training, so that each child node participating in the target shared encoder training can train the target shared encoder according to the executable program.
  • the executable program sent by the master node to each child node may include the rules for the child nodes to train the target shared encoder, the rules for returning the updated parameter set of the target shared encoder, and so on.
  • the executable program may include the number of times the target shared encoder is trained each time the child node obtains the training instruction sent by the master node; or, it may also include each time the child node obtains the training instruction sent by the master node. , The time period for returning the update parameter set of the target shared encoder to the master node, etc., which is not limited in the embodiment of the present application.
  • the master node needs to determine the initial parameter set corresponding to the target shared encoder. That is, in a possible implementation form of the embodiment of the present application, before the foregoing step 102, it may further include:
  • the target shared encoder to be trained is generated.
  • the second preset rule refers to a preset rule for obtaining the initial parameter set of the target shared encoder to be trained, and the second preset rule can be modified according to actual needs.
  • the second preset rule may be "predefined by each child node", the master node may obtain the initial parameter set predefined by each child node for the target shared encoder from each child node in advance, and The initial parameter set of the target shared encoder is defined by each child node through negotiation, so the initial parameter set predefined by each child node is the same.
  • the second preset rule may also be “determined according to the parameter set of the same type of encoder that exists in the public model”, and the master node can obtain the shared encoder with the target from the public model. For encoders of the same type, the acquired parameter set of the encoder is then determined as the initial parameter set of the target shared encoder.
  • Step 103 Obtain the updated parameter set of the target shared encoder returned by each child node.
  • the master node after the master node sends the initial parameter set of the target shared encoder to be trained to each child node, it can obtain the updated parameter set of the target shared encoder returned by each child node in real time.
  • the updated parameter set of the target shared encoder returned by each child node is obtained by each child node using its own training samples and trained according to the initial parameter set of the target shared encoder.
  • Step 104 Determine the target parameter set corresponding to the target shared encoder according to the first preset rule and the updated parameter set of the target shared encoder returned by each child node.
  • the first preset rule refers to a preset rule for fusing the update parameters of the target shared encoder returned by each child node to obtain a target parameter set corresponding to the target shared encoder.
  • the first preset rule may be to determine the mean value of each parameter in the update parameter set returned by each child node as the value of the corresponding parameter in the target parameter set; or it may also be to set each parameter in the update parameter set returned by each child node The variance, standard deviation, median, etc. of is determined as the value of the corresponding parameter in the target parameter set.
  • the first preset rule may include but is not limited to the situations listed above. In actual use, it can be preset and modified in real time according to actual needs to adapt to current use requirements and application scenarios, which is not limited in the embodiment of the present application.
  • the master node after the master node obtains the update parameter set of the target shared encoder returned by each child node, it can merge the obtained update parameter sets according to the first preset rule to determine the target shared code The target parameter set corresponding to the device.
  • the master node sends a shared encoder training instruction to each child node, so that each child node obtains training samples according to the target shared encoder type included in the training instruction, and when each child node is obtained
  • the initial parameter set of the target shared encoder to be trained is sent to each child node, so that each child node uses its own training sample to train the initial parameter set of the target shared encoder, and then obtains each child node.
  • the update parameter set of the target shared encoder returned by the node, and then the target parameter set corresponding to the target shared encoder is determined according to the first preset rule and the update parameter set of the target shared encoder returned by each child node.
  • the master node organizes each child node to use private training samples to train the target shared encoder to determine the target parameter set of the target shared encoder according to the training results of each child node on the target shared encoder.
  • the sharing of multi-domain training corpus reduces the difficulty and cost of obtaining multi-domain training corpus, and improves the performance of shared encoders.
  • the number of training samples of the sub-nodes can also be used. , Determine to give different weights to each child node.
  • Fig. 2 is a schematic flowchart of another method for generating a shared encoder provided by an embodiment of the application, and the method is applied to a master node.
  • the method for generating a shared encoder includes the following steps:
  • Step 201 The master node sends a shared encoder training instruction to each child node.
  • the training instruction includes the target shared encoder type to be trained, so that each child node obtains training samples according to the target shared encoder type.
  • Step 202 After obtaining the confirmation message returned by each child node, send the initial parameter set of the target shared encoder to be trained to each child node, so that each child node uses its own training sample to share the initial parameter of the target encoder. Set for training.
  • Step 203 Obtain the number of training samples contained in each child node.
  • the number of training samples contained in the child node may refer to the number of training samples contained in the child node that can be used to train the target shared encoder.
  • the master node can determine the weight of each sub-node when the training target shares the encoder according to the number of training samples contained in each sub-node that can be used to train the target shared encoder. Therefore, after the master node sends the initial parameter set of the target shared encoder to be trained to each child node, it can obtain the number of training samples contained in each child node that can be used to train the target shared encoder in real time.
  • Step 204 Determine the weight of each child node according to the number of training samples contained in each child node.
  • the master node since the number of training samples contained in a child node is larger, the performance of the determined target shared encoder according to the update parameter set returned by the child node is better, so that the master node can include The number of training samples determines the weight of each child node.
  • the weight of a child node can be directly proportional to the number of training samples contained in the child node, that is, the more training samples the child node contains, the greater the weight of the child node, so that the training The greater the number of samples, the greater the role of the child node in training the target shared encoder to improve the performance of the target shared encoder.
  • the ratio of the number of training samples included in each child node to the number of training samples included in all child nodes may be determined as the weight of each child node.
  • the current federated learning system has 3 child nodes X, Y, Z, the number of training samples contained in child node X is m, the number of training samples contained in child node Y is n, and the number of training samples contained in child node Z Is k, the weight of child node X is m/(m+n+k), the weight of child node Y is n/(m+n+k), and the weight of child node Z is k/(m+n+k) ).
  • Step 205 Obtain the updated parameter set of the target shared encoder returned by each child node.
  • Step 206 Determine the target parameter set corresponding to the target shared encoder according to the first preset rule, the weight of each child node, and the updated parameter set of the target shared encoder returned by each child node.
  • the weight of each child node can be adjusted according to the first preset rule and the weight of each child node.
  • the updated parameter set of the target shared encoder returned by the child node is merged to determine the target parameter set corresponding to the target shared encoder.
  • the first preset rule is "determine the mean value of each parameter in the update parameter set returned by each child node as the value of the corresponding parameter in the target parameter set".
  • the current federated learning system has three child nodes X, Y , Z, the weight of the child node X is w 1 , the updated parameter set of the returned target shared encoder is ⁇ a 1 ,b 1 ,c 1 ⁇ , the weight of the child node Y is w 2 , the returned target shared encoder’s
  • the update parameter set is ⁇ a 2 ,b 2 ,c 2 ⁇ , the weight of the child node Z is w 3 , and the update parameter set of the returned target shared encoder is ⁇ a 3 ,b 3 ,c 3 ⁇ , then the child node X
  • the revised update parameter set is ⁇ w 1 ⁇ a 1 ,w 1 ⁇ b 1 ,w 1 ⁇ c 1 ⁇ , and the revised update parameter set of the child node Y is ⁇ w 2 ⁇
  • the master node sends a shared encoder training instruction to each child node, so that each child node obtains training samples according to the target shared encoder type included in the training instruction, and when each child node is obtained
  • the initial parameter set of the target shared encoder to be trained is sent to each child node, so that each child node uses its own training sample to train the initial parameter set of the target shared encoder, and according to the obtained
  • the number of training samples contained in each child node determines the weight of each child node, and then obtains the updated parameter set of the target shared encoder returned by each child node, and then according to the first preset rule, the weight of each child node and each child node
  • the returned update parameter set of the target shared encoder determines the target parameter set corresponding to the target shared encoder.
  • the master node organizes each sub-node to use private training samples to train the target shared encoder, and adjusts the weight of each sub-node according to the number of training samples contained in each sub-node, thus not only realizing the sharing of training corpus in multiple fields , Which reduces the difficulty and cost of acquiring training corpus in multiple fields, and further improves the performance of the shared encoder.
  • the master node in order to further improve the reliability and robustness of the shared encoder, can judge whether the shared encoder meets the performance requirements based on the target parameter set obtained in the previous training process. If satisfied, the shared encoder can be retrained on the basis of the previous training.
  • FIG. 3 is a schematic flowchart of yet another method for generating a shared encoder according to an embodiment of the application, and the method is applied to a master node.
  • the shared encoder generation method includes the following steps:
  • the master node sends a shared encoder training instruction to each child node.
  • the training instruction includes the target shared encoder type to be trained, so that each child node obtains training samples according to the target shared encoder type.
  • Step 302 After obtaining the confirmation message returned by each child node, send the initial parameter set of the target shared encoder to be trained to each child node, so that each child node uses its own training sample to share the initial parameter of the target encoder. Set for training.
  • Step 303 Obtain the updated parameter set of the target shared encoder returned by each child node.
  • Step 304 Determine the target parameter set corresponding to the target shared encoder according to the first preset rule and the updated parameter set of the target shared encoder returned by each child node.
  • Step 305 Send the target parameter set corresponding to the target shared encoder to each child node, so that each child node continues to train the target shared encoder based on the target parameter set corresponding to the target shared encoder.
  • the master node can determine whether the target shared encoder meets the performance requirements each time after determining the target parameter set corresponding to the target shared encoder, and if it meets the performance requirements, it can determine to complete the target sharing.
  • the training process of the encoder if the performance requirements are not met, it can be determined that the target shared encoder needs to be retrained to improve the performance of the target shared encoder.
  • the master node can send the target parameter set corresponding to the target shared encoder to each child node, so that each child node can use its own training Samples, and based on the target parameter set of the target shared encoder, continue to train the target shared encoder.
  • the master node determines the target parameter set corresponding to the target shared encoder meets the performance requirements according to the updated parameter set returned by each child node again, it will no longer send the target parameter set corresponding to the target shared encoder to each child node, that is, the completion of this The training process of the encoder is shared with the target.
  • the weight of each child node can be determined according to the number of times each child node trains the target shared encoder, so as to balance each child node in the target shared encoder training process The role played. That is, in a possible implementation form of the embodiment of the present application, after the foregoing step 305, it may further include:
  • the target parameter set corresponding to the target shared encoder is updated according to the current weight value of the first child node and the target shared encoder parameter set that is currently returned again.
  • the first child node refers to a child node among multiple child nodes that participates in the training of the target shared encoder multiple times.
  • the master node when the master node obtains the shared encoder parameter set returned by the first child node again, it can determine the number of times the acquired first child node returns to the target shared encoder parameter set (that is, the first child node The number of times the node participates in the training of the target shared encoder), and the current weight value of the first child node is determined according to the number of times the first child node returns to the target shared encoder parameter set.
  • the parameter set of the target shared encoder returned by the child node has a greater impact on the target parameter set corresponding to the final target shared encoder.
  • the target shared encoder finally obtained fits the characteristics of the training samples of the child node too much, and the versatility of the target shared encoder becomes worse. Therefore, in a possible implementation form of the embodiment of the present application, the current weight value of the first child node may have a negative correlation with the number of times the first child node returns to the target shared encoder parameter set.
  • each child node since each child node returns to the target shared encoder parameter set at a different time, each time a target shared encoder parameter set returned by the first child node is obtained, that is, Determine the current weight value of the first child node. Furthermore, the master node modifies the target shared coding parameter set returned by the first child node according to the current weight value of the first child node, and then shares the target shared encoding parameter set returned by the first child node after the correction according to the first preset rule. The encoder parameter set and the target parameter set corresponding to the previously determined target shared encoder are merged to update the target parameter set corresponding to the target shared encoder.
  • the first preset rule is "determine the sum of the parameters in the update parameter set returned by each child node as the value of the corresponding parameter in the target parameter set.”
  • the target parameter set is ⁇ a,b,c ⁇
  • the target shared encoder parameter set returned by the first child node X is currently ⁇ a x ,b x ,c x ⁇
  • the target shared encoder is returned according to the first child node X
  • the number of encoder parameter sets to determine the weight value of the first child node X is w x , then it can be determined that the target shared encoder parameter set returned by the first child node X after the correction is ⁇ w x ⁇ a x ,w x ⁇ B x ,w x ⁇ c x ⁇ , the target parameter set corresponding to the updated target shared encoder is ⁇ a+w x ⁇ a x ,b+w x ⁇ b x ,c+w x ⁇ c x ⁇ .
  • the master node sends a shared encoder training instruction to each child node, so that each child node obtains training samples according to the target shared encoder type included in the training instruction, and when each child node is obtained After the confirmation message returned by the node, the initial parameter set of the target shared encoder to be trained is sent to each child node, so that each child node uses its own training sample to train the initial parameter set of the target shared encoder, and then obtains each child node.
  • the update parameter set of the target shared encoder returned by the node, and the target parameter set corresponding to the target shared encoder is determined according to the first preset rule and the update parameter set of the target shared encoder returned by each child node, and then the target shared encoding
  • the target parameter set corresponding to the target shared encoder is sent to each child node respectively, so that each child node continues to train the target shared encoder based on the target parameter set corresponding to the target shared encoder. Therefore, the master node organizes each child node to use private training samples to train the target shared encoder multiple times to determine the target parameter set of the target shared encoder according to the multiple training results of each child node on the target shared encoder. This not only realizes the sharing of multi-domain training corpus, reduces the difficulty and cost of obtaining multi-domain training corpus, but also further improves the accuracy and versatility of the shared encoder.
  • FIG. 4 is a schematic flowchart of another method for generating a shared encoder according to an embodiment of the application, and the method is applied to a child node.
  • the shared encoder generation method includes the following steps:
  • Step 401 The second child node obtains a shared encoder training instruction sent by the master node, and the training instruction includes the target shared encoder type to be trained.
  • the master node refers to the node used to trigger the training process of the shared encoder.
  • the master node may be a server of the federated learning system.
  • a child node refers to a node that uses its own training samples to participate in the training of the shared encoder. For example, it can be a company that participates in the training of a shared encoder, and a server used by the organization to train the model.
  • the second child node refers to one of the multiple child nodes.
  • the method for generating a shared encoder in this embodiment of the application can be applied to any scenario where joint training is performed, which is not limited in the embodiment of the application.
  • the following takes the application scenario of the shared encoder generation method of the embodiment of the present application as a federated learning system as an example for detailed description.
  • the second child node can obtain in real time that the master node sends a shared encoder training instruction to each child node when there is a need to train the shared encoder.
  • the second child node obtains the shared encoder training instruction .
  • the type of the target shared encoder to be trained included in the training instruction it is judged whether to join the training of the target shared encoder.
  • Step 402 After determining that it contains training samples that match the target shared encoder type, return a confirmation message to the master node.
  • the second child node determines that it contains training samples that match the target shared encoder type, it is determined that the training of the target shared encoder can be added, so that a confirmation message can be returned to the master node so that the master node According to the obtained confirmation message, it is determined to join the child node for training the target shared encoder, and the initial parameter set of the target shared encoder to be trained is pulled and sent to the second child node.
  • Step 403 Obtain the initial parameter set of the target shared encoder sent by the master node.
  • the initial parameter set of the target shared encoder returned by the main node can be obtained in real time, so as to share the target with the target shared encoder according to the initial parameter set of the target shared encoder.
  • the encoder is trained.
  • Step 404 Perform model training using training samples that match the type of the target shared encoder, and determine an update parameter set of the target shared encoder.
  • the second child node after the second child node obtains the initial parameter set of the target shared encoder, it can use the training samples that match the type of the target shared encoder, and the acquired initial parameter set of the target shared encoder , Perform model training to determine the updated parameter set of the target shared encoder.
  • Step 405 Return the updated parameter set of the target shared encoder to the master node.
  • the second child node after the second child node completes the training of the target shared encoder, it can return the determined update parameter set of the target shared encoder to the master node, so that the master node can follow each child node
  • the returned update parameter set determines the target parameter set corresponding to the target shared encoder.
  • the master node can also determine each child node according to the number of training samples of the child node Give different weights. That is, in a possible implementation form of the embodiment of the present application, the foregoing method may further include:
  • the number of training samples matching the target shared encoder type is sent to the master node.
  • the master node can determine the second sub-node based on the number of training samples contained in the second sub-node that can be used to train the target shared encoder, that is, the number of training samples that match the target shared encoder type.
  • the child nodes can also design private encoders and private decoders based on private training samples, and then determine the shared encoder training tasks that can be added according to the designed private encoders and private decoders. That is, in a possible implementation form of the embodiment of the present application, after the foregoing step 405, it may further include:
  • the second sub-node can design a private encoder and a private decoder based on the included training samples, and can use training samples that match the target shared encoder type to share the target.
  • the encoder After the encoder is trained, obtain the private encoder and private decoder corresponding to the target shared encoder, and then determine jointly based on the training result of the target shared encoder and the private encoder and private decoder corresponding to the target shared encoder The target shares the updated parameter set of the encoder.
  • the shared encoder training instruction sent by the main node is obtained through the second child node, and after determining that it contains the training sample matching the target shared encoder type, the confirmation message is returned to the main node, and the obtaining The initial parameter set of the target shared encoder sent by the master node, and then use the training samples that match the target shared encoder type to perform model training, determine the updated parameter set of the target shared encoder, and then return the target shared encoder's parameter set to the master node Update the parameter set. Therefore, the master node organizes each child node to use private training samples to train the target shared encoder to determine the target parameter set of the target shared encoder according to the training results of each child node on the target shared encoder.
  • the sharing of multi-domain training corpus reduces the difficulty and cost of obtaining multi-domain training corpus, and improves the performance of shared encoders.
  • the child node may return the update parameter set of the target shared encoder to the master node according to the update rule preset by the master node.
  • FIG. 5 is a schematic flowchart of another method for generating a shared encoder according to an embodiment of the application, and the method is applied to a child node.
  • the shared encoder generation method includes the following steps:
  • Step 501 The second child node obtains the shared encoder training instruction sent by the master node, and the training instruction includes the target shared encoder type to be trained.
  • Step 502 After determining that it contains training samples that match the target shared encoder type, return a confirmation message to the master node.
  • Step 503 Obtain the target shared encoder parameter update rule returned by the master node.
  • the target shared encoder parameter update rule can be included in the executable program returned by the master node.
  • the master node after the master node obtains the confirmation message returned by the child node, it can also send a preset executable program to each child node participating in the training of the target shared encoder, so that the second child node can follow
  • the executable program trains the target shared encoder, and determines whether the determined update parameter set can be returned to the main node according to the target shared encoder parameter update rule included in the executable program.
  • the target shared encoder parameter update rule may be the number of times the target shared encoder is trained each time the second child node obtains the training instruction sent by the master node; or, it may also be the second child node Each time the training instruction sent by the master node is acquired, the time period for updating the parameter set of the target shared encoder is returned to the master node, etc., which is not limited in the embodiment of the present application.
  • the target shared encoder parameter update rule can be "return to the master node to obtain the updated parameter set of the target shared code after 5 rounds of training on the target shared encoder", or it can also be “to the master node every month Return the updated parameter set of the target shared code once”.
  • Step 504 Obtain the initial parameter set of the target shared encoder sent by the master node.
  • Step 505 Use training samples matching the type of the target shared encoder to perform model training to determine an update parameter set of the target shared encoder.
  • Step 506 When it is determined that the update rule is currently satisfied, the update parameter set of the target shared encoder is returned to the master node.
  • each time the second child node completes a round of training on the target shared encoder it can determine whether the target shared encoder parameter update rule is currently met, and if it is satisfied, it can return the currently obtained target to the master node The update parameter set of the shared encoder; if it is not satisfied, the next round of training is continued on the target shared encoder until the current target shared encoder parameter update rule is met.
  • the target shared encoder parameter update rule is "the child node completes 5 rounds of training for each pair of target shared encoders and returns to the master node the updated parameter set of the target shared encoder", if the second child node currently completes the target shared encoding In the fourth round of training of the target shared encoder, it can be determined that the current target shared encoder parameter update rule is not satisfied, that is, the next round of training of the target shared encoder can be continued; if the second child node currently completes the fifth round of the target shared encoder Round training, it can be determined that the target shared encoder parameter update rule is currently met, and the updated parameter set of the target shared encoder obtained after the fifth round of training can be returned to the master node.
  • the shared encoder training instruction sent by the main node is obtained through the second child node, and after determining that it contains the training sample matching the target shared encoder type, the confirmation message is returned to the main node, and the obtaining The target shared encoder parameter update rules returned by the master node and the initial parameter set of the target shared encoder. Then, the training samples matching the target shared encoder type are used for model training to determine the update parameter set of the target shared encoder. When it is determined that the update rule is currently satisfied, the update parameter set of the target shared encoder is returned to the master node.
  • the master node organizes each child node to use private training samples to train the target shared encoder, and according to the parameter update rule returned by the master node, determines the time when the child node returns the target shared encoder update parameter set to the master node. This not only realizes the sharing of multi-domain training corpus, reduces the difficulty and cost of obtaining multi-domain training corpus, but also further improves the performance of the shared encoder.
  • this application also proposes a shared encoder generating device.
  • FIG. 6 is a schematic structural diagram of a device for generating a shared encoder according to an embodiment of the application, and the device is applied to a master node.
  • the shared encoder generating device 60 includes:
  • the first sending module 61 is used for the master node to send a shared encoder training instruction to each child node, the training instruction includes the target shared encoder type to be trained, so that each child node obtains training according to the target shared encoder type sample;
  • the second sending module 62 is configured to send the initial parameter set of the target shared encoder to be trained to each child node after obtaining the confirmation message returned by each child node, so that each child node uses its own training sample to compare the The target shares the initial parameter set of the encoder for training;
  • the first obtaining module 63 is configured to obtain the updated parameter set of the target shared encoder returned by each child node;
  • the first determining module 64 is configured to determine the target parameter set corresponding to the target shared encoder according to the first preset rule and the updated parameter set of the target shared encoder returned by each child node.
  • the shared encoder generating apparatus provided in the embodiment of the present application can be configured in any electronic device to execute the foregoing shared encoder generating method.
  • the master node sends a shared encoder training instruction to each child node, so that each child node obtains training samples according to the target shared encoder type included in the training instruction, and when each child node is obtained
  • the initial parameter set of the target shared encoder to be trained is sent to each child node, so that each child node uses its own training sample to train the initial parameter set of the target shared encoder, and then obtains each child node.
  • the update parameter set of the target shared encoder returned by the node, and then the target parameter set corresponding to the target shared encoder is determined according to the first preset rule and the update parameter set of the target shared encoder returned by each child node.
  • the master node organizes each child node to use private training samples to train the target shared encoder to determine the target parameter set of the target shared encoder according to the training results of each child node on the target shared encoder.
  • the sharing of multi-domain training corpus reduces the difficulty and cost of obtaining multi-domain training corpus, and improves the performance of shared encoders.
  • the foregoing shared encoder generating device 60 further includes:
  • the second obtaining module is used for the master node to obtain the request for obtaining the target shared encoder sent by at least one child node;
  • the third obtaining module is configured to obtain the target shared encoder update request sent by at least one child node by the master node, where the update request includes the target shared encoder identification.
  • the above-mentioned shared encoder generating device 60 further includes:
  • the fourth acquisition module is used to acquire the number of training samples contained in each sub-node
  • the second determining module is used to determine the weight of each child node according to the number of training samples contained in each child node;
  • the above-mentioned first determining module 64 is specifically used for:
  • the aforementioned shared encoder generating device 60 further includes:
  • the third sending module is configured to send the target parameter set corresponding to the target shared encoder to each child node, so that each child node encodes the target shared encoder based on the target parameter set corresponding to the target shared encoder The device continues to train.
  • the above-mentioned shared encoder generating device 60 further includes:
  • the fifth acquiring module is used to acquire the target shared encoder parameter set returned by the first child node again;
  • the third determining module is configured to determine the current weight value of the first child node according to the acquired number of times the first child node returns to the target shared encoder parameter set;
  • the update module is configured to perform, according to the first preset rule, the target parameter set corresponding to the target shared encoder according to the current weight value of the first child node and the target shared encoder parameter set that is currently returned again Update.
  • the above-mentioned shared encoder generating device 60 further includes:
  • the generating module is configured to generate the target shared encoder to be trained according to the second preset rule.
  • the master node sends a shared encoder training instruction to each child node, so that each child node obtains training samples according to the target shared encoder type included in the training instruction, and when each child node is obtained After the confirmation message returned by the node, the initial parameter set of the target shared encoder to be trained is sent to each child node, so that each child node uses its own training sample to train the initial parameter set of the target shared encoder, and then obtains each child node.
  • the update parameter set of the target shared encoder returned by the node, and the target parameter set corresponding to the target shared encoder is determined according to the first preset rule and the update parameter set of the target shared encoder returned by each child node, and then the target shared encoding
  • the target parameter set corresponding to the target shared encoder is sent to each child node respectively, so that each child node continues to train the target shared encoder based on the target parameter set corresponding to the target shared encoder. Therefore, the master node organizes each child node to use private training samples to train the target shared encoder multiple times to determine the target parameter set of the target shared encoder according to the multiple training results of each child node on the target shared encoder. This not only realizes the sharing of multi-domain training corpus, reduces the difficulty and cost of obtaining multi-domain training corpus, but also further improves the accuracy and versatility of the shared encoder.
  • this application also proposes another shared encoder generating device.
  • FIG. 7 is a schematic structural diagram of another device for generating a shared encoder according to an embodiment of the application, and the device is applied to a child node.
  • the shared encoder generating device 70 includes:
  • the first obtaining module 71 is configured to obtain the shared encoder training instruction sent by the master node by the second child node, where the training instruction includes the target shared encoder type to be trained;
  • the first return module 72 is configured to return a confirmation message to the master node after determining that it contains a training sample matching the target shared encoder type;
  • the second obtaining module 73 is configured to obtain the initial parameter set of the target shared encoder sent by the master node;
  • the first determining module 74 is configured to use training samples matching the type of the target shared encoder to perform model training to determine the update parameter set of the target shared encoder;
  • the second return module 75 is configured to return the updated parameter set of the target shared encoder to the master node.
  • the shared encoder generating apparatus provided in the embodiment of the present application can be configured in any electronic device to execute the foregoing shared encoder generating method.
  • the shared encoder training instruction sent by the main node is obtained through the second child node, and after determining that it contains the training sample matching the target shared encoder type, the confirmation message is returned to the main node, and the obtaining The initial parameter set of the target shared encoder sent by the master node, and then use the training samples that match the target shared encoder type to perform model training, determine the updated parameter set of the target shared encoder, and then return the target shared encoder's parameter set to the master node Update the parameter set. Therefore, the master node organizes each child node to use private training samples to train the target shared encoder to determine the target parameter set of the target shared encoder according to the training results of each child node on the target shared encoder.
  • the sharing of multi-domain training corpus reduces the difficulty and cost of obtaining multi-domain training corpus, and improves the performance of shared encoders.
  • the foregoing shared encoder generating device 70 further includes:
  • the third obtaining module is configured to obtain the target shared encoder parameter update rule returned by the master node
  • the above-mentioned second return module 75 is specifically used for:
  • the update parameter set of the target shared encoder is returned to the master node.
  • the aforementioned shared encoder generating device 70 further includes:
  • the sending module is configured to send the number of training samples matching the target shared encoder type to the master node.
  • the above-mentioned shared encoder generating device 70 further includes:
  • the second determining module is used to determine the private encoder and the private decoder corresponding to the target shared encoder.
  • the shared encoder training instruction sent by the main node is obtained through the second child node, and after determining that it contains the training sample matching the target shared encoder type, the confirmation message is returned to the main node, and the obtaining The target shared encoder parameter update rules returned by the master node and the initial parameter set of the target shared encoder. Then, the training samples matching the target shared encoder type are used for model training to determine the update parameter set of the target shared encoder. When it is determined that the update rule is currently satisfied, the update parameter set of the target shared encoder is returned to the master node.
  • the master node organizes each child node to use private training samples to train the target shared encoder, and according to the parameter update rule returned by the master node, determines the time when the child node returns the target shared encoder update parameter set to the master node. This not only realizes the sharing of multi-domain training corpus, reduces the difficulty and cost of obtaining multi-domain training corpus, but also further improves the performance of the shared encoder.
  • the present application also provides an electronic device and a readable storage medium.
  • FIG. 8 it is a block diagram of an electronic device of a method for generating a shared encoder according to an embodiment of the present application.
  • Electronic devices are intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers.
  • Electronic devices can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices.
  • the components shown herein, their connections and relationships, and their functions are merely examples, and are not intended to limit the implementation of the application described and/or required herein.
  • the electronic device includes: one or more processors 801, a memory 802, and interfaces for connecting various components, including a high-speed interface and a low-speed interface.
  • the various components are connected to each other using different buses, and can be installed on a common motherboard or installed in other ways as needed.
  • the processor may process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of the GUI on an external input/output device (such as a display device coupled to an interface).
  • an external input/output device such as a display device coupled to an interface.
  • multiple processors and/or multiple buses can be used with multiple memories and multiple memories.
  • multiple electronic devices can be connected, and each electronic device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system).
  • a processor 801 is taken as an example.
  • the memory 802 is a non-transitory computer-readable storage medium provided by this application.
  • the memory stores instructions executable by at least one processor, so that the at least one processor executes the shared encoder generation method provided by the present application.
  • the non-transitory computer-readable storage medium of the present application stores computer instructions, and the computer instructions are used to make a computer execute the shared encoder generation method provided by the present application.
  • the memory 802 can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as program instructions/modules corresponding to the shared encoder generation method in the embodiment of the present application (for example, , The first sending module 61, the second sending module 62, the first acquiring module 63, and the first determining module 64 shown in FIG. 6, and the first acquiring module 71, the first returning module 72, and the first acquiring module 71 shown in FIG. 7 The second acquisition module 73, the first determination module 74 and the second return module 75).
  • the processor 801 executes various functional applications and data processing of the server by running non-transitory software programs, instructions, and modules stored in the memory 802, that is, realizes the shared encoder generation method in the foregoing method embodiment.
  • the memory 802 may include a storage program area and a storage data area.
  • the storage program area can store an operating system and an application program required by at least one function; Data etc.
  • the memory 802 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices.
  • the memory 802 may optionally include a memory remotely provided with respect to the processor 801, and these remote memories may be connected to an electronic device sharing the encoder generation method through a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
  • the electronic device sharing the encoder generation method may further include: an input device 803 and an output device 804.
  • the processor 801, the memory 802, the input device 803, and the output device 804 may be connected by a bus or in other ways. In FIG. 8, the connection by a bus is taken as an example.
  • the input device 803 can receive input digital or character information, and generate key signal input related to the user settings and function control of the electronic device sharing the encoder generation method, such as touch screen, keypad, mouse, trackpad, touchpad, instructions Stick, one or more mouse buttons, trackball, joystick and other input devices.
  • the output device 804 may include a display device, an auxiliary lighting device (for example, LED), a tactile feedback device (for example, a vibration motor), and the like.
  • the display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display, and a plasma display. In some embodiments, the display device may be a touch screen.
  • Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, application specific ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: being implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, the programmable processor It can be a dedicated or general-purpose programmable processor that can receive data and instructions from the storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device. An output device.
  • machine-readable medium and “computer-readable medium” refer to any computer program product, device, and/or device used to provide machine instructions and/or data to a programmable processor ( For example, magnetic disks, optical disks, memory, programmable logic devices (PLD)), including machine-readable media that receive machine instructions as machine-readable signals.
  • machine-readable signal refers to any signal used to provide machine instructions and/or data to a programmable processor.
  • the systems and techniques described here can be implemented on a computer that has: a display device for displaying information to the user (for example, a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) ); and a keyboard and a pointing device (for example, a mouse or a trackball) through which the user can provide input to the computer.
  • a display device for displaying information to the user
  • LCD liquid crystal display
  • keyboard and a pointing device for example, a mouse or a trackball
  • Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (for example, visual feedback, auditory feedback, or tactile feedback); and can be in any form (including Acoustic input, voice input, or tactile input) to receive input from the user.
  • the systems and technologies described herein can be implemented in a computing system that includes back-end components (for example, as a data server), or a computing system that includes middleware components (for example, an application server), or a computing system that includes front-end components (for example, A user computer with a graphical user interface or a web browser, through which the user can interact with the implementation of the system and technology described herein), or includes such back-end components, middleware components, Or any combination of front-end components in a computing system.
  • the components of the system can be connected to each other through any form or medium of digital data communication (for example, a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.
  • the computer system can include clients and servers.
  • the client and server are generally far away from each other and usually interact through a communication network.
  • the relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other.
  • the master node sends a shared encoder training instruction to each child node, so that each child node obtains training samples according to the target shared encoder type included in the training instruction, and when each child node is obtained
  • the initial parameter set of the target shared encoder to be trained is sent to each child node, so that each child node uses its own training sample to train the initial parameter set of the target shared encoder, and then obtains each child node.
  • the update parameter set of the target shared encoder returned by the node, and then the target parameter set corresponding to the target shared encoder is determined according to the first preset rule and the update parameter set of the target shared encoder returned by each child node.
  • the master node organizes each child node to use private training samples to train the target shared encoder to determine the target parameter set of the target shared encoder according to the training results of each child node on the target shared encoder.
  • the sharing of multi-domain training corpus reduces the difficulty and cost of obtaining multi-domain training corpus, and improves the performance of shared encoders.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Mathematical Physics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Medical Informatics (AREA)
  • Devices For Executing Special Programs (AREA)
  • Mobile Radio Communication Systems (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Transmission And Conversion Of Sensor Element Output (AREA)
  • Feedback Control In General (AREA)

Abstract

一种共享编码器生成方法、装置及电子设备，属于计算机技术领域。其中，该方法包括：主节点向各子节点发送共享编码器训练指令，以使每个子节点根据训练指令中包括的目标共享编码器的类型获取训练样本（101）；在获取到各子节点返回的确认消息后，向各子节点发送待训练的目标共享编码器的初始参数集，以使各子节点利用各自的训练样本对所述目标共享编码器的初始参数集进行训练（102）；获取各子节点返回的目标共享编码器的更新参数集（103）；根据第一预设规则及各子节点返回的目标共享编码器的更新参数集，确定目标共享编码器对应的目标参数集（104）。由此，通过这种共享编码器生成方法，降低了多领域训练语料的获取难度和成本，提高了共享编码器的性能。

Description

共享编码器生成方法、装置及电子设备
相关申请的交叉引用
本申请要求北京百度网讯科技有限公司于2019年11月22日提交的、发明名称为“共享编码器生成方法、装置及电子设备”的、中国专利申请号“201911155509.2”的优先权。
技术领域
本申请涉及计算机技术领域，尤其涉及大数据技术领域，提出一种共享编码器生成方法、装置及电子设备。
背景技术
预训练模型已经在计算机视觉、自然语言处理等领域中的应用实践被证明有效，在大数据下进行一次预训练再根据具体任务进行模型参数微调，已经成为一种流行的做法。
但是，相关技术中，多领域训练语料的获取难度大、成本高，从而导致预训练模型的实现难度大。
发明内容
本申请提出的共享编码器生成方法、装置及电子设备，用于解决相关技术中，多领域训练语料的获取难度大、成本高，从而导致预训练模型的实现难度大的问题。
本申请一方面实施例提出的共享编码器生成方法，包括：主节点向各子节点发送共享编码器训练指令，所述训练指令中包括待训练的目标共享编码器类型，以使每个子节点根据目标共享编码器的类型获取训练样本；在获取到各子节点返回的确认消息后，向各子节点发送待训练的目标共享编码器的初始参数集，以使各子节点利用各自的训练样本对所述目标共享编码器的初始参数集进行训练；获取各子节点返回的目标共享编码器的更新参数集；根据第一预设规则及各子节点返回的目标共享编码器的更新参数集，确定所述目标共享编码器对应的目标参数集。
本申请另一方面实施例提出的共享编码器生成方法，包括：第二子节点获取主节点发送的共享编码器训练指令，所述训练指令中包括待训练的目标共享编码器类型；确定包含与所述目标共享编码器类型匹配的训练样本后，向所述主节点返回确认消息；获取所述主节点发送的目标共享编码器的初始参数集；利用与所述目标共享编码器类 型匹配的训练样本，进行模型训练，确定所述目标共享编码器的更新参数集；向所述主节点返回所述目标共享编码器的更新参数集。
本申请再一方面实施例提出的共享编码器生成装置，包括：第一发送模块，用于主节点向各子节点发送共享编码器训练指令，所述训练指令中包括待训练的目标共享编码器类型，以使每个子节点根据目标共享编码器的类型获取训练样本；第二发送模块，用于在获取到各子节点返回的确认消息后，向各子节点发送待训练的目标共享编码器的初始参数集，以使各子节点利用各自的训练样本对所述目标共享编码器的初始参数集进行训练；第一获取模块，用于获取各子节点返回的目标共享编码器的更新参数集；第一确定模块，用于根据第一预设规则及各子节点返回的目标共享编码器的更新参数集，确定所述目标共享编码器对应的目标参数集。
本申请又一方面实施例提出的共享编码器生成装置，第一获取模块，用于第二子节点获取主节点发送的共享编码器训练指令，所述训练指令中包括待训练的目标共享编码器类型；第一返回模块，用于确定包含与所述目标共享编码器类型匹配的训练样本后，向所述主节点返回确认消息；第二获取模块，用于获取所述主节点发送的目标共享编码器的初始参数集；第一确定模块，用于利用与所述目标共享编码器类型匹配的训练样本，进行模型训练，确定所述目标共享编码器的更新参数集；第二返回模块，用于向所述主节点返回所述目标共享编码器的更新参数集。
本申请又一方面实施例提出的电子设备，其包括：至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行如前所述的共享编码器生成方法。
本申请另一方面实施例提出的存储有计算机指令的非瞬时计算机可读存储介质，其特征在于，所述计算机指令用于使所述计算机执行如前所述的共享编码器生成方法。
上述申请中的任一个实施例具有如下优点或有益效果：通过主节点组织各子节点利用私有训练样本对目标共享编码器进行训练，以根据各子节点对目标共享编码器的训练结果，确定目标共享编码器的目标参数集，从而不仅实现了多领域训练语料的共享，降低了多领域训练语料的获取难度和成本，而且提高了共享编码器的性能。因为采用了通过主节点向各子节点发送共享编码器训练指令，以使每个子节点根据训练指令中包括的目标共享编码器的类型获取训练样本，并在获取到各子节点返回的确认消息后，向各子节点发送待训练的目标共享编码器的初始参数集，以使各子节点利用各自的训练样本对目标共享编码器的初始参数集进行训练，之后获取各子节点返回的目标共享编码器的更新参数集，进而根据第一预设规则及各子节点返回的目标共享编码 器的更新参数集，确定目标共享编码器对应的目标参数集，所以克服了多领域训练语料的获取难度大、成本高，预训练模型的实现难度大的问题，实现了多领域训练语料的共享，降低了多领域训练语料的获取难度和成本，而且提高了共享编码器的性能。
上述可选方式所具有的其他效果将在下文中结合具体实施例加以说明。
附图说明
附图用于更好地理解本方案，不构成对本申请的限定。其中：
图1为本申请实施例所提供的一种共享编码器生成方法的流程示意图；
图2为本申请实施例所提供的另一种共享编码器生成方法的流程示意图；
图3为本申请实施例所提供的再一种共享编码器生成方法的流程示意图；
图4为本申请实施例所提供的又一种共享编码器生成方法的流程示意图；
图5为本申请实施例所提供的又一种共享编码器生成方法的流程示意图；
图6为本申请实施例提供的一种共享编码器生成装置的结构示意图；
图7为本申请实施例提供的又一种共享编码器生成装置的结构示意图；
图8为本申请实施例提供的电子设备的结构示意图。
具体实施方式
以下结合附图对本申请的示范性实施例做出说明，其中包括本申请实施例的各种细节以助于理解，应当将它们认为仅仅是示范性的。因此，本领域普通技术人员应当认识到，可以对这里描述的实施例做出各种改变和修改，而不会背离本申请的范围和精神。同样，为了清楚和简明，以下的描述中省略了对公知功能和结构的描述。
本申请实施例针对相关技术中，多领域训练语料的获取难度大、成本高，从而导致预训练模型的实现难度大的问题，提出一种共享编码器生成方法。
下面参考附图对本申请提供的共享编码器生成方法、装置、电子设备及存储介质进行详细描述。
下面分别以主节点侧、子节点侧为例，对本申请实施例提供的共享编码器生成方法进行详细说明。
首先以主节点侧为例，对本申请实施例提供的共享编码器生成方法进行详细说明。
下面结合图1，对本申请实施例提供的共享编码器生成方法进行详细说明。
图1为本申请实施例所提供的一种共享编码器生成方法的流程示意图，该方法应用于主节点。
如图1所示，该共享编码器生成方法，包括以下步骤：
步骤101，主节点向各子节点发送共享编码器训练指令，所训练指令中包括待训练的目标共享编码器类型，以使每个子节点根据目标共享编码器的类型获取训练样本。
需要说明的是，预训练模型已经在计算机视觉、自然语言处理等领域中的应用实践被证明有效，在大数据下进行一次预训练再根据具体任务进行模型参数微调已经成为一种流行的做法。然而，由于预训练模型需要大量的预训练语料，而采集多种领域的无监督语料是一个难题。此外，不同的公司、组织拥有和自己领域业务相关的大量无监督语料，但可能并不开放，这导致共享预训练语料，训练得到一个更强大的预训练模型比较困难。本申请实施例的共享编码器生成方法，通过使用深度学习中的分布式训练技术，使得多个子节点(多方组织)共享训练语料中的信息训练得到共享编码器，并保证子节点的训练语料的安全性和隐私性。
其中，主节点，是指用于触发共享编码器训练过程的节点。比如，本申请实施例的共享编码器生成方法应用与联邦学习系统时，主节点可以是联邦学习系统的服务器。
子节点，是指利用自身的训练样本参与共享编码器训练的节点。比如，可以是参与共享编码器训练的公司、组织训练模型时所使用的服务器。
实际使用时，本申请实施例的共享编码器生成方法可以应用在任意进行联合训练的场景，本申请实施例对此不做限定。以下以本申请实施例的共享编码器生成方法的应用场景为联邦学习系统为例，进行详细说明。
在本申请实施例中，主节点可以在有训练共享编码器的需求时，向联邦学习系统中的各子节点发送共享编码器训练指令，以使各子节点在获取到共享编码器训练指令时，根据训练指令中包括的待训练的目标共享编码器的类型，判断本地是否有与该目标共享编码器类型对应的训练样本，若有，则获取与该目标共享编码器的类型对应的训练样本。
需要说明的是，子节点可能包括多种类型的训练样本，以用于训练不同类型的共享编码器，从而主节点在向各子节点发送共享编码器训练指令时，可以包括待训练的目标共享编码器类型，以使子节点从多种类型的训练样本中，获取与目标共享编码器的类型对应的训练样本。
进一步的，主节点可以根据子节点对目标共享编码器的获取请求或更新请求，触发对目标共享编码器的训练。即在本申请实施例一种可能的实现形式中，上述步骤101之前，还可以包括：
所述主节点获取到至少一个子节点发送的获取目标共享编码器的请求；
或者，
所述主节点获取到至少一个子节点发送的共享编码器更新请求，所述更新请求中 包括所述目标共享编码器的标识。
作为一种可能的实现方式，主节点可以在未对目标共享编码器进行训练之前，根据各子节点对目标共享编码器的获取请求，触发对目标共享编码器的训练过程。即主节点可以在获取到至少一个子节点发送的获取目标共享编码器的请求，并且获取请求中包括目标共享编码器对应的初始参数集时，确定获取到至少一个子节点对目标共享编码器的获取请求，从而可以向各子节点发送共享编码器训练指令，以触发对目标共享编码器的训练过程。
作为一种可能的实现方式，主节点还可以在对目标共享编码器进行一次或多次训练之后，根据各子节点对目标共享编码器的更新需求，触发对目标共享编码器的训练过程。即主节点可以在获取到至少一个子节点发送的共享编码器更新请求时，向各子节点发送共享编码器训练指令，以触发对目标共享编码器的训练过程，从而实现对目标共享编码器的更新。其中，子节点发送的共享编码器更新请求中可以包括目标共享编码器的标识，以使主节点可以根据目标共享编码器的标识确定目标共享编码器的类型，进而根据目标共享编码器的类型向各子节点发送共享编码器训练指令。
作为一种可能的实现方式，主节点还可以根据更新规则，主动触发对目标共享编码器的训练过程。比如，更新规则为按照预设的频率对目标共享编码器进行更新，主节点则可以按照预设的频率向各子节点发送共享编码器训练指令，以触发目标共享编码器的训练过程。
需要说明的是，触发目标共享编码器的训练过程的方式，可以包括但不限于以上列举的情形。实际使用时，可以根据实际需要或具体的应用场景确定合适的触发目标共享编码器的训练过程的方式，本申请实施例对此不做限定。
步骤102，在获取到各子节点返回的确认消息后，向各子节点发送待训练的目标共享编码器的初始参数集，以使各子节点利用各自的训练样本对目标共享编码器的初始参数集进行训练。
在本申请实施例中，主节点向各子节点发送共享编码器训练指令之后，各子节点可以根据自身对目标共享编码器的需求，或者是否有与目标共享编码器类型相符的训练样本，判断是否加入对目标共享编码器的训练。若可以加入对目标共享编码器的训练，则可以向主节点返回确认消息；若不可以加入对目标共享编码器的训练，则不向主节点返回确认消息。
从而，主节点可以在获取到子节点返回的确认消息时，确定参与目标共享编码器训练的各子节点(即返回确认消息的各子节点)，并拉取待训练的目标共享编码器的初始参数集，并将目标共享编码器的初始参数集发送给参与目标共享编码器训练的各 子节点，以使各子节点可以利用各自的训练样本对目标共享编码器的初始参数集进行训练。
作为一种可能的实现方式，主节点在获取到子节点返回的确认消息之后，还可以将预先设置的可执行程序与待训练的目标共享编码器的初始参数集，一同发送给参与目标共享编码器训练的各子节点，以使参与目标共享编码器训练的各子节点可以根据可执行程序对目标共享编码器进行训练。
需要说明的是，主节点发送给各子节点的可执行程序中可以包括子节点对目标共享编码器进行训练的规则、返回目标共享编码器的更新参数集的规则等。比如，可执行程序中可以包括子节点每次获取到主节点发送的训练指令时，对目标共享编码器进行训练的次数；或者，还可以包括子节点每次获取到主节点发送的训练指令后，向主节点返回目标共享编码器的更新参数集的时间周期，等等，本申请实施例对此不做限定。
进一步的，在对目标共享编码器进行训练之前，主节点需要确定目标共享编码器对应的初始参数集。即在本申请实施例一种可能的实现形式中，上述步骤102之前，还可以包括：
依据第二预设规则，生成所述待训练的目标共享编码器。
其中，第二预设规则，是指预先设置的获取待训练的目标共享编码器的初始参数集的规则，且第二预设规则可以根据实际需要进行修改。
作为一种可能的实现方式，第二预设规则可以是“通过各子节点预先定义”，则主节点可以预先从各子节点获取各子节点对目标共享编码器预先定义的初始参数集，且目标共享编码器的初始参数集是各子节点通过协商定义的，因此各子节点预先定义的初始参数集相同。
作为一种可能的实现方式，第二预设规则还可以是“根据公开模型中已有的同类型编码器的参数集确定”，则主节点可以从公开模型中，获取与目标共享编码器的类型相同的编码器，进而将获取的编码器的参数集确定为目标共享编码器的初始参数集。
步骤103，获取各子节点返回的目标共享编码器的更新参数集。
在本申请实施例中，主节点在向各子节点发送待训练的目标共享编码器的初始参数集之后，即可实时获取各子节点返回的目标共享编码器的更新参数集。其中，各子节点返回的目标共享编码器的更新参数集，是各子节点利用自身的训练样本，根据目标共享编码器的初始参数集训练得到的。
步骤104，根据第一预设规则及各子节点返回的目标共享编码器的更新参数集，确定目标共享编码器对应的目标参数集。
其中，第一预设规则，是指预先设置的将各子节点返回的目标共享编码器的更新参数进行融合，以获得目标共享编码器对应的目标参数集的规则。比如，第一预设规则可以是将各子节点返回的更新参数集中各参数的均值，确定为目标参数集中的相应参数的取值；或者还可以是将各子节点返回的更新参数集中各参数的方差、标准差、中值等，确定为目标参数集中的相应参数的取值。
需要说明的是，第一预设规则可以包括但不限于以上列举的情形。实际使用时，可以根据实际需要进行预设以及实时修改，以适应当前的使用需求和应用场景，本申请实施例对此不做限定。
在本申请实施例中，主节点获取到各子节点返回的目标共享编码器的更新参数集之后，则可以根据第一预设规则，将获取的各更新参数集进行融合，以确定目标共享编码器对应的目标参数集。
举例来说，第一预设规则为“将各子节点返回的更新参数集中各参数的均值，确定为目标参数集中的相应参数的取值”，假设当前联邦学习系统中具有3个子节点X、Y、Z，目标共享编码器的参数集中包括3个参数，子节点X返回的目标共享编码器的更新参数集为{a 1,b 1,c 1}，子节点Y返回的目标共享编码器的更新参数集为{a 2,b 2,c 2}，子节点Z返回的目标共享编码器的更新参数集为{a 3,b 3,c 3}，则目标共享编码器对应的目标参数集为{a,b,c}，其中，a＝(a 1+a 2+a 3)/3，b＝(b 1+b 2+b 3)/3，c＝(c 1+c 2+c 3)/3。
根据本申请实施例的技术方案，通过主节点向各子节点发送共享编码器训练指令，以使每个子节点根据训练指令中包括的目标共享编码器的类型获取训练样本，并在获取到各子节点返回的确认消息后，向各子节点发送待训练的目标共享编码器的初始参数集，以使各子节点利用各自的训练样本对目标共享编码器的初始参数集进行训练，之后获取各子节点返回的目标共享编码器的更新参数集，进而根据第一预设规则及各子节点返回的目标共享编码器的更新参数集，确定目标共享编码器对应的目标参数集。由此，通过主节点组织各子节点利用私有训练样本对目标共享编码器进行训练，以根据各子节点对目标共享编码器的训练结果，确定目标共享编码器的目标参数集，从而不仅实现了多领域训练语料的共享，降低了多领域训练语料的获取难度和成本，而且提高了共享编码器的性能。
在本申请一种可能的实现形式中，为进一步提高最终获得的共享编码器的性能，以及平衡各子节点在共享编码器的训练过程中的重要程度，还可以根据子节点的训练样本的数量，确定为各子节点赋予不同的权重。
下面结合图2，对本申请实施例提供的共享编码器生成方法进行进一步说明。
图2为本申请实施例所提供的另一种共享编码器生成方法的流程示意图，该方法 应用于主节点。
如图2所示，该共享编码器生成方法，包括以下步骤：
步骤201，主节点向各子节点发送共享编码器训练指令，训练指令中包括待训练的目标共享编码器类型，以使每个子节点根据目标共享编码器的类型获取训练样本。
步骤202，在获取到各子节点返回的确认消息后，向各子节点发送待训练的目标共享编码器的初始参数集，以使各子节点利用各自的训练样本对目标共享编码器的初始参数集进行训练。
上述步骤201-202的具体实现过程及原理，可以参照上述实施例的详细描述，此处不再赘述。
步骤203，获取各子节点包含的训练样本的数量。
其中，子节点包含的训练样本的数量，可以是指子节点包含的可以用于训练目标共享编码器的训练样本的数量。
作为一种可能的实现方式，主节点可以根据各子节点包含的可以用于训练目标共享编码器的训练样本的数量，确定各子节点在训练目标共享编码器时的权重。因此，主节点在向各子节点发送待训练的目标共享编码器的初始参数集之后，可以实时获取各子节点包含的可用于训练目标共享编码器的训练样本的数量。
步骤204，根据每个子节点包含的训练样本的数量，确定每个子节点的权重。
在本申请实施例中，由于子节点包含的训练样本的数量越多，则根据该子节点返回的更新参数集，确定的目标共享编码器的性能越好，从而主节点可以根据每个子节点包含的训练样本的数量，确定每个子节点的权重。
作为一种可能的实现方式，子节点的权重可以与子节点包含的训练样本的数量成正比例关系，即子节点包含的训练样本的数量越多，则该子节点的权重越大，从而使得训练样本数量越多的子节点在训练目标共享编码器时，所起的作用越大，以提高最终获得的目标共享编码器的性能。
举例来说，可以将每个子节点包含的训练样本的数量与所有子节点包含的训练样本的数量的比值，确定为每个子节点的权重。比如，当前联邦学习系统中具有3个子节点X、Y、Z，子节点X包含的训练样本的数量为m，子节点Y包含的训练样本的数量为n，子节点Z包含的训练样本的数量为k，则子节点X的权重为m/(m+n+k)，子节点Y的权重为n/(m+n+k)，子节点Z的权重为k/(m+n+k)。
步骤205，获取各子节点返回的目标共享编码器的更新参数集。
上述步骤205的具体实现过程及原理，可以参照上述实施例的详细描述，此处不再赘述。
步骤206，根据第一预设规则、每个子节点的权重及各子节点返回的目标共享编码器的更新参数集，确定目标共享编码器对应的目标参数集。
在本申请实施例中，确定出每个子节点的权重以及获取到每个子节点返回的目标共享编码器的更新参数集之后，即可以根据第一预设的规则和每个子节点的权重，对各子节点返回的目标共享编码器的更新参数集进行融合，以确定目标共享编码器对应的目标参数集。
作为一种可能的实现方式，可以首先根据每个子节点的权重对每个子节点返回的目标共享编码器的更新参数集进行修正，即将每个子节点的权重与其对应的更新参数集中各参数的乘积，确定为每个子节点修正后的更新参数集中的各参数的取值。之后，根据第一预设规则，每个子节点修正后的更新参数集进行融合，以确定目标共享编码器对应的目标参数集。
举例来说，第一预设规则为“将各子节点返回的更新参数集中各参数的均值，确定为目标参数集中的相应参数的取值”，当前联邦学习系统中具有3个子节点X、Y、Z，子节点X的权重为w 1，返回的目标共享编码器的更新参数集为{a 1,b 1,c 1}，子节点Y的权重为w 2，返回的目标共享编码器的更新参数集为{a 2,b 2,c 2}，子节点Z的权重为w 3，返回的目标共享编码器的更新参数集为{a 3,b 3,c 3}，则子节点X修正后的更新参数集为{w 1·a 1,w 1·b 1,w 1·c 1}，子节点Y修正后的更新参数集为{w 2·a 2,w 2·b 2,w 2·c 2}，子节点Z修正后的更新参数集为{w 3·a 3,w 3·b 3,w 3·c 3}，从而目标共享编码器对应的目标参数集为{a,b,c}，其中，a＝(w 1·a 1+w 2·a 2+w 3·a 3)/3，b＝(w 1·b 1+w 2·b 2+w 3·b 3)/3，c＝(w 1·c 1+w 2·c 2+w 3·c 3)/3。
根据本申请实施例的技术方案，通过主节点向各子节点发送共享编码器训练指令，以使每个子节点根据训练指令中包括的目标共享编码器的类型获取训练样本，并在获取到各子节点返回的确认消息后，向各子节点发送待训练的目标共享编码器的初始参数集，以使各子节点利用各自的训练样本对目标共享编码器的初始参数集进行训练，以及根据获取的每个子节点包含的训练样本的数量，确定每个子节点的权重，之后获取各子节点返回的目标共享编码器的更新参数集，进而根据第一预设规则、每个子节点的权重及各子节点返回的目标共享编码器的更新参数集，确定目标共享编码器对应的目标参数集。由此，通过主节点组织各子节点利用私有训练样本对目标共享编码器进行训练，并根据各子节点包含的训练样本的数量调整各子节点的权重，从而不仅实现了多领域训练语料的共享，降低了多领域训练语料的获取难度和成本，而且进一步提高了共享编码器的性能。
在本申请一种可能的实现形式中，为进一步提高共享编码器的可靠性和鲁棒性，主节点可以根据前一次训练过程获得的目标参数集，判断共享编码器是否满足性能要求，若不满足，则可以在前一次训练的基础上对共享编码器进行再次训练。
下面结合图3，对本申请实施例提供的共享编码器生成方法进行进一步说明。
图3为本申请实施例所提供的再一种共享编码器生成方法的流程示意图，该方法应用于主节点。
如图3所示，该共享编码器生成方法，包括以下步骤：
步骤301，主节点向各子节点发送共享编码器训练指令，所训练指令中包括待训练的目标共享编码器类型，以使每个子节点根据目标共享编码器的类型获取训练样本。
步骤302，在获取到各子节点返回的确认消息后，向各子节点发送待训练的目标共享编码器的初始参数集，以使各子节点利用各自的训练样本对目标共享编码器的初始参数集进行训练。
步骤303，获取各子节点返回的目标共享编码器的更新参数集。
步骤304，根据第一预设规则及各子节点返回的目标共享编码器的更新参数集，确定目标共享编码器对应的目标参数集。
上述步骤301-304的具体实现过程及原理，可以参照上述实施例的详细描述，此处不再赘述。
步骤305，将目标共享编码器对应的目标参数集分别发送给每个子节点，以使每个子节点基于目标共享编码器对应的目标参数集，对目标共享编码器继续进行训练。
在本申请实施例中，主节点可以在每次确定出目标共享编码器对应的目标参数集之后，判断目标共享编码器是否符合性能需求，若符合性能需求，则可以确定完成本次对目标共享编码器的训练过程；若不符合性能需求，则可以确定需要对目标共享编码器进行再次训练，以提升目标共享编码器的性能。
因此，主节点在确定目标共享编码器对应的目标参数集不符合性能需求之后，可以将目标共享编码器对应的目标参数集分别发送给各子节点，以使各子节点可以利用其自身的训练样本，并基于目标共享编码器的目标参数集，对目标共享编码器继续进行训练。直至主节点根据各子节点再次返回的更新参数集，确定的目标共享编码器对应的目标参数集符合性能需求，则不再向各子节点发送目标共享编码器对应的目标参数集，即完成本次对目标共享编码器的训练过程。
进一步的，在对目标共享编码器进行多次训练时，还可以根据各子节点对目标共享编码器的训练次数，确定各子节点的权重，以平衡各子节点在目标共享编码器训练过程中所起的作用。即在本申请实施例一种可能的实现形式中，上述步骤305之后， 还可以包括：
获取第一子节点再次返回的目标共享编码器参数集；
根据已获取的所述第一子节点返回目标共享编码器参数集的次数，确定所述第一子节点当前的权重值；
依据所述第一预设规则，根据所述第一子节点当前的权重值及当前再次返回的目标共享编码器参数集，对所述目标共享编码器对应的目标参数集进行更新。
其中，第一子节点，是指多个子节点中多次参与目标共享编码器训练的子节点。
作为一种可能的实现方式，主节点获取到第一子节点再次返回的共享编码器参数集时，则可以确定已获取的第一子节点返回目标共享编码器参数集的次数(即第一子节点参与目标共享编码器训练的次数)，并根据第一子节点返回目标共享编码器参数集的次数，确定第一子节点当前的权重值。
可选的，由于子节点参与目标共享编码器训练的次数越多，则该子节点返回的目标共享编码器的参数集，对最终确定的目标共享编码器对应的目标参数集的影响越大，从而使得最终获得的目标共享编码器过于贴合该子节点的训练样本的特性，从而使得目标共享编码器的通用性变差。因此，在本申请实施例一种可能的实现形式中，第一子节点当前的权重值，可以与第一子节点返回目标共享编码器参数集的次数呈负相关关系。
在本申请实施例中，由于每个子节点返回目标共享编码器参数集的时刻可能是不同的，因此，可以在每次获取到一个第一子节点再次返回的目标共享编码器参数集时，即确定该第一子节点当前的权重值。进而，主节点根据第一子节点当前的权重值对第一子节点再次返回的目标共享编码参数集进行修正，之后根据第一预设规则，将修正后的第一子节点再次返回的目标共享编码器参数集和前一次确定的目标共享编码器对应的目标参数集进行融合，以对目标共享编码器对应的目标参数集进行更新。
举例来说，第一预设规则为“将各子节点返回的更新参数集中各参数之和，确定为目标参数集中的相应参数的取值”，主节点前一次确定的目标共享编码器对应的目标参数集为{a,b,c}，当前获取到第一子节点X再次返回的目标共享编码器参数集为{a x,b x,c x}，根据第一子节点X返回目标共享编码器参数集的次数，确定第一子节点X的权重值为w x，则可以确定修正后的第一子节点X再次返回的目标共享编码器参数集为{w x·a x,w x·b x,w x·c x}，更新后的目标共享编码器对应的目标参数集为{a+w x·a x,b+w x·b x,c+w x·c x}。
根据本申请实施例的技术方案，通过主节点向各子节点发送共享编码器训练指令， 以使每个子节点根据训练指令中包括的目标共享编码器的类型获取训练样本，并在获取到各子节点返回的确认消息后，向各子节点发送待训练的目标共享编码器的初始参数集，以使各子节点利用各自的训练样本对目标共享编码器的初始参数集进行训练，之后获取各子节点返回的目标共享编码器的更新参数集，以及根据第一预设规则及各子节点返回的目标共享编码器的更新参数集，确定目标共享编码器对应的目标参数集，进而将目标共享编码器对应的目标参数集分别发送给每个子节点，以使每个子节点基于目标共享编码器对应的目标参数集，对目标共享编码器继续进行训练。由此，通过主节点组织各子节点利用私有训练样本对目标共享编码器进行多次训练，以根据各子节点对目标共享编码器的多次训练结果，确定目标共享编码器的目标参数集，从而不仅实现了多领域训练语料的共享，降低了多领域训练语料的获取难度和成本，而且进一步提高了共享编码器的准确性和通用性。
下面以子节点侧为例，对本申请实施例提供的共享编码器生成方法进行详细说明。
下面结合图4，对本申请实施例提供的共享编码器生成方法，进行进一步说明。
图4为本申请实施例所提供的又一种共享编码器生成方法的流程示意图，该方法应用于子节点。
如图4所示，该共享编码器生成方法，包括以下步骤：
步骤401，第二子节点获取主节点发送的共享编码器训练指令，训练指令中包括待训练的目标共享编码器类型。
其中，主节点，是指用于触发共享编码器训练过程的节点。比如，本申请实施例的共享编码器生成方法应用与联邦学习系统时，主节点可以是联邦学习系统的服务器。
子节点，是指利用自身的训练样本参与共享编码器训练的节点。比如，可以是参与共享编码器训练的公司、组织训练模型时所使用的服务器。第二子节点，是指多个子节点中的其中一个子节点。
实际使用时，本申请实施例的共享编码器生成方法可以应用在任意进行联合训练的场景，本申请实施例对此不做限定。以下以本申请实施例的共享编码器生成方法的应用场景为联邦学习系统为例，进行详细说明。
在本申请实施例中，第二子节点可以实时获取主节点在有训练共享编码器的需求时，向各子节点发送共享编码器训练指令，第二子节点在获取到共享编码器训练指令时，根据训练指令中包括的待训练的目标共享编码器的类型，判断是否加入对目标共享编码器的训练。
步骤402，确定包含与目标共享编码器类型匹配的训练样本后，向主节点返回确认消息。
在本申请实施例中，若第二子节点确定包含与目标共享编码器类型匹配的训练样本，则确定可以加入对目标共享编码器的训练，从而可以向主节点返回确认消息，以使主节点根据获取到的确认消息时，确定加入对目标共享编码器进行训练的子节点，并拉取待训练的目标共享编码器的初始参数集，发送给第二子节点。
步骤403，获取主节点发送的目标共享编码器的初始参数集。
在本申请实施例中，第二子节点向主节点返回确认消息之后，即可以实时获取主节点返回的目标共享编码器的初始参数集，以根据目标共享编码器的初始参数集，对目标共享编码器进行训练。
步骤404，利用与目标共享编码器类型匹配的训练样本，进行模型训练，确定目标共享编码器的更新参数集。
在本申请实施例中，第二子节点在获取到目标共享编码器的初始参数集之后，即可以利用与目标共享编码器的类型匹配的训练样本，及获取的目标共享编码器的初始参数集，进行模型训练，以确定目标共享编码器的更新参数集。
步骤405，向主节点返回目标共享编码器的更新参数集。
在本申请实施例中，第二子节点在完成对目标共享编码器的训练之后，则可以将确定的目标共享编码器的更新参数集，返回至主节点，以使主节点可以根据各子节点返回的更新参数集，确定目标共享编码器对应的目标参数集。
进一步的，为进一步提高最终获得的共享编码器的性能，以及平衡各子节点在共享编码器的训练过程中的重要程度，主节点还可以根据子节点的训练样本的数量，确定为各子节点赋予不同的权重。即在本申请实施例一种可能的实现形式中，上述方法，还可以包括：
向所述主节点发送与目标共享编码器类型匹配的训练样本的数量。
作为一种可能的实现方式，主节点可以根据第二子节点包含的可以用于训练目标共享编码器的训练样本的数量，即与目标共享编码器类型匹配的训练样本的数量，确定第二子节点在训练目标共享编码器时的权重。因此，第二子节点可以向主节点返回发送与目标共享编码器类型匹配的训练样本的数量，以使主节点可以根据第二子节点包含的与目标共享编码器类型匹配的训练样本的数量，确定第二子节点的权重。
进一步的，子节点还可以根据私有的训练样本，设计私有编码器和私有解码器，进而根据设计的私有编码器和私有解码器，确定可以加入的共享编码器训练任务。即在本申请实施例一种可能的实现形式中，上述步骤405之后，还可以包括：
确定与目标共享编码器对应的私有编码器及私有解码器。
在本申请实施例一种可能的实现形式中，第二子节点可以根据包含的训练样本， 设计私有编码器和私有解码器，并可以在利用与目标共享编码器类型匹配的训练样本对目标共享编码器进行训练之后，获取与目标共享编码器对应的私有编码器和私有解码器，进而根据对目标共享编码器的训练结果和与目标共享编码器对应的私有编码器和私有解码器，共同确定目标共享编码器的更新参数集。
根据本申请实施例的技术方案，通过第二子节点获取主节点发送的共享编码器训练指令，并在确定包含与目标共享编码器类型匹配的训练样本后，向主节点返回确认消息，以及获取主节点发送的目标共享编码器的初始参数集，之后利用与目标共享编码器类型匹配的训练样本，进行模型训练，确定目标共享编码器的更新参数集，进而向主节点返回目标共享编码器的更新参数集。由此，通过主节点组织各子节点利用私有训练样本对目标共享编码器进行训练，以根据各子节点对目标共享编码器的训练结果，确定目标共享编码器的目标参数集，从而不仅实现了多领域训练语料的共享，降低了多领域训练语料的获取难度和成本，而且提高了共享编码器的性能。
在本申请一种可能的实现形式中，子节点可以根据主节点预设的更新规则，向主节点返回目标共享编码器的更新参数集。
下面结合图5，对本申请实施例提供的共享编码器生成方法进行进一步说明。
图5为本申请实施例所提供的又一种共享编码器生成方法的流程示意图，该方法应用于子节点。
如图5所示，该共享编码器生成方法，包括以下步骤：
步骤501，第二子节点获取主节点发送的共享编码器训练指令，训练指令中包括待训练的目标共享编码器类型。
步骤502，确定包含与目标共享编码器类型匹配的训练样本后，向主节点返回确认消息。
上述步骤501-502的具体实现过程及原理，可以参照上述实施例的详细描述，此处不再赘述。
步骤503，获取主节点返回的目标共享编码器参数更新规则。
其中，目标共享编码器参数更新规则，可以包括在主节点返回的可执行程序中。
作为一种可能的实现方式，主节点在获取到子节点返回的确认消息之后，还可以将预先设置的可执行程序发送给参与目标共享编码器训练的各子节点，从而第二子节点可以根据可执行程序对目标共享编码器进行训练，并根据可执行程序中包括的目标共享编码器参数更新规则，判断是否可以将确定的更新参数集返回主节点。
需要说明的是，目标共享编码器参数更新规则，可以是第二子节点每次获取到主节点发送的训练指令时，对目标共享编码器进行训练的次数；或者，还可以是第二子 节点每次获取到主节点发送的训练指令后，向主节点返回目标共享编码器的更新参数集的时间周期，等等，本申请实施例对此不做限定。
举例来说，目标共享编码器参数更新规则可以是“对目标共享编码器进行5轮训练后向主节点返回获得的目标共享编码的更新参数集”，或者，还可以是“每月向主节点返回一次目标共享编码的更新参数集”。
步骤504，获取主节点发送的目标共享编码器的初始参数集。
步骤505，利用与目标共享编码器类型匹配的训练样本，进行模型训练，确定目标共享编码器的更新参数集。
上述步骤504-505的具体实现过程及原理，可以参照上述实施例的详细描述，此处不再赘述。
步骤506，在确定当前满足更新规则时，向主节点返回目标共享编码器的更新参数集。
在本申请实施例中，第二子节点每完成对目标共享编码器的一轮训练，则可以确定当前是否满足目标共享编码器参数更新规则，若满足，则可以向主节点返回当前获得的目标共享编码器的更新参数集；若不满足，则继续对目标共享编码器进行下一轮训练，直至当前满足目标共享编码器参数更新规则。
举例来说，目标共享编码器参数更新规则为“子节点每对目标共享编码器完成5轮训练向主节点返回目标共享编码器的更新参数集”，若第二子节点当前完成对目标共享编码器的第4轮训练，则可以确定当前不满足目标共享编码器参数更新规则，即可以继续对目标共享编码器进行下一轮训练；若第二子节点当前完成对目标共享编码器的第5轮训练，则可以确定当前满足目标共享编码器参数更新规则，则可以将第5轮训练结束后获得的目标共享编码器的更新参数集，返回至主节点。
根据本申请实施例的技术方案，通过第二子节点获取主节点发送的共享编码器训练指令，并在确定包含与目标共享编码器类型匹配的训练样本后，向主节点返回确认消息，以及获取主节点返回的目标共享编码器参数更新规则及目标共享编码器的初始参数集，之后利用与目标共享编码器类型匹配的训练样本，进行模型训练，确定目标共享编码器的更新参数集，进而在确定当前满足更新规则时，向主节点返回目标共享编码器的更新参数集。由此，通过主节点组织各子节点利用私有训练样本对目标共享编码器进行训练，并根据主节点返回的参数更新规则，确定子节点向主节点返回目标共享编码器的更新参数集的时机，从而不仅实现了多领域训练语料的共享，降低了多领域训练语料的获取难度和成本，而且进一步提高了共享编码器的性能。
为了实现上述实施例，本申请还提出一种共享编码器生成装置。
图6为本申请实施例提供的一种共享编码器生成装置的结构示意图，该装置应用于主节点。
如图6所示，该共享编码器生成装置60，包括：
第一发送模块61，用于主节点向各子节点发送共享编码器训练指令，所述训练指令中包括待训练的目标共享编码器类型，以使每个子节点根据目标共享编码器的类型获取训练样本；
第二发送模块62，用于在获取到各子节点返回的确认消息后，向各子节点发送待训练的目标共享编码器的初始参数集，以使各子节点利用各自的训练样本对所述目标共享编码器的初始参数集进行训练；
第一获取模块63，用于获取各子节点返回的目标共享编码器的更新参数集；
第一确定模块64，用于根据第一预设规则及各子节点返回的目标共享编码器的更新参数集，确定所述目标共享编码器对应的目标参数集。
在实际使用时，本申请实施例提供的共享编码器生成装置，可以被配置在任意电子设备中，以执行前述共享编码器生成方法。
根据本申请实施例的技术方案，通过主节点向各子节点发送共享编码器训练指令，以使每个子节点根据训练指令中包括的目标共享编码器的类型获取训练样本，并在获取到各子节点返回的确认消息后，向各子节点发送待训练的目标共享编码器的初始参数集，以使各子节点利用各自的训练样本对目标共享编码器的初始参数集进行训练，之后获取各子节点返回的目标共享编码器的更新参数集，进而根据第一预设规则及各子节点返回的目标共享编码器的更新参数集，确定目标共享编码器对应的目标参数集。由此，通过主节点组织各子节点利用私有训练样本对目标共享编码器进行训练，以根据各子节点对目标共享编码器的训练结果，确定目标共享编码器的目标参数集，从而不仅实现了多领域训练语料的共享，降低了多领域训练语料的获取难度和成本，而且提高了共享编码器的性能。
在本申请一种可能的实现形式中，上述共享编码器生成装置60，还包括：
第二获取模块，用于所述主节点获取到至少一个子节点发送的获取目标共享编码器的请求；
或者，
第三获取模块，用于所述主节点获取到至少一个子节点发送的目标共享编码器更新请求，所述更新请求中包括所述目标共享编码器的标识。
进一步的，在本申请另一种可能的实现形式中，上述共享编码器生成装置60，还包括：
第四获取模块，用于获取各子节点包含的训练样本的数量；
第二确定模块，用于根据每个子节点包含的训练样本的数量，确定每个子节点的权重；
相应的，上述第一确定模块64，具体用于：
根据所述第一预设规则、所述每个子节点的权重及各子节点返回的目标共享编码器的更新参数集，确定所述目标共享编码器对应的目标参数集。
进一步的，在本申请再一种可能的实现形式中，上述共享编码器生成装置60，还包括：
第三发送模块，用于将所述目标共享编码器对应的目标参数集分别发送给每个子节点，以使每个子节点基于所述目标共享编码器对应的目标参数集，对所述目标共享编码器继续进行训练。
进一步的，在本申请又一种可能的实现形式中，上述共享编码器生成装置60，还包括：
第五获取模块，用于获取第一子节点再次返回的目标共享编码器参数集；
第三确定模块，用于根据已获取的所述第一子节点返回目标共享编码器参数集的次数，确定所述第一子节点当前的权重值；
更新模块，用于依据所述第一预设规则，根据所述第一子节点当前的权重值及当前再次返回的目标共享编码器参数集，对所述目标共享编码器对应的目标参数集进行更新。
进一步的，在本申请又一种可能的实现形式中，上述共享编码器生成装置60，还包括：
生成模块，用于依据第二预设规则，生成所述待训练的目标共享编码器。
需要说明的是，前述对图1、图2、图3所示的共享编码器生成方法实施例的解释说明也适用于该实施例的共享编码器生成装置60，此处不再赘述。
根据本申请实施例的技术方案，通过主节点向各子节点发送共享编码器训练指令，以使每个子节点根据训练指令中包括的目标共享编码器的类型获取训练样本，并在获取到各子节点返回的确认消息后，向各子节点发送待训练的目标共享编码器的初始参数集，以使各子节点利用各自的训练样本对目标共享编码器的初始参数集进行训练，之后获取各子节点返回的目标共享编码器的更新参数集，以及根据第一预设规则及各子节点返回的目标共享编码器的更新参数集，确定目标共享编码器对应的目标参数集，进而将目标共享编码器对应的目标参数集分别发送给每个子节点，以使每个子节点基于目标共享编码器对应的目标参数集，对目标共享编码器继续进行训练。由此，通过 主节点组织各子节点利用私有训练样本对目标共享编码器进行多次训练，以根据各子节点对目标共享编码器的多次训练结果，确定目标共享编码器的目标参数集，从而不仅实现了多领域训练语料的共享，降低了多领域训练语料的获取难度和成本，而且进一步提高了共享编码器的准确性和通用性。
为了实现上述实施例，本申请还提出另一种共享编码器生成装置。
图7为本申请实施例提供的另一种共享编码器生成装置的结构示意图，该装置应用于子节点。
如图7所示，该共享编码器生成装置70，包括：
第一获取模块71，用于第二子节点获取主节点发送的共享编码器训练指令，所述训练指令中包括待训练的目标共享编码器类型；
第一返回模块72，用于确定包含与所述目标共享编码器类型匹配的训练样本后，向所述主节点返回确认消息；
第二获取模块73，用于获取所述主节点发送的目标共享编码器的初始参数集；
第一确定模块74，用于利用与所述目标共享编码器类型匹配的训练样本，进行模型训练，确定所述目标共享编码器的更新参数集；
第二返回模块75，用于向所述主节点返回所述目标共享编码器的更新参数集。
在实际使用时，本申请实施例提供的共享编码器生成装置，可以被配置在任意电子设备中，以执行前述共享编码器生成方法。
根据本申请实施例的技术方案，通过第二子节点获取主节点发送的共享编码器训练指令，并在确定包含与目标共享编码器类型匹配的训练样本后，向主节点返回确认消息，以及获取主节点发送的目标共享编码器的初始参数集，之后利用与目标共享编码器类型匹配的训练样本，进行模型训练，确定目标共享编码器的更新参数集，进而向主节点返回目标共享编码器的更新参数集。由此，通过主节点组织各子节点利用私有训练样本对目标共享编码器进行训练，以根据各子节点对目标共享编码器的训练结果，确定目标共享编码器的目标参数集，从而不仅实现了多领域训练语料的共享，降低了多领域训练语料的获取难度和成本，而且提高了共享编码器的性能。
在本申请一种可能的实现形式中，上述共享编码器生成装置70，还包括：
第三获取模块，用于获取所述主节点返回的目标共享编码器参数更新规则；
相应的，上述第二返回模块75，具体用于：
在确定当前满足所述更新规则时，向所述主节点返回所述目标共享编码器的更新参数集。
进一步的，在本申请另一种可能的实现形式中，上述共享编码器生成装置70，还 包括：
发送模块，用于向所述主节点发送与所述目标共享编码器类型匹配的训练样本的数量。
进一步的，在本申请再一种可能的实现形式中，上述共享编码器生成装置70，还包括：
第二确定模块，用于确定与所述目标共享编码器对应的私有编码器及私有解码器。
需要说明的是，前述对图4、图5所示的共享编码器生成方法实施例的解释说明也适用于该实施例的共享编码器生成装置70，此处不再赘述。
根据本申请实施例的技术方案，通过第二子节点获取主节点发送的共享编码器训练指令，并在确定包含与目标共享编码器类型匹配的训练样本后，向主节点返回确认消息，以及获取主节点返回的目标共享编码器参数更新规则及目标共享编码器的初始参数集，之后利用与目标共享编码器类型匹配的训练样本，进行模型训练，确定目标共享编码器的更新参数集，进而在确定当前满足更新规则时，向主节点返回目标共享编码器的更新参数集。由此，通过主节点组织各子节点利用私有训练样本对目标共享编码器进行训练，并根据主节点返回的参数更新规则，确定子节点向主节点返回目标共享编码器的更新参数集的时机，从而不仅实现了多领域训练语料的共享，降低了多领域训练语料的获取难度和成本，而且进一步提高了共享编码器的性能。
根据本申请的实施例，本申请还提供了一种电子设备和一种可读存储介质。
如图8所示，是根据本申请实施例的共享编码器生成方法的电子设备的框图。电子设备旨在表示各种形式的数字计算机，诸如，膝上型计算机、台式计算机、工作台、个人数字助理、服务器、刀片式服务器、大型计算机、和其它适合的计算机。电子设备还可以表示各种形式的移动装置，诸如，个人数字处理、蜂窝电话、智能电话、可穿戴设备和其它类似的计算装置。本文所示的部件、它们的连接和关系、以及它们的功能仅仅作为示例，并且不意在限制本文中描述的和/或者要求的本申请的实现。
如图8所示，该电子设备包括：一个或多个处理器801、存储器802，以及用于连接各部件的接口，包括高速接口和低速接口。各个部件利用不同的总线互相连接，并且可以被安装在公共主板上或者根据需要以其它方式安装。处理器可以对在电子设备内执行的指令进行处理，包括存储在存储器中或者存储器上以在外部输入/输出装置(诸如，耦合至接口的显示设备)上显示GUI的图形信息的指令。在其它实施方式中，若需要，可以将多个处理器和/或多条总线与多个存储器和多个存储器一起使用。同样，可以连接多个电子设备，各个电子设备提供部分必要的操作(例如，作为服务器阵列、一组刀片式服务器、或者多处理器系统)。图8中以一个处理器801为例。
存储器802即为本申请所提供的非瞬时计算机可读存储介质。其中，所述存储器存储有可由至少一个处理器执行的指令，以使所述至少一个处理器执行本申请所提供的共享编码器生成方法。本申请的非瞬时计算机可读存储介质存储计算机指令，该计算机指令用于使计算机执行本申请所提供的共享编码器生成方法。
存储器802作为一种非瞬时计算机可读存储介质，可用于存储非瞬时软件程序、非瞬时计算机可执行程序以及模块，如本申请实施例中的共享编码器生成方法对应的程序指令/模块(例如，附图6所示的第一发送模块61、第二发送模块62、第一获取模块63和第一确定模块64，以及附图7所示的第一获取模块71、第一返回模块72、第二获取模块73、第一确定模块74和第二返回模块75)。处理器801通过运行存储在存储器802中的非瞬时软件程序、指令以及模块，从而执行服务器的各种功能应用以及数据处理，即实现上述方法实施例中的共享编码器生成方法。
存储器802可以包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需要的应用程序；存储数据区可存储根据共享编码器生成方法的电子设备的使用所创建的数据等。此外，存储器802可以包括高速随机存取存储器，还可以包括非瞬时存储器，例如至少一个磁盘存储器件、闪存器件、或其他非瞬时固态存储器件。在一些实施例中，存储器802可选包括相对于处理器801远程设置的存储器，这些远程存储器可以通过网络连接至共享编码器生成方法的电子设备。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
共享编码器生成方法的电子设备还可以包括：输入装置803和输出装置804。处理器801、存储器802、输入装置803和输出装置804可以通过总线或者其他方式连接，图8中以通过总线连接为例。
输入装置803可接收输入的数字或字符信息，以及产生与共享编码器生成方法的电子设备的用户设置以及功能控制有关的键信号输入，例如触摸屏、小键盘、鼠标、轨迹板、触摸板、指示杆、一个或者多个鼠标按钮、轨迹球、操纵杆等输入装置。输出装置804可以包括显示设备、辅助照明装置(例如，LED)和触觉反馈装置(例如，振动电机)等。该显示设备可以包括但不限于，液晶显示器(LCD)、发光二极管(LED)显示器和等离子体显示器。在一些实施方式中，显示设备可以是触摸屏。
此处描述的系统和技术的各种实施方式可以在数字电子电路系统、集成电路系统、专用ASIC(专用集成电路)、计算机硬件、固件、软件、和/或它们的组合中实现。这些各种实施方式可以包括：实施在一个或者多个计算机程序中，该一个或者多个计算机程序可在包括至少一个可编程处理器的可编程系统上执行和/或解释，该可编程处理器可以是专用或者通用可编程处理器，可以从存储系统、至少一个输入装置、和至少 一个输出装置接收数据和指令，并且将数据和指令传输至该存储系统、该至少一个输入装置、和该至少一个输出装置。
这些计算程序(也称作程序、软件、软件应用、或者代码)包括可编程处理器的机器指令，并且可以利用高级过程和/或面向对象的编程语言、和/或汇编/机器语言来实施这些计算程序。如本文使用的，术语“机器可读介质”和“计算机可读介质”指的是用于将机器指令和/或数据提供给可编程处理器的任何计算机程序产品、设备、和/或装置(例如，磁盘、光盘、存储器、可编程逻辑装置(PLD))，包括，接收作为机器可读信号的机器指令的机器可读介质。术语“机器可读信号”指的是用于将机器指令和/或数据提供给可编程处理器的任何信号。
为了提供与用户的交互，可以在计算机上实施此处描述的系统和技术，该计算机具有：用于向用户显示信息的显示装置(例如，CRT(阴极射线管)或者LCD(液晶显示器)监视器)；以及键盘和指向装置(例如，鼠标或者轨迹球)，用户可以通过该键盘和该指向装置来将输入提供给计算机。其它种类的装置还可以用于提供与用户的交互；例如，提供给用户的反馈可以是任何形式的传感反馈(例如，视觉反馈、听觉反馈、或者触觉反馈)；并且可以用任何形式(包括声输入、语音输入或者、触觉输入)来接收来自用户的输入。
可以将此处描述的系统和技术实施在包括后台部件的计算系统(例如，作为数据服务器)、或者包括中间件部件的计算系统(例如，应用服务器)、或者包括前端部件的计算系统(例如，具有图形用户界面或者网络浏览器的用户计算机，用户可以通过该图形用户界面或者该网络浏览器来与此处描述的系统和技术的实施方式交互)、或者包括这种后台部件、中间件部件、或者前端部件的任何组合的计算系统中。可以通过任何形式或者介质的数字数据通信(例如，通信网络)来将系统的部件相互连接。通信网络的示例包括：局域网(LAN)、广域网(WAN)和互联网。
计算机系统可以包括客户端和服务器。客户端和服务器一般远离彼此并且通常通过通信网络进行交互。通过在相应的计算机上运行并且彼此具有客户端-服务器关系的计算机程序来产生客户端和服务器的关系。
根据本申请实施例的技术方案，通过主节点向各子节点发送共享编码器训练指令，以使每个子节点根据训练指令中包括的目标共享编码器的类型获取训练样本，并在获取到各子节点返回的确认消息后，向各子节点发送待训练的目标共享编码器的初始参数集，以使各子节点利用各自的训练样本对目标共享编码器的初始参数集进行训练，之后获取各子节点返回的目标共享编码器的更新参数集，进而根据第一预设规则及各子节点返回的目标共享编码器的更新参数集，确定目标共享编码器对应的目标参数集。 由此，通过主节点组织各子节点利用私有训练样本对目标共享编码器进行训练，以根据各子节点对目标共享编码器的训练结果，确定目标共享编码器的目标参数集，从而不仅实现了多领域训练语料的共享，降低了多领域训练语料的获取难度和成本，而且提高了共享编码器的性能。
应该理解，可以使用上面所示的各种形式的流程，重新排序、增加或删除步骤。例如，本发申请中记载的各步骤可以并行地执行也可以顺序地执行也可以不同的次序执行，只要能够实现本申请公开的技术方案所期望的结果，本文在此不进行限制。
上述具体实施方式，并不构成对本申请保护范围的限制。本领域技术人员应该明白的是，根据设计要求和其他因素，可以进行各种修改、组合、子组合和替代。任何在本申请的精神和原则之内所作的修改、等同替换和改进等，均应包含在本申请保护范围之内。

Claims (22)

  1. 一种共享编码器生成方法，其特征在于，包括：
    主节点向各子节点发送共享编码器训练指令，所述训练指令中包括待训练的目标共享编码器类型，以使每个子节点根据目标共享编码器的类型获取训练样本；
    在获取到各子节点返回的确认消息后，向各子节点发送待训练的目标共享编码器的初始参数集，以使各子节点利用各自的训练样本对所述目标共享编码器的初始参数集进行训练；
    获取各子节点返回的目标共享编码器的更新参数集；
    根据第一预设规则及各子节点返回的目标共享编码器的更新参数集，确定所述目标共享编码器对应的目标参数集。
  2. 如权利要求1所述的方法，其特征在于，所述主节点向各子节点发送共享编码器训练指令之前，还包括：
    所述主节点获取到至少一个子节点发送的获取目标共享编码器的请求；
    或者，
    所述主节点获取到至少一个子节点发送的目标共享编码器更新请求，所述更新请求中包括所述目标共享编码器的标识。
  3. 如权利要求1或2所述的方法，其特征在于，所述向各子节点发送待训练的目标共享编码器的初始参数集之后，还包括：
    获取各子节点包含的训练样本的数量；
    根据每个子节点包含的训练样本的数量，确定每个子节点的权重；
    所述根据第一预设规则及各子节点返回的目标共享编码器的更新参数集，确定所述目标共享编码器对应的目标参数集，包括：
    根据所述第一预设规则、所述每个子节点的权重及各子节点返回的目标共享编码器的更新参数集，确定所述目标共享编码器对应的目标参数集。
  4. 如权利要求1-3任一所述的方法，其特征在于，所述确定所述目标共享编码器对应的目标参数集之后，还包括：
    将所述目标共享编码器对应的目标参数集分别发送给每个子节点，以使每个子节点基于所述目标共享编码器对应的目标参数集，对所述目标共享编码器继续进行训练。
  5. 如权利要求4所述的方法，其特征在于，所述将所述目标共享编码器对应的目标参数集分别发送给每个子节点之后，还包括：
    获取第一子节点再次返回的目标共享编码器参数集；
    根据已获取的所述第一子节点返回目标共享编码器参数集的次数，确定所述第一子节点当前的权重值；
    依据所述第一预设规则，根据所述第一子节点当前的权重值及当前再次返回的目标共享编码器参数集，对所述目标共享编码器对应的目标参数集进行更新。
  6. 如权利要求1-5任一所述的方法，其特征在于，所述向各子节点发送待训练的目标共享编码器的初始参数集之前，还包括：
    依据第二预设规则，生成所述待训练的目标共享编码器。
  7. 一种共享编码器生成方法，其特征在于，包括：
    第二子节点获取主节点发送的共享编码器训练指令，所述训练指令中包括待训练的目标共享编码器类型；
    确定包含与所述目标共享编码器类型匹配的训练样本后，向所述主节点返回确认消息；
    获取所述主节点发送的目标共享编码器的初始参数集；
    利用与所述目标共享编码器类型匹配的训练样本，进行模型训练，确定所述目标共享编码器的更新参数集；
    向所述主节点返回所述目标共享编码器的更新参数集。
  8. 如权利要求7所述的方法，其特征在于，所述向所述主节点返回确认消息之后，还包括：
    获取所述主节点返回的目标共享编码器参数更新规则；
    所述向所述主节点返回所述目标共享编码器的更新参数集，包括：
    在确定当前满足所述更新规则时，向所述主节点返回所述目标共享编码器的更新参数集。
  9. 如权利要求7或8所述的方法，其特征在于，还包括：
    向所述主节点发送与所述目标共享编码器类型匹配的训练样本的数量。
  10. 如权利要求7-9任一所述的方法，其特征在于，所述利用与所述目标共享编码器类型匹配的训练样本，进行模型训练之后，还包括：
    确定与所述目标共享编码器对应的私有编码器及私有解码器。
  11. 一种共享编码器生成装置，其特征在于，包括：
    第一发送模块，用于主节点向各子节点发送共享编码器训练指令，所述训练指令中包括待训练的目标共享编码器类型，以使每个子节点根据目标共享编码器的类型获取训练样本；
    第二发送模块，用于在获取到各子节点返回的确认消息后，向各子节点发送待训 练的目标共享编码器的初始参数集，以使各子节点利用各自的训练样本对所述目标共享编码器的初始参数集进行训练；
    第一获取模块，用于获取各子节点返回的目标共享编码器的更新参数集；
    第一确定模块，用于根据第一预设规则及各子节点返回的目标共享编码器的更新参数集，确定所述目标共享编码器对应的目标参数集。
  12. 如权利要求11所述的装置，其特征在于，还包括：
    第二获取模块，用于所述主节点获取到至少一个子节点发送的获取目标共享编码器的请求；
    或者，
    第三获取模块，用于所述主节点获取到至少一个子节点发送的目标共享编码器更新请求，所述更新请求中包括所述目标共享编码器的标识。
  13. 如权利要求11或12所述的装置，其特征在于，还包括：
    第四获取模块，用于获取各子节点包含的训练样本的数量；
    第二确定模块，用于根据每个子节点包含的训练样本的数量，确定每个子节点的权重；
    所述第一确定模块，具体用于：
    根据所述第一预设规则、所述每个子节点的权重及各子节点返回的目标共享编码器的更新参数集，确定所述目标共享编码器对应的目标参数集。
  14. 如权利要求11-13任一所述的装置，其特征在于，还包括：
    第三发送模块，用于将所述目标共享编码器对应的目标参数集分别发送给每个子节点，以使每个子节点基于所述目标共享编码器对应的目标参数集，对所述目标共享编码器继续进行训练。
  15. 如权利要求14所述的装置，其特征在于，还包括：
    第五获取模块，用于获取第一子节点再次返回的目标共享编码器参数集；
    第三确定模块，用于根据已获取的所述第一子节点返回目标共享编码器参数集的次数，确定所述第一子节点当前的权重值；
    更新模块，用于依据所述第一预设规则，根据所述第一子节点当前的权重值及当前再次返回的目标共享编码器参数集，对所述目标共享编码器对应的目标参数集进行更新。
  16. 如权利要求11-15任一所述的装置，其特征在于，还包括：
    生成模块，用于依据第二预设规则，生成所述待训练的目标共享编码器。
  17. 一种共享编码器生成装置，其特征在于，包括：
    第一获取模块，用于第二子节点获取主节点发送的共享编码器训练指令，所述训练指令中包括待训练的目标共享编码器类型；
    第一返回模块，用于确定包含与所述目标共享编码器类型匹配的训练样本后，向所述主节点返回确认消息；
    第二获取模块，用于获取所述主节点发送的目标共享编码器的初始参数集；
    第一确定模块，用于利用与所述目标共享编码器类型匹配的训练样本，进行模型训练，确定所述目标共享编码器的更新参数集；
    第二返回模块，用于向所述主节点返回所述目标共享编码器的更新参数集。
  18. 如权利要求17所述的装置，其特征在于，还包括：
    第三获取模块，用于获取所述主节点返回的目标共享编码器参数更新规则；
    所述第二返回模块，具体用于：
    在确定当前满足所述更新规则时，向所述主节点返回所述目标共享编码器的更新参数集。
  19. 如权利要求17或18所述的装置，其特征在于，还包括：
    发送模块，用于向所述主节点发送与所述目标共享编码器类型匹配的训练样本的数量。
  20. 如权利要求17-19任一所述的装置，其特征在于，还包括：
    第二确定模块，用于确定与所述目标共享编码器对应的私有编码器及私有解码器。
  21. 一种电子设备，其特征在于，包括：
    至少一个处理器；以及
    与所述至少一个处理器通信连接的存储器；其中，
    所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行权利要求1-6或7-10中任一项所述的方法。
  22. 一种存储有计算机指令的非瞬时计算机可读存储介质，其特征在于，所述计算机指令用于使所述计算机执行权利要求1-6或7-10中任一项所述的方法。
PCT/CN2020/083551 2019-11-22 2020-04-07 共享编码器生成方法、装置及电子设备 Ceased WO2021098121A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
JP2021516453A JP7159460B2 (ja) 2019-11-22 2020-04-07 共有エンコーダ生成方法、装置及び電子機器
EP20864267.8A EP3855368B1 (en) 2019-11-22 2020-04-07 Shared encoder generation method and apparatus, and electronic device
KR1020207035376A KR102532368B1 (ko) 2019-11-22 2020-04-07 공유 엔코더 생성 방법, 장치 및 전자 기기
US17/209,576 US12229667B2 (en) 2019-11-22 2021-03-23 Method and apparatus for generating shared encoder

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911155509.2A CN110909875B (zh) 2019-11-22 2019-11-22 共享编码器生成方法、装置及电子设备
CN201911155509.2 2019-11-22

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/209,576 Continuation US12229667B2 (en) 2019-11-22 2021-03-23 Method and apparatus for generating shared encoder

Publications (1)

Publication Number Publication Date
WO2021098121A1 true WO2021098121A1 (zh) 2021-05-27

Family

ID=69818726

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/083551 Ceased WO2021098121A1 (zh) 2019-11-22 2020-04-07 共享编码器生成方法、装置及电子设备

Country Status (6)

Country Link
US (1) US12229667B2 (zh)
EP (1) EP3855368B1 (zh)
JP (1) JP7159460B2 (zh)
KR (1) KR102532368B1 (zh)
CN (2) CN115630695A (zh)
WO (1) WO2021098121A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114490647A (zh) * 2022-01-14 2022-05-13 深圳市汇川技术股份有限公司 参数值的获取方法、控制器、终端设备及可读存储介质
CN114612015A (zh) * 2022-05-09 2022-06-10 杰创智能科技股份有限公司 共享设备的控制方法、装置、电子设备及存储介质
CN120046116A (zh) * 2025-04-23 2025-05-27 杭州邻商网络科技股份有限公司 一种智能参数共享方法及系统

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115630695A (zh) * 2019-11-22 2023-01-20 北京百度网讯科技有限公司 共享编码器生成方法、装置及电子设备
JP6967099B2 (ja) * 2020-03-04 2021-11-17 株式会社日立製作所 学習モデル作成システム及び学習モデル作成方法
CN111477336A (zh) * 2020-04-07 2020-07-31 中南大学 传染病诊断数据的融合方法、系统及存储介质
US11802894B2 (en) * 2020-09-17 2023-10-31 Silicon Laboratories Inc. Compressing information in an end node using an autoencoder neural network
CN113377797B (zh) * 2021-07-02 2023-03-28 支付宝(杭州)信息技术有限公司 联合更新模型的方法、装置及系统
WO2024031420A1 (en) * 2022-08-10 2024-02-15 Qualcomm Incorporated Remote offline sequential network node encoder training
CN116313196A (zh) * 2023-01-12 2023-06-23 华能海南昌江核电有限公司 设备状态的监测方法以及装置、存储介质、电子设备
CN116843963B (zh) * 2023-06-28 2026-05-12 北京百度网讯科技有限公司 图像识别方法、装置、设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9514216B2 (en) * 2009-08-10 2016-12-06 Yahoo! Inc. Automatic classification of segmented portions of web pages
CN107622310A (zh) * 2017-08-30 2018-01-23 第四范式（北京）技术有限公司 用于执行机器学习的分布式系统及其方法
CN109472345A (zh) * 2018-09-28 2019-03-15 深圳百诺名医汇网络技术有限公司 一种权重更新方法、装置、计算机设备和存储介质
CN109670594A (zh) * 2018-12-28 2019-04-23 北京旷视科技有限公司 数据训练方法、装置及电子设备
CN110909875A (zh) * 2019-11-22 2020-03-24 北京百度网讯科技有限公司 共享编码器生成方法、装置及电子设备

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8010469B2 (en) * 2000-09-25 2011-08-30 Crossbeam Systems, Inc. Systems and methods for processing data flows
US11244242B2 (en) * 2018-09-07 2022-02-08 Intel Corporation Technologies for distributing gradient descent computation in a heterogeneous multi-access edge computing (MEC) networks
CN110442457A (zh) * 2019-08-12 2019-11-12 北京大学深圳研究生院 基于联邦学习的模型训练方法、装置及服务器

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9514216B2 (en) * 2009-08-10 2016-12-06 Yahoo! Inc. Automatic classification of segmented portions of web pages
CN107622310A (zh) * 2017-08-30 2018-01-23 第四范式（北京）技术有限公司 用于执行机器学习的分布式系统及其方法
CN109472345A (zh) * 2018-09-28 2019-03-15 深圳百诺名医汇网络技术有限公司 一种权重更新方法、装置、计算机设备和存储介质
CN109670594A (zh) * 2018-12-28 2019-04-23 北京旷视科技有限公司 数据训练方法、装置及电子设备
CN110909875A (zh) * 2019-11-22 2020-03-24 北京百度网讯科技有限公司 共享编码器生成方法、装置及电子设备

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3855368A4 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114490647A (zh) * 2022-01-14 2022-05-13 深圳市汇川技术股份有限公司 参数值的获取方法、控制器、终端设备及可读存储介质
CN114612015A (zh) * 2022-05-09 2022-06-10 杰创智能科技股份有限公司 共享设备的控制方法、装置、电子设备及存储介质
CN120046116A (zh) * 2025-04-23 2025-05-27 杭州邻商网络科技股份有限公司 一种智能参数共享方法及系统

Also Published As

Publication number Publication date
US12229667B2 (en) 2025-02-18
CN110909875B (zh) 2022-12-23
JP2022512053A (ja) 2022-02-02
EP3855368A4 (en) 2022-07-13
EP3855368A1 (en) 2021-07-28
KR20210065069A (ko) 2021-06-03
US20210209417A1 (en) 2021-07-08
EP3855368B1 (en) 2025-12-03
CN110909875A (zh) 2020-03-24
KR102532368B1 (ko) 2023-05-15
JP7159460B2 (ja) 2022-10-24
CN115630695A (zh) 2023-01-20

Similar Documents

Publication Publication Date Title
CN110909875B (zh) 共享编码器生成方法、装置及电子设备
KR102484617B1 (ko) 이종 그래프 노드를 표현하는 모델 생성 방법, 장치, 전자 기기, 저장 매체 및 프로그램
US20230022550A1 (en) Image processing method, method for training image processing model devices and storage medium
KR102566277B1 (ko) 이미지 편집 모델 구축 방법 및 장치
US20210210091A1 (en) Method, device, and storage medium for waking up via speech
US20230251898A1 (en) Federated computing method, electronic device and storage medium
CN112001180A (zh) 多模态预训练模型获取方法、装置、电子设备及存储介质
CN111695699B (zh) 用于模型蒸馏的方法、装置、电子设备及可读存储介质
US20210108931A1 (en) Method and apparatus for determining hybrid travel route, device and storage medium
WO2021068468A1 (zh) 知识图谱的向量表示生成方法、装置及设备
CN110543558B (zh) 问题匹配方法、装置、设备和介质
CN111241838B (zh) 文本实体的语义关系处理方法、装置及设备
CN112016632A (zh) 模型联合训练方法、装置、设备和存储介质
CN113762328A (zh) 基于联邦学习的模型训练方法、装置、设备以及存储介质
CN111695698A (zh) 用于模型蒸馏的方法、装置、电子设备及可读存储介质
KR102583112B1 (ko) 이미지를 처리하는 방법, 장치, 전자 기기, 저장 매체 및 프로그램
CN112184851B (zh) 图像编辑方法、网络训练方法、相关装置及电子设备
CN111667428B (zh) 基于自动搜索的噪声生成方法和装置
CN111818046B (zh) 用于交互信息的方法、装置、设备以及存储介质
CN115344753B (zh) 联邦图计算方法、联邦图计算配置方法、装置及电子设备
CN112687271B (zh) 语音翻译方法、装置、电子设备和存储介质
CN111680599B (zh) 人脸识别模型处理方法、装置、设备和存储介质
KR102783761B1 (ko) 도로 정보 결정 방법 및 장치, 전자 기기, 컴퓨터 판독 가능 저장 매체 및 컴퓨터 프로그램
CN111177479A (zh) 获取关系网络图中节点的特征向量的方法以及装置
CN112508830B (zh) 图像处理模型的训练方法、装置、设备及存储介质

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2021516453

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2020864267

Country of ref document: EP

Effective date: 20210420

NENP Non-entry into the national phase

Ref country code: DE

WWG Wipo information: grant in national office

Ref document number: 2020864267

Country of ref document: EP