CN115577112B - Event extraction method and system based on type perception gated attention mechanism - Google Patents
Event extraction method and system based on type perception gated attention mechanism Download PDFInfo
- Publication number
- CN115577112B CN115577112B CN202211576463.3A CN202211576463A CN115577112B CN 115577112 B CN115577112 B CN 115577112B CN 202211576463 A CN202211576463 A CN 202211576463A CN 115577112 B CN115577112 B CN 115577112B
- Authority
- CN
- China
- Prior art keywords
- event
- trigger word
- word
- text
- argument
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
- G06F16/353—Clustering; Classification into predefined classes
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3346—Query execution using probabilistic model
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Databases & Information Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Probability & Statistics with Applications (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Machine Translation (AREA)
Abstract
本发明涉及信息抽取技术领域,公开了一种基于类型感知门控注意力机制的事件抽取方法及系统,该事件抽取方法,利用门控信息指导,使不同事件类别下有不同信息流向触发词,过滤与触发词无关的噪声。本发明解决了现有技术存在的事件论元抽取准确度低、角色分类效果差等问题。
The invention relates to the technical field of information extraction, and discloses an event extraction method and system based on a type-aware gating attention mechanism. The event extraction method utilizes gating information guidance to make different information flow to trigger words under different event categories. Filter out noise not related to trigger words. The invention solves the problems of low event argument extraction accuracy, poor role classification effect and the like existing in the prior art.
Description
Technical Field
The invention relates to the technical field of information extraction, in particular to an event extraction method and system based on a type-aware gated attention mechanism.
Background
Event extraction is a fundamental and extremely challenging task in the field of information extraction. Event extraction generally includes two tasks, namely event detection and event argument extraction. More specifically, the event detection task comprises two subtasks of trigger word detection and event classification, and the event argument extraction comprises two subtasks of argument detection and role classification. In recent years, with the development of deep learning, event extraction methods based on deep learning have been improved to some extent, but the difficulty of event extraction has not yet been completely solved.
At present, most event extraction methods focus on solving the overlapping argument scene, but neglect the overlapping trigger word scene and trigger word ambiguity problem. In other words, rather than arguments that may play different roles in different/the same event, the trigger may have multiple event types.
Furthermore, event argument extraction is more difficult than event detection. Many methods attempt to improve the event argument extraction effect by using role information, such as role occurrence frequency (importance), role correlation such as hierarchical conceptual relationship, role grammatical relationship, and the like. The appearance frequency of the roles ignores the mutual relation among the roles, the correlation among other roles needs to be summarized and concluded based on human experience, and the correlation is not applicable to some data, so the extraction effect of event arguments is not greatly improved.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides an event extraction method and system based on a type-aware gated attention mechanism, and solves the problems of low accuracy of event argument extraction, poor role classification effect and the like in the prior art.
The technical scheme adopted by the invention for solving the problems is as follows:
an event extraction method based on a type perception gating attention mechanism utilizes gating information guidance to enable different information under different event types to flow to trigger words and filters noise irrelevant to the trigger words.
As a preferred technical scheme, the method comprises the following steps:
s1, text vectorization: mixing the sampleInputting the result into a text vectorization layer based on a language model to obtain a text vectorization result ≥>(ii) a Wherein it is present>Indicates the th in the text>Word and/or word>Represents->Corresponding vectorized result, based on the result of the evaluation>Representing the vectorized result of the text X, R representing a real number, d representing the dimensions of a vector, R d Representing a d-dimensional real number vector;
s2, event detection: vectorizing text to a resultInputting an event detection module of a gate control attention mechanism with fusion type perception to complete two subtasks of trigger word detection and event classification;
s3, argument extraction: and (3) for each trigger word under each event type in the results after the trigger word detection and the event classification are finished in the step (S2), completing two subtasks of argument extraction and argument role classification by using an argument extraction module which integrates learnable role interaction parameters.
As a preferred technical solution, the event detection module of the fusion type perception gated attention mechanism in step S2 includes the following sub-modules connected in series: a trigger word extraction layer and a gated attention event classification layer.
As a preferred technical solution, the construction process of the trigger word extraction layer includes the following steps:
s21, firstly, calculating and obtaining the probability that each character in the input text is a starting/ending character of a trigger word according to the following formula:
wherein,sigmoid is an activation function for a learnable network parameter, SIGMOID>For the th in the input text>The probability that a word is the starting character of a trigger word, <' >>Is the fifth or fifth in the text>Probability that a word is the ending character of a trigger word;
Wherein,a set of start character positions representing a trigger word, <' > or>A set of ending character positions representing trigger words;
s23, combining the result of the step S22, and obtaining a trigger word set by using a nearest matching principle;
Wherein t is a candidate trigger word, s is the position of the starting character of the candidate trigger word t in the text X,is a setIs closest to->The element (b);
the construction process of the gated attention event classification layer comprises the following steps:
s24, in the gating information filtering layer, for each event categoryDefining event class semantic vector>The corresponding gating vector is calculated as follows:
wherein,is an event category->Lower gated vector, < >>Learnable weight parameters for a gating unit>A learnable bias parameter for the gate control unit;
and S25, combining the result in the S24, and filtering the context information by using an element product function under each event category:
wherein,for the th in the input text>Word corresponding vector->For first/second in text input after passing through a gating information filter layer>Word in event category pick>The corresponding vector after information filtering;
s26, in the attention information fusion layer, the event category is obtained by using the attention calculation functionThe lower input text is ^ h or greater>Word-to-trigger word->Is greater than or equal to>;
And S27, combining the obtained importance scores calculated in S26, obtaining a final information aggregation result related to each trigger word under each event category by using the following formula:
wherein,for post-attention information fusion layer event categories>The information related to the trigger word t is added to form an aggregation vector;
s28, in the event classification layer, in combination with the information aggregation vector related to the trigger word t obtained in step S27, determining the event type to which the trigger word t belongs by using the following formula:
wherein,learnable weight parameter for an event class decision unit>Bias parameter for event class determination unit, w T Represents a transposition of w;
according to a preset threshold valueAll event classes which satisfy the following condition>All will be determined as event categories to which the trigger word t belongs:
As a preferred technical solution, in step S26, the attention calculation function formula is as follows:
wherein,a characterization vector representing the beginning character of the trigger word t, be->A token vector representing an ending character of the trigger word t;
wherein,denotes the tan h activation function, V T Means for transposing a V>Represents a weight, [; (ii) a]Representing the concatenation of the vectors. />
As a preferred technical solution, the construction process of the argument extraction module fused with learnable role interaction parameters in step S3 includes the following steps:
s31, calculating a characteristic vector of the trigger words blended into the context by using the following formula:
Wherein,is based on the input text being in the event category>Lower information filtered corresponding vector->The calculated mean value is then taken>Is based on the input text being in the event category>The corresponding vector which is filtered by the information is selected>The standard deviation of the result of the calculation, device for selecting or keeping>Indicates the ^ th or greater in the input text>Word in event category pick>The vector after information filtering and fusion of the trigger word t information is combined>、Represents an extension parameter and a translation parameter, respectively>、Respectively represent a value for calculating->Is selected, the weight parameter, the bias parameter, -of the linear layer of (4)>、Respectively indicate for calculation &>The weight parameter and the bias parameter of the linear layer;
s32, respectively calculating each character in the input text as each role event category by using the following formulaProbability size of start character/end character of argument:
wherein,indicates that the entered text is up to->Word is in event category->Is triggered by the trigger word t->Probability value of the start character of the argument below, <>Indicates that the entered text is up to->Word is in event category->Is triggered by the trigger word t->Probability value of the end character of the argument below, <>、Represents a weight parameter, <' > is selected>、Representing a bias parameter;
s33, defining a character interaction matrix capable of learningAnd designing the following decision function:
wherein,indicates an event category pick>An underlying indicator function->、A weight parameter and a bias parameter representing a linear layer of the first layer, -a>Representing a bias parameter of the second layer linear layer;
will be provided withAs the weight, in combination with the calculation result in the weight correction step S32:
after training, the judgment function can learn the mutual relation between the event roles and the mutual relation between the roles at the same time;
wherein,for entering text ^ h>Word is event category->Is triggered by the trigger word t->The final probability value of the beginning character of the argument below, based on the current location of the argument below>First ^ for inputting text>Word is event category->Is triggered by the trigger word t->The final probability value of the end character of the argument below, based on the value of the argument below>Is an event category->Down character>The weight of (c);
s34, according to the preset threshold value、Filtering the results in S33Thereby obtaining a set of positions、:
Wherein,indicates that a character is pickand place>The set of start character positions of the lower argument, <' >>Indicates that a character is pickand place>The set of ending character positions of the lower argument;
s35, combining the result of the step S34, and obtaining the role by using the nearest matching principleArgument set of(ii) a Wherein it is present>Is argument>The position of the start character of (b) in the text X, device for combining or screening>As a set +>Is closest to->Of (2) is used.
As a preferred solution, the loss function of the event extraction method is as follows:
wherein,representing input text>Event type of the middle trigger word t, < >>Representing input textThe trigger word t with medium event type c has a role of ≥>Argument of (D), D denotes all input samples, based on the value of the argument, and->Indicates that in sample x the trigger word is t and the event type is c and argument of character r is +>In the case of>Represents the probability that the trigger word in sample x is t, <' >>Represents a trigger word in a sample x->Has a probability of c, is greater than>Indicating in sample x that the event category is c and the trigger word is &>Argument of time character r is ^ 5>Is greater than or equal to>Representing all events in sample x.
As a preferred technical solution, in step S1, the language model is a BERT model.
A text vectorization module: for mixing the sampleInputting the result into a text vectorization layer based on a language model to obtain a text vectorization result ≥>(ii) a Wherein,indicates the th in the text>Word and/or word>Represents->Corresponding vectorized result, is>Representing vectorization results of a text X, R representing a real number, d representing a dimension of a vector, R d Representing a d-dimensional real number vector; />
An event detection module: to vectorize the text into a resultInputting an event detection module of a gate control attention mechanism with fusion type perception to complete two subtasks of trigger word detection and event classification;
argument extraction module: and the argument extraction module which integrates learnable role interaction parameters is used for completing two subtasks of argument extraction and argument role classification for each trigger word under each event type in the results after the event detection module completes trigger word detection and event classification.
Compared with the prior art, the invention has the following beneficial effects:
according to the invention, different information flows to the trigger words under different event types are guided by using the gating information, noise irrelevant to the trigger words is effectively filtered, and other relevant information is effectively integrated, so that the ambiguity problem of the trigger words under the event extraction scene containing the overlapped trigger words is better eliminated, the overlapped trigger word scene is dealt with, and the event classification effect is improved; meanwhile, the method considers the role co-occurrence relation ignored by people, and further improves the effect of argument extraction and role classification tasks by introducing learnable role interaction parameters to model the role co-occurrence relation.
Drawings
FIG. 1 is a schematic diagram illustrating steps of an event extraction method based on a type-aware gated attention mechanism according to the present invention;
fig. 2 is a schematic diagram of a model structure of an event detection module of a fusion type perception gated attention mechanism according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to examples and drawings, but the embodiments of the present invention are not limited thereto.
The invention provides an event extraction method based on a type perception gated attention mechanism, which designs the type perception gated attention mechanism, and is different from a pure attention mechanism, the method utilizes gated information guidance to enable different information to flow to trigger words under different event types, effectively filters noise irrelevant to the trigger words and effectively integrates other related information, thereby better eliminating the ambiguity problem of the trigger words under an event extraction scene containing overlapped trigger words, coping with the overlapped trigger word scene and improving the event classification effect; meanwhile, the method considers the role co-occurrence relation ignored by everybody, and further improves the effect of argument extraction and role classification tasks by introducing learnable role interaction parameters to model the role co-occurrence relation.
Example 1
As shown in fig. 1, an event extraction method based on a type-aware gated attention mechanism includes the steps of:
s1, mixing the sampleInputting the text vector into a text vectorization layer based on a language model such as BERT to obtain a text vectorization result->. WhereinIndicates the th in the text>Word and/or word>The corresponding vectorization result is obtained;
s2, vectorizing the text into a resultInputting an event detection module of a gate control attention mechanism with fusion type perception to complete two subtasks of trigger word detection and event classification;
s3, for each trigger word belonging to each event type in the result of S2, completing two subtasks of argument extraction and argument role classification by using an argument extraction module which integrates learnable role interaction parameters;
example 2
On the basis of embodiment 1, as shown in fig. 2, the event detection module of the fusion type perception gated attention mechanism in step S2 includes the following sub-modules in a serial order: a trigger word extraction layer and a gated attention event classification layer.
The construction process of the trigger word extraction layer comprises the following steps:
s21, firstly, calculating and obtaining the probability that each character in the input text is a starting/ending character of the trigger word according to the following formula:
wherein,sigmoid is an activation function for a learnable network parameter.Corresponding to the ^ th or greater part of the input text>Probability that a word is the starting character of a trigger word, based on the word's word score>Then corresponds to the ^ th in the text>Probability that a word is the ending character of a trigger word;
Here, the location sets、Respectively representing a starting character position set and an ending character position set of the trigger word;
s23, combining the result of the step S22, and obtaining a trigger word set by using a nearest matching principleHere +>For the position of the start character of the candidate trigger word t in the text X, ->As a set +>Is closest to->Of (2) is used.
The construction process of the gated attention event classification layer comprises the following steps:
s24, in the gating information filtering layer, for each event categoryDefining event class semantic vector>The corresponding gating vector is calculated as follows:
here, theFor an event category>Lower gated vector, < >>Is a learnable weight parameter of the gate control unit>A learnable bias parameter for the gate control unit;
and S25, combining the result in the S24, and filtering the context information by using an element-wise product (element-wise product) under each event category:
here, ,for the first ^ in the input text>Word corresponding vector->Is the first or second based on the input text after passing through the gating information filtering layer>Word in event category>The corresponding vector after information filtering;
s26, in the notesIn the intention information fusion layer, after the calculation described in step S25, the event type is obtained by using the attention calculation function designed as followsOn/in text>Word-to-trigger word->Is greater than or equal to>:
and S27, combining the obtained importance scores calculated in S26, obtaining a final information aggregation result related to each trigger word under each event category by using the following formula:
here, ,for an event category after the attention information fusion layer>Down and trigger word->A related information aggregation vector;
and S28, in the event classification layer, combining the information aggregation vector related to the trigger word t obtained in the step S27, and determining which event types the trigger word t belongs to by using the following formula:
here, ,learnable weight parameter for an event class decision unit>Bias parameter for event class determination unit, w T Represents a transposition of w; in particular according to a predetermined threshold value>All event classes that satisfy the following condition>All will be determined as event categories to which the trigger word t belongs:
Example 3
On the basis of the embodiments 1 and 2, the construction process of the argument extraction module fusing the learnable role interaction parameters in the step S3 includes the following steps:
s31, in order to better extract the event type asAll arguments associated with a given trigger word t are first blended into the trigger word characterization vector @incontext using the following formula>:
Here, ,、is based on the input text being in the event category>The corresponding vector which is filtered by the information is selected>Calculating the obtained mean value and standard deviation;
s32, respectively calculating each character in the input text as each role event category by using the following formulaProbability size of start character/end character of argument:
here, the number of the first and second electrodes,、respectively represent the input text ^ h>Word is event category->Is triggered by the trigger word t->Probability value of starting character/ending character of argument under;
s33, defining a character interaction matrix capable of learningAnd designing the following decision function:
using the calculation result as a weight, and combining the calculation result in the weight correction step S32:
after training, the judgment function can learn the mutual relation between the event roles and the mutual relation between the roles at the same time.、For entering text ^ h>Word is event category->Is triggered by the trigger word t->Final probability values for the beginning/ending characters of the argument below. Here->Is an event category->Lower role->The weight of (c).
S34, according to the preset threshold value、Filtering the result in S33 to obtain a set of locations、:
Here, the position sets、Respectively represent a role->A starting character position set and an ending character position set of a lower argument;
s35, combining the result of the step S34, and obtaining the role by using the nearest matching principleArgument set ofHere +>Is argument>The position of the start character of (b) in the text X, device for selecting or keeping>Is set->Is closest to->Of (2) is used.
Example 4
On the basis of examples 1, 2 and 3, the loss function of the event extraction method is as follows:
here, ,indicates that the input text pick>Event type in which the word t is triggered>Indicates that the input text pick>Medium event type is->Has the role of ≥>The argument of (1).
As described above, the present invention can be preferably implemented.
All features disclosed in all embodiments in this specification, or all methods or process steps implicitly disclosed, may be combined and/or expanded, or substituted, in any way, except for mutually exclusive features and/or steps.
While the invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not to be limited to the disclosed embodiment, but on the contrary, is intended to cover various modifications, equivalent arrangements, and alternatives falling within the spirit and scope of the invention as defined by the appended claims.
Claims (6)
1. An event extraction method based on a type perception gating attention mechanism is characterized in that different information flows to a trigger word under different event types by using gating information guidance, and noise irrelevant to the trigger word is filtered;
the method comprises the following steps:
s1, text vectorization: mixing the sampleInputting the result into a text vectorization layer based on a language model to obtain a text vectorization result ≥>(ii) a Wherein +>Indicates the th in the text>Word in and out apparatus>Represents->Corresponding vectorized result, is>Representing the vectorized result of the text X, R representing a real number, d representing the dimensions of a vector, R d Representing a d-dimensional real number vector;
s2, event detection: vectorizing text into resultsInputting an event detection module of a gate control attention mechanism with fusion type perception to complete two subtasks of trigger word detection and event classification;
s3, argument extraction: for each trigger word under each event type in the results after the trigger word detection and the event classification are completed in the step S2, an argument extraction module which integrates learnable role interaction parameters is utilized to complete two subtasks of argument extraction and argument role classification;
the event detection module of the fusion type perception gated attention mechanism in the step S2 comprises the following serially connected sub-modules: a trigger word extraction layer and a gate control attention event classification layer;
the construction process of the trigger word extraction layer comprises the following steps:
s21, firstly, calculating and obtaining the probability that each character in the input text is a starting/ending character of the trigger word according to the following formula:
wherein,sigmoid is an activation function for learnable network parameters>For the first ^ in the input text>The probability that a word is the starting character of a trigger word, <' >>Is the fifth or fifth in the text>The probability that a word is the ending character of a trigger word;
s22, according to the preset threshold value、The result in S21 is filtered, so that a position set @isobtained>、:
Wherein,set of start character positions, representing a trigger word>A set of ending character positions representing trigger words;
s23, combining the result of the step S22, and obtaining a trigger word set by using a nearest matching principle;
Wherein t is a candidate trigger word, s is the position of the starting character of the candidate trigger word t in the text X,is set->Is closest to->An element of (1); />
The construction process of the gated attention event classification layer comprises the following steps:
s24, in the gating information filtering layer, for each event categoryDefining event class semantic vector>The corresponding gating vector is calculated as follows:
wherein,for an event category>Lower gated vector, < >>Is a learnable weight parameter of the gate control unit>A learnable bias parameter for the gate control unit;
and S25, combining the result in the S24, and filtering the context information by using an element product function under each event category:
wherein,for the th in the input text>Word corresponding vector->Is the first or second based on the input text after passing through the gating information filtering layer>Word in event category pick>The corresponding vector after information filtering;
s26, in the attention information fusion layer, the event type is obtained by using the attention calculation functionEnter text intoWord-to-trigger word->Importance score of;
And S27, combining the obtained importance scores calculated in S26, obtaining a final information aggregation result related to each trigger word under each event category by using the following formula:
wherein,for post-attention information fusion layer event categories>Down and trigger word->A related information aggregation vector;
s28, in the event classification layer, in combination with the information aggregation vector related to the trigger word t obtained in step S27, determining the event type to which the trigger word t belongs by using the following formula:
wherein,learnable weight parameter for an event class decision unit>Bias parameter for event class determination unit, w T Represents a transposition of w;
according to a preset threshold valueAll ofEvent category £ that satisfies the following condition>All will be determined as event categories to which the trigger word t belongs:
2. The method for extracting events based on the type-aware gated attention mechanism as claimed in claim 1, wherein in step S26, the attention calculation function formula is as follows:
wherein,is a trigger word>The characterization vector of (2) is obtained by calculating the following formula:
wherein,a characterization vector representing the beginning character of the trigger word t, be->A token vector representing an ending character of the trigger word t;
3. The method for extracting events based on the type-aware gated attention mechanism according to claim 2, wherein the construction process of the argument extraction module fused with learnable role interaction parameters in step S3 includes the following steps:
s31, calculating a characteristic vector of the trigger words blended into the context by using the following formula:
Wherein,is based on the input text in the event category>The corresponding vector which is filtered by the information is selected>The calculated mean value is then taken>Is based on the input text being in the event category>Lower information filtered corresponding vector->The calculated standard deviation is->Indicates the ^ th or greater in the input text>Word in event category pick>The vector after information filtering and fusion of the trigger word t information is selected and matched>、Represents an extension parameter and a translation parameter, respectively>、Respectively represent a value for calculating->Weight parameter, bias parameter, based on the linear layer of (4), based on the number of linear layers in the device>、Respectively indicate that it is used for counting->The weight parameter and the bias parameter of the linear layer;
s32, respectively calculating each character in the input text as each role event category by using the following formulaProbability size of start/end character for argument: />
Wherein,indicates that the entered text is up to->Word is event category->Is triggered by the trigger word t->Probability value of the start character of the argument below, <>Indicating that input text is on &>Word is in event category->Is triggered by the trigger word t->Probability value of the end character of the argument below, <>、Represents a weight parameter, <' > is selected>、Representing a bias parameter;
s33, defining a character interaction matrix capable of learningAnd designing the following decision function:
wherein,indicates an event category pick>An underlying indicator function->、A weight parameter and a bias parameter representing a linear layer of the first layer, -a>Representing a bias parameter of the second layer linear layer;
will be provided withAs the weight, in combination with the calculation result in the weight correction step S32:
after training, the judgment function can learn the mutual relation between the event roles and the mutual relation between the roles at the same time;
wherein,for entering text ^ h>Word is event category->Trigger word of t role->The final probability value of the beginning character of the argument below, based on the current location of the argument below>For entering text ^ h>Word is event category->Is triggered by the trigger word t->Of the ending character of the argument belowFinal probability value->Is an event category->Lower role->The weight of (c);
s34, according to the preset threshold value、The result in S33 is filtered, so that a position set @isobtained>、:
Wherein,indicates that a character is pickand place>The set of start character positions of the lower argument, <' >>Indicates that a character is pickand place>The ending character position set of the lower argument;
s35, combining the result of the step S34, and obtaining the role by using the nearest matching principleArgument set of(ii) a Wherein it is present>Is argument>The position of the starting character in the text X, device for combining or screening>As a set +>Is closest to->The element (c) of (a). />
4. The event extraction method based on the type-aware gated attention mechanism as claimed in claim 3, wherein the loss function of the event extraction method is as follows:
wherein,indicates that the input text pick>Middle touchEvent type for the asserted t->Indicates that the input text pick>The trigger word t with medium event type c has a role of ≥>Argument of (D), D denotes all input samples, based on the value of the argument, and->Indicates that the trigger is t and the event type is c in sample x and the argument of the role r is ^ H>Is greater than or equal to>Represents the probability that the trigger word in sample x is t, <' >>Represents a trigger word in a sample x->Has a probability of c, is greater than>Indicates that in sample x the event class is c and the trigger word is ≥>Argument of time character r is ^ 5>Is greater than or equal to>Representing what is in sample xThere is an event.
5. The method for extracting events based on the type-aware gated attention mechanism as claimed in any one of claims 1 to 4, wherein in step S1, the language model is a BERT model.
6. An event extraction system based on a type-aware gated attention mechanism, for implementing the event extraction method based on the type-aware gated attention mechanism as claimed in any one of claims 1 to 5, comprising the following modules connected in sequence:
a text vectorization module: for mixing the sampleInputting into a text vectorization layer based on a language model, obtaining a text vectorization result>(ii) a Wherein +>Indicates the th in the text>Word in and out apparatus>Represents->Corresponding vectorized result, is>Representing the vectorized result of the text X, R representing a real number, d representing the dimensions of a vector, R d Representing a d-dimensional real number vector;
an event detection module: to vectorize the text into a resultInputting an event detection module of a gate control attention mechanism with fusion type perception to complete two subtasks of trigger word detection and event classification;
argument extraction module: the argument extraction module which is integrated with learnable role interaction parameters is used for completing two subtasks of argument extraction and argument role classification for each trigger word under each event type in the results of the event detection module after the trigger word detection and the event classification are completed;
the event detection module of the fusion type perception gating attention mechanism comprises the following sub-modules which are connected in series: a trigger word extraction layer and a gated attention event classification layer;
the construction process of the trigger word extraction layer comprises the following steps:
s21, firstly, calculating and obtaining the probability that each character in the input text is a starting/ending character of a trigger word according to the following formula:
wherein,sigmoid is an activation function for a learnable network parameter, SIGMOID>For the th in the input text>Probability that a word is the starting character of a trigger word, based on the word's word score>For the fifth in the text>The probability that a word is the ending character of a trigger word;
s22, according to the preset threshold value、The result in S21 is filtered, so that a position set @isobtained>、:
Wherein,a set of start character positions representing a trigger word, <' > or>A set of ending character positions representing trigger words;
s23, combining the result of the step S22, and obtaining a trigger word set by using a nearest matching principle;
Wherein t is a candidate trigger word, s is the position of the starting character of the candidate trigger word t in the text X,is set->Is closest to->The element (b);
the construction process of the gated attention event classification layer comprises the following steps:
s24, in the gating information filtering layer, for each event categoryDefining an event class semantic vector ≥>The corresponding gating vector is calculated as follows:
wherein,is an event category->A lower gated vector +>Is a learnable weight parameter of the gate control unit>A learnable bias parameter for the gate control unit;
and S25, combining the result in the S24, and filtering the context information by using an element product function under each event category:
wherein,for the first ^ in the input text>A vector corresponding to a word, <' >>Is the first or second based on the input text after passing through the gating information filtering layer>Word in event category pick>The corresponding vector after information filtering;
s26, in the attention information fusion layer, the event category is obtained by using the attention calculation functionEnter text intoWord-to-trigger word->Is greater than or equal to>;
And S27, combining the obtained importance scores calculated in S26, obtaining a final information aggregation result related to each trigger word under each event category by using the following formula:
wherein,for an event category after the attention information fusion layer>The information related to the trigger word t is aggregated to form a vector;
s28, in the event classification layer, in combination with the information aggregation vector related to the trigger word t obtained in step S27, determining the event type to which the trigger word t belongs by using the following formula:
wherein,learnable weight parameter for an event class decision unit>Bias parameter for event class determination unit, w T Represents a transposition of w;
according to a preset threshold valueAll event classes that satisfy the following condition>All will be determined as event categories to which the trigger word t belongs:
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211576463.3A CN115577112B (en) | 2022-12-09 | 2022-12-09 | Event extraction method and system based on type perception gated attention mechanism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211576463.3A CN115577112B (en) | 2022-12-09 | 2022-12-09 | Event extraction method and system based on type perception gated attention mechanism |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115577112A CN115577112A (en) | 2023-01-06 |
| CN115577112B true CN115577112B (en) | 2023-04-18 |
Family
ID=84589980
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211576463.3A Active CN115577112B (en) | 2022-12-09 | 2022-12-09 | Event extraction method and system based on type perception gated attention mechanism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115577112B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112307761A (en) * | 2020-11-19 | 2021-02-02 | 新华智云科技有限公司 | Event extraction method and system based on attention mechanism |
| CN113705218A (en) * | 2021-09-03 | 2021-11-26 | 四川大学 | Event element gridding extraction method based on character embedding, storage medium and electronic device |
| CN115238685A (en) * | 2022-09-23 | 2022-10-25 | 华南理工大学 | Combined extraction method for building engineering change events based on position perception |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10565229B2 (en) * | 2018-05-24 | 2020-02-18 | People.ai, Inc. | Systems and methods for matching electronic activities directly to record objects of systems of record |
| US10186275B2 (en) * | 2017-03-31 | 2019-01-22 | Hong Fu Jin Precision Industry (Shenzhen) Co., Ltd. | Sharing method and device for video and audio data presented in interacting fashion |
| CN109299470B (en) * | 2018-11-01 | 2024-02-09 | 成都数联铭品科技有限公司 | Method and system for extracting trigger words in text bulletin |
| CN113239142B (en) * | 2021-04-26 | 2022-09-23 | 昆明理工大学 | Trigger-word-free event detection method fused with syntactic information |
| CN115292568B (en) * | 2022-03-02 | 2023-11-17 | 内蒙古工业大学 | Civil news event extraction method based on joint model |
| CN114298053B (en) * | 2022-03-10 | 2022-05-24 | 中国科学院自动化研究所 | A joint event extraction system based on the fusion of feature and attention mechanism |
| CN114648016B (en) * | 2022-03-29 | 2025-02-11 | 河海大学 | An event argument extraction method based on event element interaction and label semantic enhancement |
| CN115392248A (en) * | 2022-06-22 | 2022-11-25 | 北京航空航天大学 | A Method for Event Extraction Based on Context and Graph Attention |
| CN115168541B (en) * | 2022-07-13 | 2025-08-26 | 山西大学 | Text event extraction method and system based on frame semantic mapping and type awareness |
-
2022
- 2022-12-09 CN CN202211576463.3A patent/CN115577112B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112307761A (en) * | 2020-11-19 | 2021-02-02 | 新华智云科技有限公司 | Event extraction method and system based on attention mechanism |
| CN113705218A (en) * | 2021-09-03 | 2021-11-26 | 四川大学 | Event element gridding extraction method based on character embedding, storage medium and electronic device |
| CN115238685A (en) * | 2022-09-23 | 2022-10-25 | 华南理工大学 | Combined extraction method for building engineering change events based on position perception |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115577112A (en) | 2023-01-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110598005B (en) | Public safety event-oriented multi-source heterogeneous data knowledge graph construction method | |
| Li et al. | A model of coherence based on distributed sentence representation | |
| CN108170816A (en) | A kind of intelligent vision Question-Answering Model based on deep neural network | |
| CN108052547A (en) | Natural language question-answering method and system based on question sentence and knowledge graph structural analysis | |
| CN114265937A (en) | Intelligent classification analysis method and system of scientific and technological information, storage medium and server | |
| US20210312133A1 (en) | Word vector-based event-driven service matching method | |
| CN103207860A (en) | Method and device for extracting entity relationships of public sentiment events | |
| CN109710744A (en) | A kind of data matching method, device, equipment and storage medium | |
| Guo et al. | Divert more attention to vision-language object tracking | |
| CN115858723B (en) | A method and system for query graph generation in complex knowledge base question answering | |
| CN113158673A (en) | Single document analysis method and device | |
| CN108846000A (en) | A kind of common sense semanteme map construction method and device based on supernode and the common sense complementing method based on connection prediction | |
| CN118821779B (en) | A method and device for extracting relations based on entity pairs and shortest dependency paths | |
| CN109117891A (en) | It merges social networks and names across the social media account matching process of feature | |
| Fan et al. | Adaptive domain-aware representation learning for speech emotion recognition | |
| CN120707944A (en) | Clothing compatibility analysis method, equipment and medium based on multimodal feature fusion | |
| Wang et al. | A survey of vision and language related multi-modal task | |
| CN121841709A (en) | CVE-ATT & CK association fusion RAG construction method for network security scene | |
| CN106570167A (en) | Knowledge-integrated subject model-based microblog topic detection method | |
| CN118350379B (en) | Method, device and medium for improving accuracy of natural language processing in knowledge system | |
| CN110377753B (en) | Relation extraction method and device based on relation trigger word and GRU model | |
| CN119513317A (en) | Zero-shot stance detection method and system based on multi-task fine-tuning and knowledge enhancement of debate data | |
| CN115577112A (en) | An event extraction method and system based on type-aware gating attention mechanism | |
| CN116522346A (en) | Intelligent contract vulnerability detection method based on key data flow graph and abstract semantic graph | |
| CN107886942A (en) | A Speech Signal Emotion Recognition Method Based on Locally Penalized Random Spectral Regression |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |















































































































































































































































































































































































































































































