EP3640935B1 - Verfahren zur ausgabe von benachrichtigungsinformationen, server und überwachungssystem - Google Patents

Verfahren zur ausgabe von benachrichtigungsinformationen, server und überwachungssystem Download PDF

Info

Publication number
EP3640935B1
EP3640935B1 EP18817001.3A EP18817001A EP3640935B1 EP 3640935 B1 EP3640935 B1 EP 3640935B1 EP 18817001 A EP18817001 A EP 18817001A EP 3640935 B1 EP3640935 B1 EP 3640935B1
Authority
EP
European Patent Office
Prior art keywords
audio information
information
feature value
type audio
type
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP18817001.3A
Other languages
English (en)
French (fr)
Other versions
EP3640935A4 (de
EP3640935A1 (de
Inventor
Zhi Cui
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Hikvision Digital Technology Co Ltd
Original Assignee
Hangzhou Hikvision Digital Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Hikvision Digital Technology Co Ltd filed Critical Hangzhou Hikvision Digital Technology Co Ltd
Publication of EP3640935A1 publication Critical patent/EP3640935A1/de
Publication of EP3640935A4 publication Critical patent/EP3640935A4/de
Application granted granted Critical
Publication of EP3640935B1 publication Critical patent/EP3640935B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/54—Interprogram communication
    • G06F9/542—Event management; Broadcasting; Multicasting; Notifications
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/60—Information retrieval; Database structures therefor; File system structures therefor of audio data
    • G06F16/65—Clustering; Classification
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/08—Speech classification or search
    • G10L15/18—Speech classification or search using natural language modelling
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/08—Speech classification or search
    • G10L15/18—Speech classification or search using natural language modelling
    • G10L15/1815—Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/20—Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0272—Voice signal separating
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G10L25/54—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination for retrieval

Definitions

  • the present application relates to the field of multimedia information processing technology, and in particular, to a method for outputting notification information, a server and a monitoring system.
  • notification information for some abnormal events to remind a relevant person to deal with the abnormal events in time. For example, when a robbery event occurs in a captured video image, the notification information needs to be output for the robbery event. Alternatively, during a process of video monitoring for a cashier desk in a mall or supermarket, if there is a property dispute, the notification information may also be outputted and so on.
  • the solution for outputting the notification information generally includes: analyzing a video image captured by a video capture device, for example, determining an active target in the video image, and a motion trajectory of the active target; based on the analysis result, determining whether an abnormal event occurs in the video image; if an abnormal event occurs in the video image, outputting the notification information.
  • EP3147902A1 relates to a sound processing apparatus which includes detection means for detecting a situation of a subject to be imaged from image data generated by imaging the subject to be imaged by imaging means, extraction means for extracting a feature amount of a sound from sound data generated by sound acquisition means corresponding to the imaging means, and determination means for executing a process of comparing the feature amount of the sound extracted by the extraction means with a feature amount of a specific sound in accordance with the situation of the subject to be imaged detected by the detection means, thereby determining whether the sound contains the specific sound.
  • EP3059733A2 relates to a method, which includes receiving sound from an area being monitored by a video surveillance system having a camera and microphone, analyzing the sound to determine a classification of the sound, and capturing video of the area being monitored responsive to the classification of the sound.
  • CN102521945A relates to a calling detection alarming device, which is characterized by comprising a voice acquisition module, a voice preprocessing module, a voice temporary storage module, a voice comparison module and an alarming module, which are sequentially connected with one another.
  • the voice acquisition module is connected with voice input equipment; the alarming module is connected with output equipment; and the device also comprises a voice preset module which is used for the voice comparison module to compare captured voices with a preset voice.
  • the device has the advantages that: calling for help can be automatically detected when somebody calls for help and the device can give an alarm to related departments or personnel timely, and help for persons calling for help can be timely offered.
  • An objective of embodiments of the present application is to provide a method for outputting notification information, a server and a monitoring system, to improve the accuracy for outputting the notification information.
  • an embodiment of the present application provides a method for outputting notification information, a server and a monitoring system.
  • the method may be applicable to the server in the monitoring system, or may be applicable to various electronic devices, which is not specifically limited.
  • FIG. 1 is a schematic flowchart of a method for outputting notification information provided by an embodiment of the present application. The method includes the following steps.
  • a device performing the solution may have an audio capture function, and the audio information acquired in S101 may be captured by the present device itself.
  • the present device may be in communication with an audio capture device to acquire the audio information from the audio capture device.
  • the solution may be implemented once every preset duration, that is, the audio information is acquired once every preset duration.
  • the solution may be implemented after receiving a trigger instruction from the user, which is not specifically limited.
  • a feature value of the audio information is extracted.
  • the acquired audio information may be filtered, denoised, etc., and then the feature value thereof is extracted.
  • the extracted feature value may include one or more types of: a speech rate, semantic information, a volume zero-crossing rate, a maximum volume value, a minimum volume value, an average volume value, a maximum volume change rate, a minimum volume change rate, an average volume change rate, a maximum sound frequency, a minimum sound frequency, an average sound frequency, a maximum sound frequency change rate, a minimum sound frequency change rate, an average sound frequency change rate, an audio curve vector, a volume curve vector, etc.
  • the extracted feature value is matched with feature value models in a preset database.
  • a database is pre-built before implementing the solution.
  • the database stores a correspondence between the feature value models and forewarning levels, and each feature value model may be a set of multiple feature values.
  • a type of the feature values included in the feature value model is consistent with the type of the feature value extracted in S102. In this way, a better matching effect can be obtained.
  • the forewarning levels include three levels, and the third-level indicates the highest level.
  • the feature value model corresponding to the first forewarning level may be: the speech rate of 200 words/minute, the average volume value of 70 dB, and the semantic information of "caution”.
  • the feature value model corresponding to the second forewarning level may be: the speech rate of 300 words/minute, the average volume value of 80 dB, and the semantic information of "somebody”.
  • the feature value model corresponding to the third forewarning level may be: the speech rate of 400 words/minute, the average volume value of 90 dB, and the semantic information of "help".
  • each forewarning level may correspond to multiple feature value models.
  • only the above models are taken as examples for description.
  • a forewarning level corresponding to the audio information is determined.
  • feature values acquired in S102 include: the speech rate of 300 words/minute, the average volume value of 80 dB, and the semantic information of "somebody”, these feature values are matched with feature value models in the above database, and the second forewarning level is matched. It is determined that the forewarning level corresponding to the audio information acquired in S101 is the second-level.
  • the criteria for successful matching may be set based on actual conditions. For example, it may be provided when a matching rate is higher than a preset value, the matching is successful.
  • the matching result may include information of successfully matching with a certain feature value model or unsuccessfully matching with a certain feature value model, or others, which is not specifically limited.
  • the feature value models stored in the preset database may include a scene sound model, and the scene sound model may be a feature value model constructed for sounds in a preset scene.
  • the scene sounds may include a gunshot sound, a crying sound, a whistling sound, etc., which are not specifically limited. It can be understood that when a disorder occurs in a scene such as a mall, a supermarket or a bank, it is usually accompanied by the gunshot sound, the whistling sound, and the crying sound. In this embodiment, these sounds are referred to as the scene sounds.
  • a machine learning algorithm may be used to perform model training on the scene sounds in advance to obtain the scene sound models. It can be understood that when these scene sounds exist, the probability of occurrence of an abnormal event is large, and therefore, the forewarning levels corresponding to the scene sound models may be set higher.
  • the feature value extracted in S 102 is matched with the scene sound models, and the forewarning level corresponding to a successfully matched scene sound model is determined as the forewarning level of the audio information.
  • the forewarning level in this step refers to the forewarning level corresponding to the above audio information determined in S104.
  • notification information corresponding to the audio information is determined.
  • the preset condition is above the first forewarning level, if the preset condition is met, the notification information corresponding to the audio information acquired in S101 is determined.
  • S106 may include: acquiring a video image and/or geographic location information corresponding to the audio information; and determining the video image and/or the geographic location information as the notification information corresponding to the audio information.
  • the present device may possess a video capture function and a positioning function, so that the present device may acquire the video image captured by itself and the geographical location information determined by itself; or, the present device may be in communication with another device and acquire the video image and/or the geographic location information corresponding to the audio information from the other device, which are not specifically limited.
  • the video image corresponding to the audio information refers to a video image that is in the same scene and is at the same capture moment as the audio information; and the geographical location information corresponding to the audio information refers to the geographical location information where the device that captures the audio information is located.
  • the present device acquires the video image and/or the geographic location information corresponding to the audio information from another device, the other device and the device that captures the audio information perform the audio or video capture for the same scene.
  • the determined notification information is output.
  • the notification information includes the video image and/or the geographical location information, so that the abnormal event may be more accurately notified to the relevant person to deal with.
  • the user may be prompted to determine whether to output the notification information; it is determined whether rejection information sent by the user is received within a preset time period; if no rejection information is received from the user within the preset time period, 5107 is performed.
  • the prompt information may include one or more of: the forewarning level, the video image, the geographic location information corresponding to the audio information, or the like, which is not specifically limited.
  • the prompt information is displayed to the user, and there are multiple display forms, such as pop-up windows, flashing reminders, etc., which are not specifically limited.
  • the user may select the confirmation of output, may select the rejection of output, or may select nothing. If the confirmation information sent by the user is received (the user selects the confirmation of output), or the user's feedback is not received within the preset time period (the user selects nothing), step S107 is performed. If the rejection information sent by the user is received (the user selects the rejection of output), the notification information is not output.
  • the above database is constructed by: acquiring analog audio information of abnormal events; extracting feature values of the analog audio information; constructing feature value models based on the extracted feature values; and storing the constructed feature value models into the database in association with corresponding forewarning levels set by a user.
  • the abnormal events may be understood as robbery events, property dispute events, etc., which are not specifically limited.
  • the above database may be constructed based on actual needs.
  • the analog audio information of the robbery events may be recorded, and the feature values of the analog audio information are extracted.
  • the extracted feature values include: the speech rate of 400 words/minute, the average volume value of 90 dB, and the semantic information of "help"; a feature value model is constructed based on the extracted feature values, and the feature value model may be a set of the above feature values; and, the feature value model is stored in association with a corresponding forewarning level set by the user. In this way, the correspondence between each feature value model and each forewarning level is stored in the database.
  • the constructed database may be updated by: receiving an adding instruction sent by the user; extracting a feature value of target audio information corresponding to the adding instruction; constructing a target feature value model based on the feature value of the target audio information; and adding the target feature value model into the database in association with a corresponding forewarning level included in the adding instruction.
  • the audio information that is considered by the user to meet the expectation is referred as the target audio information.
  • the user may send an adding instruction to the device; wherein, the adding instruction may include an identifier of the target audio information and a forewarning level set by the user for the target audio information.
  • the device determines the target audio information based on the identifier in the adding instruction, extracts a feature value of the target audio information, constructs a target feature value model based on the extracted feature value, and adds the constructed target feature value model in the database in association with a corresponding forewarning level included in the adding instruction.
  • the update of the database is implemented. Further, matching the feature value of the acquired audio information with the feature value models in the updated database may improve the matching accuracy.
  • a database including a correspondence between feature value models and forewarning levels is constructed in advance; a feature value of the audio information is acquired, the acquired feature value is matched with the feature value models in the database, and then a forewarning level corresponding to the audio information is determined; the notification information is output when the forewarning level meets a preset condition.
  • the notification information is output by analyzing the audio information, without determining the active targets in the video image; even if there are many active targets in the scene and the trajectories of the active targets are confusing, the notification information may still be accurately output by applying this solution.
  • FIG. 2 is a schematic flowchart of a second method for outputting notification information provided by an embodiment of the present application. The method includes the following steps.
  • the audio information is multi-type audio information; if the audio information is the multi-type audio information, 5203 is performed; if the audio information is not the multi-type audio information, 5204 is directly performed.
  • the multi-type audio information is decomposed into at least one piece of single-type audio information.
  • multi-type audio information includes multiple types of sounds, and each piece of single-type audio information includes one type of sound.
  • the application scene of the solution may be a single sound scene, for example, in a home scene; the captured audio information may include voice information of only one person, and such audio information is also the above single-type audio information.
  • the application scene of the solution may also be a multi-type sound scene, such as a supermarket, a mall, a bank, etc.
  • the captured audio information includes voice information of multiple persons, and such audio information is also the above multi-type audio information.
  • the captured audio information includes voice information of one person and sound information in the environment, and such audio information is also the multi-type audio information.
  • the captured audio information includes voice information of multiple persons and sound information in the environment, and such audio information is also the multi-type audio information.
  • the multi-type audio information is firstly decomposed into single-type audio information, and then the subsequent steps are performed.
  • 5203 may include: segmenting the multi-type audio information into multiple audio segments based on a preset segmentation rule; for each of the multiple audio segments, determining whether the audio segment includes multiple types of sounds; if the audio segment does not include multiple types of sounds, determining the audio segment as one piece of single-type audio information; if the audio segment includes multiple types of sounds, decomposing the audio segment into at least one piece of single-type audio information based on a sound parameter in the audio segment; wherein, the sound parameter includes one or more of: tone, loudness, timbre.
  • the multi-type audio information may be segmented into multiple audio segments each having an equal length of time; or, the multi-type audio information may be segmented into multiple audio segments each having an equal volume size; or, the number of the audio segments to be segmented into may also be determined based on the total duration of the multi-type audio information, and the multi-type audio information may be segmented into audio segments based on this number; or, the number of the audio segments to be segmented into may also be determined based on the total volume size of the multi-type audio information, and the multi-type audio information may be segmented into audio segments based on this number, or the like; the specific segmentation rule is not specifically limited.
  • the multi-type audio information may be segmented into multiple audio segments each having a duration of 1 second. It is assumed that the total duration of the multi-type audio information is 1 minute, 60 audio segments are obtained.
  • the audio segment For each audio segment, it is determined whether the audio segment includes multiple types of sounds.
  • the multi-type audio information is a conversation between a person A and a person B, the duration thereof is one minute, and the voice information of the person A does not intersect with the voice information of the person B. It is assumed that the fromt 30 audio segments obtained by the segmentation only include the voice information of the person A, and the last 30 audio segments only include the voice information of the person B, each of the 60 audio segments includes only one sound type, and thus is single-type audio information.
  • each audio segment includes voice information of only one person.
  • multiple types of sounds would appear in one audio segment.
  • the multi-type audio information is the conversation between the person A and the person B, and the duration is one minute.
  • some of the audio segments obtained by the segmentation include voice information of only one person and some of the audio segments obtained by the segmentation include voice information of two persons.
  • Each of the audio segments including voice information of one person is treated as the single-type audio information; while for each of the audio segments including voice information of two persons, the audio segment is further decomposed based on the sound parameter in the audio segment.
  • Multi-type audio information is captured form these scenes, and the multi-type audio information is segmented to obtain multiple audio segments. Since there are multiple types of sounds at the same moment, the audio segment corresponding to this moment includes multiple types of sounds. The audio segment is further decomposed based on the sound parameter therein.
  • the sound parameter may include one or more of: pitch, loudness, tone.
  • pitch a parameter for adjusting the sound parameter thereof.
  • loudness a parameter for adjusting the sound parameter thereof.
  • tone a parameter for adjusting the sound parameter thereof.
  • different sounds may be extracted using sound parameter thereof, such as the pitch, the loudness, the tone. Therefore, it is possible to continue to decompose the audio segment including multiple types of sounds to obtain respective pieces of single-type audio information.
  • S204 corresponds to S102 in FIG. 1
  • S205 corresponds to S103 in FIG. 1
  • the steps of extracting the feature value and of matching the feature value in FIG. 2 are performed for each piece of single-type audio information, therefore:
  • S206 corresponds to 5104 in FIG. 1 , and at S206,
  • each piece of single-type audio information included in the multi-type audio information corresponds to a matching result, and in this case, the weight corresponding to each piece of single-type audio information may be determined.
  • the weight is determined based on the order of the pieces of single-type audio information obtained by the decomposition; or the weight is determined based on the average volume value of each piece of single-type audio information, or the like, which is not specifically limited.
  • the multi-type audio information acquired in S201 includes the whistling sound, the crying sound, and the voice information of the multiple persons; the multi-type audio information is decomposed to obtain four pieces of single-type audio information of the "whistling sound", the "crying sound", the "voice information of the person A” and the "voice information of the person B".
  • the forewarning level is determined as the second-level.
  • the forewarning level is determined as the third-level; based on the matching result corresponding to the "voice information of the person A”, the forewarning level is determined as the third-level; and based on the matching result corresponding to the "voice information of the person B", the forewarning level is determined as the first-level.
  • the weight corresponding to the "whistling sound” is 0.7
  • the weight corresponding to the "crying sound” is 0.9
  • the weight corresponding to the "voice information of the person A” is 0.8
  • the weight corresponding to the "voice information of the person B” is 0.6
  • the weights and forewarning levels corresponding to the scene sounds may be set higher.
  • the forewarning level corresponding to the multi-type audio information by only considering forewarning levels and weights corresponding to the scene sounds.
  • 5207 is the same as S105 in FIG. 1
  • S208 is the same as S106 in FIG. 1
  • S209 is the same as 5107 in FIG. 1 .
  • the notification information corresponding to the multi-type audio information is determined, and the subsequent steps are similar to the embodiment of FIG. 1 and will not be described again.
  • the multi-types audio information is acquired, the multi-type audio information is decomposed into single-type audio information, and then the single-type audio information is analyzed to output the notification information, thereby further improving the accuracy of outputting the notification information.
  • FIG. 3 is a schematic flowchart of a third method for outputting notification information provided by an embodiment of the present application. The method includes the following steps.
  • the audio information is multi-type audio information; if the audio information is the multi-type audio information, S303 is performed; if the audio information is not the multi-type audio information, S308 is directly performed.
  • the multi-type audio information is matched with at least one preset scene sound model.
  • each of scene sounds included in the multi-type audio information is determined.
  • a forewarning level and a weight corresponding to each of the scene sounds is determined.
  • the scene sound model may include: a gunshot sound model, a whistling sound model, a crying sound model, and the like, which is not specifically limited in detail. It can be understood that when a disorder occurs in a scene such as a mall, a supermarket or a bank, it is usually accompanied by the gunshot sound, the whistling sound, and the crying sound. In the embodiment in FIG. 3 , these sounds are referred to as the scene sounds.
  • a machine learning algorithm may be used to perform model training on the scene sounds in advance to obtain the scene sound models. Before decomposing the multi-type audio information, the multi-type audio information may be firstly matched with these scene sound models.
  • the multi-type audio information acquired in S301 includes the whistling sound, the crying sound, and the voice information of multiple persons.
  • the multi-type audio information is firstly matched with each of preset scene sound models, and it is assumed that the matching result is: successfully matching with the whistling sound model and the crying sound model, that is, it is determined that the multi-type audio information includes the whistling sound and the crying sound.
  • a corresponding forewarning level and a corresponding weight may be set in advance for each of the scene sounds.
  • the set forewarning levels and weights may be stored correspondingly with the scene sound models, so that based on the matching result in S303, a forewarning level and a weight corresponding to each of the scene sounds (the whistling sound and the crying sound) may be directly determined.
  • S305 may include: extracting each of the scene sounds from the multi-type audio information; for each of the extracted scene sounds, extracting a feature value of the scene sound; matching the extracted feature value with the feature value models in the preset database; determining a forewarning level corresponding to a successfully matched feature value model as a forewarning level corresponding to the scene sound.
  • the multi-type audio information includes the whistling sound and the crying sound.
  • the whistling sound and the crying sound may be extracted respectively based on the tone, the loudness, the timbre or other sound parameters.
  • the scene sound is also handled as the single-type audio information. Specifically, feature values of the whistling sound and the crying sound are extracted and matched, and the specific processes are similar to S204 and S205 in the embodiment of FIG. 2 , which are not described herein again.
  • the database in this implementation and the database in the embodiment of FIG. 1 may be the same database, or may be different databases, which are not specifically limited.
  • the scene sounds and the voice information in the multi-type audio information are separately processed.
  • the scene sounds may be processed firstly, and then the voice information is processed; or, the voice information may be also processed firstly, and then the scene sounds may be processed. That is to say, S303-305 may be performed firstly, and then 5306-5309 are performed; or, 5306-5309 may be performed firstly, and then S303-305 are performed.
  • the specific order is not limited.
  • the voice information in this embodiment refers to "a voice that is made by a person and has semantics", and does not include a voice having no semantics, such as the above crying sound.
  • voice information included in the multi-type audio information is determined.
  • each piece of single-type audio information corresponding to the voice information is determined.
  • voice information made by a person may be extracted through the timbre, or may be extracted by other manners, which is not specifically limited.
  • S308 corresponds to S204 in FIG. 2
  • S309 corresponds to S205 in FIG. 2 . The specific process is not described again.
  • a forewarning level corresponding to the single-type audio information is determined based on the matching result; if the audio information acquired in S301 is the multi-type audio information, a matching result corresponding to each piece of single-type audio information included in the multi-type audio information is obtained; a weight corresponding to each piece of single-type audio information is determined; and, the forewarning level corresponding to the multi-type audio information is determined, based on the weight and the matching result corresponding to each piece of single-type audio information, and the forewarning level and the weight corresponding to each of the scene sounds.
  • each piece of single-type audio information included in the multi-type audio information corresponds to a matching result, and in this case, the weight corresponding to each piece of single-type audio information is determined.
  • the weight is determined based on the order of the pieces of single-type audio information obtained by the decomposition; or, the weight is randomly assigned; or, the weight is determined based on the average volume value of each piece of single-type audio information, or the like, which is not specifically limited.
  • the forewarning level and weight determined in S305, and the matching result and weight corresponding to the single-type audio information are comprehensively considered to determine the forewarning level corresponding to the multi-type audio information. That is to say, the forewarning level and weight corresponding to each of the scene sounds, and the forewarning level and weight corresponding to each piece of single-type audio information are comprehensively considered, to determine the forewarning level corresponding to the multi-type audio information.
  • the multi-type audio information acquired in S301 includes two scene sounds of the whistling sound and the crying sound, and voice information of person A and person B.
  • the multi-type audio information is firstly matched with the scene sound models, to determine that the multi-type audio information includes the "whistling sound” and the "crying sound”; then, it is determined that the voice information included in the multi-type audio information corresponds to two pieces of single-type audio information, the "voice information of the person A" and the "voice information of the person B".
  • the voice information included in the multi-type audio information corresponds to two pieces of single-type audio information, the "voice information of the person A" and the "voice information of the person B"; then, the multi-type audio information is matched with the scene sound models, to determine that the multi-type audio information includes the "whistling sound” and the "crying sound".
  • the forewarning level and the weight corresponding to the "whistling sound” is determined as the second-level and 0.7 respectively, and the forewarning level and the weight corresponding to the "crying sound” is determined as the third-level and 0.9 respectively;
  • S306-S309 the forewarning level and the weight corresponding to the "voice information of the person A” is determined as the third-level and 0.8 respectively, and the forewarning level and the weight corresponding to the "voice information of the person B" is determined as the first-level and 0.6 respectively.
  • S311 is the same as S105 in FIG. 1
  • S312 is the same as S106 in FIG. 1
  • S313 is the same as S107 in FIG. 1 .
  • the notification information corresponding to the multi-type audio information is determined, and the subsequent steps are similar to the embodiment of FIG. 1 and will not be described again.
  • the multi-type audio information is acquired, and scene sounds and voice information in the multi-type audio information are processed separately, such that the scene sounds and the voice information can be distinguished based on the difference therebetween.
  • an embodiment of the present application further provides a server.
  • FIG. 4 is a schematic structural diagram of a server provided by an embodiment of the present application.
  • the server includes: a processor 401 and a memory 402.
  • the memory 402 is configured for storing executable program code.
  • the server is defined as a part of the monitoring system that is defined by the appended claims 8-11.
  • a database including a correspondence between feature value models and forewarning levels is constructed in advance; a feature value of the audio information is acquired, the acquired feature value is matched with the feature value models in the database, and then a forewarning level corresponding to the audio information is determined; the notification information is output when the forewarning level meets a preset condition.
  • the notification information is output by analyzing the audio information, without determining the active targets in the video image; even if there are many active targets in the scene and the trajectories of the active targets are confusing, the notification information may still be accurately output by applying this solution.
  • An embodiment of the present application further provides a monitoring system.
  • the monitoring system may include only a server which has an audio capture function; or, the monitoring system may also include, as shown in FIG. 5 , a server and an audio capture device; or, the monitoring system may also include, as shown in FIG. 6 , a server and a multimedia capture device which has an audio and video capture function; or, the monitoring system may also include, as shown in FIG. 7 , a server, an audio capture device and a video capture device.
  • the audio capture device or the multimedia capture device is configured for capturing audio information and sending the captured audio information to the server.
  • the video capture device or the multimedia capture device is configured for capturing a video image, determining its geographic location information, and sending the captured video image and the determined geographic location information to the server.
  • the server is further configured for, during a process of determining notification information corresponding to the audio information, determining a video image and geographic location information corresponding to the audio information, and adding the video image and the geographic location information to the notification information.
  • the server may include a communication server and a database server; wherein,
  • the server is configured for: acquiring audio information; extracting a feature value of the audio information; matching the extracted feature value with feature value models in a preset database; wherein, the database stores a correspondence between the feature value models and forewarning levels; determining a forewarning level corresponding to the audio information based on a matching result; determining whether the forewarning level meets a preset condition; if the forewarning level meets the preset condition, determining notification information corresponding to the audio information; and outputting the notification information which is determined.
  • the feature value models include a scene sound model, and the scene sound model is a feature value model constructed for a preset scene sound; the server may be further configured for: matching the extracted feature value with the scene sound model.
  • the server is further configured for: after acquiring the audio information, determining whether the audio information is multi-type audio information; wherein, the multi-type audio information includes multiple types of sounds; if the audio information is the multi-type audio information, decomposing firstly the multi-type audio information into at least one piece of single-type audio information; wherein, each piece of single-type audio information includes one type of sound; then extracting a feature value of each piece of single-type audio information; if the audio information is not the multi-type audio information, extracting directly a feature value of the single-type audio information; for each piece of single-type audio information, matching the feature value extracted from the piece of single-type audio information with the feature value models in the preset database; if the audio information is the single-type audio information, determining a forewarning level corresponding to the single-type audio information based on the matching result; if the audio information is the multi-type audio information, obtaining a matching result corresponding to each piece of single-type audio information included in the multi-type audio information;
  • the server may be further configured for:
  • the server is further configured for: in a case that the audio information is determined as the multi-type audio information, matching the multi-type audio information with at least one preset scene sound model;
  • the server may be further configured for:
  • the server may be further configured for:
  • the server may be further configured for:
  • the process of the server constructing the database may include:
  • the server may be further configured for:
  • a database including a correspondence between feature value models and forewarning levels is constructed in advance; a feature value of the audio information is acquired, the acquired feature value is matched with the feature value models in the database, and then a forewarning level corresponding to the audio information is determined; the notification information is output when the forewarning level meets a preset condition.
  • the notification information is output by analyzing the audio information, without determining the active targets in the video image; even if there are many active targets in the scene and the trajectories of the active targets are confusing, the notification information may still be accurately output by applying this solution.
  • An embodiment of the present application further provides a computer readable storage medium.
  • the computer readable storage medium stores a computer program therein.
  • the computer program when being executed by a processor, implements any of the above methods for outputting notification information.
  • An embodiment of the present application further provides executable program code.
  • the executable program code is configured for, when being executed, implementing any of the above methods for outputting notification information.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Acoustics & Sound (AREA)
  • Computational Linguistics (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Quality & Reliability (AREA)
  • Alarm Systems (AREA)
  • Emergency Alarm Devices (AREA)

Claims (12)

  1. Verfahren zum Ausgeben von Benachrichtigungsinformationen, wobei das Verfahren umfasst:
    Erfassen (S101, S201, S301) von Audioinformationen;
    Extrahieren (S102) eines Merkmalswerts der Audioinformationen;
    Abgleichen (S103) des extrahierten Merkmalswerts mit Merkmalswertmodellen in einer voreingestellten Datenbank; wobei die Datenbank eine Entsprechung zwischen den Merkmalswertmodellen und Vorwarnstufen speichert;
    Bestimmen (S104) einer Vorwarnstufe, die den Audioinformationen entspricht, auf Grundlage eines Abgleichsergebnisses;
    Bestimmen (S105, S207, S311), ob die Vorwarnstufe eine voreingestellte Bedingung erfüllt; wenn die Vorwarnstufe die voreingestellte Bedingung erfüllt, Bestimmen (S106, S208, S312) von Benachrichtigungsinformationen, die den Audioinformationen entsprechen; und
    Ausgeben (5017, S209, S313) der Benachrichtigungsinformationen, die bestimmt werden;
    dadurch gekennzeichnet, dass
    nach dem Erfassen der Audioinformationen das Verfahren weiter umfasst:
    Bestimmen (S202, S302), ob es sich bei den Audioinformationen um Mehrtyp-Audioinformationen handelt; wobei die Mehrtyp-Audioinformationen mehrere Klangtypen umfassen;
    wenn es sich bei den Audioinformationen um die Mehrtyp-Audioinformationen handelt, zunächst Zerlegen (S203) der Mehrtyp-Audioinformationen in mindestens eine Einzeltyp-Audioinformation; wobei jede Einzeltyp-Audioinformation einen Klangtyp umfasst; Durchführen des Schritts des Extrahierens eines Merkmalswerts der Audioinformationen;
    wenn es sich bei den Audioinformationen nicht um die Mehrtyp-Audioinformationen handelt, Durchführen des Schritts des Extrahierens eines Merkmalswerts der Audioinformationen;
    wobei das Extrahieren eines Merkmalswerts der Audioinformationen umfasst:
    Extrahieren (S204, S308) eines Merkmalswerts jeder Einzeltyp-Audioinformation;
    wobei das Abgleichen des extrahierten Merkmalswerts mit Merkmalswertmodellen in einer voreingestellten Datenbank umfasst:
    für jede Einzeltyp-Audioinformation, Abgleichen (S205, S309) des aus der Einzeltyp-Audioinformation extrahierten Merkmalswerts mit den Merkmalswertmodellen in der voreingestellten Datenbank;
    wobei das Bestimmen einer Vorwarnstufe, die den Audioinformationen entspricht, auf Grundlage eines Abgleichsergebnisses umfasst:
    wenn es sich bei den Audioinformationen um die Einzeltyp-Audioinformationen handelt, Bestimmen einer Vorwarnstufe, die den Einzeltyp-Audioinformationen entspricht, auf Grundlage des Abgleichsergebnisses;
    wenn es sich bei den Audioinformationen um die Mehrtyp-Audioinformationen handelt, Erhalten eines Abgleichsergebnisses, das jeder Einzeltyp-Audioinformation entspricht, die in den Mehrtyp-Audioinformationen umfasst ist; Bestimmen einer Gewichtung, die jeder der Einzeltyp-Audioinformationen entspricht; und Bestimmen, auf Grundlage der bestimmten Gewichtungen und der Abgleichsergebnisse, einer Vorwarnstufe, die den Mehrtyp-Audioinformationen entspricht;
    falls die Audioinformationen als die Mehrtyp-Audioinformationen bestimmt werden, das Verfahren weiter umfasst:
    Abgleichen (S303) der Mehrtyp-Audioinformationen mit mindestens einem voreingestellten Szenenklangmodell; Bestimmen (S304) eines jeden von Szenenklängen, die in den Mehrtyp-Audioinformationen umfasst sind, auf Grundlage eines Abgleichsergebnisses; Bestimmen (S305) einer Vorwarnstufe und einer Gewichtung, die jedem der Szenenklänge entsprechen;
    wobei das Zerlegen der Mehrtyp-Audioinformationen in mindestens eine Einzeltyp-Audioinformation umfasst: Bestimmen (S306) von Sprachinformationen, die in den Mehrtyp-Audioinformationen umfasst sind; wobei es sich bei den Sprachinformationen um eine Sprache handelt, die von einer Person stammt und Semantik aufweist, die in den Mehrtyp-Audioinformationen umfasst ist;
    Bestimmen (S307) jeder Einzeltyp-Audioinformation, die den Sprachinformationen entspricht, auf Grundlage der Klangfarbe der Sprachinformationen;
    wobei das Bestimmen, auf Grundlage der bestimmten Gewichtungen und der Abgleichsergebnisse, einer Vorwarnstufe, die den Mehrtyp-Audioinformationen entspricht, umfasst:
    Bestimmen der Vorwarnstufe, die den Mehrtyp-Audioinformationen entspricht, auf Grundlage der Gewichtung und des Abgleichsergebnisses, die jeder Einzeltyp-Audioinformation entsprechen, und der Vorwarnstufe und der Gewichtung, die jedem der Szenenklänge entsprechen.
  2. Verfahren nach Anspruch 1, wobei die Merkmalswertmodelle ein Szenenklangmodell umfassen und das Szenenklangmodell ein Merkmalswertmodell ist, das für einen voreingestellten Szenenklang erstellt wurde;
    wobei das Abgleichen des extrahierten Merkmalswerts mit Merkmalswertmodellen in einer voreingestellten Datenbank umfasst:
    Abgleichen des extrahierten Merkmalswerts mit dem Szenenklangmodell.
  3. Verfahren nach Anspruch 1, wobei das Zerlegen der Mehrtyp-Audioinformationen in mindestens eine Einzeltyp-Audioinformation umfasst:
    Segmentieren der Mehrtyp-Audioinformationen in mehrere Audiosegmente auf Grundlage einer voreingestellten Segmentierungsregel;
    für jedes der mehreren Audiosegmente:
    Bestimmen, ob das Audiosegment mehrere Klangtypen umfasst;
    wenn das Audiosegment nicht mehrere Klangtypen umfasst, Bestimmen des Audiosegments als eine Einzeltyp-Audioinformation;
    wenn das Audiosegment mehrere Klangtypen umfasst, Zerlegen des Audiosegments in mindestens eine Einzeltyp-Audioinformation auf Grundlage eines Klangparameters im Audiosegment; wobei der Klangparameter eines oder mehrere umfasst von: Ton, Lautstärke, Klangfarbe.
  4. Verfahren nach Anspruch 1, wobei das Bestimmen von Benachrichtigungsinformationen, die den Audioinformationen entsprechen, umfasst:
    Erfassen eines Videobildes und/oder von geografischen Standortinformationen, die den Audioinformationen entsprechen; und
    Bestimmen des Videobildes und/oder der geografischen Standortinformationen als die Benachrichtigungsinformationen, die den Audioinformationen entsprechen.
  5. Verfahren nach Anspruch 1, wobei das Verfahren vor dem Ausgeben der Benachrichtigungsinformationen, die bestimmt werden, weiter umfasst:
    Auffordern eines Benutzers dazu, zu bestimmen, ob die Benachrichtigungsinformationen ausgegeben werden sollen;
    Bestimmen, ob Ablehnungsinformationen vom Benutzer innerhalb eines voreingestellten Zeitraums empfangen werden; und
    wenn innerhalb des voreingestellten Zeitraums keine Ablehnungsinformationen vom Benutzer empfangen werden, Durchführen des Schritts des Ausgebens der Benachrichtigungsinformationen, die bestimmt werden.
  6. Verfahren nach Anspruch 1, wobei die Datenbank erstellt wird durch:
    Erfassen von analogen Audioinformationen anormaler Ereignisse;
    Extrahieren von Merkmalswerten der analogen Audioinformationen;
    Erstellen von Merkmalswertmodellen auf Grundlage der extrahierten Merkmalswerte; und
    Speichern der erstellten Merkmalswertmodelle in der Datenbank in Verknüpfung mit entsprechenden, von einem Benutzer eingestellten Vorwarnstufen.
  7. Verfahren nach Anspruch 1, wobei das Verfahren weiter umfasst:
    Empfangen einer Hinzufügeanweisung, die von einem Benutzer gesendet wird;
    Extrahieren eines Merkmalswerts von Zielaudioinformationen, die der Hinzufügeanweisung entsprechen;
    Erstellen eines Zielmerkmalswertmodells auf Grundlage des Merkmalswerts der Zielaudioinformationen; und
    Hinzufügen des Zielmerkmalswertmodells zur Datenbank in Verknüpfung mit einer entsprechenden Vorwarnstufe, die in der Hinzufügeanweisung umfasst ist.
  8. Überwachungssystem, wobei das System einen Server umfasst,
    wobei der Server konfiguriert ist zum: Erfassen von Audioinformationen; Extrahieren eines Merkmalswerts der Audioinformationen; Abgleichen des extrahierten Merkmalswerts mit Merkmalswertmodellen in einer voreingestellten Datenbank, wobei die Datenbank eine Entsprechung zwischen den Merkmalswertmodellen und Vorwarnstufen speichert; Bestimmen einer Vorwarnstufe, die den Audioinformationen entspricht, auf Grundlage eines Abgleichsergebnisses; Bestimmen, ob die Vorwarnstufe eine voreingestellte Bedingung erfüllt; wenn die Vorwarnstufe die voreingestellte Bedingung erfüllt, Bestimmen von Benachrichtigungsinformationen, die den Audioinformationen entsprechen; und Ausgeben der Benachrichtigungsinformationen, die bestimmt werden;
    dadurch gekennzeichnet, dass
    wobei der Server nach dem Erfassen der Audioinformationen weiter konfiguriert ist zum:
    Bestimmen, ob es sich bei den Audioinformationen um Mehrtyp-Audioinformationen handelt; wobei die Mehrtyp-Audioinformationen mehrere Klangtypen umfassen;
    wenn es sich bei den Audioinformationen um die Mehrtyp-Audioinformationen handelt, zunächst Zerlegen der Mehrtyp-Audioinformationen in mindestens eine Einzeltyp-Audioinformation; wobei jede Einzeltyp-Audioinformation einen Klangtyp umfasst; Durchführen des Schritts des Extrahierens eines Merkmalswerts der Audioinformationen;
    wenn es sich bei den Audioinformationen nicht um die Mehrtyp-Audioinformationen handelt, Durchführen des Schritts des Extrahierens eines Merkmalswerts der Audioinformationen;
    wobei das Extrahieren eines Merkmalswerts der Audioinformationen umfasst:
    Extrahieren eines Merkmalswerts jeder Einzeltyp-Audioinformation;
    wobei das Abgleichen des extrahierten Merkmalswerts mit Merkmalswertmodellen in einer voreingestellten Datenbank umfasst:
    für jede Einzeltyp-Audioinformation, Abgleichen des aus der Einzeltyp-Audioinformation extrahierten Merkmalswerts mit den Merkmalswertmodellen in der voreingestellten Datenbank;
    wobei das Bestimmen einer Vorwarnstufe, die den Audioinformationen entspricht, auf Grundlage eines Abgleichsergebnisses umfasst:
    wenn es sich bei den Audioinformationen um die Einzeltyp-Audioinformationen handelt, Bestimmen einer Vorwarnstufe, die den Einzeltyp-Audioinformationen entspricht, auf Grundlage des Abgleichsergebnisses;
    wenn es sich bei den Audioinformationen um die Mehrtyp-Audioinformationen handelt, Erhalten eines Abgleichsergebnisses, das jeder Einzeltyp-Audioinformation entspricht, die in den Mehrtyp-Audioinformationen umfasst ist; Bestimmen einer Gewichtung, die jeder der Einzeltyp-Audioinformationen entspricht; und Bestimmen, auf Grundlage der bestimmten Gewichtungen und der Abgleichsergebnisse, einer Vorwarnstufe, die den Mehrtyp-Audioinformationen entspricht;
    falls die Audioinformationen als die Mehrtyp-Audioinformationen bestimmt werden, das Verfahren weiter umfasst:
    Abgleichen der Mehrtyp-Audioinformationen mit mindestens einem voreingestellten Szenenklangmodell; Bestimmen eines jeden von Szenenklängen, die in den Mehrtyp-Audioinformationen umfasst sind, auf Grundlage eines Abgleichsergebnisses; Bestimmen einer Vorwarnstufe und einer Gewichtung, die jedem der Szenenklänge entsprechen;
    wobei das Zerlegen der Mehrtyp-Audioinformationen in mindestens eine Einzeltyp-Audioinformation umfasst: Bestimmen von Sprachinformationen, die in den Mehrtyp-Audioinformationen umfasst sind; wobei es sich bei den Sprachinformationen um eine Sprache handelt, die von einer Person stammt und Semantik aufweist, die in den Mehrtyp-Audioinformationen umfasst ist;
    Bestimmen jeder Einzeltyp-Audioinformation, die den Sprachinformationen entspricht, auf Grundlage der Klangfarbe der Sprachinformationen;
    wobei das Bestimmen, auf Grundlage der bestimmten Gewichtungen und der Abgleichsergebnisse, einer Vorwarnstufe, die den Mehrtyp-Audioinformationen entspricht, umfasst:
    Bestimmen der Vorwarnstufe, die den Mehrtyp-Audioinformationen entspricht, auf Grundlage der Gewichtung und des Abgleichsergebnisses, die jeder Einzeltyp-Audioinformation entsprechen, und der Vorwarnstufe und der Gewichtung, die jedem der Szenenklänge entsprechen.
  9. System nach Anspruch 8, wobei das System weiter umfasst: eine Audioaufnahmevorrichtung;
    wobei die Audioaufnahmevorrichtung zum Aufnehmen der Audioinformationen und Senden der aufgenommenen Audioinformationen an den Server konfiguriert ist.
  10. System nach Anspruch 8, wobei das System weiter umfasst: eine Videoaufnahmevorrichtung;
    wobei die Videoaufnahmevorrichtung zum Aufnehmen eines Videobildes, Bestimmen seiner geografischen Standortinformationen und Senden des aufgenommenen Videobildes und der bestimmten geografischen Standortinformationen an den Server konfiguriert ist;
    wobei der Server während eines Prozesses zum Bestimmen der Benachrichtigungsinformationen, die den Audioinformationen entsprechen, weiter konfiguriert ist zum Bestimmen eines Videobildes und von geografischen Standortinformationen, die den Audioinformationen entsprechen, und Hinzufügen des Videobildes und der geografischen Standortinformationen zu den Benachrichtigungsinformationen.
  11. System nach Anspruch 8, wobei der Server weiter umfasst: einen Datenbankserver, der konfiguriert ist zum Erfassen von analogen Audioinformationen anormaler Ereignisse; Extrahieren von Merkmalswerten der analogen Audioinformationen; Erstellen von Merkmalswertmodellen auf Grundlage der extrahierten Merkmalswerte; und Speichern der erstellten Merkmalswertmodelle mit entsprechenden, von einem Benutzer eingestellten Vorwarnstufen in einer Datenbank des Datenbankservers;
    wobei die voreingestellte Datenbank die Datenbank des Datenbankservers ist.
  12. Computerlesbares Speichermedium, das ein Computerprogramm speichert; wobei das Computerprogramm das Verfahren zum Ausgeben von Benachrichtigungsinformationen nach einem der Ansprüche 1-7 implementiert, wenn es von einem Prozessor ausgeführt wird.
EP18817001.3A 2017-06-12 2018-06-08 Verfahren zur ausgabe von benachrichtigungsinformationen, server und überwachungssystem Active EP3640935B1 (de)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710436582.1A CN109036461A (zh) 2017-06-12 2017-06-12 一种通知信息的输出方法、服务器及监控系统
PCT/CN2018/090388 WO2018228280A1 (zh) 2017-06-12 2018-06-08 一种通知信息的输出方法、服务器及监控系统

Publications (3)

Publication Number Publication Date
EP3640935A1 EP3640935A1 (de) 2020-04-22
EP3640935A4 EP3640935A4 (de) 2020-06-17
EP3640935B1 true EP3640935B1 (de) 2024-02-14

Family

ID=64630058

Family Applications (1)

Application Number Title Priority Date Filing Date
EP18817001.3A Active EP3640935B1 (de) 2017-06-12 2018-06-08 Verfahren zur ausgabe von benachrichtigungsinformationen, server und überwachungssystem

Country Status (4)

Country Link
US (1) US11275628B2 (de)
EP (1) EP3640935B1 (de)
CN (1) CN109036461A (de)
WO (1) WO2018228280A1 (de)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110197663B (zh) * 2019-06-30 2022-05-31 联想(北京)有限公司 一种控制方法、装置及电子设备
CN110532888A (zh) * 2019-08-01 2019-12-03 悉地国际设计顾问（深圳）有限公司 一种监控方法、装置及系统
CN111028860B (zh) * 2019-11-22 2021-08-06 深圳市康冠智能科技有限公司 音频数据处理方法、装置、计算机设备以及存储介质
CN111178883A (zh) * 2019-12-16 2020-05-19 秒针信息技术有限公司 异常确定方法及装置、存储介质、电子装置
CN113838478B (zh) * 2020-06-08 2024-04-09 华为技术有限公司 异常事件检测方法、装置和电子设备
CN112188427A (zh) * 2020-08-19 2021-01-05 天津大学 一种公共场所群体异常事件物联传感系统和方法

Family Cites Families (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7979146B2 (en) * 2006-04-13 2011-07-12 Immersion Corporation System and method for automatically producing haptic events from a digital audio signal
CN101587710B (zh) * 2009-07-02 2011-12-14 北京理工大学 一种基于音频突发事件分类的多码本编码参数量化方法
CN102014278A (zh) * 2010-12-21 2011-04-13 四川大学 一种基于语音识别技术的智能视频监控方法
CN102810311B (zh) * 2011-06-01 2014-12-03 株式会社理光 说话人估计方法和说话人估计设备
CN102521945A (zh) * 2011-12-02 2012-06-27 无锡奥盛信息科技有限公司 一种呼叫探测报警方法与装置
CN103366740B (zh) * 2012-03-27 2016-12-14 联想(北京)有限公司 语音命令识别方法及装置
CN103456301B (zh) 2012-05-28 2019-02-12 中兴通讯股份有限公司 一种基于环境声音的场景识别方法及装置及移动终端
JP6127422B2 (ja) * 2012-09-25 2017-05-17 セイコーエプソン株式会社 音声認識装置及び方法、並びに、半導体集積回路装置
CN102970438A (zh) * 2012-11-29 2013-03-13 广东欧珀移动通信有限公司 一种手机自动报警方法及自动报警装置
CN103198838A (zh) 2013-03-29 2013-07-10 苏州皓泰视频技术有限公司 一种用于嵌入式系统的异常声音监控方法和监控装置
CN104239372B (zh) * 2013-06-24 2017-09-12 浙江大华技术股份有限公司 一种音频数据分类方法及装置
CN104347068B (zh) * 2013-08-08 2020-05-22 索尼公司 音频信号处理装置和方法以及监控系统
CN104036617B (zh) * 2014-06-11 2017-05-17 广东安居宝数码科技股份有限公司 报警方法和报警系统
CN104156297A (zh) * 2014-08-07 2014-11-19 浪潮(北京)电子信息产业有限公司 告警方法和装置
CN105812721A (zh) * 2014-12-30 2016-07-27 浙江大华技术股份有限公司 一种跟踪监控方法及跟踪监控设备
US20160241818A1 (en) * 2015-02-18 2016-08-18 Honeywell International Inc. Automatic alerts for video surveillance systems
CN104795064B (zh) * 2015-03-30 2018-04-13 福州大学 低信噪比声场景下声音事件的识别方法
CN105022835B (zh) * 2015-08-14 2018-01-12 武汉大学 一种群智感知大数据公共安全识别方法及系统
JP6682222B2 (ja) * 2015-09-24 2020-04-15 キヤノン株式会社 検知装置及びその制御方法、コンピュータプログラム
CN105679313A (zh) * 2016-04-15 2016-06-15 福建新恒通智能科技有限公司 一种音频识别报警系统及方法
CN106328134A (zh) * 2016-08-18 2017-01-11 都伊林 监狱语音数据识别及监测预警系统
CN106683361A (zh) * 2017-01-24 2017-05-17 宇龙计算机通信科技(深圳)有限公司 声音监控方法及装置

Also Published As

Publication number Publication date
US20200364097A1 (en) 2020-11-19
EP3640935A4 (de) 2020-06-17
US11275628B2 (en) 2022-03-15
WO2018228280A1 (zh) 2018-12-20
CN109036461A (zh) 2018-12-18
EP3640935A1 (de) 2020-04-22

Similar Documents

Publication Publication Date Title
EP3640935B1 (de) Verfahren zur ausgabe von benachrichtigungsinformationen, server und überwachungssystem
US20200228648A1 (en) Method and apparatus for detecting abnormality of caller
CN111310665A (zh) 违规事件检测方法及装置、电子设备和存储介质
EP3407200A1 (de) Verfahren und vorrichtung zur aktualisierung eines selbstlernenden online-modells für ereigniserkennung
WO2021136975A1 (en) Image processing methods and apparatuses, electronic devices, and storage media
CN109993044B (zh) 电信诈骗识别系统、方法、装置、电子设备及存储介质
CN108073577A (zh) 一种基于人脸识别的报警方法和系统
CN107566358A (zh) 一种风险预警提示方法、装置、介质及设备
CN106960172A (zh) 人员识别处理方法、装置及系统
JP2012048689A (ja) 異常検知装置
US20210201478A1 (en) Image processing methods, electronic devices, and storage media
US20240095862A1 (en) Method for determining dangerousness of person, apparatus, system and storage medium
CN110544491A (zh) 一种实时关联说话人及其语音识别结果的方法及装置
TWM565361U (zh) 金融交易詐騙偵測防範系統
CN111985428A (zh) 一种安全检测方法、装置、电子设备及存储介质
CN116170566A (zh) 一种智慧楼宇监控管理方法、装置、电子设备及存储介质
CN105227920A (zh) 一种通过网络摄像机进行监控的方法和装置
CN115424355A (zh) 抽烟检测方法及装置、电子设备、计算机可读存储介质
CN108694388B (zh) 基于智能摄像头的校园监控方法及设备
CN115424634A (zh) 音视频流数据处理方法、装置、电子设备及存储介质
CN110738077B (zh) 一种异物检测方法及装置
CN114596636A (zh) 异常行为识别方法、装置、电子设备和可读存储介质
JP2012058944A (ja) 異常検知装置
CN110659603A (zh) 一种数据处理方法及装置
CN115393798A (zh) 预警方法、装置、电子设备及存储介质

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20191219

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

A4 Supplementary search report drawn up and despatched

Effective date: 20200515

RIC1 Information provided on ipc code assigned before grant

Ipc: G10L 25/48 20130101ALI20200511BHEP

Ipc: G10L 15/20 20060101ALI20200511BHEP

Ipc: G10L 25/51 20130101AFI20200511BHEP

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20220411

REG Reference to a national code

Ref legal event code: R079

Free format text: PREVIOUS MAIN CLASS: G10L0015200000

Ipc: G10L0021027200

Ref country code: DE

Ref legal event code: R079

Ref document number: 602018065278

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: G10L0015200000

Ipc: G10L0021027200

RIC1 Information provided on ipc code assigned before grant

Ipc: G10L 25/51 20130101ALI20230718BHEP

Ipc: G10L 21/0272 20130101AFI20230718BHEP

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20230911

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

REG Reference to a national code

Ref country code: DE

Ref legal event code: R081

Ref document number: 602018065278

Country of ref document: DE

Owner name: HANGZHOU HIKVISION DIGITAL TECHNOLOGY CO., LTD, CN

Free format text: FORMER OWNER: HANGZHOU HIKVISION DIGITAL TECHNOLOGY CO., LTD., HANGZHOU, CN

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602018065278

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20240304

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG9D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20240214

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240614

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240515

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1657754

Country of ref document: AT

Kind code of ref document: T

Effective date: 20240214

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240514

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240514

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240514

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240614

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240515

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240614

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240614

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602018065278

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

26N No opposition filed

Effective date: 20241115

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20240608

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20240608

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240214

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20240630

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20240630

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20240630

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20250617

Year of fee payment: 8

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20250625

Year of fee payment: 8

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20250630

Year of fee payment: 8

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20180608

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20180608