EP1168298A2 - Procédé d'assemblage de messages pour la synthèse de la parole - Google Patents

Procédé d'assemblage de messages pour la synthèse de la parole Download PDF

Info

Publication number
EP1168298A2
EP1168298A2 EP01114995A EP01114995A EP1168298A2 EP 1168298 A2 EP1168298 A2 EP 1168298A2 EP 01114995 A EP01114995 A EP 01114995A EP 01114995 A EP01114995 A EP 01114995A EP 1168298 A2 EP1168298 A2 EP 1168298A2
Authority
EP
European Patent Office
Prior art keywords
segments
segment
sentence
announcement
search
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP01114995A
Other languages
German (de)
English (en)
Other versions
EP1168298B1 (fr
EP1168298A3 (fr
Inventor
Peter Buth
Simona Grothues
Amir Iman
Wolfgang Theimer
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nokia Oyj
Original Assignee
Nokia Mobile Phones Ltd
Nokia Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nokia Mobile Phones Ltd, Nokia Inc filed Critical Nokia Mobile Phones Ltd
Publication of EP1168298A2 publication Critical patent/EP1168298A2/fr
Publication of EP1168298A3 publication Critical patent/EP1168298A3/fr
Application granted granted Critical
Publication of EP1168298B1 publication Critical patent/EP1168298B1/fr
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L13/00—Speech synthesis; Text to speech systems
    • G10L13/06—Elementary speech units used in speech synthesisers; Concatenation rules
    • G10L13/07—Concatenation rules

Definitions

  • the invention relates to a method for composing announcements for voice output, in particular with the improvement of the playback quality of such voice outputs.
  • the invention has for its object to provide a method for forming announcements from segments, which takes into account the natural flow of speech and thus leads to harmonious playback results.
  • each segment is assigned at least one variable that characterizes its phonetic properties in the original sentence, and that the segments of the individual segments that characterize the phonetic properties in the original sentence are used to check whether the segments forming the playback sentence to be output as an announcement are composed according to their natural flow of speech , In this way it can be achieved that the natural flow and rhythm of an announcement is largely reconstructed in speech reproduction, without the announcement itself having to be completely stored.
  • each segment is assigned a number of variables which characterize its phonetic properties in the original sentence, the variables advantageously being selectable from the following variables: length of the respective segment, position of the respective segment in the original sentence, front and / or rear transition value of the respective segment to the preceding or following segment in the original sentence, the length of the respective assigned search criterion also being used as the length of the respective segment.
  • an advantageous development of the invention provides that the last or first letters, syllables or phonemes of the preceding or following segment in the original sentence are used as transition values.
  • a particularly high quality Playback of playback sentences composed of audio files is achieved if phonemes are used as transition values
  • An expedient development of the invention is characterized in that for a combination of segments found which forms the playback sentence to be output as an announcement, an evaluation measure is calculated from the quantities of the individual segments which characterize the phonetic properties in the original sentence according to the following formula: where f n, i (n) is a functional relationship of the nth size, i is an index designating the segment and W n is a weighting factor for the functional relationship of the nth size.
  • f n, i (n) is a functional relationship of the nth size
  • i an index designating the segment
  • W n is a weighting factor for the functional relationship of the nth size.
  • the combination of segments found is then selected as the announcement to be played back, the evaluation measure of which indicates that the segments of the combination are composed according to a natural flow of speech.
  • evaluation measure B calculates from the functional relationships f n (n) at least the following variables length L and position P as well as front and rear transition values Ü front , Ü rear of the segment according to the following formula becomes:
  • the evaluation is particularly simple when the reproduction record is in a format corresponding to the search criteria, wherein alphanumeric character strings are preferably used for the search criteria and the transmitted reproduction records.
  • search criteria are arranged hierarchically in a database.
  • the selection of the segments for the playback of an announcement is particularly simple if, to select the segments for an announcement stored as audio files, it is checked whether the playback set desired as the announcement as a whole matches a search criterion stored in a database together with an assigned audio file, if this is not the case, the end of the respective rendering set is reduced and then checked for matches with search criteria stored in the database until one or more matches have been found for the remaining part of the rendering set, if for such parts of the rendering set, which were separated in a previous step, the check specified in the last paragraph is continued if, for each combination of segments whose search criteria completely match the playback set, it is checked whether the playback set to be output as an announcement bi ld segments are composed according to their natural flow of speech, and if the audio files of the segments are used to reproduce a desired announcement, the combination of which comes closest to the natural flow of speech.
  • the data records prepared in this way can be a combination very easily be created from segments whose playback is different from a spoken playback the corresponding announcement no longer differentiates.
  • This effect is achieved in that before the announcement, that is to say before the playback of sentences, parts of sentences, requests, commands, phrases or the like, the database is searched for segments from which combinations corresponding to the desired announcement can be formed, and that then each combination found and consisting of one or more segments is evaluated based on the information for each segment used, which describes the approximation of the combination to the natural flow of speech. Once the evaluations for the combinations determined have been completed, the combination of segments that comes closest to the natural flow of speech is selected for the announcement.
  • each of these original sentences is the same Has meaningfulness and - apart from the order - there are no differences in the letters and numbers used, but there are considerable differences between the individual original sentences when they are reproduced acoustically. This is due to the fact that, depending on the position of individual words or groups of words in the sentence structure, different stresses may arise. If, for example, the sentence "Turn left in 100 meters" is to be reproduced as an announcement and segments 10.4 and 10.3 are used instead of segments 10.1 and 10.2, this does not lead to a harmonious reproduction that corresponds to the normal flow of speech.
  • the different sentences for an announcement are spoken and recorded as so-called original sentences by a speaker.
  • the original sentences recorded in this way are then divided into segments 10, each of these segments 10 being stored in an audio file.
  • a group of search criteria is also assigned to each original record. This group of search criteria is divided according to the segmentation of the original records, with each segment 10 being assigned a search criterion.
  • the mutual assignment of audio files and search criteria takes place in a database 11, which is shown in more detail in FIG. 2.
  • alphanumeric character strings are used here as search criteria, the character strings used as search criteria correspond to the textual representation of the assigned segments 10 stored as audio files.
  • search criteria correspond to the textual representation of the assigned segments 10 stored as audio files.
  • the database 11 has further entries 12. According to the column headings, these entries 12 are the length (L) of the respective segment, its position (P) within the sentence and two connecting sounds or transition values (Ü front , Ü rear ).
  • the respective entries 12, which relate to the length (L), e.g. B. obtained by determining the number of words of the assigned segment 10 for each of the search criteria.
  • the words within the assigned search criterion can be used in the present exemplary embodiment. This results in the length value 1 for the audio file or the segment 10 assigned to the search criterion “turn”, while the length value 3 is assigned to the search criterion “in 100 meters” since the number sequence “100” is regarded as a word.
  • the words contained in the search criterion do not necessarily have to be used to obtain the length information.
  • the number of characters contained in the respective search criterion can also be used. For example, this would result in a length value of 8 for the search criterion "and a length value of 13 for the search criterion" in 100 meters ", since the last search criterion also evaluates the spaces between the words and the numbers as characters. It is also conceivable to use the number of syllables or phonemes as a length value.
  • the entry 12, which represents the position (P), is obtained, for example, by first the number of segments 10 or search criteria can be determined for each original record. If, for example, an original record is divided into three segments 10 when segmented, the first segment 10 is assigned the position value 0, the second segment 10 the position value 0.5 and the last of the three segments 10 the position value 1. However, if the original sentence is only divided into two segments 10 (for example in the first two original sentences in FIG. 1), the first segment 10 receives the position value 0, while the second and last segment 10 receives the position value 1. If the original set consists of four segments 10, the first segment 10 has the position value 0, the second segment 10 the position value 0.33 and the third segment 10 the position value 0.66, while the last segment again receives the position value 1.
  • Transitional values (T) in the sense of this application are understood to mean the relationships of a segment 10 or search criterion to the segment 10 or search criterion preceding and following this segment 10 or search criterion. This relationship for the respective segment 10 is established in the present case to the last letter of the previous segment 10 and to the first letter of the following segment 10. The more detailed explanation is now to be made on the basis of the first set of original signals (turn left at 100 meters) according to FIG. 1. Since the first segment 10 or search criterion of this original sentence (in 100 meters) has no preceding segment 10 or search criterion, the entry in the data record relating to this segment 10 and bearing the index number 3 (FIG. 2) is the front transition value "empty" noted.
  • segment 10 (in 100 meters) in the original sentence is followed by segment 10 (turn left), because in the present exemplary embodiment only one letter is used as transition values (Ü), a rear transition value (Ü) is entered in the data record with index number 3 "I" noted.
  • the same procedure is followed with the second segment (10) of the original sentence (turn left), which in the data record with index number 9 leads to the front transition value (Ü) "n” and to the rear transition value (Ü) "empty” because segment 10 (in 100 meters), which precedes segment 10 (turn left) in the original sentence, ends with an "n” and segment 10 (turn left) no longer follows segment 10 in the original sentence.
  • transition values (Ü) for the respective segment 10 shown in the preceding paragraph to the last letter of the segment 10 preceding this or the first letter of the segment 10 following this segment 10 is not mandatory. Rather, instead of individual letters, it is also possible to use groups of letters or phonemes from the segment 10 preceding and following segment 10 as the respective transition values (Ü). In particular, the use of phonemes leads to a high-quality reproduction of announcements composed from audio files on the basis of the data records according to FIG. 2.
  • entries 12 shown in FIG. 2 need not be limited to the length, the position and the two transition values. Rather, further entries 12 (not shown) can also be provided in order to further improve the quality of the announcements. Since there is an emphasis difference between question and exclamation sentences, although the textual rendering of the corresponding sentence is completely identical without taking punctuation marks into account, a column can be provided as a further entry 12 in the database 11 according to FIG. 2, in which it is noted whether the respective segment 10 or search criterion comes from a question or exclamation set.
  • the entire sentence intended for reproduction "turn left in 100 meters” is brought into a format in which the search criteria of the corresponding segments 10 are also present. Since in the exemplary embodiment shown the search criteria correspond to the textual reproduction of the audio files, the sentence to be reproduced is also brought into this format if it should not already be in this format. It is then checked whether one or more search criteria are present in the database 11, which have a complete match with the correspondingly formatted sentence intended for reproduction "Turn left in 100 meters". Since this is not the case according to the database shown in FIG.
  • the search string of the sentence intended for reproduction (turn left in 100 meters) is shortened by the last word "turn” and it is examined whether this subset "left in 100 meters” in this form occurs in the database 11 as a search criterion. Since this comparison must also be negative due to the content of the database 11, the sentence intended for reproduction is reduced again by one word. Then it is checked again whether the part of the sentence "in 100 meters” which is then reduced in this way occurs in the data records of the database 11 as a search criterion. According to the content of the database 11, this can be affirmed for the data records with the indices 3 to 6. This then leads to temporary storage of the found indices 3 to 6.
  • Combinations are then formed from the indices found in each case, which each result in the sentence to be reproduced.
  • the latter is shown in more detail in Fig.3. Since in the present case the sentence to be reproduced is formed from the indices 9 and 10 and the indices 3 to 6, only the combinations in FIG. 3 with the serial numbers 1 to 8 are relevant. The other combinations in Fig. 3 have no meaning in this embodiment.
  • the length and position information and information on the transition values of the sentence to be reproduced are then created in accordance with the convention, which were relevant when the corresponding entries 12 were made in the database 11, for the sentence parts, their index in the relevant one Combination stands, the length and position information as well as the respective transition values are buffered.
  • Such a buffering is shown for the sentence to be reproduced "turn left in 100 meters" in FIG. 4, whereby the designation W indicates that it is the position and the transition values of the segments in the sentence to be reproduced and not that in the database 11 stored values.
  • the values that are entered in the data records with the indices 3 to 6 or 9 and 10 can be used, since by the fact that if the sentence to be reproduced or a tell thereof a complete match in the search criteria according to FIG. 2 has found that the length specification in the corresponding data records in the database 11 according to FIG. 2 corresponds to the length value of the part of the record to be reproduced.
  • the combinations are evaluated by determining an evaluation measure B for each of these combinations using the entries 12 for the segments 10 or search criteria in the database 11, who are involved in the respective combination.
  • the evaluation measure B is determined according to the following formula: where W n is a weighting factor for the nth entry 12, f n, i is a functional relationship of the nth entry 12, n is a running index which runs over the individual entries of a data record assigned to a segment involved in a combination, and i further Run index is, runs over all indices of the data records or segments involved in the combination.
  • a functional relationship f n, i (n) is thus created for each entry n included in the formula.
  • some or all of the functional relationships can be provided with a weighting factor W n .
  • the functional relationship f L, i (L) is formed for the length information L of a segment 10 such that the value one is divided by the value of the length L, which corresponds to the entry (length) in the respective data set i, then receive a value that is less than one for each data record whose index is involved in a combination, provided that - as assumed here - the weighting factor W L for the length is equal to one.
  • longer segments 10 provide smaller values f Li (L) due to the formula. These smaller values should be preferred because the longer segments make better use of an existing melody.
  • f Pi (P) for the position information P
  • the functional relationship for the transition values f Ü, i (Ü front ), f Ü, i (Ü rear ) can be formed analogously to the preceding paragraph by the temporarily stored transition values Ü front, W , Ü rear, W from FIG. 4 with the Transition values Ü front, D , Ü rear, D of the corresponding data records from the database are related in such a way that a zero is assigned if there is a match and a value greater than zero if there is no match.
  • a corresponding weighting factor W Ü can be used.
  • the functional relationships for the front and rear transition values should expediently each have a weighting factor Wü can be provided from 0.5.
  • the table according to FIG. 5 clearly shows that there are B values for each serial number which are between 0.8 and 4.8.
  • the table in accordance with FIG. 5 shows that double B values are also present. Since preferably only those audio files from data records of the database 11 are to be combined for speech reproduction, the combinations of which, according to FIG. 3, have the lowest B value of all combinations after an evaluation according to the above formula, all occurring B values are those according to the table 5 are greater than 0.8, meaningless. This insignificance does not exist with the combinations of the serial numbers 1 and 5 according to FIG. 5, since with these combinations the B values are 0.8 and thus represent the smallest B values.
  • the data records 3 and 5 used to form the combinations according to the serial numbers 1 and 5 are the same.
  • the data records with the indices 3 and 5 are identified as duplicate and only in accordance with a further convention the data record with the smallest index number is left in the database.
  • the deletion of the data record with the index 5 means that in Fig. 4 there are no longer any combinations which have the serial numbers 5 and 6. Consequently, the sequence numbers 5 and 6 are also omitted in the table according to FIG. 5, so that no B values are determined for these combinations and the combination 3/9 (sequence number 1) is determined as the combination with the smallest B value becomes.
  • the relevant audio files are put together and output based on the indices involved. If it has been found that the combination 3/9 is the combination with the smallest B value in the exemplary embodiment discussed above, the corresponding audio files (file 3 and file 9) are combined and output.
  • the audio files do not necessarily have to be stored in the database 11 according to FIG. 2. Much more is sufficient if there are corresponding references in the database 11 to the audio files stored at another location.
  • the phrase "Turn left in 100 meters” is used. If this sentence is received as a text string, it is first checked whether at least the beginning of this sentence matches a search criteria in the table according to FIG. 2. In this test, the table according to FIG. H. starting with the last entry. In the present case, this would be the data record with index 10. During this check, the entry "in 100 meters” with index 6 is found. Since the found entry "in 100 meters” cannot completely cover the playback record, the part which is not covered by the search criterion of the data record just found is separated. The data record with index 6 is also buffered.
  • the search for the part of the playback sentence "turn left" is continued, starting not at the end of the table in accordance with FIG. 2, but instead at the point where the last correspondence (here data record with index 10) was found. This means that the entry with index 9 is then found. After finding the data record with index 9, the one with index 6 is also copied here and cached together with the found data record with index 9 as a possible interim solution. Then the part found "turn left” is separated from the search string and the search for the rest is started. Since the search string no longer has any content when the "turn left" part is removed, the index combination 6, 9 is noted as a combination which completely exhausts the sentence to be reproduced.
  • the part found is separated again "left” and the search for the part remaining in the search string is "bend". This search then leads to the entry with the index 2 being found.
  • the combination 6, 8 cached as a partial solution in the last step is copied again and cached together with the data record with index 2 as a further partial solution.
  • the part found is then again separated from the search string. Since the search string is now empty again, the combination of the data records with the indices 6, 8, 2 is stored as a combination which completely reproduces the playback record. Then the process returns to the previous step and the search for a match of the search string "turn off" is continued, the search for the entry where the last match (here the data record with index 2) was found also started here.
  • the data record with the index 1 is found, which leads to the result that the combination of the data records with the indices 6, 8, 1 is stored as a combination which completely reproduces the playback record.
  • the search for combinations is terminated when a predetermined number of combinations, for example 10 combinations, has been found.
  • a predetermined number of combinations for example 10 combinations.
  • this measure reduces the memory requirement and the required computing power.
  • This combination limitation is particularly advantageous if the search is carried out in accordance with the last-mentioned method. This is due to the fact that with this search method, longer and longer segments are found first. This finding of the longer segments ensures that the best combination is usually recognized even under the first combinations and thus there is no loss of quality.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Electric Clocks (AREA)
  • Machine Translation (AREA)
  • Document Processing Apparatus (AREA)
  • Electrically Operated Instructional Devices (AREA)
EP01114995A 2000-06-30 2001-06-20 Procédé d'assemblage de messages pour la synthèse de la parole Expired - Lifetime EP1168298B1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE10031008A DE10031008A1 (de) 2000-06-30 2000-06-30 Verfahren zum Zusammensetzen von Sätzen zur Sprachausgabe
DE10031008 2000-06-30

Publications (3)

Publication Number Publication Date
EP1168298A2 true EP1168298A2 (fr) 2002-01-02
EP1168298A3 EP1168298A3 (fr) 2002-12-11
EP1168298B1 EP1168298B1 (fr) 2006-11-29

Family

ID=7646792

Family Applications (1)

Application Number Title Priority Date Filing Date
EP01114995A Expired - Lifetime EP1168298B1 (fr) 2000-06-30 2001-06-20 Procédé d'assemblage de messages pour la synthèse de la parole

Country Status (5)

Country Link
US (1) US6757653B2 (fr)
EP (1) EP1168298B1 (fr)
JP (1) JP2002055692A (fr)
AT (1) ATE347160T1 (fr)
DE (2) DE10031008A1 (fr)

Families Citing this family (124)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8645137B2 (en) 2000-03-16 2014-02-04 Apple Inc. Fast, language-independent method for user authentication by voice
US7089184B2 (en) * 2001-03-22 2006-08-08 Nurv Center Technologies, Inc. Speech recognition for recognizing speaker-independent, continuous speech
US8677377B2 (en) 2005-09-08 2014-03-18 Apple Inc. Method and apparatus for building an intelligent automated assistant
US8036894B2 (en) * 2006-02-16 2011-10-11 Apple Inc. Multi-unit approach to text-to-speech synthesis
US9318108B2 (en) 2010-01-18 2016-04-19 Apple Inc. Intelligent automated assistant
US8027837B2 (en) * 2006-09-15 2011-09-27 Apple Inc. Using non-speech sounds during text-to-speech synthesis
US8977255B2 (en) 2007-04-03 2015-03-10 Apple Inc. Method and system for operating a multi-function portable electronic device using voice-activation
US9330720B2 (en) 2008-01-03 2016-05-03 Apple Inc. Methods and apparatus for altering audio output signals
US8996376B2 (en) 2008-04-05 2015-03-31 Apple Inc. Intelligent text-to-speech conversion
US10496753B2 (en) 2010-01-18 2019-12-03 Apple Inc. Automatically adapting user interfaces for hands-free interaction
US20100030549A1 (en) 2008-07-31 2010-02-04 Lee Michael M Mobile device having human language translation capability with positional feedback
US8396714B2 (en) 2008-09-29 2013-03-12 Apple Inc. Systems and methods for concatenation of words in text to speech synthesis
US8712776B2 (en) 2008-09-29 2014-04-29 Apple Inc. Systems and methods for selective text to speech synthesis
US8352272B2 (en) 2008-09-29 2013-01-08 Apple Inc. Systems and methods for text to speech synthesis
US8352268B2 (en) 2008-09-29 2013-01-08 Apple Inc. Systems and methods for selective rate of speech and speech preferences for text to speech synthesis
US9959870B2 (en) 2008-12-11 2018-05-01 Apple Inc. Speech recognition involving a mobile device
US8380507B2 (en) 2009-03-09 2013-02-19 Apple Inc. Systems and methods for determining the language to use for speech generated by a text to speech engine
US10241752B2 (en) 2011-09-30 2019-03-26 Apple Inc. Interface for a virtual digital assistant
US9858925B2 (en) 2009-06-05 2018-01-02 Apple Inc. Using context information to facilitate processing of commands in a virtual assistant
US10241644B2 (en) 2011-06-03 2019-03-26 Apple Inc. Actionable reminder entries
US10255566B2 (en) 2011-06-03 2019-04-09 Apple Inc. Generating and processing task items that represent tasks to perform
US9431006B2 (en) 2009-07-02 2016-08-30 Apple Inc. Methods and apparatuses for automatic speech recognition
US10276170B2 (en) 2010-01-18 2019-04-30 Apple Inc. Intelligent automated assistant
US10679605B2 (en) 2010-01-18 2020-06-09 Apple Inc. Hands-free list-reading by intelligent automated assistant
US10705794B2 (en) 2010-01-18 2020-07-07 Apple Inc. Automatically adapting user interfaces for hands-free interaction
US10553209B2 (en) 2010-01-18 2020-02-04 Apple Inc. Systems and methods for hands-free notification summaries
DE202011111062U1 (de) 2010-01-25 2019-02-19 Newvaluexchange Ltd. Vorrichtung und System für eine Digitalkonversationsmanagementplattform
US8682667B2 (en) 2010-02-25 2014-03-25 Apple Inc. User profiling for selecting user specific voice input processing information
US10762293B2 (en) 2010-12-22 2020-09-01 Apple Inc. Using parts-of-speech tagging and named entity recognition for spelling correction
US9262612B2 (en) 2011-03-21 2016-02-16 Apple Inc. Device access using voice authentication
US10057736B2 (en) 2011-06-03 2018-08-21 Apple Inc. Active transport based notifications
US8994660B2 (en) 2011-08-29 2015-03-31 Apple Inc. Text correction processing
US9372902B2 (en) 2011-09-23 2016-06-21 International Business Machines Corporation Accessing and editing virtually-indexed message flows using structured query langauge (SQL)
US10134385B2 (en) 2012-03-02 2018-11-20 Apple Inc. Systems and methods for name pronunciation
US9483461B2 (en) 2012-03-06 2016-11-01 Apple Inc. Handling speech synthesis of content for multiple languages
US9280610B2 (en) 2012-05-14 2016-03-08 Apple Inc. Crowd sourcing information to fulfill user requests
US9721563B2 (en) 2012-06-08 2017-08-01 Apple Inc. Name recognition system
US9495129B2 (en) 2012-06-29 2016-11-15 Apple Inc. Device, method, and user interface for voice-activated navigation and browsing of a document
US9576574B2 (en) 2012-09-10 2017-02-21 Apple Inc. Context-sensitive handling of interruptions by intelligent digital assistant
US9547647B2 (en) 2012-09-19 2017-01-17 Apple Inc. Voice-based media searching
DE112014000709B4 (de) 2013-02-07 2021-12-30 Apple Inc. Verfahren und vorrichtung zum betrieb eines sprachtriggers für einen digitalen assistenten
US9368114B2 (en) 2013-03-14 2016-06-14 Apple Inc. Context-sensitive handling of interruptions
WO2014144579A1 (fr) 2013-03-15 2014-09-18 Apple Inc. Système et procédé pour mettre à jour un modèle de reconnaissance de parole adaptatif
WO2014144949A2 (fr) 2013-03-15 2014-09-18 Apple Inc. Entraînement d'un système à commande au moins partiellement vocale
WO2014197334A2 (fr) 2013-06-07 2014-12-11 Apple Inc. Système et procédé destinés à une prononciation de mots spécifiée par l'utilisateur dans la synthèse et la reconnaissance de la parole
WO2014197336A1 (fr) 2013-06-07 2014-12-11 Apple Inc. Système et procédé pour détecter des erreurs dans des interactions avec un assistant numérique utilisant la voix
US9582608B2 (en) 2013-06-07 2017-02-28 Apple Inc. Unified ranking with entropy-weighted information for phrase-based semantic auto-completion
WO2014197335A1 (fr) 2013-06-08 2014-12-11 Apple Inc. Interprétation et action sur des commandes qui impliquent un partage d'informations avec des dispositifs distants
US10176167B2 (en) 2013-06-09 2019-01-08 Apple Inc. System and method for inferring user intent from speech inputs
KR101959188B1 (ko) 2013-06-09 2019-07-02 애플 인크. 디지털 어시스턴트의 둘 이상의 인스턴스들에 걸친 대화 지속성을 가능하게 하기 위한 디바이스, 방법 및 그래픽 사용자 인터페이스
KR101809808B1 (ko) 2013-06-13 2017-12-15 애플 인크. 음성 명령에 의해 개시되는 긴급 전화를 걸기 위한 시스템 및 방법
KR101749009B1 (ko) 2013-08-06 2017-06-19 애플 인크. 원격 디바이스로부터의 활동에 기초한 스마트 응답의 자동 활성화
US9620105B2 (en) 2014-05-15 2017-04-11 Apple Inc. Analyzing audio input for efficient speech and music recognition
US10592095B2 (en) 2014-05-23 2020-03-17 Apple Inc. Instantaneous speaking of content on touch devices
US9502031B2 (en) 2014-05-27 2016-11-22 Apple Inc. Method for supporting dynamic grammars in WFST-based ASR
US10289433B2 (en) 2014-05-30 2019-05-14 Apple Inc. Domain specific language for encoding assistant dialog
US9734193B2 (en) 2014-05-30 2017-08-15 Apple Inc. Determining domain salience ranking from ambiguous words in natural speech
US10170123B2 (en) 2014-05-30 2019-01-01 Apple Inc. Intelligent assistant for home automation
US9760559B2 (en) 2014-05-30 2017-09-12 Apple Inc. Predictive text input
WO2015184186A1 (fr) 2014-05-30 2015-12-03 Apple Inc. Procédé d'entrée à simple énoncé multi-commande
US9715875B2 (en) 2014-05-30 2017-07-25 Apple Inc. Reducing the need for manual start/end-pointing and trigger phrases
US9633004B2 (en) 2014-05-30 2017-04-25 Apple Inc. Better resolution when referencing to concepts
US9842101B2 (en) 2014-05-30 2017-12-12 Apple Inc. Predictive conversion of language input
US10078631B2 (en) 2014-05-30 2018-09-18 Apple Inc. Entropy-guided text prediction using combined word and character n-gram language models
US9430463B2 (en) 2014-05-30 2016-08-30 Apple Inc. Exemplar-based natural language processing
US9785630B2 (en) 2014-05-30 2017-10-10 Apple Inc. Text prediction using combined word N-gram and unigram language models
US10659851B2 (en) 2014-06-30 2020-05-19 Apple Inc. Real-time digital assistant knowledge updates
US9338493B2 (en) 2014-06-30 2016-05-10 Apple Inc. Intelligent automated assistant for TV user interactions
US10446141B2 (en) 2014-08-28 2019-10-15 Apple Inc. Automatic speech recognition based on user feedback
US9818400B2 (en) 2014-09-11 2017-11-14 Apple Inc. Method and apparatus for discovering trending terms in speech requests
US10789041B2 (en) 2014-09-12 2020-09-29 Apple Inc. Dynamic thresholds for always listening speech trigger
US9606986B2 (en) 2014-09-29 2017-03-28 Apple Inc. Integrated word N-gram and class M-gram language models
US9646609B2 (en) 2014-09-30 2017-05-09 Apple Inc. Caching apparatus for serving phonetic pronunciations
US9668121B2 (en) 2014-09-30 2017-05-30 Apple Inc. Social reminders
US10074360B2 (en) 2014-09-30 2018-09-11 Apple Inc. Providing an indication of the suitability of speech recognition
US10127911B2 (en) 2014-09-30 2018-11-13 Apple Inc. Speaker identification and unsupervised speaker adaptation techniques
US9886432B2 (en) 2014-09-30 2018-02-06 Apple Inc. Parsimonious handling of word inflection via categorical stem + suffix N-gram language models
US10552013B2 (en) 2014-12-02 2020-02-04 Apple Inc. Data detection
US9711141B2 (en) 2014-12-09 2017-07-18 Apple Inc. Disambiguating heteronyms in speech synthesis
US9865280B2 (en) 2015-03-06 2018-01-09 Apple Inc. Structured dictation using intelligent automated assistants
US9721566B2 (en) 2015-03-08 2017-08-01 Apple Inc. Competing devices responding to voice triggers
US10567477B2 (en) 2015-03-08 2020-02-18 Apple Inc. Virtual assistant continuity
US9886953B2 (en) 2015-03-08 2018-02-06 Apple Inc. Virtual assistant activation
US9899019B2 (en) 2015-03-18 2018-02-20 Apple Inc. Systems and methods for structured stem and suffix language models
US9842105B2 (en) 2015-04-16 2017-12-12 Apple Inc. Parsimonious continuous-space phrase representations for natural language processing
US10083688B2 (en) 2015-05-27 2018-09-25 Apple Inc. Device voice control for selecting a displayed affordance
US10127220B2 (en) 2015-06-04 2018-11-13 Apple Inc. Language identification from short strings
US10101822B2 (en) 2015-06-05 2018-10-16 Apple Inc. Language input correction
US9578173B2 (en) 2015-06-05 2017-02-21 Apple Inc. Virtual assistant aided communication with 3rd party service in a communication session
US10255907B2 (en) 2015-06-07 2019-04-09 Apple Inc. Automatic accent detection using acoustic models
US11025565B2 (en) 2015-06-07 2021-06-01 Apple Inc. Personalized prediction of responses for instant messaging
US10186254B2 (en) 2015-06-07 2019-01-22 Apple Inc. Context-based endpoint detection
US10671428B2 (en) 2015-09-08 2020-06-02 Apple Inc. Distributed personal assistant
US10747498B2 (en) 2015-09-08 2020-08-18 Apple Inc. Zero latency digital assistant
US9697820B2 (en) 2015-09-24 2017-07-04 Apple Inc. Unit-selection text-to-speech synthesis using concatenation-sensitive neural networks
US11010550B2 (en) 2015-09-29 2021-05-18 Apple Inc. Unified language modeling framework for word prediction, auto-completion and auto-correction
US10366158B2 (en) 2015-09-29 2019-07-30 Apple Inc. Efficient word encoding for recurrent neural network language models
US11587559B2 (en) 2015-09-30 2023-02-21 Apple Inc. Intelligent device identification
US10691473B2 (en) 2015-11-06 2020-06-23 Apple Inc. Intelligent automated assistant in a messaging environment
US10049668B2 (en) 2015-12-02 2018-08-14 Apple Inc. Applying neural network language models to weighted finite state transducers for automatic speech recognition
US10223066B2 (en) 2015-12-23 2019-03-05 Apple Inc. Proactive assistance based on dialog communication between devices
US10446143B2 (en) 2016-03-14 2019-10-15 Apple Inc. Identification of voice inputs providing credentials
US9934775B2 (en) 2016-05-26 2018-04-03 Apple Inc. Unit-selection text-to-speech synthesis based on predicted concatenation parameters
US9972304B2 (en) 2016-06-03 2018-05-15 Apple Inc. Privacy preserving distributed evaluation framework for embedded personalized systems
US10249300B2 (en) 2016-06-06 2019-04-02 Apple Inc. Intelligent list reading
US10049663B2 (en) 2016-06-08 2018-08-14 Apple, Inc. Intelligent automated assistant for media exploration
DK179588B1 (en) 2016-06-09 2019-02-22 Apple Inc. INTELLIGENT AUTOMATED ASSISTANT IN A HOME ENVIRONMENT
US10586535B2 (en) 2016-06-10 2020-03-10 Apple Inc. Intelligent digital assistant in a multi-tasking environment
US10490187B2 (en) 2016-06-10 2019-11-26 Apple Inc. Digital assistant providing automated status report
US10192552B2 (en) 2016-06-10 2019-01-29 Apple Inc. Digital assistant providing whispered speech
US10509862B2 (en) 2016-06-10 2019-12-17 Apple Inc. Dynamic phrase expansion of language input
US10067938B2 (en) 2016-06-10 2018-09-04 Apple Inc. Multilingual word prediction
DK201670540A1 (en) 2016-06-11 2018-01-08 Apple Inc Application integration with a digital assistant
DK179343B1 (en) 2016-06-11 2018-05-14 Apple Inc Intelligent task discovery
DK179415B1 (en) 2016-06-11 2018-06-14 Apple Inc Intelligent device arbitration and control
DK179049B1 (en) 2016-06-11 2017-09-18 Apple Inc Data driven natural language event detection and classification
US10043516B2 (en) 2016-09-23 2018-08-07 Apple Inc. Intelligent automated assistant
US10593346B2 (en) 2016-12-22 2020-03-17 Apple Inc. Rank-reduced token representation for automatic speech recognition
DK201770439A1 (en) 2017-05-11 2018-12-13 Apple Inc. Offline personal assistant
DK179745B1 (en) 2017-05-12 2019-05-01 Apple Inc. SYNCHRONIZATION AND TASK DELEGATION OF A DIGITAL ASSISTANT
DK179496B1 (en) 2017-05-12 2019-01-15 Apple Inc. USER-SPECIFIC Acoustic Models
DK201770431A1 (en) 2017-05-15 2018-12-20 Apple Inc. Optimizing dialogue policy decisions for digital assistants using implicit feedback
DK201770432A1 (en) 2017-05-15 2018-12-21 Apple Inc. Hierarchical belief states for digital assistants
DK179560B1 (en) 2017-05-16 2019-02-18 Apple Inc. FAR-FIELD EXTENSION FOR DIGITAL ASSISTANT SERVICES

Family Cites Families (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3797037A (en) * 1972-06-06 1974-03-12 Ibm Sentence oriented dictation system featuring random accessing of information in a preferred sequence under control of stored codes
DE3104551C2 (de) * 1981-02-10 1982-10-21 Neumann Elektronik GmbH, 4330 Mülheim Elektronischer Textgeber zur Abgabe von Kurztexten
DE3642929A1 (de) * 1986-12-16 1988-06-23 Siemens Ag Verfahren zur natuerlich klingenden sprachausgabe
US4908867A (en) * 1987-11-19 1990-03-13 British Telecommunications Public Limited Company Speech synthesis
JPH0477962A (ja) * 1990-07-19 1992-03-12 Sanyo Electric Co Ltd 機械翻訳装置
CA2051135C (fr) * 1991-09-11 1996-05-07 Kim D. Letkeman Dictionnaire de langage comprime
CA2119397C (fr) * 1993-03-19 2007-10-02 Kim E.A. Silverman Synthese vocale automatique utilisant un traitement prosodique, une epellation et un debit d'enonciation du texte ameliores
US5664060A (en) * 1994-01-25 1997-09-02 Information Storage Devices Message management methods and apparatus
DE19518504C2 (de) * 1994-10-26 1998-08-20 United Microelectronics Corp Dynamisch programmierbare Ansage-Einrichtung
GB2296846A (en) * 1995-01-07 1996-07-10 Ibm Synthesising speech from text
US5832434A (en) * 1995-05-26 1998-11-03 Apple Computer, Inc. Method and apparatus for automatic assignment of duration values for synthetic speech
JP3050832B2 (ja) * 1996-05-15 2000-06-12 株式会社エイ・ティ・アール音声翻訳通信研究所 自然発話音声波形信号接続型音声合成装置
JPH1097268A (ja) * 1996-09-24 1998-04-14 Sanyo Electric Co Ltd 音声合成装置
JP3029403B2 (ja) * 1996-11-28 2000-04-04 三菱電機株式会社 文章データ音声変換システム
JPH1138989A (ja) * 1997-07-14 1999-02-12 Toshiba Corp 音声合成装置及び方法
US5913194A (en) * 1997-07-14 1999-06-15 Motorola, Inc. Method, device and system for using statistical information to reduce computation and memory requirements of a neural network based speech synthesis system
JPH1195796A (ja) * 1997-09-16 1999-04-09 Toshiba Corp 音声合成方法
US6047255A (en) * 1997-12-04 2000-04-04 Nortel Networks Corporation Method and system for producing speech signals
JPH11305787A (ja) * 1998-04-22 1999-11-05 Victor Co Of Japan Ltd 音声合成装置
US6266637B1 (en) * 1998-09-11 2001-07-24 International Business Machines Corporation Phrase splicing and variable substitution using a trainable speech synthesizer
US20030028380A1 (en) * 2000-02-02 2003-02-06 Freeland Warwick Peter Speech system

Also Published As

Publication number Publication date
DE10031008A1 (de) 2002-01-10
JP2002055692A (ja) 2002-02-20
EP1168298B1 (fr) 2006-11-29
US20020029139A1 (en) 2002-03-07
ATE347160T1 (de) 2006-12-15
EP1168298A3 (fr) 2002-12-11
DE50111522D1 (de) 2007-01-11
US6757653B2 (en) 2004-06-29

Similar Documents

Publication Publication Date Title
EP1168298B1 (fr) Procédé d'assemblage de messages pour la synthèse de la parole
EP0285221B1 (fr) Procédé pour la reconnaissance de la parole continue
DE69413052T2 (de) Sprachsynthese
EP0533260B1 (fr) Procédé et dispositif pour la reconnaissance des mots prononcés dans un signal de parole
DE3878541T2 (de) Verfahren und einrichtung, um ein markov-modell-referenzmuster von woertern zu erzeugen.
DE69427525T2 (de) Trainingsmethode für ein tts-system, sich daraus ergebendes gerät und methode zur bedienung des gerätes
DE4397100C2 (de) Verfahren zum Erkennen von Sprachsignalen und Spracherkennungssystem mit rekursiver Grammatik mit endlicher Zustandsanzahl
DE60035001T2 (de) Sprachsynthese mit Prosodie-Mustern
DE69814589T2 (de) Spracherkennung unter verwendung mehrerer spracherkenner
DE602004012909T2 (de) Verfahren und Vorrichtung zur Modellierung eines Spracherkennungssystems und zur Schätzung einer Wort-Fehlerrate basierend auf einem Text
DE19942178C1 (de) Verfahren zum Aufbereiten einer Datenbank für die automatische Sprachverarbeitung
EP0797185A2 (fr) Procédé et dispositif pour la reconnaissance de la parole
DE69917960T2 (de) Phonembasierte Sprachsynthese
EP1058235B1 (fr) Procédé de reproduction pour systèmes contrôlés par la voix avec synthèse de la parole basée sur texte
EP0981129A2 (fr) Procédé et système d'execution d'une requête de base de données
DE69837428T2 (de) Speichermedium mit elektronischer Schaltung und mit diesem Speichermedium augestatteter Sprachsynthesizer
EP1282897B1 (fr) Procede pour produire une banque de donnees vocales pour un lexique cible pour l'apprentissage d'un systeme de reconnaissance vocale
EP0417854B1 (fr) Procédé pour la reconnaissance de N suites différentes de mots dans un signal de parole
DE4111781A1 (de) Computersystem zur spracherkennung
EP0677835B1 (fr) Procédé pour rechercher une suite de mots
DE69908106T2 (de) Erweiterung eines spracherkennungswortschatzes unter verwendung von abgeleiteten wörtern
DE69318223T2 (de) Verfahren zur sprachanalyse
DE3215868A1 (de) Verfahren und anordnung zum erkennen der woerter in einer zusammenhaengenden wortkette
DE69329375T2 (de) Verfahren zur Realisierung von Tonkurven für Sprachnachrichten und Verfahren zur Sprachsynthese und Einrichtung zu seiner Anwendung
DE2363590A1 (de) Spracherkennungssystem mit merkmalsfolgekodierung

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE TR

AX Request for extension of the european patent

Free format text: AL;LT;LV;MK;RO;SI

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: NOKIA CORPORATION

PUAL Search report despatched

Free format text: ORIGINAL CODE: 0009013

AK Designated contracting states

Kind code of ref document: A3

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE TR

AX Request for extension of the european patent

Free format text: AL;LT;LV;MK;RO;SI

17P Request for examination filed

Effective date: 20030610

AKX Designation fees paid

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE TR

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE TR

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061129

Ref country code: IE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061129

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT;WARNING: LAPSES OF ITALIAN PATENTS WITH EFFECTIVE DATE BEFORE 2007 MAY HAVE OCCURRED AT ANY TIME BEFORE 2007. THE CORRECT EFFECTIVE DATE MAY BE DIFFERENT FROM THE ONE RECORDED.

Effective date: 20061129

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

GBT Gb: translation of ep patent filed (gb section 77(6)(a)/1977)

Effective date: 20061129

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

REF Corresponds to:

Ref document number: 50111522

Country of ref document: DE

Date of ref document: 20070111

Kind code of ref document: P

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070312

REG Reference to a national code

Ref country code: SE

Ref legal event code: TRGR

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070430

ET Fr: translation filed
REG Reference to a national code

Ref country code: IE

Ref legal event code: FD4D

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20070830

BERE Be: lapsed

Owner name: NOKIA CORPORATION

Effective date: 20070630

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20070630

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20070630

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20070630

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070301

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20070630

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20070620

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20080603

Year of fee payment: 8

Ref country code: SE

Payment date: 20080609

Year of fee payment: 8

REG Reference to a national code

Ref country code: GB

Ref legal event code: 732E

Free format text: REGISTERED BETWEEN 20090409 AND 20090415

REG Reference to a national code

Ref country code: FR

Ref legal event code: TP

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061129

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20070620

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061129

NLV4 Nl: lapsed or anulled due to non-payment of the annual fee

Effective date: 20100101

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20100101

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20090621

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 50111522

Country of ref document: DE

Representative=s name: VON ROHR PATENTANWAELTE PARTNERSCHAFT MBB, DE

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 16

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20160621

Year of fee payment: 16

Ref country code: GB

Payment date: 20160621

Year of fee payment: 16

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 17

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20170511

Year of fee payment: 17

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 50111522

Country of ref document: DE

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20170620

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180103

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170620

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180630