WO2004100164A1 - Systeme d'affichage d'un texte vocal - Google Patents

Systeme d'affichage d'un texte vocal Download PDF

Info

Publication number
WO2004100164A1
WO2004100164A1 PCT/IB2004/001336 IB2004001336W WO2004100164A1 WO 2004100164 A1 WO2004100164 A1 WO 2004100164A1 IB 2004001336 W IB2004001336 W IB 2004001336W WO 2004100164 A1 WO2004100164 A1 WO 2004100164A1
Authority
WO
WIPO (PCT)
Prior art keywords
audio
descriptor
track
phoneme
time
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/IB2004/001336
Other languages
English (en)
Inventor
William Mutual
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
UNISAY Sdn Bhd
Original Assignee
UNISAY Sdn Bhd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by UNISAY Sdn Bhd filed Critical UNISAY Sdn Bhd
Publication of WO2004100164A1 publication Critical patent/WO2004100164A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G11INFORMATION STORAGE
    • G11BINFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/19Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier
    • G11B27/28Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/222Studio circuitry; Studio devices; Studio equipment
    • H04N5/262Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/278Subtitling
    • GPHYSICS
    • G11INFORMATION STORAGE
    • G11BINFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B2220/00Record carriers by type
    • G11B2220/20Disc-shaped record carriers
    • G11B2220/25Disc-shaped record carriers characterised in that the disc is based on a specific recording technology
    • G11B2220/2537Optical discs
    • G11B2220/2562DVDs [digital versatile discs]; Digital video discs; MMCDs; HDCDs

Definitions

  • the present invention relates to a voice script system and refers particularly, though not exclusively, to such a voice process system to assist with post-production voice processes of an audio/video presentation.
  • the normal post-production processing costs about one third of the total budget. Re-recording the audio track is about one third of that cost. To facilitate the post-production audio processing may therefore not only reduce costs, but may also reduce the time taken, and thus allow the audio/video presentation to be on the market earlier.
  • the audio/video presentation may be for a motion picture, a television broadcast, streaming video, semi-professional or home video particularly digital video, or portable media including DVD, VCD, and so forth.
  • machine (and its grammatical variants) is to be taken as including a reference to one or more of: server, desktop computer, personal computer, laptop computer, notebook computer, tablet computer, and personal digital assistant.
  • a voice script system for providing a scrolling text on video content including the steps:
  • the time-stamped text data may be time-stamped by phrase start. Between steps (a) and (b) there may be included a further process step of obtaining a machine translation of the phonemes of the script. After the machine translation is obtained, manual correction for slang and idiomatic language may be performed. The text may be displayed on the scrolling track in a plurality of lines scrolling from the bottom. The scrolling may be by phrase.
  • time-stamped text may be time-stamped by word start.
  • the text may be displayed on the scrolling track in a single line scrolling from left to right.
  • the text alignment may be performed by using a punctuation filter to locate the beginnings and ends of sentences.
  • the first process step of producing time-stamped text data may be conducted using a track reading process.
  • the track reading process may include the steps: (a) breaking the script into phonemes;
  • the phonemes may be obtained using a text-to-speech process, and the script may be broken into words before the phonemes are obtained. Any unrecognised phrases in the audio track after the recognition process may be subsequently processed in the same manner with reset threshold and accuracy of the speech recognition process. Prior to the speech recognition process, parameters and next phrase logic of the speech recognition process may be set.
  • the audio track may be analysed to produce a time location for words in the audio track.
  • the speech recognition process on the script may also produce a description of words and phoneme length timings.
  • the description of words, phonemes and phoneme length timings may be further processed with a timer input to produce a timed word/phoneme list.
  • the timer input may be obtained from the audio track.
  • the audio track may be part of an audio/video input and the audio/video input may be first converted to be PEG1/MPEG2 compliant before separating the audio track from the audio/video input.
  • the timed phonemes may be subjected to further processing to provide a speaker phoneme set.
  • the speech recognition process may also produce marked-up phonemes.
  • the further processing may be a phoneme analysis.
  • the phoneme analysis may be an MPEG 7 word/phoneme analysis using sound spectrum and sound waveforms.
  • the audio track may also be input into an MPEG 7 audio low level descriptors extraction process and an MPEG 7 emotive analysis for extraction of emotion-related descriptors.
  • a computer usable medium comprising a computer program code that is configured to cause one or more processors to execute one or more functions to perform the steps and functions as described above; as well as to a video having a scrolling text prepared by the above process.
  • Figure 1 is an overall chart of a preferred embodiment of the present invention
  • Figure 2 is a flow chart for the initialising process of the track reading process of Figure 1
  • Figure 3 is a flow chart for a preferred process for phoneme extraction
  • Figure 4 is a flow chart for a preferred process for timed phoneme extraction.
  • Figure 5 is a flow chart for audio low level descriptors extraction
  • Figure 6 is a flow chart of emotive analysis
  • Figure 7 is a flow chart of word/phoneme analysis.
  • FIG. 1 there is illustrated the overall process, and the initial process steps.
  • a machine of a user logs on to the relevant web page to initiate the process.
  • the process may be performed on a stand-alone machine.
  • the feature selection is made at 98. It is assumed for the following description that the process is for a new feature and thus the entire process is required.
  • the audio, video and text files corresponding to the audio files, requiring post-processing are selected and input to the server. If a multi-track audio dubbing is to be performed, the input audio track is supplemented by a multi-track voice input 96. In that case, auxiliary text files 94 for the multi-track audio may be required.
  • time-stamped text data 92 may be sent to one or both of two streams 88 and 86.
  • a first stream 88 is used when the server has been advised by the user's machine that the text is to be in a different, nominated language.
  • the time-stamped text data is converted to XML at 84 and time-stamped at the start of each phrase. Any translation required is then performed at 82 using a machine translation.
  • the translated text from 82 may be checked for idiomatic language, slang expressions, and corrections at 80.
  • each phrase of the translated text is translated to bitmap graphics on two lines at the bottom of each frame that constitute the subtitle track.
  • bitmap graphics of the text scroll from the bottom to the top they can easily be read by the actors.
  • the actors can read the complete phrase at the time the phrase commences and can read the phrase knowing the video context so they can include all necessary enunciation, emotions and emphasis required to match the video and thus have correct audio/lip sync.
  • multiple tracks may be separately displayed. This may be in a side-by-side relationship or in consecutive lines on the scrolling text in the same manner as they would appear on a printed script.
  • the actors would be reading the top line of the displayed script, if another actor has the following line, their cue time is as long as it takes the first actor to read their line.
  • a background colour and the font, font size and font colour can be selected at 76.
  • the text as displayed is time-stamped, it will be in sync with the video and will scroll at an appropriate speed to keep pace with the audio and video. This has the script being displayed in what may be termed teleprompter mode (74).
  • the output may be a standard video file that is able to be imported in to digital audio workstations such as "Pro Tools”. It also enables the output to be used as a synchronised voice prompter.
  • the stream 86 is generally the same except the time stamp is by word rather than phrase (72) - all other aspects are the same as 84).
  • the word text scrolls left-to-right (70).
  • the opacity of the display may also need to be selected in 68 - all other selections are the same as 76.
  • the video has the text corresponding to the audio ( and matching the video) scrolling left-to-right it will be in sync when it crosses an imaginary line in the centre of the display at 66.
  • the process is initiated.
  • a preliminary check is made to ensure all parameters are correct. If not, at 111 an incorrect parameter message is generated and sent to the user's machine.
  • the five input parameters (a) to (e) can then be reselected, reset or the error corrected. If all parameters are correct, at 112 all data is sent for processing and the input stage of track reading ends.
  • the process commences with a detailed check at 113 to determine if all parameters are correct. This is a more complete test than that conducted at 110 and is also to ensure the transmission from the sender's machine to the server has been error-free. If there is an error, an error message is generated at 114, sent to the sending machine, and all processing stops.
  • This script is processed at 115 to break it down into individual words. If there is an error, an error message is generated at 114, sent to the sending machine, and all processing stops.
  • the words of the script are broken into phonemes using a text-to-speech engine such as, for example, "SAPI 5.1” as available from Microsoft Corp, or the "Viavoice” as available from IBM Corp. If there is an error, error message is generated at 114, sent to the sending machine, and all processing stops.
  • a text-to-speech engine such as, for example, "SAPI 5.1” as available from Microsoft Corp, or the "Viavoice” as available from IBM Corp.
  • a speech recognition engine is used at 117 to analyse the audio/voice track 105.
  • the parameters for the analysis are first set, and the logic for the next phrase of the audio/voice track 105 is also set.
  • a suitable speech recognition engine would be, for example, "SAPI 5.1" as available from. Microsoft Corp, or the "Viavoice” as available from IBM Corp.. If at the commencement of the audio track and script, the first phrase of the script is generated from 116. For subsequent processing, the next phrase of the script is generated. At the end of the script, there will be nothing left so the processing is diverted at 119 to the next major processing stage. Otherwise, it proceeds to 120 where the present voice track position is set.
  • the phonemes used come from the script.
  • the phoneme timing, and time location in the audio track of the phoneme come from the audio track.
  • the phonemes can be timed for duration and location with a relatively high degree of accuracy.
  • the overall length of the audio track is known and this will be the overall duration of the script. Therefore, it is possible to estimate an approximate time in the duration of the script where a particular phrase might appear.
  • the current time location is recognised from the audio track as is the current script phrase as appearing on the audio track using speech recognition.
  • a "most likely position" for that script phrase can be determined and thus its likely time location on the audio track.
  • Phonemes missing from the script may be obtained from the audio track using the voice properties of the speaker as obtained from the closely matched complete set, or by synthesising the missing phonemes that don't have a close match. Manual intervention may be required to correct errors depending on the subject of the text, number of slang expressions used, and the diction on the audio track.
  • the word times are then updated in 122 using the results of the recognition process, and the next phrase of the script prepared. The process then loops back to 118 and repeats until there is nothing left in the script.
  • the process is diverted at 119 and passes to a second stage of processing.
  • any unrecognised phrases in the script are re-examined. If there are no unrecognised phrases, the output from 122 is directly to 129 where the timed word/phoneme list is generated.
  • the speech recognition engine threshold and accuracy are set. The first (or next for subsequent phrases) phrase of the script is generated. If there are no more unrecognised phrases, at 125 the process is diverted to 129 to generate the timed word/phoneme list.
  • the voice track position is set at 126, as before.
  • the recognition process of 121 is re-performed at 127 but with the new thresholds and accuracy. Again, at 128 the recogniser results are used to update the word times, and the next unrecognised phrase prepared. The process loops back to 124.
  • Audio 167 is input at 163 for processing in accordance with the MPEG 7 sound classification model audio classifier. This extracts and outputs at 171 the timed sound classifications.
  • Sound classifications include non-word sounds such as, for example, inhalation, laughter, natural sounds, noises, background sounds and noises, music, and so forth.
  • Audio 166 is input at 117 as described above for speech recognition analysis preferably using the Hidden Markov Method ("HMM") for raw recognition and timing.
  • the output 172 is an aligned raw phoneme list.
  • the text 165 is input for processing as described above for text-to-speech (phoneme) and the output 173 is a normally timed phonetic text representation.
  • the audio and text are input at 164 into process 121 as is described above for phrase- segmented, time-assisted, speech recognition.
  • the output 174 is a verified, aligned word/phoneme list.
  • Outputs 171 , 172 and 173 are input to process 168 for initial text alignment for words and the sound classifications.
  • the output 175 from process 168 is estimated word timings and partial phoneme timings. This is also input into 121 for its output 174.
  • the output 175 is also input to process 169, as is output 174, for final text alignment (including some or all of the sound classifications).
  • the output 176 from 169 is a complete timed/word/phoneme list.
  • FIG. 4 there is shown the overall track reader processing of an audio/video input 8 as received in the machine operating the system of the present invention.
  • input 8 there is an audio/video input and a script corresponding to the audio track of the audio/video input as a text input in digital form.
  • the audio/video input is preferably in source language. This is process 96 of Figure 1; and steps 104 to 112 of Figure 3.
  • the audio/video input is split into two processing streams 4, 6.
  • stream 4 the audio/video 10 is first transcoded at 12 to MPEG1/MPEG2, if not already in the appropriate format. If in the appropriate format, transcoding is not necessary. Audio/video separation takes place and the video is output at 14 and by-passes all subsequent processing at this stage.
  • the script undergoes a track reading process at 18. These are process steps 113 to 122 of Figure 3. Also input at 18 is a lime input to enable each of the phrases, words and phonemes to be timed for likely duration.
  • the time input may be in seconds down to a preferred level of three decimal places (i.e. milliseconds).
  • the output from process 18 is timed phonemes. It may also have marked-up phonemes and descriptions of the words. Preferably, all output to 20 is in MPEG 7 representation.
  • the phoneme timing are completed in accordance with process steps 123 to 128 of Figure 3 to have a speaker phoneme set that is as complete as possible.
  • the output 30 from process 20 is passed to data storage 22 for storage and to be used later in other processes.
  • the audio output 32 from process 12, and the outputs from process steps 18, 20, are all passed for three parallel enhancement process steps.
  • the audio output 32 from process 12 is passed for MPEG 7 audio low level descriptors ("LLD") extraction.
  • LLDs may include such characteristics as the spectrum of the sound, silence, and so forth [any others?].
  • the LLD analysis may proceed in parallel to the track reading process 18, or may be subsequent to that process. If subsequent, the output from process 18 may be also input to process 24.
  • the audio output 32 from process 12 and the result 34 of the track reading process 18 are both input to the second enhancement step 26 .
  • MPEG 7 emotive analysis is performed by extracting of the audio using emotion-related descriptors.
  • An emotive analysis based on MPEG 7 LLDs is then performed.
  • the third enhancement step 28 has as its input the text and phoneme timing output 30 of process 20, the audio output 32 from process 12, and a timer 36 .
  • the timer 36 may be in seconds down to a preferred level of three decimal places (i.e. milliseconds) and may start from the commencement of the audio track, or otherwise as prescribed or required.
  • the audio track 32 is combined with the output 30 and the timer 36 and analysed and described into a timed word/phoneme list.
  • the output 30 gives the phonemes and their likely duration.
  • the audio track 32 and the timer 36 give the time location of the phonemes By matching the timed phonemes and the audio track 32, based on a likely match, a timed word/phoneme list can be prepared.
  • This is output 38 from process 28 and stored in data storage 22 for later use.
  • the outputs 40, 42 from processes 26, 24 respectively are also stored in data storage 22 for subsequent use. The processing ends.
  • Figure 5 is a flow chart of the audio LLD extraction process 24 of Figure 4.
  • the spectrums, and logarithm of the spectrum are calculated at 130.
  • Some or all of a large number of low level descriptors are then determined in either parallel (as shown) or sequentially. These include, but are not limited to:
  • audio amplitude changes detection (loudness change) is measured along each phoneme/word as a function of time;
  • rhythm detection auto-correlation of the audio power
  • spectral slope along each phoneme/word is measured along each phoneme/word as a function of time.
  • the outputs of all four are then combined at 154 to create prosodic descriptors.
  • the final enhancement process 28 of Figure 4 is illustrated in Figure 7 - the phoneme/word analysis. This may be performed using HLDs.
  • the phoneme segmentation of the audio track 155 and the diphone segmentation of the audio track 156 are combined and at 157 the phoneme and diphone data is extracted and subjected to post-processing.
  • Post-processing may include one or more of normalization of amplitude via peak or RMS methodologies, noise reduction, and frequency filtering (either adaptive or manually set).
  • the phoneme/diphone data descriptors are then encoded and passed for storage as well as for input into the translation process 90.
  • the present invention also extends to a computer usable medium comprising a computer program code that is configured to cause one or more processors to execute one or more functions to perform the steps and functions described above; as well as to a video having subtitles prepared by the above system.
  • the translation is preferably conducted using a machine translation.
  • Suitable machine translation engines include Systrans 4.0.
  • the translation may be into any suitable language in any format - using the Roman alphabet, Roman numerals, Arabic numerals, and any regional or national alphabets, scripts and symbols.
  • the present invention extends to all features disclosed both individually, and in all possible permutations and combinations.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Machine Translation (AREA)

Abstract

Système d'affichage d'un texte en défilement en superposition sur des images vidéo, comprenant les étapes suivantes : utilisation d'une piste audio correspondant aux images vidéo et d'un texte correspondant à la piste audio afin de produire des données textuelles horodatées ; alignement des données textuelles horodatées sur la piste audio et les images vidéo ; et traduction en graphique en mode point des données textuelles horodatées et alignées, et ce sur une piste en défilement indiquant les images vidéo. Les données textuelles sont horodatées en fonction du début d'une phrase ou d'un mot. Le texte peut être affiché sur la piste en défilement selon plusieurs lignes en défilement depuis le bas, ou alors sur la piste en défilement selon une ligne unique en défilement depuis la gauche et vers la droite.
PCT/IB2004/001336 2003-04-18 2004-04-15 Systeme d'affichage d'un texte vocal Ceased WO2004100164A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
MYPI20031469 2003-04-18
MYPI20031469 2003-04-18

Publications (1)

Publication Number Publication Date
WO2004100164A1 true WO2004100164A1 (fr) 2004-11-18

Family

ID=33476080

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2004/001336 Ceased WO2004100164A1 (fr) 2003-04-18 2004-04-15 Systeme d'affichage d'un texte vocal

Country Status (1)

Country Link
WO (1) WO2004100164A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007129004A1 (fr) * 2006-04-12 2007-11-15 Sysmedia Ltd Système de message à commande vocale
CN116506652A (zh) * 2023-04-23 2023-07-28 深圳新成思达教育科技有限公司 一种技能实操示范视频脚本自动生成系统
US11763099B1 (en) 2022-04-27 2023-09-19 VoyagerX, Inc. Providing translated subtitle for video content

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5649060A (en) * 1993-10-18 1997-07-15 International Business Machines Corporation Automatic indexing and aligning of audio and text using speech recognition
EP0896467A1 (fr) * 1997-08-06 1999-02-10 British Broadcasting Corporation Méthode et dispositif d'affichage de textes parlés pour utilisation pendant la génération des signaux de télévision
WO2001095631A2 (fr) * 2000-06-09 2001-12-13 British Broadcasting Corporation Production de sous-titres ou de legendes pour images animees
US6490550B1 (en) * 1998-11-30 2002-12-03 Ericsson Inc. System and method for IP-based communication transmitting speech and speech-generated text

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5649060A (en) * 1993-10-18 1997-07-15 International Business Machines Corporation Automatic indexing and aligning of audio and text using speech recognition
EP0896467A1 (fr) * 1997-08-06 1999-02-10 British Broadcasting Corporation Méthode et dispositif d'affichage de textes parlés pour utilisation pendant la génération des signaux de télévision
US6490550B1 (en) * 1998-11-30 2002-12-03 Ericsson Inc. System and method for IP-based communication transmitting speech and speech-generated text
WO2001095631A2 (fr) * 2000-06-09 2001-12-13 British Broadcasting Corporation Production de sous-titres ou de legendes pour images animees

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
WATANABE K ET AL: "Automatic caption generation for video data. Time alignment between caption and acoustic signal", 13 September 1999, MULTIMEDIA SIGNAL PROCESSING, 1999 IEEE 3RD WORKSHOP ON COPENHAGEN, DENMARK 13-15 SEPT. 1999, PISCATAWAY, NJ, USA,IEEE, US, PAGE(S) 65-70, ISBN: 0-7803-5610-1, XP010351716 *

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007129004A1 (fr) * 2006-04-12 2007-11-15 Sysmedia Ltd Système de message à commande vocale
GB2437782B (en) * 2006-04-12 2010-06-23 Sysmedia Ltd Speech driven prompt system
US11763099B1 (en) 2022-04-27 2023-09-19 VoyagerX, Inc. Providing translated subtitle for video content
US11770590B1 (en) 2022-04-27 2023-09-26 VoyagerX, Inc. Providing subtitle for video content in spoken language
US11947924B2 (en) 2022-04-27 2024-04-02 VoyagerX, Inc. Providing translated subtitle for video content
US12099815B2 (en) 2022-04-27 2024-09-24 VoyagerX, Inc. Providing subtitle for video content in spoken language
US12541656B2 (en) 2022-04-27 2026-02-03 VoyagerX, Inc. Providing subtitle for video content
CN116506652A (zh) * 2023-04-23 2023-07-28 深圳新成思达教育科技有限公司 一种技能实操示范视频脚本自动生成系统

Similar Documents

Publication Publication Date Title
US7676373B2 (en) Displaying text of speech in synchronization with the speech
US9552807B2 (en) Method, apparatus and system for regenerating voice intonation in automatically dubbed videos
CN108780643B (zh) 自动配音方法和装置
US6505153B1 (en) Efficient method for producing off-line closed captions
US20080195386A1 (en) Method and a Device For Performing an Automatic Dubbing on a Multimedia Signal
WO2020181133A1 (fr) Système et procédé de doublage multilingue simultané de programmes audio-vidéo
US20060136226A1 (en) System and method for creating artificial TV news programs
CN105378830A (zh) 音频数据的处理
CN111986656A (zh) 教学视频自动字幕处理方法与系统
CN105898556A (zh) 一种外挂字幕的自动同步方法及装置
US20250118336A1 (en) Automatic Dubbing: Methods and Apparatuses
CN114842858A (zh) 一种音频处理方法、装置、电子设备及存储介质
CN117278699A (zh) 视频生成方法、装置、计算机设备及存储介质
WO2004093078A1 (fr) Procede pour ajouter des sous-titres a un contenu video
CN119314488A (zh) 一种基于生成式ai的自动化配音方法及系统
CN119400206A (zh) 一种基于录播课程自动生成多语种慕课的方法及系统
KR101618777B1 (ko) 파일 업로드 후 텍스트를 추출하여 영상 또는 음성간 동기화시키는 서버 및 그 방법
JP4210723B2 (ja) 自動字幕番組制作システム
WO2004100164A1 (fr) Systeme d'affichage d'un texte vocal
CN119091895A (zh) 配音方法及系统
Artioli et al. Generative AI for Realistic Voice Dubbing Across Languages
CN119255009A (zh) 一种基于ai和语音鼠标的实时字幕生成方法及其系统
Penyameen et al. AI-Based Automated Subtitle Generation System for Multilingual Video Transcription and Embedding
JP3969570B2 (ja) 逐次自動字幕制作処理システム
CN121121616B (zh) 影视剧台本提取方法、装置、存储介质及计算机设备

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): BW GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 69(1) EPC DATED 15.02.2006

122 Ep: pct application non-entry in european phase