EP2362388B1 - Verfahren und Vorrichtung zur Beurteilung eines möglicherweise mit einem Wasserzeichen versehenen Signals, das über einen akustischen Pfad in einem mobilen Empfänger empfangen wird - Google Patents

Verfahren und Vorrichtung zur Beurteilung eines möglicherweise mit einem Wasserzeichen versehenen Signals, das über einen akustischen Pfad in einem mobilen Empfänger empfangen wird Download PDF

Info

Publication number
EP2362388B1
EP2362388B1 EP10305181A EP10305181A EP2362388B1 EP 2362388 B1 EP2362388 B1 EP 2362388B1 EP 10305181 A EP10305181 A EP 10305181A EP 10305181 A EP10305181 A EP 10305181A EP 2362388 B1 EP2362388 B1 EP 2362388B1
Authority
EP
European Patent Office
Prior art keywords
frequency
current
signal
data
watermark
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Not-in-force
Application number
EP10305181A
Other languages
English (en)
French (fr)
Other versions
EP2362388A1 (de
Inventor
Ulrich Gries
Peter Georg Baum
Walter Voessing
Michael Arnold
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Thomson Licensing SAS
Original Assignee
Thomson Licensing SAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Thomson Licensing SAS filed Critical Thomson Licensing SAS
Priority to EP10305181A priority Critical patent/EP2362388B1/de
Publication of EP2362388A1 publication Critical patent/EP2362388A1/de
Application granted granted Critical
Publication of EP2362388B1 publication Critical patent/EP2362388B1/de
Not-in-force legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/018—Audio watermarking, i.e. embedding inaudible data in the audio signal

Definitions

  • the invention relates to a method and to an apparatus for evaluating a possibly watermarked signal that is received via an acoustic path in a mobile watermark data receiver, wherein motion of the mobile receiver causing a Doppler shift of the received signal is taken into account.
  • audio watermarking can be used.
  • a TV or radio broadcaster will embed a unique station ID inside the audio signal transmitted by the transmitter stations.
  • Panelists use a device including a microphone, an audio WM detector and a storage and/or transmission unit for monitoring via the microphone the environmental sound, i.e. the acoustic sound signal carrying the WM and being transmitted by one or more loudspeakers, plus any kind of 'noise' signal occurring in the shop or airport hall etc. If a TV set or radio is turned on, the embedded WM is detected and the station ID is decoded and stored in the storage unit, or is directly transmitted by the WM signal receiver device transmission unit to the audience measurement company.
  • WM embedded watermark
  • Document EP 2 083 418 discloses watermark detection in a signal received via an acoustic path.
  • the correct sampling frequency of the incoming watermarked signal is determined by trying out a number of candidates in a frequency range.
  • a problem to be solved by the invention is to achieve faster and/or more reliable watermark detection for a mobile WM signal receiver.
  • This problem is solved by the method disclosed in claim 1.
  • An apparatus that utilises this method is disclosed in claim 2.
  • one or more accelerometers and/or a Galileo or GPS system signal receiver are integrated into the mobile WM signal detection device. Inside the WM signal detection device the accelerometer data signals and/or the Galileo or GPS system location data signals are used for calculating an approximation of the device speed and the real Doppler shift caused by the motion of the device.
  • the motion speed approximation value or values are used as an a-priori knowledge in the WM detection processing for reducing the number of candidate frequencies searched and thereby the search time and/or for improving the WM signal detection results and/or for reducing the CPU power consumption.
  • advantages of the inventive watermark signal detection are improved WM detection rate and increased battery life time.
  • Mobile detectors will probably include a GPS or Galileo device anyway for measuring the location of the content exposure. Then advantageously the cost of the WM receiver device will increase only by adding accelerometers and corresponding control software, which are quite cheap.
  • the inventive method is suited for evaluating a possibly watermarked signal that is received via an acoustic path in a mobile receiver device, wherein motion of said receiver device causes a Doppler shift of the received signal, and wherein for the decoding or demodulation of said signal a current decoder or demodulation frequency needs to be determined, said method including the steps:
  • the inventive apparatus is suited for evaluating a possibly watermarked signal that is received via an acoustic path in a mobile receiver device, wherein motion of said receiver device causes a Doppler shift of the received signal, and wherein for the decoding or demodulation of said signal a current decoder or demodulation frequency needs to be determined, said apparatus including:
  • the whole possible frequency range has to be searched in order to determine the correct frequency shift introduced by the Doppler shift. Therefore the maximum allowable speed or acceleration of the device needs to be specified. If the current speed is outside that specification, the WM will not be detected.
  • the maximum speed allowed in the direction to or from the acoustic sound signal transmitter i.e. loudspeaker
  • the acoustic sound signal transmitter i.e. loudspeaker
  • 300m/s being the approximated speed of acoustic waves, this will result in a Doppler shift of about ⁇ 1%.
  • a microphone MIC receives acoustic signals from a loudspeaker LSP.
  • the microphone output signal passes through an A/D converter and an optional spectral whitening step or stage SPW to a watermark demodulator or decoder stage or step WMD, which carries out - for a given frequency - a correlation with one or more reference bit sequences, and which is controlled by a frequency search step or stage FRS that receives motion speed or Doppler shift data from motion speed calculator or step MSPC.
  • a correlation result shows one or more suitable correlation result peaks, a watermark signal bit corresponding to the applied reference bit sequence has been determined.
  • the demodulator/decoder starts operation with a first candidate frequency, tries to decode a watermark. If successful, it is assumed that that candidate frequency is the correct frequency. If not successful, the following or a different candidate frequency is checked, and so on. In case no candidate frequency out of the frequency search range was successful it is assumed that at present, i.e. for a current WM frame or signal section of the received input signal, no watermark is present within the received input signal.
  • the watermark demodulator or decoder stage or step WMD sends a frequency correct signal FCS to frequency search step or stage FRS for signalling that the current frequency is the correct one.
  • the current speed of the WM signal receiver device is calculated in the motion speed calculator or step MSPC. From the acceleration values the current speed can be calculated by integrating them over a short time period, and based on the previous speed value. When switching on the mobile device the initial speed value can be assumed to be zero. In particular, if no Galileo or GPS signal is evaluated and no positive or negative acceleration is determined for a predetermined time period, the current speed value can be assumed to be zero because otherwise in the meantime the person carrying the mobile device would have left the sound field of the loudspeaker LSP.
  • the Galileo/GPS receiver can also take into account the current spatial orientation of the WM device and thereby automatically determine the share of the motion speed that is directed to, or away from, the loudspeaker LSP, following a corresponding initialisation taking into account the loudspeaker position.
  • the loudspeaker coordinates can be stored upon placing the WM device nearby the loudspeaker for a short time period, or by automatic transmission of the current loudspeaker coordinates, or by pre-stored loudspeaker coordinates, depending on the current application of the WM signal receiver device.
  • accelerometers only are used (three one-dimensional accelerometers or two two-dimensional accelerometers or a single three-dimensional accelerometer), the position of the loudspeaker is unknown.
  • the current maximum possible Doppler shift can be calculated for the frequency search in frequency search step or stage FRS.
  • This estimated Doppler shift data will normally not correspond to the real Doppler shift data because the maximum shift is reached only if the movement is directly towards or away from the loudspeaker. If the movement is more tangential to the loudspeaker position, the real Doppler shift is lower, depending on the distance to the loudspeaker.
  • the knowledge of the current maximum Doppler shift will already significantly reduce the frequency range to be searched because the maximum allowable frequency shift (i.e. motion speed) according to the device specification is rarely reached in real life. If, for example for the person carrying the WM signal receiver device a walking speed of 1m/s is detected, a maximum Doppler shift of 0.3% is calculated and 13 candidate frequencies only instead of 41 candidate frequencies need to be tested.
  • the spatial orientation of the WM signal receiver device with respect to the loudspeaker can be determined (in a spatial orientation detector SPOD that provides the motion speed calculator MSPC with corresponding data.
  • a spatial orientation detector SPOD that provides the motion speed calculator MSPC with corresponding data.
  • Such determination of the spatial orientation of the WM signal receiver device can be achieved by using an additional optical signal received from (the location of) the loudspeaker LSP, or by using several (directional) microphones in the WM device for determining from which average or main direction the sound is received by these microphones.
  • a repeated or periodic or continuous determination of the spatial orientation of the WM signal receiver device can be carried out.
  • WM signal receiver device With or without loudspeaker location determination, even more can be gained in case the WM signal receiver device is not moving and only non-watermarked sound is received. This will happen very frequently in real life, for example if the person carrying the device is sitting somewhere and no TV or radio broadcast signal is playing from loudspeaker LSP, but only some ambient noise is received by microphone MIC.
  • all 41 candidate frequencies are to be tested because the WM detector cannot differentiate between non-detection due to no WM signal being embedded and non-detection due to wrong candidate frequency selection. But in the inventive WM signal receiver device a single frequency only is to be tested, which represents an improvement by a factor of 41.
  • the station or program ID and/or the watermark data can be stored in a memory MEM for a future upload to the audience measurement data company.
  • these data can be transmitted via a transmitter TR to a corresponding audience measurement data receiver that is operated for that company.
  • the calculated speed and acceleration data can also be used for adapting other parameters of the WM detection processing, for example the correlation length.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Position Fixing By Use Of Radio Waves (AREA)
  • Measurement Of Velocity Or Position Using Acoustic Or Ultrasonic Waves (AREA)

Claims (10)

  1. Verfahren zur Bewertung eines möglicherweise mit einem Wasserzeichen versehenen Signals (AWMS), das über einen akustischen Weg (LSP, MIC) in einer mobilen Empfängervorrichtung empfangen (MIC) wird, wobei die Bewegung der Empfängervorrichtung eine Doppler-Verschiebung des empfangenen Signals verursacht, und wobei für die Dekodierung oder Demodulation (WDM) des Signals eine aktuelle Dekodierer- oder Demodulationsfrequenz bestimmt (FRS) werden muss, wobei das Verfahren die Schritte einschließt:
    a) Bestimmen (MWD) auf der Basis einer aktuellen Anwärterfrequenz aus einem gegebenen Frequenzbereich aus einer Korrelation eines aktuellen Signalabschnitts mit einer oder mehreren Referenz-Bitsequenzen, ob wenigstens ein Wasserzeichenbit des Signals (AWMS) in dem aktuellen Signalabschnitt vorhanden ist oder nicht;
    b) falls dies zutrifft, angenommen wird, dass die aktuelle Anwärterfrequenz die richtige Frequenz ist, und wenn dies nicht zutrifft, der Schritt a) unter Verwendung einer anderen Anwärterfrequenz aus dem Frequenzbereich ausgeführt wird;
    c) wenn keine richtige Frequenz nach Ausführen der Schritte a) und b) für alle Anwärterfrequenzen aus dem Frequenzbereich gefunden worden ist, angenommen wird, dass der aktuelle Signalabschnitt kein Wasserzeichen führt, dadurch gekennzeichnet, dass der Bereich der Frequenzsuche (FRS) gemäß den aktuellen maximalen Bewegungsgeschwindigkeitsdaten der Vorrichtung, die eine entsprechend begrenzte Dopplerverschiebung darstellen, begrenzt wird, wobei die aktuellen maximalen Bewegungsgeschwindigkeitsdaten der Vorrichtung für die mobile Empfängervorrichtung von Galilei- oder GPS-Daten (GSR) und/oder von Beschleunigungsmesserdaten (ACC1, ACC2, ACC3) abgeleitet werden.
  2. Vorrichtung zur Bewertung eines möglicherweise mit einem Wasserzeichen versehenen Signals (AWMS), das über einen akustischen Weg (LSP, MIC) in einer mobilen Empfängervorrichtung empfangen wird, wobei die Bewegung der Empfängervorrichtung eine Doppler-Verschiebung des empfangenen Signals verursacht, und wobei für die Dekodierung oder Demodulation (WDM) des Signals eine aktuelle Dekodierer- oder Demodulationsfrequenz bestimmt (FRS) werden muss, wobei die Vorrichtung einschließt:
    - Mittel (MIC, A/D) zum Empfang des Signals (AWMS) über den akustischen Weg und zur Erzeugung eines entsprechenden Datensignals;
    - Mittel (WMD) zum Bestimmen auf der Basis einer aktuellen Anwärterfrequenz aus einem gegebenen Frequenzbereich aus einer Korrelation eines aktuellen Signalabschnitts mit einer oder mehreren Referenz-Bitsequenzen, ob wenigstens ein Wasserzeichenbit des Signals (AWMS) in dem aktuellen Signalabschnitt vorhanden ist oder nicht, wobei,
    wenn dies zutrifft, angenommen wird, dass die aktuelle Anwärterfrequenz die richtige Frequenz ist, und,
    wenn dies nicht zutrifft, die Bestimmungsmittel (WMD) die Wasserzeichenbitbestimmung unter Verwendung einer anderen Anwärterfrequenz aus dem Frequenzbereich ausführen; und wobei, wenn keine richtige Frequenz in den Bestimmungsmitteln (MWD) für alle Anwärterfrequenzen gefunden wird, angenommen wird, dass der aktuelle Signalabschnitt kein Wasserzeichen führt, gekennzeichnet durch:
    - Einen Galileo- oder GPS-Empfänger (GSR) und/oder einen oder mehrere Beschleunigungsmesser (ACC1, ACC2, ACC3);
    - Mittel (MSPC, FRS), um für die mobile Empfängervorrichtung aus entsprechenden Daten des Galileo- oder GPS-Empfängers und/oder der Beschleunigungsmesser einen begrenzten, bei der Frequenzsuche in den Bestimmungsmitteln (WMD) anzuwendenden Bereich gemäß aktuellen maximalen Geschwindigkeitsdaten der Vorrichtung, die eine entsprechend begrenzte Doppler-Verschiebung darstellen, zu berechnen.
  3. Verfahren nach Anspruch 1 oder Vorrichtung nach Anspruch 2, bei dem bzw. bei der vor der Dekodierung oder Demodulation (WDM) ein spektraler Weißmachungsschritt oder eine spektrale Weißmachungsstufe ausgeführt bzw. arrangiert wird.
  4. Verfahren nach Anspruch 1 oder 3 oder Vorrichtung nach Anspruch 2 oder 3, bei dem bzw. bei der nur eine einzelne Frequenz aus dem Frequenzbereich geprüft wird, falls die mobile Empfängervorrichtung sich nicht gemäß den Daten von dem Galileo- oder GPS-Empfänger (GSR) und/oder dem einen oder mehreren Beschleunigungsmessern (ACC1, ACC2, ACC3) bewegt.
  5. Verfahren nach Anspruch 1, 3 oder 4 oder Vorrichtung nach einem der Ansprüche 2 bis 4, bei dem bzw. bei der bei der Bestimmung (MSPC) der aktuellen maximalen Bewegungsgeschwindigkeitsdaten der Vorrichtung der Galileo- oder GPS-Empfänger auch die aktuelle räumliche Orientierung der mobilen Vorrichtung berücksichtigt und dabei automatisch im Anschluss an eine entsprechende Initialisierung, die die Position der Quelle berücksichtigt, den Anteil der Bewegungsgeschwindigkeit bestimmt, der zu einer Quelle (LSP) des aktuellen Weges hin oder von dieser fort gerichtet wird.
  6. Verfahren nach Anspruch 5 oder Vorrichtung nach Anspruch 5, bei dem bzw. bei der für eine solche Initialisierung die Koordinaten der Quelle (LSP) bei kurzzeitiger Platzierung der mobilen Vorrichtung nahe der Quelle gespeichert werden, oder die aktuellen Koordinaten der Quelle automatisch übertragen werden oder die Koordinaten vorgespeichert werden.
  7. Verfahren nach Anspruch 5 oder 6 oder Vorrichtung nach Anspruch 5 oder 6, bei dem bzw. bei der die Bestimmung der räumlichen Orientierung der mobilen Vorrichtung wiederholt oder in einer periodischen oder kontinuierlichen Weise ausgeführt wird.
  8. Verfahren nach Anspruch 1, 3 oder 4 oder Vorrichtung nach einem der Ansprüche 2 bis 4, bei dem bzw. bei der bei der Bestimmung (MSPC) der aktuellen maximalen Bewegungsgeschwindigkeitsdaten der Vorrichtung auch die aktuelle räumliche Orientierung der mobilen Vorrichtung berücksichtigt wird und der Anteil der Bewegungsgeschwindigkeit, der zu einer Quelle (LSP) des akustischen Weges hin oder von dieser weg gerichtet wird, dadurch automatisch bestimmt wird, wobei die aktuelle räumliche Orientierung der mobilen Vorrichtung in Bezug auf die Quelle durch Verwendung eines zusätzlichen optischen Signals bestimmt (SPOD) wird, das in der mobilen Vorrichtung von dem Ort der Quelle (LSP) oder einem Ort nahe der Quelle empfangen wird oder durch Verwendung mehrerer Mikrofone in der mobilen Vorrichtung um zu bestimmen, von welcher Durchschnitts- oder Hauptrichtung das akustische Signal von diesen Mikrofonen empfangen wird.
  9. Verfahren nach einem der Ansprüche 1 und 2 bis 8 oder Vorrichtung nach einem der Ansprüche 2 bis 8, bei dem bzw. bei der Stations- oder Programm-ID-Daten und/oder Wasserzeichendaten, die von der mobilen Vorrichtung empfangen werden, in einem Speicher (MEM) gespeichert werden.
  10. Verfahren nach einem der Ansprüche 1 und 2 bis 9 oder Vorrichtung nach einem der Ansprüche 2 bis 9, bei dem bzw. bei der die Stations- oder Programm-ID-Daten und/oder Wasserzeichendaten, die von der mobilen Station empfangen werden, über einen Sender (TR) zu einem entsprechenden Audienzmessdatenempfänger gesendet werden, der für eine Audienzmessdaten sammelnde Gesellschaft betrieben wird.
EP10305181A 2010-02-24 2010-02-24 Verfahren und Vorrichtung zur Beurteilung eines möglicherweise mit einem Wasserzeichen versehenen Signals, das über einen akustischen Pfad in einem mobilen Empfänger empfangen wird Not-in-force EP2362388B1 (de)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP10305181A EP2362388B1 (de) 2010-02-24 2010-02-24 Verfahren und Vorrichtung zur Beurteilung eines möglicherweise mit einem Wasserzeichen versehenen Signals, das über einen akustischen Pfad in einem mobilen Empfänger empfangen wird

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP10305181A EP2362388B1 (de) 2010-02-24 2010-02-24 Verfahren und Vorrichtung zur Beurteilung eines möglicherweise mit einem Wasserzeichen versehenen Signals, das über einen akustischen Pfad in einem mobilen Empfänger empfangen wird

Publications (2)

Publication Number Publication Date
EP2362388A1 EP2362388A1 (de) 2011-08-31
EP2362388B1 true EP2362388B1 (de) 2013-04-03

Family

ID=42321001

Family Applications (1)

Application Number Title Priority Date Filing Date
EP10305181A Not-in-force EP2362388B1 (de) 2010-02-24 2010-02-24 Verfahren und Vorrichtung zur Beurteilung eines möglicherweise mit einem Wasserzeichen versehenen Signals, das über einen akustischen Pfad in einem mobilen Empfänger empfangen wird

Country Status (1)

Country Link
EP (1) EP2362388B1 (de)

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7016688B2 (en) * 2003-02-24 2006-03-21 Qualcomm, Incorporated Forward link repeater delay watermarking system
EP2083418A1 (de) * 2008-01-24 2009-07-29 Deutsche Thomson OHG Verfahren und Vorrichtung zur Bestimmung und Verwendung einer zufälligen Frequenz zur Entschlüsselung von Wasserzeicheninformationen, die in einem empfangenen Signal und mit einer ursprünglichen zufälligen Frequenz auf der Verschlüsselungsseite eingebettet wurde

Also Published As

Publication number Publication date
EP2362388A1 (de) 2011-08-31

Similar Documents

Publication Publication Date Title
JP5971768B2 (ja) 音声信号の符号化及び認識を使用する音声測位
US8792305B2 (en) Ultrasound detectors
KR101496067B1 (ko) 모바일 디바이스의 위치를 결정하기 위한 방법 및 장치
US9078055B2 (en) Localization of a wireless user equipment (UE) device based on single beep per channel signatures
EP3978949A3 (de) System und verfahren zur drahtlosen bewegungsüberwachung
US9026437B2 (en) Location determination system and mobile terminal
KR102162284B1 (ko) 클라우드 데이터 처리 gnss 재밍 감시 방법 및 시스템
WO2004077699A8 (en) Forward link repeater frequency watermarking system
US20140079223A1 (en) Localization of a Wireless User Equipment (UE) Device Based on Audio Masking
RU2481649C2 (ru) Способ и устройство для определения и использования частоты дискретизации для декодирования информации водяного знака, встроенной в принимаемый сигнал, выбранный с помощью исходной частоты дискретизации на стороне кодера
JP5918351B2 (ja) 信号サーチ方法、信号サーチプログラム、信号サーチ装置、ｇｎｓｓ信号受信装置、および情報機器端末
KR102242421B1 (ko) 재난을 감지하는 방법 및 이를 수행하는 전자 장치
EP4270059A1 (de) Detektion und schätzung von direkten und reflektierten navigationssatellitensignalparametern in einer mehrpfadumgebung
JP2006276856A (ja) 音声信号前処理システム
KR20160016974A (ko) 열화된 신호 환경에서 gnss 위성 신호를 검출하는 방법 및 장치
CA2996500C (en) System and method for determining a location of a vehicle relative to a stopping point
EP2823320B1 (de) Geographische stationaritätserkennung mit niedrigem stromverbrauch
EP2362388A1 (de) Verfahren und Vorrichtung zur Beurteilung eines möglicherweise mit einem Wasserzeichen versehenen Signals, das über einen akustischen Pfad in einem mobilen Empfänger empfangen wird
JP2007248110A (ja) 電波監視装置及び方法
JP2007172423A (ja) 発話状態検出装置および方法
CN114690223A (zh) 场景识别方法、装置、电子设备及计算机可读介质
KR102155419B1 (ko) 인공신경망을 기반으로 하는 적응형 거리측정장치
EP2867692A1 (de) Verfahren zur erzeugung und anwendung einer karte
WO2013140910A1 (ja) 信号サーチ方法、信号サーチプログラム、信号サーチ装置、ｇｎｓｓ信号受信装置、および情報機器端末
KR101588177B1 (ko) 상황 인지 기반의 상황 정보 추론 방법 및 이를 위한 장치

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK SM TR

AX Request for extension of the european patent

Extension state: AL BA RS

17P Request for examination filed

Effective date: 20120217

RIC1 Information provided on ipc code assigned before grant

Ipc: G10L 19/00 20060101AFI20120820BHEP

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 605152

Country of ref document: AT

Kind code of ref document: T

Effective date: 20130415

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Ref country code: GB

Ref legal event code: 746

Effective date: 20130415

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602010005901

Country of ref document: DE

Effective date: 20130529

REG Reference to a national code

Ref country code: DE

Ref legal event code: R084

Ref document number: 602010005901

Country of ref document: DE

Effective date: 20130410

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 605152

Country of ref document: AT

Kind code of ref document: T

Effective date: 20130403

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130403

REG Reference to a national code

Ref country code: NL

Ref legal event code: VDEP

Effective date: 20130403

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130403

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130714

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130803

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130403

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130703

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130403

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130403

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130805

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130403

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130403

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130704

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130403

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130403

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130403

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130703

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130403

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130403

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130403

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130403

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130403

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130403

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130403

26N No opposition filed

Effective date: 20140106

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602010005901

Country of ref document: DE

Effective date: 20140106

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140224

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130403

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20140228

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20140228

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20140224

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 7

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130403

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130403

REG Reference to a national code

Ref legal event code: R082

Country of ref document: DE

Representative=s name: HOFSTETTER, SCHURACK & PARTNER PATENT- UND REC, DE

Ref country code: DE

Ref document number: 602010005901

Ref country code: DE

Ref legal event code: R082

Ref document number: 602010005901

Country of ref document: DE

Representative=s name: KASTEL PATENTANWAELTE, DE

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 602010005901

Country of ref document: DE

Representative=s name: HOFSTETTER, SCHURACK & PARTNER PATENT- UND REC, DE

Ref country code: DE

Ref legal event code: R082

Ref document number: 602010005901

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20100224

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130403

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 8

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 9

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20180227

Year of fee payment: 9

Ref country code: DE

Payment date: 20180223

Year of fee payment: 9

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20180216

Year of fee payment: 9

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130403

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 602010005901

Country of ref document: DE

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20190224

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190224

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190903

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190228

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230526