EP2811762A1 - Système binaural de formation de faisceau fondé sur la logique - Google Patents

Système binaural de formation de faisceau fondé sur la logique Download PDF

Info

Publication number
EP2811762A1
EP2811762A1 EP20140166949 EP14166949A EP2811762A1 EP 2811762 A1 EP2811762 A1 EP 2811762A1 EP 20140166949 EP20140166949 EP 20140166949 EP 14166949 A EP14166949 A EP 14166949A EP 2811762 A1 EP2811762 A1 EP 2811762A1
Authority
EP
European Patent Office
Prior art keywords
hearing aid
signal
aid system
linear combinations
linear combination
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP20140166949
Other languages
German (de)
English (en)
Other versions
EP2811762B1 (fr
Inventor
Eghart Fischer
Homayoun Dr. Kamkar Parsi
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sivantos Pte Ltd
Original Assignee
Siemens Medical Instruments Pte Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Siemens Medical Instruments Pte Ltd filed Critical Siemens Medical Instruments Pte Ltd
Publication of EP2811762A1 publication Critical patent/EP2811762A1/fr
Application granted granted Critical
Publication of EP2811762B1 publication Critical patent/EP2811762B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
    • H04R25/00—Electric hearing aids
    • H04R25/55—Electric hearing aids using an external connection, either wireless or wired
    • H04R25/552—Binaural
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
    • H04R25/00—Electric hearing aids
    • H04R25/40—Arrangements for obtaining a desired directivity characteristic
    • H04R25/405—Arrangements for obtaining a desired directivity characteristic by combining a plurality of transducers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
    • H04R25/00—Electric hearing aids
    • H04R25/40—Arrangements for obtaining a desired directivity characteristic
    • H04R25/407—Circuits for combining signals of a plurality of transducers
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208—Noise filtering
    • G10L21/0216—Noise filtering characterised by the method used for estimating noise
    • G10L2021/02161—Number of inputs available containing the signal or the noise to be suppressed
    • G10L2021/02166—Microphone arrays; Beamforming

Definitions

  • the present invention relates to a hearing aid system, wherein the hearing aid system has a left and a right hearing aid.
  • the left hearing aid has a left akusto-electrical converter and the right hearing aid on a right akusto-electrical converter.
  • the transducers are designed to convert incoming acoustic signals into left and right electrical signals.
  • the hearing aid system has a signal processing device, wherein the signal processing device is in signal connection with the left and the right acousto-electrical converter.
  • Hearing aids are portable hearing aids that are used to care for the hearing impaired.
  • different types of hearing aids such as behind-the-ear hearing aids (BTE), hearing aid with external receiver (RIC: receiver in the canal) and in-the-ear hearing aids (ITE), e.g. Concha hearing aids or canal hearing aids (ITE, CIC).
  • BTE behind-the-ear hearing aids
  • RIC hearing aid with external receiver
  • ITE in-the-ear hearing aids
  • ITE in-the-ear hearing aids
  • ITE in-the-ear hearing aids
  • ITE concha hearing aids or canal hearing aids
  • the hearing aids listed by way of example are worn on the outer ear or in the ear canal.
  • bone conduction hearing aids, implantable or vibrotactile hearing aids are also available on the market. The stimulation of the damaged hearing takes place either mechanically or electrically.
  • Hearing aids have in principle as essential components an input transducer, an amplifier and an output transducer.
  • the input transducer is usually an acoustoelectric transducer, z. As a microphone, and / or an electromagnetic receiver, for. B. an induction coil.
  • the output transducer is usually used as an electroacoustic transducer, z. As miniature speaker, or as an electromechanical transducer, z. B. bone conduction, realized.
  • the amplifier is usually integrated in a signal processing device.
  • binaural hearing is an essential prerequisite for spatial hearing and sound wave localization. Because of the importance of binaural processes in analyzing hearing situations, it is understandable that hearing-impaired individuals benefit more from two hearing aids for binaural care than from a single hearing aid for monaural care.
  • the binaural signal processing is also used to hide noise.
  • Wiener filters are used to hide uncorrelated noise.
  • a formation of a static directional characteristic by means of a static beam shaping (beam: beam, Richtkeule) of binaural signals is not able to independently respond to changing acoustic environments, so that the wearer of the hearing aid itself must respond by settings on the device.
  • beam beam, Richtkeule
  • adaptive filters are based on certain conditions for the useful and interfering signals, so that in certain listening situations that do not meet these requirements, the intelligibility for the wearer can even be worsened by the adaptive filters and he must again manually correct them.
  • this object is achieved by a method for operating a hearing aid system according to claim 1 and a hearing aid system according to claim 8.
  • the method according to the invention relates to a method for beam shaping for hearing aid systems.
  • the hearing aid system includes a left and a right hearing aid for placement on a head of a wearer. Usually, the hearing aids are worn on or in the left or right ear.
  • the left hearing aid has a left acousto-electric transducer, which converts sound waves arriving at the left hearing aid into a left input signal.
  • the right hearing aid has a right acousto-electrical converter, which converts sound waves arriving at the right hearing aid into a right input signal.
  • the hearing aid system has a signal processing device which is in signal connection with the left and the right acousto-electrical converter and receives the left and the right input signal.
  • the linear combinations are easy to calculate and therefore require little processor power. Furthermore, the linear combinations are undistorted signals without artificial frequency components and provide a natural listening experience available. By evaluating the linear combinations and selecting one as the beam signal, the type of evaluation can be used to predict the output of the beam shaping in a deterministic manner and no undesirable effects are to be expected.
  • the device of claim 8 for carrying out the method according to the invention shares its advantages.
  • the input signals are weighted with weighting factors, wherein the sum of the weighting factors of a linear combination is equal to 1 in each case.
  • the evaluation of the linear combinations is effected by determining a signal level of the linear combinations.
  • selecting a linear combination is accomplished by selecting the linear combination having the lowest signal level.
  • the signal with the lowest energy content is selected.
  • the advantageous effect is that in this way the signal with the lowest level of noise from directions unequal in direction is chosen in front of the carrier.
  • an estimated value for the spectral power density of a useful signal and a noise signal is determined from the left and the right input signal, and depending on this, the beam signal is amplified or attenuated.
  • the steps of the method are carried out separately for a plurality of frequency ranges.
  • the selection of a linear combination is effected by switching or blending the beam signal between two linear combinations.
  • this is done automatically for the user switching to the signal with the least amount of noise.
  • the transition for the wearer is barely noticeable.
  • Fig. 1 shows the basic structure of a hearing aid system 100 according to the invention.
  • the hearing aid system 100 has two hearing aid devices 110, 110 '.
  • a hearing aid housing 1, 1 ' for carrying behind the ear, one or more microphones 2, 2' for receiving the sound or acoustic signals from the environment are installed.
  • the microphones 2, 2 ' are acousto-electrical converters 2, 2' for converting the sound into first audio signals.
  • a signal processing device 3, 3 ' which is also integrated in the hearing aid housing 1, 1', processes the first audio signals.
  • the output signal of the signal processing device 3, 3 ' is transmitted to a loudspeaker or receiver 4, 4', which outputs an acoustic signal.
  • the sound is optionally transmitted via a sound tube, which is fixed with an earmold in the ear canal, to the eardrum of the device carrier.
  • the power supply of the hearing device and in particular that of the signal processing device 3, 3 ' is effected by a likewise integrated into the hearing aid housing 1, 1' battery 5, 5 '.
  • the hearing aid system 100 has a signal connection 6, which is designed to transmit a left input signal from the signal processing device 3 to the signal processing device 3 '. Conversely, it is provided that also the signal processing device 3 'transmits a right input signal to the signal processing device 3 in the opposite direction.
  • the signal connection 6 can be galvanic. However, in a preferred embodiment, the first and second electrical signals are converted for transmission over the signal connection.
  • the signal connection can be made, for example, inductively, via Bluetooth, optically or another wireless transmission technology.
  • the signal processing device 3, 3 ' is designed to form a plurality of linear combinations from the left and the right input signal, to evaluate the linear combinations and to select one of the linear combinations as a beam signal on the basis of the evaluation. For details, see the description of the process steps Fig. 2 see below.
  • the hearing aid system 100 also includes means 7, 7 'for adjusting the gain of the beam signal.
  • the signal processing means 3, 3 'and the means 7, 7' for adjusting the gain of the beam signal can, as in Fig. 1 represented, an integral part of the signal processing means 3, 3 'be.
  • each hearing aid device its own signal processing means 3, 3 'and get the signals of both microphones 2, 2' supplied.
  • Each of the signal processing devices 3, 3 ' is then able to independently determine and compensate for the signal differences between the microphones 2, 2'.
  • only one of the hearing aid devices 110, 110 ' has a signal processing device 3, 3', which performs the signal processing, determining and compensating and the resulting signal via the signal connection 6 to the other hearing aid 110, 110 'to Forwarding the issue.
  • the device 7, 7 'for adjusting the amplification of the beam signal which is either provided in each of the hearing aids 110, 110' or even in one, common to both hearing aids 110, 110 '.
  • Fig. 2 shows a schematic flow diagram of a method according to the invention in the signal processing device 3, 3 '.
  • the method includes a step S10 of providing a plurality of different linear combinations of the left input signal and the right input signal.
  • This step includes, among other things, converting an acoustic signal at the microphones 2, 2 'into a left input signal LS and a right input signal RS, as well as its transmission to the signal processing device 3, 3'.
  • the signal processing device 3, 3 ' provides a plurality of linear combinations LKi from the input signals LS and RS.
  • each of the hearing aids two microphones 2, 2 'have, so that a linear combination of 4 signals is formed.
  • the boundary condition remains that the sum of the coefficients, in this case four coefficients, is equal to 1 in each case. Also conceivable are three or more input signals and coefficients per page.
  • a step S20 the linear combinations are evaluated from the step S10. This is preferably also done by the signal processing device 3, 3 '.
  • One possible evaluation of the linear combination is a determination of an instantaneous signal level by means of a fast level meter. This can be done, for example, by a short-term average of the amount of the linear combination, wherein the short-term averaging could each comprise a few periods of the signal. However, it is also conceivable to use in each case the maximum of the magnitude of the amplitude of the signal in a signal period for determining the level.
  • one of the linear combinations is selected on the basis of the evaluation as a beam signal.
  • the linear combination is selected in which the signal level determined as the evaluation criterion is the lowest.
  • the energy density of the signal is correlated with the square of the signal level.
  • the linear combination with the lowest signal level and the corresponding lowest Energy density also the linear combination, which has the least amount of noise.
  • FIG. 3 illustrates a flowchart of a further method according to the invention. The method is identical in steps S10 to S30 with that in FIG Fig. 2 illustrated method.
  • the procedure of Fig. 3 also has a step S40.
  • step S40 an estimation of the spectral energy density of the useful signal is carried out.
  • step S50 an estimation of the spectral energy density of the noise is performed in the same manner.
  • the amplification of the beam signal is set as a function of the estimated energy densities of the useful and the interference signals. If it is estimated that the energy density of the useful signal is low, ie no useful signal arrives from a source in the plane of symmetry, then the amplification of the beam signal is reduced and thus also the interference signals. Conversely, if it is estimated that the energy density of the useful signal is large and thus a useful signal is present, the gain of the beam signal can be increased.
  • the linear combination is selected in step S60 by switching or blending the beam signal between the previously selected linear combination and the linear combination selected from the switching time.
  • the signal connection between the beam signal output and the linear combination is changed from the previous linear combination to newly selected.
  • the digital signal processing takes place This is achieved, for example, by the signal processing device 3, 3 'passing on the result of the selected linear combination to the beam signal output starting from this point in time.
  • the sum of the previous and the selected linear combination can be passed on to the cross-fading, the previous linear combination being weighted with a factor falling to zero over time and the selected linear combination being weighted with a factor increasing to one.
  • Steps S10 to S30 or S10 to S60 are therefore also carried out separately for individual frequency ranges or frequency bands of the input signals in a possible embodiment, so that the beam signal with the lowest noise component can be selected in each frequency range.
  • Fig. 4 shows the course of the Fig. 2 shown in function blocks.
  • the linear combinations LK1, LK2 and LK 3 are evaluated in accordance with step S20.
  • the comparator 24 decides on the basis of the criterion of the minimum level which is to be selected and passes this information on to the switch 25. This selects in step S30 from the linear combinations LK1, LK2, LK3 that which is to be passed as a beam signal.
  • Fig. 5 the sequence of steps S40 to S60 is shown in function blocks.
  • the estimation block 35 executes an estimate of the spectral energy density of the desired signal in accordance with step S40 with the prefiltered signals.
  • estimation block 36 occurs Similarly, in step S50, an estimate of the spectral energy density of the interfering signal.
  • the gain for the prefiltered beam signal BS is adjusted according to step S60.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Neurosurgery (AREA)
  • Otolaryngology (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Circuit For Audible Band Transducer (AREA)
EP14166949.9A 2013-05-16 2014-05-05 Système binaural de formation de faisceau fondé sur la logique Active EP2811762B1 (fr)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
DE102013209062.5A DE102013209062A1 (de) 2013-05-16 2013-05-16 Logik-basiertes binaurales Beam-Formungssystem

Publications (2)

Publication Number Publication Date
EP2811762A1 true EP2811762A1 (fr) 2014-12-10
EP2811762B1 EP2811762B1 (fr) 2016-03-09

Family

ID=50624507

Family Applications (1)

Application Number Title Priority Date Filing Date
EP14166949.9A Active EP2811762B1 (fr) 2013-05-16 2014-05-05 Système binaural de formation de faisceau fondé sur la logique

Country Status (4)

Country Link
US (1) US9473860B2 (fr)
EP (1) EP2811762B1 (fr)
DE (1) DE102013209062A1 (fr)
DK (1) DK2811762T3 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP4149121A1 (fr) * 2021-09-13 2023-03-15 Sivantos Pte. Ltd. Procédé de fonctionnement d'un appareil auditif

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9510222B2 (en) * 2013-08-23 2016-11-29 Qualcomm Incorporated Detection of bursty WiFi interference in LTE/LTE-A communications in an unlicensed spectrum
EP3269155B1 (fr) 2015-03-13 2019-01-02 Sivantos Pte. Ltd. Système d'aide auditive binaurale
US10182299B1 (en) * 2017-12-05 2019-01-15 Gn Hearing A/S Hearing device and method with flexible control of beamforming
EP3606100B1 (fr) 2018-07-31 2021-02-17 Starkey Laboratories, Inc. Commande automatique de fonctions binaurales dans des dispositifs portables à l'oreille
US11049509B2 (en) * 2019-03-06 2021-06-29 Plantronics, Inc. Voice signal enhancement for head-worn audio devices

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040196994A1 (en) 2003-04-03 2004-10-07 Gn Resound A/S Binaural signal enhancement system
US6983055B2 (en) 2000-06-13 2006-01-03 Gn Resound North America Corporation Method and apparatus for an adaptive binaural beamforming system
EP2018079A1 (fr) * 2007-07-20 2009-01-21 Siemens Audiologische Technik GmbH Procédé de traitement du signal dans une aide auditive
WO2010091077A1 (fr) 2009-02-03 2010-08-12 University Of Ottawa Procédé et système de réduction de bruit à multiples microphones
EP2262285A1 (fr) * 2009-06-02 2010-12-15 Oticon A/S Dispositif d'écoute fournissant des repères de localisation améliorés, son utilisation et procédé

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DK1326478T3 (en) * 2003-03-07 2014-12-08 Phonak Ag Method for producing control signals and binaural hearing device system
CA2621940C (fr) * 2005-09-09 2014-07-29 Mcmaster University Procede et dispositif d'amelioration d'un signal binaural

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6983055B2 (en) 2000-06-13 2006-01-03 Gn Resound North America Corporation Method and apparatus for an adaptive binaural beamforming system
US20040196994A1 (en) 2003-04-03 2004-10-07 Gn Resound A/S Binaural signal enhancement system
EP2018079A1 (fr) * 2007-07-20 2009-01-21 Siemens Audiologische Technik GmbH Procédé de traitement du signal dans une aide auditive
WO2010091077A1 (fr) 2009-02-03 2010-08-12 University Of Ottawa Procédé et système de réduction de bruit à multiples microphones
EP2262285A1 (fr) * 2009-06-02 2010-12-15 Oticon A/S Dispositif d'écoute fournissant des repères de localisation améliorés, son utilisation et procédé

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP4149121A1 (fr) * 2021-09-13 2023-03-15 Sivantos Pte. Ltd. Procédé de fonctionnement d'un appareil auditif
US12212927B2 (en) 2021-09-13 2025-01-28 Sivantos Pte. Ltd. Method for operating a hearing device, and hearing device

Also Published As

Publication number Publication date
US20140341407A1 (en) 2014-11-20
EP2811762B1 (fr) 2016-03-09
US9473860B2 (en) 2016-10-18
DE102013209062A1 (de) 2014-11-20
DK2811762T3 (da) 2016-06-13

Similar Documents

Publication Publication Date Title
EP3222057B1 (fr) Procédé et dispositif de détection rapide de la voix naturelle
EP2180726B2 (fr) Localisation du son avec des prothèses auditives binauriculaires
DE102011006471B4 (de) Hörhilfegerät sowie Hörgerätesystem mit einem Richtmikrofonsystem sowie Verfahren zum Einstellen eines Richtmikrofons bei einem Hörhilfegerät
EP2811762B1 (fr) Système binaural de formation de faisceau fondé sur la logique
EP2840809B1 (fr) Commande de l'intensité d'effet d'un microphone directionnel binaural
DE102007017761B4 (de) Verfahren zur Anpassung eines binauralen Hörgerätesystems
EP2164283B1 (fr) Appareil auditif et fonctionnement d'un appareil auditif doté d'une transposition de fréquence
EP2991379B1 (fr) Procede et dispositif de perception amelioree de sa propre voix
DE102011006129B4 (de) Hörvorrichtung mit Rückkopplungsunterdrückungseinrichtung und Verfahren zum Betreiben der Hörvorrichtung
EP2822300B1 (fr) Reconnaissance de situations d'écoute à l'aide de différentes sources de signal
DE102008064484A1 (de) Verfahren zum Auswählen einer Vorzugsrichtung eines Richtmikrofons und entsprechende Hörvorrichtung
DE102014218672B3 (de) Verfahren und Vorrichtung zur Rückkopplungsunterdrückung
EP4149121B1 (fr) Procédé de fonctionnement d'un appareil auditif
EP2793488B1 (fr) Adaptation de microphone binaurale au moyen de sa propre voix
EP2658289B1 (fr) Procédé de commande d'une caractéristique de guidage et système auditif
EP3048813B1 (fr) Procédé et dispositif de suppression du bruit basée sur l'inter-corrélation de bandes secondaires
EP4311269B1 (fr) Procédé de fonctionnement d'un appareil auditif binaural, appareil auditif binaural et logiciel
EP2885926A1 (fr) Système auditif ainsi que procédé de transmission
WO2011107545A2 (fr) Procédé pour régler un dispositif d'audition à effet directif

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20140505

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

R17P Request for examination filed (corrected)

Effective date: 20150407

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

RIC1 Information provided on ipc code assigned before grant

Ipc: G10L 21/0216 20130101ALN20150805BHEP

Ipc: H04R 25/00 20060101AFI20150805BHEP

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20150917

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: SIVANTOS PTE. LTD.

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 780257

Country of ref document: AT

Kind code of ref document: T

Effective date: 20160315

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: CH

Ref legal event code: NV

Representative=s name: E. BLUM AND CO. AG PATENT- UND MARKENANWAELTE , CH

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 502014000432

Country of ref document: DE

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 3

REG Reference to a national code

Ref country code: DK

Ref legal event code: T3

Effective date: 20160609

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20160309

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160609

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160309

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160610

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160309

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160309

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160309

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160309

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160309

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160531

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160309

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160309

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160309

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160309

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160709

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160309

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160309

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160309

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160309

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160711

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 502014000432

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160309

Ref country code: LU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160505

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20161212

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160609

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 4

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160309

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160505

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 5

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20140505

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160309

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160309

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160309

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160309

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160309

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160309

REG Reference to a national code

Ref country code: AT

Ref legal event code: MM01

Ref document number: 780257

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190505

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190505

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20250423

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20250423

Year of fee payment: 12

Ref country code: DK

Payment date: 20250423

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20250423

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: CH

Payment date: 20250601

Year of fee payment: 12