EP2255359B1 - Dispositif et procédé d'indication acoustique - Google Patents

Dispositif et procédé d'indication acoustique Download PDF

Info

Publication number
EP2255359B1
EP2255359B1 EP09721864.8A EP09721864A EP2255359B1 EP 2255359 B1 EP2255359 B1 EP 2255359B1 EP 09721864 A EP09721864 A EP 09721864A EP 2255359 B1 EP2255359 B1 EP 2255359B1
Authority
EP
European Patent Office
Prior art keywords
objects
signal
acoustic
signals
loudspeaker
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP09721864.8A
Other languages
German (de)
English (en)
Other versions
EP2255359A1 (fr
Inventor
Thomas Sporer
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fraunhofer Gesellschaft zur Foerderung der Angewandten Forschung eV
Original Assignee
Fraunhofer Gesellschaft zur Foerderung der Angewandten Forschung eV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fraunhofer Gesellschaft zur Foerderung der Angewandten Forschung eV filed Critical Fraunhofer Gesellschaft zur Foerderung der Angewandten Forschung eV
Publication of EP2255359A1 publication Critical patent/EP2255359A1/fr
Application granted granted Critical
Publication of EP2255359B1 publication Critical patent/EP2255359B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
    • H04S7/30—Control circuits for electronic adaptation of the sound field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
    • H04S2400/11—Positioning of individual sound objects, e.g. moving airplane, within a sound field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
    • H04S2400/13—Aspects of volume control, not necessarily automatic, in stereophonic sound systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
    • H04S2420/13—Application of wave-field synthesis in stereophonic audio systems

Definitions

  • the present invention relates to an apparatus and method for acoustically displaying a position of an object in a playback room.
  • Exemplary embodiments include in particular acoustic displays for use on ships.
  • the DE 101 55 742 A1 discloses an apparatus and method for generating spatially located warning and information signals.
  • visual signals emitted wherein the respective acoustic and visual signal sources are mounted in a specific for the respective warning or information position.
  • a conventional stereo system of a vehicle is adapted as a "virtual reality" system.
  • a moving sound source is simulated realistically, the emitted information being varied in connection with the movement of the virtual sound source (eg imitation of the Doppler effect simulating an approaching siren of an emergency vehicle or intensifying the volume at the approach of a dangerous object).
  • a further advantageous use of this technique results from its integration into a lane recognition system.
  • a warning signal for example Nagelbandrattern
  • the US Pat. No. 6,097,285 discloses an apparatus for emitting acoustic signals in a vehicle compartment for notifying a driver of a vehicle of the presence of objects at predetermined locations around the vehicle.
  • the driver is alerted to the presence of objects in predetermined detection zones around the vehicle by the presence of various audible sounds corresponding to each detection zone.
  • an object present in a detection zone is caused to produce a unique, audibly different sound within the cabin to alert the driver of the presence of the particular object.
  • the present invention has the object to obtain a device or a method for the acoustic display of a position of an object.
  • an acoustic display device according to claim 1, an apparatus for scanning an environment according to claim 11, a method for acoustic display according to claim 13 or a computer program according to claim 14.
  • the core idea of the present invention is that a plurality of loudspeakers is spatially arranged so differently in a reproduction room that different positions can be acoustically represented by different activation of the loudspeakers.
  • a signal allocation device is designed to assign an acoustic signal to the object
  • a loudspeaker drive device is designed to determine one or more loudspeaker signals for the multiplicity of loudspeakers.
  • the one or more loudspeaker signals are arranged to indicate the position of the object, wherein the one or more loudspeaker signals are based on the acoustic signal associated with the object by the signal assigning means.
  • the one or more loudspeaker signals are determined such that upon playback of the one or more loudspeaker signals, the position of the object in the playback room is displayed acoustically.
  • Embodiments of the present invention also relate to how sensor signals can be displayed more easily by means of intelligent acoustic displays and thus both the security can be improved and the running costs can be reduced.
  • Another idea of the present invention is based on the fact that an essential part of the information in many detectors is a location.
  • a detector for example, a radar, a depth sounder, nautical charts or weather maps come into consideration and the location refers to, for example, a direction as well as a distance to the object.
  • a sound field generated which encodes as precisely as possible this information in a natural way.
  • the core idea of the present invention is that a plurality of loudspeakers is spatially arranged so differently in a reproduction room that different positions can be acoustically represented by different activation of the loudspeakers.
  • a signal allocation device is designed to assign an acoustic signal to the object
  • a loudspeaker drive device is designed to determine one or more loudspeaker signals for the multiplicity of loudspeakers.
  • the one or more loudspeaker signals are arranged to indicate the position of the object, wherein the one or more loudspeaker signals are based on the acoustic signal associated with the object by the signal assigning means.
  • the one or more loudspeaker signals are determined such that upon playback of the one or more loudspeaker signals, the position of the object in the playback room is displayed acoustically.
  • Embodiments of the present invention also relate to how sensor signals can be displayed more easily by means of intelligent acoustic displays and thus both the security can be improved and the running costs can be reduced.
  • Another idea of the present invention is based on the fact that an essential part of the information in many detectors is a location.
  • a detector for example, a radar, a depth sounder, nautical charts or weather maps come into consideration and the location refers to, for example, a direction as well as a distance to the object.
  • a sound field is generated, for example by means of several speakers, which encodes this information as precisely as possible in a natural way.
  • the workup includes on the one hand the detection of moving objects, such as ships and aircraft, and also the detection of static objects, such as the coastline, buoys or islands.
  • moving objects such as ships and aircraft
  • static objects such as the coastline, buoys or islands.
  • the audio signal can optionally be converted into an audio signal by means of a text-to-speech identification, so that the text signal of the transponder becomes audible.
  • Such objects are z. B. determines buoys or beacons, whose identifying information appear, for example, on the radar as text.
  • Objects can still be classified according to their hazard potential. For example, objects that come closer (from the front or faster from the back) or cross the ship's path of movement may be classified as more dangerous than objects that run parallel to the ship or are moving away from the ship. Objects that are farther away are generally considered less dangerous than those that are near or approaching at a high relative speed. Depending on the risk can thus be assigned to the objects a different identifier tone, with the identification tone for example in pitch or in the pulse repetition frequency and increase as the danger increases. Thus, a higher tone may mean greater danger or increasing volume may imply an increasing danger. Similarly, a faster beating clock pulse may mean a rising or a higher hazard than a lower clock pulse (for example, if the signature sound is represented as a rhythmic clock pulse).
  • the audio signals of the objects thus generated are then reproduced, for example, by the above-mentioned WFS or ZAP, whereby automatically far distant objects become quieter.
  • non-hazardous objects are completely blanked out (not shown) so as not to overload the helmsman or the listener with too much information.
  • the replay location may appear at the same distance as the actual distance, ie, if the object is one kilometer away by radar, the audio object is perceptible at one kilometer distance (1: 1 mapping).
  • the playback location is scaled accordingly so that, for example, a 1: 100 mapping is made and an object one kilometer away is acoustically perceptible or reproduced by an acoustic signal (virtual sound source) approximately ten meters away.
  • the former the 1: 1 image
  • Very distant objects would only be audible very late due to the speed of sound, and furthermore, in a 1: 1 representation, very distant objects are hardly distinguishable by distance.
  • Exemplary embodiments thus pursue the goal of coding objects with audio signals, so that they can be located as well as possible.
  • the audio signals should be sufficiently broadband, for example, because a sine wave is difficult to perceive. Accordingly, narrowband noise or speech should be used to identify objects, not sinusoidal ones.
  • pulsed signals are emitted instead of continuous signals (eg a continuous tone). The pulse rate can rise similarly to parking sensors in cars with increasing risk.
  • the audio signals should sound pleasant when the danger is sufficiently low.
  • the danger threshold above which there is a serious danger or below which there is no risk or hardly any danger, is set variably in accordance with the circumstances, for example.
  • the danger threshold can optionally also be adapted by the user.
  • the size and speed of a ship or the speeds of the other objects play a role.
  • the threshold value can be determined, for example, from the ratio of the time duration to a predicted collision to a braking time of the ship.
  • the pleasant sound of the audio signals can be achieved, for example, by using a low center frequency of the narrowband noise or a low pulse frequency (rare representation) for unidentified objects (eg objects that pose no danger).
  • a spectral hue of narrowband noise can be used where high frequencies have less energy than deep (cut with pink noise bandpass). For identified objects, this is done Rarely reporting reached, eg. B. at first contact to then send only a minute away a new signal.
  • the reporting signal may optionally be selected to be precisely located and distinguishable from ambient noise. Moreover, it is advantageous if the reporting signal has a pleasant sound, so that even with long trips, the system is permanently accepted.
  • An essential advantage of acoustic, spatially resolving displays is that, unlike optical displays, they can be used by one person simultaneously with the natural environment.
  • the natural environment may include, for example, driving on sight or listening to ships and buoys. Thus, a so-called augmented reality can be generated.
  • Embodiments are particularly advantageous because they provide an important synergy effect between acoustic and visual display. Namely, the audible indication is always reported and perceived, whereby prioritization for danger may occur while the visual indication requires the attention of the personnel on the bridge. For example, a helmsman sees an object on the radar screen only when he looks at the radar screen. At the same time, however, he no longer looks out of the window and thus loses some of the information about what is happening in his immediate surroundings. Acoustic displays allow him to simultaneously use the information from the radar and the view from the window. Especially in the case of non-self-identifying objects, however, the experienced evaluator is able to classify an object from the radar image (eg as a ship, island or picture disturbance). Thus, in the interaction of the acoustic perception (there is an object) and the view on the radar screen to control an important synergy effect. For remote, self-identifying objects, the identification can be read at any time by looking at the radar screen.
  • Fig. 1 shows a schematic representation of an apparatus for acoustic display 100 having an input 105, above the position information of an object in the Device 100 can be entered.
  • the device 100 further has outputs for a plurality of loudspeaker signals LS (eg for a first loudspeaker signal LS1, a second loudspeaker signal LS2, a third loudspeaker signal LS3, ..., an nth loudspeaker signal LSn).
  • the input for the position information 105 is designed to signal objects with their position to a signal allocation device 110.
  • the signal allocation device 110 is designed to assign an acoustic signal to the objects, wherein the signal allocation device 110 optionally accesses a signal database 140 in order to assign different signals to different objects, for example on the basis of their potential dangers.
  • the respectively assigned signal may, for example, depend on whether the object is moving, if so at what speed, or if it is immovable.
  • the device 100 has a loudspeaker drive device 120, which receives from the signal allocation device 110 the position of the object and the acoustic signal in order to determine one or more loudspeaker signals LS for a multiplicity of loudspeakers and these via the outputs for the loudspeaker signals LS1,. .. to spend LSn.
  • the loudspeaker driver 120 is configured to determine the one or more loudspeaker signals LS based on the acoustic signal assigned to the object. The determination is carried out in such a way that, when the one or more loudspeaker signals LS are reproduced, the position of the object in the reproduction room is indicated acoustically.
  • a listener or user then takes the position (eg, distance and direction) of the object as a virtual sound source position.
  • one embodiment relates to the reproduction of information of a radar device which determines positions of objects.
  • information from, for example, other sources, such as echosounders, or other sensors are implemented in a similar manner.
  • loudspeakers on the bridge of the ship below windows may be arranged on all walls. These loudspeakers, for example, can all be equipped with their own amplifiers or with A / D converters (analog-to-digital converters) and can also be individually controlled.
  • Fig. 2 shows a schematic representation of a playback room 210 with three speakers 220a, 220b and 220c and a radar 230.
  • the radar 230 is connected to the input 105 and provides position information about objects in a vicinity of the playback room 210. For example, the radar 230 is formed to the Position of the object 200 to the device 100 for acoustic display to pass.
  • the three speakers 220a, 220b, 220c are also connected to the outputs for the loudspeaker signals LS of the acoustic display device 100.
  • a first speaker 220a is connected to the output for the first speaker signal LS1
  • a second speaker 220b is connected to the output for the second speaker signal LS2
  • a third speaker 220c is connected to the output for the third speaker signal LS3.
  • the acoustic display device 100 evaluates the position information of the object 200 received from the radar 230 to generate three loudspeaker signals LS1, LS2, LS3 for the first, second and third loudspeakers 220a, 220b, 220c. The determination is made such that the position of the object 200 is audible to the listener in the playback room 210, which is at a position P, for example. For this purpose, first the device 100 determines an acoustic signal for the object 200 as a function of the position of the object 200. The position is determined by the distance d and the direction, which can be given for example by an angle ⁇ . Next, the apparatus 100 calculates loudspeaker signals LS for the first to third loudspeakers 220a to 220c.
  • This may include, for example, scaling the signal level and delaying the signal so that the listener at position P perceives the object 200 according to its position. In the in the Fig. 2 For example, this may be done such that the third loudspeaker 220c provides the strongest signal, during which the first loudspeaker 220a provides only a small signal and the second loudspeaker 220b does not provide a signal.
  • radar 230 may further be coupled to a sonar device, which, for example, the underwater topography abskannt and signals any existing shoals, which are also acoustically displayed.
  • a sonar device for example, the underwater topography abskannt and signals any existing shoals, which are also acoustically displayed.
  • Fig. 3a and 3b show possible variations of the acoustic signal as a function of the distance of the object and the associated danger potential.
  • a dependency of a frequency f of the signal on the distance d of the object 200 is shown.
  • a critical distance d c is less than that, there is an increased danger which requires an increased attention of the helmsman.
  • This transition from a safe to a dangerous state for example, be signaled in a changing acoustic signal.
  • the frequency f of the signal are close or only slightly above a fundamental frequency f 0 , wherein the thus defined frequency range is perceived by the helmsman as safe.
  • the object reduces the distance so that it comes below the critical distance d c , the frequency f of the acoustic signal can suddenly rise sharply, so that the increasing danger is signaled to the helmsman.
  • the increase in frequency can optionally also increase monotonously with decreasing distance of the object without causing a sudden change in the critical distance and a constantly increasing danger potential for the helmsman becomes perceptible.
  • the acoustic signal or the frequency f of the acoustic signal may include on the one hand, the audio frequency or else the clock frequency, for example, if the acoustic signal indicates a particular clock in a particular frequency (repetition rate of the clocks). Even with the clock signal can increase with decreasing distance, the clock frequency, so that acoustically an increasing risk potential for the helmsman is perceived.
  • Fig. 3b shows an embodiment in which the signal level S is shown as a function of time t.
  • the distance between two adjacent clocks decreases, so that the clock frequency increases, so that an approaching object will signal.
  • the decreasing pitch can be combined by the fact that the signal pulses louder and / or the frequencies of the signal pulses are changed.
  • the change of the signal may, for example, have a shift of the center frequency to higher frequencies, so that the increasing danger potential is also perceptible in the frequency level or audio frequency of the signal pulses.
  • the amplitude or volume of the signal can increase as the potential dangers increase.
  • the acoustic signals are barely perceptible, so that the helmsman is not disturbed by the acoustic signals.
  • Fig. 4 11 shows an embodiment in which a plurality of loudspeakers 220, a first loudspeaker 220a,..., a fourth loudspeaker 220d,..., a ninth loudspeaker 220i,... have a twelfth loudspeaker 2201.
  • the loudspeakers 220 are arranged around the position P of a listener so that the position of an object 200 or the direction of the object 200 becomes noticeable by the fact that only one loudspeaker is active.
  • the position of the active loudspeaker corresponds at the same time in the direction of the object 200. This is particularly advantageous when the position P in the reproduction room 210 is fixed.
  • a first object 200a at a distance d1 and a second object 200b at a distance d2 from the point of view P are perceived by the fourth loudspeaker 220d generating a first sound signal S1 and the ninth loudspeaker 220i generating a second sound signal S2.
  • the listener at the position P takes the first object 200a and the second object was then according to their positions.
  • the speaker can be selected, which is the shortest distance to the connecting line between the respective object and the Position P has. That would be the fourth loudspeaker 220d for the first object 200a and the ninth loudspeaker 220i for the second object 200b. All other speakers are further away from the respective connection lines (measured as a vertical distance) and, for example, can not be active in this embodiment (do not generate a sound signal).
  • the respective adjacent loudspeakers between which the connecting line between the first object 200a and the position P runs, to be active.
  • other neighbors speakers may be active. This means that, for example, in further embodiments not only the fourth loudspeaker 220d is active, but at the same time the third loudspeaker 220c and / or the second loudspeaker 220b and / or the fifth loudspeaker 220e can also be active.
  • the amplitude / phase should be selected such that for a listener at position P, the object 200 will be acoustically perceivable at its respective position.
  • Acoustic perceptibility means that the object 200 is perceived as a virtual sound source, wherein the distance in addition to the volume can also be signaled by a different clock frequency or frequency (as in the example Fig. 3a, b was shown).
  • Fig. 5 shows an embodiment in which the speakers are arranged in the context of a wave field synthesis system, so that the device for acoustic display 100, a first loudspeaker array 221a, a second loudspeaker array 221b and a third loudspeaker array 221c drives.
  • Each of the three loudspeaker arrays 221a, 221b, 221c has, for example, a multiplicity of loudspeakers which, for B.
  • each loudspeaker in a respective array can be controlled individually, so that the three arrays, which may be arranged, for example, on the sidewalls of the reproduction room 210, synthesize a wave field which would produce an object 200 as a virtual sound source in the reproduction room 210.
  • the device 100 can in turn be coupled to a radar device or a sonar device 230 which transmits the device 100 the position of the respective objects.
  • the object itself does not need to be a sound source, but instead a sound signal is specifically assigned to the object. In this sense, therefore, the acoustic display differs according to embodiments of conventional audio playback systems.
  • Wave field synthesis is an audio reproduction method developed at the TU Delft for the spatial reproduction of complex audio scenes.
  • the spatially correct rendering is not limited to a small area, but extends over a wide viewing area.
  • WFS is based on a well-founded mathematical-physical basis, namely the principle of Huygens and the Kirchhoff-Helmholtz integral.
  • a WFS reproduction system consists of a large number of loudspeakers (so-called secondary sources).
  • the loudspeaker signals are formed from delayed and scaled input signals. Since many audio objects (primary sources) are typically used in a WFS scene, many such operations are required to generate the loudspeaker signals. This requires the high computing power required for wave field synthesis.
  • WFS also offers the possibility of realistically mapping moving sources. This feature is used in many WFS systems and is for example for use in the cinema, virtual reality applications or live performances of great importance.
  • a primary goal is the development of signal processing algorithms for the playback of moving sources using WFS.
  • the real-time capability of the algorithms is an important condition.
  • the most important criterion for evaluating the algorithms is the objective perceived audio quality.
  • WFS is a very expensive audio reproduction process in terms of processing resources. This is mainly due to the large number of speakers in a WFS setup and the often high number of virtual sources used in WFS scenes. For this reason, the efficiency of the algorithms to be developed is of paramount importance.
  • Wave field synthesis systems have the advantage, in comparison to conventional multi-speaker systems, that exact positioning becomes possible as a result and exact positioning can also be determined at different positions within the reproduction space 210.
  • a basic structure of a wave field synthesis system is shown and has a speaker array 221 placed with respect to a reproduction space 210. Specifically, this includes in Fig. 6 shown speaker array, which is a 360 ° array, four array sides 221a, 221b, 221c and 221d.
  • speaker array which is a 360 ° array, four array sides 221a, 221b, 221c and 221d.
  • Each loudspeaker array 221 consists of a number of different individual loudspeakers 708, each of which is driven by its own loudspeaker signals LS, which are transmitted by a wave field synthesis module 710 via an in-line loudspeaker signal LS Fig. 6 only schematically shown data bus 712 are provided.
  • the position information is determined, for example, by a sensor for determining the position of objects (eg the radar) and provided to the wave field synthesis module via the input 105.
  • the wave field synthesis module may also receive further inputs, such as information about the room acoustics of the playback room 210, etc.
  • the signal allocator 110 is configured to associate acoustic signals to a plurality of objects 200
  • the loudspeaker driver 120 is configured to generate component signals for each of the plurality of objects 200 Combine component signals to speaker signals LS, so that the plurality of objects 200 are acoustically perceptible at different positions.
  • the various objects can appear or be perceived as virtual sources (sound sources) for the listeners.
  • boundary conditions are considered in the ships.
  • the boundary conditions include, for example, requirements for the frequency of the messages, possible positions of the loudspeakers, the required sound pressure level, the characterization of the noise (for example from the engine) and a specification of the control signals for the acoustic display.
  • optimal message signals can then be generated taking into account typical spatial sounds on the ships.
  • the acoustic drive includes techniques such as binaural coding or the wave field synthesis described above.
  • the different techniques are used on test rigs in ships (or one-to-one models of the bridge and / or the control room). For example, psychoacoustic experiments can provide clues.
  • Embodiments use reporting signals that are as well as possible to locate in the ship environment, but at the same time sound as pleasant as possible. In this case test setups in the laboratory or else a one-to-one model from the bridge and / or the control station or in vehicles as well as psychoacoustic experiments are useful.
  • Further embodiments also provide a connection of sensors and information, for example, from Radar, sounder and nautical charts are received, to the audible indicator.
  • An essential part of the connection is the selection of the relevant objects, which should be displayed for example by means of acoustic display.
  • the described systems can also be applied in automobiles, i. Further embodiments also include corresponding driver assistance systems in the car. For example, vehicles approaching laterally (eg when changing lanes) can be signaled acoustically.
  • the signal allocation device 110 is designed to assign an acoustic signal to the object 200 even if the object 200 itself is not a sound source.
  • the signal allocation device 110 further has an input 105 which can be coupled to a sensor 230 for determining the position of the object 200, wherein the sensor 230 is configured to transmit the position of the object 200 to the signal allocation device 110.
  • the sensor 230 has a radar or sonar.
  • the loudspeaker drive device 120 is designed to detect precisely one loudspeaker signal LS for exactly one loudspeaker 220d, wherein the loudspeaker 220d can be placed in the reproduction space 210 in the direction of the object 200.
  • exactly one loudspeaker signal LS drives exactly one other loudspeaker 220 when the object 200 changes position.
  • the signal allocation device 110 is designed to assign the object 200 an acoustic signal in a predetermined minimum bandwidth, so that the acoustic signal is clearly acoustically perceptible.
  • the inventive scheme can also be implemented in software.
  • the implementation may be on a digital storage medium, in particular a floppy disk or a CD with electronically readable control signals, which may interact with a programmable computer system such that the corresponding method is executed.
  • the invention thus also consists in a computer program product with program code stored on a machine-readable carrier for carrying out the method according to the invention when the computer program product runs on a computer.
  • the invention can thus be described as a computer program with a program code for carrying out the method be realized when the computer program runs on a computer.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Circuit For Audible Band Transducer (AREA)
  • Measurement Of Velocity Or Position Using Acoustic Or Ultrasonic Waves (AREA)
  • Radar Systems Or Details Thereof (AREA)

Claims (14)

  1. Dispositif (100) d'indication acoustique des positions d'au moins deux objets différents (200) dans un espace de reproduction (210), dans lequel dans l'espace de reproduction (210) sont disposés au moins trois haut-parleurs (220) en des positions spatiales différentes, de sorte que par une activation différente des haut-parleurs (220) puissent être représentées acoustiquement des positions spatiales différentes, le dispositif (100) d'indication acoustique présentant les caractéristiques suivantes:
    un moyen d'association de signal (110) qui est réalisé pour associer à chacun des objets (200) un signal acoustique en fonction de la distance de l'objet respectif (200) et du potentiel de risque y lié,
    dans lequel le moyen d'association de signal (110) présente par ailleurs une entrée (105) qui peut être couplée à un capteur (230) pour la détermination de position des objets (200), et le capteur (230) est réalisé pour transmettre la position des objets (200) au moyen d'association de signal (110); et
    un moyen d'activation de haut-parleur (120) qui est réalisé pour déterminer un ou plusieurs signaux de haut-parleur (LS) pour les au moins trois haut-parleurs (220),
    dans lequel les un ou plusieurs signaux de haut-parleur (LS) par lesquels sont indiquées les positions des objets (200) se basent sur les signaux acoustiques associés aux objets (200) par le moyen d'association de signal (110), et dans lequel les un ou plusieurs signaux de haut-parleur (LS) peuvent être déterminés de sorte que lors de la reproduction des un ou plusieurs signaux de haut-parleur (LS) soient indiquées acoustiquement les positions des objets (200) dans l'espace de reproduction (210),
    le dispositif (100) d'indication acoustique présentant une base de données de signal (140) qui est connectée au moyen d'association de signal (110), la base de données de signal (140) étant réalisée pour mettre à disposition différents signaux acoustiques pour les objets (200), les signaux acoustiques associés dépendant de si l'objet respectif (200) est déplaçable ou statique.
  2. Dispositif (100) selon la revendication 1, dans lequel les signaux acoustiques dans la base de données de signal (140) sont classifiés selon un potentiel de risque et le moyen d'association de signal (110) est réalisé pour associer à différents objets (200) en fonction de leur risque potentiels des signaux acoustiques de différentes classes.
  3. Dispositif (100) selon la revendication 2, dans lequel des signaux acoustiques à un potentiel de risque supérieur présentent une fréquence sonore supérieure ou une fréquence d'horloge supérieure.
  4. Dispositif (100) selon la revendication 2 ou la revendication 3, dans lequel un signal acoustique à haut potentiel de risque est associé à un objet (200) à faible distance et un signal acoustique à faible potentiel de risque est associé à un objet (200) à grande distance.
  5. Dispositif (100) selon l'une des revendications précédentes, dans lequel au moins l'un des objets (200) présente une vitesse relative vers l'espace de reproduction (200), et dans lequel le signal acoustique associé dépend de la vitesse relative.
  6. Dispositif (100) selon l'une des revendications précédentes, dans lequel le moyen d'activation de haut-parleur (120) est réalisé pour déterminer plusieurs signaux de haut-parleur (LS) pour la pluralité de haut-parleurs (220), dans lequel la pluralité de haut-parleurs (220) entourent une position dans l'espace de reproduction (210) au moins partiellement dans un plan.
  7. Dispositif (100) selon l'une des revendications précédentes, dans lequel au moins l'un des objets (200) est identifié par un message de texte, dans lequel le capteur (230) est réalisé pour transmettre le message de texte à l'entrée (105), et dans lequel le dispositif (100) présente par ailleurs un module texte-vocal qui est réalisé pour convertir le message texte en un signal audio et le transmettre au moyen d'activation de haut-parleur (120).
  8. Dispositif (100) selon l'une des revendications précédentes, dans lequel le moyen d'association de signal (110) est réalisé pour associer des signaux acoustiques à plusieurs objets (200), et dans lequel le moyen d'activation de haut-parleur (120) est réalisé pour générer pour chacun des plusieurs objets (200) des signaux à composantes et pour combiner les signaux à composantes pour obtenir des signaux de haut-parleur (LS), de sorte que les plusieurs objets (200) puissent être perçus acoustiquement à différentes positions.
  9. Dispositif (100) selon l'une des revendications précédentes, dans lequel le moyen d'activation de haut-parleur (120) est réalisé pour coder la distance (d) de l'objet (200) par une fréquence sonore ou une fréquence d'horloge, de sorte que la distance de l'objet (200) puisse être perçue avec un niveau prédéterminé.
  10. Dispositif (100) selon l'une des revendications précédentes, dans lequel le moyen d'activation de haut-parleur (120) présente un système de synthèse de champ d'onde, dans lequel le système de synthèse de champ d'onde est réalisé pour reproduire le signal acoustique associé à l'objet (200) comme source virtuelle.
  11. Dispositif de balayage d'un environnement, avec:
    un capteur (230) destiné à déterminer des positions d'objets (200) dans l'environnement; et
    un dispositif (100) d'indication acoustique selon l'une des revendications 1 à 10 qui est couplé au capteur (230) et obtient les positions des objets (200) du capteur (230).
  12. Dispositif selon la revendication 12, dans lequel le capteur (230) présente un radar ou un sonar.
  13. Procédé d'indication acoustique des positions d'au moins deux objets différents (200) dans un espace de reproduction (210), dans lequel dans l'espace de reproduction (210) sont disposés au moins trois haut-parleurs (220) en des positions spatiales différentes, de sorte que par une activation différente des haut-parleurs (220) puissent être représentées acoustiquement des positions spatiales différentes, aux étapes suivantes consistant à:
    associer différents signaux acoustiques aux objets (200) en fonction de la distance de l'objet respectif (200) et du potentiel de risque y lié, dans lequel, lors de l'association des différents signaux acoustiques aux objets au moyen d'une base de données de signal (140), les signaux acoustiques sont associés en fonction de si l'objet respectif (200) est déplaçable ou statique; et
    déterminer un ou plusieurs signaux de haut-parleur (LS) pour les au moins trois haut-parleurs (220), où les un ou plusieurs signaux de haut-parleur (LS) sont déterminés de sorte que, lors de la reproduction des un ou plusieurs signaux de haut-parleur (LS), les positions des objets (200) dans l'espace de reproduction (210) soient indiquées acoustiquement.
  14. Programme d'ordinateur avec un code de programme pour réaliser le procédé selon la revendication 13 lorsque le programme d'ordinateur est exécuté sur un ordinateur.
EP09721864.8A 2008-03-20 2009-03-17 Dispositif et procédé d'indication acoustique Active EP2255359B1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US3820908P 2008-03-20 2008-03-20
PCT/EP2009/001963 WO2009115299A1 (fr) 2008-03-20 2009-03-17 Dispositif et procédé d'indication acoustique

Publications (2)

Publication Number Publication Date
EP2255359A1 EP2255359A1 (fr) 2010-12-01
EP2255359B1 true EP2255359B1 (fr) 2015-07-15

Family

ID=40673888

Family Applications (1)

Application Number Title Priority Date Filing Date
EP09721864.8A Active EP2255359B1 (fr) 2008-03-20 2009-03-17 Dispositif et procédé d'indication acoustique

Country Status (6)

Country Link
US (1) US20110188342A1 (fr)
EP (1) EP2255359B1 (fr)
JP (1) JP2011516830A (fr)
KR (1) KR20100116223A (fr)
CN (1) CN101978424B (fr)
WO (1) WO2009115299A1 (fr)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101285391B1 (ko) * 2010-07-28 2013-07-10 주식회사 팬택 음향 객체 정보 융합 장치 및 방법
CN105578380B (zh) * 2011-07-01 2018-10-26 杜比实验室特许公司 用于自适应音频信号产生、编码和呈现的系统和方法
MY207992A (en) * 2011-07-01 2025-04-03 Dolby Laboratories Licensing Corp System and method for adaptive audio signal generation, coding and rendering
DE102011082310A1 (de) 2011-09-07 2013-03-07 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung, Verfahren und elektroakustisches System zur Nachhallzeitverlängerung
KR101308588B1 (ko) * 2012-02-28 2013-09-23 주식회사 부국하이텍 레이더 시스템 및 레이더 시스템을 이용한 표적의 음파 표시 방법
WO2013142657A1 (fr) * 2012-03-23 2013-09-26 Dolby Laboratories Licensing Corporation Système et procédé de conception de groupe et de rendu de haut-parleur
WO2017001611A1 (fr) * 2015-06-30 2017-01-05 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Procédé et dispositif pour affecter des bruits et les analyser
GB2542846A (en) * 2015-10-02 2017-04-05 Ford Global Tech Llc Hazard indicating system and method
PT109485A (pt) * 2016-06-23 2017-12-26 Inst Politécnico De Leiria Método e aparelho de criação de um cenário tridimensional
WO2019087646A1 (fr) * 2017-11-01 2019-05-09 ソニー株式会社 Dispositif de traitement d'informations, procédé de traitement d'informations et programme
DE102019006679B4 (de) * 2019-09-23 2025-01-02 Mbda Deutschland Gmbh Systeme und Verfahren zur Situationserkennung bezüglich sich in einem Überwachungsraum befindlicher mobiler Objekte
CN112911354B (zh) * 2019-12-03 2022-11-15 海信视像科技股份有限公司 显示设备和声音控制方法

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6097285A (en) * 1999-03-26 2000-08-01 Lucent Technologies Inc. Automotive auditory feedback of changing conditions outside the vehicle cabin
DE10155742A1 (de) * 2001-10-31 2003-05-22 Daimler Chrysler Ag Vorrichtung und Verfahren zur Generierung von räumlich lokalisierten Warn- und Informationssignalen zur vorbewussten Verarbeitung
EP1584901A1 (fr) * 2004-04-08 2005-10-12 Wolfgang Dr. Sassin Dispositif d'affichage dynamique optique, acoustique ou haptique de l'environnement d'un véhicule

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS6299879U (fr) * 1985-12-13 1987-06-25
JP2580117B2 (ja) * 1985-12-18 1997-02-12 ソニー株式会社 再生装置における音場拡大装置
JPS6325666U (fr) * 1986-03-13 1988-02-19
FR2744871B1 (fr) * 1996-02-13 1998-03-06 Sextant Avionique Systeme de spatialisation sonore, et procede de personnalisation pour sa mise en oeuvre
US5979586A (en) * 1997-02-05 1999-11-09 Automotive Systems Laboratory, Inc. Vehicle collision warning system
WO2001055833A1 (fr) * 2000-01-28 2001-08-02 Lake Technology Limited Systeme audio a composante spatiale destine a etre utilise dans un environnement geographique
JP4304845B2 (ja) * 2000-08-03 2009-07-29 ソニー株式会社 音声信号処理方法及び音声信号処理装置
FR2842064B1 (fr) * 2002-07-02 2004-12-03 Thales Sa Systeme de spatialisation de sources sonores a performances ameliorees
US20050222844A1 (en) * 2004-04-01 2005-10-06 Hideya Kawahara Method and apparatus for generating spatialized audio from non-three-dimensionally aware applications
US8494861B2 (en) * 2004-05-11 2013-07-23 The Chamberlain Group, Inc. Movable barrier control system component with audible speech output apparatus and method
JP2006005868A (ja) * 2004-06-21 2006-01-05 Denso Corp 車両用報知音出力装置及びプログラム
JP2006019908A (ja) * 2004-06-30 2006-01-19 Denso Corp 車両用報知音出力装置及びプログラム
DE102005008333A1 (de) * 2005-02-23 2006-08-31 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zum Steuern einer Wellenfeldsynthese-Rendering-Einrichtung
US20060256976A1 (en) * 2005-05-11 2006-11-16 House William N Spatial array monitoring system
JP4914057B2 (ja) * 2005-11-28 2012-04-11 日本無線株式会社 船舶用障害物警報装置
US7898423B2 (en) * 2007-07-31 2011-03-01 At&T Intellectual Property I, L.P. Real-time event notification

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6097285A (en) * 1999-03-26 2000-08-01 Lucent Technologies Inc. Automotive auditory feedback of changing conditions outside the vehicle cabin
DE10155742A1 (de) * 2001-10-31 2003-05-22 Daimler Chrysler Ag Vorrichtung und Verfahren zur Generierung von räumlich lokalisierten Warn- und Informationssignalen zur vorbewussten Verarbeitung
EP1584901A1 (fr) * 2004-04-08 2005-10-12 Wolfgang Dr. Sassin Dispositif d'affichage dynamique optique, acoustique ou haptique de l'environnement d'un véhicule

Also Published As

Publication number Publication date
WO2009115299A1 (fr) 2009-09-24
US20110188342A1 (en) 2011-08-04
JP2011516830A (ja) 2011-05-26
CN101978424B (zh) 2012-09-05
CN101978424A (zh) 2011-02-16
KR20100116223A (ko) 2010-10-29
EP2255359A1 (fr) 2010-12-01

Similar Documents

Publication Publication Date Title
EP2255359B1 (fr) Dispositif et procédé d'indication acoustique
DE69623023T2 (de) Ortungseinrichtungen
DE60217809T2 (de) Sicherheitsvorrichtung für Fahrzeuge mit einem Mehrkanal-Audio-System
DE102013204798A1 (de) Umgebungsinformationsmitteilungsvorrichtung
DE102012208825A1 (de) 3d-audiogerät
WO2013045374A1 (fr) Procédé de traitement assisté par ordinateur de l'environnement proche d'un véhicule
DE102011082886A1 (de) Stereoklangwiedergabesystem
DE112015006772T5 (de) System und Verfahren für Geräuschrichtungsdetektion in einem Fahrzeug
DE102007034029A1 (de) Verfahren zur Information eines Beobachters über ein im Einsatz befindliches Einsatzfahrzeug und Anordnung dazu
DE102012016819A1 (de) Verfahren und Vorrichtungen zur Fahrerunterstützung
DE102009057981B4 (de) Verfahren zur Steuerung der akustischen Wahrnehmbarkeit eines Fahrzeugs
DE102018209962A1 (de) Privataudiosystem für ein 3D-artiges Hörerlebnis bei Fahrzeuginsassen und ein Verfahren zu dessen Erzeugung
AT517657A1 (de) Verfahren und Einrichtung zur Warnung von Verkehrsteilnehmern für ein Schienenfahrzeug mittels Schall- oder Lichtsignalen
DE19801205A1 (de) Akustische Warneinrichtung
EP3652964A1 (fr) Émission d'informations localisée
DE102013214239A1 (de) Warnvorrichtung für ein Fahrzeug, Verfahren und Fahrzeug
DE102014217732B4 (de) Verfahren zum Assistieren eines Fahrers eines Kraftfahrzeugs, Vorrichtung und System
DE102014210932A1 (de) System und Verfahren für ein Fahrzeug zur akustischen Erkennung einer Verkehrssituation
EP2226217B1 (fr) Système d'information acoustique et tactile pour les occupants d'un véhicule
DE102018203661A1 (de) Verfahren und Vorrichtung zum Testen von direktionalem Hören in einem Fahrzeug
EP2182744B1 (fr) Retransmission d'un champ sonore dans une zone de sonorisation ciblée
EP4483158A1 (fr) Système de banc d'essai destiné à tester un système d'aide à la conduite avec un capteur de sons audibles
WO2023016924A1 (fr) Procédé et système de génération de bruits dans un habitacle sur la base de sources de bruit réelles extraites et classées, et véhicule acoustiquement transparent à des bruits cibles déterminés et comportant un système de ce type
DE102022113008A1 (de) Bordsystem und verfahren zum steuern des bordsystems
DE102016114413A1 (de) Vorrichtung zur Erzeugung von objektabhängigen Audiodaten und Verfahren zur Erzeugung von objektabhängigen Audiodaten in einem Fahrzeuginnenraum

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20101005

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA RS

DAX Request for extension of the european patent (deleted)
17Q First examination report despatched

Effective date: 20140130

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20150202

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 737158

Country of ref document: AT

Kind code of ref document: T

Effective date: 20150815

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 502009011260

Country of ref document: DE

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20150715

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150715

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150715

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151015

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150715

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151016

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150715

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150715

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150715

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151116

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150715

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 8

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 502009011260

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150715

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150715

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150715

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150715

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150715

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150715

26N No opposition filed

Effective date: 20160418

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150715

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150715

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160331

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150715

Ref country code: LU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160317

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160331

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160317

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160331

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 9

REG Reference to a national code

Ref country code: AT

Ref legal event code: MM01

Ref document number: 737158

Country of ref document: AT

Kind code of ref document: T

Effective date: 20160317

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150715

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160317

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150715

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 10

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20090317

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150715

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150715

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150715

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150715

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230524

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20260324

Year of fee payment: 18

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20260320

Year of fee payment: 18

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20260325

Year of fee payment: 18