EP1826736B1 - Véhicule automobile doté d'un dispositif de reconnaissance de signaux de trafic - Google Patents

Véhicule automobile doté d'un dispositif de reconnaissance de signaux de trafic Download PDF

Info

Publication number
EP1826736B1
EP1826736B1 EP20070001706 EP07001706A EP1826736B1 EP 1826736 B1 EP1826736 B1 EP 1826736B1 EP 20070001706 EP20070001706 EP 20070001706 EP 07001706 A EP07001706 A EP 07001706A EP 1826736 B1 EP1826736 B1 EP 1826736B1
Authority
EP
European Patent Office
Prior art keywords
output form
display
motor vehicle
output
probability
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
EP20070001706
Other languages
German (de)
English (en)
Other versions
EP1826736A1 (fr
Inventor
Jürgen Meurle
Markus Popken
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Audi AG
Original Assignee
Audi AG
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Audi AG filed Critical Audi AG
Publication of EP1826736A1 publication Critical patent/EP1826736A1/fr
Application granted granted Critical
Publication of EP1826736B1 publication Critical patent/EP1826736B1/fr
Ceased legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/09—Arrangements for giving variable traffic instructions
    • G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
    • G08G1/0967—Systems involving transmission of highway information, e.g. weather, speed limits
    • G08G1/096766—Systems involving transmission of highway information, e.g. weather, speed limits where the system is characterised by the origin of the information transmission
    • G08G1/096783—Systems involving transmission of highway information, e.g. weather, speed limits where the system is characterised by the origin of the information transmission where the origin of the information is a roadside individual element
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/09—Arrangements for giving variable traffic instructions
    • G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
    • G08G1/0967—Systems involving transmission of highway information, e.g. weather, speed limits
    • G08G1/096708—Systems involving transmission of highway information, e.g. weather, speed limits where the received information might be used to generate an automatic action on the vehicle control
    • G08G1/096716—Systems involving transmission of highway information, e.g. weather, speed limits where the received information might be used to generate an automatic action on the vehicle control where the received information does not generate an automatic action on the vehicle control

Definitions

  • the invention relates to a motor vehicle with a device for traffic sign recognition, comprising a camera, a control device for determining any traffic sign contained in the camera image and / or its regulatory content, and a display for outputting a representation of the traffic sign or information regarding its regulatory content.
  • a traffic sign recognition device serves to assist the driver while driving, since it is not ensured that the driver always sees each traffic sign or recognizes its regulatory content.
  • the vehicle apron is continuously recorded by means of the camera and continuously checked by a corresponding image analysis means by the control device, whether there is a traffic sign in the camera image.
  • suitable detection algorithms are provided. If such a traffic sign is detected or its regulatory content analyzed, the driver is issued a corresponding information on a display so that it is informed in any case about the presence of this traffic sign. This ensures that such traffic signs, which he, for example, due to the traffic situation that requires increased attention from him, or due to other, the traffic sign of concealing vehicles not covered, brought to his knowledge.
  • the traffic sign recognition device can not unequivocally recognize a traffic sign or its regulatory content, for example if the traffic sign is dirty, for example with respect to its orientation to the roadway is twisted or if, for example, a branch or the like depends on the traffic sign. In such a case, it is sometimes not ensured that the traffic sign recognition device correctly detects the traffic sign or its regulatory content. If, despite this lesser probability of correct detection, an output of the allegedly correctly detected traffic sign or regulatory content occurs, the output can sometimes be incorrect, the driver reacts incorrectly, which is disadvantageous. Or there is no output at all, so the driver is not informed at all about the presence of a traffic sign.
  • the invention is therefore based on the problem of specifying a motor vehicle in which a driver information adapted to the situation is possible even in such cases.
  • the traffic sign detection device recognizes a traffic sign or a regulatory content and assigns it a probability value regarding the correctness of recognition. This will be 100% if the detection is undoubted, but may be lower in different circumstances.
  • a probability limit is defined, eg 75% probability, above which a high probability of the correctness of the recognition is assumed, and the traffic sign or the regulatory content is displayed in a second output form. Below this probability limit is the probability that the recognition is correct, much lower, the first output form is chosen.
  • a monochrome representation or gray-scale representation e.g. a pictograph-like image of the supposedly recognized road sign or its control content
  • the second form of output the road sign with high probability or its control content
  • the driver recognizes immediately on the basis of the color of the presentation, whether the given information is secured or unsecured.
  • a second alternative provides that in the first output form, a camera image showing the traffic sign is output. If the recognition is not saved, the original image that the camera has taken is output. Optionally, the traffic sign can be enlarged in the representation and thus highlighted. In the second embodiment, in this variant of the invention a symbol showing the traffic sign or its regulatory content is displayed, so that the driver can thus recognize and distinguish the probability of the correctness of the detection on the basis of the type of representation (original image, symbol representation).
  • a symbol indicating the lower probability in particular a question mark or the like, is also output.
  • the respective output form and thus the probability of the correctness of the additional symbol such as a question mark at a probability below the limit are detected. So if, for example, a speed-limiting sign, such as a 100 km / h sign is detected and the detection is not secured, for example, this sign can be displayed in color, but also a large, eye-catching and possibly flashing question mark is displayed. When correctly detected, only the sign in the second output form is displayed.
  • the possibility of an additional symbol such as the question mark is of course also conceivable in the different embodiments described above.
  • a flashing display is made in the first output form and a permanent display in the second output form.
  • the driver can recognize that the probability of the correctness of the detection lies below the probability limit; in the case of permanent presentation, the information is considered to be secure.
  • certain traffic signs are assigned additional characters, which are usually arranged below the actual traffic sign, for example a speed-limiting traffic sign, and, for example, provide a temporal restriction of the regulation defined above the speed-limiting sign.
  • additional characters which are usually arranged below the actual traffic sign, for example a speed-limiting traffic sign, and, for example, provide a temporal restriction of the regulation defined above the speed-limiting sign.
  • a possibly unsecured detection of the shield or its regulatory content occurs.
  • the output of the same or its regulatory content between a first output form with less likelihood, in particular as information-less pictogram-like representation, and a second output form with a higher probability, in particular in the form of a pictorial representation representing its regulatory content.
  • a corresponding differentiation takes place.
  • the additional sign is not detected correctly, it is reproduced, for example, in the output representation as a white area or as a white sign together with the correctly recorded bid sign shown above. If, on the other hand, it is detected correctly, a corresponding representation of the additional sign or a corresponding pictograph-like representation, etc., takes place, similar to the assigned traffic signs.
  • the device for traffic sign recognition in particular its control device, is designed to carry out a plausibility check of its own detection result on the basis of at least one information of at least one further driver assistance or operating system or its sensors or control device.
  • the integrated navigation system indicates that the vehicle is in or shortly before a construction site area and if the traffic sign recognition device detects a speed-limiting road sign of 80 km / h, for example-regardless of the probability of the actual registration-information can be obtained From the navigation system that you are in a construction site area or just before, the speed limit is recognized as plausible. Because it is common that in the field of construction sites, especially on highways, the speed is significantly limited. Similarly, for example, can be dealt with information of an ACC system.
  • the speed limit can be defined as plausible on the ACC information regarding the distance and the speed of the vehicle ahead, after it is assumed that the preceding vehicle has reduced its speed due to the speed limit provided.
  • Fig. 1 shows a motor vehicle 1 according to the invention comprising a device 2 for traffic sign recognition, which in turn a camera 3, which is facing forward, looking through the windshield, and detects the vehicle apron, one of these associated control device 4 with an image processing and detection means and a display 5, where a detected road sign or its regulatory content is issued. Also provided is an acoustic signal means 6, for example, the speaker system of the usually integrated infotainment system of the motor vehicle is used here to output an acoustic information if necessary.
  • the control device 4 is also communicatively connected to various, here two driver assistance or operating systems 7, 8, each of which has various sensors 9, 10, are recorded on the appropriate environment or operating or situation information, if necessary the control device 4 are given and processed in the context of traffic sign recognition.
  • driver assistance systems 7, 8 for example, a navigation system and an ACC system are conceivable, but the embodiment of the invention is not limited thereto.
  • the vehicle apron is continuously detected via the camera 3.
  • the camera image is continuously given to the control device 4, which analyzes the camera image in real time for the presence of any traffic signs.
  • the control device 4 determines how probable the accuracy of the detection is.
  • the control device 4 thus virtually arranges a probability value for the determination or analysis result. Depending on this probability value, the type of output of the traffic sign or the detected regulatory content varies on the display 5.
  • Fig. 2 shows by way of example in the form of a tabular form the different display options.
  • the control unit 4 differentiates two different output forms, wherein the choice of one or the other depends on whether the defined probability, which has been assigned to the performed traffic sign recognition, is greater or less than a probability threshold.
  • the defined probability is indicated by W, while the probability threshold is indicated by W S.
  • the probability threshold can be, for example, 75%, or 80% or 90%, this can be defined by the manufacturer.
  • a traffic sign 11 here a speed-limiting sign which limits the speed to 80 km / h
  • the control device 4 now analyzes the camera image.
  • the detection can be complicated by the fact that the traffic sign 4 can be dirty, for example, it can for example be twisted to the road, so that it is only obliquely detected from the side, a branch can hang in front, etc. In such a case, with a lesser probability to count, for example, after polluted road sign, the controller 4 can not reliably detect the number sequence.
  • the representation now varies.
  • the display options (second output form) are shown in the tabular list on the left, if it is highly probable, ie if W> W S. With an assumed probability threshold of 75%, W> 75%.
  • a flashing indication is displayed here, that is to say that the traffic sign symbol shown flashes rhythmically.
  • a traffic sign or control content representation takes place without an additional symbol, while with an unsecured, first output form an additional symbol is also output, e.g. a large, conspicuous question mark, which may additionally flash or be conspicuously colored, or the like.
  • a fourth output mode d) provides for the second output form with secure recognition, a pictogram of the traffic sign or its control content output, while in the first output form with unsecured recognition, the original image of the camera 3 is shown, or a section thereof, which represents the hard to recognize traffic sign in an enlarged form. The driver thus sees the actual registered license plate.
  • the control device 4 communicates with the two driver assistance systems 7, 8. Via this information, which can be tapped off, it is possible to check the recognition result for plausibility. For example, if a speed-limiting sign is detected and a driver assistance system such as the navigation system recognizes that the vehicle is in a construction area or is approaching a construction site area where a speed limit is likely, the detection may for example be 80 km / h based on this information -Schilds plausibility, so be verified. Also, this can be used to increase the probability of detection, for example.
  • the probability value W can be increased by a certain amount and, if appropriate, the Exceed probability threshold, so that then but can be assumed that a secure detection. That is, taking into account the information of one or more driver assistance systems, despite the original likelihood less likely the traffic sign or control content display can be selected for a secure detection.

Landscapes

  • Life Sciences & Earth Sciences (AREA)
  • Atmospheric Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Traffic Control Systems (AREA)

Claims (8)

  1. Véhicule automobile doté d'un dispositif (2) de reconnaissance de panneaux de signalisation comprenant une caméra (3), un dispositif de commande (4) servant à déterminer un panneau de signalisation (11) éventuel contenu dans l'image de la caméra et/ou sa valeur de régulation et un écran (5) pour fournir une représentation du panneau de signalisation (11) ou une information concernant sa valeur de régulation,
    caractérisé en ce
    qu'en fonction de la probabilité de pertinence de la détection du panneau de signalisation ou de sa valeur de régulation, la sortie est modifiable entre une première forme de sortie avec une probabilité plus faible et une deuxième forme de sortie avec une probabilité plus élevée.
  2. Véhicule automobile selon la revendication 1,
    caractérisé en ce
    que dans la première forme de sortie, la représentation est monochrome ou en niveau de gris et dans la deuxième forme de sortie, la représentation est en couleurs.
  3. Véhicule automobile selon la revendication 1,
    caractérisé en ce
    que, dans la première forme de sortie, une image de caméra montrant le panneau de signalisation (11) est représentée et dans la deuxième forme de sortie un symbole représentant le panneau de signalisation (11) ou sa valeur de régulation est représenté.
  4. Véhicule automobile selon l'une quelconque des revendications précédentes,
    caractérisé en ce
    que, dans la première forme de sortie, un symbole montrant la probabilité plus faible, en particulier un signe d'interrogation ou similaire, est fourni.
  5. Véhicule automobile selon l'une quelconque des revendications précédentes,
    caractérisé en ce
    que, dans la première forme de sortie, la représentation clignote et dans la deuxième forme de réalisation, la représentation est durable.
  6. Véhicule automobile selon l'une quelconque des revendications précédentes,
    caractérisé en ce
    que, dans une représentation dans la première forme de sortie, un signal acoustique supplémentaire peut être donné.
  7. Véhicule automobile selon l'une quelconque des revendications précédentes,
    caractérisé en ce
    que, lors de la détection d'un signe supplémentaire associé au panneau de signalisation (11), la sortie de celui-ci ou de sa valeur de régulation est modifiable entre une première forme de sortie avec une probabilité plus faible, en particulier comme représentation sans informations de type pictogrammes, et une deuxième forme de sortie avec une probabilité plus élevée, en particulier sous la forme d'une représentation de type pictogrammes reproduisant sa valeur de régulation.
  8. Véhicule automobile selon l'une quelconque des revendications précédentes,
    caractérisé en ce qu'est formé le dispositif (1) de reconnaissance de panneaux de signalisation, en particulier son dispositif de commande servant à effectuer un examen de plausibilité du propre résultat de détection à l'aide d'au moins une information d'au moins un autre système d'assistance du conducteur ou d'exploitation (7, 8) ou de ses capteurs ou de son dispositif de commande.
EP20070001706 2006-02-24 2007-01-26 Véhicule automobile doté d'un dispositif de reconnaissance de signaux de trafic Ceased EP1826736B1 (fr)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
DE200610008656 DE102006008656A1 (de) 2006-02-24 2006-02-24 Kraftfahrzeug mit einer Einrichtung zur Verkehrszeichenerkennung

Publications (2)

Publication Number Publication Date
EP1826736A1 EP1826736A1 (fr) 2007-08-29
EP1826736B1 true EP1826736B1 (fr) 2008-05-07

Family

ID=38068407

Family Applications (1)

Application Number Title Priority Date Filing Date
EP20070001706 Ceased EP1826736B1 (fr) 2006-02-24 2007-01-26 Véhicule automobile doté d'un dispositif de reconnaissance de signaux de trafic

Country Status (2)

Country Link
EP (1) EP1826736B1 (fr)
DE (2) DE102006008656A1 (fr)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102010039634A1 (de) 2010-08-23 2012-02-23 Ford Global Technologies, Llc Anordnung und Verfahren zur Verkehrszeichenerkennung
DE102012104069B3 (de) * 2012-05-09 2013-03-14 Continental Teves Ag & Co. Ohg Verfahren zur Ermittlung einer Geschwindigkeitsempfehlung
US10380437B2 (en) 2014-10-14 2019-08-13 Toyota Motor Europe Systems and methods for traffic sign assistance
US10573175B2 (en) 2014-10-14 2020-02-25 Toyota Motor Europe Systems and methods for traffic sign validation

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102007004453A1 (de) * 2007-01-30 2008-07-31 Volkswagen Ag Verfahren und Vorrichtung zur Wiedergabe einer Verkehrszeicheninformation in einem Fahrzeug
DE102007058542A1 (de) 2007-12-06 2009-06-10 Robert Bosch Gmbh Fahrerassistenzsystem zur Überwachung der Fahrsicherheit und korrespondierendes Verfahren zur Erfassung und Bewertung einer Fahrzeugbewegung
DE102008037268A1 (de) 2008-08-11 2010-02-18 GM Global Technology Operations, Inc., Detroit Kraftfahrzeug mit einer Datenübertragungseinrichtung
DE102008054041B3 (de) * 2008-10-31 2010-06-10 Audi Ag Verfahren zur kombinierten Ausgabe eines Bildes und einer ein Verkehrszeichen betreffenden Information, sowie Kraftfahrzeug hierfür
DE102008057675A1 (de) 2008-11-17 2009-07-02 Daimler Ag Verfahren zur Verkehrszeichenerkennung
DE102010062633A1 (de) 2010-12-08 2012-06-14 Robert Bosch Gmbh Verfahren und Vorrichtung zur Erkennung von Verkehrszeichen in der Umgebung eines Fahrzeuges und Abgleich mit Verkehrszeicheninformationen aus einer digitalen Karte
DE102012011152A1 (de) * 2012-06-05 2013-12-05 GM Global Technology Operations LLC (n. d. Gesetzen des Staates Delaware) Fahrerassistenzsystem
DE102016202973B4 (de) 2016-02-25 2018-05-17 Bayerische Motoren Werke Aktiengesellschaft Verfahren zum Erkennen einer Straßenbaustelle
DE102017200545A1 (de) 2017-01-16 2018-07-19 Continental Teves Ag & Co. Ohg Vorrichtung und Verfahren zum Detektieren einer Baustelle für ein Kraftfahrzeug
FR3082533A1 (fr) 2018-09-25 2019-12-20 Continental Automotive France Systeme de vision amelioree des panneaux de signalisation
FR3086396B1 (fr) 2018-09-25 2021-01-15 Continental Automotive France Systeme d'identification par radar ou lidar des panneaux de signalisation
US10928828B2 (en) * 2018-12-14 2021-02-23 Waymo Llc Detecting unfamiliar signs

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE19736774A1 (de) * 1997-08-23 1999-02-25 Bosch Gmbh Robert Verfahren zur Informationsanzeige in einem Kraftfahrzeug
DE19852631C2 (de) * 1998-11-14 2001-09-06 Daimler Chrysler Ag Vorrichtung und Verfahren zur Verkehrszeichenerkennung
DE10200784A1 (de) * 2002-01-11 2003-07-31 Audi Ag Kraftfahrzeug
EP1383098B1 (fr) * 2002-07-09 2006-05-17 Accenture Global Services GmbH Dispositif de détection automatique de panneaux de signalisation routière

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102010039634A1 (de) 2010-08-23 2012-02-23 Ford Global Technologies, Llc Anordnung und Verfahren zur Verkehrszeichenerkennung
DE102010039634B4 (de) * 2010-08-23 2013-05-08 Ford Global Technologies, Llc Anordnung und Verfahren zur Verkehrszeichenerkennung
RU2571833C2 (ru) * 2010-08-23 2015-12-20 Форд Глобал Технолоджис, ЛЛК Устройство и способ распознавания дорожных знаков
DE102012104069B3 (de) * 2012-05-09 2013-03-14 Continental Teves Ag & Co. Ohg Verfahren zur Ermittlung einer Geschwindigkeitsempfehlung
US10380437B2 (en) 2014-10-14 2019-08-13 Toyota Motor Europe Systems and methods for traffic sign assistance
US10573175B2 (en) 2014-10-14 2020-02-25 Toyota Motor Europe Systems and methods for traffic sign validation

Also Published As

Publication number Publication date
EP1826736A1 (fr) 2007-08-29
DE502007000013D1 (de) 2008-06-19
DE102006008656A1 (de) 2007-08-30

Similar Documents

Publication Publication Date Title
EP1826736B1 (fr) Véhicule automobile doté d'un dispositif de reconnaissance de signaux de trafic
DE10045616B4 (de) Verfahren zum automatischen Einparken eines Kraftfahrzeugs
EP3622709B1 (fr) Procédé et dispositif de détection à résolution spatiale d'un objet externe à un véhicule à l'aide d'un capteur intégré dans un véhicule
DE102007021580B4 (de) Verfahren und Vorrichtung zum Ermitteln der fahrzeugrelevanten Verkehrszeichen
DE102009029439A1 (de) Verfahren und Vorrichtung zur Darstellung von Hindernissen in einem Einparkhilfesystem von Kraftfahrzeugen
DE102013225459B4 (de) Verfahren zum automatisierten Führen eines Kraftfahrzeugs und Kraftfahrzeug zum Ausführen des Verfahrens
DE102008010968A1 (de) Anzeige eines relevanten Verkehrszeichens oder einer relevanten Verkehrseinrichtung
EP1989094A1 (fr) Système d'assistance d'un conducteur
DE102013215176B4 (de) Verfahren zum Anzeigen einer Verkehrsumfeldinformation eines Verkehrsumfeldes eines Kraftfahrzeugs
DE102009019024A1 (de) Verfahren und Vorrichtung zur Vermeidung von Kollisionen beim Öffnen von Fahrzeugtüren
DE102013219023A1 (de) Verfahren und Vorrichtung zur Unterstützung eines Fahrers eines Fahrzeugs beim Spurwechsel
EP4107716B1 (fr) Procédé aidant un occupant à sortir d'un véhicule, prenant en compte les objets suivants, dispositif informatique, et système d'aide à la conduite
DE10200784A1 (de) Kraftfahrzeug
AT518940B1 (de) Verfahren und Vorrichtung zum Messen eines Abstands zwischen einem ersten Fahrzeug und einem zweiten, dem ersten Fahrzeug unmittelbar vorausfahrenden, Fahrzeug
DE102009003220B4 (de) Verfahren zur Kollisionswarnung sowie Kollisionswarneinrichtung
DE102009021283A1 (de) Vorrichtung und Verfahren zur Unterstützung eines Fahrzeugführers
DE102009041792A1 (de) Verfahren und Vorrichtung zur verbesserten Wahrnehmung eines von einem Einsatzfahrzeug ausgehenden akustischen Sondersignals
DE102018002239A1 (de) Verfahren zur Ausgabe eines Warnhinweises
DE102005057344A1 (de) Verfahren zur Detektion und Anzeige einer Parklücke und Einparkassistenzsystem eines Kraftfahrzeugs
DE102013003496A1 (de) Verfahren zur Dokumentation mindestens einer Kollisionswarnung während einer Fahrt eines Kraftfahrzeuges und ein Kraftfahrzeug
DE202014009919U1 (de) Fahrerassistenzsystem
DE102009019023A1 (de) Verfahren und Vorrichtung zur Steuerung einer Anzeigeeinheit eines Fahrzeugs
DE102020106850A1 (de) Verfahren zum Unterstützen eines Insassen bei einem Aussteigen aus einem Fahrzeug durch Anpassung einer Anzeige, Fahrerassistenzsystem sowie Fahrzeug
DE102014222627A1 (de) Verfahren zur Unterstützung eines Fahrers eines Kraftfahrzeugs beim Führen des Kraftfahrzeugs
DE102014222211B4 (de) Kraftfahrzeug mit einem einen Rückspiegel aufweisenden Anzeigesystem und Verfahren zur Informationsdarstellung

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20070620

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC NL PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA HR MK YU

17Q First examination report despatched

Effective date: 20070806

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): DE FR GB IT

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REF Corresponds to:

Ref document number: 502007000013

Country of ref document: DE

Date of ref document: 20080619

Kind code of ref document: P

ET Fr: translation filed
PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20090210

PGRI Patent reinstated in contracting state [announced from national office to epo]

Ref country code: IT

Effective date: 20110501

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 10

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 11

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20200131

Year of fee payment: 14

Ref country code: GB

Payment date: 20200128

Year of fee payment: 14

Ref country code: IT

Payment date: 20200131

Year of fee payment: 14

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20200128

Year of fee payment: 14

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 502007000013

Country of ref document: DE

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20210126

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210131

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210803

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210126

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210126