WO2012108721A2 - Procédé et dispositif pour produire une réalité augmentée au moyen de données images - Google Patents

Procédé et dispositif pour produire une réalité augmentée au moyen de données images Download PDF

Info

Publication number
WO2012108721A2
WO2012108721A2 PCT/KR2012/001006 KR2012001006W WO2012108721A2 WO 2012108721 A2 WO2012108721 A2 WO 2012108721A2 KR 2012001006 W KR2012001006 W KR 2012001006W WO 2012108721 A2 WO2012108721 A2 WO 2012108721A2
Authority
WO
WIPO (PCT)
Prior art keywords
image
augmented reality
plane
information
lane information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/KR2012/001006
Other languages
English (en)
Korean (ko)
Other versions
WO2012108721A3 (fr
Inventor
고석필
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Thinkware Systems Corp
Original Assignee
Thinkware Systems Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Thinkware Systems Corp filed Critical Thinkware Systems Corp
Publication of WO2012108721A2 publication Critical patent/WO2012108721A2/fr
Publication of WO2012108721A3 publication Critical patent/WO2012108721A3/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/222Studio circuitry; Studio devices; Studio equipment
    • H04N5/262Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/2625Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects for obtaining an image which is composed of images from a temporal image sequence, e.g. for a stroboscopic effect
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00Three-dimensional [3D] image rendering
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00Three-dimensional [3D] modelling for computer graphics
    • G06T17/05Geographic models
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/006Mixed reality

Definitions

  • the present invention relates to an apparatus and method for providing augmented reality to an image based on image information, and more particularly, to an augmented reality providing apparatus for synthesizing and providing a virtual object to an image captured by a camera based on the image information. It relates to a method for providing augmented reality.
  • Navigation route and road information providing method has been developed from the 2D map that provides only the shape of the road to the shape of the building around the road to the user to easily find the destination 3D map.
  • the building cannot be displayed in the same shape as the reality due to the limitation of data capacity or modeling time.
  • the exterior of a building is changed only for a short period of time due to events or other conditions, it is difficult to reflect the changed information in the 3D map, and thus the state of the building cannot be displayed in real time.
  • the present invention provides an apparatus and method for providing augmented reality that enhances a user's perception of a corresponding place by displaying information for guiding a route or information related to a road on an image of a real space photographed by a camera.
  • an apparatus and method for providing augmented reality for improving mapping accuracy between a 2D plane and a 3D space by mapping a 2D plane to a 3D space based on an image photographed by a camera are provided.
  • An apparatus for providing augmented reality may include: a 3D space generator configured to generate a virtual 3D space based on an image captured by a camera; A plane determination unit to determine a 2D plane based on the lane information included in the image; A mapping unit which maps the 2D plane to the virtual 3D space; And an augmented reality providing unit configured to provide an augmented reality by synthesizing an information object with the image based on the mapped 2D plane.
  • Augmented reality providing method comprises the steps of generating a virtual 3D space based on the camera parameters; Determining a 2D plane based on lane information of an image captured by the camera; Mapping the 2D plane to the virtual 3D space; And synthesizing an information object with the image based on the mapped 2D plane to provide augmented reality.
  • the user's recognition of the place can be improved by displaying information for guiding a route or information related to a road in an image of the actual space photographed by the camera.
  • the mapping accuracy between the 2D plane and the 3D space can be improved by mapping the 2D plane to the 3D space based on the image photographed by the camera.
  • FIG. 1 is a block diagram illustrating an apparatus for providing augmented reality according to an embodiment of the present invention.
  • 3 is an example of an augmented reality image provided in accordance with one embodiment of the present invention.
  • FIG. 4 is a flowchart illustrating a method of providing augmented reality according to an embodiment of the present invention.
  • the augmented reality providing method according to an embodiment of the present invention may be performed by an augmented reality providing apparatus.
  • the apparatus for providing augmented reality according to an embodiment of the present invention may be one of a mobile terminal, a navigation system, or a black box, and receives an image from a mobile terminal, a navigation system, a black box, or a separate camera to provide augmented reality. It may be a terminal.
  • FIG. 1 is a block diagram illustrating an apparatus for providing augmented reality according to an embodiment of the present invention.
  • the apparatus 100 for augmented reality may include a camera 110, a calibrator 120, a 3D space generator 130, a lane information extractor 140, and a plane.
  • the determination unit 150, the mapping unit 160, and the augmented reality providing unit 170 may be included.
  • the calibration unit 120 may estimate a camera parameter of the camera from an image photographed by the camera using calibration.
  • the camera may be a camera 110 embedded in the augmented reality providing apparatus 100 or may be a camera installed separately from the augmented reality providing apparatus 100 to photograph the front of the vehicle.
  • the camera parameter may be a parameter constituting a camera matrix which is information indicating a relationship between the actual space and the picture.
  • the 3D space generator 130 may generate a virtual 3D space based on the image photographed by the camera.
  • the 3D space generator 130 obtains depth information from an image captured by the camera based on the camera parameter estimated by the calibrator 120, and virtually based on the acquired depth information and the image. You can create 3D space.
  • the lane information extractor 140 may acquire a still image by capturing an image captured by the camera, and extract a plurality of lane information from the still image. Since the video captured by the camera is a video photographing the front of the vehicle in real time, it is difficult to extract lane information because the lane continues to move. Accordingly, the lane information extractor 140 may capture the image to obtain a still image, and extract lane information from a still image having no lane change.
  • the lane information extractor 140 may extract only one lane information. Accordingly, when the lane information extractor 140 fails to extract the plurality of lane information from the still image, the lane information extractor 140 may determine that the acquired still image is inappropriate and recapture the image. In this case, the lane information extractor 140 may extract a plurality of lane information from another still image obtained by recapturing.
  • the plane determiner 150 may determine the 2D plane based on the lane information included in the image photographed by the camera.
  • the plane determiner 150 may obtain a plurality of reference points based on the lane information extracted by the lane information extractor 140, and determine the 2D plane by using a linear equation connecting the reference points.
  • the reference point may include a bottom point, which is a point where each lane information and the bottom of the image meet, and a vanishing point.
  • the mapping unit 160 may map the 2D plane determined by the plane determination unit 150 to the virtual 3D space generated by the 3D space generation unit 130.
  • the mapping unit 160 may project the 2D plane determined by the plane determination unit 150 to the virtual 3D space generated by the 3D space generation unit 130 to form a bottom surface of the 3D space.
  • the augmented reality provider 170 may provide an augmented reality by synthesizing an information object with the image based on the 2D plane mapped by the mapping unit 160.
  • the information object may be an object that displays information related to a path on which the vehicle is traveling or a road on which the vehicle is driving.
  • the information object may include at least one of information indicating that 000M remains to a specific place, information indicating a place of attention such as a crosswalk, speeding camera position information, guide symbol, and direction information of a route.
  • the plane determiner 150 may determine the 2D plane based on the lane information 210 and 220 extracted by the lane information extractor 140.
  • the plane determination unit 150 may obtain a vanishing point 230 that is a point at which the lane information 210 and the lane information 220 cross each other.
  • the plane determination unit 150 may include a bottom point 250, which is a point where the lane information 210 meets the bottom of the image 200, and a bottom point that is a point where the lane information 220 meets the bottom of the image. 240 can be obtained.
  • the plane determination unit 150 may calculate the equation of the straight line between the vanishing point 230 and the bottom point 250 and the equation of the straight line between the vanishing point 230 and the bottom point 240.
  • the plane determination unit 150 is a triangle formed by the equation of the straight line between the vanishing point 230 and the bottom point 250, the equation of the straight line between the vanishing point 230 and the bottom point 240, and the bottom of the image 200. Can be determined as the 2D plane.
  • the apparatus 100 for augmented reality extracts lane information based on an image photographed by a camera installed in a vehicle, and generates a 2D plane based on the extracted lane information. May not be affected.
  • 3 is an example of an augmented reality image provided in accordance with one embodiment of the present invention.
  • the augmented reality providing apparatus 100 may provide an augmented reality by synthesizing an information object with an image photographed by a camera. For example, as shown in FIG. 3, the direction information 310 of the route and the information 320 indicating 500M remaining to a specific place may be displayed on the image 300 captured by the camera.
  • the apparatus 100 for providing augmented reality synthesizes and displays information related to a route guide or a road guide on a real-time image photographed by a camera to display a user's information on a corresponding place. It can increase the recognition.
  • FIG. 4 is a flowchart illustrating a method of providing augmented reality according to an embodiment of the present invention.
  • the calibrator 120 may estimate camera parameters of the camera from an image photographed by the camera using calibration.
  • the 3D space generator 130 may generate a virtual 3D space based on the camera parameter estimated in operation S410.
  • the 3D space generator 130 may obtain depth information from an image captured by the camera based on a camera parameter, and generate a virtual 3D space based on the acquired depth information and the image.
  • the lane information extractor 140 may acquire a still image by capturing an image captured by the camera.
  • the lane information extractor 140 may determine whether the still image acquired in operation S430 is a still image capable of extracting lane information. In more detail, the lane information extractor 140 may attempt to extract lane information from the still image acquired in operation S430. If the plurality of lane information is not extracted, the lane information extractor 140 may determine that the still image acquired in operation S430 is not a still image from which the lane information may be extracted. In this case, the lane information extractor 140 may return to step S430 to capture another still image from among images captured by the camera to obtain another still image.
  • the plane determination unit 150 may obtain a plurality of reference points based on the lane information extracted by the lane information extraction unit 140 in operation S440.
  • the reference point acquired by the plane determination unit 150 may include a bottom point, which is a point where each lane information and the bottom of the image meet, and a vanishing point.
  • the plane determination unit 150 may determine the 2D plane using the equations of the reference points obtained in operation S440 and a straight line connecting the reference points.
  • the mapping unit 160 may map the 2D plane determined in operation S460 to the virtual 3D space generated in operation S420.
  • the mapping unit 160 may project the 2D plane determined in operation S460 to the virtual 3D space generated in operation S460 to configure a bottom surface of the 3D space.
  • the augmented reality provider 170 may provide an augmented reality by synthesizing an information object with the image based on the 2D plane mapped in operation S470.
  • the augmented reality provider 170 may set the type and position of the information object to be displayed on the 2D plane based on the map data, and map the position of the information object set on the 2D plane to the bottom surface of the 3D space. Can be.
  • the augmented reality providing unit 170 may select a position to display the information object in the image taken by the camera according to the position mapped to the floor surface.
  • the augmented reality provider 170 may synthesize and display the information object at a selected position in the image photographed by the camera.
  • the information object may be an object that displays information related to a path on which the vehicle is traveling or a road on which the vehicle is driving.
  • the present invention can improve the user's perception of a corresponding place by displaying information for guiding a route or information related to a road in an image of a real space photographed by a camera.
  • mapping accuracy may be improved as compared with the conventional method of mapping the 2D plane to the 3D space based on the location where the sensor is installed.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Multimedia (AREA)
  • General Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computer Hardware Design (AREA)
  • Remote Sensing (AREA)
  • Traffic Control Systems (AREA)
  • Navigation (AREA)
  • Processing Or Creating Images (AREA)

Abstract

L'invention concerne un procédé et un dispositif utilisés pour conférer une réalité augmentée à une image photographiée par un appareil photographique sur la base de données images. Un dispositif de production d'une réalité augmentée comprend: une unité de génération d'espace 3D virtuel qui génère un espace 3D virtuel sur la base d'une image photographiée par un appareil photographique, une unité de détermination de plan qui détermine un plan 2D sur la base des données de voie de circulation contenues dans ladite image, une unité de mappage qui met en correspondance ledit plan 2D dans ledit plan 3D virtuel et une unité de production de réalité augmentée qui produit une réalité augmentée par synthétisation d'un objet d'information avec ladite image sur la base du plan 2D mappé.
PCT/KR2012/001006 2011-02-11 2012-02-10 Procédé et dispositif pour produire une réalité augmentée au moyen de données images Ceased WO2012108721A2 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR10-2011-0012353 2011-02-11
KR1020110012353A KR101188105B1 (ko) 2011-02-11 2011-02-11 영상 정보를 이용한 증강 현실 제공 장치 및 방법

Publications (2)

Publication Number Publication Date
WO2012108721A2 true WO2012108721A2 (fr) 2012-08-16
WO2012108721A3 WO2012108721A3 (fr) 2012-12-20

Family

ID=46639081

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2012/001006 Ceased WO2012108721A2 (fr) 2011-02-11 2012-02-10 Procédé et dispositif pour produire une réalité augmentée au moyen de données images

Country Status (2)

Country Link
KR (1) KR101188105B1 (fr)
WO (1) WO2012108721A2 (fr)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014151366A1 (fr) * 2013-03-15 2014-09-25 Elwha Llc Sélection, glissement et dépose en réalité mixte pour systèmes de réalité augmentée
DE102013210789A1 (de) 2013-06-10 2014-12-11 Robert Bosch Gmbh Augmented-Reality-System und Verfahren zum Erzeugen und Anzeigen von Augmented-Reality-Objektdarstellungen für ein Fahrzeug
US8928695B2 (en) 2012-10-05 2015-01-06 Elwha Llc Formatting of one or more persistent augmentations in an augmented view in response to multiple input factors
US9077647B2 (en) 2012-10-05 2015-07-07 Elwha Llc Correlating user reactions with augmentations displayed through augmented views
US9105126B2 (en) 2012-10-05 2015-08-11 Elwha Llc Systems and methods for sharing augmentation data
US9111384B2 (en) 2012-10-05 2015-08-18 Elwha Llc Systems and methods for obtaining and using augmentation data and for sharing usage data
US9141188B2 (en) 2012-10-05 2015-09-22 Elwha Llc Presenting an augmented view in response to acquisition of data inferring user activity
US9639964B2 (en) 2013-03-15 2017-05-02 Elwha Llc Dynamically preserving scene elements in augmented reality systems
US9671863B2 (en) 2012-10-05 2017-06-06 Elwha Llc Correlating user reaction with at least an aspect associated with an augmentation of an augmented view
US10109075B2 (en) 2013-03-15 2018-10-23 Elwha Llc Temporal element restoration in augmented reality systems
US10269179B2 (en) 2012-10-05 2019-04-23 Elwha Llc Displaying second augmentations that are based on registered first augmentations

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101285075B1 (ko) * 2011-11-24 2013-07-17 팅크웨어(주) 자이로스코프의 센서 정보와 차선 정보를 이용한 증강현실 뷰 모드 표현 방법 및 그 장치
KR102406490B1 (ko) * 2014-12-01 2022-06-10 현대자동차주식회사 전자 장치, 전자 장치의 제어 방법, 컴퓨터 프로그램 및 컴퓨터 판독 가능한 기록 매체
KR102411046B1 (ko) * 2015-06-05 2022-06-20 엘지디스플레이 주식회사 바형 디스플레이 장치 및 이를 포함하는 차량
KR102407296B1 (ko) * 2015-07-30 2022-06-10 현대오토에버 주식회사 관심지점 표시 장치 및 방법
KR102434406B1 (ko) * 2016-01-05 2022-08-22 한국전자통신연구원 공간 구조 인식을 통한 증강 현실 장치 및 그 방법
KR102766548B1 (ko) * 2016-11-07 2025-02-12 삼성전자주식회사 3차원의 도로 모델을 생성하는 방법 및 장치
KR101988128B1 (ko) 2017-10-11 2019-06-11 이태홍 증강현실을 이용한 전동휠의 철로 주행장치
US10417829B2 (en) 2017-11-27 2019-09-17 Electronics And Telecommunications Research Institute Method and apparatus for providing realistic 2D/3D AR experience service based on video image
KR102067823B1 (ko) * 2017-11-27 2020-01-17 한국전자통신연구원 비디오 영상기반 2d/3d ar 실감체험 방법 및 장치
US12196568B2 (en) 2019-11-01 2025-01-14 Lg Electronics Inc. Method of providing image by vehicle navigation device

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100472823B1 (ko) 2002-10-21 2005-03-08 학교법인 한양학원 차선 인식 방법 및 시스템
EP1958149B1 (fr) * 2005-12-02 2012-01-18 Koninklijke Philips Electronics N.V. Procede et appareil d'affichage d'images stereoscopiques, procede de generation de donnees d'image 3d a partir d'une entree de donnees d'image 2d et appareil generant des donnees d'image 3d a partir d'une entree d'image 2d
KR20090065223A (ko) * 2007-12-17 2009-06-22 한국전자통신연구원 교차로 경로안내객체 출력장치 및 그 방법

Cited By (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9448623B2 (en) 2012-10-05 2016-09-20 Elwha Llc Presenting an augmented view in response to acquisition of data inferring user activity
US10180715B2 (en) 2012-10-05 2019-01-15 Elwha Llc Correlating user reaction with at least an aspect associated with an augmentation of an augmented view
US10713846B2 (en) 2012-10-05 2020-07-14 Elwha Llc Systems and methods for sharing augmentation data
US8928695B2 (en) 2012-10-05 2015-01-06 Elwha Llc Formatting of one or more persistent augmentations in an augmented view in response to multiple input factors
US8941689B2 (en) 2012-10-05 2015-01-27 Elwha Llc Formatting of one or more persistent augmentations in an augmented view in response to multiple input factors
US9077647B2 (en) 2012-10-05 2015-07-07 Elwha Llc Correlating user reactions with augmentations displayed through augmented views
US9105126B2 (en) 2012-10-05 2015-08-11 Elwha Llc Systems and methods for sharing augmentation data
US9111384B2 (en) 2012-10-05 2015-08-18 Elwha Llc Systems and methods for obtaining and using augmentation data and for sharing usage data
US9111383B2 (en) 2012-10-05 2015-08-18 Elwha Llc Systems and methods for obtaining and using augmentation data and for sharing usage data
US9674047B2 (en) 2012-10-05 2017-06-06 Elwha Llc Correlating user reactions with augmentations displayed through augmented views
US10665017B2 (en) 2012-10-05 2020-05-26 Elwha Llc Displaying in response to detecting one or more user behaviors one or more second augmentations that are based on one or more registered first augmentations
US10269179B2 (en) 2012-10-05 2019-04-23 Elwha Llc Displaying second augmentations that are based on registered first augmentations
US9141188B2 (en) 2012-10-05 2015-09-22 Elwha Llc Presenting an augmented view in response to acquisition of data inferring user activity
US9671863B2 (en) 2012-10-05 2017-06-06 Elwha Llc Correlating user reaction with at least an aspect associated with an augmentation of an augmented view
US10254830B2 (en) 2012-10-05 2019-04-09 Elwha Llc Correlating user reaction with at least an aspect associated with an augmentation of an augmented view
US10109075B2 (en) 2013-03-15 2018-10-23 Elwha Llc Temporal element restoration in augmented reality systems
WO2014151366A1 (fr) * 2013-03-15 2014-09-25 Elwha Llc Sélection, glissement et dépose en réalité mixte pour systèmes de réalité augmentée
US10025486B2 (en) 2013-03-15 2018-07-17 Elwha Llc Cross-reality select, drag, and drop for augmented reality systems
US9639964B2 (en) 2013-03-15 2017-05-02 Elwha Llc Dynamically preserving scene elements in augmented reality systems
US10628969B2 (en) 2013-03-15 2020-04-21 Elwha Llc Dynamically preserving scene elements in augmented reality systems
DE102013210789A1 (de) 2013-06-10 2014-12-11 Robert Bosch Gmbh Augmented-Reality-System und Verfahren zum Erzeugen und Anzeigen von Augmented-Reality-Objektdarstellungen für ein Fahrzeug
EP2813999A2 (fr) 2013-06-10 2014-12-17 Robert Bosch Gmbh Système à réalité augmentée et procédé de production et d'affichage de représentations d'objet à réalité augmentée pour un véhicule

Also Published As

Publication number Publication date
KR101188105B1 (ko) 2012-10-09
KR20120092352A (ko) 2012-08-21
WO2012108721A3 (fr) 2012-12-20

Similar Documents

Publication Publication Date Title
WO2012108721A2 (fr) Procédé et dispositif pour produire une réalité augmentée au moyen de données images
JP3437555B2 (ja) 特定点検出方法及び装置
KR101269981B1 (ko) 부감 화상 생성장치, 부감 화상 생성방법 및 기록매체
US20140285523A1 (en) Method for Integrating Virtual Object into Vehicle Displays
CN104539925B (zh) 基于深度信息的三维场景增强现实的方法及系统
KR20220057577A (ko) 도로 상황 프롬프트 방법, 장치, 전자 기기, 저장 매체 및 프로그램 제품
JP2005268847A (ja) 画像生成装置、画像生成方法、および画像生成プログラム
CN104776849A (zh) 车辆定位装置与方法
JPWO2014132747A1 (ja) 物体検知装置
JP6820561B2 (ja) 画像処理装置、表示装置、ナビゲーションシステム、画像処理方法及びプログラム
JP2011170599A (ja) 屋外構造物計測装置及び屋外構造物計測方法
JP7680599B2 (ja) 情報処理装置および情報処理方法
CN113807282A (zh) 一种数据处理方法、设备以及可读存储介质
KR100934904B1 (ko) 거리 추정 장치 및 추정 방법
KR101285075B1 (ko) 자이로스코프의 센서 정보와 차선 정보를 이용한 증강현실 뷰 모드 표현 방법 및 그 장치
CN112484743A (zh) 车载hud融合实景导航显示方法及其系统
WO2018101746A2 (fr) Appareil et procédé de reconstruction d'une zone bloquée de surface de route
JPWO2020039897A1 (ja) 駅監視システム及び駅監視方法
JP2014013989A (ja) 拡張現実システム
JP2013061597A (ja) 車両用危険場面再現装置
KR101779423B1 (ko) 영상처리방법 및 영상처리장치
JP5196426B2 (ja) ナビゲーション装置
JP4696925B2 (ja) 画像処理装置
KR101351611B1 (ko) 실시간 영상을 이용하는 네비게이터
EP4506905A1 (fr) Procédé et appareil de traitement d'image, et procédé et appareil d'apprentissage de modèle

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12744612

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 17/12/2013)

122 Ep: pct application non-entry in european phase

Ref document number: 12744612

Country of ref document: EP

Kind code of ref document: A2