US11715261B2 - Method for detecting and modeling of object on surface of road - Google Patents

Method for detecting and modeling of object on surface of road Download PDF

Info

Publication number
US11715261B2
US11715261B2 US17/344,405 US202117344405A US11715261B2 US 11715261 B2 US11715261 B2 US 11715261B2 US 202117344405 A US202117344405 A US 202117344405A US 11715261 B2 US11715261 B2 US 11715261B2
Authority
US
United States
Prior art keywords
road
model
scanned
vehicles
remote server
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US17/344,405
Other languages
English (en)
Other versions
US20210304492A1 (en
Inventor
Haitao Xue
Dongbing Quan
Changhong Yang
James Herbst
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Aumovio Germany GmbH
Original Assignee
Continental Automotive Technologies GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Continental Automotive Technologies GmbH filed Critical Continental Automotive Technologies GmbH
Assigned to Continental Holding China Co., Ltd. reassignment Continental Holding China Co., Ltd. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: YANG, Changhong, QUAN, Dongbing, XUE, Haitao
Assigned to CONTINENTAL AUTOMOTIVE GMBH reassignment CONTINENTAL AUTOMOTIVE GMBH ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: HERBST, JAMES
Publication of US20210304492A1 publication Critical patent/US20210304492A1/en
Assigned to QUALCOMM TECHNOLOGIES, INC. reassignment QUALCOMM TECHNOLOGIES, INC. EXCLUSIVE LICENSE Assignors: CONTINENTAL AUTOMOTIVE GMBH
Priority to US18/208,223 priority Critical patent/US20230351687A1/en
Application granted granted Critical
Publication of US11715261B2 publication Critical patent/US11715261B2/en
Assigned to Continental Automotive Technologies GmbH reassignment Continental Automotive Technologies GmbH ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: CONTINENTAL AUTOMOTIVE GMBH, Continental Holding China Co., Ltd.
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—Three-dimensional [3D] image rendering
    • G06T15/10—Geometric effects
    • G06T15/20—Perspective computation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G06T17/05—Geographic models
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G06T17/30—Polynomial surface description
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G06T7/579—Depth or shape recovery from multiple images from motion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/2628—Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/28—Indexing scheme for image data processing or generation, in general involving image processing hardware
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20021—Dividing image into blocks, subimages or windows
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30236—Traffic on road, railway or crossing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • G06T2207/30256—Lane; Road marking
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2210/00—Indexing scheme for image generation or computer graphics
    • G06T2210/56—Particle system, point based geometry or rendering
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/09—Arrangements for giving variable traffic instructions
    • G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
    • G08G1/0967—Systems involving transmission of highway information, e.g. weather, speed limits
    • G08G1/096708—Systems involving transmission of highway information, e.g. weather, speed limits where the received information might be used to generate an automatic action on the vehicle control
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/09—Arrangements for giving variable traffic instructions
    • G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
    • G08G1/0967—Systems involving transmission of highway information, e.g. weather, speed limits
    • G08G1/096766—Systems involving transmission of highway information, e.g. weather, speed limits where the system is characterised by the origin of the information transmission

Definitions

  • the invention relates to relates to a method for detecting and modelling of an object on a surface of a road. Moreover, the disclosure relates to a system for detecting and modelling of an object on a surface of a road.
  • Advanced driver assistance systems and autonomously driving cars require high precision maps of roads and other areas on which vehicles can drive. Determining a vehicle's position on a road or even within a lane of a road with an accuracy of a few centimeters cannot be achieved using conventional satellite navigation systems, for example GPS, Galileo, GLONASS, or other known positioning techniques such as triangulation and the like. However, in particular, when a self-driving vehicle moves on a road with multiple lanes, it needs to exactly determine its lateral and longitudinal position within the lane.
  • One known way to determine a vehicle's position with high precision involves one or more cameras capturing images of road markings/road paints and comparing unique features of road markings/road paints or objects along the road in the captured images with corresponding reference images obtained from a database, in which reference images the respective position of road markings/paints or objects is provided. This way of determining a position provides sufficiently accurate results only when the database provides highly accurate position data with the images and when it is updated regularly or at suitable intervals.
  • Road markings may be captured and registered by special purpose vehicles that capture images of a road while driving, or may be extracted from aerial photographs or satellite images.
  • the latter variant may be considered advantageous since a perpendicular view or top-view image shows little distortion of road markings/paints and other features on substantially flat surfaces.
  • aerial photographs and satellite images may not provide sufficient detail for generating highly accurate maps of road markings/paints and other road features. Also, aerial photographs and satellite images are less suitable for providing details on objects and road features that are best viewed from a ground perspective.
  • the embodiments are providing a method for detecting and modelling of an object on a surface of a road, which allows to determine an accurate three-dimensional (3D) position of the object on the surface of the road.
  • Embodiments may further provide a system for detecting and modelling of an object on a surface of a road configured to provide an accurate three-dimensional position of the object on the surface of the road.
  • One embodiments relates to a method for detecting and modelling of an object on a surface of the road, in a first step, the road is scanned. In a subsequent second step, a 3D model of the scanned road is generated. The 3D model contains a description (data representation) of a 3D surface of the road. In a subsequent third step a top-view image of the road is created.
  • the object is detected on the surface of the road by evaluating the top-view image of the road.
  • the detected object is projected on the surface of the road in the 3D model of the scanned road.
  • the object projected on the surface of the road in the 3D model of the scanned road is modelled.
  • a method for detecting and modelling of an object on a surface of a road merges information regarding the 3D road surface and detected objects or road paints on the surface of the road from distributed vehicles driving along the road at different times in order to adjust and refine the road surface estimation and road paint/object detecting.
  • the framework of the method for detecting and modelling of an object on a surface of a road can be divided into four basic parts.
  • a road surface is estimated by each vehicle driving along the road.
  • Each vehicle will report the respective detected road surface to a remote server.
  • the remote server the different information obtained from the plurality of vehicles driving along the road are conflated. As a result, a more accurate road surface model is calculated in the remote server.
  • the course of the road captured by a forward-facing camera unit of a vehicle is transformed from the front camera view into a bird's-eye view.
  • an inverse perspective transformation is done first, before part of the image will be extracted to combine into a large image of the complete course of the road.
  • An object on a surface of the road or a road painting will be detected in the top-view/bird's-eye view image of the scanned road.
  • a 3D object/paint projection is performed from the 2D top-view/bird's-eye view image to the 3D model of the road surface.
  • the 3D model of the road is evaluated to obtain a 3D position of the object/road paint and a logical information of the object/road paint.
  • the detected object/road paint on the surface of the road is modelled in a 3D manner.
  • a Non-Uniform Rational B-Spline (NURBS) technique may be used for the 3D modelling of the detected object/road paint.
  • the NURBS curve fitting algorithm can advantageously represent any form of a curve so that the NURBS algorithm allows to represent any object/road paint on the surface of the road precisely.
  • a conventional method for modelling an object/road paint on a surface of a road usually represents a detected object/road paint by polylines which consumes a lot of memory capacitance.
  • the NURBS algorithm will extremely compress the data.
  • An embodiment relates to a system for detecting and modelling of an object on a surface of a road.
  • the system includes a plurality of vehicles driving along the road, and a remote server being spatially located far away from the plurality of the vehicles.
  • Each of the vehicles includes a respective camera unit to scan the road.
  • each of the vehicles may be configured to generate a 3D model of the scanned road.
  • the 3D model contains a description of the surface of the road.
  • Each of the vehicles may be configured to create a respective individual top-view of the road and to forward the respective individual top-view of the road to the remote server.
  • the remote server may be configured to create a top-view image of the scanned road by evaluating and conflating the respective individual top-view images of the scanned road.
  • the remote server may further be configured to detect an object on the surface of the road by evaluating the top-view image of the road.
  • the remote server may be configured to project the detected object on the surface of the road in the 3D model of the scanned road.
  • the remote server may further be configured to model the object projected on the surface of the road in the 3D model of the scanned road.
  • FIG. 1 shows a flowchart of a method for detecting and modelling of an object on a surface of a road
  • FIG. 2 shows a simplified block diagram of a system configured to detect and model an object on a surface of a road
  • FIG. 3 A shows a first simplified scene captured by a camera unit and a selection of an area of a captured picture of a road for further processing
  • FIG. 3 B shows a second simplified scene captured by a camera unit and a selection of an area of the captured picture of a road for further processing.
  • FIG. 1 illustrating a sequence of different steps of the method as well as with reference to FIG. 2 illustrating components of a system for detecting and modelling of an object on a surface of a road.
  • step S 1 of the method the road 40 along which a vehicle is driving is scanned or optically examined or scrutinized by the vehicle.
  • a plurality of vehicles 10 a , 10 b and 10 c drive along the road 40 and scan the course of the road during the driving process.
  • each of the vehicles includes a respective optical camera unit 11 .
  • the camera unit 11 may be a vehicle-mounted, forwardly-facing camera.
  • the respective camera unit 11 may include a CCD sensor array.
  • a simple mono-camera may be provided.
  • a stereo camera which may have two or more imaging sensors mounted at a distance (separated) from each other, may be used.
  • FIG. 3 A and FIG. 3 B show two subsequent images 50 a , 50 b of the road 40 captured by the camera unit 11 .
  • a 3D model of the so-scanned road 40 is generated.
  • the 3D model contains a description of a 3D surface of the road 40 .
  • the process of generation of a 3D model of the scanned road 40 is enabled even if the cameral unit 11 is configured as a mono-camera.
  • the generated 3D model of the scanned road 40 may be construed or configured as a point cloud.
  • a dense or semi-dense point cloud may be generated by evaluating the captured pictures with a respective processor unit 12 (of each of the vehicles 10 a , 10 b and 10 c ) while driving along the road.
  • degrees of density of the point cloud may be defined, for example, in accord with the common understanding of such degrees in related art.
  • a point cloud is considered to be sparse when its density is from about 0.5 pts/m 2 to about 1 pts/m 2 ; the density of the low-density point cloud is substantially between 1 pts/m 2 and 2 pts/m 2 ; the medium density point cloud may be characterized by the density of about 2 pts/m 2 to 5 pts/m 2 ; and the high density point cloud has a density from about 5 pts/m 2 to about 10 pts/m 2 .
  • the point cloud is considered to be extremely dense if its density exceeds 10 pts/m 2 .
  • a respective individual 3D model of the scanned road 40 may be generated by each of the vehicles 10 a , 10 b and 10 c .
  • the respective individual 3D model may be forwarded by each of the vehicles 10 a , 10 b and 10 c to a remote server 20 that is located far away (that is, spatially separated from) from these vehicles 10 a , 10 b and 10 c .
  • each of the vehicles 10 a , 10 b and 10 c includes a communication system 13 .
  • Each of the individual 3D models received from the vehicles 10 a , 10 b and 10 c is stored in a storage unit 22 of the remote server 20 .
  • the remote server 20 generates the 3D model of the scanned road 40 by evaluating and conflating (merging) the respective individual 3D models of the scanned road 40 received from the vehicles 10 a , 10 b and 10 c
  • the various point clouds generated by each of the vehicles while driving along the road are matched (that is, fitted, for example by stretching and/or bending the point clouds, as appropriate) by a processor unit 21 of the remote server 20 to provide the 3D model of the road 40 .
  • the 3D model contains information about the road surface so that road surface estimation may be performed by the remote server 20 .
  • An accurate road surface model of the scanned road may be constructed by the processor unit 21 by conflating and matching the various individual 3D models generated by each of the vehicles 10 a , 10 b and 10 c.
  • a top-view/bird's-eye view image of (that is, an image formed a vintage point directly above) the road 40 is created.
  • a respective individual top-view/bird's-eye view image of the scanned road 40 is created by each of the vehicles 10 a , 10 b and 10 c .
  • the respective individual top-view/bird's-eye view image is forwarded by each of the communication systems 13 of the vehicles 10 a , 10 b and 10 c to the remote server 20 .
  • the remote server 20 may create the top-view image of the scanned road 40 by evaluating and conflating the respective individual top-view images of the scanned road 40 .
  • Objects located on the surface of the road for example road paints, may be detected by the processor unit 21 by evaluating the 3D model of the scanned road 40 and the top-view image of the scanned road 40 .
  • FIG. 3 A shows a first image/picture 50 a of a simplified scene as captured by the camera unit 11 of one of the vehicles 10 a , 10 b and 10 c driving along the road 40 .
  • FIG. 3 B shows a second image/picture 50 b of the simplified scene captured by the camera unit 11 of the same of the vehicles 10 a , 10 b and 10 c a short time later than the first picture.
  • a dotted line in each of the captured images 50 a , 50 b designates/surrounds a zone (or region, or portion) of each of the images 50 a , 50 b in which the camera optics of the camera unit 11 cause minimum optical distortion.
  • the zone in which the camera optics cause minimum distortion is located in the central area of each of the captured pictures 50 a , 50 b.
  • FIG. 3 B the vehicle has already moved forward a certain distance (judging by comparison with the scene shown in FIG. 3 A ) so that an object/road paint 60 located on the surface of the road 40 , for example a directional arrow, is now repositioned in the foreground.
  • an object/road paint 60 located on the surface of the road 40 for example a directional arrow, is now repositioned in the foreground.
  • a traffic sign 30 shown in FIG. 3 A in the background region has moved in a central area of the image 50 b .
  • a respective first area 51 of the captured image 50 a is selected by each of the vehicles 10 a , 10 b and 10 c from the first image 50 a to be is located in a zone of the first image 50 a in which the optics of the camera unit 11 cause minimum distortion.
  • a respective second area 52 of the captured image 50 b is selected by each of the vehicles 10 a , 10 b and 10 c from the second image 50 b to be located in a zone of the second image 50 b in which the optics of the camera unit 11 cause minimum distortion.
  • the respective first selected areas 51 are then transformed by each of the vehicles 10 a , 10 b and 10 c to a respective first top-view perspective of the scanned road.
  • the respective second selected areas 52 are then transformed by each of the vehicles 10 a , 10 b and 10 c to respective second top-view perspectives of the scanned road.
  • these respective first and second top-view perspectives are stitched together (for example, with the use of an approach known in the art) by each of the vehicles 10 a , 10 b and 10 c.
  • the transformation to obtain the top-view perspective of the respective selected area and the step of stitching together the top-view perspectives may be executed by the respective processor unit 12 of each of the vehicles 10 a , 10 b and 10 c .
  • the transformation may be, for example, an inverse perspective transformation which transforms each of the areas 51 , 52 from the view of the camera unit 11 into the bird's-eye view.
  • the object/road paint 60 on the surface of the road 40 (illustrated in this example by the directional arrow shown in FIGS. 3 A and 3 B ) is detected by evaluating the top-view image of the road 40 (while searching for objects and/or changes in color and/or contours of colored portions of the top-view image).
  • This step allows to detect objects located on the surface of the road 40 such as road paints or other objects, for example, a cover of a water drain.
  • a step S 5 of the method the detected object 60 is projected on the surface of the road 40 in the 3D model of the scanned road 40 .
  • the pictures 50 a , 50 b of the road captured by the camera unit 11 , the top-view image of the road, and the point cloud of the 3D model of the scanned road are compared and matched by the processor unit 21 of the remote server 20 .
  • the matching process is configured to enable to project a detected object 60 in the 3D model of the scanned road 40 .
  • a 3D position and a logical information about the object 60 is determined after having projected the object 60 detected in the top-view image of the road 40 on the surface of the road 40 in the 3D model of the scanned road.
  • the object 60 projected on the surface of the road 40 in the 3D model of the scanned road is modelled.
  • a mathematical curve fitting algorithm may be used.
  • a Non-Uniform Rational B-Spline (NURBS) technique may be used to perform curve fitting.
  • NURBS methodology can represent any form of a curve so that it is enabled to represent a detected object/road paint precisely.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Computer Graphics (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Remote Sensing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Algebra (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Mathematical Physics (AREA)
  • Pure & Applied Mathematics (AREA)
  • Computing Systems (AREA)
  • Traffic Control Systems (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
US17/344,405 2018-12-13 2021-06-10 Method for detecting and modeling of object on surface of road Active 2039-05-15 US11715261B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/208,223 US20230351687A1 (en) 2018-12-13 2023-06-09 Method for detecting and modeling of object on surface of road

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/120886 WO2020118619A1 (fr) 2018-12-13 2018-12-13 Procédé de détection et de modélisation d'un objet sur la surface d'une route

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/120886 Continuation WO2020118619A1 (fr) 2018-12-13 2018-12-13 Procédé de détection et de modélisation d'un objet sur la surface d'une route

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/208,223 Continuation US20230351687A1 (en) 2018-12-13 2023-06-09 Method for detecting and modeling of object on surface of road

Publications (2)

Publication Number Publication Date
US20210304492A1 US20210304492A1 (en) 2021-09-30
US11715261B2 true US11715261B2 (en) 2023-08-01

Family

ID=71076195

Family Applications (2)

Application Number Title Priority Date Filing Date
US17/344,405 Active 2039-05-15 US11715261B2 (en) 2018-12-13 2021-06-10 Method for detecting and modeling of object on surface of road
US18/208,223 Abandoned US20230351687A1 (en) 2018-12-13 2023-06-09 Method for detecting and modeling of object on surface of road

Family Applications After (1)

Application Number Title Priority Date Filing Date
US18/208,223 Abandoned US20230351687A1 (en) 2018-12-13 2023-06-09 Method for detecting and modeling of object on surface of road

Country Status (7)

Country Link
US (2) US11715261B2 (fr)
EP (1) EP3895135A4 (fr)
JP (1) JP2022513830A (fr)
KR (1) KR20210102953A (fr)
CN (1) CN113196341A (fr)
CA (1) CA3122865A1 (fr)
WO (1) WO2020118619A1 (fr)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20220157196A (ko) 2021-05-20 2022-11-29 삼성전자주식회사 이미지 처리 방법 및 그 전자 장치
US12406395B2 (en) * 2021-12-07 2025-09-02 Adasky, Ltd. Vehicle to infrastructure extrinsic calibration system and method
WO2023182762A1 (fr) * 2022-03-21 2023-09-28 엘지전자 주식회사 Dispositif de transmission de données de nuage de points, procédé de transmission de données de nuage de points, dispositif de réception de données de nuage de points, et procédé de réception de données de nuage de points
KR102701224B1 (ko) * 2023-04-25 2024-08-30 국방과학연구소 다양한 험지 환경에서의 주행가능영역 및 주행비용 예측을 위한 영상 및 3차원 라이다 점군 데이터 가공 장치 및 방법

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130010074A1 (en) 2007-02-16 2013-01-10 Waseda University Measurement apparatus, measurement method, and feature identification apparatus
CN104766366A (zh) 2015-03-31 2015-07-08 东北林业大学 一种三维虚拟现实演示的建立方法
US20170203692A1 (en) 2014-05-08 2017-07-20 Continental Automotive Gmbh Method and device for the distortion-free display of an area surrounding a vehicle
WO2018015811A1 (fr) 2016-07-21 2018-01-25 Mobileye Vision Technologies Ltd. Externalisation ouverte et distribution d'une carte éparse, et mesures de voie pour la navigation d'un véhicule autonome
US20180067494A1 (en) 2016-09-02 2018-03-08 Delphi Technologies, Inc. Automated-vehicle 3d road-model and lane-marking definition system
US20180188026A1 (en) 2016-12-30 2018-07-05 DeepMap Inc. Visual odometry and pairwise alignment for high definition map creation
US20180224863A1 (en) 2016-01-18 2018-08-09 Tencent Technology (Shenzhen) Company Limited Data processing method, apparatus and terminal
US20190156128A1 (en) * 2017-11-20 2019-05-23 Here Global B.V. Automatic localization geometry generator for stripe-shaped objects
US20200109954A1 (en) * 2017-06-30 2020-04-09 SZ DJI Technology Co., Ltd. Map generation systems and methods

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105069395B (zh) * 2015-05-17 2018-10-09 北京工业大学 基于地面三维激光扫描技术的道路标线自动识别方法
EP3131020B1 (fr) * 2015-08-11 2017-12-13 Continental Automotive GmbH Système et procédé de traitement de données d'objet en deux étapes par un véhicule et un serveur de données pour générer, mettre à jour et distribuer des données de précision de propriété de route
CN105678285B (zh) * 2016-02-18 2018-10-19 北京大学深圳研究生院 一种自适应的道路鸟瞰图变换方法和道路车道检测方法

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130010074A1 (en) 2007-02-16 2013-01-10 Waseda University Measurement apparatus, measurement method, and feature identification apparatus
US20170203692A1 (en) 2014-05-08 2017-07-20 Continental Automotive Gmbh Method and device for the distortion-free display of an area surrounding a vehicle
CN104766366A (zh) 2015-03-31 2015-07-08 东北林业大学 一种三维虚拟现实演示的建立方法
US20180224863A1 (en) 2016-01-18 2018-08-09 Tencent Technology (Shenzhen) Company Limited Data processing method, apparatus and terminal
WO2018015811A1 (fr) 2016-07-21 2018-01-25 Mobileye Vision Technologies Ltd. Externalisation ouverte et distribution d'une carte éparse, et mesures de voie pour la navigation d'un véhicule autonome
US20180067494A1 (en) 2016-09-02 2018-03-08 Delphi Technologies, Inc. Automated-vehicle 3d road-model and lane-marking definition system
US20180188026A1 (en) 2016-12-30 2018-07-05 DeepMap Inc. Visual odometry and pairwise alignment for high definition map creation
US20200109954A1 (en) * 2017-06-30 2020-04-09 SZ DJI Technology Co., Ltd. Map generation systems and methods
US20190156128A1 (en) * 2017-11-20 2019-05-23 Here Global B.V. Automatic localization geometry generator for stripe-shaped objects

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
International Searching Authority. International Search Report and Written Opinion for application PCT/CN2018/120886. dated Sep. 11, 2019. 9 pages.
Supplementary European Search Report—EP18943309—Search Authority—Berlin—dated Jul. 26, 2022.

Also Published As

Publication number Publication date
CN113196341A (zh) 2021-07-30
JP2022513830A (ja) 2022-02-09
EP3895135A1 (fr) 2021-10-20
CA3122865A1 (fr) 2020-06-18
KR20210102953A (ko) 2021-08-20
WO2020118619A1 (fr) 2020-06-18
US20230351687A1 (en) 2023-11-02
US20210304492A1 (en) 2021-09-30
EP3895135A4 (fr) 2022-08-24

Similar Documents

Publication Publication Date Title
US11715261B2 (en) Method for detecting and modeling of object on surface of road
CN110832275B (zh) 基于双目图像更新高分辨率地图的系统和方法
De Silva et al. Fusion of LiDAR and camera sensor data for environment sensing in driverless vehicles
CN110859044B (zh) 自然场景中的集成传感器校准
US10909395B2 (en) Object detection apparatus
KR102249769B1 (ko) 2차원 영상의 픽셀별 3차원 좌표값 추정 방법 및 이를 이용한 자율주행정보 추정 방법
US12061252B2 (en) Environment model using cross-sensor feature point referencing
CN111436216A (zh) 用于彩色点云生成的方法和系统
CN111856491A (zh) 用于确定车辆的地理位置和朝向的方法和设备
KR20090064679A (ko) 이종 센서 통합 모델링에 의한 수치 사진 측량 방법 및장치
Parra et al. Robust visual odometry for vehicle localization in urban environments
JP6552448B2 (ja) 車両位置検出装置、車両位置検出方法及び車両位置検出用コンピュータプログラム
Vu et al. Traffic sign detection, state estimation, and identification using onboard sensors
CN113874681B (zh) 点云地图质量的评估方法和系统
US20210304518A1 (en) Method and system for generating an environment model for positioning
NL2016718B1 (en) A method for improving position information associated with a collection of images.
Novikov et al. Vehicle geolocalization from drone imagery
Yang et al. Road detection by RANSAC on randomly sampled patches with slanted plane prior
WO2022133986A1 (fr) Procédé et système d'estimation de précision

Legal Events

Date Code Title Description
FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

AS Assignment

Owner name: CONTINENTAL HOLDING CHINA CO., LTD., CHINA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:XUE, HAITAO;QUAN, DONGBING;YANG, CHANGHONG;SIGNING DATES FROM 20210625 TO 20210628;REEL/FRAME:056785/0516

Owner name: CONTINENTAL AUTOMOTIVE GMBH, GERMANY

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:HERBST, JAMES;REEL/FRAME:056785/0510

Effective date: 20210705

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

AS Assignment

Owner name: QUALCOMM TECHNOLOGIES, INC., CALIFORNIA

Free format text: EXCLUSIVE LICENSE;ASSIGNOR:CONTINENTAL AUTOMOTIVE GMBH;REEL/FRAME:063396/0428

Effective date: 20220615

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE

AS Assignment

Owner name: CONTINENTAL AUTOMOTIVE TECHNOLOGIES GMBH, GERMANY

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:CONTINENTAL AUTOMOTIVE GMBH;CONTINENTAL HOLDING CHINA CO., LTD.;REEL/FRAME:071099/0011

Effective date: 20250318