WO2018056802A1 - Procédé d'estimation de valeur de profondeur tridimensionnelle à partir d'images bidimensionnelles - Google Patents

Procédé d'estimation de valeur de profondeur tridimensionnelle à partir d'images bidimensionnelles Download PDF

Info

Publication number
WO2018056802A1
WO2018056802A1 PCT/MY2017/050057 MY2017050057W WO2018056802A1 WO 2018056802 A1 WO2018056802 A1 WO 2018056802A1 MY 2017050057 W MY2017050057 W MY 2017050057W WO 2018056802 A1 WO2018056802 A1 WO 2018056802A1
Authority
WO
WIPO (PCT)
Prior art keywords
dimensional
feature point
image
depth value
dimensional feature
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/MY2017/050057
Other languages
English (en)
Inventor
Rahmita Wirza O.K. Rahmat
Ng SENG BENG
Fatimah KHALID
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Universiti Putra Malaysia (UPM)
Original Assignee
Universiti Putra Malaysia (UPM)
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Universiti Putra Malaysia (UPM) filed Critical Universiti Putra Malaysia (UPM)
Publication of WO2018056802A1 publication Critical patent/WO2018056802A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G06T7/579—Depth or shape recovery from multiple images from motion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/269—Analysis of motion using gradient-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image

Definitions

  • This invention relates to a method for estimating depth value from at least two images, and more particularly to a method of three-dimensional depth value estimation from at least two images using optical flow and trigonometry method.
  • Three dimensional (3D) modeling of physical objects can be very useful in many areas, such as computer graphics and animation, robot vision, reverse engineering, and medical.
  • 3D modeling can be represented from the scratch using modeling software, or digitized from real world objects.
  • the basic information required for this representation is the x, y, and z coordinates. Further manipulation of these coordinates can deduce the objects' dimensions (width, height, and depth). Other attributes such as the models' surface colour, texture, lighting, shading and shadow contribute to a more realistic representation.
  • Conventional digitization methods utilize Coordinate Measuring Machine (CMMs) or laser scanners to obtain coordinate for each good feature from an object to a digital data. Nevertheless, both of these devices are very costly and require a certain amount of technical knowledge during usage and maintenance.
  • CCMMs Coordinate Measuring Machine
  • US Patent No. 7,342,669 B2 disclosed a 3D measurement system that uses laser-light and estimates depth information of an object by applying a triangulation method.
  • the system consists of an apparatus comprising a laser projecting, an image capturing device and a computer.
  • LEDs light-emitting diodes
  • a user can quickly acquire a precise 3D shape without using complex equipment and can also interactively check the regions whose positions are not measured during the measuring process by displaying the acquired 3D positions in real-time.
  • measurement of 3D shape of the target object becomes efficient.
  • said laser projecting device may face problems on objects with shiny surfaces or objects that do not reflect light, which include black colour and transparent surface.
  • the user may have the difficulties to verify the depth of the object.
  • US Patent No. 8,330,803 B2 disclosed a method for 3D digitization of an object, in which a plurality of camera images of the object are recorded and assembled to determine the 3D coordinates of the object.
  • An apparatus for performing the method for 3D digitization of the object comprises a projector and one or more cameras.
  • the projector projects a light pattern onto the object, in more particular white -light strips.
  • 2D feature points from the plurality of camera images of the objects are determined without human intervention.
  • the 2D point correspondences between the 2D feature points of a picture and the 2D feature points of another picture are determined. Several of these 2D point correspondences are selected, and an associated 3D transformation is determined.
  • US Patent No. 7,573,475 B2 disclosed a method of converting 2D image to 3D image. The method includes receiving a first 2D image comprising image data, where the first 2D image is captured from a first camera location. The method also includes projecting at least a portion of the first 2D image onto computer-generated geometry. The image data has depth value associated with the computer-generated geometry.
  • the system includes rendering, using the computer-generated geometry and a second camera location differs from the first camera location, a second 2D image that is stereoscopically complementary to the first 2D image, and infilling image data that absent from the second 2D image.
  • the cited patent uses two different images captured from two different locations for example image from the left and the right of the object. Then, geometry map is built for both objects and compared to each other to calculate depth for the geometry map. Geometry map is accurate to be used if the object is simple but when the object is complicated, said method becomes inaccurate to determine the depth of the object.
  • the present invention relates to a method for estimating three-dimensional depth value from two-dimensional images, characterised by the steps of placing an object on a rotatable plate; acquiring a first view of the object comprising a first image and a second image of the object, wherein the first image of the object is captured at an angle between 0° to 360°, and the second image is captured at an angle in a range of 1 ° to 35° relative to the first image; obtaining two-dimensional feature point coordinates by applying Good Features to Track technique and extracting colour information, simultaneously, from the first image and the second image; filtering two-dimensional feature point coordinates; applying Pyramidal Lucas-Kanade Optical Flow technique to obtain displacement magnitudes from the two-dimensional feature point coordinates; calculating World Coordinate; estimating the three-dimensional depth value of the acquired images; and implementing inverse perspective mapping algorithm on the three-dimensional feature point coordinates.
  • Fig. 1 is a flow chart of a method for estimating three-dimensional depth value from a two-dimensional image
  • Fig. 2 is diagram showing preferred arrangement of a kit for estimating three-dimensional depth value. Detailed Description of the Invention
  • the present invention relates to a method for estimating three-dimensional depth value from two-dimensional images, characterised by the steps of:
  • acquiring a first view of the object (102) comprising a first image and a second image of the object (102), wherein the first image of the object (102) is captured at an angle between 0° to 360°, and the second image is captured at an angle in a range of 1 ° to 35° relative to the first image;
  • zw is the three-dimensional depth value for the two-dimensional feature point coordinate (x w1 , y w1 ),
  • objdist is a distance of center of rotation (COR) from the image capture apparatus (101 ) (optic center),
  • (x w 2, yw2) is a matching coordinate of the two-dimensional feature point in the second image
  • a is a rotation angle
  • At least two views of the object (102) are acquired to reflect all sides of the object (102).
  • said first image is captured at an angle of 0°, 60°, 120°, 180°, 240°, 360° or a combination thereof.
  • the three-dimensional feature point coordinates from all views are merged and collected into a single set of three-dimensional feature point coordinates.
  • the views are merged by using an inverse rotation matrix with equation (8):
  • further filtering step on the single set of three-dimensional feature point coordinates is required to remove redundant points using a second filtering means.
  • the colour information is extracted using equations (2), (3) and (4):
  • Bi,j is a blue colour information of pixel (/ ' ,/)
  • Gi,j is a green colour information of pixel (/,/)
  • Ri,j is a red colour information of pixel (/,/).
  • the world coordinate is obtained using equations (5) and (6): (5)
  • (xpi, y P i) is a projected coordinate for the two-dimensional feature point coordinate i on the acquired image.
  • the inverse perspective mapping algorithm is implemented using equation (7): wherein,
  • ⁇ x'wi, y'wi) is a corrected coordinate for the two-dimensional feature point coordinate from the first image
  • (x w , 1 ywi) is the world coordinate for the two-dimensional feature point coordinate in the first image.
  • z 1 is an approximated three-dimensional depth value for the two-dimensional feature point coordinate (x w , 1 ywi),
  • objdist is a distance of center of rotation (COR) from the image capture apparatus (101 ) (optic center),
  • the step of filtering two-dimensional feature point coordinates using the first filtering means is by applying Euclidean distance method.
  • the estimation of three-dimensional depth value is carried out in a controlled environment includes controlling lighting and background colour.
  • the estimation of three-dimensional depth value is carried out in an open space.
  • further filtering step on the three-dimensional feature point coordinates is carried out to eliminate noise using a third filtering means, before the merging step.
  • a kit (100) mainly comprises a rotatable plate (103) and an image capture apparatus (101 ), where an object (102) is a blue toy bird which placed on the rotatable plate (103) with a distance from the image capture apparatus (101 ).
  • the blue toy bird has a width, height and depth of 134.03 mm, 94.90 mm, 35.12 mm respectively.
  • the distance from the image capture apparatus (101 ) to a centre of rotatable plate (103) is set at a length that is able to capture the whole image of the object.
  • An embodiment shows either use of two pieces of 180° half circle protractor for forming a circle or one piece of 360° protractor, which is placed underneath the rotatable plate (103) for ease of reference when measuring the angle of rotation of the rotatable plate (103).
  • the kit (100) of the present invention is for estimating three-dimensional depth value of the object (102) by carrying a method as shown in Figure 1 .
  • the kit (100) is placed in a controlled environment which includes controlling lighting and background colour.
  • the background colour is preferably black colour to eliminate reflections during image capturing.
  • a first image of the object (102) is acquired by rotating the rotatable plate (103) to a predetermined angle between 0° to 360°.
  • said predetermined angle is 0°, 60°, 120°, 180°, 240° and 360°.
  • a combination of predetermined angle may be selected for acquiring multiple images at different views of the object (102), for example 0° and 180° in a pair angles and 60° and 240°in a pair angles.
  • the pair angles is preferably a vertically opposite angles.
  • There is at least two views of the object (102) are acquired to obtain at least four images to reflect all sides of the object (102).
  • the rotatable plate (103) is rotated to an angle in a range of 1 ° to 35° relative to the predetermined angle of the first image for capturing a second image.
  • the first image and the second image are captured using the image capture apparatus (101 ), wherein the image capture apparatus (101 ) is preferably a camera and more preferably a webcam.
  • Two-dimensional feature point coordinates is obtained by applying Good Features to Track technique and the colour information is extracted from the two-dimensional feature point coordinates, simultaneously, from the first image and the second image. Said colour information is extracted using equations (2), (3), and (4) as followings:
  • Bi,j is a blue colour information of pixel (/ ' ,_/),
  • Gi,j is a green colour information of pixel (/ ' ,_/),
  • Ri,j is a red colour information of pixel (/,/).
  • the two-dimensional feature point coordinates is filtered to eliminate noise using first filtering means, where the detected two-dimensional feature point coordinates which are located further away from the centre of the images are assumed to have a higher probability of noise.
  • the two-dimensional feature point coordinates is filtered by applying Euclidean distance method.
  • detectedheight is a distance of highest point and lowest point of the object (102) detected in the acquired image, measured in pixels:
  • the three-dimensional depth value of the acquired images is estimated by calculating each of the two-dimensional feature point coordinates using equation (1 ), thereby producing three-dimensional feature point coordinates: (1) wherein,
  • z w is the three-dimensional depth value for the two-dimensional feature point coordinate (xwi, ywi),
  • objdist is a distance of center of rotation (COR) from the image capture apparatus (101 ) (optic center),
  • (x w , 1 ywi) is a coordinate of the two-dimensional feature point from the first image
  • (x w 2, yw2) is a matching coordinate of the two-dimensional feature point in the second image
  • a is a rotation angle
  • ⁇ x'wi, y'wi) is a corrected coordinate for the two-dimensional feature point coordinate from the first image
  • (x w , 1 ywi) is the world coordinate for the two-dimensional feature point coordinate in the first image.
  • zi is an approximated three-dimensional depth value for the two-dimensional feature point coordinate (xwi, ywi),
  • objdist is a distance of center of rotation (COR) from the image capture apparatus (101 ) (optic center).
  • the three-dimensional feature point coordinates from all individual views collected are merged into a single set of three-dimensional feature point coordinates.
  • the merging step is performed using an inverse rotation matrix with equation (8):
  • (x' y', z') are merged three-dimensional feature point coordinate after applying the inverse rotation matrix on three-dimensional feature point coordinate (x, y, z).
  • the single set of three-dimensional feature point coordinates is filtered to remove redundant points using a second filtering means.
  • the kit (100) is placed in an open space for carrying out the estimation of three-dimensional depth value of the object (102). If the kit (100) is placed in the open space, a further filtering step is required on the three-dimensional feature point coordinates to eliminate noise using a third filtering means, before the merging step of the present invention.
  • Point 2 is a matching two-dimensional feature point of Point 1 in the second image after the a° rotation.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

La présente invention concerne un procédé d'estimation de valeur de profondeur tridimensionnelle à partir d'images bidimensionnelles, caractérisé par les étapes consistant à placer un objet (102) sur une plaque rotative (103) ; à acquérir une première vue de l'objet (102) comprenant une première image et une seconde image de l'objet (102), la première image de l'objet (102) étant capturée à un angle compris entre 0° et 360°, et la seconde image étant capturée à un angle dans une plage de 1° à 35° par rapport à la première image ; à obtenir des coordonnées de point caractéristique bidimensionnel par application de bonnes caractéristiques à la technique de suivi et à extraire des informations de couleur, simultanément, de la première image et de la seconde image ; à filtrer des coordonnées de point caractéristique bidimensionnel ; à appliquer une technique de flux optique de Lucas-Kanade Pyramidal pour obtenir des amplitudes de déplacement à partir des coordonnées de point caractéristique bidimensionnel ; à calculer des coordonnées universelles ; à estimer la valeur de profondeur tridimensionnelle des images acquises ; et à implémenter un algorithme de mappage de perspective inverse sur les coordonnées de point caractéristique tridimensionnel.
PCT/MY2017/050057 2016-09-21 2017-09-14 Procédé d'estimation de valeur de profondeur tridimensionnelle à partir d'images bidimensionnelles Ceased WO2018056802A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
MYPI2016703427 2016-09-21
MYPI2016703427 2016-09-21

Publications (1)

Publication Number Publication Date
WO2018056802A1 true WO2018056802A1 (fr) 2018-03-29

Family

ID=61689941

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/MY2017/050057 Ceased WO2018056802A1 (fr) 2016-09-21 2017-09-14 Procédé d'estimation de valeur de profondeur tridimensionnelle à partir d'images bidimensionnelles

Country Status (1)

Country Link
WO (1) WO2018056802A1 (fr)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109978982A (zh) * 2019-04-02 2019-07-05 广东电网有限责任公司 一种基于倾斜影像的点云快速上色方法
CN110533036A (zh) * 2019-08-28 2019-12-03 湖南长城信息金融设备有限责任公司 一种票据扫描图像快速倾斜校正方法和系统
CN112241977A (zh) * 2019-07-16 2021-01-19 北京京东乾石科技有限公司 一种特征点的深度估计方法和装置
CN112771576A (zh) * 2020-05-06 2021-05-07 深圳市大疆创新科技有限公司 位置信息获取方法、设备及存储介质
CN114418996A (zh) * 2022-01-19 2022-04-29 北京林业大学 一种基于三维空间角度算法的小叶杨根部构型解析方法
CN116522556A (zh) * 2023-04-21 2023-08-01 国网冀北电力有限公司承德供电公司 一种基于二维建模的装表接电最优布线方法及终端机

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4969036A (en) * 1989-03-31 1990-11-06 Bir Bhanu System for computing the self-motion of moving images devices
US20090226094A1 (en) * 2006-09-13 2009-09-10 Pioneer Corporation Image correcting device and method, and computer program
JP2009222568A (ja) * 2008-03-17 2009-10-01 Konica Minolta Sensing Inc ３次元形状データの生成方法および装置ならびにコンピュータプログラム

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4969036A (en) * 1989-03-31 1990-11-06 Bir Bhanu System for computing the self-motion of moving images devices
US20090226094A1 (en) * 2006-09-13 2009-09-10 Pioneer Corporation Image correcting device and method, and computer program
JP2009222568A (ja) * 2008-03-17 2009-10-01 Konica Minolta Sensing Inc ３次元形状データの生成方法および装置ならびにコンピュータプログラム

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109978982A (zh) * 2019-04-02 2019-07-05 广东电网有限责任公司 一种基于倾斜影像的点云快速上色方法
CN112241977A (zh) * 2019-07-16 2021-01-19 北京京东乾石科技有限公司 一种特征点的深度估计方法和装置
CN110533036A (zh) * 2019-08-28 2019-12-03 湖南长城信息金融设备有限责任公司 一种票据扫描图像快速倾斜校正方法和系统
CN110533036B (zh) * 2019-08-28 2022-06-07 长城信息股份有限公司 一种票据扫描图像快速倾斜校正方法和系统
CN112771576A (zh) * 2020-05-06 2021-05-07 深圳市大疆创新科技有限公司 位置信息获取方法、设备及存储介质
CN114418996A (zh) * 2022-01-19 2022-04-29 北京林业大学 一种基于三维空间角度算法的小叶杨根部构型解析方法
CN116522556A (zh) * 2023-04-21 2023-08-01 国网冀北电力有限公司承德供电公司 一种基于二维建模的装表接电最优布线方法及终端机
CN116522556B (zh) * 2023-04-21 2024-05-24 国网冀北电力有限公司承德供电公司 一种基于二维建模的装表接电最优布线方法及终端机

Similar Documents

Publication Publication Date Title
US6781618B2 (en) Hand-held 3D vision system
CN108876926B (zh) 一种全景场景中的导航方法及系统、ar/vr客户端设备
CN106802138B (zh) 一种三维扫描系统及其扫描方法
Furukawa et al. Accurate camera calibration from multi-view stereo and bundle adjustment
US8452081B2 (en) Forming 3D models using multiple images
US8447099B2 (en) Forming 3D models using two images
JP5132832B1 (ja) 計測装置および情報処理装置
CN106408556B (zh) 一种基于一般成像模型的微小物体测量系统标定方法
WO2018056802A1 (fr) Procédé d'estimation de valeur de profondeur tridimensionnelle à partir d'images bidimensionnelles
CN110827392B (zh) 单目图像三维重建方法、系统及装置
US20100328308A1 (en) Three Dimensional Mesh Modeling
WO2021077720A1 (fr) Procédé, appareil et système d'acquisition de modèle tridimensionnel d'un objet, et dispositif électronique
US20060210146A1 (en) Creating 3D images of objects by illuminating with infrared patterns
CN107155341A (zh) 三维扫描系统和框架
WO2018235163A1 (fr) Dispositif d'étalonnage, graphique d'étalonnage, dispositif de génération de motif de graphique et procédé d'étalonnage
WO2012096747A1 (fr) Établissement de cartes télémétriques à l'aide de motifs d'éclairage périodiques
Ding et al. Fusing structure from motion and lidar for dense accurate depth map estimation
CN110059537B (zh) 一种基于Kinect传感器的三维人脸数据获取方法及装置
WO2020075252A1 (fr) Dispositif de traitement d'informations, programme, et procédé de traitement d'informations
CN118918265A (zh) 基于单目相机和线激光器的三维重建方法和系统
CN114494582A (zh) 一种基于视觉感知的三维模型动态更新方法
TWI595446B (zh) 擴充實境中基於深度攝影機之遮蔽邊緣品質改善方法
JP7033294B2 (ja) 撮像システム、撮像方法
JP6061631B2 (ja) 計測装置、情報処理装置、計測方法、情報処理方法、および、プログラム
Popescu et al. The modelcamera

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17853513

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17853513

Country of ref document: EP

Kind code of ref document: A1