WO2012160224A1 - Système et procédé de capture et de traitement d'images tridimensionnelles avec mouvement - Google Patents

Système et procédé de capture et de traitement d'images tridimensionnelles avec mouvement Download PDF

Info

Publication number
WO2012160224A1
WO2012160224A1 PCT/ES2012/070288 ES2012070288W WO2012160224A1 WO 2012160224 A1 WO2012160224 A1 WO 2012160224A1 ES 2012070288 W ES2012070288 W ES 2012070288W WO 2012160224 A1 WO2012160224 A1 WO 2012160224A1
Authority
WO
WIPO (PCT)
Prior art keywords
cameras
dimensional
captured
image
camera
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/ES2012/070288
Other languages
English (en)
Spanish (es)
Inventor
Matías LOPEZ IGLESIAS
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
UNIVERSIDAD EUROPEA MIGUEL DE CERVANTES
Original Assignee
UNIVERSIDAD EUROPEA MIGUEL DE CERVANTES
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by UNIVERSIDAD EUROPEA MIGUEL DE CERVANTES filed Critical UNIVERSIDAD EUROPEA MIGUEL DE CERVANTES
Publication of WO2012160224A1 publication Critical patent/WO2012160224A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/243—Image signal generators using stereoscopic image cameras using three or more two-dimensional [2D] image sensors
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/20—Three-dimensional [3D] animation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Definitions

  • the object of the present invention is about a system and method of capturing and processing three-dimensional images with movement, which captures the two-dimensional images through a plurality of cameras arranged around the object, and by intersecting the projections of said object. Transforms into a three-dimensional representation for later editing of textures, animation and postproduction. Background of the invention
  • the light sources used must be special because they must be able to emit structured light and are usually lasers.
  • the depth information is acquired from the relative displacement of the different points of the light line. This method needs to process multiple images to reconstruct the entire scene, either by moving the light source or leaving the lighting system fixed and touring the scene. In addition, they do not allow capturing the image from different points of view without the need for an external light.
  • Document ES 2306089 T3 describes an electronic photo-optical system capable of recognizing the external contour of any type of object and transforming it into a set of Cartesian coordinates; and to transmit the mathematical information of the geometry of these objects, through networks, to a personal computer (to make a three-dimensional graphical representation of the object and to make possible afterwards the processing of the geometry of the object by means of assisted design programs by computer in three dimensions, 3D CAD) and directly to an online device, of the type fax player, to reproduce in plastic, composite material or similar to paper that have the same volume as the recognized object, all automatically and following a default logic.
  • Description of the invention describes an electronic photo-optical system capable of recognizing the external contour of any type of object and transforming it into a set of Cartesian coordinates; and to transmit the mathematical information of the geometry of these objects, through networks, to a personal computer (to make a three-dimensional graphical representation of the object and to make possible afterwards the processing of the geometry of the object by means of assisted design programs by computer in three dimensions,
  • the proposed system tries to parameterize the coordinates of the points based on the intersection of the lines generated by the projections of the camera captures. That is, each of the pixels that each camera captures is represented as a line that, when intersecting with the line calculated by another or other cameras located with different angle of attack around the captured object, determines a three-dimensional voxel.
  • This triangulation system does not use structured light, with the advantage that it can be captured from different points of view without the need for a laser type light. It also allows the image to be captured consecutively so that sequential movement can be synthesized.
  • the proposed system is an object recording system in a sequential three-dimensional space in time, by which a succession of three-dimensional images of an object is generated located in the center of different cameras that capture or record taking two-dimensional images.
  • the process comprises the following stages:
  • Rendering Final representation of the audiovisual material.
  • the object to be captured is placed in the center of the room. All cameras focus on the object, in the center of the room sphere. Thus, by placing the object in the center of the installation its image is collected by the cameras without any hidden corner. The function of each of the cameras is to achieve simultaneous capture from different points of view of the object. All the The process must be carried out preferably on a regular and equidistant basis, to ensure the homogeneity of the capture result.
  • the system calculates the silhouette caused by each frame, separating the captured image and the background. All the information is easily analyzed autonomously since it comes digitally supplied thanks to the CCD or CMOS of the digital camera, by a two-dimensional bitmap.
  • the color contrast will generate a contour line representing the space occupied by the object.
  • the contour is transferred to a three-dimensional program, the surface being represented as an intersection of lines and vectors where a point on the plane can be defined by coordinates contained in a plane perpendicular to the angle of attack of the camera with respect to the object. Knowing, in advance, the camera from which the frame was taken, you can place the plane exactly in the virtual space and delimit the outline that represents the figure at that precise point of space.
  • each camera pixel-point indicates the position of a converging line, in the CCD or CMOS of the camera, so that delimited by the known inclination, angle of attack, position and lens values will establish a single equation of line.
  • the representation of the point in space will be done by calculating the intersection of two or more lines thanks to the parametric representation of a line based on a point and its vector.
  • the different flat representations obtained in a precise moment, thanks to the sum of the total of the cameras dispersed in the space, will generate a network of lines that in a global way will be combined to generate a structure or three-dimensional mesh.
  • Three-dimensional representations are the result of weighing, vectorially, the points of the different projections.
  • the uniformity of the generated segments will be supported by the equidistance of the cameras from each other, returning to the system a proportional number of faces related to the total number of cameras installed and their corresponding resolution, being able to expand the quality of the capture by increasing the definition of the pixels captured or by increasing the number of cameras in the installation. It is also possible to improve resolution, interpolate, a posteriori; through a mesh smoothing process to obtain an object with more faces and vertices. The reverse process enables to reduce the faces of the result, in order to optimize the operations required by reducing the storage and data processing capacity but also losing quality.
  • the images obtained by the camera will be used to generate a two-dimensional texture that will be projected to the object based on the different angles of attack of the cameras. Because the pixel information can lead to data redundancy, the system will interpolate the information of the different points generating a high definition texture even from cameras that do not have a resolution of many pixels, thus improving the final quality of the model.
  • the angle formed between a plane and the angle of attack of the camera will determine the projection of the points of the entire mapping.
  • the texel generated by a perpendicular angle will have a square proportion of the pixels, while, if the angle of attack is different from 90 and will have irregular proportions. In this way, if an area of the space whose surface is reflected by several cameras is captured, the resolution of that area will be increased proportionally to the number of those cameras that affect that area. For this, an image interpolation can be generated by a photo retouching algorithm within the frames. Once with the static calculations of the different moments of the recording, an alternate sequence of the resulting objects can be processed, which will generate the sensation of movement.
  • the result of the process will be a CGI image of great fidelity to the captured object, both when statically observed and the analysis of all its movements.
  • the viewer will be able to change the position of the camera by observing the object from different points of view and perspectives.
  • the result can be stored in a single file.
  • dwf wrml
  • 3d studio 3ds 3dxf
  • Autocad dwg etc.
  • Said document it will contain all the data (single file), or serving as an index and resource that links to the information of interest in times, lights, textures, etc.
  • Three-dimensional documents that include animation, modeling and texture have their usefulness as a tool to be independently observed through a rendering program or to be integrated into a three-dimensional post-production system such as those currently available in the market.
  • Figure 1 shows a scheme of the system described in the present invention.
  • Figure 2 shows a diagram of the arrangement of a plurality of cameras around an object in a circular room.
  • the present invention deals with a system and a method of capturing and processing three-dimensional images with movement, which captures two-dimensional images through a plurality of cameras arranged around the object, and by intersecting the projections of said object transforms them into a three-dimensional representation for later editing of textures, animation and postproduction.
  • the system for capturing and recording three-dimensional objects in motion described herein comprises a room (1) with a spherical shape, where the object (2) to be recorded is positioned in the center.
  • the entire room has good lighting through a homogeneous lighting system of the room (1) for proper recording.
  • Inside said spherical room (1) there are a plurality of cameras (3) arranged around the object (2), and all equidistant to said object. It is not necessary for all cameras to be distributed homogeneously among them, although it is preferable to obtain the best results.
  • each of the cameras (3) simultaneously captures the object to be recorded from different points of view equidistant, thus ensuring the homogeneity of the result in the capture.
  • the information captured by the cameras (3) is sent to a processing module (4) of the information with storage system.
  • Each frame captured is stored referring to the time of the shot and the place from which it was made, separating the captured image and the background through chroma.
  • a three-dimensional representation system (5) calculates the silhouette caused by each frame, separating the captured image and the background.
  • the convergent lines are calculated to the objective of each chamber (3) and the information of each of said chambers (3) is put in common, achieving the intersection of the different lines, each pair of secant lines defining a point in the three-dimensional space .
  • the known steps include the following:
  • the process described in the present invention comprises the following steps:
  • the information captured by each chamber (3) is given by the calculation of the lines converging to the objective of each chamber (3) and the intersection of each pair of drying lines at the same time and defining a point in space.
  • the cameras (3) are arranged equidistant from the object (2), and will preferably be arranged at the same distance between them, although it is not essential. With the information sent to the processing system (4) the exact position and place from where each image of the object (2) has been captured is available. This information sent for processing is the image captured from each of the cameras (3) captured at the same moment, and referring to its position and place of recording. In this way, the processing system (4) separates the image captured from the background through chroma.
  • the representation of the meshing form is done through the three-dimensional representation system (5) that calculates the lines converging to the objective of each camera (3) by trigonometry of the bitmap grid given by the CCD or CMOS and the position of each chamber (3), storing the information in data matrices, and sharing the data of each chamber (3) and intersecting each pair of secant lines at the same time, which define a point in space.
  • the texturing, or processing and representation of the color can be calculated based on the chromatic variations of the intersecting lines. If the lighting is diffuse and uniform the colors represented by cameras with different angles will be similar as long as they represent the surface of the object.
  • the process will be repeated for each of the frames captured depending on the cadence of images and the resolution of these depending on the cameras used.
  • the movement can be interpolated since the spatial variation of the points will define the animation.
  • this representation is a kind of cloud of points grouped in space that together make up the geometric figure.
  • the resulting information can be visualized by moving and rotating in space, since it is possible to export the data in standardized formats, it can be postproduced in different 3d treatment programs, and the mesh can be calculated by proximity of points.
  • the result can be improved by adding image composition, light spots, special effects, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

L'invention concerne un système de capture et de traitement d'images tridimensionnelles avec mouvement qui comprend une pièce (1) sphérique, dans laquelle l'objet (2) à enregistrer se trouve au centre, un système d'éclairage, un système d'acquisition à chambres (3) multiples autour de l'objet (2), un module de traitement (4), et un système de représentation tridimensionnelle (5), qui calcule les droites qui convergent vers l'objet de chaque chambre (3) et met en commun les informations de chaque chambre (3), définissant chaque paire de droites qui croise un point dans l'espace tridimensionnel. Le procédé de capture et de traitement d'images tridimensionnelles avec mouvement qui capture l'image initiale à travers plusieurs chambres (3) envoie l'image capturée à un module de traitement (4) et un système de représentation tridimensionnelle (5) y montre l'image à travers l'information capturée par chaque chambre (3) au même moment.
PCT/ES2012/070288 2011-05-23 2012-04-27 Système et procédé de capture et de traitement d'images tridimensionnelles avec mouvement Ceased WO2012160224A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
ESP201130834 2011-05-23
ES201130834A ES2400998B1 (es) 2011-05-23 2011-05-23 Sistema y procedimiento de captura y procesado de imágenes tridimensionales con movimiento

Publications (1)

Publication Number Publication Date
WO2012160224A1 true WO2012160224A1 (fr) 2012-11-29

Family

ID=47216640

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/ES2012/070288 Ceased WO2012160224A1 (fr) 2011-05-23 2012-04-27 Système et procédé de capture et de traitement d'images tridimensionnelles avec mouvement

Country Status (2)

Country Link
ES (1) ES2400998B1 (fr)
WO (1) WO2012160224A1 (fr)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0348247A1 (fr) * 1988-03-25 1989-12-27 Kreon Industrie Procédé de détermination et de reconstitution des coordonnées spatiales de chacun des points d'un ensemble de points échantillonnant une surface tridimensionnelle et reconstruction de cette surface à partir desdites coordonnées
US20040246333A1 (en) * 2003-06-03 2004-12-09 Steuart Leonard P. (Skip) Digital 3D/360 degree camera system
WO2005076094A2 (fr) * 2004-02-03 2005-08-18 Fintrade S.R.L. Systeme photo-optique electronique permettant de lever, de numeriser et de reproduire la surface exterieure d'un objet tridimensionnel, virtuellement ou en plastique, en composite ou en papier
US6975353B1 (en) * 2000-04-19 2005-12-13 Milinusic Tomislav F Immersive camera system

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000357244A (ja) * 1999-06-16 2000-12-26 Matsushita Electric Ind Co Ltd アニメーション生成装置
JP2003346180A (ja) * 2002-05-22 2003-12-05 Yappa Corp ３ｄ画像合成システム
JP4266233B2 (ja) * 2007-03-28 2009-05-20 株式会社東芝 テクスチャ処理装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0348247A1 (fr) * 1988-03-25 1989-12-27 Kreon Industrie Procédé de détermination et de reconstitution des coordonnées spatiales de chacun des points d'un ensemble de points échantillonnant une surface tridimensionnelle et reconstruction de cette surface à partir desdites coordonnées
US6975353B1 (en) * 2000-04-19 2005-12-13 Milinusic Tomislav F Immersive camera system
US20040246333A1 (en) * 2003-06-03 2004-12-09 Steuart Leonard P. (Skip) Digital 3D/360 degree camera system
WO2005076094A2 (fr) * 2004-02-03 2005-08-18 Fintrade S.R.L. Systeme photo-optique electronique permettant de lever, de numeriser et de reproduire la surface exterieure d'un objet tridimensionnel, virtuellement ou en plastique, en composite ou en papier

Also Published As

Publication number Publication date
ES2400998B1 (es) 2014-03-11
ES2400998A2 (es) 2013-04-16
ES2400998R1 (es) 2013-05-03

Similar Documents

Publication Publication Date Title
ES2351961T3 (es) Procedimiento basado en imágenes de representación y reproducción de objetos tridimensionales.
JP4335588B2 (ja) ３ｄオブジェクトをモデル化する方法
US8432435B2 (en) Ray image modeling for fast catadioptric light field rendering
CN108876926B (zh) 一种全景场景中的导航方法及系统、ar/vr客户端设备
CN105453139B (zh) 用于3d表面重建的稀疏gpu体素化
CN103021017B (zh) 基于gpu加速的三维场景重建方法
Yemez et al. 3D reconstruction of real objects with high resolution shape and texture
TW201915944A (zh) 圖像處理方法、裝置、系統和儲存介質
JP2004252935A (ja) ３ｄオブジェクトをモデル化する方法
CN106575448A (zh) 激光扫描数据的图像渲染
CN108513123A (zh) 一种集成成像光场显示的图像阵列生成方法
EP4361957B1 (fr) Reconstruction d'environnement basée sur des pavés d'image
MacDonald et al. Polynomial texture mapping and 3d representations
Kang et al. Image-based rendering
WO2023220807A1 (fr) Rendu de décalage de champ lumineux
JP4335589B2 (ja) ３ｄオブジェクトをモデル化する方法
JP4354708B2 (ja) 多視点カメラシステム
CN120339561A (zh) 一种多视频与三维场景融合方法、系统、设备和介质
Hornung et al. Interactive pixel‐accurate free viewpoint rendering from images with silhouette aware sampling
US12293454B2 (en) Multiresolution truncated neural radiance fields
CN117635454A (zh) 多源光场融合渲染方法、装置及存储介质
Pagani et al. Dense 3D Point Cloud Generation from Multiple High-resolution Spherical Images.
ES2400998B1 (es) Sistema y procedimiento de captura y procesado de imágenes tridimensionales con movimiento
US20240362853A1 (en) Foveated rendering using neural radiance fields
Verma et al. 3D Rendering-Techniques and challenges

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12789397

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 12789397

Country of ref document: EP

Kind code of ref document: A1