WO2020076826A1 - Système et procédés de fourniture de vue temporellement et spatialement modifiée dans des environnements de réalité augmentée et de réalité virtuelle - Google Patents

Système et procédés de fourniture de vue temporellement et spatialement modifiée dans des environnements de réalité augmentée et de réalité virtuelle Download PDF

Info

Publication number
WO2020076826A1
WO2020076826A1 PCT/US2019/055192 US2019055192W WO2020076826A1 WO 2020076826 A1 WO2020076826 A1 WO 2020076826A1 US 2019055192 W US2019055192 W US 2019055192W WO 2020076826 A1 WO2020076826 A1 WO 2020076826A1
Authority
WO
WIPO (PCT)
Prior art keywords
data set
dimensional
environment
user
dimensional data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2019/055192
Other languages
English (en)
Inventor
Priya CLOUTIER
Ursula DEELSTRA
Sean House
Michele Lee MCMULLEN
Adam Szofran
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
30 60 90 Corp
Original Assignee
30 60 90 Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 30 60 90 Corp filed Critical 30 60 90 Corp
Publication of WO2020076826A1 publication Critical patent/WO2020076826A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/20—Scenes; Scene-specific elements in augmented reality scenes

Definitions

  • An objective of this invention is to provide users to communicate changing three- dimensional data sets or multi-dimensional environments vary or change.
  • FIG. 1A is an exemplary stereoscopic view of three-dimensional data sets or multi- dimensional environments
  • Fig. 1B is an exemplary stereoscopic three-dimensional data sets or multi
  • FIG. 2A shows an exemplary fiduciary marker having an indicator
  • Fig. 2B shows an exemplary fiduciary marker having an indicator being placed in the physical world
  • FIG. 3 shows a user interacting with three-dimensional data sets or multi
  • FIG. 4 shows a user interacting with three-dimensional data sets or multi
  • Fig. 5 shows an exemplary fiduciary marker.
  • process(es)/operations heading(s) may be described under process(es)/operations heading(s) and/or process(es)/operations may be discussed under structure(s)/process(es) headings; and/or descriptions of single topics may span two or more topic headings).
  • formal outline headings is not intended to be in any way limiting.
  • Image is a three-dimensional environment or multi- dimensional data set that may be represented as a photosphere or image captured by light field technology. Image may represent a captured, recorded, or modeled real world object or scene.
  • Point of View is a locus, or vantage point, that represents a location in space which is visible to a user.
  • Hotspot is a point within an image with which a user may interact. A hotspot may allow a user to view multiple aspects of a scene and/or respond to a survey.
  • Teleporter is a point within a scene that allows a user to navigate to another scene or another location within the same scene.
  • Description may be text, sound, image, or other descriptive information.
  • Meeting is defined as more than one user interacting with three-dimensional environment or multi- dimensional data set on an immersive platform.
  • the three- dimensional environment or multi- dimensional data set may be annotated.
  • Ink means to draw a visual path or region.
  • an interface that allows users in augmented reality and virtual reality systems to communicate and collaborate both synchronously and asynchronously. More specifically, the interface allows users to visualize or otherwise communicate the manner in which three-dimensional data sets or multi dimensional environments vary or change over more than one period.
  • a three-dimensional data set or multi- dimensional environment (or“Image”) (10) may be recorded or captured using computational photography techniques.
  • an Image (10) may be captured or recorded as a photosphere using a computational photographic system such as Google® Cardboard Camera Application on a mobile phone or by using a purpose-built device, such as stereoscopic, or spherical, capture hardware, with one or more conventional or fisheye lenses.
  • Another computational method to record or capture an Image includes the use of light field camera technologies.
  • Images (10) can be modeled or generated on a computing system.
  • an Image (10) may be generated using programs, codes, or applications employing virtual geometry or digital world model.
  • a user may capture, record, or model an Image (10) in one location (11) during more than one period.
  • one period can be a time when a building or room has been built only to its beams and supports.
  • a second period may be when the same building or room has been finished.
  • Period as used herein, can be of any length of time. For example, a period can be seconds, hours, days, months, or years.
  • Each Image (10) recorded in a location (11) over more than one period is aligned with other Images (10) taken at a different period from the same or approximately the same point of view.
  • Images (10) are aligned manually.
  • Images (10) are optically aligned using geometric elements or other elements of the Images (10).
  • Images (10) are aligned using compass headings from the moment of capture.
  • Images (10) are aligned along an x-, y-, or z-axis.
  • Each aligned Image may be annotated at the time it was captured, recorded or modeled.
  • each aligned Image (10) with its annotations are registered in a database and aligned to an alpha-numeric or other descriptor (21) that would allow for easy recall of the aligned Images (10).
  • a fiducial marker (20) is provided in the location (11) where the aligned Images (10) were recorded, captured, or modeled.
  • Each fiducial marker (20) provides a unique descriptor (21) that allows a user to interact, in an immersive or non- immersive environment, with aligned Images (10) for that location (11).
  • the fiducial marker (20) does not have the same point of view as the aligned Images (10).
  • the fiducial marker (20) does have the same point of view as the aligned Images (10).
  • the fiducial marker (20) is in the same location (11) the aligned Images (10) are captured, recorded, or modeled.
  • a user interacting in an immersive or non- immersive environment using a mobile phone, or other tethered or untethered augmented reality or virtual reality hardware, recognizes a descriptor (21) at a location (11).
  • the tethered or untethered augmented reality or virtual reality hardware deploys optical character recognition and image matching software to read the descriptor (21).
  • the user may input the descriptor (21), via any usual manual input device or software aligned with the user’s tethered or untethered augmented reality or virtual reality hardware.
  • the fiducial marker (20) is comprised of at least two matrices (20a, 20b) where each matrix contains at least a portion of a descriptor (21).
  • a matrix (20a, 20b) is a n x m matrix.
  • a matrix (20a, 20b) is a n x n matrix.
  • the border of each matrix (20a, 20b) is black.
  • the user is provided with aligned Images (10) with which the user may interact in an immersive or non-immersive environment.
  • an indication (30) is provided to the user as to a point of view from which the aligned Images (10) align with the real world allowing the user to align herself for interaction with the Image (10).
  • the indication (30) is approximately orthogonal to the fiducial marker (20).
  • the indication (30) is can detect and avoid non-orthogonal planes.
  • each Image (10) that the user may interact with, in an immersive or non-immersive environment may include earlier recorded annotations.
  • the user may add additional annotation to the Image(s) (10), in an immersive or non-immersive environment.
  • more than one user may annotate the Image(s) in an immersive or non-immersive environment, synchronously or asynchronously.
  • an architect on a construction project may want to show stakeholders how a room may look when it is finished.
  • the stakeholders may be in different locations around the world or in the same location.
  • the architect capture or records an Image (10), by one of the methods described above, of the unfinished room shown in Fig 1B; and renders a model or creates and Image (10) of how the room may look in the future, e.g. a finished room, shown in Fig 1A.
  • the architect may annotate Fig. 1B to show where additional lighting will be added, for example.
  • the architect will save the Images (10) in Figs. 1A and 1B with his annotation.
  • a descriptor (21) will be used to associate the Images (10) to a point of view.
  • a fiducial marker (20) having the appropriate descriptor (21) is placed in the location (11) where the aligned images were recorded, captured, or model.
  • a fiducial marker (20) enters a location having a fiducial marker (20).
  • the descriptor (21) is recognized, by one of the methods described above; consequently, Fig. 1B is shown on the stakeholder’s device.
  • the stakeholder views and/or hears the annotation the architect has left; when the stakeholder hovers on the teleporter or hotspot (40), he is transported to the room shown in Fig. 1A.
  • the architect may have previously annotated on or more of the Images (10); the stakeholder will experience those annotations as he interacts with an Image (10). In some cases, where the stakeholder is in a space that closely matches the current location (11), it may not be necessary to show an Image (10) that represents the current period.
  • the second stakeholder may interact with the Images (10) synchronously with the first stakeholder.
  • the second stakeholder may Ink to draw another annotation regarding a lighting issue, for example; this interaction is recorded for later playback or asynchronous interaction.
  • the architect for example, may asynchronously, interact with the second stakeholder’s annotation of the Image(s) (10) later.
  • aspects of the present invention may be embodied as a system, method, or computer product. Accordingly, aspects of the present invention may take the form of an entirely hardware embodiment, entirely software embodiment (including firmware, resident software, micro-code, etc.), or an embodiment combining software and hardware aspects. Further aspects of this invention may take the form of a computer program embodied in one or more readable medium having computer readable program code/instructions thereon. Program code embodied on computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
  • the computer code may be executed entirely on a user’s computer, partly on the user’s computer, as a standalone software package, a cloud service, partly on the user’s computer and partly on a remote computer or entirely on a remote computer, remote or cloud-based server.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

L'invention concerne des systèmes et des procédés permettant de faciliter une communication et une collaboration basées sur la réalité virtuelle (VR), la réalité augmentée (AR) ou la réalité augmentée virtuelle (VAR) au moyen d'une structure d'interface utilisateur rationalisée qui permet des interactions à la fois synchrones et asynchrones dans des environnements immersifs.
PCT/US2019/055192 2018-10-08 2019-10-08 Système et procédés de fourniture de vue temporellement et spatialement modifiée dans des environnements de réalité augmentée et de réalité virtuelle Ceased WO2020076826A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201862742926P 2018-10-08 2018-10-08
US62/742,926 2018-10-08

Publications (1)

Publication Number Publication Date
WO2020076826A1 true WO2020076826A1 (fr) 2020-04-16

Family

ID=70164382

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2019/055192 Ceased WO2020076826A1 (fr) 2018-10-08 2019-10-08 Système et procédés de fourniture de vue temporellement et spatialement modifiée dans des environnements de réalité augmentée et de réalité virtuelle

Country Status (1)

Country Link
WO (1) WO2020076826A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120113274A1 (en) * 2010-11-08 2012-05-10 Suranjit Adhikari Augmented reality interface for video tagging and sharing
US20120249586A1 (en) * 2011-03-31 2012-10-04 Nokia Corporation Method and apparatus for providing collaboration between remote and on-site users of indirect augmented reality
US20170249745A1 (en) * 2014-05-21 2017-08-31 Millennium Three Technologies, Inc. Fiducial marker patterns, their automatic detection in images, and applications thereof
US20180137685A1 (en) * 2015-04-06 2018-05-17 Scope Technologies Us Inc. Method and apparatus for sharing augmented reality applications to multiple clients
WO2018152742A1 (fr) * 2017-02-23 2018-08-30 Versitech Limited Surfaces multiples pour alignement physique-à-image/image-à-physique et vérification d'image

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120113274A1 (en) * 2010-11-08 2012-05-10 Suranjit Adhikari Augmented reality interface for video tagging and sharing
US20120249586A1 (en) * 2011-03-31 2012-10-04 Nokia Corporation Method and apparatus for providing collaboration between remote and on-site users of indirect augmented reality
US20170249745A1 (en) * 2014-05-21 2017-08-31 Millennium Three Technologies, Inc. Fiducial marker patterns, their automatic detection in images, and applications thereof
US20180137685A1 (en) * 2015-04-06 2018-05-17 Scope Technologies Us Inc. Method and apparatus for sharing augmented reality applications to multiple clients
WO2018152742A1 (fr) * 2017-02-23 2018-08-30 Versitech Limited Surfaces multiples pour alignement physique-à-image/image-à-physique et vérification d'image

Similar Documents

Publication Publication Date Title
US11398080B2 (en) Methods for augmented reality applications
KR102417645B1 (ko) Ar 장면 이미지 처리 방법, 장치, 전자 기기 및 저장 매체
JP7391317B2 (ja) 情報管理装置、情報管理システム、情報管理方法および情報管理プログラム
JP5582548B2 (ja) 実環境視像における仮想情報の表示方法
CN107491174B (zh) 用于远程协助的方法、装置、系统及电子设备
CN110914872A (zh) 用认知洞察力导航视频场景
CN103335657B (zh) 一种基于图像捕获和识别技术增强导航功能的方法和系统
KR102195999B1 (ko) 이미지 주석정보의 처리 방법, 장치 및 시스템
US11302067B2 (en) Systems and method for realistic augmented reality (AR) lighting effects
WO2022095468A1 (fr) Procédé et appareil d'affichage dans une scène en réalité augmentée, dispositif, support, et programme
JP2022512525A (ja) Ａｒシーン画像処理方法及び装置、電子機器並びに記憶媒体
CN107566793A (zh) 用于远程协助的方法、装置、系统及电子设备
CN117173374A (zh) 一种基于ar技术的变电站二次系统回路缺陷查找方法
US11258945B2 (en) Interactive data visualization environment
WO2020076826A1 (fr) Système et procédés de fourniture de vue temporellement et spatialement modifiée dans des environnements de réalité augmentée et de réalité virtuelle
CN116452397B (zh) 一种警务数字沙盘用的联动控制系统和控制方法
CN114089836B (zh) 标注方法、终端、服务器和存储介质
JP7525515B2 (ja) 空間認識システム、空間認識方法、および情報端末
JP2020107152A (ja) 情報処理装置、情報処理システム、その制御方法及びプログラム
CN114323013A (zh) 用于确定设备在场景中的位置信息的方法
WO2015115720A1 (fr) Dispositif dédié à un service de génération de contenu et servant à générer une vidéo sur la base d'une pluralité de dispositifs utilisateur, procédé permettant de générer une vidéo sur la base d'une pluralité de dispositifs utilisateur, et support d'enregistrement contenant un programme d'ordinateur enregistré
CN115793864B (zh) 一种虚拟现实响应装置、方法及存储介质
CN109919121B (zh) 一种人体模型的投影方法、装置、电子设备及存储介质
US20220269889A1 (en) Visual tag classification for augmented reality display
WO2025094964A1 (fr) Procédé de traitement d'informations, dispositif de traitement d'informations et programme de traitement d'informations

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19871941

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19871941

Country of ref document: EP

Kind code of ref document: A1