WO2020042494A1 - Procédé de capture d'écran de scène de réalité virtuelle, dispositif et support de stockage - Google Patents

Procédé de capture d'écran de scène de réalité virtuelle, dispositif et support de stockage Download PDF

Info

Publication number
WO2020042494A1
WO2020042494A1 PCT/CN2018/123764 CN2018123764W WO2020042494A1 WO 2020042494 A1 WO2020042494 A1 WO 2020042494A1 CN 2018123764 W CN2018123764 W CN 2018123764W WO 2020042494 A1 WO2020042494 A1 WO 2020042494A1
Authority
WO
WIPO (PCT)
Prior art keywords
user
virtual scene
view
scene
screenshot
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2018/123764
Other languages
English (en)
Chinese (zh)
Inventor
张向军
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Goertek Inc
Original Assignee
Goertek Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Goertek Inc filed Critical Goertek Inc
Publication of WO2020042494A1 publication Critical patent/WO2020042494A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/01—Indexing scheme relating to G06F3/01
    • G06F2203/012—Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment

Definitions

  • the present invention relates to the technical field of virtual reality (VR), and in particular, to a method, a device, and a storage medium for capturing a VR scene.
  • VR virtual reality
  • Screenshot is a way to capture the displayed content.
  • the user can save the content displayed on the screen as an image file through the screenshot operation.
  • a smartphone can save the content displayed on the screen through the screenshot operation.
  • a two-dimensional image display device performs a screenshot operation
  • the content currently displayed on the screen is directly taken as a target screenshot.
  • a three-dimensional image display device such as a VR device
  • the screenshot obtained by directly intercepting the content currently displayed on the screen of the VR device is the same as that of the VR device
  • the virtual scenes shown are generally different.
  • Some embodiments of the present invention provide a VR scene screenshot method, device and storage medium, which are used to capture the 3D virtual scene actually displayed by the VR device according to the screenshot instruction during the VR device display of the VR scene.
  • Some embodiments of the present invention provide a VR scene screenshot method, including: in response to a screenshot instruction, selecting at least two reference fields of view in a three-dimensional virtual scene that can be displayed by a VR device, and the edge of the field of view of two adjacent reference fields of view Superimpose; obtain virtual scene segments corresponding to the at least two reference fields of view; perform image stitching on the virtual scene segments corresponding to the at least two reference fields of view to generate a VR scene screenshot.
  • selecting at least two reference fields of view in the three-dimensional virtual scene displayed by the VR device includes: uniformly selecting at least two different user viewpoints in the three-dimensional virtual scene; and viewing the at least two different user viewpoints.
  • the field FOV is set to obtain at least two reference fields of view, where at least one field of view satisfies FOV> 360 ° / N, where N is the number of user viewpoints included in the at least two user viewpoints.
  • uniformly selecting at least two different user viewpoints in the three-dimensional virtual scene includes: obtaining a user's current head position pose data; and determining the head position pose data separately in the three-dimensional virtual scene. Determining the left and / or right eye viewpoints of the user; determining the base user viewpoint based on the left and / or right eye viewpoints of the user; selecting at least uniformly distributed with the base user viewpoint within the three-dimensional virtual scene One viewpoint serves as a secondary user viewpoint.
  • the number of viewpoints of the auxiliary user is two.
  • obtaining virtual scene fragments corresponding to the at least two reference fields of view includes: sequentially performing scene rendering at each of the user viewpoints of the at least two different user viewpoints to render the at least two Virtual scene clips corresponding to each reference field of view.
  • image stitching the virtual scene segments corresponding to the at least two reference fields of view to generate a VR scene screenshot includes: performing edge similarity detection on the virtual scene segments corresponding to the at least two reference fields of view. ; According to the detection result of the edge similarity point detection, the virtual scene segments corresponding to the at least two reference fields of view whose edge similarity is greater than a set threshold are spliced.
  • performing image stitching on the virtual scene segments corresponding to the at least two reference fields of view to generate a VR scene screenshot includes determining the basis from the virtual scene segments corresponding to the at least two reference fields of view.
  • the virtual scene segment corresponding to the user viewpoint is the stitching center; according to the positional relationship between the auxiliary user viewpoint and the basic user viewpoint, determining the relative position of the virtual scene fragment corresponding to the auxiliary user viewpoint and the stitching center;
  • the stitching center and the relative position perform image stitching on the virtual scene segments corresponding to the at least two reference fields of view to generate a screenshot of the VR scene.
  • the method further includes: storing the VR scene screenshot to a designated path of the VR device, and notifying a user that a screen capture operation has been completed and / or displaying the designated path of the VR scene screenshot.
  • FIG. 1 A block diagram illustrating an exemplary computing environment in accordance with the present invention.
  • FIG. 1 A block diagram illustrating an exemplary computing environment in accordance with the present invention.
  • FIG. 1 A block diagram illustrating an exemplary computing environment in accordance with the present invention.
  • Still other embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which when executed can implement the VR scene screenshot method provided by the present invention.
  • At least two reference fields of view are selected in a three-dimensional virtual scene that can be displayed by the VR device, and the edges of the two adjacent reference fields of view are superimposed; then, at least two Virtual scene clips corresponding to each reference field of view, and image stitching of at least two virtual scene clips corresponding to each of the reference field of view, and further, the generated VR scene screenshot matches the 3D virtual scene actually displayed by the VR device, effectively capturing Real display content of VR equipment.
  • FIG. 1a is a schematic diagram of a screenshot process of a two-dimensional display device
  • FIG. 1b is a schematic diagram of a three-dimensional display principle of a VR device
  • FIG. 1c is a schematic diagram of a screen display content of a VR device
  • FIG. 1d is a schematic flowchart of a VR scene screenshot method provided by some embodiments of the present invention.
  • FIG. 2a is a schematic flowchart of a VR scene screenshot method provided by some embodiments of the present invention.
  • FIG. 2b is a schematic diagram of correspondence between different user viewpoints and VR camera viewpoints provided by some embodiments of the present invention.
  • 2c is a schematic diagram of splicing a virtual scene segment provided by some embodiments of the invention.
  • FIG. 3 is a schematic structural diagram of a VR device according to some embodiments of the present invention.
  • FIG. 4 is a schematic structural diagram of a VR device according to some embodiments of the present invention.
  • the screenshot operation of the two-dimensional image display device can be shown in FIG. 1a.
  • the content displayed on the screen of the two-dimensional image display device is the real display content of the two-dimensional image display device at a certain moment.
  • the two-dimensional image display device receives a screenshot instruction, it can directly capture the real content displayed on the current screen and save it as a picture (for example, a jpg file).
  • the user wearing the VR device is in a 3D virtual space at any time, and the image that the user can see in the 3D space should be in the entire 3D virtual space. image.
  • the content displayed on the screen of the VR device is only a part of the three-dimensional space that the VR device can display at a certain moment, and is usually subjected to distortion processing, as shown in FIG. 1c. Therefore, when a VR device receives a screenshot instruction, simply capturing the content being displayed on the screen cannot meet the actual screenshot requirements.
  • the present invention provides a solution, which will be described in detail below.
  • FIG. 1d is a schematic flowchart of a VR scene screenshot method provided by some embodiments of the present invention. As shown in FIG. 1d, the method includes:
  • Step 101 In response to a screenshot instruction, at least two reference fields of view are selected in a three-dimensional virtual scene that can be displayed by the VR device, and the edges of the two adjacent reference fields of view are superimposed.
  • Step 102 Obtain a virtual scene segment corresponding to each of the at least two reference fields of view.
  • Step 103 Perform image stitching on the virtual scene segments corresponding to the at least two reference fields of view to generate a VR scene screenshot.
  • the screenshot instruction may be initiated by a user, or may be initiated by a VR device.
  • the user can send a screenshot instruction through a specific physical button on the VR device, or send a screenshot instruction through a set hand or head movement, or send a screenshot instruction through a voice wake-up.
  • This implementation Examples do not limit this.
  • the screenshot instruction may be sent by an application currently running on the VR device, or may be sent by the VR device according to a set screenshot period, depending on the application scenario.
  • At least two reference fields of view can be selected within the three-dimensional virtual scene that the VR device can display.
  • the reference field of view is used to simulate the field of view of the user in the three-dimensional virtual scene.
  • at least two reference fields of view may be selected.
  • the at least two reference fields of view can simulate at least two fields of view of the user in the three-dimensional virtual scene.
  • the edges of the fields of view of two adjacent reference fields of view are superimposed, and the field of view ranges of the at least two reference fields of view can cover the panorama of the three-dimensional virtual scene.
  • the screenshot shows the 3D scene that the VR device is showing when the screenshot instruction is received.
  • At least two reference fields of view are selected in a three-dimensional virtual scene that can be displayed by the VR device, and the edges of the two adjacent reference fields of view are superimposed; then, each of the at least two reference fields of view is obtained correspondingly Virtual scene clips, and image stitching of at least two virtual scene clips corresponding to the reference field of view, and further, the generated VR scene screenshot matches the 3D virtual scene actually displayed by the VR device, effectively capturing the real display content of the VR device .
  • FIG. 2a is a schematic flowchart of a VR scene screenshot method according to another embodiment of the present invention. As shown in FIG. 2, the method includes:
  • Step 201 In response to a screenshot instruction, at least two different user viewpoints are uniformly selected in a three-dimensional virtual scene that can be displayed by the VR device.
  • Step 202 Set the field of view angles FOV of the at least two different user viewpoints to obtain at least two reference fields of view, where at least one field of view satisfies FOV> 360 ° / N, where N is the at least two The number of user viewpoints contained in each user viewpoint.
  • Step 203 Perform scene rendering in sequence at each of the user viewpoints of the at least two different user viewpoints to render and obtain virtual scene fragments corresponding to the at least two reference viewpoints.
  • Step 204 Perform edge similarity detection on the virtual scene segments corresponding to the at least two reference fields of view.
  • Step 205 According to the detection result of the edge similarity detection, among the virtual scene fragments corresponding to the at least two reference fields of view, the virtual scene fragments whose edge similarity is greater than a set threshold are stitched.
  • Step 206 Store the VR scene screenshot to a designated path of the VR device, and notify the user that the screen capture operation has been completed and / or the designated path of the VR scene screenshot is displayed.
  • the user viewpoint refers to a visual base point in the three-dimensional virtual scene when the user views the three-dimensional virtual scene, and the visual base point generally includes viewing position information and viewing direction information, as shown in viewpoint A and viewpoint shown in FIG. 2b. B and viewpoint C.
  • a VR device can display a virtual scene that matches the user's viewpoint.
  • the VR scene is developed by development tools such as Unity 3D. These development tools can create a three-dimensional virtual space, design a three-dimensional virtual scene in the three-dimensional virtual space, and design a VR virtual camera to simulate the user's Eyes, the viewpoint of the VR virtual camera is the user's viewpoint.
  • the viewpoints A, B, and C of the user can be simulated with the viewpoints A ', B', and C 'of the VR virtual camera, respectively.
  • selecting different user viewpoints in a three-dimensional virtual scene can simulate different viewing positions and viewing directions of the user, and then based on the different viewing positions and viewing directions, obtain a three-dimensional scene being displayed by the VR device when a screenshot instruction is received.
  • uniformly selecting different user viewpoints is beneficial to quickly calculate the size of the field of view corresponding to each viewpoint.
  • a method for uniformly selecting at least two different user viewpoints in a three-dimensional virtual scene includes:
  • the head position and posture data of the user can be obtained according to the inertial detection unit, multi-axis acceleration sensor, gyroscope and other devices installed on the VR device, and details are not described here. Then, according to the head position pose data, the left and / or right eye viewpoints of the user are respectively determined in the three-dimensional virtual scene, and the base user viewpoint is determined according to the left and / or right eye viewpoints of the user.
  • the user's left-eye viewpoint can be used as the basic user's viewpoint
  • the user's right-eye viewpoint can also be used as the basic user's viewpoint
  • the middle viewpoint of the user's left-eye viewpoint and right-eye viewpoint can also be selected
  • the viewpoint of the basic user is not limited in this embodiment.
  • at least one viewpoint uniformly distributed with the viewpoint of the base user is selected as the auxiliary user viewpoint.
  • the number of auxiliary user viewpoints may be two, and the two auxiliary user viewpoints may be respectively distributed on both sides of the base user viewpoint.
  • one basic user viewpoint and two auxiliary user viewpoints can provide the highest image rendering efficiency and image stitching efficiency on the premise of ensuring that the subsequent virtual scene fragments completely cover the three-dimensional virtual scene.
  • the field of view corresponding to at least one user viewpoint satisfies FOV> 360 ° / N
  • N is the number of user viewpoints
  • the field of view corresponding to each user viewpoint satisfies FOV > 360 ° / N, where N is the number of user viewpoints.
  • the selected at least two different user viewpoints are associated with the user's current head and pose data, which is beneficial to determine the virtual scene clips that the user is currently watching in the subsequent process, and obtain Screenshots of VR scenes more in line with real viewing progress.
  • step 202 may be performed.
  • the field of view of each user viewpoint is greater than N> 360 ° / N.
  • N 3
  • the field of view of each user's viewpoint can be set to 130 ° or 150 °.
  • the edge of the field of view of two adjacent fields of view appears superimposed, which is beneficial for subsequent image stitching.
  • the user's viewpoint when selecting the user's viewpoint, you can also choose arbitrarily without considering the uniformity between viewpoints. For example, it can be randomly selected in a three-dimensional virtual scene that can be displayed by a VR device. Select multiple user viewpoints. In this manner, the size of the field of view corresponding to each user's viewpoint can be calculated according to the positional relationship between the user's viewpoint and other user's viewpoints, and the sum of the angles of view corresponding to multiple user viewpoints is greater than 360 °.
  • the user may be set The viewpoints A and B have smaller field angles, and the user's viewpoint C has a larger field angle. For example, set the field angle corresponding to point A to 90 ° and set the field angle corresponding to point B to 120 °.
  • the field of view corresponding to C is 160 °.
  • scene rendering may be performed at each of the user points of view of the at least two different user viewpoints in order to render the at least two reference fields of view. Corresponding virtual scene clip.
  • the user's viewpoint can be used as the viewpoint of the VR virtual camera, and the VR virtual camera can be adjusted to each different user's viewpoint in turn, and A 3D virtual scene rendering program is run at each different user viewpoint to render a virtual scene fragment corresponding to the field of view of the user viewpoint.
  • step 204 in some embodiments, after acquiring virtual scene segments corresponding to at least two reference fields of view, edge similarity detection is performed on the virtual scene segments.
  • a picture correlation coefficient method may be used to find virtual scene fragments with similar edges.
  • the edges of each virtual scene segment can be identified separately, and the edge correlation coefficients of two adjacent virtual scene segments are calculated based on the recognition results, and virtual scene segments with similar edges are determined based on the correlation coefficients.
  • other picture edge similarity algorithms can also be used to identify virtual scene fragments with similar edges, such as Euclidean distance method, perceptual hashing method, and sliding window-based template matching method. This embodiment includes but Not only that.
  • step 205 in some embodiments, in this step, among the virtual scene fragments corresponding to the at least two reference fields of view, the virtual scene fragments whose edge similarity is greater than a set threshold may be stitched.
  • the virtual scene segments in the reference field of view corresponding to the user viewpoints A, B, and C can be stitched, where the overlapping area with the virtual scene segment is a region with similar edges.
  • the method of horizontal splicing is illustrated in FIG. 2c. In practice, the method of vertical splicing or other degrees of splicing may also be included.
  • the virtual scene segment corresponding to the user's viewpoint can be used as a center to make the VR scene screenshot obtained by the stitching center on the virtual scene segment that the user is currently watching, which is more in line with the user's actual viewing effect.
  • the virtual scene segment corresponding to the basic user viewpoint may be first determined as the splicing center from the virtual scene segments corresponding to the at least two reference fields of view; then, according to the auxiliary user viewpoint and the basic user viewpoint, The positional relationship between them determines the relative position of the virtual scene segment corresponding to the auxiliary user's viewpoint and the stitching center; then, according to the stitching center and the relative position, image stitching is performed on the virtual scene segments corresponding to at least two reference fields of view to generate VR Scene screenshot.
  • the virtual scene fragment in the field of view corresponding to viewpoint B can be used as the center of the VR screenshot.
  • the virtual scene segments in the field of view corresponding to viewpoint A and viewpoint C are spliced, respectively.
  • the spliced VR scene screenshot may also be stored in a specified path of the VR device, and the user is notified that the screen capture operation has been completed and / or the designation of the VR scene screenshot is displayed. path.
  • the method for notifying the user that the screen capture operation has been completed may be a voice mode or a text mode, which is not limited in this embodiment.
  • At least two reference fields of view are uniformly selected in the three-dimensional virtual scene that can be displayed by the VR device, and the edges of the two adjacent reference fields of view are superimposed; then, at least two reference fields of view are obtained to correspond to each other Virtual scene clips, and image stitching of at least two reference scenes corresponding to the virtual scene clips according to the edge similarity of the virtual scene clips, and further, the generated VR scene screenshot matches the 3D virtual scene actually displayed by the VR device, which is effective Captured the real display content of VR devices.
  • the execution subject of each step of the method provided in the foregoing embodiment may be the same device, or the method may also use different devices as execution subjects.
  • the execution subject of steps 201 to 204 may be device A; for example, the execution subject of steps 201 and 202 may be device A, and the execution subject of step 203 may be device B; and so on.
  • the above embodiment describes a part of the implementation method of the VR scene screenshot method provided by some embodiments of the present invention.
  • This method may be implemented by the VR device shown in FIG. 3.
  • the VR device includes: a memory 301, a processor 302, and an input.
  • the memory 301, the processor 302, the input device 303, and the output device 304 may be connected by using a bus or other methods.
  • the bus connection is taken as an example.
  • the memory 301 may be directly coupled to the processor 302, and the input device 303 and the output device 304 may be directly or indirectly connected to the processor 302 through a data line and a data interface.
  • the above connection mode is only used for illustrative description, and does not constitute any limitation on the protection scope of the embodiment of the present invention.
  • the memory 301 is used to store one or more computer instructions, and may be configured to store various other data to support operations on the VR device. Examples of such data include instructions for any application or method for operating on a VR device.
  • the memory 301 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), Programming read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
  • SRAM static random access memory
  • EEPROM electrically erasable programmable read-only memory
  • EPROM Programming read-only memory
  • PROM programmable read-only memory
  • ROM read-only memory
  • magnetic memory magnetic memory
  • flash memory magnetic disk or optical disk.
  • the memory 301 may optionally include a memory remotely set relative to the processor 302, and these remote memories may be connected to the AR display device through a network.
  • Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
  • a processor 302 coupled to the memory 301 and configured to execute the one or more computer instructions for: in response to a screenshot instruction, selecting at least two reference fields of view within a three-dimensional virtual scene that can be displayed by the VR device, and adjacent to each other The edges of the two reference fields of view are superimposed; the virtual scene segments corresponding to the at least two reference fields of view are obtained; image stitching is performed on the virtual scene segments corresponding to the at least two reference fields of view to generate a VR scene screenshot .
  • the processor 302 is specifically configured to: uniformly select at least two different user viewpoints in the three-dimensional virtual scene;
  • the FOVs of two different user viewpoints are set to obtain at least two reference FOVs, where at least one FOV satisfies FOV> 360 ° / N, where N is the user viewpoint contained in the at least two user viewpoints Quantity.
  • the processor 302 is specifically configured to: obtain current head position pose data of the user; Determining the left and / or right eye viewpoints of the user in the three-dimensional virtual scene; determining the base user viewpoints according to the left and / or right eye viewpoints of the user; At least one viewpoint with evenly distributed base user viewpoints is used as the auxiliary user viewpoint.
  • the number of the auxiliary user viewpoints is two.
  • the processor 302 when acquiring virtual scene fragments corresponding to the at least two reference fields of view, is specifically configured to perform scene rendering at each user viewpoint in the at least two different user viewpoints in sequence, A virtual scene segment corresponding to the at least two reference fields of view is obtained by rendering.
  • the processor 302 is specifically configured to: The scene segments are detected for edge similarity. According to the detection result of the edge similarity detection, virtual scene segments whose edge similarity is greater than a set threshold among the virtual scene segments corresponding to the at least two reference fields of view are stitched.
  • the processor 302 is specifically configured to: from the virtual scenes corresponding to the at least two reference fields of view Among the fragments, it is determined that the virtual scene fragment corresponding to the basic user viewpoint is a splicing center; and according to a positional relationship between the auxiliary user viewpoint and the basic user viewpoint, determining the virtual scene fragment corresponding to the auxiliary user viewpoint and the Relative position of the stitching center; according to the stitching center and the relative position, image stitching is performed on the virtual scene segments corresponding to the at least two reference fields of view to generate a screenshot of the VR scene.
  • the processor 302 is further configured to: store the VR scene screenshot to a specified path of the VR device, and notify the user that the screen capture operation has been completed and / or display the designated path of the VR scene screenshot .
  • the input device 303 can receive inputted numeric or character information, and generate key signal inputs related to user settings and function control of the VR device.
  • the output device 304 may include a display device such as a display screen.
  • the VR device further includes: a power supply component 305.
  • the power source component 305 provides power to various components of the device where the power source component is located.
  • Power components can include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the equipment in which the power components are located.
  • the above VR device can execute the VR scene screenshot method provided by the embodiment of the present application, and has corresponding function modules and beneficial effects of executing the method.
  • the above VR device can execute the VR scene screenshot method provided by the embodiment of the present application, and has corresponding function modules and beneficial effects of executing the method.
  • the present invention also provides a computer-readable storage medium storing a computer program, and the computer program, when executed, can implement the steps in the method that the VR device can execute.
  • the VR device provided by some embodiments of the present invention may be an external head-mounted display device or an integrated head-mounted display device, wherein the external head-mounted display device may be used in conjunction with an external processing system (such as a computer processing system).
  • an external processing system such as a computer processing system
  • FIG. 4 shows a schematic diagram of the internal configuration of the VR device 400 in some embodiments.
  • the display unit 401 may include a display panel.
  • the display panel is disposed on a side surface of the VR device 400 that faces the user's face, and may be a whole panel, or a left panel and a right panel corresponding to the left and right eyes of the user, respectively.
  • the display panel may be an electroluminescence (EL) element, a liquid crystal display, or a micro-display with a similar structure, or a laser scanning display with a direct retina display or the like.
  • EL electroluminescence
  • the virtual image optical unit 402 displays the image displayed by the display unit 401 to the user in an enlarged manner, and allows the user to observe the displayed image in accordance with the enlarged virtual image.
  • a display image output to the display unit 401 it may be an image of a virtual scene provided from a content reproduction device (Blu-ray disc or DVD player) or a streaming server, or an image of a real scene captured using an external camera 410.
  • the virtual image optical unit 402 may include a lens unit, such as a spherical lens, an aspherical lens, a Fresnel lens, and the like.
  • the input operation unit 403 includes at least one operation component for performing an input operation, such as a key, a button, a switch, or other components having similar functions, receives a user instruction through the operation component, and outputs the instruction to the control unit 407.
  • an input operation such as a key, a button, a switch, or other components having similar functions
  • the status information acquiring unit 404 is configured to acquire status information of a user wearing the VR device 400.
  • the status information obtaining unit 404 may include various types of sensors for detecting status information by itself, and may obtain status information from external devices (such as a smartphone, a wristwatch, and other multifunctional terminals worn by the user) through the communication unit 405.
  • the state information acquisition unit 404 may acquire position information and / or posture information of a user's head.
  • the status information acquisition unit 404 may include one or more of a gyroscope sensor, an acceleration sensor, a global positioning system (GPS) sensor, a geomagnetic sensor, a Doppler effect sensor, an infrared sensor, and a radio frequency field strength sensor.
  • GPS global positioning system
  • the status information acquisition unit 404 acquires status information of a user wearing the VR device 400, such as acquiring, for example, an operation status of the user (whether the user is wearing the VR device 400), an action status of the user (such as stationary, walking, running, and the like) State, hand or fingertip posture, eye open or closed state, line of sight, pupil size), mental state (whether the user is immersed in observing the displayed image and the like), and even a physiological state.
  • an operation status of the user whether the user is wearing the VR device 400
  • an action status of the user such as stationary, walking, running, and the like
  • State hand or fingertip posture, eye open or closed state, line of sight, pupil size
  • mental state whether the user is immersed in observing the displayed image and the like
  • the communication unit 405 performs a communication process with an external device, a modulation and demodulation process, and an encoding and decoding process of a communication signal.
  • the control unit 407 may transmit transmission data from the communication unit 405 to an external device.
  • the communication method can be wired or wireless, such as mobile high-definition link (MHL) or universal serial bus (USB), high-definition multimedia interface (HDMI), wireless fidelity (Wi-Fi), Bluetooth communication or Bluetooth low energy communication, And the IEEE802.11s standard mesh network.
  • the communication unit 405 may be a cellular wireless transceiver operating according to Wideband Code Division Multiple Access (W-CDMA), Long Term Evolution (LTE), and similar standards.
  • W-CDMA Wideband Code Division Multiple Access
  • LTE Long Term Evolution
  • the VR device 400 may further include a storage unit, and the storage unit 406 is a mass storage device configured with a solid state drive (SSD) or the like.
  • the storage unit 406 may store an application program or various types of data. For example, content viewed by a user using the VR device 400 may be stored in the storage unit 406.
  • the VR device 400 may further include a control unit and a storage unit (for example, the ROM 407A and the RAM 407B shown in the figure).
  • the control unit 407 may include a computer processing unit (CPU) or other devices with similar functions.
  • the control unit 407 may be used to execute an application program stored in the storage unit 406, or the control unit 407 may also be used to execute the methods, functions, and circuits disclosed in some embodiments of the present invention.
  • the image processing unit 408 is used to perform signal processing, such as image quality correction related to the image signal output from the control unit 407, and to convert its resolution to a resolution according to the screen of the display unit 401. Then, the display driving unit 409 sequentially selects each row of pixels of the display unit 401 and sequentially scans each row of pixels of the display unit 401 one by one, thereby providing a pixel signal based on the image signal subjected to signal processing.
  • the VR device 400 may further include an external camera.
  • the external cameras 410 may be disposed on the front surface of the main body of the VR device 400, and the external cameras 410 may be one or more.
  • the external camera 410 can acquire three-dimensional information and can also be used as a distance sensor.
  • a position sensitive detector (PSD) or other type of distance sensor that detects a reflected signal from an object may be used with the external camera 410.
  • PSD position sensitive detector
  • the external camera 410 and the distance sensor may be used to detect a body position, posture, and shape of a user wearing the VR device 400.
  • the user can directly view or preview the real scene through the external camera 410.
  • the VR device 400 may further include a sound processing unit, and the sound processing unit 411 may perform sound quality correction or sound amplification of a sound signal output from the control unit 407, signal processing of an input sound signal, and the like. Then, the sound input / output unit 412 outputs sound to the outside after sound processing and inputs sound from a microphone.
  • the sound processing unit 411 may perform sound quality correction or sound amplification of a sound signal output from the control unit 407, signal processing of an input sound signal, and the like. Then, the sound input / output unit 412 outputs sound to the outside after sound processing and inputs sound from a microphone.
  • the structure or component shown by the dashed box in FIG. 4 may be independent of the VR device 400, for example, it may be provided in an external processing system (such as a computer system) for use with the VR device 400; or The resulting structure or component may be disposed inside or on the surface of the VR device 400.
  • RAM random access memory
  • ROM read-only memory
  • electrically programmable ROM electrically erasable programmable ROM
  • registers hard disks, removable disks, CD-ROMs, or in technical fields Any other form of storage medium known in the art.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Software Systems (AREA)
  • Processing Or Creating Images (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

Des modes de réalisation de la présente invention concernent un procédé de capture d'écran d'une scène, un dispositif et un support de stockage. Dans un mode de réalisation de l'invention, au moins deux champs de vision de référence sont sélectionnés dans une scène virtuelle tridimensionnelle pouvant être affichée par un dispositif de réalité virtuelle (RV) et des bords de deux champs de vision de référence adjacents sont superposés; ensuite, des segments de scène virtuelle correspondant auxdits deux champs de vision de référence au moins sont acquis et des images des segments de scène virtuelle correspondant auxdits deux champs de vision de référence au moins sont jointes; puis, les captures d'écran de scène RV correspondant à la scène virtuelle tridimensionnelle réellement affichée par le dispositif RV sont générées, ce qui permet de capturer efficacement le contenu d'affichage réel du dispositif RV.
PCT/CN2018/123764 2018-08-31 2018-12-26 Procédé de capture d'écran de scène de réalité virtuelle, dispositif et support de stockage Ceased WO2020042494A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811013485.2 2018-08-31
CN201811013485.2A CN109002248B (zh) 2018-08-31 2018-08-31 Vr场景截图方法、设备及存储介质

Publications (1)

Publication Number Publication Date
WO2020042494A1 true WO2020042494A1 (fr) 2020-03-05

Family

ID=64591425

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/123764 Ceased WO2020042494A1 (fr) 2018-08-31 2018-12-26 Procédé de capture d'écran de scène de réalité virtuelle, dispositif et support de stockage

Country Status (2)

Country Link
CN (1) CN109002248B (fr)
WO (1) WO2020042494A1 (fr)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109002248B (zh) * 2018-08-31 2021-07-20 歌尔光学科技有限公司 Vr场景截图方法、设备及存储介质
CN111949113B (zh) * 2019-05-15 2024-10-29 阿里巴巴集团控股有限公司 应用于虚拟现实vr场景中的图像交互方法及装置
CN114697302B (zh) * 2020-12-31 2024-09-27 伊姆西Ip控股有限责任公司 用于分发虚拟视觉内容的方法
CN112732088B (zh) * 2021-01-18 2023-01-20 海信视像科技股份有限公司 一种虚拟现实设备及单目截屏方法
CN114286142B (zh) * 2021-01-18 2023-03-28 海信视像科技股份有限公司 一种虚拟现实设备及vr场景截屏方法
CN113126942B (zh) * 2021-03-19 2024-04-30 北京城市网邻信息技术有限公司 一种封面图片的显示方法、装置、电子设备及存储介质
CN115268650B (zh) * 2022-08-09 2025-10-17 海信视像科技股份有限公司 画面截屏方法、装置、头戴式虚拟现实设备和存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104835118A (zh) * 2015-06-04 2015-08-12 浙江得图网络有限公司 通过两路鱼眼摄像头采集全景图像的方法
CN105959666A (zh) * 2016-06-30 2016-09-21 乐视控股（北京）有限公司 一种虚拟现实系统中分享3d影像的方法和装置
US20160353090A1 (en) * 2015-05-27 2016-12-01 Google Inc. Omnistereo capture and render of panoramic virtual reality content
CN109002248A (zh) * 2018-08-31 2018-12-14 歌尔科技有限公司 Vr场景截图方法、设备及存储介质

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105138245B (zh) * 2015-09-30 2018-06-29 北京奇虎科技有限公司 一种智能终端截屏图片的去重处理方法和装置
CN105847672A (zh) * 2016-03-07 2016-08-10 乐视致新电子科技（天津）有限公司 虚拟现实头盔抓拍方法及系统
US20180068574A1 (en) * 2016-09-08 2018-03-08 Jim LaCrosse Method of and system for facilitating structured block play in a virtual reality environment

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160353090A1 (en) * 2015-05-27 2016-12-01 Google Inc. Omnistereo capture and render of panoramic virtual reality content
CN104835118A (zh) * 2015-06-04 2015-08-12 浙江得图网络有限公司 通过两路鱼眼摄像头采集全景图像的方法
CN105959666A (zh) * 2016-06-30 2016-09-21 乐视控股（北京）有限公司 一种虚拟现实系统中分享3d影像的方法和装置
CN109002248A (zh) * 2018-08-31 2018-12-14 歌尔科技有限公司 Vr场景截图方法、设备及存储介质

Also Published As

Publication number Publication date
CN109002248A (zh) 2018-12-14
CN109002248B (zh) 2021-07-20

Similar Documents

Publication Publication Date Title
WO2020042494A1 (fr) Procédé de capture d'écran de scène de réalité virtuelle, dispositif et support de stockage
US12100229B2 (en) Object scanning for subsequent object detection
US10832448B2 (en) Display control device, display control method, and program
JP6720341B2 (ja) バーチャルリアリティデバイス及びそのコンテンツの調整方法
US20250104183A1 (en) Camera-based Transparent Display
JP6126271B1 (ja) 仮想空間を提供する方法、プログラム及び記録媒体
WO2017028498A1 (fr) Procédé et appareil d'affichage de scénario tridimensionnel (3d)
CN107610044A (zh) 图像处理方法、计算机可读存储介质及虚拟现实头戴设备
CN113870213B (zh) 图像显示方法、装置、存储介质以及电子设备
WO2018000619A1 (fr) Procédé d'affichage de données, dispositif, dispositif électronique et dispositif de réalité virtuelle
WO2019015249A1 (fr) Procédé et appareil d'affichage d'image basés sur la réalité virtuelle, et dispositif de casque de réalité virtuelle
CN107678546A (zh) 虚拟场景切换方法及头戴显示设备
EP3805899A1 (fr) Système de visiocasque et son procédé de balayage de scène
JPWO2019026919A1 (ja) 画像処理システム、画像処理方法、及びプログラム
EP3062506B1 (fr) Procédé et appareil de commutation d'image
CN112702533A (zh) 视线修正方法及视线修正装置
CN115268650B (zh) 画面截屏方法、装置、头戴式虚拟现实设备和存储介质
JP6126272B1 (ja) 仮想空間を提供する方法、プログラム及び記録媒体
JPWO2018168823A1 (ja) 画像処理装置および電子機器
US11113379B2 (en) Unlocking method and virtual reality device
JP2017208808A (ja) 仮想空間を提供する方法、プログラム及び記録媒体
CN107704397B (zh) 应用程序测试方法、装置及电子设备
US12277729B2 (en) Method for providing visual content, host, and computer readable storage medium
US20200342833A1 (en) Head mounted display system and scene scanning method thereof
CN107426522B (zh) 基于虚拟现实设备的视频方法和系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18931380

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18931380

Country of ref document: EP

Kind code of ref document: A1