WO2026009727A1 - Dispositif de traitement d'informations, système de traitement d'informations, programme de traitement d'informations et procédé de traitement d'informations - Google Patents

Dispositif de traitement d'informations, système de traitement d'informations, programme de traitement d'informations et procédé de traitement d'informations

Info

Publication number
WO2026009727A1
WO2026009727A1 PCT/JP2025/022259 JP2025022259W WO2026009727A1 WO 2026009727 A1 WO2026009727 A1 WO 2026009727A1 JP 2025022259 W JP2025022259 W JP 2025022259W WO 2026009727 A1 WO2026009727 A1 WO 2026009727A1
Authority
WO
WIPO (PCT)
Prior art keywords
virtual object
mixed reality
image
reality space
virtual
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/JP2025/022259
Other languages
English (en)
Japanese (ja)
Inventor
勇磨 北清
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Canon Inc
Original Assignee
Canon Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Canon Inc filed Critical Canon Inc
Publication of WO2026009727A1 publication Critical patent/WO2026009727A1/fr
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04812—Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics

Definitions

  • This disclosure relates to an information processing device that displays virtual objects to a user in a virtual space or mixed reality space.
  • mixed reality (MR) technology has become known as a technology that seamlessly blends the real world and the virtual world in real time.
  • MR technology is an MR system that uses a video see-through HMD (Head Mounted Display).
  • HMD Head Mounted Display
  • a camera built into the HMD captures an image of a subject that roughly matches the subject observed from the user's pupil position.
  • the captured image is then superimposed with CG (Computer Graphics, virtual objects), creating a composite image that is presented to the user, allowing the user to experience an MR space (mixed reality space).
  • CG Computer Graphics, virtual objects
  • occlusion is an image processing technique that calculates the anteroposterior relationship between virtual and real objects, and renders virtual objects behind real objects as hidden or missing.
  • Patent Document 1 discloses an image processing technique that represents the anteroposterior relationship between virtual and real objects.
  • This disclosure has been made in light of these points, and aims to improve the user experience in MR spaces for users of information processing devices.
  • One aspect of the present disclosure is an information processing device comprising: an acquisition means for acquiring information regarding the placement of real objects in real space; an image acquisition means for acquiring a captured image of the real space in a predetermined direction; and a control means for controlling the generation of an image of a mixed reality space that is an image obtained by combining the captured image with a virtual object; wherein the control means controls the generation of an image of the mixed reality space in which, when the position at which the virtual object is placed in the mixed reality space is a first position where at least a portion of the virtual object is behind the real object or embedded inside the real object, the image of the mixed reality space in which the position of the virtual object in the mixed reality space is a second position that is in front of the real object as viewed from the predetermined direction.
  • This disclosure makes it possible to improve the user experience of information processing devices in MR spaces.
  • FIG. 2 is a diagram illustrating the configuration of an information processing device according to the first embodiment.
  • FIG. 2 is a diagram illustrating a real space in which a user exists in the first embodiment.
  • 1 is a diagram illustrating a virtual space viewed by a user via an HMD in a first embodiment.
  • FIG. This is a diagram showing a bird's-eye view of the mixed reality space in which the user 202 exists, assuming a scene in the first embodiment when switching from the virtual space 301 in Figure 3 to an image of the mixed reality space, leaving the virtual window 305.
  • FIG. 4B is a diagram showing a scene in which the part of the image that the user views through the HMD in the scene of FIG. 4A is also displayed that is hidden in the wall in the first embodiment.
  • FIG. 4B is a diagram showing a scene in which the portion of the image that the user views through the HMD in the scene of FIG. 4A is not displayed, the portion being hidden behind a wall, in the first embodiment.
  • FIG. 4B and 4C in the first embodiment the virtual window is translated forward and reduced in size in mixed reality space so that the appearance of the virtual window from the user does not change.
  • 5B is a diagram illustrating an image of a mixed reality space generated by a control unit A101 in the scene of FIG. 5A according to the first embodiment.
  • FIG. 4 is a flowchart illustrating a detailed operation of the information processing device according to the first embodiment.
  • 10 is a flowchart illustrating a detailed operation of the information processing device according to the second modification of the first embodiment.
  • FIG. 10 is a diagram illustrating an example of a confirmation screen that is displayed after a portion of a virtual object that is buried in a wall is displayed, and that asks the user whether or not to rearrange the virtual object, in a second modification of the first embodiment.
  • FIG. 10 is a diagram illustrating an example of a confirmation screen that is displayed before a portion of a virtual object that is embedded in a wall is displayed, and that asks the user whether or not to rearrange the virtual object, in a second modification of the first embodiment.
  • FIG. 10 is a diagram illustrating a scene in which a virtual window is displayed indoors as a virtual object on a large screen in the second embodiment.
  • 9A and 9B are diagrams illustrating a case in which the virtual object rearrangement method of the first embodiment is applied to a virtual window 903 in the situation of FIG. 9 according to the second embodiment.
  • 9A and 9B are diagrams illustrating a case where the virtual object rearrangement method according to the second embodiment is applied to a virtual window 903 in the situation of FIG. 9 according to the second embodiment.
  • FIG. 10 is a diagram illustrating a scene in which a virtual window is displayed indoors as a virtual object on a large screen in the second embodiment.
  • 12A and 12B are diagrams illustrating a case in which the virtual object rearrangement method of the first embodiment is applied to a virtual window 1203 in the situation of FIG. 12 according to the second embodiment.
  • FIGS. 12A and 12B are diagrams illustrating a case where the virtual object rearrangement method according to the second embodiment is applied to a virtual window 1203 in the situation of FIG. 12 according to the second embodiment.
  • FIG. 11 is a diagram illustrating a scene in which three virtual windows are displayed indoors in MR mode in the third embodiment.
  • FIG. 10 is a diagram illustrating a scene in which three virtual windows have been rearranged according to the rearrangement method of the first embodiment in the third embodiment.
  • FIG. 15 is a diagram illustrating a scene in which the entire screen is rearranged using a virtual window 1505 as a reference for rearrangement in the third embodiment.
  • a use case in which a user wears a head-mounted display (HMD), which is an information processing device worn on the head, and experiences a mixed reality space.
  • the user watches videos of movies, sports, etc. on a virtual display (virtual window) arranged in the mixed reality space.
  • the description will also assume a situation in which the virtual window (virtual object) is repositioned when at least a portion of the virtual window (virtual object) is embedded in a wall.
  • the information processing device A100 is composed of a control unit A101, a storage unit A102, a memory A103, an input unit A104, an output unit A105, a sensor A106, and a communication unit A107.
  • the control unit A101 is a CPU that controls each component of the information processing device A100.
  • the control unit A101 combines an image captured by an imaging sensor (camera) with a virtual object to generate a composite image (an image of mixed reality space).
  • the control unit A101 also generates an image of a virtual space, which is an image that does not include real space and is composed only of CG, based on the image captured by the imaging sensor.
  • the control unit A101 identifies a floor or real objects from the captured image and determines the placement of the floor and virtual objects in the virtual space according to the position of the real objects.
  • the control unit A101 may also obtain information on the distances from the two cameras to real objects as distance information and determine the placement of virtual objects based on the distance information.
  • the control unit A101 controls the generated image of mixed reality space and image of virtual space to be displayed on a display such as a liquid crystal panel or an organic EL panel. Furthermore, if an HMD is assumed as the information processing device A100, the control unit A101 controls the generation of an image for the right eye and an image for the left eye and displays them on the respective displays.
  • the control unit A101 estimates the position or orientation of the camera (the position or orientation of the information processing device A100) from the captured image. In this embodiment, the position and orientation is estimated using a method for estimating the camera's self-position and orientation, such as SLAM (Simultaneous Localization and Mapping).
  • SLAM Simultaneous Localization and Mapping
  • the position and orientation estimation unit 104 may estimate its self-position using LiDAR SLAM (LiDAR; Light Detection and Ranging) that uses a laser, in addition to Visual SLAM, which estimates its self-position from camera images.
  • LiDAR SLAM LiDAR; Light Detection and Ranging
  • the self-position may also be estimated using Depth SLAM, which uses a ToF sensor (ToF; Time of Flight).
  • ToF ToF; Time of Flight
  • the storage unit A102 is an electrically erasable and recordable non-volatile storage medium such as an SSD (Solid State Drive) or flash memory, and stores programs executed by the control unit A101, as well as databases and user settings.
  • SSD Solid State Drive
  • flash memory stores programs executed by the control unit A101, as well as databases and user settings.
  • Memory A103 is, for example, RAM (Random Access Memory) and is used as a buffer memory for temporarily storing various data, a working area for control unit A101, etc.
  • RAM Random Access Memory
  • the input unit A104 is used to input instructions to the information processing device A100.
  • the input unit A104 includes, for example, a power button for instructing the information processing device A100 to power on/off, and operation buttons for instructing screen transitions. Note that the input unit A104 does not necessarily have to be built into the information processing device A100. It may also be configured so that input is made via the communication unit A107, which will be described later.
  • the output unit A105 may be, for example, a display (display unit) viewed by the user, a GUI (Graphical User Interface) display for interactive operation, or a light-emitting device such as an LED (Light Emitting Diode).
  • the output unit A105 may also include a sound output device such as a speaker, and a vibration device that provides a haptic effect to the user.
  • the output unit A105 may also be configured to output via the communication unit A107, which will be described later.
  • the sensor A106 is, for example, an imaging sensor that captures images of the surroundings, or a sensor such as LiDAR (Light Detection and Ranging) or ToF (Time of Flight) that measures the surrounding conditions. It is also, for example, a sensor such as an IMU (Inertial Measurement Unit) or geomagnetic sensor that measures the attitude and position of the information processing device A100. Note that the sensor A106 does not necessarily have to be built into the information processing device A100.
  • the communication unit A107 is, for example, a NIC (Network Interface Card) that incorporates a serial bus or parallel bus for connecting to other devices, a connector (RJ45) for connecting to Ethernet, and a communication IC. It is also a communication unit for wireless connection with a controller or external device.
  • the communication unit A107 outputs a weighted wireless signal from an antenna and demodulates the wireless signal received by the antenna to achieve short-range wireless communication in accordance with the IEEE 802.15 standard (also known as Bluetooth (registered trademark)).
  • the communication unit A107 may be wired, such as a USB cable (registered trademark), or wireless, such as Wi-Fi (Wireless Fidelity) (registered trademark).
  • the terminal configuration has been described assuming that the information processing device A100 is an HMD, but the terminal configuration described above is merely an example, and the information processing device A100 does not have to be an HMD, and the terminal configuration may also be different.
  • the information processing device A100 may also be an information processing device such as a PC (personal computer), tablet terminal, or smartphone connected to an HMD. It may also be an information processing system equipped with each of the components of the information processing device A100 described above.
  • Information about the position and size of the virtual object is obtained by the system or application that controls the display of the virtual object.
  • the control unit A101 obtains information about the position and size of the virtual object from the system or application.
  • spatial information such as the position and size of surrounding walls, ceilings, furniture, etc. in real space must be acquired by capturing images of the surroundings and calculating them through spatial recognition.
  • One method of spatial recognition is to measure the distance to an object by emitting laser light and measuring the reflected light, then perform calculations through mapping.
  • Another method involves extracting feature points from images captured by a camera and combining them with the camera's movement to determine depth, and then making inferences based on the determined depth.
  • Other known methods include using machine learning such as Deep Learning to estimate depth from images captured by a camera and then making calculations based on this.
  • the control unit A101 uses these methods to acquire spatial information about real space.
  • the control unit A101 uses the position and size information of the virtual object and the spatial information of the real space to calculate the front-to-back relationship between the virtual object and the real object in the mixed reality space as seen from the HMD. In other words, it determines which object is in front in the mixed reality space as seen from the camera's imaging direction.
  • the virtual object If the virtual object is in front, it will not be obscured by surrounding walls, ceilings, furniture, etc., so it can be determined that there is no need to reposition the virtual object.
  • the virtual object is behind the virtual object, or if the virtual object is obscured by the surrounding walls, ceiling, furniture, etc., it is determined that at least part of the virtual object will become invisible and that it needs to be relocated. In other words, it is determined that relocation is necessary if the virtual object interferes with the surrounding real objects such as walls, ceiling, furniture, etc. Or, it is determined that relocation is necessary if at least part of the virtual object is embedded in the surrounding real objects such as walls, ceiling, furniture, etc.
  • the method for determining whether a virtual object should be rearranged described above is merely an example, and different methods for determining whether a virtual object should be rearranged may be used.
  • Figure 2 illustrates a bird's-eye view of the real space in which user 202 exists, with user 202 existing in room 201 separated by walls.
  • the room 201 in which user 202 exists is assumed to be furnished with multiple interior items (real objects) including a television 203, a sofa 204, a table 205, a shelf 206, and a sofa 207.
  • the user 202 is assumed to be sitting on sofa 207 and facing the television 203.
  • Figure 3 illustrates a scene in which an avatar 302, representing the user 202, exists in virtual space 301, and shows a bird's-eye view of the virtual space in which avatar 302 exists.
  • virtual space 301 it is possible to represent the space as if it were expanding independently of real objects.
  • virtual space 301 is a space larger than area 303, which corresponds to the area surrounded by the walls of room 201 in the real space in Figure 2.
  • a virtual object 304 and a virtual window 305 are arranged in virtual space 301.
  • Virtual object 304 is a virtual object superimposed on sofa 207 in room 201 in Figure 2.
  • virtual window 305 is a virtual object whose width is larger than room 201 (area 303) as seen from the user 202.
  • the user 202 can watch videos of movies, sports, etc. in a virtual window 305 that is larger than the display that can be placed in the room 201.
  • Figure 4A is a bird's-eye view of the mixed reality space in which user 202 exists, imagining a scene when switching from virtual space 301 in Figure 3 to an image of mixed reality space, leaving virtual window 305.
  • virtual window 305 is placed in the real space at the same position and size as it was in virtual space 301, and mixed reality space 401 is imagined to be expressed.
  • virtual window 305 is wider than room 201 when viewed from the user, it extends beyond the walls of room 201. In other words, both ends of virtual window 305 are positioned so that they are embedded in the walls of room 201.
  • Figure 4B shows the image viewed by the user through the HMD in the scene of Figure 4A.
  • both ends of virtual window 305 are positioned so that they are embedded in the walls of room 201. Therefore, of virtual window 305, virtual window 402 is positioned within the mixed reality space.
  • virtual windows 403 and 404 which correspond to both ends of virtual window 305, are positioned so that they are embedded in the walls in the mixed reality space.
  • virtual windows 403 and 404 which are embedded in the walls, are also displayed. A face is reflected in virtual window 402, but part of the face is also displayed in virtual window 403, which is embedded in the wall.
  • virtual object 412 which represents the shadow of virtual window 402
  • virtual object 413 which is the shadow of virtual window 403, and virtual object 414, which is the shadow of virtual window 404, are also placed in the image of mixed reality space 401. That is, control unit A101 combines virtual window 402, virtual window 403, virtual window 404, and virtual object 412, virtual object 413, and virtual object 414, which correspond to the shadows, with the captured image. In this way, an image of mixed reality space 401 is generated. In this way, if an image of mixed reality space 401 is generated so that virtual window 403 and virtual window 404, which are embedded in the wall, are also displayed, it may be difficult for the user to get the feeling that virtual window 305 is actually present in real space.
  • FIG. 4C is a diagram showing the image viewed by the user through the HMD in the scene of FIG. 4A.
  • virtual windows 403 and 404 which correspond to both ends of virtual window 305 embedded in the wall of the real space, are displayed in the image of mixed reality space 401.
  • the virtual object embedded in the wall is not displayed.
  • the only virtual window displayed is virtual object 402, so virtual object 412, which corresponds to the shadow of virtual object 402, is also displayed.
  • a face is reflected in virtual window 402, but part of the face is embedded in the wall, so that part of the face is not visible.
  • control unit A101 combines virtual window 402, virtual object 412, which corresponds to the shadow, with the captured image to generate an image of mixed reality space 401. If the image of mixed reality space 401 is generated in this manner, when the user switches from the virtual space to the mixed reality space, they will no longer be able to view the entire virtual window 305 that they were viewing when the image of the virtual space was being displayed, which may impair the user experience. For example, if video content is displayed in the virtual window 305, parts of the content that are hidden in the wall may be missed.
  • ⁇ Method for rearranging virtual objects> 4B or 4C the virtual object is displayed so that it is in front of the real object. That is, occlusion is adjusted so that the virtual object is not hidden by the real object.
  • the size of a virtual object in mixed reality space if the size of a virtual object in mixed reality space is fixed, the size of the virtual object will be represented as increasing from the user's perspective as it gets closer to the user. That is, although the size of the virtual object in mixed reality space does not change, when an image of the mixed reality space is generated, by increasing the size of the virtual object with the same angle of view in the mixed reality space, the virtual object will appear to be getting closer to the user.
  • the size of the virtual object in mixed reality space is reduced.
  • the virtual object when an image in mixed reality space is generated, the virtual object is displayed in front of the real object without changing its size in the composite image.
  • the size of the virtual object will be the same before and after adjusting the positional relationship between the real object and virtual object when viewed from the same position.
  • the virtual object before adjusting the positional relationship between the real object and virtual object, the virtual object is farther from the user, and after adjusting the positional relationship between the real object and virtual object, the virtual object is closer to the user.
  • the size of the virtual object when viewing the virtual object from a position the same distance from each virtual object before and after adjusting the positional relationship between the virtual objects, the size of the virtual object will be smaller after adjusting the positional relationship of the virtual objects.
  • the apparent size of the virtual object remains the same, but the size of the virtual object in the mixed reality space becomes smaller after the positional relationship of the virtual object has been adjusted.
  • the size of the virtual object synthesized by control unit A101 the size of the virtual object in the synthesized image remains the same, but the size of the virtual object in the mixed reality space becomes smaller after the positional relationship of the virtual object has been adjusted.
  • Figure 5A illustrates a scenario in which, when a virtual object is displayed as in Figures 4B and 4C, the virtual window is translated forward and shrunk in mixed reality space so that the way the virtual window appears to the user does not change.
  • Figure 5A also illustrates a bird's-eye view of the mixed reality space in which user 202 exists.
  • the virtual object In mixed reality space 501, the virtual object is moved from the position of virtual window 502, which was placed so that it was embedded in the wall, to the position of virtual window 503 in the foreground, and is shrunk so that its apparent size to the user does not change. If the virtual object is moved closer to the user at its original size, its apparent size to the user will increase.
  • the apparent size to the user can be maintained.
  • FIG. 5B is an example of a screen viewed by user 202 in the scene described in FIG. 5A. That is, FIG. 5B is an image of the mixed reality space generated by control unit A101.
  • FIG. 5B assumes a scene in which a virtual object is moved closer to the user at the same position and orientation as the user in FIG. 4B or 4C.
  • virtual window 503 in mixed reality space 501, is positioned closer than the virtual window in FIG. 4B or 4C and does not interfere with the wall of the room. That is, at least a portion of the virtual window is not embedded in the wall.
  • the shadow of virtual window 503 was located at the position of virtual window shadow 512, but in this mixed reality space image, virtual window shadow 512 is not composited, and virtual window shadow 513 is drawn according to the position after the rearrangement.
  • FIG. 5B a virtual window of the same size as the virtual window in FIG. 4B or 4C is composited on the image of the mixed reality space generated by control unit A101. The positional relationship in mixed reality space has changed. If the distance between the virtual window and the user in Figure 4B is the same as the distance between the virtual window and the user in Figure 5B, the size of the virtual window seen by the user will be larger in Figure 4B.
  • control unit A101 will essentially composite a virtual object of the same size with the captured image before and after rearrangement when generating an image in mixed reality space.
  • the method for rearranging virtual objects in this embodiment is performed based on values such as information about the position and size of the virtual object and information about the surrounding space.
  • any affine transformation or other different method may be used as long as the rearrangement method minimizes degradation of the user experience.
  • the size of the virtual object may not be changed in mixed reality space, and an image of the virtual object from the user's viewpoint may be generated and composited with the captured image.
  • FIG. 6 is a flowchart of the information processing device A100 in this embodiment. This flowchart starts when a system or app that controls the display of virtual objects is launched.
  • step S601 the control unit A101 determines whether to end the processing based on the status of each unit of the information processing device A100. For example, if an end operation is performed via the input unit A104, the control unit A101 ends the processing of this flowchart. If it determines that the processing should continue, the control unit A101 proceeds to step S602.
  • step S602 the control unit A101 photographs the surroundings via the sensor A106 and acquires the image. Photographing the surroundings means, for example, capturing images with a camera or measuring distances with laser light using LiDAR. The captured information is recorded in memory A103.
  • the control unit A101 activates the camera to acquire captured images in real time, and then proceeds to step S603.
  • step S603 the control unit A101 recognizes the real space (spatial recognition) based on the surrounding data captured in step S602.
  • the result of this spatial recognition is spatial information such as the positions and sizes of real objects such as surrounding walls, ceilings, and furniture.
  • step S604 the control unit A101 obtains information about the display position and size of the virtual object from the memory A103, compares it with the surrounding space information calculated in step S603, and determines whether the virtual object needs to be rearranged. If the control unit A101 determines that the virtual object needs to be rearranged, it proceeds to step S605. If the control unit A101 determines that the virtual object does not need to be rearranged, it proceeds to step S606. While the image of the virtual space is being displayed, the control unit A101 determines that the virtual object does not need to be rearranged because there is no interference with real objects. Furthermore, while the image of the mixed reality space is being displayed, the control unit A101 determines that the virtual object needs to be rearranged if there is interference with real objects or if the virtual object is hidden behind a real object.
  • step S605 the control unit A101 updates the display position and size information of the virtual object stored in memory A103 according to the virtual object rearrangement method described above, and rearranges the virtual object. If at least a portion of the virtual object is embedded in (buried in) the real object, the control unit A101 rearranges the virtual object as if it were in front of the real object.
  • step S606 the control unit A101 combines the virtual object read from memory A103 with the image captured by the camera in step S602 to generate an image in mixed reality space. Alternatively, the control unit A101 generates an image in virtual space from the virtual object read from memory A103. Once the control unit A101 has generated an image in mixed reality space or an image in virtual space, which is an image for display, it proceeds to step S607.
  • step S607 the control unit A101 displays the display image generated in step S606 via the output unit A105, and then proceeds to step S601.
  • the information processing device A100 of this embodiment determines whether or not a virtual object needs to be rearranged based on the spatial information obtained by capturing an image of the surroundings and understanding the spatial relationship between the virtual object and the image. Furthermore, if rearrangement is necessary, the information processing device A100 of this embodiment rearranges the virtual object by shrinking it in mixed reality space and moving it forward.
  • the thickness of the screen may not be reduced.
  • the vertical and horizontal lengths of the screen may be reduced before and after rearrangement, but the thickness may remain constant before and after rearrangement.
  • the virtual object is a three-dimensional virtual object, it may be reduced three-dimensionally.
  • virtual objects that cast shadows on virtual objects may not be displayed.
  • the shadowed portion may be displayed in accordance with the reduction ratio and movement amount of the virtual object.
  • an image in which a virtual object and a real object are interfering with each other may be displayed, and then an image in mixed reality space in which the virtual object has been rearranged may be displayed.
  • the control unit A101 determines whether rearrangement of the virtual object is necessary and whether an image in mixed reality space in which a virtual object and a real object are interfering with each other has been displayed once.
  • control unit A101 determines that rearrangement of the virtual object is necessary and that an image of the mixed reality space in which the virtual object and real object are interfering has been displayed once, it proceeds to step S605. On the other hand, if it does not determine that rearrangement of the virtual object is necessary and that an image of the mixed reality space in which the virtual object and real object are interfering has been displayed once, it proceeds to step S606. This allows the user to visually recognize once that the virtual object and real object are interfering, and can predict that the virtual object will be rearranged.
  • ⁇ Modification 2 of the First Embodiment> after displaying an image in which a virtual object and a real object interfere with each other, if it is determined that the virtual object needs to be rearranged, a screen may be displayed to confirm with the user whether or not to rearrange the virtual object. The flow in this case will be described with reference to FIG. 7 .
  • FIG. 7 is a flowchart of the information processing device A100 in this embodiment. This flowchart starts when a system or application that controls the display of virtual objects is launched. Here, explanations of steps S601 to S603 and step S606, which are the same as those in the flow of FIG. 6, will be omitted.
  • step S604 the control unit A101 obtains information about the display position and size of the virtual object from memory A103, compares it with the surrounding space information calculated in step S603, and determines whether the virtual object needs to be rearranged. Furthermore, in step S702, which will be described below, the control unit A101 determines whether the user has instructed not to rearrange the virtual object. If there has been an instruction not to rearrange the virtual object, the control unit A101 determines that the virtual object does not need to be rearranged, and if there has been no instruction not to rearrange the virtual object, the control unit A101 determines that the virtual object needs to be rearranged. If the control unit A101 determines that the virtual object needs to be rearranged, the process proceeds to step S701. If the control unit A101 determines that the virtual object does not need to be rearranged, the process proceeds to step S606.
  • step S701 the control unit A101 determines whether or not the user has instructed the user to rearrange the virtual objects. If the control unit A101 determines that the user has instructed the user to rearrange the virtual objects, it proceeds to step S605; if the control unit A101 does not determine that the user has instructed the user to rearrange the virtual objects, it proceeds to step S702. For example, if the user instructs the user to rearrange the virtual objects on a confirmation screen generated in step S703 below to ask the user whether or not to rearrange the virtual objects, it is determined that the user has instructed the user to rearrange the virtual objects. Furthermore, if the user does not instruct the user to rearrange the virtual objects on a confirmation screen generated in step S703 below to ask the user whether or not to rearrange the virtual objects, it is not determined that the user has instructed the user to rearrange the virtual objects.
  • the portion of the virtual object that is embedded in the wall is also composited with the mixed reality space image.
  • the image of FIG. 8A may be displayed after the image of FIG. 4B is displayed.
  • the image of FIG. 8A may be displayed without first displaying the image of FIG. 4B.
  • a confirmation screen 802 that asks the user whether or not to rearrange the virtual object is further composited with the mixed reality space image of FIG. 4B.
  • a shadow virtual object 812 of the confirmation screen 802 and a virtual object 813 indicating the user's designated position are composited into the mixed reality space image 801.
  • the confirmation screen 802 displays a button that allows the user to select whether or not to rearrange the virtual window that is embedded in the wall.
  • the user can instruct to rearrange the virtual object by moving the virtual object 813 to the "Yes" position and performing a confirm operation.
  • the user can instruct not to rearrange the virtual object by moving the virtual object 813 to the "No" position and performing a confirm operation.
  • Figure 8B if a virtual object has a portion that is embedded in a wall, that virtual object is not composited into the image of the mixed reality space.
  • a confirmation screen 802 that asks the user whether or not to rearrange the virtual object, a virtual object 812 that is the shadow of the confirmation screen 802, and a virtual object 813 that indicates the position indicated by the user are composited into the captured image.
  • the image of Figure 8A may be displayed after the image of the mixed reality space of Figure 4B is displayed.
  • the image of Figure 8A may be displayed without first displaying the image of the mixed reality space of Figure 4B.
  • step S702 the control unit A101 determines whether the user has instructed not to rearrange the virtual objects. If the control unit A101 determines that the user has instructed not to rearrange the virtual objects, it proceeds to step S606. If the control unit A101 does not determine that the user has instructed not to rearrange the virtual objects, it proceeds to step S703. For example, if the user instructs not to rearrange the virtual objects on a confirmation screen generated in step S703 below to confirm whether the user will rearrange the virtual objects, it is determined that the user has instructed not to rearrange the virtual objects.
  • step S703 if the user does not instruct not to rearrange the virtual objects on a confirmation screen generated in step S703 below to confirm whether the user will rearrange the virtual objects, it is not determined that the user has instructed not to rearrange the virtual objects. In other words, if the user has not yet instructed either to rearrange or not to rearrange the virtual objects on the confirmation screen generated in step S703 below to confirm whether the user will rearrange the virtual objects, it proceeds to step S703.
  • control unit A101 determines that the user has instructed not to rearrange the virtual objects, it determines in the subsequent step S604 that there is no need to rearrange the virtual objects.
  • the user can move virtual object 813 to the "Yes" position and perform a confirm operation to instruct the rearrangement of the virtual object.
  • the user can move virtual object 813 to the "No" position and perform a confirm operation to instruct the rearrangement of the virtual object.
  • this corresponds to a case where the user has not yet instructed the rearrangement or the user has not instructed the rearrangement.
  • step S703 the control unit A101 generates a display image on which a confirmation screen for the user is superimposed.
  • the control unit A101 combines the virtual object read from memory A103, the virtual object corresponding to the confirmation screen for the user, and the captured image captured by the camera in step S602 to generate an image of mixed reality space.
  • the control unit A101 proceeds to step S607.
  • an image of mixed reality space like the ones shown in Figures 8A and 8B above is generated as the display image.
  • the virtual objects that need to be rearranged may or may not be displayed.
  • parts of the virtual objects that are hidden by or embedded in real objects may not be displayed, or the hidden or embedded parts may be displayed transparently. For example, if a virtual object that requires rearrangement is also displayed, but parts of the virtual object that are hidden by or contained within a real object are not to be displayed, an image in mixed reality space is generated in which a confirmation screen is also superimposed on the image in Figure 4C.
  • a confirmation screen may be displayed to confirm whether or not to relocate the virtual window hidden by the real object.
  • Second Embodiment In the first embodiment, a method for rearranging a virtual object by shrinking it in mixed reality space without changing the appearance of the virtual object and moving it closer to the user has been described. However, depending on the user's position and the arrangement of walls, ceilings, furniture, etc., the rearrangement may result in the virtual object being reduced too much or being moved too far forward, making it too close.
  • the virtual object is made too small, it may become difficult to perform touch operations, gaze input, or ray input. Also, if the virtual object is too close, it may block your view and make it impossible to see the real object in front of you.
  • whether to rearrange a virtual object is determined using spatial information such as the position and size of the virtual object and the positions and sizes of surrounding walls, ceilings, furniture, etc.
  • spatial information such as the positions and sizes of the virtual object and surrounding walls, ceilings, furniture, etc. is used to calculate which of the virtual object or real object is in front as viewed from the HMD (or user). In other words, it is calculated which of the virtual object or real object is in front as viewed from the imaging position.
  • the virtual object is closer than the real object, it can be determined that there is no need to reposition the virtual object, since it will not be obscured by surrounding walls, ceilings, furniture, etc.
  • the virtual object is reduced more than the reduction ratio limit or moved more than the movement amount limit upon relocation, it is determined that the virtual object will not be relocated to the intended position. Note that if the virtual object is reduced more than the reduction ratio limit or moved more than the movement amount limit upon relocation, it will only be reduced until the reduction ratio reaches its limit and moved until both the movement amount values reach their limits.
  • the reduction ratio value has reached its limit means that the virtual object has been reduced to the smallest possible size in the mixed reality space within the set limits.
  • the movement amount value has reached its limit means that the virtual object has been positioned closest to the HMD within the set limits.
  • the virtual object is rearranged to a position where the values for the reduction ratio and movement amount reach their limits. In this way, by shrinking the virtual object to its limit and moving it forward, it may be possible to display part of the virtual object in front of the surrounding walls, ceiling, furniture, etc. Alternatively, the visibility of the virtual object may be improved compared to before it was rearranged.
  • the method for determining whether a virtual object should be rearranged described above is merely an example, and different methods for determining whether a virtual object should be rearranged may be used.
  • FIG 9 is a diagram illustrating a scene in which a virtual window is displayed as a virtual object on a large screen indoors.
  • Virtual window 903 is a virtual object that is treated as a target for rearrangement in this embodiment.
  • the user is using the HMD while sitting on a sofa, and there is a sufficient distance to the wall in front of them.
  • Figure 9 of this embodiment the user is using the HMD while sitting at a counter table, and the distance to the wall in front of them is short.
  • the virtual window 903 is displayed as if it exists behind the wall of the real object in front of the user.
  • FIG. 10 is a diagram illustrating the application of the virtual object rearrangement method of the first embodiment to virtual window 903 in the situation of FIG. 9.
  • virtual window 903 is moved so that it is closer to the front wall, and its size is reduced to approximately one-seventh of its original size. In other words, in FIG. 10, it is placed at the position of virtual window 1003. If the virtual object shrinks too much like this, it may become difficult to perform touch operations, gaze input, ray input, and other inputs.
  • FIG. 11 is a diagram illustrating the application of the virtual object rearrangement method of the second embodiment to a virtual window 903 in the situation of FIG. 9.
  • the lower limit of the virtual object reduction rate is set to half of the original size.
  • the virtual window 903 is still moved so that it is closer to the front wall through rearrangement, but its size is only reduced to half of its original size.
  • the virtual object rearrangement method of this embodiment changes the appearance of the virtual object, but it can prevent the virtual object from shrinking so much that it becomes difficult to perform touch operations, gaze input, ray input, etc.
  • Figure 12 is a diagram illustrating a scene in which a virtual window is displayed indoors on a large screen as a virtual object.
  • a virtual window 1203 is a virtual object that is treated as a target for rearrangement in this embodiment, and is placed in a mixed reality space 1201.
  • Figure 12 shows a bird's-eye view of the mixed reality space 1201 in which a user 1202 exists.
  • FIG. 13 is a diagram illustrating the application of the virtual object rearrangement method of the first embodiment to a virtual window 1203 in the situation of FIG. 12.
  • the virtual window 1303 is a virtual object rearranged in this embodiment, and is placed in a mixed reality space 1301.
  • FIG. 13 is a diagram looking down from above at the mixed reality space 1301 in which the user 1202 exists.
  • the virtual window 1203 is moved so that it is closer to the left and right walls and the toilet paper holder, and its size is reduced accordingly, and it is placed at the position and size of the virtual window 1303. If the virtual object is rearranged in this way too close to the user, there is a risk that the virtual object will block the user's view and make it impossible to see the real object in front of them.
  • FIG. 14 is a diagram illustrating the application of the virtual object rearrangement method of the second embodiment to virtual window 1203 in the situation of FIG. 12.
  • the upper limit of the virtual object's movement is set to half the distance from the user.
  • Virtual window 1403 is a virtual object rearranged in this embodiment, and is placed in mixed reality space 1401.
  • FIG. 14 is a diagram illustrating a top-down view of mixed reality space 1401 in which user 1202 exists. Through rearrangement, virtual window 1203 is moved only to a position half the distance from the user, which corresponds to the upper limit of the movement, and is placed at the position of virtual window 1303.
  • virtual window 1203 is reduced to a size that will not be obscured by the left and right walls or the toilet paper holder, and is placed at the size of virtual window 1303.
  • the virtual object appears differently, but it is possible to prevent the virtual object from blocking the view and obscuring the real object in front of the user.
  • the method for rearranging a virtual object in the second embodiment can be determined from information about the position and size of the virtual object, information about the surrounding space, and values such as the reduction ratio and movement amount limit at the time of rearrangement.
  • a limit is set for the amount of movement from the original virtual object position, but this is not limited to this. Because there is a possibility that the virtual object will block the view and make the real object in front of the user invisible, a limit (threshold) may be set for the position at which the virtual object is rearranged, depending on the distance from the HMD or the user. In other words, even if the virtual object is rearranged, the position at which the virtual object is rearranged will be farther away from the HMD or the user than a predetermined distance.
  • the above-described method for rearranging virtual objects is merely one example, and the method for rearranging virtual objects, as well as the restrictions on the reduction ratio and movement amount during rearrangement, may be different.
  • the explanations have been given on the assumption that there is always one virtual object that appears and that it is always subject to rearrangement.
  • the explanation will be given on the assumption that multiple virtual objects are displayed, and that among these there are virtual objects that should not be subject to rearrangement.
  • the user can set whether the virtual object is a target for rearrangement.
  • the user determines for each virtual object whether the virtual object is a target for rearrangement, and this is recorded as a user setting in memory A103.
  • the information processing device A100 can determine whether the virtual object is a target for rearrangement by checking the user setting recorded in memory A103.
  • Another method is to make the determination based on the type of virtual object.
  • the user experience does not deteriorate even if a virtual window is reduced in size when it is relocated, but reducing a 3DCG object displayed at life-size can have an impact on the user experience. For this reason, this method determines that if the type of virtual object is a virtual window, it is a candidate for relocation, but if it is 3DCG, it is not a candidate for relocation.
  • Another method is to make a judgment based on the situation in which the virtual object is placed.
  • An image of the virtual space is displayed, and whether the virtual object is to be subject to relocation is determined based on whether it was placed in VR mode, which normally does not include real space in the image, or MR mode, which displays an image in mixed reality space.
  • the virtual object is placed as if it exists in real space.
  • MR mode which normally does not include real space in the image
  • one method of determination is to treat virtual objects placed in VR mode as subjects to relocation, and not virtual objects placed in MR mode.
  • the virtual object to be treated as the target for rearrangement is identified using the method described above for determining whether the virtual object is the target for rearrangement.
  • the method for determining whether the virtual object should be rearranged is the same as that used in the first or second embodiment. If no restrictions are placed on the reduction ratio or movement amount when rearranging the virtual object, the method used in the first embodiment is used; if restrictions are placed, the method used in the second embodiment is used.
  • the method for determining whether a virtual object should be rearranged described above is merely an example, and different methods for determining whether a virtual object should be rearranged may be used.
  • Figure 15 is a diagram illustrating a scene in which three virtual windows are displayed indoors in MR mode.
  • Figure 15 is a top-down view of mixed reality space 1501 in which user 1502 is present and, from left to right, virtual window 1503, virtual window 1504, and virtual window 1505 are located.
  • Virtual window 1503, virtual window 1504, and virtual window 1505 are all treated as candidates for rearrangement.
  • FIG. 16 is a diagram illustrating a scene in which three virtual windows have been rearranged according to the rearrangement method of the first embodiment.
  • FIG. 16 is a top-down view of mixed reality space 1601 in which user 1502 is present and in which, from left to right, virtual window 1603, virtual window 1504, and virtual window 1605 are located.
  • virtual window 1503 and virtual window 1505 have been rearranged and are now located at the positions of virtual window 1603 and virtual window 1605, respectively.
  • virtual window 1503 Since the left side of virtual window 1503 is embedded in the wall, it is shrunk slightly and moved forward so that it is no longer embedded in the wall. Because virtual window 1503 is rearranged to the position of virtual window 1603, it becomes approximately two-thirds its size and the distance from the user becomes approximately two-thirds of its original distance. Next, virtual window 1504 does not need to be rearranged because it is not embedded in the wall, so it is not shrunk or moved. Furthermore, virtual window 1505 has its right side embedded in the wall and is hidden by shelf 1606 in the foreground. Therefore, it needs to be shrunk even more than virtual window 1503 and moved even further forward. Because virtual window 1505 is rearranged to the position of virtual window 1605, it becomes approximately half its size and the distance from the user becomes approximately half of its original distance.
  • the rearrangement method of the first embodiment when the rearrangement method of the first embodiment is applied individually to each virtual object, if the user is in the same position before and after the rearrangement, the size of the virtual object appears to remain unchanged. However, when the rearrangement method of the first embodiment is applied individually to each virtual object, three virtual objects that should have been at the same depth are rearranged to different positions. If the user intentionally places the virtual objects with the same depth, this type of rearrangement method can be a factor that degrades the user experience.
  • the rearrangement method of other virtual objects is determined in accordance with the rearrangement method of the virtual object selected as the reference.
  • Figure 17 is a diagram illustrating a scene where an entire rearrangement has been performed using virtual window 1505 as the rearrangement reference.
  • Figure 17 is a top-down view of mixed reality space 1701 in which user 1502 is present and in which, from left to right, virtual window 1703, virtual window 1704, and virtual window 1705 are located.
  • virtual window 1503, virtual window 1504, and virtual window 1505 have been rearranged and are now located at the positions of virtual window 1703, virtual window 1704, and virtual window 1705, respectively.
  • virtual window 1505 Since the left side of virtual window 1505 is hidden by the wall, it is shrunk so that it is no longer hidden by the wall, and is moved forward and placed in the position of virtual window 1705. After being rearranged, virtual window 1505 becomes approximately half its size, and the distance from the user is also approximately half of its original distance.
  • Virtual windows 1503 and 1504 are moved forward while shrinking by the same amount as virtual window 1505, in accordance with the shrink ratio and movement amount when virtual window 1505 is rearranged, and are thereby placed at the positions of virtual windows 1703 and 1704, respectively.
  • virtual window 1504 is selected as the reference
  • virtual window 1503 and virtual window 1505 will not be repositioned and will remain embedded in the wall.
  • virtual window 1505 will be repositioned and will no longer be embedded in the wall, but will instead be embedded in shelf 1606. Therefore, virtual window 1505 is selected as the reference.
  • this embodiment has described the operation of the information processing device A100 when multiple virtual objects are displayed simultaneously.
  • the relative positional relationships between virtual objects can be maintained before and after rearrangement, allowing virtual objects to be rearranged while minimizing degradation of the user experience.
  • the HMD completes the repositioning of a virtual object within one frame of updating the screen, the user will not notice that the virtual object has been repositioned, and the position of the virtual object will differ from the user's perception. This could lead to a poor user experience.
  • the method for rearranging virtual objects in this embodiment is basically the same as the methods described in the first, second, and third embodiments. The difference is that rearrangement is performed over time while animating.
  • the virtual object when it is determined that a virtual object needs to be relocated due to a scene change, the virtual object is relocated with animation in time with the scene change.
  • the animation may be linear (constant speed), ease-in (gradual acceleration), ease-out (gradual deceleration), ease-in-out (acceleration and deceleration), etc., as long as it conveys to the user that the virtual object is gradually changing in size or moving. Furthermore, by matching the time it takes for the animation to the time it takes for the scene change, the relocation can be performed more naturally.
  • the above-described method of rearranging virtual objects with animation is merely one example, and different methods of rearranging virtual objects with animation may be used.
  • this embodiment has described the operation of the information processing device A100 when rearranging virtual objects with animation to prevent a deterioration in the user experience.
  • step S604 if an animation in which a virtual object is being rearranged is in progress, the control unit A101 determines that the virtual object needs to be rearranged until the rearrangement is complete. Furthermore, in step S605, if an animation in which a virtual object is being rearranged is in progress, the control unit A101 rearranges the virtual object in accordance with the animation. Furthermore, in step S606, the control unit A101 combines the virtual object with a captured image of real space in accordance with the arrangement of the virtual object rearranged in step S605, and generates an image of mixed reality space as an image to be displayed.
  • the virtual object is shrunk when it is rearranged.
  • the position of the virtual object may be adjusted so that it is placed in front of the real object without shrinking the virtual object.
  • the present disclosure can also be realized by executing the following process: That is, software (programs) that realize the functions of the above-described embodiments are supplied to a system or device via a network or various storage media, and a computer (or a control unit, MPU, etc.) of the system or device reads and executes the program code.
  • the program and the storage media storing the program constitute the present disclosure.
  • each functional unit in each of the above embodiments may or may not be separate hardware.
  • the functions of two or more functional units may be realized by common hardware.
  • Each of multiple functions of a single functional unit may be realized by separate hardware.
  • Two or more functions of a single functional unit may be realized by common hardware.
  • each functional unit may or may not be realized by hardware such as an ASIC, FPGA, or DSP.
  • the device may have a processor and memory (storage medium) in which a control program is stored. Then, the functions of at least some of the functional units of the device may be realized by the processor reading and executing the control program from the memory.
  • the present disclosure can also be realized by supplying a program that realizes one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that realizes one or more functions.
  • a circuit e.g., an ASIC
  • processor refers to a processor in a broad sense, and includes general-purpose processors (e.g., CPUs) and dedicated processors (e.g., GPUs, ASICs, FPGAs, programmable logic devices, etc.).
  • general-purpose processors e.g., CPUs
  • dedicated processors e.g., GPUs, ASICs, FPGAs, programmable logic devices, etc.
  • the disclosure of this embodiment includes the following configuration, method, and program.
  • control means controls the information processing device to generate, in the mixed reality space, an image of a first mixed reality space in which the virtual object is placed at the first position, and then generate, in the mixed reality space, an image of a second mixed reality space in which the virtual object is placed at the second position.
  • control means controls the information processing device to generate, after generating the image of the first mixed reality space, an image of a third mixed reality space in which the virtual object is placed at a position between the first position and the second position, until generating the image of the second mixed reality space.
  • Control means generates an image in the mixed reality space in which the virtual object is placed at the second position at a position and size that match the first position and the second position as seen by the user.
  • control means controls a display unit to display the generated image of the mixed reality space, and after displaying an image of a first mixed reality space in which the virtual object is located at the first position in the mixed reality space, controls a display unit to display an image of a second mixed reality space in which the virtual object is located at the second position in the mixed reality space.
  • Control means controls to display a screen that prompts a user to select whether or not to move the virtual object after displaying the image of the first mixed reality space, and when the user selects to move the virtual object, controls to generate an image of the second mixed reality space.
  • Control means controls a display unit to display the generated image of the mixed reality space, and controls a display unit to not display an image of a first mixed reality space in which the virtual object is located at the first position in the mixed reality space, but to display an image of a second mixed reality space in which the virtual object is located at the second position in the mixed reality space.
  • Control means controls to display a screen that prompts a user to select whether or not to move the virtual object before displaying the image of the first mixed reality space, and when the user selects to move the virtual object, controls to display the image of the second mixed reality space.
  • Control method an acquisition step of acquiring information about the arrangement of real objects in a real space; an image acquisition step of acquiring a captured image of the real space in a predetermined direction; a control step of controlling the generation of an image in a mixed reality space, which is an image obtained by combining the captured image and a virtual object; and when the position of the virtual object in the mixed reality space is a first position where at least a part of the virtual object is behind the real object or embedded inside the real object, the control step controls to generate an image of the mixed reality space where the position of the virtual object in the mixed reality space is a second position that is in front of the real object when viewed from the predetermined direction.
  • program A program for causing a computer to function as each of the means of the information processing device according to any one of configurations 1 to 17.
  • system an acquisition device that acquires information about the arrangement of real objects in real space; an image capture device that captures an image of the real space in a predetermined direction; a control device that controls the generation of an image in a mixed reality space that is an image obtained by combining the captured image and a virtual object, the control device controls to generate an image of the mixed reality space in which the position of the virtual object in the mixed reality space is a second position that is in front of the real object when viewed from the specified direction, when the position of the virtual object in the mixed reality space is a first position where at least a part of the virtual object is behind the real object or embedded inside the real object.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Software Systems (AREA)
  • Processing Or Creating Images (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

Ce dispositif de traitement d'informations comprend un moyen d'acquisition servant à l'acquisition d'informations concernant l'emplacement d'un objet réel dans un espace réel, et un moyen d'acquisition d'image servant à l'acquisition d'une image capturée par capture d'une image de l'espace réel dans une direction prédéterminée. Lorsqu'une position dans laquelle un objet virtuel est placé dans un espace de réalité mixte est une première position dans laquelle au moins une partie de l'objet virtuel est positionnée derrière l'objet réel ou est dans une position de recouvrement à l'intérieur de l'objet réel, le dispositif de traitement d'informations met en œuvre une commande de génération d'une image de l'espace de réalité mixte sur laquelle la position de l'objet virtuel dans l'espace de réalité mixte est une seconde position qui est une position située devant l'objet réel comme vu depuis la direction prédéterminée.
PCT/JP2025/022259 2024-07-03 2025-06-20 Dispositif de traitement d'informations, système de traitement d'informations, programme de traitement d'informations et procédé de traitement d'informations Pending WO2026009727A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2024-107716 2024-07-03
JP2024107716A JP2026007671A (ja) 2024-07-03 2024-07-03 情報処理装置、情報処理システム、情報処理プログラム及び情報処理方法

Publications (1)

Publication Number Publication Date
WO2026009727A1 true WO2026009727A1 (fr) 2026-01-08

Family

ID=98318359

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2025/022259 Pending WO2026009727A1 (fr) 2024-07-03 2025-06-20 Dispositif de traitement d'informations, système de traitement d'informations, programme de traitement d'informations et procédé de traitement d'informations

Country Status (2)

Country Link
JP (1) JP2026007671A (fr)
WO (1) WO2026009727A1 (fr)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2022098268A (ja) * 2020-12-21 2022-07-01 富士フイルムビジネスイノベーション株式会社 情報処理装置及びプログラム

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2022098268A (ja) * 2020-12-21 2022-07-01 富士フイルムビジネスイノベーション株式会社 情報処理装置及びプログラム

Also Published As

Publication number Publication date
JP2026007671A (ja) 2026-01-16

Similar Documents

Publication Publication Date Title
US10293252B2 (en) Image processing device, system and method based on position detection
JP6956104B2 (ja) 短待ち時間レンダリングのための広ベースラインステレオ
US10373392B2 (en) Transitioning views of a virtual model
US7292240B2 (en) Virtual reality presentation device and information processing method
US11695908B2 (en) Information processing apparatus and information processing method
KR101227255B1 (ko) 마커 크기 기반 인터렉션 방법 및 이를 구현하기 위한 증강 현실 시스템
CN113711109A (zh) 具有直通成像的头戴式显示器
US10755486B2 (en) Occlusion using pre-generated 3D models for augmented reality
US12141339B2 (en) Image generation apparatus and information presentation method
US20100225743A1 (en) Three-Dimensional (3D) Imaging Based on MotionParallax
JP2019510321A (ja) 仮想現実用パススルーカメラのユーザインターフェイス要素
EP3504686A1 (fr) Déformation temporelle continue et déformation temporelle binoculaire pour systèmes et procédés d'affichage de réalité virtuelle et augmentée
US11995784B2 (en) Image processing device and image processing method
CN111466113B (zh) 图像捕获的装置和方法
JP2017174125A (ja) 情報処理装置、情報処理システム、および情報処理方法
JP2016539398A (ja) 未知の対象物に対する拡張現実コンテンツの生成
WO2019097410A1 (fr) Fourniture de contenu de réalité virtuelle
US20130057574A1 (en) Storage medium recorded with program, information processing apparatus, information processing system, and information processing method
JP2018526716A (ja) 媒介現実
US10748341B2 (en) Terminal device, system, program and method for compositing a real-space image of a player into a virtual space
JP7815353B2 (ja) ３次元仮想現実表示システム
JP2022032483A (ja) 画像処理装置、画像処理方法、およびプログラム
EP3454304A1 (fr) Dispositif de traitement d'image
CN118648293A (zh) 信息处理设备和信息处理方法
JP6126272B1 (ja) 仮想空間を提供する方法、プログラム及び記録媒体

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 25832808

Country of ref document: EP

Kind code of ref document: A1