JP2020004263A - Program, computer and method for providing virtual experience to user - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a novel method for reducing a sensory deviation in a virtual experience. A program according to an embodiment of the present disclosure includes a step of defining a virtual space for providing a virtual experience to a computer for providing a virtual experience to a user via an image display device and a housing, and a virtual space. A step of specifying a view in space, a step of detecting an operation by a user on a first device included in a housing, a step of specifying a mode of changing the view based on the detected operation, and a specified mode. Based on the above, a step of updating the field of view and a step of identifying the stimulus corresponding to the specified embodiment and applying the stimulus to the user through the housing in the real space are executed. [Selection diagram] FIG. 13

Description

  The present disclosure relates to a program, a computer, and a method for providing a virtual experience to a user.

  Patent Literature 1 discloses a ski simulator as a simulation system that provides a user with a virtual reality (VR) experience using an HMD (Head-Mounted Device). This ski simulator is designed to alleviate 3D sickness or VR sickness caused when the movement mode of the user in the real space is significantly different from the movement mode of the virtual user or the boarding moving body in the virtual space. A movable housing 40 is provided. For example, the movable casing 40 moves the base 41 of the movable casing 40 in the vertical direction according to the unevenness of the snow surface of the ski course in the virtual space, or rotates the base 41 according to the inclination of the ski course. The play position PPL is changed by moving (rolling, pitching, etc.).

  In order to compensate for a sense shift between the user's virtual space and the real space, this ski simulator employs a movable housing based on objects constituting the VR space, such as unevenness of the snow surface of a ski course and inclination of a ski course. Move your body.

  Regarding the user's operation input, this ski simulator matches the sense of reality due to the user's operation input with the change of the visual field in the VR space, so that the user performs the same operation as the actual ski as much as possible. The simulator is configured as However, in the VR game, it may be inconvenient to request a reproduction of an actual operation in the real space as an operation input by the user. For example, as mentioned in Patent Literature 1, there is a problem of code entanglement due to a large movement of a body in a real space. Further, when a user is required to perform a complicated operation in accordance with the operation in the real space, the change in the VR field of view is complicated due to the complicated movement of the user, and on the contrary, the VR game is satisfactory. There is also the problem that VR view changes are not provided.

JP-A-2017-188827

  An object of the present disclosure is to provide at least a novel technique for reducing a sensory shift in a virtual experience.

According to an embodiment of the present disclosure, a computer for providing a user with a virtual experience via an image display device and a housing includes:
Defining a virtual space to provide a virtual experience;
Identifying a field of view in the virtual space;
Detecting a user operation on a first device included in the housing;
Based on the detected operation, identifying an aspect that changes the field of view,
Updating the field of view based on the specified aspect;
Providing a stimulus corresponding to the specified aspect and providing the stimulus to the user via the housing in the real space.

According to another embodiment of the present disclosure, a computer comprises:
By a processor provided in the computer, by an image display device associated with the head of the user, by control for providing a virtual experience to the user via the housing,
Defining a virtual space to provide a virtual experience;
Identifying a field of view in the virtual space;
Detecting a user operation on a first device included in the housing;
Based on the detected operation, identifying an aspect that changes the field of view,
Updating the field of view based on the specified aspect;
Specifying a stimulus corresponding to the specified aspect and providing the stimulus to the user via the housing in the real space.

According to another embodiment of the present disclosure, an image display device associated with a user's head and a computer-implemented method for providing a virtual experience to a user via a housing,
Defining a virtual space to provide a virtual experience;
Identifying a field of view in the virtual space;
Detecting a user operation on a first device included in the housing;
Based on the detected operation, identifying an aspect that changes the field of view,
Updating the field of view based on the specified aspect;
Identifying a stimulus corresponding to the identified aspect, and applying the stimulus to the user via the housing in the real space.

  According to the embodiment of the present disclosure, at least a novel method for reducing a sensory shift in a virtual experience can be provided.

  Other features and advantages of the present disclosure will be apparent from the following description of the embodiments, the accompanying drawings, and the claims.

It is a figure which shows the structure of a system schematically. 1 is a block diagram illustrating an example of a basic hardware configuration of a computer according to an embodiment of the present disclosure. FIG. 4 is a diagram conceptually illustrating a uvw view coordinate system set in the HMD according to an embodiment. FIG. 3 is a diagram conceptually illustrating an aspect of expressing a virtual space according to an embodiment. FIG. 3 is a top view of a head of a user wearing the HMD according to one embodiment. It is a figure showing the yz section which looked at the field of view from the x direction in virtual space. FIG. 4 is a diagram illustrating an xz cross section of a view area in a virtual space viewed from a y direction. FIG. 2 is a diagram illustrating a schematic configuration of a controller according to an embodiment. FIG. 2 is a diagram illustrating a schematic configuration of a controller according to an embodiment. FIG. 4 is a block diagram illustrating functions of a computer including a sensory shift reduction function for providing a virtual experience to a user via a system according to an embodiment of the present disclosure. FIG. 9 is a flowchart of a general process for displaying an image of a virtual space in which a user is immersed on a display unit. It is a block diagram showing the outline of VR game system 1100 by one embodiment of this indication. It is the schematic which shows the structure of one Embodiment of a housing | casing. 5 is a flowchart of a sense shift reduction method according to an embodiment of the present disclosure. FIG. 4 is a diagram schematically illustrating a relationship between a virtual space, a view in the virtual space, a moving direction of the view, a handrail, and an operation on a board in the VR game system. It is a schematic diagram showing a more specific embodiment of a snowboard mechanism. It is the schematic which shows another more specific embodiment of a snowboard mechanism. It is the schematic which shows the handrail mechanism which is another embodiment of the combination of a stimulating device mechanism and a stimulating device drive mechanism. It is the schematic which shows the seat-like mechanism which is another embodiment of the combination of a stimulating device mechanism and a stimulating device drive mechanism. It is the schematic which shows one specific embodiment of a two-axis rotation mechanism. FIG. 20 is a schematic cross-sectional view taken along line II of FIG. 19 for describing details of a mechanism for rotating the turntable with respect to the movable table.

[Description of Embodiment of the Present Disclosure]
First, the configuration of an exemplary embodiment of the present disclosure will be listed and described. A program, a computer, and a method according to an embodiment of the present disclosure may have the following configurations.

(Item 1)
A computer for providing a virtual experience to the user via the image display device and the housing,
Defining a virtual space to provide a virtual experience;
Identifying a field of view in the virtual space;
Detecting a user operation on a first device included in the housing;
Based on the detected operation, identifying an aspect that changes the field of view,
Updating the field of view based on the specified aspect;
Specifying a stimulus corresponding to the specified aspect, and applying the stimulus to the user via the housing in the real space.

(Item 2)
The first device is a device on which a user rides,
The operation by the user is an operation of applying force to the first device with a foot of the user,
The detecting step includes detecting a force applied by a user to the first device,
The program according to item 1.

(Item 3)
The operation of applying the force includes at least one of an edging operation and a center of gravity moving operation on the first device,
When the operation for applying the force is an edging operation on the first device, the detecting step includes a step of detecting a tilt of the first device,
When the operation of applying the force is an operation of moving the center of gravity to the first device, the detecting step includes a step of detecting a pressure applied to the first device.
The program according to item 2.

(Item 4)
The program according to any one of items 1 to 3, wherein the housing includes a second device for giving the stimulus to a user.

(Item 5)
5. The program according to item 4, wherein the second device is a support member that supports a user's body.

(Item 6)
The program according to item 5, wherein the support member is a handrail supporting a user's hand or a seat-like member supporting a user's waist.

(Item 7)
The program according to item 4, wherein the second device is the first device.

(Item 8)
The aspect includes movement of at least one of yawing, pitching, and rolling,
8. The program according to any one of items 1 to 7, wherein the stimulus corresponding to the aspect includes at least one movement of yaw axis rotation, pitch axis rotation, and roll axis rotation.

(Item 9)
9. The program according to item 8, wherein when the aspect is yawing, the stimulus corresponding to the aspect is a yaw axis rotation movement.

(Item 10)
9. The program according to item 8, wherein when the aspect is pitching, the stimulus corresponding to the aspect is a movement of a pitch axis rotation.

(Item 11)
The stimulus corresponding to the aspect is a compensation that compensates for a deviation of a stimulus received by the user from the housing with respect to a stimulus that should be applied to the user's body when a change corresponding to a change in the visual field of the aspect occurs in a real space. The program according to any one of items 1 to 10, which is a stimulus.

(Item 12)
The program according to item 11, wherein the compensation stimulus is a stimulus in the same direction but smaller than a stimulus to be applied to the body of the user when a change corresponding to the change in the visual field of the aspect occurs in a real space.

(Item 13)
A computer,
By a processor provided in the computer, by an image display device associated with the head of the user, by control for providing a virtual experience to the user via the housing,
Defining a virtual space to provide a virtual experience;
Identifying a field of view in the virtual space;
Detecting a user operation on a first device included in the housing;
Based on the detected operation, identifying an aspect that changes the field of view,
Updating the field of view based on the specified aspect;
A step of specifying a stimulus corresponding to the specified aspect and providing the stimulus to the user via the housing in the real space.

(Item 14)
An image display device associated with a user's head and a computer-implemented method for providing a virtual experience to a user via a housing,
Defining a virtual space to provide a virtual experience;
Identifying a field of view in the virtual space;
Detecting a user operation on a first device included in the housing;
Based on the detected operation, identifying an aspect that changes the field of view,
Updating the field of view based on the specified aspect;
Identifying a stimulus corresponding to the identified aspect, and applying the stimulus to the user via the housing in the real space.

[Details of Embodiment of the Present Disclosure]
Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the following description, similar elements are denoted by similar reference numerals. The same applies to their names and functions. A duplicate description of such elements will be omitted.

  The configuration of the image display device system 100 will be described with reference to FIG. Hereinafter, a system in which a head-mounted device (HMD) is used as an example of an image display device will be specifically described. However, those skilled in the art will appreciate that various devices for providing virtual experiences to users, not just HMDs, can be applied to embodiments of the present disclosure.

  FIG. 1 schematically shows a configuration of the system 100. In one example, system 100 is provided as a home or business system. The HMD may be a so-called head-mounted display having a display unit, or may be a head-mounted device to which a terminal such as a smartphone having a display unit can be attached.

  The system 100 includes an HMD 110, an HMD sensor 120, a controller 160, and a computer 200. The HMD 110 includes a display unit 112 and a gaze sensor 140. The controller 160 may include the motion sensor 130.

  In one example, the computer 200 may be connectable to a network 192 such as the Internet, or may be communicable with a computer such as a server 150 connected to the network 192. In another aspect, the HMD 110 may include a sensor 114 instead of the HMD sensor 120.

  The HMD 110 may be worn on the head of the user 190 and provide a virtual space to the user during operation. More specifically, HMD 110 displays a right-eye image and a left-eye image on display unit 112, respectively. When each eye of the user visually recognizes each image, the user can recognize the image as a three-dimensional image based on the parallax of both eyes.

  The display unit 112 is realized, for example, as a non-transmissive display device. In one example, the display unit 112 is arranged on the main body of the HMD 110 so as to be located in front of both eyes of the user. Therefore, when the user visually recognizes the three-dimensional image displayed on the display unit 112, the user can immerse himself in the virtual space. In one embodiment, the virtual space includes, for example, a background, a user-operable object, a user-selectable menu image, and the like. In an embodiment, the display unit 112 can be realized as a liquid crystal display unit or an organic EL (Electro Luminescence) display unit included in an information display terminal such as a smartphone.

  In one example, the display unit 112 may include a sub-display unit for displaying a right-eye image and a sub-display unit for displaying a left-eye image. In another aspect, the display unit 112 may be configured to display the right-eye image and the left-eye image integrally. In this case, the display unit 112 includes a high-speed shutter. The high-speed shutter operates so that an image for the right eye and an image for the left eye can be alternately displayed so that the image is recognized only by one of the eyes.

  In one example, HMD 110 includes multiple light sources (not shown). Each light source is realized by, for example, an LED (Light Emitting Diode) that emits infrared light. The HMD sensor 120 has a position tracking function for detecting the movement of the HMD 110. More specifically, the HMD sensor 120 may read a plurality of infrared rays emitted from the HMD 110 and detect the position and the inclination of the HMD 110 in the real space.

  In one aspect, the HMD sensor 120 may be implemented by a camera. In this case, the HMD sensor 120 can detect the position and the inclination of the HMD 110 by executing image analysis processing using the image information of the HMD 110 output from the camera.

  In another aspect, the HMD 110 may include a sensor 114 instead of the HMD sensor 120 as a position detector. The HMD 110 can use the sensor 114 to detect the position and the inclination of the HMD 110 itself. For example, when the sensor 114 is an angular velocity sensor, a geomagnetic sensor, an acceleration sensor, a gyro sensor, or the like, the HMD 110 detects its own position and inclination using any of these sensors instead of the HMD sensor 120. obtain. As an example, when the sensor 114 is an angular velocity sensor, the angular velocity sensor detects the angular velocity around the three axes of the HMD 110 in the real space over time. The HMD 110 calculates a temporal change of an angle around three axes of the HMD 110 based on each angular velocity, and further calculates a tilt of the HMD 110 based on a temporal change of the angle. Further, the HMD 110 may include a transmissive display device. In this case, the transmissive display device may be temporarily configurable as a non-transmissive display device by adjusting its transmittance. Further, the view image may include a configuration for presenting the real space in a part of the image forming the virtual space. For example, an image captured by a camera mounted on the HMD 110 may be displayed so as to be superimposed on a part of the view image, or the transmittance of a part of the transmissive display device may be set to be high so that the view image can be displayed. The real space may be made visible from a part.

  The gaze sensor 140 detects the direction (line of sight) to which the line of sight of the right and left eyes of the user 190 is directed. The detection of the direction is realized by, for example, a known eye tracking function. The gaze sensor 140 is realized by a sensor having the eye tracking function. In one embodiment, the gaze sensor 140 preferably includes a sensor for the right eye and a sensor for the left eye. The gaze sensor 140 may be, for example, a sensor that irradiates infrared light to the right and left eyes of the user 190 and detects the rotation angle of each eyeball by receiving reflected light from the cornea and the iris with respect to the irradiated light. . The gaze sensor 140 can detect the line of sight of the user 190 based on the detected rotation angles.

  The server 150 may transmit the program to the computer 200. In another aspect, server 150 may communicate with other computers 200 to provide virtual reality to HMDs used by other users. For example, when a plurality of users play a participatory game at an amusement facility, each computer 200 communicates a signal based on the operation of each user with another computer 200, and a plurality of users share a common virtual space in the same virtual space. Enables you to enjoy the game.

  The controller 160 is connected to the computer 200 by wire or wirelessly. Controller 160 accepts an input of a command from user 190 to computer 200. In one aspect, the controller 160 is configured to be grippable by the user 190. In another aspect, the controller 160 is configured to be attachable to a part of the body or clothing of the user 190. In another aspect, the controller 160 may be configured to output at least one of vibration, sound, and light based on a signal transmitted from the computer 200. In another aspect, the controller 160 receives an operation from the user 190 for controlling the position and movement of an object placed in the virtual space.

  In one aspect, the motion sensor 130 is attached to a user's hand and detects movement of the user's hand. For example, the motion sensor 130 detects a rotation speed, a rotation speed, and the like of the hand. The detected signal is sent to the computer 200. The motion sensor 130 is provided in, for example, a glove-type controller 160A. In some embodiments, for safety in the real world, it is desirable that the controller 160A be mounted on a glove-shaped object that does not easily fly by being mounted on the hand of the user 190. In another embodiment, a controller 160B having a general structure having a plurality of operation buttons may be used. In another aspect, a sensor that is not worn by the user 190 may detect movement of the user 190's hand. Optical sensors that detect the position, orientation, direction of movement, distance of movement, etc. of various parts of the body of the user 190 may be used. For example, a signal from a camera that captures an image of the user 190 may be input to the computer 200 as a signal indicating the operation of the user 190. The motion sensor 130 and the computer 200 are, for example, wirelessly connected to each other. In the case of wireless communication, the communication mode is not particularly limited, and for example, Bluetooth (registered trademark) or another known communication method is used.

  With reference to FIG. 2, a computer 200 according to an embodiment of the present disclosure will be described. FIG. 2 is a block diagram illustrating an example of a basic hardware configuration of the computer 200 according to an embodiment of the present disclosure. The computer 200 includes, as main components, a processor 202, a memory 204, a storage 206, an input / output interface 208, and a communication interface 210. Each component is connected to the bus 212, respectively.

  The processor 202 executes a series of instructions included in a program stored in the memory 204 or the storage 206 based on a signal given to the computer 200 or when a predetermined condition is satisfied. In an embodiment, the processor 202 is realized as a device such as a CPU (Central Processing Unit), an MPU (Micro Processor Unit), or an FPGA (Field-Programmable Gate Array).

  The memory 204 temporarily stores programs and data. The program is loaded from the storage 206, for example. The data includes data input to computer 200 and data generated by processor 202. In one embodiment, the memory 204 is implemented as a volatile memory such as a RAM (Random Access Memory).

  The storage 206 permanently stores programs and data. The storage 206 is realized as a non-volatile storage device such as a ROM (Read-Only Memory), a hard disk device, and a flash memory. The programs stored in the storage 206 include a program for providing a virtual space in the system 100, a simulation program, a game program, a user authentication program, a program for realizing communication with another computer 200, and the like. The data stored in the storage 206 includes data for defining a virtual space, objects, and the like.

  In another aspect, the storage 206 may be implemented as a removable storage device such as a memory card. In still another embodiment, a configuration using programs and data stored in an external storage device may be used instead of the storage 206 built in the computer 200. According to such a configuration, for example, in a situation where a plurality of systems 100 are used as in an amusement facility, it is possible to update programs and data collectively.

  In some embodiments, input / output interface 208 communicates signals between HMD 110, HMD sensor 120, and motion sensor 130. In one embodiment, the input / output interface 208 is realized using a terminal such as a USB (Universal Serial Bus, USB), a DVI (Digital Visual Interface), or an HDMI (registered trademark) (High-Definition Multimedia Interface). Note that the input / output interface 208 is not limited to the above.

  In certain embodiments, input / output interface 208 may further communicate with controller 160. For example, the input / output interface 208 receives signals output from the controller 160 and the motion sensor 130. In another aspect, input / output interface 208 sends instructions output from processor 202 to controller 160. The command instructs the controller 160 to perform vibration, sound output, light emission, and the like. Upon receiving the command, the controller 160 executes vibration, sound output, light emission, and the like according to the command.

  The communication interface 210 is connected to the network 192 and communicates with another computer (for example, the server 150) connected to the network 192. In one embodiment, the communication interface 210 is implemented as a wired communication interface such as a LAN (Local Area Network), or a wireless communication interface such as WiFi (Wireless Fidelity), Bluetooth (registered trademark), NFC (Near Field Communication), or the like. Is done. The communication interface 210 is not limited to the above.

  In an aspect, the processor 202 accesses the storage 206, loads one or more programs stored in the storage 206 into the memory 204, and executes a series of instructions included in the program. The one or more programs may include an operating system of the computer 200, an application program for providing a virtual space, game software executable in the virtual space, and the like. The processor 202 sends a signal for providing a virtual space to the HMD 110 via the input / output interface 208. The HMD 110 displays an image on the display unit 112 based on the signal.

  In the example illustrated in FIG. 2, the computer 200 is provided outside the HMD 110. However, in another aspect, the computer 200 may be built in the HMD 110. As an example, a portable information communication terminal (for example, a smartphone) including the display unit 112 may function as the computer 200.

  Further, the computer 200 may have a configuration commonly used by a plurality of HMDs 110. According to such a configuration, for example, the same virtual space can be provided to a plurality of users, so that each user can enjoy the same application as another user in the same virtual space.

  In one embodiment, in the system 100, a global coordinate system is preset. The global coordinate system has three reference directions (axes) parallel to the vertical direction in the real space, a horizontal direction orthogonal to the vertical direction, and a front-rear direction orthogonal to both the vertical direction and the horizontal direction. In the present embodiment, the global coordinate system is one of the viewpoint coordinate systems. Therefore, the horizontal direction, the vertical direction (vertical direction), and the front-back direction in the global coordinate system are defined as x-axis, y-axis, and z-axis, respectively. More specifically, in the global coordinate system, the x-axis is parallel to the horizontal direction of the real space. The y-axis is parallel to the vertical direction of the real space. The z-axis is parallel to the front-back direction of the real space.

  In one aspect, the HMD sensor 120 includes an infrared sensor. When the infrared sensor detects infrared rays emitted from each light source of the HMD 110, the presence of the HMD 110 is detected. The HMD sensor 120 further detects the position and inclination of the HMD 110 in the real space according to the movement of the user 190 wearing the HMD 110, based on the value of each point (each coordinate value in the global coordinate system). More specifically, the HMD sensor 120 can detect a temporal change in the position and the inclination of the HMD 110 using the values detected over time.

  The global coordinate system is parallel to the coordinate system in the real space. Therefore, each inclination of the HMD 110 detected by the HMD sensor 120 corresponds to each inclination around three axes of the HMD 110 in the global coordinate system. The HMD sensor 120 sets the uvw visual field coordinate system to the HMD 110 based on the inclination of the HMD 110 in the global coordinate system. The uvw visual field coordinate system set on the HMD 110 corresponds to a viewpoint coordinate system when the user 190 wearing the HMD 110 views an object in a virtual space.

  The uvw visual field coordinate system will be described with reference to FIG. FIG. 3 is a diagram conceptually illustrating a uvw view coordinate system set in the HMD 110 according to an embodiment. The HMD sensor 120 detects a position and an inclination of the HMD 110 in the global coordinate system when the HMD 110 is activated. The processor 202 sets the uvw visual field coordinate system on the HMD 110 based on the detected value.

  As shown in FIG. 3, the HMD 110 sets a three-dimensional uvw visual field coordinate system centered on the head (origin) of the user wearing the HMD 110. More specifically, the HMD 110 adjusts the horizontal direction, the vertical direction, and the front-rear direction (x-axis, y-axis, and z-axis) that define the global coordinate system by tilts around the respective axes of the HMD 110 in the global coordinate system. Three directions newly obtained by inclining the respective axes are set as a pitch direction (u axis), a yaw direction (v axis), and a roll direction (w axis) of the uvw visual field coordinate system in the HMD 110.

  In an aspect, when the user 190 wearing the HMD 110 is standing upright and viewing the front, the processor 202 sets the uvw view coordinate system parallel to the global coordinate system on the HMD 110. In this case, the horizontal direction (x-axis), the vertical direction (y-axis), and the front-back direction (z-axis) in the global coordinate system are the pitch direction (u-axis) and the yaw direction (v-axis) of the uvw visual field coordinate system in the HMD 110. And the roll direction (w-axis).

  After the uvw field-of-view coordinate system is set to the HMD 110, the HMD sensor 120 can detect the inclination (the amount of change in the inclination) of the HMD 110 in the set uvw field-of-view coordinate system based on the movement of the HMD 110. In this case, the HMD sensor 120 detects the pitch angle (θu), the yaw angle (θv), and the roll angle (θw) of the HMD 110 in the uvw visual field coordinate system as the inclination of the HMD 110. The pitch angle (θu) represents the inclination angle of the HMD 110 around the pitch direction in the uvw visual field coordinate system. The yaw angle (θv) represents the inclination angle of the HMD 110 around the yaw direction in the uvw visual field coordinate system. The roll angle (θw) represents the tilt angle of the HMD 110 around the roll direction in the uvw visual field coordinate system.

  The HMD sensor 120 sets the uvw view coordinate system of the HMD 110 after the HMD 110 has moved to the HMD 110 based on the detected inclination angle of the HMD 110. The relationship between the HMD 110 and the uvw visual field coordinate system of the HMD 110 is always constant regardless of the position and inclination of the HMD 110. When the position and the inclination of the HMD 110 change, the position and the inclination of the uvw view coordinate system of the HMD 110 in the global coordinate system change in conjunction with the change in the position and the inclination.

  In one embodiment, the HMD sensor 120 determines the HMD 110 based on the light intensity of the infrared light acquired based on the output from the infrared sensor and the relative positional relationship between the plurality of points (for example, the distance between the points). May be specified as a relative position with respect to the HMD sensor 120. Further, the processor 202 may determine the origin of the uvw view coordinate system of the HMD 110 in the real space (global coordinate system) based on the specified relative position.

  The virtual space will be further described with reference to FIG. FIG. 4 is a diagram conceptually illustrating one aspect of expressing the virtual space 400 according to an embodiment. The virtual space 400 has a spherical shape that covers the entire center 406 in the 360-degree direction. FIG. 4 illustrates an upper half celestial sphere of the virtual space 400 in order not to complicate the description. In the virtual space 400, each mesh is defined. The position of each mesh is defined in advance as a coordinate value in the XYZ coordinate system defined in the virtual space 400. The computer 200 associates each partial image constituting the content (still image, moving image, etc.) that can be developed in the virtual space 400 with each corresponding mesh in the virtual space 400, and generates a virtual space image that can be visually recognized by the user. The developed virtual space 400 is provided to the user.

  In an embodiment, in the virtual space 400, an xyz coordinate system having the center 406 as an origin is defined. The xyz coordinate system is, for example, parallel to the global coordinate system. Since the xyz coordinate system is a kind of viewpoint coordinate system, the horizontal direction, the vertical direction (vertical direction), and the front-back direction in the xyz coordinate system are defined as an x-axis, a y-axis, and a z-axis, respectively. Therefore, the x axis (horizontal direction) of the xyz coordinate system is parallel to the x axis of the global coordinate system, the y axis (vertical direction) of the xyz coordinate system is parallel to the y axis of the global coordinate system, and the xyz coordinate system The z-axis (front-back direction) is parallel to the z-axis of the global coordinate system.

  When the HMD 110 is activated, that is, in the initial state of the HMD 110, the virtual camera 404 is arranged at the center 406 of the virtual space 400. In an embodiment, the processor 202 displays an image captured by the virtual camera 404 on the display unit 112 of the HMD 110. The virtual camera 404 similarly moves in the virtual space 400 in conjunction with the movement of the HMD 110 in the real space. Thus, the change in the position and the orientation of the HMD 110 in the real space can be similarly reproduced in the virtual space 400.

  As in the case of the HMD 110, the uvw view coordinate system is defined for the virtual camera 404. The uvw view coordinate system of the virtual camera 404 in the virtual space 400 is defined so as to be linked to the uvw view coordinate system of the HMD 110 in the real space (global coordinate system). Therefore, when the tilt of the HMD 110 changes, the tilt of the virtual camera 404 changes accordingly. The virtual camera 404 can also move in the virtual space 400 in conjunction with the movement of the user wearing the HMD 110 in the real space.

  The processor 202 of the computer 200 defines a view area 410 in the virtual space 400 based on the arrangement position of the virtual camera 404 and the reference line of sight 408. The view area 410 corresponds to an area in the virtual space 400 that is visually recognized by the user wearing the HMD 110.

  The line of sight of the user 190 detected by the gaze sensor 140 is the direction in the viewpoint coordinate system when the user 190 visually recognizes the object. The uvw view coordinate system of the HMD 110 is equal to the viewpoint coordinate system when the user 190 visually recognizes the display unit 112. The uvw view coordinate system of the virtual camera 404 is linked to the uvw view coordinate system of the HMD 110. Accordingly, the system 100 according to an aspect may regard the gaze of the user 190 detected by the gaze sensor 140 as the gaze of the user in the uvw view coordinate system of the virtual camera 404.

  The determination of the user's line of sight will be described with reference to FIG. FIG. 5 is a diagram showing the head of the user 190 wearing the HMD 110 according to an embodiment from above.

  In one embodiment, the gaze sensor 140 detects each line of sight of the right eye and the left eye of the user 190. In an aspect, when the user 190 is looking near, the gaze sensor 140 detects the lines of sight R1 and L1. In another aspect, when the user 190 is looking far, the gaze sensor 140 detects the lines of sight R2 and L2. In this case, the angle between the lines of sight R2 and L2 with respect to the roll direction w is smaller than the angle between the lines of sight R1 and L1 with respect to the roll direction w. The gaze sensor 140 transmits the detection result to the computer 200.

  When the computer 200 receives the detection values of the sight lines R1 and L1 from the gaze sensor 140 as the sight line detection result, the computer 200 specifies the gazing point N1 which is the intersection of the sight lines R1 and L1 based on the detection values. On the other hand, when the computer 200 receives the detection values of the sight lines R2 and L2 from the gaze sensor 140, the computer 200 specifies the intersection of the sight lines R2 and L2 as the gazing point. The computer 200 specifies the line of sight N0 of the user 190 based on the specified position of the gazing point N1. The computer 200 detects, for example, a direction in which a straight line passing through the midpoint of the straight line connecting the right eye R and the left eye L of the user 190 and the gazing point N1 as the line of sight N0. The line of sight N0 is the direction in which the user 190 is actually turning his or her line of sight with both eyes. The line of sight N0 corresponds to the direction in which the user 190 is actually pointing the line of sight to the field of view 410.

  In another aspect, the system 100 may include a microphone and a speaker in any part of the system 100. The user can give a voice instruction to the virtual space 400 by speaking to the microphone.

  In another aspect, system 100 may include a television broadcast receiving tuner. According to such a configuration, system 100 can display a television program in virtual space 400.

  In yet another aspect, the system 100 may include a communication circuit for connecting to the Internet, or a call function for connecting to a telephone line.

  The visibility region 410 will be described with reference to FIGS. FIG. 6 is a diagram illustrating a yz cross section of the visual field 410 viewed in the x direction in the virtual space 400. FIG. 7 is a diagram illustrating an xz cross section of the visual field 410 in the virtual space 400 when viewed from the y direction.

  As shown in FIG. 6, the visibility region 410 in the yz section includes a region 602. The area 602 is defined by the arrangement position of the virtual camera 404, the reference line of sight 408, and the yz section of the virtual space 400. The processor 202 defines a range including the polar angle α around the reference line of sight 408 in the virtual space as the region 602.

  As shown in FIG. 7, the view region 410 in the xz section includes a region 702. The area 702 is defined by the arrangement position of the virtual camera 404, the reference line of sight 408, and the xz cross section of the virtual space 400. The processor 202 defines a range including the azimuth β around the reference line of sight 408 in the virtual space 400 as a region 702. The polar angles α and β are determined according to the arrangement position of the virtual camera 404 and the orientation of the virtual camera 404.

  In an embodiment, the system 100 provides a view in the virtual space to the user 190 by displaying a view image on the display unit 112 based on a signal from the computer 200. The view image corresponds to a portion of the virtual space image 402 that overlaps with the view area 410. When the user 190 moves the HMD 110 mounted on the head, the virtual camera 404 also moves in conjunction with the movement. As a result, the position of the view area 410 in the virtual space 400 changes. As a result, the view image displayed on the display unit 112 is updated to an image of the virtual space image 402 that is superimposed on the view area 410 in the direction in which the user is facing in the virtual space 400. The user can visually recognize a desired direction in the virtual space 400.

  As described above, the direction (inclination) of the virtual camera 404 corresponds to the user's line of sight (the reference line of sight 408) in the virtual space 400, and the position where the virtual camera 404 is arranged corresponds to the user's viewpoint in the virtual space 400. Therefore, by moving the virtual camera 404 (including an operation of changing the arrangement position and an operation of changing the direction), the image displayed on the display unit 112 is updated, and the field of view (including the viewpoint and the line of sight) of the user 190 moves. Is done.

  The user 190 can visually recognize only the virtual space image 402 developed in the virtual space 400 without visually recognizing the real world while wearing the HMD 110. Therefore, the system 100 can provide the user with a high feeling of immersion in the virtual space 400.

  In an aspect, the processor 202 may move the virtual camera 404 in the virtual space 400 in conjunction with the movement of the user 190 wearing the HMD 110 in the real space. In this case, the processor 202 specifies the image area projected on the display unit 112 of the HMD 110 (that is, the view area 410 in the virtual space 400) based on the position and orientation of the virtual camera 404 in the virtual space 400.

  According to an embodiment, virtual cameras 404 may include two virtual cameras, a virtual camera for providing an image for the right eye and a virtual camera for providing an image for the left eye. Further, appropriate parallax may be set for the two virtual cameras so that the user 190 can recognize the three-dimensional virtual space 400.

  Referring to FIG. 8A, a controller 160A, which is an example of the controller 160, will be described. FIG. 8A is a diagram illustrating a schematic configuration of a controller 160A according to an embodiment.

  In certain aspects, controller 160A may include a right controller and a left controller. For simplicity, controller 160A in FIG. 8A shows the right controller. The right controller is operated by the right hand of the user 190. The left controller is operated by the left hand of the user 190. In some embodiments, the right and left controllers are configured symmetrically as separate devices. Therefore, the user 190 can freely move the right hand holding the right controller and the left hand holding the left controller. In another aspect, the controller 160 may be an integrated controller that accepts two-handed operation. Hereinafter, the right controller 160A will be described.

  The right controller 160A includes a grip 802, a frame 804, and a top surface 806. The grip 802 is configured to be gripped by the right hand of the user 190. For example, the grip 802 can be held by the palm of the right hand of the user 190 and three fingers (a middle finger, a ring finger, and a little finger).

  Grip 802 includes buttons 808 and 810 and motion sensor 130. The button 808 is arranged on the side surface of the grip 802, and receives an operation by the middle finger of the right hand. Button 810 is arranged on the front surface of grip 802, and accepts operation by the index finger of the right hand. In one aspect, buttons 808, 810 are configured as trigger-type buttons. The motion sensor 130 is built in the housing of the grip 802. Note that when the movement of the user 190 can be detected from around the user 190 by a camera or other device, the grip 802 may not include the motion sensor 130.

  Frame 804 includes a plurality of infrared LEDs 812 arranged along its circumferential direction. The infrared LED 812 emits infrared light during the execution of the program using the controller 160A in accordance with the progress of the program. The infrared rays emitted from the infrared LED 812 can be used to detect the positions and postures (tilt, direction) of the right controller 160A and the left controller (not shown). In the example shown in FIG. 8, the infrared LEDs 812 arranged in two rows are shown, but the number of arrays is not limited to that shown in FIG. One or three or more arrays may be used.

  Top surface 806 includes buttons 814 and 816 and analog stick 818. Buttons 814 and 816 are configured as push buttons. Buttons 814 and 816 accept an operation by the user's 190 right thumb. In a certain mode, the analog stick 818 receives an operation in any direction of 360 degrees from the initial position (neutral position). The operation includes, for example, an operation for moving an object arranged in the virtual space 400.

  In some embodiments, the right controller 160A and the left controller include batteries for driving components such as infrared LED 812. The battery may be any of a primary battery and a secondary battery, and its shape may be arbitrary such as a button type, a dry cell type and the like. In another aspect, right controller 160A and left controller may be connected to a USB interface of computer 200, for example. In this case, the right controller 800 and the left controller can be powered via the USB interface.

  Referring to FIG. 8B, a controller 160B, which is an example of the controller 160, will be described. FIG. 8B is a diagram illustrating a schematic configuration of a controller 160B according to an embodiment.

  The controller 160B includes a plurality of buttons 820 (820a, 820b, 820c, 820d) and 822 (822a, 822b, 822c, 822d), and left and right analog sticks 824L and 824R. Each button 820 and 822 is configured as a push button. Buttons 820 and 822 accept an operation by the thumb of user 190's hand. A trigger-type button (not shown) capable of accepting an operation by the index finger or the middle finger of the user 190 may be further provided on the controller 160B. In a certain mode, the analog sticks 824L and 824R each receive an operation in any direction of 360 degrees from the initial position (neutral position). The operation includes, for example, an operation for moving an object arranged in the virtual space 400. Buttons 820 (820a, 820b, 820c, 820d) and 822 (822a, 822b, 822c, 822d) and analog sticks 824L and 824R (and, if included, trigger-type buttons not shown) have separate operations. Commands are assigned. The operation commands include, for example, a command for giving a command to an object in the virtual space 400, a command for performing various settings on a menu screen of a game, and the like, and a computer when the user 190 is experiencing the virtual space 400. 200 includes any other commands that may be entered. The operation command assigned to each button or analog stick may be dynamically changed according to, for example, the progress of a game or a change in a scene.

  In an embodiment, the controller 160B may include a plurality of infrared LEDs (not shown) disposed on an outer surface thereof. The infrared LED emits infrared light during the execution of the program using the controller 160B in accordance with the progress of the program. Infrared light emitted from the infrared LED can be used to detect the position and posture (tilt, direction) of the controller 160B. Controller 160B includes a battery for driving internal electronic components. The battery may be any of a primary battery and a secondary battery, and its shape may be arbitrary such as a button type, a dry cell type and the like. In another aspect, controller 160B may be connected to, for example, a USB interface of computer 200. In this case, controller 160B may be powered via a USB interface.

  FIG. 9 is a block diagram illustrating functions of a computer 200 for providing a virtual experience to a user via the system 100 according to an embodiment of the present disclosure. The computer 200 executes various processes mainly based on inputs from the HMD sensor 120, the motion sensor 130, the gaze sensor 140, and the controller 160.

  The computer 200 includes a processor 202, a memory 204, and a communication control unit 205. The processor 202 includes a virtual space specifying unit 902, an HMD operation detecting unit 904, a visual line detecting unit 906, a reference visual line determining unit 908, a visual field determining unit 910, a virtual visual point specifying unit 912, and a visual image generating unit 914. , A virtual camera control unit 916, a sense shift reduction unit 918, and a visual field image output unit 924. The memory 204 can be configured to store various information. In one example, the memory 204 may include virtual space data 926, object data 928, application data 930, user operation determination data 932, view change mode data 934, stimulus data 936, and other data 938. The memory 204 also includes various data necessary for calculation for providing output information corresponding to inputs from the HMD sensor 120, the motion sensor 130, the gaze sensor 140, the controller 160, and the like to the display unit 112 associated with the HMD 110. May be. The object data 928 may include data relating to various objects arranged in the virtual space. The user operation determination data 932 may include various data necessary for determining a user operation on a housing described later. The view change mode data 934 may include various data for determining a mode that changes the view in response to the detected user operation. The stimulus data 936 may include various data for determining a stimulus corresponding to an aspect that changes the field of view. The display unit 112 may be built in the HMD 110, or may be a display of another device (for example, a smartphone) attachable to the HMD 110.

  The components included in the processor 202 in FIG. 9 are only one example of expressing a function executed by the processor 202 as a specific module. The functions of a plurality of components may be realized by a single component. Processor 202 may be configured to perform the functions of all components.

  FIG. 10 is a flowchart of a general process for displaying an image of a virtual space in which a user is immersed on the display unit 112.

  The general processing of the system 100 for providing an image of a virtual space will be described with reference to FIGS. The virtual space 400 may be provided by interaction of the HMD sensor 120, the gaze sensor 140, the computer 200, and the like.

  The process starts at step 1002. As an example, the game application included in the application data 930 may be executed by the computer 200. In step 1004, the processor 202 (or the virtual space specifying unit 902) generates the celestial virtual space image 402 constituting the virtual space 400 in which the user is immersed by referring to the virtual space data 926. The position and inclination of the HMD 110 are detected by the HMD sensor 120. Information detected by the HMD sensor 120 is transmitted to the computer 200. In step 1006, the HMD operation detection unit 904 acquires position information, inclination information, and the like of the HMD 110. In step 1008, the direction of view is determined based on the acquired position information and inclination information.

  When the gaze sensor 140 detects the movement of the left and right eyes of the user, the information is transmitted to the computer 200. In step 1010, the line-of-sight detection unit 906 specifies the direction in which the line of sight of the right eye and the left eye is directed, and determines the line-of-sight direction N0. In step 1012, the reference line-of-sight determination unit 908 determines, as the reference line-of-sight 408, the view direction determined by the inclination of the HMD 110 or the user's line-of-sight direction N 0. The reference line of sight 408 may also be determined based on the position and inclination of the virtual camera 404 that follows the position and inclination of the HMD 110.

  In step 1014, the view area determining unit 910 determines the view area 410 of the virtual camera 404 in the virtual space 400. As shown in FIG. 4, the view area 410 is a part of the virtual space image 402 that constitutes the view of the user. The field of view 410 is determined based on the reference line of sight 408. The yz sectional view of the viewing area 410 as viewed from the x direction and the xz sectional view of the viewing area 410 as viewed from the y direction are shown in FIGS. 6 and 7, respectively, which have already been described.

  In step 1016, the view image generating unit 914 generates a view image based on the view area 410. The view image includes two two-dimensional images for the right eye and the left eye. These two-dimensional images are superimposed on the display unit 112 (more specifically, the right-eye image is output to the right-eye display unit, and the left-eye image is output to the left-eye display unit). A virtual space 400 as an image is provided to the user. In step 1018, the view image output unit 924 outputs information related to the view image to the display unit 112. The display unit 112 displays the view image based on the received view image information. The process ends at step 1020.

  FIG. 11 is a block diagram schematically illustrating a VR game system 1100 according to an embodiment of the present disclosure. As illustrated, the VR game system 1100 includes a housing 1110 in addition to the computer 200 and the HMD 110 illustrated in FIG. Although the VR game system 1100 further includes other elements shown in FIG. 9, for simplification of description, illustration is omitted, and only a portion related to the housing 1110 will be described in detail. Although the computer 200 is illustrated as including one processor in FIG. 9, the computer 200 may include two or more processors and control the housing 1110 with separate processors. In the following description, it is assumed that the VR game is to go around the world while flying in the air on a snowboard. However, the present disclosure is not limited to such games and vehicles such as snowboards, but can be applied to other games and vehicles such as skis and skateboards.

  Specifically, the housing 1110 includes a first device, that is, a user operation device mechanism 1112 (including a snowboard in this example) which is a target operated by the user in the VR game, and performs a user operation on the device mechanism. A user operation detection mechanism 1114 for detection is provided. The detection data output from the user operation detection mechanism 1114 is supplied to the computer 200. The housing 1110 also includes a second device, i.e., a stimulating device mechanism 1116 for applying a stimulus to the user, and further includes a stimulating device driving mechanism 1118 for driving the applying device mechanism. I have. Computer 200 provides an output that controls stimulus delivery device drive 1118.

  FIG. 12 is a schematic diagram illustrating the structure of an embodiment of the housing 1110. As illustrated, the housing 1200 of this embodiment includes a snowboard mechanism 1220 on which a user 1210 rides, a stimulus imparting device mechanism 1116, and an example of a set of a user operation device mechanism 1112 and a user operation detection mechanism 1114. A handrail mechanism 1230 as an example of a set of the stimulating device driving mechanism 1118. The snowboard mechanism 1220 has a slender shape as a whole in the reference direction 1212 with the traveling direction of the snowboard set as a reference direction 1212 (indicated by an arrow). The snowboard mechanism 1220 also includes a board 1222 having a flat top surface on which a user rides, a roll axis parallel to the reference direction 1212 (Z axis in FIG. 12), and a floor orthogonal to the roll axis and on which the housing 1110 is placed. A rotation mechanism 1224 capable of rotating the board 1222 on two axes, a pitch axis (X-axis in FIG. 12) parallel to the axis, and an inclination sensor 1226 for detecting the inclination of the board due to the rotation. This tilt sensor may be realized using the controller 160 described above. Details of the rotation mechanism 1224 and the inclination sensor 1226 will be described later. In this example, a rotation mechanism for a yaw axis perpendicular to the roll axis and perpendicular to the floor is not provided, but such a rotation mechanism may be added depending on the content of the game. By providing the rotation mechanism 1224, the user 1210 performs an edging operation to raise a left edge when the user wants to turn left in the VR space, and performs an edging operation to raise a right edge when the user wants to turn right. The user 1210 also says to tilt the board backward when he wants to rise in the VR space, ie raises the front part (or lowers the rear part), and leans forward when he wants to lower, ie lowers the front part (or raises the rear part). Perform pitching operation. The moving speed in the VR space may be performed by any means.

  Next, as shown, the handrail mechanism 1230 includes a handrail 1232 composed of two bars extending in parallel to each other and gripped by the user 1210, an annular platform 1234, and a handrail 1232 on the annular platform. There are provided four vertically extending support rods 1236 for supporting and fixing the support, and a two-axis rotating mechanism 1238 for rotating the annular base with two axes. The two handrails 1232 extend parallel to the reference direction 1212, and have a length and arrangement such that one is grasped by the user's right hand and the other is grasped by the user's left hand. As shown, the two-axis rotating mechanism 1238 has an annular structure that is the same as the shape of an annular platform, and houses a snowboard mechanism 1220 in the center space of the annular structure. The annular structure of the annular base and the two-axis rotating mechanism 1238 has two axes of a yaw axis and a pitch axis based on the reference direction 1212 with the center of the annular structure (near the position where the user 1210 rides the snowboard mechanism 1220) as the center. Can rotate. Details of the biaxial rotation mechanism 1238 will be described later.

  FIG. 13 is a flowchart of a sensation shift reducing method 1300 according to an embodiment of the present disclosure. In one embodiment of the present disclosure, a computer program may cause processor 202 (or computer 200) to execute each step illustrated in FIG. In addition, another embodiment of the present disclosure can be implemented as a computer that includes at least a processor and executes the sensation shift reduction method 1300 under the control of the processor.

  Hereinafter, embodiments of the present disclosure will be specifically described. Here, as a specific example to which the embodiment of the present disclosure can be applied, as described above, a VR game in which the user goes around the world while flying in the air on a snowboard represented by board 1222 is assumed. However, embodiments of the present disclosure are not necessarily limited to such an aspect. It will be apparent to those skilled in the art that embodiments of the present disclosure may take various aspects that fall within the scope defined in the claims.

  13 begins with step 1302, where a virtual space for a VR game is defined. Next, in step 1304, the field of view in the virtual space is specified. Then, in step 1306, the identified view is provided to the user. Details of these steps 1302 to 1306 are as described with reference to FIG. 10, but in the VR game of this example, the inclination of the board 1222 is also used for specifying the view, that is, determining the view area. Initially, a default tilt (ie, no tilt) is used as the tilt of the board 1222.

  The process proceeds to step 1308, and the sensation shift reducing unit 918 determines whether the user has operated the board 1222 of the housing 1110 based on the output from the tilt sensor 1226. If no user operation is detected, it is determined that the inclination of the board 1222 has not changed. In this case, the process returns to step 1306 and continues to provide the user with the initially identified view. After the provision of the initially specified field of view is started, if the position and / or angle of the HMD or the direction of the user's line of sight changes, even if the inclination of the board 1222 does not change, the change is not changed. The field of view is changed based on this. On the other hand, when the inclination of the board 1222 by the user operation is detected in step 1308, the process proceeds to step 1310. In step 1310, the sensation shift reducing unit 918 specifies a view change mode based on the detected operation. Here, the view change mode refers to a mode in which the view is changed. In this example, the mode of the board edging operation, that is, the rotation operation of the roll axis, is the yawing of the field of view, that is, the change in the horizontal direction of the field of view. In the case of the operation of raising the part or the operation of tilting forward (or the operation of lowering the front part), the aspect of the pitching of the field of view, that is, the change in the vertical direction of the field of view, is the rotation operation of both the roll axis and the pitch axis. As a mode at that time, yawing and pitching of the field of view are simultaneously executed with the yawing amount and the pitching amount according to the operation amount.

  Next, in step 1312, the view area determination unit 910 updates the view based on the view change mode specified as described above. The process proceeds to step 1314, where the sensation shift reducing unit 918 refers to the stimulus data 936, specifies a stimulus corresponding to the specified visual field change mode, and gives the specified stimulus to the user. This application is performed by controlling the two-axis rotation mechanism 1238 of the handrail mechanism 1230. Here, the relationship between the target device operated by the user and the behavior of the device caused by the user operation on the device depends on the device. For example, when the device is a snowboard as in the present example, the edging operation causes a behavior of turning in the left-right direction. In view of this point, the stimulus data 936 referred to by the sensation shift reducing unit 918 may be configured to include data that defines a device and a stimulus to be given to a user according to a user operation on the device. In the VR game, a behavior of the device that does not occur in the real space can be determined for a specific user operation on the device. For example, in the game of the present example, the snowboard is tilted forward and backward so that the snowboard moves up and down in a space that does not occur in the real space. Therefore, the stimulus data 936 may include data that determines the stimulus of pitch axis rotation by the forward and backward tilting operations of the snowboard.

  Here, an example of the relationship between the visual field change mode and the corresponding stimulus will be described in detail with reference to FIG. FIG. 14 shows a virtual space, a view in the virtual space, a moving direction of the view, a handrail 1230 (in FIG. 14, only one of the left side of the board is shown for simplicity of illustration), and a board 1222 (FIG. 14). FIG. 4 is a diagram schematically showing a relationship between operations with respect to a rectangular frame of a board for simplicity of illustration. In this example, since the device operated by the user is a snowboard, for example, when turning to the left, an edging operation for raising the left edge of the board is performed. At this time, the view change mode is yaw in the left direction YL of the view to indicate that the view is turning left. In this case, when the reference direction 1212 is set to the front, a stimulus to move the front side of the handrail to the left direction TL or a stimulus to rotate the yaw axis in a counterclockwise direction when viewed from above the user are given to the user. give. Thus, the orientation of the board on which the user is riding does not change in the real space, but the stimulus of leftward yaw axis rotation is applied to the user via the handrail, so that the user is given the feeling of turning leftward. be able to. That is, a stimulus in the same direction is given to a stimulus that should be applied to the user's body when a change corresponding to the view change mode occurs in the real space. The magnitude of the stimulus at this time can be smaller than the magnitude of the stimulus that should be applied to the user's body when the change corresponding to the view change mode occurs in the real space. That is, the amount or angle of leftward yaw axis rotation may be an amount corresponding to the amount of erecting the left edge or the amount of left inclination of the board, or may be a fixed amount. May be small. Even with a small stimulus, it is possible to compensate or at least reduce the deviation from the feeling that the user is bending leftward from the virtual space, and to exhibit the effect of reducing VR sickness. The use of small stimuli also reduces the problem of code entanglement. In addition, it is possible to avoid or reduce the problem that the movement of the user is further complicated by the application of the stimulus.

  On the other hand, when turning right, an edging operation is performed to raise the right edge of the board. In order to indicate that the vehicle is turning right, the view change mode is yaw of the view in the right direction YR. The direction of the stimulus at that time is the right direction TR when the reference direction 1212 is set to the front, or the clockwise direction when viewed from above the user. By applying such a stimulus, as in the above-described example, a deviation from the sense that the user receives a rightward turn from the virtual space can be compensated or at least reduced, and an effect of reducing VR sickness occurs.

  Further, a stimulus when a pitching operation for raising a front portion or a rear portion of the snowboard is performed will be described. In this game, as described above, when the board is tilted backward (or the front portion is raised), it rises in the virtual space, and when it is tilted forward (or lowers the front portion), it descends in the virtual space. The view change mode is set. Such a change in the field of view due to the pitching operation cannot occur in the real space, but is set for playing a game of traveling around the world while flying in the air. First, when the user tilts the board backward to ascend, the view change mode at that time is pitching of the view in the upward direction PU. At this time, as a stimulus applied to the user, the front part of the handrail is moved in the upward direction TU or rotated in the pitch axis so as to give the user a feeling of moving upward. Conversely, when the user tilts the board forward to descend, the view change mode at that time is pitching in the downward direction PD of the view. At this time, as a stimulus applied to the user, the front part of the handrail is moved in the downward direction TD or the pitch axis is rotated in order to give the user a feeling of moving downward. By providing such a stimulus to the user, a stimulus corresponding to a forward leaning or backward leaning operation of the board is compensated for a deviation from a feeling that the user is moving upward or downward from the virtual space. Alternatively, it can be at least reduced, which can reduce VR sickness.

  When the stimulus applying step 1314 described above ends, the process returns to step 1306, the updated field of view is provided to the user in step 1312, and the subsequent steps 1308 to 1314 are repeated. As a result, it is possible to reduce the shift of the sense received from the real space and the virtual space and reduce VR sickness.

  FIG. 15 is a schematic diagram illustrating a more specific embodiment of the snowboard mechanism 1220 of FIG. As shown, the snowboard mechanism 1500 of this embodiment includes a flat board 1502, a two-axis rotation mechanism 1504 on which the board 1502 is mounted to allow the board 1502 to rotate in two axes, A controller 1506 that acts as a tilt sensor installed on the board. The two rotation shafts of the two-axis rotation mechanism 1504 include a roll shaft 1508 for enabling rotation by an edging operation and a pitch shaft 1510 for enabling rotation by a forward or backward tilt operation. Here, since the two-axis rotating mechanism is well known per se, a detailed description of its structure is omitted. If necessary, use dampers or springs to increase the operability of the board by applying force to keep the board flat or to suppress sudden movement of the board. You may. The controller 1506 may have the same configuration as the controller 160 as described above. From the data indicating the board inclination from the controller 1506, the type of user operation performed on the board is determined by referring to the data included in the user operation determination data as necessary. The user operation determination data may include, for example, data such as a threshold value for determining that there is a slope and an upper limit value of the slope.

  FIG. 16 is a schematic diagram illustrating another more specific embodiment of the snowboard mechanism 1220 of FIG. As shown in the figure, the snowboard mechanism 1600 of this embodiment detects a movement of the center of gravity or a weight of the user on the board as a user operation input. This example is different from an example in which the inclination of the board is detected as a user operation input, such as the snowboard mechanism 1220 in FIG. 12 and the snowboard mechanism 1500 in FIG. Specifically, the snowboard mechanism 1600 includes a flat board 1602, a table 1604 on which the board is mounted, and four thin flat plate-like pressure sensors 1606 disposed between the table and the board 1602. It has. Four pressure sensors 1606 are arranged at four corners of the board 1602, and sensor outputs are supplied to the computer 200.

  With this configuration, based on the difference between the pressures detected by the one or more pressure sensors on the left side in the traveling direction of the board and the one or more pressure sensors on the right side in the traveling direction, the user shifts weight to the left when the left side is large. It is determined that the user has shifted the weight to the right when the right pressure is large. As described above, when the user shifts his weight to the left in the traveling direction when he wants to turn left, the same process as when the pressure sensor detects the weight shift in the left direction and the above-described operation of setting the left edge is performed. Done. Conversely, if the user shifts his / her weight to the right in the traveling direction when he / she wants to turn right, the same processing as that performed when the pressure sensor detects this rightward weight shift and the above-described operation of setting the right edge is performed. Is

  Regarding the weight shift forward or backward in the traveling direction, based on the difference between the pressures detected by the one or more pressure sensors in the front and the one or more pressure sensors in the rear, when the front side is large, the user moves forward in weight. When it is determined that the user has moved, and when the pressure on the rear side is large, it is determined that the user has shifted the weight backward. As described above, when the user moves his / her weight backward in the traveling direction when he / she wants to ascend, the pressure sensor detects the weight shift to the rear side, and the same processing as when the above-described backward tilt operation is detected is performed. Conversely, if the user moves his weight forward in the traveling direction when he / she wants to descend, the pressure sensor detects this forward weight movement and the same processing as when the above-described forward leaning operation is detected is performed. The snowboard mechanism of this embodiment has an advantage that the structure is simple because there are almost no movable parts. In the example described above, the four pressure sensors are arranged at the four corners of the board, but the embodiment is not limited to this configuration. The pressure sensor may be arranged at the center of each side of the rectangular shape of the board. The method of detecting the pressure difference in that case may be changed according to the sensor arrangement position.

  As still another specific embodiment of the snowboard mechanism 1220 of FIG. 12, the snowboard mechanism 1500 may include four pressure sensors in the snowboard mechanism 1600. By arranging these four pressure sensors between the board 1502 and the biaxial rotation mechanism 1504, it is possible to detect not only a tilt operation of the board, but also a user operation due to the movement of the center of gravity on the board, and more finely. It is possible to enable a more or more precise user operation input.

  FIG. 17 is a schematic diagram showing a handrail mechanism 1700, which is another embodiment of the set of the stimulating device mechanism 1116 and the stimulating device driving mechanism 1118 shown in FIG. As shown, the handrail mechanism 1700 includes a handrail 1704 formed of a single semicircular bar extending in the horizontal direction, which is gripped by the user 1702, an annular platform 1706, and an annular platform 1706. It includes three indicator rods 1708 for supporting and fixing the handrail 1704, and a two-axis rotating mechanism 1710 for rotating the annular base with two axes. The annular base 1706 and the biaxial rotation mechanism 1710 may be similar to the annular base 1234 and the biaxial rotation mechanism 1238 in FIG. In the embodiment of FIG. 12, a pair of handrails 1232 are provided, including one handrail on each side of the snowboard mechanism 1220. On the other hand, in the embodiment of FIG. 17, one handrail 1704 is provided on the front side of a snowboard mechanism 1712 similar to the snowboard mechanism 1220 of FIG. In the present embodiment, by disposing the handrail in front of 1712, the same handrail can be grasped by both hands, so that it is easy to convey the stimulation of the yaw axis rotation and the pitch axis rotation to the user.

  FIG. 18 is a schematic diagram showing another embodiment of a seat-like mechanism 1800 of the set of the stimulating device mechanism 1116 and the stimulating device driving mechanism 1118 of FIG. The seat-like mechanism 1800 includes a semicircular seat-like member 1804 viewed from above for supporting the waist of the user 1802, a T-shaped bar 1806 extending in the horizontal direction to which the seat-like member is fixed, , A vertically extending support rod 1810 for supporting and fixing a T-shaped bar 1806 to the annular base 1808, and a two-axis rotation for rotating the annular base 1808 on two axes And a mechanism 1812. The seat-like member 1804 is generally oriented in the width direction of the board, that is, in the X-axis direction, in order to easily support the waist of the user riding the board while spreading his or her feet in the traveling direction. The annular base 1808 and the two-axis rotating mechanism 1812 may be similar to the annular base 1234 and the two-axis rotating mechanism 1238 in FIG. In the embodiments of FIGS. 12 and 17, a handrail is provided for the user to grasp. On the other hand, in the present embodiment of FIG. 18, since the user's waist can be supported, the safety of the user during playing the VR game can be further secured. Further, by providing a seat belt (not shown) on the seat-like member 1804 to prevent the user's body from detaching from the seat-like member 1804, safety can be further improved. In the present embodiment, a seat-like member is used, but a shape like a seat is not essential, and any other shape may be used as long as the stimulus can be given to the user while supporting the body of the user. .

  FIG. 19 is a schematic view showing a two-axis rotating mechanism 1900 which is a specific embodiment of the two-axis rotating mechanism shown in FIGS. 12, 17, and 18. As shown in the figure, a two-axis rotating mechanism 1900 includes an annular base 1902 fixed to the floor, an annular movable base 1904 movably mounted in a direction perpendicular to the base 1902, and an annular movable base. And a turntable 1906 that is mounted rotatably with respect to 1904. On the rotating table 1906, the circular table 1234 in FIG. 12, the circular table 1706 in FIG. 17, or the circular table 1808 in FIG. 18 is fixed.

  Specifically, the movable base 1904 is supported on a base 1902 by four electric cylinders 1908 extending in the vertical direction (see also FIG. 20). As shown, the electric cylinders 1908 are arranged at equal intervals on an annular ring, and are symmetrical with respect to the center axis in the width direction of the board (not shown) (the X axis which is the horizontal direction in FIG. 19). May be arranged. As a result, when the two electric cylinders on the front side of the board extend more than the two electric cylinders on the rear side, the movable base 1904 tilts backward by rotation of the pitch axis (X axis in FIG. 19). On the other hand, when the two electric cylinders on the rear side of the board extend more than the two electric cylinders on the front side, the movable base 1904 tilts forward due to the pitch axis rotation. As a result, the movable platform 19 and the rotating platform 1906 placed thereon, and the annular platform 1234, 1706 or 1808 placed on the rotating platform 1906 are rotated by the pitch axis. Therefore, in response to a change in visibility in the vertical direction due to the operation of tilting the board forward or backward, the user can be stimulated to rotate the pitch axis in the same direction as the direction.

  In the VR game used for the description, the roll axis (the Z axis which is the front-back direction in FIG. 19) rotation is not used. However, depending on the content of the VR game, a three-axis rotation mechanism that also provides such roll axis rotation may be provided based on the configuration of the two-axis rotation mechanism 1900 in FIG. In this case, the roll axis rotation can be generated by extending one of the electric cylinders on the front right side and the front left side of the board more than the other electric cylinder.

  FIG. 20 is a schematic cross-sectional view taken along line II of FIG. 19 to explain the details of the mechanism for rotating the turntable 1906 of FIG. 19 with respect to the movable base 1904. As shown, an electric motor 1910 is fixed to the turntable 1906, and a rack 1916 of a rack and pinion mechanism is fixed to the inner circumference 1912 of the annular portion of the movable table 1904. A pinion gear 1918 engaging with the rack 1916 is driven by the electric motor 1910. With this configuration, the rotary table 1906 is rotated with respect to the movable table 1904 by a yaw axis (Y axis which is a vertical direction in FIG. 19). Further, an annular base 1234, 1706, or 1808 placed on the rotary base 1906 is also rotated in the yaw axis. Therefore, in response to a change in the field of view in the left-right direction due to the edging operation of the board, a stimulus of yaw axis rotation in the same direction as the direction can be given to the user.

  In the two-axis rotation mechanism 1900 in FIG. 19 described above, two-axis rotation is realized using a combination of an electric cylinder, a rack and pinion mechanism, and an electric motor as an example of the stimulus imparting device driving mechanism 1118 in FIG. However, the present disclosure is not limited to such means, and any other two-axis rotating mechanism may be used. For example, instead of a combination of a rack and pinion mechanism and an electric motor, one electric cylinder is disposed between the turntable 1906 and the movable table 1904, and the expansion and contraction thereof imparts relative rotation between the turntable and the movable table. May be done. In addition, instead of the electric cylinder, a biaxial rotation mechanism may be realized by using various other methods such as a hydraulic or pneumatic cylinder and a gear mechanism other than the rack and pinion.

  Although the embodiments of the present disclosure have been described above, it should be understood that they are merely examples, and do not limit the scope of the present disclosure. It should be understood that changes, additions, improvements, etc., of the embodiments can be made without departing from the spirit and scope of the present disclosure. The scope of the present disclosure should not be limited by any of the above-described embodiments, but should be defined by the appended claims and equivalents thereof.

Reference numeral 100: system, 112: display unit, 114: sensor, 120: HMD sensor, 130: motion sensor, 140: gaze sensor, 150: server, 160A, 160B: controller, 190: head, 192: network, 200: computer , 202 processor, 204 memory, 205 communication controller, 206 storage, 208 input / output interface, 210 communication interface, 212 bus, 400 virtual space, 402 virtual space image, 404 virtual camera, 406: Center, 408: Reference line of sight, 410: Viewing area, 800: Right controller, 802: Grip, 804: Frame, 806: Top surface, 808, 810, 814, 816 ... Button, 812: Infrared LED, 818: Analog Stick, 820, 822 Button, 824R, 824L: analog stick, 902: virtual space specifying unit, 904: HMD operation detecting unit, 906: visual line detecting unit, 908: reference visual line determining unit, 910: view area determining unit, 912: virtual viewpoint specifying unit, 914: view image generation unit, 916 ... virtual camera control unit, 918 ... sense displacement reduction unit, 924 ... view image output unit, 926 ... virtual space data, 928 ... object data, 930 ... application data, 932 ... user operation determination data 934, view change mode data, 936 stimulus data, 1100 VR game system, 1110 housing, 1112 user operation device mechanism, 1114 user operation detection mechanism, 1116 stimulus application device mechanism, 1118 stimulus application device drive Mechanism, 1220 ... snowboard mechanism, 1230 ... Sliding mechanism, 1700 ... handrail mechanism, 1800 ... seat like mechanism, 1900 ... two-axis turning mechanism

Claims (14)

A computer for providing a virtual experience to the user via the image display device and the housing,
Defining a virtual space to provide a virtual experience;
Identifying a field of view in the virtual space;
Detecting a user operation on a first device included in the housing;
Based on the detected operation, identifying an aspect that changes the field of view,
Updating the field of view based on the specified aspect;
Specifying a stimulus corresponding to the specified aspect, and applying the stimulus to the user via the housing in the real space. The first device is a device on which a user rides,
The operation by the user is an operation of applying force to the first device with a foot of the user,
The detecting step includes detecting a force applied by a user to the first device,
The program according to claim 1. The operation of applying the force includes at least one of an edging operation and a center of gravity moving operation on the first device,
When the operation for applying the force is an edging operation on the first device, the detecting step includes a step of detecting a tilt of the first device,
When the operation of applying the force is an operation of moving the center of gravity to the first device, the detecting step includes a step of detecting a pressure applied to the first device.
The program according to claim 2.   The program according to claim 1, wherein the housing includes a second device for giving the stimulus to a user.   The program according to claim 4, wherein the second device is a support member that supports a user's body.   The program according to claim 5, wherein the support member is a handrail supporting a user's hand or a seat-like member supporting a user's waist.   The program according to claim 4, wherein the second device is the first device. The aspect includes movement of at least one of yawing, pitching, and rolling,
The program according to any one of claims 1 to 7, wherein the stimulus corresponding to the aspect includes at least one movement of yaw axis rotation, pitch axis rotation, and roll axis rotation.   9. The program according to claim 8, wherein when the aspect is yawing, the stimulus corresponding to the aspect is a yaw axis rotation movement.   The program according to claim 8, wherein when the aspect is pitching, the stimulus corresponding to the aspect is a pitch axis rotation motion.   The stimulus corresponding to the aspect is a compensation that compensates for a deviation of a stimulus received by the user from the housing with respect to a stimulus that should be applied to the user's body when a change corresponding to a change in the visual field of the aspect occurs in a real space. The program according to any one of claims 1 to 10, which is a stimulus.   The program according to claim 11, wherein the compensation stimulus is a stimulus in the same direction but smaller than a stimulus to be applied to a user's body when a change corresponding to the visual field change in the aspect occurs in a real space. A computer,
By a processor provided in the computer, by an image display device associated with the head of the user, by control for providing a virtual experience to the user via the housing,
Defining a virtual space to provide a virtual experience;
Identifying a field of view in the virtual space;
Detecting a user operation on a first device included in the housing;
Based on the detected operation, identifying an aspect that changes the field of view,
Updating the field of view based on the specified aspect;
A step of specifying a stimulus corresponding to the specified aspect and providing the stimulus to the user via the housing in the real space. An image display device associated with a user's head and a computer-implemented method for providing a virtual experience to a user via a housing,
Defining a virtual space to provide a virtual experience;
Identifying a field of view in the virtual space;
Detecting a user operation on a first device included in the housing;
Based on the detected operation, identifying an aspect that changes the field of view,
Updating the field of view based on the specified aspect;
Identifying a stimulus corresponding to the identified aspect, and applying the stimulus to the user via the housing in the real space.