WO2024252695A1 - Programme, procédé de traitement d'informations, et système de traitement d'informations - Google Patents

Programme, procédé de traitement d'informations, et système de traitement d'informations Download PDF

Info

Publication number
WO2024252695A1
WO2024252695A1 PCT/JP2023/045378 JP2023045378W WO2024252695A1 WO 2024252695 A1 WO2024252695 A1 WO 2024252695A1 JP 2023045378 W JP2023045378 W JP 2023045378W WO 2024252695 A1 WO2024252695 A1 WO 2024252695A1
Authority
WO
WIPO (PCT)
Prior art keywords
input
user
output control
control unit
virtual space
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2023/045378
Other languages
English (en)
Japanese (ja)
Inventor
義也 田中
陽介 金屋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
GREE Holdings Inc
Original Assignee
GREE Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by GREE Inc filed Critical GREE Inc
Priority to JP2024575686A priority Critical patent/JP7787533B2/ja
Publication of WO2024252695A1 publication Critical patent/WO2024252695A1/fr
Priority to JP2025203382A priority patent/JP2026026164A/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/0485—Scrolling or panning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics

Definitions

  • This disclosure relates to a program, an information processing method, and an information processing system.
  • a service is already being provided that displays a space, such as an event venue, created in a virtual space on a user device. For example, in this virtual space, the user can gather information by viewing exhibits (see, for example, Patent Document 1).
  • the user device can accept instructions for exhibits based on user input operations, such as selecting an exhibit or enlarging the display.
  • the user device must also accept user instructions for things other than exhibits that are related to the virtual space. The user device must distinguish between these instructions depending on the input operation. Increasing the number of acceptable instructions makes the input operation more complicated. This may impair usability, such as ease of operation. Note that similar issues exist not only in virtual spaces at event venues where exhibits are placed, but also in other virtual spaces where objects are placed that display information for users to view.
  • the program for solving the above problem causes one or more computers to function as a first output control unit that outputs a virtual space image of a three-dimensional virtual space seen from a virtual camera to a display viewed by a user, a second output control unit that outputs a projection screen that can be operated by the user within the virtual space image, an input coordinate acquisition unit that acquires input coordinates to an input device of the user, and an operation determination unit that determines whether or not an input operation by the user is an operation on an input area corresponding to the projection screen based on the acquired input coordinates, and when the operation determination unit determines that the input operation is not an operation on the input area, the first output control unit performs a first display control process for the three-dimensional virtual space other than control of the projection screen, and when the operation determination unit determines that the input operation is an operation on the input area, the second output control unit performs a second display control process on the projection screen according to the input coordinates.
  • the information processing method for solving the above problem includes a first output control process in which one or more computers output a virtual space image of a three-dimensional virtual space seen from a virtual camera to a display viewed by a user, a second output control process in which a projection screen that can be operated by the user is output within the virtual space image, an input coordinate acquisition process in which input coordinates are acquired for the user's input device, and an operation determination process in which, based on the acquired input coordinates, it is determined whether the user's input operation is an operation on an input area corresponding to the projection screen, and if it is determined in the operation determination process that the input operation is not an operation on the input area, a first display control process is performed for the three-dimensional virtual space other than control over the projection screen, and if it is determined in the operation determination process that the input operation is an operation on the input area, a second display control process is performed on the projection screen according to the input coordinates.
  • the information processing system that solves the above problem includes a first output control unit that outputs a virtual space image of a three-dimensional virtual space viewed from a virtual camera to a display viewed by a user, a second output control unit that outputs a projection screen that can be operated by the user within the virtual space image, an input coordinate acquisition unit that acquires input coordinates to an input device of the user, and an operation determination unit that determines whether or not the input operation of the user is an operation on an input area corresponding to the projection screen based on the acquired input coordinates, and when the operation determination unit determines that the input operation is not an operation on the input area, the first output control unit performs a first display control process for the three-dimensional virtual space other than control of the projection screen, and when the operation determination unit determines that the input operation is an operation on the input area, the second output control unit performs a second display control process on the projection screen according to the input coordinates.
  • FIG. 1 is a schematic diagram illustrating an example of an information processing system according to a first embodiment.
  • FIG. 2 is a diagram illustrating hardware of a device such as a user device in the first embodiment.
  • FIG. 2 is a block diagram illustrating functions of a user device according to the first embodiment.
  • FIG. 2 is a block diagram illustrating functions of a management server according to the first embodiment.
  • FIG. 2 is a diagram illustrating user management information according to the first embodiment.
  • FIG. 2 is a diagram illustrating view tally information according to the first embodiment.
  • FIG. 4 is a diagram illustrating performance information according to the first embodiment.
  • FIG. 2 is a diagram illustrating a virtual space according to the first embodiment.
  • 4A to 4E are diagrams illustrating examples of input operations received by the user device of the first embodiment.
  • FIG. 5 is a flowchart showing a processing procedure of a user device according to the first embodiment
  • 5 is a flowchart showing a processing procedure of a user device according to the first embodiment
  • FIG. 2 is a diagram illustrating a virtual space screen displayed on a user device according to the first embodiment.
  • FIG. 2 is a diagram illustrating a virtual space screen displayed on a user device according to the first embodiment.
  • FIG. 2 is a diagram illustrating a virtual space screen displayed on a user device according to the first embodiment.
  • FIG. 2 is a diagram illustrating a virtual space screen displayed on a user device according to the first embodiment.
  • 13A to 13C are diagrams illustrating transmission and reception of input operation data in a second embodiment.
  • FIG. 11 is a diagram illustrating a virtual space screen displayed on a user device according to a second embodiment.
  • the information processing system 11 includes a management server 12 and a user device 20.
  • the information processing system 11 provides a service of outputting an image of a three-dimensional virtual space (hereinafter referred to as virtual space) to a user device 20.
  • virtual space the virtual position of the user 10 is movable.
  • the virtual space is also called a world or a field.
  • the information processing system 11 is a system that distributes a video of the virtual space displayed by one user 10 to other users 10, or a system in which the user device 20 outputs an image of the three-dimensional virtual space without distributing such a video.
  • the information processing system 11 is described as a system that distributes videos.
  • the management server 12 is managed by a PF provider that provides a platform (PF) for distributing and viewing videos in a virtual space.
  • the web server 13 is managed by a publisher.
  • the publisher uses the platform to provide resources, which are information on products or services, to users 10.
  • the resources are associated in advance with the virtual space in which the resources are published.
  • the resources published in a specified area of the virtual space are associated with this publisher.
  • the resources published in a specified area of the virtual space are associated with the multiple publishers.
  • the publisher has the advantage of being able to advertise its own products or services by providing its own information to users 10.
  • the PF provider may receive compensation from the publisher for making the platform available to the publisher. Alternatively, the PF provider may be the publisher.
  • the management server 12 comprises one or more devices.
  • the management server 12 may be a group of servers.
  • the management server 12 is connected to user devices 20 via a network such as the Internet.
  • the user devices 20 are devices on which a program for distributing and viewing videos is installed (implemented), or devices used by users 10 who have registered with a video providing service for distributing and viewing videos.
  • the management server 12 is responsible for relaying data related to video distribution between the user devices 20.
  • User 10A is a user who distributes videos using user device 20A, i.e., a distribution user
  • user 10B is a user who watches the distributed videos on user device 20B, i.e., a viewing user.
  • One user 10 can be a distribution user 10A at one time and a viewing user 10B at another time. That is, when user 10 distributes videos, he is a distribution user 10A, and when he watches videos, he is a viewing user 10B.
  • user device 20A that distributes videos is called a distribution user device
  • user device 20 that outputs videos for viewing by users is called a viewing user device 20B.
  • User device 20 can be switched between a mode for distributing videos, i.e., a distribution mode, and a mode for outputting videos for viewing, i.e., a viewing mode.
  • a mode for distributing videos i.e., a distribution mode
  • a mode for outputting videos for viewing i.e., a viewing mode.
  • user 10A and viewing user 10B a mode for outputting videos for viewing 10.
  • user devices 20 when describing without distinguishing between distribution user 10A and viewing user 10B, they will simply be called user 10.
  • the broadcasting user device 20A and the viewing user device 20B without distinguishing between them they will simply be referred to as user devices 20.
  • the user device 20 is a smartphone (multi-function phone terminal), a tablet terminal, a personal computer, a console game machine, a wearable computer, or any other information processing device capable of playing videos other than these devices.
  • the wearable computer may have a screen on which the user can watch videos, and may be, for example, a head-mounted display worn on the user's head, or a glasses-type wearable terminal.
  • the user device 20 may be a device that operates independently, or may be a set of multiple devices that are connected so that various data can be transmitted and received between each other.
  • An example of the latter user device 20 is, for example, a system that performs tracking using an outside-in method.
  • the user device 20 includes a processing circuit 100, a storage 104, a sensor unit 112, a display 109, an input device 110, and peripheral devices 111.
  • the user device 20 illustrated in FIG. 2 includes a sensor unit 112 that detects the movement of the broadcast user 10A. Note that the sensor unit 112 may be a separate device from the user device 20 and may be communicably connected to the user device 20.
  • the processing circuit 100 includes a CPU 101, which is a processor that executes one or more of the control processes disclosed in this specification, and a memory 102. Processing data and instructions may be stored in the memory 102. In addition, these processing data and instructions may be stored in a storage 104, which is a storage medium disk such as a hard drive (HDD) or a portable storage medium, or may be stored in a storage medium provided separately from the processing circuit 100.
  • the storage 104 is connected to the CPU 101 via a storage controller 103 and a bus 108.
  • an application is stored in the storage 104.
  • the application includes a program for distributing and viewing videos, and various data necessary for executing the program.
  • Each function disclosed herein may be implemented using circuitry that may include, but is not limited to, a CPU 101, general-purpose processors, special-purpose processors, integrated circuits, ASICs (Application Specific Integrated Circuits), other conventional circuits, and/or combinations thereof, configured or programmed to perform the disclosed functions.
  • a processor is a processing circuit or circuitry because it includes transistors and other circuits therein.
  • a processor may be a programmed processor that executes a program stored in a memory.
  • a unit such as a processing circuit, or means, is hardware that performs or is programmed to perform the functions mentioned.
  • the hardware may be any hardware disclosed herein or otherwise known that is programmed or configured to perform the functions mentioned.
  • the processor is not limited by the form of computer-readable medium on which the instructions of the process are stored.
  • the instructions may be stored on a CD, DVD, FLASH memory, RAM, ROM, PROM, EPROM, EEPROM, hard disk or other non-transitory computer-readable medium of an information processing device with which the processing circuit 100 communicates, such as a server or computer.
  • the process may also be stored in network-based storage, cloud-based storage, or other mobile-accessible storage and executable by the processing circuit 100.
  • the hardware elements for implementing the processing circuit 100 may be implemented by various circuit elements. Furthermore, each function disclosed in this specification may be implemented by a circuit including one or more processing circuits.
  • the processing circuit 100 also includes a network controller 106 for connecting to the network NW.
  • the network NW may be a public network such as the Internet, or a private network such as a local area network (LAN) or a wide area network (WAN), or any combination thereof, and may also include a public switched telephone network (PSTN) or an integrated services digital network (ISDN, registered trademark) or a sub-network.
  • PSTN public switched telephone network
  • ISDN integrated services digital network
  • the network NW may also be a wired network such as an Ethernet network, a universal serial bus (USB) cable, or a wireless network such as a cellular network including 3G, 4G and 5G wireless cellular systems.
  • the wireless network may also be Wi-Fi, a wireless LAN, Bluetooth, or other known forms of wireless communication.
  • the network controller 106 may be compliant with other direct communication standards such as Bluetooth, near field communication (NFC), infrared, etc.
  • the processing circuit 100 further includes a display controller 105 and an input/output interface 107.
  • the display controller 105, the network controller 106, and the input/output interface 107 are connected to a bus 108.
  • the display controller 105 is connected to a display 109.
  • the input/output interface 107 is connected to an input device 110 and a peripheral device 111.
  • the sensor unit 112 is one or more sensors that detect data indicating the movement of the user 10.
  • the movement of the user 10 includes face motion indicating a change in the user's facial expression and body motion indicating a change in the relative position of the user's body with respect to the sensor unit 112. Face motion includes movements such as blinking and opening and closing of the mouth.
  • a publicly known sensor unit can be used as the sensor unit 112.
  • the sensor unit 112 has a sensor that measures the distance to the measurement object by irradiating the measurement object with light.
  • a True Depth sensor or a "LIDAR" Light Detection and Ranging, or Laser Imaging Detection and Ranging
  • the light-emitting section of the sensor unit 112 projects a dot pattern consisting of tens of thousands of invisible dots (points) onto the user's face, etc., using a dot projector.
  • the sensor unit 112 detects and analyzes the reflected light of the dot pattern to form a depth map of the face and capture an infrared image of the face, etc. This allows the sensor unit 112 to capture accurate face data.
  • the calculation processing unit of the sensor unit 112 generates various information based on the depth map and the infrared image, and compares the information with registered reference data to calculate the depth of each point on the face (the distance between each point and the near-infrared camera) and positional deviations other than in the depth direction.
  • the sensor unit 112 includes a ToF sensor that measures the time of flight (time of flight) from when light is irradiated toward a measurement target such as a user's face until it is reflected back, or the phase difference between the irradiated light and the reflected light, using a distance measurement method other than the above-mentioned method.
  • the sensor unit 112 may include a camera that captures the user's face and an image processing unit that performs image processing on the data captured by the camera. The ToF sensor calculates the distance to the measurement target based on the time of flight or phase difference.
  • the sensor unit 112 outputs the facial motion and body motion to the processing circuit 100 as tracking data.
  • the tracking data is included in the motion data.
  • Motion data refers to data in general for moving an avatar object corresponding to the user 10.
  • the motion data includes emote data in addition to tracking data.
  • Emo data is data for making the avatar object perform a pre-registered specific movement such as "clap”. Below, this pre-registered movement of the avatar object is referred to as an "emote".
  • the sensor unit 112 may also have a function of tracking not only the user's face but also their hands (hand tracking).
  • the sensor unit 112 may also include a sensor that detects the position or orientation of a human body part other than the hand.
  • the sensor unit 112 may further include a sensor that detects speed or acceleration, a sensor that detects direction or orientation (such as a gyro sensor), etc.
  • the sensor unit 112 may have a spatial mapping function that recognizes objects in the real space where the user exists based on the detection results of the various sensors described above, and maps the recognized objects onto a spatial map, i.e., creates a map of the positions of the recognized objects in the real space.
  • the input device 110 may be a touch panel, a keyboard, a mouse, a controller operated by the user's hands (e.g., a game controller such as a game pad or joystick), or an operation button provided on the housing of the user device 20.
  • the controller may have various known sensors built in, such as an acceleration sensor, an inertial measurement sensor (IMU: Inertial Measurement Unit) such as a gyroscope, etc.
  • the input device 110 may be a tracking device that identifies the user's hand movements, eye movements, head movements, gaze direction, etc.
  • the user's instructions can be determined based on the user's hand movements, and various operations can be performed, such as starting or ending video distribution, rating messages and videos, and displaying specified objects.
  • the peripheral device 111 includes a microphone that collects sounds in the real world, such as the voice of the user 10, and a speaker that outputs sound effects and the voices of other users 10, etc.
  • a program included in the application stored in the storage 104 will be described.
  • the program includes a native part implemented in the user device 20.
  • the native part runs on an OS (Operating System) of the user device 20, and can access devices (physical resources) such as the storage 104 and a camera of the user device 20.
  • the native part displays a video on the display 109 based on video data received from the management server 12.
  • the program includes an in-app browser.
  • the in-app browser is a component, such as WebView, that is incorporated into a program to realize a browser function.
  • the WebView is, for example, an API (Application Programming Interface) such as SFSafariView or WKWebView.
  • the in-app browser includes an analysis unit and an output control unit.
  • the in-app browser acquires resources.
  • the resources are, for example, web data.
  • the web server 13 provides resources associated with the virtual space to the user device 20.
  • the in-app browser is capable of acquiring resources from the web server 13 via the network.
  • the web data is data necessary for displaying a browser screen on the display 109.
  • the web data may include, for example, data written in HTML (Hyper Text Markup Language), data written in JavaScript (registered trademark) or the like, a CSS (Cascading Style Sheets) file, etc.
  • HTML Hyper Text Markup Language
  • JavaScript registered trademark
  • CSS CSS
  • the in-app browser's analysis unit interprets HTML, CSS, etc.
  • the in-app browser's output control unit executes JavaScript (registered trademark).
  • the program may launch an independent web browser (i.e., a web browser installed on the user device 20 separately from the in-app browser of the application) when a predetermined user operation is received.
  • an independent web browser i.e., a web browser installed on the user device 20 separately from the in-app browser of the application.
  • the program is described as having an in-app browser.
  • the management server 12 has a processing circuit 100 and storage 104 similar to those of the user device 20.
  • the management server 12 can omit the input device 110, the peripheral device 111, the sensor unit 112, the display 109, etc.
  • the functions of the user device 20 will be described with reference to Fig. 3.
  • the user device 20 functions as an input coordinate acquisition unit 21, an operation determination unit 22, a first output control unit 23, and a second output control unit 24 by the processing circuit 100 executing a program recorded in the storage 104.
  • the first output control unit 23 outputs a virtual space image of the virtual space viewed from the virtual camera to a display 109 which is visually recognized by the user.
  • the second output control unit 24 outputs a projection screen that can be operated by the user within the virtual space image.
  • the resources obtained from the web server 13 are displayed on the projection screen.
  • the input coordinate acquisition unit 21 acquires input coordinates, which are the coordinates of an input made to the input device 110 by a user.
  • the input coordinate acquisition unit 21 acquires a series of input coordinates corresponding to the trajectory of a touch operation on the touch panel.
  • the input coordinate acquisition unit 21 acquires a series of input coordinates corresponding to the trajectory of a drag of the mouse and input coordinates corresponding to a click.
  • the operation determination unit 22 determines whether or not the input operation by the user 10 is an operation on an input area corresponding to the projection screen, based on the acquired input coordinates.
  • the input area is a predetermined area within the virtual space screen displayed on the display 109.
  • the first output control unit 23 performs a first display control process regarding the virtual space.
  • the second output control unit 24 When the operation determination unit 22 determines that the input operation is an operation on the input area, the second output control unit 24 performs a second display control process on the projection screen according to the acquired coordinates. Furthermore, when the operation determination unit 22 determines that the acquired series of coordinates changes continuously along the first axis, i.e., that the input operation corresponds to a first scroll operation, the second output control unit 24 scrolls the projection screen in the direction of the first scroll operation.
  • the second output control unit 24 scrolls the projection screen in the direction of the second scroll operation.
  • the second scroll operation is an input operation in which the acquired series of coordinates changes continuously along the second axis.
  • the second output control unit 24 When the operation determination unit 22 determines that the input operation includes only the second scroll operation, the second output control unit 24 performs a first display control process. When the operation determination unit 22 determines that the input operation is a selection operation for selecting an arbitrary position on the projection screen, the second output control unit 24 executes a process according to the position of the selection operation.
  • the management server 12 will be described with reference to Fig. 4.
  • the management server 12 functions as a distribution management unit 30, a browsing history information receiving unit 32, and a performance recording unit 33 by the processing circuit 100 executing a program recorded in the storage 104.
  • the management server 12 also includes a user management information storage unit 34, a browsing tally information storage unit 36, and a performance information storage unit 37.
  • the distribution management unit 30 transmits video data to the user device 20 for outputting a virtual space screen including a virtual space image to the display 109.
  • the distribution management unit 30 transmits asset data to the user device 20 for drawing objects in the virtual space.
  • the distribution management unit 30 receives data for drawing avatar objects, including motion data, from the distribution user device 20A, and transmits the received data to other user devices 20.
  • the result recording unit 33 calculates the number of times each resource has been viewed based on the viewing history information, and records the number of times each resource has been viewed in the result information storage unit 37 as the result information of the resource.
  • ⁇ Data structure> Each piece of information recorded in the management server 12 will be described with reference to FIGS.
  • the user management information 40 is recorded for each user.
  • the user management information 40 includes a user ID (identifier), which is user identification information, accumulated points, and coins.
  • the accumulated points are points that are awarded for activities on an application or a platform that provides the application, such as distribution and viewing of videos, and are accumulated for each user 10.
  • the accumulated points can be used as a consideration for a lottery (gacha) that can be executed on the platform, a consideration for a gift object, or a consideration for an item.
  • a gift object is an object that a viewing user 10B provides to a distribution user 10A, etc. Items include items that are worn by an avatar object and items used in a game provided by the platform.
  • the accumulated points may also be exchangeable for currency used in the real world. Coins differ from the accumulated points in that the user 10 can purchase them. Coins may be awarded according to the activity (distribution status or viewing status) of the user 10. The coins may also be used as a consideration for a lottery, a consideration for a gift object, etc., like the accumulated points.
  • FIG. 6 shows an example of the viewing summary information 42 recorded in the viewing summary information storage unit 36.
  • the viewing summary information 42 is updated based on the viewing history information transmitted from the user device 20.
  • the viewing summary information 42 includes a user ID, a URL (Uniform Resource Locator), a room ID, a viewing start time, a viewing end time, and a behavior history.
  • the viewing start time is the time when the user device 20 connects to the web server 13 or the time when the user device 20 displays a resource associated with the virtual space.
  • the viewing end time is the time when the user device 20 disconnects from the web server 13 or the time when the user device 20 ends displaying the resource.
  • the behavior history indicates the behavior performed by the user 10 on the displayed resource. Examples of the behavior history include requesting materials on a resource, purchasing a product or the like on a resource, etc.
  • FIG. 7 shows an example of performance information 43 recorded in performance information storage unit 37.
  • Performance information 43 is recorded for each publisher.
  • Publishers publish resources. Multiple resources may be associated with one publisher. Publishers can advertise products and services to users through resources displayed in the virtual space.
  • the performance information 43 includes the publisher identifier, URL, user stay time, questionnaire response information, and number of views.
  • the publisher identifier is an identifier assigned to the publisher.
  • the user stay (view) time indicates the time each user 10 viewed the resource, or statistical information on those viewing times. This time can be calculated from the viewing start time and viewing end time in the viewing summary information 42.
  • the user stay time may be recorded as the time that the user viewed the resource individually. Alternatively, the user stay time may be the average stay time of the user, that is, the average value of those individually recorded times.
  • the questionnaire response information indicates the content of the questionnaire that the user 10 answered in the virtual space, that is, the user 10's response to the questionnaire in the virtual space.
  • the number of views is the number obtained by aggregating the viewing history information and is the total number of times the resource was displayed.
  • the management server 12 receives a video distribution request from the user device 20 of the user 10. At this time, the user 10 can specify a virtual space. When the management server 12 receives the distribution request and identification information of the specified virtual space, it generates a room (distribution slot) for the virtual space specified by the user 10. In addition, the management server 12 transmits data (asset data) for displaying the virtual space to the user device 20.
  • the video can be delivered and viewed using one of the following methods: client rendering, browser rendering, video delivery, and server delivery.
  • client rendering method is used.
  • each user device 20 renders objects by executing a program.
  • the user device 20 stores three-dimensional model data of avatar objects and objects other than avatar objects (hereinafter referred to as physical objects).
  • the three-dimensional model data of avatar objects includes video data of the main body parts that make up the avatar object itself and the attachable parts that can be attached to the avatar object.
  • the data of the main body parts includes polygon data, skeletal data (bones) for expressing the movement of the avatar object, texture data, etc.
  • the attached parts include texture data, etc.
  • the broadcasting user 10A can set the attached parts of the avatar object corresponding to him/her according to his/her preferences.
  • the model data of physical objects other than avatar objects includes polygon data, texture data, etc.
  • the first output control unit 23 performs drawing using video data including motion data and identification information (parts IDs) indicating parts of each object.
  • the first output control unit 23 also transmits video data including motion data and coordinates of the avatar object to the management server 12.
  • the first output control unit 23 also transmits part IDs indicating each part of the avatar object to the management server 12 at a predetermined timing (e.g., when distribution starts).
  • the management server 12 transmits the video data to the viewing user device 20B, etc.
  • the viewing user device 20B receives the data from the management server 12.
  • the viewing user device 20B uses the received data to draw images of the avatar object and physical object of the broadcasting user 10A.
  • the first output control unit 23 also transmits audio data based on the sound picked up by the microphone to the management server 12.
  • the management server 12 transmits the audio data to the viewing user device 20B.
  • a timestamp is attached to the motion data, audio data, etc.
  • the first output control unit 23 of the viewing user device 20B receives video data and audio data for displaying the video from the management server 12. The first output control unit 23 then uses the video data to draw an image including an avatar object corresponding to the broadcast user 10A.
  • the first output control unit 23 of the broadcasting user device 20A and the viewing user device 20B also receives data such as messages posted by other users from the management server 12.
  • the first output control unit 23 generates video data by combining rendered images of avatar objects and other objects with posted messages, notifications, etc., and outputs the generated video data to the display 109.
  • the first output control unit 23 also synchronizes audio data with the video data based on a timestamp and outputs it from the speaker.
  • each user device 20 generates video using a client rendering method.
  • the broadcasting user device 20A and the viewing user device 20B may display video using different methods.
  • the broadcasting user device 20A may display video using a client rendering method
  • the viewing user device 20B may display video using a browser rendering method.
  • the user may be able to select the method for displaying video.
  • some of the viewing user devices 20B may display video using a client rendering method, while other devices may display video using a browser rendering method.
  • a virtual space 50 in this embodiment will be described with reference to Fig. 8.
  • the virtual space 50 includes a plurality of objects 51.
  • the objects 51 located in the virtual space 50 include an avatar object 52 and other physical objects 53.
  • a world coordinate system 90 is set, which indicates a position in the virtual space.
  • the world coordinate system is a coordinate system having an X-axis, a Y-axis, and a Z-axis.
  • the world coordinate system 90 is also called a global coordinate system.
  • the position of the object 51 can be specified by the coordinates of the world coordinate system 90.
  • the coordinates specified in the world coordinate system 90 include coordinates (X, Y, Z) expressed in a Cartesian coordinate system. These coordinates may also include rotation coordinates (roll, pitch, yaw) for each coordinate axis.
  • Each object 51 has its own object coordinate system 91 set.
  • the object coordinate system 91 is a coordinate system having an X-axis, a Y-axis, and a Z-axis.
  • the orientation, movement, etc. of the avatar object 52 can be specified by the coordinates of the Cartesian coordinate system or the rotation coordinates.
  • a virtual camera 54 associated with each broadcast user 10A is positioned.
  • a camera coordinate system 92 is set for the virtual camera 54.
  • the camera coordinate system 92 includes coordinates (X, Y, Z) specified in a three-dimensional Cartesian coordinate system, as well as rotational coordinates (X ⁇ , Y ⁇ , Z ⁇ ) centered on each coordinate axis.
  • the first output control unit 23 performs rendering based on the coordinates of objects included in a predetermined field of view (for example, a field of view cone having an approximately pyramidal shape) centered on the virtual optical axis of the virtual camera 54.
  • the first output control unit 23 then outputs a two-dimensional image generated by rendering to the display 109.
  • the rendering here refers to a drawing process that includes obtaining the position of the virtual camera 54, perspective projection, and hidden surface removal based on depth information associated with each object.
  • the rendering may be at least one of these processes, or may further include processes such as shading and texture mapping.
  • the position of the virtual camera 54 can be changed by at least one of the broadcasting user 10A and the viewing user 10B.
  • the broadcasting user 10A can select a first-person perspective (avatar position) or a third-person perspective as a setting for the position of the virtual camera 54.
  • the first output control unit 23 performs rendering with the position of the avatar's eyes coinciding or nearly coinciding with the position of the virtual camera 54.
  • the line of sight extending from the front of the avatar object 52 nearly coincides with the optical axis of the virtual camera 54.
  • an image is displayed on the display 109 as if the virtual space 50 is being viewed from the avatar object 52, i.e., through the eyes of the avatar.
  • the first output control unit 23 sets the virtual camera 54 at a position around the avatar object 52.
  • an image of the avatar object 52 can be displayed on the screen.
  • the first output control unit 23 can change the position of the virtual camera 54 in the third-person perspective.
  • an image may be generated using a multi-view camera in which one (or more) of virtual cameras 54 positioned at three or more positions, including a first-person viewpoint position, a third-person viewpoint position, and one or more other positions, is selected.
  • the camera coordinate system 92 determines the camera orientation and operations such as panning and tilting.
  • the first output control unit 23 changes at least one of the position and direction of the virtual camera 54 based on the operation of the user 10.
  • the first output control unit 23 changes at least one of the position and direction of the virtual camera 54 based on the operation of the user 10 to move the avatar object 52.
  • the virtual camera 54 may be associated with the avatar object 52 so as to maintain a positional relationship with respect to the avatar object 52.
  • the first output control unit 23 causes the virtual camera 54 to follow the movement of the avatar object 52 during a period in which no camera operation is being performed.
  • the first output control unit 23 acquires the movement direction indicated by the distribution user 10A as a first direction, converts the first direction into a second direction as a movement direction in the world coordinate system 90, and moves the avatar object 52 in the second direction.
  • the first output control unit 23 moves the virtual camera 54 in the second direction while keeping the relative distance between the avatar object 52 and the virtual camera 54 constant.
  • the user 10 can change at least one of the position and direction of the virtual camera 54 by an input operation.
  • the first output control unit 23 changes the position of the virtual camera 54 based on a slide operation by the user 10.
  • a browser screen 57 is output as a projection screen in the virtual space 50.
  • the browser screen 57 is a screen projected onto a display object 58.
  • the display object 58 is an object that has coordinates in the world coordinate system in the virtual space 50. Only one display object 58 may be provided in the virtual space 50, or multiple display objects 58 may be provided.
  • a display object 58 may be provided for each of a number of exhibition booths.
  • the display object 58 is an object for displaying resources made public by the exhibition booth owner.
  • the display object 58 is, for example, a two-dimensional object with no thickness, or a three-dimensional object with a display surface. Note that an exhibition object 51 other than the display object 58 may be set as an exhibit in the exhibition booth.
  • the second output control unit 24 reads web data identified by a specific URL using an in-app browser, and outputs a browser screen 57 based on the web data to a display object 58 using a technique such as texture mapping.
  • the browser screen 57 may include images such as still images in JPEG, GIF, PNG, etc., and videos based on video data such as MPEG4.
  • the browser screen 57 is displayed in the same manner as if it were displayed by a web browser separate from the in-app browser of the application. Note that details of the browser screen 57 are omitted in Figure 8.
  • the first output control unit 23 converts coordinates P1 in the display area 97 displayed on the screen output to the display 109 into coordinates P2 on the browser screen 57 (see arrow 95 in Figure 8). An example of this coordinate conversion process will be described.
  • An input coordinate system 93 is set in the virtual space 50.
  • the input coordinate system 93 is a two-dimensional Cartesian coordinate system that specifies coordinates on the screen of the display 109.
  • the Y axis of the input coordinate system 93 corresponds to the first axis.
  • the X axis corresponds to the second axis.
  • a UI area 55 and a browser area 56 are set.
  • the browser area 56 corresponds to the input area.
  • the coordinates on these areas are specified by the input coordinate system 93.
  • the first output control unit 23 or the second output control unit 24 detects the position and direction of the input operation by acquiring the coordinates of the input operation by the user 10 for these areas.
  • the first output control unit 23 or the second output control unit 24 determines the content of the user instruction corresponding to the input operation according to at least one of the position and direction of the input operation, and performs control according to the user instruction.
  • the UI area 55 is an area (layer) that accepts operations on GUI (Graphical User Interface) elements such as buttons, text boxes, and icons.
  • FIG. 8 shows UI areas 55A to 55D as the UI area 55. Although the UI area 55 itself is hidden, GUI elements (not shown in FIG. 8) are displayed within the UI area 55. That is, while the GUI elements within the UI area 55 are displayed on the virtual space screen, the UI area 55 itself is not displayed. In the example of FIG. 8, a GUI element such as a setting button is displayed within the UI area 55A. In the UI areas 55B and 55C, GUI elements such as a button for moving the avatar object 52 and a button for jumping are displayed, respectively. In the UI area 55D, a GUI element such as a text box in which a message can be input is displayed. Note that the UI areas 55A to 55D in FIG. 8 are simplified views of the actual screens.
  • the browser area 56 is an area (layer) that accepts input operations for the browser screen 57.
  • the coordinates of the browser area 56 can be specified by the input coordinate system 93.
  • the first output control unit 23 sets the browser area 56 when the display conditions for the browser screen 57 are satisfied. At this time, the first output control unit 23 sets the browser area 56 so that the browser area 56 overlaps with the browser screen 57 when viewed from the virtual camera 54.
  • the first output control unit 23 sets the browser area 56 to match the shape and size of the browser screen 57 as seen by the virtual camera 54. For example, when the browser screen 57 becomes larger on the screen as a result of the virtual camera 54 approaching the display object 58 in response to the operation of the user 10, the first output control unit 23 enlarges the browser area 56 to match the size of the browser screen 57. Similarly, when the browser screen 57 becomes smaller on the screen, the first output control unit 23 reduces the browser area 56 to match the size of the browser screen 57. Furthermore, when the relative angle between the virtual camera 54 and the browser screen 57 changes in response to the operation of the user 10, the first output control unit 23 deforms the browser area 56 to match the shape of the browser screen 57 on the screen.
  • the operation determination unit 22 identifies the coordinates of the input to the browser area 56. In other words, the operation determination unit 22 identifies an input trajectory that corresponds to the trajectory of continuous input in the browser area 56. The operation determination unit 22 also converts the identified coordinates into coordinates on the browser screen 57. The coordinates on the browser screen 57 are identified in the browser screen coordinate system 94.
  • the second output control unit 24 receives a request for display control based on input coordinates, which are coordinates in the browser screen coordinate system 94, from the operation determination unit 22.
  • the second output control unit 24 performs second output control processing based on user instructions for controlling the browser screen 57. For example, when a series of input coordinates on the browser screen 57 are consecutive coordinates along the Y axis (positive Y direction and negative Y direction) of the browser screen coordinate system 94, the second output control unit 24 determines that an input operation for scrolling the browser screen 57 has been performed. Then, the second output control unit 24 scrolls the browser screen 57 in the same direction as the direction of the input operation.
  • the operation determination unit 22 When the operation determination unit 22 acquires coordinates in the input coordinate system 93 other than the UI area 55 and the browser area 56 as input coordinates, it requests the first output control unit 23 to perform display control based on the acquired coordinates.
  • the first output control unit 23 performs display control processing related to the virtual space 50.
  • the input device 110 is a touch panel.
  • An input operation on the touch panel is performed by contact with a finger or a stylus or the like.
  • An input operation on the browser area 56 only needs to be executable by at least one of the broadcasting user 10A and the viewing user 10B.
  • the operation determination unit 22 of that user device 20 accepts an input operation into the browser area 56.
  • the operation determination unit 22 of that user device 20 does not accept an input operation into the browser area 56. In other words, only the broadcast user 10A can perform an input operation into the browser area 56.
  • the virtual space screen 60 includes a virtual space image 62 including other objects 51 (not shown in FIG. 9), and GUI elements (not shown in FIG. 9).
  • the direction parallel to the X axis of the input coordinate system 93 (positive X direction and negative X direction) is referred to as the horizontal direction.
  • the positive X direction is the direction in which the X coordinate increases
  • the negative X direction is the direction in which the X coordinate decreases.
  • the direction parallel to the Y axis of the input coordinate system 93 (positive Y direction and negative Y direction) and perpendicular to the X axis is referred to as the vertical direction.
  • the positive Y direction is the direction in which the Y coordinate increases
  • the negative Y direction is the direction in which the Y coordinate decreases.
  • the first output control unit 23 determines the moving direction of the position of the virtual camera 54 based on a predetermined condition.
  • the determination of the direction indicated by the user 10 may be performed using a known method, and is not particularly limited. For example, when the user 10 performs an input operation that is diagonal to the X-axis and Y-axis, the first output control unit 23 compares the amount of change in the X-coordinate and the amount of change in the Y-coordinate of the input trajectory (vector). The first output control unit 23 then determines that the input operation is an input operation along the one of the two coordinate axes with the greater amount of change.
  • the first output control unit 23 determines that the input operation is an input operation along the X-axis.
  • the first output control unit 23 may determine the direction of the input operation based on the angle of the vector of the input trajectory.
  • the user 10 can specify an input start position PA outside the UI area (not shown in FIG. 9) and outside the browser area 56 by a touch operation.
  • the operation determination unit 22 acquires the input coordinates of the input start position PA.
  • the user 10 may slide the touch position in the X-positive direction from the input start position PA to the input end position PB.
  • the operation determination unit 22 acquires a series of input coordinates that change continuously from the input start position PA to the input end position PB in response to this sliding operation.
  • the operation determination unit 22 requests the first output control unit 23 to perform display control based on the acquired input coordinates.
  • the first output control unit 23 performs a first display control process.
  • the position of the virtual camera 54 is changed. More specifically, the first output control unit 23 moves the position of the virtual camera 54 in a direction from the input start position PA toward the input end position PB (the X-positive direction in FIG. 9).
  • the position of the virtual camera 54 may be moved parallel to the X-axis of the camera coordinate system 92, or may be moved in an arc so as to maintain a constant relative distance to a predetermined object 51 (e.g., an avatar object 52).
  • the position of the virtual camera 54 may be moved in the positive Y direction (or negative Y direction).
  • the user 10 can specify an input start position PC in the browser area 56.
  • the operation determination unit 22 acquires the input coordinates of the input start position PC.
  • the user 10 may slide the touch position in the negative Y direction from the input start position PC to the input end position PD.
  • the operation determination unit 22 acquires a series of input coordinates that change continuously from the input start position PC to the input end position PD in response to this slide operation.
  • the operation determination unit 22 requests the second output control unit 24 to perform display control based on the acquired input coordinates.
  • the second output control unit 24 scrolls the browser screen 57 in the negative Y direction of the browser screen coordinate system 94 (the same direction as the negative Y direction of the input coordinate system 93). In other words, the second output control unit 24 displays a range that is not displayed on the browser screen 57 shown in FIG.
  • the first output control unit 23 may determine that the operation is a vertical scroll operation on the browser screen 57 even if the input end position PD is outside the browser area 56, as long as part of the input trajectory of the vertical slide operation is included in the browser area 56.
  • the operation determination unit 22 may determine that the input operation is a combination of a designation operation, which is a long press operation, and a horizontal slide operation. At this time, the operation determination unit 22 requests the second output control unit 24 to perform display control based on the acquired input coordinates.
  • the second output control unit 24 determines whether the input start position PC is included in the horizontally scrollable display range 61.
  • the horizontally scrollable display range is the range of the screen in which the display content can be switched by moving it left and right, and is also called a carousel.
  • the second output control unit 24 determines that the input start position PC is included in the horizontally scrollable display range 61, it scrolls that range of the browser screen 57 in the direction from the input start position PC toward the input end position PE.
  • the user 10 may perform a slide operation in the positive X direction from the input start position PC to the input end position PF without performing a long press operation.
  • the operation determination unit 22 acquires a series of input coordinates that change continuously from the input start position PC to the input end position PF in response to this slide operation.
  • the operation determination unit 22 requests the first output control unit 23 to perform display control based on the acquired input coordinates.
  • the first output control unit 23 moves the position of the virtual camera 54 in the positive X direction.
  • the user 10 can move the position of the virtual camera 54 by performing a horizontal sliding operation outside the UI area 55 and outside the browser area 56, or a sliding operation that does not include a long press operation within the browser area 56.
  • the operation determination unit 22 outputs a determination result including the input coordinates to the second output control unit 24.
  • the second output control unit 24 performs display control according to the input coordinates acquired from the operation determination unit 22. For example, when the input coordinates are included in a video display area, the second output control unit 24 plays a stopped video or stops a video being played.
  • the video display area is an area where thumbnails (small-sized images) of videos are displayed.
  • the second output control unit 24 acquires a resource specified by the link from the web server 13. The second output control unit 24 then displays the acquired link on the browser screen 57.
  • the operation determination unit 22 determines that it is a pinch operation, and outputs the determination result including the input coordinates to the first output control unit 23.
  • the first output control unit 23 reduces or enlarges the entire virtual space image 62, regardless of whether the acquired input coordinates are included in the browser area 56. In other words, even if the user 10 performs a pinch out operation within the browser area 56, the virtual space image 62 is enlarged. As a result, the browser screen 57 included in the virtual space 50 is also enlarged.
  • the broadcast user device 20A When the broadcast user device 20A performs display control in response to the input operation of the broadcast user 10A in this way, the broadcast user device 20A transmits input operation data indicating the content of the input operation to the viewing user device 20B via the management server 12.
  • the input operation data includes an operation instruction for the browser screen 57, identification information of the display object 58 to be operated, the operation amount, the position of the virtual camera 54, etc.
  • the operation instruction is vertical scrolling, horizontal scrolling, link selection, stopping or playing a video, etc.
  • the operation amount is the operation amount of scrolling, etc.
  • the viewing user device 20B performs display control of the browser screen 57 and controls the virtual camera 54 based on the input operation data.
  • the screen output to the broadcast user device 20A and the screen output to the viewing user device 20B are synchronized.
  • the broadcast user device 20A may transmit data indicating the display position of the browser screen 57 to the viewing user device 20B in order to synchronize the browser screens 57 between the broadcast user device 20A and the viewing user device 20B.
  • the broadcast user device 20A displays a virtual space screen 60 including a browser screen 57 on the display 109.
  • the display condition of the browser screen 57 is that the position of the avatar object 52 is within a predetermined area set for the display object 58, or that the broadcast user 10A has performed an operation to display the browser screen 57.
  • the second output control unit 24 determines that the display condition is satisfied, it requests a resource specified by the URL from the web server 13 based on the URL associated with the display object 58.
  • the second output control unit 24 receives the resource from the web server 13, it launches an in-app browser and outputs the browser screen 57 based on the resource to the display surface of the display object 58 in the virtual space 50.
  • the input coordinate acquisition unit 21 acquires input coordinates, which are the coordinates of an input made by the broadcast user 10A to the input device 110 (step S10).
  • the operation determination unit 22 determines which area of the input coordinate system 93 the input coordinates are an operation for. For example, the operation determination unit 22 determines whether the acquired input coordinates are included in the UI area 55 (step S11). If the operation determination unit 22 determines that the input coordinates are included in the UI area 55 (step S11: YES), it requests the first output control unit 23 to perform display control processing corresponding to that UI area 55. The first output control unit 23 performs display control processing corresponding to that UI area 55 (step S12). For example, if the first output control unit 23 determines that a GUI element for moving the avatar object 52 has been operated, it moves the avatar object 52 in the direction indicated by the broadcast user 10A.
  • step S11 determines whether or not the input coordinates are included within the browser area 56 (step S13).
  • step S13 determines that the input coordinates are included within the browser area 56 (step S13: YES)
  • it requests the second output control unit 24 to perform display control based on the input coordinates.
  • the second output control unit 24 performs operation determination processing based on the input coordinates (step S14).
  • the second output control unit 24 converts the input coordinates into coordinates on the browser screen 57.
  • the second output control unit 24 determines the input operation that corresponds to the converted coordinates.
  • the second output control unit 24 determines that the input operation is a pinch-in operation, it reduces the virtual space image 62. If the second output control unit 24 determines that the input operation is a pinch-out operation, it enlarges the virtual space image 62. If the second output control unit 24 determines that the input operation is a selection operation, it transitions the screen, or stops or plays the video, depending on the input coordinates.
  • the second output control unit 24 determines whether the user instruction is for vertical scrolling, horizontal scrolling, or control of the virtual camera 54.
  • the second output control unit 24 determines whether the slide operation is a vertical operation (step S20). If the second output control unit 24 determines that the slide operation is a vertical operation (step S20: YES), it scrolls the browser screen 57 vertically (step S21). If the second output control unit 24 determines that the slide operation is a horizontal operation rather than a vertical operation (step S20: YES), it determines whether the input operation includes a designation operation (step S22).
  • step S22 determines that the input operation includes a designation operation (step S22: YES)
  • the broadcast user device 20A When the broadcast user device 20A outputs the browser screen 57, it transmits browsing history information to the management server 12.
  • the browsing history information includes the broadcast user's 10A user ID, URL, room ID, and browsing start time. Furthermore, the broadcast user 10A may transmit browsing information including the input operation as a behavior history to the management server 12 each time an input operation is performed. Furthermore, when the broadcast user device 20A no longer outputs the browser screen 57, it transmits browsing history information including the viewing end time to the management server 12.
  • the browsing history information receiving unit 32 uses the browsing history information to record the viewing tally information 42 in the viewing tally information storage unit 36.
  • the performance recording unit 33 generates performance information 43 using the viewing count information 42. At this time, the number of views may be calculated based on the number of viewing users 10B. After generating the performance information 43, the performance recording unit 33 records it in the performance information storage unit 37.
  • the administrator of the management server 12 e.g., the PF provider
  • the publisher can check the user response by checking the frequency with which the information they published was viewed and the responses to questionnaires, etc. For example, the administrator can receive compensation from the publisher according to the performance information.
  • FIG. 12 The virtual space screen 60 (distribution screen) displayed by the broadcasting user device 20A will be described with reference to Figures 12 to 15. Note that Figures 12 to 15 show a screen when the position of the virtual camera 54 is a third-person perspective. Also, the virtual space screen (viewing screen) displayed by the viewing user device 20B is almost the same as the virtual space screen 60 displayed by the broadcasting user device 20A, except for the difference in GUI elements. However, in this embodiment, the viewing user 10B cannot operate the browser screen 57 as described above.
  • an avatar object 52 is displayed on the virtual space screen 60.
  • the first output control unit 23 includes the display object 58 in the virtual space screen 60.
  • the first output control unit 23 determines that the display conditions are met, it outputs a browser screen 57.
  • the browser screen 57 displays a screen of a website identified by a URL associated with the display object 58. For example, if an exhibition is being held in the virtual space 50, a screen of a website related to the contents of the exhibition is displayed.
  • the website is a site that displays information about the exhibitor, details of a product or service, etc.
  • the virtual space screen 60 also displays a plurality of GUI elements 65 and icons.
  • the virtual space screen 60 also displays a movement button 66 and a jump button 67, which are GUI elements.
  • the first output control unit 23 moves the avatar object 52.
  • the jump button 67 the first output control unit 23 makes the avatar object 52 jump. This makes it possible to move the avatar object 52 to any exhibition booth, for example.
  • the broadcast user 10A When the broadcast user 10A performs a vertical slide operation on the browser screen 57 (see arrow 96 in FIG. 12), the second output control unit 24 scrolls the browser screen 57 in the direction of the operation.
  • the broadcast user 10A can perform actions similar to those performed on an independent browser. For example, the broadcast user 10A can view information about products, purchase products, request documents, etc.
  • the broadcasting user 10A when the broadcasting user 10A views information about a product or the like on the browser screen 57, the viewing user 10B who watches the broadcasted video will also see that information. For this reason, the publisher can advertise the product or the like by exhibiting in the virtual space 50 and having the broadcasting user 10A broadcast a video. Furthermore, if the broadcasting user 10A is able to operate the browser screen 57 within the virtual space 50, more information can be provided to the broadcasting user 10A and the viewing user 10B compared to when a non-operable image is displayed in the display object 58. For this reason, the publisher can further increase the advertising effect of the information they publish.
  • FIG. 13 shows a virtual space screen 60 in which the position of the virtual camera 54 has been moved.
  • the first output control unit 23 changes the position 54A of the virtual camera 54 to position 54B based on the direction of the input operation.
  • the first output control unit 23 may move the virtual camera 54 a distance according to the amount of the input operation while keeping the virtual optical axis of the virtual camera 54 facing the avatar object 52.
  • the first output control unit 23 outputs to the display 109 a virtual space image 62 included in the field of view Z1 of the virtual camera 54 at position 54B.
  • the second output control unit 24 scrolls the display range 99 horizontally (see arrow 98 in FIG. 14).
  • FIG. 15 shows the virtual space screen 60 when the broadcast user 10A pinches out in the browser area 56.
  • the second output control unit 24 enlarges only the virtual space image 62 on the virtual space screen 60.
  • the browser screen 57 is also enlarged and displayed.
  • a browser screen 57 that allows input operations by the user 10 is displayed on the virtual space screen 60.
  • the user 10 can efficiently collect information that interests him or her while exploring the virtual space 50.
  • the publisher can provide information to users 10 who tend to behave differently from customers in the real world as potential customers through the browser screen 57.
  • the operation determination unit 22 distinguishes between operations for controlling the display of the browser screen 57 and operations for controlling the virtual camera 54 according to the input coordinates. Therefore, the user 10 can intuitively operate the browser screen 57 without performing complicated operations.
  • the operation determination unit 22 determines that the input coordinates are a scroll operation that changes continuously along the first axis (vertical direction)
  • the second output control unit 24 scrolls the browser screen 57 in the direction of the operation. This allows the user 10 to scroll the browser screen 57 along the first axis with an intuitive operation.
  • the operation determination unit 22 determines that the input operation is a combination of a predetermined designation operation and a scroll operation in which the input coordinates change continuously along the second axis (horizontal direction)
  • the second output control unit 24 scrolls the horizontally scrollable display area 97 of the browser screen 57 in the direction of the operation.
  • the operation determination unit 22 determines that the input operation includes only a scroll operation in which the input coordinates change continuously along the second axis
  • the first output control unit 23 changes the position of the virtual camera 54 in the direction of the operation. This allows the user device 20 to distinguish between a slide operation that changes the position of the virtual camera 54 and an operation for scrolling along the second axis.
  • the operation determination unit 22 determines that the input operation is not an input operation within the UI area 55 or the browser area 56, the first output control unit 23 changes the position of the virtual camera 54 in the direction of the operation. This allows the user 10 to control the virtual camera 54 by intuitive operations.
  • the operation determination unit 22 determines that the input operation is a selection operation in the browser area 56, the second output control unit 24 executes processing according to the position of the selection operation. This allows the user 10 to control the virtual camera 54 with an intuitive operation.
  • the first output control unit 23 outputs an avatar object 52 corresponding to the user 10 into the virtual space. As a result, through the avatar object 52, it is possible to stimulate at least one of the interactions between the broadcast user 10A and the viewing users 10B, and interactions between the broadcast users 10A.
  • the second output control unit 24 accepts input operations into the browser area 56 in the distribution mode, but does not accept input operations into the browser area 56 in the viewing mode. This allows the viewing user 10B to view the browser screen 57 being viewed by the distribution user 10A on the viewing user device 20B, making it easier for them to interact with the distribution user 10A.
  • the broadcast user 10A performs input operations in the browser area 56.
  • a viewing user 10B who has authority can operate the browser screen 57.
  • multiple broadcast users 10A perform collaborative broadcasting in which multiple broadcast user devices 20 broadcast together. Even in collaborative broadcasting, each broadcast user 10A can independently perform input operations in the browser area 56.
  • Collaboration distribution refers to multiple users distributing videos in the same virtual space 50.
  • the distribution user 10A who requests the creation of a room will be referred to as the host user
  • the distribution user 10A who participates in the room created by the host user will be referred to as the guest user.
  • User 10 can participate in the collaboration distribution as a guest user by accepting the participation request from the host user, or by having the participation request sent by user 10 approved by the host user.
  • the broadcasting user device 20A used by the host user is referred to as the host user device 20H
  • the broadcasting user device 20A used by the guest user is referred to as the guest user device 20G.
  • the host user device 20H outputs a virtual space screen 60 seen from a virtual camera 54 set by the host user.
  • the guest user device 20G outputs a virtual space screen 60 seen from a virtual camera 54 set by the guest user.
  • the host user device 20H and the guest user device 20G broadcast within the same virtual space 50, but the display range of each virtual space screen 60 may differ.
  • each broadcasting user 10A may output a virtual image of the same display range.
  • the second output control units 24 of the host user device 20H and the guest user device 20G each control the display of the browser screen 57 based on the input operation of the broadcast user 10A.
  • the input operation of the browser screen 57 here includes operations that control the display state of the browser screen 57, such as operations for scrolling vertically, scrolling horizontally, and selection operations, as in the first embodiment. Note that when multiple display objects 58 are located within the virtual space 50, the host user and the guest user may each operate the browser screen 57 displayed on a different display object 58.
  • FIG. 16 shows the state in which data is transmitted and received between a broadcasting user device 20A and a viewing user 20B. Note that the management server 12 that relays the transmission and reception of data between the user devices 20 is not shown. Also, video data, gift display requests, message data, position information of the virtual camera 54, etc. transmitted between the user devices 20 are not shown.
  • One or more of the viewing users 10B have the authority to perform input operations on the browser screen 57, while the remaining viewing users 10B do not have the authority to perform input operations.
  • Viewing users 10B who meet a certain condition have the authority to perform input operations.
  • the certain condition may be, for example, that the viewing user 10B is associated with the broadcasting user 10A in advance (following or being a friend) before watching the video, that the viewing user 10B has made a request to the broadcasting user 10A to display a gift object (tipping), or that the viewing user device 20B of the viewing user 10B who meets the certain condition accepts input operations on the browser screen 57.
  • the authority of distributor user 10A and viewing user 10B to operate browser screen 57, and reflection of the operated browser screen 57 on other devices are as follows.
  • the host user device 20H accepts input operations on the browser screen 57, and transmits data for reflecting the display state of the operated browser screen 57 on other devices.
  • the guest user device 20G accepts input operations on the browser screen 57, and transmits data for reflecting the display state of the operated browser screen 57 on other devices.
  • the viewing user device 20B of the authorized viewing user 10B accepts input operations on the browser screen 57, but does not transmit data for reflecting the display state of the operated browser screen 57 on another device.
  • the host user device 20H When the host user device 20H receives an input operation on the browser screen 57 of the host user, it transmits input operation data D1 including an input operation instruction, identification information of the display object 58 to be operated, and a URL to the guest user device 20G and the viewing user device 20B.
  • the input operation instruction includes a scroll direction and scroll amount.
  • the guest user device 20G and the viewing user device 20B Based on the input operation data D1, the guest user device 20G and the viewing user device 20B output the browser screen 57 to the display object 58 to be operated by the host user. If the input operation data D1 does not include a URL, the guest user device 20G and the viewing user device 20B obtain a URL associated with the display object 58 to be displayed from the storage 104 or the management server 12 of their own device.
  • the guest user device 20G and the viewing user device 20B scroll the browser screen 57.
  • the guest user device 20G and the viewing user device 20B control the display of the browser screen 57 in response to the input operation of the host user. This causes the browser screens 57 to be synchronized between the users 10.
  • the guest user device 20G accepts an input operation on the guest user's browser screen 57, it transmits input operation data D1 to the host user device 20H and the viewing user device 20B. Based on the input operation data D1, the host user device 20H and the viewing user device 20B output the browser screen 57 to the display object 58 that is the target of the guest user's operation.
  • the viewing user device 20B of the authorized viewing user 10B when the viewing user device 20B of the authorized viewing user 10B accepts an input operation on the viewing user's browser screen 57, it controls the display of the browser screen 57 based on the input operation. However, this viewing user device 20B does not transmit the input operation data D1 to the broadcasting user device 20A and other viewing user devices 20B. Furthermore, the viewing user device 20B of the authorized viewing user 10B accepts the input operation when it has not received the input operation data D1 from the broadcasting user device 20A. In other words, when this viewing user device 20B receives the input operation data D1 from the broadcasting user device 20A, it controls the browser screen 57 by prioritizing the received input operation data D1. When the viewing user 10B and the broadcasting user 10A operate the browser screens 57 output to different display objects 58, the display of each browser screen 57 is controlled by the input operation of the viewing user 10B and the input operation of the broadcasting user 10A, respectively.
  • the viewing user device 20B of the unauthorized viewing user 10B does not accept input operations on the viewing user's browser screen 57.
  • This viewing user device 20B makes a resource transmission request D2 to the web server 13 based on the input operation data of each distribution user device 20A, and receives D3 the resources in response to the transmission request.
  • the input operation that is received first may be given priority. That is, for example, if the host user device 20H receives an input operation by the host user immediately after receiving input operation data D1 transmitted from the guest user device 20G, the input operation by the guest user is given priority. If the host user device 20H receives input operation data transmitted from the guest user device 20G immediately after receiving an input operation by the host user, the input operation by the host user is given priority. The same applies to the guest user device 20G.
  • the other distribution user 10A and viewing user 10B can also perform input operations.
  • a browser screen 57 corresponding to each input operation is output to each display object 58.
  • FIG. 17 is an example of a virtual space screen 60 output to the display 109 of the host user device 20H in collaboration distribution.
  • the virtual space screen 60 includes an avatar object 52A corresponding to the host user. If an avatar object 52B of a guest user is included in the field of view of the virtual camera 54, the avatar object 52B is drawn.
  • the virtual space screen 60 also includes a mini-avatar 52C, which is a simplified avatar object 52 corresponding to an authorized viewing user 10B.
  • the mini-avatar 52C has a smaller processing load for drawing than the avatar object 52 of the distribution user 10A.
  • the mini-avatar 52C is a two-dimensional image.
  • the authorized viewing user 10B can also perform input operations to control the browser screen 57. This makes it possible to provide the display contents of the browser screen 57 to a larger number of users, thereby improving the advertising effect.
  • the first output control unit 23 of the user device 20 distributes and views the video using the client rendering method.
  • the video may be distributed and viewed using any of the above methods other than the client rendering method.
  • the browser rendering method is a method in which the broadcasting user device 20A and the viewing user device 20B display the video using independent web browsers stored in the storage 104. At this time, the broadcasting user device 20A transmits video data, etc. to the management server 12.
  • the user device 20 that displays the video receives web data from the management server 12 that is written in a markup language such as HTML (Hyper Text Markup Language) and uses CSS (Cascading Style Sheets), JavaScript (registered trademark), etc.
  • the web browser program started on the user device 20 draws avatar objects and other objects using JavaScript running on the browser. In this method, model data of the avatar object, etc. is not stored in the user device 20.
  • an application does not need to be installed on the user device 20.
  • an application including an in-app browser may be installed on the user device 20 .
  • At least one of the well-known methods such as static rendering (Static Server-Side Rendering), server-side rendering (Server-Side Rendering), client-side rendering (Client-Side Rendering), pre-rendering (CSR with Prerendering), and hydration rendering (SSR with (Re)Hydration) can be used as a method for generating a file to display the virtual space screen 60 by rendering.
  • Rendering here refers to interpreting data written in a markup language or the like and calculating the arrangement of image elements and characters to be displayed on the virtual space screen 60.
  • the web server 13 stores a static HTML file in advance.
  • the user device 20 downloads the static HTML file from the web server 13.
  • the user device 20 outputs the rendered HTML file to the display 109 to display the virtual space screen 60.
  • server-side rendering SSR
  • the web server 13 performs rendering etc. in response to a request from the user device 20, and transmits the rendered HTML file to the user device 20.
  • the user device 20 outputs the downloaded HTML file to the display 109 to display the virtual space screen 60.
  • the web server 13 includes an application server.
  • client-side rendering CSR
  • the user device 20 performs rendering by executing JavaScript (registered trademark).
  • the user device 20 downloads a simple HTML file from the web server 13, and also downloads the necessary JavaScript (registered trademark) and CSS files, etc.
  • pre-rendering a type of CSR
  • the user device 20 generates and records some HTML files in advance, and downloads and renders the necessary content from the web server 13 in response to a request.
  • hydration rendering the web server 13 generates and records HTML files that have not been rendered.
  • the user device 20 downloads HTML files from the web server 13, and also downloads and renders the necessary data based on the HTML files.
  • the user device 20 may obtain the necessary data from the management server 12, rather than from the web server 13.
  • the video distribution method is a method in which the broadcasting user device 20A generates video data. Specifically, the broadcasting user device 20A uses motion data and the like to draw objects and generate video data. The broadcasting user device 20A also transmits the encoded video data and audio data to the management server 12. The management server 12 transmits the video data and audio data to the viewing user device 20B. The viewing user device 20B displays video on the display 109 based on the video data received from the broadcasting user device 20A.
  • the server distribution method is a method in which the management server 12 generates video data.
  • the management server 12 receives motion data and audio data from the broadcasting user device 20A.
  • the management server 12 then generates output data for outputting the video based on that data.
  • the management server 12 transmits the generated output data to the broadcasting user device 20A and the viewing user device 20B.
  • the broadcasting user device 20A and the viewing user device 20B output the video to the display 109 based on the received output data.
  • the user device 20 or the management server 12 is responsible for generating data for outputting a video.
  • the data to be generated differs depending on the distribution method.
  • the device responsible for generating the data can be changed depending on the distribution method.
  • the virtual space 50 in which an exhibition is held is exemplified, but the present invention is not limited to this virtual space 50.
  • the virtual space 50 may be one in which a projection screen is displayed, and may be one in which events such as live performances and matches are held.
  • the virtual space 50 may be a virtual store or an entertainment facility such as an amusement park.
  • the virtual space 50 may not hold any events, and may simply allow users 10 to explore or communicate with each other. There may not be any objects corresponding to exhibits in the virtual space 50.
  • the first display control process is a process of controlling the position of the virtual camera 54.
  • the first display control process may be a process of controlling the direction of the virtual camera 54 without changing its position. For example, when the operation direction is the X-positive direction, the orientation of the virtual camera 54 is moved in the X-positive direction while maintaining the position of the virtual camera 54.
  • the first display control process may also be a process of moving an object 51, such as the avatar object 52, in a direction indicated by the user 10.
  • the first output control unit 23 outputs the browser screen 57 as the projection screen.
  • the projection screen may be a screen on which information recorded in the storage 104 of the user device 20 or information acquired from an external server is output. This information is, for example, image data including still images and videos, an image list, a document file, etc. Or, this information is video data acquired from an external server and playable on the user device 20, call data for conversation through a video of another user, etc. Or, it may be the top page of an application that plays videos.
  • image data such as photos or videos recorded by the album function of the storage 104, books such as comics, data created by presentation software such as "Microsoft PowerPoint (registered trademark)", etc.
  • the user 10 can vertically scroll, horizontally scroll, transition between screens, and play or stop videos on this projection screen as well, in the same way as the browser screen 57.
  • the second output control unit 24 scrolls the display area 97 of the browser screen 57 horizontally. Furthermore, when the operation determination unit 22 determines that the input operation does not include a designation operation and includes only a horizontal sliding operation, the second output control unit 24 controls the virtual camera 54. The second output control unit 24 may perform the opposite process depending on these input operations. In other words, when the input operation includes a designation operation and a horizontal sliding operation, the second output control unit 24 may control the virtual camera 54, and when the input operation does not include a designation operation and includes only a horizontal sliding operation, the second output control unit 24 may perform display control of the browser screen 57.
  • the second output control unit 24 may determine that the input operation during the period between the input of the lock operation and the input of the unlock operation is an operation on the browser screen 57. For example, the second output control unit 24 determines that all coordinates input between a long press operation on the input device 110 and a long press operation for unlocking are operations on the browser screen 57. The second output control unit 24 performs processing on the browser screen 57 based on the input operation during the period from the lock operation to the input of the unlock operation for unlocking the lock operation. Therefore, the user 10 can scroll the display area 97 of the browser screen 57 horizontally by simply performing a horizontal slide operation in the browser area 56, without performing a designation operation for designating horizontal scrolling of the browser screen 57.
  • the second output control unit 24 synchronizes the projection screens such as the browser screen 57 by transmitting input operation data to the other user device 20.
  • the second output control unit 24 may transmit data (image data) capturing the browser screen 57 to the other user device 20.
  • the user device 20 that receives this data projects the received data onto the display object 58.
  • the second output control unit 24 performs, as the second display process, vertical scrolling, horizontal scrolling, link selection, and video playback or stopping of the browser screen 57 in response to an input operation of the user 10.
  • the second output control unit 24 may perform at least one of these processes.
  • the second output control unit 24 may perform a process other than the above-mentioned processes as long as the process is a process for a projection screen including the browser screen 57.
  • the second output control unit 24 may close or open the browser screen 57 in response to an input operation of the user 10.
  • the second output control unit 24 may also operate a GUI element, such as selecting one of the items displayed in a pull-down menu, in response to an input operation of the user 10.
  • the input coordinate acquisition unit 21 acquires coordinates where a touch of a finger or a stylus is detected on a touch panel that is the input device 110.
  • the input coordinate acquisition unit 21 acquires input coordinates according to the input device 110.
  • the input coordinate acquisition unit 21 acquires coordinates on the screen that the user 10 specifies using the mouse or controller.
  • the input coordinate acquisition unit 21 detects the direction of the user 10's gaze and detects the position on the screen that the user 10 specifies.
  • the input coordinate acquisition unit 21 detects the direction in which the user 10 points the finger and detects the position on the screen that the user 10 specifies.
  • the operation determination unit 22 determines whether or not the input operation is an operation on the browser screen 57.
  • the second output control unit 24 may determine whether or not the input operation is an operation on the browser screen 57 based on the input coordinates acquired from the operation determination unit 22.
  • the management server 12 may be a single server device equipped with multiple virtual servers, or may be a set of multiple physical servers.
  • the management server 12 may be a set of a combination of a server having a distribution management unit 30, a server having a browsing history information receiving unit 32, and a server having a performance recording unit 33.
  • the browsing history information receiving unit 32 may receive browsing information from the web server 13.
  • the web server 13 records an access log of a resource specified by a URL that the user 10 can register.
  • the access log includes the connection date and time (browsing start time), the disconnection date and time (browsing end time), the identification information of the connected user device 20, the operation content, etc.
  • the web server 13 transmits the access log to the management server 12.
  • the browsing history information receiving unit 32 of the management server 12 records the received access log in the browsing tally information storage unit 36 as browsing tally information 42.
  • the performance recording unit 33 records performance information 43 based on the access log in the performance information storage unit 37.
  • the distribution user 10A can operate the projection screen.
  • the distribution user 10A including the host user and the guest user and the viewing user 10B having authority can operate the projection screen, and only the operation of the distribution user 10A is reflected (synchronized) in the video of the other users 10.
  • the attributes of the user 10 who can operate the projection screen and the attributes of the user whose input operation is synchronized with the other user devices 20 can be changed as appropriate. In other words, for each attribute of each user 10, whether or not the browser screen 57 can be operated is set, and if the browser screen 57 can be operated, whether or not the browser screen 57 can be synchronized with the other devices is set. Note that the browser screen 57 may be synchronized with all other user devices 20 other than the own device, or the browser screen 57 may be synchronized with only some of the other user devices 20.
  • the attributes of these users 10 may be in the following forms: [First aspect] (A) The host user device 20H accepts an input operation on the browser screen 57, and transmits data for reflecting the operated browser screen 57 on another device. (B) The guest user device 20G and the viewing user device 20B of the authorized viewing user 10B accept input operations on the browser screen 57, but do not transmit data for reflecting the operated browser screen 57 on another device. [Second aspect] (C) The host user device 20H accepts an input operation on the browser screen 57 and transmits data for reflecting the operated browser screen 57 on another device. (D) The guest user device 20G and the viewing user device 20B of an authorized viewing user 10B accept input operations on the browser screen 57.
  • a synchronization instruction which is an operation different from the input operation on the browser screen 57
  • the user device 20 transmits data for reflecting the browser screen 57 on another device.
  • the user device 20 does not transmit data for reflecting the browser screen 57 on another device unless a synchronization instruction is input.
  • the synchronization instruction is input by operating a synchronization button or the like displayed on the virtual space screen 60.
  • the data to be reflected in the other device is data generated by the user device 20 or the management server 12, and it is sufficient that the user device 20 receiving this data can determine the operation contents of the user 10, who is the operator of the resource and the browser screen 57.
  • the data to be reflected in the other device may include a URL that identifies the resource.
  • the data to be reflected in the other device may include data indicating the operation contents of the user 10, which is the first user device 20 used by the user 10, who is the operator.
  • the first user device 20 Upon detecting the operation of the user 10, transmits data indicating the operation contents to the management server 12.
  • the management server 12 generates an identifier as needed based on the data indicating the operation contents.
  • the management server 12 transmits the identifier to the other device.
  • the second user device 20, which is the other device that receives the identifier determines the operation contents based on the identifier and displays the browser screen 57 according to the operation contents.
  • the input operation of the host user takes priority.
  • the user device 20 may take priority over the input operation of the browser screen 57 of the user 10 of the own device.
  • all users 10 may operate the projected screens, and synchronization of the projected screens may not be performed.
  • the browser screen 57 that the user 10 has started (displayed) or the browser screen 57 that the user 10 has operated once may be operated by another user 10. That is, in one aspect, only the host user may be able to both operate the projected screen and synchronize the operation, and the user 10 other than the host user may be able to operate the projected screen. For example, after the first user 10 starts the browser screen 57, if a predetermined condition is satisfied, the second user 10 may be able to operate the browser screen 57.
  • the user device 20 of the second user 10 who has performed the operation halfway through transmits data for synchronizing the browser screen 57 to the other device if the second user 10 has the authority to synchronize the browser screen 57. If the second user 10 does not have the authority to synchronize the browser screen 57, the user device 20 of the second user 10 does not transmit data for synchronizing the browser screen 57 to the other device.
  • the predetermined condition is that the host user is not performing the operation. Or, the predetermined condition is that the user device 20 has accepted the operation of the user 10.
  • the user device 20 draws the avatar object 52 corresponding to the broadcast user 10A using tracking data or emote data that detects the movement of the broadcast user 10A. This allows the movement of the broadcast user 10A or the movement selected by the broadcast user 10A to be reflected in the avatar object 52. Alternatively or in addition to this, the user device 20 may draw the avatar object 52 temporarily or continuously without using tracking data and emote data.
  • the first output control unit 23 controls the avatar object 52 based on the input operation of the user 10.
  • the avatar object 52 performs an action in response to the input operation of the user 10. In this aspect, the load of the drawing process on the user device 20 can be reduced when drawing a large number of avatar objects 52 in the virtual space 50.
  • the user device 20 renders an avatar object 52 corresponding to the broadcast user 10A.
  • the user device 20 may render an image of the virtual space 50 from a first-person perspective, and may not need to render the avatar object 52.
  • the broadcast user device 20A broadcasts videos.
  • the application implemented by the user device 20 may be an application that allows the user 10 to move around the virtual space 50 without broadcasting videos.
  • the management server 12 does not perform processing for broadcasting videos, but provides asset data of the virtual space 50, etc., in response to a request from the user device 20.
  • the user device 20 outputs a virtual space screen 60.
  • an avatar object 52 corresponding to another user 10 is located in the same virtual space 50, the avatar object 52 may be displayed on the virtual space screen 60.
  • the user 10 launches an application and selects a desired virtual space 50.
  • the user device 20 displays a virtual space screen of the virtual space 50 as seen from each virtual camera 54.
  • the user 10 can operate the projection screen in the same manner as the broadcast user 10A in the first embodiment.
  • the broadcasting user 10A refers to a user who transmits at least one of information related to images (video) and information related to audio.
  • the broadcasting user 10A is a user who broadcasts videos alone (first embodiment), or a user who performs collaboration broadcasting in which multiple people can participate (second embodiment).
  • the broadcasting user 10A may be a user who hosts or organizes a video chat or voice chat in which multiple people can participate and/or watch, or an event (party, etc.) in a virtual space in which multiple people can participate and/or watch.
  • the broadcasting user 10A can also be called a host user, organizer user, or organizer user.
  • viewing user 10B refers to a user who receives at least one of video-related information and audio-related information.
  • viewing user 10B not only receives the above information, but can also react to the information.
  • viewing user 10B is a user who watches a video broadcast or collaboration broadcast.
  • viewing user 10B may be a user who participates in and/or watches video chat, voice chat, or an event. Therefore, viewing user 10B can also be referred to as a guest user, participating user, listener, viewing user, supporting user, etc.
  • the management server 12 is managed by the PF provider, and the web server 13 is managed by the publisher.
  • the user 10 may display an arbitrary web page on the browser screen 57, instead of a resource previously associated with the virtual space 50 as in each of the above embodiments.
  • the user 10 may input a search word or the like in a predetermined area and execute a search, thereby outputting an arbitrary browser screen 57.
  • the predetermined area is, for example, a text box displayed in the UI area 55.
  • the user device 20 is an information processing device such as a smartphone, a mobile phone, a tablet terminal, a personal computer, a console game machine, or a wearable computer such as a head-mounted display.
  • the information processing system may be a system provided in a studio for video distribution.
  • the information processing system has an information processing device, a server, a sensor unit worn on the body of the distribution user 10A, a tracking system that detects the position of the sensor unit, an operation unit, a speaker, a display, etc.
  • the tracking system may be equipped with a multi-axis laser emitter that emits pulsed laser light for synchronization.
  • the sensor unit has a sensor that detects laser light, and detects its own position and orientation while synchronizing with a synchronization pulse.
  • the user device 20 functions as an input coordinate acquisition unit 21, an operation determination unit 22, a first output control unit 23, and a second output control unit 24. At least one of these functions may be executed by the management server 12 or another device in the information processing system 11. Furthermore, the management server 12 functions as a distribution management unit 30, a browsing history information receiving unit 32, and a performance recording unit 33 by executing a program. At least one of these functions may be executed by the user device 20 or another device in the information processing system 11. Furthermore, the management server 12 includes a user management information storage unit 34, a browsing aggregation information storage unit 36, and a performance information storage unit 37. At least one of these functions may be included in the user device 20 or another device in the information processing system 11.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Software Systems (AREA)
  • User Interface Of Digital Computer (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Processing Or Creating Images (AREA)

Abstract

La présente invention fournit une pluralité d'informations à un utilisateur dans un espace virtuel sans dégrader la facilité d'utilisation. Un dispositif utilisateur est amené à fonctionner grâce à : une première unité de commande de sortie qui délivre, à un dispositif d'affichage reconnu visuellement par un utilisateur, une image d'espace virtuel d'un espace virtuel tridimensionnel vue depuis une caméra virtuelle ; une seconde unité de commande de sortie qui délivre, à l'intérieur de l'image d'espace virtuel, un écran de projection qui peut être actionné par un utilisateur ; une unité d'acquisition de coordonnées d'entrée qui acquiert des coordonnées d'entrée sur un dispositif d'entrée de l'utilisateur ; et une unité de détermination d'opération qui, sur la base des coordonnées d'entrée acquises, détermine si l'opération d'entrée de l'utilisateur est une opération pour une région d'entrée correspondant à l'écran de projection, la première unité de commande de sortie effectuant un premier traitement de commande d'affichage relatif à l'espace virtuel tridimensionnel autre que la commande sur l'écran de projection lorsque l'unité de détermination d'opération détermine que l'opération d'entrée n'est pas pour la région d'entrée, et la seconde unité de commande de sortie effectuant un second traitement de commande d'affichage sur l'écran de projection en fonction des coordonnées d'entrée lorsque l'unité de détermination d'opération détermine que l'opération d'entrée est pour la région d'entrée.
PCT/JP2023/045378 2023-06-08 2023-12-18 Programme, procédé de traitement d'informations, et système de traitement d'informations Ceased WO2024252695A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2024575686A JP7787533B2 (ja) 2023-06-08 2023-12-18 プログラム、情報処理方法及び情報処理システム
JP2025203382A JP2026026164A (ja) 2023-06-08 2025-11-26 プログラム、情報処理方法及び情報処理システム

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2023-094517 2023-06-08
JP2023094517 2023-06-08

Publications (1)

Publication Number Publication Date
WO2024252695A1 true WO2024252695A1 (fr) 2024-12-12

Family

ID=93795771

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/045378 Ceased WO2024252695A1 (fr) 2023-06-08 2023-12-18 Programme, procédé de traitement d'informations, et système de traitement d'informations

Country Status (2)

Country Link
JP (2) JP7787533B2 (fr)
WO (1) WO2024252695A1 (fr)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH10154244A (ja) * 1996-09-30 1998-06-09 Sony Corp ３次元仮想現実空間表示処理装置、３次元仮想現実空間表示処理方法および情報提供媒体
JP2012212340A (ja) * 2011-03-31 2012-11-01 Sony Corp 情報処理装置、画像表示装置、および情報処理方法
US20140033078A1 (en) * 2010-03-10 2014-01-30 Xmobb, Inc. Instantiating media into a virtual social venue to share with friends
JP2021135582A (ja) * 2020-02-25 2021-09-13 株式会社コロプラ プログラム、情報処理方法、及び情報処理装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH10154244A (ja) * 1996-09-30 1998-06-09 Sony Corp ３次元仮想現実空間表示処理装置、３次元仮想現実空間表示処理方法および情報提供媒体
US20140033078A1 (en) * 2010-03-10 2014-01-30 Xmobb, Inc. Instantiating media into a virtual social venue to share with friends
JP2012212340A (ja) * 2011-03-31 2012-11-01 Sony Corp 情報処理装置、画像表示装置、および情報処理方法
JP2021135582A (ja) * 2020-02-25 2021-09-13 株式会社コロプラ プログラム、情報処理方法、及び情報処理装置

Also Published As

Publication number Publication date
JPWO2024252695A1 (fr) 2024-12-12
JP2026026164A (ja) 2026-02-16
JP7787533B2 (ja) 2025-12-17

Similar Documents

Publication Publication Date Title
US11402964B1 (en) Integrating artificial reality and other computing devices
US11460970B2 (en) Meeting space collaboration in augmented reality computing environments
US11050977B2 (en) Immersive interactive remote participation in live entertainment
KR102735623B1 (ko) 인터넷 전화 기반 통화 중 리액션을 표시하는 방법, 시스템, 및 컴퓨터 프로그램
JP2020091504A (ja) 仮想空間中のアバター表示システム、仮想空間中のアバター表示方法、コンピュータプログラム
US12289561B2 (en) Parallel video call and artificial reality spaces
JP7847781B2 (ja) プログラム、情報処理方法、及び情報処理装置
JP6883140B1 (ja) 情報処理システム、情報処理方法およびコンピュータプログラム
JP7145266B2 (ja) 情報処理システム、情報処理方法およびコンピュータプログラム
US12200320B2 (en) Moving image distribution system, moving image distribution method, and moving image distribution program
US12567203B2 (en) Information processing system, information processing method, and information processing program
JP7199791B2 (ja) 情報処理システム、情報処理方法およびコンピュータプログラム
JP7244450B2 (ja) コンピュータプログラム、サーバ装置、端末装置、及び方法
JP7787533B2 (ja) プログラム、情報処理方法及び情報処理システム
JP7281012B1 (ja) プログラム、情報処理方法及び情報処理システム
CN114984574B (zh) 信息显示方法、装置、存储介质及设备
JP7550411B1 (ja) プログラム、情報処理方法、サーバ、サーバの情報処理方法及び情報処理システム
US12586304B2 (en) Program, information processing method, and information processing system
JP7804290B2 (ja) 情報処理システム、情報処理方法及びプログラム
HK40073678B (zh) 信息显示方法、装置、存储介质及设备

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 2024575686

Country of ref document: JP

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23940789

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE