WO2019245003A1 - Système de jeu, procédé de commande de système de jeu, et programme de jeu - Google Patents

Système de jeu, procédé de commande de système de jeu, et programme de jeu Download PDF

Info

Publication number
WO2019245003A1
WO2019245003A1 PCT/JP2019/024603 JP2019024603W WO2019245003A1 WO 2019245003 A1 WO2019245003 A1 WO 2019245003A1 JP 2019024603 W JP2019024603 W JP 2019024603W WO 2019245003 A1 WO2019245003 A1 WO 2019245003A1
Authority
WO
WIPO (PCT)
Prior art keywords
game
unit
input
story
voice
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2019/024603
Other languages
English (en)
Japanese (ja)
Inventor
崇 片山
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mintflag Inc
Original Assignee
Mintflag Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mintflag Inc filed Critical Mintflag Inc
Priority to JP2020525808A priority Critical patent/JP7330518B2/ja
Publication of WO2019245003A1 publication Critical patent/WO2019245003A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
    • A63F13/214—Input arrangements for video game devices characterised by their sensors, purposes or types for locating contacts on a surface, e.g. floor mats or touch pads
    • A63F13/2145—Input arrangements for video game devices characterised by their sensors, purposes or types for locating contacts on a surface, e.g. floor mats or touch pads the surface being also a display device, e.g. touch screens
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
    • A63F13/215—Input arrangements for video game devices characterised by their sensors, purposes or types comprising means for detecting acoustic signals, e.g. using a microphone
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/40—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment
    • A63F13/42—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
    • A63F13/424—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle involving acoustic input signals, e.g. by using the results of pitch or rhythm extraction or voice recognition
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/45—Controlling the progress of the video game
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/70—Game security or game management aspects
    • A63F13/79—Game security or game management aspects involving player-related data, e.g. identities, accounts, preferences or play histories
    • A63F13/798—Game security or game management aspects involving player-related data, e.g. identities, accounts, preferences or play histories for assessing skills or for ranking players, e.g. for generating a hall of fame
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B19/00—Teaching not covered by other main groups of this subclass
    • G09B19/06—Foreign languages
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B5/00—Electrically-operated educational appliances
    • G09B5/04—Electrically-operated educational appliances with audible presentation of the material to be studied
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition

Definitions

  • the present invention relates to a game system for recognizing input voice, a control method for the game system, and a game program.
  • the present invention has been made in order to solve the above-described problem, and it is possible to enhance training for vocalizing in a natural atmosphere and timing in real life while enhancing entertainment by incorporating vocal evaluation in language learning into a game. It is an object to provide a game system, a game system control method, and a game program.
  • the present invention is a game system that advances a game in accordance with an operation input and a voice input by a player, and a voice input unit that inputs a voice signal.
  • a voice recognition unit for recognizing a player's voice by comparing the input voice signal with predetermined voice recognition dictionary data and generating a control signal related to the game based on the voice recognition result;
  • a scenario control unit for sequentially proceeding the game according to a scenario describing an event process to be proceeded; a game control unit and a scenario control unit for proceeding the scenario based on an input signal from an operation input and a control signal generated by a voice recognition unit
  • a game mode for evaluating the accuracy of pronunciation by a player based on a control signal generated by a speech recognition unit in accordance with the control by the player.
  • a game mode execution unit that executes a game as one of the event processes, and a story mode execution unit that executes a story mode in which a story related to the action of the character proceeds according to the control of the scenario control unit as one of the event processes.
  • the game mode execution unit detects the accuracy of pronunciation by the player for the question to be recognized, and evaluates the points by level or level up according to the detected accuracy. It has a function of executing a reaction event process relating to the action of the character in response to the voice signal evaluated by the mode execution unit being input by the player.
  • the present invention is a game system for advancing a game in response to an operation input and a voice input by a player, comprising: a voice input unit for inputting a voice signal;
  • the game is sequentially recognized in accordance with a scenario that describes a player's voice by comparing it with dictionary data and generates a control signal for the game based on the recognition result of the voice, and a scenario describing event processing that advances the story.
  • An audio dubbing data generating unit that acquires audio input in synchronization with the video reproduction of the dubbing video obtained from the story mode video played back from the story progressing in the execution unit and generates the dubbing data.
  • the game mode execution unit detects the accuracy of pronunciation by the player with respect to the voice recognition target problem set using the post-recording video or the voice input in synchronization with the post-recording video or the post-recording data, Evaluation based on points or level-up according to the detected accuracy And having a function of performing.
  • the present invention is a control method for a game system that advances a game in response to an operation input and a voice input by a player, (1)
  • the voice input unit receives the input of the voice signal, and the voice recognition unit recognizes the voice of the player by comparing the input voice signal with predetermined voice recognition dictionary data, and recognizes the voice of the player.
  • a voice recognition step of generating a control signal for the game based on the recognition result (2) a game control step in which, based on an input signal from the operation input and a control signal generated by the voice recognition unit, the scenario control unit sequentially advances the game in accordance with a scenario describing event processing for advancing the story; (3) A game in which the game mode execution unit executes, as one of the event processes, a game mode in which the accuracy of pronunciation by a player is evaluated based on a control signal generated by the voice recognition unit in accordance with control by the scenario control unit.
  • a mode execution step (4) a story mode in which a story relating to the action of the character proceeds in accordance with the control by the scenario control unit, and a story mode execution step executed by the story mode execution unit as one of the event processes; (5)
  • the accuracy of pronunciation by the player with respect to the question on which the speech recognition target is set is detected, and evaluation is performed by a point or a level-up according to the detected accuracy.
  • a reaction event process related to the action of the character is executed in response to the voice signal evaluated by the game mode execution unit being input by the player.
  • the present invention is a control method of a game system for advancing a game in response to an operation input and a voice input by a player,
  • a voice input unit receives an input of a voice signal
  • a voice recognition unit recognizes a player's voice by comparing the input voice signal with predetermined dictionary data for voice recognition, and Voice recognition step of generating a control signal for the game based on the recognition result of
  • a game control step in which, based on an input signal from an operation input and a control signal generated by the voice recognition unit, the scenario control unit causes the game to sequentially progress according to a scenario describing event processing for advancing the story.
  • a game mode in which the game mode execution unit executes a game mode for evaluating the accuracy of pronunciation by the player based on a control signal generated by the voice recognition unit as one of the event processes.
  • Steps and (4) a story mode execution step in which the story mode execution section executes a story mode in which a story related to the action of the character progresses as one of the event processes according to control by the scenario control section;
  • the audio dubbing data generation unit acquires the audio input in synchronization with the video reproduction of the audio dubbing video obtained from the video for the story mode reproduced from the story progressed by the story mode execution unit, and performs the audio dubbing.
  • the post-record data generation step of generating data and (6) the game mode execution step in the post-record video, a voice input in synchronization with the post-record video, or a voice recognition target question set using the post-record data, It is characterized in that the accuracy of pronunciation is detected, and the evaluation is performed by a point or a level-up according to the detected accuracy.
  • the story mode two or more languages are used, and in the game mode, the accuracy of the speech recognition target problem relating to the two or more languages is evaluated.
  • the character is operated according to the points or the level-up acquired in the game mode.
  • two or more languages are used in the story mode, and a speech recognition target problem relating to the two or more languages is presented in the game mode.
  • the character in the event processing that is generated in response to the input of the audio signal by the player in the story mode, it is preferable that the character be operated according to the points or the level-up acquired in the game mode.
  • the above-described system and method according to the present invention can be realized by executing a program of the present invention described in a predetermined language on a computer. That is, the program of the present invention is installed on a mobile terminal device, a smartphone, a wearable terminal, a mobile PC or other information processing terminal, an IC chip of a general-purpose computer such as a personal computer or a server computer, or a memory device, and executed on a CPU. Accordingly, it is possible to construct a system having the above-described functions and implement the method according to the present invention.
  • the program of the present invention can be distributed, for example, through a communication line, and can be transferred as a package application that operates on a stand-alone computer by recording it on a computer-readable recording medium.
  • the recording medium can be recorded on various recording media such as a magnetic recording medium such as a flexible disk and a cassette tape, an optical disk such as a CD-ROM and a DVD-ROM, and a RAM card.
  • the above-described system and method can be easily implemented using a general-purpose computer or a special-purpose computer. Installation can be performed easily.
  • FIG. 1 is a functional block diagram illustrating an overall configuration of a game system according to an embodiment.
  • FIG. 3 is a block diagram showing an internal configuration of the game server according to the embodiment.
  • FIG. 2 is a block diagram illustrating an internal configuration of a user terminal according to the embodiment.
  • FIG. 7 is a sequence diagram illustrating an operation of the game system according to the embodiment. It is an explanatory view showing a screen configuration of a theater space in the game system according to the embodiment. It is an explanatory view showing a screen configuration of a game mode in the game system according to the embodiment. It is an explanatory view showing a screen configuration in a story mode in the game system according to the embodiment.
  • FIG. 7 is a flowchart illustrating an operation in a post-recording mode according to the embodiment.
  • FIG. 4 is an explanatory diagram showing a screen configuration in a post-recording mode in the game system according to the embodiment.
  • FIG. 1 is a conceptual diagram showing the overall configuration of the game system according to the present embodiment.
  • a game system according to the present embodiment includes a smartphone 1 (1a, 1b) used by a plurality of players U (Ua, Ub) as players, and a game installed on the Internet 2. And a server 3.
  • the smartphone 1 will be described as an example of a mobile terminal device.
  • the game server 3 is a server that performs game progress processing, and can be realized by a single server device or a plurality of server device groups, and virtually constructs a plurality of functional modules on a CPU. Then, the respective function modules cooperate to execute the processing.
  • the game server 3 can transmit and receive data through the Internet 2 by a communication function, and can present a Web page through browser software by a Web server function.
  • the smartphone 1 is a portable information processing terminal device using wireless communication.
  • the mobile phone can wirelessly communicate with a relay point such as the wireless base station 22 and receive communication services such as telephone calls and data communication while moving. Can be.
  • a communication method of the mobile phone for example, a 3G (3rd. Generation) method, an LTE (Long Term Evolution) method, a 4G method, an FDMA method, a TDMA method, a CDMA method, a W-CDMA, and a PHS (Personal Handyphone System) ) System.
  • the smartphone 1 is equipped with various functions such as a digital camera function, an application software execution function, and a position information acquisition function using a GPS (Global Positioning System) in addition to a communication function. included.
  • GPS Global Positioning System
  • the position information acquisition function is a function of acquiring and recording position information indicating the position of the own device.
  • the position information acquisition function includes, for example, detecting the position of the own device based on a signal from a satellite, such as a GPS. 1, and a method of detecting a position based on radio wave intensity from a wireless base station 22 of a mobile phone or an access point of Wi-Fi communication as shown in FIG.
  • the smartphone 1 includes a liquid crystal display as a display unit for displaying information and an operation device such as an operation button for a user to perform an input operation.
  • the operation device is superimposed on the liquid crystal display.
  • a touch panel is provided as an input unit that is arranged and acquires an operation signal by a touch operation or the like that specifies a coordinate position on the liquid crystal display.
  • this touch panel is an input device that inputs operation signals by detecting pressure, static electricity, etc. by touch operation using a user's fingertip or pen, etc., and a liquid crystal display that displays graphics and a liquid crystal display that is displayed on this liquid crystal display And a touch sensor that receives an operation signal corresponding to the coordinate position of the graphic.
  • FIG. 2 is a block diagram illustrating an internal configuration of the game server 3 according to the present embodiment
  • FIG. 3 is a block diagram illustrating an internal configuration of the smartphone 1 according to the present embodiment.
  • the “module” used in the description is a functional unit for achieving a predetermined operation, which is configured by hardware such as an apparatus or a device, software having the function thereof, or a combination thereof. .
  • the game server 3 is a server device arranged on the Internet 2, and can transmit and receive data to and from each smartphone 1 via the Internet 2.
  • the game server 3 includes a communication interface 31 for performing data communication through the Internet 2, an authentication unit 33 for authenticating the authority of a user or a user terminal, an access management unit 32 for managing access from each user terminal, A game progress processing unit 36 that executes the entire game progress process and a game progress process of each user, a voice learning management unit 37 that manages voice learning of each user, and a game that distributes game data to each user.
  • a data distribution unit 34 and various database groups 35 (35a to 35c) are provided.
  • the database group 35 includes a voice dictionary database 35a that stores a learning game executed during the game, a voice dictionary file used for voice recognition performed in a story movie, and a user database 35b that stores information about users. And a game database 35c for storing scenario data and other data as information on the game progress processing of the entire game and the game progress processing of each user.
  • Each of these databases may be a single database, or may be divided into a plurality of databases and a relational database in which each data is linked by setting a relation with each other.
  • the voice dictionary database 35a is a storage device that stores a learning game executed during the game or a voice dictionary file used for voice recognition performed in a story movie, and stores a foreign language (here, English) to be learned.
  • a dictionary of a native language here, Japanese
  • the English translation and the bilingual translation in English and Japanese are also set as questions for speech recognition. Therefore, the speech dictionary file is composed of two types, foreign language and native language. Accordingly, any one of the audio dictionary files may be omitted, and three or more types of audio dictionary files may be installed in correspondence with multiple languages.
  • the information stored in the user database 35b includes authentication information in which an identifier (user ID, terminal ID) for identifying a user or a portable terminal device used by the user, a password, and the like are included. It also includes personal information of the user associated with the ID, the model of the terminal device, and the like. Further, the user database 35b has an authentication history (access history) for each user or user terminal, a learning level for the game progress for each user by a relation with the game database 35c, a status during the game, a score, and usage. History, etc.), settlement information relating to the game being played, and the like.
  • an identifier user ID, terminal ID
  • the user database 35b has an authentication history (access history) for each user or user terminal, a learning level for the game progress for each user by a relation with the game database 35c, a status during the game, a score, and usage. History, etc.), settlement information relating to the game being played, and the like.
  • the information stored in the game database 35c includes game scenario data, characteristics of characters and objects, information on event processing, graphic information, and the like as game data.
  • the authentication unit 33 is a module that establishes a communication session with each smartphone 1 through the communication interface 31 and performs an authentication process for each established communication session.
  • the authentication information is acquired from the smartphone 1 of the user who is the access user, the user and the like are specified with reference to the user database 35b, and their authority is authenticated.
  • the authentication result (user ID, authentication time, session ID, etc.) by the authentication unit 33 is sent to the game progress processing unit 36, and is stored as an authentication history in the user database 35b through the access management unit 32.
  • the game progress processing unit 36 is a module that generates various event processes to develop the game mode and the story mode according to the scenario data and progresses the game, and executes a game program including a certain rule logic algorithm, In accordance with the positional relationship (approach, contact, etc.) between characters and objects, event processing such as battles, mini-games, and movie playback is generated.
  • the game progress processing unit 36 of the game server 3 cooperates with the game progress processing unit 141 of the smartphone 1, and a part of the game progress processing is performed by the game server 3. Then, a part of the graphic processing and the event processing is executed by the game progress processing unit 141 of the smartphone 1.
  • event processing that can occur is predicted based on the position of the user's character and the position of the object, and the generation conditions are determined. 3 and the conditions are transmitted to the smartphone 1, and the actual event processing and the graphic processing therefor are executed on the smartphone 1 based on the occurrence conditions received from the game server 3. .
  • the utterance learning management unit 37 is a module that presents questions for utterance to the user in accordance with the game progress by the game progress processing unit 36, and updates the evaluation to the user according to the accuracy of the utterance.
  • the utterance problem may be generated on the game server 3 side, generated on the smartphone 1 side, or generated in cooperation with both the game server 3 and the smartphone 1.
  • the game data distribution unit 34 Based on the learning level of the user, the game data distribution unit 34 synchronizes the generation problem generated by the utterance learning management unit 37 and the dictionary file for speech recognition with these files according to the control of the utterance learning management unit 37. , And a module for distributing to each user through the communication interface 31.
  • the smartphone 1 includes a communication interface 11, an input interface 12, an output interface 13, an application execution unit 14, and a memory 15.
  • the communication interface 11 is a communication interface for performing data communication, and has a function of performing non-contact communication such as wireless communication, and performing contact (wired) communication using a cable, an adapter, or the like.
  • the input interface 12 is a device for inputting a user operation such as a mouse, a keyboard, an operation button, and a touch panel 12a.
  • the output interface 13 is a device that outputs video and audio, such as a display and a speaker.
  • the output interface 13 includes a display unit 13a such as a liquid crystal display, and the display unit is superimposed on the touch panel 12a as an input interface.
  • the memory 15 is a storage device that stores an OS (Operating System), firmware, programs for various applications, other data, and the like.
  • the memory 15 has a user ID for identifying a user and a game server.
  • the memory 15 stores scenario data, movie data used in the story mode, questions for voice learning, dictionary files for recognition, and the like, obtained from the game server 3.
  • the application execution unit 14 is a module that executes an application such as a general OS, a game application, and browser software, and is generally realized by a CPU or the like.
  • the application execution unit 14 executes the game program according to the present invention, so that a game progress processing unit 141, a scenario control unit 142, an event control unit 143, a voice recognition processing unit 144, a display control unit 145, a display data generation unit
  • the unit 146 and the after-recording data control unit 147 are virtually constructed.
  • the game progress processing unit 141 executes, as one of the event processes, a game mode in which the accuracy of pronunciation by the player U is evaluated based on the control signal generated by the voice recognition processing unit 144, under the control of the scenario control unit 142.
  • a game mode execution unit In the present embodiment, the game progress processing unit 141 sequentially develops the story mode and the game mode by using the same rule logic algorithm as the game program executed on the game server 3 to generate various event processes.
  • the game has a function to advance the game.
  • the scenario progression, the learning level of the user, and the positional relationship between characters and objects (approach / contact) are synchronized with the game progress processing unit 36 of the game server 3 through the scenario control unit 142. ), Event processing such as battle game or movie playback is generated.
  • the game progress processing section 141 cooperates with the game progress processing section 141 of the game server 3, and performs a part of the game progress processing on the game server 3 side, and performs graphic processing and event processing.
  • a part of the processing is executed by the game progress processing unit 141 of the smartphone 1a.
  • an event occurrence condition or the like is generated on the game server 3 side, the condition is transmitted to the smartphone 1 side, and the actual event processing generation and graphic processing therefor are executed on the smartphone 1 side.
  • the game progress processing unit 141 includes a scenario control unit 142 and an event control unit 143.
  • the event control unit 143 includes a game mode execution unit 143a, a story mode execution unit 143b, and a post-record mode execution unit 143c.
  • the story mode execution unit 143b is a module that executes, as one of the event processes, a story mode in which a story relating to the action of the character proceeds under the control of the scenario control unit 142.
  • the game mode execution unit 143a executes This module detects the accuracy of pronunciation by the player for (problem to be recognized by speech) and evaluates the points based on the detected accuracy or a level-up.
  • a character's reaction operation corresponding to an event process generated in response to the input of a voice signal by the player U is displayed. 146a.
  • the post-recording mode execution unit 143c acquires a post-recording video from the story mode video reproduced with respect to the story progressed by the story mode execution unit 143b, and synchronizes the acquired post-recording video with the video reproduction of the obtained post-recording video.
  • the post-recording data generation unit 144b records the input voice data D3, analyzes the voice waveform thereof, and performs a voice changer function or the like as necessary.
  • the post-record data generation unit 144b combines the input audio data D3 subjected to the audio processing and the lip sync processing with the post-record data D2 to generate the post-recorded moving image D5.
  • the scenario control unit 142 is a module that advances a scenario based on an input signal from an operation input and a control signal generated by the voice recognition processing unit 144.
  • a scenario mode and a story mode are described.
  • the game server 3 has a function of synchronizing the game progress processing on the smartphone 1 side with the game progress processing on the game server 3 side.
  • event processing that can occur is predicted based on the degree of game progress in the scenario data, the positions of characters of other users, the positions of objects, and the like. 3 and sends the condition to the smartphone 1a.
  • the game progress processing unit 141 receives the occurrence condition transmitted from the server side by the scenario control unit 142, and executes the actual event processing and the graphic processing therefor based on the occurrence condition received from the game server 3. This is executed by the game progress processing section 141 on the first side.
  • the result of the event processing (the progress of the story, the outcome of the battle or the mini-game, the score, etc.) executed by the game progress processing unit 141 of the smartphone 1 is transmitted to the game progress processing unit 141 of the game server 3 through the scenario control unit 142. And is reflected in the subsequent game progress processing.
  • the voice recognition processing unit 144 recognizes the voice of the player U by comparing the voice signal input through the input interface 12 with predetermined voice recognition dictionary data, and controls the game based on the voice recognition result.
  • This is a module for generating a signal.
  • the module includes an evaluation unit 144a and an after-recording data generation unit 144b.
  • the evaluation unit 144a is a module that evaluates the accuracy of pronunciation by the player U based on the control signal generated by the voice recognition processing unit 144 under the control of the scenario control unit 142.
  • the evaluation result by the evaluation unit is sent to the game progress processing unit 141.
  • the post-record data generation unit 144b is a module that acquires audio input in synchronization with video reproduction of a post-record video that is reproduced with respect to the story progressed by the story mode execution unit 143b, and generates post-record data. .
  • the display data generation unit 146 is a module that generates display data to be displayed on the display unit 13a.
  • the display data is data generated by combining graphic data, image data, character data, moving image data, audio, and other data.
  • the display data generation unit 146 includes a reaction operation processing unit 146a.
  • the reaction operation processing unit 146a is a module that generates a moving image indicating the operation of the character of the user according to the game progress by the game progress processing unit 141. .
  • the display control unit 145 according to the present embodiment includes a display mode switching unit 145a.
  • the display mode switching unit 145a displays a graphic according to the display mode.
  • the post-record data control unit 147 is a module for managing and controlling various data used in the post-record mode. Specifically, the post-recording data control unit 147 manages and controls the post-recording video D5, and in the post-recording mode, reproduces the post-recording video D5 as an execution result of the post-recording mode. Also in the moving image reproduction processing, the post-recording moving image D5 is reproduced, or the post-recording moving image D5 is converted into the question moving image D41 or the question sound in the game mode, and is reproduced by the question processing in the game mode.
  • FIG. 4 is a sequence diagram showing the operation of the system. Note that the processing procedure described below is merely an example, and each processing may be changed as much as possible. In addition, in the processing procedure described below, steps can be omitted, replaced, and added as appropriate according to the embodiment.
  • a game application is activated on the smartphone 1 to start a learning game (S201).
  • the scenario data is referred to, the current learning level of the user and the degree of progress of the game are read out, and the processing is started from the ending point of the previous time.
  • the scenario data is referred to as the game progresses, the event occurrence condition is monitored (S202), and the process is repeated until the event occurrence condition is satisfied ("N" in S203).
  • the event process to be generated is a game mode, a story mode, or a post-recording mode (S204). These modes are determined based on the scenario data.
  • the story mode a battle is started by touching an object such as an enemy or a monster on the map, or a predetermined story branch point on the scenario data is reached. It is selected as appropriate when it is done.
  • game mode When the game mode is selected (“game mode” in S204), the game is switched to a battle scene screen as shown in FIGS. 6A and 6B, and in order to defeat the enemy character Ob1 in this battle scene screen. Then, an attack is executed using the item I1 or the item I2 displayed on the graphic user interface G1.
  • an item to be used for the attack is selected by a user operation, and according to the selection, the user's level (physical strength, magic, language, etc.) at that time is selected. It can be executed by type and intensity.
  • the game mode execution unit 143a displays a window G3 in which a word or phrase to be generated is set, as shown in FIG. Is detected, an attack of a type and intensity corresponding to the detected accuracy is performed, and an evaluation is performed based on points or level-up based on damage given to the enemy by the attack.
  • step S204 when the story mode is selected in step S204 (“story mode” in S204), the story mode is executed by the story mode execution unit 143b (S205), and the story mode is generated in response to the input of the audio signal by the player.
  • the character is operated according to the event processing performed (S206). More specifically, the story mode in the present embodiment takes the form of a so-called "drama play", and the theater St1 as shown in FIGS. 5 (a) and 5 (b) corresponds to the virtual space of the game story.
  • the story unfolds in such a way that the virtual characters Ca1 to Ca3 perform a play on the stage St2 in the theater St1.
  • the video that shows the theater St1 can be displayed three-dimensionally using so-called VR (Virtual Reality) or AR (Augmented Reality) technology.
  • the player of the game can participate in the game as one of the spectators sitting in the audience seat St10 in the theater St1, and the player can display the character on the stage St2 displayed on the display unit 13a through the microphone of the smartphone 1 or the like. It is possible to call out to Ca1-3.
  • the girl character Ca1 is acting on the stage St2 to develop the story, and the player is seated on the audience seat St10. It is set to participate in the game as one person. Under such a setting, when the player speaks to the character Ca1 on the stage St2 through the microphone of the smartphone 1 regardless of the story of the game, an event is generated so as to take a reaction according to the voice.
  • the character when a player performs so-called “hearing” such as cheering, diaper, or caution toward a microphone, the character is caused to take an irregular action corresponding to the voice.
  • this irregular movement include, for example, a character looking at a camera facing a virtual camera projecting the character, and a character corresponding to a player sitting on a stage in a virtual space.
  • pointing in the direction of the audience the direction of sound generation as seen from the character
  • when the character on the stage is approaching danger, call the character's name or warn, "Behind! Behind!” And the like.
  • the timing of the call by the player may be, for example, by displaying a mark representing a microphone on the screen as a graphic, or by using "Watch” as an example of the call as shown in FIG.
  • a character string such as “Out!” May be displayed together with the utterance from the audience seat (the side where the character corresponding to the player is sitting).
  • a voice of a voice of another spectator may be automatically uttered to make it easier for the player to play the voice.
  • the microphone may be turned on / off in synchronization with the timing of the voice call, and the event processing of the timing of the voice call may not be generated while another character is speaking. You may.
  • two or more languages such as English and Japanese
  • the speech recognition target for these two or more languages is used.
  • English and Japanese pronunciation questions are given.
  • a Japanese phrase is displayed as a character string, and English corresponding to the character string is pronounced.
  • an English character string is displayed, and Japanese corresponding to the English is pronounced.
  • Two-way translation problems are being issued.
  • the scenario is configured such that the words or phrases including the pronunciation questions set in this game mode are used in the conversation with the characters Ca1 to Ca3 in the play in the story mode.
  • the player may utter a voice related to the pronunciation problem that has already been set and has been evaluated, and may also input a word or phrase that has not been set. Can utter.
  • the evaluation unit 144a determines whether the word or phrase uttered by the player is a question or a foreign language or a native language for the player among the two or more languages.
  • the character Ca1 has a function of changing the type of reaction according to the evaluation result.
  • the game mode in the event processing generated in response to the shouting to the characters Ca1 to Ca3 on the stage St2 due to the input of the voice signal of the player, the development of the story, the stage, the position on the map, the game mode Is operated such that the reactions of the characters Ca1 to Ca3 and the like differ according to the points or the level-up obtained in.
  • the effect on the game by the voice in the story mode may be value information related to the progress of the game such as points, items, level up, etc. in addition to the reaction of the character. Etc.) may be synchronized with the levels, stages, and points of the game mode that is linked to the story mode.
  • value information such as points and items, effects in the game, variations in the reaction of the character, and the like may be changed.
  • post-recording mode is selected in step S204 ("post-recording mode" in S204)
  • the post-recording mode is executed by the post-recording mode execution unit 143c (S212), and the post-recording mode is input in synchronization with the video reproduction of the post-recording video.
  • the obtained voice is acquired and evaluated (S213), and post-record data is generated and stored (S214).
  • the post-record data generation unit 144b reads the story mode moving image D1 from the memory 15 or the like (S301), and acquires the post-record data D2 (S302).
  • a predetermined scene in the story mode moving image D1 may be cut out, or a moving image corresponding to the predetermined scene of the after-recording data D2 in advance.
  • the post-recording data D2 in addition to the moving image data representing the background and the character motion, graphic data of the lip for lip sync, sound effect or background sound, sound data such as sample sound, voice guide, or the like.
  • Subtitle text data and the like are in a project file format in which each data is decomposed into different layers and files.
  • the post-recording data D2 is sent to the post-recording data generation unit 144b of the voice recognition processing unit 144, and is stored in the memory 15 so that it can be used in a game executed by the game mode execution unit 143a.
  • the post-recording data D2 generated by the post-recording data generation unit 144b is reproduced when the post-recording mode is executed by the post-recording mode execution unit 143c (S303) and displayed on the display unit 13a or the like (S304).
  • the post-recording data D2 draws text data Txt1 for utterance guide or text data Txt2 for subtitles on different layers in moving image data representing the motion of the background or the character Ca4. Each layer is superimposed and displayed.
  • the post-recording data D2 also includes data for displaying a GUI (Graphical User Interface) Ob21 for operating to start and stop synchronous recording and a graphic Ob22 indicating that recording is in progress. .
  • GUI Graphic User Interface
  • a voice is input to the player U, and the input voice is recorded (S402).
  • the speech input in step S401 is evaluated by the evaluator 144a for correctness of pronunciation based on a control signal generated by the speech recognition processor 144 under the control of the scenario controller 142.
  • the evaluation of the accuracy is performed based on a numerical value indicating the degree of matching with the waveform of the sample voice included in the after-recording data D2, and when sufficient accuracy is ensured (e.g., exceeding a predetermined threshold).
  • the synchronous recording is ended, and when the accuracy is insufficient (for example, when the value falls below a predetermined threshold), processing such as prompting re-recording is executed.
  • the audio waveform of the synchronously recorded input audio data D3 is analyzed by the post-record data generation unit 144b (S403), and processed by a voice changer function or the like as necessary (S404).
  • the processed input audio data D3 is passed to the lip sync process S405 and is stored in the memory 15 as the question audio D42.
  • the audio data D3 input in accordance with the post-recording data D2 is matched with the audio waveform of the input audio data D3 in order to further improve the synchronization accuracy with the post-recording data D2. Adjust the movement of the character.
  • the input audio data D3 that has been subjected to the audio processing and the lip sync processing as described above is combined with the after-recording data D2 to form an after-recorded moving image D5.
  • the post-recording video D5 is managed and controlled by the post-recording data control unit 147.
  • the post-recording video D5 is reproduced in the post-recording video reproduction process (S407). It is also used in (S406).
  • the post-recording moving image D5 is converted into a question moving image D41 in the game mode, and is also used in the question processing (S408) in the game mode.
  • the learning level is updated (S209) according to the voice evaluation during the execution of the game mode (S208) and the event processing (S206) during the execution of the story mode, and the scenario proceeds (S210).
  • the processes S202 to S210 described above are repeated until the game ends (“Y” in S211) (“N” in S211).
  • a system for evaluating the utterance of a foreign language by a player is incorporated into a game application that can be executed on a terminal device such as the smartphone 1 so as to enhance the entertaining ability and enhance the language.
  • Learning can be easily performed, and by adopting settings such as speaking to the character on the stage St2 during the stage, training for speaking in a natural atmosphere and timing in real life can be enhanced.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Business, Economics & Management (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • General Physics & Mathematics (AREA)
  • Educational Technology (AREA)
  • Educational Administration (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Computer Security & Cryptography (AREA)
  • General Business, Economics & Management (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Electrically Operated Instructional Devices (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

La présente invention vise à incorporer une évaluation d'énoncé dans un jeu pour rendre l'apprentissage linguistique plus divertissant, et à fournir un entraînement satisfaisant pour réaliser des énoncés à un moment ou dans une atmosphère naturelle dans la vie réelle. À cet effet, l'invention concerne un système de jeu comprenant : une interface d'entrée 12 pour entrer un signal vocal ; une unité de traitement de reconnaissance vocale 144 qui reconnaît la voix d'un joueur et génère un signal de commande concernant un jeu sur la base des résultats de reconnaissance de la voix ; une unité de commande de scénario 142 qui amène le jeu à progresser successivement selon un scénario décrivant un processus d'événement pour effectuer une progression d'histoire ; une unité de traitement de progression de jeu 141 qui amène le scénario à progresser sur la base d'un signal d'entrée dû à une entrée opérationnelle et du signal de commande ; une unité d'exécution de mode de jeu 143a qui exécute un mode de jeu pour évaluer la précision de la prononciation du joueur ; et une unité d'exécution de mode d'histoire 143b qui amène une histoire concernant le comportement d'un personnage à progresser en tant que mode d'histoire. L'évaluation est réalisée en termes de points ou de niveaux en fonction de la précision de la prononciation du joueur par rapport à un problème de reconnaissance vocale qui a été présenté.
PCT/JP2019/024603 2018-06-22 2019-06-20 Système de jeu, procédé de commande de système de jeu, et programme de jeu Ceased WO2019245003A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2020525808A JP7330518B2 (ja) 2018-06-22 2019-06-20 ゲームシステム、ゲームシステムの制御方法及びゲームプログラム

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2018118808 2018-06-22
JP2018-118808 2018-06-22

Publications (1)

Publication Number Publication Date
WO2019245003A1 true WO2019245003A1 (fr) 2019-12-26

Family

ID=68983701

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2019/024603 Ceased WO2019245003A1 (fr) 2018-06-22 2019-06-20 Système de jeu, procédé de commande de système de jeu, et programme de jeu

Country Status (3)

Country Link
JP (1) JP7330518B2 (fr)
TW (1) TWI825111B (fr)
WO (1) WO2019245003A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2022125393A (ja) * 2021-02-17 2022-08-29 ゲシピ株式会社 言語学習装置及びプログラム

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012050791A (ja) * 2010-09-03 2012-03-15 Konami Digital Entertainment Co Ltd キャラクター表示装置、キャラクター表示方法、ならびに、プログラム
JP2012100798A (ja) * 2010-11-09 2012-05-31 Tomohiro Maki 携帯型ゲーム機
US20160307453A1 (en) * 2015-04-16 2016-10-20 Kadho Inc. System and method for auditory capacity development for language processing

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3491833B2 (ja) 2001-02-16 2004-01-26 株式会社ナムコ プログラム、情報記憶媒体及び遊戯装置
AU2008333827A1 (en) * 2007-12-05 2009-06-11 Ol2, Inc. System and method for compressing video by adjusting tile size based on detected intraframe motion or scene complexity

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012050791A (ja) * 2010-09-03 2012-03-15 Konami Digital Entertainment Co Ltd キャラクター表示装置、キャラクター表示方法、ならびに、プログラム
JP2012100798A (ja) * 2010-11-09 2012-05-31 Tomohiro Maki 携帯型ゲーム機
US20160307453A1 (en) * 2015-04-16 2016-10-20 Kadho Inc. System and method for auditory capacity development for language processing

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
"Magna and a mysterious girl", SMARTPHONE ENGLISH LEARNING APP, 25 July 2017 (2017-07-25), Retrieved from the Internet <URL:https://prtimes.jp/main/html/rd/p/000000001.000027398.html> [retrieved on 20190816] *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2022125393A (ja) * 2021-02-17 2022-08-29 ゲシピ株式会社 言語学習装置及びプログラム
JP7452867B2 (ja) 2021-02-17 2024-03-19 ゲシピ株式会社 言語学習装置及びプログラム

Also Published As

Publication number Publication date
TW202005688A (zh) 2020-02-01
JP7330518B2 (ja) 2023-08-22
JPWO2019245003A1 (ja) 2021-07-15
TWI825111B (zh) 2023-12-11

Similar Documents

Publication Publication Date Title
US12159026B2 (en) Audio associations for interactive media event triggering
EP1064975B1 (fr) Méthode de contrôle de jeu vidéo et appareil de jeu vidéo et support d&#39;enregistrement correspondant
US20200251089A1 (en) Contextually generated computer speech
KR20220123170A (ko) 인공지능 아바타 튜터를 활용한 회화 학습 시스템 및 그 방법
US11967248B2 (en) Conversation-based foreign language learning method using reciprocal speech transmission through speech recognition function and TTS function of terminal
CN113450783A (zh) 用于渐进式自然语言理解的系统和方法
KR102659886B1 (ko) 가상 현실 및 ai 인식 영어 학습 시스템
JP7330518B2 (ja) ゲームシステム、ゲームシステムの制御方法及びゲームプログラム
KR101920653B1 (ko) 비교음 생성을 통한 어학학습방법 및 어학학습프로그램
US20240024783A1 (en) Contextual scene enhancement
US11765435B2 (en) Text tagging and graphical enhancement
US12518736B2 (en) Non-transitory computer-readable medium and voice generating system
US12387000B1 (en) Privacy-preserving avatar voice transmission
US20240203435A1 (en) Information processing method, apparatus and computer program
US20250041733A1 (en) Modifying gameplay experiences
JP6867543B1 (ja) 情報処理装置、情報処理方法及びプログラム
JP2024164644A (ja) 対話システム、プログラムおよび対話方法
JP2001005476A (ja) プレゼンテーション装置
EP4631589A1 (fr) Systèmes et procédés de modification dynamique de la voix d&#39;un personnage de jeu
WO2024232297A1 (fr) Appareil de traitement d&#39;informations et procédé de traitement d&#39;informations
WO2026028751A1 (fr) Système de traitement d&#39;informations, procédé de traitement d&#39;informations et programme
JP2026047562A (ja) ゲームプログラム及びゲームサーバ
KR20260024888A (ko) Ai 아바타 리얼타임 대화 학습 시스템
JP2024069156A (ja) 音声生成プログラム及び音声生成システム
US11088976B2 (en) Information processing system capable of promoting communication between/among users, information processing apparatus, non-transitory storage medium encoded with computer readable program, and method of controlling information processing apparatus

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19822246

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020525808

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19822246

Country of ref document: EP

Kind code of ref document: A1