WO2024089602A2 - Application et système de double commande d'un jeu, application et système d'affichage de boutons virtuels, système de détermination de la valeur d'au moins un paramètre du doigt de l'utilisateur, et système de détermination et de présentation de la position du doigt de l'utilisateur sur un écran - Google Patents

Application et système de double commande d'un jeu, application et système d'affichage de boutons virtuels, système de détermination de la valeur d'au moins un paramètre du doigt de l'utilisateur, et système de détermination et de présentation de la position du doigt de l'utilisateur sur un écran Download PDF

Info

Publication number
WO2024089602A2
WO2024089602A2 PCT/IB2023/060726 IB2023060726W WO2024089602A2 WO 2024089602 A2 WO2024089602 A2 WO 2024089602A2 IB 2023060726 W IB2023060726 W IB 2023060726W WO 2024089602 A2 WO2024089602 A2 WO 2024089602A2
Authority
WO
WIPO (PCT)
Prior art keywords
finger
user
data
display
control device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/IB2023/060726
Other languages
English (en)
Other versions
WO2024089602A3 (fr
Inventor
Kamil WARULIK
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to EP23813056.1A priority Critical patent/EP4608527A2/fr
Priority to CN202380088619.XA priority patent/CN120417975A/zh
Priority to KR1020257016067A priority patent/KR20250088760A/ko
Priority to JP2025522743A priority patent/JP2025541589A/ja
Publication of WO2024089602A2 publication Critical patent/WO2024089602A2/fr
Publication of WO2024089602A3 publication Critical patent/WO2024089602A3/fr
Priority to PCT/IB2024/060393 priority patent/WO2025088497A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
    • A63F13/211—Input arrangements for video game devices characterised by their sensors, purposes or types using inertial sensors, e.g. accelerometers or gyroscopes
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
    • A63F13/212—Input arrangements for video game devices characterised by their sensors, purposes or types using sensors worn by the player, e.g. for measuring heart beat or leg activity
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
    • A63F13/213—Input arrangements for video game devices characterised by their sensors, purposes or types comprising photodetecting means, e.g. cameras, photodiodes or infrared cells
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
    • A63F13/214—Input arrangements for video game devices characterised by their sensors, purposes or types for locating contacts on a surface, e.g. floor mats or touch pads
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/23—Input arrangements for video game devices for interfacing with the game device, e.g. specific interfaces between game controller and console
    • A63F13/235—Input arrangements for video game devices for interfacing with the game device, e.g. specific interfaces between game controller and console using a wireless connection, e.g. infrared or piconet
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/90—Constructional details or arrangements of video game devices not provided for in groups A63F13/20 or A63F13/25, e.g. housing, wiring, connections or cabinets
    • A63F13/92—Video game devices specially adapted to be hand-held while playing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/014—Hand-worn input/output arrangements, e.g. data gloves
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/033—Indexing scheme relating to G06F3/033
    • G06F2203/0331—Finger worn pointing device

Definitions

  • An application and a system for dual control of a game an application and a system for displaying virtual buttons, a system for determining the value of at least one parameter of the user's finger, and a system for determining and presenting the position of the user's finger on a display
  • the invention relates to an application and a system for dual control of a game, an application and a system for displaying virtual buttons, a system for determining the value of at least one parameter of the user's finger, and a system for determining and presenting the position of the user's finger on a display.
  • Systems including software installed on a smartphone, allow for precisely displaying a pointer on a screen under the fingertip, after using it to hover over the screen, and for controlling a game by means of specified parameters of the finger. This is achieved by providing feedback information about the precise location of the finger with respect to the telephone screen, and about the kind of movement made by the finger.
  • WO2019165139 An example of such a device is disclosed in document WO2019165139.
  • This invention discloses a wearable device which, by using only the magnetic field, is capable of determining the position and the gestures of the user's hands and fingers on the basis of the bending of the joint.
  • the device is capable of displaying a contour of the hand on the screen, while the tracing of the position of the device with respect to the screen has not been disclosed.
  • Said method for locating elements provided with magnetic sensors may also be found useful in the technology of tablets.
  • WO2019005586 discloses a device wearable on one of the fingers. It has optical, acceleration and pressure sensors, and it may connect wirelessly to other devices. Said device may trace the movement of fingers in a VR environment.
  • the publication does not disclose any use of the invention with respect to touch screens, or to determining the position of the finger before touching the screen.
  • FIG. 10642356 Another example of a device determining the position of the user’s hand is disclosed in American patent US10642356.
  • This invention discloses a device in the form of a flexible material comprising an acceleration sensor and a gyroscope. It is intended to determine the position and movements of the hands in space in order to transfer them to a VR environment.
  • the method of monitoring the user’s movements involves tracing the relative movement of at least two fingers by means of the device. No tracing of the movement with respect to external devices, such as a touch screen, is disclosed.
  • Document US10579099B2 discloses a system comprising a ring device.
  • the ring device may have a housing adapted to be wearable on the user's finger.
  • Sensors which can be used to collect input data from the user include force sensors, ultrasonic sensors, inertial measurement units, optical sensors, touch sensors and other elements.
  • the control circuitry may wirelessly transmit information collected from sensors and other input devices to the associated electronic device. This information may be used to control the operation of the electronic device.
  • the housing of the ring device may have an annular main body, and an expandable part connected to the main body.
  • the expandable part may comprise a flap with a hinge, a rotatable housing element, a housing with an internal adjustable frame and a lid, an expandable housing made of an expandable tube connected between the first and the second ring device, and other expandable structures.
  • the presented solution does not define the type of input which will be accomplished after touching and/or pressing the screen.
  • the inventions disclosed in the above documents focus on applications related to VR environments, detecting gestures on the basis of the movement of the hands. Moreover, the user does not receive any feedback information about the position of the finger. In the case of devices with a touch screen, the user is forced to enter a command by touching the touch screen, and subsequently correct the inputting position in accordance with the user’s intentions.
  • Prior art does not provide any solution allowing for the determination of the precise position of the user's fingers over the touch screen, and for displaying the determined position on the screen. Therefore, one of the objectives of the present invention is to enable the user to stream the telephone screen to an external displaying device, and to perform precise actions on the display without looking at it, and instead by only looking at the external displaying device.
  • gestures input into the application displayed on an external display, with no need to check the position of the finger over the control device.
  • a button e.g. disposed under the fingertip
  • the object of the invention is an application for controlling a video game, configured to receive information from a device comprising a touch layer, and data from an additional device for inputting data, and to control the progress of the game depending on this information and these data.
  • the application is additionally configured to transmit data, in particular video data, to a display or to a device with a display, in which the display has the form of a monitor, a smart TV, a projector or a VR or AR device, the screen of a smartphone or the screen of a tablet, and/or to an external device, in particular the device for inputting data, in particular if the device for inputting data is a game controller, a gamepad, a joystick or a device wearable on the user's finger.
  • data in particular video data
  • the display has the form of a monitor, a smart TV, a projector or a VR or AR device, the screen of a smartphone or the screen of a tablet, and/or to an external device, in particular the device for inputting data, in particular if the device for inputting data is a game controller, a gamepad, a joystick or a device wearable on the user's finger.
  • the invention also relates to a system for controlling a video game, comprising:
  • a first control device comprising a processor, memory and the application for controlling a video game according to the invention
  • a device comprising a touch layer configured to determine the position of the user's finger on this touch layer and transmit the information about this position to the first control device
  • an additional device for inputting data configured to transmit the data to the first control device
  • the first control device is configured to receive said information from the device comprising a touch layer, and data from the additional device for inputting data, and to control the progress of the game depending on this information and these data, and to transmit the data, in particular video data, to the display, wherein the display is configured to receive data, in particular video data, from the first control device, and to display a video image on their basis.
  • the first control device, the device comprising a touch layer, and the display are a single integrated device, in particular a smartphone or a tablet.
  • the device comprising a touch layer is a smartphone or a tablet, and the touch layer is the screen of the smartphone or the tablet.
  • the first control device is a smartphone, a tablet, a smart TV or a video game console.
  • the additional device for inputting data is a game controller, a gamepad, a joystick or a device wearable on the user's finger.
  • the device comprising a touch layer is configured to transmit information about the position of the user's finger and/or the input and/or the data, and/or the triggered actions inputted by means of the finger on the touch layer directly to the first control device.
  • the device comprising a touch layer is configured to transmit information about the position of the user's finger and/or the input and/or the data, and/or the triggered actions input by means of the finger on the touch layer to the first control device via the additional device for inputting data.
  • the additional device for inputting data is configured to transmit the data directly to the first control device.
  • the additional device for inputting data is configured to transmit the data to the first control device via the device comprising a touch layer.
  • the display is configured to receive additional data, in particular video data, from an additional external source, for example a video game console, a smartphone, a tablet, or to receive video streaming from the internet, and to display a video image on their basis.
  • the display has the form of a monitor, a smart TV, a projector or a VR or AR device, the screen of a smartphone or the screen of a tablet.
  • the device comprising a touch layer and/or the additional device for inputting data and/or the first control device and/or the display are configured to communicate wirelessly, preferably via Bluetooth and/or Wi-Fi and/or near-field communication NFC and/or Li-Fi.
  • the object of the invention is also an application for generating virtual buttons, configured so as to display them over an application or a game.
  • the virtual buttons have the form of a context menu, and it is configured to receive information from the device comprising a touch layer and possibly data from the additional device, and to generate data, in particular video data, in the form of a context menu dependent on this information and these data.
  • the application is additionally configured in such a way that the virtual buttons can be assigned selected actions of the game or the physical buttons of game controllers, gamepads, mouse buttons or keyboard keys.
  • the application is additionally configured to transmit information about the activated buttons to the display or the additional device for inputting data.
  • the application is additionally configured to transmit data, in particular video data, to the display.
  • the object of the invention is also a system for displaying virtual buttons, comprising:
  • a second control device comprising a processor, memory and the application for generating virtual buttons according to the invention
  • a device comprising a touch layer, configured to determine the position of the user's finger on this touch layer and transmit the information about this position to the second control device, - possibly, an additional device for inputting data, configured to transmit the data to the second control device,
  • the second control device is configured to receive said information from the device comprising a touch layer and possibly data from the additional device, and to generate data, in particular video data, in the form of a context menu depending on this information and these data, and to transmit these data, in particular video data, to the display, wherein the display is configured to receive data, in particular video data, from the second control device, and to display a video image on their basis.
  • the context menu comprises any number of buttons, distributed over a circle around a place corresponding to the position of the user's finger determined by the device comprising a touch layer.
  • said buttons are configured in such a manner that, following their activation, they transmit data to an external device, and in particular to the additional device for inputting data, and these data are transmitted to the second control device, in particular a tablet, a telephone or a television set, as information about pressing physical buttons corresponding to the selected buttons of a game controller or a keyboard and/or a mouse.
  • the device comprising a touch layer is a smartphone or a tablet, and the touch layer is the screen of the smartphone or the tablet.
  • the second control device is a smartphone, a tablet, a smart TV or a video game console.
  • the additional device for inputting data is a game controller, a gamepad, a joystick or a device wearable on the user's finger.
  • the display is configured to receive additional data, in particular video data, from an additional external source, for example a video game console, a smartphone, a tablet, or to receive video streaming from the internet, and to display a video image on their basis.
  • additional data in particular video data
  • an additional external source for example a video game console, a smartphone, a tablet, or to receive video streaming from the internet, and to display a video image on their basis.
  • the display has the form of a monitor, a smart TV, a projector or a VR or AR device, the screen of a smartphone or the screen of a tablet.
  • the device comprising a touch layer and/or the additional device for inputting data and/or the second control device and/or the display are configured to communicate wirelessly, preferably via Bluetooth and/or Wi-Fi and/or near-field communication NFC and/or Li-Fi.
  • the invention also relates to a system for determining the value of at least one parameter of the user's finger, comprising:
  • a third control device comprising a processor, memory and a third control application
  • said at least one sensor is adapted to transmit a signal to the third control device
  • an arrangement for determining the value of at least one parameter of the user's finger comprising at least one sensor, and it is configured to determine the value of at least one parameter of the user's finger, and to transmit information about the determined value of at least one parameter of the user's finger to the third control device, wherein the third control device is configured to receive information about the determined value of at least one parameter of the user's finger.
  • the senor has the form of a motion sensor and/or a pressure sensor and/or an optical sensor and/or a proximity sensor.
  • the system for determining the value of at least one parameter of the user's finger additionally comprises a navigation surface, in particular in the form of the touch screen of a smartphone or a tablet, or a touchpad or a trackpad.
  • the navigation surface is configured to determine the value of at least one parameter of the user's finger, wherein the navigation surface is configured to transmit the value of at least one parameter of the user's finger to the third control device.
  • the system for determining the value of at least one parameter of the user's finger is configured to determine the value of a parameter of the user's finger — meaning the angle between the distal phalanx and the proximal phalanx.
  • the system for determining the value of at least one parameter of the user's finger is configured to determine the value of a parameter of the user's finger — meaning the pressure exerted by the finger on the sensor.
  • the system for determining the value of at least one parameter of the user's finger is configured to determine the value of a parameter of the user's finger — meaning the pressure exerted by the finger on the navigation surface.
  • the system for determining the value of at least one parameter of the user's finger additionally comprises a display.
  • the display is a projector, a monitor, a smart TV, a VR or AR device, the screen of a smartphone or the screen of a tablet.
  • the display is configured to receive data from the third control device, and to display an image on their basis.
  • the display is configured to receive additional data, in particular video data, from an additional external source, for example a video game console, a smartphone, a tablet, or to receive video streaming from the internet, and to display a video image on their basis, including the image.
  • additional data in particular video data
  • an additional external source for example a video game console, a smartphone, a tablet, or to receive video streaming from the internet, and to display a video image on their basis, including the image.
  • the third control device is configured to perform actions based on the value of a parameter of the user's finger.
  • the third control device is a tablet or a smartphone.
  • the third control device is integrated with the display, and in particular the third control device is a smart TV.
  • the third control device is a video game console.
  • the device wearable on the user's finger has the form of a clip meant to be placed on the user's finger from the top or from a side, or of an open or closed ring with a sensor.
  • the device wearable on the user's finger comprises a sensor adapted to measure the angle between the distal phalanx and the proximal phalanx of the user's finger.
  • the device wearable on the user's finger comprises a motion sensor adapted to measure the difference in the position and the angle between the finger and the navigation surface, so as to measure the pressure exerted by the finger on the navigation surface.
  • the device wearable on the user's finger comprises a pressure sensor and/or an optical sensor adapted to measure the pressure exerted by the finger on the navigation surface.
  • the device wearable on the user's finger has an additional part extending over the distal phalanx of the finger, with a motion sensor located in the additional part.
  • the device wearable on the user's finger comprises a button under the finger tip, in particular a pressure sensor, which is configured to generate and transmit a signal to the third control device when pressed.
  • the third control device and/or the device wearable on the user's finger and/or the arrangement for determining the value of at least one parameter of the user's finger and/or the navigation surface and/or the display are configured to communicate wirelessly, preferably via Bluetooth and/or Wi-Fi and/or near-field communication NFC and/or Li-Fi.
  • the system for determining the value of at least one parameter of the user's finger is configured in such a way that the user defines a portion of the surface of the touch layer in the form of the touch screen of a smartphone or a tablet, which constitutes the navigation surface.
  • the object of the invention is also a system for determining and presenting the position of the user's finger on a display, comprising: - a fourth control device configured to determine the position of the finger over the navigation surface, comprising a processor, memory and a fourth control application,
  • a navigation surface configured to transmit information about its position to the fourth control device
  • an arrangement for determining the position of the user's finger over the navigation surface comprising at least one sensor and it is configured to determine said position of the user's finger, and to transmit information about the determined position to the fourth control device
  • the fourth control device is configured to receive said information from the arrangement for determining the position of the user's finger, and to transmit data, in particular video data, to the display, wherein these data, and in particular video data, comprise a finger position indicator corresponding to the position of the user's finger over the navigation surface
  • the display is configured to receive data, in particular video data, from the fourth control device, and to display a video image on their basis, including the finger position indicator.
  • the fourth control device is a tablet or a smartphone or a video game console.
  • the fourth control device is integrated with the display, and in particular the fourth control device is a smart TV.
  • the display has the form of a monitor, a smart TV, a projector or a VR or AR device, the screen of a smartphone or the screen of a tablet.
  • the display is configured to receive additional data, in particular video data, from an additional external source, for example a video game console, a smartphone, a tablet, or to receive video streaming from the internet, and to display a video image on their basis, including the image.
  • the navigation surface is the surface of a touch layer, in particular the touch screen of a tablet or a smartphone.
  • the system for determining and presenting the position of the user's finger on a display is configured in such a way that the user defines a portion of the surface of the touch layer in the form of the touch screen of a smartphone or a tablet, which constitutes the navigation surface.
  • the arrangement for determining the position of the user's finger over the navigation surface is adapted to be put on at least one of the user’s fingers, wherein said arrangement comprises an accelerometer and/or a gyroscope, and it is configured to determine the position of the user's finger over the navigation surface in real time, using the data from said accelerometer and/or gyroscope.
  • the arrangement for determining the position of the user's finger over the navigation surface additionally comprises an optical sensor configured to assist in determining the position of the user's finger over the navigation surface by determining the finger bend level via determining the angle between the distal phalanx and the proximal phalanx.
  • the fourth control device and/or the arrangement for determining the position of the user's finger over the navigation surface, and/or the navigation surface and/or the display are configured to communicate wirelessly, preferably via Bluetooth and/or Wi-Fi and/or near-field communication NFC and/or Li-Fi.
  • the system for determining and presenting the position of the user's finger on a display is configured to recognise the gestures performed by the user's finger over the navigation surface and/or on the navigation surface and/or in the form of a specific configuration of one or more of the user's fingers.
  • the first control device and/or the second control device and/or the third control device and/or the fourth control device are the same device.
  • the system according to the invention assumes the combination of touch navigation known from mobile devices, such as a telephone or a tablet, with additional external input and feedback originating from one or more external devices worn on the finger, in order to create a controller, in particular, but not exclusively, for games, which combines the positive properties of touch navigation known from mobile games, such as a touch interface and precision of control, with additional input allowing for the performance of numerous actions at the same time, with no need to use several fingers, and with a feedback function, which allows for controlling the system, applications and games with no need to look at the fingers.
  • buttons displayed on a transparent layer over the game and configured in such a way that, following activation, they would transmit information corresponding to the information about pressing physical buttons corresponding to the selected buttons of a game controller or a keyboard and/or a mouse, allows the players to use touch navigation for playing any games streamed from a cloud on any device, even when these games officially do not support touch navigation.
  • Figs. 1-3 present a system for determining the value of a parameter of the user's finger — meaning the angle between the distal phalanx and the proximal phalanx.
  • Figs. 4-6 present a system for determining the value of a parameter of the user's finger — meaning the pressure exerted by the finger on the sensor,
  • Figs. 7-1 1 present a system for determining and presenting the position of the user's finger on a display, configured in such a way that the user defines a portion of the surface of the touch layer,
  • Figs. 12-13 present a device wearable on the user's finger in the form of a clip meant to be placed on the user's finger from the top or from a side, or of an open or closed ring with a sensor,
  • Figs. 14-15 present a device wearable on the user's finger comprising a sensor adapted to measure the angle between the distal phalanx and the proximal phalanx of the user's finger
  • Figs. 17-19 present a device wearable on the user's finger having an additional part extending over the distal phalanx of the finger, with a motion sensor located in the additional part.
  • the present invention has the form of a system for controlling a video game.
  • the system comprises a first control device, which comprises a processor, memory and an application for controlling a video game.
  • the application for controlling a video game is configured to receive information from a device comprising a touch layer, and data from an additional device for inputting data, and to control the progress of the game depending on this information and these data.
  • the application for controlling a video game is additionally configured to transmit data, in particular video data, to a display or to a device with a display, in which the display has the form of a monitor, a smart TV, a projector or a VR or AR device, the screen of a smartphone or the screen of a tablet, and/or to an external device, in particular the device for inputting data, in particular if the device for inputting data is a game controller, a gamepad, a joystick or a device wearable on the user's finger.
  • the system further comprises a device comprising a touch layer configured to determine the position of the user's finger on this touch layer and transmit the information about this position to the first control device.
  • the system also has the additional device for inputting data, configured to transmit the data to the first control device, and the display.
  • the first control device is configured to receive said information from the device comprising a touch layer, and data from the additional device for inputting data, and to control the progress of the game depending on this information and these data, and to transmit the data, in particular video data, to the display.
  • the display is in turn configured to receive the data, in particular video data, from the first control device, and to display a video image on their basis.
  • the device comprising a touch layer is a smartphone or a tablet, and the touch layer is the screen of the smartphone or the tablet.
  • the first control device is a smartphone, a tablet, a smart TV or a video game console.
  • the additional device for inputting data may be a game controller, a gamepad, a joystick or a device wearable on the user's finger.
  • the device comprising a touch layer is configured to transmit information about the position of the user's finger and/or the input and/or the data, and/or the triggered actions inputted by means of the finger on the touch layer directly to the first control device.
  • the device comprising a touch layer is configured to transmit information about the position of the user's finger on the touch layer to the first control device via the additional device for inputting data.
  • the additional device for inputting data is configured to transmit the data directly to the first control device.
  • the additional device for inputting data is configured to transmit the data to the first control device via the device comprising a touch layer.
  • the display is configured to receive additional data, in particular video data, from an additional external source, for example a video game console, a smartphone, a tablet, or to receive video streaming from the internet, and to display a video image on their basis.
  • additional data in particular video data
  • an additional external source for example a video game console, a smartphone, a tablet, or to receive video streaming from the internet, and to display a video image on their basis.
  • the display may have the form of a monitor, a smart TV, a projector or a VR or AR device, the screen of a smartphone or the screen of a tablet.
  • the device comprising a touch layer and/or the additional device for inputting data and/or the first control device and/or the display are configured to communicate wirelessly, preferably via Bluetooth and/or Wi-Fi and/or near-field communication NFC and/or Li-Fi.
  • the invention has the form of a system for displaying virtual buttons in the form of a context menu, comprising a second control device comprising a processor, memory and an application for generating virtual buttons.
  • An application for generating data, in particular video data, in the form of a context menu is configured so as to display them over an application or a game.
  • the virtual buttons have the form of a context menu, and it is configured to receive information from the device comprising a touch layer and possibly data from the additional device, and to generate data, in particular video data, in the form of a context menu dependent on this information and these data.
  • the application is additionally configured in such a way that the virtual buttons can be assigned selected actions of the game or the physical buttons of game controllers, gamepads, mouse buttons or keyboard keys.
  • the application may also be configured to transmit information about the activated buttons to the display or the additional device for inputting data, and to transmit data, in particular video data, to the display.
  • the system further comprises the device comprising a touch layer, configured to determine the position of the user's finger on this touch layer and transmit the information about this position to the second control device.
  • the system may possibly comprise the additional device for inputting data, configured to transmit the data to the second control device.
  • the system also has the display in which e.g. virtual buttons in the form of a context menu are displayed.
  • the second control device is configured to receive said information from the device comprising a touch layer and possibly data from the additional device, and to generate data, in particular video data, in the form of a context menu that is dependent on this information and these data, and to transmit these data, in particular video data, to the display.
  • the display is configured to receive data, in particular video data, from the second control device, and to display a video image on their basis.
  • the system may comprise the system for controlling a video game comprising the first control device, which comprises a processor, memory and an application for controlling a video game.
  • the application for controlling a video game is configured to receive information from the device comprising a touch layer, and data from the additional device for inputting data, and to control the progress of the game depending on this information and these data.
  • the application for controlling a video game is additionally configured to transmit data, in particular video data, to the display.
  • the system further comprises the device comprising a touch layer configured to determine the position of the user's finger on this touch layer and transmit the information about this position to the first control device.
  • the system also has the additional device for inputting data, configured to transmit the data to the first control device, and the display.
  • the first control device is configured to receive said information from the device comprising a touch layer, and data from the additional device for inputting data, and to control the progress of the game depending on this information and these data, and to transmit the data, in particular video data, to the display.
  • the display is in turn configured to receive the data, in particular video data, from the first control device, and to display a video image on their basis.
  • the context menu comprises any number of buttons, preferably four buttons, distributed over a circle around a place corresponding to the position of the user's finger determined by the device comprising a touch layer. Due to the above, the user may select a proper command from the context menu, depending on the needs or on the situation in the game.
  • buttons correspond to the physical buttons of a standard game controller.
  • the standard game controller is usually a pad or a joystick.
  • the buttons are configured in such a manner that, following activation, they transmit data to an external device, and in particular to the additional device for inputting data, and these data are transmitted to the second control device, in particular a tablet, a telephone or a television set, as information about pressing physical buttons corresponding to selected buttons of a game controller or a keyboard and/or a mouse.
  • the device comprising a touch layer is a smartphone or a tablet, and the touch layer is the screen of the smartphone or the tablet.
  • the second control device may be a smartphone, a tablet, a smart TV or a video game console.
  • the additional device for inputting data is a game controller, a gamepad, a joystick or a device wearable on the user's finger.
  • the display is configured to receive additional data, in particular video data, from an additional external source, for example a video game console, a smartphone, a tablet, or to receive video streaming from the internet, and to display a video image on their basis.
  • additional data in particular video data
  • an additional external source for example a video game console, a smartphone, a tablet, or to receive video streaming from the internet, and to display a video image on their basis.
  • the display has the form of a monitor, a smart TV, a projector or a VR or AR device, the screen of a smartphone or the screen of a tablet.
  • the device comprising a touch layer and/or the additional device for inputting data and/or the second control device and/or the display are configured to communicate wirelessly, preferably via Bluetooth and/or Wi-Fi and/or near-field communication NFC and/or Li-Fi.
  • the purpose of the context menu is to add a larger amount of additional input for performing side actions.
  • the context menu may have the form of a menu displayed on the interface in the location of the finger. The finger is located by the place in which the finger touches the navigation surface, or by feedback functions.
  • the menu may be activated with one of the buttons of the device wearable on the user's finger, b.
  • the menu may be activated by pressing a button on the interface of a game or an application, c.
  • the menu may be activated by an external mechanical button, which mechanically presses the button on the interface of a game or an application.
  • these may be triggers which, after being put on a phone, when pressed by a finger, mechanically touch the screen in the place where there is a button on the interface, d.
  • the menu may be activated with a gesture.
  • the context menu may be displayed on the screen of the navigation space and/or only on an external screen.
  • the menu is displayed within the range of one of the fingers, e.g. around a given finger (the place of touching the navigation space, or a place in the space above which the finger is located). This may be the same finger which activates the menu, as well as another finger. E.g. one of the buttons on the finger tip of the left thumb activates the menu around the right finger.
  • the context menu may have one or more buttons, which correspond to the actions assigned to the buttons of traditional controllers.
  • the buttons of a traditional controller are assigned to the menu, and the game / application performs an action which was previously assigned in the configuration of the traditional controller.
  • the context menu may also have one or more buttons which correspond to the actions assigned in the game / application.
  • the application displaying virtual buttons in the form of a context menu communicates back with the controllers which, after receiving information about the selected position from the context menu, transmit information about the input, meaning about the button which was pressed, to the device.
  • the main buttons and the side actions from the context menu may be assigned to specific buttons of a traditional controller supported by a game or an application.
  • the controller upon receiving information about the selection of an action in the input menu, the controller returns information about input corresponding to, e.g. pressing button A on the controller.
  • a motion sensor which will recognise the direction of movement towards the button/action displayed in the menu which is being approached by the finger.
  • the data about the type of the transmitted input e.g. buttons A or B
  • the context menu displayed in the application or on an external screen becomes only a form of visual feedback information when making the selection.
  • Activation of the context menu proceeds by touching/pressing the selected button, or by swiping the finger and releasing it in the location of the button.
  • the context menu may take on any form which arranges the action icons within a short distance from the finger with which the user may choose the actions.
  • the simplest example is the form of a circle, divided into any number of parts.
  • the form of a context menu with multiple levels is possible, in which the selection of one of the buttons of the context menu displays the next level of the context menu with additional actions.
  • An interface with virtual buttons on a transparent layer over a game or an application allows for configuring virtual buttons in such a way in which they would correspond to the actions in the game, or cause input corresponding to any buttons of popular game controllers, game pads, mouse buttons or keyboard keys.
  • the solution is in particular intended to use game-supported navigation using game controllers, keyboards and mouses, in order to generate on their basis virtual navigation which provides the ability to play games streamed from a cloud, which do not have native touch navigation, on devices with a touch screen.
  • the system for displaying data, in particular video data, in the form of a context menu is within the range of one of the user's fingers which touches the touch layer, which allows for selecting any/each action of this menu only by moving the finger, with no significant hand movements.
  • the system for displaying data, in particular video data, in the form of a context menu is placed around one of the fingers, which touches the touch layer.
  • the selection of actions from the context menu takes place by swiping and releasing the finger over the selected button, and by releasing the finger, without withdrawing the finger from the moment when the menu appears on the touch layer.
  • the context menu may be activated in an external application, which is activated over another application, and in particular a game or a game streaming application.
  • the context menu may also be activated in a game streaming application, over a layer with the streamed image of the game. It may also be displayed on a layer over a game or an application / the graphical layer of the game or the application.
  • the activation of a virtual button of the context menu is assigned in the application code to triggering an action indirectly by the button of a game controller or a keyboard and/or a mouse assigned thereto.
  • the information about input from the context menu received by an external device is transmitted further to another device, e.g. a smart TV, or back to the device from which it was received, e.g. a telephone.
  • another device e.g. a smart TV
  • the device from which it was received e.g. a telephone.
  • the information about the button selected from the context menu is transmitted to an external device, and in particular to the additional device for inputting data, and returned to the control device, in particular a tablet, a telephone or a television set, as information about pressing a physical button/physical input corresponding to the buttons of a game controller or a keyboard and/or a mouse.
  • the purpose of adding external input is the ability to perform several actions at the same time during the game, with no need to use additional fingers.
  • the solution is intended to improve the ergonomics, efficiency, comfort and experience of playing more complex mobile titles, as well as to enable playing console and PC titles using touch navigation, including, but not exclusively, via cloud gaming.
  • Yet another aspect of the invention constitutes a system for determining the value of at least one parameter of the user's finger, comprising a device wearable on the user's finger, at least one sensor disposed in or on the device wearable on the user's finger, a third control device comprising a processor, memory and a third control application.
  • the at least one sensor is adapted to transmit a signal to the third control device.
  • the system additionally comprises an arrangement for determining the value of at least one parameter of the user's finger, wherein said arrangement comprises at least one sensor, and it is configured to determine the value of at least one parameter of the user's finger, and to transmit information about the determined value of at least one parameter of the user's finger to the third control device.
  • the third control device is configured to receive information about the determined value of at least one parameter of the user's finger.
  • the system for determining the value of at least one parameter of the user's finger may comprise a system for controlling a video game, comprising a first control device, which comprises a processor, memory and an application for controlling a video game.
  • the application for controlling a video game is configured to receive information from a device comprising a touch layer, and data from an additional device for inputting data, and to control the progress of the game depending on this information and these data.
  • the application for controlling a video game is additionally configured to transmit data, in particular video data, to a display. Due to this, the system further comprises the device comprising a touch layer configured to determine the position of the user's finger on this touch layer and transmit the information about this position to the first control device.
  • the system also has the additional device for inputting data, configured to transmit the data to the first control device, and the display.
  • the first control device is configured to receive said information from the device comprising a touch layer, and data from the additional device for inputting data, and to control the progress of the game depending on this information and these data, and to transmit the data, in particular video data, to the display.
  • the display is in turn configured to receive the data, in particular video data, from the first control device, and to display a video image on their basis.
  • the system for determining the value of at least one parameter of the user's finger may comprise a system for displaying virtual buttons comprising a second control device comprising a processor, memory and an application for the context menu, comprising a second control device comprising a processor, memory and an application for generating virtual buttons.
  • An application for generating data, in particular video data, in the form of a context menu is configured so as to display them over an application or a game.
  • the virtual buttons have the form of a context menu, and it is configured to receive information from the device comprising a touch layer and possibly data from the additional device, and to generate data, in particular video data, in the form of a context menu dependent on this information and these data.
  • the application is additionally configured in such a way that the virtual buttons can be assigned selected actions of the game or the physical buttons of game controllers, gamepads, mouse buttons or keyboard keys.
  • the application may also be configured to transmit information about the activated buttons to the display or the additional device for inputting data, and to transmit data, in particular video data, to the display.
  • the system further comprises the device comprising a touch layer, configured to determine the position of the user's finger on this touch layer and transmit the information about this position to the second control device.
  • the system may possibly comprise the additional device for inputting data, configured to transmit the data to the second control device.
  • the system also has the display which displays, e.g. the context menu.
  • the second control device is configured to receive said information from the device comprising a touch layer and possibly data from the additional device, and to generate data, in particular video data, in the form of a context menu depending on this information and these data, and to transmit these data, in particular video data, to the display.
  • the display is configured to receive data, in particular video data, from the second control device, and to display a video image on their basis.
  • At least one sensor may contact the surface of the finger.
  • the senor has the form of a camera and/or a proximity sensor and/or a motion sensor and/or a pressure sensor and/or an optical sensor.
  • the system for determining the value of at least one parameter of the user's finger may additionally comprise a navigation surface, in particular in the form of a touch layer, in particular in the form of the touch screen of a smartphone or a tablet, or a touchpad or a trackpad, configured to determine the value of at least one parameter of the user's finger, the navigation surface being configured to transmit the value of at least one parameter of the user's finger to the third control device.
  • a navigation surface in particular in the form of a touch layer, in particular in the form of the touch screen of a smartphone or a tablet, or a touchpad or a trackpad, configured to determine the value of at least one parameter of the user's finger, the navigation surface being configured to transmit the value of at least one parameter of the user's finger to the third control device.
  • the system for determining the value of at least one parameter of the user's finger is configured to determine the value of a parameter of the user's finger — meaning the angle between the distal phalanx and the proximal phalanx, which is presented in Figs. 1- 3.
  • the system for determining the value of at least one parameter of the user's finger is configured to determine the value of a parameter of the user's finger — meaning the pressure exerted by the finger on the sensor, which is presented in Figs. 4-6.
  • the system for determining the value of at least one parameter of the user's finger is configured to determine the value of a parameter of the user's finger — meaning the pressure exerted by the finger on the navigation surface and/or the angle between the distal phalanx and the proximal phalanx.
  • information about the parameter is collected when touching the navigation surface with the finger, preferably through the value of the pressure exerted by the finger on the navigation surface.
  • the system for determining the value of at least one parameter of the user's finger additionally comprises a display.
  • the display may be a projector, a monitor, a smart TV, a VR or AR device, the screen of a smartphone or the screen of a tablet.
  • the display is configured to receive data from the third control device and to display an image on their basis.
  • the display is configured to receive additional data, in particular video data, from an additional external source, for example a video game console, a smartphone, a tablet, or to receive video streaming from the internet, and to display a video image on their basis, including the image acquired based on the data received from the third control device.
  • an additional external source for example a video game console, a smartphone, a tablet, or to receive video streaming from the internet.
  • the third control device is configured to perform actions based on the value of a parameter of the user's finger.
  • the third control device may be a tablet or a smartphone. Furthermore, the third control device may be integrated with the display, and in particular the third control device is a smart TV. Preferably, the third control device is a video game console.
  • the device wearable on the user's finger has the form of a clip meant to be placed on the user's finger from the top or from a side, or of an open or closed ring with a sensor, which is presented in Figs. 12 and 13.
  • a ring with only a sensor e.g., but not exclusively, an EMG
  • the pressing of the finger can be detected, and the input can be read due to the information about changes in the tension, and the finger bend angle, meaning the type of input, can be identified due to the information about the tension of the finger.
  • This ring may be supplemented with an additional sensor, e.g. a Camera/ Proximity Sensor / Infrared sensor, which will help to assess the bending of the finger.
  • the device wearable on the user's finger comprises a sensor adapted to measure the angle between the distal phalanx and the proximal phalanx of the user's finger, which is presented in Figs. 14 and 15.
  • the additional element extends from the top, and it is made of a flexible material, e.g. rubber, in such a manner that it always presses against the top part of the distal phalanx of the finger.
  • An important feature is that the motion sensor should abut on the distal phalanx and move along with it during bending.
  • the device wearable on the user's finger has an additional part extending over the distal phalanx of the finger, with a motion sensor located in the additional part, which is presented in Figs. 16-19.
  • this solution may be derived from the solution presented in Figs. 14 and 15.
  • the solution has the form of a clip or an open ring, in this case put on from the side.
  • the motion sensor is still placed on the distal phalanx of the finger, and it serves the same function.
  • the “fingertip button” element under the fingertip is an addition which generates input when pressed.
  • the device wearable on the user's finger may comprise a button under the fingertip, in particular a pressure sensor, which is configured to generate and transmit a signal to the third control device when pressed.
  • the device wearable on the user's finger comprises a pressure sensor and/or an optical sensor adapted to measure the pressure exerted by the finger on the navigation surface.
  • the device wearable on the user's finger may have an additional part extending over the distal phalanx of the finger, with a motion sensor located in the additional part.
  • the device wearable on the user's finger comprises a button under the fingertip, in particular a pressure sensor, which is configured to generate and transmit a signal to the third control device when pressed.
  • the third control device and/or the device wearable on the user's finger and/or the arrangement for determining the value of at least one parameter of the user's finger and/or the navigation surface and/or the display are configured to communicate wirelessly, preferably via Bluetooth and/or Wi-Fi and/or near-field communication NFC and/or Li-Fi.
  • the system for determining the value of at least one parameter of the user's finger is configured in such a way that the user defines a portion of the surface of the touch layer in the form of the touch screen of a smartphone or a tablet, which constitutes a navigation surface, in particular in the form of a touch layer, in particular in the form of the touch screen of a smartphone or a tablet, or a touchpad or a trackpad, configured to determine the value of at least one parameter of the user's finger, the navigation surface being configured to transmit the value of at least one parameter of the user's finger to the third control device.
  • Another aspect of the invention is a system for determining and presenting the position of the user's finger on a display, comprising a fourth control device configured to determine the position of the finger over the navigation surface, comprising a processor, memory and a fourth control application, a navigation surface configured to transmit information about its position to the fourth control device, and a display.
  • the system for determining and presenting the position of the user's finger also comprises an arrangement for determining the position of the user's finger over the navigation surface, wherein said arrangement comprises at least one sensor and it is configured to determine said position of the user's finger and to transmit information about the determined position to the fourth control device.
  • the fourth control device is configured to receive said information from the arrangement for determining the position of the user's finger, and to transmit data, in particular video data, to the display, wherein these video data comprise a finger position indicator corresponding to the position of the user's finger over the navigation surface.
  • the display is configured to receive data, in particular video data, from the fourth control device, and to display a video image on their basis, including the finger position indicator.
  • the system for determining and presenting the position of the user's finger on a display may also comprise a system for controlling a video game, comprising a first control device, which comprises a processor, memory and an application for controlling a video game.
  • the application for controlling a video game is configured to receive information from a device comprising a touch layer, and data from an additional device for inputting data, and to control the progress of the game depending on this information and these data.
  • the application for controlling a video game is additionally configured to transmit data, in particular video data, to the display. Due to this, the system further comprises the device comprising a touch layer configured to determine the position of the user's finger on this touch layer and transmit the information about this position to the first control device.
  • the system also has the additional device for inputting data, configured to transmit the data to the first control device, and the display.
  • the first control device is configured to receive said information from the device comprising a touch layer, and data from the additional device for inputting data, and to control the progress of the game depending on this information and these data, and to transmit the data, in particular video data, to the display.
  • the display is in turn configured to receive the data, in particular video data, from the first control device, and to display a video image on their basis.
  • the system for determining and presenting the position of the user's finger on a display may comprise a system for displaying virtual buttons, comprising a second control device comprising a processor, memory and an application for generating virtual buttons.
  • An application for generating data, in particular video data, in the form of a context menu is configured so as to display them over an application or a game.
  • the virtual buttons have the form of a context menu, and it is configured to receive information from the device comprising a touch layer and possibly data from the additional device, and to generate data, in particular video data, in the form of a context menu dependent on this information and these data.
  • the application is additionally configured in such a way that the virtual buttons can be assigned selected actions of the game or the physical buttons of game controllers, gamepads, mouse buttons or keyboard keys.
  • the application may also be configured to transmit information about the activated buttons to the display or the additional device for inputting data, and to transmit data, in particular video data, to the display.
  • the system further comprises the device comprising a touch layer, configured to determine the position of the user's finger on this touch layer and transmit the information about this position to the second control device.
  • the system may possibly comprise the additional device for inputting data, configured to transmit the data to the second control device.
  • the system also has the display which displays, e.g. the context menu.
  • the second control device is configured to receive said information from the device comprising a touch layer and possibly data from the additional device, and to generate data, in particular video data, in the form of a context menu depending on this information and these data, and to transmit these data, in particular video data, to the display.
  • the display is configured to receive data, in particular video data, from the second control device, and to display a video image on their basis.
  • system for determining and presenting the position of the user's finger on a display may also comprise a system for determining the value of at least one parameter of the user's finger comprising a device wearable on the user's finger, at least one sensor disposed in or on the device wearable on the user's finger, a third control device comprising a processor, memory and a third control application.
  • the at least one sensor is adapted to transmit a signal to the third control device.
  • the system additionally comprises an arrangement for determining the value of at least one parameter of the user's finger, wherein said arrangement comprises at least one sensor, and it is configured to determine the value of at least one parameter of the user's finger, and to transmit information about the determined value of at least one parameter of the user's finger to the third control device.
  • the third control device is configured to receive information about the determined value of at least one parameter of the user's finger.
  • the fourth control device may be a tablet or a smartphone or a video game console.
  • the fourth control device is integrated with the display, and in particular the fourth control device is a smart TV.
  • the display may have the form of a monitor, a smart TV, a projector or a VR or AR device, the screen of a smartphone or the screen of a tablet.
  • the display is configured to receive additional data, in particular video data, from an additional external source, for example a video game console, a smartphone, a tablet, or to receive video streaming from the internet, and to display a video image on their basis, including the image.
  • the navigation surface of the system for determining and presenting the position of the user's finger on a display is the surface of the touch layer, in particular the touch screen of a tablet or a smartphone.
  • the system for determining and presenting the position of the user's finger on a display is configured in such a way that the user defines a portion of the surface of the touch layer in the form of the touch screen of a smartphone or a tablet, which constitutes the navigation surface, which is presented in Figs. 7-1 1 .
  • the arrangement for determining the position of the user's finger over the navigation surface is adapted to be put on at least one of the user’s fingers, wherein said arrangement comprises an accelerometer and/or a gyroscope, and it is configured to determine the position of the user's finger over the navigation surface in real time, using the data from said accelerometer and/or gyroscope.
  • the arrangement for determining the position of the user's finger over the navigation surface is adapted to be put on at least one of the user’s fingers, wherein said arrangement comprises a magnetometer, and it is configured to determine the position of the user's finger over the navigation surface in real time, using the data from the magnetometer.
  • the arrangement for determining the position of the user's finger over the navigation surface comprises at least one camera, and it is configured to determine the position of the user's finger over the navigation surface in real time, using the data from the at least one camera, wherein preferably said arrangement comprises a device wearable on at least one of the user's fingers, having an indicator.
  • the arrangement for determining the position of the user's finger over the navigation surface comprises a device configured to detect the intensity of light, and configured to determine the position of the user's finger over the navigation surface in real time, using the data from said device, wherein preferably said arrangement comprises a device wearable on at least one of the user's fingers, which constitutes a light barrier or a reflector for the light emitted by the navigation surface.
  • the arrangement for determining the position of the user's finger over the navigation surface additionally comprises an optical sensor configured to assist in determining the position of the user's finger over the navigation surface by determining the finger bend level via determining the angle between the distal phalanx and the proximal phalanx.
  • the fourth control device and/or the arrangement for determining the position of the user's finger over the navigation surface, and/or the navigation surface and/or the display are configured to communicate wirelessly, preferably via Bluetooth and/or Wi-Fi and/or near-field communication NFC and/or Li-Fi.
  • the system for determining and presenting the position of the user's finger on a display is configured to recognise the gestures performed by the user's finger over the navigation surface and/or on the navigation surface and/or in the form of a specific configuration of one or more of the user's fingers.
  • the first control device and/or the second control device and/or the third control device and/or the fourth control device are the same device.
  • Each of the systems according to the present invention i.e. the system for dual control of a game, the system for displaying virtual buttons, the system for determining the value of at least one parameter of the user's finger and the system for determining and presenting the position of the user's finger on a display, may be implemented individually, as the only system, without the implementation of the remaining systems, or any two of them may be implemented, i.e.
  • an application is a computer program which, when activated on proper hardware, will perform the actions mentioned in the present specification. They refer to a computer program product comprising instructions which, when the program is executed by the computer, cause the computer to perform proper actions.
  • the above explanation of what an application is refers to all the applications mentioned in the specification, i.e. for dual control of a game, for displaying a context menu, and for displaying the third and the fourth application.
  • touch navigation takes place in contact with a navigation surface, in particular touch-sensitive, in particular a telephone or a tablet, but this may also be a different surface covered with any touch sensor, or a surface which is not touch-sensitive.
  • the purpose of the navigation surface is to return the information about the position of the user's finger and/or the input and/or the data and/or the triggered actions inputted by means of the finger.
  • touch-sensitive this takes place analogically to the operation of touch screens in mobile devices or trackpads. For a surface which is not touch-sensitive, this takes place on the basis of measurements of the system sensors.
  • the purpose of the navigation surface is to serve a function analogical to touch control in mobile games, or a mouse in a computer, or analogue knobs in traditional pads.
  • the player need not withdraw any of the thumbs and touch the virtual buttons in order to trigger targeting or shooting actions. They may still move their character by using the left thumb, and rotate the camera by using the right thumb. At the same time, slightly stronger pressing of the left thumb will trigger the targeting actions, and slightly stronger pressing of the right thumb will trigger the shooting actions.
  • a sample use may be such that the user puts on two devices wearable on the user's finger, one per the thumb of each hand, and plays a “first person shooter”-type mobile game on a telephone.
  • Basic control takes place by means of the native capabilities of the telephone and the functions of the game — the left thumb is used to move the character by means of a virtual knob; the right thumb is used to rotate the camera by means of swiping the finger against the right part of the screen. They may also freely use the actions assigned to virtual buttons placed on an interface in the form of a context menu. At the same time, they may use additional input, which is allowed by the device wearable on the user's finger.
  • Another use may be such that the user plays a “first person shooter” -type game, which is streamed from a cloud to a television set.
  • They use a telephone and two devices wearable on the user's finger, each one worn on the thumb of each hand.
  • a set consisting of these three devices and an application installed on the telephone constitutes the entire controller.
  • the player controls the movement of the character and the rotation of cameras in the application on the telephone, which transmits information about the tilt of the virtual knob and the movement of the finger against the screen directly to the television set, or to the device wearable on the user's finger, which transmits it further to the television set along with additional input made by the player via the device wearable on the user's finger.
  • Controlling a game streamed to a television set requires a larger number of actions than just moving the character and rotating the camera. Due to external main actions, the player may additionally perform other necessary actions, such as targeting and shooting.
  • Main external input involves pressing the finger against the navigation plane.
  • Several separate inputs can be, but do not have to be distinguished depending on the angle at which the finger, and in particular the distal phalanx, has been pressed against the navigation plane, or depending on the specific part of the fingertip which is pressed against the navigation plane.
  • Additional external input involves choosing an action from the context menu.
  • the menu is displayed on the screen of the navigation surface and/or on an external screen, preferably in a place which corresponds to the position of one of the fingers.
  • the selection of a position from the context menu involves swiping the finger or the pointer displayed on the external screen, by means of the finger, towards the selected element, and releasing it when the element has been selected, or normally pressing the selected element.
  • the external spatial input involves the performance of specific gestures in the space with one or more fingers wearing the device wearable on the user's finger.
  • the gestures may be performed without reference to another object in the space, and/or with respect to the navigation surface and/or the second or more devices wearable on the user's finger.
  • the purpose of the external input is the ability to perform a larger number of actions at the same time, without interrupting other actions, with a single finger, e.g. to allow for using screen touch or a control plane in order to control the pointer analogically to a mouse pointer.
  • the purpose of adding external input is the ability to perform several actions at the same time during the game, with no need to use additional fingers.
  • the solution is also aimed at improving the ergonomics, efficiency, comfort and experience of playing, e.g. so that the player touching the screen with their right thumb in order to rotate the camera could start shooting without resigning from rotation of the camera even for a split second.
  • Feedback involves collecting data from the motion sensor in order to acquire feedback information about the position of the fingers with respect to the navigation surface, and in particular over the telephone or tablet, but also over another external navigation plane, e.g. a trackpad, or another navigation surface, in particular, but not necessarily, having a touch layer.
  • the purpose is the ability to precisely touch any points of the navigation surface without looking at it and/or the hands.
  • the only information used involves the positions of the fingers, as presented by pointers visible on the external screen, and the ability to display additional information regarding the interface elements of the game, application or system (e.g. the application icon), over which the finger or fingers are positioned, analogically to hovering over certain interface elements with a mouse pointer, but even if the pointers are not displayed.
  • the main purpose of the feedback is to display pointers indicating the position of the fingers with respect to the navigation surface, in particular a telephone or a tablet, and to allow the player to easily hit the interface elements of the game or the application with their finger, without looking at the navigation plane, in particular at the telephone or the tablet.
  • the pointers as well as the interface elements may be displayed directly on the navigation surface, in particular in mobile devices, or only on the external screen. Hovering over an interface element with the pointer (visible or not) may display additional information.
  • the user has put on two devices wearable on the user's finger, one per the thumb of each hand, and they are playing a “first person shooter”-type mobile game on a telephone, transmitting the image displayed thereon to a television set.
  • the pointers displayed on the screen of the telephone, or directly on the screen of the television set present the positions of the thumbs over the screen of the telephone. Due to this, the player may look only at the television set, and still hit the virtual buttons displayed on the interface, on the telephone, with their thumbs, with no need to look at the telephone or the hands.
  • Another sample use may be such that the user plays a “first person shooter”-type game streamed from a cloud to a television set. They use a telephone and two devices wearable on the user's finger, each one worn on the thumb of each hand. A set consisting of these three devices and an application installed on the telephone constitutes the entire controller. Apart from transmitting information about the tilt of the knob or the rotation of the camera, the telephone also serves as an external navigation surface and point of reference for displaying pointers on the television set. The application transforms the entire screen of the telephone, or a part thereof, into a space corresponding to the proportions of the screen of the television set.
  • One of the additional uses of the system for determining and presenting the position of the user's finger on a display is to use the system in the form of the so-called feedback, i.e. in the form of pointers visible on an external screen.
  • the position of the pointers corresponds to the position of the thumbs over the screen of the telephone. Due to this, the players may touch selected buttons on the interface while focusing only on the external screen, without looking at their hands.
  • Another application is the use of additional input in the form of one or more buttons placed on the fingertips, due to which players may perform several main actions at the same time by using only their thumbs.
  • the input is additionally supported by the context menu, which allows for performing an even larger number of side actions.
  • Methods for achieving additional input a) Mechanical buttons — switches — placing one or more mechanical buttons of any kind on the fingertip.
  • the controller element abutting on the fingertip may be flexible and/or replaceable, so as to enable matching a proper size to one’s finger, or replacing it when worn out. It can also be fixedly attached to the rest of the device.
  • Pressure sensors mounting one or more pressure sensors of any kind on the fingertip. Each of the pressure sensors may transmit one or more inputs depending on the exerted pressure.
  • the controller element abutting on the fingertip may be flexible and/or replaceable, so as to enable matching a proper size to one’s finger, or replacing it when worn out. It can also be fixedly attached to the rest of the device.
  • Touch sensors Mounting one or more touch sensors of any kind on the fingertip.
  • the controller element abutting on the fingertip may be flexible and/or replaceable, so as to enable matching a proper size to one’s finger, or replacing it when worn out. It can also be fixedly attached to the rest of the device.
  • a motion sensor it may be just a gyroscope, or a gyroscope with an accelerometer, or an entire set of sensors making up a motion sensor located in any place on the finger. When performing a tap/click, the fingers perform a specific micro-movement. The collection of data from the motion sensor allows for discerning this movement/gesture from a normal touch and swipe against the screen.
  • the angle of inclination of the distal phalanx relative to the navigation surface allows for discerning various types of inputs, in particular, but not exclusively, when the motion sensor is placed on the distal phalanx.
  • input A means pressing it with a flat finger, arranged parallel to the navigation surface, with the distal phalanx straightened.
  • Input B is a finger bent at an angle of 45 degrees.
  • Input C means fingers bent at an angle of 90 degrees with respect to the navigation surface and pressing it with the end of the finger (the part of the fingertip directly under the nail).
  • motion sensors are additionally supported by machine learning for reaching better precision.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biophysics (AREA)
  • Cardiology (AREA)
  • General Health & Medical Sciences (AREA)
  • Heart & Thoracic Surgery (AREA)
  • User Interface Of Digital Computer (AREA)
  • Position Input By Displaying (AREA)
  • Slot Machines And Peripheral Devices (AREA)

Abstract

L'invention concerne une application et un système de double commande d'un jeu, une application et un système d'affichage de données, et en particulier de données vidéo, sous la forme d'un menu contextuel, un système de détermination de la valeur d'au moins un paramètre du doigt de l'utilisateur, et un système de détermination et de présentation de la position du doigt de l'utilisateur sur un dispositif d'affichage.
PCT/IB2023/060726 2022-10-24 2023-10-24 Application et système de double commande d'un jeu, application et système d'affichage de boutons virtuels, système de détermination de la valeur d'au moins un paramètre du doigt de l'utilisateur, et système de détermination et de présentation de la position du doigt de l'utilisateur sur un écran Ceased WO2024089602A2 (fr)

Priority Applications (5)

Application Number Priority Date Filing Date Title
EP23813056.1A EP4608527A2 (fr) 2022-10-24 2023-10-24 Application et système de double commande d'un jeu, application et système d'affichage de boutons virtuels, système de détermination de la valeur d'au moins un paramètre du doigt de l'utilisateur, et système de détermination et de présentation de la position du doigt de l'utilisateur sur un écran
CN202380088619.XA CN120417975A (zh) 2022-10-24 2023-10-24 用于游戏的双重控制的应用程序及系统
KR1020257016067A KR20250088760A (ko) 2022-10-24 2023-10-24 게임의 이중 제어를 위한 애플리케이션 및 시스템
JP2025522743A JP2025541589A (ja) 2022-10-24 2023-10-24 ゲームのデュアル制御のためのアプリケーションおよびシステム、仮想ボタンを表示するためのアプリケーションおよびシステム、ユーザの指の少なくとも１つのパラメータの値を決定するためのシステム、ならびにディスプレイ上でユーザの指の位置を決定し提示するためのシステム
PCT/IB2024/060393 WO2025088497A1 (fr) 2023-10-24 2024-10-23 Dispositif, en particulier pour commander un jeu vidéo

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP22461626.8 2022-10-24
EP22461626 2022-10-24

Publications (2)

Publication Number Publication Date
WO2024089602A2 true WO2024089602A2 (fr) 2024-05-02
WO2024089602A3 WO2024089602A3 (fr) 2024-07-18

Family

ID=83996554

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2023/060726 Ceased WO2024089602A2 (fr) 2022-10-24 2023-10-24 Application et système de double commande d'un jeu, application et système d'affichage de boutons virtuels, système de détermination de la valeur d'au moins un paramètre du doigt de l'utilisateur, et système de détermination et de présentation de la position du doigt de l'utilisateur sur un écran

Country Status (1)

Country Link
WO (1) WO2024089602A2 (fr)

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI755608B (zh) * 2018-12-07 2022-02-21 致伸科技股份有限公司 遊戲裝置之輔助裝置
US10953312B1 (en) * 2020-07-02 2021-03-23 Blok Party, Inc. Virtual game console using multiple NFC devices simultaneously
US11554323B2 (en) * 2020-09-11 2023-01-17 Riot Games, Inc. System and method for precise positioning with touchscreen gestures
US11541305B2 (en) * 2020-09-28 2023-01-03 Snap Inc. Context-sensitive remote eyewear controller

Also Published As

Publication number Publication date
WO2024089602A3 (fr) 2024-07-18

Similar Documents

Publication Publication Date Title
CN111638801B (zh) 用于手势辨识的控制器及其手势辨识的方法
Gong et al. Wristwhirl: One-handed continuous smartwatch input using wrist gestures
US8576171B2 (en) Systems and methods for providing haptic feedback to touch-sensitive input devices
TWI408572B (zh) Information processing device, input device, information processing system, information processing method and computer readable storage medium
US20120127070A1 (en) Control signal input device and method using posture recognition
CN210573659U (zh) 计算机系统、头戴式设备、手指设备和电子设备
US20080010616A1 (en) Spherical coordinates cursor, mouse, and method
US20070149282A1 (en) Interactive gaming method and apparatus with emotion perception ability
JP2012503244A (ja) 指に装着される装置および相互作用方法および通信方法
KR102184243B1 (ko) Imu 센서를 이용한 손가락 동작 기반 인터페이스 제어 시스템
US20170087455A1 (en) Filtering controller input mode
EP3134802B1 (fr) Dispositifs périphériques sensibles à la pression, et procédés associés d'utilisation
McArthur et al. An empirical comparison of" wiimote" gun attachments for pointing tasks
US20120133581A1 (en) Human-computer interaction device and an apparatus and method for applying the device into a virtual world
US20080248871A1 (en) Interface device
JP2019522855A (ja) ハンドヘルド電子デバイスのための適応的ユーザ・インターフェース
CN112791382A (zh) Vr场景的控制方法、装置、设备及存储介质
CN112237736A (zh) 使用触摸感应以使轨迹球表现得像操纵杆
EP4345584A1 (fr) Dispositif de commande, procédé de commande et programme
US20050264522A1 (en) Data input device
KR20190005038A (ko) 유저 인터페이스 제공 방법 및 이를 수행하기 위한 단말
EP4608527A2 (fr) Application et système de double commande d'un jeu, application et système d'affichage de boutons virtuels, système de détermination de la valeur d'au moins un paramètre du doigt de l'utilisateur, et système de détermination et de présentation de la position du doigt de l'utilisateur sur un écran
KR101686585B1 (ko) 비행 가상 시뮬레이터 환경에서 회전형 꼭지 손잡이를 구동하기 위한 손 움직임 추적 시스템
JP7672474B2 (ja) プログラム
EP2450776A1 (fr) Interface utilisateur permettant de contrôler un système de traitement de données à l'aide d'une manette de jeu

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23813056

Country of ref document: EP

Kind code of ref document: A2

ENP Entry into the national phase

Ref document number: 2025522743

Country of ref document: JP

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 2025522743

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 202547046466

Country of ref document: IN

ENP Entry into the national phase

Ref document number: 20257016067

Country of ref document: KR

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 2023813056

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2023813056

Country of ref document: EP

Effective date: 20250526

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23813056

Country of ref document: EP

Kind code of ref document: A2

WWP Wipo information: published in national office

Ref document number: 202547046466

Country of ref document: IN

WWP Wipo information: published in national office

Ref document number: 1020257016067

Country of ref document: KR

WWE Wipo information: entry into national phase

Ref document number: 202380088619.X

Country of ref document: CN

WWP Wipo information: published in national office

Ref document number: 202380088619.X

Country of ref document: CN

WWP Wipo information: published in national office

Ref document number: 2023813056

Country of ref document: EP