EP4677552A1 - Système pour convertir une entrée d'expression en une animation de corps entier complexe en temps réel - Google Patents

Système pour convertir une entrée d'expression en une animation de corps entier complexe en temps réel

Info

Publication number
EP4677552A1
EP4677552A1 EP23926581.2A EP23926581A EP4677552A1 EP 4677552 A1 EP4677552 A1 EP 4677552A1 EP 23926581 A EP23926581 A EP 23926581A EP 4677552 A1 EP4677552 A1 EP 4677552A1
Authority
EP
European Patent Office
Prior art keywords
movements
user
facial
character
animation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP23926581.2A
Other languages
German (de)
English (en)
Inventor
Colin Brady
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Amgi Animation LLC
Original Assignee
Amgi Animation LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US18/117,844 external-priority patent/US12608866B2/en
Application filed by Amgi Animation LLC filed Critical Amgi Animation LLC
Publication of EP4677552A1 publication Critical patent/EP4677552A1/fr
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/20—Three-dimensional [3D] animation
    • G06T13/40—Three-dimensional [3D] animation of characters, e.g. humans, animals or virtual beings
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/40—Business processes related to social networking or social networking services
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/20—Three-dimensional [3D] animation
    • G06T13/205—Three-dimensional [3D] animation driven by audio data
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/23—Recognition of whole body movements, e.g. for sport training
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/23—Recognition of whole body movements, e.g. for sport training
    • G06V40/25—Recognition of walking or running movements, e.g. gait recognition
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G10L25/63—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination for estimating an emotional state
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Definitions

  • the present invention is in the technical field of animation systems and more particularly to a real time system to turn ordinary expression input into a complex full body animation.
  • Fig. l is a workflow diagram of a system to turn ordinary expression input into a complex full body animation in real time or from recordings analyzed over time, according to one embodiment of the present invention
  • Fig. 2 is a flowchart diagram of state machine character options to turn ordinary expression input into a complex full body animation useful for the system of Fig. 1;
  • Fig. 3 is a flowchart diagram of a real time system to turn ordinary expression input into a complex full body animation of Fig. 1;
  • Fig. 4 is a driver process for a flowchart diagram of a real time system to turn ordinary expression input into a complex full body animation of Fig. 1.
  • the system may includes one or more than one recording device, one or more than one processor capable of executing instructions for: selecting an animated character for a user to use in the animation; recording one or more than one facial movements, body movements, or both facial and body movements from the user on the recording device; transferring the real-time information or recording from the recording device to a storage for processing by a computing platform, wherein the recording device, the storage and the computing platform may include the same device; identifying the user’s facial movements, body movements, or both facial and body movements from the stored recording; determining the user’s potential next facial movements, body movements, or both facial and body movements using a state machine; and generating a complete full body animated character that interprets the one or more than one user’s facial movement, body movement, or both facial and body movement input in real time or from the recording, wherein the animated character is pre-selected by the user for social media, real time avatar
  • the one or more than one facial and body movements may be input from the user’s face, body, voice, heartbeats, brainwaves or any biological or neural link data, or any data that can be analyzed over time.
  • the user movements may also be determined from the input of text or voice.
  • the step of identifying the user’s facial movements, body movements, or both facial and body movements may track the user’s head position, eye direction, eyebrows and mouth or may user artificial intelligence and/or machine learning to determine the body responses by analyzing previous body responses submitted previously by the user or other users.
  • a computer-implemented method may be provided for a system to turn ordinary expression input into a complex full body animation in real time or from recordings analyzed over time.
  • the method may include causing one or more than one processor to execute a plurality of instructions for: selecting an animated character a user to using in the animation; recording one or more than one facial movements, body movements, or both facial and body movements from a user on a smart device or a recording device; storing the real-time or recording to a storage for processing in real-time; identifying the user’s facial movements, body movements, or both facial and body movements; analyzing the user’s potential next facial movements, body movements, or both facial and body movements using a state machine; and generating a complete full body animated character that interprets the one or more than one user’s facial movement, body movement, or both facial and body movement input, wherein the animated character is pre-selected by the user.
  • a second user may select the animated character for creating a final full body animation.
  • the method also may include instructions for a driver process for
  • the state machine may include instructions to turn real-time or recorded user expressions into a complex full body animation including a set of states, a set of transitions and a storage variable to remember the current state
  • the state machine may be driven by a user’s expression input.
  • a state may occur when the user of a character is engaged in a particular kind of action at any given time.
  • the character may include limited state transitions that integrate restrictions on a next available state that the character can enter from its current state.
  • the state may be a preprogrammed action based on the type of character, and may include actions including: idling, walking, running, and jumping among other actions.
  • the next state of the character may be pre-animated and then applied to the character in real time using the next available state and the limited state transitions. This transition may be accomplished by analyzing the user’s expression input over time.
  • the state machine may determine the next probable character animations. The determination of the next probable state may decreases the amount of computational power required and the time needed to prepare the animation sequence as there are only a limited number of possible next motions for the character and the user.
  • the state machine may include a library of probable character animations including but not limited to body and facial animation clips covering varying degrees of basic human and animal emotions and actions such as happy, sad, surprise, angry, and neutral, for each unique character.
  • the probable character animations may be discreet actions ranging from a fraction of a second to several minutes, and preferably, less than two (2) seconds in duration.
  • Each body and facial animation clip may be further subdivided into smaller more specific regions, wherein the specific regions comprise eyes, mouth, upper body, lower body, or hands and comprise their own sub-clip category of body and facial animation clips.
  • Expression input data may be represented as a complex graph or curve, such that the curves may be recognized as patterns or signatures that relate to certain emotions and specific personalities.
  • audio waves may be analyzed into graphs of expression input using a live running tally of the last five seconds of expression input to determine more specific emotions to reproduce. The live running tally of the last five seconds of expression input may trigger different body gestures animation clips.
  • the live running tally of the last five seconds of expression input may be combined with the user’s speech pattern analysis into a resulting motion signature that may be communicated to a state machine containing pre-animated stylized motions to produce unique and nuanced full body animations based off of the harmonization of the input with the state library.
  • the present claimed system and method may overcome the limitations of the previous systems by providing a system to turn ordinary expression input into a complex full body animation in real time or from recordings analyzed over time.
  • the claimed system and method s may call for taking limited human input from a user’s face, body, voice, heartbeats, brainwaves or any biological or neural link data, and producing extremely expressive full body animation for social media, real time avatar communication and content creation in any medium.
  • Any person from age three to ninety-three may be able to talk into a wireless device (i.e. smartphone, tablet, webcam etc.) or record video and produce original high-quality animation that looks as good as a professionally animated character.
  • the user’s face may be all that’s needed to create unique motion of an entire body.
  • text or voice may be all that is needed to create unique motion of an entire body.
  • the present system and method may also be capable of rendering full body animations from animals as well as humans which may enable content creators the ability to add lifelike animated animals as characters without spending weeks or months trying to mathematically model an animals motion.
  • each block in the flowchart or block diagrams may represent a module, segment, or portion of code, that may include one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures.
  • a flowchart may describe the operations as a sequential process, many of the operations may be performed in parallel or concurrently. In addition, the order of the operations may be rearranged.
  • a process may be terminated when its operations are completed.
  • a process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination may correspond to a return of the function to the calling function or the main function.
  • each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration may be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
  • a storage may represent one or more devices for storing data, including readonly memory (ROM), random access memory (RAM), magnetic disk storage mediums, optical storage mediums, flash memory devices and/or other non-transitory machine readable mediums for storing information.
  • ROM readonly memory
  • RAM random access memory
  • magnetic disk storage mediums magnetic disk storage mediums
  • optical storage mediums flash memory devices and/or other non-transitory machine readable mediums for storing information.
  • machine readable medium may include, but is not limited to portable or fixed storage devices, optical storage devices, wireless channels and various other non-transitory mediums capable of storing, comprising, containing, executing or carrying instruction(s) and/or data.
  • embodiments may be implemented by hardware, software, firmware, middleware, microcode, or a combination thereof.
  • the program code or code segments to perform the necessary tasks may be stored in a machine-readable medium such as a storage medium or other storage(s).
  • One or more than one processor may perform the necessary tasks in series, distributed, concurrently or in parallel.
  • a code segment may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or a combination of instructions, data structures, or program statements.
  • a code segment may be coupled to another code segment or a hardware circuit by passing and/or receiving information, data, arguments, parameters, or memory contents.
  • Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted through a suitable means including memory sharing, message passing, token passing, network transmission, etc. and are also referred to as an interface, where the interface is the point of interaction with software, or computer hardware, or with peripheral devices.
  • the term “synthesize” may refer to the combination of data to form a system of variables.
  • the term “hypothesize” may refer to a supposition or proposed explanation made on the basis of limited evidence for starting point for future investigation.
  • the term “extrapolate” may refer to extend the application of a conclusion based on statistics to an unknown situation by assuming that existing trends will continue.
  • transformation may refer to cause two or more pieces of data to change places with each other or transfer to a different context.
  • the term “harmonize” may refer to producing a pleasing visual combination of color, texture and shape.
  • interpret may refer to imitate someone or their actions and/or motions in order to entertain.
  • recording device may refer to any device capable of recording or storing a user’s voice or face/body motion.
  • smart device may refer to any device capable of recording a user’s voice or face/body motion, and executing instructions operable on at least one processor and communicatively enabled either wired, wireless or both wired and wireless with other similar devices and/or the Internet.
  • expression input may refer to images or data captured by a recording device from a user’s face, body, voice, movement, heartbeats, brainwaves or any biological or neural link data or data created by an algorithm to convert text to speech and where the speech is correlated to a user’s face, body, voice, movement, heartbeats, brainwaves or any biological or neural link data.
  • recording may refer to any real time, pre-recorded, or previously recorded motion pictures that comprise humans or animals that can be analyzed and converted into animated characters.
  • the term “player” may refer to any device capable of displaying the resulting animation created by the system.
  • Various embodiments provide a system to convert expression input into a complex full body animation.
  • One embodiment of the present invention provides a system to convert expression input into a complex full body animation.
  • the expression input may be a video.
  • the expression input may be a voice recording.
  • the expression input may be text input.
  • the voice or text may be interpreted to emotion using software such as applications from nice.com or imotions.com as examples and not limitations.
  • the text may converted to voice using software such as naturalreaders.com or balabolka.com but other applications are possible and are contemplated.
  • text may come from a secondary sources such as from another web site or application such as from Google, Bing, Wikipedia, or from CHAT GPT, for example and not limitation.
  • An API may be used to communicate terms to additional web sites and receiving results back in known formats such that the data may be accurately and efficiently be used by the system and methods.
  • a user 102 may access facial expressions from a memory.
  • users may record their facial expressions with a recording device or a smart device 103.
  • the facial expressions may not be uploaded but may be created by an interpretation system by analyzing the text as the expression input or the verbal input as the expression input to determine an appropriate facial expression.
  • past verbal inputs or past text inputs and the related facial expressions communicated by the current or other users may be analyzed by an artificial intelligence system in the interpretation system to determine an appropriate facial expression as the recorded information.
  • Sample artificial intelligence systems may include but are not limited to Google Cloud Al Platform or Azure Machine Learning Studio.
  • previous users may make a smiling face when saying “that is great news” and may make a frowning face when saying “that is bad news.”
  • the artificial intelligence system in the interpretation system may associate the text or verbal input with determined appropriate facial expression which may include facial movements, body movements, or both facial and body movements including but not limited to the user’s head position, eye direction, eyebrows and mouth and may communicate the facial expression as the recorded information.
  • the recording device or the smart device 103 may communicate the recorded information to a computing platform comprising at least one processor, memory, and storage capable of executing instructions operable on the at least one processor to identify 104 the user’s 102 facial and/or body movements, analyze 106 the potential next movements using a state machine, and generate 108 a complete full body animated character selected by the user 102 that interprets the user’s 102 expression input in real time.
  • the real time animation may be used for uploading to a player 112 through the Internet 110.
  • the user’s 102 real-time animation may be used to communicate with other users on social media and the like through the Internet 110 or displayed on various players.
  • the animated character maybe selected by a second user, such as, for example, a fdm director, for creating the final full body animation.
  • a second user such as, for example, a fdm director
  • Fig. 2 there is shown a flowchart diagram of state machine 200 character options that may turn ordinary expression input into a complex full body animation.
  • Most video games may use a state machine 200 to animate their game characters.
  • Most state machines 200 may be controlled by joysticks which control the video game character’s body motion.
  • Sample state machine are sold by Unity (unity3d.com) and by Unreal (www.unrealengine.com) but other state engines are possible and are contemplated.
  • a state machine 200 may be driven by a user’s 102 expression input. For example, while the user 102 is smiling, only happy full body and face motions may be allowed in the state machine 200. Or when the state machine 200 recognizes the user’s 102 head bobbing back and forth to a rhythm, the state machine 200 may only allow dance motions to be played on the full body animation.
  • a state 202, 204, 206, 208, 210 and 212 may occur when a character is engaged in a particular kind of action at any given time.
  • the actions available depend on the type of character, but typical actions include idling 201, walking 206, running 210, jumping 212, falling 208, etc.
  • These actions may be referred to as states 202-212, in the sense that the character is in a “state” where the character is walking 206, idling 204, etc.
  • the character may have restrictions on the next state 202-212 the character may go to rather than being able to switch immediately from any state to 202-212 any other.
  • a running jump 212 may only be taken when the character is already running 210 and not when it is at a standstill or idle 204, so the character could never switch straight from the idle 204 state to the running jump 212 state.
  • the options for the next state 202-214 that a character may enter from its current state 202-212 are referred to as state transitions 214, 216, 218, 220, and 222. Taken together, the set of states 202-212, the set of transitions 214-222 and a storage variable or marker to remember the current state 224 may form a state machine 200.
  • the states 202-212 and transitions 214-222 of a state machine 200 may be represented using a graph diagram, where the nodes represent the states 202-212 and the arcs, arrows between nodes, represent the transitions 214-222.
  • the current state 224 may be a marker or highlight that is placed on one of the nodes and can then only move to another node 206, 208, or 212 along one of the transition arrows 214-222.
  • the states 202-212 that the character may be in at any given point in time are idle 204, walk 206, run 210, standing jump 202, fall 208 and running jump 212.
  • the character may transition 214-222 between idle 204 and standing jump 202; idle 204 and walk 206; walk 206 and fall 208; walk 206 and run 210; run 210 and fall 208; run 210 and running jump 212; running jump 212 and fall 208.
  • the next state 202-212 of this character may be pre-animated and then applied to the character in real time.
  • the state machine 200 may have already provided the next probable animations of the character to the system 300. This anticipation of next states may decrease the amount of computational power required and the time needed to prepare the animation sequence as there are only a limited number of possible next motions or states 202-212 for the character and user 102.
  • Fig. 3 there may be shown a flowchart diagram of a real time system 300 to turn ordinary expression input into a complex full body animation.
  • Computer generated characters for video games, episodic shows and feature fdms may usually be comprised of a 3D polygonal model and an underlying skeletal control system called a rig.
  • the rig may be animated by setting key poses at certain moments in time, or using more advanced characters techniques such as motion capture where a human performer’s body movement controls the rig.
  • the animation may be saved as discreet actions ranging from a fraction of a second to several minutes. Most actions created for video games may be roughly 1-2 seconds in duration.
  • the present invention utilizes the capabilities of a game engine that may include a library of animation that may contain body and facial animation clips covering the basic human emotions such as happy, sad, surprise, angry, neutral, etc. for each unique character.
  • Each emotion or state 202-212 may include several clips ranging from subtle expression to extreme. Within a happy emotional state, the character’s face and body may range from a subtle smile to laughing to jumping for joy.
  • the animation may be an animated character that was created in advance and does not reflect the person using the system. The person using the system may be able to select the animated character or the system may select the character. The character selected by the system may be based on the input or the action inputted to the system.
  • the animation may be a simulation of the person using the system which may be created from an image processed by the system such as by MetaHuman option available at unrealengine.com.
  • the animation may be in two dimensions or in three dimensions and may be appropriate to be viewed in a virtual reality headset or display device.
  • the system 300 may trigger a happy emotional state in the game engine and may only blend between happy animation clips. If the user’ s eyebrows move down for more than half a second, the software may trigger an angry emotional state and only may play clips from the angry library of clips. Combined with other factors such as rapid eye darts, head motion and jaw movement, the system 300 may recognize 304 a unique complex signature movement which may trigger a unique blend of body and facial movement which ranges from a mild to extreme emotional states.
  • Each animation clip may be further subdivided into smaller more specific regions.
  • the eyes, mouth, upper body, lower body, or hands may contain their own sub clip category of motion animations.
  • the user 102 may dial up or down the level of exaggeration in their performance. In most cases the final performance may be some blend between the user’s 102 live unfiltered motion blended with the animated performance.
  • Fig. 4 there may be shown a flowchart diagram of a driver process 400 for a real time system 300 to turn ordinary expression input into a complex full body animation.
  • Expression input 402 may also be analyzed 404 over time. All expression input 402 data may be represented as a complex graph or curve. Such curves may be recognized as patterns or signatures that relate to certain emotions and even specific personalities.
  • audio wave analysis such as Fast Fourier transform or audio spectrogram, the present invention may apply these techniques to graphs of expression input 402.
  • a running live tally of the last five (5) seconds of expression input 402, such as, for example, facial movement may determine more specific emotions to reproduce.
  • a high frequency of blinking, rapid eye darts, or even a slow blink may be used to trigger a different body gestures.
  • the motion signatures may be communicated to a state machine 200 containing pre-animated stylized motions to produce unique and nuanced full body animated results.
  • the system may read fifty-one unique facial shapes in real time and then apply a filter, in real time, over the recognized facial shape.
  • the present system 300 may sample facial data over time and then may then transpose 414 that facial data into variables that may identify the user’s 102 expression which cannot be done with instantaneous systems.
  • the time based data analysis may be used to produce a full body character animation.
  • the data analysis may continue as a running tally over time.
  • any time period may be used for analysis.
  • Each animated character may have their own individual motions and limitations on those motions that are signatures to the characters themselves.
  • the system 300 may harmonize the user’s 102 movement with the library clips of the motion of the animated character and the animated character’s limitations. For example, even though a user 102 raises their hands above their head, if the animated character is motion limited to raising their hands only to their shoulders, then the data may be harmonized 416, so that the final output does not break the character’s limitations.
  • the user’s 102 background may be used as data input and harmonized. For example, if a projection of a volcano is shown behind the user (or if the user 102 is at a volcano), the heat from the volcano may be used to make the animated character sweat.
  • Machine learning or artificial intelligence may also be used, with sufficient data points to harmonize 416 with the final output of the system 300.
  • the system 300 may be used with a library of video clips to bring people back to “life” by taking the clips library and applying the person’s personal characteristics to the data stream performed by another actor.
  • Humphrey Bogart s many films may be used to make an animated video of him for a new purpose.
  • voice analysis may also be part of these systems for analysis and machine learning.
  • the system 300 may analyze data points over time to make full body animations.
  • the transposed 414 variables may be shown or displayed on any wireless device, over the Internet 422 and the final output may be shown on the same wired, wireless screen 242 displays.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Business, Economics & Management (AREA)
  • Signal Processing (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Computational Linguistics (AREA)
  • Hospice & Palliative Care (AREA)
  • Child & Adolescent Psychology (AREA)
  • Economics (AREA)
  • Acoustics & Sound (AREA)
  • Human Resources & Organizations (AREA)
  • Marketing (AREA)
  • Operations Research (AREA)
  • Quality & Reliability (AREA)
  • Strategic Management (AREA)
  • Tourism & Hospitality (AREA)
  • General Business, Economics & Management (AREA)
  • Processing Or Creating Images (AREA)

Abstract

L'invention concerne un système pour convertir une entrée d'expression ordinaire en une animation de corps entier complexe à l'aide d'un dispositif d'enregistrement, une plateforme informatique avec un ou plusieurs processeurs exécutant des instructions pour sélectionner un personnage animé pour un utilisateur à utiliser dans l'animation, enregistrer les mouvements de visage de l'utilisateur, les mouvements de corps, ou les mouvements de visage et de corps, transférer les informations en temps réel ou l'enregistrement du dispositif d'enregistrement à une mémoire pour un traitement par une plateforme informatique, identifier les mouvements de l'utilisateur à partir de l'enregistrement stocké, déterminer les mouvements de visage et/ou de corps suivants potentiels de l'utilisateur à l'aide d'une machine d'état, et générer un personnage animé de corps entier complet qui interprète les mouvements de l'utilisateur.
EP23926581.2A 2023-03-06 2023-03-08 Système pour convertir une entrée d'expression en une animation de corps entier complexe en temps réel Pending EP4677552A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US18/117,844 US12608866B2 (en) 2020-09-09 2023-03-06 System to convert expression input into a complex full body animation, in real time or from recordings, analyzed over time
PCT/US2023/014826 WO2024186317A1 (fr) 2023-03-06 2023-03-08 Système pour convertir une entrée d'expression en une animation de corps entier complexe en temps réel

Publications (1)

Publication Number Publication Date
EP4677552A1 true EP4677552A1 (fr) 2026-01-14

Family

ID=96917953

Family Applications (1)

Application Number Title Priority Date Filing Date
EP23926581.2A Pending EP4677552A1 (fr) 2023-03-06 2023-03-08 Système pour convertir une entrée d'expression en une animation de corps entier complexe en temps réel

Country Status (2)

Country Link
EP (1) EP4677552A1 (fr)
WO (1) WO2024186317A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119376586B (zh) * 2024-10-29 2025-11-28 南通大学 一种采用gpt的多语言3d数字人交互方法

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10852838B2 (en) * 2014-06-14 2020-12-01 Magic Leap, Inc. Methods and systems for creating virtual and augmented reality
US10775880B2 (en) * 2016-11-30 2020-09-15 Universal City Studios Llc Animated character head systems and methods
WO2022056151A1 (fr) * 2020-09-09 2022-03-17 Colin Brady Système pour convertir une entrée d'expression en une animation du corps entier complexe, en temps réel ou à partir d'enregistrements analysés dans le temps

Also Published As

Publication number Publication date
WO2024186317A1 (fr) 2024-09-12

Similar Documents

Publication Publication Date Title
US20250218099A1 (en) A system to convert expression input into a complex full body animation, in real time of from recordings, analyzed over time
US8581911B2 (en) Training system and methods for dynamically injecting expression information into an animated facial mesh
US8648863B1 (en) Methods and apparatus for performance style extraction for quality control of animation
US12198470B2 (en) Server device, terminal device, and display method for controlling facial expressions of a virtual character
CN111724457A (zh) 基于ue4的真实感虚拟人多模态交互实现方法
CN119071521A (zh) 一种基于ai数字人的直播交互方法以及系统
De Lima et al. Video-based interactive storytelling using real-time video compositing techniques
WO2025161923A1 (fr) Procédé et appareil de traitement de données basés sur l'intelligence artificielle, dispositif électronique, produit-programme informatique et support de stockage lisible par ordinateur
CN115631270A (zh) 虚拟角色的直播方法及装置、计算机存储介质、终端
EP4677552A1 (fr) Système pour convertir une entrée d'expression en une animation de corps entier complexe en temps réel
KR20130130625A (ko) 비디오 생성 시스템 및 방법
WO2025195064A1 (fr) Procédé de génération d'animation, support de stockage, dispositif électronique et produit-programme informatique
JP2026038066A (ja) システム
JP2024160978A (ja) 仮想環境のシステム及び方法におけるマルチモーダルデータストリームベースの人工知能介入
JP2026018498A (ja) システム
WO2025236876A1 (fr) Procédé et appareil pour créer un personnage de jeu, dispositif, support et produit-programme
JP2026028919A (ja) システム
JP2026022477A (ja) システム
JP2026029326A (ja) システム
JP2026029892A (ja) システム
JP2026033781A (ja) システム
CN118714409A (zh) 特效生成方法、电子设备、存储介质及程序产品
JP2026018603A (ja) システム
Ergüzen Personality Transfer in Human Animation: Comparing Handcrafted and Data-Driven Approaches
JP2026027980A (ja) システム

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20250828

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR