WO2024125478A1 - Procédé et dispositif de présentation audio - Google Patents

Procédé et dispositif de présentation audio Download PDF

Info

Publication number
WO2024125478A1
WO2024125478A1 PCT/CN2023/138019 CN2023138019W WO2024125478A1 WO 2024125478 A1 WO2024125478 A1 WO 2024125478A1 CN 2023138019 W CN2023138019 W CN 2023138019W WO 2024125478 A1 WO2024125478 A1 WO 2024125478A1
Authority
WO
WIPO (PCT)
Prior art keywords
audio content
user
audio
information
posture information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/138019
Other languages
English (en)
Chinese (zh)
Inventor
高玮隆
徐艺晨
金烨鑫
韩佳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony China Ltd
Sony Group Corp
Original Assignee
Sony China Ltd
Sony Group Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony China Ltd, Sony Group Corp filed Critical Sony China Ltd
Priority to CN202380083647.2A priority Critical patent/CN120322747A/zh
Publication of WO2024125478A1 publication Critical patent/WO2024125478A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/016—Input arrangements with force or tactile feedback as computer generated output to the user
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G06F3/162—Interface to dedicated audio devices, e.g. audio drivers, interface to CODECs

Definitions

  • the present disclosure relates to audio signal processing, and in particular to audio signal rendering.
  • the present disclosure provides for optimizing audio signal presentation, in particular optimizing audio signal presentation for a specific user.
  • the present disclosure also provides optimized interactive audio signal presentation.
  • a receiving-side device for interactive audio presentation comprising a processing circuit configured to: receive relevant information about audio content to be presented from a control-side device for interactive audio presentation, wherein the audio content to be presented comprises audio content set based on user posture information, and present the audio content, wherein presenting the audio content comprises presenting the audio content in a tactile manner.
  • a control-side device for interactive audio presentation comprising a processing circuit, configured to: obtain audio content presentation indication information, the audio content presentation indication information comprising indication information based on posture information of a user to whom the audio is to be presented, send relevant information of the audio content to be presented to a receiving-side device for audio interactive presentation, wherein the audio content to be presented comprises indication information based on posture information of a user to whom the audio is to be presented, The audio content of the posture information setting.
  • a method for a receiving side for interactive audio presentation comprising: receiving relevant information of audio content to be presented from a control side device for interactive audio presentation, wherein the audio content to be presented comprises audio content set based on user posture information, and presenting the audio content, wherein presenting the audio content comprises presenting the audio content in a tactile manner.
  • a control side method for interactive audio presentation comprising: obtaining audio content presentation indication information, the audio content presentation indication information including indication information based on posture information of a user to whom the audio is to be presented, and sending relevant information of the audio content to be presented to a receiving side device for audio interactive presentation, wherein the audio content to be presented includes audio content set based on the posture information of the user.
  • a device comprising at least one processor and at least one storage device, wherein the at least one storage device has program codes and/or instructions stored thereon, which when executed by the at least one processor may enable the at least one processor to perform the method described herein.
  • a storage medium storing program codes and/or instructions.
  • the program codes and/or instructions are executed by a processor, the method described herein may be performed.
  • a program product includes program codes and/or instructions.
  • the processor may perform the method as described herein.
  • a computer program comprising program codes and/or instructions, which when executed by a processor may cause the processor to perform the method as described herein.
  • FIG. 1 shows a conceptual diagram of audio presentation according to an embodiment of the present disclosure.
  • FIG. 2A shows a conceptual diagram of an interactive audio presentation according to an embodiment of the present disclosure.
  • FIG. 2B shows a flowchart of an interactive audio presentation according to an embodiment of the present disclosure.
  • FIG3A shows a block diagram of a receiving-side device for interactive audio presentation according to an embodiment of the present disclosure.
  • FIG3B shows a flowchart of a receiving-side method for interactive audio presentation according to an embodiment of the present disclosure.
  • FIG. 4A shows a block diagram of a control-side device for interactive audio presentation according to an embodiment of the present disclosure.
  • FIG4B shows a flowchart of a control-side method for interactive audio presentation according to an embodiment of the present disclosure.
  • FIG. 5 shows a conceptual flow chart of setting audio content to be presented according to an embodiment of the present disclosure.
  • 6A to 6C show schematic diagrams of exemplary gesture detection.
  • FIG. 7A is a schematic diagram showing an exemplary gesture of a first user (a performer or player) according to the present disclosure.
  • FIG. 7B is a schematic diagram showing an exemplary gesture of a second user (listener) according to the present disclosure.
  • FIG. 8 illustrates a graph of an exemplary audio conversion according to an embodiment of the present disclosure.
  • FIG. 9 shows an exemplary implementation of a receiving-side device according to an embodiment of the present disclosure.
  • FIG. 10 shows an exemplary implementation scenario according to an embodiment of the present disclosure.
  • FIG. 11 illustrates a block diagram showing an exemplary hardware configuration of a computer system capable of implementing an embodiment of the present disclosure.
  • first the terms “first”, “second”, etc. are used merely to distinguish elements or steps, but are not intended to indicate time sequence, priority or importance.
  • the system provides an improved interactive audio presentation solution, especially for the hearing impaired.
  • the present disclosure provides an improved audio presentation solution, in particular, capable of presenting audio content to hearing-impaired people in a tactile manner. More particularly, the audio content to be presented can be provided to the listener as corresponding vibrations via a tactile providing device worn on the listener.
  • the present disclosure proposes an improved interactive audio presentation solution, in particular, the audio content, such as the audio content being played, can be affected by detecting specific inputs of the listener, so that the audio content can be presented to the listener in a more user-desired manner, rhythm, etc. More particularly, the present disclosure proposes to affect the audio content according to gesture information, so as to achieve more convenient interaction.
  • the audio content mentioned in the context of the present disclosure can be in various suitable forms, and as an example can be related to audio, which can cover any suitable type of music signal, such as music melody, track, phoneme, sequence, sound effect, etc.
  • the audio content can correspond to a complete piece of music or a part thereof, or even to a music clip corresponding to a specific user input, such as a specific user gesture.
  • Fig. 1 shows a conceptual diagram of audio presentation according to an embodiment of the present disclosure.
  • the audio presentation according to an embodiment of the present disclosure is particularly suitable for hearing-impaired people, and the audio presentation can be implemented based on gesture information.
  • data/information related to audio presentation is collected.
  • gesture information of members participating in the audio presentation is collected.
  • the personnel participating in the audio presentation may include listeners, especially hearing-impaired people.
  • the personnel participating in the audio presentation may also include specific members responsible for the audio presentation, such as a host, DJ, performer, performer, etc.
  • the collected data/information related to the audio presentation may also include other data/information, such as parameter information of members (including such as identity ID, etc.), instructions for starting and/or stopping the audio presentation, other data/information related to the audio presentation control, etc.
  • audio processing is performed based on the collected data/information.
  • the audio content to be presented can be set based on the collected data/information, especially the gesture information, which will be described in detail later.
  • the set audio content to be presented is presented to the audience.
  • the audio content presentation can be achieved through tactile means.
  • audio presentation can also be achieved through other means. For example, it can be presented to the audience through video, visual effects, lighting effects, etc., so as to enrich the audio presentation effect.
  • Figure 2A shows a conceptual diagram of the interactive audio presentation according to an embodiment of the present disclosure
  • Figure 2B shows a flowchart of the interactive audio presentation according to an embodiment of the present disclosure.
  • the interactive audio presentation according to some embodiments of the present disclosure can be applicable to various application scenarios, such as live music scenes in various music bars and gathering places where music interactions are participated in.
  • a scene includes a first user and a second user.
  • the first user can be a person who is responsible for or leads the audio presentation in the scene, such as a host, DJ, etc., who can start, pause, end, set, and adjust the audio content to be presented to the user.
  • the second user can be the audio presentation object of the scene, such as a customer, participant, listener, etc. in a music bar or gathering place.
  • at least one of the first and second users can especially be a hearing-impaired person.
  • the first and second users may not be people participating in the scene, for example, they may be people who participate in music through the network, cloud, etc.
  • the audio content is set by acquiring the posture information of the first user, and then the set audio content is presented to at least one of the first user and the second user.
  • audio content can be generated or created based on the posture information of the first user, such as audio content corresponding to the posture information of the first user, such as audio content combined from audio units corresponding to the posture information of the first user.
  • the posture information of the first user may only indicate the start, pause, stop, etc. of the audio presentation, so that when the user posture information indicates the start, specific audio content can be presented, such as playing specific music, such as pre-set audio/music.
  • the audio content can be presented to the user in various appropriate ways, such as tactile, video, visual, etc. Special effects, lighting, etc. will not be described in detail here.
  • the audio content is adjusted by acquiring the gesture information of the second user, for example, the audio/music being played is adjusted, and then the adjusted audio content is presented to at least one of the first user and the second user.
  • the adjustment of the audio content can be implemented in various appropriate ways, such as adjusting the volume, melody, etc. of the audio playback, and such adjustment can be reflected in the tactile implementation accordingly.
  • audio data processing can be implemented in an appropriate manner, for example, it can be implemented by software, hardware, firmware, etc. It can be located on the control side of a system for audio presentation, and implemented by a control side device for audio presentation, such as a server, a control device, etc. in a network.
  • a control side device for audio presentation such as a server, a control device, etc. in a network.
  • the user to whom the audio is presented can correspond to the receiving side of the system, which can be configured with a receiving side device to receive audio content so as to present it to the user in an appropriate manner.
  • the receiving side device can cooperate with various presentation devices, such as tactile, visual effects, lighting and other presentation devices to present it to the user.
  • the presentation device can also be included in the receiving side device.
  • the receiving side devices for the first user and the second user may be different.
  • different receiving side devices and/or audio content presentation devices may be used for different users according to the needs of the users.
  • the receiving side devices for the first user and the second user may also be the same.
  • such a receiving side device and/or an audio content presentation device may be able to set functions separately, so that different function configurations can be set for different users according to the needs of the users, for example, certain functions may be turned on or off accordingly for different users.
  • the receiving-side device 300 includes a processing circuit 302, which is configured to receive relevant information about audio content to be presented, wherein the audio content to be presented may include audio content set based on user gesture information, and to cause the audio content to be presented, wherein causing the audio content to be presented may include causing the audio content to be presented in a tactile manner.
  • audio content may be provided to a receiving-side device in an appropriate manner, so that the audio content-related information may be in various appropriate forms accordingly.
  • the audio content can be in various suitable formats and directly provided to the receiving side as audio content related information.
  • the audio content is music to be played, which can be in various suitable music formats, such as mp3, midi, other suitable formats, etc., and is directly sent to the receiving side.
  • the audio content related information may be information indicating the audio content, such as an index of the audio content.
  • the audio content and the audio index may be pre-associated and stored, and the corresponding audio content may be called according to the audio index during the application process.
  • the audio content related information may be information/data obtained by converting the audio content.
  • the audio content can be converted into information/data suitable for the presentation method on the control side and then transmitted to the receiving side as relevant information.
  • the audio content to be presented can be set in various appropriate ways, including but not limited to generation, creation, adjustment, etc. In particular, it can be set based on the user's posture information.
  • the user's posture information includes at least one of the user's posture (including, for example, the posture of a specific part, the spatial position, etc.), and the posture motion information, wherein the posture motion information includes at least one of the posture motion trajectory and the motion acceleration.
  • the posture motion information may include the moving direction, moving speed, moving acceleration, moving frequency, etc. of a specific posture.
  • the user posture may include a specific gesture, a spatial position, etc.
  • the action of the gesture may refer to the action of the gesture, such as how a specific gesture swings, the speed of the swing, the direction of the swing, the frequency of the swing, etc.
  • the application scenario of the receiving side device may include various types of users, especially including the first user and the second user, as described above.
  • the audio content may include audio content set based on the posture information of at least one of the first user and the second user.
  • Fig. 5 shows a conceptual flow chart of the setting of the audio content to be presented according to an embodiment of the present disclosure.
  • the user's posture is acquired or detected, and information and/or data related to the user's posture is generated, thereby setting the audio content to be presented so as to be presented to the user.
  • the setting of the audio content to be presented can usually be implemented on the control side of the system, especially by the control side device.
  • the acquisition or detection of the user gesture may be performed in various appropriate ways.
  • the user's gesture may be acquired by video acquisition, image capture, and the like.
  • the user's movements may be acquired by a camera/camera, and then the user's gesture analysis may be performed from the acquired image or video of the user's movements to acquire information/data related to the user's gesture.
  • this may be achieved by camera motion capture, camera color capture, and the like.
  • a specific color or a specific label may be set on a specific part of the user, and then the movement of the corresponding part may be acquired by camera color recognition.
  • a patch of a specific color may be attached to at least one finger of the user, and then the information/data related to the user's finger gesture/movement may be captured by camera color recognition, as shown in FIG6A .
  • the pre-installed cameras in the party venue can be used to capture the postures of the users in the party scene; in remote scenarios, the postures of each user can be captured using their own dedicated cameras and then uploaded to the network; This can be done on the server side or in the cloud for gesture capture.
  • the user's gesture may be acquired through camera skeleton capture.
  • the movement of a specific part of the user's hand such as the overall outline of the finger, the skeleton, etc.
  • the movement state of the finger skeleton may be detected by a device such as a projector through a specific algorithm to acquire the finger gesture.
  • the user may wear a specific gesture capture device, such as a motion capture sensor, a gyroscope, etc., and then obtain information/data related to the user's gesture based on the data of the gesture capture device, as shown in Figure 6C.
  • the user gesture information to be obtained is the user's hand gesture information
  • the gesture capture device may include a motion capture device that can be worn on at least one finger of the user, and the gesture information is based on the gesture information of each finger wearing the motion capture device and/or a combination thereof.
  • the information/data related to the user's posture can be regarded as being obtained on the receiving side and provided to the control side for audio content setting.
  • the control side device is further configured to obtain the user's posture information determined by the posture capture device, and send the obtained user posture information to the control side device.
  • the receiving side device can also provide other appropriate information, such as parameter information of the user, such as user ID, etc.
  • gesture capture and conversion can be an exemplary implementation in a network scenario.
  • the user waves his hand in front of the camera, so that the movement of the user's fingers can be captured by the computer camera.
  • the movement state and trajectory of the user's fingers can be determined by comparing the pixel differences between adjacent pictures, and finger motion data is generated accordingly.
  • Such data can be represented and stored in various appropriate ways, for example, including the data number of each finger, and corresponding data, including but not limited to swing speed, swing position, time point, etc.
  • the user's finger posture, etc. can be determined in this way. This can be achieved in various ways known in the art, which will not be described in detail here.
  • the corresponding audio content is determined based on the determined motion data of the user's fingers, such as converting it into MIDI and musical processing.
  • corresponding audio content may be set based on the acquired relevant information/data of the user's gesture to be presented to the user.
  • the audio content to be presented may include audio content constructed based on audio units or a specific combination corresponding to the posture information of the first user.
  • the audio content may be set based on the association or correspondence between the posture data and the audio unit.
  • the audio unit may be a component unit of the audio content, for example, corresponding to at least one of a phoneme, a sequence of sounds, an audio segment, etc.
  • at least one of the user's gesture information may be obtained.
  • the user can generate audio content by performing a combination of audio units.
  • the audio units can be combined to generate audio content.
  • the audio units corresponding to the user's various gestures can be combined to obtain audio content to be presented.
  • the combined audio content can also be appropriately processed, such as filtering, smoothing, etc.
  • the correlation/correspondence between user gestures and audio units can be pre-constructed, for example, various gestures can be trained and corresponding audio units can be set for each gesture.
  • the first user can provide relatively fine gesture information, such as gesture information of multiple fingers, and perform corresponding control for audio content, such as controlling multiple tracks of audio, generating more accurate audio content, thereby more accurately presenting audio.
  • user gestures can be stored in a database in association with corresponding audio units.
  • User gestures and audio units can be stored in various appropriate ways. For example, each user gesture and the audio unit corresponding thereto can be stored in a list manner, in a mapping manner, etc.
  • user gestures, corresponding audio units, user gesture change modes, corresponding audio unit change modes, etc. can be included in the database, but are not limited thereto. As long as it is possible to generate audio content and/or change audio content based on the acquired user gestures from the data stored in the database.
  • the audio content setting can be performed in an appropriate manner.
  • a machine learning or deep learning algorithm can be used to set the audio content based on the posture data, so that the audio MIDI signal converted according to the posture data is better filtered and smoothed to enhance its musicality.
  • the machine learning or deep learning algorithm may include various algorithms known in the art, which will not be described in detail here.
  • the machine learning or deep learning algorithm may also be pre-trained based on the training data, and the training can be performed in various appropriate ways, which will not be described in detail here.
  • the trained AI model inputs the posture data and outputs a MIDI signal for presenting audio.
  • the trained AI model inputs the posture data of multiple users and the initial audio content on the performance side, and outputs an adjusted MIDI signal for presenting audio, so as to enable the audience to co-create music.
  • the audio content to be presented may include specific audio content specified by the user gesture information.
  • the specific user gesture may correspond to the specific audio content, so that when the specific user gesture is detected, the complete audio content can be directly sent to the receiving side device for presentation.
  • the user's gesture information may also correspond to audio content presentation indication information, which may, for example, indicate a specific operation of the audio presentation, such as start, pause, stop, etc., so that when the gesture information is detected, a corresponding operation may be performed on the audio content presentation.
  • audio content here may be pre-set or associated with the user's gesture.
  • FIG. 7A shows a schematic diagram of an exemplary posture of the first user (performer) according to the present disclosure, wherein different postures may correspond to different music presentation operations, such as continuous movement may correspond to performance, clenching a fist may correspond to drumming, and music recording start, pause, end, and other operations may also correspond to other postures.
  • different postures may correspond to different music presentation operations, such as continuous movement may correspond to performance, clenching a fist may correspond to drumming, and music recording start, pause, end, and other operations may also correspond to other postures.
  • it may correspond to the generation and/or creation of audio content dominated by the first user such as a performer, performer, host, etc. in the audio content presentation scenario.
  • the created or generated audio content can be presented to the user in various appropriate ways.
  • the relevant information of the audio content is converted into data suitable for an audio presentation device; and the converted data is provided to the audio presentation device.
  • the converted data can be driving data or input data of the audio presentation device, so that the audio presentation device can present the audio content to the user in a specific manner.
  • Data conversion can be achieved in various appropriate ways.
  • audio data can be changed into tactile data in various appropriate ways, such as using an analog signal method, an FFT (Fast Fourier Transform) filtering method, and the like.
  • FFT Fast Fourier Transform
  • Fig. 8 shows a schematic diagram of data conversion, wherein different types of music data are converted to obtain respective waveform data and used to drive audio presentation data. Since the obtained waveform data can often reflect the characteristics of different types of music data, the audio presentation device can also accurately present the characteristics, melody, etc. of the music to the user.
  • the receiving side device may be separated from the audio presentation device, and in other embodiments, the receiving side device may be integrated with the audio presentation device. In particular, the receiving side device may include the audio presentation device.
  • the audio content may be provided to the receiving side in various appropriate timing modes. In some embodiments, once a playable/presentable audio unit/segment is available based on the user's gesture information, it is sent to the receiving side. In other embodiments, a predetermined number of audio units/segments, or even the entire audio content, may be sent to the receiving side each time.
  • the audio content may also be audio content to be presented that is set in other ways, such as audio content that starts playing/presenting upon receiving a specific play/presentation instruction, or audio content that starts playing/presenting according to a preset order/instruction, such as audio content predetermined in a concert hall, on-site, etc., which will not be described in detail here.
  • the audio presentation device is a tactile sensation providing device, so that the audio presentation device is The device provides the audio content to the user in a tactile manner.
  • the tactile providing device may include a tactile feedback device worn on at least one of the user's hand, wrist, arm, etc., such as a glove, wristband, armband, etc., which can provide tactile feedback to at least one finger, back of hand, wrist, arm, etc. of the user.
  • the received information/data can be directly forwarded to the tactile device.
  • the receiving side device can convert the audio content into information/data suitable for a tactile presentation mode, and then forward it to the tactile device.
  • the receiving side device can include a conversion unit, which is configured to convert the information/data obtained for the audio presentation mode for audio presentation.
  • the tactile sensation providing device may be implemented in various appropriate ways.
  • the tactile sensation providing device may include a vibrator that can provide vibrations corresponding to the characteristics of the audio content, such as melody, to the user, so that the hearing-impaired person can feel the melody of the music.
  • the tactile sensation providing device can be implemented by an inertial actuator, a piezoelectric semiconductor transducer, an electro-active polymer actuator (EAP), etc., which will not be described in detail here.
  • each user finger may correspond to a specific audio track to set (eg, generate or influence) a different timbre of the audio content.
  • the tactile sensation providing device includes at least one tactile unit, wherein each tactile unit may correspond to a specific audio track in the audio content to be presented.
  • the tactile sensation providing device may include a tactile feedback device in the form of a glove or a finger sleeve, and a vibration motor may be provided for at least one finger component to provide vibration feedback.
  • the vibration motor of each finger component of the glove or finger sleeve style tactile feedback device may provide vibration feedback according to the sound intensity and rhythm of the corresponding audio track.
  • the tactile feedback providing device can be set to provide tactile feedback only for audio tracks that are difficult for the hearing-impaired person to hear, such as tactile feedback for audio tracks of a specific frequency (such as a high-frequency audio track).
  • a specific frequency such as a high-frequency audio track.
  • the correspondence between multiple fingers and multiple audio tracks can be predefined, so that the gesture of each collected finger can control the corresponding audio track, but the tactile feedback unit is only provided on the finger corresponding to the specific audio track, such as the high-frequency audio track, so that only the corresponding audio content of the specific audio track is tactilely fed back to the user.
  • the feedback can be mainly provided for audio frequencies and rhythms that the user is sensitive to.
  • the feedback device can be set to provide tactile feedback only for the audio track where the drum beats are located. This can enhance the user experience of the user feeling the music. For example, when the user listens to music, the user feels feedback at a specific rhythm melody, which further improves the user experience.
  • the tactile sensation providing device includes a glove or finger sleeve style tactile feedback device
  • it can also be appropriately configured to facilitate identification, control, and ease of user operation.
  • it can be pre-specified that a specific one or more fingers are dedicated to gesture control, while another one or more fingers are dedicated to tactile feedback.
  • the presentation device worn by the performer's finger may correspond to the audience.
  • the presentation device worn by the performer's finger may be the same as that worn by the audience, where the performer's finger corresponds to the audience's finger, for example, the same finger corresponds to the same track.
  • the performer's finger-worn device may be different from that worn by the audience, but the correspondence is pre-set.
  • the receiving side device can also make the audio content be presented to the user in other appropriate ways.
  • it can be presented to the user in the form of sound, video display, lighting, visual effects, etc.
  • the receiving side device can convert the audio content into information/data suitable for other presentation modes, and then provide it to be forwarded to the corresponding presentation device.
  • the audio content can be converted and used in various presentation devices in common.
  • audio content can be provided to the user in an audio manner.
  • the audio content can be played to the user through a speaker, etc.
  • the audio content can be further processed before playing, such as converting the audio content into low-frequency content suitable for hearing-impaired people.
  • necessary audio playback software, audio playback equipment, etc. may also be included, which will not be described in detail here.
  • a speaker can be a speaker of a portable device, a speaker in a theater scene, a speaker set up in a KTV, a bar, a gathering place, etc., or a speaker of other appropriate types, etc.
  • the audio content may be provided to the user in a video format.
  • it may be provided to the user via a video presentation device.
  • it may be presented to the user via various types of screens, such as a projector, a computer screen, a screen of a portable device, etc., in various appropriate videos.
  • Such videos may be video tracks, special effects, pictures, short videos, etc. corresponding to the audio content, and may be pre-set and stored.
  • audio can be presented through lighting effects, in particular, the lights of the presentation device can flash accordingly according to the rhythm of the audio content.
  • a presentation device can be fixedly arranged, such as a fixed screen, a flashing device, etc., or it can be portable, such as a screen of a portable device, a flashing device, such as a wristband, an ornament, etc.
  • the lighting effect can be achieved by an LED on an electronic wristband.
  • the present disclosure further proposes optimized interactive audio presentation.
  • audio interaction can be achieved based on user gestures.
  • the user's gestures can be obtained to adjust the audio content.
  • the performer can present the audio content to the audience as described above, and after the audience obtains the audio content, they can give feedback through their actions, such as expressing the user's emotions through actions, adaptively adjusting the audio content according to the user's actions, etc. In this way, user interaction can be achieved.
  • the audio content to be presented includes audio content obtained by adjusting the audio content based on the user's gesture information.
  • the adjustment includes at least one of the following: increasing or decreasing the volume of the audio content; adjusting the rhythm of the audio content; enhancing the effect of the audio content; adding additional effects to the audio content.
  • a user gesture may correspond to a specific audio unit, an audio clip, etc.
  • a specific user gesture may correspond to a modification to the specific audio unit, such as increasing or decreasing the intensity of the specific audio unit, changing the rhythm of the audio unit, etc.
  • the presentation effect of the audio clip may be adjusted accordingly.
  • a specific action may indicate that the presentation effect of the audio clip is to be increased, such as increasing the volume, increasing the tactile effect, etc.; or that the presentation effect of the audio clip is to be decreased, such as decreasing the volume, decreasing the tactile effect, etc.
  • the adjustment here may be performed as described above for the audio content modification.
  • the entire hand indicates a bass when on the left, a treble when on the right, a high octave of a specific note when upward, and a low octave of a specific note when downward.
  • the user can express his/her emotion of liking the specific audio content through actions, such as through specific waving actions, etc. In this way, the emotion can be presented through video in the party scene.
  • the application scenario of interactive audio presentation is particularly suitable for achieving an influence on the presented audio content based on the gesture of the second user (e.g., the audience).
  • the second user can provide relatively rough gesture information, such as gesture information of only one finger, to perform corresponding control on the audio content, such as controlling the drum beat and volume, thereby simplifying the user's operation.
  • FIG. 7B shows a schematic diagram of an exemplary gesture of the second user (audience) according to the present disclosure, for example, audio content control can be performed by continuous movement.
  • the first user may also participate in the scene of interactive audio presentation, for example, the music content may also be controlled or adjusted based on the gesture of the first user.
  • the first user may also be treated as a specific second user, and the audio content may be adjusted based on the gestures of both users.
  • adjusting the audio content based on the user's gesture can be performed in accordance with various appropriate criteria.
  • the presentation of the audio content can be adjusted based on the statistical value of the gesture information of at least one second user. In this way, the needs of the second user can be more comprehensively considered to achieve the impact on the audio content.
  • the statistical value of the posture information of at least one second user includes a statistical value about the priority of the user posture information, and the presentation of the audio content is adjusted according to the highest priority posture information in the posture information of at least one second user.
  • the statistical value about the priority of the user posture information is determined as follows: the posture information of at least one audio user is weighted, wherein the weighted processing is performed based on at least one of the number of each posture information, the priority of each posture information, and the priority of the user corresponding to each posture information.
  • the presentation of the audio content is adjusted based on the statistical values of the posture information of the multiple second users. This allows the audio setting to take into account the group feelings of the multiple second users, and realizes audio presentation feedback jointly created by the group, thereby enhancing the audience's sense of presence.
  • priorities may be set for user gestures, and feedback may be provided based on the priorities of the user gestures. For example, the gestures of each user may be aggregated and sorted based on priorities, and then the audio content may be adjusted based on the gesture with the highest priority.
  • feedback may be provided based on the number of user actions. For example, the actions of each user may be aggregated, the number of identical or similar actions may be counted, and the audio content may be adjusted accordingly based on the action with the largest number.
  • the priority of the user may be further considered.
  • the priority may be set for the user, and the corresponding audio content adjustment may be performed according to the action of the user with the highest priority.
  • feedback may be further provided based on at least two of the priority of user actions, the priority of the user, the number of user actions, etc.
  • the actions of each user may be summarized and mathematically counted to obtain feedback results, so as to make corresponding audio content adjustments according to the feedback results.
  • a priority value for each user may be set, a priority value for each user's action may be set, and then the acquired user actions may be counted to obtain a statistical value for each user action, for example, by multiplying the user priority value or the action priority value by the number of the actions, thereby obtaining the statistical value of the user action. Then, the corresponding audio content adjustment may be performed according to the action with the highest statistical value.
  • user feedback can be easily obtained to achieve interaction, especially in the case of both online and offline scenarios, user feedback can also be obtained.
  • the audio content can be adjusted in a timely manner according to the user's feedback to meet the user's modification.
  • the audio content can also be adjusted in an appropriate manner, so that the audio content can be adjusted more appropriately to obtain a better presentation effect.
  • the processing circuit 302 can be in the form of a general-purpose processor or a dedicated processing circuit, such as an ASIC.
  • the processing circuit 202 can be constructed by a circuit (hardware) or a central processing device (such as a central processing unit (CPU)).
  • the processing circuit 302 can carry a program (software) for making the circuit (hardware) or the central processing device work.
  • the program can be stored in a memory (such as arranged in the memory) or an external storage medium connected from the outside, and downloaded via a network (such as the Internet).
  • the processing circuit 302 may include various units for implementing the above functions, such as a receiving unit 304, which is configured to receive relevant information of the audio content to be presented from a control side device for interactive audio presentation, wherein the audio content to be presented includes audio content set based on the user's posture information, and a control unit 306, which is configured to present the audio content, wherein presenting the audio content includes presenting the audio content in a tactile manner.
  • the control unit 306 may control the sending unit 308 to provide the audio content or its relevant information to the audio presentation device for audio content presentation.
  • the audio presentation device may be included in the receiving side device, in particular, included in the control unit, so that the audio presentation device can be directly controlled by the control unit to present the audio content.
  • the processing circuit 302 may further include an acquisition unit 310, which is configured to acquire user gesture information determined via a gesture capture device, and send the acquired user gesture information to the control side device via the sending unit 308.
  • the acquisition unit 310 may be separated from the gesture capture device, and acquire the user gesture information from the gesture capture device.
  • the acquisition unit 310 may include a gesture capture device.
  • the processing channel 302 may further include a conversion unit 312 configured to convert relevant information of the audio content into data suitable for the audio presentation device; and provide the converted data to the audio presentation device via the sending unit 308.
  • a conversion unit 312 configured to convert relevant information of the audio content into data suitable for the audio presentation device; and provide the converted data to the audio presentation device via the sending unit 308.
  • each unit is shown as a discrete unit in FIG3 , one or more of these units may be combined into one unit, or split into multiple units.
  • some units may not be included in the processing circuit or even the receiving side device, and thus may be shown with dotted lines.
  • the acquisition unit 310 and the conversion unit 312 may even be outside the processing circuit 302, and thus may also be shown with dotted lines.
  • the above-mentioned units are only logical modules divided according to the specific functions they implement, rather than The specific implementation method is limited, for example, it can be implemented in software, hardware, or a combination of software and hardware.
  • the above-mentioned various units can be implemented as independent physical entities, or can also be implemented by a single entity (for example, a processor (CPU or DSP, etc.), an integrated circuit, etc.).
  • the above-mentioned various units are shown with dotted lines in the drawings to indicate that these units may not actually exist, and the operations/functions implemented by them can be implemented by the processing circuit itself.
  • Fig. 3A is only a schematic structural configuration of the receiving side device for audio presentation, and the device 300 may also include other possible components, such as memory, network interface, controller, communication unit, etc., which are not shown for clarity.
  • the processing circuit can be associated with the memory.
  • the processing circuit can be directly or indirectly (for example, other components may be connected in the middle) connected to the memory to access the image processing related data.
  • the memory can store various data and/or information generated by the processing circuit 302.
  • the memory can also be located in the optimization device but outside the processing circuit, or even outside the optimization device.
  • the memory can be a volatile memory and/or a non-volatile memory.
  • the memory can include but is not limited to a random access memory (RAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a read-only memory (ROM), and a flash memory.
  • RAM random access memory
  • DRAM dynamic random access memory
  • SRAM static random access memory
  • ROM read-only memory
  • step S311 receiving step
  • relevant information of audio content to be presented is received from a control side device for interactive audio presentation, wherein the audio content to be presented includes audio content set based on the user's posture information
  • step S313 controlling step
  • the audio content is presented, wherein presenting the audio content includes presenting the audio content in a tactile manner.
  • step S312 conversion step
  • step S312 conversion step
  • these steps can be performed by any appropriate device or device element, such as the aforementioned receiving side device, the processing circuit in the receiving side device, the corresponding element in the processing circuit, etc. It should be noted that the audio presentation method according to the embodiment of the present disclosure may also include other steps, such as the various further processing described above. Moreover, these further processing can also be performed by appropriate devices or device elements, which will not be described in detail here.
  • the control side device 400 includes a processing circuit 402, which is configured to: obtain audio content presentation indication information, the audio content presentation indication information includes indication information based on the posture information of the user to whom the audio is to be presented, and send relevant information of the audio content to be presented to a receiving side device for audio interactive presentation, wherein the audio content to be presented includes audio content set based on the posture information of the user.
  • the processing circuit 402 may be further configured to: obtain user posture information, and set the audio content to be presented based on the obtained user posture information.
  • the processing circuit 402 can be further configured to: determine statistical values of posture information of at least one second user, the statistical values including statistical values about the priority of user posture information, and set the audio content to be presented according to the highest priority posture information in the posture information of at least one second user.
  • the processing circuit 402 can be implemented in various appropriate ways, such as the processing circuit 302 described above, and will not be described in detail here.
  • the processing circuit 402 may include various units for implementing the above functions, such as an acquisition unit 404, which is configured to acquire audio content presentation indication information, wherein the audio content presentation indication information includes indication information based on the posture information of the user to whom the audio is to be presented, and a sending unit 406, which is configured to send relevant information of the audio content to be presented to a receiving side device for audio interactive presentation, wherein the audio content to be presented includes audio content set based on the posture information of the user.
  • an acquisition unit 404 which is configured to acquire audio content presentation indication information, wherein the audio content presentation indication information includes indication information based on the posture information of the user to whom the audio is to be presented
  • a sending unit 406 which is configured to send relevant information of the audio content to be presented to a receiving side device for audio interactive presentation, wherein the audio content to be presented includes audio content set based on the posture information of the user.
  • the processing circuit 402 may include a setting unit 408 configured to set the audio content to be presented based on the acquired user's posture information.
  • the processing circuit may include a determination unit 410 configured to determine statistical values of posture information of at least one second user, wherein the statistical values include statistical values regarding the priority of the user posture information, whereby the setting unit 408 may set the audio content to be presented based on the highest priority posture information in the posture information of at least one second user.
  • the processing channel 402 may further include a conversion unit 412 configured to convert the audio content into information suitable for reception by a receiving device, or even into data suitable for an audio presentation device.
  • a conversion unit 412 configured to convert the audio content into information suitable for reception by a receiving device, or even into data suitable for an audio presentation device.
  • step S411 acquisition step
  • step S413 sending step
  • the relevant information of the audio content to be presented is sent to the receiving side device for audio interactive presentation, wherein the audio content to be presented includes audio content set based on the posture information of the user.
  • step S412 setting step
  • the audio content to be presented is set based on the acquired posture information of the user.
  • a system for interactive audio presentation may include a control side device and a receiving side device as described above, wherein the receiving side device may be associated with at least one user, for example, associated with multiple users including a first user and a second user, wherein each user wears a corresponding or associated receiving side device.
  • control side device receives the posture information of the audio presenting user, and sets the audio content to be presented based on the posture information of the audio presenting user.
  • the receiving side device receives the relevant information of the audio content to be presented, and makes the audio content presented, wherein in particular, presenting the audio content includes presenting the audio content to the user in a tactile manner.
  • a method for interactive audio presentation is also provided, which is based on the control side method and the receiving side method as described above.
  • the gesture e.g., finger gesture, etc.
  • the first user e.g., the host in a party, bar, etc., the performer in a concert, the performer in various activities, etc.
  • the second user e.g., the listener, the audience, etc.
  • the receiving device of the second user e.g., the listener, the audience, etc.
  • the gesture e.g., finger gesture, etc.
  • the music can be adjusted according to the gesture of the second user to be more suitable for the user, thereby further improving the user's experience.
  • Specific devices can be called interactive devices, and the posture of the device can also be used to generate/adjust audio content.
  • it can be a specific device, such as an anthropomorphic doll, a handheld device, various devices worn on the body, etc., and the specific postures of these devices can be collected to achieve audio content presentation and/or feedback.
  • the audience can release specific handheld devices, such as glow sticks, etc., so that the posture/movement of the handheld devices of the audience can be used as audience feedback to adjust the audio content accordingly.
  • FIG9 shows an implementation of a receiving device according to the present disclosure. It can be implemented as a receiving device that can be worn on a user. In the form of cots/gloves on the fingers.
  • 901 indicates a data receiving and sending unit of a receiving side device, which may, for example, receive information related to audio content and provide specific data to drive a tactile providing device 903 and a light special effect presenting device 904. Additionally, 901 may also implement data conversion of audio content data. Optionally, the tactile providing device 903 and the light special effect presenting device 904 may also be included in the receiving side device.
  • the receiving side device may further include a posture acquisition device 902, which may acquire finger motion data and provide the finger motion data to the control side device via 901.
  • a posture acquisition device 902 may acquire finger motion data and provide the finger motion data to the control side device via 901.
  • FIG. 9 shows that the receiving side device includes only a single posture acquisition device 902, and the posture acquisition device 902 is only worn on one finger, this is merely exemplary, and the device may be worn on other fingers, or on more fingers.
  • the gloves worn by the user may be the same or different.
  • two or more fingers may be configured with gesture capture devices, so that the performer's gestures can be detected more accurately, so as to more accurately set, create or synthesize audio content.
  • the gesture capture device may be worn on only one finger, and the tactile providing device may also be worn on another finger, which can simplify the listener's operation and facilitate the listener's use.
  • the receiving-side device may also include an integrated antenna.
  • the receiving side device may also include a battery, a data transmission and receiving device, such as an antenna, and an optional data processing unit, such as a phase shifter, a filter, etc., which will not be described in detail here.
  • a data transmission and receiving device such as an antenna
  • an optional data processing unit such as a phase shifter, a filter, etc., which will not be described in detail here.
  • FIG. 10 is a schematic diagram showing an exemplary implementation of interactive audio presentation between performers and listeners according to an embodiment of the present disclosure.
  • the performer performs relatively fine gesture operations, and then sets the music based on the acquired gesture information, such as converting the gesture into music, so as to present it to the performer in an appropriate manner. And such music can be presented to the audience. For example, in a live scene.
  • the audience can also perform gesture operations, in particular, in order to perform relatively simple gesture operations, and then influence the music based on the acquired gesture information, such as adjusting the rhythm, melody, etc. of the music, so as to present it to the audience in an appropriate manner, such as by sound, touch, visual feedback, etc.
  • the music adjusted in this way can also be presented to the performer. Interactive audio presentation is achieved in this way.
  • Such implementation can be embodied in various appropriate application scenarios.
  • a music bar especially a music bar suitable for or able to accommodate hearing-impaired people.
  • listeners can At the entrance, the user receives a suitable receiving device, such as the glove-type device described above, and then during activities in the music bar, the user sets the music to be listened to based on the gesture corresponding to the finger swing through appropriate body movements, especially finger swings wearing the glove-type device.
  • a suitable receiving device such as the glove-type device described above
  • the technical solution of the present disclosure can be applied to various appropriate tasks, including but not limited to hearing-impaired environments.
  • the task includes generating music and providing the music to the user in a video, audio, or the like.
  • the technical solution of the present disclosure can also be applied to provide other content in other ways. For example, provide audio content, or provide audio content in a video, such as dialogues in a movie or TV series, to a hearing-impaired user. In other embodiments, the technical solution of the present disclosure can also be used for users with normal hearing.
  • the technology disclosed herein can be used in many applications.
  • the technology disclosed herein can be used in live audio presentation applications. It can also be used in remote concerts, recitals, etc., and can also capture the user's gestures, set or adjust the audio content in the cloud, and then provide the audio content to the user through the network.
  • the disclosed solution can be implemented by a software algorithm, so that it can be easily integrated into various types of devices including presentation devices, such as devices including various presentation devices, such as finger sleeves.
  • the disclosed method can be executed as a computer program, instruction, etc. by a processor of a portable device to perform audio presentation enhancement processing.
  • FIG. 11 is a block diagram showing an example structure of a personal computer of an optimization device that can be adopted in an embodiment of the present disclosure.
  • the personal computer can correspond to the above exemplary optimization device according to the present disclosure.
  • a central processing unit (CPU) 1101 performs various processes according to a program stored in a read-only memory (ROM) 1102 or a program loaded from a storage section 1108 to a random access memory (RAM) 1103.
  • ROM read-only memory
  • RAM random access memory
  • the CPU 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104.
  • An input/output interface 1105 is also connected to the bus 1104.
  • the following components are connected to the input/output interface 1105: an input section 1106 including a keyboard, a mouse, etc.; an output section 1107 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1108 including a hard disk, etc.; and a communication section 1109 including a network interface card such as a LAN card, modem, etc.
  • the communication section 1109 performs communication processing via a network such as the Internet.
  • a drive 1110 is also connected to the input/output interface 1105 as needed.
  • a removable medium 1111 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the drive 1110 as needed so that a computer program read therefrom is installed into the storage section 1108 as needed.
  • a program constituting the software is installed from a network such as the Internet or a storage medium such as the removable medium 1111 .
  • such storage medium is not limited to the removable medium 1111 shown in FIG. 11 in which the program is stored and distributed separately from the device to provide the program to the user.
  • the removable medium 1111 include magnetic disks (including floppy disks (registered trademark)), optical disks (including compact disk read-only memory (CD-ROM) and digital versatile disks (DVD)), magneto-optical disks (including minidiscs (MD) (registered trademark)), and semiconductor memories.
  • the storage medium may be ROM 1102, a hard disk included in the storage portion 1108, or the like, in which the program is stored and distributed to the user together with the device containing them.
  • the method and system of the present disclosure may be implemented in a variety of ways.
  • the method and system of the present disclosure may be implemented by software, hardware, firmware, or any combination thereof.
  • the order of the steps of the method described above is illustrative only, and unless otherwise specifically stated, the steps of the method of the present disclosure are not limited to the order specifically described above.
  • the present disclosure may also be embodied as a program recorded in a recording medium, including machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also encompasses a recording medium storing a program for implementing the method according to the present disclosure.
  • Such storage media may include, but are not limited to, a floppy disk, an optical disk, a magneto-optical disk, a memory card, a memory stick, and the like.
  • embodiments of the present disclosure may also include the following exemplary embodiment implementations (EEE).
  • a receiving-side device for interactive audio presentation comprising a processing circuit configured to: receive information related to audio content to be presented from a control-side device for interactive audio presentation, wherein The audio content to be presented includes audio content set based on the user's posture information, and the audio content is presented, wherein presenting the audio content includes presenting the audio content in a tactile manner.
  • EEE 2 The receiving side device according to EEE 1, wherein the user's posture information includes at least one of the user's posture and posture motion information, wherein the posture motion information includes at least one of the posture motion direction, trajectory, and motion acceleration.
  • EEE 3 The receiving side device according to EEE 1, wherein the processing circuit is further configured to: obtain the user's posture information determined by a posture capture device, and send the obtained user posture information to the control side device.
  • EEE 4 The receiving side device according to EEE 3, wherein the gesture capture device includes a motion capture device that can be worn on at least one finger of a user, and the gesture information is based on the gesture information of each finger wearing the motion capture device and/or their combination.
  • EEE 5 The receiving side device according to EEE 1, wherein the user's posture information includes posture information of a first user, and wherein the audio content to be presented includes at least one of specific audio content specified by the first user's posture information, audio content constructed based on an audio unit corresponding to the posture information of the first user, or a specific combination.
  • EEE 6 The receiving side device according to EEE 1, wherein the user's posture information includes posture information of a second user, and wherein the audio content to be presented includes audio content obtained by adjusting the audio content based on the posture information of the second user.
  • EEE 7 The receiving device according to EEE 6, wherein adjusting the audio content based on the posture information of the second user comprises at least one of the following:
  • EEE 8 The receiving device according to EEE 7, wherein adjusting the audio content based on the posture information of the second user comprises:
  • the presentation of the audio content is adjusted based on the statistical value of the gesture information of the plurality of second users.
  • EEE 9 The receiving-side device according to EEE 8, wherein the statistical value of the posture information of the plurality of second users includes a statistical value about the priority of the user posture information, and the presentation of the audio content is adjusted according to the highest priority posture information among the posture information of the plurality of second users.
  • EEE 10 The receiving device according to EEE 9, wherein the statistical value of the priority of the user posture information is determined as follows:
  • the posture information of multiple audio users is weighted, wherein the weighted processing is performed based on at least one of the quantity of each posture information, the priority of each posture information, and the priority of the user corresponding to each posture information.
  • EEE 11 The receiving side device according to EEE 1, wherein presenting the audio content further includes: converting relevant information of the audio content into data suitable for an audio presentation device; and providing the converted data to the audio presentation device.
  • EEE 12 A receiving side device according to any one of EEE 1-11, wherein the audio presentation device is a tactile providing device, so that the audio content is provided to the user in a tactile manner via the tactile providing device.
  • EEE 13 The receiving side device according to EEE 12, wherein the tactile sensation providing device comprises at least one tactile unit, wherein each tactile unit corresponds to a specific audio track in the audio content to be presented.
  • a control side device for interactive audio presentation comprising a processing circuit, configured to: obtain audio content presentation indication information, the audio content presentation indication information comprising indication information based on posture information of a user to whom the audio is to be presented, and send relevant information of the audio content to be presented to a receiving side device for audio interactive presentation, wherein the audio content to be presented comprises audio content set based on the posture information of the user.
  • EEE 15 The control side device according to EEE 14, wherein the processing circuit is further configured to: obtain the user's posture information, and set the audio content to be presented based on the obtained user's posture information.
  • EEE 16 The control side device according to EEE 15, wherein the user's posture information includes posture information of a first user, and wherein the audio content to be presented includes at least one of specific audio content specified by the first user's posture information, audio content constructed based on an audio unit corresponding to the posture information of the first user or a specific combination, and/or
  • the user's posture information includes posture information of a second user
  • the audio content to be presented includes audio content obtained by adjusting the audio content based on the posture information of the second user.
  • EEE 17 The control side device according to EEE 16, wherein the processing circuit is further configured to:
  • the statistical values comprising statistical values regarding priorities of the user gesture information
  • the audio content to be presented is set according to the highest priority gesture information among the gesture information of the plurality of second users.
  • a method for a receiving side of interactive audio presentation comprising: receiving information related to audio content to be presented from a control side device for interactive audio presentation, wherein the audio content to be presented includes The method further comprises: providing an audio content configured based on the user's gesture information, and causing the audio content to be presented, wherein presenting the audio content comprises presenting the audio content in a haptic manner.
  • EEE 19 The method according to EEE 18 also includes: acquiring posture information of the user determined by a posture capture device, and sending the acquired user posture information to the control side device.
  • EEE 20 The method according to EEE 18, wherein presenting the audio content further includes: converting relevant information of the audio content into data suitable for an audio presentation device; and providing the converted data to the audio presentation device.
  • a control side method for interactive audio presentation comprising: obtaining audio content presentation indication information, the audio content presentation indication information including indication information based on posture information of a user to whom the audio is to be presented, and sending relevant information of the audio content to be presented to a receiving side device for audio interactive presentation, wherein the audio content to be presented includes audio content set based on the posture information of the user.
  • EEE 22 The method according to EEE 21 also includes: obtaining user posture information, and setting the audio content to be presented based on the obtained user posture information.
  • the method according to EEE 21 also includes: determining statistical values of posture information of multiple second users, the statistical values including statistical values about the priority of user posture information, and setting the audio content to be presented according to the highest priority posture information among the posture information of the multiple second users.
  • An interactive audio presentation system comprising: a control-side device for interactive audio presentation, configured to receive posture information of an audio presentation user, and set audio content to be presented based on the posture information of the audio presentation user; and a receiving-side device for interactive audio presentation, configured to receive relevant information of the audio content to be presented, and present the audio content, wherein presenting the audio content includes presenting the audio content to the user in a tactile manner.
  • EEE 25 The system according to EEE 24, wherein the receiving side device is further configured to: obtain the user's posture information and send the posture information to the controlling side device.
  • EEE 26 A system according to EEE 24, wherein the control side device is further configured to: obtain posture information of multiple users, and set the audio content to be presented based on the statistical values of the posture information of the multiple users.
  • An interactive audio presentation method comprising: receiving posture information of an audio presentation user, setting audio content to be presented based on the posture information of the audio presentation user; and presenting the audio content, wherein presenting the audio content includes presenting the audio content to the user in a tactile manner.
  • EEE 28 A device comprising at least one processor; and at least one storage device storing instructions thereon that, when executed by the at least one processor, cause the at least one The processor executes a method according to any one of EEE 18-23 and 27.
  • EEE 29 A storage medium storing instructions that, when executed by a processor, enable the processor to execute a method according to any one of EEE 18-23 and 27.
  • EEE 30 A computer program product comprising instructions which, when executed by a processor, enable the processor to perform a method according to any one of EEEs 18-23 and 27.
  • EEE 31 A computer program comprising instructions which, when executed by a processor, enable the processor to perform a method according to any one of EEE 18-23 and 27.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

La présente divulgation concerne un procédé et un dispositif de présentation audio. Est proposé un dispositif côté réception utilisé pour une présentation audio interactive, comprenant un circuit de traitement qui est configuré pour recevoir, en provenance d'un dispositif côté commande utilisé pour une présentation audio interactive, des informations relatives à un contenu audio devant être présenté, dans lequel le contenu audio devant être présenté comprend un ensemble de contenus audio sur la base d'informations de posture d'utilisateur, et présenter le contenu audio, dans lequel la présentation du contenu audio consiste à présenter le contenu audio d'une manière tactile.
PCT/CN2023/138019 2022-12-12 2023-12-12 Procédé et dispositif de présentation audio Ceased WO2024125478A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202380083647.2A CN120322747A (zh) 2022-12-12 2023-12-12 音频呈现方法和设备

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211599599.6 2022-12-12
CN202211599599.6A CN118226946A (zh) 2022-12-12 2022-12-12 音频呈现方法和设备

Publications (1)

Publication Number Publication Date
WO2024125478A1 true WO2024125478A1 (fr) 2024-06-20

Family

ID=91484384

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/138019 Ceased WO2024125478A1 (fr) 2022-12-12 2023-12-12 Procédé et dispositif de présentation audio

Country Status (2)

Country Link
CN (2) CN118226946A (fr)
WO (1) WO2024125478A1 (fr)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110228962A1 (en) * 2008-09-19 2011-09-22 National University Of Singapore Haptic Chair Sound Enhancing System With Audiovisual Display
US20150103154A1 (en) * 2013-10-10 2015-04-16 Sony Corporation Dual audio video output devices with one device configured for the sensory impaired
US20190373355A1 (en) * 2018-05-30 2019-12-05 Bose Corporation Audio eyeglasses with gesture control
CN112817557A (zh) * 2021-02-08 2021-05-18 海信视像科技股份有限公司 一种基于多人手势识别的音量调节方法及显示设备

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110228962A1 (en) * 2008-09-19 2011-09-22 National University Of Singapore Haptic Chair Sound Enhancing System With Audiovisual Display
US20150103154A1 (en) * 2013-10-10 2015-04-16 Sony Corporation Dual audio video output devices with one device configured for the sensory impaired
US20190373355A1 (en) * 2018-05-30 2019-12-05 Bose Corporation Audio eyeglasses with gesture control
CN112817557A (zh) * 2021-02-08 2021-05-18 海信视像科技股份有限公司 一种基于多人手势识别的音量调节方法及显示设备

Also Published As

Publication number Publication date
CN120322747A (zh) 2025-07-15
CN118226946A (zh) 2024-06-21

Similar Documents

Publication Publication Date Title
US11625994B2 (en) Vibrotactile control systems and methods
US8987571B2 (en) Method and apparatus for providing sensory information related to music
US10290291B2 (en) Information processing apparatus, method, and program for controlling output of a processing pattern in association with reproduced content
US20200243055A1 (en) Method and System for Musical Communication
CN104700860B (zh) 律动图像化方法及系统
JP2010092065A (ja) ローカライズされたオーディオ・ネットワークおよび関連するディジタル・アクセサリ
Hunt et al. Multiple media interfaces for music therapy
CN106383676B (zh) 用于声音的即时光色渲染系统及其应用
JP2014123085A (ja) カラオケにおいて歌唱に合わせて視聴者が行う身体動作等をより有効に演出し提供する装置、方法、およびプログラム
Clarke Rhythm/body/motion: Tricky's contradictory dance music
JP2023025013A (ja) 音楽療法のための歌唱補助装置
JP7343129B2 (ja) アプリケーション制御プログラム、アプリケーション制御システム及びアプリケーション制御方法
US20260094509A1 (en) Vibrotactile control systems and methods
WO2022163137A1 (fr) Dispositif de traitement d'informations, procédé de traitement d'informations et programme
WO2024125478A1 (fr) Procédé et dispositif de présentation audio
CN109151515B (zh) 表演场景下的互动系统和方法
CN114639394B (zh) 一种虚拟演奏伙伴的实现方法和装置
CN116149476A (zh) 音乐响应控制方法、智能眼镜及计算机可读存储介质
WO2025121219A1 (fr) Appareil, procédé et programme de traitement d'informations
US20230237981A1 (en) Method and apparatus for implementing virtual performance partner
Turchet Musical haptics for the listener, Part I: tactile music enhancement systems for hearing and DHH users
Hólmgeirsson Enhancing the Performer-Spectator Communication at Electronic Concerts
Civit et al. A Framework for AI assisted Musical Devices
JP2023174364A (ja) カラオケ装置
JP2026068428A (ja) システム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23902674

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 202380083647.2

Country of ref document: CN

NENP Non-entry into the national phase

Ref country code: DE

WWP Wipo information: published in national office

Ref document number: 202380083647.2

Country of ref document: CN

122 Ep: pct application non-entry in european phase

Ref document number: 23902674

Country of ref document: EP

Kind code of ref document: A1