WO2020155976A1 - 一种音频信号处理方法及装置 - Google Patents

一种音频信号处理方法及装置 Download PDF

Info

Publication number
WO2020155976A1
WO2020155976A1 PCT/CN2019/128908 CN2019128908W WO2020155976A1 WO 2020155976 A1 WO2020155976 A1 WO 2020155976A1 CN 2019128908 W CN2019128908 W CN 2019128908W WO 2020155976 A1 WO2020155976 A1 WO 2020155976A1
Authority
WO
WIPO (PCT)
Prior art keywords
audio signal
listening
audio signals
information
audio
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/128908
Other languages
English (en)
French (fr)
Inventor
王宾
吉布斯•乔纳森•阿拉斯泰尔
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to EP19912993.3A priority Critical patent/EP3909656A4/en
Priority to KR1020217027277A priority patent/KR102696575B1/ko
Priority to BR112021015040-4A priority patent/BR112021015040B1/pt
Publication of WO2020155976A1 publication Critical patent/WO2020155976A1/zh
Priority to US17/386,800 priority patent/US20210358506A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/30—Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
    • A63F13/35—Details of game servers
    • A63F13/355—Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/50—Controlling the output signals based on the game progress
    • A63F13/54—Controlling the output signals based on the game progress involving acoustic signals, e.g. for simulating revolutions per minute [RPM] dependent engine sounds in a driving game or reverberation against a virtual wall
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G10L25/60—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination for measuring the quality of voice signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/239—Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests
    • H04N21/2393—Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests involving handling client requests
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/438—Interfacing the downstream path of the transmission network originating from a server, e.g. retrieving encoded video stream packets from an IP network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/439—Processing of audio elementary streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/4728—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/8106—Monomedia components thereof involving special audio data, e.g. different tracks for different languages
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
    • H04S7/30—Control circuits for electronic adaptation of the sound field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
    • H04S7/30—Control circuits for electronic adaptation of the sound field
    • H04S7/302—Electronic adaptation of stereophonic sound system to listener position or orientation
    • H04S7/303—Tracking of listener position or orientation
    • H04S7/304—For headphones
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/60—Methods for processing data by generating or executing the game program
    • A63F2300/6063—Methods for processing data by generating or executing the game program for sound processing
    • A63F2300/6081—Methods for processing data by generating or executing the game program for sound processing generating an output signal, e.g. under timing constraints, for spatialization
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
    • H04S2400/01—Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
    • H04S2400/15—Aspects of sound capture and related signal processing for recording or reproduction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
    • H04S2420/01—Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
    • H04S7/30—Control circuits for electronic adaptation of the sound field
    • H04S7/302—Electronic adaptation of stereophonic sound system to listener position or orientation
    • H04S7/303—Tracking of listener position or orientation

Definitions

  • the embodiments of the present application relate to the field of audio signal processing, and in particular, to an audio signal processing method and device.
  • immersive audio can meet people's needs in this regard.
  • 4G 4th generation mobile communication technology
  • 5G 5th generation mobile communication technology
  • VR virtual reality
  • AR Augmented reality
  • MR Mixed Reality
  • An immersive audio-visual system requires not only stunning visual effects, but also realistic auditory effects. The fusion of audio-visual can greatly improve the immersive experience.
  • the core of audio is three-dimensional audio technology.
  • There are many playback methods for three-dimensional audio including speaker-based speaker playback and earphone-based binaural playback.
  • the earphone-based binaural playback is mainly realized by head-related transfer function (HRTF) or binaural room impulse response (BRIR).
  • HRTF head-related transfer function
  • BRIR binaural room impulse response
  • BRIR takes into account the impact of environmental reflections on the sound source for reproduction BRIR can be regarded as the impulse response of a system composed of sound source, indoor environment, and binaural (including head, torso, and pinna), which is composed of direct sound, early reflection, and late reverberation.
  • the principle of earphone-based binaural playback is: when the sound source is assumed to be at a certain position, select the HRTF or BRIR corresponding to the center position of the listener's head from this position to perform convolution processing to obtain the binaural output signal.
  • the corresponding device is selected according to the application scenario to collect the audio signals corresponding to all useful sound sources in the sound field, and all the audio signals collected are encoded, and then transmitted to the decoding end and decoded according to the playback device selected by the player end Perform rendering.
  • the power consumption of the decoding end is large.
  • the embodiments of the present application provide an audio signal processing method and device, which reduce decoding power consumption under the premise of ensuring the quality of the audio signal.
  • the first aspect of the present application provides an audio signal processing method applied to the encoding end.
  • the method may include: first collecting audio signals at the listening position, each collected audio signal has a corresponding sound source, and the collected audio signal It includes at least audio signals that meet the filtering conditions; where the filtering conditions may include one or more of the following conditions: the volume at the listening position is greater than or equal to the volume threshold, or the corresponding sound source and listening position The listening distance is less than or equal to the preset listening distance, or the corresponding sound source is within the range corresponding to the listening direction; then the collected audio signal is encoded to obtain the encoded audio signal; and then the encoded audio signal is sent to the decoder Audio signal.
  • the audio signal processing method provided by the present application, at least the audio signal that meets the screening condition at the listening position is collected, and the collected audio signal is encoded and sent to the decoder for decoding and playing the audio signal.
  • the encoder can only collect audio signals that meet the screening conditions and encode and send them, so that the decoder can only decode audio signals that meet the screening conditions. Therefore, it is possible to configure screening conditions to ensure the quality of the audio signal.
  • the transmission of the audio signal not only ensures the quality of the audio signal, but also greatly reduces the power consumption of the decoding end.
  • the solution provided by this application can be applied to the sound field, of course, can also be used in other application scenarios, such as game scenes, movie scenes, conference scenes, and so on.
  • the encoding end described in this application refers to the part of the audio-visual system that encodes the audio signal corresponding to the sound source.
  • the encoding end collects and encodes the audio signal corresponding to the sound source, transmits it to the decoding end, and the decoding end decodes and renders it before playing.
  • audio-visual systems include but are not limited to VR systems, AR systems, MR systems, or other streaming systems. This application does not specifically limit the type of audio-visual system to which the solution provided in this application is applied.
  • audio signal processing includes real-time audio signal processing scenarios and non-real-time audio signal processing scenarios. No matter in which scene, due to the mobility of the user, the listening position is not fixed, and the listening direction is not fixed.
  • the range corresponding to the listening direction may include an omnidirectional or a preset angle range, which is not specifically limited in this application.
  • the listening position is the current position of the listener in the sound field, and the listening orientation refers to the direction the listener faces at the listening position.
  • the listening position and/or listening orientation are from the decoding end to the encoding end
  • the encoding end processes the audio signal by collecting and transmitting, so as to realize the realistic restoration of the sound field and make the listener immersive.
  • the listening position is any possible position in the sound field, and the listening direction is any possible direction.
  • the audio signal processing method provided in this application may further include: receiving listening position information sent by the decoding end, where the listening position information is used for Indicates the listening position.
  • the audio signal processing method provided by the present application may further include: receiving listening orientation information sent by the decoding end, where the listening orientation information is used to indicate the listening orientation of the listener.
  • multiple listening positions are preset or determined, and there are multiple listening positions in each listening position. Listening direction, the encoding end needs to execute the audio signal processing method provided in this application at each listening position to collect and encode audio signals. The decoding end selects the matched encoded audio signal for decoding and rendering according to the actual listener's position, so as Make the listener immersive.
  • the listening orientation may be any listening orientation at any listening position.
  • the filtering conditions include content related to the listening direction
  • the encoding end needs to obtain the audio signal corresponding to any listening direction, so that the decoding end can obtain the audio signal that meets the current filtering conditions from the encoding end, so as to restore the sound field on the decoding end;
  • Multiple audio signals of listening orientations so that when the listener is in any listening orientation, the decoder can obtain the audio signal corresponding to the listener's current listening orientation from the encoding end, thereby making the listener immersive.
  • the audio signal whose volume exceeds a certain threshold can be heard, and when the positional relationship between the sound source and the listening direction is different, this
  • the thresholds are different, and the volume threshold can be configured as a threshold determined by the positional relationship between the sound source corresponding to the audio signal and the listening direction.
  • different position relationships are configured with different thresholds, which more accurately adapts to listening requirements, and makes the listening effect more realistic.
  • the audio signal whose volume exceeds a certain threshold can be heard, and the volume threshold can be configured as a preset volume threshold to collect the volume Large sound source signal.
  • the audio signal processing method provided in this application may further include: the encoding terminal collects audio signals corresponding to all sound sources at the listening position, Encode all the collected audio signals to obtain the encoded audio signal and send it to the decoding end, and send instruction information to the decoding end.
  • the instruction information is used to indicate the audio signals that meet the filtering conditions among the encoded audio signals. According to the instruction information, only the audio signals that meet the screening conditions are decoded, so that the decoding end only decodes the audio signals corresponding to the sound sources that meet the screening conditions at the listening position.
  • the indication information may include: feature information of each audio signal in the encoded audio signal; wherein, the feature information includes volume information Or the location information of the sound source corresponding to the audio signal.
  • the decoder selects audio signals that meet the screening conditions for decoding according to the characteristic information.
  • the indication information may include: the identifier of the audio signal whose encoded audio signal meets the screening condition.
  • an audio signal processing method is provided, which is applied to a decoding end.
  • the method may include: receiving a plurality of encoded audio signals, the audio signals included in the plurality of audio signals are audio signals at a listening position , Each audio signal has a corresponding sound source, and the multiple audio signals include at least audio signals that meet the screening conditions; wherein the screening conditions include one or more of the following conditions: the volume at the listening position is greater than or equal to Volume threshold, or the listening distance between the corresponding sound source and the listening position is less than or equal to the preset listening distance, or the corresponding sound source is within the range corresponding to the listening direction; decode the audio signal that meets the filtering conditions to obtain Decode the signal; render the decoded signal according to the channel transfer function in the playback device to obtain the output signal.
  • the decoder only decodes the audio signals corresponding to the sound sources that meet the filtering conditions.
  • the filtering conditions are configured to ensure the audio quality, when the number of sound sources is relatively large, the transmission of audio signals can ensure audio
  • the signal quality greatly reduces the power consumption of the decoder.
  • rendering the decoded signal according to the channel transfer function in the playback device to obtain the output signal includes: connecting each decoded signal to its channel transfer function in the playback device. Perform convolution to obtain one or more output signals.
  • the audio signal processing method provided in this application may further include: a decoding end to an encoding end Sending the listening position information, so that the encoding terminal obtains multiple encoded audio signals according to the listening position information, and the listening position information is used to indicate the listening position.
  • the audio signal processing method provided by the present application may further include: the decoding end sends listening orientation information to the encoding end, so that the encoding end obtains multiple encoded pieces of information according to the orientation information. Audio signal, the listening direction information is used to indicate the listening direction.
  • the listening position is an application scenario, such as any position in the sound field. Therefore, in non-real-time audio signal processing scenarios, multiple listening positions need to be preset or determined. There are multiple listening directions at each listening position. The encoder needs to collect audio signals at each listening position where the audio signal processing method provided in this application is executed.
  • the decoder can obtain the corresponding audio signal from the encoder according to the listener’s current listening position, execute the audio signal processing method provided by this application on the decoder to restore the sound field, and obtain the audio signal of each listening position in advance, so that the listener At any position, the decoder can obtain the encoded audio signal at the current listening position from the encoding end for decoding, so that the listener is on the scene.
  • the listening orientation may be any listening orientation at any listening position.
  • the filtering conditions include content related to the listening direction
  • the encoding end needs to obtain the encoded audio signal corresponding to any listening direction, so that the decoding end can obtain the encoded audio signal that meets the filtering conditions from the encoding end, thereby restoring the sound field on the decoding end ;
  • the audio signal whose volume exceeds a certain threshold can be heard, and when the positional relationship between the sound source and the listening direction is different, this The threshold is different, and the volume threshold can be configured as a threshold determined by the positional relationship between the sound source and the listening direction. In this implementation manner, different positional relationships are configured with different volume thresholds, which can more accurately adapt to the listening requirements and also make the listening effect more realistic.
  • the audio signal whose volume exceeds a certain threshold can be heard, and the volume threshold can be configured as a preset volume threshold to collect the volume Large sound source signal.
  • the encoded multiple audio signals only include audio signals that meet the filtering conditions at the listening position;
  • the sound source signal is decoded to obtain a decoded signal, which is specifically implemented as follows: the decoder decodes each audio signal among the multiple audio signals to obtain a decoded signal.
  • the encoded multiple audio signals include audio signals corresponding to all sound sources at the listening position.
  • the audio signal processing provided in this application is The method may further include: the decoding end receives instruction information sent by the encoding end, where the instruction information is used to indicate a sound source signal that satisfies the filtering condition among the encoded multiple audio signals; the decoding end obtains information from the multiple encoded audio signals according to the instruction information. The audio signal is screened, and the audio signal indicated by the indication information is obtained as an audio signal that meets the screening condition.
  • the indication information may include: feature information of each audio signal in the encoded multiple audio signals; wherein, the feature information includes Volume information or location information of the sound source corresponding to the audio signal.
  • the decoder selects audio signals that meet the screening conditions for decoding according to the characteristic information.
  • the indication information may include: an identifier of an audio signal that meets the screening condition among the encoded multiple audio signals.
  • the decoder selects the audio signal indicated by the identifier from the multiple encoded audio signals as the audio signal that meets the screening condition.
  • the channel transmission function may include: an HRTF function or a BRIR function. This application does not specifically limit the type of channel transmission function.
  • the decoding end decodes the audio signal that meets the filtering conditions to obtain the decoded signal, which specifically includes: The decoder downloads the audio signal that meets the screening conditions, and decodes the downloaded audio signal to obtain a decoded signal.
  • a third aspect of the present application provides an audio signal processing device, which may include: a collection unit, an encoding unit, and a sending unit.
  • the collection unit is used to collect audio signals at the listening position, each collected audio signal has a corresponding sound source, and the collected audio signals include at least audio signals that meet the screening conditions; wherein the screening conditions may include the following One or more of the conditions: the volume at the listening position is greater than or equal to the volume threshold, or the listening distance between the corresponding sound source and the listening position is less than or equal to the preset listening distance, or the corresponding sound source is listening Toward the corresponding range; the encoding unit is used to encode the collected audio signal to obtain the encoded audio signal; the sending unit is used to send the encoded audio signal to the decoding end.
  • the audio signal that meets the filtering conditions at the listening position is collected, so that the decoder only decodes the audio signals that meet the filtering conditions.
  • the filtering conditions to ensure the quality of the audio signals.
  • the audio signal processing device provided in the third aspect of the present application is used to implement the audio signal processing method provided in the first aspect and any possible implementation manner of the first aspect. Therefore, the third aspect of the present application
  • the specific implementation of the provided audio signal processing device reference may be made to the foregoing first aspect and the specific implementation of the audio signal processing method provided by any one of the possible implementation manners of the first aspect, and details are not described herein again.
  • a fourth aspect of the present application provides another audio signal processing device, which may include a receiving unit, a decoding unit, and a rendering unit.
  • the receiving unit is configured to receive multiple encoded audio signals
  • the multiple audio signals are audio signals at the listening position
  • each audio signal has a corresponding sound source
  • the multiple audio signals include at least those that meet the filtering conditions.
  • the filtering conditions include one or more of the following conditions: the volume at the listening position is greater than or equal to the volume threshold, or the listening distance between the corresponding sound source and the listening position is less than or equal to the preset listening distance, Or, the corresponding sound source is in the range corresponding to the listening direction; the decoding unit decodes the audio signal that meets the filtering conditions received by the receiving unit to obtain the decoded signal; the rendering unit is used to transfer the function according to the channel in the playback device , Rendering the decoded signal obtained by the decoding unit to obtain the output signal.
  • the decoder only decodes the audio signals corresponding to the sound sources that meet the filtering conditions in the sound field. As long as the filtering conditions are configured to ensure the audio quality, the audio signal is transmitted when the number of sound sources in the sound field is relatively large. This not only guarantees the audio signal quality, but also greatly reduces the power consumption of the decoder.
  • the audio signal processing device provided in the fourth aspect of the present application is used to implement the audio signal processing method provided in the second aspect and any possible implementation manner of the second aspect. Therefore, the fourth aspect of the present application
  • the present application provides an audio signal processing device, which can implement the function of the encoding end in the above method example.
  • the function can be implemented by hardware, or the corresponding software can be executed by hardware.
  • the hardware or software includes one or more modules corresponding to the aforementioned functions.
  • the audio signal processing device can exist in the form of a chip product.
  • the structure of the audio signal processing device includes a processor and a transceiver, and the processor is configured to support the audio signal processing device to perform corresponding functions in the foregoing method.
  • the transceiver is used to support the communication between the audio signal processing device and other devices.
  • the audio signal processing device may further include a memory, which is used for coupling with the processor and stores necessary program instructions and data of the audio signal processing device.
  • the present application provides another audio signal processing device, which can implement the function of the decoding end in the above method example.
  • the function can be implemented by hardware, or can be executed by hardware.
  • Software Implementation The hardware or software includes one or more modules corresponding to the aforementioned functions.
  • the audio signal processing device can exist in the form of a chip product.
  • the structure of the audio signal processing device includes a processor and a transceiver, and the processor is configured to support the audio signal processing device to perform corresponding functions in the foregoing method.
  • the transceiver is used to support the communication between the audio signal processing device and other devices.
  • the audio signal processing device may further include a memory, which is used for coupling with the processor and stores necessary program instructions and data of the audio signal processing device.
  • a seventh aspect of the present application provides an encoder, which includes an audio signal processing device that executes the audio signal processing method provided in the first aspect or any one of the possible implementation manners of the first aspect.
  • the encoder can be used by the encoding terminal in the embodiment of the present application.
  • An eighth aspect of the present application provides a decoder, which includes an audio signal processing device that executes the audio signal processing method provided by the second aspect or any one of the possible implementation manners of the second aspect.
  • the decoder can be used by the decoder in the embodiment of the present application.
  • a ninth aspect of the present application provides an audio signal processing system, including the encoder provided in the seventh aspect, and/or the decoder provided in the eighth aspect.
  • a tenth aspect of the present application provides a computer-readable storage medium, including instructions, which when run on a computer, cause the computer to execute the audio signal processing method provided by any one of the foregoing aspects or any possible implementation manner.
  • the eleventh aspect of the present application provides a computer program product containing instructions, which when run on a computer, enables the computer to execute the audio signal processing method provided by any one of the foregoing aspects or any possible implementation manners.
  • FIG. 1 is a schematic diagram of an audio signal processing method provided by the prior art
  • FIG. 2 is a schematic diagram of the architecture of an audio-visual system provided by the prior art
  • FIG. 3 is a schematic structural diagram of an audio signal processing device provided by an embodiment of the application.
  • FIG. 4 is a schematic flowchart of an audio signal processing method provided by an embodiment of this application.
  • FIG. 5 is a schematic flowchart of another audio signal processing method provided by an embodiment of the application.
  • FIG. 6 is a schematic structural diagram of another audio signal processing device provided by an embodiment of the application.
  • FIG. 7 is a schematic structural diagram of still another audio signal processing device provided by an embodiment of the application.
  • FIG. 8 is a schematic structural diagram of yet another audio signal processing device provided by an embodiment of the application.
  • FIG. 9 is a schematic structural diagram of another audio signal processing device provided by an embodiment of the application.
  • words such as “exemplary” or “for example” are used as examples, illustrations, or illustrations. Any embodiment or design solution described as “exemplary” or “for example” in the embodiments of the present application should not be construed as being more preferable or advantageous than other embodiments or design solutions. To be precise, words such as “exemplary” or “for example” are used to present related concepts in a specific manner.
  • A, B, and C described in the embodiments of this application are used to represent the following concepts: A, or B, or C, or A and B, or A and C, or B And C, or A, B, and C.
  • the conventional audio signal processing process includes: the encoding end device selects the corresponding device according to the application scenario to collect the audio signals corresponding to all the sound sources in the current sound field, encodes all the collected audio signals, and transmits them to the decoding end; the decoding end responds to the received Decode and render all audio signals to obtain output signals. If the playback device is a speaker array, the output signal is rendered as a multi-channel speaker signal; if the playback device is a headphone device, the output signal is rendered as a binaural output signal.
  • the codec can use the current common enhanced voice service (enhance voice services, EVS), adaptive multi-rate wideband (AMRWB), adaptive multi-rate (Adaptive Multi-Rate, AMR), etc.
  • EVS enhanced voice services
  • AMRWB adaptive multi-rate wideband
  • AMR adaptive Multi-Rate
  • the multi-channel speaker signal is rendered by the renderer according to the specific number of speakers
  • the binaural output signal is rendered by the renderer using the multi-channel speaker signal and HRTF/BRIR.
  • the encoding methods used in the aforementioned encoding and decoding are only examples, and other audio encoding methods can also be used in practical applications.
  • Fig. 1 illustrates the current audio signal processing process. Taking 5 sound sources (1-5 in Fig. 1) in the sound field as an example, the current audio signal processing process will be described in conjunction with Fig. 1.
  • the audio signals corresponding to the 5 sound sources are recorded as S1(n), S2(n), S3(n), S4(n), S5(n), respectively, after EVS encoding
  • Obtain 5 code stream signals S1S(n), S2S(n), S3S(n), S4S(n), S5S(n) connect the 5 code stream signals in a frame-to-frame adjacent manner after channel coding Send it out.
  • the decoder After receiving the coded stream signal, the decoder first performs channel decoding to obtain 5 code streams S1S(n), S2S(n), S3S(n), S4S(n), S5S(n), and the code stream passes through EVS The decoder gets 5 signals DS1(n), DS2(n), DS3(n), DS4(n), DS5(n) after decoding.
  • the 5 signals are respectively corresponding to the HRTF signal (HRTF1(n), HRTF2(n), HRTF3(n), HRTF3(n), HRTF4(n)) of the channel transfer function (HRTF function) selected by the listening device Perform convolution processing and get the convolution result as Y1(n), Y2(n), Y3(n), Y4(n), Y5(n).
  • HRTF function channel transfer function
  • Y(n) Y1(n)+Y2(n)+Y3(n)+Y4(n)+Y5(n).
  • the encoding end collects and encodes all audio signals corresponding to all sound sources to the decoding end, and the decoding end decodes and renders all received audio signals.
  • the power consumption of the decoding end is very large.
  • the embodiments of the present application provide an audio signal processing method, the basic principle of which is that the decoder selects audio signals that meet the filtering conditions for decoding, instead of decoding the audio corresponding to all sound sources in the prior art.
  • Signal decoding reduces the power consumption of the decoding end when there are many sound sources.
  • the filtering conditions are properly configured, while reducing power consumption, the quality of audio signal transmission can still be well guaranteed to ensure the listening effect.
  • the position of the listener refers to the position of the listener; the listening direction refers to the direction the listener is facing at the listening position.
  • the listener can be a user who actually uses the playback device or a virtual user who uses the playback device.
  • a user who virtually uses the playback device can be in any possible listening position, and can have any possible listening orientation.
  • the listening distance refers to the linear distance between the sound source and the listening position.
  • the listening distance may also be the distance of a preset path between the sound source and the listening position.
  • the preset path may be the distance to bypass the obstacle or others, which is not specifically limited in this application.
  • the audio signal processing method provided in this application is applied to the audio-visual system 10 shown in FIG. 2.
  • the audio-visual system 10 includes an end-to-end audio signal processing part and an end-to-end video signal processing part.
  • the audio-visual system 10 may be a VR system, an AR system, or an MR system or other streaming systems.
  • the embodiment of the present application does not specifically limit the actual form of the audiovisual system 10.
  • the end-to-end audio signal processing part of the audio-visual system 10 may include an acquisition module (acquisition) 201, an audio preprocessing module (audio preprocessing) 202, an audio encoding module (audio encoding) 203, and an encapsulation module (file/ segment encapsulation) 204, transmission module (delivery) 205/218, decapsulation module (file/segment decapsulation) 206, audio decoding module (audio decoding) 207, audio rendering module (audio rendering) 208, and speakers/headphones (loudspeakers/headphones) ) 209.
  • the end-to-end video signal processing part included in the audio-visual system 10 may include: an acquisition module (acquisition) 201, a video combination module (visual stitching) 210, a predictive drawing module (projection and mapping) 211, and video coding Module (video encoding) 212, image encoding module (image encoding) 213, encapsulation module (file/segment encapsulation) 204, transmission module (delivery) 205/218, decapsulation module (file/segment decapsulation) 206, video decoding module ( video decoding 214, image decoding module 215, video rendering module (visual rendering) 216, and player (display) 217.
  • end-to-end audio signal processing part and the end-to-end video signal processing part included in the audio-visual system 10 are shared modules.
  • the collection module 201 is used to collect the audio signal corresponding to the sound source and transmit the audio signal to the audio preprocessing module 202.
  • the audio preprocessing module 202 is configured to preprocess the audio signal, for example, filter processing, etc., and transmit the preprocessed audio signal to the audio encoding module 203.
  • the audio encoding module 203 is configured to encode the preprocessed audio signal, and transmit the encoded audio signal to the packaging module 204.
  • the collection module 201 is also used to collect video signals. After the video signal is processed by the video combining module 210, the predictive drawing module 211, the video encoding module 212, and the image encoding module 213, the encoded video signal is transmitted to the packaging module 204.
  • the encapsulation module 204 is configured to encapsulate the encoded audio signal and the encoded video signal to obtain a code stream, which is transmitted to the transmission module 218 of the decoding end through the transmission module 205, and then transmitted to the decapsulation module 206 by the transmission module 218.
  • the transmission module 205 and the transmission module 218 may be a wired communication module or a wireless communication module, which is not specifically limited in the embodiment of the present application.
  • the transmission module 205 can be specifically implemented in the form of a server, that is, the encoding end uploads the code stream to the server, and the decoding end downloads the code stream from the server as required to achieve The function of the transmission module 205, this process will not be repeated.
  • the decapsulation module 206 is used to decapsulate the code stream to obtain an encoded audio signal and an encoded video signal, and transmit the encoded audio signal to the audio decoding module 207, and transmit the encoded video signal to the video decoder Module 214 and image decoding module 215.
  • the audio decoding module 207 is configured to decode the encoded audio signal and transmit the decoded audio signal to the audio rendering module 208.
  • the audio rendering module 208 is used for rendering the decoded audio signal and transmitting the rendered output signal to the speaker/headphone 209.
  • the video decoding module 214, the image decoding module 215, and the video rendering module 216 process the encoded video signal, and transmit the processed video signal to the player 217 for playback.
  • the structure shown in FIG. 2 does not constitute a limitation on the audio-visual system 10, and may include more or less components than shown in the figure, or a combination of certain components, or different component arrangements.
  • the audio-visual system 10 may also include a sensor, etc., which are used to obtain the listening position and/or listening orientation, which will not be repeated here.
  • end-to-end video signal processing part and the end-to-end audio signal processing system in the audio-visual system 10 shown in FIG. 2 can be deployed independently, and will not be repeated in this application.
  • FIG. 3 shows an audio signal processing device 30 related to various embodiments of the application.
  • the audio signal processing device 30 can be deployed on the encoding end of the audiovisual system 10 as a functional unit, or can be deployed on the decoding end of the audiovisual system 10 as a functional unit, which is not specifically limited in the embodiment of the present application.
  • the audio signal processing device 30 may also be an encoding end or a decoding end in an audio-visual system.
  • the audio signal processing device 30 may be a chip system. In the embodiments of the present application, the chip system may be composed of chips, or may include chips and other discrete devices.
  • the audio signal processing device 30 may include: a processor 301, a memory 302, and a transceiver 303.
  • the components of the audio signal processing device 30 are specifically introduced below in conjunction with FIG. 3:
  • the memory 302 may be a volatile memory (volatile memory), such as RAM; or a non-volatile memory, such as read-only memory (ROM), flash memory (flash memory), HDD or SSD; or the above
  • volatile memory such as RAM
  • non-volatile memory such as read-only memory (ROM), flash memory (flash memory), HDD or SSD; or the above
  • the processor 301 is the control center of the audio signal processing device 30, and can be a CPU, or an ASIC, or one or more integrated circuits configured to implement the embodiments of the present application, such as one or more DSPs, or , One or more FPGAs.
  • the processor 301 may execute various functions of the audio signal processing apparatus 30 by running or executing software programs and/or modules stored in the memory 302, and calling data stored in the memory 302.
  • the transceiver 303 is used for the audio signal processing device 30 to interact with other units.
  • the transceiver 303 may be a transceiver port of the audio signal processing device 30.
  • the audio signal processing device 30 may further include a sensor 304 for acquiring the listening position and/or listening orientation.
  • the sensor 304 may be a gyroscope, an external camera, a motion detection device or an image detection device, etc., which is not limited in the embodiment of the present application.
  • the processor 301 runs or executes the software programs and/or modules stored in the memory 302, and calls the storage
  • the data in the memory 302 performs the following functions:
  • each collected audio signal has a corresponding sound source, and the collected audio signal includes at least the audio signal that meets the filtering conditions;
  • the filtering conditions can include one or more of the following conditions: The volume at the listening position is greater than or equal to the volume threshold, or the listening distance between the corresponding sound source and the listening position is less than or equal to the preset listening distance, or the corresponding sound source is within the range corresponding to the listening direction; the audio to be collected
  • the signal is encoded to obtain an encoded audio signal; the encoded audio signal is sent to the decoding end through the transceiver 303.
  • the method, device, device or system provided in the embodiments of the present application can be applied to real-time audio signal processing scenarios and non-real-time audio signal processing scenarios.
  • the listening position is the decoding end, that is, the position where the listener is currently located.
  • the position where the listener is currently located may be the range of the game scene. Sound field range, movie scene range, meeting scene range, etc.
  • the position within the application scene range; in order to better reproduce the audio of the encoding end for the listener at the decoding end, the encoding end can be based on the current position of the listener sent by the decoding end (I.e.
  • the listening position to collect audio signals, so that the audio collected by the encoder can match the current position of the listener; when applied to non-real-time audio signal processing scenes, the listening position is the preset position.
  • the user may be in any position within the scope of the application scene, so some listening positions can be preset.
  • the encoder can collect audio signals according to the preset listening positions.
  • the decoder can decode the audio signal according to the listening position.
  • the audio signal with the best match for example, the same or the closest position
  • the processor 301 runs or executes the software programs and/or modules stored in the memory 302 and calls the storage
  • the data in the memory 302 performs the following functions:
  • the multiple audio signals are audio signals at the listening position, each audio signal has a corresponding sound source, and the multiple audio signals include at least audio signals that meet the screening conditions;
  • the filtering conditions may include one or more of the following conditions: the volume at the listening position is greater than or equal to the volume threshold, or the listening distance between the corresponding sound source and the listening position is less than or equal to the preset listening distance, or, The corresponding sound source is in the range corresponding to the listening direction; the audio signal that meets the filtering condition is decoded to obtain the decoded signal; the decoded signal is rendered according to the channel transmission function in the playback device, and the output signal is obtained.
  • the embodiment of the present application provides an audio signal processing method, which is applied to the interaction between the encoding end and the decoding end of the end-to-end audio signal processing part of the audio-visual system 10 shown in FIG. 2 to process audio signals in the sound field.
  • the encoding end/decoding end described below performs a certain operation, which can be performed for the encoding end/decoding end itself, or can be performed for the functional modules in the encoding end/decoding end, and will not be described one by one in the following.
  • the two can directly communicate and interact, or they can interact by uploading and downloading interactive content to a shared medium.
  • This application is uniformly described as sending/ Receive, follow-up will not explain one by one.
  • the shared medium may be a server or a cloud server or others.
  • the audio signal processing method may include:
  • the encoding terminal obtains the listening position.
  • audio signal processing includes real-time audio signal processing scenarios and non-real-time audio signal processing scenarios. No matter in which scene, the listening position is not fixed due to the user's mobility.
  • S401 is described below in real-time audio signal processing scenarios and non-real-time audio signal processing scenarios.
  • the listening position is the current position of the listener in the sound field, and the listening orientation refers to the direction the listener faces at the listening position.
  • the listening position can be provided by the decoding end to the encoding end.
  • the audio signal processing method provided in the embodiment of the present application may further include S401a.
  • the decoding terminal obtains the listening position, and sends the listening position information to the encoding terminal.
  • the decoding end may use a sensor or a device for obtaining a position to obtain the listener's current position in the sound field.
  • the embodiment of the present application does not specifically limit the implementation of obtaining the listening position by the decoding end.
  • the listening position information is used to indicate the listening position obtained by the decoding end, and the content of the listening position information can be configured according to actual requirements, which is not limited in the embodiment of the present application.
  • the listening position information may be coordinate information, or a position identification.
  • the identification of each position in the sound field is pre-configured, and the specific listening position can be uniquely determined through the identification.
  • S401 can be specifically implemented as: the encoding end receives the listening position information from the decoding end, the listening position information is used to indicate the listening position, and the encoding end determines the position of the listener in the sound field according to the listening position information.
  • the decoding end may provide one or more listening positions to the decoding end, which is not specifically limited in the embodiment of the present application.
  • the listening position can be any position in the sound field. Therefore, in the non-real-time audio signal processing scene, the encoder needs to pre-collect the audio signals of multiple possible listening positions. At this time, the listening position It can be any of multiple preset positions.
  • S401 may be specifically implemented as: the encoding end determines each possible listening position in the sound field. Specifically, each possible listening position may be determined according to multiple preset positions, or each possible listening position may be determined by using a preset algorithm according to the situation of the current processing scene.
  • the audio signal processing method provided in the embodiment of the present application may only consider the listening position, and then S403 is executed after S401.
  • the audio signal processing method provided by the embodiment of the present application may refer to the listening orientation based on the listening position, so after S401, it is necessary to perform S402 and then S403.
  • the listening orientation refers to the direction the user faces at the listening position. Due to the mobility of the user, the listening orientation is not fixed.
  • the listening direction is provided by the decoding end to the encoding end. Therefore, before S402, as shown in FIG. 4, the audio signal processing method provided in the embodiment of the present application may further include S402a.
  • the decoding end obtains the listening direction, and sends the listening direction information to the encoding end.
  • the decoding end may use a sensor or a device for obtaining orientation to obtain the direction the listener is facing at the current position in the sound field as the listening orientation.
  • the embodiment of the present application does not implement the decoding end obtaining the listening orientation. Make specific restrictions.
  • the listening direction information is used to indicate the listening direction obtained by the decoding end, and the content of the listening direction information can be configured according to actual needs, which is not limited in the embodiment of the present application.
  • the listening orientation information may be an orientation indicator, or the listening orientation information may be an angle with the reference orientation.
  • S402 can be specifically implemented as: the encoding end receives the listening direction information from the decoding end, and the encoding end determines the listening direction of the listener in the sound field according to the listening direction information.
  • S401 and S402 can be implemented in combination, and this application does not specifically limit the process.
  • the listening orientation refers to every possible orientation of each listening position. Therefore, in the scene of non-real-time audio signal processing, there are multiple listening positions, and there are multiple listening positions in each listening position. Listening direction.
  • S402 may be specifically implemented as: the encoding end determines each listening orientation at each possible listening position in the sound field.
  • the encoding terminal collects audio signals at the listening position in the sound field, and the collected audio signals include at least audio signals that meet the screening conditions.
  • each audio signal collected in S403 has a corresponding sound source in the sound field.
  • the encoder only collects audio signals that meet the filtering conditions at the listening position in the sound field.
  • the encoding end executes S403 at each listening position.
  • the process of executing S403 at one listening position is described, but it is not specifically limited.
  • the encoding end performs S403 through at least one audio signal acquisition device.
  • the audio signal collection device may be a microphone or other devices, which is not specifically limited in the embodiment of the present application.
  • the filtering conditions may include one or more of the following conditions: the volume at the listening position is greater than or equal to the volume threshold, or the listening distance between the sound source corresponding to the audio signal and the listening position is less than or equal to the preset listening distance, or , The sound source corresponding to the audio signal is in the range corresponding to the listening direction.
  • the content of the filtering conditions can also be configured according to actual needs, which is not specifically limited in the embodiment of the present application.
  • audio signals whose volume exceeds a certain threshold can be heard, and the threshold is different when the positional relationship between the sound source and the listening direction is different, and the volume threshold can be configured to determine the positional relationship between the sound source and the listening direction.
  • the threshold can be configured to determine the positional relationship between the sound source and the listening direction.
  • volume thresholds corresponding to different position relationships may be pre-configured, and the volume threshold is determined according to the position relationship in S403.
  • the positional relationship may refer to at least one of a relative direction relationship and a distance relationship between the sound source and the listening direction.
  • the sound source is located forward or behind the listening direction, and the sound source is located on the left or right side of the listening direction.
  • the positional relationship may refer to the angular relationship between the sound source and the listening direction.
  • the sound source is located at 50 degrees to the right of the listening direction, and the sound source is located at 0 degrees to the right of the listening direction (in front).
  • the volume threshold corresponding to the position relationship is obtained according to a search method. For example, when the sound source is located 30 degrees to the right of the listening direction, and the linear distance between the sound source and the listening position is 0.35 meters, look up Table 1, and determine that the corresponding volume threshold is the threshold value 20.
  • Table 1 is only an example to illustrate a method for determining the volume threshold, and is not specifically limited.
  • a preset algorithm for the position relationship of the volume threshold can be pre-configured, and then the volume threshold can be obtained according to the preset algorithm according to the position relationship between the sound source and the listening position.
  • the volume threshold when a sound source whose volume exceeds a certain threshold can be heard, can be configured as a preset volume threshold to collect audio signals with loud volume. In this possible implementation, the influence of the listening orientation on the listening effect is not considered.
  • volume threshold may also be set by the listener and sent from the decoding end to the encoding end.
  • listening distance can also be set by the listener and sent from the decoding end to the encoding end.
  • volume threshold can be configured according to actual needs, and the embodiment of the present application does not specifically limit this.
  • the range corresponding to the listening direction may include an omnidirectional or a preset angle range, which is not specifically limited in this application.
  • the specific value of the preset angle can be configured according to actual requirements, which is not specifically limited in the embodiment of the present application.
  • the encoding terminal encodes the collected audio signal to obtain an encoded audio signal.
  • the encoding end sends the encoded audio signal to the decoding end.
  • the encoding end uses the transmission module 205 in the audio-visual system 10 shown in FIG. 2 to send the encoded audio signal to the decoding end.
  • the transmission module 205 may be a transmission line or others, which is not specifically limited in the embodiment of the present application. Among them, multiple encoded audio signals may also be referred to as an audio signal set.
  • the decoding terminal receives multiple encoded audio signals.
  • the multiple encoded audio signals received by the decoding end in S406 are the encoded audio signals sent by the encoding end in S405.
  • the multiple encoded audio signals only include audio signals that meet the filtering conditions at the listening position in the sound field.
  • the listening position here may be the listening position provided by the decoder in the real-time audio signal processing scene, or may include each listening position in the sound field in the non-real-time audio signal processing scene.
  • the decoding terminal decodes the audio signal that meets the screening condition to obtain a decoded signal.
  • the decoding end decodes each received audio signal to obtain a decoded signal.
  • the decoding process reference may be made to the decoding in the audio signal processing process shown in FIG. 1, which will not be repeated here.
  • the decoding end renders the decoded signal according to the channel transmission function in the playback device, and obtains the output signal.
  • the decoding end convolves each decoded signal with its channel transmission function in the playback device to obtain one or more output signals.
  • the decoded signal is rendered, as shown in the audio signal processing process shown in Figure 1. If the playback device is a speaker array, the decoded signal is rendered as a multi-channel speaker signal; if the playback device is a headphone device, then Render the decoded signal as a binaural output signal.
  • the channel transmission function may include: an HRTF function, or a BRIR function.
  • HRTF function a function that is not specifically limited in the embodiment of the present application.
  • BRIR function a function that is not specifically limited in the embodiment of the present application.
  • the decoder can play the output signal through the playback device.
  • the playback device may be inside the decoding end or outside the decoding end, which is not specifically limited in the embodiment of the present application.
  • the encoder only collects audio signals that meet the screening conditions in the sound field, then encodes and sends them to the decoder, and the decoder decodes the received audio signals that meet the screening conditions.
  • the configuration guarantees The audio signal quality screening conditions ensure that the audio signal collected at the encoder end not only guarantees the audio quality, but also does not require the decoding end to decode all audio signals.
  • an embodiment of the present application provides another audio signal processing method.
  • the encoding end and the decoding end of the end-to-end audio signal processing part are interacted to process audio signals in the sound field.
  • the encoding end/decoding end described below performs a certain operation, which can be performed for the encoding end/decoding end itself, or can be performed for the functional modules in the encoding end/decoding end, and will not be described one by one in the following.
  • the audio signal processing method may include:
  • the encoding terminal obtains the listening position.
  • S501 is the same as S401, and the specific implementation can refer to S401, which will not be repeated here.
  • the audio signal processing method provided in the embodiment of the present application may only consider the listening position, and then S503 is executed after S501.
  • the audio signal processing method provided by the embodiment of the present application may refer to the listening orientation based on the listening position, so after S501, it is necessary to perform S502 and then S503.
  • S502 is the same as S402, and the specific implementation can refer to S402, which will not be repeated here.
  • the encoding terminal collects audio signals corresponding to all sound sources at the listening position in the sound field.
  • the encoding end performs S503 through at least one audio signal collection device.
  • the audio signal collection device may be a microphone or other devices, which is not specifically limited in the embodiment of the present application.
  • the encoding terminal encodes the collected audio signal to obtain an encoded audio signal.
  • the encoding terminal determines indication information used to indicate audio signals that meet the screening conditions among the encoded audio signals.
  • the indication information may include characteristic information of each audio source signal in the encoded audio signal, where the characteristic information includes volume information or position information of the sound source corresponding to the audio signal.
  • the encoding end may collect the characteristic information of the audio signal through a sensor or other device, and this application does not limit the specific implementation.
  • the volume information is used to indicate the volume of the sound source.
  • the volume information can be a specific volume value, or the volume information can also be a volume level.
  • the volume information can also be other content, as long as it can be used to indicate the volume of the sound source.
  • the size information belongs to the volume information described here.
  • the position information of the sound source corresponding to the volume signal is used to indicate the specific position of the sound source in the sound field.
  • the position information can be coordinates or position identification.
  • the position information can also be other content, as long as it can be used to indicate that the sound source is in the sound field.
  • the specific location information in is all the location information described here.
  • the indication information may include an identifier of an audio signal that meets the screening condition in the encoded audio signal.
  • the identification of the audio signal is used to uniquely indicate the audio signal, and the type of the identification of the audio signal is not specifically limited in this application.
  • the identification of the audio signal may be the position or sequence of the audio signal in the encoded audio signal.
  • the encoding end sends the encoded audio signal to the decoding end.
  • the encoding end uses the transmission module 205 in the audio-visual system 10 shown in FIG. 2 to send the encoded audio signal to the decoding end.
  • the transmission module 205 may be a transmission line or others, which is not specifically limited in the embodiment of the present application.
  • S506 and S507 can be executed in combination or separately, which is not specifically limited in this application.
  • S506 and S507 are executed separately, the order of the two is not specifically limited, and FIG. 5 is only an example.
  • the decoding end receives multiple encoded audio signals.
  • the multiple encoded audio signals received by the decoding end in S508 are the encoded audio signals sent by the encoding end in S506.
  • the multiple encoded audio signals received in S508 include not only audio signals that meet the screening conditions at the listening position in the sound field, but also other audio signals that do not meet the screening conditions.
  • the encoding terminal receives the instruction information sent by the encoding terminal.
  • S508 and S509 can be executed in combination or separately, which is not specifically limited in this application.
  • S508 and S509 are executed separately, the order of the two is not specifically limited, and FIG. 5 is only an example.
  • the decoding terminal decodes the audio signal that meets the screening condition to obtain a decoded signal.
  • the decoding end filters from the multiple encoded audio signals received in S508 according to the received instruction information to obtain audio signals that meet the screening conditions at the listening position, and perform decoding to obtain decoded signals.
  • the encoding end in S510 determines whether each audio signal meets the screening condition according to the feature information, and screens the audio signals that meet the screening condition.
  • the encoding end screens the audio signal indicated by the indication information in S510 as the audio signal that meets the screening condition.
  • the decoding end renders the decoded signal according to the channel transmission function in the playback device, and obtains the output signal.
  • S511 is the same as S408.
  • S408 For specific implementation, refer to S408, which will not be repeated here.
  • the encoding terminal collects all audio signals corresponding to the sound source and encodes it and sends it to the decoding terminal. At the same time, it indicates to the decoding terminal the audio signals that meet the screening conditions through the instruction information, and the decoding terminal uses the instruction information provided by the encoding terminal. , Select the audio signal decoding that meets the screening conditions. As long as the screening conditions are configured to ensure the audio signal quality, when the number of sound sources in the sound field is relatively large, the audio signal transmission not only ensures the audio signal quality, but also greatly reduces the power consumption of the decoder.
  • the decoding end and the encoding end include hardware structures and/or software modules corresponding to the respective functions.
  • the functional unit that implements the above audio signal processing method in the decoding end and the encoding end is called an audio signal processing device.
  • the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed by hardware or computer software-driven hardware depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of this application.
  • the embodiment of the present application may divide the audio signal processing device into functional modules according to the foregoing method examples.
  • each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module.
  • the above-mentioned integrated modules can be implemented in the form of hardware or software functional modules. It should be noted that the division of modules in the embodiments of the present application is illustrative, and is only a logical function division, and there may be other division methods in actual implementation.
  • FIG. 6 shows a possible structural schematic diagram of the audio signal processing device 60 deployed in the encoding end involved in the foregoing embodiment.
  • the audio signal processing device 60 may be deployed on the encoding end of the audio-visual system 10 as a functional unit.
  • the audio signal processing device 60 may be the encoding end itself, or a functional module or chip in the encoding end.
  • the audio signal processing device 60 may include: a collection unit 601, an encoding unit 602, and a sending unit 603.
  • the acquisition unit 601 is used to execute the process S403 in FIG. 4 and the process S503 in FIG. 5; the encoding unit 602 is used to execute the process S404 in FIG.
  • the sending module 603 is used to execute the process S404 in FIG. Process S405 or processes S506 and S507 in FIG. 5.
  • the acquisition unit 601 may include the acquisition module 201 in the encoding end of the audio-visual system 10
  • the encoding unit 602 may include the encoding of the audio-visual system 10.
  • the audio encoding module 203 and the sending unit 603 in the audio-visual system 10 may include a transmission module 205 in the encoding end of the audiovisual system 10.
  • the audio signal processing device 60 may further include a receiving unit 604 and an acquiring unit 605.
  • the acquiring unit 605 may execute the processes S401 and S402 in FIG. 4 or the processes S501 and S502 in FIG. 5 through the receiving unit 604; or the acquiring unit 605 may be configured to execute the processes S401 and S402 in FIG. 4 or the processes in FIG. 5 S501, S502.
  • FIG. 7 shows a possible structural schematic diagram of the audio signal processing device 70 involved in the foregoing embodiment.
  • the audio signal processing device 70 may be deployed on the encoding end of the audiovisual system 10 as a functional unit.
  • the audio signal processing device 70 may also be the encoding end itself, or may also be a functional module or chip in the encoding end.
  • the audio signal processing device 70 may include: a processing module 701 and a communication module 702.
  • the processing module 701 is used to control and manage the actions of the audio signal processing device 70.
  • the processing module 701 is used to perform the procedures S401, S402, S403, and S404 in FIG. 4, or the procedures S501, S502, S503, S504, and S505 in FIG.
  • the audio signal processing device 70 may further include a storage module 703 for storing program codes and data of the audio signal processing device 70.
  • the processing module 701 may include a collection module 201, an audio preprocessing module 202, and an audio encoding module 203 in the encoding end of the audio-visual system 10.
  • the communication module 702 may include a transmission module 205 at the encoding end of the audiovisual system 10.
  • the processing module 701 may be the processor 301 in the physical structure of the audio signal processing apparatus 30 shown in FIG. 3, and may be a processor or a controller. For example, it may be a CPU, a general-purpose processor, DSP, ASIC, FPGA or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logical blocks, modules and circuits described in conjunction with the disclosure of this application.
  • the processor 701 may also be a combination that implements computing functions, for example, includes a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and so on.
  • the communication module 702 may be the transceiver 303 in the physical structure of the audio signal processing device 30 shown in FIG.
  • the communication module 702 may be a communication port, or may be a transceiver, a transceiver circuit, or a communication interface.
  • the above-mentioned communication interface may realize communication with other devices through the above-mentioned element having a transceiver function.
  • the above-mentioned elements with transceiving functions can be implemented by antennas and/or radio frequency devices.
  • the storage module 703 may be the memory 302 in the physical structure of the audio signal processing device 30 shown in FIG. 3.
  • the audio signal processing device 70 involved in FIG. 7 of the embodiment of the present application may be the audio signal processing device 30 shown in FIG. 3.
  • the audio signal processing device 60 or the audio signal processing device 70 provided by the embodiments of the present application can be used to implement the functions of the encoding end in the methods implemented by the various embodiments of the present application.
  • the audio signal processing device 60 or the audio signal processing device 70 provided by the embodiments of the present application can be used to implement the functions of the encoding end in the methods implemented by the various embodiments of the present application.
  • only the same as those implemented in the present application are shown.
  • the specific technical details are not disclosed, please refer to the embodiments of this application.
  • FIG. 8 shows a possible structural schematic diagram of the audio signal processing device 80 deployed in the decoding end involved in the foregoing embodiment.
  • the audio signal processing device 80 can be deployed on the decoding end of the audiovisual system 10 as a functional unit.
  • the audio signal processing device 80 can be the decoding end itself, or a functional module or chip in the decoding end.
  • the audio signal processing device 80 may include: a receiving unit 801, a decoding unit 802, and a processing unit 803.
  • the receiving unit 801 is used to execute the process S406 in FIG. 4 or the processes S508 and S509 in FIG. 5;
  • the decoding unit 802 is used to execute the process S407 in FIG.
  • the processing unit 803 is used to execute Process S408 in FIG. 4 or process S511 in FIG. 5.
  • the signal processing device 80 When the signal processing device 80 is deployed on the decoding end of the audio-visual system 10, the receiving unit 801 may include the transmission module 218 in the decoding end of the audio-visual system 10, and the decoding unit 802 may include the decoding of the audio-visual system 10.
  • the audio decoding module 207 in the terminal and the processing unit 803 may include the audio rendering module 208 in the decoding terminal in the audio-visual system 10.
  • the audio signal processing device 80 may further include an acquiring unit 804 and a sending unit 805. Among them, the acquiring unit 804 and the sending unit 805 are used to execute the processes S401a and S402a in FIG. 4.
  • FIG. 9 shows a schematic diagram of a possible structure of the audio signal processing device 90 deployed in the decoding end involved in the foregoing embodiment.
  • the audio signal processing device 90 may be deployed on the decoding end of the audio-visual system 10 as a functional unit.
  • the audio signal processing device 90 may be the decoding end itself, or a functional module or chip in the decoding end.
  • the audio signal processing device 90 may include: a processing module 901 and a communication module 902.
  • the processing module 901 is used to control and manage the actions of the audio signal processing device 90.
  • the processing module 901 is used to execute the processes S407, S408, and S401a in FIG. 4, or the processes S510, S511 in FIG.
  • the audio signal processing device 90 may further include a storage module 903 for storing program codes and data of the audio signal processing device 90.
  • the processing module 901 may include the audio decoding module 207 and the audio rendering module 208 in the decoding end of the audio-visual system 10
  • the communication module 902 may include The transmission module 218 in the audio-visual system 10.
  • the processing module 901 may be the processor 301 in the physical structure of the audio signal processing apparatus 30 shown in FIG. 3, and may be a processor or a controller. For example, it may be a CPU, a general-purpose processor, DSP, ASIC, FPGA or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logical blocks, modules and circuits described in conjunction with the disclosure of this application.
  • the processor 901 may also be a combination that implements computing functions, for example, includes a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and so on.
  • the communication module 902 may be the transceiver 303 in the physical structure of the audio signal processing device 30 shown in FIG.
  • the communication module 902 may be a communication port, or may be a transceiver, a transceiver circuit, or a communication interface.
  • the above-mentioned communication interface may realize communication with other devices through the above-mentioned element having a transceiver function.
  • the above-mentioned elements with transceiving functions can be implemented by antennas and/or radio frequency devices.
  • the storage module 903 may be the memory 302 in the physical structure of the audio signal processing device 30 shown in FIG. 3.
  • the audio signal processing device 90 involved in FIG. 9 of the embodiment of the present application may be the audio signal processing device 30 shown in FIG. 3.
  • the audio signal processing device 80 or the audio signal processing device 90 provided by the embodiments of the present application can be used to implement the functions of the decoding end in the methods implemented by the various embodiments of the present application.
  • the specific technical details are not disclosed, please refer to the embodiments of this application.
  • a computer-readable storage medium is provided, and an instruction is stored thereon.
  • the instruction is executed, the audio signal processing method in the foregoing method embodiment is executed.
  • a computer program product containing instructions is provided, and when the instructions are executed, the audio signal processing method in the foregoing method embodiment is executed.
  • the disclosed system, device, and method may be implemented in other ways.
  • the device embodiments described above are merely illustrative.
  • the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or It can be integrated into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may be separately physically included, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit can be realized in the form of hardware, or in the form of hardware plus software functional unit.
  • the above-mentioned integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium.
  • the above-mentioned software function unit is stored in a storage medium, and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device, etc.) to execute part of the steps of the method described in each embodiment of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program code .

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Human Computer Interaction (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Mathematical Physics (AREA)
  • Quality & Reliability (AREA)
  • Databases & Information Systems (AREA)
  • Stereophonic System (AREA)

Abstract

一种音频信号处理方法及装置，该方法包括：采集收听位置处满足筛选条件的音频信号（S403），其中，该筛选条件包括下述条件中一项或多项：在收听位置处音量大于或等于音量阈值，或者，声源与收听位置的收听距离小于或等于预设收听距离，或者，声源处于收听朝向对应的范围内；将采集的音频信号进行编码获取经过编码的音频信号（S404）；向解码端发送经过编码的音频信号（S405）。该方法能够在保证音频信号质量的前提下，降低解码功耗。

Description

一种音频信号处理方法及装置
本申请要求于2019年01月31日提交国家知识产权局、申请号为201910103595.6、发明名称为“一种音频信号处理方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及音频信号处理领域，尤其涉及一种音频信号处理方法及装置。
背景技术
随着高性能计算机和信号处理技术的飞速发展，人们对语音、音频体验提出了越来越高的要求，浸入式音频能够满足人们在这方面的需求。例如，第四代移动通信技术(the 4th generation mobile communication technology，4G)/第五代移动通信技术(the5th generation mobile communication technology，5G)通信语音，音频服务和虚拟现实(virtual reality，VR)、增强现实(Augmented reality，AR)、混合显示(Mixed Reality，MR)等影音技术受到越来越多的关注。一个具有沉浸感的影音系统，不仅需要震撼的视觉效果，还需要逼真的听觉效果，视听的融合能大大提高沉浸感体验。
音频的核心是三维音频技术，目前实现三维音频有多种重放方法，包括使用基于扬声器的扬声器重放和基于耳机的双耳重放。以基于耳机的双耳重放主要借助头部相关传输函数(head related transfer function，HRTF)或者双耳房间冲激响应(binaural room impulse response，BRIR)来实现。其中，HRTF参考声源所产生的声波传播到耳道时，受到头部、躯干、耳廓等器官散射、反射及折射等影响进行重放；BRIR考虑环境反射声对声源的影响进行重放，BRIR可以看作声源、室内环境、双耳(包括头部、躯干、耳廓)所组成的系统的脉冲响应，其由直达声、早期反射、后期混响组成。基于耳机的双耳重放的原理为：当假定声源在某一位置后，选择从此位置到收听者头中心位置对应的HRTF或者BRIR进行卷积处理，得到的双耳的输出信号。
当前的三维音频技术中，根据应用场景选择相应的设备采集声场中所有的有用声源对应的音频信号，将采集到所有音频信号进行编码，传输到解码端经解码后根据播放端选择的播放设备进行渲染。但是，当声场中声源数量比较大时，导致解码端功耗很大。
发明内容
本申请实施例提供一种音频信号处理方法及装置，在保证音频信号质量的前提下，降低解码功耗。
为达到上述目的，本申请的实施例采用如下技术方案：
本申请的第一方面，提供一种音频信号处理方法，应用于编码端，该方法可以包括：先采集收听位置处的音频信号，每个采集的音频信号有对应的声源，采集的音频信号至少包括了满足筛选条件的音频信号；其中，该筛选条件可以包括下述条件中一项或多项：在该收听位置处的音量大于或等于音量阈值，或者，其对应的声源与收听位置的收听距离小于或等于预设收听距离，或者，其对应的声源处于收听朝向对应的 范围内；然后将采集的音频信号进行编码，以获得经过编码的音频信号；再向解码端发送经过编码的音频信号。
通过本申请提供的音频信号处理方法，通过至少采集收听位置处满足筛选条件的音频信号，并将采集的音频信号编码发送给解码端用于解码端解码播放音频信号。这样一来，编码端可以仅采集满足筛选条件的音频信号并编码发送，以使得解码端仅解码满足筛选条件的音频信号。因此，可以通过配置保证音频信号质量的筛选条件，在声源数量比较大时，音频信号的传输既保证了音频信号质量，又大大降低了解码端的功耗。
需要说明的是，本申请所提供的方案可以应用于声场中，当然也可以用在别的应用场景下，例如游戏场景，电影场景，会议场景等等。本申请所描述的编码端是指影音系统中对声源对应的音频信号进行编码的部分，编码端将声源对应的音频信号采集编码后，传输至解码端，由解码端解码渲染后播放。其中，影音系统包括但不限于VR系统、AR系统、MR系统或其他流式传输系统。本申请实对于本申请提供的方案所应用的影音系统的类型，不进行具体限定。
具体的，音频信号处理包括实时音频信号处理场景以及非实时音频信号处理场景。无论在哪个场景中，由于用户的移动性，收听位置并不是固定不变的，收听朝向也不是固定不变的。收听朝向对应的范围可以包括全向或者预设角度的范围，本申请对此不进行具体限定。
在实时音频信号处理的场景，收听位置是收听者当前在声场中所处的位置，收听朝向是指收听者在收听位置面朝的方向，收听位置和/或收听朝向均由解码端向编码端提供，编码端根据解码端提供的收听位置和/或收听朝向，对音频信号的处理方式为边采集边传输，实现声场的逼真还原，使收听者身临其境。在非实时音频信号处理的场景，收听位置是声场中任一可能的位置，收听朝向是任一可能的方向。
结合第一方面，在一种可能的实现方式中，在实时音频信号处理的场景中，本申请提供的音频信号处理方法还可以包括：接收解码端发送的收听位置信息，该收听位置信息用于指示收听位置。
结合第一方面或上述任一种可能的实现方式，在另一种可能的实现方式中，当筛选条件中需使用收听朝向时，例如筛选条件包括对应的声源处于收听朝向对应的范围内，在实时音频信号处理的场景中，本申请提供的音频信号处理方法还可以包括：接收解码端发送的收听朝向信息，该收听朝向信息用于指示的收听者的收听朝向。
结合第一方面或上述任一种可能的实现方式，在另一种可能的实现方式中，在非实时音频信号处理的场景，预设或确定多个收听位置，在每个收听位置存在多个收听朝向，编码端需要在每个收听位置执行本申请提供的音频信号处理方法采集音频信号并编码，解码端根据实际收听者所处的位置，选择匹配的经过编码的音频信号进行解码渲染，从而使收听者身临其境。
结合第一方面或上述任一种可能的实现方式，在另一种可能的实现方式中，在非实时音频信号处理的场景，收听朝向可以是任意一个收听位置的任意一个收听朝向。当筛选条件包括收听朝向相关的内容时，编码端需要获取任意一个收听朝向对应的音频信号，使得解码端可以从编码端获取满足当前筛选条件的音频信号，从而在解码端 还原声场；预先获取各个收听朝向的多个音频信号，使得收听者在任意收听朝向时，解码器都可以从编码端获取收听者当前的收听朝向对应的音频信号，从而使收听者身临其境。
结合第一方面或上述任一种可能的实现方式，在另一种可能的实现方式中，当音量超过一定门限的音频信号才能被听到，并结合声源与收听朝向的位置关系不同时该门限不同，可以配置音量阈值为音频信号对应的声源与收听朝向的位置关系确定的阈值。在该实现方式中，不同的位置关系配置不同的阈值，更加准确的适应收听需求，使得收听效果更加逼真。
结合第一方面或上述任一种可能的实现方式，在另一种可能的实现方式中，当音量超过一定门限的音频信号才能被听到，可以配置音量阈值为预设音量阈值，以采集音量大的声源信号。
结合第一方面或上述任一种可能的实现方式，在另一种可能的实现方式中，本申请提供的音频信号处理方法还可以包括：编码端采集收听位置处所有声源对应的音频信号，将采集的所有音频信号编码，以获得经过编码的音频信号发送给解码端，并向解码端发送指示信息，该指示信息用于指示经过编码的音频信号中满足筛选条件的音频信号，由解码端根据指示信息，只解码满足筛选条件的音频信号，实现了解码端只解码收听位置处满足筛选条件的声源对应的音频信号。
结合第一方面或上述任一种可能的实现方式，在另一种可能的实现方式中，指示信息可以包括：经过编码的音频信号中每个音频信号的特征信息；其中，特征信息包括音量信息或者音频信号对应的声源的位置信息。解码端根据特征信息选择满足筛选条件的音频信号进行解码。
结合第一方面或上述任一种可能的实现方式，在另一种可能的实现方式中，指示信息可以包括：经过编码的音频信号满足筛选条件的音频信号的标识。
本申请的第二方面，提供一种音频信号处理方法，应用于解码端，该方法可以包括：接收经过编码的多个音频信号，该多个音频信号包括的音频信号是收听位置处的音频信号，每个音频信号有对应的声源，所述多个音频信号至少包括满足筛选条件的音频信号；其中，筛选条件包括下述条件中一项或多项：在收听位置处的音量大于或等于音量阈值，或者，其对应的声源与收听位置的收听距离小于或等于预设收听距离，或者，其对应的声源处于收听朝向对应的范围内；对满足筛选条件的音频信号进行解码以得到解码信号；根据播放设备中的声道传输函数渲染解码信号，获取输出信号。
通过本申请提供的音频信号处理方法，解码端仅解码满足筛选条件的声源对应的音频信号，只要配置保证音频质量的筛选条件，在声源数量比较大时，音频信号的传输既保证了音频信号质量，又大大降低了解码端的功耗。
需要说明的是，对于收听位置、收听朝向已经在第一方面中进行了详细描述，此处不再进行赘述。
结合第二方面，在一种可能的实现方式中，根据播放设备中的声道传输函数渲染解码信号，获取输出信号，包括：将每个解码信号，分别与其在播放设备中的声道传输函数进行卷积，获取一个或多个输出信号。
结合第二方面或上述任一种可能的实现方式，在另一种可能的实现方式中，在实 时音频信号处理的场景中，本申请提供的音频信号处理方法还可以包括：解码端向编码端发送收听位置信息，以使编码端根据收听位置信息获取经过编码的多个音频信号，该收听位置信息用于指示收听位置。
结合第二方面或上述任一种可能的实现方式，在另一种可能的实现方式中，当筛选条件中需使用收听朝向时，例如筛选条件包括音频信号对应的声源处于收听朝向对应的范围内，在实时音频信号处理的场景中，本申请提供的音频信号处理方法还可以包括：解码端向编码端发送收听朝向信息，以使所述编码端根据所述朝向信息获取经过编码的多个音频信号，该收听朝向信息用于指示收听朝向。
结合第二方面或上述任一种可能的实现方式，在另一种可能的实现方式中，在非实时音频信号处理的场景，收听位置是应用场景，例如声场中的任意一个位置，因此，在非实时音频信号处理的场景中需要预设或确定多个收听位置，在每个收听位置存在多个收听朝向，编码端需要在每个收听位置执行本申请提供的音频信号处理方法处采集音频信号编码，解码端可以根据收听者当前所处的收听位置从编码端获取对应的音频信号，在解码端执行本申请提供的音频信号处理方法还原声场，预先获取各个收听位置的音频信号，使得收听者在任意位置时，解码器都可以从编码端获取当前收听位置处的经过编码的音频信号进行解码，从而使收听者身临其境。
结合第二方面或上述任一种可能的实现方式，在另一种可能的实现方式中，在非实时音频信号处理的场景，收听朝向可以是任意一个收听位置的任意一个收听朝向。当筛选条件包括收听朝向相关的内容时，编码端需要获取任意一个收听朝向对应的经过编码音频信号，使得解码端可以从编码端获取满足筛选条件的经过编码的音频信号，从而在解码端还原声场；预先获取各个收听朝向的经过编码的音频信号，使得收听者在任意收听朝向时，解码器都可以从编码端获取当前收听朝向对应的经过编码的音频信号进行解码，从而使收听者身临其境。
结合第二方面或上述任一种可能的实现方式，在另一种可能的实现方式中，当音量超过一定门限的音频信号才能被听到，并结合声源与收听朝向的位置关系不同时该门限不同，可以配置音量阈值为声源与收听朝向的位置关系确定的阈值。在该实现方式中，不同的位置关系配置不同的音量阈值，更加准确的适应收听需求，也能使得收听效果更加逼真。
结合第二方面或上述任一种可能的实现方式，在另一种可能的实现方式中，当音量超过一定门限的音频信号才能被听到，可以配置音量阈值为预设音量阈值，以采集音量大的声源信号。
结合第二方面或上述任一种可能的实现方式，在另一种可能的实现方式中，经过编码的多个音频信号仅包括收听位置处满足筛选条件的音频信号；解码端对满足筛选条件的声源信号进行解码以得到解码信号，具体实现为：解码端将多个音频信号中每个音频信号进行解码以得到解码信号。
结合第二方面或上述任一种可能的实现方式，在另一种可能的实现方式中，经过编码的多个音频信号包括收听位置处所有声源对应的音频信号，本申请提供的音频信号处理方法还可以包括：解码端接收编码端发送的指示信息，该指示信息用于指示经过编码的多个音频信号中满足筛选条件的声源信号；解码端根据该指示信息，从经过 编码的多个音频信号中筛选，获取该指示信息指示的音频信号作为满足筛选条件的音频信号。
结合第二方面或上述任一种可能的实现方式，在另一种可能的实现方式中，指示信息可以包括：经过编码的多个音频信号中每个音频信号的特征信息；其中，特征信息包括音量信息或者音频信号对应的声源的位置信息。解码端根据特征信息选择满足筛选条件的音频信号进行解码。
结合第二方面或上述任一种可能的实现方式，在另一种可能的实现方式中，指示信息可以包括：经过编码的多个音频信号中满足筛选条件的音频信号的标识。解码端从经过编码的多个音频信号中筛选标识指示的音频信号作为满足筛选条件的音频信号。
结合第二方面或上述任一种可能的实现方式，在另一种可能的实现方式中，声道传输函数可以包括：HRTF函数，或者，BRIR函数。本申请对于声道传输函数的类型不进行具体限定。
结合第二方面或上述任一种可能的实现方式，在另一种可能的实现方式中，在流式传输场景中，解码端对满足筛选条件的音频信号进行解码以得到解码信号，具体包括：解码端对满足筛选条件的音频信号下载，对下载的音频信号进行解码以得到解码信号。
本申请的第三方面，提供一种音频信号处理装置，该装置可以包括：采集单元、编码单元以及发送单元。其中，采集单元，用于采集收听位置处的音频信号，每个采集的音频信号有对应的声源，所述采集的音频信号至少包括了满足筛选条件的音频信号；其中，筛选条件可以包括下述条件中一项或多项：在收听位置处音量大于或等于音量阈值，或者，其对应的声源与收听位置的收听距离小于或等于预设收听距离，或者，其对应的声源处于收听朝向对应的范围内；编码单元，用于将采集的音频信号进行编码，以获得经过编码的音频信号；发送单元，用于向解码端发送经过编码的音频信号。
通过本申请提供的音频信号处理装置，通过采集收听位置处满足筛选条件的音频信号，使得解码端仅解码满足筛选条件的音频信号，这样一来，通过配置保证音频信号质量的筛选条件，在声源数量比较大时，音频信号的传输既保证了音频信号质量，又大大降低了解码端的功耗。
需要说明的是，本申请第三方面提供的音频信号处理装置，用于实现上述第一方面以及第一方面中任一种可能的实现方式提供的音频信号处理方法，因此，本申请第三方面提供的音频信号处理装置的具体实现，可以参照上述第一方面以及第一方面中任一种可能的实现方式提供的音频信号处理方法的具体实现，此处不再进行赘述。
本申请的第四方面，提供另一种音频信号处理装置，该装置可以包括：接收单元、解码单元以及渲染单元。其中，接收单元，用于接收经过编码的多个音频信号，该多个音频信号是收听位置处的音频信号，每个音频信号有对应的声源，该多个音频信号至少包括满足筛选条件的音频信号；其中，筛选条件包括下述条件中一项或多项：在收听位置处音量大于或等于音量阈值，或者，其对应的声源与收听位置的收听距离小于或等于预设收听距离，或者，其对应的声源处于收听朝向对应的范围内；解码单元，对接收单元接收的满足筛选条件的音频信号进行解码以得到解码信号；渲染单元，用 于根据播放设备中的声道传输函数，渲染解码单元得到的解码信号，获取输出信号。
通过本申请提供的音频信号处理装置，解码端仅解码声场中满足筛选条件的声源对应的音频信号，只要配置保证音频质量的筛选条件，在声场中声源数量比较大时，音频信号的传输既保证了音频信号质量，又大大降低了解码端的功耗。
需要说明的是，本申请第四方面提供的音频信号处理装置，用于实现上述第二方面以及第二方面中任一种可能的实现方式提供的音频信号处理方法，因此，本申请第四方面提供的音频信号处理装置的具体实现，可以参照上述第二方面以及第二方面中任一种可能的实现方式提供的音频信号处理方法的具体实现，此处不再进行赘述。
本申请的第五方面，本申请提供了一种音频信号处理装置，该音频信号处理装置可以实现上述方法示例中的编码端的功能，所述功能可以通过硬件实现，也可以通过硬件执行相应的软件实现。所述硬件或软件包括一个或多个上述功能相应的模块。该音频信号处理装置可以以芯片的产品形态存在。
结合第五方面，在一种可能的实现方式中，该音频信号处理装置的结构中包括处理器和收发器，该处理器被配置为支持该音频信号处理装置执行上述方法中相应的功能。该收发器用于支持该音频信号处理装置与其他设备之间的通信。该音频信号处理装置还可以包括存储器，该存储器用于与处理器耦合，其保存该音频信号处理装置必要的程序指令和数据。
本申请的第六方面，本申请提供了另一种音频信号处理装置，该音频信号处理装置可以实现上述方法示例中的解码端的功能，所述功能可以通过硬件实现，也可以通过硬件执行相应的软件实现。所述硬件或软件包括一个或多个上述功能相应的模块。该音频信号处理装置可以以芯片的产品形态存在。
结合第六方面，在一种可能的实现方式中，该音频信号处理装置的结构中包括处理器和收发器，该处理器被配置为支持该音频信号处理装置执行上述方法中相应的功能。该收发器用于支持该音频信号处理装置与其他设备之间的通信。该音频信号处理装置还可以包括存储器，该存储器用于与处理器耦合，其保存该音频信号处理装置必要的程序指令和数据。
本申请的第七方面，提供一种编码器，该编码器包括执行上述第一方面或第一方面的任一种可能的实现方式提供的音频信号处理方法的音频信号处理装置。该编码器可以被本申请实施例中的编码端使用。
本申请的第八方面，提供一种解码器，该解码器包括执行上述第二方面或第二方面的任一种可能的实现方式提供的音频信号处理方法的音频信号处理装置。该解码器可以被本申请实施例中的解码端使用。
本申请的第九方面，提供一种音频信号处理系统，包括第七方面提供的编码器，和/或第八方面提供的解码器。
本申请的第十方面，提供了一种计算机可读存储介质，包括指令，当其在计算机上运行时，使得计算机执行上述任一方面或任一种可能的实现方式提供的音频信号处理方法。
本申请的第十一方面，提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行上述任一方面或任一种可能的实现方式提供的音频信号处理 方法。
其中，需要说明的是，上述各个方面中的任意一个方面的各种可能的实现方式，在方案不矛盾的前提下，均可以进行组合。
附图说明
图1为现有技术提供的一种音频信号处理方法的示意图；
图2为现有技术提供的一种影音系统的架构示意图；
图3为本申请实施例提供的一种音频信号处理装置的结构示意图；
图4为本申请实施例提供的一种音频信号处理方法的流程示意图；
图5为本申请实施例提供的另一种音频信号处理方法的流程示意图；
图6为本申请实施例提供的另一种音频信号处理装置的结构示意图；
图7为本申请实施例提供的再一种音频信号处理装置的结构示意图；
图8为本申请实施例提供的又一种音频信号处理装置的结构示意图；
图9为本申请实施例提供的又一种音频信号处理装置的结构示意图。
具体实施方式
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”、“第三”和“第四”等是用于区别不同对象，而不是用于限定特定顺序。
在本申请实施例中，“示例性的”或者“例如”等词用于表示作例子、例证或说明。本申请实施例中被描述为“示例性的”或者“例如”的任何实施例或设计方案不应被解释为比其它实施例或设计方案更优选或更具优势。确切而言，使用“示例性的”或者“例如”等词旨在以具体方式呈现相关概念。
在本申请实施例中描述的A、B、C中的一项或多项，用于表示下述概念：A，或者B，或者C，或者A和B，或者，A和C，或者，B和C，或者，A、B以及C。
在介绍本申请提供的方案之前，先对的音频信号处理的常规过程进行说明。
常规的音频信号处理过程包括：编码端设备根据应用场景选择相应的设备采集当前声场中所有的声源对应的音频信号，将采集到所有音频信号进行编码，传输到解码端；解码端对接收到的所有音频信号进行解码渲染获取输出信号。如果播放设备为扬声器阵列，则将输出信号渲染为多声道扬声器信号；如果播放设备为耳机设备，则将输出信号渲染为双耳输出信号。
其中，编解码可以使用目前常见的增强型语音通话服务(enhance voice services，EVS)、自适应多速率宽带(adaptive multi-rate wideband，AMRWB)、自适应多速率(Adaptive Multi-Rate，AMR)等编码方式，多声道扬声器信号是由渲染器根据具体扬声器个数渲染得到，双耳输出信号是由渲染器利用多声道扬声器信号和HRTF/BRIR渲染得到。需要说明的是，前述列举的编解码所使用的编码方式仅仅是举例，在实际应用中也可以使用其他的音频编码方式。
图1示意了当前音频信号处理的过程，以声场中有5个声源(图1中的1-5)为例，结合图1对当前的音频信号处理过程进行描述。
以选择EVS编解码器为例：5个声源对应的音频信号分别记录为S1(n)，S2(n)，S3(n)，S4(n)，S5(n)，分别经过EVS编码后得到5个码流信号S1S(n)，S2S(n)，S3S(n)，S4S(n)，S5S(n)，将5个码流信号按照帧与帧相邻方式连接起来经信道编码之后发送出 去。
解码端接收到编码后的码流信号后，先进行信道解码得到5个码流S1S(n)，S2S(n)，S3S(n)，S4S(n)，S5S(n)，码流经EVS decoder解码后得到5个信号DS1(n)，DS2(n)，DS3(n)，DS4(n)，DS5(n)。
将5个信号分别与收听装置选择的声道传输函数(HRTF函数)中各自对应的HRTF信号(HRTF1(n)、HRTF2(n)、HRTF3(n)、HRTF3(n)、HRTF4(n))进行卷积处理，得到卷积结果为Y1(n)，Y2(n)，Y3(n)，Y4(n)，Y5(n)。具体的卷积表示为：
Figure PCTCN2019128908-appb-000001
Figure PCTCN2019128908-appb-000002
Figure PCTCN2019128908-appb-000003
Figure PCTCN2019128908-appb-000004
Figure PCTCN2019128908-appb-000005
最后，将卷积结果叠加得到最终的双耳(binaural)输出信号Y(n)。其中，Y(n)的计算表达式可以为：
Y(n)＝Y1(n)+Y2(n)+Y3(n)+Y4(n)+Y5(n)。
在上述音频信号处理过程中，编码端将所有的声源对应的音频信号全部进行采集编码传输至解码端，解码端将接收到的所有音频信号解码渲染。当声源信号个数比较多时，解码端的功耗非常大。
为了解决上述问题，本申请实施例提供一种音频信号信号处理方法，其基本原理是：解码端选择满足筛选条件的音频信号进行解码，以替代现有技术中解码端将所有声源对应的音频信号解码，在声源个数多时，降低了解码端的功耗。另外，只要合理配置筛选条件，在降低功耗的同时，依然可以很好的保证音频信号传输的质量，以保证收听效果。
在描述本申请的方案之前，先对本申请涉及的名字进行解释说明。
收听位置(position of the listener)，是指收听者(listener)所处的位置；收听朝向，是指收听者在收听位置面朝的方向。收听者可以为实际使用播放设备的用户，也可以为虚拟的使用播放设备的用户。虚拟的使用播放设备的用户可以处于任一个可能的收听位置，可以有任一种可能的收听朝向。
收听距离，是指声源与收听位置之间的直线距离。或者，收听距离也可以为声源与收听位置之间的预设路径的距离。该预设路径可以为绕过障碍物的距离或者其他，本申请不进行具体限定。
下面将结合附图对本申请实施例的实施方式进行详细描述。
本申请提供的音频信号处理方法应用于图2示意的影音系统10中。影音系统10中包括端到端的音频信号处理部分以及端到端的视频信号处理部分。影音系统10可以为VR系统、或AR系统、或MR系统或其他流式传输系统。当然，本申请实施例对于影音系统10的实际形态不进行具体限定。
如图2所示，影音系统10中端到端的音频信号处理部分可以包括采集模块(acquisition)201、音频预处理模块(audio preprocessing)202、音频编码模块(audio encoding)203、封装模块(file/segment encapsulation)204、传输模块(delivery)205/218、 解封装模块(file/segment decapsulation)206、音频解码模块(audio decoding)207、音频渲染模块(audio rendering)208和扬声器/耳机(loudspeakers/headphones)209。
如图2所示，影音系统10中包括的端到端的视频信号处理部分可以包括：采集模块(acquisition)201、视频组合模块(visual stitching)210、预测绘图模块(projection and mapping)211、视频编码模块(video encoding)212、图像编码模块(image encoding)213、封装模块(file/segment encapsulation)204、传输模块(delivery)205/218、解封装模块(file/segment decapsulation)206、视频解码模块(video decoding)214、图像解码模块(image decoding)215、视频渲染模块(visual rendering)216和播放器(display)217。
需要说明的是，影音系统10中包括的端到端的音频信号处理部分以及端到端的视频信号处理部分，部分模块为共用模块。
下面对图2示意的影音系统10中各个模块功能进行简单描述：
采集模块201用于采集声源对应的音频信号，将音频信号传输至音频预处理模块202。
音频预处理模块202用于对音频信号进行预处理，例如，滤波处理等，并将预处理后的音频信号传输至音频编码模块203。
音频编码模块203用于对预处理后的音频信号进行编码，将编码后的音频信号传输至封装模块204。
采集模块201还用于采集视频信号。视频信号经过视频组合模块210、预测绘图模块211、视频编码模块212和图像编码模块213的处理后，将编码后的视频信号传输至封装模块204。
封装模块204用于将编码后的音频信号和编码后的视频信号进行封装得到码流，该码流通过传输模块205传输至解码端的传输模块218，再由传输模块218传输至解封装模块206。示例性的，传输模块205和传输模块218可以是有线通信模块或者无线通信模块，本申请实施例对此不进行具体限定。
需要说明的是，当影音系统10为流式传输系统时，传输模块205具体可以实现为服务器的形式，即编码端将码流上传至服务器，由解码端按照需求从服务器下载码流，以实现传输模块205的功能，该过程不再进行赘述。
解封装模块206用于对码流进行解封装，获得编码后的音频信号和编码后的视频信号，并将编码后的音频信号传输至音频解码模块207，将编码后的视频信号传输至视频解码模块214和图像解码模块215。
音频解码模块207用于对编码后的音频信号进行解码，并将解码后的音频信号传输至音频渲染模块208。
音频渲染模块208用于对解码后的音频信号进行渲染处理，并将渲染的输出信号传输至扬声器/耳机209。
视频解码模块214、图像解码模块215和视频渲染模块216对编码后的视频信号进行处理，并将处理后的视频信号传输至播放器217播放。
需要说明的是，图2中示出的结构并不构成对影音系统10的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。尽管未示出，影音 系统10还可以包括传感器等，传感器用于获取收听位置和/或收听朝向，在此不再赘述。
还需要说明的是，图2示意的影音系统10中的端到端的视频信号处理部分和端到端的音频信号处理系统可以单独的独立部署，本申请不再进行赘述。
下面结合附图，对本申请提供的方案进行详细描述。
一方面，本申请提供一种音频信号处理装置，图3示出的是与本申请各实施例相关的一种音频信号处理装置30。该音频信号处理装置30可以部署在影音系统10中的编码端作为功能单元，也可以部署在影音系统10中的解码端作为功能单元，本申请实施例对此不进行具体限定。当然，该音频信号处理装置30也可以为影音系统中编码端或者解码端。或者，该音频信号处理装置30可以为芯片系统。本申请实施例中，芯片系统可以由芯片构成，也可以包含芯片和其他分立器件。
如图3所示，音频信号处理装置30可以包括：处理器301、存储器302、收发器303。
下面结合图3对音频信号处理装置30的各个构成部件进行具体的介绍：
存储器302，可以是易失性存储器(volatile memory)，例如RAM；或者non-volatile memory，例如只读存储器(read-only memory，ROM)，快闪存储器(flash memory)，HDD或SSD；或者上述种类的存储器的组合，用于存储可实现本申请方法的程序代码、以及配置文件。
处理器301是音频信号处理装置30的控制中心，可以是一个CPU，也可以是ASIC，或者是被配置成实施本申请实施例的一个或多个集成电路，例如：一个或多个DSP，或，一个或者多个FPGA。处理器301可以通过运行或执行存储在存储器302内的软件程序和/或模块，以及调用存储在存储器302内的数据，执行音频信号处理装置30的各种功能。
收发器303用于音频信号处理装置30与其他单元进行交互。示例性的，收发器303可以为音频信号处理装置30的收发端口。
进一步的，音频信号处理装置30还可以包括传感器304，用于获取收听位置和/或收听朝向。示例的，传感器304具可以是陀螺仪、外置摄像头、运动检测装置或图像检测装置等，本申请实施例对此不作限定。
具体的，当音频信号处理装置30部署在编码端，或者，当音频信号处理装置30为编码端时，处理器301通过运行或执行存储在存储器302内的软件程序和/或模块，以及调用存储在存储器302内的数据，执行如下功能：
采集收听位置处的音频信号，每个采集的音频信号有对应的声源，采集的音频信号至少包括满足筛选条件的音频信号；其中，筛选条件可以包括下述条件中一项或多项：在收听位置处音量大于或等于音量阈值，或者，其对应的声源与收听位置的收听距离小于或等于预设收听距离，或者，其对应的声源处于收听朝向对应的范围内；将采集的音频信号进行编码，以获得经过编码的音频信号；通过收发器303向解码端发送经过编码的音频信号。
其中，本申请实施例提供的方法，装置，设备或系统可以应用于实时音频信号处理场景以及非实时音频信号处理场景。在应用于实时音频信号处理场景时，所述的收 听位置是解码端，即收听者(listener)当前所处的位置，具体地，收听者(listener)当前所处的位置可以是游戏场景范围，声场范围，电影场景范围，会议场景范围等等应用场景范围内的位置；为了在解码端更好地为收听者重现编码端的音频，编码端可以根据解码端发送的收听者当前所处的位置(即收听位置)进行音频信号的采集，从而使得编码端采集的音频能够与收听者当前所处的位置匹配；在应用于非实时音频信号处理场景时，收听位置是预设的位置，由于收听者可能在应用场景范围内任何位置，因此可以预设一些收听位置，编码端可以根据预设的收听位置进行音频信号的采集，此时，解码端在进行音频信号的解码处理时，可以根据收听者当前所处的位置，选择最匹配(例如，位置相同或最相近)的音频信号进行解码处理。
具体的，当音频信号处理装置30部署在解码端，或者，当音频信号处理装置30为解码端时，处理器301通过运行或执行存储在存储器302内的软件程序和/或模块，以及调用存储在存储器302内的数据，执行如下功能：
通过收发器303接收经过编码的多个音频信号，该多个音频信号是收听位置处的音频信号，每个音频信号有对应的声源，该多个音频信号至少包括满足筛选条件的音频信号；其中，筛选条件可以包括下述条件中一项或多项：在收听位置处音量大于或等于音量阈值，或者，其对应的声源与收听位置的收听距离小于或等于预设收听距离，或者，其对应的声源处于收听朝向对应的范围内；对满足筛选条件的音频信号进行解码以得到解码信号；根据播放设备中的声道传输函数，渲染解码信号，获取输出信号。
另一方面，本申请实施例提供一种音频信号处理方法，应用于图2示意的影音系统10中端到端的音频信号处理部分的编码端与解码端交互过程中，以处理声场中的音频信号。下文中描述的编码端/解码端执行某操作，可以为编码端/解码端自身执行该操作，也可以为编码端/解码端中的功能模块执行该操作，后续不再一一说明。
需要说明的是，本文所描述的编码端与解码端交互时，两者可以直接通信交互，也可以通过将交互内容上传至共用媒介，从共用媒介下载的方式交互，本申请统一描述为发送/接收，后续不再一一说明。其中，共用媒介可以为服务器或者云服务器或者其他。
如图4所示，该音频信号处理方法可以包括：
S401、编码端获取收听位置。
其中，音频信号处理包括实时音频信号处理场景以及非实时音频信号处理场景。无论在哪个场景中，由于用户的移动性，收听位置并不是固定不变的。
下面分实时音频信号处理场景和非实时音频信号处理场景对S401的实现进行说明。
一、实时音频信号处理场景。
在实时音频信号处理的场景，收听位置是收听者当前在声场中所处的位置，收听朝向是指收听者在收听位置面朝的方向，收听位置可以由解码端向编码端提供。
因此，在S401之前，如图4所示，本申请实施例提供的音频信号处理方法还可以包括S401a。
S401a、解码端获取收听位置，向编码端发送收听位置信息。
具体的，解码端可以使用传感器或者用于获取位置的装置，获取收听者当前在声 场中所处的位置，本申请实施例对于解码端获取收听位置的实现方式不进行具体限定。
其中，收听位置信息用于指示解码端获取的收听位置，收听位置信息的内容可以根据实际需求配置，本申请实施例对此不进行限定。
示例性的，收听位置信息可以为坐标信息，或者，位置标识。预先配置了声场中每个位置的标识，通过标识可以唯一确定具体的收听位置。
对应于S401a，S401具体可以实现为：编码端从解码端接收收听位置信息，收听位置信息用于指示收听位置，编码端根据收听位置信息确定收听者在声场中所处的位置。
在实时音频信号处理场景中，解码端可以向解码端提供一个或多个收听位置，本申请实施例对此不进行具体限定。
二、非实时音频信号处理场景。
在非实时音频信号处理场景，收听位置可以是声场中的任意一个位置，因此，在非实时音频信号处理的场景中，编码端需要预先采集多个可能的收听位置的音频信号，此时收听位置可以是多个预设位置中的任意一个。
因此，在非实时音频信号处理场景中，S401具体可以实现为：编码端确定声场中每一个可能的收听位置。具体地，可以根据多个预设位置来确定每一个可能的收听位置，也可以根据当前处理场景的情况，采用预设算法确定每一个可能的收听位置。
可选的，本申请实施例提供的音频信号处理方法，可以仅考虑收听位置，那么在S401之后则执行S403。或者，本申请实施例提供的音频信号处理方法，可以在收听位置的基础上参考收听朝向，那么在S401之后需先执行S402再执行S403。
S402、编码端获取收听朝向。
其中，收听朝向是指用户在收听位置面朝的方向，由于用户的移动性，收听朝向也不是固定不变的。
下面分实时音频信号处理场景和非实时音频信号处理场景对S402的实现进行说明。
在实时音频信号处理的场景，收听朝向由解码端向编码端提供，因此，在S402之前，如图4所示，本申请实施例提供的音频信号处理方法还可以包括S402a。
S402a、解码端获取收听朝向，向编码端发送收听朝向信息。
具体的，解码端可以使用传感器或者用于获取朝向的装置，获取收听者当前在声场中所处的位置处面朝的方向作为收听朝向，本申请实施例对于解码端获取收听朝向的实现方式不进行具体限定。
其中，收听朝向信息用于指示解码端获取的收听朝向，收听朝向信息的内容可以根据实际需求配置，本申请实施例对此不进行限定。
示例性的，收听朝向信息可以为朝向标识，或者，收听朝向信息可以为与基准朝向之间的夹角。
对应于S402a，S402具体可以实现为：编码端从解码端接收收听朝向信息，编码端根据收听朝向信息确定收听者在声场中的收听朝向。
需要说明的是，在实时音频信号处理场景中，S401和S402可以合并实现，本申请对该过程不进行具体限定。
在非实时音频信号处理的场景，收听朝向是指每个收听位置的每一种可能的朝向，因此，在非实时音频信号处理的场景中存在多个收听位置，在每个收听位置存在多个收听朝向。
因此，在非实时信号处理场景中，S402具体可以实现为：编码端确定声场中每一个可能的收听位置处每一个收听朝向。
S403、编码端采集声场中收听位置处的音频信号，采集的音频信号至少包括了满足筛选条件的音频信号。
其中，S403中每个采集的音频信号在声场中有对应的声源。
具体的，在S403中，编码端仅采集声场中收听位置处满足筛选条件的音频信号。
需要说明的是，当存在多个收听位置时，编码端在每个收听位置执行S403，此处仅描述一个收听位置处执行S403的过程，但并不是具体限定。
具体的，编码端通过至少一个音频信号采集装置执行S403。其中，音频信号采集装置可以为麦克风或者其他装置，本申请实施例对此不进行具体限定。
其中，筛选条件可以包括下述条件中一项或多项：在收听位置处音量大于或等于音量阈值，或者，音频信号对应的声源与收听位置的收听距离小于或等于预设收听距离，或者，音频信号对应的声源处于收听朝向对应的范围内。当然，还可以根据实际需求配置筛选条件的内容，本申请实施例对此不进行具体限定。
一种可能的实现中，当音量超过一定门限的音频信号才能被听到，并结合声源与收听朝向的位置关系不同时该门限不同，可以配置音量阈值为声源与收听朝向的位置关系确定的阈值。
一种可能的实现中，可以预先配置不同的位置关系对应的音量阈值，在S403中根据位置关系确定音量阈值。
其中，位置关系可以是指声源与收听朝向的相对方向关系、距离关系中的至少一种。例如，声源位于收听朝向的前方或者后方，声源位于收听朝向的左侧或右侧。或者，位置关系可以是指声源与收听朝向的角度关系，例如，声源位于收听朝向的右侧50度，声源位于收听右侧0度(正前方)。
例如，如表1所示，示例了一种预设的位置关系与音量门限的对应关系，在实际应用中，根据查找法，获取与位置关系对应的音量阈值。例如，当声源位于收听朝向的右侧30度，且声源距离收听位置直线距离为0.35米时，查找表1，确定对应的音量阈值为阈值20。
表1
Figure PCTCN2019128908-appb-000006
需要说明的是，表1只是以举例的形式说明一种音量阈值的确定方式，并不是具体限定。
另一种可能的实现中，可以预先配置音量阈值关于位置关系的预设算法，然后根据声源与收听位置的位置关系，按照预设算法获取音量阈值。
当然，对于获取不同的位置关系对应的音量阈值的具体实现，可以根据实际需求选择实现方案，本申请实施例不进行具体限定。
另一种可能的实现中，当音量超过一定门限的声源才能被听到，可以配置音量阈值为预设音量阈值，以采集音量大的音频信号。在该可能的实现中，不考虑收听朝向对收听效果的影响。
可以理解的是，该音量阈值也可以由收听者设定后由解码端发送给编码端。同理，收听距离也可以由收听者设定后由解码端发送给编码端。
需要说明的是，对于音量阈值的具体取值，可以根据实际需求配置，本申请实施例对此不进行具体限定。
其中，收听朝向对应的范围可以包括全向或者预设角度的范围，本申请对此不进行具体限定。对于预设角度的具体取值，可以根据实际需求配置，本申请实施例对此不进行具体限定。
S404、编码端将采集的音频信号进行编码，以获得经过编码的音频信号。
需要说明的是，对于编码过程，可以参照图1示意的音频信号处理过程中的编码，此处不再进行赘述。
S405、编码端向解码端发送经过编码的音频信号。
具体的，编码端采用图2示意的影音系统10中的传输模块205向解码端发送经过编码的音频信号，该传输模块205可以为传输线或者其他，本申请实施例对此不进行具体限定。其中，经过编码的多个音频信号也可以称为音频信号集合。
S406、解码端接收经过编码的多个音频信号。
需要说明的是，S406中解码端接收的经过编码的多个音频信号，即为S405中编码端发送的经过编码的音频信号。
其中，该经过编码的多个音频信号仅包括声场中收听位置处满足筛选条件的音频信号。此处的收听位置可以为实时音频信号处理场景中解码端提供的收听位置，也可以包括非实时音频信号处理场景中声场中每一个收听位置。
S407、解码端对满足筛选条件的音频信号进行解码以得到解码信号。
具体的，由于S406中接收的经过编码的多个音频信号中仅包括满足筛选条件的声源信号，在S407中，解码端对接收的每个音频信号进行解码以得到解码信号。对于解码过程，可以参照图1示意的音频信号处理过程中的解码，此处不再进行赘述。
S408、解码端根据播放设备中的声道传输函数渲染解码信号，获取输出信号。
具体的，在S408中解码端将每个解码信号，分别与其在播放设备中的声道传输函数进行卷积，获取一个或多个输出信号。
具体的，S408中是将解码信号进行渲染，如图1示意的音频信号信号处理过程，如果播放设备为扬声器阵列，则将解码信号渲染为多声道扬声器信号；如果播放设备为耳机设备，则将解码信号渲染为双耳输出信号。
可选的，声道传输函数可以包括：HRTF函数，或者，BRIR函数。当然，可以根据实际需求配置声道传输函数的类型，本申请实施例对此不进行具体限定。
在S408之后，解码端即可通过播放设备播放输出信号。当然，播放设备可以在解码端内部，也可以在解码端外部，本申请实施例对此不进行具体限定。
通过本申请提供的音频信号处理方法，编码端仅采集声场中满足筛选条件的音频信号，再编码发送给解码端，解码端解码接收到的满足筛选条件的音频信号，这样一来，通过配置保证音频信号质量的筛选条件，使得编码端采集的音频信号既保证音频质量，又无需解码端解码所有的音频信号，在声场中声源数量比较大时，与现有技术中解码所有声源信号相比，大大降低了解码端的功耗。
再一方面，本申请实施例提供另一种音频信号处理方法，图2示意的影音系统10中端到端的音频信号处理部分的编码端与解码端交互过程中，以处理声场中的音频信号。下文中描述的编码端/解码端执行某操作，可以为编码端/解码端自身执行该操作，也可以为编码端/解码端中的功能模块执行该操作，后续不再一一说明。
如图5所示，该音频信号处理方法可以包括：
S501、编码端获取收听位置。
具体的，S501与S401相同，具体实现可以参照S401，此处不再进行赘述。
可选的，本申请实施例提供的音频信号处理方法，可以仅考虑收听位置，那么在S501之后则执行S503。或者，本申请实施例提供的音频信号处理方法，可以在收听位置的基础上参考收听朝向，那么在S501之后需先执行S502再执行S503。
S502、编码端获取收听朝向。
具体的，S502与S402相同，具体实现可以参照S402，此处不再进行赘述。
S503、编码端采集声场中收听位置处所有声源对应的音频信号。
需要说明的是，当存在多个收听位置时，编码端在每个收听位置执行S503，此处仅描述一个收听位置处执行S503的过程，但并不是具体限定。
具体的，编码端通过至少一个音频信号采集装置执行S503。其中，音频信号采集装置可以为麦克风或者其他装置，本申请实施例对此不进行具体限定。
S504、编码端将采集的音频信号进行编码，以获得经过编码的音频信号。
需要说明的是，对于编码过程，可以参照图1示意的音频信号处理过程中的编码，此处不再进行赘述。
S505、编码端确定用于指示经过编码的音频信号中满足筛选条件的音频信号的指示信息。
需要说明的是，对于筛选条件，已经在S403中进行了详细描述，此处不再进行赘述。
一种可能的实现中，该指示信息可以包括经过编码的音频信号中每个音频源信号的特征信息，其中，该特征信息包括音量信息或者音频信号对应的声源的位置信息。
具体的，编码端可以通过传感器或者其他装置采集音频信号的特征信息，本申请对于具体实现不进行限定。
其中，音量信息用于指示声源音量的大小，音量信息可以为具体的音量值，或者，音量信息也可以为音量等级，当然，音量信息也可以为其他内容，只要可以用于指示 声源音量大小的信息，均属于此处描述的音量信息。
音量信号对应的声源的位置信息用于指示声源在声场中的具体位置，位置信息可以为坐标或者位置的标识，当然，位置信息也可以为其他内容，只要可以用于指示声源在声场中的具体位置的信息，均属于此处描述的位置信息。
另一种可能的实现中，该指示信息可以包括经过编码的音频信号中满足筛选条件的音频信号的标识。
其中，音频信号的标识用于唯一指示音频信号，本申请对于音频信号的标识的类型不进行具体限定。例如，音频信号的标识可以为音频信号在经过编码的音频信号中的位置或者顺序。
S506、编码端向解码端发送经过编码的音频信号。
具体的，编码端采用图2示意的影音系统10中的传输模块205向解码端发送经过编码的音频信号，该传输模块205可以为传输线或者其他，本申请实施例对此不进行具体限定。
S507、编码端向解码端发送指示信息。
需要说明的是，S506和S507可以合并执行，也可以单独执行，本申请对此不进行具体限定。当S506和S507单独执行时，对于两者的先后顺序也不进行具体限定，图5中仅为示例。
S508、解码端接收经过编码的多个音频信号。
需要说明的是，S508中解码端接收的经过编码的多个音频信号，即为S506中编码端发送的经过编码的音频信号。
其中，S508中接收的经过编码的多个音频信号，不仅包括声场中收听位置处满足筛选条件的音频信号，还包括其他不满足筛选条件的音频信号。
S509、编码端接收编码端发送的指示信息。
需要说明的是，S508和S509可以合并执行，也可以单独执行，本申请对此不进行具体限定。当S508和S509单独执行时，对于两者的先后顺序也不进行具体限定，图5中仅为示例。
S510、解码端对满足筛选条件的音频信号进行解码以得到解码信号。
在S510中，解码端根据接收的指示信息，从S508中接收的经过编码的多个音频信号中筛选，以获取收听位置处满足筛选条件的音频信号，进行解码以得到解码信号。
一种可能的实现中，当指示信息为多个音频信号中每个音频信号的特征信息时，S510中编码端根据特征信息判断每个音频信号是否满足筛选条件，筛选满足筛选条件的音频信号。
另一种可能的实现中，当指示信息为满足筛选条件的音频信号的标识时，S510中编码端筛选指示信息指示的音频信号，作为满足筛选条件的音频信号。
S511、解码端根据播放设备中的声道传输函数渲染解码信号，获取输出信号。
具体的，S511与S408相同，具体实现可以参照S408，此处不再进行赘述。
通过本申请提供的音频信号处理方法，编码端采集所有声源对应的音频信号编码发送给解码端的同时，通过指示信息向解码端指示满足筛选条件的音频信号，解码端根据编码端提供的指示信息，选择满足筛选条件的音频信号解码，只要配置保证音频 信号质量的筛选条件，在声场中声源数量比较大时，音频信号的传输既保证了音频信号质量，又大大降低了解码端的功耗。
需要说明的是，本申请实施例提供的音频信号处理方法中包括的各个步骤的执行先后顺序，可以根据需求配置，附图中的执行顺序只是示例，并不是具体限定。
上述主要从各个网元之间交互的角度对本申请实施例提供的方案进行了介绍。可以理解的是，上述解码端、编码端为了实现上述功能，其包含了执行各个功能相应的硬件结构和/或软件模块。将解码端、编码端中实现上述音频信号处理方法的功能单元，称之为音频信号处理装置。本领域技术人员应该很容易意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，本申请能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
本申请实施例可以根据上述方法示例对音频信号处理装置进行功能模块的划分，例如，可以对应各个功能划分各个功能模块，也可以将两个或两个以上的功能集成在一个处理模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。需要说明的是，本申请实施例中对模块的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。
在采用对应各个功能划分各个功能模块的情况下，图6示出了上述实施例中所涉及的编码端中部署的音频信号处理装置60的一种可能的结构示意图。音频信号处理装置60可以部署在影音系统10中的编码端作为功能单元，该音频信号处理装置60可以为编码端本身，也可以为编码端中的功能模块或者芯片。如图6所示，音频信号处理装置60可以包括：采集单元601、编码单元602、发送单元603。采集单元601用于执行图4中的过程S403、图5中的过程S503；编码单元602用于执行图4中的过程S404或图5中的过程S504；发送模块603用于执行图4中的过程S405或图5中的过程S506、S507。其中，上述方法实施例涉及的各步骤的所有相关内容均可以援引到对应功能模块的功能描述，在此不再赘述。当将所述信号处理装置60部署在影音系统10中的编码端时，所述采集单元601可以包括影音系统10中的编码端中的采集模块201，编码单元602可以包括影音系统10中的编码端中的音频编码模块203，发送单元603可以包括归属于影音系统10中的编码端中的传输模块205。
进一步的，如图6所示，音频信号处理装置60还可以包括接收单元604、获取单元605。获取单元605可以通过接收单元604执行图4中的过程S401、S402或图5中的过程S501、S502；或者，获取单元605可以用于执行图4中的过程S401、S402或图5中的过程S501、S502。
在采用集成的单元的情况下，图7示出了上述实施例中所涉及的音频信号处理装置70的一种可能的结构示意图。音频信号处理装置70可以部署在影音系统10中的编码端作为功能单元，该音频信号处理装置70也可以为编码端本身，或者，也可以为编码端中的功能模块或者芯片。音频信号处理装置70可以包括：处理模块701、通信模块702。处理模块701用于对音频信号处理装置70的动作进行控制管理。例如，处理模块701用于执行图4中的过程S401、S402、S403、S404，或者，图5中的过程S501、 S502、S503、S504、S505；通信模块702用于执行图4中的过程S405，或图5中的过程S506、S507。音频信号处理装置70还可以包括存储模块703，用于存储音频信号处理装置70的程序代码和数据。当将所述信号处理装置70部署在影音系统10中的编码端时，所述处理模块701可以包括影音系统10中的编码端中的采集模块201、音频预处理模块202、音频编码模块203，通信模块702可以包括归属于影音系统10中的编码端中的传输模块205。
其中，处理模块701可以为图3所示的音频信号处理装置30的实体结构中的处理器301，可以是处理器或控制器。例如可以是CPU，通用处理器，DSP，ASIC，FPGA或者其他可编程逻辑器件、晶体管逻辑器件、硬件部件或者其任意组合。其可以实现或执行结合本申请公开内容所描述的各种示例性的逻辑方框，模块和电路。处理器701也可以是实现计算功能的组合，例如包含一个或多个微处理器组合，DSP和微处理器的组合等等。通信模块702可以为图3所示的音频信号处理装置30的实体结构中的收发器303，通信模块702可以是通信端口，或者可以是收发器、收发电路或通信接口等。或者，上述通信接口可以通过上述具有收发功能的元件，实现与其他设备的通信。上述具有收发功能的元件可以由天线和/或射频装置实现。存储模块703可以是图3所示的音频信号处理装置30的实体结构中的存储器302。
当处理模块701为处理器，通信模块702为收发器，存储模块703为存储器时，本申请实施例图7所涉及的音频信号处理装置70可以为图3所示的音频信号处理装置30。
如前述，本申请实施例提供的音频信号处理装置60或音频信号处理装置70可以用于实施上述本申请各实施例实现的方法中编码端的功能，为了便于说明，仅示出了与本申请实施例相关的部分，具体技术细节未揭示的，请参照本申请各实施例。
在采用对应各个功能划分各个功能模块的情况下，图8示出了上述实施例中所涉及的解码端中部署的音频信号处理装置80的一种可能的结构示意图。音频信号处理装置80可以部署在影音系统10中的解码端作为功能单元，该音频信号处理装置80可以为解码端本身，也可以为解码端中的功能模块或者芯片。如图8所示，音频信号处理装置80可以包括：接收单元801、解码单元802、处理单元803。接收单元801用于执行图4中的过程S406，或图5中的过程S508、S509；解码单元802用于执行图4中的过程S407、或图5中的过程S510；处理单元803用于执行图4中的过程S408、或图5中的过程S511。其中，上述方法实施例涉及的各步骤的所有相关内容均可以援引到对应功能模块的功能描述，在此不再赘述。当将所述信号处理装置80部署在影音系统10中的解码端时，所述接收单元801可以包括影音系统10中的解码端中的传输模块218，解码单元802可以包括影音系统10中的解码端中的音频解码模块207，处理单元803可以包括影音系统10中的解码端中的音频渲染模块208。
进一步的，如图8所示，音频信号处理装置80还可以包括获取单元804、发送单元805。其中，获取单元804及发送单元805用于执行图4中的过程S401a、S402a。
在采用集成的单元的情况下，图9示出了上述实施例中所涉及的解码端中部署的音频信号处理装置90的一种可能的结构示意图。音频信号处理装置90可以部署在影音系统10中的解码端作为功能单元，该音频信号处理装置90可以为解码端本身，也 可以为解码端中的功能模块或者芯片。音频信号处理装置90可以包括：处理模块901、通信模块902。处理模块901用于对音频信号处理装置90的动作进行控制管理。例如，处理模块901用于执行图4中的过程S407、S408、S401a，或者，图5中的过程S510、S511；通信模块902用于执行图4中的过程S406，或图5中的过程S508、S509。音频信号处理装置90还可以包括存储模块903，用于存储音频信号处理装置90的程序代码和数据。当将所述信号处理装置90部署在影音系统10中的解码端时，所述处理模块901可以包括影音系统10中的解码端中的音频解码模块207、音频渲染模块208，通信模块902可以包括影音系统10中的传输模块218。
其中，处理模块901可以为图3所示的音频信号处理装置30的实体结构中的处理器301，可以是处理器或控制器。例如可以是CPU，通用处理器，DSP，ASIC，FPGA或者其他可编程逻辑器件、晶体管逻辑器件、硬件部件或者其任意组合。其可以实现或执行结合本申请公开内容所描述的各种示例性的逻辑方框，模块和电路。处理器901也可以是实现计算功能的组合，例如包含一个或多个微处理器组合，DSP和微处理器的组合等等。通信模块902可以为图3所示的音频信号处理装置30的实体结构中的收发器303，通信模块902可以是通信端口，或者可以是收发器、收发电路或通信接口等。或者，上述通信接口可以通过上述具有收发功能的元件，实现与其他设备的通信。上述具有收发功能的元件可以由天线和/或射频装置实现。存储模块903可以是图3所示的音频信号处理装置30的实体结构中的存储器302。
当处理模块901为处理器，通信模块902为收发器，存储模块903为存储器时，本申请实施例图9所涉及的音频信号处理装置90可以为图3所示的音频信号处理装置30。
如前述，本申请实施例提供的音频信号处理装置80或音频信号处理装置90可以用于实施上述本申请各实施例实现的方法中解码端的功能，为了便于说明，仅示出了与本申请实施例相关的部分，具体技术细节未揭示的，请参照本申请各实施例。
作为本实施例的另一种形式，提供一种计算机可读存储介质，其上存储有指令，该指令被执行时执行上述方法实施例中的音频信号处理方法。
作为本实施例的另一种形式，提供一种包含指令的计算机程序产品，该指令被执行时执行上述方法实施例中的音频信号处理方法。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统，装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
本领域技术人员应该可以意识到，在上述一个或多个示例中，本申请所描述的功能可以用硬件、软件、固件或它们的任意组合来实现。当使用软件实现时，可以将这些功能存储在计算机可读介质中或者作为计算机可读介质上的一个或多个指令或代码进行传输。计算机可读介质包括计算机存储介质和通信介质，其中通信介质包括便于从一个地方向另一个地方传送计算机程序的任何介质。存储介质可以是通用或专用计算机能够存取的任何可用介质。所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统，装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统，装置和方法，可 以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理包括，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用硬件加软件功能单元的形式实现。
上述以软件功能单元的形式实现的集成的单元，可以存储在一个计算机可读取存储介质中。上述软件功能单元存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
最后应说明的是：以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围。

Claims (29)

  1. 一种音频信号处理方法，其特征在于，包括：
    采集收听位置处的音频信号，每个采集的音频信号有对应的声源，所述采集的音频信号至少包括了满足筛选条件的音频信号；其中，所述筛选条件包括下述条件中一项或多项：在所述收听位置处的音量大于或等于音量阈值，或者，声源与所述收听位置的收听距离小于或等于预设收听距离，或者，声源处于收听朝向对应的范围内；
    将采集的音频信号进行编码，以获得经过编码的音频信号；
    向解码端发送所述经过编码的音频信号。
  2. 根据权利要求1所述的方法，其特征在于，所述音量阈值包括：
    预设音量阈值，或者，根据声源与所述收听朝向的位置关系确定的阈值。
  3. 根据权利要求1或2所述的方法，其特征在于，在所述采集收听位置处的音频信号前，所述方法还包括：
    接收所述解码端发送的收听位置信息，所述收听位置信息用于指示所述收听位置。
  4. 根据权利要求1-3任一项所述的方法，其特征在于，所述采集的音频信号仅包括满足筛选条件的音频信号，当所述筛选条件包括所述声源处于收听朝向对应的范围内时，在所述采集收听位置处的音频信号前，所述方法还包括：
    接收所述解码端发送的收听朝向信息，所述收听朝向信息用于指示所述收听朝向。
  5. 根据权利要求1-4任一项所述的方法，其特征在于，所述采集的音频信号包括了不满足所述筛选条件的音频信号，所述方法还包括：
    向所述解码端发送指示信息，所述指示信息用于指示所述经过编码的音频信号中满足所述筛选条件的音频信号。
  6. 根据权利要求5所述的方法，其特征在于，所述指示信息包括：
    所述经过编码的音频信号中每个音频信号的特征信息；其中，所述特征信息包括音量信息或者音频信号对应的声源的位置信息；
    或者，
    所述经过编码的音频信号中满足筛选条件的音频信号的标识。
  7. 一种音频信号处理方法，其特征在于，包括：
    接收经过编码的多个音频信号，所述多个音频信号是收听位置处的音频信号，每个音频信号有对应的声源，所述多个音频信号至少包括满足筛选条件的音频信号；其中，所述筛选条件包括下述条件中一项或多项：在所述收听位置处的音量大于或等于音量阈值，或者，声源与所述收听位置的收听距离小于或等于预设收听距离，或者，声源处于收听朝向对应的范围内；
    对满足所述筛选条件的音频信号进行解码以得到解码信号；
    根据播放设备中的声道传输函数渲染所述解码信号，获取输出信号。
  8. 根据权利要求7所述的方法，其特征在于，所述音量阈值包括：
    预设音量阈值，或者，根据声源与所述收听朝向的位置关系确定的阈值。
  9. 根据权利要求7或8所述的方法，其特征在于，在所述接收经过编码的多个音频信号前，所述方法还包括：
    向编码端发送收听位置信息，以使所述编码端根据所述收听位置信息获取所述经 过编码的多个音频信号，所述收听位置信息用于指示所述收听位置。
  10. 根据权利要求7-9任一项所述的方法，其特征在于，所述多个音频信号包括了不满足所述筛选条件的音频信号，所述对满足所述筛选条件的音频信号进行解码以得到解码信号前，所述方法还包括：
    接收编码端发送的指示信息，所述指示信息用于指示所述多个音频信号中满足所述筛选条件的音频信号；
    所述对满足所述筛选条件的音频信号进行解码以得到解码信号，包括：
    根据所述指示信息，从所述多个音频信号中筛选出满足所述筛选条件的音频信号；
    对筛选出的满足所述筛选条件的音频信号进行解码以得到所述解码信号。
  11. 根据权利要求10所述的方法，其特征在于，所述指示信息包括：
    所述多个音频信号中每个音频信号的特征信息，所述特征信息包括音量信息或者音频信号对应的声源的位置信息；
    或者，
    所述多个音频信号中满足所述筛选条件的音频信号的标识。
  12. 根据权利要求7-11任一项所述的方法，其特征在于，所述多个音频信号仅包括满足所述筛选条件的音频信号，当所述筛选条件包括声源处于收听朝向对应的范围内时，在所述接收经过编码的多个音频信号前，所述方法还包括：
    向编码端发送收听朝向信息，以使所述编码端根据所述朝向信息获取所述经过编码的多个音频信号，所述收听朝向信息用于指示所述收听朝向。
  13. 根据权利要求7-12任一项所述的方法，其特征在于，所述声道传输函数包括：
    头部相关传输HRTF函数，或者，双耳房间冲激响应BRIR函数。
  14. 一种音频信号处理装置，其特征在于，包括：
    采集单元，用于采集收听位置处的音频信号，每个采集的音频信号有对应的声源，所述采集的音频信号至少包括了满足筛选条件的音频信号；其中，所述筛选条件包括下述条件中一项或多项：在所述收听位置处的音量大于或等于音量阈值，或者，声源与所述收听位置的收听距离小于或等于预设收听距离，或者，声源处于收听朝向对应的范围内；
    编码单元，用于将采集的音频信号进行编码，以获得经过编码的音频信号；
    发送单元，用于向解码端发送所述经过编码的音频信号。
  15. 根据权利要求14所述的装置，其特征在于，所述音量阈值包括：
    预设音量阈值，或者，根据声源与所述收听朝向的位置关系确定的阈值。
  16. 根据权利要求14或15所述的装置，其特征在于，所述装置还包括：
    接收单元，用于接收所述解码端发送的收听位置信息，所述收听位置信息用于指示所述收听位置。
  17. 根据权利要求14-16任一项所述的装置，其特征在于，所述采集的音频信号仅包括满足筛选条件的音频信号，当所述筛选条件包括所述声源处于收听朝向对应的范围内时，所述装置还包括：
    接收单元，用于接收所述解码端发送的收听朝向信息，所述收听朝向信息用于指示所述收听朝向。
  18. 根据权利要求14-17任一项所述的装置，其特征在于，所述采集的音频信号包括了不满足所述筛选条件的音频信号，所述发送单元还用于：
    向所述解码端发送指示信息，所述指示信息用于指示所述经过编码的音频信号中满足所述筛选条件的音频信号。
  19. 根据权利要求18所述的装置，其特征在于，所述指示信息包括：
    所述经过编码的音频信号中每个音频信号的特征信息；其中，所述特征信息包括音量信息或者音频信号对应的声源的位置信息；
    或者，
    所述经过编码的音频信号中满足筛选条件的音频信号的标识。
  20. 一种音频信号处理装置，其特征在于，包括：
    接收单元，用于接收经过编码的多个音频信号，所述多个音频信号是收听位置处的音频信号，每个音频信号有对应的声源，所述多个音频信号至少包括满足筛选条件的音频信号；其中，所述筛选条件包括下述条件中一项或多项：在所述收听位置处的音量大于或等于音量阈值，或者，声源与所述收听位置的收听距离小于或等于预设收听距离，或者，声源处于收听朝向对应的范围内；
    解码单元，用于对满足所述筛选条件的音频信号进行解码以得到解码信号；
    渲染单元，用于根据播放设备中的声道传输函数，渲染所述解码单元得到的所述解码信号，获取输出信号。
  21. 根据权利要求20所述的装置，其特征在于，所述音量阈值包括：
    预设音量阈值，或者，根据声源与所述收听朝向的位置关系确定的阈值。
  22. 根据权利要求20或21所述的装置，其特征在于，所述装置还包括：
    发送单元，用于向编码端发送收听位置信息，以使所述编码端根据所述收听位置信息获取所述经过编码的多个音频信号，所述收听位置信息用于指示所述收听位置。
  23. 根据权利要求20-22任一项所述的装置，其特征在于，所述多个音频信号包括了不满足所述筛选条件的音频信号，所述接收单元还用于：
    接收编码端发送的指示信息，所述指示信息用于指示所述多个音频信号中满足所述筛选条件的音频信号；
    所述解码单元具体用于：根据所述接收单元接收的所述指示信息，从所述多个音频信号中筛选出满足所述筛选条件的音频信号，对筛选出的满足所述筛选条件的音频信号进行解码以得到所述解码信号。
  24. 根据权利要求23所述的装置，其特征在于，所述指示信息包括：
    所述多个音频信号中每个音频信号的特征信息，所述特征信息包括音量信息或者音频信号对应的声源的位置信息；
    或者，
    所述多个音频信号中满足所述筛选条件的音频信号的标识。
  25. 根据权利要求20-24任一项所述的装置，其特征在于，所述多个音频信号仅包括满足所述筛选条件的音频信号，当所述筛选条件包括声源处于收听朝向对应的范围内时，所述装置还包括发送单元，用于：
    向编码端发送收听朝向信息，以使所述编码端根据所述朝向信息获取所述经过编 码的多个音频信号，所述收听朝向信息用于指示所述收听朝向。
  26. 根据权利要求20-25任一项所述的装置，其特征在于，所述声道传输函数包括：
    头部相关传输HRTF函数，或者，双耳房间冲激响应BRIR函数。
  27. 一种音频信号处理装置，其特征在于，包括：包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的程序，其特征在于，所述处理器执行所述程序时实现权利要求1-13中任一项所述的音频信号处理方法。
  28. 一种计算机可读存储介质，其特征在于，包括指令，当其在计算机上运行时，使得计算机执行如权利要求1-13中任一项所述的音频信号处理方法。
  29. 一种计算机程序，用于使得计算机执行如权利要求1-13中任一项所述的音频信号处理方法。
PCT/CN2019/128908 2019-01-31 2019-12-26 一种音频信号处理方法及装置 Ceased WO2020155976A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
EP19912993.3A EP3909656A4 (en) 2019-01-31 2019-12-26 AUDIO SIGNAL PROCESSING METHOD AND APPARATUS
KR1020217027277A KR102696575B1 (ko) 2019-01-31 2019-12-26 오디오 신호 처리 방법 및 장치
BR112021015040-4A BR112021015040B1 (pt) 2019-01-31 2019-12-26 Método e aparelho de processamento de sinal de áudio e meio de armazenamento legível por computador
US17/386,800 US20210358506A1 (en) 2019-01-31 2021-07-28 Audio signal processing method and apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910103595.6A CN111508507B (zh) 2019-01-31 2019-01-31 一种音频信号处理方法及装置
CN201910103595.6 2019-01-31

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/386,800 Continuation US20210358506A1 (en) 2019-01-31 2021-07-28 Audio signal processing method and apparatus

Publications (1)

Publication Number Publication Date
WO2020155976A1 true WO2020155976A1 (zh) 2020-08-06

Family

ID=71841612

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/128908 Ceased WO2020155976A1 (zh) 2019-01-31 2019-12-26 一种音频信号处理方法及装置

Country Status (5)

Country Link
US (1) US20210358506A1 (zh)
EP (1) EP3909656A4 (zh)
KR (1) KR102696575B1 (zh)
CN (2) CN116312572A (zh)
WO (1) WO2020155976A1 (zh)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114420160B (zh) * 2020-10-28 2025-08-29 华为技术有限公司 处理音频信号的方法和装置
DE102021201668A1 (de) * 2021-02-22 2022-08-25 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung eingetragener Verein Signaladaptives Neumischen von getrennten Audioquellen
CN116939473A (zh) * 2022-03-30 2023-10-24 腾讯科技（深圳）有限公司 一种音频生成方法和相关装置
CN116724351B (zh) * 2023-03-27 2026-06-12 北京小米移动软件有限公司 一种量化编码方法、装置、设备及存储介质
WO2025173919A1 (ko) * 2024-02-16 2025-08-21 삼성전자 주식회사 근거리 음성을 제거하기 위한 오디오 신호 처리 방법 및 전자 장치

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105959905A (zh) * 2016-04-27 2016-09-21 北京时代拓灵科技有限公司 混合模式空间声生成系统与方法
CN106843801A (zh) * 2017-01-10 2017-06-13 福建省天奕网络科技有限公司 音效的拟合方法及其系统
WO2018084770A1 (en) * 2016-11-04 2018-05-11 Dirac Research Ab Methods and systems for determining and/or using an audio filter based on head-tracking data
CN108744516A (zh) * 2018-05-29 2018-11-06 腾讯科技（深圳）有限公司 获取定位信息的方法和装置、存储介质及电子装置

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5659691A (en) * 1993-09-23 1997-08-19 Virtual Universe Corporation Virtual reality network with selective distribution and updating of data to reduce bandwidth requirements
US5736982A (en) * 1994-08-03 1998-04-07 Nippon Telegraph And Telephone Corporation Virtual space apparatus with avatars and speech
AUPR989802A0 (en) * 2002-01-09 2002-01-31 Lake Technology Limited Interactive spatialized audiovisual system
CN101350931B (zh) * 2008-08-27 2011-09-14 华为终端有限公司 音频信号的生成、播放方法及装置、处理系统
US20130226324A1 (en) * 2010-09-27 2013-08-29 Nokia Corporation Audio scene apparatuses and methods
CA2837893C (en) * 2011-07-01 2017-08-29 Dolby Laboratories Licensing Corporation System and method for adaptive audio signal generation, coding and rendering
EP2839460A4 (en) * 2012-04-18 2015-12-30 Nokia Technologies Oy STEREOTONSIGNALCODIERER
US10986457B2 (en) * 2017-07-09 2021-04-20 Lg Electronics Inc. Method and device for outputting audio linked with video screen zoom
GB2567172A (en) * 2017-10-04 2019-04-10 Nokia Technologies Oy Grouping and transport of audio objects
KR102653185B1 (ko) * 2018-04-12 2024-04-02 소니그룹주식회사 정보 처리 장치 및 방법, 그리고 프로그램이 저장된 컴퓨터 판독 가능한 기록매체
JP2023515886A (ja) * 2020-03-02 2023-04-14 マジック リープ， インコーポレイテッド 没入型のオーディオプラットフォーム

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105959905A (zh) * 2016-04-27 2016-09-21 北京时代拓灵科技有限公司 混合模式空间声生成系统与方法
WO2018084770A1 (en) * 2016-11-04 2018-05-11 Dirac Research Ab Methods and systems for determining and/or using an audio filter based on head-tracking data
CN106843801A (zh) * 2017-01-10 2017-06-13 福建省天奕网络科技有限公司 音效的拟合方法及其系统
CN108744516A (zh) * 2018-05-29 2018-11-06 腾讯科技（深圳）有限公司 获取定位信息的方法和装置、存储介质及电子装置

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
See also references of EP3909656A4
XIN LI; DEHUI KONG; YONG ZHANG: "OsgAL-Based Sound Source Management in Simulation Games", INTELLIGENCE COMPUTATION AND EVOLUTIONARY COMPUTATION, vol. 180, 13 August 2012 (2012-08-13), pages 723 - 732, XP009529492, ISBN: 978-3-642-31655-5, DOI: 10.1007/978-3-642-31656-2_97 *

Also Published As

Publication number Publication date
EP3909656A4 (en) 2022-03-23
US20210358506A1 (en) 2021-11-18
CN111508507A (zh) 2020-08-07
BR112021015040A2 (pt) 2021-10-05
CN111508507B (zh) 2023-03-03
KR20210120063A (ko) 2021-10-06
KR102696575B1 (ko) 2024-08-19
CN116312572A (zh) 2023-06-23
EP3909656A1 (en) 2021-11-17

Similar Documents

Publication Publication Date Title
CN111385728B (zh) 一种音频信号处理方法及装置
TWI819344B (zh) 音訊訊號渲染方法、裝置、設備及電腦可讀存儲介質
CN111508507B (zh) 一种音频信号处理方法及装置
CN115211146B (zh) 音频表示和相关联的渲染
US11109177B2 (en) Methods and systems for simulating acoustics of an extended reality world
WO2022021898A1 (zh) 音频处理方法、装置、系统以及存储介质
CN117396957A (zh) 基于链路数据的音频编码
US12177644B2 (en) Signalling of audio effect metadata in a bitstream
CN112313972B (zh) 用于音频呈现的装置和相关联的方法
CN116472725A (zh) 用于增强现实/虚拟现实音频的智能混合渲染
BR112021015040B1 (pt) Método e aparelho de processamento de sinal de áudio e meio de armazenamento legível por computador
WO2024212637A1 (zh) 场景音频解码方法及电子设备
GB2639006A (en) A multi-participant, spatial audio service
WO2024212634A1 (zh) 场景音频编码方法及电子设备
WO2024212894A1 (zh) 场景音频信号的解码方法和装置
WO2024212897A1 (zh) 场景音频信号的解码方法和装置
WO2025011361A1 (zh) 解码方法及电子设备
WO2025011383A1 (zh) 编码方法及电子设备
WO2024212895A1 (zh) 场景音频信号的解码方法和装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19912993

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

REG Reference to national code

Ref country code: BR

Ref legal event code: B01A

Ref document number: 112021015040

Country of ref document: BR

ENP Entry into the national phase

Ref document number: 2019912993

Country of ref document: EP

Effective date: 20210811

ENP Entry into the national phase

Ref document number: 20217027277

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 112021015040

Country of ref document: BR

Kind code of ref document: A2

Effective date: 20210730