WO2022247651A1 - 多声道音频信号的编码方法和装置 - Google Patents

多声道音频信号的编码方法和装置 Download PDF

Info

Publication number
WO2022247651A1
WO2022247651A1 PCT/CN2022/092518 CN2022092518W WO2022247651A1 WO 2022247651 A1 WO2022247651 A1 WO 2022247651A1 CN 2022092518 W CN2022092518 W CN 2022092518W WO 2022247651 A1 WO2022247651 A1 WO 2022247651A1
Authority
WO
WIPO (PCT)
Prior art keywords
channel
channel signals
energy
fluctuation interval
balance mode
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/092518
Other languages
English (en)
French (fr)
Inventor
王智
王喆
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to EP22810378.4A priority Critical patent/EP4336494A4/en
Publication of WO2022247651A1 publication Critical patent/WO2022247651A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S3/00—Systems employing more than two channels, e.g. quadraphonic
    • H04S3/008—Systems employing more than two channels, e.g. quadraphonic in which the audio signals are in digital form, i.e. employing more than two discrete digital channels
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
    • G10L19/16—Vocoder architecture
    • G10L19/18—Vocoders using multiple modes

Definitions

  • the present application relates to audio processing technologies, and in particular to a method and device for encoding multi-channel audio signals.
  • the codec of multi-channel audio is a technology for encoding or decoding audio containing more than two channels.
  • Common multi-channel audio includes 5.1-channel audio, 7.1-channel audio, 7.1.4-channel audio, and 22.2-channel audio.
  • MPS MPEG Surround
  • the present application provides a coding method and device for a multi-channel audio signal, so as to improve coding efficiency of audio frames.
  • the present application provides a method for encoding a multi-channel audio signal, comprising: acquiring a first audio frame to be encoded, the first audio frame including at least five channel signals; acquiring the target channel pair set The sum of correlation values of all channel pairs of the target channel pair set includes at least one channel pair, one channel pair includes two channel signals in the at least five channel signals, and the one channel pair The channel pair has a correlation value, and the correlation value is used to represent the correlation between the two channel signals of the one channel pair; when the sum of the correlation values is greater than a preset threshold, the at least five performing energy equalization processing on at least two channel signals among the channel signals to obtain at least two equalized channel signals; encoding the at least two equalized channel signals to obtain an encoded code stream.
  • At least five channel signals included in the audio frame are grouped to obtain a target channel pair set.
  • the sum of the correlation values of the target channel pair set is greater than a preset threshold.
  • the method further includes: when the sum of the correlation values is less than or equal to the preset threshold, encoding the at least five channel signals to obtain an encoded code stream .
  • the sum of the correlation values is less than or equal to the preset threshold, it means that the correlation between the two channel signals in the channel pair in the target channel pair set is low, and there is no need for group pair coding. Furthermore, there is no need to perform energy equalization processing on the at least five channel signals, and the object of encoding at this time is the at least five channel signals, not the equalized channel signals.
  • the performing energy equalization processing on at least two channel signals of the at least five channel signals to obtain at least two equalized channel signals includes: obtaining the at least five The fluctuation interval value of the channel signal; determine the energy balance mode according to the fluctuation interval value of the at least five channel signals; perform energy equalization processing on the at least two channel signals according to the energy balance mode to obtain the At least two equalized channel signals.
  • the fluctuation interval value is used to represent the difference in energy or amplitude between at least five channel signals.
  • the energy balance mode includes a first energy balance mode and a second energy balance mode, wherein the first energy balance mode uses two channel signals in a channel pair to obtain two equalized channel signals corresponding to a channel pair.
  • the second energy equalization mode uses two channel signals in a channel pair and at least one channel signal outside a channel to obtain two equalized channel signals corresponding to a channel pair.
  • the determining the energy balance mode according to the fluctuation interval values of the at least five channel signals includes: when the fluctuation interval values meet a preset condition, determining that the energy balance mode is The first energy balance mode; or, when the fluctuation interval value does not meet the preset condition, determine the energy balance mode as the second energy balance mode.
  • the fluctuation interval value includes the energy flatness of the first audio frame; the fluctuation interval value meeting a preset condition means that the energy flatness is smaller than a first threshold; or, the The fluctuation interval value includes the amplitude flatness of the first audio frame; the fluctuation interval value meets the preset condition means that the amplitude flatness is smaller than the second threshold; or, the fluctuation interval value includes the first audio frame
  • the energy deviation degree of the frame; the fluctuation interval value meets the preset condition means that the energy deviation degree is not within the first preset range; or, the fluctuation interval value includes the amplitude deviation degree of the first audio frame;
  • the fluctuation interval value meeting the preset condition means that the amplitude deviation degree is not within the second preset range.
  • performing energy balance processing on at least two channel signals among the at least five channel signals to obtain The at least two equalized channel signals include: performing energy equalization processing on the channel signals corresponding to the target channel pair set to obtain the at least two equalized channel signals.
  • the performing energy equalization processing on the channel signals corresponding to the target channel pair set to obtain the at least two equalized channel signals includes: for the target channel pair set The current channel pair in the current channel pair, calculate the average value of the energy value or amplitude value of the two channel signals contained in the current channel pair, and calculate the two channels contained in the current channel pair according to the average value The signal is subjected to energy equalization processing to obtain two corresponding equalized channel signals.
  • the coding noise of the channel pair with high energy may be much greater than that of the channel pair with low energy due to insufficient bits, and the bits of the channel pair with low energy will have redundant problems .
  • performing energy balance processing on at least two channel signals among the at least five channel signals to obtain At least two equalized channel signals including: calculating the average value of the energy value or amplitude value of the at least five channel signals, and performing energy equalization processing on the at least five channel signals according to the average value to obtain the obtained at least five equalized channel signals.
  • determining the energy balance mode according to the fluctuation interval values of the at least five channel signals before determining the energy balance mode according to the fluctuation interval values of the at least five channel signals, it further includes: judging whether the coding bit rate corresponding to the first audio frame is greater than the bit rate Threshold; when the code rate is greater than the code rate threshold, determine the energy balance mode as the second energy balance mode; when the code rate is less than or equal to the code rate threshold, only according to the fluctuation The interval value determines the energy balance mode.
  • the method further includes: encoding channel signals that have not been subjected to energy equalization processing among the at least five channel signals.
  • the present application provides an encoding device, including: an acquisition module configured to acquire a first audio frame to be encoded, the first audio frame including at least five channel signals;
  • the sum of correlation values of all channel pairs, the target channel pair set includes at least one channel pair, one channel pair includes two channel signals in the at least five channel signals, and the one channel
  • the pair has a correlation value, and the correlation value is used to represent the correlation between the two channel signals of the one channel pair;
  • the processing module is configured to, when the sum of the correlation values is greater than a preset threshold value, pair At least two of the at least five channel signals are subjected to energy equalization processing to obtain at least two equalized channel signals;
  • an encoding module is configured to encode the at least two equalized channel signals to obtain Encoded stream.
  • the encoding module is further configured to encode the at least five channel signals to obtain encoded stream.
  • the processing module is specifically configured to acquire fluctuation interval values of the at least five channel signals; determine an energy balance mode according to the fluctuation interval values of the at least five channel signals;
  • the energy equalization mode respectively performs energy equalization processing on the at least two channel signals to obtain the at least two equalized channel signals.
  • the processing module is specifically configured to determine that the energy balance mode is the first energy balance mode when the fluctuation interval value meets a preset condition; or, when the fluctuation interval value When the preset condition is not met, the energy balance mode is determined to be the second energy balance mode.
  • the fluctuation interval value includes the energy flatness of the first audio frame; the fluctuation interval value meeting a preset condition means that the energy flatness is smaller than a first threshold; or, the The fluctuation interval value includes the amplitude flatness of the first audio frame; the fluctuation interval value meets the preset condition means that the amplitude flatness is smaller than the second threshold; or, the fluctuation interval value includes the first audio frame
  • the energy deviation degree of the frame; the fluctuation interval value meets the preset condition means that the energy deviation degree is not within the first preset range; or, the fluctuation interval value includes the amplitude deviation degree of the first audio frame;
  • the fluctuation interval value meeting the preset condition means that the amplitude deviation degree is not within the second preset range.
  • the processing module is specifically configured to perform energy balance processing on channel signals corresponding to the target channel pair set to obtain the at least two equalized channel signals.
  • the processing module is specifically configured to, for the current channel pair in the target channel pair set, calculate the energy value or the energy value of the two channel signals contained in the current channel pair The average value of the amplitude value, according to the average value, performs energy equalization processing on the two channel signals included in the current channel pair to obtain two corresponding equalized channel signals.
  • the processing module is specifically configured to calculate an average of energy values or amplitude values of the at least five channel signals value, performing energy equalization processing on the at least five channel signals respectively according to the average value to obtain the at least five balanced channel signals.
  • the processing module is further configured to determine whether the encoding bit rate corresponding to the first audio frame is greater than a bit rate threshold; when the encoding bit rate is greater than the bit rate threshold, Determining the energy balance mode as the second energy balance mode; and determining the energy balance mode according to the fluctuation interval value when the encoding code rate is less than or equal to the code rate threshold.
  • the encoding module is further configured to encode channel signals that have not been subjected to energy equalization processing among the at least five channel signals.
  • the present application provides a device, including: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, The one or more processors are made to implement the method as described in any one of the above first aspects.
  • the present application provides a computer-readable storage medium, including a computer program.
  • the computer program When the computer program is executed on a computer, the computer executes the method described in any one of the above-mentioned first aspects.
  • the present application provides a computer-readable storage medium, including an encoded code stream obtained according to the method for encoding a multi-channel audio signal according to any one of the above-mentioned first aspects.
  • FIG. 1 exemplarily provides a schematic block diagram of an audio decoding system 10 applied in the present application
  • FIG. 2 exemplarily provides a schematic block diagram of an audio decoding device 200 applied in the present application
  • Fig. 3 is the flow chart of an exemplary embodiment of the encoding method of the multi-channel audio signal provided by the present application
  • Fig. 4a is an exemplary structural diagram of an encoding device applied to the encoding method of a multi-channel audio signal provided by the present application;
  • Fig. 4 b is an exemplary structural diagram of multi-channel adaptive group pair module
  • Fig. 4c is an exemplary structural diagram of the pair processing module
  • FIG. 5 is an exemplary structural diagram of a decoding device applied to the multi-channel audio decoding method provided by the present application
  • FIG. 6 is a schematic structural diagram of an embodiment of an encoding device of the present application.
  • Fig. 7 is a schematic structural diagram of an embodiment of the device of the present application.
  • At least one (item) means one or more, and “multiple” means two or more.
  • “And/or” is used to describe the association relationship of associated objects, indicating that there can be three types of relationships, for example, “A and/or B” can mean: only A exists, only B exists, and A and B exist at the same time , where A and B can be singular or plural.
  • the character “/” generally indicates that the contextual objects are an “or” relationship.
  • At least one of the following” or similar expressions refer to any combination of these items, including any combination of single or plural items.
  • At least one item (piece) of a, b or c can mean: a, b, c, "a and b", “a and c", “b and c", or "a and b and c ", where a, b, c can be single or multiple.
  • Audio frame Audio data is streamed.
  • the amount of audio data within a period is usually taken as a frame of audio. This period is called “sampling time", which can be determined according to the codec Its value is determined according to requirements of the device and specific applications, for example, the duration is 2.5ms to 60ms, and ms is milliseconds.
  • Audio signal is the frequency and amplitude change information carrier of regular sound waves with voice, music and sound effects. Audio is a continuously changing analog signal that can be represented by a continuous curve called a sound wave. Audio is a digital signal generated by analog-to-digital conversion or computer generation. Sound waves have three important parameters: frequency, amplitude, and phase, which determine the characteristics of the audio signal.
  • Channel signal Refers to mutually independent audio signals that are collected or played back at different spatial locations during recording or playback.
  • the number of channels is therefore the number of sound sources when sound is recorded or the number of speakers when it is played back.
  • FIG. 1 exemplarily shows a schematic block diagram of an audio decoding system 10 applied in this application.
  • an audio decoding system 10 may include a source device 12 and a destination device 14.
  • the source device 12 generates an encoded code stream. Therefore, the source device 12 may be called an audio encoding device.
  • the destination device 14 can decode the coded stream generated by the source device 12, therefore, the destination device 14 can be called an audio decoding device.
  • the source device 12 includes an encoder 20 and, optionally, an audio source 16 , an audio preprocessor 18 , and a communication interface 22 .
  • Audio source 16 may include or be any type of audio capture device for capturing real world speech, music, sound effects, etc., and/or any type of audio generation device, such as an audio processor for generating speech, music, sound effects, etc. or device.
  • the audio source may be any type of memory or storage that stores audio as described above.
  • the audio preprocessor 18 is used to receive (original) audio data 17 and preprocess the audio data 17 to obtain preprocessed audio data 19 .
  • preprocessing performed by audio preprocessor 18 may include clipping or denoising. It can be understood that the audio preprocessing unit 18 may be an optional component.
  • the encoder 20 is adapted to receive pre-processed audio data 19 and provide encoded audio data 21 .
  • Communication interface 22 in source device 12 may be used to receive encoded audio data 21 and to transmit encoded audio data 21 to destination device 14 via communication channel 13 for storage or direct reconstruction.
  • Destination device 14 includes a decoder 30 and, optionally, a communication interface 28 , an audio post-processor 32 and a playback device 34 .
  • the communication interface 28 in the destination device 14 is used to receive the encoded audio data 21 directly from the source device 12 and to provide the encoded audio data 21 to the decoder 30 .
  • the communication interface 22 and the communication interface 28 can be used to pass through a direct communication link between the source device 12 and the destination device 14, such as a direct wired or wireless connection, etc., or through any type of network, such as a wired network, a wireless network, or any other combination, any type of private network and public network or any type of combination thereof, to send or receive encoded audio data 21 .
  • communication interface 22 may be used to encapsulate encoded audio data 21 into a suitable format such as a message, and/or process encoded audio data 21 using any type of transfer encoding or processing for transmission over a communication link or network .
  • the communication interface 28 corresponds to the communication interface 22 and can be used, for example, to receive transmission data and process the transmission data to obtain encoded audio data 21 using any type of corresponding transmission decoding or processing and/or decapsulation.
  • Both the communication interface 22 and the communication interface 28 can be configured as a one-way communication interface as indicated by an arrow from the source device 12 to the corresponding communication channel 13 of the destination device 14 in FIG. 1, or a two-way communication interface, and can be used to send and receive messages etc., to establish the connection, confirm and exchange any other information related to the communication link and/or data transmission such as encoded audio data, etc.
  • the decoder 30 is configured to receive encoded audio data 21 and provide decoded audio data 31 .
  • the audio post-processor 32 is used to post-process the decoded audio data 31 to obtain post-processed audio data 33 .
  • Post-processing performed by audio post-processor 32 may include, for example, trimming or resampling, among others.
  • Playback device 34 is used to receive post-processed audio data 33 to play the audio to a user or listener.
  • Playback device 34 may be or include any type of player for playing the reconstructed audio, such as integrated or external speakers.
  • speakers may include horns, stereos, and the like.
  • FIG. 2 exemplarily shows a schematic block diagram of an audio decoding device 200 applied in this application.
  • the audio decoding device 200 may be an audio decoder (such as the decoder 30 of FIG. 1 ) or an audio encoder (such as the encoder 20 of FIG. 1 ).
  • the audio decoding device 200 includes: an input port 210 and a receiving unit (Rx) 220 for receiving data, a processor, a logic unit or a central processing unit 230 for processing data, and a transmitting unit (Tx) 240 for transmitting data And output port 250, and memory 260 for storing data.
  • the audio decoding device 200 may further include a photoelectric conversion component and an electro-optic (EO) component coupled with the input port 210 , the receiving unit 220 , the transmitting unit 240 and the output port 250 for output or input of optical or electrical signals.
  • EO electro-optic
  • the processor 230 is realized by hardware and software.
  • Processor 230 may be implemented as one or more CPU chips, cores (eg, multi-core processors), FPGAs, ASICs, and DSPs.
  • Processor 230 is in communication with ingress port 210 , receive unit 220 , transmit unit 240 , egress port 250 and memory 260 .
  • the processor 230 includes a decoding module 270 (eg, an encoding module or a decoding module).
  • the decoding module 270 implements the embodiments disclosed in this application, so as to realize the encoding method of a multi-channel audio signal provided in this application. For example, the decoding module 270 implements, processes or provides various encoding operations.
  • decoding module 270 is implemented as instructions stored in memory 260 and executed by processor 230 .
  • Memory 260 including one or more magnetic disks, tape drives, and solid-state drives, may be used as an overflow data storage device for storing programs while those programs are selectively executed, and for storing instructions and data that are read during program execution.
  • Memory 260 may be volatile and/or nonvolatile, and may be read-only memory (ROM), random-access memory (RAM), random-access memory (ternary content-addressable memory, TCAM), and/or static Random Access Memory (SRAM).
  • ROM read-only memory
  • RAM random-access memory
  • TCAM ternary content-addressable memory
  • SRAM static Random Access Memory
  • the present application provides a method for encoding a multi-channel audio signal.
  • Fig. 3 is a flow chart of an exemplary embodiment of a method for encoding a multi-channel audio signal provided in the present application.
  • the process 300 may be performed by the source device 12 or the audio coding device 200 in the audio coding system 10 .
  • the process 300 is described as a series of steps or operations. It should be understood that the process 300 may be executed in various orders and/or concurrently, and is not limited to the execution order shown in FIG. 3 .
  • the method includes:
  • Step 301 Acquire a first audio frame to be encoded.
  • the first audio frame in this embodiment may be any frame in the multi-channel audio to be encoded, and the first audio frame includes five or more channel signals.
  • 5.1 channels include center channel (C), front left channel (left, L), front right channel (right, R), rear left surround channel (left surround, LS), rear The right surround channel (right surround, RS) and the 0.1 channel low frequency effects (low frequency effects, LFE) have a total of six channel signals.
  • the 7.1 channel includes eight channel signals of C, L, R, LS, RS, LB, RB and LFE.
  • LFE is an audio channel from 3-120Hz, which is usually sent to the Designed speakers.
  • Step 302. Obtain the sum of correlation values of all channel pairs in the target channel pair set.
  • the target channel pair set is obtained for the purpose of obtaining the maximum sum of correlation values, the target channel pair set includes at least one channel pair, and the one channel pair includes two channel signals in at least five channel signals .
  • a channel pair has a correlation value representing the correlation between two channel signals of a channel pair.
  • the pairing is determined according to the correlation value between the two channel signals.
  • the correlation values between any two of the at least five channel signals in the first audio frame may be calculated first to obtain a correlation value set of the first audio frame.
  • five channel signals may form 10 channel pairs in total, and correspondingly, the correlation value set may include 10 correlation values.
  • the correlation value can be normalized, so that the correlation value of all channel pairs is limited within a specific range, so as to set a unified judgment standard for the correlation value, such as a group pair threshold, which can be Set to a value greater than or equal to 0.2 and less than or equal to 1, for example, it can be 0.3, so as long as the normalized correlation value of the two channel signals is less than the group pair threshold, it is considered that the correlation of the two channel signals is relatively high. Poor, no need for group pair encoding.
  • a group pair threshold which can be Set to a value greater than or equal to 0.2 and less than or equal to 1, for example, it can be 0.3, so as long as the normalized correlation value of the two channel signals is less than the group pair threshold, it is considered that the correlation of the two channel signals is relatively high. Poor, no need for group pair encoding.
  • the correlation value between two channel signals may be calculated using the following formula:
  • corr(ch1, ch2) represents the normalized correlation value between channel signal ch1 and channel signal ch2
  • spec_ch1(i) represents the frequency domain coefficient of the i-th frequency point of channel signal ch1
  • spec_ch2(i ) is the frequency domain coefficient of the i-th frequency point of the channel signal ch2
  • N represents an integer value not exceeding the total frequency points of one audio frame.
  • the pairing method of obtaining the target channel pair set includes: for the purpose of obtaining the maximum sum of correlation values, selecting a channel pair from the channel pairs corresponding to at least five channel signals to add to the target channel pair set.
  • the sum of the correlation values of the target channel pair set is the sum of the correlation values of all channel pairs in the target channel pair set obtained by pairing at least five channel signals according to the above pairing manner.
  • the pairing mode of this embodiment may include the following two implementation modes:
  • N can be an integer greater than or equal to 2, and the maximum value of N cannot exceed the number of all channel pairs corresponding to all channel signals of the first audio frame. The larger the value of N, the accompanying calculation amount will increase, and the smaller the value of N, the channel pair set may be lost, thereby reducing the coding efficiency.
  • each channel pair set includes at least one of the M channel pairs corresponding to the M correlation values, and when the channel pair set includes more than two channels When paired, more than two channel pairs do not contain the same channel signal.
  • the three channel pairs corresponding to the maximum correlation value selected according to the correlation value set are (L, R), (R, C) and (LS, RS), where the correlation of (LS, RS) The value is less than the group pair threshold, so it is excluded, then the remaining two channel pairs (L, R) and (R, C) can get two channel pair sets, one of which includes (L ,R), the other includes (R,C).
  • the method for obtaining the set of M channel pairs in this embodiment may include: The first channel pair is added to the target channel pair set, and M channel pair sets include the target channel pair set.
  • the threshold channel pair selects a channel pair with the largest correlation value from other channel pairs to add to the target channel pair set, and the associated channel pair includes the channels included in the channel pair that has been added to the target channel pair set any one of the signals.
  • the channel pair whose correlation value is greater than the group pair threshold is included, select a channel pair with the largest correlation value from other channel pairs and add it to the target channel pair set.
  • the above step b can be iteratively performed.
  • correlation values smaller than the group pair threshold may be deleted from the correlation value set, so that the number of channel pairs can be reduced, thereby reducing the number of iterations.
  • the correlation value set includes the correlation values of multiple channel pairs of at least five channel signals of the first audio frame, and the multiple channel pairs are regularly combined (that is, multiple channel pairs in the same channel pair set) The same channel signal cannot be included between the channel pairs), and multiple channel pair sets corresponding to the at least five channel signals can be obtained.
  • the following formula may be used to calculate the number of sets of all channel pairs:
  • the following formula may be used to calculate the number of sets of all channel pairs:
  • Pair_num represents the number of all channel pairs
  • CH represents the number of channel signals participating in multi-channel processing in the first audio frame, which is the result of filtering through the multi-channel mask.
  • multiple channel pair sets can be obtained according to other channel pairs in the multiple channel pairs except for the relevant channel, and the correlation value of the non-correlated channel pair is less than the group pair threshold, so that the number of channel pairs involved in the calculation can be reduced when obtaining the channel pair set, thereby reducing the number of channel pair sets, and the calculation amount of the sum of correlation values can also be reduced in subsequent steps.
  • Step 303 when the sum of the correlation values is greater than the preset threshold, perform energy equalization processing on at least two channel signals among the at least five channel signals to obtain at least two equalized channel signals.
  • the fluctuation interval values of at least five channel signals may be obtained first, and then the energy balance mode is determined according to the fluctuation interval values of at least five channel signals, and then at least five The channel signals are subjected to energy equalization processing to obtain at least five equalized channel signals.
  • the fluctuation interval value is used to represent the difference in energy or amplitude between at least five channel signals.
  • the energy balance mode includes a first energy balance mode and a second energy balance mode, wherein the first energy balance mode uses two channel signals in a channel pair to obtain two equalized channel signals corresponding to a channel pair.
  • the second energy balance mode uses two channel signals in a channel pair and at least one channel signal outside a channel to obtain two equalized channel signals corresponding to a channel pair.
  • Determining the energy balance mode according to the fluctuation interval values of at least five channel signals may include: when the fluctuation interval values meet the preset conditions, determining the energy balance mode as the first energy balance mode; when the fluctuation interval values do not meet the preset conditions, Determine the energy balance mode as the second energy balance mode.
  • the above fluctuation interval value includes the energy flatness of the first audio frame, and the fluctuation interval value meets the preset condition means that the energy flatness is less than the first threshold; or, the fluctuation interval value includes the amplitude flatness of the first audio frame, and the fluctuation interval value meets the The preset condition means that the amplitude flatness is less than the second threshold; or, the fluctuation interval value includes the energy deviation degree of the first audio frame, and the fluctuation interval value meets the preset condition means that the energy deviation degree is not within the first preset range; or, The fluctuation interval value includes the amplitude deviation of the first audio frame, and the fluctuation interval value meeting the preset condition means that the amplitude deviation is not within the second preset range.
  • the energy flatness indicates the fluctuation of the frame energy normalized by the energy of the current frame frequency domain coefficients of multiple channels filtered by the multi-channel screening unit, which can be calculated by the flatness formula to measure.
  • the energy flatness of the current frame is 1; when the energy of a certain channel of the current frame is 0, the energy flatness of the current frame is 0, so the inter-channel
  • the value range of energy flatness is [0,1]. The greater the energy fluctuation between channels, the smaller the value of its energy flatness.
  • a unified first threshold may be set for all channel formats (such as 5.1, 7.1, 9.1, and 11.1), for example, it may be 0.483, 0.492, or 0.504 and so on.
  • different first thresholds are set for different channel formats. For example, the first threshold for the 5.1-channel format is 0.511, the first threshold for the 7.1-channel format is 0.563, the first threshold for the 9.1-channel format is 0.608, and the first threshold for the 11.1-channel format is 0.654.
  • the amplitude flatness indicates the fluctuation of the frame amplitude normalized by the frequency-domain coefficient amplitudes of the current frame of multiple channels filtered by the multi-channel screening unit, which can be measured by the flatness calculation formula.
  • the flatness is 1; when the frame width of one of the channels is 0, the flatness is 0. Therefore the range of amplitude flatness is between [0,1].
  • a unified second threshold may be set for all channel formats (such as 5.1, 7.1, 9.1, and 11.1), for example, it may be 0.695, 0.701, or 0.710 and so on.
  • different second thresholds may be given for different channel formats, for example, the second threshold of the 5.1 channel format may be 0.715, the second threshold of the 7.1 channel format may be 0.753, 9.1 The second threshold for the channel format may be 0.784, and the second threshold for the 11.1 channel format may be 0.809.
  • the energy balance mode may be determined by using the above-mentioned various information representing fluctuation interval values of the at least five channel signals, including energy flatness, amplitude flatness, energy deviation or amplitude deviation.
  • (1) Calculate the energy values of at least five channel signals, obtain the energy flatness of the first audio frame according to the energy values of at least five channel signals, and determine when the energy flatness of the first audio frame is less than the first threshold
  • the energy balance mode is the first energy balance mode; when the energy flatness of the first audio frame is greater than or equal to the first threshold, the energy balance mode is determined to be the second energy balance mode.
  • the energy balance mode is the first energy balance mode; when the amplitude flatness of the first audio frame is greater than or equal to the second threshold, the energy balance mode is determined to be the second energy balance mode.
  • the energy balance mode before determining the energy balance mode according to the fluctuation interval values of at least five channel signals, the energy balance mode may also be determined according to the encoding bit rate corresponding to the first audio frame, that is, to determine the encoding bit rate rate is greater than the code rate threshold, when the code rate is greater than the code rate threshold, determine the energy balance mode as the second energy balance mode; when the code rate is less than or equal to the code rate threshold, according to at least five channel signals
  • the fluctuation interval value determines the energy balance mode.
  • the average of the energy or amplitude values of the two channel signals contained in the current channel pair can be calculated for the current channel pair in the target channel pair set corresponding to the pairing mode value, respectively perform energy equalization processing on the two channel signals according to the average value to obtain two corresponding equalized channel signals.
  • the coding noise of the channel pair with high energy may be much greater than that of the channel pair with low energy due to insufficient bits, and the bits of the channel pair with low energy will have redundant problems .
  • the energy balance mode is the second energy balance mode
  • the average value of the energy or amplitude values of the at least five channel signals can be calculated, and the energy balance processing is performed on the at least five channel signals according to the average value to obtain at least five balanced sound road signal.
  • step 303 is mainly to perform energy equalization processing on at least two channel signals among at least five channel signals to obtain at least two equalized channel signals, and the at least two channel signals are the target channel pair For the paired channel signals in the set, except for the paired channel signals in the target channel pair set, the remaining ungrouped channel signals are directly encoded.
  • Step 304 encode at least two equalized channel signals to obtain encoded code streams.
  • Step 305 when the sum of the correlation values is less than or equal to the preset threshold, encode at least five channel signals to obtain an encoded code stream.
  • the sum of the correlation values is less than or equal to the preset threshold, it means that the correlation between the two channel signals in the channel pair in the target channel pair set is low, and there is no need for group pair coding. Furthermore, there is no need to perform energy equalization processing on the at least five channel signals, and the object of encoding at this time is the at least five channel signals, not the equalized channel signals.
  • At least five channel signals included in the audio frame are grouped to obtain a target channel pair set.
  • the sum of the correlation values of the target channel pair set is greater than a preset threshold.
  • the 5.1 channel includes a center channel (C), a front left channel (left, L), a front right channel (right, R), a rear left surround channel (left surround , LS), rear right surround channel (right surround, RS) and 0.1 channel low frequency effects (low frequency effects, LFE), as shown in Table 1 for example, set the channel index for the above six channel signals.
  • C center channel
  • L front left channel
  • R front right channel
  • R rear left surround channel
  • LFE low frequency effects
  • Fig. 4a is an exemplary structural diagram of an encoding device applied to the encoding method of a multi-channel audio signal provided by the present application.
  • the encoding device may be the encoder 20 of the source device 12 in the audio decoding system 10, or may be is the decoding module 270 in the audio decoding device 200 .
  • the encoding device may include a multi-channel adaptive group pair module, a channel encoding module and a code stream multiplexing interface, wherein,
  • the input of the multi-channel adaptive group pair module includes six channel signals of 5.1 channels (L, R, C, LS, RS, LFE), and the multi-channel processing indicator (MultiProcFlag), and the output includes Six channel signals (M1, S1, M2, S2, C, LFE), wherein, M1 and S1 are a pair of channel pairs obtained by pairing, M2 and S2 are a pair of channel pairs obtained by grouping, and multi-channel side information (sideInfoMc), which includes a set of channel pairs.
  • M1, S1, M2, S2, C, LFE Six channel signals
  • M1 and S1 are a pair of channel pairs obtained by pairing
  • M2 and S2 are a pair of channel pairs obtained by grouping
  • multi-channel side information which includes a set of channel pairs.
  • the channel coding module uses the mono coding unit (or mono channel box, mono tool) to the channel signals (M1, S1, M2, S2, C, LFE) output by the multi-channel adaptive group pair module ) to encode and output corresponding coded channel signals (E1-E6).
  • the channel encoding module may also use a stereo encoding unit, such as a parametric stereo encoder or a lossy stereo encoder, to encode the processed channel signals output by the multi-channel processing module.
  • unpaired channel signals (such as C and LFE) can be directly input into the channel encoding module to obtain encoded channel signals E5 and E6.
  • the code stream multiplexing interface generates coded multi-channel signals, and the coded multi-channel signals include coded channel signals (E1-E6) output by the channel coding module and side information (including multi-channel side information).
  • the stream multiplexing interface can process the coded multi-channel signal into a serial signal or a serial bit stream.
  • Fig. 4 b is an exemplary structural diagram of the multi-channel adaptive group pair module, as shown in Fig. 4 b, the multi-channel adaptive group pair module includes: multi-channel screening unit, global correlation value statistics unit, multi-channel An energy balance selection module and a pair processing module.
  • the multi-channel screening unit screens out five channel signals participating in multi-channel processing from six channel signals (L, R, C, LS, RS, LFE) according to the multi-channel processing indicator (MultiProcFlag), namely L, R, C, LS, RS.
  • the global correlation value statistical unit first calculates the channel signals participating in the multi-channel processing, that is, the normalized correlation value between any two channel signals among L, R, C, LS, and RS.
  • the present application may use the following formula to calculate the correlation value between two channel signals (for example, channel signal ch1 and channel signal ch2):
  • corr(ch1, ch2) represents the normalized correlation value between channel signal ch1 and channel signal ch2
  • spec_ch1(i) represents the frequency domain coefficient of the i-th frequency point of channel signal ch1
  • spec_ch2(i ) represents the frequency domain coefficient of the i-th frequency point of the channel signal ch2
  • N represents an integer value not exceeding the total number of frequency points of one audio frame.
  • the global correlation value statistics unit can filter the correlation value according to the set group pair threshold, that is, it is greater than or equal to the group pair threshold Correlation values of are retained, while correlation values smaller than the group pair threshold are deleted, or their values are set to 0. This reduces computation.
  • the multi-channel energy equalization selection module determines whether to perform energy equalization processing on the five channel signals according to the encoding bit rate and the five channel signals.
  • the grouping method of the five channel signals is a global grouping method, and the purpose of this method is to obtain the maximum sum of correlation values. For details, refer to the description of step 302 .
  • the sum of the correlation values of the target channel pairs is greater than the preset threshold, it is determined that energy equalization processing needs to be performed on the five channel signals; when the sum of the correlation values of the target channel pairs is less than or equal to the preset threshold, it is determined No energy equalization processing is required for the five-channel signals.
  • an energy equalization mode is determined.
  • Fig. 4c is an exemplary structural diagram of the group pair processing module.
  • the group pair processing module includes a group pair decider, an energy equalization unit and a stereo processing box.
  • the group pair decider first calculates the energy or amplitude value of each channel signal, and the application can use the following formula to calculate the energy or amplitude value of the channel signal (ch):
  • energy(ch) represents the energy or amplitude value of the channel signal ch
  • sepc_coeff(ch,i) represents the frequency domain coefficient of the i-th frequency point of the channel signal ch
  • N represents the number of total frequency points not exceeding one audio frame integer value.
  • the application can use the following formula to calculate the normalized energy or amplitude value of the channel signal (ch):
  • energy_uniform(ch) represents the normalized energy or amplitude value of the channel signal ch
  • the fluctuation interval value may refer to the energy flatness.
  • the application may use the following formula to calculate the energy flatness of the five channel signals:
  • efm represents the energy flatness of the five channel signals
  • efm represents the energy flatness of the five channel signals
  • the channel indexes of L, R, C, LS, and RS are shown in Table 1.
  • the fluctuation interval value may also refer to the energy deviation degree.
  • the application can use the following formula to calculate the average energy or amplitude of the five channel signals value:
  • avg_energy_uniform represents the average energy or amplitude value of the five channel signals, and the channel indexes of L, R, C, LS, and RS are shown in Table 1.
  • the energy deviation of the channel signal (ch) is calculated using the following formula:
  • deviation(ch) represents the degree of energy deviation of the channel signal ch.
  • the largest of the energy deviations of L, R, C, LS, and RS is determined as the energy deviation deviation of the five channel signals.
  • the fluctuation interval value may also refer to an amplitude value or an amplitude deviation degree, the principle of which is similar to the above-mentioned energy-related value, and will not be repeated here.
  • the energy balance mode of the present application includes two implementations, wherein, the Pair energy balance mode is for each channel pair in the target channel pair set corresponding to the pairing mode determined by the module selection unit, using one channel Two equalized channel signals corresponding to the one channel pair are obtained by pairing the two channel signals.
  • the overall energy equalization mode two channel signals in a channel pair and at least one channel signal outside a channel are used to obtain two equalized channel signals corresponding to the channel pair.
  • the corresponding equalized channel signal is the channel signal itself.
  • the group pair decider determines the energy balance mode according to the fluctuation interval value, including the following two judgment methods:
  • the energy balance mode When efm is less than the first threshold, the energy balance mode is the Pair energy balance mode; when efm is greater than or equal to the first threshold, the energy balance mode is the overall energy balance mode.
  • the energy balance mode is the overall energy balance mode; when the deviation is not within the value range [threshold, 1/threshold], the energy balance mode is the pair energy balanced mode.
  • the value range of threshold can be (0,1).
  • the deviation may represent a ratio of the frequency-domain amplitude of each channel of the current frame to the average value of the frequency-domain amplitude of each channel of the current frame, that is, the amplitude deviation.
  • the frequency domain amplitude of the current channel is greater than the average value of the frequency domain amplitudes of each channel in the current frame, and the "current sound The frequency
  • the value of threshold can be between (0, 1), and the value of threshold The smaller the value, the greater the fluctuation of the frequency domain amplitude of the current channel relative to the average value of the frequency domain amplitudes of each channel in the current frame, and the larger the value of threshold, it indicates that the frequency domain amplitude of the current channel is relative to the current frame.
  • the fluctuation of the average value of the frequency domain amplitude of each channel is smaller.
  • the value of threshold can be 0.2, 0.15, 0.125, 0.11, or 0.1 and so on.
  • the deviation may also represent the ratio of the frequency-domain energy of each channel to the average value of the frequency-domain energy of each channel, that is, the energy deviation.
  • the frequency-domain energy of the current channel Less than or equal to the average value of the frequency-domain energy of each channel in the current frame, and the "frequency-domain energy of the current channel/average value of the frequency-domain energy of each channel in the current frame" that satisfies the condition is between (0.04, 1] interval, that is (threshold, 1];
  • the frequency domain energy of the current channel is greater than the average value of the frequency domain energy of each channel in the current frame, and the "frequency domain energy of the current channel/each The average value of the frequency domain energy of the channel" is between (1,25); combining the above two situations, when the frequency domain energy of the current channel is proportional to the average value of the frequency domain energy of each channel in the current frame When it is less than 25, the range of "the frequency domain energy of the current channel/the average frequency domain energy of each channel of the current frame” that satisfies the condition is between (0.04,25), that is, (threshold,1/threshold ), (threshold, 1/
  • threshold can be between (0,1), and the smaller the value of threshold, it means that the frequency domain energy of the current channel is relative to The greater the fluctuation of the average value of the frequency-domain energy of each channel in the current frame, the greater the value of threshold, which means that the fluctuation of the frequency-domain energy of the current channel relative to the average value of the frequency-domain energy of each channel in the current frame is greater.
  • the value of Threshold can be 0.04, 0.0225, 0.015625, 0.0121, or 0.01, etc.
  • the above-mentioned first preset range can also be extended to (0,1/threshold), at this time, the range of Pair energy balance is [1/threshold,+ ⁇ ), which indicates that when the current When the frequency domain energy of the channel is greater than the average frequency domain energy of each channel in the current frame, and "the frequency domain energy of the current channel/the average frequency domain energy of each channel in the current frame" is greater than 1/threshold , the Pair energy balance is performed.
  • the above-mentioned second preset range can also be extended to (0,1/threshold), at this time, the interval range of Pair amplitude equalization is [1/threshold,+ ⁇ ), which indicates that when the current When the frequency domain amplitude of the channel is greater than the average frequency domain amplitude of each channel in the current frame, and "frequency domain amplitude of the current channel/average frequency domain amplitude of each channel in the current frame" is greater than 1/threshold , before performing Pair amplitude equalization.
  • the group pair determiner can calculate the normalized energy or amplitude value based on the five channel signals, and then obtain the energy flatness or energy deviation, or it can only calculate the normalized value based on the successfully grouped channel signals
  • the normalized energy or amplitude value can be obtained to obtain the energy flatness or energy deviation. It can also calculate the normalized energy or amplitude value based on part of the five channel signals to obtain the energy flatness or energy deviation. . This application does not specifically limit it.
  • the stereo processing unit may adopt prediction-based or Karhunen-Loeve Transform (KLT)-based processing, that is, the input two channel signals are rotated (e.g. via a 2 ⁇ 2 rotation matrix) to maximize energy compression, Thereby concentrating the signal energy in one channel.
  • KLT Karhunen-Loeve Transform
  • the stereo processing unit After the stereo processing unit processes the input two channel signals, it outputs the processed channel signals (P1-P4) and multi-channel side information corresponding to the two channel signals, and the multi-channel side information includes the sum of correlation values and A collection of target channel pairs.
  • FIG. 5 is an exemplary structural diagram of a decoding device applied to the multi-channel audio decoding method provided by the present application.
  • the decoding device may be the decoder 30 of the destination device 14 in the audio decoding system 10, or may be The decoding module 270 in the audio decoding device 200 .
  • the decoding device may include a code stream demultiplexing interface, a channel decoding module and a multi-channel processing module, wherein,
  • the code stream demultiplexing interface receives coded multi-channel signals (such as serial bitstream bitstream) from the coding device, and obtains coded channel signals (E) and multi-channel parameters (SIDE_PAIR) after demultiplexing.
  • E coded channel signals
  • SIDE_PAIR multi-channel parameters
  • the channel decoding module uses a mono decoding unit (or a mono channel box, a mono tool) to decode the coded channel signal output by the stream demultiplexing interface and output the decoded channel signal (D). For example, E1 , E2 , E3 , E4 , .
  • the multi-channel processing module includes a plurality of stereo processing units, and the stereo processing unit can adopt prediction-based or KLT-based processing, that is, the input two channel signals are derotated (for example, via a 2 ⁇ 2 rotation matrix), so that the signal Transform to original signal direction.
  • the decoded channel signal output by the channel decoding module can identify which two decoded channel signals are paired by the multi-channel parameter, and input the paired decoded channel signal into the stereo processing unit, and the stereo processing unit decodes the input two After the channel signal is processed, the channel signal (CH) corresponding to the two decoded channel signals is output.
  • stereo processing unit 1 processes D1 and D2 according to SIDE_PAIR1 to obtain CH1 and CH2
  • stereo processing unit 2 processes D3 and D4 according to SIDE_PAIR2 to obtain CH3 and CH4
  • stereo processing unit m processes Di-1 and Di according to SIDE_PAIRm treatment to obtain CHi-1 and CHi.
  • unpaired channel signals (such as CHj) do not need to be processed by the stereo processing unit in the multi-channel processing module, and can be directly output after being decoded.
  • FIG. 6 is a schematic structural diagram of an embodiment of an encoding device in the present application. As shown in FIG. 6 , the device can be applied to the source device 12 or the audio decoding device 200 in the above embodiments.
  • the encoding device in this embodiment may include: an acquisition module 601 , a processing module 602 and an encoding module 603 . in,
  • the obtaining module 601 is used to obtain the first audio frame to be encoded, the first audio frame includes at least five channel signals; obtain the sum of the correlation values of the target channel pair set, the target channel pair set is Obtaining the sum of the maximum correlation value is obtained for the purpose, the target channel pair set includes at least one channel pair, one channel pair includes two channel signals in the at least five channel signals, and the one channel pair The channel pair has a correlation value, and the correlation value is used to represent the correlation between the two channel signals of the one channel pair; the processing module 602 is configured to when the sum of the correlation values is greater than a preset threshold Perform energy equalization processing on the at least five equalized channel signals to obtain at least five equalized channel signals; the encoding module 603 is configured to encode the at least five equalized channel signals.
  • the encoding module 603 is further configured to encode the at least five channel signals when the sum of the correlation values is less than or equal to the preset threshold.
  • the processing module 602 is specifically configured to acquire fluctuation interval values of the at least five channel signals; determine an energy balance mode according to the fluctuation interval values of the at least five channel signals; Perform energy equalization processing on the at least five channel signals according to the energy equalization mode to obtain the at least five equalized channel signals.
  • the processing module 602 is specifically configured to determine that the energy balance mode is the first energy balance mode when the fluctuation interval value meets a preset condition; or, when the fluctuation interval When the value does not meet the preset condition, it is determined that the energy balance mode is the second energy balance mode.
  • the fluctuation interval value includes the energy flatness of the first audio frame; the fluctuation interval value meeting a preset condition means that the energy flatness is smaller than a first threshold; or, the The fluctuation interval value includes the amplitude flatness of the first audio frame; the fluctuation interval value meets the preset condition means that the amplitude flatness is smaller than the second threshold; or, the fluctuation interval value includes the first audio frame
  • the energy deviation degree of the frame; the fluctuation interval value meets the preset condition means that the energy deviation degree is not within the first preset range; or, the fluctuation interval value includes the amplitude deviation degree of the first audio frame;
  • the fluctuation interval value meeting the preset condition means that the amplitude deviation degree is not within the second preset range.
  • the processing module 602 is specifically configured to, for the current channel pair in the target channel pair set, calculate The average value of the energy or amplitude values of the two channel signals contained in the current channel pair, and perform energy equalization processing on the two channel signals according to the average value to obtain two corresponding equalized channel signals .
  • the processing module 602 is specifically configured to calculate the average of the energy or amplitude values of the at least five channel signals value, performing energy equalization processing on the at least five channel signals respectively according to the average value to obtain the at least five balanced channel signals.
  • the processing module 602 is further configured to determine whether the encoding bit rate corresponding to the first audio frame is greater than a bit rate threshold; when the encoding bit rate is greater than the bit rate threshold , determining that the energy balance mode is the second energy balance mode; when the encoding code rate is less than or equal to the code rate threshold, the energy balance mode is determined according to the fluctuation interval value.
  • the device of this embodiment can be used to execute the technical solution of the method embodiment shown in FIG. 3 , and its implementation principle and technical effect are similar, and will not be repeated here.
  • FIG. 7 is a schematic structural diagram of a device embodiment of the present application.
  • the device may be the encoding device in the foregoing embodiment.
  • the device in this embodiment may include: a processor 701 and a memory 702, the memory 702 is used to store one or more programs; when the one or more programs are executed by the processor 701, the processor 701 realizes The technical solution of the method embodiment is shown in FIG. 3 .
  • each step of the above-mentioned method embodiments may be completed by an integrated logic circuit of hardware in a processor or instructions in the form of software.
  • the processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other possible Program logic devices, discrete gate or transistor logic devices, discrete hardware components.
  • a general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.
  • the steps of the method disclosed in the present application can be directly implemented by a hardware coded processor, or executed by a combination of hardware and software modules in the coded processor.
  • the software module can be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other mature storage media in this field.
  • the storage medium is located in the memory, and the processor reads the information in the memory, and completes the steps of the above method in combination with its hardware.
  • the memories mentioned in the above embodiments may be volatile memories or nonvolatile memories, or may include both volatile and nonvolatile memories.
  • the non-volatile memory can be read-only memory (read-only memory, ROM), programmable read-only memory (programmable ROM, PROM), erasable programmable read-only memory (erasable PROM, EPROM), electrically programmable Erases programmable read-only memory (electrically EPROM, EEPROM) or flash memory.
  • Volatile memory can be random access memory (RAM), which acts as external cache memory.
  • RAM random access memory
  • SRAM static random access memory
  • DRAM dynamic random access memory
  • DRAM synchronous dynamic random access memory
  • SDRAM double data rate synchronous dynamic random access memory
  • ESDRAM enhanced synchronous dynamic random access memory
  • SLDRAM direct memory bus random access memory
  • direct rambus RAM direct rambus RAM
  • the disclosed systems, devices and methods may be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of the units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.
  • the functions described above are realized in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium.
  • the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (personal computer, server, or network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (read-only memory, ROM), random access memory (random access memory, RAM), magnetic disk or optical disc and other media that can store program codes. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Abstract

本申请提供一种多声道音频信号的编码方法和装置。该多声道音频信号的编码方法，包括：获取待编码的第一音频帧，第一音频帧包括至少五个声道信号；获取目标声道对集合中的所有声道对的相关值之和，目标声道对集合包括至少一个声道对，一个声道对包括至少五个声道信号中的两个声道信号，一个声道对具有一个相关值，相关值用于表示一个声道对的两个声道信号之间的相关性；当相关值之和大于预设阈值时，对至少五个声道信号中的至少两个声道信号进行能量均衡处理以得到至少两个均衡声道信号；对至少两个均衡声道信号进行编码以得到经编码的码流。本申请可以提高音频帧的编码效率。

Description

多声道音频信号的编码方法和装置
本申请要求于2021年5月28日提交中国专利局、申请号为202110595367.2、申请名称为“多声道音频信号的编码方法和装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及音频处理技术，尤其涉及一种多声道音频信号的编码方法和装置。
背景技术
多声道音频的编解码是对包含两个以上声道的音频进行编码或解码的技术。常见的多声道音频有5.1声道音频、7.1声道音频、7.1.4声道音频以及22.2声道音频等。
MPEG环绕声(MPEG Surround，MPS)标准规定了针对四个声道的联合编码，但仍需有可以针对上述各种多声道音频信号的编解码方法。
发明内容
本申请提供一种多声道音频信号的编码方法和装置，以提高音频帧的编码效率。
第一方面，本申请提供一种多声道音频信号的编码方法，包括：获取待编码的第一音频帧，所述第一音频帧包括至少五个声道信号；获取目标声道对集合中的所有声道对的相关值之和，所述目标声道对集合包括至少一个声道对，一个声道对包括所述至少五个声道信号中的两个声道信号，所述一个声道对具有一个相关值，所述相关值用于表示所述一个声道对的两个声道信号之间的相关性；当所述相关值之和大于预设阈值时，对所述至少五个声道信号中的至少两个声道信号进行能量均衡处理以得到至少两个均衡声道信号；对所述至少两个均衡声道信号进行编码以得到经编码的码流。
本实施例以获取最大相关值之和为目的对音频帧包含的至少五个声道信号进行组对得到目标声道对集合，当目标声道对集合的相关值之和大于预设阈值时，对至少五个声道信号中的至少两个声道信号进行能量均衡处理，进而编码，可以提高音频帧的编码效率。
在一种可能的实现方式中，所述方法还包括：当所述相关值之和小于或等于所述预设阈值时，对所述至少五个声道信号进行编码以得到经编码的码流。
本实施例中，如果相关值之和小于或等于预设阈值，表示目标声道对集合中的声道对中的两个声道信号之间的相关性较低，没有组对编码的必要，进而也不需要对至少五个声道信号进行能量均衡处理，此时编码的对象是该至少五个声道信号，而非均衡声道信号。
在一种可能的实现方式中，所述对所述至少五个声道信号中的至少两个声道信号进行能量均衡处理以得到至少两个均衡声道信号，包括：获取所述至少五个声道信号的波动区间值；根据所述至少五个声道信号的波动区间值确定能量均衡模式；根据所述能量均衡模式分别对所述至少两个声道信号进行能量均衡处理以得到所述至少两个均衡声道信号。
波动区间值用于表示至少五个声道信号之间的能量或幅度的差异大小。能量均衡模式包括第一能量均衡模式和第二能量均衡模式，其中，第一能量均衡模式使用一个声道对中两个声道信号获取一个声道对对应的两个均衡声道信号。第二能量均衡模式使用一个声道 对中两个声道信号以及一个声道对外至少一个声道信号来获取一个声道对对应的两个均衡声道信号。
在一种可能的实现方式中，所述根据所述至少五个声道信号的波动区间值确定能量均衡模式，包括：当所述波动区间值符合预设条件时，确定所述能量均衡模式为第一能量均衡模式；或者，当所述波动区间值不符合预设条件时，确定所述能量均衡模式为第二能量均衡模式。
在一种可能的实现方式中，所述波动区间值包括所述第一音频帧的能量平整度；所述波动区间值符合预设条件是指所述能量平整度小于第一阈值；或者，所述波动区间值包括所述第一音频帧的幅度平整度；所述波动区间值符合预设条件是指所述幅度平整度小于第二阈值；或者，所述波动区间值包括所述第一音频帧的能量偏离度；所述波动区间值符合预设条件是指所述能量偏离度不在第一预设范围内；或者，所述波动区间值包括所述第一音频帧的幅度偏离度；所述波动区间值符合预设条件是指所述幅度偏离度不在第二预设范围内。
在一种可能的实现方式中，当所述能量均衡模式为所述第一能量均衡模式时，所述对所述至少五个声道信号中的至少两个声道信号进行能量均衡处理以得到至少两个均衡声道信号，包括：对所述目标声道对集合对应的声道信号进行能量均衡处理以得到所述至少两个均衡声道信号。
在一种可能的实现方式中，所述对所述目标声道对集合对应的声道信号进行能量均衡处理以得到所述至少两个均衡声道信号，包括：针对所述目标声道对集合中的当前声道对，计算所述当前声道对包含的两个声道信号的能量值或幅度值的平均值，根据所述平均值分别对所述当前声道对包含的两个声道信号进行能量均衡处理以得到对应的两个均衡声道信号。
这样当至少五个声道信号的波动区间值较大时，可以只在相关的两个声道信号之间进行能量均衡，使得立体声处理时对于比特的分配更符合声道信号的波动区间值，避免在低码率的编码环境中能量大的声道对因比特不足导致编码噪声可能会远大于能量小的声道对的编码噪声，而能量小的声道对的比特会有冗余的问题。
在一种可能的实现方式中，当所述能量均衡模式为所述第二能量均衡模式时，所述对所述至少五个声道信号中的至少两个声道信号进行能量均衡处理以得到至少两个均衡声道信号，包括：计算所述至少五个声道信号的能量值或幅度值的平均值，根据所述平均值分别对所述至少五个声道信号进行能量均衡处理得到所述至少五个均衡声道信号。
在一种可能的实现方式中，所述根据所述至少五个声道信号的波动区间值确定能量均衡模式之前，还包括：判断与所述第一音频帧对应的编码码率是否大于码率阈值；当所述编码码率大于所述码率阈值时，确定所述能量均衡模式为第二能量均衡模式；当所述编码码率小于或等于所述码率阈值时，才根据所述波动区间值确定所述能量均衡模式。
在一种可能的实现方式中，所述方法还包括：对所述至少五个声道信号中未进行能量均衡处理的声道信号进行编码。
第二方面，本申请提供一种编码装置，包括：获取模块，用于获取待编码的第一音频帧，所述第一音频帧包括至少五个声道信号；获取目标声道对集合中的所有声道对的相关值之和，所述目标声道对集合包括至少一个声道对，一个声道对包括所述至少五个声道信 号中的两个声道信号，所述一个声道对具有一个相关值，所述相关值用于表示所述一个声道对的两个声道信号之间的相关性；处理模块，用于当所述相关值之和大于预设阈值时，对所述至少五个声道信号中的至少两个声道信号进行能量均衡处理以得到至少两个均衡声道信号；编码模块，用于对所述至少两个均衡声道信号进行编码以得到经编码的码流。
在一种可能的实现方式中，所述编码模块，还用于当所述相关值之和小于或等于所述预设阈值时，对所述至少五个声道信号进行编码以得到经编码的码流。
在一种可能的实现方式中，所述处理模块，具体用于获取所述至少五个声道信号的波动区间值；根据所述至少五个声道信号的波动区间值确定能量均衡模式；根据所述能量均衡模式分别对所述至少两个声道信号进行能量均衡处理以得到所述至少两个均衡声道信号。
在一种可能的实现方式中，所述处理模块，具体用于当所述波动区间值符合预设条件时，确定所述能量均衡模式为第一能量均衡模式；或者，当所述波动区间值不符合预设条件时，确定所述能量均衡模式为第二能量均衡模式。
在一种可能的实现方式中，所述波动区间值包括所述第一音频帧的能量平整度；所述波动区间值符合预设条件是指所述能量平整度小于第一阈值；或者，所述波动区间值包括所述第一音频帧的幅度平整度；所述波动区间值符合预设条件是指所述幅度平整度小于第二阈值；或者，所述波动区间值包括所述第一音频帧的能量偏离度；所述波动区间值符合预设条件是指所述能量偏离度不在第一预设范围内；或者，所述波动区间值包括所述第一音频帧的幅度偏离度；所述波动区间值符合预设条件是指所述幅度偏离度不在第二预设范围内。
在一种可能的实现方式中，当所述能量均衡模式为所述第一能量均衡模式时，所述处理模块，具体用于对所述目标声道对集合对应的声道信号进行能量均衡处理以得到所述至少两个均衡声道信号。
在一种可能的实现方式中，所述处理模块，具体用于针对所述目标声道对集合中的当前声道对，计算所述当前声道对包含的两个声道信号的能量值或幅度值的平均值，根据所述平均值分别对所述当前声道对包含的两个声道信号进行能量均衡处理以得到对应的两个均衡声道信号。
在一种可能的实现方式中，当所述能量均衡模式为所述第二能量均衡模式时，所述处理模块，具体用于计算所述至少五个声道信号的能量值或幅度值的平均值，根据所述平均值分别对所述至少五个声道信号进行能量均衡处理得到所述至少五个均衡声道信号。
在一种可能的实现方式中，所述处理模块，还用于判断与所述第一音频帧对应的编码码率是否大于码率阈值；当所述编码码率大于所述码率阈值时，确定所述能量均衡模式为第二能量均衡模式；当所述编码码率小于或等于所述码率阈值时，才根据所述波动区间值确定所述能量均衡模式。
在一种可能的实现方式中，所述编码模块，还用于对所述至少五个声道信号中未进行能量均衡处理的声道信号进行编码。
第三方面，本申请提供一种设备，包括：一个或多个处理器；存储器，用于存储一个或多个程序；当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如上述第一方面中任一项所述的方法。
第四方面，本申请提供一种计算机可读存储介质，包括计算机程序，所述计算机程序在计算机上被执行时，使得所述计算机执行上述第一方面中任一项所述的方法。
第五方面，本申请提供一种计算机可读存储介质，包括根据如上述第一方面中任一项所述的多声道音频信号的编码方法获得的编码码流。
附图说明
图1示例性地给出了本申请所应用的音频译码系统10的示意性框图；
图2示例性地给出了本申请所应用的音频译码设备200的示意性框图；
图3是本申请提供的多声道音频信号的编码方法的一个示例性的实施例的流程图；
图4a是本申请提供的多声道音频信号的编码方法所应用的编码装置的一个示例性的结构图；
图4b是多声道自适应组对模块的一个示例性的结构图；
图4c是组对处理模块的一个示例性的结构图；
图5是本申请提供的多声道音频的解码方法所应用的解码装置的一个示例性的结构图；
图6为本申请编码装置实施例的结构示意图；
图7为本申请设备实施例的结构示意图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚，下面将结合本申请中的附图，对本申请中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获取的所有其他实施例，都属于本申请保护的范围。
本申请的说明书实施例和权利要求书及附图中的术语“第一”、“第二”等仅用于区分描述的目的，而不能理解为指示或暗示相对重要性，也不能理解为指示或暗示顺序。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元。方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
应当理解，在本申请中，“至少一个(项)”是指一个或者多个，“多个”是指两个或两个以上。“和/或”，用于描述关联对象的关联关系，表示可以存在三种关系，例如，“A和/或B”可以表示：只存在A，只存在B以及同时存在A和B三种情况，其中A，B可以是单数或者复数。字符“/”一般表示前后关联对象是一种“或”的关系。“以下至少一项(个)”或其类似表达，是指这些项中的任意组合，包括单项(个)或复数项(个)的任意组合。例如，a，b或c中的至少一项(个)，可以表示：a，b，c，“a和b”，“a和c”，“b和c”，或“a和b和c”，其中a，b，c可以是单个，也可以是多个。
本申请涉及到的相关名词解释：
音频帧：音频数据是流式的，在实际应用中，为了便于音频处理和传输，通常取一时长内的音频数据量作为一帧音频，该时长被称为“采样时间”，可以根据编解码器和具体应用的需求确定其值，例如该时长为2.5ms～60ms，ms为毫秒。
音频信号：音频信号是带有语音、音乐和音效的有规律的声波的频率、幅度变化信息载体。音频是一种连续变化的模拟信号，可用一条连续的曲线来表示，称为声波。音频通过模数转换或计算机生成的数字信号即为音频信号。声波有三个重要参数：频率、幅度和相位，这也就决定了音频信号的特征。
声道信号：是指声音在录制或播放时在不同空间位置采集或回放的相互独立的音频信号。因此声道数也就是声音录制时的音源数量或回放时的扬声器数量。
以下是本申请所应用的系统架构。
图1示例性地给出了本申请所应用的音频译码系统10的示意性框图。如图1所示，音频译码系统10可包括源设备12和目的设备14，源设备12产生经编码的码流，因此，源设备12可被称为音频编码装置。目的设备14可对由源设备12所产生的经编码的码流进行解码，因此，目的设备14可被称为音频解码装置。
源设备12包括编码器20，可选地，可包括音频源16、音频预处理器18、通信接口22。
音频源16可包括或可以为任意类型的用于捕获现实世界语音、音乐和音效等的音频捕获设备，和/或任意类型的音频生成设备，例如用于生成语音、音乐和音效的音频处理器或设备。所述音频源可以为存储上述音频的任意类型的内存或存储器。
音频预处理器18用于接收(原始)音频数据17，并对音频数据17进行预处理，得到预处理音频数据19。例如，音频预处理器18执行的预处理可包括修剪或去噪。可以理解的是，音频预处理单元18可以为可选组件。
编码器20用于接收预处理音频数据19并提供编码音频数据21。
源设备12中的通信接口22可用于接收编码音频数据21并通过通信信道13向目的设备14发送编码音频数据21，以便存储或直接重建。
目的设备14包括解码器30，可选地，可包括通信接口28、音频后处理器32和播放设备34。
目的设备14中的通信接口28用于直接从源设备12接收编码音频数据21，并将编码音频数据21提供给解码器30。
通信接口22和通信接口28可用于通过源设备12与目的设备14之间的直连通信链路，例如直接有线或无线连接等，或者通过任意类型的网络，例如有线网络、无线网络或其任意组合、任意类型的私网和公网或其任意类型的组合，发送或接收编码音频数据21。
例如，通信接口22可用于将编码音频数据21封装为报文等合适的格式，和/或使用任意类型的传输编码或处理来处理编码音频数据21，以便在通信链路或通信网络上进行传输。
通信接口28与通信接口22对应，例如，可用于接收传输数据，并使用任意类型的对应传输解码或处理和/或解封装，对传输数据进行处理，得到编码音频数据21。
通信接口22和通信接口28均可配置为如图1中从源设备12指向目的设备14的对应通信信道13的箭头所指示的单向通信接口，或双向通信接口，并且可用于发送和接收消息等，以建立连接，确认并交换与通信链路和/或编码音频数据等数据传输相关的任何其它信息，等等。
解码器30用于接收编码音频数据21并提供解码音频数据31。
音频后处理器32用于对解码音频数据31进行后处理，得到后处理后的后处理音频数据33。音频后处理器32执行的后处理可以包括例如修剪或重采样等。
播放设备34用于接收后处理音频数据33，以向用户或收听者播放音频。播放设备34可以为或包括任意类型的用于播放重建后音频的播放器，例如，集成或外部扬声器。例如，扬声器可包括喇叭、音响等。
图2示例性地给出了本申请所应用的音频译码设备200的示意性框图。在一个实施例中，音频译码设备200可以是音频解码器(例如图1的解码器30)或音频编码器(例如图1的编码器20)。
音频译码设备200包括：用于接收数据的入端口210和接收单元(Rx)220，用于处理数据的处理器、逻辑单元或中央处理器230，用于传输数据的发射单元(Tx)240和出端口250，以及，用于存储数据的存储器260。音频译码设备200还可以包括与入端口210、接收单元220、发射单元240和出端口250耦合的光电转换组件和电光(EO)组件，用于光信号或电信号的出口或入口。
处理器230通过硬件和软件实现。处理器230可以实现为一个或多个CPU芯片、核(例如，多核处理器)、FPGA、ASIC和DSP。处理器230与入端口210、接收单元220、发射单元240、出端口250和存储器260通信。处理器230包括译码模块270(例如编码模块或解码模块)。译码模块270实现本申请中所公开的实施例，以实现本申请所提供的多声道音频信号的编码方法。例如，译码模块270实现、处理或提供各种编码操作。因此，通过译码模块270为音频译码设备200的功能提供了实质性的改进，并影响了音频译码设备200到不同状态的转换。或者，以存储在存储器260中并由处理器230执行的指令来实现译码模块270。
存储器260包括一个或多个磁盘、磁带机和固态硬盘，可以用作溢出数据存储设备，用于在选择性地执行这些程序时存储程序，并存储在程序执行过程中读取的指令和数据。存储器260可以是易失性和/或非易失性的，可以是只读存储器(ROM)、随机存取存储器(RAM)、随机存取存储器(ternary content-addressable memory，TCAM)和/或静态随机存取存储器(SRAM)。
基于上述实施例的描述，本申请提供了一种多声道音频信号的编码方法。
图3是本申请提供的多声道音频信号的编码方法的一个示例性的实施例的流程图。该过程300可由音频译码系统10中的源设备12或音频译码设备200执行。过程300描述为一系列的步骤或操作，应当理解的是，过程300可以以各种顺序执行和/或同时发生，不限于图3所示的执行顺序。如图3所示，该方法包括：
步骤301、获取待编码的第一音频帧。
本实施例的第一音频帧可以是待编码的多声道音频中的任意一个帧，该第一音频帧包括了五个或五个以上的声道信号。例如，5.1声道包括中央声道(C)、前置左声道(left，L)、前置右声道(right，R)、后置左环绕声道(left surround，LS)、后置右环绕声道(right surround，RS)以及0.1声道低频效果(low frequency effects，LFE)共六个声道信号。7.1声道包括C、L、R、LS、RS、LB、RB和LFE共八个声道信号，其中，LFE是从3-120Hz的音频声道，该声道通常发送到专门为低音调而设计的扬声器。
步骤302、获取目标声道对集合中的所有声道对的相关值之和。
目标声道对集合是以获取最大相关值之和为目的得到的，该目标声道对集合包括至少一个声道对，该一个声道对包括至少五个声道信号中的两个声道信号。一个声道对具有一个相关值，该相关值用于表示一个声道对的两个声道信号之间的相关性。
将相关性越高的两个声道信号放在一起编码可以减少冗余，提升编码效率，因此本实施例在组对时，是依据两个声道信号之间的相关值来确定的。为了尽可能找寻相关性最高的组对方式，可以先计算第一音频帧中的至少五个声道信号中两两之间的相关值得到第一音频帧的相关值集合。例如五个声道信号一共可以组成10个声道对，相对应的，相关值集合中可以包括10个相关值。
可选的，可以对相关值做归一化处理，这样所有声道对的相关值都限定在一特定范围内，以便于设置相关值的统一判断标准，例如组对阈值，该组对阈值可以设置为大于或等于0.2、且小于或等于1的值，例如可以是0.3，这样只要两个声道信号的归一化相关值小于组对阈值，就认为该两个声道信号的相关性较差，不需要组对编码。
在一种可能的实现方式中，可以采用以下公式计算两个声道信号(例如ch1和ch2)之间的相关值：
Figure PCTCN2022092518-appb-000001
其中，corr(ch1,ch2)表示声道信号ch1和声道信号ch2之间归一化的相关值，spec_ch1(i)表示声道信号ch1的第i个频点的频域系数，spec_ch2(i)是声道信号ch2的第i个频点的频域系数，N表示不超过一个音频帧总频点数的整数值。
需要说明的是，还可以采用其他的算法或公式计算两个声道信号之间的相关值，本申请对此不做具体限定。
获取目标声道对集合的组对方式包括：以获取最大相关值之和为目的，从至少五个声道信号对应的声道对中选取声道对加入目标声道对集合。目标声道对集合的相关值之和是根据上述组对方式对至少五个声道信号进行组对得到的目标声道对集合中的所有声道对的相关值之和。本实施例的组对方式可以包括以下两种实现方式：
(1)从相关值集合中选取最大的M个相关值，该M个相关值必须是大于或等于组对阈值的，这是因为小于组对阈值的相关值，表示其所对应的声道对中的两个声道信号之间的相关性较低，没有组对编码的必要。而为了提高编码效率，无需把所有大于或等于组对阈值的相关值全都选出来，因此设定了一个M的上限N，即最多选取N个相关值即可。
N可以选取大于或等于2的整数，N的最大值也不能超过第一音频帧的所有声道信号对应的所有声道对的个数。N的值越大，伴随的计算量会增加，而N的值越小，可能会出现声道对集合丢失的情况，从而降低编码效率。
可选的，可以将N设置为最大声道对数加一，即
Figure PCTCN2022092518-appb-000002
CH表示第一音频帧包含的声道信号的个数。例如，5.1声道包含五个声道信号，则N＝3；7.1声道包含七个声道信号，则N＝4。
然后根据M个相关值获取M个声道对集合，每个声道对集合至少包括M个相关值对应的M个声道对的其中之一，且当声道对集合包括两个以上声道对时，两个以上声道对不包含相同的声道信号。例如，5.1声道，根据相关值集合选出来的最大相关值对应的3 个声道对是(L,R)、(R,C)和(LS,RS)，其中(LS,RS)的相关值小于组对阈值，因此排除，那么剩余的两个声道对(L,R)和(R,C)可以得到两个声道对集合，这两个声道对集合的其中一个包括(L,R)，另一个包括(R,C)。
以M个大于或等于组对阈值的相关值对应的M个声道对中的任意一个(例如第一声道对)为例，本实施例获取M个声道对集合的方法可以包括：将第一声道对加入目标声道对集合，M个声道对集合包括该目标声道对集合，当多个声道对中除关联声道对外的其他声道对中包括相关值大于组对阈值的声道对时，从其他声道对中选取相关值最大的一个声道对加入目标声道对集合，关联声道对包括已加入目标声道对集合的声道对所包括的声道信号中的任意一个。
上述过程除将第一声道对加入目标声道对集合的步骤外，均为迭代处理步骤。即
a、判断多个声道对中除关联声道对外的其他声道对中是否包括相关值大于组对阈值的声道对。
b、若包括相关值大于组对阈值的声道对，则从其他声道对中选取相关值最大的一个声道对加入目标声道对集合。
此时只要其他声道对中包括相关值大于组对阈值的声道对，就可以迭代执行上述步骤b。
可选的，为了减少计算量，可以从相关值集合中将小于组对阈值的相关值删除，这样可以减少声道对的个数，进而减少迭代的次数。
(2)根据多个声道对获取至少五个声道信号对应的所有声道对集合，根据相关值集合获取所有声道对集合中任意一个声道对集合包含的所有声道对的相关值之和，将所有声道对集合中对应于最大的相关值之和的声道对集合确定为目标声道对集合。
相关值集合包括了第一音频帧的至少五个声道信号的多个声道对的相关值，将该多个声道对进行有规则的组合(即同一声道对集合中的多个声道对之间不能包含相同的声道信号)，可以得到该至少五个声道信号对应的多个声道对集合。
在一种可能的实现方式中，当声道信号的个数为奇数时，可以采用以下公式计算所有声道对集合的个数：
Figure PCTCN2022092518-appb-000003
在一种可能的实现方式中，当声道信号的个数为偶数时，可以采用以下公式计算所有声道对集合的个数：
Figure PCTCN2022092518-appb-000004
其中，Pair_num表示所有声道对集合的个数，CH表示第一音频帧里参与多声道处理的声道信号的个数，是经过多声道掩码筛选后的结果。
可选的，为了减少计算量，得到相关值集合之后，可以根据多个声道对中除非相关声道对外的其他声道对获取多个声道对集合，该非相关声道对的相关值小于组对阈值，这样在获取声道对集合时可以减少参与计算的声道对的个数，进而减少声道对集合的个数，在后续步骤也可以减少相关值之和的计算量。
步骤303、当相关值之和大于预设阈值时，对至少五个声道信号中的至少两个声道信 号进行能量均衡处理以得到至少两个均衡声道信号。
在一种可能的实现方式中，可以先获取至少五个声道信号的波动区间值，然后根据至少五个声道信号的波动区间值确定能量均衡模式，再根据能量均衡模式分别对至少五个声道信号进行能量均衡处理以得到至少五个均衡声道信号。
波动区间值用于表示至少五个声道信号之间的能量或幅度的差异大小。
能量均衡模式包括第一能量均衡模式和第二能量均衡模式，其中，第一能量均衡模式使用一个声道对中两个声道信号获取一个声道对对应的两个均衡声道信号。第二能量均衡模式使用一个声道对中两个声道信号以及一个声道对外至少一个声道信号来获取一个声道对对应的两个均衡声道信号。
根据至少五个声道信号的波动区间值确定能量均衡模式可以包括：当波动区间值符合预设条件时，确定能量均衡模式为第一能量均衡模式；当波动区间值不符合预设条件时，确定能量均衡模式为第二能量均衡模式。
上述波动区间值包括第一音频帧的能量平整度，波动区间值符合预设条件是指能量平整度小于第一阈值；或者，波动区间值包括第一音频帧的幅度平整度，波动区间值符合预设条件是指幅度平整度小于第二阈值；或者，波动区间值包括第一音频帧的能量偏离度，波动区间值符合预设条件是指能量偏离度不在第一预设范围内；或者，波动区间值包括第一音频帧的幅度偏离度，波动区间值符合预设条件是指幅度偏离度不在第二预设范围内。
在本发明实施例中，能量平整度表示的是经过多声道筛选单元筛选后的多个声道的当前帧频域系数能量归一化后的帧能量的波动性，可以通过平整度计算公式来衡量。当当前帧的所有的声道的能量相同时，当前帧的能量平整度为1；当当前帧的某个声道的能量为0时，当前帧的能量平整度为0，因此声道间的能量平整度的取值范围是[0,1]。声道间的能量的波动性越大，其能量平整度的值越小。在一种实施方式中，可以针对所有的声道格式(比如5.1、7.1、9.1、11.1)设置一个统一的第一阈值，例如可以为0.483，0.492，或0.504等等。在另一种实施方式中，针对不同的声道格式设置不同的第一阈值。比如，5.1声道格式的第一阈值为0.511，7.1声道格式的第一阈值为0.563，9.1声道格式的第一阈值为0.608，11.1声道格式的第一阈值为0.654。
幅度平整度表示的是经过多声道筛选单元筛选后的多个声道的当前帧频域系数幅度归一化后的帧幅度的波动性，可以通过平整度计算公式来衡量。当所有的声道的帧幅度相同时，其平整度为1；当其中某个声道的帧幅度为0时，其平整度为0。因此幅度平整度的范围在[0,1]之间。声道间的幅度的波动性越大，其平整度的值越小。在一种实施方式中，可以针对所有的声道格式(比如5.1、7.1、9.1、11.1)设置一个统一的第二阈值，例如可以为0.695，0.701,或0.710等等。在另一种实施方式中，可以针对不同的声道格式给出不同的第二阈值，例如，5.1声道格式的第二阈值可以为0.715，7.1声道格式的第二阈值可以为0.753，9.1声道格式的第二阈值可以为0.784，11.1声道格式的第二阈值可以为0.809。
由于幅度和能量之间存在平方关系，因此幅度平整度和能量平整度也存在平方的关系，即幅度平整度的平方对应的声道间的帧幅度的波动性近似等同于能量平整度对应的声道间帧能量的波动性。
本实施例可以通过至少五个声道信号的上述多种表示波动区间值的信息确定能量均衡模式，其包括能量平整度、幅度平整度、能量偏离度或者幅度偏离度。
(1)计算至少五个声道信号的能量值，根据至少五个声道信号的能量值获取第一音频帧的能量平整度，当第一音频帧的能量平整度小于第一阈值时，确定能量均衡模式为第一能量均衡模式；当第一音频帧的能量平整度大于或等于第一阈值时，确定能量均衡模式为第二能量均衡模式。
(2)计算至少五个声道信号的幅度值，根据至少五个声道信号的幅度值获取第一音频帧的幅度平整度，当第一音频帧的幅度平整度小于第二阈值时，确定能量均衡模式为第一能量均衡模式；当第一音频帧的幅度平整度大于或等于第二阈值时，确定能量均衡模式为第二能量均衡模式。
(3)计算至少五个声道信号的能量值，根据至少五个声道信号的能量值获取第一音频帧的能量偏离度，当第一音频帧的能量偏离度不在第一预设范围内时，确定能量均衡模式为第一能量均衡模式；当第一音频帧的能量偏离度在第一预设范围内时，确定能量均衡模式为第二能量均衡模式。
(4)计算至少五个声道信号的幅度值，根据至少五个声道信号的幅度值获取第一音频帧的幅度偏离度，当第一音频帧的幅度偏离度不在第二预设范围内时，确定能量均衡模式为第一能量均衡模式；当第一音频帧的幅度偏离度在第二预设范围内时，确定能量均衡模式为第二能量均衡模式。
需要说明的是，本申请还可以采用其他的能量均衡模式，对此不做具体限定。
在一种可能的实现方式中，根据至少五个声道信号的波动区间值确定能量均衡模式之前，还可以先根据与第一音频帧对应的编码码率确定能量均衡模式，即判断该编码码率是否大于码率阈值，当该编码码率大于码率阈值时，确定能量均衡模式为第二能量均衡模式；当该编码码率小于或等于码率阈值时，根据至少五个声道信号的波动区间值确定能量均衡模式。
当能量均衡模式为第一能量均衡模式时，可以针对组对方式对应的目标声道对集合中的当前声道对，计算当前声道对包含的两个声道信号的能量或幅度值的平均值，根据平均值分别对两个声道信号进行能量均衡处理以得到对应的两个均衡声道信号。
这样当至少五个声道信号的波动区间值较大时，可以只在相关的两个声道信号之间进行能量均衡，使得立体声处理时对于比特的分配更符合声道信号的波动区间值，避免在低码率的编码环境中能量大的声道对因比特不足导致编码噪声可能会远大于能量小的声道对的编码噪声，而能量小的声道对的比特会有冗余的问题。
当能量均衡模式为第二能量均衡模式时，可以计算至少五个声道信号的能量或幅度值的平均值，根据平均值分别对至少五个声道信号进行能量均衡处理得到至少五个均衡声道信号。
需要说明的是，步骤303主要是对至少五个声道信号中的至少两个声道信号进行能量均衡处理以得到至少两个均衡声道信号，该至少两个声道信号是目标声道对集合中已经组对的声道信号，除目标声道对集合中已经组对的声道信号外，剩余没组队的声道信号则直接进行编码处理。
步骤304、对至少两个均衡声道信号进行编码以得到经编码的码流。
步骤305、当相关值之和小于或等于预设阈值时，对至少五个声道信号进行编码以得到经编码的码流。
本实施例中，如果相关值之和小于或等于预设阈值，表示目标声道对集合中的声道对中的两个声道信号之间的相关性较低，没有组对编码的必要，进而也不需要对至少五个声道信号进行能量均衡处理，此时编码的对象是该至少五个声道信号，而非均衡声道信号。
本实施例以获取最大相关值之和为目的对音频帧包含的至少五个声道信号进行组对得到目标声道对集合，当目标声道对集合的相关值之和大于预设阈值时，对至少五个声道信号进行能量均衡处理，进而编码，可以提高音频帧的编码效率。
以下通过两个具体的实施例对图3所示方法实施例中如何确定组对方式和能量均衡模式的过程进行描述。以5.1声道为例，该5.1声道包括中央声道(C)、前置左声道(left，L)、前置右声道(right，R)、后置左环绕声道(left surround，LS)、后置右环绕声道(right surround，RS)以及0.1声道低频效果(low frequency effects，LFE)，按照例如表1所示，给上述六个声道信号设置声道索引。
表1
声道索引 声道信号
0 L
1 R
2 LS
3 RS
4 C
5 LFE
图4a是本申请提供的多声道音频信号的编码方法所应用的编码装置的一个示例性的结构图，该编码装置可以是音频译码系统10中的源设备12的编码器20，也可以是音频译码设备200中的译码模块270。该编码装置可以包括多声道自适应组对模块、声道编码模块和码流复用接口，其中，
多声道自适应组对模块的输入包括5.1声道的六个声道信号(L、R、C、LS、RS、LFE)，以及多声道处理指示符(MultiProcFlag)，输出包括组对后的六个声道信号(M1、S1、M2、S2、C、LFE)，其中，M1和S1是组对得到的一对声道对，M2和S2是组对得到的一对声道对，以及多声道边信息(sideInfoMc)，该多声道边信息包括声道对集合。
声道编码模块使用单声道编码单元(或者单声道声道盒、单声道工具)对多声道自适应组对模块输出的声道信号(M1、S1、M2、S2、C、LFE)进行编码输出对应的编码声道信号(E1-E6)。单声道编码单元对声道信号编码过程中，对具有较高能量(或较高振幅)的声道信号分配较多的比特数，对具有较少能量(或较少振幅)的声道信号分配较少的比特数。可选的，声道编码模块也可以采用立体声编码单元，例如参数立体声编码器或损耗立体声编码器对多声道处理模块输出的处理声道信号进行编码。
需要说明的是，未组对的声道信号(例如C和LFE)可以直接输入声道编码模块得到编码声道信号E5和E6。
码流复用接口产生编码多声道信号，该编码多声道信号包括声道编码模块输出的编码声道信号(E1-E6)和边信息(包括多声道边信息)。可选的，码流复用接口可以将编码多声道信号处理成串行信号或串行比特流。
图4b是多声道自适应组对模块的一个示例性的结构图，如图4b所示，多声道自适应组对模块包括：多声道筛选单元、全局相关值统计单元、多声道能量均衡选择模块和组对处理模块。
多声道筛选单元根据多声道处理指示符(MultiProcFlag)从六个声道信号(L、R、C、LS、RS、LFE)中筛选出参与多声道处理的五个声道信号，即L、R、C、LS、RS。
全局相关值统计单元先计算参与多声道处理的声道信号，即L、R、C、LS、RS中任意两个声道信号之间归一化的相关值。本申请可以采用以下公式计算两个声道信号(例如声道信号ch1和声道信号ch2)之间的相关值：
Figure PCTCN2022092518-appb-000005
其中，corr(ch1,ch2)表示声道信号ch1和声道信号ch2之间归一化的相关值，spec_ch1(i)表示声道信号ch1的第i个频点的频域系数，spec_ch2(i)表示声道信号ch2的第i个频点的频域系数，N表示不超过一个音频帧总频点数的整数值。然后根据任意两个声道信号之间归一化的相关值确定参与多声道处理的声道信号对应的所有声道对集合中，相关值之和(即声道对集合中包含的所有声道对的相关值之和)最大者、以及该最大者对应的声道对集合(视为目标声道对集合)。最后输出全局相关值边信息，该全局相关值边信息包括最大相关值之和corr_sum_max和目标声道对集合。假设目标声道对集合包括(R,C)和(LS,RS)，最大相关值之和corr_sum_max＝corr(L,R)+corr(LS,RS)。
需要说明的是，全局相关值统计单元在得到任意两个声道信号之间归一化的相关值之后，可以根据设定的组对阈值对相关值进行筛选，即，大于或等于组对阈值的相关值保留，而小于组对阈值的相关值删除，或者将其值设置为0。这样可以减少计算量。
多声道能量均衡选择模块根据编码码率和五个声道信号确定是否需要对五个声道信号进行能量均衡处理。五个声道信号的组队方式为全局组对方式，该方式以获取最大相关值之和为目的，具体可以参照步骤302的描述。当目标声道对集合的相关值之和大于预设阈值时，确定需要对五个声道信号进行能量均衡处理；当目标声道对集合的相关值之和小于或等于预设阈值时，确定不需要对五个声道信号进行能量均衡处理。当确定需要对五个声道信号进行能量均衡处理时，确定能量均衡模式。
图4c是组对处理模块的一个示例性的结构图，如图5b所示，组对处理模块包括组对判决器、能量均衡单元和立体声处理盒。
组对判决器先计算各个声道信号的能量或幅度值，本申请可以采用以下公式计算声道信号(ch)的能量或幅度值：
Figure PCTCN2022092518-appb-000006
其中，energy(ch)表示声道信号ch的能量或幅度值，sepc_coeff(ch,i)表示声道信号ch的第i个频点的频域系数，N表示不超过一个音频帧总频点数的整数值。
然后计算各个声道信号的归一化的能量或幅度值，本申请可以采用以下公式计算声道信号(ch)的归一化的能量或幅度值：
Figure PCTCN2022092518-appb-000007
其中，energy_uniform(ch)表示声道信号ch的归一化的能量或幅度值，energy_max表示五个声道信号的能量或幅度值(即energy(L)、energy(R)、energy(C)、energy(LS)和energy(RS))中的最大者。若energy_max＝0，则energy_uniform(ch)均为0。
接下来计算五个声道信号的波动区间值，可选的，波动区间值可以是指能量平整度，本申请可以采用以下公式计算五个声道信号的能量平整度：
Figure PCTCN2022092518-appb-000008
其中，efm表示五个声道信号的能量平整度，L、R、C、LS、RS的声道索引参见表1。
可选的，波动区间值也可以是指能量偏离度，基于上述计算得到的归一化的能量或幅度值energy_uniform(ch)，本申请可以采用以下公式计算五个声道信号的平均能量或幅度值：
Figure PCTCN2022092518-appb-000009
其中，avg_energy_uniform表示五个声道信号的平均能量或幅度值，L、R、C、LS、RS的声道索引参见表1。
采用以下公式计算声道信号(ch)的能量偏离度：
Figure PCTCN2022092518-appb-000010
其中，deviation(ch)表示声道信号ch的能量偏离度。将L、R、C、LS、RS的能量偏离度中的最大者确定为五个声道信号的能量偏离度deviation。
可选的，波动区间值还可以是指幅度值或幅度偏离度，其原理和上述能量相关的值类似，此处不再赘述。
如上所述，本申请的能量均衡模式包括两种实现方式，其中，Pair能量均衡模式是针对模块选择单元确定的组对方式对应的目标声道对集合中的各个声道对，使用一个声道对中两个声道信号获取该一个声道对对应的两个均衡声道信号。整体能量均衡模式是使用一个声道对中两个声道信号以及一个声道对外至少一个声道信号来获取该一个声道对对应的两个均衡声道信号。而对于没有组对的声道信号，其对应的均衡声道信号即为该声道信号本身。
组对判决器根据波动区间值确定能量均衡模式，包括以下两种判断方式：
(1)当efm小于第一阈值时，能量均衡模式为Pair能量均衡模式；当efm大于或等于第一阈值时，能量均衡模式为整体能量均衡模式。
(2)当deviation在值区间[threshold,1/threshold]之内时，能量均衡模式为整体能量均衡模式；当deviation不在值区间[threshold,1/threshold]之内时，能量均衡模式为Pair能量均衡模式。threshold的取值范围可以是(0,1)。
deviation可以表示当前帧的各声道的频域幅度相对于当前帧的各声道的频域幅度的平均值的比值，即幅度偏离度。当当前帧的当前声道的频域幅度和当前帧的各声道的频域幅度的平均值之间的比例关系小于5(对应于threshold＝0.2)时，可以分为两种情况：一、 当前声道的频域幅度小于或等于当前帧的各声道的频域幅度的平均值，满足条件的“当前声道的频域幅度/当前帧的各声道的频域幅度的平均值”在(0.2,1]之间，也就是在(threshold,1]之间；二，当前声道的频域幅度大于当前帧的各声道的频域幅度的平均值，满足条件的“当前声道的频域幅度/当前帧的各声道的频域幅度的平均值”在(1,5)之间；综合以上两种情况，当当前声道的频域幅度和当前帧的各声道的频域幅度的平均值的比例关系小于5时，满足条件的“当前声道的频域幅度/当前帧的各声道的频域幅度的平均值”的范围在(0.2,5)之间，也就是在(threshold,1/threshold)之间，(threshold,1/threshold)即为上述的第二预设范围。其中，threshold的取值可以在(0,1)之间，threshold的值越小，表示当前声道的频域幅度相对于当前帧的各声道的频域幅度的平均值的波动越大，threshold的值越大，表示当前声道的频域幅度相对于当前帧的各声道的频域幅度的平均值的波动越小。其中，threshold的取值可以是0.2，0.15，0.125，0.11，或0.1等等。
deviation也可以表示的是各声道的频域能量相对于各声道的频域能量的平均值的比值，即能量偏离度。当当前帧的当前声道的频域能量和各声道的频域能量的平均值的比例关系小于25(threshold＝0.04)时，可以分为两种情况：一、当前声道的频域能量小于或等于当前帧的各声道的频域能量的平均值，满足条件的“当前声道的频域能量/当前帧的各声道的频域能量的平均值”在(0.04,1]之间，也就是(threshold,1]；二，当前声道的频域能量大于当前帧的各声道的频域能量的平均值，满足条件的“当前声道的频域能量/当前帧的各声道的频域能量的平均值”在(1,25)之间；综合以上两种情况，当当前声道的频域能量和当前帧的各声道的频域能量的平均值的比例关系小于25时，满足条件的“当前声道的频域能量/当前帧的各声道的频域能量的平均值”的范围在(0.04,25)之间，也就是在(threshold,1/threshold)之间，(threshold,1/threshold)即为上述的第一预设范围。其中，threshold可以在(0,1)之间，threshold的值越小，表示当前声道的频域能量相对于当前帧的各声道的频域能量的平均值的波动越大，threshold的值越大，表示当前声道的频域能量相对于当前帧的各声道的频域能量的平均值的波动越小。Threshold的取值可以是0.04，0.0225，0.015625，0.0121，或0.01等等。
由于幅度和能量之间存在平方关系，因此幅度偏离度和能量偏离度也存在平方的关系，即幅度偏离度的平方对应的声道间的帧幅度的波动性近似等同于能量偏离度对应的声道间帧能量的波动性。
在另一种实施方式中，上述的第一预设范围也可以扩展成(0,1/threshold)，此时Pair能量均衡的区间范围为[1/threshold,+∞)，此时表明当当前声道的频域能量大于当前帧的各声道的频域能量的平均值，并且“当前声道的频域能量/当前帧的各声道的频域能量的平均值”大于1/threshold时，才进行Pair能量均衡。
在另一种实施方式中，上述的第二预设范围也可以扩展成(0,1/threshold)，此时Pair幅度均衡的区间范围为[1/threshold,+∞)，此时表明当当前声道的频域幅度大于当前帧的各声道的频域幅度的平均值，并且“当前声道的频域幅度/当前帧的各声道的频域幅度的平均值”大于1/threshold时，才进行Pair幅度均衡。
需要说明的是，组对判决器可以根据五个声道信号计算归一化的能量或幅度值，进而得到能量平整度或能量偏离度，也可以只根据组对成功的声道信号计算归一化的能量或幅度值，进而得到能量平整度或能量偏离度，还可以根据五个声道信号中的部分声道信号计 算归一化的能量或幅度值，进而得到能量平整度或能量偏离度。本申请对此不做具体限定。
立体声处理单元可以采用基于预测的或者基于Karhunen-Loeve变换(Karhunen-Loeve Transform，KLT)的处理，即输入的两个声道信号被旋转(例如经由2×2旋转矩阵)以最大化能量压缩，从而将信号能量集中于一个声道内。
立体声处理单元对输入的两个声道信号处理后，输出该两个声道信号对应的处理声道信号(P1-P4)以及多声道边信息，多声道边信息包括相关值之和和目标声道对集合。
图5是本申请提供的多声道音频的解码方法所应用的解码装置的一个示例性的结构图，该解码装置可以是音频译码系统10中的目的设备14的解码器30，也可以是音频译码设备200中的译码模块270。该解码装置可以包括码流解复用接口、声道解码模块和多声道处理模块，其中，
码流解复用接口接收来自编码装置的编码多声道信号(例如串行比特流bitstream)，解复用后得到编码声道信号(E)和多声道参数(SIDE_PAIR)。例如，E1、E2、E3、E4、…、Ei1、Ei，以及SIDE_PAIR1，SIDE_PAIR2，…，SIDE_PAIRm。
声道解码模块使用单声道解码单元(或者单声道声道盒、单声道工具)对码流解复用接口输出的编码声道信号进行解码输出解码声道信号(D)。例如，E1、E2、E3、E4、…、Ei1、Ei分别通过一个单声道解码单元进行解码得到E1解码得D1、D2、D3、D4、…、Di1、Di。
多声道处理模块包括多个立体声处理单元，立体声处理单元可以采用基于预测的或者基于KLT的处理，即输入的两个声道信号被反旋转(例如经由2×2旋转矩阵)，从而将信号变换到原始信号方向。
声道解码模块输出的解码声道信号藉由多声道参数可以识别哪两个解码声道信号组对，将组对的解码声道信号输入立体声处理单元，立体声处理单元对输入的两个解码声道信号处理后，输出该两个解码声道信号对应的声道信号(CH)。例如，立体声处理单元1根据SIDE_PAIR1对D1和D2处理，得到CH1和CH2，立体声处理单元2根据SIDE_PAIR2对D3和D4处理，得到CH3和CH4，…，立体声处理单元m根据SIDE_PAIRm对Di-1和Di处理，得到CHi-1和CHi。
需要说明的是，针对未组对的声道信号(例如CHj)不需要经过多声道处理模块中的立体声处理单元处理，可以解码后直接输出。
图6为本申请编码装置实施例的结构示意图，如图6所示，该装置可以应用于上述实施例中的源设备12或音频译码设备200。本实施例的编码装置可以包括：获取模块601、处理模块602和编码模块603。其中，
获取模块601，用于获取待编码的第一音频帧，所述第一音频帧包括至少五个声道信号；获取目标声道对集合的相关值之和，所述目标声道对集合是以获取最大相关值之和为目的得到的，所述目标声道对集合包括至少一个声道对，一个声道对包括所述至少五个声道信号中的两个声道信号，所述一个声道对具有一个相关值，所述相关值用于表示所述一个声道对的两个声道信号之间的相关性；处理模块602，用于当所述相关值之和大于预设阈值时，对所述至少五个声道信号进行能量均衡处理以得到至少五个均衡声道信号；编码模块603，用于对所述至少五个均衡声道信号进行编码。
在一种可能的实现方式中，所述编码模块603，还用于当所述相关值之和小于或等于 所述预设阈值时，对所述至少五个声道信号进行编码。
在一种可能的实现方式中，所述处理模块602，具体用于获取所述至少五个声道信号的波动区间值；根据所述至少五个声道信号的波动区间值确定能量均衡模式；根据所述能量均衡模式分别对所述至少五个声道信号进行能量均衡处理以得到所述至少五个均衡声道信号。
在一种可能的实现方式中，所述处理模块602，具体用于当所述波动区间值符合预设条件时，确定所述能量均衡模式为第一能量均衡模式；或者，当所述波动区间值不符合预设条件时，确定所述能量均衡模式为第二能量均衡模式。
在一种可能的实现方式中，所述波动区间值包括所述第一音频帧的能量平整度；所述波动区间值符合预设条件是指所述能量平整度小于第一阈值；或者，所述波动区间值包括所述第一音频帧的幅度平整度；所述波动区间值符合预设条件是指所述幅度平整度小于第二阈值；或者，所述波动区间值包括所述第一音频帧的能量偏离度；所述波动区间值符合预设条件是指所述能量偏离度不在第一预设范围内；或者，所述波动区间值包括所述第一音频帧的幅度偏离度；所述波动区间值符合预设条件是指所述幅度偏离度不在第二预设范围内。
在一种可能的实现方式中，当所述能量均衡模式为所述第一能量均衡模式时，所述处理模块602，具体用于针对所述目标声道对集合中的当前声道对，计算所述当前声道对包含的两个声道信号的能量或幅度值的平均值，根据所述平均值分别对所述两个声道信号进行能量均衡处理以得到对应的两个均衡声道信号。
在一种可能的实现方式中，当所述能量均衡模式为所述第二能量均衡模式时，所述处理模块602，具体用于计算所述至少五个声道信号的能量或幅度值的平均值，根据所述平均值分别对所述至少五个声道信号进行能量均衡处理得到所述至少五个均衡声道信号。
在一种可能的实现方式中，所述处理模块602，还用于判断与所述第一音频帧对应的编码码率是否大于码率阈值；当所述编码码率大于所述码率阈值时，确定所述能量均衡模式为第二能量均衡模式；当所述编码码率小于或等于所述码率阈值时，才根据所述波动区间值确定所述能量均衡模式。
本实施例的装置，可以用于执行图3所示方法实施例的技术方案，其实现原理和技术效果类似，此处不再赘述。
图7为本申请设备实施例的结构示意图，如图7所示，该设备可以是上述实施例中的编码设备。本实施例的设备可以包括：处理器701和存储器702，存储器702，用于存储一个或多个程序；当所述一个或多个程序被所述处理器701执行，使得所述处理器701实现如图3所示方法实施例的技术方案。
在实现过程中，上述方法实施例的各步骤可以通过处理器中的硬件的集成逻辑电路或者软件形式的指令完成。处理器可以是通用处理器、数字信号处理器(digital signal processor,DSP)、特定应用集成电路(application-specific integrated circuit，ASIC)、现场可编程门阵列(field programmable gate array,FPGA)或其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。本申请公开的方法的步骤可以直接体现为硬件编码处理器执行完成，或者用编码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储 器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器，处理器读取存储器中的信息，结合其硬件完成上述方法的步骤。
上述各实施例中提及的存储器可以是易失性存储器或非易失性存储器，或可包括易失性和非易失性存储器两者。其中，非易失性存储器可以是只读存储器(read-only memory，ROM)、可编程只读存储器(programmable ROM，PROM)、可擦除可编程只读存储器(erasable PROM，EPROM)、电可擦除可编程只读存储器(electrically EPROM，EEPROM)或闪存。易失性存储器可以是随机存取存储器(random access memory，RAM)，其用作外部高速缓存。通过示例性但不是限制性说明，许多形式的RAM可用，例如静态随机存取存储器(static RAM，SRAM)、动态随机存取存储器(dynamic RAM，DRAM)、同步动态随机存取存储器(synchronous DRAM，SDRAM)、双倍数据速率同步动态随机存取存储器(double data rate SDRAM，DDR SDRAM)、增强型同步动态随机存取存储器(enhanced SDRAM，ESDRAM)、同步连接动态随机存取存储器(synchlink DRAM，SLDRAM)和直接内存总线随机存取存储器(direct rambus RAM，DR RAM)。应注意，本文描述的系统和方法的存储器旨在包括但不限于这些和任意其它适合类型的存储器。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(read-only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (23)

  1. 一种多声道音频信号的编码方法，其特征在于，包括：
    获取待编码的第一音频帧，所述第一音频帧包括至少五个声道信号；
    获取目标声道对集合中的所有声道对的相关值之和，所述目标声道对集合包括至少一个声道对，一个声道对包括所述至少五个声道信号中的两个声道信号，所述一个声道对具有一个相关值，所述相关值用于表示所述一个声道对的两个声道信号之间的相关性；
    当所述相关值之和大于预设阈值时，对所述至少五个声道信号中的至少两个声道信号进行能量均衡处理以得到至少两个均衡声道信号；
    对所述至少两个均衡声道信号进行编码以得到经编码的码流。
  2. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    当所述相关值之和小于或等于所述预设阈值时，对所述至少五个声道信号进行编码以得到经编码的码流。
  3. 根据权利要求1或2所述的方法，其特征在于，所述对所述至少五个声道信号中的至少两个声道信号进行能量均衡处理以得到至少两个均衡声道信号，包括：
    获取所述至少五个声道信号的波动区间值；
    根据所述至少五个声道信号的波动区间值确定能量均衡模式；
    根据所述能量均衡模式分别对所述至少两个声道信号进行能量均衡处理以得到所述至少两个均衡声道信号。
  4. 根据权利要求3所述的方法，其特征在于，所述根据所述至少五个声道信号的波动区间值确定能量均衡模式，包括：
    当所述波动区间值符合预设条件时，确定所述能量均衡模式为第一能量均衡模式；或者，
    当所述波动区间值不符合预设条件时，确定所述能量均衡模式为第二能量均衡模式。
  5. 根据权利要求4所述的方法，其特征在于，所述波动区间值包括所述第一音频帧的能量平整度；所述波动区间值符合预设条件是指所述能量平整度小于第一阈值；或者，
    所述波动区间值包括所述第一音频帧的幅度平整度；所述波动区间值符合预设条件是指所述幅度平整度小于第二阈值；或者，
    所述波动区间值包括所述第一音频帧的能量偏离度；所述波动区间值符合预设条件是指所述能量偏离度不在第一预设范围内；或者，
    所述波动区间值包括所述第一音频帧的幅度偏离度；所述波动区间值符合预设条件是指所述幅度偏离度不在第二预设范围内。
  6. 根据权利要求4或5所述的方法，其特征在于，当所述能量均衡模式为所述第一能量均衡模式时，所述对所述至少五个声道信号中的至少两个声道信号进行能量均衡处理以得到至少两个均衡声道信号，包括：
    对所述目标声道对集合对应的声道信号进行能量均衡处理以得到所述至少两个均衡声道信号。
  7. 根据权利要求6所述的方法，其特征在于，所述对所述目标声道对集合对应的声道信号进行能量均衡处理以得到所述至少两个均衡声道信号，包括：
    针对所述目标声道对集合中的当前声道对，计算所述当前声道对包含的两个声道信号的能量值或幅度值的平均值，根据所述平均值分别对所述当前声道对包含的两个声道信号进行能量均衡处理以得到两个均衡声道信号。
  8. 根据权利要求4或5所述的方法，其特征在于，当所述能量均衡模式为所述第二能量均衡模式时，所述对所述至少五个声道信号中的至少两个声道信号进行能量均衡处理以得到至少两个均衡声道信号，包括：
    计算所述至少五个声道信号的能量值或幅度值的平均值，根据所述平均值分别对所述至少五个声道信号进行能量均衡处理得到所述至少五个均衡声道信号。
  9. 根据权利要求3-8中任一项所述的方法，其特征在于，所述根据所述至少五个声道信号的波动区间值确定能量均衡模式之前，还包括：
    判断与所述第一音频帧对应的编码码率是否大于码率阈值；
    当所述编码码率大于所述码率阈值时，确定所述能量均衡模式为第二能量均衡模式；
    当所述编码码率小于或等于所述码率阈值时，才根据所述波动区间值确定所述能量均衡模式。
  10. 根据权利要求1-9中任一项所述的方法，其特征在于，所述方法还包括：
    对所述至少五个声道信号中未进行能量均衡处理的声道信号进行编码。
  11. 一种编码装置，其特征在于，包括：
    获取模块，用于获取待编码的第一音频帧，所述第一音频帧包括至少五个声道信号；获取目标声道对集合中的所有声道对的相关值之和，所述目标声道对集合包括至少一个声道对，一个声道对包括所述至少五个声道信号中的两个声道信号，所述一个声道对具有一个相关值，所述相关值用于表示所述一个声道对的两个声道信号之间的相关性；
    处理模块，用于当所述相关值之和大于预设阈值时，对所述至少五个声道信号中的至少两个声道信号进行能量均衡处理以得到至少两个均衡声道信号；
    编码模块，用于对所述至少两个均衡声道信号进行编码以得到经编码的码流。
  12. 根据权利要求11所述的装置，其特征在于，所述编码模块，还用于当所述相关值之和小于或等于所述预设阈值时，对所述至少五个声道信号进行编码以得到经编码的码流。
  13. 根据权利要求11或12所述的装置，其特征在于，所述处理模块，具体用于获取所述至少五个声道信号的波动区间值；根据所述至少五个声道信号的波动区间值确定能量均衡模式；根据所述能量均衡模式分别对所述至少两个声道信号进行能量均衡处理以得到所述至少两个均衡声道信号。
  14. 根据权利要求13所述的装置，其特征在于，所述处理模块，具体用于当所述波动区间值符合预设条件时，确定所述能量均衡模式为第一能量均衡模式；或者，当所述波动区间值不符合预设条件时，确定所述能量均衡模式为第二能量均衡模式。
  15. 根据权利要求14所述的装置，其特征在于，所述波动区间值包括所述第一音频帧的能量平整度；所述波动区间值符合预设条件是指所述能量平整度小于第一阈值；或者，
    所述波动区间值包括所述第一音频帧的幅度平整度；所述波动区间值符合预设条件是指所述幅度平整度小于第二阈值；或者，
    所述波动区间值包括所述第一音频帧的能量偏离度；所述波动区间值符合预设条件是指所述能量偏离度不在第一预设范围内；或者，
    所述波动区间值包括所述第一音频帧的幅度偏离度；所述波动区间值符合预设条件是指所述幅度偏离度不在第二预设范围内。
  16. 根据权利要求14或15所述的装置，其特征在于，当所述能量均衡模式为所述第一能量均衡模式时，所述处理模块，具体用于对所述目标声道对集合对应的声道信号进行能量均衡处理以得到所述至少两个均衡声道信号。
  17. 根据权利要求16所述的装置，其特征在于，所述处理模块，具体用于针对所述目标声道对集合中的当前声道对，计算所述当前声道对包含的两个声道信号的能量值或幅度值的平均值，根据所述平均值分别对所述当前声道对包含的两个声道信号进行能量均衡处理以得到对应的两个均衡声道信号。
  18. 根据权利要求14或15所述的装置，其特征在于，当所述能量均衡模式为所述第二能量均衡模式时，所述处理模块，具体用于计算所述至少五个声道信号的能量值或幅度值的平均值，根据所述平均值分别对所述至少五个声道信号进行能量均衡处理得到所述至少五个均衡声道信号。
  19. 根据权利要求13-18中任一项所述的装置，其特征在于，所述处理模块，还用于判断与所述第一音频帧对应的编码码率是否大于码率阈值；当所述编码码率大于所述码率阈值时，确定所述能量均衡模式为第二能量均衡模式；当所述编码码率小于或等于所述码率阈值时，才根据所述波动区间值确定所述能量均衡模式。
  20. 根据权利要求11-19中任一项所述的装置，其特征在于，所述编码模块，还用于对所述至少五个声道信号中未进行能量均衡处理的声道信号进行编码。
  21. 一种设备，其特征在于，包括：
    一个或多个处理器；
    存储器，用于存储一个或多个程序；
    当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如权利要求1-10中任一项所述的方法。
  22. 一种计算机可读存储介质，其特征在于，包括计算机程序，所述计算机程序在计算机上被执行时，使得所述计算机执行权利要求1-10中任一项所述的方法。
  23. 一种计算机可读存储介质，其特征在于，包括根据如权利要求1-10中任一项所述的多声道音频信号的编码方法获得的码流。
PCT/CN2022/092518 2021-05-28 2022-05-12 多声道音频信号的编码方法和装置 Ceased WO2022247651A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP22810378.4A EP4336494A4 (en) 2021-05-28 2022-05-12 METHOD AND APPARATUS FOR ENCODING MULTICHANNEL AUDIO SIGNALS

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110595367.2A CN115410584A (zh) 2021-05-28 2021-05-28 多声道音频信号的编码方法和装置
CN202110595367.2 2021-05-28

Publications (1)

Publication Number Publication Date
WO2022247651A1 true WO2022247651A1 (zh) 2022-12-01

Family

ID=84155844

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/092518 Ceased WO2022247651A1 (zh) 2021-05-28 2022-05-12 多声道音频信号的编码方法和装置

Country Status (3)

Country Link
EP (1) EP4336494A4 (zh)
CN (1) CN115410584A (zh)
WO (1) WO2022247651A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118591840A (zh) * 2024-04-25 2024-09-03 北京小米移动软件有限公司 音频数据编码方法、设备、系统及存储介质

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0964751A (ja) * 1995-08-21 1997-03-07 Samsung Electron Co Ltd 多チャネルオーディオ符号化器及び符号化方法
EP1175030A2 (en) * 2000-07-07 2002-01-23 Nokia Mobile Phones Ltd. Method and system for multichannel perceptual audio coding using the cascaded discrete cosine transform or modified discrete cosine transform
US20040109471A1 (en) * 2000-09-15 2004-06-10 Minde Tor Bjorn Multi-channel signal encoding and decoding
WO2009146734A1 (en) * 2008-06-03 2009-12-10 Nokia Corporation Multi-channel audio coding
US20120029916A1 (en) * 2009-02-13 2012-02-02 Nec Corporation Method for processing multichannel acoustic signal, system therefor, and program
WO2015009040A1 (ko) * 2013-07-15 2015-01-22 한국전자통신연구원 다채널 신호를 위한 인코더 및 인코딩 방법, 다채널 신호를 위한 디코더 및 디코딩 방법
CN107592937A (zh) * 2015-03-09 2018-01-16 弗劳恩霍夫应用研究促进协会 用于对多声道信号进行编码或解码的装置与方法
US20200168232A1 (en) * 2017-06-01 2020-05-28 Panasonic Intellectual Property Corporation Of America Encoder and encoding method

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7983922B2 (en) * 2005-04-15 2011-07-19 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Apparatus and method for generating multi-channel synthesizer control signal and apparatus and method for multi-channel synthesizing
US7831434B2 (en) * 2006-01-20 2010-11-09 Microsoft Corporation Complex-transform channel coding with extended-band frequency coding
CN107731238B (zh) * 2016-08-10 2021-07-16 华为技术有限公司 多声道信号的编码方法和编码器
WO2018058379A1 (zh) * 2016-09-28 2018-04-05 华为技术有限公司 一种处理多声道音频信号的方法、装置和系统
ES3059239T3 (en) * 2018-07-04 2026-03-19 Fraunhofer Ges Forschung Multisignal encoder, multisignal decoder, and related methods using signal whitening or signal post processing

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0964751A (ja) * 1995-08-21 1997-03-07 Samsung Electron Co Ltd 多チャネルオーディオ符号化器及び符号化方法
EP1175030A2 (en) * 2000-07-07 2002-01-23 Nokia Mobile Phones Ltd. Method and system for multichannel perceptual audio coding using the cascaded discrete cosine transform or modified discrete cosine transform
US20040109471A1 (en) * 2000-09-15 2004-06-10 Minde Tor Bjorn Multi-channel signal encoding and decoding
WO2009146734A1 (en) * 2008-06-03 2009-12-10 Nokia Corporation Multi-channel audio coding
US20120029916A1 (en) * 2009-02-13 2012-02-02 Nec Corporation Method for processing multichannel acoustic signal, system therefor, and program
WO2015009040A1 (ko) * 2013-07-15 2015-01-22 한국전자통신연구원 다채널 신호를 위한 인코더 및 인코딩 방법, 다채널 신호를 위한 디코더 및 디코딩 방법
CN107592937A (zh) * 2015-03-09 2018-01-16 弗劳恩霍夫应用研究促进协会 用于对多声道信号进行编码或解码的装置与方法
US20200168232A1 (en) * 2017-06-01 2020-05-28 Panasonic Intellectual Property Corporation Of America Encoder and encoding method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4336494A4

Also Published As

Publication number Publication date
EP4336494A1 (en) 2024-03-13
CN115410584A (zh) 2022-11-29
EP4336494A4 (en) 2024-10-02

Similar Documents

Publication Publication Date Title
KR101719094B1 (ko) 컨텐츠 분석 및 가중치를 이용한 바이노럴 룸 임펄스 응답들에 의한 필터링
CN101390443B (zh) 音频编码和解码
KR100928311B1 (ko) 오디오 피스 또는 오디오 데이터스트림의 인코딩된스테레오 신호를 생성하는 장치 및 방법
US12437767B2 (en) Multi-channel audio signal encoding and decoding method and apparatus
EP3762923B1 (en) Audio coding
CN110556118B (zh) 立体声信号的编码方法和装置
CN115691521B (zh) 一种音频信号的编解码方法和装置
CN115691514B (zh) 一种多声道信号的编解码方法和装置
KR102288111B1 (ko) 스테레오 신호의 인코딩 및 디코딩 방법과, 인코딩 및 디코딩 장치
WO2021208792A1 (zh) 音频信号编码方法、解码方法、编码设备以及解码设备
US11696075B2 (en) Optimized audio forwarding
JP2024063226A (ja) ＤｉｒＡＣベースの空間オーディオ符号化のためのパケット損失隠蔽
JP2023533367A (ja) マルチ・チャネル・オーディオ信号符号化方法及び装置
CN115346537A (zh) 一种音频编码、解码方法及装置
KR102929558B1 (ko) 멀티 채널 오디오 신호 코딩 방법 및 장치
EP4336494A1 (en) Encoding method and apparatus for multi-channel audio signals
ES3054167T3 (en) Audio coding with adaptive gain control of downmixed signals
KR20230062836A (ko) 파라미터적으로 코딩된 오디오 처리
CN116798438B (zh) 一种多声道信号的编解码方法和编解码设备以及终端设备
EP4462426A1 (en) Multi-channel signal encoding and decoding methods, encoding and decoding devices, and terminal device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22810378

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2022810378

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2022810378

Country of ref document: EP

Effective date: 20231205

NENP Non-entry into the national phase

Ref country code: DE