Classifications

• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
  • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
  • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
  • G10L21/038—Speech enhancement, e.g. noise reduction or echo cancellation using band spreading techniques
  • G10L21/0388—Details of processing therefor
• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
  • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
  • G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing

Definitions

• the present disclosure relates generally to communications, and more particularly to communication methods and related devices and nodes supporting wireless communications for speech and audio coding.
• BACKGROUND [0002]
• Most existing telecommunication systems operate on a limited audio bandwidth. Stemming from the limitations of the land-line telephony systems, most voice services are limited to only transmitting the lower end of the frequency spectrum and only in a single mono channel. Although the lower bandwidth mono signal is enough for most conversations, there is a desire to increase bandwidth and spatial reproduction to improve intelligibility and sense of presence. The capacity in telecommunication networks is continuously increasing, but it is still of great interest to limit the required bandwidth per communication channel.
• a stereo channel pair typically also includes a non-correlated component which cannot be reconstructed from the down-mix.
• This component may be represented with an inter-channel coherence parameter (ICC).
• ICC inter-channel coherence parameter
• the non-correlated component may be artificially synthesized at a stereo decoder by running the decoded down-mix channel through a decorrelator filter, which aims to create a signal which has low correlation with the decoded down-mix. The strength of the decorrelated component is then controlled with the ICC parameter.
• the same principles apply for multichannel audio such as 5.1 and 7.1.4, and spatial audio representations such as Ambisonics or Spatial Audio Object Coding.
• the number of channels is reduced by exploiting the correlation between the channels and bundling the reduced channel set with metadata or parameters for channel reconstruction or spatial audio rendering at the decoder.
• BWE techniques may be used in combination with parametric methods of spatial reconstruction.
• a down-mix is also produced for the BWE target band.
• SUMMARY There currently exist certain challenge(s).
• the target generation from frequency domain is computationally complex.
• one solution is to inject zeros instead of synthesizing the target when operating in a mode where the BWE is not used, but this may lead to zeros in the target frame when switching back to a mode using the BWE.
• an assumption is that the frames are extracted with an overlap in the encoder such that the decoder may reconstruct the stereo signals using an overlap add strategy.
• the input signals on left and right channels are windowed with a suitable windowing function ⁇ ( ⁇ ) and transformed to DFT (discrete Fourier transform) domain.
• ⁇ ( ⁇ ) windowing function
• DFT discrete Fourier transform
• other frequency domain representation may be used here, such as a QMF (quadrature mirror filter) filter bank, a Hybrid QMF filter bank or an odd DFT (ODFT) representation which is composed of the MDCT (modified discrete cosine transform) and MDST (modified discrete sine transform) transform components.
• the frequency spectrum is partitioned into bands ⁇ , where each band corresponds to a range of frequency coefficients where ⁇ ⁇ denote the total number of bands.
• the band limits are typically set to reflect the resolution of the human auditory perception which suggests narrow bands for low frequencies and wider bands for high frequencies. Note that different band resolution may be used for different parameters.
• the signals are then analyzed within the parametric analysis block to extract the ITD, IPD and ILD parameters.
• the channel coherence may be analyzed, and an ICC parameter may be derived.
• the parameters are encoded by a parameter encoder 218 and added to the bitstream to be stored or transmitted to a decoder.
• a stereo residual bitstream may be produced by a residual encoder 216 and added to the bitstream to be stored or transmitted to a decoder.
• the ITD compensation may be implemented both in time domain before the frequency transform or in frequency domain, but it essentially performs a time shift on one or both channels to eliminate the ITD.
• the phase alignment may be implemented in different ways, but the purpose is to align the phase such that the cancellation is minimized. This ensures maximum energy in the down- mix.
• the ITD and IPD adjustments may be done in frequency bands or on the full frequency spectrum and it should preferably be done using the quantized ITD and IPD parameters to ensure that the modification can be inverted in the decoder stage.
• the various embodiments described herein are independent of the realization of the IPD and ITD parameter analysis and compensation.
• the ITD and IPD adjusted channels are denoted ⁇ ⁇ ⁇ ⁇ ( ⁇ , ⁇ ) ⁇ ⁇ ⁇ ( ⁇ , ⁇ ) [0031]
• the down-mix signal ⁇ ⁇ ( ⁇ , ⁇ ) is encoded by a down-mix encoder 214 to be stored or transmitted to a decoder. This encoding may be done in frequency domain, but it may also be done in time domain.
• a DFT synthesis stage is required to produce at least one time domain version of the down-mix signal ⁇ ⁇ ( ⁇ , ⁇ ) , which is in turn fed to the down-mix encoder. If the down-mix encoder operates on several frequency bands of the down-mix signal, several DFT syntheses may be done to generate the time domain signals of the required bands. [0032]
• the down-mix encoder 214 is described in further detail in Figure 3.
• the down-mix encoder 214 may contain at least two encoding modes, where at least one of the encoding modes 310 operates on at least two frequency bands using a low band encoder 312 and a BWE encoder 314.
• the encoding mode may for instance be selected using a signal analysis which selects the most appropriate mode, or it may be selected by running all possible modes and selecting the mode which gives the best performance for the current frame ⁇ .
• the low band encoder 312 receives a low band down-mix signal ⁇ ⁇ , ⁇ ( ⁇ , ⁇ ) and encodes a representation of this frequency band.
• the BWE encoder 314 receives a delay adjusted high band down-mix signal ⁇ ′ ⁇ , ⁇ ( ⁇ , ⁇ ) from a BWE target signal buffer 316. The buffer allows an alignment of the analysis frame which gives the desired analysis length and alignment, compensating any possible delay from the BWE process.
• the BWE encoder also uses parameters from the low-band encoder, such as the low-band excitation signal in a low band ACELP (algebraic code-excited linear prediction) encoder.
• the BWE target buffer 316 is updated with the high band down-mix signal ⁇ ⁇ , ⁇ ( ⁇ , ⁇ ) which is generated by a DFT synthesis.
• the full band down-mix signal ⁇ ⁇ ( ⁇ , ⁇ ) is used as input to the full band encoder 322.
• the full band signal is also added to a low complex BWE target signal extractor 324.
• the down- mix decoder 224 is configured to decode and reconstruct the down-mix signal encoded by the down-mix encoder 214.
• the down-mix decoder comprises at least a decoding mode 420 that has a full band decoder 422, and a decoding mode 410 having a low band decoder 412 and a BWE decoder 414. Similar to the encoder, the BWE decoder may use parameters from the low-band decoder, such as the low-band excitation signal of an ACELP decoder.
• the output of the decoding mode used for the current frame is a reconstructed down- mix.
• the reconstructed down-mix, the reconstructed stereo parameters and optionally a reconstructed residual signal is fed to a stereo up-mixer 222 to produce a reconstructed stereo signal.
• An efficient realization of the BWE target extractor 324 can be seen in Figure 5, following the steps outlined in Figure 6.
• optional initial step 610 may be done by intermediate length determiner 510 to determine an intermediate length of the interpolated signal.
• the intermediate length is determined by rescaling the length of the input frame ⁇ in the input sampling frequency ⁇ ⁇ such that the upper limit of the target band matches the Nyquist frequency in the interpolated frame.
• Different target bands may depend on the bandwidth of the low band encoder. If the low band encoder uses a lower bandwidth, the start of the BWE target band should match the end of the low band bandwidth. Typically, the BWE target band is extracted to have a small overlap with the low band encoder bandwidth to ensure a smooth frequency transition between the encoded bands.
• the interpolator 520 performs an interpolation of the full band input frame ⁇ ⁇ ( ⁇ , ⁇ ) to an intermediate frame ⁇ ⁇ ( ⁇ , ⁇ ).
• the interpolator may use linear interpolation, as illustrated in Figure 8.
• the sample points are assumed to be connected by straight lines, and the resampling uses the sampling point intersecting with the line.
• the first two lines of the equality for ⁇ ⁇ ( ⁇ ) is handling the cases on the edges of the frame where the new sampling point is extrapolated from the two last points of the frame or two first points of the frame.
• a displacement is desirable, here denoted ⁇ .
• ⁇ may be defined according to 0.3
• the spectrum reverser 530 performs a reversal of the spectrum of the intermediate frame. This may be implemented by changing the sign of every second sample. 0,1, ... , ⁇ ⁇ ⁇ 1 [0042] Alternatively, a spectrum reversal equivalent for this purpose can be obtained with [0043] The result of the spectrum reversal is illustrated in Figure 7E with its reversed spectrum in Figure 7F.
• the interpolator 540 adjusts the frame length to match the sampling frequency of step 650, where a low-pass filter and decimator 550 performs a low-pass filter and decimation to produce the BWE target signal.
• the decimator performs a decimation by 2, which gives an output sampling frequency of half the input sampling frequency.
• the low-pass filter has a cut-off frequency in the middle of the spectrum.
• the decimator frame ⁇ ⁇ ( ⁇ , ⁇ ) is illustrated in Figure 7G.
• the target band is now aligned in the lower half of the spectrum. This is the frequency band that the low-pass filter and decimator 550 then extracts to give the extracted BWE target signal. Note that the spectrum of the BWE target signal is now reversed.
• the down-mix encoder outputs the encoded representation from the down-mix encoding mode into a down-mix bitstream. This bitstream is then joined with the bitstream of the parameter encoder 218. Optionally, a stereo residual bitstream may be produced by a residual encoder 216.
• the bitstream components of the active modules is joined into a compound bitstream to be stored or transmitted to a decoder.
• Figure 9 is a block diagram illustrating elements of the encoder 210 configured to encode audio frames according to the various embodiments herein.
• encoder 210 may include a network interface circuitry 905 (also referred to as a network interface) configured to provide communications with other devices/entities/functions/etc.
• the encoder 210 may also include processing circuitry 901 (also referred to as a processor and processor circuitry) coupled to the network interface circuitry 905, and a memory circuitry 903 (also referred to as memory) coupled to the processing circuit.
• the memory circuitry 903 may include computer readable program code that when executed by the processing circuitry 901 causes the processing circuit to perform operations according to embodiments disclosed herein.
• processing circuitry 901 may be defined to include memory so that a separate memory circuit is not required. As discussed herein, operations of the encoder 210 may be performed by processing circuitry 901 and/or network interface 905. For example, processing circuitry 901 may control network interface 905 to transmit communications to decoder 220 and/or to receive communications through network interface 905 from one or more other network nodes/entities/servers such as other encoder nodes, depository servers, etc. Moreover, modules may be stored in memory 903, and these modules may provide instructions so that when instructions of a module are executed by processing circuitry 901, processing circuitry 901 performs respective operations.
• processing circuitry 1001 may be defined to include memory so that a separate memory circuit is not required.
• operations of the decoder 220 may be performed by processor 1001 and/or network interface 1005.
• processing circuitry 1001 may control network interface circuitry 1005 to receive communications from encoder 210.
• modules may be stored in memory 1003, and these modules may provide instructions so that when instructions of a module are executed by processing circuitry 1001, processing circuitry 1001 performs respective operations.
• various operations of the encoder 210 and/or decoder 220 may be distributed across various components.
• Figure 11 is a block diagram illustrating a virtualization environment 1100 in which functions implemented by some embodiments may be virtualized.
• virtualizing means creating virtual versions of apparatuses or devices such as encoders and/or decoders which may include virtualizing hardware platforms, storage devices and networking resources.
• virtualization can be applied to any device described herein, or components thereof, and relates to an implementation in which at least a portion of the functionality is implemented as one or more virtual components.
• Some or all of the functions described herein may be implemented as virtual components executed by one or more virtual machines (VMs) implemented in one or more virtual environments 1100 hosted by one or more of hardware nodes, such as a hardware computing device that operates as an encoder, a decoder, a network node, UE, core network node, etc.
• VMs virtual machines
• Hardware 1204 includes processing circuitry, memory that stores software and/or instructions executable by hardware processing circuitry, and/or other hardware devices as described herein, such as a network interface, input/output interface, and so forth.
• Software may be executed by the processing circuitry to instantiate one or more virtualization layers 1106 (also referred to as hypervisors or virtual machine monitors (VMMs)), provide VMs 1108A and 1108B (one or more of which may be generally referred to as VMs 1108), and/or perform any of the functions, features and/or benefits described in relation with some embodiments described herein.
• the virtualization layer 1106 may present a virtual operating platform that appears like networking hardware to the VMs 1108.
• the VMs 1108 comprise virtual processing, virtual memory, virtual networking or interface and virtual storage, and may be run by a corresponding virtualization layer 1106.
• NFV network function virtualization
• NFV may be used to consolidate many network equipment types onto industry standard high volume server hardware, physical switches, and physical storage, which can be located in data centers, and customer premise equipment.
• a VM 1108 may be a software implementation of a physical machine that runs programs as if they were executing on a physical, non-virtualized machine.
• Each of the VMs 1108, and that part of hardware 1104 that executes that VM forms separate virtual network elements. Still in the context of NFV, a virtual network function is responsible for handling specific network functions that run in one or more VMs 1108 on top of the hardware 1104 and corresponds to the application 1102. [0055]
• Hardware 1104 may be implemented in a standalone network node with generic or specific components. Hardware 1104 may implement some functions via virtualization.
• hardware 1104 may be part of a larger cluster of hardware (e.g., such as in a data center or CPE) where many hardware nodes work together and are managed via management and orchestration 1110, which, among others, oversees lifecycle management of applications 1102.
• hardware 1104 is coupled to one or more radio units that each include one or more transmitters and one or more receivers that may be coupled to one or more antennas. Radio units may communicate directly with other hardware nodes via one or more appropriate network interfaces and may be used in combination with the virtual components to provide a virtual node with radio capabilities, such as a radio access node or a base station.
• some signaling can be provided with the use of a control system 1112 which may alternatively be used for communication between hardware nodes and radio units.
• a control system 1112 which may alternatively be used for communication between hardware nodes and radio units.
• modules may be stored in memory 903 of Figure 9, and these modules may provide instructions so that when the instructions of a module are executed by respective encoder processing circuitry 901, the encoder 210 performs respective operations of the flow chart.
• the encoder 210 determines an intermediate length of an intermediate frame.
• the encoder 210 receives the full band input frame by receiving a full band down-mix signal generated by a discrete Fourier transform, DFT, synthesis as described above. [0060] In block 1205, the encoder 210 interpolates the full band input frame to an intermediate frame. In some embodiments, the encoder 210 interpolates the full band input frame by interpolating the full band input frame using a linear interpolation.
• ⁇ is defined in accordance with 0.3 [0063] Other displacements may be used.
• the encoder 210 performs a reversal of the spectrum of the intermediate frame to produce a spectrally reversed intermediate frame.
• the encoder 210 selects a cut-off frequency of the low pass filtering to be substantially near a middle of the spectrum as illustrated in block 1301.
• the encoder 210 performs the low pass filtering and decimation of the reversed spectrum to produce the BWE target signal by performing the low pass filtering and decimation to align the BWE target signal in a lower half of the spectrum.
• the encoder 210 determines the decimator frame length in accordance with where ⁇ ⁇ is the decimator frame length, ⁇ ⁇ is an intermediate length, ⁇ ⁇ is a frequency of operation of a decimator, and ⁇ ⁇ is a frequency of the intermediate frame.
• the encoder 210 inputs the BWE target signal to a BWE encoder 314 via a BWE target buffer 316, where the BWE encoder 314 and the BWE target buffer 316 are part of encoder 210 operating on at least two frequency bands.
• the BWE encoder 314 and BWE target buffer 316 are part of a down-mix encoder 214.
• the down-mix encoder 214 of the encoder 210 in some embodiments has at least two encoding modes, wherein at least one of the at least two encoding modes has a BWE encoder 314.
• the encoder 210 operates on a down-mix signal in a parametric stereo encoder.
• Determining, calculating, obtaining or similar operations described herein may be performed by processing circuitry, which may process information by, for example, converting the obtained information into other information, comparing the obtained information or converted information to information stored in the network node, and/or performing one or more operations based on the obtained information or converted information, and as a result of said processing making a determination.
• processing circuitry may process information by, for example, converting the obtained information into other information, comparing the obtained information or converted information to information stored in the network node, and/or performing one or more operations based on the obtained information or converted information, and as a result of said processing making a determination.
• computing devices may comprise multiple different physical components that make up a single illustrated component, and functionality may be partitioned between separate components.
• a communication interface may be configured to include any of the components described herein, and/or the functionality of the components may be partitioned between the processing circuitry and the communication interface.
• non-computationally intensive functions of any of such components may be implemented in software or firmware and computationally intensive functions may be implemented in hardware.
• some or all of the functionality described herein may be provided by processing circuitry executing instructions stored on in memory, which in certain embodiments may be a computer program product in the form of a non-transitory computer- readable storage medium.
• some or all of the functionality may be provided by the processing circuitry without executing instructions stored on a separate or discrete device-readable storage medium, such as in a hard-wired manner.
• the processing circuitry can be configured to perform the described functionality.
• the benefits provided by such functionality are not limited to the processing circuitry alone or to other components of the computing device but are enjoyed by the computing device as a whole, and/or by end users and a wireless network generally.
• Embodiment 1 A method performed in an encoder (210, 1108A, 1108B), the method comprising: receiving (1203) a full band input frame; interpolating (620, 1205) the full band input frame to an intermediate frame; performing (630, 1207) a reversal of a spectrum of the intermediate frame to produce a spectrally reversed intermediate frame; interpolating (640,1209) the spectrally reversed intermediate frame to match a sampling frequency of a low pass filter and decimation process; and performing (650, 1211) a low pass filtering and decimation of the interpolated spectrally reversed intermediate frame to produce a band width extension, BWE, target signal.
• Embodiment 1 wherein interpolating the full band input frame comprises using a linear interpolation in interpolating the full band input frame.
• Embodiment 4 The method of Embodiment 3 wherein ⁇ is defined in accordance with 0.3 Embodiment 5.
• receiving the full band input frame comprises receiving a full band down-mix signal generated by a discrete Fourier transform, DFT, synthesis.
• the method of any of Embodiments 1-5, wherein performing the reversal of the spectrum of the intermediate frame comprises performing the reversal of the spectrum in accordance with: 0,1, ... , ⁇ ⁇ ⁇ 1 Embodiment 7.
• the method of any of Embodiments 1-5, wherein performing the reversal of the spectrum of the intermediate frame comprises performing the reversal of the spectrum in accordance with: Embodiment 8.
• Embodiment 9 The method of any of Embodiments 1-7, further comprising: determining (610, 1201) an intermediate length of the intermediate frame.
• Embodiment 9 The method of Embodiment 8 wherein determining the intermediate length comprises rescaling a length of an input frame ⁇ in the input sampling frequency ⁇ ⁇ until an upper limit of a target band matches a Nyquist frequency in the intermediate frame.
• Embodiment 21 The encoder (210, 1108A, 1108B) of Embodiment 20 wherein ⁇ is defined in accordance with 0.3 Embodiment 22.
• Embodiment 23 The encoder (210, 1108A, 1108B) of any of Embodiments 18-22, wherein performing the reversal of the spectrum of the intermediate frame comprises performing the reversal of the spectrum in accordance with: 0,1, ... , ⁇ ⁇ ⁇ 1 Embodiment 24.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• Computational Linguistics (AREA)
• Signal Processing (AREA)
• Health & Medical Sciences (AREA)
• Audiology, Speech & Language Pathology (AREA)
• Human Computer Interaction (AREA)
• Acoustics & Sound (AREA)
• Multimedia (AREA)
• Quality & Reliability (AREA)
• Mathematical Physics (AREA)
• Compression, Expansion, Code Conversion, And Decoders (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=87971811

Family Applications (1)

Country Status (4)

Family Cites Families (7)

• 2023
  • 2023-09-06 EP EP23765498.3A patent/EP4584781A1/de active Pending
  • 2023-09-06 WO PCT/EP2023/074394 patent/WO2024052378A1/en not_active Ceased
  • 2023-09-06 CN CN202380050495.6A patent/CN119452413A/zh active Pending
  • 2023-09-06 JP JP2025514258A patent/JP2025528549A/ja active Pending

Also Published As

Similar Documents

Legal Events