Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04S—STEREOPHONIC SYSTEMS 
  • H04S3/00—Systems employing more than two channels, e.g. quadraphonic
  • H04S3/002—Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution
  • H04S3/004—For headphones
• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
  • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
  • G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04S—STEREOPHONIC SYSTEMS 
  • H04S3/00—Systems employing more than two channels, e.g. quadraphonic
  • H04S3/02—Systems employing more than two channels, e.g. quadraphonic of the matrix type, i.e. in which input signals are combined algebraically, e.g. after having been phase shifted with respect to each other
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04S—STEREOPHONIC SYSTEMS 
  • H04S1/00—Two-channel systems
  • H04S1/002—Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04S—STEREOPHONIC SYSTEMS 
  • H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
  • H04S2400/01—Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04S—STEREOPHONIC SYSTEMS 
  • H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
  • H04S2420/01—Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04S—STEREOPHONIC SYSTEMS 
  • H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
  • H04S2420/03—Application of parametric coding in stereophonic audio systems
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04S—STEREOPHONIC SYSTEMS 
  • H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
  • H04S2420/05—Application of the precedence or Haas effect, i.e. the effect of first wavefront, in order to improve sound-source localisation
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04S—STEREOPHONIC SYSTEMS 
  • H04S3/00—Systems employing more than two channels, e.g. quadraphonic
  • H04S3/002—Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution

Definitions

• the invention relates to the processing of sound data for spatialized reproduction.
• 3D rendering of compressed audio signals occurs especially during the decompression of a 3D audio signal, for example encoded in compression and represented on a number of channels, to a number of different channels (two for example to allow the rendering of 3D audio effects on a headset).
• binaural aims at the reproduction on a stereophonic headphones of a sound signal with nevertheless effects of spatialization.
• the invention is however not limited to the aforementioned technique and applies, in particular, to techniques derived from the "binaural", such as so-called technical rendering techniques TRANSAU RAL (registered trademark), that is to say on distant speakers.
• Such techniques can then use a "crosstalk cancellation” (or “cross-talk cancellation” in English), which consists in canceling the crossed acoustic paths, so that a sound, thus processed and then emitted by the loudspeakers, speakers, can be perceived only by one of the two ears of a listener.
• these two binaural and transaural rendering techniques are commonly referred to as binaural restitution.
• the invention relates to the transmission of multichannel audio signals and their conversion for a spatialized rendering (with 3D rendering) on two channels.
• the rendering device simple headset with earflaps for example
• the conversion can aim for example the case of a restitution of a sound scene initially in multichannel format 5.1 (or
• the invention also relates to the restitution, in the frame of a game or a video recording, for example, of one or more sound samples stored in files, with a view to their spatialization.
• binaural two-channel synthesis consists, with reference to FIG. 1 relating to the prior art, to:
• HRTF Head Related Transfer Functions
• N denotes the number of sound sources or incident audio streams to be spatialized
• the number of filters, or transfer functions, necessary for the binaural synthesis is 2xN for rendering in static binaural spatialization and 4xN for rendering in dynamic binaural spatialization (with transitions of transfer functions).
• the treatment described above with reference to FIG. 1 and involving the HRTF transfer functions is conventional. It is often used for 3D rendering from two speakers. It may be the basis of an implementation implemented by the present invention, as will be seen later. It is for this reason that it is introduced here.
• an encoder compresses the multichannel signal on only one or two channels (typically according to the bit rate offered on the network telecommunication) and also provides spatialization information. This embodiment is illustrated in FIG.
• FIG. 2A where, as an example for a signal in a multichannel 5.1 format, five channels (C for a center speaker, FL for a front left speaker, FR for a speaker right front, BL for a left rear speaker and BR for a right rear speaker) are encoded in compression by an ENCOD module capable of delivering two compressed L and R channels, as well as SPAT spatialization information.
• the compressed channels L and R, as well as the spatialization information SPAT are then conveyed through one or more telecommunication networks RES, on one or two channels according to the offered bit rate (FIG. 2B).
• a decoder reconstitutes the original signal in the initial multichannel format by means of the SPAT spatialization information delivered by the coder and, in the example of Figures 2A and 2C, there are still five channels, after decoding, feeding five speakers (HP-FL, HP-FR, HP-C, HP-BL and HP-BR) for a 5.1 format playback.
• DECOD decoder
• Audio encoders use time-frequency representations of signals to compress information. These representations are based on an analysis by filter banks or by time-frequency transformation of the MDCT type (for "Modified Discrete Cosine Transform"). In the case where a binaural spatialization must be performed after an audio decoding, the filtering operations are advantageously performed from the outset in the transformed domain. Recent work on filtering in the transformed subband domain has formalized the filtering architecture for a bank of filters commonly used in audio coders. One can usefully refer to the document: "A Generic Framework for Filtering in Subband Domain", A.Benjelloun Touimi, Proceeding IEEE - 9th Workshop on Digital Signal Processing, Hunt, Texas, USA, October 2000.
• a more recent filtering technique in the transformed domain of complex QMFs (for "Quadrature Mirror Filters") has been proposed in the "MPEG Surround” standard.
• This technique aims at converting the impulse response (finite) of the temporal filter noted h (v) into a set of M complex filters denoted h m (i), where M is the number of sub-frequency bands.
• the conversion is performed by analyzing the time filter h (v) by a complex filter bank similar to the QMF filterbank used for signal analysis.
• the prototype filter q (v) used to generate the conversion filter bank may be of length 192.
• the transcoding implemented by the DECOD BIN module of FIG. 3 must consider ten transfer functions: a for path A between the left front speaker HP-FL and the left ear OL of the AU listener, one for the path B between the front left speaker HP-FL and the right ear OR of the AU listener, - one for path C between the left rear loudspeaker HP-BL and the left ear OL of the listener AU,
• the binaural spatialization can then be advantageously achieved by applying these binaural filters directly to the transformed domain at the heart of the decoder.
• DECOD BIN audio as shown in Figure 3.
• this type of DECOD BIN decoder uses a monophonic or stereophonic representation (compressed L, R channels) of the multichannel audio scene, representation to which are associated SPAT spatialization parameters (which may consist, for example, of energy differences between channels and correlation indices between channels). These SPAT parameters are used at decoding to reproduce the original multichannel sound scene.
• the decoding can use representations decorrelated from these L, R signals (which are obtained, for example, by the application of all-pass decorrelation filters or reverb filters). These signals are then adjusted in energy thanks to interchannel energy differences, then recombined to obtain the multichannel signal for restitution.
• the parametric encoder (ENCOD - FIG. 2A) of the multichannel compressed two-way format (stereo or mono) according to the draft standard "MPEG Surround” delivers a decorrelation information between channels in the initial multichannel format and this information of decorrelation can be taken over by the homologous parametric decoder (DECOD - figure 2C) during the rendering in the initial multichannel format.
• h ⁇ _ L is the filter corresponding to the contributions of the left front and rear channels
• - gm is the gain associated with the set of left channels
• ⁇ 2 FL and ⁇ 2 BL are the useful energies of the channels respectively front and back left
• h ⁇ _, FL and IIL.BL are the transfer functions in the sub domain bands between the left ear and the speakers respectively front and rear left (paths A and C of FIG. 4)
• BL is the phase shift corresponding to the delay between the forward and backward time filters h L , FL and h L , BL-
• phase compensation a function of the target energy of the channels, is to avoid a so-called "coloring" effect resulting from the addition of two filters shifted in time (comb filtering).
• a decoder receives the SPAT spatialization parameters accompanying the compressed signals on two L and R channels in the example shown, and FIG. 5A illustrates how the aforementioned filter h L , ⁇ _s applies to the compressed channel L to form a component of the L-BIN signal, intended for binaural restitution. Nevertheless, as shown again in FIG. 5A, account should also be taken of the compressed signal on the channel R, which must, for its part, be filtered by a filter involving HRTF transfer functions (denoted h ⁇ _, FR ⁇ t ⁇ IL. BR) relating to the crossed paths H and E of FIG. 4, always towards the left ear.
• HRTF transfer functions decoder
• the filter corresponding to these crossed paths (denoted h ⁇ _, R) is calculated according to the gains, target energies and phase shifts, taken from the spatialization parameters SPAT, using an expression equivalent to the relation (1) given previously.
• This filter h L , R is finally applied to the compressed signal on the channel R. It is also necessary to take into account the
• an equivalent treatment is provided for the construction of the R-BIN signal intended for restitution.
• binaural for the right ear OD with three contributions given by: the compressed signal on the channel R filtered by the filter ⁇ I R, R representing the HRTF functions of the right speakers (direct paths G and F of FIG. 4) ; the compressed signal on the channel L filtered by the filter ⁇ I RL representing the HRTF functions of the left speakers (crossed paths B and D of FIG. 4); and
• FIG. 5B shows another example, in which a decoder receives the compressed signal on a single channel M, accompanying the spatialization parameters SPAT.
• the M channel is duplicated in two L and R channels and the further processing is strictly equivalent to the processing shown in FIG. 5A.
• the two signals L-BIN and R-BIN resulting from these filterings can then be applied to two loudspeakers intended respectively for the left ear and the right ear of the listener after a transition from the transformed domain to the time domain.
• a sound data processing method for a spatialized reproduction in three dimensions on two rendering channels for the respective ears of a listener the sound data being initially represented in a multichannel format and then encoded in compression on a a reduced number of channels (for example one or two channels), said initial multichannel format consisting of providing more than two channels capable of supplying respective loudspeakers, the method comprising the steps of: obtaining, with the data compressed on said reduced number, of channels, spatialization parameters, - for each playback channel associated with an ear of the listener, forming, from said spatialization parameters, a combination of filters representative each of transfer functions between this listener's ear and speakers that can be powered by respective channels of the multichannel init format ial, and - apply to the compressed data the filter combination associated with each rendition channel.
• the method according to the invention further comprises the steps: for each playback channel associated with an ear of the listener, determining from said spatialization parameters at least one transfer function of a loudspeaker located at the back of the listener's ear and representative of a decorrelation between the channels of the multichannel format respectively associated with the rear loudspeaker and at least one speaker located in front of the listener's ear, and for each rendering channel, integrating said transfer function representative of a decorrelation in said combination of filters associated with this rendering path. Spatial restitution on two paths, within the meaning of the invention, can be in binaural or transaural format as well.
• the initial multichannel format can be of ambiophonic type (or "ambisonic" in English and aiming at the decomposition of the sound signal in a spherical harmonics base). Alternatively, it can be a type of 5.1 or 7.1, or 10.2. It will be understood that for the latter types of format implementing channels for respectively supplying at least pairs of speakers front-left / left-back, on the one hand, and front-right / rear-right, d On the other hand, the decorrelation information can be directed to the respective channels of the front / rear speakers preferably associated with the same ear (left or right).
• this decorrelation information at the rear of a 3D scene is represented in the binaural or transaural restitution, a better representation of the ambiences is obtained, for example crowd noise or reverberation at the back of a scene, or otherwise, contrary to the achievements of the prior art.
• the combination of filters comprises a weighting, according to a chosen coefficient, between:
• the combination of filters associated with a restitution channel comprises at least one filter grouping from: the front speaker transfer function, the rear speaker transfer function, and the representative transfer function of a channel decorrelation, and these front speakers and back are located on the same side with respect to the listener.
• This may include, for example, the front and rear speakers located to the left (or both to the right) of the 5.1 format listener (as shown in Figure 4).
• the weighting between the uncorrelated version and the raw version of the transfer functions it may be advantageous to prefer the decorrelated version in the combination of filters of the left speakers for the playback channel on the left. right ear (and vice versa) and prefer the raw version (not decorrelated) in the right (left) speaker filter combination for the right ear (left) feedback channel.
• the encoding in compression implements a parametric encoder delivering, in the compressed stream including the spatialization parameters, cross-channel decorrelation information of the multichannel format, from which the aforementioned weighting can be determined dynamically.
• the aforementioned combination of transfer functions takes advantage of the information already present concerning the correlation between multichannel channel signals, this information simply being provided by the coder parametric, with the aforementioned spatialization parameters.
• FIG. 6A and 6B illustrate, by way of example, a filtering treatment of compressed data (on two channels in the example shown), the filtering being determined by the setting implementation of the method in the sense of the invention for delivering L-BIN and R-BIN signals for respectively feeding the left and right channels of a binaural restitution device such as a headset with two atria, taking into account the a front / rear decorrelation, and
• FIG. 7 schematically illustrates the structure of a module implementing the method in the sense of the invention.
• the compressed signal is first recovered on two L and R channels in the example represented, as well as the SPAT spatialization parameters that an encoder such as the ENCOD module of FIG. 2A previously described.
• transfer functions are determined to construct a combination of filters ("+" sign of FIG. 6A), each filter being to be applied to an L channel (filter L , L of FIG. 5A). ), or R (filter h L , R of FIG. 5A), or a combination of these channels (filter h L , c of FIG. 5A) to construct a signal supplying one of the two binaural reproduction channels L- BIN.
• transfer functions are representative of the disturbances experienced by an acoustic wave on a path between a loudspeaker that would have been fed by a channel of the initial multichannel format and an ear of the listener. For example, if the audio content is initially in 5.1 format, as described above with reference to FIG. 4, a total of ten HRTF transfer functions, five HRTF functions for the right ear (on paths B, D, G, F and I in Figure 4) and five HRTF functions for the left ear (on paths A, C, H, E and J).
• the HRTF functions of front and rear speakers, located on the same side of the listener are thus grouped to construct each filter of a filter combination specific to a playback channel on an ear of a listener.
• a grouping of HRTF functions to construct a filter is for example an addition, by means of multiplicative coefficients, an example of which will be described later.
• multiplicative coefficients an example of which will be described later.
• the initial sound data can be in multichannel format 5.1 and, with reference to FIG. 6A, a first grouping comprises: the function HRTF-A (for the front left speaker according to a direct path towards the left ear OL of Figure 4), - the HRTF-C function (for the left rear speaker in a direct path to the left ear),
• HRTF-C * the decorrelated version of this HRTF-C function
• a second grouping includes: - the HRTF-H function (for the right front speaker in a cross path to the left ear), the HRTF-E function (for the right rear speaker in a cross path),
• HRTF-E * the decorrelated version of this HRTF-E function
• Similar processing is provided to construct the signal for feeding the other binaural R-BIN rendering path of Figure 6B.
• HRTF functions of the paths leading to the right ear OD of the listener AU (FIG. 4).
• a first grouping includes the HRTF-G functions (for the right front speaker in a direct path), HRTF-F (for the right rear speaker in a direct path) and the decorrelated version HRTF-F * of the HRTF-F function to form the filter to be applied to the compressed channel R.
• a second grouping includes the function HRTF-B (for the front left speaker according to a cross path), the function HRTF-D (for the rear speaker left in a cross path) and the decorrelated version, denoted HRTF-D *, of the HRTF-D function, to form the filter to be applied to the compressed channel L.
• the filter combinations integrating the decorrelated versions of the HRTF functions of the rear loudspeakers are applied to the L and R compressed channels to deliver the L-BIN and R-BIN rendering channels, for spatial binaural rendering with 3D rendering.
• the received sound data are encoded in compression on two stereo channels L and R as illustrated in the example of FIG. 5A.
• they could be encoded in compression on a monophonic single channel M, as illustrated in FIG. 5B, in which case the filter combinations are applied to the monophonic channel (duplicated) as illustrated in FIG. 5B, to deliver two signals again. supplying respectively the two L-BIN, R-BIN restitution channels.
• the initial sound data is in the 5.1 multichannel format and is encoded in compression by a parametric encoder according to the aforementioned draft standard MPEG Surround. More particularly, during such encoding, it is possible to obtain, among the spatialization parameters provided, decorrelation information between the right rear channel and the right front channel (respective loudspeakers HP-BR and HP-FR of Figure 4), as well as homologous decorrelation information between the left rear channel and the left front channel (respective speakers HP-FR and HP-BR of Figure 4).
• This decorrelation information in a 5.1 format, is intended to make the reproduction of the rear speakers as independent as possible from the reproduction of the front speakers, to enrich, in 5.1 format, the enveloping effect by noise of reverb or audience for concert recordings for example. It is recalled that this enrichment of the 3D envelopment has not been proposed in binaural restitution and an advantage of the invention is to take advantage of the availability of the decorrelation information among the SPAT spatialization parameters to build uncorrelated versions of the functions. HRTF that integrate advantageously with filter combinations for binaural restitution.
• these filter combinations can be calculated directly in the transformed domain, for example in the field of sub-bands, and the filters representing the decorrelated versions of the HRTF functions of the rear loudspeakers can be obtained for example by applying the initial HRTF functions a phase shift function of the sub-frequency band considered.
• the decorrelation filters can be so-called "natural” reverb filters (recorded in a particular acoustic environment such as a concert hall for example), or "synthetic" (created by summation of multiple reflections of decreasing amplitudes in the time).
• the application of a decorrelated filter can therefore return to apply to the signal broken down into frequency subbands a different phase difference in each of the subbands, combined with the addition of a global delay.
• weighting is provided by different coefficients CM, (1-cti) and a 2 , (1-ct2) depending on whether the rear loudspeaker is on the same side as the ear in question. giving the filters U L, L and h RjR ) or not giving the filters U L, R and h Rj / _).
• the decorrelated version is preferred for crossed paths (rear right speaker for the left ear and left rear speaker for the right ear), so that in general the CM coefficient can often be greater than coefficient 02.
• the coefficients ⁇ are given by variable weighting functions so as to dynamically favor the raw version of the HRTF function of the rear loudspeaker or its uncorrelated version depending on whether the back signal is correlated or not with the front signal. This gives a better representation of the ambiences (crowd noise, reverb, or other) in the 3D rendering.
• the weighting function ⁇ can be defined dynamically by means of the decorrelation information provided with the spatialization parameters.
• ⁇ B ⁇ _ represents the target energy of the left rear channel when it is necessary to determine the coefficient ⁇ to calculate the filter
• an equivalent expression can be applied to calculate the weighting factor ⁇ involved in the homologous filter ⁇ R, R specific to direct acoustic paths to the right ear.
• the "sqrt" function no longer applies for the crossed paths and for the calculation of the corresponding coefficient ⁇ 2 , in the example described.
• the target energies and the correlation indices are terms between 0 and 1 so that the coefficient ⁇ 2 is generally lower than the coefficient cti.
• the global filter combination for the L-BIN channel, has many groupings of HRTF functions forming filters ⁇ L, L and ⁇ L, R obtained by the formulas given previously, and, in each grouping, intervene well.
• the present invention also relates to a DECOD decoding module
• BIN as represented by way of example in FIG. 7, for three-dimensional spatialized reproduction on two L-BIN and R-BIN reproduction channels, and in particular comprising means for processing sound data
• Compressed channels L possibly R in stereophonic mode and SPAT spatialization parameters for the implementation of the method described above.
• These means can typically comprise: an input E to receive the compressed channels and the spatialization parameters,
• a working memory MEM and a processor PROC for constructing the filter combinations from the parameters SPAT and applying these combinations respectively to the compressed channels L and R, and an output S for delivering the compressed and filtered signals for spatialized binaural reproduction on the respective L-BIN and R-BIN restitution routes.
• the present invention also aims at a computer program, intended to be stored in a memory of a decoding module, such as the memory MEM of the DECOD-BIN module of FIG. 7, for a three-dimensional spatialized restitution on two restoration channels. L-BIN and R-BIN.
• the program then comprises instructions for executing the method according to the invention and, in particular, to construct the filter combinations integrating the uncorrelated versions as illustrated in FIGS. 6A and 6B described above.
• one or the other of these figures can constitute a flowchart representing the algorithm underlying the program.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• Acoustics & Sound (AREA)
• Signal Processing (AREA)
• Mathematical Physics (AREA)
• Human Computer Interaction (AREA)
• Health & Medical Sciences (AREA)
• Audiology, Speech & Language Pathology (AREA)
• Computational Linguistics (AREA)
• Multimedia (AREA)
• Algebra (AREA)
• General Physics & Mathematics (AREA)
• Mathematical Analysis (AREA)
• Mathematical Optimization (AREA)
• Pure & Applied Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Stereophonic System (AREA)
• Golf Clubs (AREA)

Applications Claiming Priority (2)

Publications (2)

Family

ID=37684981

Family Applications (1)

Country Status (7)

Families Citing this family (16)

Family Cites Families (5)

• 2006
  • 2006-07-07 FR FR0606212A patent/FR2903562A1/fr not_active Withdrawn
• 2007
  • 2007-06-19 EP EP07803885A patent/EP2042001B1/de active Active
  • 2007-06-19 WO PCT/FR2007/051457 patent/WO2008003881A1/fr not_active Ceased
  • 2007-06-19 ES ES07803885T patent/ES2334856T3/es active Active
  • 2007-06-19 AT AT07803885T patent/ATE446652T1/de not_active IP Right Cessation
  • 2007-06-19 US US12/309,074 patent/US8880413B2/en active Active
  • 2007-06-19 DE DE602007002917T patent/DE602007002917D1/de active Active

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events