EP2145167B1 - Audiokodierungsverfahren sowie entsprechendes Audiokodierungsgerät, kodierten Signal und Computerprogramme dafür - Google Patents

Audiokodierungsverfahren sowie entsprechendes Audiokodierungsgerät, kodierten Signal und Computerprogramme dafür Download PDF

Info

Publication number
EP2145167B1
EP2145167B1 EP08788186A EP08788186A EP2145167B1 EP 2145167 B1 EP2145167 B1 EP 2145167B1 EP 08788186 A EP08788186 A EP 08788186A EP 08788186 A EP08788186 A EP 08788186A EP 2145167 B1 EP2145167 B1 EP 2145167B1
Authority
EP
European Patent Office
Prior art keywords
spectral
elements
coded
function
signals
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP08788186A
Other languages
English (en)
French (fr)
Other versions
EP2145167A2 (de
Inventor
Adil Mouhssine
Abdellatif Benjelloun Touimi
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Orange SA
Original Assignee
France Telecom SA
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by France Telecom SA filed Critical France Telecom SA
Publication of EP2145167A2 publication Critical patent/EP2145167A2/de
Application granted granted Critical
Publication of EP2145167B1 publication Critical patent/EP2145167B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
    • G10L19/16—Vocoder architecture
    • G10L19/18—Vocoders using multiple modes
    • G10L19/24—Variable rate codecs, e.g. for generating different qualities using a scalable representation such as hierarchical encoding or layered encoding
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing

Definitions

  • the present invention relates to audio signal coding devices, intended in particular to take place in applications for transmission or storage of digitized and compressed audio signals.
  • the invention relates more specifically to audio hierarchical coding systems, having the capacity to provide varied bit rates, by distributing the information relating to an audio signal to be coded in hierarchical subsets, so that they can be used in order of importance in terms of audio quality.
  • the criterion taken into account for determining the order is a criterion for optimizing (or rather reducing) the quality of the coded audio signal.
  • Hierarchical coding is particularly suited to transmission over heterogeneous networks or having variable available rates over time, or to transmission to terminals having different or variable characteristics.
  • the invention more particularly relates to the hierarchical coding of a 3D sound stage.
  • a 3D sound scene comprises a plurality of audio channels corresponding to monophonic audio signals and is still referred to as its spatialized sound.
  • a coded sound stage is intended to be reproduced on a sound rendering system, which may include a simple headset, two speakers of a computer or a home theater system 5.1 (Home Cinema) with five loudspeakers (a loudspeaker on the screen and on the front of the intended listener: a speaker on the left and a speaker on the right, on the back of the theoretical listener a speaker on the left and a speaker on the right), etc.
  • a sound rendering system may include a simple headset, two speakers of a computer or a home theater system 5.1 (Home Cinema) with five loudspeakers (a loudspeaker on the screen and on the front of the intended listener: a speaker on the left and a speaker on the right, on the back of the theoretical listener a speaker on the left and a speaker on the right), etc.
  • a technique used includes the determination of elements of description of the sound stage, then compression operations of each of the monophonic signals. The data resulting from these compressions and the description elements are then supplied to the decoder.
  • Scalability also known as scalability
  • Scalability is therefore achievable, by adapting the bit rate during compression operations, but it is performed according to criteria for optimizing the quality of each signal considered individually.
  • Another coding technique which is used in the "MPEG Audio Surround” encoder (see “Text of ISO / IEC FDIS 23003-1, MPEG Surround”, ISO / IEC JTC1 / SC29 / WG11 N8324, July 2006, Klagenfurt, Austria), includes the extraction and coding of spatial parameters from all monophonic audio signals on the different channels. These signals are then mixed to obtain a monophonic or stereophonic signal, which is then compressed by a conventional mono or stereo encoder (for example of the MPEG-4 AAC, HE-AAC type, etc.). At the level of the decoder, the synthesis of the 3D sound scene is made from the spatial parameters and the decoded mono or stereo signal.
  • a conventional mono or stereo encoder for example of the MPEG-4 AAC, HE-AAC type, etc.
  • Adaptability in flow with this other technique is thus achievable using a mono or hierarchical stereo encoder, but it is performed according to a criterion of optimizing the quality of the monophonic or stereophonic signal.
  • the PSMAC Progressive Syntax-Rich Multichannel Audio Codec
  • KLT Kerhunen Loeve Transform
  • the adaptability in flow is based on a cancellation of the least energy components. However, these components can sometimes be of great importance to overall audio quality.
  • none of the known 3D sound stage coding techniques makes it possible to adapt to the bit rate on the basis of a criterion for optimizing the spatial resolution, during the rendering of the 3D sound scene. This adaptability makes it possible to guarantee that each reduction in flow rate will degrade as little as possible the accuracy of the location of the sound sources in the space, as well as the size of the restitution area, which must be as wide as possible around the head of the room. 'auditor.
  • none of the known 3D sound stage coding techniques allows rate adaptability that directly guarantees optimal quality regardless of the sound rendering system used for the rendering of the 3D sound stage.
  • the current coding algorithms are defined to optimize the quality with respect to a particular configuration of the sound rendering system. Indeed, for example in the case of the encoder "MPEG Audio Surround” described above implemented with a hierarchical coding, a direct listening on headphones or two speakers, or monophonic is possible. If it is desired to use the compressed bit stream with a sound rendering system of 5.1 or 7.1 type, it is necessary to implement additional processing at the decoder, for example using OTT boxes (in English "One-To-One"). Two ”) to generate the five signals from the two decoded signals.
  • the present invention improves the situation.
  • the present invention aims at providing, according to a first aspect as defined in claim 1, a method for scheduling spectral components of elements to be coded from an audio scene comprising N signals with N> 1, a element to be encoded comprising spectral components relating to respective spectral bands.
  • a method according to the invention thus makes it possible to order in order of importance as to the overall audio quality of the element components to be coded.
  • a binary sequence is constituted after comparison between them of the different spectral components of the different elements to be encoded in the global scene compared with each other in terms of their contribution to the overall perceived audio quality.
  • the interaction between signals is thus taken into account to compress them together.
  • the bit stream can thus be ordered in such a way that each rate reduction degrades as little as possible the perceived overall audio quality of the 3D sound scene, since the least important elements in terms of their contribution to overall audio quality are detected, so that they can not be inserted (when the bit rate allocated for transmission is insufficient to transmit all the components of the elements to be coded) or end of binary sequence (to minimize the defects generated by a subsequent truncation).
  • Such a process thus makes it possible to determine at least one component of an element to be coded which is the least important in terms of the contribution to overall audio quality, compared to all the other components of elements to be coded to be ordered. .
  • steps a to g are repeated with a set of spectral components of elements to be coded to be restricted by deleting the spectral components for which an order of priority has been assigned.
  • steps a to g are repeated with a set of spectral components of elements to be coded to be ordered in which the spectral components for which an order of priority has been assigned are assigned a lower quantization rate when using a nested quantizer.
  • the elements to be encoded comprise the spectral parameters calculated for the N channels. These are then, for example, the spectral components of the signals that are coded directly.
  • the elements to be encoded comprise elements obtained by spatial transformation, for example of the ambisonic type, called in English "ambisonic", spectral parameters calculated for the N signals.
  • This arrangement makes it possible on the one hand to reduce the number of data to be transmitted since, in general, the N signals can be very satisfactorily described by a reduced number of surround components (for example, a number equal to 3 or 5 ), less than N.
  • This provision also allows adaptability to any type of sound rendering system, since it is sufficient at the decoder, to apply a reverse surround transform of size Q'x (2p '+ 1), (where Q 'is equal to the number of loudspeakers of the sound rendering system used at the output of the decoder and 2p' + 1 the number of received surround components), to determine the signals to be supplied to the sound rendering system, while preserving the overall audio quality.
  • the mask-to-noise ratios are determined as a function of the errors due to the coding and relating to elements to be encoded and in addition according to a spatial transformation matrix and a matrix determined according to the transpose of said spatial transformation matrix.
  • a method according to the invention thus makes it possible to order at least some spectral parameters of ambiophonic components of the set to be ordered, according to their relative importance in terms of contribution to spatial accuracy.
  • Spatial resolution or spatial precision measures the fineness of the location of sound sources in space. An increased spatial resolution allows a finer localization of the sound objects in the room and allows for a wider playback area around the listener's head.
  • the bit stream can thus be ordered in such a way that each rate reduction degrades as little as possible the perceived spatial accuracy of the 3D sound scene, since the least important elements in terms of their input are detected, in order to be at the end of the binary sequence (to minimize the defects generated by a subsequent truncation).
  • the angles ⁇ V and ⁇ E associated with the velocity vectors are exploited as indicated below.
  • V and energy E Gerzon's criteria for identifying elements to be coded that are least relevant in terms of contribution, in terms of spatial accuracy, to the 3D sound scene. Thus, contrary to usual usage, velocity vectors V and energy E are not used to optimize a sound rendering system considered.
  • This arrangement makes it possible, in a limited number of calculations, to determine the spectral parameter of the component to be determined whose contribution to the spatial accuracy is minimum.
  • steps a to g are repeated with a set of spectral parameters of components to be coded to be restricted by deleting the spectral parameters for which an order of priority has been assigned.
  • steps a to g are repeated with a set of spectral parameters of components to be coded to be ordered in which the spectral parameters for which an order of priority has been assigned are assigned a lower quantization rate. when using a nested quantizer.
  • a first coordinate of the energy vector is a function of the formula ⁇ 1 ⁇ i ⁇ Q ⁇ Ti 2 ⁇ cos ⁇ i ⁇ 1 ⁇ i ⁇ Q ⁇ Ti 2
  • a second coordinate of the energy vector is a function of the formula ⁇ 1 ⁇ i ⁇ Q ⁇ Ti 2 ⁇ sin ⁇ i ⁇ 1 ⁇ i ⁇ Q ⁇ Ti 2
  • a first coordinate of the velocity vector is a function of the formula ⁇ 1 ⁇ i ⁇ Q ⁇ Ti ⁇ cos ⁇ i ⁇ 1 ⁇ i ⁇ Q ⁇ Ti
  • a first coordinate of an angle vector indicates an angle depending on the sign of the second coordinate of the velocity vector and the arccosinus of the first coordinate of the velocity vector and that a second coordinate of the velocity vector an angle vector indicates an angle depending on the sign of the second coordinate of the energy vector and the arccosinus of the first coordinate of the energy vector.
  • the invention proposes a computer program to be installed in a scheduling module, said program comprising instructions for implementing the steps of a method according to the first aspect of the invention during a program execution by processing means of said module.
  • the invention proposes a signal comprising spectral components relating to respective spectral bands of elements to be coded coming from an audio scene comprising N signals with N> 1, in this respect. in that said spectral components are ordered for the implementation of the scheduling method according to the first aspect of the invention.
  • the figure 1 represents an audio coder 1 in one embodiment of the invention.
  • the encoder 1 comprises a time / frequency transformation module 3, a masking curve calculation module 7, a spatial transformation module 4, a module 5 for defining the least relevant coding elements combined with a quantization module 10, a module 6 for scheduling elements, a module 8 for constituting a binary sequence, for the transmission of a bit stream ⁇ .
  • a 3D sound scene comprises N channels on each respective signal S1, ..., SN is delivered.
  • the figure 2 represents an audio decoder 100 in one embodiment of the invention.
  • the decoder 100 comprises a bit sequence reading module 104, an inverse quantization module 105, a reverse surround conversion module 101, a frequency / time transformation module 102.
  • the decoder 100 is adapted to receive as input the bitstream ⁇ transmitted by the encoder 1 and to output Q 'signals S'1, S'2,..., S'Q' intended to feed the Q 'loudspeakers. respective speakers H1, H2 ..., HQ 'of a sound rendering system 103.
  • the time / frequency conversion module 3 of the encoder 1 receives as input the N signals S1,..., SN of the 3D sound scene to be encoded.
  • the time / frequency transformation module 3 On each time frame of each of these signals indicating the different values taken over time by the acoustic pressure Pi, the time / frequency transformation module 3 performs a time / frequency transformation, in this case a modified discrete cosine transform. (MDCT).
  • MDCT modified discrete cosine transform.
  • the definition elements of these masking curves are delivered to the module 5 for defining the least relevant coding elements.
  • the spatial transformation module 4 is adapted to perform a spatial transformation of the input signals provided, that is to say to determine the spatial components of these signals resulting from the projection on a spatial repository depending on the order of the transformation. .
  • the order of a spatial transformation is related to the angular frequency according to which it "scans" the sound field.
  • the spatial transformation module 4 performs an ambiophonic transformation, which gives a compact spatial representation of a 3D sound scene, by making projections of the sound field on the associated spherical or cylindrical harmonic functions.
  • (J m ) represent the functions of Bessel, r the distance between the center of the marker and the position of a listener placed at a point M, Pi the acoustic pressure of the signal Si, ⁇ i the angle of propagation of the wave acoustic signal corresponding to the signal Si and ⁇ the angle between the position of the listener and the axis of the marker.
  • This module 5 for defining the least relevant elements is adapted to implement operations, following the execution on the processing means of the module 5, of a first algorithm and / or a second algorithm, with a view to defining elements to code the least relevant and to order the elements to be coded between them.
  • This scheduling of the elements to be coded is used later when constituting a binary sequence to be transmitted.
  • the first algorithm comprises instructions adapted to implement, when executed on the processing means of the module 5, the steps of the Proc1 process described below with reference to FIG. figure 4 .
  • the principle of the Proc1 process is as follows: the respective influence of at least some computable spectral components is calculated as a function of spectral parameters derived from at least some of the N signals, on mask-to-noise ratios determined on the spectral bands in question. function of a coding of said spectral components. Then, an order of priority is assigned to at least one spectral component as a function of the calculated influence for said spectral component compared to the other calculated influences.
  • the detailed Proc1 process is as follows:
  • d k, j be the rate attributed to the element to be coded A (k, j), ( k, j ) ⁇ E 0 , during this allocation (the sum of these flows d k, j
  • the elements to be coded A (k, j), ( k , j ) ⁇ E 0 are quantized by the quantization module 10 as a function of the allocation defined for the bit rate D 0 .
  • the figure 8 represents a processing chain 200 comprising a p-order ambiophonic transformation module 201 (similar to the p-order ambiophonic transformation module 4 of the figure 1 ) followed by a module 202 of p-type inverse ambiophonic transformation.
  • the p-order ambiophonic transformation module 201 receives as input the spectral representations X1,..., XN of the signals S1,..., SN, performs on these signals a p-order ambiophonic transformation, delivers the ambiophonic signals obtained.
  • the figure 9 represents a processing chain 210 comprising the p-order ambiophonic transformation module 201 followed by a quantization module 203, then an inverse quantization module 204, and a p-order inverse ambiophonic transformation module 202.
  • the p-channel ambiophonic transformation module 201 at the input of the processing chain 210 receives as input the spectral representations X1,..., XN of the signals S1,..., SN and delivers the ambiophonic signals obtained A1 to AQ. which are provided at the input of the quantization module 203.
  • Signals AT 1, ..., AT Q are the signals delivered to the inverse ambiophonic transformation module 202 by the inverse quantization module 204, resulting from the inverse quantization performed on the signals delivered by the quantization module 203.
  • the processing chain 210 of the figure 9 provides the same acoustic pressures ⁇ 'i output that the processing chain 211 shown in figure 10 wherein the p-order ambiophonic transformation module 201 is located between the inverse quantization module 204 and the p-order inverse ambiophonic transformation module 202.
  • the quantization module 203 at the input of the processing chain 211 receives as input the spectral representations X1, ..., XN, quantizes them and then delivers the result of this quantization to the inverse quantization module 204, which delivers the N signals X 1, ..., X N.
  • These signals X 1, ..., X N are then provided to the cascaded surround back-up and surround-back transform modules 201 and 202.
  • the element to be encoded A (i 1 , j 1 ) is thus identified as the least relevant element as to the overall audio quality among the set of elements to be coded A (i, j) with ( i , j ) ⁇ E 0 .
  • the identifier of the pair (i 1 , j 1 ) is delivered to the scheduling module 6 as a result of the first iteration of the process Proc1.
  • Priority indices are then allocated by the scheduling module 6 to the different frequency bands, for the purpose of inserting the coding data into a binary sequence.
  • the scheduling module 6 on the sole basis of the results successively provided by the successive iterations of the process Proc1 implemented by the module 5 for defining the elements to be coded least relevant to the exclusion of the results provided by the Proc2 process, this last defines an order of said elements to be encoded, reflecting the importance of the elements to be coded in terms of overall audio quality.
  • the element to be encoded A (i 1 , j 1 ) corresponding to the pair (i 1 , j 1 ) determined during the first iteration of Proc1 is considered the least relevant in terms of overall audio quality. It is therefore assigned a minimum priority index Prio1 by the module 5.
  • the element to be encoded A (i 2 , j 2 ) corresponding to the pair (i 2 , j 2 ) determined during the second iteration of Proc1, is considered as the element to be coded the least relevant in terms of audio quality. overall, after the one assigned to priority Prio1. It is therefore assigned a minimum priority index Prio2, with Prio2> Prio1.
  • the scheduling module 6 thus successively orders r elements to be coded, each assigned to increasing priority indices Prio1, Prio2 to Prio r.
  • the elements to be coded that have not been assigned to an order of priority during an iteration of the process Proc1 are more important in terms of the overall audio quality than the elements to be coded to which priority orders have been assigned.
  • the priority order assigned to an element to be encoded A (k, j) is also assigned to the encoded element AT ( k , j ) resulting from a quantization of this element to be coded.
  • the binary sequence thus consists of spectral components relating to respective spectral bands, elements to be coded from of an audio scene comprising N signals with N> 1, and which are ordered according to their influence on determined mask-to-noise ratios on the spectral bands.
  • the spectral components of the binary sequence are, for example, ordered according to the method of the invention.
  • only some of the spectral components included in the constituted binary sequence are ordered using a method according to the invention.
  • a nested quantizer is used for the quantization operations.
  • the spectral component of an element to be encoded A (i 0 , j 0 ) identified is not suppressed, but a reduced bit rate is assigned to the coding of this component with respect to the coding of the other spectral components of elements to be coded to be ordered.
  • the encoder 1 is thus an encoder allowing a rate adaptability taking into account the interactions between the different monophonic signals. It allows to define compressed data optimizing the overall perceived audio quality.
  • the first iteration of the process 1 is assigned, for example, a minimum priority index (minimum of the elements remaining to be ordered) to the element to be encoded X (i 1 , j 1 ) such that the suppression of the spectral component X (i 1 , j 1 ) gives rise to a minimum variation of mask to noise ratio. Then we repeat the Proc1 process.
  • a minimum priority index minimum of the elements remaining to be ordered
  • Gerzon's criteria are generally used to characterize the location of virtual sound sources synthesized by the reproduction of signals from the loudspeakers of a given sound rendering system.
  • V ⁇ ⁇ x
  • V ⁇ 1 ⁇ i ⁇
  • the Ti cos ⁇ i ⁇ 1 ⁇ i ⁇
  • the Ti there V ⁇ 1 ⁇ i ⁇
  • the operations described below in one embodiment of the invention use the Gerzon vectors in an application other than that of seeking the best angles ⁇ i , characterizing the position of the speakers of the sound rendering system under consideration.
  • Gerzon's criteria are based on the study of the velocity and energy vectors of the acoustic pressures generated by a sound rendering system used.
  • the second algorithm comprises instructions adapted to implement, when executed on processing means of the module 5, the steps of the Proc2 process described below with reference to FIG. figure 6 .
  • the principle of the Proc2 process is as follows: the influence of each spectral parameter, among a set of spectral parameters to be ordered, on an angle vector defined as a function of energy and velocity vectors associated with Gerzon and calculated according to an inverse surround transformation on said quantized surround components. And an order of priority is assigned to at least one spectral parameter according to the calculated influence for said spectral parameter compared to the other calculated influences.
  • the detailed Proc2 process is as follows:
  • d k, j be the rate attributed to the element to be coded A (k, j), (k, j) e E o , during this initial allocation (the sum of these flows d k, j
  • each element to be coded A (k, j), (k, j) ⁇ E o is quantized by the quantization module 10 as a function of the flow rate d k, j which has been allocated to it in step 2a.
  • Each element AT ( k, j ) is the result of quantization, with the rate d k, j , of the parameter A (k, j), relative to the spectral band F j , of the ambiophonic component A (k).
  • the element AT ( k, j ) therefore defines the quantized value of the spectral representation for the frequency band F j , of the ambiophonic component Ak considered.
  • AT ⁇ AT ⁇ 1 0 AT ⁇ 1 1 1 . . . AT ⁇ ⁇ 1 , M - 1 AT ⁇ 2 0 AT ⁇ ⁇ 2 , M - 1 . . . AT ⁇ Q 0 AT ⁇ Q 1 . . . AT ⁇ ⁇ Q , M - 1 ,
  • each quantified element AT ( k , j ) is the sum of the spectral parameter A (k, j) of the ambiophonic component to be quantized and the quantization noise relative to said parameter).
  • an ambiophonic decoding matrix has been considered for a regular sound reproduction device and which comprises a number of loudspeakers equal to the number of input signals, which simplifies the calculation of the ambiophonic decoding matrix. Nevertheless, this step can be implemented by considering an ambiophonic decoding matrix corresponding to non-regular sound rendering devices and also for a number of speakers different from the number of input signals.
  • each element to be coded A (k, j), (k, j) e E 0 is quantized by the quantization module 10 as a function of the bit rate that has been allocated to it in step 2d.
  • AT is now the updated matrix of quantified elements AT (k, j), (k, j) e E 0 each resulting from the latter quantization according to the overall flow D 1 , parameters A (k, j).
  • step 2c After calculating a new p-order ambiophonic decoding performed as a function of the elements quantized with the overall bit rate D 1 , it is calculated, for the iteration n ° 1 of the process.
  • This standard represents the variation of the generalized Gerzon angle vector following the reduction of the rate of D 0 to D 1 in each frequency band F j .
  • This standard represents the variation of the generalized Gerzon angle vector in the frequency band F j 1 when for a rate D1, the frequency surround component A (i, j 1 ) is suppressed.
  • the component A (i 1 , j 1 ) is thus identified as the element to be coded of smaller importance in terms of spatial accuracy, compared to the other elements to be coded A (k, j), (k, j) e eo.
  • This redefined generalized Gerzon angle vector established for a quantization rate equal to D 1 , takes into account the deletion of the element to be encoded A (i 1 , j 1 ) and will be used for the next iteration of the process. Proc2.
  • the identifier of the pair (i 1, j 1) is supplied to the sequencing module 6 as a result of one iteration of the process Proc2 era.
  • the element to be encoded A (i 1 , j 1 ) is then removed from all the elements to be coded in the following process Proc2.
  • ⁇ 1 min d k, j , for (k, j) e E 1 .
  • the process Proc2 is repeated as many times as desired to order between them some or all of the elements to be coded A (k, j), (k, j) e E 1 remaining to be ordered.
  • E not - 1 E 0 ⁇ i 1 j 1 ... i not - 1 j not - 1 .
  • each element to be coded A (k, j), (k, j) ⁇ E n-1 is quantized by the quantization module 10 as a function of the bit rate allocated in step 2d above.
  • the result of this quantization of the element to be coded A (k, j) is AT (k, j), ( k , j ) ⁇ E n-1 .
  • This standard represents the variation of the vector of Gerzon angles generalized in each frequency band F j , following the reduction of flow rate from D n to D n-1 (the parameters A ( i 1 , j 1 ), ..., A ( i n -1 , j n -1 ) and AT ( i 1 , j 1 ), ..., AT (i n -1 , j n -1 ) being deleted).
  • This standard represents the variation, in the frequency band F j not , of the generalized Gerzon angle vector and for a rate D n , due to the removal of the ambiophonic component A (i, j n ) during the nth iteration of the Proc2 process.
  • the component A (i n , j n ) is thus identified as the element to be coded of smaller importance in terms of spatial accuracy, compared to the other elements to be coded A (k, j), (k, j) e E n-1 .
  • This redefined generalized Gerzon angle, established for a quantization rate equal to D n takes into account the deletion of the element to be encoded A (i n , j n ) and will be used for the next iteration.
  • the identifier of the pair (i n , j n ) is delivered to the scheduling module 6 as a result of the nth iteration of the process Proc2.
  • the band (i n , j n ) of the set of elements to be coded is then eliminated in the rest of the process Proc2, that is, the element to be coded A (i n , j n ).
  • the Proc2 process is repeated once and at most Q * M-1 times.
  • Priority indices are thus assigned by the scheduling module 6 to the different elements to be encoded, for the purpose of inserting the coding data into a binary sequence.
  • the scheduling module 6 defines an order of said elements to be encoded, reflecting the importance of the elements to be coded in terms of spatial accuracy.
  • the element to be encoded A (i 1 , j 1 ) corresponding to the pair (i 1 , j 1 ) determined during the first iteration of the process Proc2 is considered the least relevant in terms of spatial accuracy. It is therefore assigned a minimum priority index Prio1 by the module 5.
  • the element to be encoded A (i 2 , j 2 ) corresponding to the pair (i 2 , j 2 ) determined during the second iteration of the process Proc2, is considered as the element to be coded that is the least relevant in terms of precision. space, after the one assigned to priority Prio1. It is therefore assigned a minimum priority index Prio2, with Prio2> Prio1.
  • the scheduling module 6 thus successively orders r elements to be coded each assigned to increasing priority indices Prio1, Prio2 to Prio r.
  • the elements to be coded that have not been assigned to an order of priority during an iteration of the process Proc2 are more important in terms of spatial accuracy than the elements to be coded to which an order of priority has been assigned.
  • the priority order assigned to an element to be encoded A (k, j) is also assigned to the coded element according to the result AT (k, y) of the quantification of this element to code.
  • AT (k, j ) the coded element corresponding to the element to be coded A (k, j).
  • the binary sequence Seq is ordered in accordance with the scheduling performed by the module 6.
  • a nested quantizer is used for the quantization operations.
  • the spectral component of an element to be coded A (i, j) identified as the least important in terms of spatial precision during an iteration of the Proc2 process is not suppressed, but a reduced flow rate. is assigned to the coding of this component with respect to the coding of the other spectral components of elements to be coded to be ordered.
  • the encoder 1 is thus an encoder allowing a rate adaptability taking into account the interactions between the different monophonic signals. It allows to define compressed data optimizing perceived spatial precision.
  • the least important coding elements are defined using a Proc process combining the Proc1 and Proc2 processes described above, according to criteria taking into account the overall audio quality and the spatial relevance. .
  • the initialization of the Proc process includes the initializations of Proc1 and Proc2 processes as described above.
  • This rate and set of elements to be coded are determined during previous iterations of the Proc process on the basis of previous iterations of the Proc process using Proc1 and Proc2 processes.
  • the previous iterations made it possible to determine elements of coding determined the least important according to defined criteria.
  • step 300 selects in step 300, only one of the two elements to be coded identified or the two elements to be coded identified.
  • a (i n , j n ) that or each element to be coded.
  • the identifier or identifiers of the pair (i n , j n ) are provided to the scheduling module 6 as a result of the nth iteration of the process Proc2, which assigns it a priority Prion in view of the criteria defined.
  • the assigned Prion priority is greater than the priority of the elements to be coded during the previous iterations of the Proc process according to the defined criteria. This step replaces steps 1f of process Proc1 and 2k of process Proc2 as previously described.
  • the element or elements to be encoded A (i n , j n ) selected are then inserted in the binary sequence to be transmitted before the elements to be coded selected during the previous iterations of the Proc process (because the element to be coded A (i n , j n ) is more important in terms of the defined criteria than the elements to be coded previously selected by the process Proc).
  • the element or elements to be encoded A (i n , j n ) selected are inserted in the binary sequence to be transmitted after the other elements to be coded from the set E n-1 (because the element to be coded A (i n , j n ) is less important in terms of the defined criteria than those other elements to be coded).
  • This step 301 replaces steps 1g of process Proc1 and 2m of process Proc2 as described above.
  • the defined criteria make it possible to select the one or the least relevant elements identified respectively in step 300 of the Proc process.
  • the element identified by the process Proc1 is deleted at each iteration n, with n even and the element identified by the process Proc2 is deleted at each iteration n with n odd, which makes it possible to preserve the best overall audio quality and spatial accuracy.
  • An encoding implementing such a Proc process thus makes it possible to obtain an adaptable bitstream in terms of audio quality and in terms of spatial accuracy.
  • the decoder 100 comprises a bit sequence reading module 104, an inverse quantization module 105, a reverse surround conversion module 101 and a frequency / time transformation module 102.
  • the decoder 100 is adapted to receive as input the bitstream ⁇ transmitted by the encoder 1 and to output Q 'signals S'1, S'2,..., S'Q' intended to feed the Q 'loudspeakers. respective speakers H1, ..., HQ 'of a sound rendering system 103.
  • the number of speakers Q' may in one embodiment be different from the number Q of transmitted surround components.
  • FIG. figure 7 the configuration of a sound rendering system comprising 8 loudspeakers h1, h2, ..., h8 is represented in FIG. figure 7 .
  • the inverse quantization module 105 performs an inverse quantization operation.
  • At least some of the operations performed by the decoder are in one embodiment implemented following the execution on decoder processing means of computer program instructions.
  • An advantage of the coding of the components resulting from the ambiophonic transformation of the signals S1,..., SN as described is that in the case where the number of signals N of the sound scene is large, it is possible to represent them by a number Q of ambiophonic components much lower than N, degrading very little the spatial quality of the signals. The volume of data to be transmitted is reduced and this without significant degradation of the audio quality of the sound scene.
  • Another advantage of coding according to the invention is that such coding allows adaptability to the different types of sound rendering systems, regardless of the number, arrangement and type of loudspeakers including the sound rendering system. is equipped.
  • a decoder receiving a binary sequence comprising Q ambiophonic components operates on them an inverse surround transformation of any order p 'and corresponding to the number Q' of loudspeakers of the sound rendering system for which the signals are intended once. decoded.
  • Coding as performed by the coder 1 makes it possible to order the elements to be coded according to their respective contribution to the audio quality using the first process Proc1 and / or according to their respective contribution to the spatial accuracy and respect for the reproduction of the directions contained in the sound scene, using the second Proc2 process.
  • Proc1 and Proc2 can be implemented, according to the embodiments, in combination or else alone, independently of one another to define a binary sequence.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Computational Linguistics (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Mathematical Physics (AREA)
  • Quality & Reliability (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Stereophonic System (AREA)

Claims (12)

  1. Verfahren zur Ordnung von Spektralkomponenten zu codierender Elemente (A1, ..., ΔQ}, die von einer Audioszene kommen, welche N Signale (Sii=1 bis N) mit N>1 enthält, wobei ein zu codierendes Element Spektralkomponenten aufweist, die sich auf jeweilige Spektralbänder beziehen, dadurch gekennzeichnet, dass es die folgenden Schritte aufweist:
    - Berechnung des jeweiligen Einflusses mindestens bestimmter Spektralkomponenten, die abhängig von Spektralparametern berechnet werden können, welche von mindestens bestimmten der N Signale stammen, auf Maske-zu-Rauschen-Verhältnisse, die in den Spektralbändern abhängig von einer Codierung der Spektralkomponenten bestimmt werden;
    - Zuweisung einer Prioritätsreihenfolge zu mindestens einer Spektralkomponente abhängig von dem für die Spektralkomponente berechneten Einfluss verglichen mit den anderen berechneten Einflüssen,
    wobei der Einfluss einer gegebenen Komponente berechnet wird durch Schätzung einer Variation zwischen:
    - einem ersten Maske-zu-Rauschen-Verhältnis, das abhängig von einer Codierung der Spektralkomponenten gemäß einem ersten Durchsatz bestimmt wird, und
    - einem zweiten Maske-zu-Rauschen-Verhältnis, das abhängig von einer Codierung von Spektralkomponenten, von denen die gegebene Komponente entfernt wurde, gemäß einem zweiten Durchsatz geringer als der erste Durchsatz bestimmt wird.
  2. Verfahren nach Anspruch 1, dadurch gekennzeichnet, dass die Berechnung des Einflusses einer Spektralkomponente gemäß den folgenden Schritten ausgeführt wird:
    a- Codierung einer ersten Einheit von Spektralkomponenten zu codierender Elemente gemäß einem ersten Durchsatz;
    b- Bestimmung eines ersten Maske-zu-Rauschen-Verhältnisses pro Spektralband;
    c- Bestimmung eines zweiten Durchsatzes geringer als der erste;
    d- Entfernen der laufenden Spektralkomponente aus den zu codierenden Elementen und Codierung der verbleibenden Spektralkomponenten der zu codierenden Elemente gemäß dem zweiten Durchsatz;
    e- Bestimmung eines zweiten Maske-zu-Rauschen-Verhältnisses pro Spektralband;
    f- Berechnung einer Variation eines Maske-zu-Rauschen-Verhältnisses abhängig von den bestimmten Abständen zwischen dem ersten und dem zweiten Maske-zu-Rauschen-Verhältnis für den ersten und zweiten Durchsatz pro Spektralband;
    g- Wiederholung der Schritte d bis f für jede der Spektralkomponenten der Einheit von zu ordnenden Spektralkomponenten zu codierender Elemente und Bestimmung einer Mindestvariation des Maske-zu-Rauschen-Verhältnisses; wobei die der der Mindestvariation entsprechenden Spektralkomponente zugewiesene Prioritätsreihenfolge eine Mindest-Prioritätsreihenfolge ist.
  3. Verfahren nach Anspruch 2, dadurch gekennzeichnet, dass die Schritte a bis g mit einer Einheit von zu ordnenden Spektralkomponenten zu codierender Elemente wiederholt werden, die durch Entfernen der Spektralkomponenten eingeschränkt wird, für die eine Prioritätsreihenfolge zugewiesen wurde.
  4. Verfahren nach Anspruch 2, dadurch gekennzeichnet, dass die Schritte a bis g mit einer Einheit von zu ordnenden Spektralkomponenten zu codierender Elemente wiederholt werden, bei der den Spektralkomponenten, denen eine Prioritätsreihenfolge zugewiesen wurde, bei der Nutzung eines verschachtelten Quantifikators ein reduzierterer Quantisierungsdurchsatz zugeteilt wird.
  5. Verfahren nach einem der vorhergehenden Ansprüche, gemäß dem die zu codierenden Elemente die für die N Signale berechneten Spektralparameter enthalten.
  6. Verfahren nach einem der vorhergehenden Ansprüche, gemäß dem die zu codierenden Elemente Elemente (A1, ..., AQ) enthalten, die durch räumliche Transformation der für die N Signale berechneten Spektralparameter erhalten werden.
  7. Verfahren nach Anspruch 6, gemäß dem die räumliche Transformation eine Surround-Sound-Transformation ist.
  8. Verfahren nach Anspruch 6 oder 7, gemäß dem die Maske-zu-Rauschen-Verhältnisse abhängig von den Fehlern aufgrund der Codierung und bezüglich der zu codierenden Elemente, einer räumlichen Transformationsmatrix und einer Matrix bestimmt werden, die abhängig von der Transponierten der räumlichen Transformationsmatrix bestimmt wird.
  9. Verfahren nach einem der vorhergehenden Ansprüche und Anspruch 6, wobei bestimmte der Spektralkomponenten dann Spektralparameter von Surround-Sound-Komponenten sind, wobei das Verfahren außerdem die folgenden Schritte aufweist:
    a. Berechnung des jeweiligen Einflusses mindestens bestimmter der Spektralparameter auf einen Winkelvektor, der abhängig von Energie- und Geschwindigkeitsvektoren definiert wird, die Gerzon-Kriterien zugeordnet sind und abhängig von einer inversen Surround-Sound-Transformation an den quantisierten Surround-Sound-Komponenten berechnet werden;
    b. Zuweisung einer Prioritätsreihenfolge (Prio0) zu mindestens einem Spektralparameter abhängig vom für den Spektralparameter berechneten Einfluss verglichen mit den anderen berechneten Einflüssen.
  10. Audiocodierer (1), der geeignet ist, um eine 3D-Audioszene zu codieren, die N Signale und einen Ausgangs-Binärstrom enthält, mit N>1, der enthält:
    - ein Transformationsmodul (3, 4), das geeignet ist, um abhängig von den N Signalen Spektralkomponenten zu bestimmen, die sich auf jeweilige Spektralbänder beziehen;
    - ein Ordnungsmodul (6), das geeignet ist, um mindestens bestimmte der Spektralkomponenten, die sich auf bestimmte Spektralbänder beziehen, durch die Anwendung des Verfahrens nach einem der Ansprüche 1 bis 9 zu ordnen;
    - ein Modul (8) zur Bildung einer Binärfolge, das geeignet ist, um eine Binärfolge zu bilden, die Daten aufweist, welche Spektralkomponenten anzeigen, die sich auf jeweilige Spektralbänder beziehen, abhängig von der vom Ordnungsmodul ausgeführten Ordnung.
  11. Computerprogramm, das in einem Ordnungsmodul (6) eingerichtet werden soll, wobei das Programm Anweisungen enthält, um die Schritte eines Verfahrens nach einem der Ansprüche 1 bis 9 bei einer Ausführung des Programms durch Verarbeitungseinrichtungen des Moduls durchzuführen.
  12. Signal, das Spektralkomponenten aufweist, die sich auf jeweilige Spektralbänder zu codierender Elemente beziehen, die von einer Audioszene kommen, die N Signale enthält, mit N>1, dadurch gekennzeichnet, dass die Spektralkomponenten durch die Durchführung des Ordnungsverfahrens nach einem der Ansprüche 1 bis 9 geordnet werden.
EP08788186A 2007-05-10 2008-04-16 Audiokodierungsverfahren sowie entsprechendes Audiokodierungsgerät, kodierten Signal und Computerprogramme dafür Active EP2145167B1 (de)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
FR0703349A FR2916079A1 (fr) 2007-05-10 2007-05-10 Procede de codage et decodage audio, codeur audio, decodeur audio et programmes d'ordinateur associes
PCT/FR2008/050671 WO2008145893A2 (fr) 2007-05-10 2008-04-16 Procede de codage et decodage audio, codeur audio, decodeur audio et programmes d'ordinateur associes

Publications (2)

Publication Number Publication Date
EP2145167A2 EP2145167A2 (de) 2010-01-20
EP2145167B1 true EP2145167B1 (de) 2011-12-21

Family

ID=38858968

Family Applications (1)

Application Number Title Priority Date Filing Date
EP08788186A Active EP2145167B1 (de) 2007-05-10 2008-04-16 Audiokodierungsverfahren sowie entsprechendes Audiokodierungsgerät, kodierten Signal und Computerprogramme dafür

Country Status (6)

Country Link
US (1) US8488824B2 (de)
EP (1) EP2145167B1 (de)
CN (1) CN101730832B (de)
AT (1) ATE538369T1 (de)
FR (1) FR2916079A1 (de)
WO (1) WO2008145893A2 (de)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2688066A1 (de) 2012-07-16 2014-01-22 Thomson Licensing Verfahren und Vorrichtung zur Codierung von Mehrkanal-HOA-Audiosignalen zur Rauschreduzierung sowie Verfahren und Vorrichtung zur Decodierung von Mehrkanal-HOA-Audiosignalen zur Rauschreduzierung
EP2901667B1 (de) 2012-09-27 2018-06-27 Dolby Laboratories Licensing Corporation Räumliches multiplexing in einem telekonferenzsystem
US9685163B2 (en) * 2013-03-01 2017-06-20 Qualcomm Incorporated Transforming spherical harmonic coefficients
US10020000B2 (en) 2014-01-03 2018-07-10 Samsung Electronics Co., Ltd. Method and apparatus for improved ambisonic decoding
EP3090574B1 (de) * 2014-01-03 2019-06-26 Samsung Electronics Co., Ltd. Verfahren und vorrichtung für verbesserte ambisonic-decodierung
US10405126B2 (en) * 2017-06-30 2019-09-03 Qualcomm Incorporated Mixed-order ambisonics (MOA) audio data for computer-mediated reality systems
CN115715470B (zh) 2019-12-30 2025-11-18 卡姆希尔公司 用于提供空间化声场的方法
US11743670B2 (en) 2020-12-18 2023-08-29 Qualcomm Incorporated Correlation-based rendering with multiple distributed streams accounting for an occlusion for six degree of freedom applications
CN115497485B (zh) * 2021-06-18 2024-10-18 华为技术有限公司 三维音频信号编码方法、装置、编码器和系统

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
SE519985C2 (sv) * 2000-09-15 2003-05-06 Ericsson Telefon Ab L M Kodning och avkodning av signaler från flera kanaler
US7277765B1 (en) * 2000-10-12 2007-10-02 Bose Corporation Interactive sound reproducing
FR2820573B1 (fr) * 2001-02-02 2003-03-28 France Telecom Methode et dispositif de traitement d'une pluralite de flux binaires audio
CA2437927A1 (en) * 2003-08-14 2005-02-14 Ramesh Mantha Adaptive coding for a shared data communication channel
AU2004320207A1 (en) * 2004-05-25 2005-12-08 Huonlabs Pty Ltd Audio apparatus and method
WO2007098258A1 (en) * 2006-02-24 2007-08-30 Neural Audio Corporation Audio codec conditioning system and method
US8379868B2 (en) * 2006-05-17 2013-02-19 Creative Technology Ltd Spatial audio coding based on universal spatial cues
CN100458788C (zh) * 2006-09-25 2009-02-04 北京搜狗科技发展有限公司 一种互联网音频文件的聚类方法、搜索方法及系统

Also Published As

Publication number Publication date
EP2145167A2 (de) 2010-01-20
CN101730832B (zh) 2014-05-28
ATE538369T1 (de) 2012-01-15
WO2008145893A2 (fr) 2008-12-04
US8488824B2 (en) 2013-07-16
WO2008145893A3 (fr) 2009-12-03
US20100305952A1 (en) 2010-12-02
FR2916079A1 (fr) 2008-11-14
CN101730832A (zh) 2010-06-09

Similar Documents

Publication Publication Date Title
EP2143102B1 (de) Verfahren zur audiokodierung und -dekodierung, audiokodierer, audiodekodierer und zugehörige computerprogramme
EP2145167B1 (de) Audiokodierungsverfahren sowie entsprechendes Audiokodierungsgerät, kodierten Signal und Computerprogramme dafür
EP2374123B1 (de) Verbesserte codierung von mehrkanaligen digitalen audiosignalen
EP2374124B1 (de) Verwaltete codierung von mehrkanaligen digitalen audiosignalen
EP1600042B1 (de) Verfahren zum bearbeiten komprimierter audiodaten zur räumlichen wiedergabe
EP2168121B1 (de) Quantifizierung nach linearer umwandlung durch kombination von audiosignalen einer klangszene und kodiergerät dafür
EP2304721B1 (de) Raumsynthese mehrkanaliger tonsignale
WO2007104883A1 (fr) Dispositif et procede de codage gradue d'un signal audio multi-canal selon une analyse en composante principale
EP2005420A1 (de) Einrichtung und verfahren zur codierung durch hauptkomponentenanalyse eines mehrkanaligen audiosignals
JP2008517339A (ja) 空間音声パラメータの効率的符号化のためのエネルギー対応量子化
WO2020177981A1 (fr) Codage audio spatialisé avec interpolation et quantification de rotations
FR3049084A1 (de)
EP2198425A1 (de) Verfahren, modul und computerprogramm mit quantifizierung auf der basis von gerzon-vektoren
EP4042418B1 (de) Bestimmung von korrekturen zur anwendung auf ein mehrkanalaudiosignal, zugehörige codierung und decodierung
EP4172986B1 (de) Optimierte codierung einer für ein räumliches bild eines mehrkanaligen audiosignals repräsentativen information
EP4533449A1 (de) Titel räumliche audiokodierung mit konfiguration einer dekorrelationsverarbeitungsoperation

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20091022

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA MK RS

DAX Request for extension of the european patent (deleted)
17Q First examination report despatched

Effective date: 20110315

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

RTI1 Title (correction)

Free format text: AUDIO ENCODING METHOD, AUDIO ENCODER, CODED SIGNAL AND ASSOCIATED COMPUTER PROGRAM

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 538369

Country of ref document: AT

Kind code of ref document: T

Effective date: 20120115

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602008012219

Country of ref document: DE

Effective date: 20120308

REG Reference to a national code

Ref country code: NL

Ref legal event code: VDEP

Effective date: 20111221

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120321

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111221

LTIE Lt: invalidation of european patent or patent extension

Effective date: 20111221

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120322

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111221

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111221

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111221

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111221

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111221

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111221

REG Reference to a national code

Ref country code: IE

Ref legal event code: FD4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111221

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111221

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111221

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120321

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120421

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111221

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111221

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120423

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111221

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 538369

Country of ref document: AT

Kind code of ref document: T

Effective date: 20111221

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

BERE Be: lapsed

Owner name: FRANCE TELECOM

Effective date: 20120430

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111221

26N No opposition filed

Effective date: 20120924

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111221

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20120430

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602008012219

Country of ref document: DE

Effective date: 20120924

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20120430

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111221

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20120430

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20120430

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120401

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111221

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111221

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111221

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20120416

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080416

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 8

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 9

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 10

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 11

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20250319

Year of fee payment: 18

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20260320

Year of fee payment: 19

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20260319

Year of fee payment: 19