EP3762923B1 - Codage audio - Google Patents

Codage audio Download PDF

Info

Publication number
EP3762923B1
EP3762923B1 EP18723570.0A EP18723570A EP3762923B1 EP 3762923 B1 EP3762923 B1 EP 3762923B1 EP 18723570 A EP18723570 A EP 18723570A EP 3762923 B1 EP3762923 B1 EP 3762923B1
Authority
EP
European Patent Office
Prior art keywords
audio
doa
parameters
parameter
directional sound
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP18723570.0A
Other languages
German (de)
English (en)
Other versions
EP3762923A1 (fr
Inventor
Miikka Vilermo
Lasse Juhani Laaksonen
Mikko Tammi
Juha Tapio VILKAMO
Mikko-Ville Laitinen
Adriana Vasilache
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nokia Technologies Oy
Original Assignee
Nokia Technologies Oy
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nokia Technologies Oy filed Critical Nokia Technologies Oy
Publication of EP3762923A1 publication Critical patent/EP3762923A1/fr
Application granted granted Critical
Publication of EP3762923B1 publication Critical patent/EP3762923B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0272—Voice signal separating
    • G10L21/028—Voice signal separating using properties of sound source
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
    • G10L19/0204—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders using subband decomposition

Definitions

  • the example and embodiments of the present invention relate to processing of audio signals.
  • various embodiments of the present invention relate to aspects of encoding and/or decoding of audio signals that represent a spatial audio image, i.e. an audio scene that involves one or more directional sound components possibly together with an ambient sound component.
  • audio encoders and audio decoders are typically employed to encode and/or decode audio-based signals, such as music, ambient sounds or a combination thereof.
  • audio codecs typically do not assume an audio input of certain characteristics and, in particular, do not utilize a speech model for the encoding-decoding process but rather make use of encoding and decoding procedures that are suitable for representing all types of audio signals, including speech.
  • speech encoders and speech decoders can be considered as audio codecs that are optimized for speech signals via utilization of a speech production model in the encoding-decoding process. Relying on the speech production model enables, for speech signals, a lower bit rate at perceivable sound quality comparable to that achievable by an audio codec or an improved perceivable sound quality at a bit rate comparable to that of an audio codec.
  • speech codecs such signals typically represent background noise.
  • An audio codec or a speech codec may operate at either a fixed or variable bit rate.
  • a multi-channel audio signal may convey an audio scene that represents both directional sound components at specific positions of the audio scene as well as the ambience of the audio scene.
  • directional sound components represent distinct sound sources that have certain position within the audio scene (e.g. a certain direction of arrival and a certain relative intensity with respect to a listening point), whereas the ambience represents environmental sounds within the audio scene. Listening to such an audio scene enables the listener to experience the audio environment as if he or she was at the location the audio scene serves to represent.
  • the audio scene may also be referred to as a spatial audio image.
  • An audio scene may be stored into a predefined spatial format that enables rendering the audio scene for the listener via headphones and/or via a loudspeaker arrangement.
  • Non-limiting examples of applicable spatial audio formats include a multi-channel audio signal according to a predefined loudspeaker configuration (such as two-channel stereo, 5.1 surround sound, 7.1 surround sound, 22.2 surround sound, etc.), a multi-channel audio signal from a microphone array, an ambisonics audio signal, a binaural audio signal for headphone listening.
  • a predefined loudspeaker configuration such as two-channel stereo, 5.1 surround sound, 7.1 surround sound, 22.2 surround sound, etc.
  • An audio scene may be obtained by using a microphone arrangement that includes a plurality of microphones to capture a respective plurality of audio signals and processing the audio signals into a desired spatial audio format that represents the audio scene.
  • the audio scene may be created on basis of one or more arbitrary source signals by processing them into a desired spatial audio format that represents the audio scene of desired characteristics (e.g. with respect to directionality of sound sources and ambience of the audio scene).
  • a combination of a captured and artificially generated audio scene may be provided e.g. by complementing an audio scene captured by a plurality of microphones via introduction of one or more further sound sources at desired spatial positions of the audio scene.
  • Some recently developed audio codecs are able to encode a multi-channel input audio signal into an encoded audio signal that is accompanied by spatial information and to decode the encoded audio signal with the aid of the spatial information into reconstructed audio signal such that the spatial audio image represented by the input audio signal is re-created in the reconstructed audio signal.
  • a spatial audio codec which may include a spatial audio encoder and a spatial audio decoder.
  • a spatial audio encoder may provide the encoded audio signal on basis of a single-channel or multi-channel intermediate audio signal derived on basis of one or more channels of the input audio signal.
  • Such an intermediate audio signal has a smaller number of channels than the input audio signal (typically one or two channels) and it is commonly referred to as a downmix audio signal.
  • the spatial information may also be referred to, for example, as spatial data, as spatial metadata, as spatial parameters or as spatial attributes.
  • An example spatial audio codec is disclosed e.g. in the US patent application US2007/0269063 , M. Goodwin et al., "Spatial Audio Coding Based On Universal Spatial Cues", 22.11.2007 .
  • FIG. 1 illustrates a block diagram of some elements of a spatial audio encoder 100 according to an example.
  • the spatial audio encoder 100 includes a downmix entity 101 for creating a downmix signal 112 on basis of a multi-channel input audio signal 111 and an audio encoder 102 for processing the downmix signal 112 into an encoded audio signal 113.
  • the spatial audio encoder 100 further includes a metadata derivation entity 103 for deriving spatial metadata 114 on basis of the multi-channel input audio signal 111 and a metadata encoder 104 for processing the spatial metadata 114 into compressed spatial metadata 115.
  • the spatial metadata 113 derived for a single input frame may comprise a plurality of spatial parameters.
  • the spatial parameters for a given input frame may comprise one or more direction parameters that serve to indicate a perceivable direction of arrival of a sound represented by the respective input frame and one or more directionality parameters that serve to indicate a relative strength of a directional sound component in the respective input frame.
  • the spatial parameters included in the compressed spatial metadata 115 should be derived and encoded at a sufficient accuracy.
  • accurate representation of the compressed spatial metadata 115 may require an excessive bit-rate, which may not be feasible in scenarios where the bandwidth available for the audio bitstream 116 is limited.
  • the audio encoding entity 220 employs an audio encoding algorithm, referred herein to as an audio encoder, to process the input audio signal 215 into the audio bitstream 225.
  • the audio encoding entity 220 may further include a pre-processing entity for processing the input audio signal 215 from a format in which it is received from the audio capturing entity 210 into a format suited for the audio encoder.
  • This pre-processing may involve, for example, level control of the input audio signal 215 and/or modification of frequency characteristics of the input audio signal 215 (e.g. low-pass, high-pass or bandpass filtering).
  • the pre-processing may be provided as a pre-processing entity that is separate from the audio encoder, as a sub-entity of the audio encoder or as a processing entity whose functionality is shared between a separate pre-processing and the audio encoder.
  • the audio processing system 200 may include a storage means for storing pre-captured or pre-created audio signals, among which the audio input signal 215 for provision to the audio encoding entity 220 may be selected.
  • the audio encoding entity 220 may receive the input audio signal 215 from another entity via a communication channel (e.g. via a communication network) instead of receiving it from the audio capturing entity 210.
  • the audio processing system 200 may comprise a storage means for storing the reconstructed audio signal 235 provided by the audio decoding entity 230 for subsequent analysis, processing, playback and/or transmission to a further entity.
  • the audio decoding entity 230 may transmit the reconstructed audio signal 235 to a further entity via a communication channel (e.g. via a communication network) instead of providing it for playback by the audio reproduction entity 240.
  • the audio encoding entity 220 may further comprise a (first) network interface for encapsulating the audio bitstream 225 into a sequence of protocol data units (PDUs) for transfer to the decoding entity 230 over a network/channel
  • the audio decoding entity 230 may further comprise a (second) network interface for decapsulating the audio bitstream 225 from the sequence of PDUs received from the audio encoding entity 220 over the network/channel.
  • PDUs protocol data units
  • the multi-channel input audio signal 215 serves to represent an audio scene, e.g. one captured by the microphone assembly of the audio capturing entity 210.
  • the audio scene may also be referred to as a spatial audio image.
  • the audio scene conveyed by the multi-channel input audio signal 215 may represent both one or more directional sound components as well as the ambience of the audio scene, where a directional sound component represents a respective distinct sound source that has certain position within the audio scene whereas the ambience represents environmental sounds within the audio scene.
  • the spatial audio encoder is arranged to process the multi-channel input audio signal 215 into an encoded audio signal 305 and spatial metadata that are descriptive of the audio scene represented by the input audio signal 215.
  • the spatial audio encoder 300 may be arranged to process the multi-channel input audio signal 215 arranged into a sequence of input frames, each input frame including a respective segment of digital audio signal for each of the channels, provided as a respective time series of input samples at a predefined sampling frequency.
  • the spatial audio encoder 300 employs a fixed predefined frame length.
  • the frame length may be a selectable frame length that may be selected from a plurality of predefined frame lengths, or the frame length may be an adjustable frame length that may be selected from a predefined range of frame lengths.
  • a frame length may be defined as number samples L included in the frame for channel of the input audio signal 215, which at the predefined sampling frequency maps to a corresponding duration in time.
  • the spatial audio encoder 300 includes a downmixing entity 302 for creating a downmix signal 303 on basis of the multi-channel input audio signal 215.
  • the downmix signal 303 serves as an intermediate signal audio signal derived on basis of one or more channels of the input audio signal 215.
  • the downmix signal 303 typically has a smaller number of channels than the input audio signal 215, typically one or two channels.
  • the downmix signal 303 may have the same number of channels as in the input audio signal 215.
  • Various techniques for creating the downmix signal 303 are known in the art and a technique suitable for the intended usage of the spatial audio encoder 300 may be selected.
  • a channel of the downmix signal may be created, for example, as a linear combination (e.g. a sum, a difference, an average, etc.) of two or more channels of the input audio signal 215 or by selecting or processing one of the channels of the input audio signal 215 into a respective channel of the downmix signal 303.
  • the multi-channel input audio signal 215 may be processed by the downmixing entity 302 into one or more first signals that represent respective directional components of the audio scene conveyed by the input audio signal 215 and into a second signal that represents the ambient component of the audio scene, the first and second signals thereby constituting the respective channels of the downmix signal 302.
  • Operation of the audio encoder 304 typically results in a set of audio parameters that represent a frame of the audio signal, which set of audio parameters is provided as (a component of) the encoded audio signal 305 that enables reconstruction of a perceptually similar audio signal by an audio decoder.
  • the audio encoder 304 may process each channel of the downmix signal 303 separately into a respective set of audio parameters or it may process two or more channels of the downmix signal 303 jointly into a single set of audio parameters, depending on the characteristics of the downmix signal 303.
  • Various audio encoding techniques are known in the art, and a technique suitable for the intended usage of the spatial audio encoder 300 may be employed.
  • Non-limiting examples in this regard in include MPEG Advanced Audio Coding (AAC) encoder, Enhanced Voice Service (EVS) encoder, Adaptive Multi Rate (AMR) encoder, Adaptive Multi Rate Wide Band (AMR-WB) encoder, etc.
  • AAC MPEG Advanced Audio Coding
  • EVS Enhanced Voice Service
  • AMR Adaptive Multi Rate
  • AMR-WB Adaptive Multi Rate Wide Band
  • the spatial audio encoder 300 further includes an energy estimator 308' for estimating the overall signal energy of a reconstructed transform-domain downmix derived on basis of the encoded audio signal 305.
  • the energy estimator 308' may employ a (local) audio decoder to derive a local copy of a reconstructed downmix signal on basis of the encoded audio signal 305, which is further transformed into a transform-domain downmix signal and divided into a plurality of frequency sub-bands.
  • the energy estimator 308' further operates to derive, for a plurality of time-frequency tiles, a respective quantized energy (QEN) parameter 310' that indicates the overall signal energy (e.g. total signal energy) in a respective time-frequency tile of the transform-domain downmix audio signal.
  • QEN quantized energy
  • the spatial audio encoder 300 further includes a ratio encoder 312 for quantizing and encoding the ER parameters 311 and a direction encoder 314 for quantizing and encoding the DOA parameters 309.
  • the ratio encoder 312 operates to encode one or more ER parameters 311 derived by the spatial analysis entity 308 into respective one or more encoded ER parameters 315
  • the direction encoder 314 operates to encode zero or more DOA parameters derived by the spatial analysis entity 308 into respective one or more encoded DOA parameters 313.
  • the encoded ER parameter(s) 315 and possible encoded DOA parameter(s) 313 are provided as (part of) the spatial metadata for provision in the audio bitstream 225 to the spatial decoder. In the following, non-limiting examples of deriving the quantized and encoded ER parameters 315 and the DOA parameters 313 are described.
  • quantization and encoding of the DOA parameters is dependent on energy levels of one or more directional sound components represented by the input audio signal 215.
  • a DOA parameter for a given time-frequency tile may be quantized in dependence of the EN parameter 310 obtained for the given time-frequency tile, where applicable EN parameters 310 may be obtained from the spatial analysis entity 308.
  • a given DOA parameter for a given time-frequency tile may be quantized in dependence of a directional energy (DEN) parameter that indicates the absolute energy level for the directional sound source corresponding to the given DOA parameter in the given time-frequency tile.
  • DEN directional energy
  • the ER quantizer may comprise a variable bit-rate quantizer that assigns shorter codewords for those ER parameter values that occur more frequently and assigns longer codewords for those ER parameter values that occur less frequently.
  • the codewords and their lengths may be pre-assigned based on experimental data using techniques known in the art.
  • the quantization and encoding of an ER parameter 311 may rely, for example, on an ER quantization table that maps a plurality of table entries that each store a pair of a quantized ER parameter value and a codeword (e.g. a bit-pattern) assigned thereto. If using such an ER quantization table, the ratio encoder 312 operates to identify the table entry that holds quantized ER parameter value that is closest to the value of the ER parameter 311 under quantization/encoding and sets the value of the quantized ER parameter 316 and the value of the encoded ER parameter 315 to values found in the identified table entry.
  • an ER quantization table that maps a plurality of table entries that each store a pair of a quantized ER parameter value and a codeword (e.g. a bit-pattern) assigned thereto.
  • the ratio encoder 312 provides the encoded ER parameters 315 to a multiplexer 318 for inclusion in the audio bitstream 225 and provides the quantized ER parameters 316 for the direction encoder 314 to serve as control information in quantization and encoding of the DOA parameters 309 therein.
  • the direction encoder 314 operates to quantize the DOA parameters 309 in dependence of respective (absolute) energy levels of one or more sound components of the multi-channel transform-domain audio signal 307 in the corresponding time-frequency tile.
  • the direction encoder 314 operates to quantize one or more DOA parameters 309 using a suitable quantizer known in the art.
  • This quantizer employed by the direction encoder 314 may be referred to as a DOA quantizer, which may serve to encode the quantized value of a DOA parameter 309 using a fixed predefined number of bits or using a variable number of bits in dependence of the value of the DOA parameter.
  • the quantization and encoding of a DOA parameter 309 may rely, for example, on a DOA quantization table that maps a plurality of table entries that each store a pair of a quantized DOA parameter value and a codeword (e.g. a bit-pattern) assigned thereto. If using such a DOA quantization table, the direction encoder 314 operates to identify the table entry that holds quantized DOA parameter value that is closest to the value of the DOA parameter 309 under quantization/encoding and sets the value of the quantized DOA parameter and the value of the encoded DOA parameter 313 to respective values found in the identified table entry. The direction encoder 314 provides the encoded DOA parameters 313 to the multiplexer 318 for inclusion in the audio bitstream 225 therein.
  • a DOA quantization table that maps a plurality of table entries that each store a pair of a quantized DOA parameter value and a codeword (e.g. a bit-pattern) assigned thereto.
  • DOA parameters 309 for a given time-frequency tile pertain to a single sound source, in other words there is at most a single directional sound component in the given time-frequency tile.
  • there may be one or more DOA parameters 309 derived for the given time-frequency tile e.g. a DOA parameter that indicates an azimuth angle derived for the single direction sound component and/or a DOA parameter that indicates an elevation angle derived for the single directional sound component in the given time-frequency tile.
  • the direction encoder 314 operates to make a decision, for a plurality of time-frequency tiles considered in the spatial analysis, between including and omitting the respective encoded DOA parameter(s) 313 in/from the audio bitstream 225.
  • the decision is made in dependence of one or more criteria that pertain to the respective (absolute) energy levels of one or more sound components of the multi-channel transform-domain audio signal 307 in the respective time-frequency tile:
  • the direction encoder 314 may respond to a failure to meet the one or more criteria by using the DOA quantizer therein to quantize and encode predefined default value(s) for the DOA parameters instead of completely omitting the encoded DOA parameters 313 for the given time-frequency tile from the audio bitstream 225.
  • the default DOA parameters may serve to indicate, for example, zero azimuth and/or zero elevation (i.e. a sound source positioned directly in front of the assumed listening point).
  • the DOA quantizer employed for processing e.g.
  • the plurality of DOA quantizers provide different bit-rates, thereby providing a respective different tradeoff between accuracy of the quantization and the number of bits employed to define quantization codewords.
  • Each of the plurality of DOA quantizers operates to quantize a value of a DOA parameter 309 using a suitable quantizer known in the art using a fixed predefined number of bits or using a variable number of bits in dependence of the value of the DOA parameter.
  • each of the plurality of DOA quantizers may rely, for example, on a respective DOA quantization table that maps each of a plurality quantized DOA parameter values to a respective one of a plurality codewords assigned thereto.
  • the spatial audio encoder 400 includes a downmixing entity 402 for creating a transform-domain downmix signal 403 on basis of the multi-channel transform-domain audio signal 307.
  • the transform-domain downmix signal 403 serves as an intermediate signal audio signal derived on basis of the multi-channel transform-domain audio signal 307 such that it has a smaller number of channels than the multi-channel transform-domain audio signal 307, typically one or two channels.
  • the downmixing entity 402 operates on a transform-domain signal, while otherwise its operating principle is similar to that of the downmixing entity 302 of the spatial audio encoder 300.
  • the spatial audio encoder 400 further includes an energy estimator 308b for estimating the overall signal energy of the reconstructed transform-domain downmix signal 403'.
  • the energy estimator 308b may derive, for each time-frequency tile considered in the spatial analysis, a respective QEN parameter 410 that indicates estimated overall signal energy in a respective time-frequency tile.
  • the ratio decoder 362 operates in a manner similar to that described in context of the first example, whereas the operation of the direction decoder 364 is different. Also in the second example it is assumed that the encoded DOA parameter(s) 313 for a given time-frequency tile pertain to a single sound source and the direction decoder 364 operates to find the quantized DOA parameters 309' in dependence of (absolute) energy levels of one or more sound components of the audio signal represented by the audio bitstream 225 in the respective time-frequency tile.
  • the direction decoder 364 evaluates one or more criteria that pertain to the respective (absolute) energy levels of one or more sound components of the audio signal represented by the audio bitstream 225 in the respective time-frequency tile a in order to determine whether the first DOA quantizer or the second DOA quantizer is to applied for decoding the respective encoded DOA parameter 313:
  • the selected DOA quantizer is the one that uses the highest number of bits that does not exceed the predetermined number of bits available for encoding the ER parameter 311 and the DOA parameter(s) 309 for the given directional sound component.
  • the direction decoder 364 may detect the selected DOA quantizer based on knowledge of the predefined total number of bits available for encoding the ER and DOA parameters for the given directional sound component and the bits employed for representing the encoded ER parameter 316 via usage of the variable rate ER quantizer.
  • the directional energy-level dependent bit allocation for quantizing the DOA parameters 309 for two or more directional sound components of a given time-frequency tile is provided further in view of the third threshold: the predetermined fixed number of bits available for encoding the two or more DOA parameters 309 of the given time-frequency tile is allocated for the two or more encoded DOA parameters 313 of the given time-frequency tile in view of their respective relationships with the third threshold.
  • Components of the spatial audio encoder 300, 400 may be arranged to operate, for example, in accordance with a method 500 illustrated by a flowchart depicted in Figure 7A .
  • the method 500 serves as a method for encoding a multi-channel input audio signal that represents an audio scene as an encoded audio signal and spatial audio parameters, wherein the spatial audio parameters are descriptive of said audio scene.
  • Components of the spatial audio decoder 350, 450 may be arranged to operate, for example, in accordance with a method 550 illustrated by a flowchart depicted in Figure 7B .
  • the method 550 serves as a method for reconstructing a spatial audio signal that represents an audio scene on basis of an encoded audio signal and encoded spatial audio parameters that are descriptive of said audio scene.
  • the apparatus 600 comprises a processor 616 and a memory 615 for storing data and computer program code 617.
  • the memory 615 and a portion of the computer program code 617 stored therein may be further arranged to, with the processor 616, to implement at least some of the operations, procedures and/or functions described in the foregoing in context of the spatial audio encoder 300, 400 and/or in context of the spatial audio decoder 350, 450.
  • the apparatus 600 may further comprise user I/O (input/output) components 618 that may be arranged, possibly together with the processor 616 and a portion of the computer program code 617, to provide a user interface for receiving input from a user of the apparatus 600 and/or providing output to the user of the apparatus 600 to control at least some aspects of operation of the spatial audio encoder 300, 400 and/or spatial audio decoder 350, 450 that are implemented by the apparatus 600.
  • the user I/O components 618 may comprise hardware components such as a display, a touchscreen, a touchpad, a mouse, a keyboard, and/or an arrangement of one or more keys or buttons, etc.
  • the user I/O components 618 may be also referred to as peripherals.
  • processor 616 is depicted as a single component, it may be implemented as one or more separate processing components.
  • memory 615 is depicted as a single component, it may be implemented as one or more separate components, some or all of which may be integrated/removable and/or may provide permanent / semi-permanent! dynamic/cached storage.
  • the computer program code 617 stored in the memory 615 may comprise computer-executable instructions that control one or more aspects of operation of the apparatus 600 when loaded into the processor 616.
  • the computer-executable instructions may be provided as one or more sequences of one or more instructions.
  • the processor 616 is able to load and execute the computer program code 617 by reading the one or more sequences of one or more instructions included therein from the memory 615.
  • the one or more sequences of one or more instructions may be configured to, when executed by the processor 616, cause the apparatus 600 to carry out at least some of the operations, procedures and/or functions described in the foregoing in context of the spatial audio encoder 300, 400 and/or the spatial audio decoder 350, 450.
  • references(s) to a processor should not be understood to encompass only programmable processors, but also dedicated circuits such as field-programmable gate arrays (FPGA), application specific circuits (ASIC), signal processors, etc.
  • FPGA field-programmable gate arrays
  • ASIC application specific circuits
  • signal processors etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Quality & Reliability (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Claims (11)

  1. Appareil de codage d'un signal audio d'entrée multicanal qui représente une scène audio en signal audio codé et paramètres audio spatiaux, dans lequel les paramètres audio spatiaux décrivent ladite scène audio, l'appareil comprenant :
    un moyen pour coder une trame d'un signal de mixage réducteur en une trame du signal audio codé, dans lequel le signal de mixage réducteur est généré à partir du signal audio d'entrée multicanal ;
    un moyen pour dériver, à partir de la trame du signal audio d'entrée multicanal, une pluralité de paramètres audio spatiaux qui décrivent la scène audio, lesdits paramètres audio spatiaux comprenant une pluralité de paramètres de direction d'arrivée (DOA), dans lequel un paramètre DOA indique une position spatiale d'une composante sonore directionnelle donnée de la scène audio dans une sous-bande de fréquence donnée ; et
    un moyen pour coder les paramètres audio spatiaux, comprenant un moyen pour coder un paramètre DOA pour une composante sonore directionnelle donnée dans une sous-bande de fréquence donnée, dans lequel l'appareil est caractérisé en ce que le moyen pour coder un paramètre DOA pour une composante sonore directionnelle donnée dans une sous-bande de fréquence donnée dépend d'un niveau d'énergie absolu de la composante sonore directionnelle donnée dans la sous-bande de fréquence donnée dépassant un premier seuil, et dépend d'une énergie totale dudit signal audio d'entrée multicanal dans la sous-bande de fréquence donnée dépassant un deuxième seuil.
  2. Appareil selon la revendication 1,
    dans lequel lesdits paramètres audio spatiaux comprennent une pluralité de paramètres de rapport énergétique (ER), dans lequel un paramètre ER indique un niveau d'énergie relatif d'une composante sonore directionnelle donnée du scène audio dans un sous-bande de fréquence donnée, et
    dans lequel le moyen pour coder un paramètre DOA comprend :
    un moyen pour dériver une pluralité de paramètres d'énergie directionnelle (DEN), dans lequel un paramètre DEN indiquant ledit niveau d'énergie absolu d'une composante sonore directionnelle donnée dans une sous-bande de fréquence donnée est dérivé sur la base d'une énergie totale dudit signal audio d'entrée multicanal dans la sous-bande de fréquence donnée et du paramètre ER obtenu pour la composante sonore directionnelle donnée dans la sous-bande de fréquence donnée, et
    un moyen pour coder ledit paramètre DOA en fonction dudit paramètre DEN dérivé pour la composante sonore directionnelle donnée dans la sous-bande de fréquence donnée dépassant ledit premier seuil.
  3. Appareil selon la revendication 2, comprenant
    un moyen pour calculer l'énergie totale dudit signal audio d'entrée multicanal pour la sous-bande de fréquence donnée sur la base d'une trame du signal audio reconstruit dérivé en décodant la trame du signal audio codé.
  4. Appareil selon la revendication 2 ou 3, dans lequel
    l'appareil comprend un moyen pour coder ladite pluralité de paramètres ER, agencé pour dériver une pluralité respective de paramètres ER quantifiés ; et
    le moyen pour dériver la pluralité de paramètres DEN est agencé pour dériver un paramètre DEN qui indique le niveau d'énergie absolu de la composante sonore directionnelle donnée dans la sous-bande de fréquence donnée en fonction du paramètre ER quantifié obtenu pour la composante sonore directionnelle donnée pour la sous-bande de fréquence donnée.
  5. Appareil selon l'une des revendications 2 à 4, dans lequel un paramètre ER indique un rapport entre le niveau d'énergie d'une composante sonore directionnelle donnée dans une sous-bande de fréquence donnée et l'énergie totale dudit signal audio d'entrée multicanal dans la sous-bande de fréquence donnée.
  6. Appareil selon l'une des revendications 1 à 5, dans lequel au moins l'un des premier et deuxième seuils est un seuil prédéfini respectif attribué à la sous-bande de fréquence donnée.
  7. Appareil permettant de reconstruire un signal audio spatial qui représente une scène audio sur la base d'un signal audio codé et de paramètres audio spatiaux codés, dans lequel les paramètres audio spatiaux décrivent ladite scène audio, l'appareil comprenant :
    un moyen pour décoder une trame du signal audio codé en une trame d'un signal de mixage réducteur reconstruit ;
    un moyen pour recevoir une pluralité de paramètres audio spatiaux codés qui décrivent ladite scène audio dans une trame du signal audio spatial, lesdits paramètres audio spatiaux codés comprenant une pluralité de paramètres de direction d'arrivée, DOA, dans lequel un paramètre DOA indique une position spatiale d'une composante sonore directionnelle donnée de la scène audio dans une sous-bande de fréquence donnée ; et
    un moyen pour décoder lesdits paramètres audio spatiaux codés, comprenant un moyen pour décoder un paramètre DOA pour une composante sonore directionnelle donnée dans une sous-bande de fréquence donnée, dans lequel l'appareil est caractérisé en ce que le moyen pour décoder un paramètre DOA pour une composante sonore directionnelle donnée dans une sous-bande de fréquence donnée dépend d'un niveau d'énergie absolu de la composante sonore directionnelle donnée dans la sous-bande de fréquence donnée dépassant un premier seuil, et dépend d'une énergie totale dudit signal audio d'entrée multicanal dans la sous-bande de fréquence donnée dépassant un deuxième seuil.
  8. Appareil selon la revendication 7,
    dans lequel lesdits paramètres audio spatiaux comprennent une pluralité de paramètres de rapport énergétique, ER, dans lequel un paramètre ER indique un niveau d'énergie relatif d'une composante sonore directionnelle donnée du scène audio dans un sous-bande de fréquence donnée,
    dans lequel le moyen pour décoder un paramètre DOA comprend un moyen pour dériver une pluralité de paramètres d'énergie directionnelle, DEN, dans lequel un paramètre DEN indiquant ledit niveau d'énergie absolu d'une composante sonore directionnelle donnée dans une sous-bande de fréquence donnée est dérivé sur la base d'une énergie totale de la trame du signal audio spatial dans la sous-bande de fréquence donnée et du paramètre ER reçu pour la composante sonore directionnelle donnée dans la sous-bande de fréquence donnée, et
    un moyen pour décoder ledit paramètre DOA en fonction dudit paramètre DEN dérivé pour la composante sonore directionnelle donnée dans la sous-bande de fréquence donnée dépassant ledit premier seuil.
  9. Appareil selon la revendication 8, comprenant en outre un moyen pour calculer l'énergie totale de ladite trame du signal audio spatial pour la sous-bande de fréquence donnée sur la base de ladite trame du signal de mixage réducteur reconstruit.
  10. Appareil selon la revendication 8 ou 9, dans lequel un paramètre ER indique un rapport entre le niveau d'énergie d'une composante sonore directionnelle donnée dans une sous-bande de fréquence donnée et l'énergie totale dans la sous-bande de fréquence donnée.
  11. Appareil selon l'une des revendications 7 à 10, dans lequel au moins l'un des premier et deuxième seuils est un seuil prédéfini respectif attribué à la sous-bande de fréquence donnée.
EP18723570.0A 2018-03-08 2018-03-08 Codage audio Active EP3762923B1 (fr)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/FI2018/050171 WO2019170955A1 (fr) 2018-03-08 2018-03-08 Codage audio

Publications (2)

Publication Number Publication Date
EP3762923A1 EP3762923A1 (fr) 2021-01-13
EP3762923B1 true EP3762923B1 (fr) 2024-07-10

Family

ID=62143216

Family Applications (1)

Application Number Title Priority Date Filing Date
EP18723570.0A Active EP3762923B1 (fr) 2018-03-08 2018-03-08 Codage audio

Country Status (2)

Country Link
EP (1) EP3762923B1 (fr)
WO (1) WO2019170955A1 (fr)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11062716B2 (en) 2017-12-28 2021-07-13 Nokia Technologies Oy Determination of spatial audio parameter encoding and associated decoding
JP7262593B2 (ja) * 2019-01-13 2023-04-21 華為技術有限公司 ハイレゾリューションオーディオ符号化
GB2582916A (en) * 2019-04-05 2020-10-14 Nokia Technologies Oy Spatial audio representation and associated rendering
GB2587196A (en) * 2019-09-13 2021-03-24 Nokia Technologies Oy Determination of spatial audio parameter encoding and associated decoding
WO2021053266A2 (fr) * 2019-09-17 2021-03-25 Nokia Technologies Oy Codage de paramètres audio spatiaux et décodage associé
GB2592896A (en) 2020-01-13 2021-09-15 Nokia Technologies Oy Spatial audio parameter encoding and associated decoding
GB2598773A (en) * 2020-09-14 2022-03-16 Nokia Technologies Oy Quantizing spatial audio parameters
US20240185869A1 (en) * 2021-03-22 2024-06-06 Nokia Technologies Oy Combining spatial audio streams
CN115273870B (zh) * 2022-06-24 2025-04-25 安克创新科技股份有限公司 音频处理方法、装置、介质及电子设备

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8379868B2 (en) * 2006-05-17 2013-02-19 Creative Technology Ltd Spatial audio coding based on universal spatial cues
EP2249334A1 (fr) * 2009-05-08 2010-11-10 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Transcodeur de format audio
WO2011104463A1 (fr) * 2010-02-26 2011-09-01 France Telecom Compression de flux audio multicanal
US9313599B2 (en) * 2010-11-19 2016-04-12 Nokia Technologies Oy Apparatus and method for multi-channel signal playback

Also Published As

Publication number Publication date
EP3762923A1 (fr) 2021-01-13
WO2019170955A1 (fr) 2019-09-12

Similar Documents

Publication Publication Date Title
EP3762923B1 (fr) Codage audio
JP7091411B2 (ja) マルチチャネル信号の符号化方法およびエンコーダ
CN111656442B (zh) 使用量化和熵编码来编码或解码定向音频编码参数的装置和方法
JP6641018B2 (ja) チャネル間時間差を推定する装置及び方法
US11594231B2 (en) Apparatus, method or computer program for estimating an inter-channel time difference
CN113678199B (zh) 空间音频参数的重要性的确定及相关联的编码
CN114846541B (zh) 空间音频参数的合并
US9479886B2 (en) Scalable downmix design with feedback for object-based surround codec
CN114846542B (zh) 空间音频参数的组合
RU2439718C1 (ru) Способ и устройство для обработки звукового сигнала
JP2022548038A (ja) 空間オーディオパラメータ符号化および関連する復号化の決定
IL323655A (en) Methods and devices for encoding and/or decoding spatial background noise within a multichannel input signal
US11355131B2 (en) Time-domain stereo encoding and decoding method and related product
RU2648632C2 (ru) Классификатор многоканального звукового сигнала
GB2587614A (en) Audio encoding and audio decoding
CN116508332A (zh) 空间音频参数编码和相关解码
CN116508098A (zh) 量化空间音频参数
US20250191593A1 (en) Audio Codec Bitrate Using Directional Loudness
RU2836622C1 (ru) Способы и устройства для кодирования и/или декодирования пространственного фонового шума в многоканальном входном сигнале
RU2793703C2 (ru) Обработка аудиоданных на основе карты направленной громкости
JP2025041781A (ja) 空間音声パラメータの量子化
KR100891665B1 (ko) 믹스 신호의 처리 방법 및 장치
HK40038483B (en) Apparatus, method or computer program for estimating an inter-channel time difference
HK40038483A (en) Apparatus, method or computer program for estimating an inter-channel time difference

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: UNKNOWN

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20201008

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20220419

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20231115

GRAJ Information related to disapproval of communication of intention to grant by the applicant or resumption of examination proceedings by the epo deleted

Free format text: ORIGINAL CODE: EPIDOSDIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTC Intention to grant announced (deleted)
INTG Intention to grant announced

Effective date: 20240305

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602018071551

Country of ref document: DE

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG9D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20240710

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20241111

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1702769

Country of ref document: AT

Kind code of ref document: T

Effective date: 20240710

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240710

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20241111

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240710

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20241010

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240710

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240710

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20241011

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240710

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240710

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20241110

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240710

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240710

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240710

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20241010

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20241010

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240710

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20241010

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240710

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20241110

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240710

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20241011

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240710

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240710

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240710

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240710

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602018071551

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240710

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240710

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240710

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240710

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240710

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240710

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240710

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20250411

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240710

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240710

REG Reference to a national code

Ref country code: CH

Ref legal event code: H13

Free format text: ST27 STATUS EVENT CODE: U-0-0-H10-H13 (AS PROVIDED BY THE NATIONAL OFFICE)

Effective date: 20251023

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20250308

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20250331

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20250331

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20250331

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20250331

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20250308

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20260202

Year of fee payment: 9

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20260204

Year of fee payment: 9