Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
  • H04N19/188—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a video data packet, e.g. a network abstraction layer [NAL] unit
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
  • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
  • H04N19/174—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a slice, e.g. a line of blocks or a group of blocks
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/46—Embedding additional information in the video signal during the compression process
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards

Definitions

• Video coding systems are widely used to compress digital video signals to reduce the storage need and/or transmission bandwidth of such signals.
• video coding systems such as block-based, wavelet-based, and object-based systems
• block-based hybrid video coding systems are the most widely used and deployed.
• a block-based video coding system may include a video coding layer (VCL) and a network abstraction layer (NAL).
• VCL video coding layer
• NAL network abstraction layer
• Video data encoded or otherwise processed at the VCL may be output to the NAL as an encoded video bit stream.
• multiple NAL units may be formed and strung together as a NAL unit stream; which stream may form one or more access units (e.g., video frames).
• the multiple NAL units may be various types of NAL units, including a NAL unit formed using the encoded video bit stream from the VCL.
• Such NAL unit may include a header and a payload.
• the NAL unit may be formed by populating the payload with one or more bits (e.g., a bit string) of the encoded video bit stream; populating the header, as appropriate; and combining the header with the payload.
• This type of NAL unit may be classified under a NAL unit type class for video data provided from the VCL.
• An example of the VCL and NCL are shown in Figure 1.
• the access units may be formed into packets for communication via a communication channel. As shown in Figure 1 , the access units may be populated in to various payloads of the packets, and appropriate lower layer, such as communication protocol layer (L2/L3/L4, etc.), headers may be combined with payload.
• L2/L3/L4, etc. communication protocol layer
• adaptation parameters may include, for example, one or more parameters for facilitating reconstruction of, and/or used for reconstructing, decoded video.
• the adaptation parameters may include one or more parameters for adapting filters for (or otherwise facilitating) in- loop filtering of decoded video data. Examples of such adaptation parameters may include sample adaptive offset (SAO) parameters, and adaptive loop filter (ALF) parameters.
• SAO sample adaptive offset
• ALF adaptive loop filter
• a method may include selectively conveying adaptation parameters in a video bit stream by any of (i) inserting the adaptation parameters in a slice header, and (ii) inserting the adaptation parameters in a dedicated NAL unit.
• the dedicated NAL unit may be, for example, an adaptation parameter set (APS) NAL unit type.
• a method may include selectively conveying adaptation parameters in a video bit stream by a signaling method selected from a set including (i) inserting the adaptation parameters in a slice header, and (ii) inserting the adaptation parameters in a dedicated NAL unit (e.g., an APS NAL unit).
• a signaling method selected from a set including (i) inserting the adaptation parameters in a slice header, and (ii) inserting the adaptation parameters in a dedicated NAL unit (e.g., an APS NAL unit).
• the method may include signaling the selection by a flag inserted to the slice header to indicate whether the adaptation parameters for a current slice are obtainable from the dedicated NAL unit, or included in a current slice header.
• the flag may be set to first and second values.
• the flag when set to the first value, may indicate that the adaptation parameters are retrievable or otherwise obtainable from the dedicated NAL unit.
• Such dedicated NAL unit may be an APS NAL unit associated with a given APS identifier (e.g., set forth in an aps id field) included in the bitstream.
• the flag when set to the second value, may indicate that the adaptation parameters are retrievable or otherwise obtainable from the current slice.
• the selection may be made with reference to a threshold value.
• the threshold value may be established based on one or more of a number of factors, including: an amount of frames between intra frames (I-frames); an amount of frames between a current frame and a preceding intra frame; video statistics; and/or error likelihoods associated with a transmission channel, for example.
• the error likelihoods may be obtained from a physical (PHY) layer, a media access control (MAC) layer, etc. of a wireless transmit and/or receive unit (WTRU) or a base station).
• the statistics may include color component analysis information.
• the color component analysis information may include color saturation and/or hue information. If the color satisfies (e.g., is above) a second threshold, then the selection may be a selection of inserting the adaptation parameters in slice header. If the color fails to satisfy (e.g., is below) the second threshold, then the selection may be a selection of inserting the adaptation parameters in a dedicated NAL unit. Other factors may alter the value of the second threshold.
• the selection may be based, at least in part, on a frequency of use as a reference picture to code subsequent pictures. In an embodiment, the selection may be based, at least in part, on values of QPs (such as, wherein slice header level signaling of the adaptation parameters is used more frequently for pictures having lower QP). In an embodiment, the selection may be based, at least in part, on long term reference pictures. Slice header level signaling of the adaptation parameters may be used for longer term reference pictures, for instance.
• a method of decoupling a region of interest (ROI) slice from other slices may include signaling the adaptation parameters in a slice header of the ROI slice ("ROI-slice header"), and using dedicated NAL units (e.g., APS NAL units) for the other slices.
• the adaptation parameters signaled in the ROI-slice header may be adaptation parameters for a first combination of one or more types of in-loop filters (e.g., SAO and ALF parameters).
• the adaptation parameters signaled in at least one of the dedicated NAL units may include adaptation parameters for a second combination of one or more types of in-loop filter (e.g., SAO parameters).
• Figure 1 is a block diagram illustration an example of a video coding layer (VCL) and a network abstraction layer (NAL) for a block-based video coding system;
• Figure 2A is a block diagram illustrating an example video encoding and decoding system in which one or more embodiments may be carried out and/or implemented;
• Figure 2B is a block diagram illustrating an example video encoder unit for use with a video encoding and/or decoding system, such as the system of Figure 2A;
• Figure 2D is a block diagram illustrating an example of a video encoder for use with a video encoder unit, such as the video encoder unit of Figures 2A-2B;
• Figure 2E is a block diagram illustrating an example of a video decoder for use with a video decoder unit, such as the video decoder unit of Figures 2A-2B;
• Figure 3 is a flow diagram illustrating a flow 300 for performing high-level layer signaling of adaptation parameters
• Figure 4 is a block diagram illustrating an example of an in-loop filtering unit for performing in-loop filtering during video coding
• FIG. 5 is a block diagram illustrating an example of a Network Abstraction Layer (NAL) unit stream in which adaptation parameters are only conveyed using a slice-header signaling method;
• NAL Network Abstraction Layer
• Figure 6 is a block diagram illustrating an example of a NAL unit stream in which adaptation parameters are only conveyed using a dedicated-NAL-unit signaling method
• Figure 7 is a block diagram illustrating an example of a NAL unit stream in which adaptation parameters are only conveyed using a dedicated-NAL-unit signaling method
• Figure 8 is a block diagram illustrating an example of a NAL unit stream including different adaptation parameter set (APS) NAL units sharing the same APS identifier;
• APS adaptation parameter set
• Figure 9 is a block diagram illustrating an example of a NAL unit stream in which APSs are conveyed using a dedicated-NAL-unit signaling method
• Figure 10 is a block diagram illustrating an example of a NAL unit stream in which adaptation parameters are selectively conveyed using a signaling method selected from a set including multiple APS signaling methods;
• Figures 1 1A-1 1B are flow diagrams illustrating examples of flows for selectively conveying adaptation parameters using a signaling method selected from a set including multiple APS signaling methods;
• Figure 12A is a system diagram of an example communications system in which one or more disclosed embodiments may be implemented;
• FIG 12B is a system diagram of an example wireless transmit/receive unit (WTRU) that may be used within the communications system illustrated in Figure 12A; and
• WTRU wireless transmit/receive unit
• FIGS. 12C, 12D, and 12E are system diagrams of example radio access networks and example core networks that may be used within the communications system illustrated in Figure 12A.
• FIG. 2A is a block diagram illustrating an example video encoding and decoding system 100 in which one or more embodiments may be carried out and/or implemented.
• the system 100 may include a source device 102 that transmits encoded video information to a destination device 104 via a communication channel 106.
• the source device 102 and destination device 104 may be any of a wide range of devices.
• the source device 102 and the destination device 104 may include wireless transmit and/or receive units (WTRUs), such as wireless handsets or any wireless devices that can communicate video information over the communication channel 106, in which case, communication channel 106 includes a wireless link.
• WTRUs wireless transmit and/or receive units
• the communication channel 106 may include and/or may be any combination of wireless or wired media suitable for transmission of encoded video data.
• the source device 102 may include a video encoder unit 108, a transmit and/or receive (Tx/Rx) unit 110 and a Tx/Rx element 1 12. As shown, the source device 102 may optionally include a video source 1 14.
• the destination device 104 may include a Tx/Rx element 1 16, a Tx/Rx unit 1 18 and a video decoder unit 120. As shown, the destination device 104 may optionally include a display device 122.
• Each of the Tx/Rx units 110, 1 18 may be or include a transmitter, a receiver or a combination of a transmitter and a receiver (e.g., a transceiver or transmitter-receiver).
• Each of the Tx/Rx elements 112, 1 16 may be, for example, an antenna.
• video encoder unit 108 of source device 102 and/or the video decoder unit 120 of the destination device may be configured and/or adapted (collectively "adapted") to apply the coding techniques provided herein.
• the source and destination devices 102, 104 may include other elements/components or arrangements.
• the source device 102 may be adapted to receive video data from an external video source.
• the destination device 104 may interface with an external display device (not shown), rather than including and/or using the (e.g., integrated) display device 122.
• a data stream generated by the video encoder unit 108 may be conveyed to other devices without the need for modulating the data onto a carrier signal, such as by direct digital transfer, wherein the other devices may or may not modulate the data for transmission.
• the illustrated system 100 of Figure 2 is merely one example. The techniques provided herein may be performed by any digital video encoding and/or decoding device.
• the techniques provided herein are performed by separate video encoding and/or video decoding devices, the techniques may also be performed by a combined video encoder/decoder, typically referred to as a "CODEC.” Moreover, the techniques provided herein may also be performed by a video preprocessor or the like.
• the source device 102 and the destination device 104 are merely examples of such coding devices in which the source device 102 generates (and/or receives video data and generates) the encoded video information for transmission to the destination device 14.
• the devices 102, 104 may operate in a substantially symmetrical manner such that each of the devices 102, 104 include both video encoding and decoding components and/or elements (collectively "elements").
• the system 100 may support any of one-way and two-way video transmission between the devices 102, 104, e.g., for any of video streaming, video playback, video broadcasting, video telephony and video conferencing.
• the source device 102 may be, for example, a video streaming server adapted to generate (and/or receive the video data and generate) the encoded video information for one or more destination devices, where the destination devices may be in communication with the source device 102 over wired and/or wireless communication systems.
• the external video source and/or the video source 114 may be and/or include a video capture device, such as a video camera, a video archive containing previously captured video, and/or a video feed from a video content provider.
• the external video source and/or the video source 1 14 may generate computer graphics-based data as the source video, or a combination of live video, archived video, and computer-generated video.
• the source device 102 and destination device 104 may be or embody camera phones or video phones.
• the techniques provided herein may be applicable to video coding in general, and may be applied to wireless and/or wired applications.
• the captured, pre-captured, computer-generated video, video feed, or other type of the video data may be encoded by the video encoder unit 108 so as to form the encoded video information.
• the Tx/Rx unit 1 10 may modulate the encoded video information, e.g., according to a communication standard, so as to form one or more modulated signals carrying the encoded video information.
• the Tx/Rx unit 1 10 may also pass the modulated signals to its transmitter for transmission.
• the transmitter may transmit the modulated signals via the Tx/Rx element 1 12 to the destination device 14.
• the Tx/Rx unit 1 18 may receive the modulated signals from over channel 106 via the Tx/Rx element 1 16.
• the Tx/Rx unit 118 may demodulate the modulated signals to obtain the encoded video information.
• the Tx/RX unit 1 18 may pass the encoded video information to the video decoder unit 120.
• the video decoder unit 120 may decode the encoded video information to obtain decoded video data.
• the encoded video information may include syntax information defined by the video encoder unit 108. This syntax information may include one or more elements ("syntax elements"); some or all of which may be useful for decoding the encoded video information.
• syntax elements may include, for example, characteristics of the encoded video information.
• the syntax elements may also include characteristics, and/or describe the processing, of the un-encoded video used to form the encoded video information.
• the video decoder unit 120 may output the decoded video data for later storage and/or display on the external display (not shown). Alternatively, the video decoder unit 120 may output the decoded video data to the display device 122.
• the display device 122 may be and/or include any individual, multiple, combination, combination of multiples of a variety of display devices adapted to display the decoded video data to a user. Examples of such display devices include a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, a cathode ray tube (CRT), etc.
• LCD liquid crystal display
• OLED organic light emitting diode
• CRT cathode ray tube
• the communication channel 106 may be any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines, or any combination of wireless and wired media.
• the communication channel 106 may form part of a packet-based network, such as a local area network, a wide-area network, or a global network, such as the Internet.
• the communication channel 106 generally represents any suitable communication medium, or collection of different communication media, for transmitting video data from the source device 102 to the destination device 104, including any suitable combination of wired or wireless media.
• the communication channel 106 may include routers, switches, base stations, or any other equipment that may be useful to facilitate communication from the source device 102 to the destination device 14. Details of an example communications system, which may facilitate such communication between the devices 102, 104, are provided below with reference to Figures 12A- 12E. Details of devices that may be representative of the devices 102, 104 are provided below, as well.
• the video encoder unit 108 and the video decoder unit 120 may operate according to one or more standards and/or specifications, such as, for example, MPEG-2, H.261, H.263, H.264, H.264/AVC, H.264 as extended according to SVC extensions ("H.264/SVC"), etc. It is understood, however, the methods, apparatuses and systems provided herein are applicable to other video encoders, decoders and/or CODECs implemented according to (and/or compliant with) different standards, or to proprietary video encoders, decoders and/or CODECs, including future video encoders, decoders and/or CODECs yet to be developed. Further, the techniques provided herein, however, are not limited to any particular coding standard.
• H.264/AVC Relevant portions of H.264/AVC noted above are available from the International Telecommunications Union as ITU-T Recommendation H.264, or more specifically, "ITU-T Rec. H.264 and ISO/IEC 14496- 10 (MPEG4-AVC), Advanced Video Coding for Generic Audiovisual Services,' v5, March, 2010;” which is incorporated herein by reference, and which may be referred to herein as the H.264 standard or H.264 specification, or the H.264/AVC standard or specification.
• the H.264/AVC standard was formulated by the ITU-T Video Coding Experts Group (VCEG) together with the ISO/IEC MPEG as a product of a collective partnership, known as the Joint Video Team (JVT).
• JVT Joint Video Team
• the techniques provided herein may be applied to devices that generally conform to the H.264 standard.
• the JVT continues to work on extensions to H.264/AVC standard.
• H.264/AVC Advanced Video Coding
• KTA Key Technologies Area
• the techniques provided herein may provide for coding improvements relative to and/or in accordance with the H.264/AVC and/or HEVC (currently draft) standards.
• each of the video encoder and video decoder units 108, 120 may include and/or be integrated with an audio encoder and/or decoder (as appropriate).
• the video encoder and video decoder units 108, 120 may include appropriate MUX- DEMUX units, or other hardware and/or software, to handle encoding of both audio and video in a common data stream or, alternatively, separate data streams.
• the MUX-DEMUX units may conform, for example, to the ITU-T Recommendation H.223 multiplexer protocol, or other protocols such as the user datagram protocol (UDP).
• Each or multiples of the video encoder and video decoder units 108, 120 may be included in one or more encoders or decoders; any of which may be integrated as part of a CODEC, and may be integrated or otherwise combined with a respective camera, computer, mobile device, subscriber device, broadcast device, set- top box, server, and the like.
• the video encoder unit 108 and the video decoder unit 120 may be implemented as any of a variety of suitable encoder and decoder circuitries, respectively, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware or any combinations thereof.
• either or both of the video encoder and video decoder units 108, 120 may be implemented substantially in software, and accordingly, operations of the elements of the video encoder unit 108 and/or the video decoder unit 120 may be performed by appropriate software instructions executed by one or more processors (not shown).
• processors may also contain off-chip components, such as external storage (for example, in the form of non-volatile memory), input/output interfaces, etc.
• each of the video encoder unit 108 and the video decoder unit 120 may be implemented as hardware, while others may be implemented using appropriate software instructions executed by one or more processors (not shown).
• operations of elements of the of the video encoder and/or video decoder units 108, 120 may be performed by software instructions executed by one or more processors
• software instructions may be maintained on a computer readable medium including magnetic disks, optical disks, and any other volatile (e.g., Random Access Memory (“RAM”)) or non-volatile (e.g., Read-Only Memory (“ROM”)) mass storage system readable by the CPU.
• RAM Random Access Memory
• ROM Read-Only Memory
• the computer readable medium may include cooperating or interconnected computer readable medium, which exist exclusively on the processing system or are distributed among multiple interconnected processing systems that may be local or remote to the processing system.
• FIG. 2B is a block diagram illustrating an example video encoder unit 108 for use with a video encoding and/or decoding system, such as the system 100 of Figure 2A.
• the video encoder unit 108 may include a video encoder 124, an output buffer 126 and a system controller 128.
• the video encoder 124 (or one or more elements thereof) may be implemented according to one or more standards and/or specifications, such as, for example, H.261, H.263, H.264, H.264/AVC, H.264/SVC, HEVC, etc. It is understood, however, the methods, apparatuses and systems provided herein are applicable to other video encoders implemented according to different standards or to proprietary CODECs, including future CODECs yet to be developed.
• the video encoder 124 may receive a video signal provided from a video source, such as, the video source 1 14 and/or the external video source. This video signal may include the un-encoded video. The video encoder 124 may encode the un-encoded video, and provide an encoded (i.e., compressed) video bit stream (BS) at its output.
• a video source such as, the video source 1 14 and/or the external video source.
• This video signal may include the un-encoded video.
• the video encoder 124 may encode the un-encoded video, and provide an encoded (i.e., compressed) video bit stream (BS) at its output.
• BS video bit stream
• the encoded video bit stream BS may be provided to the output buffer 126.
• the output buffer 126 may buffer the encoded video bit stream BS, and provide such encoded video bit stream BS as a buffered bit stream (BBS) for transmission via the communication channel 106.
• BBS buffered bit stream
• the buffered bit stream BBS output from the output buffer 126 may be provided to a storage device (not shown) for later viewing or transmission.
• the video encoder unit 108 may be configured for visual communication in which the buffered bit stream BBS may be transmitted via the communication channel 106 at a specified constant and/or variable bit rate (e.g., with very low or minimal delay).
• the encoded video bit stream BS may carry bits of the encoded video information.
• the bits of the encoded video bit stream BS and/or the buffered bit stream BBS may be arranged as a stream of encoded video frames.
• the encoded video frames may be intra-coded frames (e.g., I-frames), inter-coded frames (e.g., B-frames or P-frames), etc., as understood by those skilled in the art.
• the stream of encoded video frames may be arranged, for example, as a series of GOPs, with the encoded video frames of each GOP arranged in a specified order.
• each GOP starts with an intra-coded frame (e.g., an I-frame) followed by one or more inter-coded frames (e.g., P-frames and/or B-frames).
• Each GOP typically includes only a single intra-coded frame; although any of the GOPs may include multiple. It is noted that traditional B-frames might not be used for real-time, low delay applications (because, for example, bi-directional prediction typically causes extra coding delay as compared to uni-directional prediction (P- frames), for instance). Additional and/or other frame types are supported and the particular ordering of the encoded video frames may be modified as understood by those skilled in the art.
• Each GOP may include syntax data ("GOP syntax data").
• the GOP syntax data may be disposed in a header of the GOP, in a header of one or more frames of the GOP, or elsewhere.
• the GOP syntax data may indicate ordering, quantity and/or type of, and/or otherwise describe, the encoded video frames of the respective GOP.
• Each encoded video frame may include syntax data ("encoded- frame syntax data").
• the encoded- frame syntax data may indicate or otherwise describe an encoding mode for the respective encoded video frame.
• the video encoder 124 may include a video coding layer (VCL) and a network abstraction layer (NAL).
• VCL video coding layer
• NAL network abstraction layer
• the video encoder 124, at the VCL, may generate a raw encoded video bit stream on, for example, a block-by -block basis, and output the raw encoded video bit stream from the VCL to the NAL.
• Output of the raw encoded video bit stream output from the VCL may occur on a block-by-block, multiple-block, segment (e.g., slice, partition, etc.) or other basis.
• the raw encoded video bit stream may include encoded video data along with corresponding syntax information.
• syntax information may include, for example, one or more adaptation parameters for facilitating reconstruction of, and/or use with reconstructing, decoded video data.
• the adaptation parameters may include, for example, one or more parameters for use with, or for otherwise facilitating, in-loop filtering of the decoded video data.
• the types of adaptation parameters in the raw encoded video bit stream may vary, e.g., based on processing at the VCL.
• the raw encoded video bit stream may include adaptation parameters for use with, or for otherwise facilitating, a single type of in-loop filtering, such as sample adaptive offset (SAO) parameters, adaptive loop filter (ALF) parameters or adaptation parameters for use with, or for otherwise facilitating, another type of in-loop filtering.
• the raw encoded video bit stream may include adaptation parameters for use with, or for otherwise facilitating, multiple types of in-loop filtering, such as any combination of two or more of the SAO parameters, adaptive loop filter (ALF) parameters and adaptation parameters for use with, or for otherwise facilitating, other types of in-loop filtering.
• the raw encoded video bit stream in some instances, might not include adaptation parameters.
• the video encoder 124 may package or otherwise process the raw encoded video bit stream obtained from the VCL into one or more NAL units.
• the NAL units may include one or more coded slice NAL units, one or more NAL units ("dedicated NAL units") specifically dedicated for signaling adaptation parameters and/or one or more other types of NAL units.
• the coded slice NAL units may be, for example, IDR slice, CRA slice, non-IDR, non-CRA slice, etc., NAL units.
• the coded slice NAL units may take other forms, as well.
• Each of the coded slice NAL units may include a header ("slice header") and a payload ("slice payload") appropriately populated, by the video encoder 124, from the raw encoded video bit stream.
• Each slice payload may include data ("slice data") of a particular slice, and each slice header may include information (“slice-header information”) associated with its corresponding slice payload.
• Each of the dedicated NAL units and/or other types of NAL units may include syntax elements appropriately populated, by the video encoder 124, from, e.g., the raw encoded video bit stream.
• the dedicated NAL units may be, for example, adaptation parameter set (APS) NAL units in accordance with one or more standards and/or specifications, such as, for example, H.264, H.264/AVC, H.264/SVC, HEVC, etc.
• the dedicated NAL units may take other forms, as well.
• the video encoder 124 may string the NAL units together as a NAL unit stream, and form the stream of encoded video frames (e.g., access units) from the NAL unit stream.
• the video encoder 124 may output the stream of encoded video frames in the encoded video bit stream BS.
• the NAL units and/or the encoded video frames may undergo packetization at other layers, such as a transport layer and a network layer, for transmission via the communication channel 106.
• the NAL units and/or the encoded video frames may be, for example, populated in to payloads of various packets, and appropriate lower layer, such as communication protocol layer (L2/L3/L4, etc.), headers may be combined with payload.
• the system controller 128 may monitor various parameters and/or constraints associated with the channel 106, computational capabilities of the video encoder unit 108, demands by the users, etc., and may establish target parameters to provide an attendant quality of experience (QoE) suitable for the specified constraints and/or conditions of the channel 106.
• QoE quality of experience
• One or more of the target parameters may be adjusted from time to time depending upon the specified constraints and/or channel conditions.
• the QoE may be assessed quantitatively using one or more metrics for assessing video quality, including, for example, a metric commonly referred to as relative perceptive quality of encoded video sequences.
• the relative perceptive quality of encoded video sequences is controlled by a bit rate (BR) of the encoded bit stream BS.
• One or more of the target parameters may be adjusted to maximize the relative perceptive quality of video within the constraints associated with the bit rate BR of the encoded bit stream BS.
• FIG. 2C is a block diagram illustrating an example video decoder unit 120 for use with a video encoding and/or decoding system, such as the system 100 of Figure 2A.
• the video decoder unit 120 may include a video decoder 130, an input buffer 132 and a system controller 134.
• the video decoder 130 (or one or more elements thereof) may be implemented according to one or more standards and/or specifications, such as, for example, H.261, H.263, H.264, H.264/AVC, H.264/SVC, HEVC, etc. It is understood, however, the methods, apparatuses and systems provided herein are applicable to other video encoders implemented according to different standards or to proprietary CODECs, including future CODECs yet to be developed.
• the input buffer 132 may receive the buffered bit stream BBS from, for example, the storage device and/or the communication channel 106.
• the input buffer 130 may buffer the buffered bit stream BBS and provide an encoded video bit stream BS to the video decoder 130.
• This encoded video bit stream BS may be provided at a bit rate that is the same as the received bit rate or some other bit rate value.
• the video decoder 130 may receive encoded video bit stream BS.
• the video decoder 130 may decode the encoded video bit stream BS so as to provide a reconstructed video signal.
• the decoding of the encoded video bit stream BS may be, generally, a reciprocal of an encoding process carried out by the video encoder 124.
• the reconstructed video signal provided by the video decoder 130 may be (e.g., formatted) for presentation on a display device, such as the display device 122, and/or for storage.
• the system controller 134 may monitor various parameters and/or constraints associated with the channel 106, computational capabilities of the video decoder unit 120, demands by the users, etc., and may establish target parameters to provide an attendant quality of experience (QoE) suitable for the specified constraints and/or conditions of the channel 106.
• QoE quality of experience
• One or more of the target parameters may be adjusted from time to time depending upon the specified constraints and/or channel conditions.
• the QoE may be assessed quantitatively using one or more metrics for assessing video quality, including, for example, a metric commonly referred to as relative perceptive quality of encoded video sequences.
• One or more of the target parameters may be signaled or otherwise provided to the video encoder unit 108 and/or network elements associated with the channel 106 to request and/or effect adjustment to maximize the relative perceptive quality of video.
• QP quantization parameter
• FIG. 2D is a block diagram illustrating an example of the video encoder 124 for use with a video encoder unit, such as the video encoder unit 108 of Figures 2A-2B.
• the video encoder 124 may include an input 136, a mode decision and encoder controller unit 138, a spatial prediction unit 140, a motion/temporal prediction unit 142, a first summer 144, a transform unit 146, a quantization unit 148, an entropy coding unit 150, an inverse quantization unit 152, an inverse transform unit 154, a second summer 156, filters 158, a reference picture store 160, an output 162 and a bitstream parser 164.
• the video encoder 124 may include additional and/or different elements. Such elements are not illustrated for the sake of brevity and clarity.
• video encoder 124 shown in Figure 2D and provided herein are meant merely to be illustrative, and real world implementations may differ.
• a real world implementation may include more, fewer and/or different elements, and/or be arranged differently from the arrangement shown in Figure 2D.
• some or all functionality of both of the transform unit 146 and quantization unit 148 may be highly integrated in some of the real-world implementations, such as, for example, implementations that use the core transform of the H.264 standard.
• the inverse quantization unit 152 and inverse transform unit 154 may be highly integrated in some of the real-world implementations (e.g., H.264-standard-compliant implementations), but are likewise illustrated separately for conceptual purposes.
• the video encoder 124 may receive the video signal at its input 136.
• the video encoder 124 may generate the encoded video information from the received un-encoded video, and output the encoded video information (e.g., any of intra- frames or inter- frames) from its output 162 in the form of the encoded video bit stream BS.
• the video encoder 124 may operate, for example, as a hybrid video encoder, and employ a block-based coding process for encoding the un-encoded video. When performing such encoding process, the video encoder 124 typically operates on individual frames, pictures or images (collectively "un-encoded pictures") of the un-encoded video.
• the video encoder 124 may slice, partition, divide or otherwise segment (collectively "segment") each un-encoded picture received at its input 136 into multiple un-encoded video blocks.
• the video encoder 124 may first segment the un-encoded picture into multiple un-encoded video segments (e.g., slices), and then segment each of the un-encoded video segments into the un-encoded video blocks.
• the video encoder 124 may pass, supply, send or otherwise provide the un-encoded video blocks to the spatial prediction unit 140, the motion/temporal prediction unit 142 and/or the first summer 144.
• the un-encoded video blocks may be provided on a block-by-block basis.
• the spatial prediction unit 140 may receive the un-encoded video blocks, and encode such video blocks in intra-mode.
• Intra-mode refers to any of several modes of spatial-based compression, and encoding in intra-mode endeavors to provide spatial-based compression of the un-encoded picture.
• the spatial-based compression if any, may result from reducing or removing spatial redundancy of video information within the un-encoded picture.
• the spatial prediction unit 140 may perform spatial prediction (or "intra-prediction") of each un-encoded video block relative to one or more video blocks of the un-encoded picture that have been already encoded ("encoded video blocks") and/or reconstructed (“reconstructed video blocks").
• the encoded and/or reconstructed video blocks may be neighbors of, adjacent to, or in close proximity to, the un-encoded video block.
• the motion/temporal prediction unit 142 may receive the un-encoded video blocks from the input 136, and encode them in inter-mode.
• Inter-mode refers to any of several modes of temporal-based compression, including, for example, P-mode (uni-directional prediction) or B-mode (bi-directional prediction).
• P-mode uni-directional prediction
• B-mode bi-directional prediction
• Encoding in inter-mode endeavors to provide temporal-based compression of the un-encoded picture.
• the temporal-based compression if any, may result from reducing or removing temporal redundancy of video information among the un-encoded picture and one or more reference (e.g., adjacent) pictures.
• the motion/temporal prediction unit 142 may perform temporal prediction (or "inter-prediction") of each un- encoded video block relative to one or more video blocks of the reference pictures ("reference video blocks").
• the temporal prediction carried out may be uni-directional prediction (e.g., for P-mode) or, alternatively, bidirectional prediction (e.g., for B-mode).
• the reference video blocks may be from previously encoded and/or reconstructed pictures, and in some instances, from only one previously encoded and/or reconstructed picture.
• the encoded and/or reconstructed pictures may be neighbors of, adjacent to, or in close proximity to, the un- encoded picture.
• the reference video blocks may be from one or more previously encoded and/or reconstructed pictures and one or more other un-encoded pictures of the video stream.
• the encoded and/or reconstructed pictures and the other un-encoded pictures may be neighbors of, adjacent to, or in close proximity to, the un-encoded picture.
• reference picture index may be sent to the entropy coding unit 150 for subsequent output and/or transmission.
• the reference index may be used to identify from which reference picture(s) in the reference picture store 160 the temporal prediction comes.
• motion estimation may be carried out to estimate motion for each un-encoded video block relative to the reference-picture video blocks, and may involve generating a motion vector for the un-encoded video block.
• the motion vector may indicate a displacement of a prediction block relative to the un-encoded video block being coded.
• This prediction block is the reference-picture video block that is found to closely match, in terms of pixel difference, the un-encoded video block being coded.
• the matching may be determined by sum of absolute difference (SAD), sum of square difference (SSD), or other difference metrics.
• Motion compensation may involve fetching or generating the prediction block based on the motion vector determined by motion estimation.
• the motion/temporal prediction unit 142 may calculate the motion vector for the un-encoded video block by comparing the un-encoded video block to the reference video blocks from reference pictures stored in the reference picture store 160.
• the motion/temporal prediction unit 142 may calculate values for fractional pixel positions of a reference picture included in reference picture store 160.
• the summer 156 or another unit of video encoder 124 may calculate the fractional pixel position values for reconstructed video blocks, and then store the reconstructed video blocks, with the calculated values for the fractional pixel positions, in reference picture store 160.
• the motion/temporal prediction unit 142 may also interpolate sub- integer pixels of the reference picture (e.g., of an I-frame or a P-frame).
• the motion/temporal prediction unit 142 may be configured to encode the motion vector relative to a selected motion predictor.
• the motion predictor selected by the motion/temporal prediction unit 142 may be, for example, a vector equivalent to a mean of motion vectors of neighboring blocks that have already been encoded.
• the motion/temporal prediction unit 142 may calculate the difference between the motion vector and the motion predictor to form a motion vector difference value.
• H.264 refers to a set of potential reference frames as a "list.”
• a set of reference pictures stored in reference picture store 160 may correspond to such list of reference frames.
• the motion/temporal prediction unit 142 may compare the reference video blocks of the reference pictures from reference picture store 160 to the un-encoded video block (e.g., of a P-frame or a B-frame).
• the motion vector calculated by motion/temporal prediction unit 142 may refer to a sub-integer pixel location of the reference picture.
• the motion/temporal prediction unit 142 may send the calculated motion vector to the entropy coding unit 150 and to the motion compensation functions of the motion/temporal prediction unit 142.
• the motion/temporal prediction unit 142 (or the motion compensation functions thereof) may calculate error values for the prediction block relative to the un-encoded video block being coded.
• the motion/temporal prediction unit 142 may calculate prediction data based on the prediction block.
• the mode decision and encoder controller unit 138 may select one of the coding modes, intra-mode or inter-mode. The mode decision and encoder controller unit 138 may do so based on a rate- distortion optimization method and/or on error results produced in each mode, for example.
• the video encoder 124 may form a block of residuals ("residual video block") by subtracting the prediction data provided from motion/temporal prediction unit 140 from the un-encoded video block being coded.
• the summer 144 represents an element or multiple elements that perform this subtraction operation.
• the transform unit 146 may apply a transform to the residual video block to convert such residual video block from a pixel value domain to a transform domain, such as a frequency domain.
• the transform may be, for example, any of the transforms provided herein, a discrete cosine transform (DCT), or a conceptually similar transform. Other examples of the transform include those defined in H.264, wavelet transforms, integer transforms, sub-band transforms, etc.
• application of the transform to the residual video block by the transform unit 146 produces a corresponding block of transform coefficients of the residual video block ("residual-transform coefficients"). These residual-transform coefficients may represent magnitudes of frequency components of the residual video block.
• the transform unit 146 may forward the residual-transform coefficients and motion vectors to the quantization unit 148.
• the quantization unit 148 may quantize the residual- transform coefficients to further reduce the encoded bit rate.
• the quantization process may, for example, reduce the bit depth associated with some or all of the residual-transform coefficients.
• the quantization unit 148 may divide the values of residual-transform coefficients by a quantization level corresponding to the QP so as to form a block of quantized transform coefficients.
• the degree of quantization may be modified by adjusting the QP value.
• the quantization unit 148 applies the quantization so as to represent the residual- transform coefficients using a desired number of quantization steps; the number of steps used (or correspondingly the value of the quantization level) may determine the number of encoded video bits used to represent the residual video block.
• the quantization unit 148 may obtain the QP value from a rate controller (not shown).
• the quantization unit 148 may provide the quantized transform coefficients and the motion vectors to the entropy coding unit 150 and to the inverse quantization unit 152.
• the entropy coding unit 150 may apply entropy coding to the quantized transform coefficients so as to form entropy-coded coefficients (i.e., a bit stream).
• the entropy coding unit 150 may use adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC) or another entropy coding technique to form the entropy-coded coefficients.
• CABAC as understood by those skilled in the art, requires input of contextual information ("context"). This context may be based on neighboring video blocks, for instance.
• the entropy coding unit 150 may provide the entropy-coded coefficients along with the motion vectors in the form of a raw encoded video bit stream to an internal bit-stream formatter (not shown).
• This bit- stream formatter may form the encoded video bit stream BS provided to the output buffer 126 ( Figure 2B) by appending to the raw encoded video bit stream additional information, including headers and/or other information to enable the video decoder unit 120 to decode the encoded video block from the raw encoded video bit stream.
• the encoded video bit stream BS provided from the entropy coding unit 150 to the output buffer 126 may be transmitted to the destination device 104 via the channel 106 or archived for later transmission or retrieval.
• the entropy coding unit 150 or another unit of video encoder 124 may be configured to perform other coding functions, in addition to entropy coding.
• the entropy coding unit 150 may be configured to determine the CBP values for the video blocks.
• the entropy coding unit 150 may perform run length coding of the quantized transform coefficients in a video block.
• the entropy coding unit 150 may apply a zigzag scan or other scan pattern to arrange the quantized transform coefficients in a video block and encode runs of zeros for further compression.
• the entropy coding unit 150 also may construct the header information with appropriate syntax elements for transmission in the encoded video bit stream BS.
• the inverse quantization unit 152 and inverse transform unit 154 may apply inverse quantization and inverse transformation, respectively, to reconstruct a residual video block in the pixel domain, e.g., for later use as one of the reference video blocks (e.g., within one of the reference pictures in the reference picture list).
• the mode decision and encoder controller unit 138 may calculate the reference video block by adding the reconstructed residual video block to the prediction block of one of the reference pictures stored in the reference picture store 160.
• the mode decision and encoder controller unit 138 may also apply one or more interpolation filters to the reconstructed residual video block to calculate sub-integer pixel values (e.g., for half-pixel positions) for use in motion estimation.
• the summer 156 may add the reconstructed residual video block to the motion compensated prediction video block to produce a reconstructed video block for storage in the reference picture store 160.
• the reconstructed (pixel value domain) video block may be used by the motion/temporal prediction unit 142 (or motion estimation functions thereof and/or the motion compensation functions thereof) as one of the reference blocks for inter-coding an un-encoded video block in subsequent un-encoded video.
• the filters 158 may be configured to filter (e.g., perform in-loop filtering of) the reconstructed video blocks output from the summer 156.
• the BS parser 164 may be configured to parse from the encoded video bit stream BS, and provide to the filters 158, information to facilitate operation of such filters 158.
• This information may include, for example, control information and/or adaptation parameters.
• the control information may include, for a particular type of filtering, a flag for signaling whether to disable or enable such particular type of filtering ("disable/enable flag").
• the control information may include multiple disable/enable flags; each of which for signaling whether to disable or enable a particular type of filtering, for instance.
• FIG. 2E is a block diagram illustrating an example of a video decoder 35 for use with a video decoder unit, such as the video decoder unit 120 of Figures 2A-2B.
• the video decoder 35 may include an input 166, entropy decoding unit 168, a motion compensated prediction unit 170, a spatial prediction unit 172, an inverse quantization unit 174, an inverse transformation unit 176, a reference picture store 180, filters 182, a summer 178, an output 184 and a bitstream parser 186.
• the video decoder 35 may perform a decoding process that is, generally, reciprocal to the encoding process provided with respect to video encoder 124 ( Figure 2D). This decoding process may be carried out as follows.
• the motion compensated prediction unit 170 may generate prediction data based on motion vectors received from the entropy decoding unit 168.
• the motion vectors may be encoded relative to a motion predictor for a video block corresponding to the encoded motion vector.
• the motion compensated prediction unit 170 may determine the motion predictor, for example, as the median of motion vectors of blocks neighboring the video block to be decoded.
• the motion compensated prediction unit 170 may decode the encoded motion vector by extracting a motion vector difference value from the encoded video bit stream BS and adding the motion vector difference value to the motion predictor.
• the motion compensated prediction unit 170 may quantize the motion predictor to the same resolution as the encoded motion vector.
• the motion compensated prediction unit 170 may use the same precision for all encoded motion predictors.
• motion compensated prediction unit 170 may be configured to use either of the above methods, and to determine which method to use by analyzing data included in a sequence parameter set, slice parameter set, or picture parameter set obtained from the encoded video bit stream BS.
• the motion compensated prediction unit 170 may extract a prediction video block identified by the motion vector from a reference picture of reference picture store 180. If the motion vector points to a fractional pixel position, such as a half-pixel, the motion compensated prediction unit 170 may interpolate values for the fractional pixel positions.
• the motion compensated prediction unit 170 may use adaptive interpolation filters or fixed interpolation filters to interpolate these values. Furthermore, the motion compensated prediction unit 170 may obtain indicia of which of the filters 182 to use, and in some embodiments, coefficients for the filters 182, from the received encoded video bit stream BS.
• the spatial prediction unit 172 may use intra prediction modes received in the encoded video bit stream BS to form a prediction video block from spatially adjacent blocks.
• Inverse quantization unit 174 may inverse quantize, i.e., de-quantize, quantized block coefficients provided in the encoded video bit stream BS and decoded by the entropy decoding unit 168.
• the inverse quantization process may include a conventional process, e.g., as defined by H.264.
• the inverse quantization process may also include use of a quantization parameter QP calculated by the video encoder 124 for each video block to determine a degree of quantization and, likewise, a degree of inverse quantization that should be applied.
• the inverse transform unit 176 may apply an inverse transform, e.g., an inverse of any of the transforms provided herein, inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process, to the transform coefficients to produce residual video blocks in the pixel domain.
• the motion compensated prediction unit 170 may produce motion compensated blocks, possibly performing interpolation based on interpolation filters. Identifiers for interpolation filters to be used for motion estimation with sub-pixel precision may be included in syntax elements of the video block.
• the motion compensated prediction unit 170 may use the interpolation filters as used by video encoder 124 during encoding of the video block to calculate interpolated values for sub-integer pixels of a reference block.
• the motion compensated prediction unit 170 may determine the interpolation filters used by the video encoder 124 according to received syntax information and use the interpolation filters to produce prediction blocks.
• the motion compensated prediction unit 170 may use some of the syntax information to determine sizes of the video blocks used to encode picture(s) of the encoded video sequence, partition information that describes how each video block of a frame of the encoded video sequence is partitioned, modes indicating how each partition is encoded, one or more reference pictures for each inter-encoded video block, and other information to decode the encoded video sequence.
• the summer 178 may sum the residual blocks with the corresponding prediction blocks generated by the motion compensated prediction unit 170 or the spatial prediction unit 172 to form decoded video blocks.
• the filters 182 may be configured to filter (e.g., perform in-loop filtering of) the reconstructed video blocks output from the summer 178.
• the BS parser 186 may be configured to parse from the encoded video bit stream BS, and provide to the filters 182, information to facilitate operation of such filters 182.
• This information may include, for example, control information and/or adaptation parameters.
• the control information and/or adaptation parameters may be the same as those parsed from the encoded video bit stream BS by the BS parser 164 ( Figure 2D).
• the filters 182 and filters 158 may process the reconstructed video blocks using the same adaptation parameters.
• the filters 182 may output the reconstructed video blocks after application of one or more types of filtering.
• the reconstructed video blocks are then stored in reference picture store 180, which provides the reference video blocks for subsequent motion compensation and also produces decoded video for presentation on a display device (such as display device 122 of Figure 2A).
• the video information of such video block may be represented differently.
• the video block may include (i) pixel data in the pixel domain; (ii) residual data ("residuals") representing pixel differences between an un-encoded video blocks and a prediction block; (iii) transform coefficients in the transform domain (e.g., following application of a transform); and (iv) quantized transform coefficients in the quantized transform domain.
• Each video block may have given dimensions or, collectively "size”.
• the video block size may depend on the coding standard.
• the H.264 standard supports intra prediction in various video block sizes, such as 16x 16, 8x8, or 4x4 for luma components, and 8x8 for chroma components, and supports inter prediction in various block sizes, such as 16x16, 16x8, 8x16, 8x8, 8x4, 4x8 and 4x4 for luma components and corresponding scaled sizes for chroma components.
• a video block whose dimensions are 16 pixels by 16 pixels is generally referred to as a macroblock (MB), and a video block with less than 16 pixels by 16 pixels is generally referred to as a partition of an MB ("MB partition").
• a video block called a "coding unit” or “CU” may be used to compress high resolution (e.g., 1080p and beyond) video signals more efficiently.
• the CU size is set in a parameter sequence set, and may be set as large as 64x64 pixels or as small as 4x4 pixels.
• the CU can be further partitioned into prediction units (PUs), for which separate prediction methods are applied.
• Each video block (whether a MB, CU, PU, etc.) may be processed by using the spatial prediction unit 140 and/or motion/temporal prediction unit 142.
• Figure 3 is a flow diagram illustrating a flow 300 for performing high-level layer signaling of adaptation parameters.
• the flow 300 of Figure 3 is described with reference to the example system 100 of Figure 2.
• the flow 300 may be carried out using other architectures, as well.
• the video encoder 124 may obtain an APS.
• the video encoder 124 may obtain the APS; the APS may contain SAO and ALF parameters that may be obtained during encoding operations, for example, according to the encoder in Figure 2D.
• the video encoder 124 may selectively convey the APS in an encoded video bit stream BS by inserting the APS in a slice header, inserting the APS in a dedicated NAL unit, or both, as shown in process block 304.
• the video encoder 124 may selectively convey the adaption parameters in the encoded video bit stream BS using a signaling method selected from a set including (i) inserting the APS in the slice header ("slice-header APS signaling method") and (ii) inserting the APS in a dedicated NAL unit (“NAL-unit APS signaling method").
• the set may include other high-level APS signaling methods, as well.
• Selection of the signaling method may be indicated by a flag inserted to the slice header by the video encoder 124 so as to indicate whether the APS is obtainable from the dedicated NAL unit, or obtainable from a current slice header.
• the flag if set to a first value, may indicate the APS is obtainable from the dedicated NAL unit.
• the flag if set to a second value, may indicate the APS is obtainable from the current slice.
• the dedicated NAL unit may be an APS NAL unit associated with an APS identifier (ID) included in the encoded video bit stream BS.
• the APS NAL unit may include a syntax element for identifying the APS NAL unit, and the APS ID included in the encoded video bit stream BS may correspond to a value of the syntax element.
• the signaling-method selection may be made with reference to a threshold.
• the threshold may be based on at least one of a number of factors, including any of (i) an amount of frames between intra frames, (ii) an amount of frames following an intra- frame, (iii) an amount of frames between a current frame and a preceding intra frame; (iv) a video statistic, and (v) an error likelihood associated with a transmission channel.
• the video encoder 124 may determine, and establish the threshold.
• the video statistics may include one or more criteria for a color component analysis.
• the criteria may be for a color component analysis including color saturation and/or hue.
• the signaling-method selection made, responsive to a color satisfying the threshold may be the slice-header APS signaling method, the NAL-unit APS signaling method, or both the slice-header APS signaling method and NAL-unit APS signaling method.
• the signaling-method selection made, responsive to a color failing to satisfy the threshold may be the slice-header APS signaling method, the NAL-unit APS signaling method, or both the slice-header APS signaling method and NAL-unit APS signaling method.
• the signaling- method selection made, responsive to a color satisfying the threshold may be the slice-header APS signaling method or the NAL-unit APS signaling method, and the signaling-method selection made, responsive to a color failing to satisfy the threshold, may be APS signaling method not selected responsive to the color satisfying the threshold.
• the signaling-method selection made, responsive to a color satisfying the threshold might not be mutually exclusive from the signaling-method selection made, responsive to the color failing to satisfy the threshold.
• the signaling-method selection may be based, in part, on a frequency of use of a current picture as a reference picture to code subsequent pictures.
• the signaling-method selection may be based, in part, on values of quantization parameters (QPs). For pictures having lower values of QPs, for example, the signaling- method selection made or more frequently made may be may be the slice-header APS signaling method.
• QPs quantization parameters
• the signaling-method selection may be based on long term reference pictures.
• the signaling-method selection made may be the slice-header APS signaling method.
• the video encoder 124 may output the encoded bit stream BS with the selectively conveyed APS.
• the video encoder 124 may output the encoded bit stream BS from the NAL to a lower layer (e.g., any of one or more communication protocol layers).
• FIG. 4 is a block diagram illustrating a combined logical and structural view of an example of an in- loop filtering unit 400 for performing in-loop filtering during video coding (e.g., video encoding and/or video decoding carried out by a video encoder and/or a video decoder, respectively). Also shown in Figure 4 is an example of a bitstream parser 402. For convenience and simplicity of exposition, the in-loop filtering unit 400 and the bitstream parser 402 are described with respect to the video encoder and video decoder units 108, 120 of Figure 2. The in-loop filtering unit 400 and bitstream parser 402 may be applicable to other architectures, as well.
• the in-loop filtering unit 400 may be representative embodiment of the filters 158 and/or filters 182 of the video encoder and video decoder 124, 130, respectively, of Figures 2D, 2E, for instance.
• the bitstream parser 402 may be representative of the bitstream parsers 164, 184 of the video encoder and video decoder 124, 130, respectively.
• the in-loop filtering unit 400 may be configured to apply in-loop filtering on reconstructed video blocks received by the in-loop filtering unit 400 (e.g., after receipt from summers 156, 178 and prior to such reconstructed video blocks being stored in the reference picture stores 160, 180).
• the in-loop filtering unit 400 may include a deblocking filter (DBF) unit 410, a sample adaptive offset (SAO) filter unit (hereinafter "SAO unit”) 412, an adaptive loop filter (ALF) unit 414 and control logic (not shown).
• DBF deblocking filter
• SAO unit sample adaptive offset
• ALF adaptive loop filter
• the in-loop filtering unit 200 may include more, fewer and/or different filter units.
• the in- loop filtering unit 200 might include, for example, the DBF unit 410 and the SAO unit 412 or the ALF unit 414 (or any other permutation of the three filter units).
• the control logic may control or otherwise cause invocation of the DBF, SAO and/or ALF units 410, 412 and 414.
• the control logic may invoke all of the DBF, SAO and ALF units 410, 412 and 414 by default (e.g., in absence of control information, such as disable/enable flags, indicating otherwise).
• the control logic may invoke the DBF, SAO and/ or ALF units 410, 412 and 414 based on the control information obtained from the bitstream parser 202.
• the control logic may invoke the DBF, SAO and ALF units 410, 412 and 414 in absence of the control information including disable/enable flags indicating otherwise.
• control logic may invoke the DBF unit 410 in absence of the control information including a disable flag ("disable-DBF flag”) for disabling the DBF unit 410 from applying deblocking filtering.
• the control logic may invoke the SAO unit 412 in absence of the control information including a disable flag ("disable-SAO flag") for disabling the SAO unit 412 from applying SAO filtering.
• the control logic may invoke the SAO unit 412 may invoke the ALF unit 414 in absence of the control information including a disable flag ("disable -ALF flag") for disabling the ALF unit 414 from applying adaptive loop filtering.
• control logic may invoke the DBF, SAO and ALF units 410, 412 and 414 based on values of the disable/enable flags included in the control information. For example, the control logic may invoke and/or disable the DBF unit 410 responsive to different values of a corresponding disable/enable flag. The control logic may invoke and/or disable the SAO and ALF units 412 and 414 in the same way.
• the reconstructed video blocks may undergo in-loop filtering in a particular order.
• the reconstructed video blocks undergo deblocking filtering by the DBF unit 410, followed by SAO filtering by the SAO unit 412 and then adaptive loop filtering by the ALF unit 414, if all of the DBF, SAO and ALF units 410, 412 and 414 are invoked. If any of the DBF, SAO and ALF units 410, 412 and 414 are not invoked, the order may remain the same except that the corresponding filtering is skipped.
• the reconstructed video blocks may undergo the in-loop filtering in a different order, as well. For example, the reconstructed video blocks undergo adaptive loop filtering by the ALF unit 414 followed by SAO filtering by the SAO unit 412.
• the DBF unit 410 when invoked, may operate to remove visual artifacts that may be present in the reconstructed video blocks. These artifacts may be introduced in the encoding process due to, for example, use of different modes of encoding such as intra coding mode or inter coding mode. Artifacts may be present, for example, at boundaries and/or edges of the received video blocks, and the DBF unit 410 may operate to smoothen the boundaries and/or edges of the video blocks to improve visual quality.
• the SAO unit 412 may operate to restore certain high frequency components present in an original video signal, but lost due to (e.g., relatively severe) quantization.
• the SAO unit 412 may receive offset values to be added to the reconstructed video blocks, and may filter the reconstructed blocks by adding the offsets adaptively. SAO filtering may improve subjective video quality, as well as coding efficiency.
• the SAO unit 412 when invoked, may receive the reconstructed video blocks, and perform SAO filtering of the reconstructed video blocks in accordance with a set of SAO parameters (e.g., offsets) obtained from the bitstream parser 202.
• SAO parameters e.g., offsets
• the ALF unit 414 may operate to restore frequency components present in an original video signal, but lost due to (e.g., relatively severe) quantization using adaptive loop filtering.
• the ALF filtering improves coding efficiency, especially for high resolution video content.
• the ALF unit 414 when invoked, may receive the reconstructed video blocks, and perform ALF filtering of the reconstructed video blocks in accordance with a set of ALF parameters obtained from the bitstream parser 202.
• the video encoder 124 may determine to invoke the SAO filtering, and derive the set of SAO parameters.
• the video encoder 124 may insert the set of SAO parameters into the encoded video bit stream BS, and not set the disable-SAO flag carried in the encoded video bit stream BS.
• the bitstream parser 202 may parse the disable-SAO flag from the encoded video bit stream BS, and provide it the control logic.
• the control logic may check the disable-SAO flag, and determine that the SAO unit 412 is enabled (or not disabled).
• the bitstream parser 202 may parse the set of SAO parameters from the encoded video bit stream BS, and provide the set of SAO parameters to the SAO unit 412.
• the SAO unit 412 may use the set of SAO parameters when applying the SAO filtering.
• the video decoder 130 may determine that the video encoder 124 invoked the SAO filtering.
• the bitstream parser 202 may parse the disable-SAO flag from the encoded video bit stream BS, and provide it the control logic.
• the control logic may check the disable-SAO flag, and determine that the SAO unit 412 is enabled (or not disabled).
• the bitstream parser 202 may parse the set of SAO parameters from the encoded video bit stream BS, and provide them to the SAO unit 412.
• the SAO unit 412 may use the set of SAO parameters when applying the SAO filtering.
• the video encoder 124 may determine to invoke the adaptive-loop filtering, and derive the set of ALF parameters.
• the video encoder 124 may insert the set of ALF parameters into the encoded video bit stream BS, and not set the disable-ALF flag carried in the encoded video bit stream BS.
• the bitstream parser 202 may parse the disable-ALF flag from the encoded video bit stream BS, and provide it the control logic.
• the control logic may check the disable-ALF flag, and determine that the ALF unit 414 is enabled (or not disabled).
• the bitstream parser 202 may parse the set of ALF parameters from the encoded video bit stream BS, and provide the set of ALF parameters to the ALF unit 414.
• the ALF unit 414 may use the set of ALF parameters when applying the adaptive-loop filtering.
• the video decoder 130 may determine that the video encoder 124 invoked the adaptive-loop filtering.
• the bitstream parser 202 may parse the disable-ALF flag from the encoded video bit stream BS, and provide it the control logic.
• the control logic may check the disable-ALF flag, and determine that the ALF unit 414 is enabled (or not disabled).
• the bitstream parser 202 may parse the set of ALF parameters from the encoded video bit stream BS, and provide the set of ALF parameters to the ALF unit 414.
• the ALF unit 414 may use the set of ALF parameters when applying the ALF filtering.
• both of the video encoder and the video decoder 124, 130 may cause the video decoder 130 to produce the reconstructed video blocks, and in turn, the reference video blocks, that are consistent with the same produced by the video encoder 124. Having consistent reference blocks at both of the video encoder and the video decoder 124, 130, may minimize or otherwise limit drift between the video encoder and video decoder 124, 130 when predicting other decoded video blocks.
• FIG. 5 is a block diagram illustrating an example of a NAL unit stream 500 in which adaptation parameters are only conveyed by inserting the adaptation parameters in slice headers ("a slice-header only APS signaling method").
• the NAL unit stream 500 may include first, second and third coded slice NAL units 502, 504 and 506 for respective coded slices, namely, slice 0, slice 1 and slice 0, of a picture.
• the first NAL unit 502 may include a slice header 502- 1 and slice data 502-2 for slice 0.
• the second NAL unit 504 may include a slice header 504-1 and slice data 504-2 for slice 1.
• the third NAL unit 506 may include a slice header 506- 1 and slice data 506-2 for slice 2.
• the slice headers 502- 1, 504-1 and 506-2 include respective APSs (e.g., SAO and/or ALF parameters).
• APSs e.g., SAO and/or ALF parameters.
• bits of each of the slice headers 502- 1, 504- 1 and 506-2 are allocated to the respective APSs notwithstanding values of such APSs may be the same for all of the coded slices 0-2 of the picture.
• FIG. 6 is a block diagram illustrating an example of a NAL unit stream 600 in which adaptation parameters are only conveyed by inserting the adaptation parameters in a dedicated NAL unit ("a dedicated- NAL-unit only APS signaling method").
• the NAL unit steam 600 of Figure 6 is similar to the NAL unit stream 500 of Figure 5 in that it may include first, second and third NAL units, 602, 604 and 606 for respective coded slices, namely, slice 0, slice 1 and slice 0, of a picture.
• the first NAL unit 602 may include a slice header 602- 1 and slice data 602-2 for slice 0.
• the second NAL unit 604 may include a slice header 604- 1 and slice data 604-2 for slice 1.
• the third NAL unit 606 may include a slice header 606- 1 and slice data 606-2 for slice 2.
• the NAL unit stream 600 may also include a dedicated NAL unit 608.
• the dedicated NAL unit 608 may carry a common APS for slices 0-2.
• the dedicated NAL unit 608 may carry the common APS for slices 0-2 and every other slice, if any, of the picture (i.e., one dedicated NAL unit per coded picture).
• the dedicated NAL unit 608 may also include an identifier ("APS ID") for identifying the common APS.
• the APS ID may be, for example, a number assigned to the common APS.
• the APS ID may be maintained in a syntax element of the dedicated NAL unit 608.
• slice headers 602-1, 604- 1 and 606-2 include the APS, but each includes a flag (“slice-header APS flag") that indicates the APS is retrievable from the dedicated NAL unit 608. If the slice-header APS flag is set, then a syntax element, for example, the APS ID, may be included to identify the dedicated NAL unit that may contain the common APS.
• the dedicated NAL unit 608 may be, for example, an APS NAL unit in accordance with one or more standards and/or specifications, such as, for example, H.264, H.264/AVC, H.264/SVC, HEVC, etc.
• the APS ID may be maintained in a syntax element, aps id.
• the dedicated NAL unit 408 may take other forms, as well.
• FIG. 7 is a block diagram illustrating an example of a NAL unit stream 700 in which adaptation parameters are only conveyed using the dedicated-NAL-unit only APS signaling method.
• the NAL unit stream 700 includes, for each of three consecutive coded pictures, a dedicated NAL unit and a coded slice NAL unit for each slice of the corresponding picture. As shown, the first picture is coded using three slices, the second picture is coded using one slice, and the third picture is coded using two slices.
• a unique APS is signaled for each of the pictures, and all slices within the same picture use the same APS, which APS is in the dedicated NAL unit associated with the given picture, identified by the APS IDs present in the slice headers of the picture and in the dedicated NAL unit.
• the slices in all the pictures, picture 0, 1 and 2 include integral numbers of rows of video blocks, and "dedicated NAL unit” is abbreviated as "DNU".
• the slices may include non- integral numbers of rows of video blocks and/or be different shapes, as well. Non- integral numbers of rows of video blocks and/or be different shapes for slices may be useful for slices when coupled with the use of tiles.
• the dedicated NAL units may be APS NAL units in accordance with one or more standards and/or specifications, such as, for example, H.264, H.264/AVC, H.264/SVC, HEVC, etc.
• Table 1 lists, in detail, syntax elements for an APS NAL unit in accordance with an HEVC draft ("HEVC APS NAL unit"). As listed in Table 1 , the APS NAL unit may also include scaling list parameters. Table 1. Syntax Elements for a HEVC APS NAL Unit
• the aps id identifies the APS that is referred to in the slice header.
• the aps_scaling_list_data_present_flag 1 specifies that the scaling list parameters exist in this APS, and equal to 0 specifies that scaling list parameters do not exist in this APS.
• the aps sample adaptive offset flag 1 specifies that the SAO is on for slices referred to the current APS; and equal to 0 specifies that the SAO is off for slices referred to the current APS. If there is no active APS, the aps sample adaptive offset flag value is inferred to be 0.
• the aps adaptive loop filter flag 1 specifies that the ALF is on for slices referred to in the current APS; equal to 0 specifies that the ALF is off for slices referred to the current APS. If there is no active APS, the aps adaptive loop filter flag value is inferred to be 0.
• the aps extension flag 0 specifies that no aps extension data flag syntax elements are present in the picture parameter set RBSP syntax structure.
• the aps extension flag shall be equal to 0 in bitstreams conforming to the HEVC Recommendation
• the value of 1 for the aps extension flag is reserved for future use by ITU-T
• the aps extension data flag may have any value. It shall not affect the conformance to profiles specified in the HEVCRecommendation
• Table 2 (below) lists syntax elements for a slice header in accordance with an HEVC draft.
• the entropy slice flag 1 specifies that the value of slice header syntax elements not present shall be inferred to be equal to the value of slice header syntax elements in a preceding slice, entropy slice flag shall be equal to 0 when LCUAddress equal to 0.
• the slice type specifies the coding type of the slice according to the following table.
• the pic_parameter_set_id specifies the picture parameter set in use.
• the value of pic_parameter_set_id shall be in the range of 0 to 255, inclusive.
• the aps id identifies the adaptation parameter set in use to reconstruct the coded blocks in the current slice.
• NAL unit streams in which adaptation parameters are only conveyed using the dedicated-NAL-unit only APS signaling method may present error resilience challenges.
• the encoded video bit stream BS output may include a multitude of NAL units, including NAL unit types, such as Sequence Parameter Set (SPS), Picture Parameter Set (PPS), APS, coded slice and /or the like.
• SPS Sequence Parameter Set
• PPS Picture Parameter Set
• APS coded slice and /or the like.
• the NAL units in the bitstream may undergo additional packetization at other layers, such as the transport layer and the network layer, where NAL units are further packetized into network packets for transmission.
• the transmission channel 106 might not be reliable and some video packets might be lost.
• Using re-transmission to ensure all packets are delivered and delivered correctly may be impractical for low-delay video applications, such as mobile video calling, video conferencing, etc, and attendant real-time requirements of such applications may make.
• Reconstruction error at picture 0 may continue to propagate in the temporal domain through picture 1 , picture 2, and so on, as all these later pictures may use the reconstructed picture 0 for temporal prediction (as shown).
• the dedicated-NAL-unit only APS signaling method for conveying APSs might be more efficient and more aligned with common usage of SAO/ALF filtering than the slice-header only APS signaling method,
• the dedicated-NAL-unit only APS signaling method might make it more difficult to guarantee error- free delivery of the APSs and all slice data that rely on it simultaneously. And in an event of losing the dedicated NAL units or the APSs, reconstruction error may occur even if all slice data are correctly received.
• modifications at the network layer may be used to prevent loss of the APS when conveying the APS using the dedicated-NAL-unit only APS signaling method.
• the foregoing approach may place additional burden on the network layer, and in some instances, might introduce reconstruction error.
• a unique APS ID value is used for each of the APSs.
• the HEVC standard allows different APS NAL units to share the same aps id, as shown in Figure 8 by way of example.
• Each picture in Figure 8 is coded using 1 slice, and one APS is sent for each picture/slice.
• the video decoder 130 decodes the first picture, it may use the APS sent in the first APS NAL unit.
• the video decoder 130 may replace the first
• APS with new APS when they have the same value for aps id) in bitstream arrangement may make carrying out steps 1 -6 at network layer more difficult (e.g., more processing and storage of APSs), and make correctly pairing APS NAL units with corresponding slice data NAL units more difficult without mimicking the NAL.
• Support coding a picture with multiple slices when each slice intends to use its own set of APS may be difficult and/or inefficient when conveying the APS using the dedicated-NAL-unit only APS signaling method.
• the NAL unit stream 900 may incur additional overhead including signaling of additional NAL header, and additional parsing of more NAL units, etc. Note that there are various ways to arrange the order of the dedicated NAL units and the coded slice NAL units, and Figure 9 illustrates only one possible example.
• the NAL unit stream may include various intervening NAL units between a dedicated NAL unit and a coded slice NAL unit(s) with the slice(s) which refer to such dedicated NAL unit.
• FIG. 10 is a block diagram illustrating an example of a NAL unit stream 1000 in which adaptation parameters are selectively conveyed using a signaling method selected from a set including (i) the slice- header APS signaling method and (ii) the NAL-unit APS signaling method.
• a flag (“use_sao_alf_param_in_aps_flag") added by the video encoder 124 to the slice header is present to indicate whether the APS used for reconstruction of the current slice is obtained from a dedicated NAL unit, or included in the current slice header.
• the aps id may be included in the encoded bit stream B S, and the APS are obtainable from the dedicated NAL unit associated with the aps id. If the flag is set to one, then the APS are included in the current slice.
• Selectively conveying adaptation parameters using a signaling method selected from a set including (i) the slice-header APS signaling method and (ii) the NAL-unit APS signaling method may provide the video encoder 124 flexibility to decide which slices use the APS inserted in dedicated NAL units, and which slices use the APS inserted in the slice header.
• the video encoder 124 may use the slice-header APS signaling method more frequently when closer to a last intra refresh point; and use the NAL-unit APS signaling method more frequently when closer to a next intra refresh point. Due to signaling inside slice header being more error robust, using the slice-header APS signaling method more frequently toward a beginning of the temporal prediction chain may reduces a probability of error propagation. Toward the later part of the temporal prediction chain, error propagation might be less problematic, and the video encoder 130 may use the NAL- unit APS signaling method (e.g., to take advantage of efficiency gains).
• Certain pictures may be used more frequently as references to code subsequent pictures (subsequent in coding order).
• the video encoder 124 may find it beneficial to use pictures coded with lower QPs as reference. For these pictures, the video encoder 124 may use the slice-header APS signaling method more frequently (e.g., for robust delivery). As another example, the video encoder 124 may decide to use certain pictures as long term reference pictures. The video encoder 124 may use the slice-header APS signaling method more frequently for such pictures.
• the video encoder 124 may want to apply different APSs for different slices in the picture.
• Region of interest (ROI) coding may be a use case for which different APSs may be applied for different slices in the picture.
• the video encoder 124 may consider a center part of a current picture to be an ROI (e.g., the video encoder 124 may consider the center part more important than other parts of the picture).
• the video encoder 124 may want to apply different APS to the slice that contains the ROI.
• the video encoder 124 may want to apply both SAO filtering and ALF to the ROI, and apply only SAO filtering to the rest of the picture.
• the video encoder 124 may use the slice-header APS signaling method to signal the SAO and ALF parameters in the slice header of the ROI slice, and use the NAL-unit APS signaling method to signal the SAO parameters in all other slices.
• Application of the slice- header APS signaling method and the NAL-unit APS signaling method in this way may not only de-couple the ROI slice from the other slices representing the rest of the picture, but may also provide better error resilience (e.g., particularly for the ROI slice).
• Table 3 (below) lists example syntax elements for a slice header in accordance with selectively conveying adaptation parameters using a signaling method selected from a set including (i) the slice-header APS signaling method and (ii) the NAL-unit APS signaling method. Some or all of the syntax elements listed in Table 3 may be used inside a slice header.
• the use sao alf jaram in aps flag identifies if the syntax element aps id is present. If use_sao_alf_param_in_aps_flag is equal to 1 , then the syntax element aps id is present, and the ALF and SAO parameters are retrieved from the APS associated with the aps id. If use_sao_alf_param_in_aps_flag is equal to 0, then the syntax element aps id is not present, and the ALF and SAO parameters are decoded directly from the slice header.
• the use sample adaptive offset flag 1 specifies that the SAO is on for the current; equal to 0 specifies that the SAO is off for the current slice.
• the use adaptive loop filter flag 1 specifies that the ALF is on for the current slice; equal to 0 specifies that the ALF is off for the current slice.
• Figure 1 IB is a flow chart illustrating an example of a flow 1 100 for selectively conveying adaptation parameters using a signaling method selected from a set including (i) the slice-header APS signaling method and (ii) the NAL-unit APS signaling method.
• the flow 1 100 may be carried out by the video encoder 124, video decoder 130, or other video encoder, decoder and/or CODEC.
• a check is performed on a slice header to determine if the slice header includes a flag (e.g., a one-bit flag) indicating to use the APS from the dedicated NAL unit having the particular aps id. If a determination is made that the slice header includes the flag indicating to use the APS from the dedicated NAL unit having the particular aps id, then the value of the particular aps id may be decoded as shown in process block 1 104.
• the APS retrieved from the dedicated NAL unit may include, for example, SAO and/or ALF parameters, and may be used for reconstruction of decoded video blocks.
• the APS may be obtained from the current slice header.
• a check is performed on the current slice header to determine whether SAO filtering is invoked. If a determination is made that SAO filtering is invoked, the SAO parameters may be decoded from the APS obtained from the current slice header, as shown in process block 1 108. If a determination is made that SAO filtering is not invoked, the APS obtained from the current slice header might not be examined for SAO parameters.
• an aps id is decoded from a NAL-unit APS .
• decision block 1 154 a check is performed to determine whether SAO filtering is invoked. If a determination is made that SAO filtering is invoked, then the SAO parameters may be decoded from the APS from the dedicated NAL unit, as shown in process block 1 156. If a determination is made that SAO filtering is not invoked, the dedicated NAL unit might not be examined for SAO parameters.
• a check is performed to determine whether adaptive loop filtering is invoked.
• the ALF parameters may be decoded from an APS obtainable from the APS of the dedicated NAL unit, as shown in process block 1 160. If a determination is made that adaptive loop filtering is not invoked, then the dedicated NAL unit might not be examined for ALF parameters.
• the APS e.g., the SAO/ALF parameters
• the APS may be used to reconstruct the current slice; otherwise, the APS in the dedicated NAL may be used.
• base stations such as base transceiver station (BTS), a Node-B, an eNode B, a Home Node B, a Home eNode B, a site controller, an access point (AP), a wireless router, a media aware network element (MANE), as well as a wireless transmit/receive units (WTRUs) may generate and/or process the signaling described above to convey coded video data from one entity to another.
• BTS base transceiver station
• AP access point
• MANE media aware network element
• WTRUs wireless transmit/receive units
• FIG 12A is a diagram of an example communications system 1200 in which one ormore disclosed embodiments may be implemented.
• the communications system 1200 may be a multiple access system that provides content, such as voice, data, video, messaging, broadcast, etc., to multiple wireless users.
• the communications system 1200 may enable multiple wireless users to access such content through the sharing of system resources, including wireless bandwidth.
• the communications systems 1200 may employ one or more channel access methods, such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), single- carrier FDMA (SC-FDMA), and the like.
• CDMA code division multiple access
• TDMA time division multiple access
• FDMA frequency division multiple access
• OFDMA orthogonal FDMA
• SC-FDMA single- carrier FDMA
• the communications system 1200 may include wireless transmit/receive units (WTRUs) 1202a, 1202b, 1202c, 1202d, a radio access network (RAN) 1204, a core network 1206, a public switched telephone network (PSTN) 1208, the Internet 1210, and other networks 1212, though it will be appreciated that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and/or network elements.
• WTRUs 1202a, 1202b, 1202c, 1202d may be any type of device configured to operate and/or communicate in a wireless environment.
• the WTRUs 1202a may be any type of device configured to operate and/or communicate in a wireless environment.
• 1202b, 1202c, 1202d may be configured to transmit and/or receive wireless signals and may include user equipment (UE), a mobile station, a fixed or mobile subscriber unit, a pager, a cellular telephone, a personal digital assistant (PDA), a smartphone, a laptop, a netbook, a personal computer, a wireless sensor, consumer electronics, or any other terminal capable of receiving and processing compressed video communications.
• UE user equipment
• PDA personal digital assistant
• smartphone a laptop, a netbook, a personal computer, a wireless sensor, consumer electronics, or any other terminal capable of receiving and processing compressed video communications.
• the communications systems 1200 may also include a base station 1214a and a base station 1214b.
• Each of the base stations 1214a, 1214b may be any type of device configured to wirelessly interface with at least one of the WTRUs 1202a, 1202b, 1202c, 1202d to facilitate access to one or more communication networks, such as the core network 1206, the Internet 1210, and/or the networks 1212.
• the base stations 1214a, 1214b may be a base transceiver station (BTS), a Node-B, an eNode B, a Home Node B, a Home eNode B, a site controller, an access point (AP), a wireless router, and the like. While the base stations 1214a, 1214b are each depicted as a single element, it will be appreciated that the base stations 1214a, 1214b may include any number of interconnected base stations and/or network elements.
• the base station 1214a may be part of the RAN 1204, which may also include other base stations and/or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), relay nodes, etc.
• the base station 1214a and/or the base station 1214b may be configured to transmit and/or receive wireless signals within a particular geographic region, which may be referred to as a cell (not shown).
• the cell may further be divided into cell sectors.
• the cell associated with the base station 1214a may be divided into three sectors.
• the base station 1214a may include three transceivers, i.e., one for each sector of the cell.
• the base station 1214a may employ multiple-input multiple output (MIMO) technology and, therefore, may utilize multiple transceivers for each sector of the cell.
• MIMO multiple-input multiple output
• the base stations 1214a, 1214b may communicate with one or more of the WTRUs 1202a, 1202b, 1202c, 1202d over an air interface 1216, which may be any suitable wireless communication link (e.g. , radio frequency (RF), microwave, infrared (IR), ultraviolet (UV), visible light, etc.).
• the air interface 1216 may be established using any suitable radio access technology (RAT).
• RAT radio access technology
• the communications system 1200 may be a multiple access system and may employ one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC- FDMA, and the like.
• the base station 1214a in the RAN 1204 and the WTRUs 1202a, 1202b, 1202c may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may establish the air interface 1216 using wideband CDMA (WCDMA).
• WCDMA may include communication protocols such as High-Speed Packet Access (HSPA) and/or Evolved HSPA (HSPA+).
• HSPA may include High-Speed Downlink Packet Access (HSDPA) and/or High-Speed Uplink Packet Access (HSUPA).
• the base station 1214a and the WTRUs 1202a, 1202b, 1202c may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may establish the air interface 1216 using Long Term Evolution (LTE) and/or LTE-Advanced (LTE-A).
• E-UTRA Evolved UMTS Terrestrial Radio Access
• LTE Long Term Evolution
• LTE-A LTE-Advanced
• the base station 1214a and the WTRUs 1202a, 1202b, 1202c may implement radio technologies such as IEEE 802.16 (i.e., Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 IX, CDMA2000 EV-DO, Interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile communications (GSM), Enhanced Data rates for GSM Evolution (EDGE), GSM EDGE (GERAN), and the like.
• IEEE 802.16 i.e., Worldwide Interoperability for Microwave Access (WiMAX)
• CDMA2000, CDMA2000 IX, CDMA2000 EV-DO Code Division Multiple Access 2000
• IS-95 Interim Standard 95
• IS-856 Interim Standard 856
• GSM Global System for Mobile communications
• GSM Global System for Mobile communications
• EDGE Enhanced Data rates for GSM Evolution
• GERAN GSM EDGERAN
• the base station 1214b in Figure 12A may be a wireless router, Home Node B, Home eNode B, or access point, for example, and may utilize any suitable RAT for facilitating wireless connectivity in a localized area, such as a place of business, a home, a vehicle, a campus, and the like.
• the base station 1214b and the WTRUs 1202c, 1202d may implement a radio technology such as IEEE 802.1 1 to establish a wireless local area network (WLAN).
• the base station 1214b and the WTRUs 1202c, 1202d may implement a radio technology such as IEEE 802.15 to establish a wireless personal area network (WPAN).
• WLAN wireless local area network
• WPAN wireless personal area network
• the base station 1214b and the WTRUs 1202c, 1202d may utilize a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, etc.) to establish a picocell or femtocell.
• a cellular-based RAT e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, etc.
• the base station 1214b may have a direct connection to the Internet 1212.
• the base station 1214b may not be required to access the Internet 1212 via the core network 1206.
• the RAN 1204 may be in communication with the core network 1206, which may be any type of network configured to provide voice, data, applications, and/or voice over internet protocol (VoIP) services to one or more of the WTRUs 1202a, 1202b, 1202c, 1202d.
• the core network 1206 may provide call control, billing services, mobile location-based services, pre-paid calling, Internet connectivity, video distribution, etc., and/or perform high-level security functions, such as user authentication.
• the RAN 1204 and/or the core network 1206 may be in direct or indirect communication with other RANs that employ the same RAT as the RAN 1204 or a different RAT.
• the core network 1206 may also be in communication with another RAN (not shown) employing a GSM radio technology.
• the core network 1206 may also serve as a gateway for the WTRUs 1202a, 1202b, 1202c, 1202d to access the PSTN 1208, the Internet 1210, and/or other networks 1212.
• the PSTN 1208 may include circuit- switched telephone networks that provide plain old telephone service (POTS).
• POTS plain old telephone service
• the Internet 1212 may include a global system of interconnected computer networks and devices that use common communication protocols, such as the transmission control protocol (TCP), user datagram protocol (UDP) and the internet protocol (IP) in the TCP/IP internet protocol suite.
• the networks 1212 may include wired or wireless communications networks owned and/or operated by other service providers.
• the networks 1212 may include another core network connected to one or more RANs, which may employ the same RAT as the RAN 1204 or a different RAT.
• Some or all of the WTRUs 1202a, 1202b, 1202c, 1202d in the communications system 1200 may include multi-mode capabilities, i.e., the WTRUs 1202a, 1202b, 1202c, 1202d may include multiple transceivers for communicating with different wireless networks over different wireless links.
• the WTRU 1202c shown in Figure 12A may be configured to communicate with the base station 1214a, which may employ a cellular-based radio technology, and with the base station 1214b, which may employ an IEEE 802 radio technology.
• FIG. 12B is a system diagram of an example WTRU 1202.
• the WTRU 1202 may include a processor 1218, a transceiver 1220, a transmit/receive element 1222, a speaker/microphone 1224, a keypad 1226, a display/touchpad 1228, non-removable memory 1206, removable memory 1232, a power source 1234, a global positioning system (GPS) chipset 1236, and other peripherals 1238.
• GPS global positioning system
• the processor 1218 may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a graphics processing unit (GPU), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Array (FPGAs) circuits, any other type of integrated circuit (IC), a state machine, and the like.
• the processor 1218 may perform signal coding, data processing, power control, input/output processing, and/or any other functionality that enables the WTRU 1202 to operate in a wireless environment.
• the processor 1218 may be coupled to the transceiver 1220, which may be coupled to the transmit/receive element 1222. While Figure 12B depicts the processor 1218 and the transceiver 1220 as separate components, it will be appreciated that the processor 1218 and the transceiver 1220 may be integrated together in an electronic package or chip.
• the transmit/receive element 1222 may be configured to transmit signals to, or receive signals from, a base station (e.g., the base station 1214a) over the air interface 1216.
• a base station e.g., the base station 1214a
• the transmit/receive element 1222 may be an antenna configured to transmit and/or receive RF signals.
• the transmit/receive element 1222 may be an emitter/detector configured to transmit and/or receive IR, UV, or visible light signals, for example.
• the transmit/receive element 1222 may be configured to transmit and receive both RF and light signals. It will be appreciated that the transmit/receive element 1222 may be configured to transmit and/or receive any combination of wireless signals.
• the WTRU 1202 may include any number of transmit/receive elements 1222. More specifically, the WTRU 1202 may employ MIMO technology. Thus, in one embodiment, the WTRU 1202 may include two or more transmit/receive elements 1222 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 1216.
• the WTRU 1202 may include two or more transmit/receive elements 1222 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 1216.
• the transceiver 1220 may be configured to modulate the signals that are to be transmitted by the transmit/receive element 1222 and to demodulate the signals that are received by the transmit/receive element 1222.
• the WTRU 1202 may have multi-mode capabilities.
• the transceiver 1220 may include multiple transceivers for enabling the WTRU 1202 to communicate via multiple RATs, such as UTRA and IEEE 802.1 1, for example.
• the processor 1218 ofthe WTRU 1202 may be coupled to, and may receive user input data from, the speaker/microphone 1224, the keypad 1226, and/or the display/touchpad 1228 (e.g., a liquid crystal display (LCD) display unit or organic light-emitting diode (OLED) display unit).
• the processor 1218 may also output user data to the speaker/microphone 1224, the keypad 1226, and/or the display/touchpad 1228.
• the processor 1218 may access information from, and store data in, any type of suitable memory, such as the non-removable memory 1206 and/or the removable memory 1232.
• the non-removable memory 1206 may include random-access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device.
• the removable memory 1232 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like.
• SIM subscriber identity module
• SD secure digital
• the processor 1218 may access information from, and store data in, memory that is not physically located on the WTRU 1202, such as on a server or a home computer (not shown).
• the processor 1218 may receive power from the power source 1234, and may be configured to distribute and/or control the power to the other components in the WTRU 1202.
• the power source 1234 may be any suitable device for powering the WTRU 1202.
• the power source 1234 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, and the like.
• the processor 1218 may also be coupled to the GPS chipset 1236, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU 1202.
• location information e.g., longitude and latitude
• the WTRU 1202 may receive location information over the air interface 1216 from a base station (e.g., base stations 1214a, 1214b) and/or determine its location based on the timing of the signals being received from two or more nearby base stations. It will be appreciated that the WTRU 1202 may acquire location information by way of any suitable location- determination method while remaining consistent with an embodiment.
• the processor 1218 may further be coupled to other peripherals 1238, which may include one or more software and/or hardware modules that provide additional features, functionality and/or wired or wireless connectivity.
• the peripherals 1238 may include an accelerometer, an e-compass, a satellite transceiver, a digital camera (for photographs or video), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands free headset, a Bluetooth® module, a frequency modulated (FM) radio unit, a digital music player, a media player, a video game player module, an Internet browser, and the like.
• the peripherals 1238 may include an accelerometer, an e-compass, a satellite transceiver, a digital camera (for photographs or video), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands free headset, a Bluetooth® module, a frequency modulated (FM) radio unit, a digital music player, a media player, a video game player
• FIG. 12C is a system diagram of the RAN 1204 and the core network 1206 according to an embodiment.
• the RAN 1204 may employ a UTRA radio technology to communicate with the WTRUs 1202a, 1202b, 1202c over the air interface 1216.
• the RAN 1204 may also be in communication with the core network 1206.
• the RAN 1204 may include Node-Bs 1240a, 1240b, 1240c, which may each include one or more transceivers for communicating with the WTRUs 1202a, 1202b, 1202c over the air interface 1216.
• the Node-Bs 1240a, 1240b, 1240c may each be associated with a particular cell (not shown) within the RAN 1204.
• the RAN 1204 may also include RNCs 1242a, 1242b. It will be appreciated that the RAN 1204 may include any number of Node-Bs and RNCs while remaining consistent with an embodiment.
• the Node-Bs 1240a, 1240b may be in communication with the RNC
• the Node-B 1240c may be in communication with the RNC 1242b.
• the RNCs 1240b, 1240c may communicate with the respective RNCs 1242a, 1242b via an Iub interface.
• the RNCs 1242a, 1242b may communicate with the respective RNCs 1242a, 1242b via an Iub interface.
• Each of the RNCs 1242a, 1242b may be in communication with one another via an Iur interface.
• Each of the RNCs 1242a, 1242b may be in communication with one another via an Iur interface.
• Each of the RNCs 1242a, 1242b may be in communication with one another via an Iur interface.
• the core network 1206 shown in Figure 12C may include a media gateway (MGW) 1244, a mobile switching center (MSC) 1246, a serving GPRS support node (SGSN) 1248, and/or a gateway GPRS support node (GGSN) 1250. While each of the foregoing elements are depicted as part of the core network 1206, it will be appreciated that any one of these elements may be owned and/or operated by an entity other than the core network operator.
• MGW media gateway
• MSC mobile switching center
• SGSN serving GPRS support node
• GGSN gateway GPRS support node
• the RNC 1242a in the RAN 1204 may be connected to the MSC 1246 in the core network 1206 via an IuCS interface.
• the MSC 1246 may be connected to the MGW 1244.
• the MSC 1246 and the MGW 1244 may provide the WTRUs 1202a, 1202b, 1202c with access to circuit-switched networks, such as the PSTN 1208, to facilitate communications between the WTRUs 1202a, 1202b, 1202c and traditional land-line communications devices.
• the RNC 1242a in the RAN 1204 may also be connected to the SGSN 1248 in the core network 1206 via an luPS interface.
• the SGSN 1248 may be connected to the GGSN 1250.
• the SGSN 1248 and the GGSN 1250 may provide the WTRUs 1202a, 1202b, 1202c with access to packet-switched networks, such as the Internet 1212, to facilitate communications between and the WTRUs 1202a, 1202b, 1202c and IP-enabled devices.
• the core network 1206 may also be connected to the networks 1212, which may include other wired or wireless networks that are owned and/or operated by other service providers.
• FIG. 12D is a system diagram of the RAN 1204 and the core network 1206 according to another embodiment.
• the RAN 1204 may employ an E-UTRA radio technology to communicate with the WTRUs 1202a, 1202b, 1202c over the air interface 1216.
• the RAN 1204 may also be in communication with the core network 1206.
• the RAN 1204 may include eNode-Bs 1260a, 1260b, 1260c, though it will be appreciated that the RAN 1204 may include any number of eNode-Bs while remaining consistent with an embodiment.
• the eNode-Bs 1260a, 1260b, 1260c may each include one or more transceivers for communicating with the WTRUs 1202a, 1202b, 1202c over the air interface 1216.
• the eNode-Bs 1260a, 1260b, 1260c may implement MIMO technology.
• the eNode-B 1260a for example, may use multiple antennas to transmit wireless signals to, and receive wireless signals from, the WTRU 1202a.
• Each of the eNode-Bs 1260a, 1260b, 1260c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in the uplink and/or downlink, and the like. As shown in Figure 12D, the eNode-Bs 1260a, 1260b, 1260c may communicate with one another over an X2 interface.
• the core network 1206 shown in Figure 12D may include a mobility management gateway (MME) 1262, a serving gateway 1264, and a packet data network (PDN) gateway 1266. While each of the foregoing elements are depicted as part of the core network 1206, it will be appreciated that any one of these elements may be owned and/or operated by an entity other than the core network operator.
• MME mobility management gateway
• PDN packet data network
• the MME 1262 may be connected to each of the eNode-Bs 1260a, 1260b, 1260c in the RAN 1204 via an S I interface and may serve as a control node.
• the MME 1262 may be responsible for authenticating users of the WTRUs 1202a, 1202b, 1202c, bearer activation/ deactivation, selecting a particular serving gateway during an initial attach of the WTRUs 1202a, 1202b, 1202c, and the like.
• the MME 1262 may also provide a control plane function for switching between the RAN 1204 and other RANs (not shown) that employ other radio technologies, such as GSM or WCDMA.
• the serving gateway 1264 may be connected to each of the eNode Bs 1260a, 1260b, 1260c in the RAN 1204 via the S 1 interface.
• the serving gateway 1264 may generally route and forward user data packets to/from the WTRUs 1202a, 1202b, 1202c.
• the serving gateway 1264 may also perform other functions, such as anchoring user planes during inter-eNode B handovers, triggering paging when downlink data is available for the WTRUs 1202a, 1202b, 1202c, managing and storing contexts of the WTRUs 1202a, 1202b, 1202c, and the like.
• the serving gateway 1264 may also be connected to the PDN gateway 1266, which may provide the WTRUs 1202a, 1202b, 1202c with access to packet-switched networks, such as the Internet 1210, to facilitate communications between the WTRUs 1202a, 1202b, 1202c and IP-enabled devices.
• PDN gateway 1266 may provide the WTRUs 1202a, 1202b, 1202c with access to packet-switched networks, such as the Internet 1210, to facilitate communications between the WTRUs 1202a, 1202b, 1202c and IP-enabled devices.
• the core network 1206 may facilitate communications with other networks.
• the core network 1206 may provide the WTRUs 1202a, 1202b, 122c with access to circuit-switched networks, such as the PSTN 1208, to facilitate communications between the WTRUs 1202a, 1202b, 1202c and traditional land- line communications devices.
• the core network 1206 may include, or may communicate with, an IP gateway (e.g., an IP multimedia subsystem (IMS) server) that serves as an interface between the core network 1206 and the PSTN 1208.
• IMS IP multimedia subsystem
• the core network 1206 may provide the WTRUs 1202a, 1202b, 1202c with access to the networks 1212, which may include other wired or wireless networks that are owned and/or operated by other service providers.
• FIG. 12E is a system diagram of the RAN 1204 and the core network 1206 according to another embodiment.
• the RAN 1204 may be an access service network (ASN) that employs IEEE 802.16 radio technology to communicate with the WTRUs 1202a, 1202b, 1202c over the air interface 1216.
• ASN access service network
• the communication links between the different functional entities of the WTRUs 1202a, 1202b, 1202c, the RAN 1204, and the core network 1206 may be defined as reference points.
• the RAN 1204 may include base stations 1270a, 1270b, 1270c, and an ASN gateway 1272, though it will be appreciated that the RAN 1204 may include any number of base stations and ASN gateways while remaining consistent with an embodiment.
• the base stations 1270a, 1270b, 1270c may each be associated with a particular cell (not shown) in the RAN 1204 and may each include one or more transceivers for communicating with the WTRUs 1202a, 1202b, 1202c over the air interface 1216.
• the base stations 1270a, 1270b, 1270c may implement MIMO technology.
• the base station 1270a may use multiple antennas to transmit wireless signals to, and receive wireless signals from, the WTRU 1202a.
• the base stations 1270a, 1270b, 1270c may also provide mobility management functions, such as handoff triggering, tunnel establishment, radio resource management, traffic classification, quality of service (QoS) policy enforcement, and the like.
• the ASN gateway 1272 may serve as a traffic aggregation point and may be responsible for paging, caching of subscriber profiles, routing to the core network 1206, and the like.
• the air interface 1216 between the WTRUs 1202a, 1202b, 1202c and the RAN 1204 maybe defined as an Rl reference point that implements the IEEE 802.16 specification.
• each of the WTRUs 1202a, 1202b, 1202c may establish a logical interface (not shown) with the core network 1206.
• the logical interface between the WTRUs 1202a, 1202b, 1202c and the core network 1206 may be defined as an R2 reference point, which may be used for authentication, authorization, IP host configuration management, and/or mobility management.
• the communication link between each of the base stations 1270a, 1270b, 1270c may be defined as an R8 reference point that includes protocols for facilitating WTRU handovers and the transfer of data between base stations.
• the communication link between the base stations 1270a, 1270b, 1270c and the ASN gateway 1272 may be defined as an R6 reference point.
• the R6 reference point may include protocols for facilitating mobility management based on mobility events associated with each of the WTRUs 1202a, 1202b, 1202c.
• the RAN 1204 may be connected to the core network 1206.
• the communication link between the RAN 124 and the core network 1206 may defined as an R3 reference point that includes protocols for facilitating data transfer and mobility management capabilities, for example.
• the core network 1206 may include a mobile IP home agent (MIP-HA) 1274, an authentication, authorization, accounting (AAA) server 1276, and a gateway 1278. While each of the foregoing elements are depicted as part of the core network 1206, it will be appreciated that any one of these elements may be owned and/or operated by an entity other than the core network operator.
• MIP-HA mobile IP home agent
• AAA authentication, authorization, accounting
• the MIP-HA 1274 may be responsible for IP address management, and may enable the WTRUs 1202a, 1202b, 1202c to roam between different ASNs and/or different core networks.
• the MIP-HA 1274 may provide the WTRUs 1202a, 1202b, 1202c with access to packet-switched networks, such as the Internet 1212, to facilitate communications between the WTRUs 1202a, 1202b, 1202c and IP-enabled devices.
• the AAA server 1276 may be responsible for user authentication and for supporting user services.
• the gateway 1278 may facilitate interworking with other networks.
• the gateway 1278 may provide the WTRUs 1202a, 1202b, 1202c with access to circuit-switched networks, such as the PSTN 1208, to facilitate communications between the WTRUs 1202a, 1202b, 1202c and traditional land-line communications devices.
• the gateway 1278 may provide the WTRUs 1202a, 1202b, 1202c with access to the networks 1212, which may include other wired or wireless networks that are owned and/or operated by other service providers.
• the RAN 1204 may be connected to other ASNs and the core network 1206 may be connected to other core networks.
• the communication link between the RAN 1204 the other ASNs may be defined as an R4 reference point, which may include protocols for coordinating the mobility of the WTRUs 1202a, 1202b, 1202c between the RAN 1204 and the other ASNs.
• the communication link between the core network 1206 and the other core networks may be defined as an R5 reference, which may include protocols for facilitating interworking between home core networks and visited core networks.
• a method may include selectively conveying one or more adaptation parameters (APs) in a video bit stream (e.g., the encoded bit stream BS) by any of (i) inserting adaptation parameters in a slice header, and (ii) inserting the adaptation parameters in a dedicated NAL unit (referred to hereinafter as a "DNU").
• APs adaptation parameters
• DNU dedicated NAL unit
• a method may include selectively conveying APs in a video bit stream (e.g., the encoded bit stream BS) using a signaling method selected from a set including (i) inserting adaptation parameters in a slice header ("a slice-header APs signaling method"), and (ii) inserting adaptation parameters in a dedicated network abstraction layer (NAL) unit ("a NAL-unit APs signaling method").
• a signaling method selected from a set including (i) inserting adaptation parameters in a slice header ("a slice-header APs signaling method"), and (ii) inserting adaptation parameters in a dedicated network abstraction layer (NAL) unit ("a NAL-unit APs signaling method").
• NAL network abstraction layer
• the selection (e.g., of the slice header, the DNU, the signaling method (collectively “(signaling-method selection”)) may be indicated by a flag, inserted to the slice header, to indicate whether the APs are obtainable from the DNU, or obtainable from a current slice header.
• the flag if set to a first value, may indicate the adaptation parameters are obtainable from the DNU.
• the flag if set to a second value, may indicate the APs are obtainable from the current slice.
• the DNU may be an adaptation parameter set (APS) NAL unit associated with an APS identifier (ID) included in the video bit stream.
• APS adaptation parameter set
• ID APS identifier
• the APS NAL unit may include at least one syntax element for identifying the APS NAL unit, and the APS ID may correspond to a value of the syntax element.
• the method may further include: making the signaling- method selection with reference to a threshold.
• the signaling-method selection may be made with reference to a threshold.
• the threshold may be based on an amount of frames between a current frame and a preceding intra frame.
• the threshold may be based on an amount of frames between intra frames.
• the threshold may be based on an amount of frames following an intra frame.
• the threshold may be based on an error likelihood associated with a transmission channel.
• the threshold may be based on one or more video statistics.
• the threshold may be based on at least one of a number of factors.
• the factors may include any of (i) an amount of frames between intra frames, (iii) an amount of frames following an intra-frame, (iii) an amount of frames between a current frame and a preceding intra frame; (iv) a video statistic, and (iv) an error likelihood associated with a transmission channel.
• the video statistics may include at least one criterion for a color component analysis.
• the video statistics may include at least one criterion for a color component analysis including any of a color saturation and hue.
• the signaling-method selection made, responsive to a color satisfying the threshold may be the slice-header APs signaling method, the NAL-unit APs signaling method, or both the slice-header APs signaling method and NAL-unit APs signaling method.
• the signaling-method selection made, responsive to a color failing to satisfy the threshold may be the slice-header APs signaling method, the NAL-unit APs signaling method, or both the slice-header APs signaling method and NAL-unit APs signaling method.
• the signaling- method selection made, responsive to a color satisfying the threshold may be the slice-header APs signaling method or the NAL-unit APs signaling method, and the signaling-method selection made, responsive to a color failing to satisfy the threshold, may be APs signaling method not selected responsive to the color satisfying the threshold.
• the signaling-method selection made, responsive to a color satisfying the threshold might not be mutually exclusive from the signaling-method selection made, responsive to the color failing to satisfy the threshold.
• the method may further include: determining whether a color satisfies or fails to satisfy the threshold.
• the method may further include: establishing the threshold.
• establishing the threshold may include: establishing the threshold based on at least one of a number of factors.
• the factors may include any of (i) an amount of frames between intra frames, (ii) an amount of frames following an intra- frame, (iii) an amount of frames between a current frame and a preceding intra frame; (iv) a video statistic, and (iv) an error likelihood associated with a transmission channel.
• the signaling-method selection may be based, in part, on a frequency of use of a (e.g., current) picture as a reference picture to code subsequent pictures.
• the method may further include: making the signaling- method selection based, in part, on a frequency of use of a (e.g., current) picture as a reference picture to code subsequent pictures.
• the signaling-method selection may be based, in part, on one or more values of one or more quantization parameters (QPs).
• QPs quantization parameters
• the signaling-method selection made based, in part, on values of QPs may depend on a threshold level of QP ("QP threshold level").
• QP threshold level a threshold level of QP
• the signaling-method selection made, for values of QPs satisfying (e.g., at or below) the QP threshold level may be the slice-header APs signaling method.
• the signaling-method selection made, for values of QPs failing to satisfy (e.g., above) the QP threshold level may be the NAL-unit APs signaling method.
• the QP threshold level may be set at a low value of QP, and may be dynamically adjusted downwards or upwards. Adjustment may be based on various conditions, such the factors noted above.
• the signaling-method selection made based, in part, on values of QPs may depend on whether the values fall within a range of values of QP ("QP value range").
• the signaling-method selection made, for values of QPs falls within the QP value range may be the slice-header APs signaling method.
• the signaling-method selection made, for values of QPs failing to fall within the QP value range may be the NAL-unit APs signaling method.
• the QP value range may include a set of low values of QP, and the set may be dynamically adjusted to expand or contract the number of values in the set and/or to shift the values downwards or upwards. Adjustment may be based on various conditions, such the factors noted above.
• the signaling-method selection made, for a picture having a low value of QP may be a slice-header APs signaling method.
• the signaling-method selection more frequently made, for a picture having a low value of QP (or for multiple pictures having low values of QPs), the selection more frequently made is inserting the adaptation parameters in a slice header.
• the method may further include: making the signaling- method selection based, in part, on one or more values of one or more QPs.
• the signaling-method selection may be based on long term reference pictures.
• the signaling-method selection made, for long term reference pictures may be the slice-header APs signaling method.
• the method may further include: making the signaling- method selection based on long term reference pictures.
• the APs may include, or be, one or more parameters used for reconstructing decoded video data.
• the APs may include, or be, one or more parameters for in-loop filtering of decoded video data.
• the APs may include, or be, one or more parameters for adapting filters for in-loop filtering of decoded video data.
• the APs may include, or be, any of (i) adaptive offset (SAO) parameters, and (ii) adaptive loop filter (ALF) parameters.
• SAO adaptive offset
• ALF adaptive loop filter
• the DNU may be an adaptation parameter set (APS) NAL unit.
• APS adaptation parameter set
• a method may include: receiving a video bit stream (e.g. the encoded video bit stream BS), and the video bit stream may include APs selectively conveyed using a signaling method selected from a set including (i) inserting adaptation parameters in a slice header (the " slice-header APs signaling method "), and (ii) inserting adaptation parameters in a DNU (the "NAL-unit APs signaling method");
• the method may further include determining the signaling-method selection.
• the method may further include obtaining the APs based on the signaling-method selection.
• the signaling-method selection may be indicated by a flag, inserted to the slice header, to indicate whether the APs are obtainable from the dedicated NAL unit, or obtainable from a current slice header.
• the flag if set to a first value, may indicate the adaptation parameters are obtainable from the DNU.
• the flag if set to a second value, may indicate the adaptation parameters are obtainable from the current slice.
• determining the signaling-method selection may include determining whether the flag is set to the first value or the second value.
• the DNU may be an adaptation parameter set (APS) NAL unit associated with an APS identifier (ID) included in the video bit stream.
• APS adaptation parameter set
• ID APS identifier
• the APS NAL unit may include at least one syntax element for identifying the APS NAL unit, and wherein the APS ID may correspond to a value of the syntax element.
• a method may include: decoupling a region of interest (ROl) slice from other slices by signaling APs in a slice header of the ROl slice, and using at least one DNU for the other slices.
• ROI region of interest
• the APs may include, or be, one or more parameters used for reconstructing decoded video data.
• the APs may include, or be, one or more parameters for in-loop filtering of decoded video data.
• the APs may include, or be, one or more parameters for adapting filters for in-loop filtering of decoded video data.
• the APs may include, or be, any of (i) adaptive offset (SAO) parameters, and (ii) adaptive loop filter (ALF) parameters.
• SAO adaptive offset
• ALF adaptive loop filter
• the APs signaled in the slice header of the ROl slice may include, or be, APs for a first combination of one or more types of in-loop filtering
• the APs signaled in the at least one DNU may include, or be, APs for a second combination of one or more types of in-loop filtering.
• the APs for multiple types of in-loop filtering may include, or be, sample adaptive offset (SAO) parameters, and (ii) adaptive loop filter (ALF) parameters.
• the APs for a particular type of in-loop filtering may include, or be, sample adaptive offset (SAO) parameters or, alternatively, adaptive loop filter (ALF) parameters.
• SAO sample adaptive offset
• ALF adaptive loop filter
• an apparatus such as the source device 102 ( Figure 2), the destination device 104 ( Figure 2), the WTRU 1202 ( Figure 12) or the like, having any of a transmitter, receiver and processor may be configured to generate and/or receive and process messages using the signaling of a method of at least one of the preceding embodiments.
• an apparatus may include a wireless transmit/receive unit (WTRU) configured to generate and/or receive and process messages using the signaling of a method of at least one of the preceding embodiments.
• WTRU wireless transmit/receive unit
• an apparatus may include a video encoding unit configured to perform a method of at least one of the preceding embodiments.
• an apparatus may include a video decoding unit configured to perform a method of at least one of the embodiments set forth in paragraphs [0253]-[0261].
• an apparatus may include a video decoding unit configured to obtain the APs selectively conveyed in a video bit stream by a process reciprocal of a method of at least one of the preceding embodiments except those set forth in paragraphs [0253]-[0261].
• an apparatus may include a video encoding unit, wherein the video encoding unit may include any of a video coding layer (VCL) and a network abstraction layer
• VCL video coding layer
• VNL network abstraction layer
• NAL NAL
• the NAL may be configured to selectively convey adaptation parameters in a video bit stream using a signaling method selected from a set including (i) inserting adaptation parameters in a slice header, and (ii) inserting adaptation parameters in a DNU.
• an apparatus may include a video encoding unit, wherein the video encoding unit may include any of a VCL and a NAL, and wherein the NAL may be configured to: signal adaptation parameters in a slice header of the ROI slice so as to decouple a region of interest (ROI) slice from other slices; and for the other slices, signal adaptation parameters in at least one dedicated NAL unit.
• the video encoding unit may include any of a VCL and a NAL
• the NAL may be configured to: signal adaptation parameters in a slice header of the ROI slice so as to decouple a region of interest (ROI) slice from other slices; and for the other slices, signal adaptation parameters in at least one dedicated NAL unit.
• ROI region of interest
• an apparatus may include a network element or base station configured to generate and/or receive and process messages using the signaling of a method of at least one of the preceding embodiments.
• a computer readable medium having stored thereon instructions that when executed by a processor may cause a system to generate and/or receive and process messages using the signaling of a method of at least one of the preceding embodiments.
• a computer readable medium having stored thereon instructions that when executed by a processor may cause a video encoding unit to perform a method of at least one of the preceding embodiments.
• a computer readable medium having stored thereon instructions that when executed by a processor may cause a video decoding unit to perform a method of at least one of the embodiments set forth in paragraphs [0253]-[0261].
• a computer readable medium having stored thereon instructions that when executed by a processor may cause a video decoding unit to obtain the APs selectively conveyed in a video bit stream by a process reciprocal of a method of at least one of the preceding embodiments except those set forth in paragraphs [0253]-[0261].
• a computer readable medium having stored thereon instructions that when executed by a processor may cause a video encoding unit, comprising any of a VCL and a NAL, to use the NAL to selectively convey APs in a video bit stream using a signaling method selected from a set including (i) inserting adaptation parameters in a slice header, and (ii) inserting adaptation parameters in a dedicated NAL unit.
• a computer readable medium having stored thereon instructions that when executed by a processor may cause a video encoding unit, comprising a VCL and NAL, to use the NAL to: signal adaptation parameters in a slice header of the ROI slice so as to decouple a region of interest (ROI) slice from other slices; and for the other slices, signal adaptation parameters in at least one dedicated NAL unit.
• a video encoding unit comprising a VCL and NAL
• ROM read only memory
• RAM random access memory
• register cache memory
• semiconductor memory devices magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks, and digital versatile disks (DVDs).
• a processor in association with software may be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer.
• an electrical system represents data bits that can cause a resulting transformation or reduction of the electrical signals and the maintenance of data bits at memory locations in a memory system to thereby reconfigure or otherwise alter the CPU's operation, as well as other processing of signals.
• the memory locations where data bits are maintained are physical locations that have particular electrical, magnetic, optical, or organic properties corresponding to or representative of the data bits. It should be understood that the exemplary embodiments are not limited to the above-mentioned platforms or CPUs and that other platforms and CPUs may support the provided methods.
• the data bits may also be maintained on a computer readable medium including magnetic disks, optical disks, and any other volatile (e.g., Random Access Memory (“RAM”)) or non-volatile (e.g., Readonly Memory (“ROM”)) mass storage system readable by the CPU.
• RAM Random Access Memory
• ROM Readonly Memory
• the computer readable medium may include cooperating or interconnected computer readable medium, which exist exclusively on the processing system or are distributed among multiple interconnected processing systems that may be local or remote to the processing system. It should be understood that the exemplary embodiments are not limited to the above- mentioned memories and that other platforms and memories may support the provided methods.
• any of followed by a listing of a plurality of items and/or a plurality of categories of items are intended to include “any of,” “any combination of,” “any multiple of,” and/or “any combination of multiples of the items and/or the categories of items, individually or in conjunction with other items and/or other categories of items.
• the term “set” is intended to include any number of items, including zero.
• the term “number” is intended to include any number, including zero.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Compression Or Coding Systems Of Tv Signals (AREA)

Applications Claiming Priority (2)

Publications (2)

Family

ID=47604246

Family Applications (1)

Country Status (1)

Cited By (11)

• 2013
  • 2013-01-14 WO PCT/US2013/021450 patent/WO2013109505A2/fr not_active Ceased

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events