HK1237171A1 - Method, device, and computer-readable storage medium for encoding and decoding video data - Google Patents

Description

Palette index grouping for high throughput CABAC coding

This application claims the benefit of U.S. provisional patent application No. 62/110,302, filed on 30/1/2015, the entire contents of which are hereby incorporated by reference herein.

Technical Field

The present disclosure relates to encoding and decoding content, and more particularly, to encoding and decoding content according to a palette-based coding mode.

Background

Digital video capabilities can be incorporated into a wide range of devices, including digital televisions, digital direct broadcast systems, wireless broadcast systems, Personal Digital Assistants (PDAs), laptop or desktop computers, tablet computers, e-book readers, digital cameras, digital recording devices, digital media players, video gaming devices, video game consoles, cellular or satellite radio telephones, so-called "smart phones," video teleconferencing devices, video streaming devices, and the like. Digital video devices implement video compression techniques such as those described in standards defined by MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264/MPEG-4, part 10 Advanced Video Coding (AVC), ITU-T H.265, High Efficiency Video Coding (HEVC), and extensions of these standards. Video devices may transmit, receive, encode, decode, and/or store digital video information more efficiently by implementing these video compression techniques.

Video compression techniques perform spatial (intra-picture) prediction and/or temporal (inter-picture) prediction to reduce or remove redundancy inherent in video sequences. For block-based video coding, a video slice (i.e., a video frame or a portion of a video frame) may be partitioned into video blocks. Video blocks in an intra-coded (I) slice of a picture are encoded using spatial prediction with respect to reference samples in neighboring blocks in the same picture. Video blocks in inter-coded (P or B) slices of a picture may use spatial prediction with respect to reference samples in neighboring blocks in the same picture or temporal prediction with respect to reference samples in other reference pictures. A picture may be referred to as a frame and a reference picture may be referred to as a reference frame.

Spatial or temporal prediction generates a predictive block for the block to be coded. The residual data represents pixel differences between the original block to be coded and the predictive block. An inter-coded block is encoded according to a motion vector that points to a block of reference samples that forms a predictive block, and residual data indicates differences between the coded block and the predictive block. An intra-coded block is encoded according to an intra-coding mode and residual data. For further compression, the residual data may be transformed from the pixel domain to the transform domain, producing residual coefficients, which may then be quantized. The quantized coefficients, initially arranged in a two-dimensional array, may be scanned in order to generate a one-dimensional vector of coefficients, and entropy coding may be applied to achieve even more compression.

Content such as images may be encoded and decoded using palette mode. In general, palette mode is a technique that involves representing content using a palette. The content may be encoded such that the content is represented by an index map that includes values corresponding to a palette. The index map may be decoded to reconstruct the content.

Disclosure of Invention

The techniques of this disclosure relate to palette-based content coding. For example, in palette-based content coding, a content coder (e.g., a content coder such as a video encoder or a video decoder) may form a "palette" as a table of colors for representing video data of a particular region (e.g., a given block). Palette-based content coding may be particularly useful, for example, for coding regions of video data having a relatively small number of colors. Rather than coding actual pixel values (or their residuals), the content coder may code, for one or more of the pixels, palette indices (e.g., index values) that relate the pixels to entries in a palette representing the colors of the pixels. The techniques described in this disclosure may include techniques for signaling various combinations of one or more of a palette-based coding mode, transmitting a palette, deriving values for non-transmitted syntax elements, transmitting palette-based coding maps and other syntax elements, predicting palette entries, coding runs of palette indices, entropy coding palette information, and various other palette coding techniques.

In one example, this disclosure describes a method of decoding video data comprising receiving a palette mode encoded block of video data of a picture from an encoded video bitstream; receiving, from an encoded video bitstream, encoded palette mode information for a palette mode encoded block of video data, wherein the encoded palette mode information includes a plurality of instances of a first syntax element and a plurality of syntax elements that are different from the first syntax element; decoding a plurality of instances of a first syntax element using bypass mode prior to decoding a plurality of syntax elements different from the first syntax element using context mode; decoding, using context mode, a plurality of syntax elements different from the first syntax element after decoding the plurality of instances of the first syntax element using bypass mode; and decoding the palette mode encoded block of video data using the decoded plurality of instances of the first syntax element and the decoded plurality of syntax elements that are different from the first syntax element.

In another example, this disclosure describes a device for decoding video data, the device comprising a memory configured to store video data; and a video decoder in communication with the memory, the video decoder configured to receive a palette mode encoded block of video data of a picture from an encoded video bitstream; receiving, from an encoded video bitstream, encoded palette mode information for a palette mode encoded block of video data, wherein the encoded palette mode information includes a plurality of instances of a first syntax element and a plurality of syntax elements that are different from the first syntax element; decoding a plurality of instances of a first syntax element using bypass mode prior to decoding a plurality of syntax elements different from the first syntax element using context mode; decoding, using context mode, a plurality of syntax elements different from the first syntax element after decoding the plurality of instances of the first syntax element using bypass mode; and decoding the palette mode encoded block of video data using the decoded plurality of instances of the first syntax element and the decoded plurality of syntax elements that are different from the first syntax element.

In another example, this disclosure describes a non-transitory computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors to receive a palette mode encoded block of video data of a picture from an encoded video bitstream; receiving, from an encoded video bitstream, encoded palette mode information for a palette mode encoded block of video data, wherein the encoded palette mode information includes a plurality of instances of a first syntax element and a plurality of syntax elements that are different from the first syntax element; decoding a plurality of instances of a first syntax element using bypass mode prior to decoding a plurality of syntax elements different from the first syntax element using context mode; decoding, using context mode, a plurality of syntax elements different from the first syntax element after decoding the plurality of instances of the first syntax element using bypass mode; and decoding the palette mode encoded block of video data using the decoded plurality of instances of the first syntax element and the decoded plurality of syntax elements that are different from the first syntax element.

In another example, this disclosure describes a method of encoding video data, the method comprising determining that a block of video data is to be coded in palette mode; encoding a block of video data into an encoded bitstream using a palette mode, wherein encoding the block of video data using the palette mode comprises: generating palette mode information for a block of video data, wherein the palette mode information includes a plurality of instances of a first syntax element and a plurality of syntax elements different from the first syntax element; encoding, using bypass mode, a plurality of instances of a first syntax element into an encoded bitstream before encoding, using context mode, a plurality of syntax elements that are different from the first syntax element into the encoded bitstream; and encoding, using the context mode, a plurality of syntax elements that are different from the first syntax element into the encoded bitstream after encoding the plurality of instances of the first syntax element into the encoded bitstream using the bypass mode.

In another example, this disclosure describes a device for encoding video data, the device comprising a memory configured to store video data; and a video encoder in communication with the memory, the video encoder configured to: determining that a block of video data stored in memory is to be encoded in palette mode; encoding a block of video data into an encoded bitstream using a palette mode, wherein a video encoder configured to encode the block of video data using the palette mode comprises the video encoder configured to: generating palette mode information for a block of video data, wherein the palette mode information includes a plurality of instances of a first syntax element and a plurality of syntax elements different from the first syntax element; encoding, using bypass mode, a plurality of instances of a first syntax element into an encoded bitstream before encoding, using context mode, a plurality of syntax elements that are different from the first syntax element into the encoded bitstream; and encoding, using the context mode, a plurality of syntax elements that are different from the first syntax element into the encoded bitstream after encoding the plurality of instances of the first syntax element into the encoded bitstream using the bypass mode.

The details of one or more examples of the invention are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the invention will be apparent from the description and drawings, and from the claims.

Drawings

FIG. 1 is a block diagram illustrating an example video coding system that may utilize the techniques described in this disclosure.

FIG. 2 is a block diagram illustrating an example video encoder that may perform the techniques described in this disclosure.

Fig. 3 is a block diagram illustrating an example video decoder that may perform the techniques described in this disclosure.

Fig. 4 is a conceptual diagram illustrating an example of determining palette entries for palette-based video coding consistent with the techniques of this disclosure.

Fig. 5 is a conceptual diagram illustrating an example of determining an index to a palette for a block of pixels consistent with the techniques of this disclosure.

FIG. 6 is a conceptual diagram illustrating an example of determining a maximum copy-over-run length, assuming a raster scan order, consistent with the techniques of this disclosure.

Fig. 7 is a table illustrating a change in coding order for syntax elements of a palette mode.

Fig. 8 is a flow diagram illustrating an example process for decoding video data consistent with the techniques for palette-based video coding of this disclosure.

Fig. 9 is a flow diagram illustrating an example process for encoding video data consistent with the techniques for palette-based video coding of this disclosure.

Detailed Description

Aspects of this disclosure are directed to techniques for content coding (e.g., video coding). In particular, this disclosure describes techniques for palette-based coding of content data (e.g., video data) and techniques for context-based adaptive binary arithmetic coding (CABAC) of palette coding information. In various examples of this disclosure, as described in more detail below, techniques of this disclosure may be directed to a process of predicting or coding a block in palette mode to improve coding efficiency and/or reduce codec complexity. For example, this disclosure describes techniques related to palette index grouping (e.g., advanced palette index grouping).

In CABAC processes (e.g., as described in d.marpe, h.schwarz, and t.wiegand, IEEE trans.cir. & sys. videotech., "context-based adaptive binary arithmetic coding in the h.264/AVC video compression standard" No. 7/2003, volume 13), there are two modes: (1) bypass mode and (2) context mode. In bypass mode, there is no context update procedure. Thus, bypass mode may achieve higher data throughput than context-based mode by employing hardware or ISA level parallelism. This benefit of the bypass mode becomes greater as the number of bypass binary numbers that can be processed together increases.

In current palette mode coding designs, "High Efficiency Video Coding (HEVC) screen content coding as r.joshi and j.xu: as described in draft 2, "JCTVC-S1005, in screen content coding, syntax elements of palette _ index _ idc and palette _ escape _ val are CABAC bypass mode coded and interleaved with other syntax elements (e.g., palette _ run _ msb _ id _ plus1) coded by CABAC context mode. This disclosure describes techniques to group syntax elements coded with bypass mode together. As used herein, "bypass-mode coded" and "context-mode coded" may be interchanged with "bypass-coded" and "context-coded", respectively.

As used herein, examples of the term "content" may change to the term "video" and examples of the term "video" may change to the term "content". This is true regardless of whether the term "content" or "video" is used as an adjective, noun, or other part of a word. For example, a reference to a "content coder" also includes a reference to a "video coder," and a reference to a "video coder" also includes a reference to a "content coder. Similarly, references to "content" also include references to "video" and references to "video" also include references to "content".

As used herein, "content" refers to any type of content. For example, "content" may refer to video, screen content, images, any graphical content, any displayable content, or any data corresponding thereto (e.g., video data, screen content data, image data, graphical content data, displayable content data, and the like).

As used herein, the term "video" may refer to screen content, movable content, a plurality of images that may be presented in sequence, or any data corresponding thereto (e.g., screen content data, movable content data, video data, image data, and the like).

As used herein, the term "image" may refer to a single image, one or more images of a plurality of images corresponding to a video, one or more images of a plurality of images not corresponding to a video, a plurality of images corresponding to a video (e.g., corresponding to all images of a video or to less than all images of a video), a subsection of a single image, a plurality of subsections corresponding to a plurality of images, one or more graphics primitives, image data, graphics data, and the like.

In conventional video coding, it is assumed that the image is continuous-tone and spatially smooth. Based on these assumptions, various tools such as block-based transform, filtering, and other coding tools have been developed and have demonstrated good performance for natural content video. However, in applications like remote desktop computers, collaborative and wireless displays, computer generated screen content may be the primary content to be compressed. This type of screen content tends to have discrete tones, steep lines, and high contrast object boundaries. The assumptions of continuous tone and smoothness may no longer apply, and thus conventional video coding techniques may be inefficient in compressing content (e.g., screen content).

In one example of palette-based video coding, a video encoder may encode a block of video data by determining a palette for the block (e.g., explicitly coding the palette, predicting the palette, or a combination thereof), locating an entry in the palette to represent a value of one or more pixels, and encoding both the palette and the block using index values that indicate entries in the palette for representing pixel values of the block. In some examples, a video encoder may signal palette and/or index values in an encoded bitstream. In turn, the video decoder may obtain, from the encoded bitstream, a palette for a block, as well as index values for individual pixels of the block. A video decoder may correlate index values of pixels with entries of a palette to reconstruct various pixel values of a block.

For example, a particular region of video data may be assumed to have a relatively small number of colors. A video coder (e.g., a video encoder or a video decoder) may code (e.g., encode or decode) a so-called "palette" to represent a particular region of video data. The palette may be expressed as an index (e.g., a table) of colors or pixel values representing the video data of a particular region (e.g., a given block). The video coder may code an index that correlates one or more pixel values with appropriate values in the palette. Each pixel may be associated with an entry in the palette that represents the color of the pixel. For example, a palette may include the most dominant pixel values in a given block. In some cases, the most dominant pixel values may include one or more pixel values that occur most frequently within the block. Additionally, in some cases, the video coder may apply a threshold value to determine whether a pixel value should be included as one of the most dominant pixel values in the block. According to various aspects of palette-based coding, a video coder may code index values that indicate one or more of the pixel values of a current block, rather than coding actual pixel values or a residual thereof for the current block of video data. In the case of palette-based coding, the index values indicate corresponding entries in the palette that are used to represent individual pixel values of the current block. The above description is intended to provide an overview of palette-based video coding.

Palette-based coding may be particularly suitable for screen-generated content coding or other content in which one or more conventional coding tools are inefficient. The techniques for palette-based coding of video data may be used with one or more other coding techniques, such as techniques for inter or intra predictive coding. For example, as described in more detail below, an encoder or decoder or a combined encoder-decoder (codec) may be configured to perform inter and intra predictive coding, as well as palette-based coding.

In some examples, palette-based coding techniques may be configured to be used with one or more video coding standards. For example, High Efficiency Video Coding (HEVC) is a new video coding standard developed by the joint collaborative group of video coding (JCT-VC) of the ITU-T Video Coding Experts Group (VCEG) and the ISO/IEC Moving Picture Experts Group (MPEG). The finalized HEVC standard document was published by the telecommunication standardization sector of the International Telecommunication Union (ITU) in 2013 in 4 as "ITU-T h.265, series H: audio-visual AND MULTIMEDIA system Infrastructure for audio-visual services-decoding of motion video-High efficiency video decoding (SERIES H: Audio video AND MULTIMEDIA System information Coding of audio visual services) ".

To provide more efficient coding of screen generated content, JCT-VC will be developed to extend to the HEVC standard, referred to as the HEVC Screen Content Coding (SCC) standard. A new working draft of the HEVC SCC standard, called "HEVC SCC draft 2" or "WD 2", is described in the document JCTVC-S1005 "HEVC screen content coding draft letter 2" of R.Joshi and J.xu (Union team on video coding (JCT-VC) of ITU-T SG16WP 3 and ISO/IEC JTC 1/SC 29/WG 11, conference 19: France, St.Lasburg, 10 months, 17 days to 24 days 2014).

With the HEVC framework, as an example, palette-based coding techniques may be configured to be used as Coding Unit (CU) modes. In other examples, palette-based coding techniques may be configured to be used as a Prediction Unit (PU) mode in the framework of HEVC. Accordingly, all of the following disclosed processes described in the context of CU mode may additionally or alternatively apply to PU. However, these HEVC-based examples should not be viewed as constraining or limiting the palette-based coding techniques described herein, as such these techniques may be applied to work independently or as part of other existing or yet-to-be developed systems/standards. In these cases, the unit for palette coding may be a square block, a rectangular block, or even a non-rectangular shaped region.

In some examples, a palette may be derived from one or more CUs, PUs, or any region of data (e.g., any block of data). For example, a palette may comprise (and may consist of): the most dominant pixel value in the current CU, where for this particular example, the CU is a region of data. The size and elements of the palette are first transmitted from the video encoder to the video decoder. The size and/or elements of the palette may be directly coded or predictively coded using the size and/or elements of the palette in a neighboring CU (e.g., a coded CU above and/or to the left). Thereafter, pixel values in the CU are encoded based on the palette according to a particular scanning order. For each pixel position in the CU, a flag (e.g., palette _ flag or escape _ flag) may be first transmitted to indicate whether the pixel value is included in the palette. For those pixel values that map to an entry in the palette, the palette index associated with that entry is signaled for a given pixel position in the CU. For those pixel values that do not exist in the palette, special indices may be allocated to the pixels and the actual pixel values (possibly in quantized form) may be transmitted for a given pixel location in the CU, rather than sending a flag (e.g., a palette _ flag or escape _ flag). These pixels are referred to as "escape pixels". Escape pixels may be coded using any existing entropy coding method (e.g., fixed length coding, unary coding, etc.). In some examples, one or more techniques described herein may utilize a flag such as a palette _ flag or escape _ flag. In other examples, one or more techniques described herein may not utilize a flag such as a palette _ flag or escape _ flag.

The samples in a block of video data may be processed (e.g., scanned) using a horizontal raster scan order or other scan order. For example, a video encoder may convert a two-dimensional block of palette indices into a one-dimensional array by scanning the palette indices using a horizontal raster scan order. Likewise, the video decoder may reconstruct the blocks of palette indices using horizontal raster scan order. Accordingly, this disclosure may refer to the previous samples as samples prior to the currently coded sample in the block in scan order. It should be appreciated that scans other than horizontal raster scanning (e.g., vertical raster scan order) may also be applicable. The above examples, as well as other examples set forth in this disclosure, are intended to provide an overview of palette-based video coding.

FIG. 1 is a block diagram illustrating an example video coding system 10 that may utilize techniques of this disclosure. As used herein, the term "video coder" generally refers to both video encoders and video decoders. In this disclosure, the term "video coding" or "coding" may generally refer to video encoding or video decoding. Video encoder 20 and video decoder 30 of video coding system 10 represent examples of devices that may be configured to perform for palette-based video coding and entropy coding (e.g., CABAC) in accordance with various examples described in this disclosure. For example, video encoder 20 and video decoder 30 may be configured to selectively code various blocks of video data, such as CUs or PUs in HEVC coding, using palette-based coding or non-palette-based coding. Non-palette-based coding modes may refer to various inter-predictive temporal coding modes or intra-predictive spatial coding modes, such as the various coding modes specified by the HEVC standard.

As shown in fig. 1, video coding system 10 includes a source device 12 and a destination device 14. Source device 12 generates encoded video data. Accordingly, source device 12 may be referred to as a video encoding device or a video encoding apparatus. Destination device 14 may decode the encoded video data generated by source device 12. Destination device 14 may, therefore, be referred to as a video decoding device or a video decoding apparatus. Source device 12 and destination device 14 may be examples of video coding devices or video coding apparatuses.

Source device 12 and destination device 14 may comprise a wide range of devices, including desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, handheld phones such as so-called "smart" phones, televisions, cameras, display devices, digital media players, video game consoles, in-car computers (in-car computers), or the like.

Destination device 14 may receive encoded video data from source device 12 via channel 16. Channel 16 may comprise one or more media or devices capable of moving encoded video data from source device 12 to destination device 14. In one example, channel 16 may comprise one or more communication media that enable source device 12 to transmit encoded video data directly to destination device 14 in real-time. In this example, source device 12 may modulate the encoded video data according to a communication standard, such as a wireless communication protocol, and may transmit the modulated video data to destination device 14. The one or more communication media may include wireless and/or wired communication media such as a Radio Frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network, such as a local area network, a wide area network, or a global network (e.g., the internet).

In another example, channel 16 may include a storage medium that stores encoded video data generated by source device 12. In this example, destination device 14 may access the storage medium, such as via disk access or card access. The storage medium may comprise a variety of locally-accessed data storage media such as blu-ray discs, DVDs, CD-ROMs, flash memory, or other suitable digital storage media for storing encoded video data.

In yet another example, channel 16 may include a file server or another intermediate storage device that stores the encoded video data generated by source device 12. In this example, destination device 14 may access encoded video data stored at a file server or other intermediate storage device via streaming or download. The file server may be a server of the type capable of storing encoded video data and transmitting the encoded video data to destination device 14. Example file servers include web servers (e.g., for a website), File Transfer Protocol (FTP) servers, Network Attached Storage (NAS) devices, and local disk drives.

Destination device 14 may access the encoded video data via a standard data connection, such as an internet connection. Example types of data connections may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., DSL, cable modem, etc.), or a combination of both, suitable for accessing encoded video data stored on a file server. The transmission of the encoded video data from the file server may be a streaming transmission, a download transmission, or a combination of both.

Source device 12 and destination device 14 may be configured to perform palette-based coding and entropy coding (e.g., CABAC) consistent with this disclosure. However, the techniques of this disclosure for palette-based coding or CABAC are not limited to wireless applications or settings. The techniques may be applicable to, for example, video coding to support a variety of multimedia applications (e.g., over the internet such as over-the-air television broadcasts, cable television transmissions, satellite television transmissions, streaming video transmissions), encoding for video data stored on a data storage medium, decoding of video data stored on a data storage medium, or other applications. In some examples, video coding system 10 may be configured to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcasting, and/or video telephony.

The video coding system 10 shown in fig. 1 is merely an example, and the techniques of this disclosure may be applied to video coding settings (e.g., video encoding or video decoding) that do not necessarily include any data communication between an encoding device and a decoding device. In other examples, the data is retrieved from local memory or the like that is streamed over a network. A video encoding device may encode and store data to memory and/or a video decoding device may retrieve data from memory and decode data. In many examples, encoding and decoding are performed by devices that do not communicate with each other, but only encode data to memory and/or retrieve and decode data from memory.

In the example of fig. 1, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. In some examples, output interface 22 may include a modulator/demodulator (modem) and/or a transmitter. Video source 18 may include a video capture device such as a video camera, a video archive containing previously captured video data, a video feed interface to receive video data from a video content provider, and/or a computer graphics system for generating video data, or a combination of these sources of video data.

Video encoder 20 may encode video data from video source 18. In some examples, source device 12 transmits the encoded video data directly to destination device 14 via output interface 22. In other examples, the encoded video data may also be stored on a storage medium or file server for later access by destination device 14 for decoding and/or playback.

In the example of fig. 1, destination device 14 includes an input interface 28, a video decoder 30, and a display device 32. In some examples, input interface 28 includes a receiver and/or a modem. Input interface 28 may receive encoded video data over channel 16. The display device 32 may be integrated with the destination device 14 or external to the destination device 14. In general, display device 32 displays the decoded video data. The display device 32 may include various display devices, such as a Liquid Crystal Display (LCD), a plasma display, an Organic Light Emitting Diode (OLED) display, or another type of display device.

This disclosure may generally refer to video encoder 20 "signaling" or "transmitting" certain information to another device, such as video decoder 30. The terms "signaling" or "transmitting" may generally refer to communication of syntax elements and/or other data used to decode compressed video data. This communication may occur in real time or near real time. Alternatively, such communication may occur over a span of time, e.g., such communication may occur when a syntax element is stored to a computer-readable storage medium in an encoded bitstream at an encoding time, which may then be retrieved by a decoding device at any time after storage of such medium. Thus, while video decoder 30 may be referred to as "receiving" certain information, the reception of the information does not necessarily occur in real time or near real time and may be retrieved from the media at some time after storage.

Video encoder 20 and video decoder 30 may each be implemented as any of a variety of suitable circuits, such as one or more microprocessors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), discrete logic, hardware, or any combinations thereof. If the techniques are implemented in part in software, the device may store instructions for the software in a suitable non-transitory computer-readable storage medium and may execute the instructions in hardware using one or more processors to perform the techniques of this disclosure. Any of the above, including hardware, software, a combination of hardware and software, etc., can be considered one or more processors. Each of video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder/decoder (CODEC) in a corresponding device.

In some examples, video encoder 20 and video decoder 30 operate according to a video compression standard such as and described in the HEVC standard mentioned above. In addition to the basic HEVC standard, there is a continuing effort to generate scalable video coding, multiview video coding, and 3D coding extensions for HEVC. In addition, palette-based coding modes (e.g., as described in this disclosure) may be provided for extending the HEVC standard. In some examples, the techniques described in this disclosure for palette-based coding may be applicable to encoders and decoders configured to operate according to other video coding standards. Thus, for purposes of example, application of palette-based coding modes for coding of Coding Units (CUs) or Prediction Units (PUs) in HEVC codecs is described.

In HEVC and other video coding standards, a video sequence typically includes a series of pictures. Pictures may also be referred to as "frames". The picture may include three sample arrays, denoted as S_L、S_CbAnd S_Cr。S_LIs a two-dimensional array (i.e., block) of luma samples. S_CbIs a two-dimensional array of Cb chroma samples. S_CrIs a two-dimensional array of Cr chroma samples. Chroma samples may also be referred to herein as "chroma" samples. In other cases, a picture may be monochrome and may include only an array of luma samples.

To generate an encoded representation of a picture, video encoder 20 may generate a set of Coding Tree Units (CTUs). Each of the CTUs may be a coding tree block of luma samples, two corresponding coding tree blocks of chroma samples, and syntax structures used to code the samples of the coding tree blocks. The coding tree block may be an nxn block of samples. A CTU may also be referred to as a "tree block" or a "largest coding unit" (LCU). The CTU of HEVC may be broadly similar to macroblocks of other standards such as h.264/AVC. However, the CTUs are not necessarily limited to a particular size and may include one or more Coding Units (CUs). A slice may include an integer number of CTUs ordered consecutively in a raster scan. A coded slice may include a slice header and slice data. A slice header of a slice may be a syntax structure that includes syntax elements that provide information about the slice. The slice data may include coded CTUs of the slice.

This disclosure may use the terms "video unit" or "video block" or "block" to refer to one or more blocks of samples and syntax structures used to code the samples of the one or more blocks of samples. Example types of video units or blocks may include CTUs, CUs, PUs, Transform Units (TUs), macroblocks, macroblock partitions, and so forth. In some cases, the discussion of a PU may be interchanged with that of a macroblock or macroblock partition.

To generate a coded CTU, video encoder 20 may recursively perform quadtree partitioning on a coding tree block of the CTU to divide the coding tree block into coding blocks, hence the name "coding tree unit. The coded block is an nxn block of samples. A CU may be a coding block of luma samples and two corresponding coding blocks of chroma samples of a picture having an array of luma samples, an array of Cb samples, and an array of Cr samples, as well as syntax structures used to code the samples of the coding block. Video encoder 20 may partition the coding block of the CU into one or more prediction blocks. A prediction block may be a rectangular (i.e., square or non-square) block of samples to which the same prediction applies. A Prediction Unit (PU) of a CU may be a prediction block of luma samples of a picture, two corresponding prediction blocks of chroma samples of the picture, and syntax structures used to predict the prediction block samples. Video encoder 20 may generate, for a luma prediction block, a Cb prediction block, and a Cr prediction block for each PU of the CU, a predictive luma block, a predictive Cb block, and a predictive Cr block.

Video encoder 20 may use intra prediction or inter prediction to generate the predictive blocks for the PU. If video encoder 20 generates the predictive blocks of the PU using intra prediction, video encoder 20 may generate the predictive blocks of the PU based on decoded samples of a picture associated with the PU.

If video encoder 20 uses inter prediction to generate the predictive blocks for the PU, video encoder 20 may generate the predictive blocks for the PU based on decoded samples of one or more pictures other than the picture associated with the PU. Video encoder 20 may use uni-prediction or bi-prediction to generate the predictive blocks for the PU. When video encoder 20 uses uni-prediction to generate the predictive blocks for the PU, the PU may have a single Motion Vector (MV). When video encoder 20 uses bi-prediction to generate the predictive blocks for a PU, the PU may have two MVs.

After video encoder 20 generates predictive blocks (e.g., predictive luma blocks, predictive Cb blocks, and predictive Cr blocks) for one or more PUs of the CU, video encoder 20 may generate residual blocks for the CU. Each sample in the residual block of the CU may indicate a difference between a sample in a predictive block of a PU of the CU and a corresponding sample in a coding block of the CU. For example, video encoder 20 may generate a luma residual block for a CU. Each sample in the luma residual block of the CU indicates a difference between luma samples in one of the predictive luma blocks of the CU and corresponding samples in the original luma coding block of the CU. In addition, video encoder 20 may generate a Cb residual block for the CU. Each sample in the Cb residual block of the CU may indicate a difference between the Cb sample in one of the predictive Cb blocks of the CU and the corresponding sample in the original Cb coding block of the CU. Video encoder 20 may also generate a Cr residual block for the CU. Each sample in the Cr residual block of the CU may indicate a difference between a Cr sample in one of the predictive Cr blocks of the CU and a corresponding sample in the original Cr coding block of the CU.

Moreover, video encoder 20 may decompose residual blocks of the CU (e.g., luma, Cb, and Cr residual blocks) into one or more transform blocks (e.g., luma, Cb, and Cr transform blocks) using quad-tree partitioning. The transform block may be a rectangular block of samples to which the same transform is applied. A Transform Unit (TU) of a CU may be a transform block of luma samples, two corresponding transform blocks of chroma samples, and syntax structures used to transform the transform block samples. Thus, each TU of a CU may be associated with a luma transform block, a Cb transform block, and a Cr transform block. A luma transform block associated with a TU may be a sub-block of a luma residual block of a CU. The Cb transform block may be a sub-block of a Cb residual block of the CU. The Cr transform block may be a sub-block of a Cr residual block of the CU.

Video encoder 20 may apply one or more transforms to the transform block to generate a coefficient block for the TU. The coefficient block may be a two-dimensional array of transform coefficients. The transform coefficients may be scalar. For example, video encoder 20 may apply one or more transforms to a luma transform block of a TU to generate a luma coefficient block of the TU. Video encoder 20 may apply one or more transforms to a Cb transform block of a TU to generate a Cb coefficient block of the TU. Video encoder 20 may apply one or more transforms to a Cr transform block of a TU to generate a Cr coefficient block of the TU.

After generating the coefficient block (e.g., the luma coefficient block, the Cb coefficient block, or the Cr coefficient block), video encoder 20 may quantize the coefficient block. Quantization generally refers to the process of quantizing transform coefficients to possibly reduce the amount of data used to represent the transform coefficients, providing further compression. After video encoder 20 quantizes the coefficient block, video encoder 20 may entropy encode syntax elements indicating the quantized transform coefficients. For example, video encoder 20 may perform Context Adaptive Binary Arithmetic Coding (CABAC) on syntax elements that indicate quantized transform coefficients.

In the case of CABAC, as an example, video encoder 20 and video decoder 30 may select a probability model (also referred to as a context model) to code symbols associated with a block of video data based on context. For example, the context model (Ctx) may be an index or a delta applied to select one of a plurality of different contexts, each of which may correspond to a particular probability model. Thus, a different probabilistic model is typically defined for each context. After encoding or decoding the bins, the probability model is further updated based on the values of the bins to reflect the most recent probability assessment for the bins. For example, the probabilistic model may be maintained as a state in a finite state machine. Each particular state may correspond to a particular likelihood value. The next state corresponding to the update of the probability model may depend on the value of the current bin (e.g., the current coded bin). Thus, the selection of the probability model may be affected by the value of the previously coded binary, as the value is at least partially indicative of the probability that the binary has a given value. The context coding process described above may be generally referred to as a context adaptive coding mode.

Thus, video encoder 20 may encode the target symbol using a probability model. Likewise, video decoder 30 may parse the target symbol using the probability model. In some cases, video encoder 20 may code the syntax element using a combination of context adaptive coding and non-context adaptive coding. For example, video encoder 20 may context code binaries by selecting a probability model or "context model" that operates on context to code some binaries. In contrast, for other binaries, video encoder 20 may bypass coding the binary by bypassing or omitting conventional arithmetic coding processes when coding the binary. In these examples, video encoder 20 may bypass coding the bins using a fixed probability model. That is, bypass coded binaries do not include context or probability updates.

Video encoder 20 may output a bitstream that includes the entropy-encoded syntax elements. The bitstream may also include syntax elements that are not entropy encoded. The bitstream may include a sequence of bits that form a representation of coded pictures and associated data. The bitstream may include a sequence of Network Abstraction Layer (NAL) units. Each of the NAL units includes a NAL unit header and encapsulates a Raw Byte Sequence Payload (RBSP). The NAL unit header may include a syntax element indicating a NAL unit type code. The NAL unit type code specified by the NAL unit header of the NAL unit indicates the type of the NAL unit. An RBSP may be a syntax structure containing an integer number of bytes encapsulated within a NAL unit. In some cases, the RBSP includes a zero bit.

Different types of NAL units may encapsulate different types of RBSPs. For example, a first type of NAL unit may encapsulate RBSPs of a Picture Parameter Set (PPS), a second type of NAL unit may encapsulate RBSPs of a coded slice, a third type of NAL unit may encapsulate RBSPs of Supplemental Enhancement Information (SEI), and so on. The NAL units that encapsulate the RBSP of the video coding data (as opposed to the RBSP of the parameter set and SEI message) may be referred to as Video Coding Layer (VCL) NAL units.

Video decoder 30 may receive a bitstream generated by video encoder 20. Furthermore, video decoder 30 may parse the bitstream to decode the syntax elements from the bitstream. Video decoder 30 may reconstruct pictures of the video data based at least in part on syntax elements decoded from the bitstream. The process of reconstructing the video data may be substantially reciprocal to the process performed by video encoder 20. For example, video decoder 30 may use the MVs of the PUs to determine predictive blocks of inter-predicted PUs of the current CU. Likewise, video decoder 30 may generate intra-predicted blocks for PUs of the current CU. In addition, video decoder 30 may inverse quantize transform coefficient blocks associated with TUs of the current CU. Video decoder 30 may perform an inverse transform on the transform coefficient blocks to reconstruct the transform blocks associated with the TUs of the current CU. Video decoder 30 may reconstruct the coding blocks of the current CU by adding samples of predictive blocks of PUs of the current CU to corresponding residual values obtained from inverse quantization and inverse transformation of transform blocks of TUs of the current CU. By reconstructing the coding blocks of each CU of a picture, video decoder 30 may reconstruct the picture.

In some examples, video encoder 20 and video decoder 30 may be configured to perform palette-based coding. For example, in palette-based coding, rather than performing the intra-prediction or inter-prediction coding techniques described above, video encoder 20 and video decoder 30 may code a so-called palette as a table of color or pixel values that represents video data of a particular region (e.g., a given block). In this way, the video coder may code index values for one or more of the pixel values of the current block, rather than coding the actual pixel values of the current block of video data or its residuals, where the index values indicate entries in the palette that are used to represent the pixel values of the current block.

For example, video encoder 20 may encode a block of video data by determining a palette for the block, locating entries in the palette to represent values for each pixel, and encoding the palette and index values of the pixels that correlate pixel values with the palette. Video decoder 30 may obtain, from the encoded bitstream, a palette for the block, as well as index values for the pixels of the block. Video decoder 30 may match the index values of the individual pixels with entries of the palette to reconstruct the pixel values of the block. In the event that the index values associated with an individual pixel do not match any index values of the corresponding palette of the block, video decoder 30 may identify such pixel as an escape pixel for purposes of palette-based coding.

As described in more detail below, the basic idea of palette-based coding is: for a given block of video data to be coded, video encoder 20 may derive a palette that includes the most dominant pixel values in the current block. For example, a palette may refer to a number of pixel values that are determined or assumed to be primary and/or representative of the current CU. Video encoder 20 may first transmit the size and elements of the palette to video decoder 30. In addition, video encoder 20 may encode pixel values in a given block according to a particular scan order. For each pixel included in a given block, video encoder 20 may signal an index value that maps the pixel value to a corresponding entry in the palette. A pixel is defined as an "escape pixel" if the pixel value is not included in the palette (i.e., there are no palette entries specifying the particular pixel value of the palette coded block). According to palette-based coding, video encoder 20 may encode and signal index values reserved for escape pixels. In some examples, video encoder 20 may also encode and signal pixel values (or quantized versions thereof) for escape pixels included in a given block. For example, video decoder 30 may be configured to determine whether a pixel value matches or is otherwise close to a palette entry based on a distortion metric (e.g., MSE, SAD, and the like).

Upon receiving the encoded video bitstream signaled by video encoder 20, video decoder 30 may first determine the palette based on information received from video encoder 20. Video decoder 30 may then map the received index values associated with the pixel locations in the given block to entries of the palette to reconstruct the pixel values of the given block. In some cases, video decoder 30 may determine that the pixels of the palette coded block are escape pixels, e.g., by determining that the pixels are palette coded by the index values reserved for the escape pixels. In the case where video decoder 30 identifies escape pixels in palette coded blocks, video decoder 30 may receive pixel values (or quantized versions thereof) for the escape pixels included in a given block. Video decoder 30 may reconstruct the palette coded block by mapping individual pixel values to corresponding palette entries and by using the pixel values (or quantized versions thereof) to reconstruct any escape pixels included in the palette coded block.

As stated above, in an example palette coding mode, a palette may include entries numbered by indices. Each entry may represent a color component value or intensity (e.g., in a color space such as YCbCr, RGB, YUV, CMYK, or other format) that may be used as a predictor of a block or a final reconstructed block sample. The Palette may contain entries copied from the predictor Palette as described in the standard filing file JCTVC-Q0094(Wei Pu et al, "AHG 10: suggested software for Palette Coding based on RExt6.0 (Suggested software for Palette Coding based on RExt6.0)" JCTVC-Q0094, Balsamine, Spanish, Valencia, 3 months 27 days 2014 to 4 months 4 days 2014). The predictor palette may include palette entries from blocks previously coded using the palette mode or from other reconstructed samples. For each entry in the predictor palette, a binary flag is sent to indicate whether the entry is copied to the current palette (indicated by flag ═ 1). This is called a binary palette predictor vector. Additionally, the current palette may include (e.g., consist of) the explicitly signaled new entry. The number of new entries may also be signaled.

As another example, in palette mode, a palette may include entries numbered by indices representing color component values that may be used as predictors for block samples or as finally reconstructed block samples. Each entry in the palette may contain, for example, one luma component (e.g., luma value), two chroma components (e.g., two chroma values), or three color components (e.g., RGB, YUV, etc.). The previously decoded palette entries may be stored in a list. For example, such a list may be used to predict palette entries in the current palette mode CU. The binary prediction vector may be signaled in bit streams to indicate which entries in the list are reused in the current palette. In some examples, run-length coding may be used to compress the binary palette predictor. For example, run length values may be coded using an exponential Golomb code of order 0 (Exp-Golomb code).

In this disclosure, it will be assumed that each palette entry specifies the values of all color components of a sample. However, the concepts of the present disclosure are applicable to using separate palettes and/or separate palette entries for each color component. Also, assume that samples in a block are processed using a horizontal raster scan order. However, other scans, such as a vertical raster scan order, may also be suitable. As mentioned above, a palette may contain predicted palette entries (e.g., predicted from the palette used to code the aforementioned block) and new entries that are specifically and explicitly signaled for the current block. The encoder and decoder may know the number of predicted and new palette entries and their sum may indicate the total palette size in the block.

As proposed in the example of JCTVC-Q0094 referenced above, each sample in a block coded using a palette may belong to one of three modes, as described below:

● escape the mode. In this mode, sample values are not included as palette entries into the palette, and quantized sample values are explicitly signaled for all color components. The situation is similar to the signaling of new palette entries, although the color component values are not quantized for the new palette entries.

● CopyAbove mode (also called CopyFromTop mode). In this mode, the palette entry index for the current sample is copied from the sample located directly above the current sample in the block of samples. In other examples, for an above copy mode, a block of video data may be transposed such that samples above the block are actually samples to the left of the block.

● value pattern (also referred to as index pattern). In this mode, the value of the palette entry index is explicitly signaled.

As described herein, the palette entry index may be referred to as a palette index or simply an index. These terms are used interchangeably to describe the techniques of the present invention. In addition, as described in more detail below, the palette index may have one or more associated color or intensity values. For example, the palette index may have a single associated color or intensity value associated with a single color or intensity component of the pixel (e.g., the red component of RGB data, the Y component of YUV data, or the like). In another example, a palette index may have a plurality of associated color or intensity values. In some cases, palette-based video coding may be applied to code monochrome video. Thus, "color value" may generally refer to any color or achromatic component used to generate a pixel value.

The run value may indicate a run of palette indices coded using the same palette coding mode. For example, with respect to a value mode, a video coder (e.g., video encoder 20 or video decoder 30) may code an index value and a run value that indicates a plurality of consecutive subsequent samples in a scan order that have the same index value and are coded using palette indices. With respect to CopyAbove mode, a video coder may code an indication that an index value of a current sample value is the same as an index value of an above-neighboring sample (e.g., a sample positioned above the sample is currently coded in a block) and a run value indicating a number of consecutive subsequent samples in scan order that the index value is also copied from the above-neighboring sample. Thus, in the above example, a run of palette index values refers to a run of palette values having the same value or a run of index values copied from an upper adjacent sample.

Thus, a run may specify, for a given pattern, the number of subsequent samples belonging to the same pattern. In some cases, signaling the index value and the run value may be similar to run length coding. In an example for purposes of illustration, strings of consecutive palette index values corresponding to a block of indexes of a block of video data may be 0,2,2,2,2, 5. Each index value corresponds to a sample in a block of video data. In this example, the video coder may code the second sample (e.g., the first palette index value of "2") using a value mode. After coding the index value of 2, the video coder may code a run of 3 indicating that three subsequent samples also have the same palette index value of 2. In a similar manner, coding a run of four palette indices after coding the indices using CopyAbove mode may indicate: a total of five palette indices are copied from corresponding palette index values in the row above the current coded sample position.

Using palettes, video encoder 20 and/or video decoder 30 may be configured to code a block of samples (e.g., a block of video data) into a block of indices, where a block of indices is a block that includes index values mapped to one or more palette entries and, in some examples, one or more escape pixel values. Video encoder 20 may be configured to entropy encode the block of indices to compress the block of indices. Similarly, video decoder 30 may be configured to entropy decode the encoded block of indices to generate a block of indices from which video decoder 30 may generate a block of samples (e.g., a block of video data encoded by encoder 20). For example, run-length based entropy coding may be used to compress and decompress blocks of indexes. In some examples, video encoder 20 and video decoder 30 may be configured to entropy encode and decode the block of indices using CABAC, respectively.

To apply CABAC coding to information (e.g., syntax elements, index blocks such as index values of the index blocks, or other information), video coders (e.g., video encoder 20 and video decoder 30) may perform binarization on the information. Binarization refers to the process of converting information into a series of one or more bits. Each series of one or more bits may be referred to as a "binary". Binarization is a lossless process and may include one or a combination of the following coding techniques: fixed length coding, unary coding, truncated Rice (Rice) coding, golomb coding, exponential golomb coding, golomb-Rice coding, any form of golomb coding, any form of Rice coding, and any form of entropy coding. For example, binarization may include representing integer value 5 as 00000101 using an 8-bit fixed length technique, or representing integer value 5 as 11110 using a unary coding technique.

After binarization, the video coder may identify the coding context. The coding context may identify a probability of coding a bin having a particular value. For example, the coding context may indicate a 0.7 probability of coding a 0-valued binary and a 0.3 probability of coding a 1-valued binary. After identifying the coding context, the video coder may arithmetically code the binary based on the context, which is known as context mode coding. Binaries coded using CABAC context mode coding may be referred to as "context binaries.

Moreover, rather than performing context mode coding on all bins, video coders (e.g., video encoder 20 and video decoder 30) may use bypass CABAC coding (e.g., bypass mode coding) to code some bins. Bypass mode coding refers to a process of arithmetically coding a binary without using adaptive contexts (e.g., coding contexts). That is, the bypass coding engine does not select a context, and can assume that the probability of both symbols (0 and 1) is 0.5. Although bypass mode coding may be less bandwidth efficient than context mode coding, it may be computationally less costly when bypass mode coding is performed on the bins rather than context mode coding. Furthermore, performing bypass mode coding may allow for higher parallelism and throughput. Binaries that are coded using bypass mode coding may be referred to as "bypass binaries".

Video encoder 20 and video decoder 30 may be configured with CABAC coders (e.g., CABAC encoder and CABAC decoder, respectively). A CABAC coder may include a context mode coding engine to perform CABAC context mode coding and a bypass mode coding engine to perform bypass mode coding. If a binary is context mode coded, a context mode coding engine is used to code such a binary. A context mode coding engine may require more than two processing cycles to code a single binary. However, due to proper pipeline design, the context mode coding engine may only require n + M cycles to encode n binaries, where M is the additional load of the starting pipeline. M is typically greater than 0.

At the beginning of the CABAC coding process (i.e., each transition from bypass mode to context mode and vice versa), pipeline overhead is introduced. If a binary is bypass mode coded, a bypass mode coding engine is used to code this binary. The bypass mode coding engine may expect to require only one cycle to do with n bits of information, where n may be greater than one. Thus, if all bypass binaries within a set of bypass and context binaries are coded together and all context binaries within the set are coded together, the total number of cycles to code the set of bypass and context binaries may be reduced. In particular, coding bypass binaries together before or after transitioning to context mode coding may save the additional load required to restart the context mode coding engine. For example, video encoder 20 and video decoder 30 may be configured to switch between bypass mode to context mode when encoding or decoding a block of video data using palette mode, respectively. In another example, video encoder 20 and video decoder 30 may be configured to reduce the number of times an encoding or decoding process switches between bypass mode to context mode when a block of video data is encoded or decoded using palette mode.

The techniques described in this disclosure may include techniques for signaling various combinations of one or more of a palette-based video coding mode, transmitting a palette, deriving a palette, signaling a scan order, deriving a scan order, and transmitting a palette-based video coding map and other syntax elements. For example, the techniques of this disclosure may be directed to entropy coding palette information. In some examples, the techniques of this disclosure may be used, among other things, to improve coding efficiency and reduce coding inefficiencies associated with palette-based video coding. Thus, as described in more detail below, in some cases, the techniques of this disclosure may improve efficiency and improve bitrate when video data is coded using a palette mode.

As described above, in the current palette mode design in screen content coding, the syntax elements of palette _ index _ idc and palette _ escape _ val are CABAC bypass coded and interleaved with other syntax elements (e.g., palette _ run _ msb _ id _ plus1) that are CABAC context coded. However, it may be beneficial to group bypass-coded information (e.g., syntax elements) together, which may improve coding efficiency and/or reduce codec complexity.

The syntax element of palette _ index _ idc may be an indication of an index to an array represented as currentpalette entries, e.g., as defined in JCTVC-S1005. The value of palette _ index _ idc may range from 0 to (adoustedIndexMax-1) (inclusive). For example, as defined in JCTVC-S1005, the syntax element of palette escape _ val may specify the quantized escape coded sample value of a component. For example, as defined in JCTVC-S1005, the palette _ run _ msb _ id _ plus1 minus 1 may specify the index of the most significant bit in the binary representation of the palette run. For example, as defined in JCTVC-S1005, the variable palette run may specify the number of consecutive positions having the same palette INDEX as the position in the ABOVE row minus 1 when the palette _ run _ type _ flag is equal to COPY _ ABOVE _ MODE, or the number of consecutive positions having the same palette INDEX minus 1 when the palette _ run _ type _ flag is equal to COPY _ INDEX _ MODE. Additional details regarding palette _ index _ idc, palette _ escape _ val, palette _ run _ msb _ plus1, currentpalette entries, adjustedIndexMax, and palette run may be found in JCTVC-S1005.

In some examples, this disclosure describes methods of grouping all syntax elements palette _ index _ idc in front of the palette index block coded portion to improve CABAC throughput. For example, video encoder 20 may be configured to encode all syntax elements palette _ index _ idc in front of the palette index block coding portion. For example, video encoder 20 may be configured to encode all syntax elements palette _ index _ idc prior to encoding the syntax elements to be context mode encoded. Similarly, video decoder 30 may be configured to decode all syntax elements palette _ index _ idc prior to the palette index block coding portion. For example, video decoder 30 may be configured to decode all syntax elements palette _ index _ idc prior to decoding the context-mode encoded syntax elements.

As another example, video encoder 20 may be configured to bypass mode encode all syntax elements palette _ index _ idc prior to the palette index block coding portion such that all syntax elements palette _ index _ idc are encoded prior to encoding syntax elements relating to palette run types (e.g., CopyAbove mode or index mode) and/or run lengths (e.g., palette _ run _ msb _ id _ plus 1). Similarly, video decoder 30 may be configured to decode all syntax elements palette _ index _ idc of the block in front of the palette index block coded portion of the block such that all syntax elements palette _ index _ idc are decoded before decoding syntax elements relating to palette run types (e.g., CopyAbove mode or index mode) and/or run lengths (e.g., palette _ run _ msb _ id _ plus 1).

Syntax elements relating to palette run type (e.g., CopyAbove mode or index mode) and/or run length (e.g., palette run msb id plus1)

As another example, example video encoder 20 may be configured to encode all syntax elements palette _ index _ idc prior to context encoding the syntax elements for a palette run type (e.g., CopyAbove mode or index mode) and/or run length (e.g., palette run msb id plus 1). Similarly, video decoder 30 may be configured to decode all syntax elements palette _ index _ idc prior to context decoding syntax elements for palette run types (e.g., CopyAbove mode or index mode) and/or run lengths (e.g., palette run msb id plus 1).

As another example, video encoder 20 may be configured to encode all syntax elements palette index idc within the palette block coding portion prior to encoding the syntax element to be context mode encoded. Similarly, video decoder 30 may be configured to decode all syntax elements palette _ index _ idc within the palette block coding portion prior to decoding the context-mode encoded syntax elements. As another example, video encoder 20 may be configured to encode all syntax elements palette _ index _ idc within the palette block coding portion prior to context encoding the syntax elements for the palette run type (e.g., CopyAbove mode or index mode) and/or run length (e.g., palette run msb id plus 1). Similarly, video decoder 30 may be configured to decode all syntax elements palette _ index _ idc within the palette block coding portion prior to context decoding syntax elements relating to palette run types (e.g., CopyAbove mode or index mode) and/or run lengths (e.g., palette run msb id plus 1).

In general, video encoder 20 and video decoder 30 may be configured such that encoding or decoding palette _ index _ idc in a bypass mode that encodes or decodes syntax elements using context mode, respectively, is not interleaved. For example, video encoder 20 and video decoder 30 may be configured such that encoding or decoding palette _ index _ idc using a bypass mode that encodes or decodes syntax elements for palette run types (e.g., CopyAbove mode or index mode) and/or run lengths (e.g., palette run msb id plus1), respectively, is not interleaved. As another example, video encoder 20 may be configured to bypass encode all instances of the palette _ index _ idc syntax element before context encoding the syntax element requiring a context mode. Similarly, video decoder 30 may be configured to bypass decode all instances of the palette _ index _ idc syntax element before context decoding the syntax element that requires context mode. As another example, video encoder 20 may be configured to bypass encode all instances of syntax element palette _ index _ idc prior to context encoding the syntax elements for the palette run type (e.g., CopyAbove mode or index mode) and/or run length (e.g., palette run msb id plus 1). Similarly, video decoder 30 may be configured to bypass decode all instances of the palette _ index _ idc syntax element prior to context decoding syntax elements relating to palette run types (e.g., CopyAbove mode or index mode) and/or run lengths (e.g., palette _ run _ msb _ id _ plus 1).

Video encoder 20 and video decoder 30 may also encode and decode values representing the number of occurrences of palette _ index _ idc, respectively. Video encoder 20 and video decoder 30 may use the value representing the number of occurrences of palette _ index _ idc to encode or decode each of the syntax elements palette _ index _ idc, respectively. The techniques described in this disclosure may also eliminate redundancy for palette run length related syntax elements, and eliminate redundancy for palette _ run _ type _ flag and palette _ index _ idc.

In some examples, this disclosure describes methods of grouping all syntax elements palette _ escape _ val in front of the palette index block coding portion of a block (e.g., PU or CU) to improve CABAC throughput. For example, video encoder 20 may be configured to encode all syntax elements palette _ escape _ val in front of the palette index block coded portion of the block. For example, video encoder 20 may be configured to bypass mode encode all syntax elements palette _ escape _ val in front of the palette index block coding portion such that all syntax elements palette _ escape _ val are encoded before encoding syntax elements relating to palette run types (e.g., CopyAbove mode or index mode) and/or run lengths (e.g., palette _ run _ msb _ id _ plus 1). Similarly, video decoder 30 may be configured to decode all syntax elements palette _ escape _ val of the block prior to the palette index block coded portion of the block such that all syntax elements palette _ escape _ val are decoded prior to decoding syntax elements relating to the palette run type (e.g., CopyAbove mode or index mode) and/or run length (e.g., palette run msb id plus 1). As another example, video encoder 20 may be configured to encode all syntax elements palette escape val prior to encoding the syntax element to be context mode encoded. For example, video encoder 20 may be configured to encode all syntax elements pattern escape val prior to context encoding the syntax elements for the palette run type (e.g., CopyAbove mode or index mode) and/or run length (e.g., pattern run msb id plus 1). Similarly, video decoder 30 may be configured to decode all syntax elements palette _ escape _ val in front of the palette index block coded portion of the block. For example, video decoder 30 may be configured to decode all syntax elements palette _ escape _ val prior to decoding the context mode encoded syntax elements in the block.

As another example, video encoder 20 may be configured to encode all syntax elements palette _ escape _ val within the palette block coding portion of the block prior to encoding the syntax element to be context mode encoded. Similarly, video decoder 30 may be configured to decode all syntax elements palette _ escape _ val within the palette block coding portion of the block prior to decoding the context mode encoded syntax elements of the block.

In general, video encoder 20 and video decoder 30 may be configured to de-interleave palette _ escape _ val that encodes or decodes a block (e.g., a PU or CU) with a bypass mode that encodes or decodes syntax elements using context mode for the block, respectively. For example, video encoder 20 and video decoder 30 may be configured such that palette _ escape _ val is encoded or decoded without interleaving in a bypass mode that encodes or decodes syntax elements for palette run types (e.g., CopyAbove mode or index mode) and/or run lengths (e.g., palette _ run _ msb _ id _ plus1), respectively, using context mode. As another example, video encoder 20 may be configured to bypass encode all instances of the palette _ escape _ val syntax element of the block prior to context encoding the syntax element requiring the context mode. Similarly, video decoder 30 may be configured to bypass decode all instances of the palette _ escape _ val syntax element of a block (e.g., PU or CU) before context decoding the syntax element that requires the context mode of the block.

Video encoder 20 and video decoder 30 may also encode and decode a value representing the number of occurrences of palette _ escape _ val for the block, respectively. Video encoder 20 and video decoder 30 may use a value representing the number of occurrences of palette _ escape _ val to encode or decode each of the syntax elements palette _ escape _ val of the block, respectively. The techniques described in this disclosure may reduce the dynamic range of the palette _ index _ idc of a block, which may result in improved coding efficiency.

The techniques, aspects, and/or examples described herein may be utilized in conjunction with each other in any combination or independently of each other. For example, video encoder 20 and video decoder 30 may be configured to perform any one or any suitable combination of one or more of the techniques, aspects, and/or examples described herein.

In some examples, to improve CABAC throughput, a video coder (e.g., video encoder 20) may be configured to group all occurrences of the syntax element palette _ index _ idc, as described above. For example, a video coder (e.g., video encoder 20) may be configured to group all occurrences of a syntax element palette _ index _ idc in a current block (e.g., a PU or a CU) before an index coded portion of the current block. Similarly, as described above, a video decoder (e.g., video decoder 30) may be configured to decode all syntax elements palette _ index _ idc. Fig. 7 illustrates an example in which video encoder 20 may be configured to, for example, with respect to r.joshi and j.xu "High Efficiency Video Coding (HEVC) screen content coding: draft 2 "JCTVC-S1005, 7.3.3.8 section, groups one instance of all occurrences of the syntax element palette _ index _ idc in the current block (e.g., CU) before the index coded block. This aspect of the invention is referred to as aspect 1. In particular, fig. 7 illustrates an example of video encoder 20 that relocates an example of the syntax element palette _ index _ idc to be in front of an index coding block (which may also be referred to as the palette coding portion or in front of an index coding block). By repositioning the illustrated examples of the illustrated syntax element palette _ index _ idc, video encoder 20 may be configured to improve CABAC throughput by coding all instances of the syntax element palette _ index _ idc using bypass mode and switching to context mode to code palette information that occurs after all instances of the syntax element palette _ index _ idc in the index coded block are bypass mode encoded.

According to the disclosure of JCTVC-S1005, one instance of palette _ index _ idc will be coded in bypass mode, then one instance of syntax elements for palette run type and one instance of palette _ run _ msb _ id _ plus1 will be coded in context mode, and the process will repeat while (scanPos < nCbS × nCbS), meaning that the video encoder will switch back and forth between bypass mode coding and context mode coding because syntax elements to be coded using bypass mode are not grouped together. This is depicted in fig. 7, where the ellipse is directly below the loop of "while (scanPos < nCbS x nCbS)" (i.e., the ellipse does not include information showing that syntax elements regarding palette run types are encoded using context patterns), and the square of the if-statement surrounding the following palette _ index _ idc syntax element is below the loop of "while (scanPos < nCbS x nCbS)" and the following pseudo-code. However, as described above, fig. 7 also depicts aspect 1 of this disclosure, which is grouping (may also be referred to as relocating) one or more instances of the syntax element palette _ index _ idc, e.g., ahead of the index coding block. By repositioning one or more syntax elements (e.g., or other palette information) to be encoded using bypass mode, a video encoder (e.g., video encoder 20) may increase the throughput of entropy coding by reducing the number of times a video encoder or video decoder must switch between bypass mode encoding and context mode encoding. Similarly, by repositioning one or more syntax elements in this manner, the throughput of a video decoder (e.g., video decoder 30) may be increased because the number of times the video decoder must switch between bypass mode decoding and context mode decoding is reduced. In some examples of the techniques described in this disclosure, all instances of the palette _ index _ idc syntax element will be coded in bypass mode before an instance of the palette _ run _ msb _ id _ plus1 will be coded in the context mode.

In some examples, video encoder 20 may be configured to signal the number of occurrences (e.g., an example) of syntax element palette _ index _ idc using a syntax element referred to as, for example, num _ palette _ index. For example, video encoder 20 may signal a value of num _ palette _ index in a bitstream, where the value represents the number of occurrences of the syntax element palette _ index _ idc. In some examples, video encoder 20 may be configured to not signal an index value such as palette _ index _ idc. In these examples, video decoder 30 may be configured to infer the index value. For example, the occurrence of palette _ INDEX _ idc may be counted in num _ palette _ INDEX, which may be equal to the number of times the run type (e.g., COPY _ INDEX _ MODE) occurs in a particular block. Even when a run type (e.g., COPY _ INDEX _ MODE) is inferred or a palette _ INDEX _ idc is inferred, it still counts num _ palette _ INDEX. As used herein, in some examples, a reference that is parsed, decoded, or holds a plurality of indices to be decoded may refer to the number of COPY INDEX MODEs regardless of whether a MODE or INDEX is inferred. Video decoder 30 may be configured to determine the number of occurrences (e.g., instances) of syntax element palette _ index _ idc by, for example, decoding the encoded value from the bitstream that corresponds to the num _ palette _ index syntax element. This aspect of the invention is referred to as aspect 2. Video encoder 20 and video decoder 30 may be configured to implement aspect 1 using aspect 2 or without aspect 2. In terms of syntax, according to some examples, aspect 2 may be defined as:

indices_idc_coding(){
		num_palette_index	ae(v)
for(i＝0；i<num_palette_index；i++)
		palette_index_idc	ae(v)
}

in some examples, video encoder 20 and video decoder 30 may be configured to implement (e.g., by enabling) aspects 1 and 2 only when the variable indexMax is greater than 1. This aspect of the invention is referred to as aspect 3. The variable indexMax may specify the number of different values the palette index has for the current coding unit. In some examples, indexMax may refer to the number of (palette size + palette _ escape _ val _ present _ flag).

In some examples, aspect 1 and aspect 2 may be disabled when: (a) no escape pixel exists in the current block (i.e., palette _ escape _ val _ present _ flag ═ 0) and the palette size is less than 2; or (b) there may be at least one escape pixel in the current block (i.e., palette _ escape _ val _ present _ flag ═ 1) and the palette size is equal to 0. In other examples, video encoder 20 and video decoder 30 may be configured to implement (e.g., by enabling) aspects 1 and 2 only when the variable indexMax is greater than 2. Similarly, in examples where indexMax is equal to (palette size + palette _ escape _ val _ present _ flag), aspects 1 and 2 may be enabled (e.g., implemented) when indexMax is greater than 1. For example, if the palette size is 0 and the palette _ escape _ val _ present _ flag is 1, then all pixels in the block are escape pixels; and, thus, the index is already known. As another example, if palette _ escape _ val _ present _ flag is 0 and the palette size is 1, then, likewise, each pixel has an index of 0; and, thus, the index may not be unnecessarily signaled.

In some examples, video encoder 20 may be configured to implement aspects 1 and 2 such that the last occurrence (e.g., an instance) of the syntax element palette _ run _ type _ flag [ xC ] [ yC ] is signaled by video encoder 20 in front of the palette index block coding portion. This aspect of the invention is referred to as aspect 4. Specifically, according to some examples, the syntax table may be updated by adding a new syntax element palette _ last _ run _ type _ flag, as follows:

indices_idc_coding(){
		num_palette_index	ae(v)
for(i＝0；i<num_palette_index；i++)
		palette_index_idc	ae(v)
palette_last_run_type_flag	ae(v)
		}

video decoder 30 may be configured to determine a last occurrence (e.g., an example) of the syntax element palette _ run _ type _ flag [ xC ] [ yC ] by, for example, decoding an encoded palette _ last _ run _ type _ flag syntax element from the bitstream. Syntax elements of the palette _ last _ run _ type _ flag may be bypass mode coded or context mode coded in CABAC, for example. In examples where the palette _ last _ run _ type _ flag syntax element is context mode coded, the palette _ last _ run _ type _ flag syntax element may share the same context with palette _ run _ type _ flag [ xC ] [ yC ], or the palette _ last _ run _ type _ flag syntax element may have its own context independent of the context of palette _ run _ type _ flag [ xC ] [ yC ].

In some examples, video decoder 30 may be configured to decode the syntax element palette _ index _ idc such that the dynamic range adjustment process is disabled for the first occurrence (e.g., an example) of the palette _ index _ idc syntax element. This aspect of the invention is referred to as aspect 5. Specifically, a very similar procedure is used as the adjustedIndexMax variable export procedure specified in JCTVC-S1005 section 7.4.9.6. For comparison purposes, JCTVC-S1005 describes that variable adjustedIndexMax can be derived as follows:

adjustedIndexMax＝indexMax

if(scanPos>0)

adjustedIndexMax-＝1

however, according to aspect 5 of the present invention, the variable adjustIndexMax may be derived as set forth below. For example, for each block, the variable isFirstIndex is initialized to 1 prior to parsing. In some examples, the variable adjustedIndexMax may be derived as follows:

adjustedIndexMax＝indexMax

palette_index_idc

if(isFirstIndex){

adjustedIndexMax-＝isFirstIndex

isFirstIndex＝0

}

in some examples, prior to parsing and decoding the palette run, video decoder 30 may be configured to check one or more conditions. This aspect of the invention is referred to as aspect 6. As disclosed in, for example, JCTVC-S1005, the variable palette run may specify the number of consecutive positions having the same palette INDEX as the position in the ABOVE-described row minus 1 when the palette _ run _ type _ flag is equal to COPY _ ABOVE _ MODE, or the number of consecutive positions having the same palette INDEX minus 1 when the palette _ run _ type _ flag is equal to COPY _ INDEX _ MODE.

Referring to one or more conditions that video decoder 30 may be configured to check, if video decoder 30 determines that one or more of the conditions are satisfied, video decoder 30 may be configured to bypass the parsing and decoding process for syntax elements related to the current palette run (i.e., palette _ run _ msb _ id _ plus1 and palette _ run _ refinement _ bits). In this example, video decoder 30 may be configured to implicitly derive the current palette run as it runs to the end of the current block (i.e., equal to maxpaletteeren). The list of one or more conditions related to aspect 6 includes: (i) a number of parsed/decoded palette _ index _ idc syntax elements equal to num _ palette _ index; or, alternatively, a variable palette index left may be defined equal to num _ palette _ index minus the number of received indices, and using that definition, this condition may be stated as palette index left being equal to zero; and/or (ii) the current palette run type palette _ run _ type _ flag [ xC ] [ yC ] is equal to the last palette run type palette _ last _ run _ type _ flag.

In some examples, video encoder 20 may be configured to code the palette run length into the bitstream if conditions (i) and (ii) described above for aspect 6 are not satisfied at the same time. This aspect of the invention is referred to as aspect 7. In other examples, video encoder 20 may be configured to code the palette run length into the bitstream if conditions (i) and (ii) described above for aspect 6 are not satisfied at the same time. According to the current draft specification JCTVC-S1005, it is necessary to specify as input the parameter of the maximum achievable run length, where the parameter is equal to maxpaletteer ═ nCbS × nCbS-scanPos-1. However, in accordance with this disclosure, video encoder 20 may be configured to reduce the parameter specifying the maximum achievable run length to maxpaletteer ═ nCbS-scanPos-1-palette indendecicep left to improve coding efficiency. As used herein, nCbS specifies the size of the current block.

In some examples, if the block is not in palette sharing mode (i.e., palette _ share _ flag x0 y0 ═ 0), then a normative constraint may be imposed on video encoder 20 that requires it to never signal a palette with unused entries. This aspect of the invention is referred to as aspect 8.

In some examples, for a palette mode that does not use palette sharing, when one or more of the following conditions are met: where num _ palette _ index equals indexMax condition 1 and where palette indendedceinforcesleft ═ 1 condition 2, video decoder 30 may be configured to bypass decoding of the current occurrence (e.g., instance) of syntax element palette _ index _ idc. In these examples, video decoder 30 may be configured to implicitly derive the currently occurring value of syntax element palette _ index _ idc as an index in the palette, but already in the index map during the decoding process (e.g., until this point does not appear in the index map in the decoding process). This aspect of the invention is referred to as aspect 9.

Video decoder 30 may be configured to derive the currently occurring value of syntax element palette _ index _ idc described above for aspect 9, since condition 1 requires that each index between 0 and (indexMax-1) be signaled, inclusively, and only once. Thus, after the first (indexMax-1) index value is signaled, video decoder 30 may be configured to derive a number between the last index value of 0 and (indexMax-1), which has occurred during the decoding process of the current index map.

In some examples, video decoder 30 may be configured to bypass decoding of a current occurrence (e.g., an example) of syntax element palette _ run _ type _ flag [ xC ] [ yC ], when one or both of the following conditions are met: condition 1, where palette endicendices left is equal to 0, and condition 2, where the current pixel is at the last position of the block in scan order. In these examples, video decoder 30 may be configured to implicitly derive a currently occurring value of syntax element palette _ run _ type _ flag [ xC ] [ yC ]. For example, when condition 1 is satisfied, palette _ run _ type _ flag [ xC ] [ yC ] video decoder 30 may be configured to derive the currently occurring value of syntax element palette _ run _ type _ flag [ xC ] [ yC ] as COPY _ ABOVE _ MODE. As another example, when condition 1 is satisfied, if palette indicesleft >0, palette _ run _ type _ flag [ xC ] [ yC ] video decoder 30 may be configured to derive a currently occurring value of syntax element palette _ run _ type _ flag [ xC ] [ yC ] as COPY _ INDEX _ MODE, and if palette indicesleft is 0, as COPY _ ABOVE _ MODE. This aspect of the invention is referred to as aspect 10.

As described herein, video encoder 20 and video decoder 30 may be configured to determine when a condition is satisfied. For example, with respect to aspect 10, video decoder 30 may be configured to determine whether condition 1 is satisfied. Similarly, video decoder 30 may be configured to determine whether condition 2 is satisfied. In response to determining that condition 1 or condition 2 is satisfied, video decoder 30 may be configured to derive a currently occurring value of a syntax element palette _ run _ type _ flag [ xC ] [ yC ], as described above.

In some examples, video encoder 20 and video decoder 30 may be configured to encode or decode the num _ palette _ index syntax element using any family of golomb codes, respectively. For example, video encoder 20 and video decoder 30 may be configured to encode or decode the num _ palette _ index syntax element using, for example, a golomb rice code, an exponential golomb code, a truncated rice code, an unary code, or a concatenation of golomb rice codes and exponential golomb codes, respectively. This aspect of the invention is referred to as aspect 11.

In other examples, video encoder 20 and video decoder 30 may be configured to encode or decode the num _ palette _ index syntax element using any truncated version of any golomb family, respectively. For example, video encoder 20 and video decoder 30 may be configured to encode or decode num _ palette _ index syntax elements using, for example, a truncated golomb code, a truncated exponential golomb code, a truncated rice code, a truncated unary code, or a concatenation of a truncated rice code and an exponential golomb code (e.g., program code for coding coeff _ abs _ level _ remaining syntax elements), respectively. This aspect of the invention is referred to as aspect 12.

In some examples, any golomb parameter related to aspect 11 or aspect 12 depends on CU size, indexMax, palette size, and/or palette _ escape _ val _ present _ flag. The dependencies may be expressed as equations or as look-up tables. In some examples, video encoder 20 may be configured to signal parameters in a lookup table or equation such that they are received by video decoder 30, for example, in an SPS/PPS/slice header. Alternatively or additionally, the parameters may be adaptively updated on a block-by-block basis. This aspect of the invention is referred to as aspect 13. In some examples, the golomb parameter cRiceParam may depend on indexMax, palette size, and/or palette _ escape _ val _ present _ flag. The golomb parameter cRiceParam may vary from block to block.

In some examples, video encoder 20 may be configured to predictively encode num _ palette _ index by signaling a difference between a value of num _ palette _ index, which may be expressed by a syntax element referred to as, for example, numpacketindexcoded, and the delta value. This aspect of the invention is referred to as aspect 14. For example, video encoder 20 may be configured to predictively encode num _ palette _ index by signaling a value for num _ palette _ index, where num _ palette _ index-index _ IndexOffsetValue. Similarly, video decoder 30 may be configured to predictively decode num _ palette _ index by, for example, determining a value of numscalable index coded from the bitstream. Because numpacketindexcoded is num _ palette _ index-indexevalue, video decoder 30 may be configured to determine the value of num _ palette _ index based on the determined value of numpacketindexcoded and the value of indexevalue.

In some examples, the variable indexeffsetvalue may be a constant. For example, indexeffsetvalue may be equal to a constant value of X for the palette sharing mode or may be equal to a constant value of Y for the non-palette sharing mode, where X and Y are integers. In some examples, X and Y may be the same (e.g., X equals Y, such as equals 1). In other examples, X and Y may be different (e.g., X is not equal to Y). For example, indexafsetvalue may be equal to 9 when palette shared mode is used and may be equal to 33 when non-shared mode is used. In some examples, the variable indexevactive may depend on the syntax element palette _ share _ flag [ x0] [ y0 ]. In other examples, the variable indexeffsetvalue may depend on the variable indexMax. For example, indexafsetvalue may be equal to indexMax. In some examples, video encoder 20 may be configured to signal indexafsetvalue in the SPS/PPS/slice header. Alternatively or additionally, variable indexevalue may be adaptively updated block-by-block, meaning that the value corresponding to variable indexevalue may be adaptively updated block-by-block.

In some examples, video encoder 20 and video decoder 30 may be configured to encode or decode, respectively, numpraltetelndexcoded that may be coded using any family of golomb codes or any family of truncated golomb, such as a concatenation of golomb rice codes and exponential golomb codes. For example, nummetendexcoded is equal to num _ palette _ index-1 when indexatevalue is equal to 1.

In some examples, video encoder 20 and video decoder 30 may be configured to encode or decode numscalable indexcoded using any family of golomb codes, respectively. For example, video encoder 20 and video decoder 30 may be configured to encode or decode numPaletteIndexCoded using, for example, golomb codes, exponential golomb codes, truncation rice codes, unary codes, or a concatenation of golomb rice codes and exponential golomb codes, respectively.

In other examples, video encoder 20 and video decoder 30 may be configured to encode or decode numscalable indexcoded using any truncated version of any family of golomb codes, respectively. For example, video encoder 20 and video decoder 30 may be configured to encode or decode numbedetendexcoded using, for example, a truncated golomb code, a truncated exponential golomb code, a truncated rice code, a truncated unary code, or a concatenation of a truncated rice code and an exponential golomb code (e.g., a code for coding coeff _ abs _ level _ remaining syntax elements), respectively.

To code nummeteindexcoded, video encoder 20 may be configured to determine a sign of nummeteindexcoded. Video encoder 20 may be configured to signal a flag indicating whether the value of nummetendexcoded is negative or not (e.g., whether the determined sign is positive or negative). This aspect of the invention is referred to as aspect 15. In some examples, video encoder 20 may be configured to signal a flag, and then signal a value of numsealtedindexcoded. In other examples, video encoder 20 may be configured to signal the value of numsealtedindexcoded, and then signal the flag. Video encoder 20 may be configured to encode the flag using either bypass mode or context mode. If context coded, the context may depend on the CU size, indexMax, palette size, and/or palette escape val present flag.

As described above, video encoder 20 may be configured to determine the sign of nummeteindexcoded according to some examples. If the sign of the determined nummeteindexcoded is negative, video encoder 20 may be configured to encode the value of (1-nummeteindexcoded) into the bitstream. If the sign of nummeteindexcoded is determined to be positive, video encoder 20 may be configured to encode the value of nummeteindexcoded into the bitstream. Video encoder 20 may be configured to encode the value of (1-numscalable indexcoded) or the numscalable indexcoded) value using different golomb code parameters that depend on, for example, the sign of numscalable indexcoded, CU size, indexMax, palette size, and/or palette _ escape _ val _ present _ flag.

In some examples, video encoder 20 may be configured to use a mapping operation to represent the negative of numsealeindexcoded, which may replace aspect 15 in addition to aspect 15. This aspect of the invention is referred to as aspect 16. For example, a mapping interval may be introduced and defined as a variable maplnterval. Video encoder 20 may be configured to map negative values of numpatterndexcoded to equally spaced equal to: a positive value of maprinterval x (nummeteIndexCoded) -1. The corresponding positive value of nummeteindexcoded may thus be shifted to accommodate the position obtained by the mapped negative value.

For example, if maplnterval is 2, and numpatelndexcoded is selected from { -3, -2, -1,0,1,2,3}, the mapping may be as illustrated in table I below. In this example, video encoder 20 may be configured to encode the value of numsealteindexcode using the mapped values in table I. For example, video encoder 20 may be configured to entropy encode the mapped values into a binary form.

Table i codeword mapping example

In some examples, video encoder 20 may be configured to represent the negative of numsealeindexcoded using the mapping operations described with respect to aspect 16. Video encoder 20 may also be configured to remove one or more redundancies that may occur when implementing aspect 16. This aspect of the invention is referred to as aspect 17. For example, the number of negative values of nummeteindexcoded may be in the range of a { -1, -2., -indexofsetvalue +1 }. As another example, the number of negative values of numpralletteindexcode may be in the range of a { -1, -2., -indexeffsetvalue +1, indexeffsetvalue }. In either of these examples, the mapped value only needs to retain the (IndexOffsetValue-1) or IndexOffsetValue position of the negative numBeteteIndexCoded value. For example, if maplnterval is 2, and numpatelndexcoded is selected from { -3, -2, -1,0,1,2,3,4,5,6,7,8}, then the mapping may be as illustrated in table II below. In this example, video encoder 20 may be configured to encode the value of numsealteindexcode using the mapped values in table II. For example, video encoder 20 may be configured to entropy encode the mapped values into a binary form.

Table ii codeword mapping example

As shown in table II above, video encoder 20 may be configured to encode the mapped value corresponding to the value of nummetendindexcode such that negative and positive values of nummetendexcode are not interleaved after a certain value. For example, in the example of Table II above, there is no interleaving of positive and negative values of nummeteIndexCoded through mapped values starting with a value of 3 for nummeteIndexCoded (i.e., positive values 3-8 for nummeteIndexCoded map to mapped values 6-11).

As described above, video encoder 20 may also be configured to remove one or more redundancies that may occur when implementing aspect 16. Another redundant example different from that described above includes: since num _ palette _ index is upper-limited by the total number of pixels in the current block, numpatterndindexcoded also has an upper limit. Thus, after allocating positions with all probabilities of positive codewords, negative values may be mapped to the following positions without interleaving. For example, if maplnterval is 2, and numpatelndexcoded is selected from { -5, -4, -3, -2, -1,0,1,2,3}, the mapping may be as illustrated in table III below. In this example, video encoder 20 may be configured to encode the value of numsealteindexcode using the mapped values in table III. For example, video encoder 20 may be configured to entropy encode the mapped values into a binary form.

Table iii codeword mapping example

As shown in table III above, video encoder 20 may be configured to encode the mapped value corresponding to the value of numsealteindexcode such that negative and positive values of numsealteindexcode are not interleaved after a certain value. For example, in the example of table III above, there is no interleaving of positive and negative values of nummetendexcoded through mapped values starting with a value of 4 for nummetendexcoded (i.e., negative values-4 and-5 for nummetendexcoded map to mapped values of 7 and 8).

In some examples, video encoder 20 may be configured to further decouple the relationship between palette indices and palette runs. This aspect of the invention is referred to as aspect 18. For example, video encoder 20 may be configured such that the context of palette run coding depends on the foregoing palette run length or on the palette run msb plus1, indexMax, and/or CU size of the foregoing run, rather than allowing the context of palette run coding to depend on the parsed or decoded index.

In some examples, to further packet bypass the binary, video encoder 20 may be configured to signal the number of escape indices in the palette block and the escape value prior to signaling the palette run type (i.e., palette run type flag xC yC) as follows. This aspect of the invention is referred to as aspect 19. The italicized part illustrates the variation of the previous version with respect to JCT-VC S1005, and the bold part and "ae (v)" in the right column indicate the signaling of syntax elements.

In the above example, escape _ idc _ coding () consists of signaling the number of escape indices and the escape value corresponding to each escape index. If palette _ escape _ val _ present _ flag is 0 or if indexMax is equal to 0, the number of escape indices in the palette block may not be signaled. In the former case, the number of escape indices is inferred to be 0, and no escape value is signaled. In the latter case, where indexMax is equal to 0, the number of escape indices is inferred to be equal to the block size when palette _ escape _ val _ present _ flag is equal to 1 and the escape value is signaled, or zero when palette _ escape _ val _ present _ flag is equal to 0.

In some examples, video encoder 20 may be configured to signal the number of escape indices using the family of golomb codes. This aspect of the invention is referred to as aspect 20. For example, video encoder 20 may be configured to signal the number of escape indices using, for example, golomb codes, exponential golomb codes, truncated rice codes, unary codes, or a concatenation of golomb rice codes and exponential golomb codes. A truncated version of the above code may be used with the largest set equal to the block size.

In some examples, it is proposed to enforce a normative constraint on the palette _ escape _ val _ present _ flag, when the palette _ escape _ val _ present _ flag is equal to 0, no escape pixels exist in the current block. This aspect of the invention is referred to as aspect 21. When palette _ escape _ val _ present _ flag is equal to 1, at least one escape pixel exists in the current block. Due to this limitation, in escape _ idc _ coding (), the number of escape indexes minus 1 can be coded instead of the number of escape indexes to improve coding efficiency. In that case, the maximum value of the truncated family of Golomb codes can thus be adjusted to (blockSize-1).

In some examples, when the number of escape indices is signaled prior to coding an index tile and when all escape indices have been coded, then indexMax may be reduced by 1. Furthermore, if indexMax becomes 1, index, run, and mode coding is terminated because the index of all remaining samples can be inferred. This aspect of the invention is referred to as aspect 22. As one example of aspect 22, assume that the palette size is equal to 1 and the palette _ escape _ val _ present _ flag is equal to 1. Typically, the possible index values are 0 and 1, with 1 being used for escape samples. Under aspect 22, video encoder 20 may be configured to signal the number of escape values/samples. Subsequently, when the index is signaled and the last escape value/sample is encountered, both video encoder 20 and/or video decoder 30 may be configured to infer (e.g., determine) that the escape value/sample is no longer present. As such, video encoder 20 and/or video decoder 30 may be configured to determine that the only index value that may be generated from the last escape value/sample to the end of the block is 0, meaning that video encoder 20 may be configured not to signal the mode, index value, and/or run value from the last escape value/sample to the end of the block.

In some examples, escape _ idc _ coding () is used in combination with indices _ idc _ coding (). This aspect of the invention is referred to as aspect 23. In one example, the number of escape indices may be signaled before the number of indices is signaled. In this case, it is only necessary to signal the number of non-escape indexes with indices _ idc _ coding (). In one example, the number of escape indices may be signaled after the number of indices is signaled. In this case, the maximum value of the truncated family of golomb codes may thus be adjusted to num _ palette _ index.

Video encoder 20 and/or video decoder 30 may be configured to operate in accordance with the techniques described in this disclosure. In general, video encoder 20 and/or video decoder 30 may be configured to determine that the current block is coded in palette mode, bypass mode code the plurality of instances of the first syntax element for reconstructing the current block, and, after bypass mode coding the plurality of instances of the first syntax element, context mode decode the plurality of instances of the second syntax element for reconstructing the current block.

FIG. 2 is a block diagram illustrating an example video encoder 20 that may implement the techniques of this disclosure. Fig. 2 is provided for purposes of explanation and should not be taken as a limitation on the technology as broadly exemplified and described in this disclosure. For purposes of explanation, this disclosure describes video encoder 20 in the context of HEVC coding and SCC extensions such as HEVC. However, the techniques of this disclosure may be applicable to other coding standards or methods.

Video encoder 20 represents an example of a device that may be configured to perform techniques for palette-based coding and entropy coding (e.g., CABAC) according to various examples described in this disclosure.

In the example of fig. 2, video encoder 20 includes a block encoding unit 100, a video data memory 101, a residual generation unit 102, a transform processing unit 104, a quantization unit 106, an inverse quantization unit 108, an inverse transform processing unit 110, a reconstruction unit 112, a filter unit 114, a decoded picture buffer 116, and an entropy encoding unit 118. Block encoding unit 100 includes inter-prediction processing unit 120 and intra-prediction processing unit 126. Inter prediction processing unit 120 includes a motion estimation unit and a motion compensation unit (not shown). Video encoder 20 also includes palette-based encoding unit 122 configured to perform various aspects of the palette-based coding techniques described in this disclosure. In other examples, video encoder 20 may include more, fewer, or different functional components.

Video data memory 101 may store video data to be encoded by components of video encoder 20. The video data stored in video data memory 101 may be obtained, for example, from video source 18. Decoded picture buffer 116 may be a reference picture memory that stores reference video data for use when video encoder 20 encodes video data, e.g., in intra or inter coding modes). Video data memory 101 and decoded picture buffer 116 may be formed from any of a variety of memory devices, such as Dynamic Random Access Memory (DRAM), including synchronous DRAM (sdram), magnetoresistive ram (mram), resistive ram (rram), or other types of memory devices. Video data memory 101 and decoded picture buffer 116 may be provided by the same memory device or separate memory devices. In various examples, video data memory 101 may be on-chip with other components of video encoder 20, or off-chip with respect to those components.

Video encoder 20 may receive video data. Video encoder 20 may encode each CTU in a slice of a picture of video data. Each of the CTUs may be associated with a luma Coding Tree Block (CTB) and a corresponding CTB of the picture having equal sizes. As part of encoding the CTUs, block encoding unit 100 may perform quadtree partitioning to divide the CTBs of the CTUs into progressively smaller blocks. The smaller block may be a coding block of the CU. For example, the block encoding unit 100 may partition a CTB associated with a CTU into four equally sized sub-blocks, partition one or more of the sub-blocks into four equally sized sub-blocks, and so on.

Video encoder 20 may encode a CU of a CTU to generate an encoded representation of the CU (i.e., a coded CU). As part of encoding a CU, block encoding unit 100 may partition coding blocks associated with the CU among one or more PUs of the CU. Thus, each PU may be associated with a luma prediction block and a corresponding chroma prediction block. Video encoder 20 and video decoder 30 may support PUs having various sizes. As indicated above, the size of a CU may refer to the size of a luma coding block of the CU, and the size of a PU may refer to the size of a luma prediction block of the PU. Assuming that the size of a particular CU is 2 nx 2N, video encoder 20 and video decoder 30 may support 2 nx 2N or nxn PU sizes for intra prediction, and symmetric PU sizes of 2 nx 2N, 2 nx N, N x 2N, N x N, or the like for inter prediction. Video encoder 20 and video decoder 30 may also support asymmetric partitions of PU sizes of 2 nxnu, 2 nxnd, nlx 2N, and nrx 2N for inter prediction.

Inter-prediction processing unit 120 may generate predictive data for a PU by performing inter-prediction on each PU of the CU. The predictive data for the PU may include predictive blocks of the PU and motion information of the PU. Inter prediction unit 121 may perform different operations for a PU of a CU depending on whether the PU is in an I slice, a P slice, or a B slice. In an I slice, all PUs are intra predicted. Therefore, if the PU is in an I slice, inter prediction unit 121 does not perform inter prediction on the PU. Thus, for blocks encoded in I-mode, the predicted block is formed from previously encoded neighboring blocks within the same frame using spatial prediction.

If the PU is in a P slice, the motion estimation unit of inter-prediction processing unit 120 may search for a reference picture in a reference picture list (e.g., "RefPicList 0") of the reference region of the PU. The reference region of the PU may be a region within the reference picture that contains a sample block that most closely corresponds to the sample block of the PU. The motion estimation unit of inter-prediction processing unit 120 may generate a reference index that indicates the location in RefPicList0 of the reference picture of the reference region containing the PU. In addition, the motion estimation unit may generate MVs that indicate spatial shifts between the coding blocks of the PU and reference locations associated with the reference region. For example, the MV may be a two-dimensional vector that provides an offset from coordinates in the current decoded picture to coordinates in a reference picture. The motion estimation unit may output the reference index and the MV as motion information for the PU. The motion compensation unit of inter prediction processing unit 120 may generate the predictive blocks of the PU based on actual or interpolated samples at the reference location indicated by the motion vector of the PU.

If the PU is in a B slice, the motion estimation unit may perform uni-directional prediction or bi-directional prediction for the PU. To perform uni-directional prediction for a PU, the motion estimation unit may search for a reference picture in RefPicList0 or a second reference picture list ("RefPicList 1") for the reference region of the PU. The motion estimation unit may output, as the motion information of the PU, each of: a reference index indicating a position in RefPicList0 or RefPicList1 of a reference picture containing the reference region, an MV indicating a spatial displacement between a prediction block of the PU and a reference position associated with the reference region, and one or more prediction direction indicators indicating whether the reference picture is in RefPicList0 or RefPicList 1. The motion compensation unit of inter prediction processing unit 120 may generate the predictive block for the PU based at least in part on actual or interpolated samples at a reference region indicated by the motion vector of the PU.

To perform bi-directional inter prediction for a PU, the motion estimation unit may search for a reference picture in RefPicList0 for a reference region of the PU, and may also search for a reference picture in RefPicList1 for another reference region of the PU. The motion estimation unit may generate a reference picture index that indicates the position of the reference picture containing the reference region in RefPicList0 and RefPicList 1. In addition, the motion estimation unit may generate MVs that indicate spatial shifts between reference locations associated with the reference region and the sample blocks of the PU. The motion information for the PU may include a reference index and a MV for the PU. The motion compensation unit may generate the predictive block for the PU based at least in part on actual or interpolated samples at a reference region indicated by a motion vector of the PU.

According to various examples of this disclosure, video encoder 20 may be configured to perform palette-based coding. With respect to the HEVC architecture, as an example, palette-based coding techniques may be configured to be used at the CU level. In other examples, palette-based video coding techniques may be configured for use at the PU level. In other examples, palette-based coding techniques may be configured to be used at a sub-prediction unit (sub-PU) level (e.g., a sub-block of a prediction unit). Thus, all of the disclosed processes described herein (throughout this disclosure) in the context of a CU level may additionally or alternatively be applied to a PU level or a sub-PU level. However, these HEVC-based examples should not be viewed as restricting or limiting the palette-based video coding techniques described herein, as these techniques may be applicable to work independently or as part of other existing or yet-to-be developed systems/standards. In these cases, the unit for palette coding may be a square block, a rectangular block, or even a non-rectangular shaped region.

Palette-based encoding unit 122 may, for example, perform palette-based decoding when a palette-based encoding mode is selected, e.g., for a CU or PU. For example, palette-based encoding unit 122 may be configured to generate a palette having entries indicating pixel values, select pixel values in the palette to represent pixel values of at least some locations in a block of video data, and signal information associating at least some of the locations of the block of video data with entries in the palette that respectively correspond to the selected pixel values. Although various functions are described as being performed by palette-based encoding unit 122, some or all of these functions may be performed by other processing units or a combination of different processing units.

According to aspects of this disclosure, palette-based encoding unit 122 may be configured to perform any combination of the techniques described herein for palette coding.

Intra-prediction processing unit 126 may generate predictive data for the PU by performing intra-prediction on the PU. The predictive data for the PU may include predictive blocks for the PU and various syntax elements. Intra-prediction processing unit 126 may perform intra-prediction on PUs in I-slices, P-slices, and B-slices.

To perform intra-prediction for a PU, intra-prediction processing unit 126 may use multiple intra-prediction modes to generate multiple sets of predictive data for the PU. Intra-prediction processing unit 126 may use samples from sample blocks of neighboring PUs to generate predictive blocks for the PU. For PUs, CUs, and CTUs, assuming left-to-right top-to-bottom coding order, neighboring PUs may be above, above-right, above-left, or left to a PU. Intra-prediction processing unit 126 may use various numbers of intra-prediction modes, e.g., 33 directional intra-prediction modes. In some examples, the number of intra-prediction modes may depend on the size of the area associated with the PU.

Block encoding unit 100 may select predictive data for a PU of the CU from the predictive data for the PU generated by inter prediction processing unit 120 or the predictive data for the PU generated by intra prediction processing unit 126. In some examples, block encoding unit 100 selects predictive data for PUs of the CU based on a rate/distortion metric of the predictive data set. The predictive block of the selected predictive data may be referred to herein as the selected predictive block.

Residual generation unit 102 may generate the luma, Cb, and Cr residual blocks of the CU based on the luma, Cb, and Cr coding blocks of the CU and the selected predictive luma, predictive Cb, and predictive Cr blocks of the PUs of the CU. For example, residual generation unit 102 may generate the residual block of the CU such that each sample in the residual block has a value equal to a difference between a sample in the coding block of the CU and a corresponding sample in the corresponding selected predictive block of the PU of the CU.

Transform processing unit 104 may perform quadtree partitioning to partition a residual block associated with a CU into transform blocks associated with TUs of the CU. Thus, in some examples, a TU may be associated with a luma transform block and two chroma transform blocks. The size and position of luma and chroma transform blocks of a TU of a CU may or may not be based on the size and position of a prediction block of a PU of the CU. A quadtree structure, referred to as a "residual quadtree" (RQT), may include nodes associated with each of the regions. The TUs of a CU may correspond to leaf nodes of a RQT.

Transform processing unit 104 may generate transform coefficient blocks for each TU of the CU by applying one or more transforms to the transform blocks of the TU. Transform processing unit 104 may apply various transforms to transform blocks associated with TUs. For example, transform processing unit 104 may apply a Discrete Cosine Transform (DCT), a directional transform, or a conceptually similar transform to the transform blocks. In some examples, transform processing unit 104 does not apply the transform to the transform block. In these examples, the transform block may be processed into a block of transform coefficients.

Quantization unit 106 may quantize transform coefficients in the coefficient block. The quantization process may reduce the bit depth associated with some or all of the transform coefficients. For example, during quantization, an n-bit transform coefficient may be rounded down to an m-bit transform coefficient, where n is greater than m. Quantization unit 106 may quantize coefficient blocks associated with TUs of the CU based on Quantization Parameter (QP) values associated with the CU. Video encoder 20 may adjust the degree of quantization applied to the coefficient block associated with the CU by adjusting the QP value associated with the CU. Quantization may introduce information loss, so the quantized transform coefficients may have lower precision than the original transform coefficients.

Inverse quantization unit 108 and inverse transform processing unit 110 may apply inverse quantization and inverse transform, respectively, to the coefficient block to reconstruct the residual block from the coefficient block. Reconstruction unit 112 may add the reconstructed residual block to corresponding samples of one or more predictive blocks generated by block encoding unit 100 to generate a reconstructed transform block associated with the TU. By reconstructing the transform blocks for each TU of the CU in this manner, video encoder 20 may reconstruct the coding blocks of the CU.

Filter unit 114 may perform one or more deblocking operations to reduce block artifacts in coding blocks associated with CUs. Filter unit 114 may perform other filtering operations including Sample Adaptive Offset (SAO) filtering and/or Adaptive Loop Filtering (ALF). Decoded picture buffer 116 may store the reconstructed coded block after filter unit 114 performs one or more deblocking operations on the reconstructed coded block. Inter-prediction processing unit 120 may use the reference picture containing the reconstructed coding block to perform inter-prediction on PUs of other pictures. In addition, intra-prediction processing unit 126 may use reconstructed coding blocks in decoded picture buffer 116 to perform intra-prediction on other PUs in the same picture as the CU.

Entropy encoding unit 118 may receive data from other functional components of video encoder 20. For example, entropy encoding unit 118 may receive coefficient blocks from quantization unit 106 and may receive syntax elements from block encoding unit 100. Entropy encoding unit 118 may perform one or more entropy encoding operations on the data to generate entropy encoded data. For example, entropy encoding unit 118 may perform a context adaptive coding operation (e.g., a CABAC operation), a Context Adaptive Variable Length Coding (CAVLC) operation, a variable-to-variable (V2V) length coding operation, a syntax-based context adaptive binary arithmetic coding (SBAC) operation, a Probability Interval Partition Entropy (PIPE) coding operation, an exponential Golomb encoding operation, or another type of entropy encoding operation on the data. Video encoder 20 may output a bitstream that includes the entropy-encoded data generated by entropy encoding unit 118. For example, the bitstream may include data representing a RQT for a CU.

In some examples, residual coding is not performed with palette coding. Thus, when coded using the palette coding mode, video encoder 20 may not perform transform or quantization. In addition, video encoder 20 may entropy encode data generated from the residual data using the palette coding mode alone.

In accordance with one or more of the techniques of this disclosure, video encoder 20, and in particular palette-based encoding unit 122, may perform palette-based video coding of the predicted video block. As described above, the palette generated by video encoder 20 may be explicitly encoded and sent to video decoder 30, predicted from previous palette entries, predicted from previous pixel values, or a combination thereof.

In accordance with one or more techniques of this disclosure, video encoder 20 may be configured to determine that the current block is coded in palette mode, bypass mode encode the plurality of instances of the first syntax element for reconstructing the current block, and bypass mode encode the plurality of instances of the first syntax element for reconstructing the current block after context mode encoding the plurality of instances of the second syntax element, e.g., using a CABAC coding process. Video encoder 20 may be configured to bypass mode encode any two instances of the plurality of instances of the first syntax element, e.g., using a bypass mode of a CABAC coding process, without interleaving with context mode encoding of any instance of the plurality of instances of the second syntax element. In one example, the first syntax element comprises one of a palette _ index _ idc syntax element or a palette _ escape _ val syntax element, and the second syntax element comprises a palette _ run _ msb _ plus1 syntax element. Video encoder 20 may be configured to bypass encode the plurality of instances of the first syntax element prior to the index block coding portion for the current block.

Video encoder 20 may be configured to encode a third syntax element indicating a number of instances of the first syntax element, wherein bypass mode encoding the plurality of instances of the first syntax element comprises bypass mode encoding the plurality of instances of the first syntax element based on the third syntax element. Video encoder 20 may encode the third syntax element using golomb rice code, exponential golomb code, truncated rice code, unary code, or a concatenation of golomb rice code and exponential golomb code, or a truncated version of any of the foregoing codes.

Fig. 3 is a block diagram illustrating an example video decoder 30 that may be configured to perform the techniques of this disclosure. Fig. 3 is provided for purposes of explanation, and is not limiting of the techniques as broadly illustrated and described in this disclosure. For purposes of explanation, this disclosure describes video decoder 30 in the context of HEVC coding. However, the techniques of this disclosure may be applicable to other coding standards or methods.

The details of palette coding described above with respect to encoder 20 are not repeated herein with respect to decoder 30, but it should be understood that decoder 30 may perform a reciprocal decoding process with respect to any encoding process described herein with respect to encoder 20.

Video decoder 30 represents an example of a device that may be configured to perform techniques for palette-based coding and entropy coding (e.g., CABAC) according to various examples described in this disclosure.

In the example of fig. 3, video decoder 30 includes an entropy decoding unit 150, a video data memory 151, a block decoding unit 152, an inverse quantization unit 154, an inverse transform processing unit 156, a reconstruction unit 158, a filter unit 160, and a decoded picture buffer 162. Block decoding unit 152 includes motion compensation unit 164 and intra-prediction processing unit 166. Video decoder 30 also includes a palette-based decoding unit 165 configured to perform various aspects of the palette-based coding techniques described in this disclosure. In other examples, video decoder 30 may include more, fewer, or different functional components.

Video data memory 151 may store video data, such as an encoded video bitstream, to be decoded by components of video decoder 30. The video data stored in video data memory 151 may be obtained, for example, from computer-readable media 16 (e.g., from a local video source, such as a camera), via wired or wireless network communication of video data, or by accessing a physical data storage medium. Video data memory 151 may form a Coded Picture Buffer (CPB) that stores encoded video data from an encoded video bitstream. Decoded picture buffer 162 may be a reference picture memory that stores reference video data for use in decoding video data by video decoder 30 (e.g., in intra or inter coding modes). Video data memory 151 and decoded picture buffer 162 may be formed from any of a variety of memory devices, such as Dynamic Random Access Memory (DRAM), including synchronous DRAM (sdram), magnetoresistive ram (mram), resistive ram (rram), or other types of memory devices. Video data memory 151 and decoded picture buffer 162 may be provided by the same memory device or separate memory devices. In various examples, video data memory 151 may be on-chip with other components of video decoder 30, or off-chip with respect to those components.

A Coded Picture Buffer (CPB), which may be provided by video data memory 151, may receive and store encoded video data (e.g., NAL units) of a bitstream. Entropy decoding unit 150 may receive encoded video data (e.g., NAL units) from the CPB and parse the NAL units to decode the syntax elements. Entropy decoding unit 150 may entropy decode entropy-encoded syntax elements in the NAL unit. Block decoding unit 152, inverse quantization unit 154, inverse transform processing unit 156, reconstruction unit 158, and filter unit 160 may generate decoded video data based on syntax elements extracted from the bitstream.

Video decoder 30 may be configured to perform a process that is substantially reciprocal to the process of video encoder 20 described herein. Similarly, video encoder 20 may be configured to perform a process that is substantially reciprocal to the process of video decoder 30 described herein. For example, disclosure that video decoder 30 may be configured to decode an encoded syntax element in a bitstream also necessarily discloses that video encoder 20 may be configured to encode the syntax element into the bitstream.

As another example, entropy decoding unit 150 may be configured to perform a process that is substantially reciprocal to the process of entropy encoding unit 118 described herein. According to aspects of this disclosure, entropy decoding unit 150 may be configured to entropy decode any codewords generated by entropy encoding unit 118. For example, entropy decoding unit 150 may be configured to entropy decode uniform and non-uniform kth order truncated exponential golomb (TEGk) encoded values, such as binary palette prediction vectors and/or palette maps for CUs. As another example, entropy decoding unit 150 may be configured to entropy decode a kth order exponential golomb (EGk) codeword, a kth order truncated exponential golomb (TEGk) codeword, a kth order non-uniform truncated exponential golomb (TEGk) codeword, or any combination thereof.

NAL units of a bitstream may include coded slice NAL units. As part of decoding the bitstream, entropy decoding unit 150 may extract and entropy decode syntax elements from the coded slice NAL units. Each of the coded slices may include a slice header and slice data. The slice header may contain syntax elements for the slice. The syntax elements in the slice header may include syntax elements that identify a PPS associated with the picture containing the slice.

In addition to decoding syntax elements from the bitstream, video decoder 30 may perform reconstruction operations on the undivided CUs. To perform a reconstruction operation on an undivided CU, video decoder 30 may perform a reconstruction operation on each TU of the CU. By performing a reconstruction operation for each TU of the CU, video decoder 30 may reconstruct residual blocks of the CU.

As part of performing the reconstruction operation on the TUs of the CU, inverse quantization unit 154 may inverse quantize (i.e., dequantize) coefficient blocks associated with the TUs. Inverse quantization unit 154 may use the QP value associated with the CU of the TU to determine the degree of quantization and, likewise, the degree of inverse quantization that inverse quantization unit 154 applies. That is, the compression ratio, i.e., the ratio of the number of bits used to represent the original sequence and the compressed sequence, may be controlled by adjusting the QP value used in quantizing the transform coefficients. The compression ratio may also depend on the method of entropy coding applied.

After inverse quantization unit 154 inverse quantizes the coefficient block, inverse transform processing unit 156 may apply one or more inverse transforms to the coefficient block in order to generate a residual block associated with the TU. For example, inverse transform processing unit 156 may apply an inverse DCT, an inverse integer transform, an inverse Karhunen-Loeve transform (KLT), an inverse rotation transform, an inverse directional transform, or another inverse transform to the coefficient block.

If the PU is encoded using intra prediction, intra prediction processing unit 166 may perform intra prediction to generate the predictive blocks for the PU. Intra-prediction processing unit 166 may use the intra-prediction mode to generate, based on the prediction blocks of the spatially neighboring PUs, the predictive luma block, the predictive Cb block, and the predictive Cr block for the PU. Intra-prediction processing unit 166 may determine the intra-prediction mode for the PU based on one or more syntax elements decoded from the bitstream.

The block decoding unit 152 may construct a first reference picture list (RefPicList0) and a second reference picture list (RefPicList1) based on syntax elements extracted from the bitstream. Furthermore, if the PU is encoded using inter prediction, entropy decoding unit 150 may extract motion information for the PU. Motion compensation unit 164 may determine one or more reference regions of the PU based on the motion information of the PU. Motion compensation unit 164 may generate, based on the sample blocks at the one or more reference blocks of the PU, predictive luma, Cb, and predictive Cr blocks for the PU.

Reconstruction unit 158 may reconstruct the luma, Cb, and Cr coding blocks of the CU using the luma, Cb, and Cr transform blocks associated with the TUs of the CU and the predictive luma, Cb, and Cr blocks of the PUs of the CU (i.e., the intra-prediction data or the inter-prediction data), as appropriate. For example, reconstruction unit 158 may add samples of the luma, Cb, and Cr transform blocks to corresponding samples of the predictive luma, Cb, and Cr blocks to reconstruct luma, Cb, and Cr coding blocks of the CU.

Filter unit 160 may perform deblocking operations to reduce block artifacts associated with luma, Cb, and Cr coding blocks of a CU. Video decoder 30 may store luma, Cb, and Cr coding blocks of the CU in decoded picture buffer 162. Decoded picture buffer 162 may provide reference pictures for subsequent motion compensation, intra prediction, and presentation on a display device, such as display device 32 of fig. 1. For example, video decoder 30 may perform intra-prediction operations or inter-prediction operations on PUs of other CUs based on luma, Cb, and Cr blocks in decoded picture buffer 162.

According to various examples of this disclosure, video decoder 30 may be configured to perform palette-based coding. Palette-based decoding unit 165 may, for example, perform palette-based decoding when a palette-based decoding mode is selected, e.g., for a CU or PU. For example, palette-based decoding unit 165 may be configured to generate a palette having entries indicating pixel values, receive information associating at least some pixel locations in a block of video data with entries in the palette, select pixel values in the palette based on the information, and reconstruct pixel values of the block based on the selected pixel values in the palette. Although various functions are described as being performed by palette-based decoding unit 165, some or all of these functions may be performed by other processing units or a combination of different processing units.

Palette-based decoding unit 165 may receive palette coding mode information and perform the above-described operations when the palette coding mode information indicates that the palette coding mode applies to the block. When the palette coding mode information indicates that the palette coding mode is not applicable to the block, or when other mode information indicates use of a different mode, the palette-based decoding unit 165 decodes the block of video data using a non-palette-based coding mode (e.g., an HEVC inter-prediction coding mode or an HEVC intra-prediction coding mode). A block of video data may be, for example, a CU or PU generated according to an HEVC coding process. The palette-based coding mode may comprise one of a plurality of different palette-based coding modes, or there may be a single palette-based coding mode.

According to aspects of this disclosure, palette-based decoding unit 165 may be configured to perform any combination of the techniques described herein for palette coding. The details of palette coding described above with respect to encoder 20 are not repeated herein with respect to decoder 30, but it should be understood that decoder 30 may perform a reciprocal palette-based decoding process with respect to any palette-based encoding process described herein with respect to encoder 20.

Video decoder 30 may be configured to determine that the current block is coded in palette mode, bypass mode decode the plurality of instances of the first syntax element for reconstructing the current block, e.g., using a bypass mode of a CABAC coding process, and, after bypass mode decoding the plurality of instances of the first syntax element, context mode decode the plurality of instances of the second syntax element for reconstructing the current block, e.g., using a CABAC coding process. Video decoder 30 may bypass mode decode any two instances of the plurality of instances of the first syntax element without interleaving with context mode decoding of any of the plurality of instances of the second syntax element. In some examples, the first syntax element comprises one of a palette _ index _ idc syntax element or a palette _ escape _ val syntax element, and the second syntax element comprises a palette _ run _ msb _ plus1 syntax element. Video decoder 30 may bypass decoding the multiple instances of the first syntax element prior to the index block coded portion for the current block.

Video decoder 30 may decode a third syntax element that indicates a number of instances of the first syntax element, wherein bypass mode decoding the plurality of instances of the first syntax element comprises bypass mode decoding the plurality of instances of the first syntax element based on the third syntax element. Video decoder 30 may decode the third syntax element using golomb rice code, exponential golomb code, truncated rice code, unary code, or a concatenation of golomb rice code and exponential golomb code, or a truncated version of any of the foregoing codes.

Fig. 4 is a conceptual diagram illustrating an example of determining a palette for coding video data consistent with the techniques of this disclosure. The example of fig. 4 includes picture 178 having a first PAL (palette) Coding Unit (CU)180 associated with a first palette 184 and a second PAL CU 188 associated with a second palette 192. As will be described in greater detail below and in accordance with the techniques of this disclosure, second palettes 192 are based on first palettes 184. Picture 178 also includes block 196 coded with an intra-prediction coding mode and block 200 coded with an inter-prediction coding mode.

For purposes of explanation, the techniques of fig. 4 are described in the context of video encoder 20 (fig. 1 and 2) and video decoder 30 (fig. 1 and 3) and with respect to the HEVC video coding standard. However, it should be understood that the techniques of this disclosure are not limited in this manner and may be applied in other video coding processes and/or standards by other video coding processors and/or devices.

In general, a palette refers to a plurality of pixel values that are primary and/or representative for a currently coded CU (CU 188 in the example of fig. 4). First palette 184 (also may be referred to as index 184) and second palette 192 (also may be referred to as index 192) are shown to include multiple palettes (also may be referred to as multiple indices). In some examples, according to aspects of this disclosure, a video coder (e.g., video encoder 20 or video decoder 30) may code a palette (e.g., an index) separately for each color component of a CU. For example, video encoder 20 may encode a palette for a luma (Y) component of the CU, another palette for a chroma (U) component of the CU, and yet another palette for a chroma (V) component of the CU. In this example, an entry of the Y palette may represent a Y value for a pixel of the CU, an entry of the U palette may represent a U value for a pixel of the CU, and an entry of the V palette may represent a V value for a pixel of the CU.

In other examples, video encoder 20 may encode a single palette for all color components of the CU. In this example, video encoder 20 may encode a palette having an ith entry that is a triple value (including Yi, Ui, and Vi). In this case, the palette includes a value for each of the components of the pixel. Accordingly, the representation of palettes 184 and 192 as a palette set having multiple independent palettes is merely one example and is not intended to be limiting.

In the example of fig. 4, first palette 184 includes three entries 202-206 having an entry index value of 1, an entry index value of 2, and an entry index value of 3, respectively. First palette 184 correlates index values (e.g., values shown in the left column of first palette 184) with pixel values. For example, as shown in fig. 4, one of first palettes 184 correlates index values 1,2, and 3 with pixel values A, B and C, respectively. As described herein, a video coder (e.g., video encoder 20 or video decoder 30) may code pixels of a block using indices 1-3 (which may also be expressed as index values 1-3) using palette-based coding, rather than coding actual pixel values of first CU 180. That is, for each pixel position of first CU 180, video encoder 20 may encode an index value for the pixel, where the index value is associated with the pixel value in one or more of first palettes 184. Video decoder 30 may obtain index values from the bitstream and reconstruct the pixel values using the index values and one or more of first palettes 184. Accordingly, first palette 184 is transmitted by video encoder 20 in an encoded video data bitstream for use by video decoder 30 in palette-based decoding.

In some examples, video encoder 20 and video decoder 30 may determine second palettes 192 based on first palettes 184. For example, video encoder 20 and/or video decoder 30 may locate one or more blocks from which a predictive palette (in this example, first palette 184) is determined. In some examples, such as the example illustrated in fig. 4, video encoder 20 and/or video decoder 30 may locate a previously coded CU, such as the left-neighboring CU (first CU 180), when determining the predictive palette for second CU 188.

In the example of fig. 4, second palette 192 includes three entries 208-212 having an entry index value of 1, an entry index value of 2, and an entry index value of 3, respectively. Second palette 192 correlates index values (e.g., the values shown in the left column of first palette 192) with pixel values. For example, as shown in fig. 4, one of second palettes 192 correlates index values 1,2, and 3 with pixel values A, B and D, respectively. In this example, video encoder 20 may code one or more syntax elements that indicate which entries of first palette 184 are included in second palette 192. In the example of fig. 4, one or more syntax elements are illustrated as vectors 216. Vector 216 has a plurality of associated bins (or bits), where each bin indicates whether the palette predictor associated with that bin is used to predict an entry of the current palette. For example, vector 216 indicates that the first two entries (202 and 204) in first palette 184 are included in second palette 192 (value "1" in vector 216), while the third entry in first palette 184 is not included in second palette 192 (value "0" in vector 216). In the example of fig. 4, the vector is a Boolean vector.

In some examples, video encoder 20 and video decoder 30 may determine a palette predictor list (which may also be referred to as a palette predictor table) when performing palette prediction. The palette predictor list may include entries from a palette of one or more neighboring blocks used to predict one or more entries of the palette for coding the current block. Video encoder 20 and video decoder 30 may construct the list in the same manner. Video encoder 20 and video decoder 30 may code data (e.g., vector 216) to indicate which entries of the palette predictor list are to be included in the palette used to code the current block.

FIG. 5 is a conceptual diagram illustrating an example of determining an index to a palette for a block of pixels consistent with the techniques of this disclosure. For example, fig. 5 includes index block 240 (which may also be referred to as map 240 or index map 240) that includes index values (e.g., index values 1,2, and 3) that correlate corresponding positions of pixels associated with the index values to entries of palette 244.

Although index block 240 is illustrated in the example of fig. 5 as including an index value for each pixel position, it should be understood that in other examples, not all pixel positions may be associated with index values that correlate pixel values with entries of palette 244. That is, as mentioned above, in some examples, video encoder 20 may encode (and video decoder 30 may obtain from the encoded bitstream) an indication of the actual pixel values (or quantized versions thereof) for locations in index block 240 if the pixel values are not included in palette 244.

In some examples, video encoder 20 and video decoder 30 may be configured to code additional mappings that indicate which pixel positions are associated with which index values. For example, assume that the (i, j) entry in index block 240 corresponds to the (i, j) location of the CU. Video encoder 20 may encode one or more syntax elements for each entry of the index block (i.e., each pixel location) that indicates whether the entry has an associated index value. For example, video encoder 20 may encode a flag having a value of one to indicate that the pixel value at the (i, j) position in the CU is one of the values in palette 244.

In this example, video encoder 20 may also encode a palette (shown as 244 in the example of fig. 5). In the case where palette 244 includes a single entry and associated pixel values, video encoder 20 may skip signaling of index values. Video encoder 20 may encode a flag with a value of zero to indicate that the pixel value at the (i, j) position in the CU is not one of the values in palette 244. In this example, video encoder 20 may also encode an indication of the pixel values for use by video decoder 30 in reconstructing the pixel values. In some cases, pixel values may be coded in a lossy manner.

The value of a pixel in one location of a CU may provide an indication of the value of one or more other pixels in other locations of the CU. For example, there may be a relatively high probability that neighboring pixel locations of a CU will have the same pixel value or may map to the same index value (in the case of lossy coding, where more than one pixel value may map to a single index value).

Accordingly, video encoder 20 may encode one or more syntax elements indicating a plurality of consecutive pixels or index values having the same pixel value or index value in a given scan order. As mentioned above, strings of identical-valued pixels or index values may be referred to herein as runs. In an example for purposes of illustration, a run is equal to zero if two consecutive pixels or indices in a given scan order have different values. A run is equal to one if two consecutive pixels or indices in a given scan order have the same value but the third pixel or index in the scan order has a different value. For three consecutive indices or pixels with the same value, the run is two, and so on. Video decoder 30 may obtain syntax elements indicating runs from the encoded bitstream and may use the data to determine the number of consecutive positions having the same pixel or index value.

In some examples in accordance with the techniques of this disclosure, entropy encoding unit 118 and entropy decoding unit 150 may be configured to entropy code index block 240. For example, entropy encoding unit 118 and entropy decoding unit 150 may be configured to entropy code run lengths (e.g., run length values or run length codes) and/or binary palette prediction vectors related to blocks of indices in a palette mode.

FIG. 6 is a conceptual diagram illustrating an example of determining a maximum copy-over-run length, assuming a raster scan order, consistent with the techniques of this disclosure. In the example of fig. 6, if none of the pixels covered by dashed line 280 are coded as escape samples, the maximum possible run length is 35 (i.e., the number of pixel positions that are unshaded). If one or more of the pixels within dashed line 280 are coded as escape samples, assuming that the pixel labeled as an escape pixel (the pixel position having an "X") is the first escape pixel in the scan order within dashed line 280, then the most likely coded uppercopy run length is five.

In some examples, video decoder 30 may determine the run mode (e.g., palette mode in which pixels are coded) only for pixels within dashed line 280. Thus, in the worst case, video decoder 30 makes a determination for a BlockWidth-1 pixel. In some examples, video decoder 30 may be configured to implement certain restrictions on the maximum number of pixels for which the run pattern is checked. For example, video decoder 30 may only check pixels within dashed line 280 if the pixels are in the same row as the current pixel. Video decoder 30 may infer that all other pixels within dashed line 280 are not coded as escape samples. The example in fig. 6 assumes a raster scan order. However, the techniques may be applied to other scanning orders, such as vertical, horizontal, and vertical traverses.

According to an example of the present invention, if the current run mode is 'copy-over', the context for the run length of the current pixel may depend on the index value relative to the index of the upper neighboring pixel of the current pixel. In this example, if the top neighboring pixel with respect to the current pixel is outside the current CU, the video decoder assumes that the corresponding index is equal to a predefined constant k. In some examples, k is 0.

During entropy coding, an entropy encoder or decoder may place bits of a symbol to be encoded or decoded as one or more binaries. The binary value may indicate whether the value of the symbol is equal to zero. An entropy coder or entropy decoder may use the value of the binary to adjust the entropy coding process. In some examples, the entropy encoder or entropy decoder may also use a binary to indicate whether a value is greater than a particular value, e.g., greater than zero, greater than one, etc.

In some examples, if the current mode is 'copy-over-above', the first bin of the run-length codeword selects one of the two candidate CABAC contexts based on whether an above-neighboring sample (e.g., pixel) with respect to the current sample (e.g., pixel) is equal to 0.

As another example, if the current mode is 'previous copy', the first bin of the run-length codeword selects one of the four candidate CABAC contexts based on whether the index value is equal to 0, equal to 1, equal to 2, or greater than 2.

Fig. 8 is a flow diagram illustrating an example process for decoding video data consistent with the techniques of this disclosure. For purposes of illustration, the process of fig. 8 is generally described as being performed by video decoder 30, although a variety of other processors may also perform the process shown in fig. 8. In some examples, block decoding unit 152, palette-based decoding unit 165, and/or entropy decoding unit 150 may perform one or more processes shown in fig. 8.

In the example of fig. 8, video decoder 30 may be configured to receive a palette mode encoded block of video data of a picture from an encoded video bitstream (800). Video decoder 30 may be configured to receive encoded palette mode information for a palette mode encoded block of video data from an encoded video bitstream (802). In some examples, the encoded palette mode information may include a plurality of instances of the first syntax element and a plurality of syntax elements different from the first syntax element. For example, the first syntax element may include a palette _ index _ idc or palette _ escape _ val, and the plurality of syntax elements different from the first syntax element may include a palette _ run _ msb _ id _ plus1 syntax element. As another example, the first syntax element may be an indication of an index to an array of palette entries, or the first syntax element may specify quantized escape coded sample values for color components corresponding to the escape samples. The plurality of syntax elements different from the first syntax element may include a syntax element that specifies an index of a most significant bit in a binary representation of a variable representing the run length and a syntax element that specifies the run type mode.

As another example, the plurality of syntax elements that are different from the first syntax element may be any syntax element and all syntax elements that are different from the first syntax element. As described herein with respect to some examples, the plurality of syntax elements that are different from the first syntax element may also be different from the second syntax element, the third syntax element, and/or the fourth syntax element. In these examples, the plurality of syntax elements that are different from the first syntax element, the second syntax element, the third syntax element, and the fourth syntax element can be any syntax element and all syntax elements that are different from the first syntax element, the second syntax element, the third syntax element, and/or the fourth syntax element. In some examples, the plurality of syntax elements that are different from the first syntax element may be any syntax element and all syntax elements that are not decoded by bypass mode and/or are not decoded by bypass mode.

Video decoder 30 may be configured to decode multiple instances of the first syntax element using a bypass mode (e.g., a bypass mode of a CABAC coding process) before decoding multiple syntax elements different from the first syntax element using a context mode (804). Video decoder 30 may be configured to decode, using context mode (e.g., conventional CABAC mode (instead of bypass mode)) after decoding the plurality of instances of the first syntax element using bypass mode, a plurality of syntax elements that are different from the first syntax element (806). In some examples, the plurality of instances of the first syntax element includes all instances of the first syntax element for the palette mode encoded block of video data. In these examples, all instances of the first syntax element are decoded using bypass mode prior to decoding any subsequent data (e.g., a plurality of syntax elements different from the first syntax element). Stated otherwise, video decoder 30 may be configured to decode, using context mode, a plurality of syntax elements different from a first syntax element for a palette mode encoded block of video data after decoding all instances of the first syntax element using bypass mode.

Video decoder 30 may be configured to decode the palette mode encoded block of video data using the decoded plurality of instances of the first syntax element and the decoded plurality of syntax elements that are different from the first syntax element (808). In some examples, multiple instances of the first syntax element are grouped together such that switching between bypass mode and context mode is reduced when decoding the palette mode encoded block of video data.

In some examples, the encoded palette mode information may include a second syntax element that indicates a number of instances of the first syntax element (e.g., indicating how many instances of the first syntax element are present for the palette mode encoded block of video data). The plurality of syntax elements that are different from the first syntax element may also be different from the second syntax element. In these examples, video decoder 30 may be configured to decode the second syntax element using bypass mode prior to decoding a plurality of syntax elements different from the first syntax element and the second syntax element. In some examples, the instances without the second syntax element are interleaved between any two instances of the first syntax element for the palette mode encoded block of video data. In some examples, video decoder 30 may be configured to, after decoding a plurality of instances of the first syntax element equal to the number indicated by the second syntax element, determine that subsequent data in the encoded video bitstream after the number of instances of the first syntax element correspond to a plurality of syntax elements that are different from the first syntax element and the second syntax element. In some examples, video decoder 30 may be configured to decode the second syntax element using a concatenation of a truncated rice code and an exponential golomb code.

In some examples, the encoded palette mode information may include a third syntax element and a fourth syntax element. In these examples, video decoder 30 may be configured to decode the third syntax element to determine whether the value corresponding to the third syntax element indicates that the palette mode encoded block of video data includes escape pixels. Video decoder 30 may be configured to decode the fourth syntax element to determine that the value corresponding to the fourth syntax element indicates the palette size. Video decoder 30 may be configured to, after decoding the plurality of instances of the first syntax element and the second syntax element using bypass mode, decode a plurality of syntax elements that are different from the first syntax element and the second syntax element using context mode based on the determined values corresponding to the third syntax element and the fourth syntax element, respectively.

In some examples, the encoded palette mode information may include another syntax element, and video decoder 30 may be configured to decode this other syntax element to determine a value corresponding to this other syntax element that specifies a number of different values that the palette index has for the palette mode encoded block of video data. Video decoder 30 may be configured to decode, using context mode, a plurality of syntax elements that are different from the first syntax element and the second syntax element based on the determined values corresponding to this other syntax element after decoding the plurality of instances of the first syntax element and the second syntax element using bypass mode.

In some examples, the encoded palette mode information may include another syntax element, and video decoder 30 may be configured to decode this other syntax element to determine that the value corresponding to this other syntax element indicates the last instance of the syntax element for the palette _ run _ type _ flag [ xC ] [ yC ] for the palette mode encoded block of video data.

In some examples, video decoder 30 may be configured to determine that an encoded block of video data has one or more escape samples. In these examples, video decoder 30 may be configured to decode a last escape sample of the one or more escape samples in the encoded block of video data. Video decoder 30 may be configured to infer an index value applicable to a sample of the encoded block of video data that follows the last escape sample. Video decoder 30 may be configured to decode samples following the last escape sample of the encoded block of video data using the inferred index values for each sample of the samples following the last escape sample.

In some examples, video decoder 30 may be configured to determine a number of palette indices received. In these examples, video decoder 30 may be configured to determine a number of palette indices remaining based on the number of palette indices received and the number of instances of the first syntax element. Video decoder 30 may be configured to determine a maximum possible run value for the encoded block of video data based on the number of palette indices received and the number of instances of the first syntax element. In some examples, video decoder 30 may be configured to determine the maximum possible run value for the encoded block of video data from nCbS × nCbS-scanPos-1-palette indendecicep left, where nCbS specifies the size of the encoded block of video data, scanPos specifies the scan position, and palette indendecicep specifies the number of palette indices remaining.

Fig. 9 is a flow diagram illustrating an example process for encoding video data consistent with the techniques of this disclosure. For purposes of illustration, the process of fig. 9 is generally described as being performed by video encoder 20, although a variety of other processors may also perform the process shown in fig. 9. In some examples, block encoding unit 100, palette-based encoding unit 122, and/or entropy encoding unit 118 may perform one or more processes shown in fig. 9.

In the example of fig. 9, video encoder 20 may be configured to determine that a block of video data is to be encoded in palette mode (900). Video encoder 20 may be configured to encode the block of video data into an encoded bitstream using a palette mode (902). In some examples, video encoder 20 may be configured to generate palette mode information for a block of video data (904). The palette mode information may include a plurality of instances of the first syntax element and a plurality of syntax elements different from the first syntax element. For example, the first syntax element may include a palette _ index _ idc or palette _ escape _ val, and the plurality of syntax elements different from the first syntax element may include a palette _ run _ msb _ id _ plus1 syntax element. As another example, the first syntax element may be an indication of an index to an array of palette entries, or the first syntax element may specify quantized escape coded sample values for color components corresponding to the escape samples. The plurality of syntax elements different from the first syntax element may include a syntax element that specifies an index of a most significant bit in a binary representation of a variable representing the run length and a syntax element that specifies the run type mode.

As another example, the plurality of syntax elements that are different from the first syntax element may be any syntax element and all syntax elements that are different from the first syntax element. As described herein with respect to some examples, the plurality of syntax elements that are different from the first syntax element may also be different from the second syntax element, the third syntax element, and/or the fourth syntax element. In these examples, the plurality of syntax elements that are different from the first syntax element, the second syntax element, the third syntax element, and the fourth syntax element can be any syntax element and all syntax elements that are different from the first syntax element, the second syntax element, the third syntax element, and/or the fourth syntax element. In some examples, the plurality of syntax elements that are different from the first syntax element may be any syntax element and all syntax elements that are not encoded by the bypass mode and/or are not encoded by the bypass mode.

Video encoder 20 may be configured to encode the plurality of instances of the first syntax element into an encoded bitstream using a bypass mode (e.g., a bypass mode of a CABAC coding process) before encoding the plurality of syntax elements different from the first syntax element into the encoded bitstream using a context mode (906). Video encoder 20 may be configured to encode a plurality of syntax elements that are different from the first syntax element into an encoded bitstream using a context mode (e.g., a conventional CABAC context-based mode) after encoding the plurality of instances of the first syntax element into the encoded bitstream using the bypass mode (908). In some examples, multiple instances of the first syntax element are grouped together such that switching between bypass mode and context mode is reduced when encoding the palette mode encoded block of video data.

In some examples, the plurality of instances of the first syntax element includes all instances of the first syntax element for the block of video data. In these examples, all instances of the first syntax element are encoded using the bypass mode prior to encoding any subsequent data (e.g., a plurality of syntax elements different from the first syntax element). Stated otherwise, video encoder 20 may be configured to encode a plurality of syntax elements different from the first syntax element using context mode after encoding all instances of the first syntax element for the block of video data using bypass mode.

In some examples, the palette mode information may include a second syntax element that indicates a number of instances of the first syntax element (e.g., indicating a number of instances of the first syntax element that are present for the block of video data). The plurality of syntax elements that are different from the first syntax element may also be different from the second syntax element. In these examples, video encoder 20 may be configured to encode the second syntax element into the encoded bitstream using the bypass mode prior to encoding the plurality of syntax elements that are different from the first syntax element and the second syntax element. In some examples, video encoder 20 may be configured to encode the plurality of instances of the first syntax element such that no instances of the second syntax element are interleaved between any two instances of the first syntax element for the palette mode encoded block of video data in the encoded bitstream. In some examples, video encoder 20 may be configured to encode the second syntax element into the encoded bitstream after the encoded plurality of instances of the first syntax element in the encoded bitstream. For example, video encoder 20 may be configured to first encode all instances of the first syntax element, and then encode the second syntax element into an encoded bitstream. In some examples, video encoder 20 may be configured to encode the second syntax element using a concatenation of a truncated rice code and an exponential golomb code.

In some examples, the palette mode information may include a third syntax element and a fourth syntax element. In these examples, video encoder 20 may be configured to encode a value corresponding to a third syntax element into an encoded bitstream, the third syntax element indicating whether the block of video data includes escape pixels. Video encoder 20 may be configured to cause the value corresponding to the fourth syntax element that indicates the palette size to be an encoded bitstream. In some examples, the palette mode information may include another syntax element, and video encoder 20 may be configured to encode a value of this other syntax element corresponding to a number of different values that the specified palette index has for the block of video data into an encoded bitstream.

In some examples, the encoded palette mode information may include another syntax element, and video encoder 20 may be configured to encode a value corresponding to this other syntax element that indicates a last instance of a syntax element for a palette _ run _ type _ flag [ xC ] [ yC ] for a block of video data.

In some examples, video encoder 20 may be configured to encode a last escape sample of the one or more escape samples in the block of video data. In these examples, video encoder 20 may be configured to infer an index value applicable to a sample of the block of video data that follows the last escape sample. Video encoder 20 may be configured to encode the samples of the block of video data that follow the last escape sample using the inferred index value for each of the samples that follow the last escape sample.

It should be understood that all of the techniques described herein may be used alone or in combination. For example, video encoder 20 and/or one or more components thereof and video decoder 30 and/or one or more components thereof may perform the techniques described in this disclosure in any combination.

It should be recognized that depending on the example, certain acts or events of any of the techniques described herein may be performed in a different sequence, may be added, merged, or omitted entirely (e.g., not all described acts or events are necessary for the practice of the techniques). Further, in some examples, acts or events may be performed concurrently, e.g., via multi-threaded processing, interrupt processing, or multiple processors, rather than sequentially. Additionally, although certain aspects of this disclosure are described as being performed by a single module or unit for purposes of clarity, it should be understood that the techniques of this disclosure may be performed by a combination of units or modules associated with a video coder.

For purposes of illustration, certain aspects of this disclosure have been described with respect to the developing HEVC standard. However, the techniques described in this disclosure may be applicable to other video coding processes, including other standard or proprietary video coding processes that have not yet been developed.

The techniques described above may be performed by video encoder 20 (fig. 1 and 2) and/or video decoder 30 (fig. 1 and 3), both of which may be referred to generally as video coders. Likewise, video coding, where applicable, may refer to video encoding or video decoding.

In accordance with this disclosure, the term "or" may be interpreted as "and/or," where the context does not otherwise dictate. Additionally, while phrases such as "one or more" or "at least one" or the like may have been used for some features disclosed herein (but not others); features not using this language may be construed as having such meaning to imply that the context does not dictate otherwise.

While specific combinations of various aspects of the techniques are described above, these combinations are provided merely to illustrate examples of the techniques described in this disclosure. Thus, the techniques of this disclosure should not be limited to these example combinations and may encompass any conceivable combination of the various aspects of the techniques described in this disclosure.

In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. The computer-readable medium may comprise a computer-readable storage medium, corresponding to a tangible medium, such as a data storage medium, or a communication medium comprising any medium that facilitates transfer of a computer program from one place to another, such as according to a communication protocol. In this manner, the computer-readable medium may generally correspond to (1) a tangible computer-readable storage medium that is not transitory, or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, program code, and/or data structures for implementing the techniques described in this disclosure. The computer program product may include a computer-readable medium.

By way of example, and not limitation, such computer-readable storage media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if the instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. It should be understood, however, that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but rather pertain to non-transitory tangible storage media. Disk and disc, as used herein, includes Compact Disc (CD), laser disc, optical disc, Digital Versatile Disc (DVD), floppy disk and blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

The instructions may be executed by one or more processors, such as one or more Digital Signal Processors (DSPs), general purpose microprocessors, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Thus, the term "processor," as used herein may refer to any of the foregoing structure or any other structure suitable for implementation of the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided within dedicated hardware and/or software modules configured for encoding and decoding, or incorporated in a combined codec. Also, the techniques may be fully implemented in one or more circuits or logic elements.

The techniques of this disclosure may be implemented in various means or devices including a wireless handset, an Integrated Circuit (IC), or a collection of ICs (e.g., a chipset). Various components, modules, or units are described in this disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require realization by different hardware units. Instead, as described above, the various units may be combined in a codec hardware unit, or provided by a collection of interoperability hardware units (including one or more processors as described above) in conjunction with suitable software and/or firmware.

Various examples have been described herein. Any combination of the described systems, operations, functions, or examples is contemplated. These and other examples are within the scope of the following claims.

Claims (45)

1. A method of decoding video data, the method comprising:

receiving a palette mode encoded block of video data of a picture from an encoded video bitstream;

receiving, from the encoded video bitstream, encoded palette mode information for the palette mode encoded block of video data, wherein the encoded palette mode information includes a plurality of instances of a first syntax element and a plurality of syntax elements that are different from the first syntax element;

decoding the plurality of instances of the first syntax element using bypass mode prior to decoding the plurality of syntax elements different from the first syntax element using context mode;

decoding the plurality of syntax elements that are different from the first syntax element using context mode after decoding the plurality of instances of the first syntax element using bypass mode; and

decoding the palette mode encoded block of video data using the decoded plurality of instances of the first syntax element and the decoded plurality of syntax elements that are different from the first syntax element.

2. The method of claim 1, wherein the plurality of instances of the first syntax element includes all instances of the first syntax element for the palette mode encoded block of video data.

3. The method of claim 1, wherein the first syntax element is an indication of an index to an array of palette entries or specifies a quantized escape coded sample value for a color component corresponding to an escape sample, and wherein the plurality of syntax elements that are different from the first syntax element includes a syntax element that specifies an index of a most significant bit in a binary representation of a variable representing a run length and a syntax element that specifies a run type mode.

4. The method of claim 1, wherein the first syntax element is a palette _ index _ idc or palette _ escape _ val, and wherein the plurality of syntax elements different from the first syntax element includes a palette _ run _ msb _ id _ plus1 syntax element.

5. The method of claim 1, wherein the plurality of instances of the first syntax element are grouped together such that switching between bypass mode and context mode is reduced when decoding the palette mode encoded block of video data.

6. The method of claim 1, wherein the encoded palette mode information includes a second syntax element indicating a number of instances of the first syntax element, wherein the plurality of syntax elements that are different from the first syntax element are different from the second syntax element, and wherein the method further comprises:

decoding the second syntax element using bypass mode prior to the decoding of the plurality of syntax elements that are different from the first syntax element and the second syntax element.

7. The method of claim 6, wherein no instance of the second syntax element is interleaved between any two instances of the first syntax element for the palette mode encoded block of video data.

8. The method of claim 6, further comprising:

determining that subsequent data in the encoded video bitstream that is subsequent to the number of instances of the first syntax element after decoding a number of instances of the first syntax element that is equal to the number indicated by the second syntax element corresponds to the plurality of syntax elements that is different from the first syntax element and the second syntax element.

9. The method of claim 6, wherein the encoded palette mode information includes a third syntax element and a fourth syntax element, wherein the method further comprises:

decoding the third syntax element to determine whether a value corresponding to the third syntax element indicates whether the palette-mode encoded block of video data includes escape samples;

decoding the fourth syntax element to determine that a value corresponding to the fourth syntax element indicates a palette size; and

decoding, after decoding the plurality of instances of the first syntax element and the second syntax element using bypass mode, the plurality of syntax elements that are different from the first syntax element and the second syntax element using context mode based on the determined values corresponding to the third syntax element and the fourth syntax element, respectively.

10. The method of claim 6, wherein the encoded palette mode information includes a third syntax element, wherein the method further comprises:

decoding the third syntax element to determine a value corresponding to the third syntax element, the third syntax element specifying a number of different values that a palette index has for the palette mode encoded block of video data; and

decoding, using context mode, the plurality of syntax elements that are different from the first syntax element and the second syntax element based on the determined value corresponding to the third syntax element after decoding the plurality of instances of the first syntax element and the second syntax element using bypass mode.

11. The method of claim 6, wherein the encoded palette mode information includes a third syntax element, wherein the method further comprises:

decoding the third syntax element to determine that a value corresponding to the third syntax element indicates a last instance of a syntax element of a palette _ run _ type _ flag [ xC ] [ yC ] for the palette mode encoded block of video data.

12. The method of claim 6, further comprising:

decoding the second syntax element using a concatenation of a truncated Rice code and an exponential Golomb code.

13. The method of claim 1, further comprising:

determining that the encoded block of video data has one or more escape samples;

decoding a last escape sample of the one or more escape samples in the encoded block of video data;

inferring an index value applicable to a sample of the encoded block of video data that follows the last escape sample; and

decoding the samples of the encoded block of video data that follow the last escape sample using the inferred index values for each of the samples that follow the last escape sample.

14. The method of claim 6, further comprising:

determining a number of palette indices received;

determining a number of palette indices remaining based on the number of palette indices received and the number of instances of the first syntax element; and

determining a maximum possible run value for the encoded block of video data based on the number of palette indices received and the number of instances of the first syntax element.

15. The method of claim 14, further comprising:

determining the maximum possible run value for the encoded block of video data from nCbS-scanPos-1-palette indendecicep left, wherein nCbS specifies a size of the encoded block of video data, scanPos specifies a scan position, and palette indendecicep left specifies the number of remaining palette indices.

16. A device for decoding video data, the device comprising:

a memory configured to store the video data; and

a video decoder in communication with the memory, the video decoder configured to:

receiving, from the memory, a palette mode encoded block of video data of a picture;

receiving encoded palette mode information for the palette mode encoded block of video data, wherein the encoded palette mode information includes a plurality of instances of a first syntax element and a plurality of syntax elements that are different from the first syntax element;

decoding the plurality of instances of the first syntax element using bypass mode prior to decoding the plurality of syntax elements different from the first syntax element using context mode;

decoding the plurality of syntax elements that are different from the first syntax element using context mode after decoding the plurality of instances of the first syntax element using bypass mode; and

decoding the palette mode encoded block of video data using the decoded plurality of instances of the first syntax element and the decoded plurality of syntax elements that are different from the first syntax element.

17. The device of claim 16, wherein the plurality of instances of the first syntax element includes all instances of the first syntax element for the palette mode encoded block of video data.

18. The device of claim 16, wherein the first syntax element is an indication of an index to an array of palette entries or specifies a quantized escape coded sample value for a color component corresponding to an escape sample, and wherein the plurality of syntax elements that are different from the first syntax element includes a syntax element that specifies an index of a most significant bit in a binary representation of a variable representing a run length and a syntax element that specifies a run type mode.

19. The device of claim 16, wherein the first syntax element is a palette _ index _ idc or a palette _ escape _ val, and wherein the plurality of syntax elements different from the first syntax element includes a palette _ run _ msb _ id _ plus1 syntax element.

20. The device of claim 16, wherein the plurality of instances of the first syntax element are grouped together such that switching between bypass mode and context mode is reduced when decoding the palette mode encoded block of video data.

21. The device of claim 16, wherein the encoded palette mode information includes a second syntax element indicating a number of instances of the first syntax element, wherein the plurality of syntax elements that are different from the first syntax element are different from the second syntax element, and wherein the video decoder is further configured to:

decoding the second syntax element using bypass mode prior to the decoding of the plurality of syntax elements that are different from the first syntax element and the second syntax element.

22. The device of claim 21, wherein no instance of the second syntax element is interleaved between any two instances of the first syntax element for the palette mode encoded block of video data.

23. The device of claim 21, wherein the video decoder is further configured to determine, after decoding a number of instances of the first syntax element equal to the number indicated by the second syntax element, that subsequent data in the encoded video bitstream that follows the number of instances of the first syntax element correspond to the plurality of syntax elements that are different from the first syntax element and the second syntax element.

24. The device of claim 21, wherein the encoded palette mode information includes a third syntax element and a fourth syntax element, wherein the video decoder is further configured to:

decoding the third syntax element to determine whether a value corresponding to the third syntax element indicates whether the palette-mode encoded block of video data includes escape samples;

decoding the fourth syntax element to determine that a value corresponding to the fourth syntax element indicates a palette size; and

decoding, after decoding the plurality of instances of the first syntax element and the second syntax element using bypass mode, the plurality of syntax elements that are different from the first syntax element and the second syntax element using context mode based on the determined values corresponding to the third syntax element and the fourth syntax element, respectively.

25. The device of claim 21, wherein the encoded palette mode information includes a third syntax element, wherein the video decoder is further configured to:

decoding the third syntax element to determine a value corresponding to the third syntax element, the third syntax element specifying a number of different values that a palette index has for the palette mode encoded block of video data; and

decoding, using context mode, the plurality of syntax elements that are different from the first syntax element and the second syntax element based on the determined value corresponding to the third syntax element after decoding the plurality of instances of the first syntax element and the second syntax element using bypass mode.

26. The device of claim 21, wherein the encoded palette mode information includes a third syntax element, wherein the video decoder is further configured to:

decoding the third syntax element to determine that a value corresponding to the third syntax element indicates a last instance of a syntax element of a palette _ run _ type _ flag [ xC ] [ yC ] for the palette mode encoded block of video data.

27. The device of claim 21, wherein the video decoder is further configured to:

decoding the second syntax element using a concatenation of a truncated Rice code and an exponential Golomb code.

28. The device of claim 16, wherein the video decoder is further configured to:

determining that the encoded block of video data has one or more escape samples;

decoding a last escape sample of the one or more escape samples in the encoded block of video data;

inferring an index value applicable to a sample of the encoded block of video data that follows the last escape sample; and

decoding the samples of the encoded block of video data that follow the last escape sample using the inferred index values for each of the samples that follow the last escape sample.

29. The device of claim 21, wherein the video decoder is further configured to:

determining a number of palette indices received;

determining a number of palette indices remaining based on the number of palette indices received and the number of instances of the first syntax element; and

determining a maximum possible run value for the encoded block of video data based on the number of palette indices received and the number of instances of the first syntax element.

30. The device of claim 29, wherein the video decoder is further configured to:

determining the maximum possible run value for the encoded block of video data from nCbS-scanPos-1-palette indendecicep left, wherein nCbS specifies a size of the encoded block of video data, scanPos specifies a scan position, and palette indendecicep left specifies the number of remaining palette indices.

31. A non-transitory computer-readable storage medium having instructions stored thereon that, when executed, cause one or more processors to:

receiving, from a memory, a palette mode encoded block of video data for a picture;

receiving encoded palette mode information for the palette mode encoded block of video data, wherein the encoded palette mode information includes a plurality of instances of a first syntax element and a plurality of syntax elements that are different from the first syntax element;

decoding the plurality of instances of the first syntax element using bypass mode prior to decoding the plurality of syntax elements different from the first syntax element using context mode;

decoding the plurality of syntax elements that are different from the first syntax element using context mode after decoding the plurality of instances of the first syntax element using bypass mode; and

decoding the palette mode encoded block of video data using the decoded plurality of instances of the first syntax element and the decoded plurality of syntax elements that are different from the first syntax element.

32. A method of encoding video data, the method comprising:

determining that a block of video data is to be coded in palette mode;

encoding the block of video data into an encoded bitstream using a palette mode, wherein encoding the block of video data using the palette mode comprises:

generating palette mode information for the block of video data, wherein the palette mode information includes a plurality of instances of a first syntax element and a plurality of syntax elements that are different from the first syntax element;

encoding the plurality of instances of the first syntax element into the encoded bitstream using bypass mode before encoding the plurality of syntax elements different from the first syntax element into the encoded bitstream using context mode; and

encoding the plurality of syntax elements that are different from the first syntax element into the encoded bitstream using context mode after encoding the plurality of instances of the first syntax element into the encoded bitstream using bypass mode.

33. The method of claim 32, wherein the plurality of instances of the first syntax element includes all instances of the first syntax element for the block of video data.

34. The method of claim 32, wherein the first syntax element is an indication of an index to an array of palette entries or specifies a quantized escape coded sample value for a color component corresponding to an escape sample, and wherein the plurality of syntax elements that are different from the first syntax element includes a syntax element that specifies an index of a most significant bit in a binary representation of a variable representing a run length and a syntax element that specifies a run type mode.

35. The method of claim 32, wherein the first syntax element is a palette _ index _ idc or palette _ escape _ val, and wherein the plurality of syntax elements other than the first syntax element includes a palette _ run _ msb _ id _ plus1 syntax element.

36. The method of claim 32, wherein the plurality of instances of the first syntax element are grouped together such that switching between bypass mode and context mode is reduced when encoding the palette mode encoded block of video data.

37. The method of claim 32, wherein the palette mode information includes a second syntax element indicating a number of instances of the first syntax element, wherein the plurality of syntax elements that are different from the first syntax element are different from the second syntax element, and wherein the method further comprises:

encoding the second syntax element into the encoded bitstream using bypass mode prior to the encoding of the plurality of syntax elements that are different from the first syntax element and the second syntax element.

38. The method of claim 37, wherein no instance of the second syntax element is interleaved between any two instances of the first syntax element for the block of video data.

39. The method of claim 37, further comprising:

encoding the second syntax element into the encoded bitstream after the encoded plurality of instances of the first syntax element in the encoded bitstream.

40. The method of claim 37, wherein the palette mode information includes a third syntax element and a fourth syntax element, wherein the method further comprises:

encoding a value corresponding to the third syntax element that indicates whether the block of video data includes escape samples into the encoded bitstream; and

encoding a value corresponding to the fourth syntax element that indicates a palette size into the encoded bitstream.

41. The method of claim 37, wherein the palette mode information includes a third syntax element, wherein the method further comprises:

encoding a value corresponding to the third syntax element into the encoded bitstream, the third syntax element specifying a number of different values that a palette index has for the block of video data.

42. The method of claim 37, wherein the palette mode information includes a third syntax element, wherein the method further comprises:

encoding a value corresponding to the third syntax element indicating a last instance of a syntax element of palette _ run _ type _ flag [ xC ] [ yC ] for the block of video data.

43. The method of claim 37, further comprising:

encoding the second syntax element using a concatenation of a truncated Rice code and an exponential Golomb code.

44. The method of claim 32, further comprising:

encoding a last escape sample of the one or more escape samples in the block of video data;

inferring an index value applicable to a sample of the block of video data that follows the last escape sample; and

encoding the samples of the block of video data that follow the last escape sample using the inferred index values for each of the samples that follow the last escape sample.

45. A device for encoding video data, the device comprising:

a memory configured to store the video data; and

a video encoder in communication with the memory, the video encoder configured to:

determining that a block of video data stored in the memory is to be encoded in palette mode;

encoding the block of video data into an encoded bitstream using a palette mode, wherein the video encoder configured to encode the block of video data using the palette mode comprises the video encoder configured to:

generating palette mode information for the block of video data, wherein the palette mode information includes a plurality of instances of a first syntax element and a plurality of syntax elements that are different from the first syntax element;

encoding the plurality of instances of the first syntax element into the encoded bitstream using bypass mode before encoding the plurality of syntax elements different from the first syntax element into the encoded bitstream using context mode; and

encoding the plurality of syntax elements that are different from the first syntax element into the encoded bitstream using context mode after encoding the plurality of instances of the first syntax element into the encoded bitstream using bypass mode.