CN120569964A

CN120569964A - Method and apparatus for improving transform information coding in video coding based on intra chroma cross-element prediction model

Info

Publication number: CN120569964A
Application number: CN202480007567.3A
Authority: CN
Inventors: 庄政彦; 蔡佳铭; 曾馨仪; 江嫚书; 徐志玮; 陈渏纹; 陈庆晔; 庄子德
Original assignee: MediaTek Inc
Current assignee: MediaTek Inc
Priority date: 2023-01-12
Filing date: 2024-01-10
Publication date: 2025-08-29
Also published as: WO2024149293A1

Abstract

A method and apparatus for video encoding and decoding, using a codec tool including one or more cross-component model-related modes. According to a method on the decoder side, codec syntax data of one or more syntaxes related to transform coding or residual coding applied to a current block are parsed. Based on information related to cross-component prediction (Cross-Component Prediction, abbreviated as CCP), the codec syntax data is decoded using one or more context models to generate the one or more syntaxes. The current block is decoded using the CCP, wherein the decoding includes transform coding or residual coding of the current block using the one or more syntaxes. On the encoder side, one or more syntaxes related to transform coding or residual coding are encoded to generate codec syntax data, and one or more context models are used based on information related to the CCP.

Description

Method and apparatus for improving transform information coding in video coding based on intra chroma cross-element prediction model

[ Cross-reference ]

The present invention is a non-provisional application of U.S. provisional patent application No. 63/479,556 (application date 2023, 1 month, 12) and U.S. provisional patent application No. 63/487,633 (application date 2023, 3 month, 1), and claims priority. The above-mentioned U.S. provisional patent application is incorporated by reference herein in its entirety.

[ Field of technology ]

The present invention relates to video encoding and decoding systems. In particular, the present invention relates to a codec syntax for transform or residual coding using one or more context models based on information related to cross-element prediction in a video codec system.

[ Background Art ]

Multifunctional video codec (VERSATILE VIDEO CODING, VVC for short) is the latest international video codec standard developed by the joint video expert team (Joint Video Experts Team, JVET) of ITU-T video codec expert group (Video Coding Experts Group, VCEG for short) and ISO/IEC dynamic picture expert group (Moving Picture Experts Group, MPEG for short). The standard has been published as an ISO standard, ISO/IEC 23090-3:2021, information technology-coded representation of immersive media-part 3 multifunctional video codec, published in 2021, month 2. VVC was developed on the basis of its advanced High Efficiency Video Coding (HEVC), which increases Coding efficiency by adding more Coding tools, and handles various types of Video sources, including three-dimensional (3D) Video signals.

Fig. 1A illustrates an exemplary adaptive inter/intra video coding system that includes loop processing. For intra prediction 110, the prediction data is derived based on previously encoded video data in the current picture. For inter prediction 112, motion estimation (Motion Estimation, abbreviated ME) is performed at the encoder side and motion compensation (Motion Compensation, abbreviated MC) is performed based on the results of ME to provide prediction data derived from other pictures and motion data. The switch 114 selects either the intra prediction 110 or the inter prediction 112, and the selected prediction data is provided to the adder 116 to form a prediction error, also referred to as a residual. The prediction error is then processed by Transform (T) 118, followed by Quantization (Q) 120. The transformed and quantized residual is then encoded by entropy encoder (EntropyEncoder) 122 for inclusion in a video bitstream corresponding to the compressed video data. The bitstream associated with the transform coefficients is then packaged with additional information such as motion and codec modes associated with intra and inter prediction, and other information such as loop filter related parameters applied to the underlying image region. Additional information related to intra prediction 110, inter prediction 112, and loop filter 130 is provided to entropy encoder 122 as shown in fig. 1A. When inter prediction modes are used, reference pictures or pictures must also be reconstructed at the encoder side. Thus, the transformed and quantized residual is processed through inverse quantization (Inverse Quantization, IQ) 124 and inverse transformation (Inverse Transformation, IT) 126 to recover the residual. The residual is then added back to the prediction data 136 at Reconstruction (REC) 128 to reconstruct the video data. The reconstructed video data may be stored in a reference picture buffer (REFERENCE PICTURE BUFFER) 134 and used for prediction of other frames.

As shown in fig. 1A, input video data undergoes a series of processes in an encoding system. The reconstructed video data from the REC128 may suffer from various impairments due to a series of processing. Thus, loop filter 130 is typically applied to the reconstructed video data before it is stored in reference picture buffer 134 to improve video quality. For example, a deblocking filter (Deblocking Filter, DF for short), a sample adaptive Offset (SAMPLE ADAPTIVE Offset, SAO for short), and an adaptive loop filter (Adaptive Loop Filter, ALF for short) may be used. Loop filter information may need to be incorporated into the bitstream in order for the decoder to be able to correctly recover the required information. Thus, loop filter information is also provided to entropy encoder 122 for incorporation into the bitstream. In fig. 1A, loop filter 130 is applied to the reconstructed video before the reconstructed samples are stored in reference picture buffer 134. The system in fig. 1A is intended to show an exemplary architecture of a typical video encoder. It may correspond to a High Efficiency Video Coding (HEVC) system, VP8, VP9, h.264, or VVC.

As shown in fig. 1B, the decoder may use the same or partially the same functional blocks as the encoder except for the transform 118 and quantization 120, as the decoder only requires inverse quantization 124 and inverse transform 126. The decoder uses an entropy decoder (Entropy Decoder) 140 to decode the video bitstream into quantized transform coefficients and required codec information (e.g., ILPF information, intra-prediction information, and inter-prediction information). The intra prediction 150 at the decoder side does not need to perform a mode search. Instead, the decoder need only generate intra prediction from the intra prediction information received from the entropy decoder 140. In addition, for inter prediction, the decoder only needs to perform motion compensation (MC 152) based on inter prediction information received from the entropy decoder 140, without motion estimation.

According to VVC, an input picture is divided into non-overlapping square areas called codec tree units (Coding Tree Units, CTUs for short), similar to HEVC. Each CTU may be divided into one or more smaller sized codec Units (Coding Units, CUs for short). The generated CU partition may be square or rectangular in shape. In addition, the VVC divides CTUs into Prediction Units (PUs) as Units to which a Prediction process is applied, such as inter Prediction, intra Prediction, and the like.

The VVC standard incorporates various new codec tools to further improve codec efficiency beyond the HEVC standard. Some new tools relevant to the present invention are as follows.

Partition CTUs using tree structures

In HEVC, CTUs are divided into CUs, called coding trees, by using a quad-tree (QT) structure to accommodate various local characteristics. The decision whether to encode a picture region using inter (temporal) or intra (spatial) prediction is made at the leaf CU level. Each leaf CU may be further divided into one, two, or four PUs according to the PU division type. Within one PU, the same prediction process is applied and related information is transmitted to the decoder on a PU basis. After the residual block is obtained after the prediction process is applied according to the PU partition type, the leaf CU may be partitioned into Transform Units (TUs) according to another quadtree structure similar to the CU's coding tree. One key feature of the HEVC structure is that it has a multiple partition concept that includes CUs, PUs, and TUs.

In VVC, the split structure of quadtrees and nested multi-type trees using binary and ternary splitting replaces the concept of multiple split unit types, i.e. removes the separation of CU, PU and TU concepts and supports more flexibility of CU split shapes, in addition to the maximum transform length that needs to handle oversized CUs. In the coding tree structure, a CU may be square or rectangular. A Codec Tree Unit (CTU) is first partitioned by a quad-tree (also called quadtree) structure. The quaternary leaf nodes may then be further partitioned by a multi-type tree structure. As shown in FIG. 2, there are four partition types in the multi-type tree structure, vertical binary partition (SPLIT_BT_VER 210), horizontal binary partition (SPLIT_BT_HOR 220), vertical ternary partition (SPLIT_TT_VER 230), and horizontal ternary partition (SPLIT_TT_HOR 240). The multi-type leaf nodes are called Codec Units (CUs), and this partitioning is used for prediction and transform processing without further partitioning unless the CU size is too large to exceed the maximum transform length. This means that in most cases, the CUs, PUs, and TUs have the same block size in a quadtree codec block structure with nested multi-type trees. An exception occurs when the maximum supported transform length is smaller than the width or height of the color component of the CU.

Fig. 3 illustrates a signaling mechanism for partitioning information in a quadtree codec tree structure with nested multi-type trees. A Codec Tree Unit (CTU) is considered the root of a quad-tree and is first partitioned by a quad-tree structure. Each quaternary leaf node (when large enough) is then further partitioned by a multi-type tree structure. In a quadtree codec tree structure with nested multi-type trees, for each CU node, a first flag (split_cu_flag) is signaled to indicate whether the node is further partitioned. If the current CU node is a quadtree CU node, a second flag (split_qt_flag) is signaled to indicate whether it is QT split mode or MTT split mode. When a node performs a split in the MTT split mode, a third flag (MTT _split_cu_vertical_flag) is signaled to indicate the split direction, and then a fourth flag (MTT _split_cu_binary_flag) is signaled to indicate whether the split is a binary split or a ternary split. The multi-type tree partition mode (MttSplitMode) of the CU is shown in table 1 according to the values of mtt _split_cu_vertical_flag and mtt _split_cu_binary_flag.

TABLE 1-MttSplitMode derivation based on Multi-type Tree syntax elements

MttSplitMode	mtt_split_cu_vertical_flag	mtt_split_cu_binary_flag
			SPLIT_TT_HOR	0	0
SPLIT_BT_HOR	0	1
			SPLIT_TT_VER	1	0
SPLIT_BT_VER	1	1

Fig. 4 shows that one CTU is partitioned into multiple CUs with quadtree and nested multi-type tree codec block structures, where bold block edges represent quadtree partitions and the remaining edges represent multi-type tree partitions. Quadtrees with nested multi-type tree partitioning provide a content-adaptive codec tree structure consisting of CUs. The size of a CU may be as large as a CTU or as small as 4×4 in luminance sample units. For a 4:2:0 chroma format, the maximum chroma CB size is 64 x 64 and the minimum size chroma CB consists of 16 chroma samples.

In VVC, the maximum supported luminance transform size is 64×64, and the maximum supported chrominance transform size is 32×32. When the width or height of the CB is greater than the maximum transition width or height, the CB may automatically partition in the horizontal and/or vertical direction to meet the transition size limit of that direction.

The following parameters are defined for a quadtree codec tree scheme with nested multi-type trees. These parameters are specified by a Sequence Parameter Set (SPS) PARAMETER SET syntax element and may be further refined by an image header syntax element.

CTU size: root node size of quad tree

-MinQTSize minimum allowed quaternary leaf node size

-MaxBtSize maximum allowed binary tree root node size

-MaxTtSize maximum allowed ternary tree root node size

-MaxMttDepth maximum allowable hierarchical depth of multi-type tree partitioning starting from quadtree leaf nodes

-MinCbSize minimum allowed codec block node size

In one example of a quadtree codec tree structure with nested multi-type trees, CTU size is set to 128 x 128 luma samples, with two corresponding 64 x 64 4:2:0 chroma sample blocks, minQTSize is set to 16 x 16, maxbt size is set to 128 x 128, maxttsize is set to 64 x 64, mincbsize (width and height) is set to 4 x 4, maxmttdepth is set to 4. Quaternary tree segmentation is first applied to CTUs to generate quaternary leaf nodes. The size of the quaternary leaf nodes may range from 16×16 (i.e., minQTSize) to 128×128 (i.e., CTU size). If the leaf QT node is 128 x 128, it is not further partitioned by the binary tree because the size exceeds MaxBtSize and MaxTtSize (i.e., 64 x 64). Otherwise, she Sicha tree nodes may be further partitioned by multi-type trees. Thus, the quad-leaf node is also the root node of the multi-type tree, whose multi-type tree depth (mttDepth) is 0. When the multi-type tree depth reaches MaxMttDepth (i.e., 4), no further segmentation is considered. When the width of the multi-type tree node is equal to MinCbsize, no further horizontal splitting is considered. Also, when the height of the multi-type tree node is equal to MinCbsize, no further vertical partitioning is considered.

In VVC, the codec tree scheme supports luminance and chrominance having independent block tree structures. For P and B slices, the luma and chroma CTBs in one CTU must share the same codec tree structure. However, for I slices, luminance and chrominance may have independent block tree structures. When the independent block tree mode is applied, the luminance CTBs are divided into CUs by one codec tree structure, and the chrominance CTBs are divided into chrominance CUs by another codec tree structure. This means that a CU in an I slice may consist of a codec block of a luma element or a codec block of two chroma elements, while a CU in a P or B slice always consists of codec blocks of all three color elements unless the video is monochrome.

Virtual Pipe Data Unit (VPDUs)

The Virtual Pipe Data Unit (VPDUs) is defined as a non-overlapping unit in the image. In a hardware decoder, multiple pipeline stages process consecutive VPDUs simultaneously. The size of the VPDU is approximately proportional to the buffer size in most pipeline stages, so it is important to keep the size of the VPDU small. In most hardware decoders, the size of the VPDU may be set to the maximum Transform Block (TB) size. However, in VVC, trigeminal Tree (TT) and Binary Tree (BT) partitioning may result in an increase in VPDUs sizes.

To keep the VPDU size at 64x64 luma samples, the following canonical division limit (with syntax signal modification) is applied in the VTM, as shown in fig. 5:

for a CU of width or height, or width and height equal to 128, TT segmentation is not allowed (as indicated by an "X" in fig. 5).

For 128xN CU and N≤64 (i.e. width is equal to 128 and height is less than 128), no horizontal BT is allowed.

For Nx128 CU and N≤64 (i.e., height equal to 128 and width less than 128), vertical BT is not allowed. In fig. 5, the luminance block size is 128×128. The dashed line indicates a block size of 64x64. According to the above constraints, the impermissible partitioning examples are shown in various examples (510-580) in FIG. 5, denoted by an "X".

Intra chroma partitioning and prediction limiting

In typical hardware video encoders and decoders, when an image has smaller intra blocks, processing throughput is reduced due to sample processing data dependencies between adjacent intra blocks. The predictor generation of intra blocks requires reconstruction samples from the top and left boundaries of neighboring blocks. Thus, intra prediction must be processed block by block order.

In HEVC, the smallest intra CU is 8x8 luma samples. The luminance element of the smallest intra CU may be further partitioned into four 4x4 luminance intra Prediction Units (PUs), but the chrominance element of the smallest intra CU cannot be further partitioned. Thus, the worst case hardware processing throughput occurs when processing 4x4 chroma intra blocks or 4x4 luma intra blocks. In VVC, to improve worst-case throughput, intra-chroma CBs of less than 16 chroma samples (sizes 2x2, 4x2, and 2x 4) and intra-chroma CBs of less than 4 chroma samples (size 2 xN) are prohibited by limiting the division of the intra-chroma CBs.

In a single coding tree, a minimum chroma intra prediction unit (SCIPU) is defined as a coding tree node whose chroma block size is greater than or equal to 16 chroma samples and at least one sub-luma block is less than 64 luma samples, or a coding tree node whose chroma block size is not 2xN and at least one sub-luma block is 4xN luma samples. It is required that in each SCIPU, all CBs are inter-frame, or all CBs are non-inter, i.e., intra or Intra Block Copy (IBC). In the case of non-inter SCIPU, it is further required that the chromaticity of the non-inter SCIPU is not further divided, and the luminance of SCIPU may be further divided. In this way, CBs within small chroma frames of size 2xN or less than 16 chroma samples are removed. Furthermore, no chroma scaling is applied in the case of non-inter SCIPU. Here, whether or not the non-inter-frame SCIPU is non-inter-frame can be deduced by the prediction mode of the first luma CB in SCIPU without additional syntax signaling. If the current slice is an I-slice or the current SCIPU has a 4x4 luma partition after further segmentation (because inter 4x4 is not allowed in VVC), then the type of SCIPU is inferred to be non-inter, otherwise the type of SCIPU (inter or non-inter) is indicated by a flag prior to CUs in parse SCIPU.

For the dual tree in the intra image, the 2xN intra chroma blocks are removed by disabling the vertical binary and vertical trigeminal splitting of the 4xN and 8xN chroma partitions. Small chroma blocks of sizes 2x2, 4x2 and 2x4 are also removed by the partition restriction.

Further, considering that the image width and height are multiples of max (8, mincbsizey), the limitation of the image size is considered to avoid the occurrence of chroma blocks within 2x2/2x4/4x2/2xN frames at the corners of the image.

The intra mode codec has 67 intra prediction modes.

To capture any edge direction presented in natural video, the number of directional intra modes in VVC extends from 33 used in HEVC to 65. The new direction mode not found in HEVC is indicated in fig. 6 with a red dashed arrow, while the planar and DC modes remain unchanged. These denser directional intra prediction modes are applicable to all block sizes and luminance and chrominance intra predictions.

In VVC, several conventional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes of non-square blocks.

In HEVC, each intra-coded block is square, with each side being a power of 2 in length. Therefore, no division operation is required to generate the intra predictor using DC mode. In VVC, the blocks may be rectangular, and division operations are typically required for each block. To avoid division of the DC prediction, only the longer side is used to calculate the average of the non-square blocks.

In order to keep the complexity of Most Probable Mode (MPM) list generation low, an intra-mode codec method with 6 MPMs is used, considering two available neighboring intra-modes. The following three aspects are considered when constructing the MPM list:

-default intra mode

-Adjacent intra mode

-Deriving intra modes.

Whether or not MRL and ISP codec tools are applied, intra blocks use a unified 6-MPM list. The MPM list is built based on intra modes of left and upper neighboring blocks. Assuming that the Left mode is denoted Left and the upper square mode is denoted Above, a unified MPM list is constructed as follows:

when a neighboring block is not available, its intra mode default is set to a plane.

If both Left and Above modes are non-angular modes:

MPM list → { Planar, DC, V, H, V-4, V+4}

If one of the Left and Above modes is an angular mode, the other is a non-angular mode:

setting the larger mode as Max

MPM list → { Planar, max, max-1, max+1, max-2, max+2}

If Left and Above are both angular modes and are different:

setting the larger mode as Max

If Max-Min is equal to 1:

MPM list → { Planar, left, above, min-1, max+1, min-2}

Otherwise, if Max-Min is greater than or equal to 62:

MPM list → { Planar, left, above, min+1, max-1, min+2}

Otherwise, if Max-Min is equal to 2:

MPM list → { Planar, left, above, min+1, min-1, max+1}

Otherwise:

MPM list → { Planar, left, above, min-1, min+1, max-1}

If Left and Above are both in angular mode and are the same:

MPM list → { Planar, left-1, left+1, left-2, left+2}

In addition, the first bin of the MPM index codeword is CABAC context encoded. A total of three contexts are used, corresponding to whether the current intra block has MRL, ISP or normal intra block enabled, respectively.

In the 6MPM list generation process, pruning is used to remove duplicate patterns so that only unique patterns are included in the MPM list. For entropy coding of 61 non-MPM modes, a Truncated Binary Code (TBC) is used.

Wide-angle intra prediction of non-square blocks

The conventional angular intra prediction direction is defined as a clockwise direction from 45 degrees to-135 degrees. In VVC, several conventional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes of non-square blocks. The alternate pattern is signaled using the original pattern index, parsed and then remapped to the index of the wide angle pattern. The total number of intra prediction modes remains unchanged, namely 67, and the intra mode codec method remains unchanged.

To support these prediction directions, a top reference of length 2w+1 and a left reference of length 2h+1 are defined, as shown in fig. 7A and 7B, respectively.

The number of modes to replace in the wide angle direction mode depends on the aspect ratio of the block. Alternative intra prediction modes are illustrated in table 2.

TABLE 2 intra prediction modes replaced by Wide Angle modes

In VVC, 4:2:2 and 4:4:4 chroma formats are supported, as well as 4:2:0. A chroma Deriving Mode (DM) derivation table in 4:2:2 chroma format is initially migrated from HEVC, expanding the number of entries from 35 to 67 to align with the expansion of intra prediction modes. Since HEVC specifications do not support prediction angles below-135 degrees and above 45 degrees, the luminance intra-prediction mode range maps from 2 to 5 to 2. Thus, the chroma DM derivation table of the 4:2:2 chroma format is updated by replacing some values of the mapping table entries to more accurately convert the prediction angle of the chroma block.

Cross-element linear model (CCLM) prediction

To reduce cross-element redundancy, a cross-element linear model (CCLM) prediction mode is used in VVC, in which chroma samples are predicted using a linear model based on reconstructed luma samples of the same CU, as follows:

pred_C(i,j)＝α·rec_L′(i,j)+β(1)

in one Coding Unit (CU), pred _C (i, j) represents predicted chroma samples, rec _L' (i, j) represents downsampled reconstructed luma samples of the same CU.

The CCLM parameters (α and β) are derived from up to four neighboring chroma samples and their corresponding downsampled luma samples. Assuming that the current chroma block size is w×h, when the cclm_lt mode is applied, W 'and H' are set to

W’=W,H’=H;

When the cclm_t mode is applied, W' =w+h;

when the cclm_l mode is applied, H' =h+w.

The above adjacent positions are denoted as S [0, -1]. S [ W '-1, -1], and the left adjacent position is denoted as S [ 1,0]. S [ 1, H' -1]. Then select four samples as

When the cclm_lt mode is applied and both the above and left neighbor samples are available, select S W/4,

-1],S[3*W’/4,-1],S[-1,H’/4],S[-1,3*H’/4];

-Selecting S [ W '/8, -1], S [3*W'/8, -1], S [5*W '/8, -1], S [7*W'/8, -1] when cclm_t mode is applied or only the above-mentioned neighbor samples are available;

-selecting S-1, h '/8, S-1, 3 h'/8, S-1, 5h '/8, S-1, 7 h'/8 when cclm_l mode is applied or only left neighbor samples are available.

Four adjacent luminance samples at selected locations are downsampled and compared four times to find two larger values, x ⁰ _A and x ¹ _A, and two smaller values, x ⁰ _B and x ¹ _B. Its corresponding chroma sample values are denoted as y ⁰ _A,y¹ _A,y⁰ _B and y ¹ _B. X _A,x_B,y_A and y _B are then derived as:

x_A＝(x⁰ _A+x¹ _A+1)>>1;

x_B＝(x⁰ _B+x¹ _B+1)>>1;

y_A＝(y⁰ _A+y¹ _A+1)>>1;

y_B＝(y⁰ _B+y¹ _B+1)>>1 (2)

Finally, the linear model parameters α and β are obtained according to the following equation.

β=y_B-α·x_B (4)

Fig. 8 shows examples of positions of left and upper samples and current block samples involved in the cclm_lt mode. Fig. 8 shows the relative sample positions of an nxn chroma block 810, a corresponding 2 nx2N luma block 820, and its neighboring samples (shown as filled circles).

The division operation for calculating the parameter alpha is implemented by a look-up table. In order to reduce the memory required to store the table, the difference (difference between maximum and minimum, diff) and the parameter α are represented exponentially. For example, the difference is approximated by a 4-bit significant portion and an exponent. Thus, the 1/diff table is reduced to 16 elements, applicable to 16 significant values, as follows:

DivTable[]={0,7,6,5,5,4,4,3,3,2,2,1,1,1,1,0} (5)

This will help reduce the complexity of the computation and the memory size for storing the required tables.

In addition to the above templates and left templates that can be used together to calculate linear model coefficients, they can be used alternately in other 2 LM modes, called cclm_t and cclm_l modes.

In cclm_t mode, only the upper template is used to calculate the linear model coefficients. To obtain more samples, the upper template is expanded to (w+h) samples. In cclm_l mode, only the left template is used to calculate the linear model coefficients. To obtain more samples, the left template is expanded to (H+W) samples.

In the CCLM __ LT mode, the left and upper templates are used to calculate the linear model coefficients.

To match the chroma sample positions of a 4:2:0 video sequence, two types of downsampling filters are applied to the luma samples to achieve a downsampling ratio of 2 to 1 in the horizontal and vertical directions. The choice of downsampling filter is specified by the SPS level flag. These two downsampling filters correspond to "type-0" and "type-2" content, respectively.

Rec_L′(i,j)＝[rec_L(2i-1,2j-1)+2·rec_L(2i,2j-1)+rec_L(2i+1,2j-1)+rec_L(2i-1,2j)+2·rec_L(2i,2j)+rec_L(2i+1,2j)+4]＞＞3 (6)

Rec_L′(i,j)＝rec_L(2i,2j-1)+rec_L(2i-1,2j)+4·rec_L(2i,2j)+rec_L(2i+1,2j)+rec_L(2i,2j+1)+4]＞＞3 (7)

Note that when the upper reference line is located at the CTU boundary, only one luminance line (a common line buffer in intra prediction) is used to generate downsampled luminance samples.

This parameter calculation is performed as part of the decoding process and not just the encoder search operation. Therefore, the alpha and beta values are passed to the decoder without using syntax.

For chroma intra mode codec, a total of 8 intra modes are allowed for chroma intra mode codec. These modes include five traditional intra modes and three cross-element linear mode modes (cclm_lt, cclm_t, and cclm_l). The chroma mode signaling and derivation procedure is shown in table 3. Chroma mode codec directly depends on the intra prediction mode of the corresponding luma block. Since the independent block partitioning structure of luminance and chrominance elements is enabled in an I slice, one chrominance block may correspond to multiple luminance blocks. Thus, for the chroma DM mode, the intra prediction mode of the corresponding luma block covering the center position of the current chroma block is directly inherited.

TABLE 3 deriving chroma prediction modes from luma modes when CCLM is enabled

Regardless of the value of sps cclm enabled flag, a single binarization table is used, as shown in Table 4.

TABLE 4 unified binarization table for chroma prediction modes

In table 4, the first bin indicates whether normal mode (0) or CCLM mode (1). If LM mode, then the next bin indicates whether CCLM_LT (0) or not. If not CCLM_LT, the next 1 bin indicates whether CCLM_L (0) or CCLM_T (1). For this case, when sps_ cclm _enabled_flag is 0, the first bin of the binarization table of the corresponding intra_chroma_pred_mode may be discarded before entropy encoding. In other words, the first bin is inferred to be 0 and therefore not encoded. This single binarization table is used for the case where sps_ cclm _enabled_flag is equal to 0 and 1. The first two bins in table 4 are context coded using their own context model, and the remaining bins are bypass coded.

Furthermore, to reduce luma-chroma delay in dual-tree, when the 64x64 luma coding tree node is not partitioned (and ISP is not used for 64x64 CUs) or QT partition, the chroma CUs in the 32x32/32x16 chroma coding tree node are allowed to use CCLM, as follows:

if the 32x32 chroma node is not partitioned or partitioned into QT partitions, then all chroma CUs in the 32x32 node may use CCLM

If the 32x32 chroma node is partitioned into horizontal BT and the 32x16 child node is not partitioned or partitioned using vertical BT, then all chroma CUs in the 32x16 chroma node may use CCLM.

The use of CCLM by chroma CUs is not allowed under all other luma and chroma codec tree partitioning conditions.

Multi-model CCLM (MMLM)

In JEM (J.Chen, E.Alshina, G.J.Sullivan, j.—r.ohm, and j. Boyce, algorithmic description of joint exploration test model 7, document JVET-G1001, ITU-T/ISO/IEC Joint Video Exploration Team (JVET), month 7, 2017), a multi-model CCLM mode (MMLM) is proposed for predicting chroma samples of an entire CU from luma samples using two models. In MMLM, the neighboring luma samples and neighboring chroma samples of the current block are classified into two groups, each of which is used as a training set to derive a linear model (i.e., deriving specific α and β for a specific group). In addition, samples of the current luminance block are also classified based on the same rule to classify neighboring luminance samples. Three MMLM model modes (MMLM _lt, mmlm_t, and MMLM _l) are allowed to select neighboring samples from left and top, top only, and left only, respectively.

Fig. 9 shows an example of classifying adjacent samples into two groups. The threshold is calculated as the average of neighboring reconstructed luminance samples. Neighboring samples with a Rec '_L [ x, y ] < = threshold are classified as group 1, while neighboring samples with a Rec' _L [ x, y ] > threshold are classified as group 2.

Therefore MMLM uses two models according to the sample level of the neighboring samples.

Slope adjustment of CCLM

CCLM uses a model with two parameters to map luminance values to chrominance values, as shown in fig. 10A. The slope parameter "a" and the bias parameter "b" define a map as follows:

chromaVal=a*lumaVal+b

the adjustment "u" of the slope parameter signals the update model to the following form, as shown in fig. 10B:

chromaVal=a’*lumaVal+b’

Wherein the method comprises the steps of

a’=a+u,

b’=b-u*y_r。

By this selection, the mapping function is tilted or rotated around a point with a luminance value y _r. The average value of the reference luminance samples used for model creation is y _r in order to make meaningful modifications to the model. Fig. 10A and 10B illustrate this process.

Implementation of CCLM slope adjustment

The slope adjustment parameter is provided as an integer between-4 and signaled in the bitstream. The unit of the slope adjustment parameter is the (1/8) chroma sample value (for 10-bit content) for each luma sample value.

The adaptation is applicable to CCLM models (e.g., "lm_chromaidx" and "MMLM _chromaidx") that use both above and left side reference samples of the block, but is not applicable to "single side" modes. This choice is based on codec efficiency versus complexity tradeoff considerations. "LM_CHROMA_IDX" and "MMLM _CHROMA_IDX" refer to CCLM_LT and MMLM _LT in the present invention. The "one-sided" mode is referred to herein as CCLM_ L, CCLM _ T, MMLM _L and MMLM _T.

When slope adjustment is applied to a multi-mode CCLM model, both models may be adjusted so that at most two slope updates are signaled for a single chroma block.

Encoder method for CCLM slope adjustment

The proposed encoder method performs an SATD (sum of absolute transformed differences) based search to find the best slope update value for Cr and performs a similar SATD based search for Cb. If either result is a non-zero slope adjustment parameter, the combined slope adjustment pair (SATD-based update of Cr, SATD-based update of Cb) is included in the RD (rate distortion) check list of TU.

Convolution cross-element model (CCCM) -single model and multiple models

In CCCM, a convolution model is applied to improve chroma prediction performance. The convolution model has a 7-tap filter consisting of a 5-tap plus sign space component, a nonlinear term and an offset term. The inputs to the spatial 5-tap component of the filter include the center (C) luminance sample and its up/north (N), down/south (S), left/west (W) and right/east (E) neighbors co-located with the chroma samples to be predicted, as shown in fig. 11.

The nonlinear term (denoted P) is expressed as the square of the center luminance sample C and scales to the sample value range of the content:

P=(C*C+midVal)>>bitDepth.

For example, for 10-bit content, the nonlinear term is calculated as:

P=(C*C+512)>>10

the offset term (denoted B) represents the scalar offset between the input and output (similar to the offset term in CCLM) and is set to an intermediate chroma value (512 for 10-bit content).

The output of the filter is calculated as the convolution between the filter coefficient c _i and the input value and clipped to the range of valid chroma samples:

predChromaVal=c₀C+c₁N+c₂S+c₃E+c₄W+c₅P+c₆B

The filter coefficients c _i are calculated by minimizing the MSE between the predicted and reconstructed chroma samples in the reference region. Fig. 12 illustrates an example of a reference region consisting of 6 lines of chroma samples above and to the left of the PU. The reference area is extended one PU width to the right and one PU height downward. The region is adjusted to include only available samples. The expansion of the region (denoted "padding") is to support the "side samples" of the plus-shaped spatial filter in fig. 11 and to pad in the unavailable region.

Mean Square Error (MSE) minimization is achieved by computing an autocorrelation matrix of the luminance input and a cross-correlation vector between the luminance input and the chrominance output. The autocorrelation matrix is decomposed by LDL and the final filter coefficients are calculated by back-substitution. The process generally follows the calculation of ALF filter coefficients in the ECM, but LDL decomposition rather than Cholesky decomposition is chosen to avoid the use of square root operations. In newer ECMs, MSE minimization for CCM uses Gaussian elimination based methods.

Furthermore, CCCM has the option of single-model or multiple-model variants, similar to CCLM. The multiple model variant uses two models, one for samples above the average luminance reference and one for the remaining samples (following the spirit of the CCLM design). For PUs for which at least 128 reference samples are available, the multi-model CCCM mode may be selected.

Gradient Linear Model (GLM)

In contrast to CCLM, GLM uses luma sample gradients to derive a linear model instead of downsampled luma values. Specifically, when GLM is applied, the input of the CCLM process, i.e., the downsampled luma samples L, is replaced by a luma sample gradient G. The other parts of the CCLM (e.g., parameter derivation, predictive sample linear transformation) remain unchanged.

C=α·G+β

For signaling, when the current CU enables CCLM mode, two flags are transmitted for Cb and Cr components, respectively, to indicate whether GLM is enabled for each component. If GLM is enabled for a component, a syntax element is further conveyed to select one of the 4 gradient filters (1310-1340 in FIG. 13) for gradient computation. GLM may be used in conjunction with an existing CCLM by transmitting an additional flag in the bitstream. When this combination is applied, the filter coefficients used to derive the linear model input luminance samples are calculated as a combination of the GLM selected gradient filter and the downsampling filter of the CCLM.

Spatial candidate derivation

The spatial merge candidate derivation in VVC is the same as in HEVC, except for the position exchange of the first two merge candidates. A maximum of four merge candidates (B _0,A_0,B₁ and a ₁) of the current CU 1410 are selected from candidates of the positions shown in fig. 14. The derivation sequences are B _0,A_0,B_1,A₁ and B ₂. Position B ₂ is only considered when one or more neighboring CUs of position B ₀、A₀、B₁、A₁ are not available (e.g., belong to another slice or tile) or are intra-coded. After the candidates of the position a ₁ are added, redundancy check is performed to ensure that candidates having the same motion information are excluded from the list, thereby improving the codec efficiency. To reduce the computational complexity, not all possible candidate pairs are considered in the redundancy check described above. Instead, only the pair connected by an arrow in fig. 15 is considered, and candidates are added to the list only when the corresponding candidates for redundancy check do not have the same motion information.

Time candidate derivation

In this step, only one candidate is added to the list. In particular, in the temporal merging candidate derivation of the current CU 1610, a scaled motion vector is derived based on the co-located CU 1620 belonging to the co-located reference picture, as shown in fig. 16. The reference picture list and reference index for co-located CU derivation are explicitly conveyed in the slice header. The scaled motion vector 1630 of the temporal merging candidate is scaled from the motion vector 1640 of the co-located CU by using POC (picture order count) distances tb and td, as indicated by the dashed line in fig. 16, where tb is defined as the POC difference between the reference picture of the current picture and td is defined as the POC difference between the reference picture of the co-located picture and the co-located picture. The reference picture index of the temporal merging candidate is set to zero.

The location of the temporal candidate is selected between candidates C ₀ and C ₁ as shown in fig. 17. If the CU at position C ₀ is not available, is intra-coded, or is outside the current CTU row, then position C ₁ is used. Otherwise, position C ₀ is used in the derivation of the temporal merging candidates.

Non-contiguous spatial candidates

In the development of the VVC standard, a codec tool called non-contiguous motion vector prediction (NAMVP) was proposed, see JVET-L0399 (Yu Han et al, "CE4.4.6: improvement on Merge/Skipmode", joint video exploration team of ITU-T SG 16WP 3 and ISO/IEC JTC 1/SC 29/WG 11 (JVET), conference 12, australian, CN,2018, 10 months 3-12, document JVET-L0399). According to NAMVP techniques, non-neighboring spatial merge candidates are inserted after the TMVP (i.e., temporal MVP) in the conventional merge candidate list. The pattern of spatial merge candidates is shown in fig. 18. The distance between the non-neighboring spatial candidate and the current codec block is based on the width and height of the current codec block. In fig. 18, each small square corresponds to one NAMVP candidate, which is ordered by distance (as indicated by the numbers within the square). Line buffering restrictions are not applicable. In other words, NAMVP candidates that are far from the current block may need to be stored, which may require a larger buffer.

In the present invention, methods and apparatus are disclosed for improving cross-element predictive codec performance.

[ Invention ]

A method and apparatus for video encoding and decoding uses a codec tool that includes one or more cross-element model correlation modes. According to a decoder-side method, codec data associated with a current block is received, including a first color block and a second color block, wherein the current block is encoded using Cross-element prediction (CCP for short). Codec syntax data of one or more syntax related to a transform codec or a residual codec applied to the current block is parsed. The codec syntax data is decoded using one or more context models to generate the one or more grammars based on the information related to the CCP. The current block is decoded using CCP, wherein the decoding includes transform coding of the current block or residual coding of the current block using the one or more syntax.

In one embodiment, the one or more grammars are from the group consisting of root_cbf、cb_cbf、cr_cbf、transform_skip_flag、lfnst_idx、mts_idx、tu_joint_cbcr_residual_flag、last_significant_position、cu_qp_delta_abs、cu_qp_delta_sign_flag、cu_chroma_qp_offset_flag、cu_chroma_qp_offset_idx or a combination thereof.

In one embodiment, the one or more context models correspond to a plurality of context models, and the selection among the plurality of context models depends on the CCP type of the current block. In another embodiment, the selection of the plurality of context models depends on a prediction mode of a neighboring block of the current block. For example, the prediction mode of the neighboring block corresponds to one CCP mode. In another embodiment, the selection among the plurality of context models depends on the number of neighboring blocks that are encoded and decoded in CCP mode.

In one embodiment, the one or more context models correspond to a CCP single Model, a CCP multiple Model, a convolution cross-element Model (Convolutional Cross-Component Model, CCCM), a gradient linear Model (GRADIENT LINEAR Model, GLM), or a combination thereof.

A method corresponding to the encoder side is also disclosed.

According to another method, a current block is encoded or decoded using a CCP, wherein the encoding or decoding uses information including a chroma syntax corresponding to one or more codec block flags, the one or more codec block flags including a root chroma codec block flag.

In one embodiment, CCP candidates inherited from neighboring blocks are included in a merge candidate list of the current block, and a merge flag indicates whether the current block inherits one or more cross-element model parameters from the inherited CCP candidates.

In one embodiment, the one or more codec block flags further include two or more element codec block flags corresponding to the two or more chroma blocks. In one embodiment, each element codec block flag corresponding to the two or more chroma blocks is equal to 0 if the root chroma codec block flag is false, or at least one of the two or more element codec block flags corresponding to the two or more chroma blocks is equal to 1 if the root chroma codec block flag is true.

According to another method, a merge candidate list is generated for the current block, wherein the merge candidate list includes CCP candidates inherited from neighboring blocks. A merge flag is signaled or parsed indicating whether the current block inherits one or more cross-element model parameters from the inherited CCP candidates. The skip flag is signaled or parsed before the merge flag, wherein the merge flag is inferred when the skip flag is true. The current block is encoded or decoded using information including the merge candidate list.

In one embodiment, when the skip flag is true, an index associated with the inherited CCP candidate is signaled or parsed. In one embodiment, when the merge flag is inferred to be 1, the current block inherits one or more cross-element model parameters from the inherited CCP candidate relay. In one embodiment, the CCP mode index is signaled or parsed when the merge flag is inferred to be 0.

In one embodiment, a high level syntax is signaled or parsed in a Sequence Parameter Set (SPS) PARAMETER SET, picture Parameter Set (PPS) PARAMETER SET, picture Header (PH) or slice header (SLICE HEADER, SH) to indicate whether the current Sequence, picture or slice is allowed to use a skip flag.

[ Description of the drawings ]

Fig. 1A illustrates an adaptive inter/intra video codec system including loop processing.

Fig. 1B shows a corresponding decoder of the encoder of fig. 1A.

Fig. 2 illustrates an example of a multi-type tree structure corresponding to vertical binary segmentation (split_bt_ver), horizontal binary segmentation (split_bt_hor), vertical ternary segmentation (split_tt_ver), and horizontal ternary segmentation (split_tt_hor).

Fig. 3 illustrates an example of a signaling mechanism for partition splitting information with a nested multi-type tree codec tree structure in a quadtree.

Fig. 4 illustrates an example in which one CTU is divided into multiple CUs, using a quadtree and nested multi-type tree codec block structure, where bold block edges represent quadtree partitions and the remaining edges represent multi-type tree partitions.

Fig. 5 illustrates some TT split examples that are prohibited when the width or height of the luma codec block is greater than 64.

Fig. 6 illustrates intra prediction modes adopted by the VVC video codec standard.

Fig. 7A-B show examples of wide-angle intra prediction, where fig. 7A is a block with a width greater than a height and fig. 7B is a block with a height greater than a width.

Fig. 8 illustrates examples of positions of left and upper samples and current block samples involved in the cclm_lt mode.

Fig. 9 illustrates an example of classifying adjacent samples into two groups.

Fig. 10A illustrates an example of a CCLM model.

Fig. 10B shows an effect example of the slope adjustment parameter "u" for model update.

Fig. 11 shows a spatial portion example of a convolution filter.

Fig. 12 shows an example of a reference region with padding for deriving filter coefficients.

Fig. 13 shows 4 gradient modes of a gradient linear Model (GRADIENT LINEAR Model, abbreviated GLM).

Fig. 14 shows neighboring blocks used to derive VVC spatial merge candidates.

Fig. 15 shows possible candidate pairs for redundancy check in VVC.

Fig. 16 shows an example of temporal candidate derivation, in which scaled motion vectors are derived from POC (Picture Order Count) distances.

Fig. 17 shows the location of the time candidate selected between candidates C ₀ and C ₁.

Fig. 18 illustrates an exemplary pattern of non-adjacent spatial merging candidates.

Fig. 19 illustrates an example of CCM information propagation, where the dashed blocks (i.e., A, E, G) are encoded in a cross-element mode (e.g., CCLM, MMLM, GLM, CCCM).

FIG. 20 illustrates an example of inheriting temporal proximity model parameters.

Fig. 21A-B illustrate two search modes that inherit non-adjacent spatial proximity models.

Fig. 22A-B illustrate examples of constructing a current region history table from a history table of regions having the same starting geometric position of the current region (fig. 22A) or from a history table of regions containing the central geometric position of the current region (fig. 22B).

Fig. 23 illustrates an example of mapping motion information of a position to be referred to in an unavailable area to a predefined position, wherein the predefined position is located on the upper row of the first CTU row.

Fig. 24 illustrates an example of mapping motion information of a position to be referred to in an unavailable area to a predefined position, wherein the predefined position is located at the bottom row of a corresponding CTU row.

Fig. 25 illustrates an example of mapping motion information of a position to be referred to in an unavailable area to a predefined position, wherein the predefined position is located at a bottom row or a center row of a corresponding CTU row.

Fig. 26 illustrates an example of mapping motion information of a position to be referred in an unavailable area to a predefined position, wherein the predefined position is located at a bottom row of a corresponding CTU row or a last CTU row of the corresponding CTU row.

FIG. 27 illustrates an example of a neighborhood template for calculating model errors.

Fig. 28 illustrates an example of inheriting candidates from candidates in a candidate list of neighboring blocks.

Fig. 29 illustrates an example of sub-sampling inter-frame codec or CCM information at the top left position of each 2x2 grid in the CTU level buffer before the information is saved to the image level buffer.

Fig. 30 illustrates an embodiment of the present invention in which if the skip flag is true, the merge flag will be inferred to be 1 and CBF for both Cb and Cr components will be inferred to be 0.

Fig. 31 illustrates an embodiment of the present invention in which if the skip flag is true, the merge flag will be inferred to be 1 and CBF for both Cb and Cr components will be inferred to be 0, and CBF for both Cb and Cr components will be inferred to be 0.

FIG. 32 illustrates a flow diagram of an exemplary video decoding system that decodes syntax data using one or more context models based on information related to cross-element prediction, in accordance with an embodiment of the invention.

FIG. 33 illustrates a flow diagram of an exemplary video coding system that encodes syntax data using one or more context models based on information related to cross-element prediction, in accordance with an embodiment of the invention.

Fig. 34 illustrates a flow chart of an exemplary video codec system for encoding or decoding a current block using information, including a chroma syntax corresponding to a coded block flag shared by one or more chroma blocks, in accordance with an embodiment of the present invention.

FIG. 35 illustrates a flow chart of an exemplary video codec system that uses a merge flag to indicate whether a current block inherits one or more cross-element model parameters from an inherited CCP candidate in accordance with one embodiment of the present invention.

[ Detailed description ] of the invention

It will be readily understood that the components of the present invention, as generally described and illustrated in the figures herein, could be arranged and designed in a wide variety of different configurations. Thus, the following more detailed description of the embodiments of the present systems and methods, as represented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. Reference throughout this specification to "one embodiment," "an embodiment," or similar language means that a particular feature, structure, or characteristic may be included in at least one embodiment of the present invention in connection with the embodiment. Thus, appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment.

Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. One skilled in the relevant art will recognize, however, that the invention may be practiced without one or more of the specific details, or with other methods, components, etc. In other instances, well-known structures or operations are not shown or described in detail to avoid obscuring aspects of the invention. Embodiments of the invention will be best understood by reference to the drawings, wherein like parts are designated by like numerals throughout. The following description is by way of example only and is merely illustrative of certain selected apparatus and method embodiments consistent with the invention as claimed herein.

To improve the codec performance of cross-element prediction, various schemes are disclosed.

In ECM, various methods of cross-element prediction (CCP) are proposed to improve intra chroma coding efficiency. For example, the number of the cells to be processed,

-CCLM_L,CCLM_T,CCLM_LT,MMLM_L,MMLM_T,MMLM_LT

-CCCM_L,CCCM_T,CCCM_LT,MMCCCM_L,MMCCCM_T,MMCCCM_LT

-GLM_L,GLM_T,GLM_LT,MMGLM_L,MMGLM_T,MMGLM_LT

Glm2_l, glm2_t, glm2_lt, mmglm2_l, MMGLM2_t, MMGLM2_lt (glm2 is GLM with corresponding luminance term)

For each GLM mode, 4 gradient kernels may be selected.

With these cross-element models, intra chroma prediction becomes more accurate and entropy of the residual becomes smaller. Based on this observation, the present invention proposes the following method to improve the codec performance of intra chroma prediction:

Improved CBF codec

In previous codec standards, root_cbf (i.e., indicating whether any transform blocks of all color components have transform coefficients) was not signaled when the current block is coded by intra-mode, since intra-coding is typically not perfectly predicted. However, by using various cross-element models, which utilize luma reconstructed samples as model inputs, chroma components can be predicted more accurately, and the probability that CBFs (codec block flags) or coding flags of all chroma components are equal to 0 in intra codec mode is increasing. Based on this assumption, various methods have been proposed to improve CBF codec efficiency.

In one embodiment, a root_cbf syntax may be used for intra-coded blocks to indicate whether any transform blocks of each color component have any transform coefficients.

In one embodiment, a new flag root_chroma_cbf is presented to indicate whether any transform block of the chroma component has any transform coefficients. For example, if the root_chroma_cbf flag is true, it means that at least one of the Cb or Cr components has a CBF flag equal to 1. Thus, when the CBF of the Cb component is 0, the CBF of the Cr component is implicitly inferred to be 1 and is not signaled. Otherwise, if the root_chroma_cbf flag is false, it means that the CBF of the Cb component and the CBF of the Cr component are both equal to 0.

Transform codec and residual codec enhancement by adding context models for intra chroma CCP modes and inherited cross-element models

Since intra chroma prediction using CCP mode may be more accurate than using non-CCP mode and the inherited cross-element model is also a CCP mode, syntax in transform codec and residual codec may be improved by adding one or more context models and/or additional context models according to CCP mode related information, depending on whether the current block uses the inherited cross-element model.

In one embodiment, the syntax associated with the transform codec and the residual codec may include, but is not limited to, root_cbf, CBF of Cb component, CBF、transform_skip_flag、lfnst_idx、mts_idx、tu_joint_cbcr_residual_flag、last_significant_position、cu_qp_delta_abs、cu_qp_delta_sign_flag、cu_chroma_qp_offset_flag of Cr component, and/or cu_chroma_qp_offset_idx.

In one embodiment, all syntax in the transform codec and the residual codec may have one or more context models based on CCP mode related information. In another embodiment, part of the syntax in the transform codec and the residual codec may have one or more context models according to CCP mode related information. In another embodiment, all syntax elements in the transform codec and the residual codec may have additional context models depending on whether the current block uses an inherited cross-element model.

In another embodiment, the partial syntax elements in the transform codec and the residual codec may have one or more context models depending on whether the current block uses inherited cross-element models.

In one embodiment, when the current block is coded by intra mode and the intra prediction mode is one of all CCP modes or inherited cross-element model, an additional context model of syntax related to transform coding and residual coding is used.

In another embodiment, when the current block is coded by intra mode and the intra prediction mode is one of the sub-sets of all CCP modes, an additional context model of syntax related to transform coding and residual coding is used.

In another embodiment, there are multiple additional context models, and the decision of the context model may depend on the CCP type of the current codec block. For example, if the current block is encoded by the CCP single model, a first additional context model is used, and if the current block is encoded by the CCP multi-model, a second additional context model is used. Another example is to use a first additional context Model if the current block is coded by the LM Model, a second additional context Model if the current block is coded by the convolutional cross-element Model (Convolutional Cross-Component Model, CCCM), and a third additional context Model if the current block is coded by the gradient linear Model (GRADIENT LINEAR Model, GLM).

In another embodiment, there are multiple additional context models, and context model decisions for syntax related to Transform Coding (Transform Coding) and Residual Coding (Residual Coding) may be related to prediction modes of neighboring codec blocks. For example, if only the upper block is coded by CCP mode, a first additional context model is used, if only the left block is coded by CCP mode, a second additional context model is used, and if both the upper block and the left block are coded by CCP mode, a third additional context model is used.

In another embodiment, there are multiple additional context models, and the decision of the context models may depend on the number of neighboring blocks that are encoded and decoded by the CCP mode. For example, if the number of neighboring blocks encoded by CCP mode is equal to a first number, a first additional context model is used, if the number of neighboring blocks encoded by CCP mode is equal to a second number, a second additional context model is used, and so on.

To improve prediction accuracy or codec performance of cross-element prediction, various schemes related to inheritance cross-element models are disclosed.

Guide parameter set for refining cross-element model parameters

According to this method, a set of boot parameters is used to refine the derived model parameters by a specified CCLM mode. For example, the pilot parameter set is explicitly signaled in the bitstream, after deriving the model parameters, the pilot parameter set is added to the derived model parameters as final model parameters. The pilot parameter set comprises at least one differential scaling parameter (dA), one differential offset parameter (dB) and one differential shift parameter (dS). For example, equation (1) can be rewritten as:

pred_C(i,j)＝((α′·rec_L′(i,j))>>s)+β,

if dA is signaled, then the final prediction is:

pred_C(i,j)＝(((α′+dA)·rec_L′(i,j)}>>s)+β。

similarly, if dB is signaled, the final prediction is:

pred_C(i,j)＝((α′·rec_L′(i,j))>>s)+(β+dB)。

If dS is signaled, then the final prediction is:

pred_c(i,j)＝((α′·rec_L′(i,j))>>(s+dS))+β。

if dA and dB are signaled, then the final prediction is:

pred_C(i,j)＝(((α′+dA)·rec_L′(i,j))>>s)+(β+dB)。

the pilot parameter set may be signaled for each color component. For example, one pilot parameter set is transmitted for Cb component signals and another pilot parameter set is transmitted for Cr component signals. Or a set of pilot parameters may be signaled and shared between the color components. The signaled dA and dB may be positive or negative values. When signaling dA, a bin is signaled to indicate the sign of dA. Similarly, when a signal conveys dB, the signal conveys a bin to indicate the sign of dB.

For another embodiment, dA and dB may be the Least Significant Bit (LSB) portion of the final scaling and offset parameters. For example, if m bits are required to represent the final scaling parameters, dA is the LSB portion of the final scaling parameters, and n bits (m > n) are used to represent dA, where the most significant bits (m-n bits) of the final scaling parameters are implicitly derived. In other words, for the final scaling parameter, the most significant bits of the final scaling parameter are taken from the most significant bit portion of α', and the LSB portion of the final scaling parameter is from the signaled dA. Similarly, if p bits are required to represent the final offset parameter, then dB is the LSB portion of the final offset parameter, and q bits (p > q) are used to represent dB, with the most significant bits (p-q bits) of the final offset parameter being implicitly derived. In other words, for the final offset parameter, the most significant bits of the final offset parameter are taken from the most significant bit portion of β, and the LSB portion of the final offset parameter is from the dB of signal transmission.

For another embodiment, dB may be implicitly derived from the average of neighboring (e.g., L-shaped) reconstructed samples if dA is signaled. For example, in VVC, four neighboring luminance and chrominance reconstruction samples are selected to derive model parameters. Assuming that the average values of neighboring luminance and chrominance samples are lumaAvg and chromaAvg, respectively, β is derived by β= chromaAvg- (α' +da) · lumaAvg. The average value of neighboring luminance samples (i.e., lumaAvg) may be determined by the average value of all selected luminance samples, the luminance DC mode value of the current luminance CB, or the average value of the maximum and minimum luminance samples (e.g., Or (b)And (5) calculating. Similarly, the average value of neighboring chroma samples (i.e., chromaAvg) may be determined by all selected chroma samples, the chroma DC mode value of the current chroma CB, or the average of the maximum and minimum chroma samples (e.g., Or (b) And (5) calculating. Note that for non-4:4:4 color sub-sampling formats, the selected neighboring luma reconstruction samples may come from the output of the CCLM downsampling process.

For another embodiment, the shift parameter s may be a constant value (e.g., s may be 3,4, 5, 6, 7, or 8), and dS is equal to 0 and no signaling is required.

For another embodiment, the pilot parameter set may also be signaled per model in MMLM. For example, one pilot parameter set is one model signal transmission and another pilot parameter set is another model signal transmission. Or a set of pilot parameters is signaled and shared between the linear models. Or only one set of guide parameters is signaled for one selected model, while the other model is not further refined by the guide parameter set.

In another embodiment, the MSB portion of α' is selected according to the cost of possible final scaling parameters. That is, a possible final scaling parameter is derived from a possible MSB value of the signaled dA and α'. For each possible final scaling parameter, the cost defined by the sum of absolute differences between neighboring reconstructed chroma samples and corresponding chroma values generated by the CCLM model is calculated, the final scaling parameter being the one with the smallest cost. In one embodiment, the cost function is defined as the sum of squared errors.

Inheritance of neighboring model parameters to refine cross-element model parameters

The final scaling parameters of the current block are inherited from neighboring blocks and further refined by dA (e.g., the derivation or signaling of dA may be similar or identical to the method in the previous "guide parameter set for refinement of the cross-element model parameters"). Once the final scaling parameters are determined, an offset parameter (e.g., β in CCLM) is derived based on the inherited scaling parameters and the average of neighboring luma and chroma samples of the current block. For example, if the final scaling parameter is inherited from the selected neighboring block and the inherited scaling parameter is α '_nei, then the final scaling parameter is (α' _nei +dA). For another embodiment, the final scaling parameters are inherited from the history list and further refined by dA. For example, the history list records the last j final scaling parameter entries for the previous CCLM encoded block. The final scaling parameter is then inherited from a selected entry of the history list, α '_list, and is (α' _list +dA). For another embodiment, the final scaling parameters are inherited from the history list or neighboring blocks, but only the MSB (most significant bit) part of the inherited scaling parameters is taken, and the LSB (least significant bit) of the final scaling parameters is from dA. For another embodiment, the final scaling parameters are inherited from the history list or neighboring blocks, but are not further refined by dA.

For another embodiment, the offset may be further optimized by dB after inheriting the model parameters. For example, if the final offset parameter is inherited from a selected neighboring block and the inherited offset parameter is β '_nei, then the final scaling parameter is (β' _nei +db). For yet another embodiment, the final offset parameter is inherited from the history list and is further optimized by dB. For example, the history list records the last j final scaling parameter entries for the previous CCLM encoded block. The final scaling parameter is then inherited from the selected one of the entries in the history list, β '_list, and the final scaling parameter is (β' _list +db).

For another embodiment, if the inherited neighboring block is encoded using a convolution cross-element model (Convolutional Cross-Component Mode, CCCM for short), the filter coefficients (c _i) are inherited. The offset parameter (e.g., c ₆ x B or c ₆ in CCCM) may be re-derived based on the inherited parameter and the average of the neighboring corresponding position luma and chroma samples of the current block. For yet another embodiment, only a portion of the filter coefficients are inherited (e.g., only n out of 6 filter coefficients, where 1+.n < 6), with the remaining filter coefficients being further re-derived using neighboring luma and chroma samples of the current block.

For yet another embodiment, if the inherited candidate applies a gradient linear Model (GRADIENT LINEAR Model, GLM for short) gradient pattern to its luma reconstructed samples, the current block should also inherit the candidate GLM gradient pattern and apply to the current luma reconstructed samples.

For yet another embodiment, if the inherited neighboring blocks are encoded using multiple cross-element models (e.g., multi-model linear model (Multi-Model Linear Model, MMLM for short) or CCCM Multi-model), then the classification threshold is also inherited to classify the neighboring samples of the current block into multiple groups, and the inherited multiple cross-element model parameters are further assigned to each group. For another embodiment, the classification threshold is an average of neighboring reconstructed luma samples, and inherited multiple cross-element model parameters are further assigned to each group. Also, once the final scaling parameters for each group are determined, the offset parameters for each group are re-derived based on the inherited scaling parameters and the average of the neighboring luma and chroma samples for each group of the current block. For another example, if CCCM multiple models are used, once the final coefficient parameters for each group are determined (e.g., c ₀ to c ₅, except for c ₆ in CCCM), the offset parameters for each group (e.g., c ₆ ×b or c ₆ in CCCM) are re-derived based on the inherited coefficient parameters and the neighboring luma and chroma samples for each group of the current block.

For yet another embodiment, inherited model parameters may depend on color components. For example, the Cb and Cr components may be derived from the same or different candidate relay model parameters or models. For another example, only one color component inherits model parameters, and the other color component derives model parameters based on the inherited model derivation method (e.g., if the inherited candidate is encoded by MMLM or CCCM, the current block derives model parameters based on MMLM or CCCM using the current neighbor reconstructed sample as well). For yet another example, only one color component inherits model parameters, and the other color component derives its model parameters using the current neighbor reconstructed sample.

For yet another example, if Cb and Cr components may be derived from different candidate relay model parameters or models. The inheritance model of Cr may depend on the inheritance model of Cb. For example, possible scenarios include, but are not limited to, (1) if the Cb inheritance model is CCCM, then the Cr inheritance model should be CCCM, (2) if the Cb inheritance model is CCLM, then the Cr inheritance model should be CCLM, (3) if the Cb inheritance model is MMLM, then the Cr inheritance model should be MMLM, (4) if the Cb inheritance model is CCLM, then the Cr inheritance model should be CCLM or MMLM, (5) if the Cb inheritance model is MMLM, then the Cr inheritance model should be CCLM or MMLM, and (6) if the Cb inheritance model is GLM, then the Cr inheritance model should be GLM.

For another embodiment, after decoding a block, cross-element model (CCM) information for the current block is derived and stored for use in reconstructing neighboring blocks using inherited neighboring model parameters. CCM information referred to in this disclosure includes, but is not limited to, prediction modes (e.g., CCLM, MMLM, CCCM), GLM mode indexes, model parameters, or classification thresholds. For example, even though the current block is encoded by inter prediction, the cross-element model parameters of the current block may be derived by reconstructing or predicting samples using the current luminance and chrominance. Thereafter, if another block predicts by using the inherited neighboring model parameters, it can inherit the model parameters from the current block. Another example is that the current block is encoded by cross-element prediction, and the cross-element model parameters of the current block are re-derived by using the current luma and chroma reconstruction or prediction samples. For another example, the stored cross-element model may be CCCM, lm_la (i.e., a single model LM using the top and left neighbor derived models) or MMLM _la (a multiple model LM using the top and left neighbor derived models). As another example, even though the current block is encoded by non-cross-element intra prediction (e.g., DC, planar, intra angle mode, MIP, or ISP), the cross-element model parameters of the current block may be derived by reconstructing or predicting samples using the current luminance and chrominance. Another example is that even though the current block is encoded by cross-element prediction, the cross-element model parameters of the current block are re-derived by using the current luma and chroma reconstruction or prediction samples. The re-derived model parameters are then combined with the original cross-element model used to reconstruct the current block. To combine with the original cross-element model, the model combining methods mentioned in the section entitled "model generated based on other inheritance model" and "inheritance multiple cross-element model" can be used. For example, assume that the original cross-element model parameters areThe re-derived cross-element model parameters areThe final cross-element model is Where α is a weighting factor that can be predefined or implicitly derived from the cost of the neighboring templates.

For another embodiment, when inheriting a cross-element model from neighboring merge candidates encoded by the cross-element mode (e.g., CCLM and CCCM, etc.), a flag may be signaled to indicate/select whether to use the re-derived model. If the flag is 0, a cross-element model for encoding neighboring merge candidates is inherited. If the flag is 1, then the cross-element model re-derived based on luma and chroma reconstruction or prediction samples of neighboring merge candidates is inherited.

For another example, when the current slice is a non-intra slice (e.g., a P-slice or a B-slice), the cross-element model of the current block is derived and stored for use in reconstructing the neighboring block using inherited neighboring model parameters. For another embodiment, when the current block is inter-coded, the CCM information for the current inter-coded block is derived by copying the CCM information in the reference picture with the CCM information from its reference block. For example, as shown in fig. 19, block B in P/B picture 1920 is inter-coded, and CCM information for block B is then obtained by copying the CCM information from reference block a in I picture 1910. It should be noted that the current block may also copy CCM information from an intra-coded block in a P/B picture. For example, as shown in fig. 19, block D in P/B picture 1930 is inter-coded, and then CCM information for block D is obtained by copying CCM information from reference block E intra-coded in P/B picture 1920. For another embodiment, if the reference block in the reference picture is also inter-coded, the CCM information for the reference block is obtained by copying the CCM information from another reference block in another reference picture. For example, as shown in fig. 19, the current block C in the current P/B picture 1930 is inter-coded, and its reference block B is also inter-coded, and since the CCM information of the block B is obtained by copying the CCM information from the block a, the CCM information of the block a is also propagated to the current block C. For another embodiment, when the current block is inter-coded by bi-prediction, if one of its reference blocks is intra-coded and has CCM information, the CCM information of the current block is obtained by copying the CCM information from the intra-coded reference block in the reference picture. For example, assume that block F is inter-coded by bi-prediction and has reference blocks G and H. Block G is intra-coded and has CCM information. The CCM information of the block F is obtained by copying the CCM information from the block G encoded in the CCM mode. For another embodiment, when the current block is inter-coded by bi-prediction, the CCM information for the current block is a combination of the CCM models of its reference block (as noted in the section entitled "inherit multiple Cross-element models").

When deriving a cross-element model for a current block by using current luma and chroma reconstruction or prediction samples, in one embodiment, if the current derived model error is greater than a threshold, the current derived model is discarded and not stored. For example, the current luma reconstructed sample may be input into a model, the distortion between the model output and the current chroma reconstructed sample is calculated, and then the calculated distortion is normalized by the size of the current block or the number of samples used to calculate the distortion. If the normalized distortion is greater than or equal to the threshold, the currently derived model is discarded and not stored.

Whether a cross-element model is derived for a current block may depend on the size or area of the current block. For example, for small tiles (e.g., tile width/height less than or equal to a threshold, or tile area less than or equal to a threshold), the cross-element model is not allowed to be derived. For another example, for large blocks (e.g., block width/height greater than or equal to a threshold, or block area greater than or equal to a threshold), the cross-element model is not allowed to be derived.

Inheriting CCM information

In one embodiment, cross-element model (CCM) information of the inherited cross-element model may be stored with inherited model parameters. As previously described in this disclosure, CCM information includes, but is not limited to, prediction modes (e.g., CCLM, MMLM, CCCM), model indices for indicating which model shape to use in a convolution model, classification thresholds for multiple models, downsampling filter flags, downsampling filter indices, number of neighboring lines to use to derive a model, template type to use to derive a model, post-filtering flags, or model parameters.

In one embodiment, the CCLM model may be inherited. In addition to storing model parameters, prediction modes may also be stored in the CCM information to indicate that the inherited model is a CCLM model.

In another embodiment, the CCLM model with non-linear terms may be inherited. In addition to storing model parameters, prediction modes may also be stored in the CCM information to indicate that the inherited model is a CCLM model with non-linear terms.

In one embodiment, the CCCM model may be inherited. In addition to storing model parameters, prediction modes may also be stored in the CCM information to indicate that the inherited model is the CCCM model. The luminance offset and the chrominance offset used to adjust CCCM model inputs may also be stored in the CCM information.

In another embodiment, CCCM models with different convolution filter shapes may be inherited. In addition to the model parameters and prediction modes, CCCM mode indices may also be stored in the CCM information to indicate the convolution filtered shape of the CCCM model for inheritance. For example, CCCM models with different convolution filter shapes may contain only spatial terms in the horizontal direction. For another example, CCCM models with different convolution filter shapes may contain only spatial terms in the vertical direction. For another example, CCCM models with different convolution filter shapes may contain only spatial terms in the diagonal direction. For another example, CCCM models with different convolution filter shapes may contain only spatial terms in the anti-diagonal direction. For another example, CCCM models with different convolution filter shapes may contain X-shaped spatial terms.

In another embodiment, CCCM models using non-downsampled samples may be inherited. In addition to storing model parameters, prediction modes may also be stored in the CCM information to indicate that the inherited model is a CCCM model using non-downsampled samples.

In another embodiment, CCCM models with multiple downsampling filters may be inherited. In addition to storing model parameters, a prediction mode may be stored in the CCM information to indicate that the inherited model is a CCCM model with multiple downsampling filters, and a model index may also be stored in the CCM information to indicate which variant of the CCCM model with multiple downsampling filters is inherited.

In another embodiment, a hybrid CCCM model consisting of various terms (e.g., spatial, gradient, positional, nonlinear, and bias terms) may be inherited. The gradient term may be calculated in the downsampled domain or in the non-downsampled domain. The position item may be calculated with respect to the upper left coordinates of the current block or image. In addition to storing model parameters, prediction modes may also be stored in the CCM information to indicate that the inherited model is a hybrid CCCM model consisting of various terms. If there are multiple types of hybrid CCCM models, a model index may also be stored in the CCM information to indicate which type of hybrid CCCM model is inherited. For example, gradient and position-based CCCM (GL-CCCM) set forth in JVET-AB0119 is a hybrid CCCM model that contains a spatial term for the center position, two gradient terms for the horizontal and vertical directions, two position terms X and Y for the relative horizontal and relative vertical positions, a nonlinear term, and a bias term. In addition to storing model parameters, prediction modes may also be stored in the CCM information to indicate that the inherited model is a GL-CCCM model.

In one embodiment, the GLM model may be inherited. In addition to storing model parameters, prediction modes may be stored in the CCM information to indicate that the inherited model is a GLM model, and downsampling filter indices may also be stored in the CCM information to indicate gradient downsampling filters for the inherited GLM model.

In another embodiment, a gradient linear model (Gradient LinearModel, abbreviated GLM) model with luminance terms may be inherited. In addition to storing model parameters, a prediction mode may be stored in cross-element model (CCM) information to indicate that the inherited model is a GLM model with luminance terms, and a downsampling filter index may be stored in CCM information to indicate a gradient downsampling filter for the inherited GLM model with luminance terms.

In one embodiment, any type of cross-element multi-model may be inherited. In addition to storing model parameters and prediction modes, a multimode on/off flag may be stored in the CCM information to indicate whether the inherited CCM model is multimode. If the multi-model on/off flag is true, the multi-model classification threshold is also stored in the CCM information.

In one embodiment, the CCM information may include information to indicate how the inherited model was derived. For example, CCM information may include a number of adjacent rows used to derive a cross-element model and/or a template type used to derive the model. For example, a set of templates may be used to derive a convolution cross-element model (CCCM) model. The set of templates includes templates having different positions, sizes, and shapes. The CCM information may store an index of the template on which the inherited CCCM model is based. For example, the inheritance CCCM model may be derived based on top template only, left template only, or left and top templates. As another example, the inheritance CCCM model may be derived based on a 6-line template or a 2-line template.

In one embodiment, a post-filter flag may be stored in the CCM information. This information describes how the inherited model is used in its source block. If the post-filter flag is on, this indicates that a filter is applied to the prediction of the inherited model source block.

Optimization of inherited model parameters

In one embodiment, the inherited model parameters may be further optimized based on the inherited CCM information. The inherited CCM information may include how the inherited model was derived, e.g., the type of template and/or the number of adjacent rows used to derive the model. The optimized parameters are derived based on local information. The optimization process may follow the derivation of the inheritance model and use the same type of template and/or the same number of adjacent rows. For example, if the inherited model is a cross-chroma luma model (CCLM) and is derived based on only the left-side template (i.e., the inherited model is cclm_l), the offset parameter β may be derived from the average of the neighboring left-side template reconstructed samples of the current block. For example, if the inherited model is CCLM and is derived based on only the top template (i.e., the inherited model is cclm_t), the offset parameter β may be derived from the average of the neighboring top template reconstruction samples of the current block. Another example is that if the inherited model is CCCM and derived based on a 2-line template, the offset value (e.g., c ₆ in CCCM) may be re-derived based on 2-line template reconstruction samples of the current block. Another example is if the inherited model is multi-model (MMLM, CCCM with multiple models) and derived based on only the left template, then the classification threshold may be re-derived based on the left template reconstruction sample of the current block.

In one embodiment, the inherited model parameters are further optimized by different types of templates and/or different numbers of rows, with the final model parameters being determined by the template cost. The template cost is calculated by applying candidate optimization model parameters to neighboring templates to predict template samples and calculating the difference (e.g., SAD or SATD) between the predicted samples and reconstructed samples. For example, if the inherited model is a CCLM, the optimized offset parameters β' _L,β′_T,β′_LT are derived using the left template, top template, and left top template, respectively, of the reconstructed samples of the current block. If the template cost of applying β ' _L is minimal in β ' _L,β′_T,β′_LT, then β ' _L is chosen as the final offset parameter.

In another embodiment, the inherited model parameters may be further optimized by predefined values. The template cost is used to determine whether the inherited model parameters are further optimized. The template cost is calculated by applying candidate optimization model parameters to neighboring templates to predict template samples and calculating the difference (e.g., SAD or SATD) between the predicted samples and reconstructed samples. For example, for CCCM mode, for each inherited model parameter c _i, this value is optimized by dc _i, and the template costs of applying c _i+dc_i and c _i are compared to determine which is the final model parameter value.

Inheritance of spatially-adjacent model parameters

For another embodiment, the inherited model parameters may be from a directly adjacent block. Models from blocks at predefined locations are added to the candidate list in a predefined order. For example, the predefined locations may be the locations depicted in fig. 14, the predefined order may be B ₀,A₀,B₁,A₁ and B ₂, or a ₀,B₀,B₁,A₁ and B ₂.

For another embodiment, the predefined positions include those at immediately above (W > > 1) or ((W > > 1) -1) positions, if W is greater than or equal to TH, and those at immediately left (H > > 1) or ((H > > 1) -1) positions, if H is greater than or equal to TH, where W and H are the width and height of the current block, TH is a threshold that may be 4, 8, 16, 32, or 64. For another embodiment, the maximum number of models inherited from spatial neighbors is less than the number of predefined locations. For example, if the predefined locations are as shown in fig. 14, there are 5 predefined locations. If the predefined order is B _0,A_0,B_1,A₁ and B ₂, and the maximum number of models inherited from the spatial neighbors is 4, then the model from B2 is added to the candidate list only if one of the previous blocks is not available or is not encoded in the cross-element model.

Inherited temporal proximity model parameters

For another embodiment, if the current slice/image is a non-intra slice/image, the inherited model parameters may be from blocks in a previously encoded slice/image. For example, as shown in fig. 20, the current block is located at (x, y) and the block size is w×h. The inherited model parameters may be from blocks in the previously encoded slice/image that are located (x ', y'), (x ', y' +h/2), (x '+w/2, y' +h/2), (x '+w, y'), (x ', y' +h) or (x '+w, y' +h), where x '=x+Δx and y' =y+Δy. In one embodiment, if the prediction mode of the current block is intra, Δx and Δy are set to 0. If the prediction mode of the current block is inter prediction, Δx and Δy are set to horizontal and vertical motion vectors of the current block. In another embodiment, if the current block is inter bi-prediction, Δx and Δy are set to the horizontal and vertical motion vectors in reference picture list 0. In another embodiment, if the current block is inter bi-prediction, Δx and Δy are set to the horizontal and vertical motion vectors in reference picture list 1.

For another embodiment, if the current block is inter bi-prediction, the inherited model parameters may be from blocks in previously encoded slices/pictures in the reference list. For example, if the horizontal and vertical motion vectors in reference picture list 0 are Δx _L0 and Δy _L0, the motion vectors may be scaled to other reference pictures in reference lists 0 and 1. If the motion vector is scaled to the i ^th reference picture in reference list 0, it is (Δx _L0,i0,Δy_L0,i0). The model may be from the blocks in the i ^th reference images in reference list 0, and Δx and Δy are set to (Δx _L0,i0,Δy_L0,i0). For another example, if the horizontal and vertical motion vectors in reference picture list 0 are Δx _L0 and Δy _L0, and the motion vector is scaled to the i ^th reference picture in reference list 1 to be (Δx _L0,i1,Δy_L0,i1). The model may be from the blocks in the i ^th reference images in reference list 1, and Δx and Δy are set to (Δx _L0,i1,Δy_L0,i1).

Inheritance non-adjacent spatial neighbor model

For another embodiment, the inherited model parameters may be from spatially neighboring blocks. Models from blocks at predefined locations are added to the candidate list in a predefined order. For example, the pattern of positions and sequences may be as shown in fig. 18, where the distance between each position is the width and height of the current codec block. For another embodiment, the distance between locations closer to the current encoded block is less than the distance between locations farther from the current block.

For another embodiment, the maximum number of models inherited from non-adjacent spatial neighbors is less than the number of predefined locations. For example, if the predefined locations are as shown in FIGS. 21A-B, two modes are shown (mode 2110 in FIG. 21A and mode 2120 in FIG. 21B). If the maximum number of models inherited from non-adjacent spatial neighbors is N, search pattern 2 is only used when the number of available models from search pattern 1 is less than N.

Inheriting model parameters from history tables

In one embodiment, the inherited model parameters may be from a cross-element model history table. The cross-element model in the history table may be added to the candidate list in a predefined order. In one embodiment, the order of addition of history candidates may be from the beginning of the table to the end of the table. In another embodiment, the order of addition of history candidates may be from some predefined location to the end of the table. In another embodiment, the order of addition of history candidates may be from the end of the table to the beginning of the table. In another embodiment, the order of addition of history candidates may be from some predefined location to the beginning of the table. In another embodiment, the order of addition of history candidates may be in an interleaved fashion (e.g., the first addition is from the beginning of the table, the second addition is from the end of the table, and so on).

In one embodiment, a single cross-element model history table may be maintained to store previous cross-element models, and the cross-element model history table may be reset at the beginning of a current picture, a current slice, a current tile, every M CTU rows, or every N CTUs, where N and M may be any value greater than 0. In another embodiment, the cross-element model history table may be reset at the end of the current picture, current slice, current tile, current CTU row, or current CTU.

In another embodiment, an image may be divided into a plurality of regions, and a history table is maintained for each region. During encoding/decoding, history table 0 and an additional history table will be updated. The additional history table may be determined by the current location. For example, if the current CU is located in the second region, the additional history table to be updated is history table 2.

In another embodiment, multiple history tables are used for different update frequencies. For example, the first history table is updated every CU, the second history table is updated every two CUs, the third history table is updated every four CUs, and so on.

In another embodiment, multiple history tables are used to store different types of cross-element models. For example, a first history table is used to store a single model and a second history table is used to store multiple models. As another example, a first history table is used to store gradient models and a second history table is used to store non-gradient models. As another example, a first history table is used to store a simple linear model (e.g., y=ax+b) and a second history table is used to store a complex model (e.g., CCCM).

In another embodiment, multiple history tables are used for different reconstructed luminance intensities. For example, if the average value of the reconstructed luma samples in the current block is greater than a predefined threshold, the cross-element model will be stored in a first history table, otherwise, the cross-element model will be stored in a second history table. In another embodiment, multiple history tables are used for different reconstructed chroma intensities. For example, if the average of neighboring reconstructed chroma samples in the current block is greater than a predefined threshold, the cross-element model will be stored in a first history table, otherwise, the cross-element model will be stored in a second history table.

In one embodiment, when a history candidate is added to a candidate list from a plurality of history tables, the order of addition may be from the beginning of a certain table to the end of a certain table, and then the next history table is added in the same order or in the reverse order. In another embodiment, the order of addition may be from the end of a table to the beginning of a table, and then the next history table is added in the same order or in the reverse order. In another embodiment, the order of addition may be from some predefined position of a table to the end of a table, and then the next history table is added in the same order or in the reverse order. In another embodiment, the order of addition may be from some predefined position of some table to the beginning of some table, then adding the next history table in the same order or in the reverse order. In another embodiment, the order of addition of history candidates may be performed in an interleaved fashion in a certain history table (e.g., a first added candidate from the beginning of a certain history table, a second added candidate from the end of a certain history table, and so on), and then the next history table is added in the same order or in the opposite order.

In another embodiment, the order of addition may be from the beginning of each history table to the end of each history table. In another embodiment, the order of addition may be from the end of each history table to the beginning of each history table. In another embodiment, the order of addition may be from some predefined position in each history table to the end of each history table. In another embodiment, the order of addition may be from some predefined position of each history table to the beginning of each history table. In another embodiment, the order of addition of history candidates may be performed in an interleaved fashion in each particular history table (e.g., the first added candidate starts from all history tables, the second added candidate ends from all history tables, and so on).

In one embodiment, multiple cross-element model history tables are used, but not all history tables are used to create candidate lists. Only the history table whose area is close to the current block area can be used to create the candidate list.

In one embodiment, if history candidates are used, the range of selecting non-adjacent candidates may be reduced by using smaller distances between non-adjacent candidate locations. In another embodiment, if history candidates are used, the number of non-neighboring candidates may be reduced by measuring the distance from the top left position of the current block to the candidate position and then excluding candidates whose distance is greater than a predefined threshold. In another embodiment, if history candidates are used, the number of non-adjacent candidates may be reduced by skipping candidates that are not located in the same area. In another embodiment, if history candidates are used, the number of non-adjacent candidates may be reduced by skipping candidates that are not located in the adjacent region. The range of the neighborhood is predefined and may be an M by N region, where M and N may be any value greater than 0. In another embodiment, if history candidates are used, the range of selecting non-adjacent candidates may be reduced by skipping the second search mode.

In another embodiment, a picture may be divided into a plurality of regions, and at least one history table is maintained in each region. For a region of the current picture, it may use or incorporate a history table of one or more regions in the previous encoded picture as the initial history table. For example, if a picture is divided into N regions, it may implicitly or explicitly select a history table of one of the N regions in the previous encoded picture as the initial history table. The index of one of the N regions may be signaled or implicitly derived from the corresponding region in the previous coded picture. As shown in fig. 22A-B, where current picture 2220 is a P/B encoded picture, previous picture 2210 is an intra encoded picture. Each picture is divided into 4 regions, as shown by the 4 rectangular boxes. According to an embodiment of the present invention, the corresponding region in the previous encoded picture may be the region 2212 having the same starting geometric position as the current region 2222 as shown in fig. 22A, or the region including the central geometric position of the current region 2222 as shown in fig. 22B. For another example, it may construct a history table for the current region in combination with multiple history tables in the previous encoded region/picture (e.g., a method in the section entitled "inherit candidates from candidate list of neighbors").

Available region of non-contiguous spatial candidates

To limit the required buffering/storage resources, the available range including non-contiguous spatial candidates should be limited. In one embodiment, only Cross-Component Model (CCM) information in the current CTU may be referenced by non-contiguous spatial candidates. In another embodiment, only CCM information in the current CTU or the left M CTUs may be referenced by non-contiguous spatial candidates. M may be any integer greater than 0. In another embodiment, only CCM information in the current CTU row may be referenced by non-contiguous spatial candidates. In another embodiment, only the locations to be referenced within the current CTU row or the N CTU rows above may be referenced. N may be any integer greater than 0. Note that CCM information referred to in this disclosure includes, but is not limited to, prediction modes (e.g., CCLM, MMLM, CCCM), GLM mode indices, model parameters, or classification thresholds.

In another embodiment, CCM information in the current CTU, the current CTU row, the current CTU row+the N CTU rows above, the current ctu+the left M CTUs, or the current ctu+the N CTU rows above+the left M CTUs may be referenced without limitation. Furthermore, CCM information in other areas can only be referenced by larger predefined units. For example, CCM information in the current CTU row is stored in a 4x4 grid, while other CCM information outside the current CTU row is stored in a 16x16 grid. In other words, a 16x16 region need only store one CCM information, so the location to be referenced should be rounded to the 16x16 grid, or changed to the nearest location of the 16x16 grid.

In another embodiment, CCM information in the current CTU row or current CTU row+m CTU rows may be referenced without limitation, and for the locations to be referenced in the upper CTU row, the locations will be mapped to a row above the current CTU or the current CTU row+m CTU rows reference. This design may preserve most of the codec efficiency and not add too much buffering to store CCM information for the CTU row above. For example, CCM information in the current CTU row (2310) and the upper first CTU row (2312) may be referenced without limitation, while for locations to be referenced in the upper second (2320), upper third (2322), upper fourth CTU row, etc., locations will be mapped to a row (2330) above the upper first CTU row (2312) (as shown in FIG. 23). In fig. 23, dark circles represent unavailable candidates 2340, dot-filled circles represent available candidates 2342, and empty circles represent mapped candidates 2344. For example, the unavailable candidate 2350 in the upper third (2322) CTU row is mapped to the available candidate 2352 in a row (2330) above the upper first CTU row (2312).

In the above example, the region that may be unrestricted-referenced is close to the current CTU (e.g., the current CTU row or the first CTU row above). However, the area according to the present invention is not limited to the above exemplary area. The area may be larger or smaller than the examples described above. In general, the region may be limited to one or more predefined distances from the current CTU in a vertical direction, a horizontal direction, or both. In the above example, the area is limited to 1CTU height in the upper vertical direction, and can be extended to 2 or 3CTU heights if desired. In the case of using the left M CTUs, the M CTU width of the current CTU row is limited. The horizontal position of the location to be referenced and the horizontal position of the mapped predefined location may be the same (e.g., location 2350 and location 2352 are at the same horizontal position). However, other horizontal positions may be used.

In another embodiment, CCM information in the current CTU row or the current CTU row+m CTU row may be referenced without limitation. Further, for the locations to be referenced in the CTU row described above, the locations will map to the last row of the corresponding CTU row for referencing. For example, as shown in fig. 24, CCM information in the current CTU row (2310) and the upper first CTU row (2312) may be referenced without limitation, and for the locations to be referenced in the upper second CTU row (2320), the locations will map to the bottom line (2330) of the upper second CTU row (2320). For the locations to be referenced in the upper third CTU row (2322), the locations will map to the bottom line (2420) of the upper third CTU row (2322). For example, the unavailable candidate 2350 in the upper third CTU row (2322) maps to the map candidate 2430 in the bottom line (2420) of the upper third CTU row (2322). The legend of the candidate types (i.e., 2340, 2342, and 2344) of fig. 24 is the same as in fig. 23. In this example, the unconstrained region may include one or more upper CTU rows (e.g., 1CTU in fig. 24). The upper second CTU row is located above the unconstrained region. The upper third CTU row is also referred to as the upper CTU row because it is located above the CTU row above the unconstrained region (i.e., the upper second CTU row).

In another embodiment, CCM information in the current CTU row or current CTU row+m CTU row may be referenced without limitation, and for the locations to be referenced in the upper CTU row, the locations will map to the last row or bottom line or centerline of the corresponding CTU row for referencing according to the locations of the CCM information to be referenced. For example, as shown in fig. 25, CCM information in the current CTU row (2310) and the upper first CTU row (2312) may be referenced without limitation, for position 1 to be referenced in the upper second CTU row (2320), the position will map to the bottom line of the upper second CTU row (2330) before referencing. However, for position 2 to be referenced in the upper second CTU row, the position will map to the centerline (2510) of the upper second CTU row (2320) before referencing because it is closer to the centerline (2510) than the bottom line (2330). The legend of the candidate types (i.e., 2340, 2342, and 2344) of fig. 25 is the same as in fig. 23.

In another embodiment, CCM information in the current CTU row or current CTU row+m CTU row may be referenced without limitation, and for the locations to be referenced in the upper CTU row, the locations will map to the last row or bottom line of the corresponding CTU row for referencing according to the location of the CCM information to be referenced. For example, as shown in fig. 26, CCM information in the current CTU row (2310) and the upper first CTU row (2312) may be referenced without limitation, for position 1 to be referenced in the upper second CTU row (2320), the position will map to the bottom line (2330) of the upper second CTU row (2320) before referencing. However, for position 2 to be referenced in the upper second CTU row (2320), the position will map to the bottom line (2420) of the upper third CTU row (2322) before referencing because it is closer to the bottom line (2420) of the upper third CTU row than the bottom line (2330) of the upper second CTU row, as shown in fig. 26. The legend of candidate types (i.e., 2340, 2342, and 2344) is the same as in fig. 23.

In another embodiment, CCM information in the current Coding Tree Unit (CTU) or CCM information in the current CTU plus left N CTUs may be referenced without limitation, for the left CTU, the location to be referenced will be mapped to the rightmost row closest to the current CTU or current CTU plus left N CTUs. For example, CCM information in the current CTU and the first left CTU may be referenced without limitation, and if the locations to be referenced are in the second left CTU, then these locations will be mapped to a row to the left of the first left CTU. If the locations to be referenced are in the third left CTU, then these locations will be mapped to a row to the left of the first left CTU. For example, CCM information in the current CTU and the first left CTU may be referenced without limitation, and if the locations to be referenced are in the second left CTU, then these locations will be mapped to the rightmost row of the second left CTU. If the locations to be referenced are in the third left CTU, then these locations will be mapped to the right-most row of the third left CTU.

In another embodiment, when the available range limits inclusion of non-neighboring candidates, if the non-neighboring candidates are located beyond the available range, the candidates will be skipped and will not be inserted into the candidate list. The available region may be the current CTU, the current CTU row plus the top N CTU rows, the current CTU plus the left M CTUs, or the current CTU plus the top N CTU rows plus the left M CTUs.

Models generated based on other inheritance models

In another embodiment, a single cross-element model may be generated from multiple cross-element models. For example, if one candidate is encoded using multiple cross-element models (e.g., MMLM or CCCM with multiple models), a single cross-element model may be generated by selecting the first or second one of the multiple cross-element models.

Candidate list construction

In one embodiment, the candidate list is constructed by adding candidates in a predefined order until a maximum number of candidates is reached. The added candidates may include all or part of the above candidates, but are not limited to the above candidates. For example, the candidate list may include spatial proximity candidates, temporal proximity candidates, historical candidates, non-adjacent proximity candidates, single model candidates generated based on other inheritance models or combined models (as mentioned in the following section: inheritance multiple cross-element models). In another case, the candidate list may include the same candidates as the previous example, but the candidates are added to the list in a different order.

In another embodiment, if all predefined neighboring and historical candidates have been added but the maximum number of candidates has not been reached, some default candidates are added to the candidate list until the maximum number of candidates is reached.

In one sub-embodiment, the default candidates include, but are not limited to, the candidates described below. The final scaling parameter α is from the set {0,1/8, -1/8, +2/8, -2/8, +3/8, -3/8, +4/8, -4/8}, the offset parameter β=1/(1 < < bit_depth) being derived based on neighboring luma and chroma samples. For example, if the average of neighboring luminance and chrominance samples is lumaAvg and chromaAvg, respectively, β is derived by β= chromaAvg- α· lumaAvg. The average value (lumaAvg) of neighboring luminance samples may be determined by all selected luminance samples, the luminance DC mode value of the current luminance CB, or the average value of the maximum and minimum luminance samples (e.g., Or (b)And (5) calculating to obtain the product. Likewise, the average value (chromaAvg) of neighboring chroma samples may be determined by all selected chroma samples, the chroma DC mode value of the current chroma CB, or the average of the maximum and minimum chroma samples (e.g., Or (b)And (5) calculating to obtain the product.

In another sub-embodiment, the default candidates include, but are not limited to, the candidates described below. The default candidate is α·g+β, where G is the luminance sample gradient instead of the downsampled luminance sample L. The 16 GLM filters described in the section "gradient Linear Model (GRADIENT LINEAR Model, GLM for short)" are applied. The final scaling parameter α is from the set {0,1/8, -1/8, +2/8, -2/8, +3/8, -3/8, +4/8, -4/8}. The offset parameter β=1/(1 < < bit_depth) or derived based on neighboring luminance and chrominance samples.

In another embodiment, the default candidate may be an early candidate with incremental scaling parameter optimization. For example, if the scaling parameter of the early candidate is α, then the scaling parameter of the default candidate is (α+Δα), where Δα may be from the set {1/8, -1/8, +2/8, -2/8, +3/8, -3/8, +4/8, -4/8}. The default candidate offset parameter will be derived from (α+Δα) and the average of neighboring luma and chroma samples of the current block.

In another embodiment, instead of inheriting parameters from neighbors, the default candidate may be a shortcut that indicates a cross-element mode (i.e., deriving a cross-element model using current neighboring luma/chroma reconstruction samples). For example, the default candidate may be CCLM_LA, CCLM_ L, CCLM _ A, MMLM _LA, MMLM_ L, MMLM _A, single model CCCM, multi-model CCCM, or a cross-element model with a specified GLM mode.

In another embodiment, the default candidate may be a cross-element mode (i.e., deriving a cross-element model using current neighboring luma/chroma reconstruction samples) instead of inheriting parameters from neighbors, and also with a scaling parameter update (Δα). Then, the scaling parameter of the default candidate is (α+Δα). For example, the default candidate may be cclm_la, cclm_ L, CCLM _ A, MMLM _la, mmlm_l, or MMLM _a. For another example, Δα may be from the set {1/8, -1/8, +2/8, -2/8, +3/8, -3/8, +4/8, -4/8}. The default candidate offset parameter will be derived from (α+Δα) and the average of neighboring luma and chroma samples of the current block. For another example, Δα may be different for each color component.

In another embodiment, the default candidate may be an early candidate with a portion of the selected model parameters. For example, assuming that the early candidate has m parameters, it may select k of the m parameters from the early candidates as default candidates, where 0< k < m and m >1.

In another embodiment, the default candidate may be the first model of the early MMLM candidates (i.e., the model used when the sample value is less than or equal to the classification threshold). In another embodiment, the default candidate may be the second model of the early MMLM candidate (i.e., the model used when the sample value is greater than or equal to the classification threshold). In another embodiment, the default candidate may be a combination of two models of the early MMLM candidates. For example, if the model of the early MMLM candidate isAndThe default candidate model parameters may be Where a is a weighting factor, which may be predefined or implicitly derived based on the cost of the neighboring templates,Is the x-th parameter of the y-th model.

In another embodiment, default candidates may be derived from reconstructed samples of non-adjacent neighboring regions. Let the current block position be (x, y) and the block size be w×h. If reconstructed samples in the MxN region of (x+dx, y+dy) are "available", then the reconstructed luma and chroma samples in that region may be used to derive default candidates. For example, mxN may be 8x8. As another example, mxN may be 16x8. As yet another example, mxN may be 16x16. As yet another example, mxN may be w.times.h. "available" means that reconstructed samples within the current block are available, or that reconstructed samples within k rows of neighboring samples are available. k may be defined by IBC neighbor search regions or by neighbor buffer regions of other intra-coding tools (e.g., multi-reference line intra prediction, CCLM or CCCM).

In another embodiment, assuming the current block is (x, y) in position and the block size is w×h, if reconstructed samples in the MxN region of (x _mid+dx,y_mid +dy) are available, then the reconstructed samples in that region can be used to derive the default candidates, where (x _mid,y_mid) = (x+w/2, y+h/2).

In another embodiment, the default candidates derived from reconstructed samples of non-adjacent neighboring regions may be any type of cross-element model or some particular type of cross-element model. For example, the derived model may be a CCLM, MMLM, CCCM, CCCM multiple model or other cross-element model. As another example, the derived model is the CCCM model. As yet another example, the derived model is a CCLM model. As yet another example, the derived model is CCCM or CCCM multiple models.

In another embodiment, assume that two value sets α _x and α _y are defined:

α_x＝{α_x1,α_x2,α_x3,…,α_xn},α_xi<α_xjif i<j

α_y＝{α_y1,α_y2,α_y3,…,α_yn},α_yi<α_yjif i<j。

All values in α _x and α _y are positive numbers. (dx, dy) may be (α_xi×w,-α_yi×h),(-α_xi×w,α_yi×h),(-α_xi×w,-α_yi×h),(α_xi×w,0),(-α_xi×w,0),(0,α_yi×h),(0,y_mid-α_yi×h).

In another embodiment, the current block is located at (x, y) and the block size is w×h. Let δx and δy be two fixed positive numbers, (dx, dy) may be (α_xi×δx,-α_yi×δy),(-α_xi×δx,+α_yi×δy),(-α_xi×δx,-α_yi×δy),(α_xi×δx,0),(-α_xi×δx,0),(0,α_yi×δy),or(0,-α_yi×δy).

In constructing the candidate list, candidates are inserted into the list according to a predefined order. For example, the predefined order may be spatially adjacent candidates, temporal candidates, spatially non-adjacent candidates, historical candidates, and then default candidates. In one embodiment, if a cross-element model is derived for a non-LM encoded block (e.g., as mentioned in the section entitled "inherit neighboring model parameters to optimize cross-element model parameters"), then candidate models for the non-LM encoded block are included in the list after including the candidate models for the LM encoded block. In another embodiment, if a cross-element model is derived for a non-LM encoded block, candidate models for the non-LM encoded block are included in the list before the default candidate is included. In yet another embodiment, if a cross-element model is derived for a non-LM encoded block, then the candidate model for the non-LM encoded block is included in the list with a lower priority than the candidate model from the LM encoded block.

In constructing the candidate list, only candidates having a particular prediction mode may be added to the list. For example, it may be restricted that only candidates derived by CCLM or MMLM modes are allowed to be added to the list. As another example, it may be restricted that only candidates derived by a single model mode (e.g., CCLM, or CCLM with a single model) are allowed to be added to the list. As yet another example, it may be restricted that only candidates derived by multiple model modes (e.g., MMLM, or CCCM with multiple models) are allowed to be added to the list. As yet another example, it may be restricted that only candidates derived by GLM mode are allowed to be added to the list. As yet another example, it may be restricted that only candidates derived by a particular mode (e.g., CCLM, MMLM, CCCM, CCCM with multiple models, or GLM) are allowed to be added to the list. In one embodiment, if only candidates with a particular prediction mode can be added to the list, the prediction mode may be signaled first and then whether the proposed cross-element merge mode is used when the prediction mode signaling is performed. If the proposed cross-element merge mode is used, the candidate index is signaled.

In the chroma intra fusion mode, non-CCLM encoded intra prediction and CCLM encoded intra prediction are fused together to obtain the final intra prediction. In one embodiment, when inheriting cross-element model parameters from blocks/locations encoded by chroma intra fusion modes, the model parameters used to obtain intra prediction for CCLM encoding are inherited and further optimized. In another embodiment, the fusion weights, the codec mode of the non-CCLM encoded intra prediction, and the model parameters used to obtain the CCLM encoded intra prediction are inherited and further optimized. In yet another embodiment, the codec mode of non-CCLM-encoded intra prediction is implicitly derived (e.g., as DM or planar mode), and the fusion weights and model parameters used to obtain CCLM-encoded intra prediction are inherited and further optimized. In yet another embodiment, if non-CCLM encoded intra prediction of a block/location encoded by a chroma intra fusion mode can be implicitly derived (e.g., the non-CCLM encoded intra prediction is DM or planar mode), then fusion weights and model parameters for obtaining CCLM encoded intra prediction are inherited and further optimized.

Removing or modifying similar proximity model parameters

When inheriting cross-element model parameters from other blocks, the similarity between the inherited model and existing models in the candidate list or those model candidates derived from neighboring reconstructed samples of the current block may be further checked (e.g., models derived using neighboring reconstructed samples of the current block, such as CCLM, MMLM, or CCCM). If the model of the candidate parameter is similar to the existing model, the model will not be included in the candidate list. In one embodiment, (α× lumaAvg +β) or similarity of α between existing candidates may be compared to determine whether to include a model of the candidate. For example, if either (α× lumaAvg +β) or α of the candidates is the same as one of the existing candidates, then no model of the candidate is included. Another example is a model that does not contain candidates if the difference in (a x lumaAvg + β) or a between the candidate and one of the existing candidates is less than a threshold. Further, the threshold may be adaptively adjusted based on the codec information (e.g., the size or area of the current block). As yet another example, when comparing similarities, if both the candidate and the existing model use CCCM, it may be determined whether the model of the candidate is contained by examining the value of (c ₀C+c₁N+c₂S+c₃E+c₄W+c₅P+c₆ B). In another embodiment, the model of the candidate parameter is not included if the candidate location points to the same CU as the existing candidate. In yet another embodiment, if the model of the candidate is similar to one of the existing candidate models, the inherited model parameters may be adjusted so that the inherited model is different from the existing candidate model. For example, if the inherited scaling parameters are similar to one of the existing candidate models, a predefined offset (e.g., 1> > S or- (1 > > S), where S is a shift parameter) may be added so that the inherited parameters are different from the existing candidate models.

In another embodiment, only a portion of the model parameters of the existing models in the candidate list are compared. For example, one CCLM candidate has scaling and offset parameters, and it may only be compared whether the scaling or offset parameters are the same as or similar to existing candidates. If the scaling or offset parameters are the same or similar, the model will not be included in the candidate list. As yet another example, one CCCM candidate has the c ₀ to c ₆ parameters, and it is possible to compare only if n parameters (n < 7) are the same or similar to the existing candidate. If the scaling or offset parameters are the same or similar, the model will not be included in the candidate list.

In another embodiment, the candidate model may be applied to neighboring reconstructed samples of the current block and the differences compared to existing candidate models. If the variance value is less than or equal to the threshold, the model will not be included in the candidate list. For example, assume that the application results inAnd the corresponding result of the existing model in the candidate list is thatTo the point ofIf it isOr (b)The model will not be included in the candidate list. For selection of the neighboring reconstructed samples, the neighboring reconstructed sample having the maximum value, the neighboring reconstructed sample having the minimum value, the average/median/mode of the neighboring reconstructed samples, the left neighboring reconstructed sample, the upper neighboring reconstructed sample, or the upper left neighboring reconstructed sample may be selected.

In another embodiment, the number of candidates of the same type (e.g., MMLM, CCCM, or GLM) is limited when the candidates are included in the list. For example, if there are k MMLM types of candidates in the current list, then no further MMLM types of candidates are allowed to be included in the list. Another example is that if there are k CCCM types of candidates in the current list, then no further CCCM types of candidates are allowed to be included in the list. Yet another example is that if there are k GLM type candidates in the current list, no further GLM type candidates are allowed to be included in the list.

In another embodiment, the default candidate will not be compared to existing models in the candidate list and will be included in the candidate list.

Candidates in the reordered list

Candidates in the list may be reordered to reduce syntax overhead in signaling the selected candidate index. The reordering rules may depend on the codec information or model errors of neighboring blocks. For example, if a neighboring upper or left block is coded by MMLM, the MMLM candidate in the list may be moved to the beginning of the current list. Likewise, if an adjacent upper or left block is being encoded by a single model LM or CCCM, the single model LM or CCCM candidate in the list may be moved to the beginning of the current list. Likewise, if a GLM is used by a neighboring upper or left block, the GLM related candidate in the list may be moved to the beginning of the current list.

In another embodiment, the reordering rules are based on comparing errors with reconstructed samples of neighboring templates by applying the candidate model to neighboring templates of the current block. For example, as shown in FIG. 27, the size of upper adjacent template 2720 of current block is w _a×h_a, and the size of left adjacent template 2730 of current block 2710 is w _b×h_b. Assuming that there are K models in the current candidate list, α _k and β _k are the final scale and offset parameters after inheriting candidate K. The model error of candidate k corresponding to the upper neighbor template is:

wherein, the AndIs the luma and chroma reconstructed samples at position (i, j) in the upper template (e.g., after a downsampling process or GLM mode is applied), 0≤i < w _a and 0≤j < h _a.

Likewise, the model error of candidate k through the left-hand neighbor template is:

Wherein the method comprises the steps of AndIs the luminance reconstruction sample (e.g., after a downsampling process or gradient linear Model (GRADIENT LINEAR Model, GLM) mode processing) and the chrominance reconstruction sample at position (m, n) in the left template, 0≤m < w _b and 0≤n < h _b. The model error for candidate k is then: After calculation of all candidate model errors, a model error list e= { E ⁰,e¹,e²,…,e^k,…,e^K }, can be obtained. The candidate indexes in the inheritance candidate list may then be rearranged by ascending order of the model error list.

In another embodiment, if candidate k is predicted using a convolution cross-element model (ConvolutionalCross-Component Mode, CCCM for short),AndThe definition is as follows:

Where c0 _k,c1_k,C2_k,c3_k,c4_k,c5_k and c6 _k are the final filter coefficients after inheriting candidate k. P and B are nonlinear terms and bias terms.

In another embodiment, if the above-mentioned proximity template is not available, thenSimilarly, if a left-hand neighbor template is not available, thenIf neither template is available, then the candidate index reordering method using model errors is not applied.

In another embodiment, not all positions within the top and left neighbor templates are used to calculate the model error. Partial positions within the top and left adjacent templates may be selected to calculate model errors. For example, a starting position and a sampling interval may be defined, depending on the width of the current block, to partially select the position within the upper neighbor template. Likewise, a second starting position and a second sampling interval may be defined, depending on the height of the current block, to partially select a position within the left-hand neighboring template. As another example, h _a or w _b may be a constant value (e.g., h _a or w _b may be 1, 2, 3, 4, 5, or 6). As another example, h _a or w _b may depend on the size of the block. If the current block size is greater than or equal to a threshold, h _a or w _b is equal to the first value. Otherwise, h _a or w _b is equal to the second value.

In another embodiment, the different types of candidates are reordered separately before being added to the final candidate list. For each type of candidate, the candidate is added to a primary candidate list having a predefined size N ₁. Candidates in the primary list are reordered. The candidate with the least cost (N ₂)) is then added to the final candidate list, where N ₂≤N₁. In another embodiment, candidates are classified into different types according to their origin, including but not limited to spatial proximity models, temporal proximity models, non-adjacent spatial proximity models, and historical candidates. In another embodiment, candidates are classified into different types according to cross-element model patterns. For example, the types may be CCLM, MMLM, CCCM and CCCM multiple models. As another example, the type may be GLM inactive or GLM active.

In another embodiment, after the candidates are reordered according to the template cost, the redundancy of the candidates may be further checked. A candidate is considered redundant if the difference in template cost between the candidate and its previous candidate in the list is less than a threshold. If a candidate is considered redundant, it may be removed from the list or may be moved to the end of the list.

Inheritance of candidates from candidate list of neighboring blocks

The candidates in the current inheritance candidate list may be from neighboring blocks. For example, the first k candidates in the inheritance candidate list of the neighboring block may be inherited. As shown in fig. 28, the current block may inherit the first two candidates in the inheritance candidate list of the upper neighboring block and the first two candidates in the inheritance candidate list of the left neighboring block. In one embodiment, after adding the neighboring spatial candidate and the non-neighboring spatial candidate, if the current inheritance candidate list is not full, the candidates in the candidate list of the neighboring block are included in the current inheritance candidate list. In another embodiment, when a candidate in the candidate list of a neighboring block is included, the candidate in the candidate list of the left neighboring block is included before the candidate in the candidate list of the upper neighboring block. In another embodiment, when a candidate in the candidate list of a neighboring block is included, the candidate in the candidate list of the upper neighboring block is included before the candidate in the candidate list of the left neighboring block.

Signaling inheritance candidate indexes in a list

An on/off flag may be signaled to indicate whether the current block inherits cross-element model parameters from neighboring blocks. The flag may be signaled on each CU/CB, each PU, each TU/TB, each color element, or each chroma color element. The high level syntax may be signaled in SPS, PPS (picture parameter set), PH (picture header) or SH (slice header) to indicate whether the proposed method is allowed for the current sequence, picture or slice.

The maximum allowed number of candidates is signaled to indicate the maximum size of the merge candidate list. The number may be signaled on each CU/CB, each PU, each TU/TB, each color element, or each chroma color element. A high level syntax may be signaled in SPS, PPS, PH or SH to indicate whether the proposed method is allowed for the current sequence, picture or slice. The maximum allowed candidate number of the proposed method (i.e. CCP merge mode) can be shared with the maximum allowed candidate number for inter merge mode.

If the current block inherits cross-element model parameters from neighboring blocks, the inherited candidate index is signaled. The index may be signaled (e.g., signaled using a truncated unary code, exp-Golomb code, or fixed length code) and shared between the current Cb and Cr blocks. As another example, the index may be signaled on each color element. For example, one inheritance index is signaled for Cb elements and another inheritance index is signaled for Cr elements. As another example, a chroma intra prediction syntax (e.g., intraPredModeC [ xCb ] [ yCb ]) may be used to store the inheritance index.

If the current block inherits the cross-element model parameters from neighboring blocks, then the current chroma intra prediction mode (e.g., intraPredModeC [ xCb ] [ yCb ] defined in the VVC standard) is temporarily set to the cross-element mode (e.g., CCLM_LA) during the bitstream syntax parsing phase. Subsequently, in the prediction phase or reconstruction phase, the candidate list is derived and the inherited candidate model is determined by the inherited candidate index. After the inheritance model is obtained, the encoding and decoding information of the current block is updated according to the inheritance candidate model. The codec information of the current block includes, but is not limited to, a prediction mode (e.g., cclm_la or MMLM _la), a related sub-mode flag (e.g., CCCM mode flag), a prediction mode (e.g., GLM mode index), and current model parameters. Then, a prediction of the current block is generated according to the updated codec information.

Removing or modifying similar model parameters when adding candidates to a history table

When adding the cross-element model to the history table, the similarity between the model to be added and the existing model in the history table may be further checked. If the model to be added is similar to the existing model, the model to be added will not be included in the history table. In one embodiment, (α× lumaAvg +β) or the similarity of α to existing candidates may be compared to determine whether to include the model to be added. For example, if either (α× lumaAvg +β) or α of the model to be added is the same as one of the existing candidates, the model to be added does not contain. Another example is that if the difference in (α× lumaAvg +β) or α between the model to be added and the existing model is less than a threshold, the model to be added does not contain. Further, the threshold may be adaptively adjusted based on the codec information (e.g., the size or area of the current block). Another example is that in comparing the similarity, if both the model to be added and the existing model use CCCM, it can be decided whether to include the model to be added by checking the value of (c ₀C+c₁N+c₂S+c₃E+c₄W+c₅P+c₆ B). In another embodiment, the model parameters to be added do not include if the current CU position of the model to be added is the same as the current candidate CU position. In another embodiment, if the model to be added is similar to one of the existing candidate models, the inherited model parameters may be adjusted to make the model to be added different from the existing candidate model. For example, if the scaling parameter to be added is similar to one of the existing candidate models, the scaling parameter to be added may be added by a predefined offset (e.g., 1> > S or- (1 > > S), where S is a shift parameter), making the model to be added different from the existing candidate model.

In another embodiment, only a portion of the model parameters are compared to existing models in the history table. For example, one CCLM candidate has scaling and offset parameters, and only whether the scaling or offset parameters are the same as or similar to existing candidates may be compared. If the scaling or offset parameters are the same or similar, the model to be added is not included in the history table. As another example, one CCCM candidate with the c ₀ to c ₆ parameters, only n parameters (n < 7) may be compared to if they are the same or similar to the existing candidate. If the scaling or offset parameters are the same or similar, the model to be added is not included in the history table.

In another embodiment, the model to be added may be applied to neighboring reconstructed samples of the current block and the differences compared to existing candidate models. If the variance value is less than or equal to the threshold value, the model to be added is not included in the history table. For example, assume that the application results inWhile the corresponding result of the existing model in the history table isTo the point ofIf it isOr (b)The model to be added is not included in the history table. For selection of the neighboring reconstructed samples, the neighboring reconstructed sample having the maximum value, the neighboring reconstructed sample having the minimum value, the average/median/mode of the neighboring reconstructed samples, the left neighboring reconstructed sample, the upper neighboring reconstructed sample, or the upper left neighboring reconstructed sample may be selected.

In another embodiment, the number of candidates having the same type (e.g., MMLM, CCCM, or GLM) is limited when the candidates are included in the history table. For example, if there are k MMLM types of candidates in the current history table, then no further candidates of MMLM types are allowed to be included in the history table. Another example is that if there are k CCCM types of candidates in the current history table, then no further candidates of CCCM types are allowed to be included in the history table. As yet another example, if there are k GLM type candidates in the current history table, then no further GLM type candidates are allowed to be included in the history table.

In another embodiment, the constraints or rules for preventing the addition of redundancy candidates to the history table will be the same as the constraints or rules for preventing the addition of redundancy candidates to the candidate list (e.g., the constraints or rules mentioned in the section entitled "remove or modify similar proximity model parameters").

Inheritance of multiple cross-element models

The final prediction of the current block may be a combination of multiple cross-element models, or a fusion of the selected cross-element model with predictions of non-cross-element codec tools (e.g., intra-angle prediction mode, intra-plane/DC mode, or inter-prediction mode). In one embodiment, if the size of the current candidate list is N, k candidates (where k≤N) may be selected from a total of N candidates. Then, k predictions are generated separately by applying the selected k candidate cross-element models to the corresponding luma reconstruction samples. The final prediction of the current block is the combined result of these k predictions. For example, if two candidate predictions (denoted p _cand1 and p _cand2) are combined, the final prediction of the current block at the (x, y) position is p _final(x,y)＝(1-α)×p_cand1(x,y)+α×p_cand2 (x, y), where α is a weighting factor. Furthermore, the weighting factor α may be predefined or implicitly derived by the proximity template cost. For example, by using the template costs defined in the section entitled "inheritance non-adjacent spatial proximity model", the corresponding template costs for two candidates are e ^cand1 and e ^cand2, then α is e ^cand1/(e^cand1+e^cand2). In another embodiment, if two candidate models are combined, the selected model is from the first two candidates in the list. In yet another embodiment, if i candidate models are combined, the selected model is from the first i candidates in the list.

In another embodiment, if the size of the current candidate list is N, k candidates (where k≤N) may be selected from a total of N candidates. The k cross-element models can be combined into a final cross-element model by weighted averaging of the corresponding model parameters. For example, if one cross-element model has M parameters, the j-th parameter of the final cross-element model is a weighted average of the j-th parameters of the k candidates selected, where j is 1. The final prediction is then generated by applying the final cross-element model to the corresponding luma reconstruction samples. For example, if two candidate models areAndThe final cross-element model is Where a is a weighting factor, can be predefined or implicitly derived from the cost of the neighboring templates,Is the xth model parameter of the yth candidate. For example, by using the template costs defined in the section entitled "inheritance non-adjacent spatial proximity model", the corresponding template costs for two candidates are e ^cand1 and e ^cand2, then α is e ^cand1/(e^cand1+e^cand2). For another example, two candidate models are one from spatially adjacent neighboring candidates and the other from non-adjacent spatial candidates or historical candidates. If spatially adjacent neighboring candidates are not available, both candidate models are from non-adjacent spatial candidates or historical candidates. In another embodiment, if two candidate models are combined, the selected model is from the first two candidates in the list. In yet another embodiment, if i candidate models are combined, the selected model is from the first i candidates in the list.

In another embodiment, two cross-element models are combined into one final model by weighted averaging of the corresponding model parameters, wherein the two cross-element models are from the above spatial proximity candidate and the left spatial proximity candidate, respectively. The above spatial neighboring candidates refer to neighboring candidates having vertical positions less than or equal to the top boundary position of the current block. The left spatial neighboring candidate refers to a neighboring candidate whose horizontal position is less than or equal to the left side position of the current block. The weighting factor alpha is determined from the horizontal and vertical spatial positions within the current block. For example, if two candidate predictions (denoted p _above and p _left) are combined, the final prediction of the current block at the (x, y) position is p _final(x,y)＝(1-α)×p_above(x,y)+α×p_left (x, y), where α=y/(x+y). In another embodiment, the spatial neighboring candidate is the first candidate in the list having a vertical position less than or equal to the top boundary position of the current block. The left spatial neighboring candidate is the first candidate in the list whose horizontal position is less than or equal to the left edge position of the current block.

In another embodiment, cross-element model candidates may be combined with predictions of non-cross-element codec tools. For example, a cross-element model candidate is selected from the list, the prediction of which is denoted as p _ccm. Another prediction may be from chroma DM, chroma DIMD, or intra angle modes, denoted p _non-ccm. The final prediction of the current block at the (x, y) position is p _final(x,y)＝(1-α)×p_ccm(x,y)+α×p_non-ccm (x, y), where α is a weighting factor, which can be derived either by predefining or implicitly by the neighbor template cost. For still the same example, the predictions of the non-cross-element codec tools may be predefined or signaled. The prediction of the non-cross-element codec is chroma DM or chroma DIMD. For another example, the predictions of the non-cross-element codec tool are signaled, but the indices of the cross-element model candidates are predefined or determined by the codec modes of neighboring blocks. For still the same example, if at least one neighboring spatial block is coded using CCCM modes, then the first candidate with CCCM model parameters is selected. If at least one neighboring spatial block is encoded using GLM mode, a first candidate with GLM mode parameters is selected. Similarly, if at least one neighboring spatial block is coded using MMLM modes, then the first candidate with MMLM parameters is selected.

In another embodiment, the cross-element model candidates may be combined with predictions of the current cross-element model. For example, a cross-element model candidate is selected from the list, the prediction of which is denoted as p _ccm. Another prediction may be from a cross-element prediction mode of the current neighboring reconstructed samples, denoted as p _curr-ccm. The final prediction of the current block at the (x, y) position is p _final(x,y)＝(1-α)×p_ccm(x,y)+α×p_curr-ccm (x, y), where α is a weighting factor, which can be derived either by pre-defining or implicitly from the neighbor template cost. For still the same example, the predictions of the current cross-element model may be predefined or signaled. The prediction of the non-cross-element codec is chroma DM or chroma DIMD. For another example, the predictions of the non-cross-element codec tool are signaled, but the indices of the cross-element model candidates are predefined or determined by the codec modes of neighboring blocks. For still the same example, if at least one neighboring spatial block is coded using CCCM modes, then the first candidate with CCCM model parameters is selected. If at least one neighboring spatial block is encoded using GLM mode, a first candidate with GLM mode parameters is selected. Similarly, if at least one neighboring spatial block is coded using MMLM modes, then the first candidate with MMLM parameters is selected.

In another embodiment, multiple cross-element models may be combined into one final cross-element model. For example, one model may be selected from one candidate and a second model may be selected from another candidate as the multi-model pattern. The selected candidate may be a CCLM/MMLM/GLM/CCCM encoding candidate. The multimodal classification threshold may be an average of the offset parameters (e.g., offset/β in CCLM, or c ₆ xb or c ₆ in CCCM) for two selected modes. In one embodiment, if two candidate models are combined, the selected model is the first two candidates in the list. In another embodiment, the classification threshold is set to the average of neighboring luma and chroma samples of the current block.

Refining inheritance candidate locations

In one embodiment, the final inheritance model of the current block is derived from the cross-element model of the indicated candidate location and has one location increment. For example, if the currently selected candidate location isA position increment may be further signaled,To indicate the location of the final inheritance model. That is, the final inheritance model of the current block comes fromIs a cross-element model. In one embodiment, the signal position increment can be only a horizontal position increment or a vertical position increment, i.eOr (b)Further, the signal position delta may be shared among multiple color components or signal for each color component. For example, the signal position delta is shared for the current Cb and Cr blocks, or the signal position delta is only for the current Cb block or the current Cr block. Furthermore, the signalOr (b)There may be a sign bit to indicate either a positive position increment or a negative position increment. When indicatingOr (b)Can index the signal through a look-up table. For example, the look-up table is {1,2,4,8,16, }, ifEqual to 8, signal table index 3 (first table index 0).

In one embodiment, when a candidate is selected from the candidate list, a model of the proximity of the selected candidate is further searched. The final inheritance model may be from a neighborhood of the selected candidate. The location of the predefined search pattern within the selected candidate surrounding area is searched. In one embodiment, the search for neighboring locations is different from the selected candidate in the horizontal or vertical direction, i.e., the location increment isOr (b)In another embodiment, the searched neighbor locations are diagonally different from the selected candidates, i.e., the location increment isWherein the method comprises the steps ofNote that the position increment may be a positive number or a negative number.

In another embodiment, the model of the neighboring location of the candidate is further searched only when the selected candidate is a non-neighboring candidate. The location of the predefined search pattern within the selected candidate surrounding area is searched. For example, assume that the distance between non-neighboring candidates is the width and height of the current codec block. After the non-adjacent candidates are selected, the positions where both the horizontal distance and the vertical distance are smaller than the width and the height of the current codec block, that is,Within the range of + width,Within + -height. In one embodiment, the search for adjacent locations differs from the selected candidate in the horizontal or vertical direction, i.e., the difference in location isOr (b)In another embodiment, the searched neighboring locations are diagonally different from the selected candidate, i.e., the location difference isWherein the method comprises the steps of

Inheritance from shared cross-element model

In one embodiment, the current picture is partitioned into a plurality of non-overlapping regions, each region having a size of mxn. A shared cross-element model is derived for each region separately. The neighboring available luma/chroma reconstruction samples of the current region are used to derive a shared cross-element model of the current region. Then, for blocks within the current region, it may be determined whether to inherit the shared cross-element model or derive the cross-element model from the neighboring available luma/chroma reconstruction samples of the block. In one embodiment, mxn may be a predefined value (e.g., 32x32 for chroma format), a signaled value (e.g., signaled at sequence/picture/slice/tile level), a derived value (e.g., depending on CTU size), or a maximum allowed transform block size.

In another embodiment, there may be multiple shared cross-element models per region. For example, various proximity templates (e.g., top and left side proximity samples, top-only proximity samples, left-only proximity samples) may be used to derive multiple shared cross-element models. Further, the shared cross-element model of the current region may inherit from previously used cross-element models. For example, the shared model may inherit from models in neighboring spatial neighbors, non-neighboring spatial neighbors, temporal neighbors, or history lists.

In signaling, a first flag may be used to determine whether the current cross-element model inherits from the shared cross-element model. If the current cross-element model inherits from the shared cross-element model, the second syntax indicates an inheritance index of the shared cross-element model (e.g., signaled using truncated unary code, exp-Golomb code, or fixed length code).

Sharing buffering resources with existing codec tools

A buffer for storing inter-element model (CCM) information (e.g., prediction mode, related sub-mode flags, prediction modes, or model parameters) for further model inheritance is shared with the inter-element merge mode to store CCM information. By sharing buffers between different codec tools, the buffer size may be reduced. Otherwise, buffer space must be allocated to store CCM information and inter-frame codec information, respectively. The key idea of the shared buffer is that one block is encoded using only one selected codec mode of the multiple candidates. Thus, the codec information of the various codec modes may share one common buffer. Assuming that the minimum block size allowed is m×n, the current CTU size is p×q, and the current picture size is r×s. The CTU-level buffer and the picture-level buffer are used to store inter-frame codec and CCM information of the current CTU and each picture, respectively. Creating a CTU-level buffer for storing final inter-frame codec or CCM information, the CTU-level buffer being of a size ofDue toCorresponding to the number of blocks in the horizontal direction,The number of blocks corresponding to the vertical direction,Corresponding to the total number of blocks in the CTU. Creating a picture-level buffer for storing final inter-frame codec or CCM information for the current picture, the picture-level buffer being of a sizeWherein i≥m and j≥n. In other words, the codec information is stored in the picture buffer in units of i×j. Due toCorresponding to the second number of blocks in the horizontal direction,Corresponding to the second number of blocks in the vertical direction,Corresponding to the second total number of blocks in the picture. After encoding or decoding the current block, inter-frame codec or CCM information of the current block is first stored in units of mxn to a corresponding position of the CTU level buffer, where the corresponding position is a position where the current block is covered in units of mxn. Later, after encoding or decoding the current CTU, inter-frame codec or CCM information in the current CTU level buffer is saved to a corresponding location of the picture level buffer in units of i×j.

However, if the units of the CTU level buffer and the picture level buffer are not identical (e.g., i > m or j > n), the inter-frame codec or CCM information in the CTU level buffer should be sub-sampled to save to the picture level buffer. Assuming i/m=g and j/n=h, one is selected from each gxh grid of the CTU level buffer to save inter-frame codec or CCM information to the corresponding location of the picture level buffer. For example, as shown in fig. 29, if g=2 and h=2, one location is selected from each 2×2 mesh to save inter-frame codec or CCM information to a corresponding location of the picture-level buffer. In one embodiment, the selected location may be an upper left, lower left, upper right, or lower right location of each 2x2 grid. As shown in fig. 29, the inter-frame codec or CCM information marked as the upper left position of the diagonal line in each 2x2 grid is saved to the picture-level buffer. In another embodiment, the prediction mode within the gxh grid may be conditionally checked when the CCM information in the CTU level buffer is sub-sampled for saving to the picture level buffer. For example, if more than a certain percentage of the locations within the gxh grid are intra-mode (e.g., more than 50% or 75%), then the selected and saved data is CCM information. Otherwise (i.e., most of the positions within the gxh grid are inter modes), the data selected and saved is inter codec information. Upon selection of a candidate to save to the picture level buffer, a first allowed candidate may be selected following a predefined scan order. For example, if the selected and stored data is CCM information, the first cell with CCM information within the gxh cell may be selected by a predefined scanning order. For another example, if the selected and saved data is inter-frame codec information, the first grid with inter-frame codec information within the gXh grid may be selected by a predefined scan order.

Since the buffer for storing the inter-frame codec information is shared with the cross-element merge mode, a CU prediction mode (e.g., intra prediction or inter prediction) may be checked to identify whether the information stored at a certain buffer location is inter-frame codec or CCM information. In one embodiment, if the CU prediction mode is intra prediction, the stored information is CCM information. Otherwise (i.e., the CU prediction mode is non-intra prediction), the stored information is inter-coding information. In another embodiment, an invalid inter prediction reference index or an invalid MV value (e.g., horizontal or vertical MV value) may be set to identify that the stored information is CCM information. Otherwise (i.e., a valid inter prediction index), the stored information is inter-coding information. For example, in the VVC standard specification, if the inter prediction reference index is not valid more than 2, the inter prediction reference index may be set to a value more than 2 to identify that the stored information is CCM information (e.g., the inter prediction reference index is 3).

According to the method, a current block is divided into two or more prediction regions/sub-blocks, where each region may be predicted by an inter or intra codec. Furthermore, at least one prediction region is encoded by a CC merge mode, wherein a cross-element model of the at least one region inherits from spatially, historically, or temporally neighboring blocks/locations. In one embodiment, the current block is partitioned by quadtree, binary tree, or trigeminal tree partitioning. The segmentation may be a symmetrical or asymmetrical segmentation.

In another embodiment, the current block is divided into two regions, one of which is predicted by an inter or intra codec and the other is predicted by a CC merge mode. The inheritance candidate index of the region predicted by the CC merge mode may be indicated explicitly or implicitly. For example, the candidate index may be explicitly signaled by a method in the section entitled "signaling inherited candidate index in list". As another example, the first candidate in the list may be implicitly selected as the candidate index. Candidates in the list may be reordered by the method mentioned in the section entitled "reorder candidates in list".

In another embodiment, the current block is divided into two regions, both regions being predicted by the CC merge mode, the first two candidates in the list being candidate indexes of the two regions. The candidate index for the first region (e.g., the region with the upper left corner sample of the current block) may be implicitly set as the first candidate and the candidate index for the second region may be set as the second candidate. In addition, the list may be reordered by the method mentioned in the section entitled "reorder candidates in list". Another example is if both regions are predicted by CC merge mode, then an index is explicitly signaled to indicate the candidate index for the first region, the candidate index for the second region is signaled index + k or signaled index-k, where k may be 1, 2, 3, 4 or 5. For the still same example, the candidate index for the first region is implicitly derived from a cross-element model stored in the upper left corner position of the current block relative to the previously encoded slice/image, as the method mentioned in the section entitled "inherited temporal proximity model parameters". In another embodiment, if both regions are predicted by the CC merge mode, the first candidate in the list is the candidate index for both regions.

Cross-element skip mode

With many cross-element models, intra chroma prediction becomes more accurate and entropy of the residual becomes smaller. Furthermore, the probability that all CBFs (codec block flags) or coding flags of chroma components in intra codec mode are equal to 0 increases compared to the previous video codec standard. Based on this observation, several methods related to skip mode shortcuts are proposed to improve the codec performance of intra chroma prediction.

In one embodiment, a skip flag indicating whether the current encoded block inherits cross-element model parameters from neighboring blocks may be signaled before the merge flag.

In one embodiment, if the skip flag is true (i.e., using skip mode), the merge flag will be inferred to be 1, the index of inheritance candidates will be signaled directly, and the CBF of both the Cb component and the Cr component will be inferred to be 0, as shown in fig. 30. In fig. 30, when the skip flag (i.e., skipFlag) is true (i.e., the value is 1), the index of the inheritance candidate (i.e., mergeIdx) is signaled (3010). The merge flag is inferred to be 1, CBF for both cb and Cr components is inferred to be 0, indicated by dashed box 3012.

In another embodiment, if the skip flag is true, the merge flag will be inferred to be 0, the ccp mode index will be signaled, and both the CBF for the Cb component and the CBF for the Cr component will be inferred to be 0, as shown in fig. 31. In fig. 31, when the skip flag (i.e., skipFlag) is true (i.e., the value is 1), the index of the inheritance candidate (i.e., LMModeIdx) is signaled (3110). The merge flag is inferred to be 0, CBF for both cb and Cr components is inferred to be 0, indicated by dashed box 3112.

In one embodiment, a high level syntax may be signaled in SPS, PPS, PH or SH to indicate whether the current sequence, picture, or slice is allowed to use the proposed skip flag.

The above-described improved cross-element predicted syntax codec may be implemented at either the encoder side or the decoder side. For example, any of the proposed syntax codec methods for cross-element prediction may be implemented in an Intra/Inter codec module in the decoder (e.g., intra pred.150/MC 152 in fig. 1B), or the Intra/Inter codec module is a module in the encoder (e.g., intra pred.110/Inter pred.112 in fig. 1A). Any of the proposed shared buffers for storing codec information between multiple codec tools, including CCM mode, may also be implemented as circuitry coupled with an intra/inter codec module of a decoder or encoder. However, the decoder or encoder may also use additional processing units to achieve the desired cross-element prediction processing. Although the intra-prediction units (e.g., units 110/112 in fig. 1A and 150/152 in fig. 1B) are shown as separate processing units, they may correspond to executable software or firmware code stored on a medium such as a hard disk or flash memory for a central processing unit (Central Processing Unit, abbreviated CPU) or a programmable device such as a digital signal Processor (DIGITAL SIGNAL Processor, abbreviated DSP) or a field programmable gate array (Field Programmable GATE ARRAY, abbreviated FPGA).

FIG. 32 illustrates a flow diagram of an exemplary video decoding system that decodes syntax data through one or more context models using information related to cross-element prediction in accordance with an embodiment of the invention. The steps shown in the flowcharts may be implemented as program code executable by one or more processors (e.g., one or more central processing units) at the encoder side. The steps shown in the flowcharts may also be based on a hardware implementation, such as one or more electronic devices or processors arranged to perform the steps in the flowcharts. According to the decoder-side method, encoded data associated with a current block is received in step 3210, comprising a first color block and a second color block, wherein the current block is encoded using Cross-element prediction (CCP). The encoded syntax data for one or more syntax associated with the transform codec or residual codec applied to the current block is parsed in step 3220. In step 3230, the encoded grammar data is decoded using one or more context models to generate the one or more grammars based on the information related to the CCP. In step 3240, the current block is decoded using CCP, wherein decoding comprises transform coding of the current block or residual coding of the current block using the one or more syntax.

FIG. 33 illustrates a flow diagram of an exemplary video coding system that encodes syntax data through one or more context models using information related to cross-element prediction in accordance with an embodiment of the invention. According to the decoder-side method, input data associated with a current block is received in step 3310, including a first color block and a second color block. In step 3320, the current block is encoded using Cross-element prediction (CCP for short), wherein the encoding process of the current block includes transform coding of the current block to generate transform data or residual coding of the current block to generate residual data. In step 3330, one or more grammars associated with the transform codec or the residual codec are encoded using one or more context models to generate encoded grammar data based on the information associated with the CCP. The encoded syntax data is signaled in step 3340.

Fig. 34 illustrates a flow chart of an exemplary video codec system that encodes or decodes a current block using information including a chroma syntax corresponding to one or more codec block flags according to an embodiment of the present invention. Input data associated with the current block is received at step 3410, including a luma block and two or more chroma blocks, wherein the input data includes pixel data to be encoded at an encoder side or encoded data associated with the current block for decoding at a decoder side, and the current block is intra-coded using Cross-element prediction (CCP-Component Prediction for short). In step 3420, the current block is encoded or decoded using the CCP, wherein the encoding or decoding usage information includes a chroma syntax corresponding to one or more codec block flags including a root chroma codec block flag.

FIG. 35 illustrates a flow chart of an exemplary video codec system that uses merge flags to indicate whether a current block inherits one or more cross-element model parameters from inherited CCP candidates, according to one embodiment of the present invention. According to the method, input data associated with a current block is received in step 3510, comprising one luma block and two or more chroma blocks, wherein the input data comprises pixel data to be encoded at an encoder side or encoded data associated with the current block for decoding at a decoder side, and the current block is intra-coded using Cross-element prediction (CCP). A merge candidate list is generated for the current block in step 3520, wherein the merge candidate list comprises CCP candidates inherited from neighboring blocks. The merge flag is signaled or parsed in step 3530 to indicate whether the current block inherits one or more cross-element model parameters from the inherited CCP candidates. The skip flag is signaled or parsed before the merge flag in step 3540, wherein the merge flag is inferred when the skip flag is true. The current block is encoded or decoded using information including the merge candidate list in step 3550.

The flow chart shown is intended to illustrate one example of video codec according to the present invention. One skilled in the art may modify each step, rearrange steps, split steps, or combine steps to practice the invention without departing from the spirit of the invention. In this disclosure, specific grammars and semantics are used to illustrate examples of implementing embodiments of the present invention. Those skilled in the art may practice the invention by substituting equivalent syntax and semantics without departing from the spirit of the invention.

The above description is intended to enable one of ordinary skill in the art to practice the invention in the context of a particular application and its requirements. Various modifications to the described embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments. Thus, the present invention is not intended to be limited to the particular embodiments shown and described, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein. In the above detailed description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, those skilled in the art will appreciate that the present invention may be practiced.

The english control and rules according to the preceding claim, the following are Spec translations:

Embodiments of the invention, as described above, may be implemented in various hardware, software code, or a combination of both. For example, one embodiment of the invention may be one or more circuits integrated into a video compression chip, or program code integrated into video compression software to perform the processes described herein. An embodiment of the invention may also be program code to be executed on a digital signal Processor (DIGITAL SIGNAL Processor, DSP for short) to perform the processes described herein. The invention may also relate to a number of functions performed by a computer processor, a digital signal processor, a microprocessor, or a field programmable gate array (Field Programmable GATE ARRAY, FPGA for short). These processors may be configured by executing machine-readable software code or firmware code to perform certain tasks embodied in accordance with the invention. The software code or firmware code may be developed in different programming languages and different formats or styles. The software code may also be compiled for different target platforms. However, the different code formats, styles and languages of software code and other methods of configuring code to perform tasks consistent with the invention do not depart from the spirit and scope of the invention.

The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is, therefore, indicated by the appended claims rather than by the foregoing description. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.

Claims

1. A method for decoding a color image using a codec tool, comprising one or more cross-component model related modes (CCM-Related Modes), the method comprising:

Receiving codec data associated with a current block, including a first color block and a second color block, wherein the current block is coded and decoded using Cross-Component Prediction (CCP);

Parsing codec syntax data of one or more syntaxes related to transform coding or residual coding applied to the current block;

Decoding the codec syntax data using one or more context models based on information associated with the CCP to generate the one or more syntaxes; and

The current block is decoded using the CCP, wherein the decoding includes transform coding or residual coding of the current block using the one or more syntaxes.

2. The method of claim 1 , wherein the one or more syntax elements are selected from the group consisting of root_cbf, cb_cbf, cr_cbf, transform_skip_flag, lfnst_idx, mts_idx, tu_joint_cbcr_residual_flag, last_significant_position, cu_qp_delta_abs, cu_qp_delta_sign_flag, cu_chroma_qp_offset_flag, cu_chroma_qp_offset_idx, or a combination thereof.

3 . The method of claim 1 , wherein the one or more context models correspond to a plurality of context models, and selection among the plurality of context models depends on a CCP type of the current block.

4 . The method of claim 1 , wherein the one or more context models correspond to a plurality of context models, and selection among the plurality of context models depends on prediction modes of neighboring blocks of the current block.

The method of claim 4 , wherein the prediction mode of the neighboring block corresponds to a CCP mode.

6. The method of claim 1, wherein the one or more context models correspond to a plurality of context models, and selection between the plurality of context models depends on a number of neighboring blocks encoded or decoded in CCP mode.

7. The method of claim 1, wherein the one or more context models correspond to a CCP single model, a CCP multi-model, a convolutional cross-component model (CCCM), a gradient linear model (GLM), or a combination thereof.

8. An apparatus for video decoding, the apparatus comprising one or more electronic devices or processors arranged to:

The current block is decoded using the CCP, wherein a decoding process of the current block includes transform coding or residual coding of the current block using the one or more syntaxes.

9. A method for encoding a color image using a codec tool comprising one or more Cross Component Models Related Modes, the method comprising:

receiving input data associated with a current block, including a first color block and a second color block;

encoding the current block using cross-component prediction (CCP), wherein the encoding includes transform coding the current block to generate transform data or residual coding the current block to generate residual data; and

Encoding one or more syntaxes related to the transform codec or the residual codec using one or more context models according to the information related to the CCP to generate codec syntax data; and

The signal transmits the codec syntax data.

10. An apparatus for video encoding, the apparatus comprising one or more electronic devices or processors arranged to:

encoding the current block using cross-component prediction (CCP), wherein the encoding process of the current block includes transform coding (transform coding) of the current block to generate transform data or residual coding (residual coding) of the current block to generate residual data; and

The signal transmits the codec syntax data.

11. A method for encoding and decoding a color image using a codec tool comprising one or more cross component model related modes, the method comprising:

receiving input data associated with a current block, including a luma block and two or more chroma blocks, wherein the input data includes pixel data to be encoded at an encoder or encoded data associated with the current block for decoding at a decoder, and the current block is intra-coded and decoded using Cross-Component Prediction (CCP); and

The current block is encoded or decoded using the CCP, wherein the encoding or decoding uses information including a chroma syntax corresponding to one or more coded block flags, the one or more coded block flags including a root chroma coded block flag.

12. The method of claim 11, wherein CCP candidates inherited from neighboring blocks are included in a merge candidate list (Merge Candidate List) of the current block, and a merge flag indicates whether the current block inherits one or more cross-component model parameters from the inherited CCP candidates.

13. The method of claim 11, wherein the one or more codec block flags further include two or more component codec block flags corresponding to the two or more chroma blocks.

14. The method of claim 13 , wherein if the root chroma codec block flag is false, each element codec block flag corresponding to the two or more chroma blocks is equal to 0; or if the root chroma codec block flag is true, at least one of the two or more element codec block flags corresponding to the two or more chroma blocks is equal to 1.

15. A device for video encoding and decoding, the device comprising one or more electronic devices or processors arranged to:

The current block is encoded or decoded using the CCP, wherein the encoding or decoding process of the current block uses information including syntax of one or more Coded Block Flags, and the one or more Coded Block Flags include a Root Chroma Coded Block Flag.

16. The apparatus of claim 15, wherein the one or more codec block flags further include two or more element codec block flags corresponding to the two or more chroma blocks.

17. The apparatus of claim 16 , wherein if the root chroma codec block flag is false, each of the two or more component codec block flags corresponding to the two or more chroma blocks is equal to 0; or if the root chroma codec block flag is true, at least one of the two or more component codec block flags corresponding to the two or more chroma blocks is equal to 1.

18. A method for encoding and decoding a color image using a codec tool comprising one or more Cross Component Models Related Modes, the method comprising:

receiving input data associated with a current block, including a luma block and two or more chroma blocks, wherein the input data includes pixel data to be encoded at an encoder or encoded data associated with the current block for decoding at a decoder, and the current block is intra-encoded and decoded using Cross-Component Prediction (CCP);

generating a merge candidate list (Merge Candidate List) for the current block, wherein the merge candidate list includes CCP candidates inherited from neighboring blocks;

Signaling or parsing a Merge Flag indicating whether the current block inherits one or more cross-component model parameters from the inherited CCP candidate;

signaling or parsing a Skip Flag before the Merge Flag, wherein the Merge Flag is inferred when the Skip Flag is true; and

The current block is encoded or decoded using information including the merge candidate list.

19. The method of claim 18, wherein when the skip flag is true, an index associated with the inherited CCP candidate is signaled or resolved.

20 . The method of claim 18 , wherein when the merge flag is inferred to be 1, the current block inherits one or more cross-component model parameters (Cross-Component Model Parameters) from the inherited CCP candidate.

21. The method of claim 18, wherein when the merge flag is inferred to be 0, a CCP mode index is signaled or parsed.

22. The method of claim 18, wherein a high-level syntax is signaled or parsed in a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), a Picture Header (PH), or a Slice Header (SH) to indicate whether the skip flag is allowed to be used for the current sequence, picture, or slice.

23. An apparatus for video encoding and decoding, the apparatus comprising one or more electronic devices or processors configured to:

receiving input data associated with a current block, including a luma block and two or more chroma blocks, wherein the input data includes pixel data to be encoded at an encoder or codec data associated with the current block to be decoded at a decoder, and the current block is intra-coded and decoded using Cross-Component Prediction (CCP);

generating a merge candidate list (Merge Candidate List) of the current block, wherein the merge candidate list includes inherited CCP candidates from neighboring blocks;

Signaling or parsing a Merge Flag indicating whether the current block inherits one or more cross-element model parameters from the inheriting CCP candidate;

signaling or parsing a Skip Flag before the Merge Flag, wherein the Merge Flag is inferred when the Skip Flag is true;