WO2020180151A1 - Procédé et appareil de traitement de signal vidéo - Google Patents

Procédé et appareil de traitement de signal vidéo Download PDF

Info

Publication number
WO2020180151A1
WO2020180151A1 PCT/KR2020/003179 KR2020003179W WO2020180151A1 WO 2020180151 A1 WO2020180151 A1 WO 2020180151A1 KR 2020003179 W KR2020003179 W KR 2020003179W WO 2020180151 A1 WO2020180151 A1 WO 2020180151A1
Authority
WO
WIPO (PCT)
Prior art keywords
mode
isp
intra prediction
intra
current block
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/KR2020/003179
Other languages
English (en)
Korean (ko)
Inventor
최장원
허진
유선미
최정아
김승환
이령
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
LG Electronics Inc
Original Assignee
LG Electronics Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by LG Electronics Inc filed Critical LG Electronics Inc
Publication of WO2020180151A1 publication Critical patent/WO2020180151A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/11—Selection of coding mode or of prediction mode among a plurality of spatial predictive coding modes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/119—Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/625—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using discrete cosine transform [DCT]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards

Definitions

  • Embodiments of the present specification relate to a video/video compression coding system, and more particularly, to a method and apparatus for performing intra prediction and transformation in an encoding/decoding process of a video signal.
  • Compression coding refers to a series of signal processing techniques for transmitting digitized information through a communication line or storing it in a format suitable for a storage medium.
  • Media such as video, image, and audio may be subject to compression encoding.
  • a technique for performing compression encoding on an image is referred to as video image compression.
  • Next-generation video content will be characterized by high spatial resolution, high frame rate, and high dimensionality of scene representation. In order to process such content, it will bring a tremendous increase in terms of memory storage, memory access rate, and processing power.
  • Intra prediction is a method of performing prediction by referring to reconstructed samples around a current block to be encoded/decoded in a current picture.
  • addition/extension of an intra prediction mode along with a new intra prediction method is being discussed.
  • transformation between the spatial domain and the frequency domain can be applied to efficiently transmit residual data, and various techniques are being discussed for efficient conversion.
  • An embodiment of the present specification provides a method and apparatus for efficiently signaling information about a prediction mode in a process of applying intra prediction and transformation.
  • Embodiments of the present specification provide a method and apparatus for processing a video signal.
  • a method of encoding a video signal according to an embodiment of the present specification includes the steps of dividing a current block into a plurality of subblocks, an intra sub-partition (ISP) mode flag indicating whether to divide the current block, and the current block.
  • ISP intra sub-partition
  • determining an intra prediction mode of the block and encoding intra prediction information including the intra prediction mode of the current block.
  • the candidate modes include a non-directional mode and at least one directional mode, and the at least one directional mode may be determined among all directional modes based on a division direction of the current block.
  • the non-directional mode includes a planar mode, and when the current block is divided in a horizontal direction, the at least one directional mode includes a horizontal mode, and the current block is divided in a vertical direction. If so, the at least one directional mode may include a vertical mode.
  • the method of encoding a video signal includes the steps of deriving prediction samples of the current block based on the intra prediction mode, and subtracting the prediction samples from original samples of the current block. By doing so, it may further include deriving residual samples of the current block, and applying a horizontal transform and a vertical transform to each of subblocks in the current block including the residual samples.
  • the horizontal transformation kernel for the horizontal transformation and the vertical transformation kernel for the vertical transformation may be determined based on the intra prediction mode.
  • the horizontal transformation kernel for the horizontal transformation and the vertical transformation kernel for the vertical transformation may be determined as DST-7 (Discrete Sine Transform type 7) for all intra prediction modes.
  • a method of decoding a video signal includes ISP information including an intra sub-partition (ISP) mode flag indicating whether to divide a current block and an ISP direction flag indicating a dividing direction of the current block.
  • ISP intra sub-partition
  • a horizontal transform kernel and a vertical transform kernel for transforming a subblock divided from the current block may be set to DST-7 (Discrete Sine Transform type 7) for all intra prediction modes.
  • the determining of the intra prediction mode of the current block includes obtaining ISP intra mode information based on the ISP mode flag, and the intra prediction mode based on the ISP intra mode information and the ISP direction flag. It may include the step of determining the prediction mode.
  • the ISP intra mode information may be parsed when the current block is divided by the ISP and a multi reference line (MRL) is not applied.
  • MTL multi reference line
  • the intra prediction mode is determined as a PLANAR mode, and if the ISP intra mode information corresponds to a second value and the ISP direction flag corresponds to a horizontal direction, The intra prediction mode is determined as a horizontal mode, and when the ISP intra mode information corresponds to a second value and the ISP direction flag corresponds to a vertical direction, the intra prediction mode may be determined as a vertical mode.
  • the intra prediction mode is determined as a PLANAR mode, and if the ISP intra mode information corresponds to a second value and the ISP direction flag corresponds to a horizontal direction, The intra prediction mode is determined as a horizontal mode, and when the ISP intra mode information corresponds to a second value and the ISP direction flag corresponds to a vertical direction, the intra prediction mode is determined as a vertical mode, and the ISP intra mode information is If it corresponds to a third value and the ISP direction flag corresponds to a horizontal direction, the intra prediction mode is determined as a first directional mode located between a horizontal mode and a top-left diagonal mode, and the ISP intra mode information is If it corresponds to a third value and the ISP direction flag corresponds to a vertical direction, the intra prediction mode may be determined as a second directional mode located between a vertical mode and an upper left diagonal mode.
  • the intra prediction mode is determined as a PLANAR mode, and if the ISP intra mode information corresponds to a second value and the ISP direction flag corresponds to a horizontal direction, The intra prediction mode is determined as a horizontal mode, and when the ISP intra mode information corresponds to a second value and the ISP direction flag corresponds to a vertical direction, the intra prediction mode is determined as a vertical mode, and the ISP intra mode information is If it corresponds to a third value and the ISP direction flag corresponds to a horizontal direction, the intra prediction mode is determined as a first directional mode located between a horizontal mode and a top-left diagonal mode, and the ISP intra mode information is If it corresponds to a third value and the ISP direction flag corresponds to the vertical direction, the intra prediction mode is determined as a second directional mode located between the vertical mode and the upper left diagonal mode, and the ISP intra mode information corresponds to the fourth value.
  • the intra prediction mode is determined as a third directional mode located between the horizontal mode and the bottom-left diagonal mode, and the ISP intra mode information corresponds to a fourth value.
  • the intra prediction mode may be determined as a fourth directional mode located between a vertical mode and a top-right diagonal mode.
  • An embodiment of the present specification provides a non-transitory computer-readable medium storing one or more instructions.
  • the one or more instructions executed by one or more processors divide the current block into a plurality of subblocks, and an intra sub-partition (ISP) mode flag indicating whether the current block is divided, and the Encoding ISP information including an ISP direction flag indicating a division direction of the current block, determining candidate modes corresponding to some of all intra modes based on the ISP information, and selecting the current block among the candidate modes.
  • the video signal processing apparatus is controlled to determine an intra prediction mode and encode intra prediction information including the intra prediction mode of the current block.
  • the one or more instructions executed by one or more processors include an intra sub-partition (ISP) mode flag indicating whether to divide the current block and an ISP direction flag indicating the direction of division of the current block.
  • ISP intra sub-partition
  • Obtain ISP information including, divide the current block into a plurality of subblocks based on the ISP information, determine an intra prediction mode of the current block from the intra prediction mode information, and based on the intra prediction mode
  • the video signal processing apparatus is controlled to perform restoration through prediction and transformation of the subblocks.
  • a horizontal transform kernel and a vertical transform kernel for transforming a subblock divided from the current block may be set to DST-7 (Discrete Sine Transform type 7) for all intra prediction modes.
  • the amount of data and processing time required for signaling of the intra prediction mode may be reduced by limiting the intra prediction mode for a block to which the intra sub-partition (ISP) intra prediction is applied.
  • ISP intra sub-partition
  • FIG. 1 shows an example of an image coding system according to an embodiment of the present specification.
  • FIG. 2 shows an example of a schematic block diagram of an encoding apparatus in which encoding of a video signal is performed according to an embodiment of the present specification.
  • FIG. 3 shows an example of a schematic block diagram of a decoding apparatus for decoding an image signal according to an embodiment of the present specification.
  • FIG. 4 shows an example of a content streaming system according to an embodiment of the present specification.
  • FIG 5 shows an example of a video signal processing apparatus according to an embodiment of the present specification.
  • FIG. 6 illustrates an example of a picture division structure according to an embodiment of the present specification.
  • FIG. 7A to 7D illustrate an example of a block division structure according to an embodiment of the present specification.
  • FIG 8 shows an example of a case in which the ternary tree (TT) and the binary tree (BT) are divided according to an embodiment of the present specification.
  • FIG. 9 is a block diagram for entropy encoding according to an embodiment of the present specification.
  • 10A and 10B illustrate an entropy encoding method and related components according to an embodiment of the present specification.
  • 11A and 11B illustrate an entropy decoding method and related components according to an embodiment of the present specification.
  • FIG. 12 is an example of a flowchart for encoding a picture constituting a video signal according to an embodiment of the present specification.
  • FIG. 13 is an example of a flowchart for decoding a picture constituting a video signal according to an embodiment of the present specification.
  • FIG. 14 illustrates an example of a hierarchical structure for a coded image according to an embodiment of the present specification.
  • FIG. 15 illustrates an example of a flowchart for intra prediction in an encoding process of a video signal according to an embodiment of the present specification.
  • 16 illustrates an example of an intra prediction unit in an encoding device according to an embodiment of the present specification.
  • FIG. 17 shows an example of a flowchart for intra prediction in a process of decoding a video signal according to an embodiment of the present specification.
  • FIG. 18 illustrates an example of an intra prediction unit in a decoding apparatus according to an embodiment of the present specification.
  • FIG. 19 shows an example of a flowchart for encoding intra prediction information in a process of encoding a video signal according to an embodiment of the present specification.
  • FIG. 20 illustrates an example of a flowchart for decoding intra prediction information in a process of decoding a video signal according to an embodiment of the present specification.
  • FIG. 21 shows an example of a flowchart for determining an intra prediction mode in a process of decoding a video signal according to an embodiment of the present specification.
  • 22 illustrates an example of 33 directional intra prediction modes according to an embodiment of the present specification.
  • 23 illustrates an example of 65 directional intra prediction modes according to an embodiment of the present specification.
  • MCM most probable mode
  • MRL multi-reference line
  • FIGS. 26A and 26B illustrate an example of block division according to an intra sub-partition (ISP) according to an embodiment of the present specification.
  • ISP intra sub-partition
  • FIG. 27 is a diagram for describing multiple direct mode (DM) for intra prediction of a chroma block according to an embodiment of the present specification.
  • CCLM cross-component linear model
  • 29 illustrates a procedure for decoding intra prediction information according to an embodiment of the present specification.
  • FIG. 30 illustrates an example of a procedure for decoding intra prediction information using a 1-bit flag when an intra prediction mode is limited for a block to which an ISP is applied according to an embodiment of the present specification.
  • 31 illustrates an example of a procedure for determining an intra prediction mode using a 1-bit flag when an intra prediction mode is limited for a block to which an ISP is applied according to an embodiment of the present specification.
  • FIG. 32 shows an example of a procedure for decoding intra prediction information using 1 or 2 bits of information when an intra prediction mode is limited for a block to which an ISP is applied according to an embodiment of the present specification.
  • 33 illustrates an example of a procedure for determining an intra prediction mode using 1 or 2 bits of information when an intra prediction mode is limited for a block to which an ISP is applied according to an embodiment of the present specification.
  • FIG. 34 illustrates another example of a procedure for determining an intra prediction mode using 1 or 2 bits of information when an intra prediction mode is limited for a block to which an ISP is applied according to an embodiment of the present specification.
  • FIG. 35 illustrates an example of a procedure for decoding intra prediction information using 1-3 bits or 2 bits of information when an intra prediction mode is limited for a block to which an ISP is applied according to an embodiment of the present specification.
  • FIG. 36 shows an example of a procedure for decoding intra prediction information using a most probable mode (MPM) flag when an intra prediction mode is limited for a block to which an ISP is applied according to an embodiment of the present specification.
  • MPM most probable mode
  • FIG. 37 shows an example of a video signal encoding procedure according to an embodiment of the present specification.
  • a'processing unit' means a unit in which an encoding/decoding process such as prediction, transformation, and/or quantization is performed.
  • the processing unit may be interpreted as including a unit for a luma component and a unit for a chroma component.
  • the processing unit may correspond to a block, a coding unit (CU), a prediction unit (PU), or a transform unit (TU).
  • the processing unit may be interpreted as a unit for a luminance component or a unit for a color difference component.
  • the processing unit may correspond to a coding tree block (CTB), a coding block (CB), a PU, or a transform block (TB) for a luminance component.
  • the processing unit may correspond to CTB, CB, PU or TB for the color difference component.
  • the present invention is not limited thereto, and the processing unit may be interpreted as including a unit for a luminance component and a unit for a color difference component.
  • processing unit is not necessarily limited to a square block, and may be configured in a polygonal shape having three or more vertices.
  • pixels or pixels are collectively referred to as samples.
  • using a sample may mean using a pixel value or a pixel value.
  • the image coding system may include a source device 10 and a reception device 20.
  • the source device 10 may transmit the encoded video/video information or data in a file or streaming format to the receiving device 20 through a digital storage medium or a network.
  • the source device 10 may include a video source 11, an encoding device 12, and a transmitter 13.
  • the receiving device 20 may include a receiver 21, a decoding device 22 and a renderer 23.
  • the encoding device 12 may be referred to as a video/image encoding device, and the decoding device 22 may be referred to as a video/image decoding device.
  • the transmitter 13 may be included in the encoding device 12.
  • the receiver 21 may be included in the decoding device 22.
  • the renderer 23 may include a display unit, and the display unit may be configured as a separate device or an external component.
  • the video source 11 may acquire a video/image through a process of capturing, synthesizing, or generating a video/image.
  • the video source 11 may include a video/image capturing device and/or a video/image generating device.
  • the video/image capturing device may include, for example, one or more cameras, and a video/image archive including previously captured video/images.
  • Video/image generating devices may include, for example, computers, tablets, and smartphones, and may (electronically) generate video/images.
  • a virtual video/image may be generated through a computer, and in this case, a video/image capturing process may be substituted as a process of generating related data.
  • the encoding device 12 may encode an input video/video.
  • the encoding apparatus 12 may perform a series of procedures such as prediction, transformation, and quantization for compression and coding efficiency.
  • the encoded data (encoded video/video information) may be output in the form of a bitstream.
  • the transmitter 13 may transmit the encoded video/video information or data output in the form of a bitstream to the receiver 21 of the receiving device 20 through a digital storage medium or a network in a file or streaming form.
  • Digital storage media include USB (universal serial bus), SD card (secure digital card), CD (compact disc), DVD (digital versatile disc), Blu-ray disc, HDD (hard disk drive), SSD (solid state drive) may include a variety of storage media.
  • the transmitter 13 may include an element for generating a media file through a predetermined file format, and may include an element for transmission through a broadcast/communication network.
  • the receiver 21 may extract the bitstream and transmit it to the decoding device 22.
  • the decoding device 22 may decode the video/video by performing a series of procedures such as inverse quantization, inverse transformation, and prediction corresponding to the operation of the encoding device 12.
  • the renderer 23 may render the decoded video/image.
  • the rendered video/image may be displayed through the display unit.
  • FIG. 2 shows an example of a schematic block diagram of an encoding apparatus in which encoding of a video signal is performed according to an embodiment of the present specification.
  • the encoding device 100 of FIG. 2 may correspond to the encoding device 12 of FIG. 1.
  • the encoding apparatus 100 includes an image partitioning module 110, a subtraction module 115, a transform module 120, and a quantization module. (130), a de-quantization module (140), an inverse-transform module (150), an addition module (155), a filtering module (160), a memory A (memory) 170, an inter prediction module 180, an intra prediction module 185, and an entropy encoding module 190 may be included.
  • the inter prediction unit 180 and the intra prediction unit 185 may be collectively referred to as a prediction unit. That is, the prediction unit may include an inter prediction unit 180 and an intra prediction unit 185.
  • the transform unit 120, the quantization unit 130, the inverse quantization unit 140, and the inverse transform unit 150 may be included in a residual processing unit.
  • the residual processing unit may further include a subtraction unit 115.
  • the above-described image segmentation unit 110, subtraction unit 115, transform unit 120, quantization unit 130, inverse quantization unit 140, inverse transform unit 150, addition unit 155, filtering unit 160 ), the inter prediction unit 180, the intra prediction unit 185, and the entropy encoding unit 190 may be configured by one hardware component (eg, an encoder or a processor) according to an embodiment.
  • the memory 170 may include a decoded picture buffer (DPB) 175 and may be configured by a digital storage medium.
  • DPB decoded picture buffer
  • the image segmentation unit 110 may divide an input image (or picture, frame) input to the encoding apparatus 100 into one or more processing units.
  • the processing unit may be referred to as a coding unit (CU).
  • the coding unit may be recursively partitioned from a coding tree unit (CTU) or a largest coding unit (LCU) according to a quad-tree binary-tree (QTBT) structure.
  • CTU coding tree unit
  • LCU largest coding unit
  • QTBT quad-tree binary-tree
  • one coding unit may be divided into a plurality of coding units of a deeper depth based on a quad tree structure and/or a binary tree structure.
  • a quad tree structure may be applied first and a binary tree structure may be applied later.
  • the binary tree structure may be applied first.
  • a coding procedure according to an embodiment of the present specification may be performed based on a final coding unit that is no longer divided.
  • the maximum coding unit may be directly used as the final coding unit based on coding efficiency according to image characteristics.
  • the coding unit is recursively divided into coding units of a lower depth, so that a coding unit having an optimal size may be used as a final coding unit.
  • the coding procedure may include procedures such as prediction, transformation, and restoration described below.
  • the processing unit may further include a prediction unit (PU) or a transform unit (TU).
  • the prediction unit and the transform unit may be divided from the above-described coding units, respectively.
  • the prediction unit may be a unit of sample prediction
  • the transform unit may be a unit for inducing a transform coefficient or a unit for inducing a residual signal from the transform coefficient.
  • the term "unit” used in this document may be used interchangeably with terms such as "block” or "area” in some cases.
  • the MxN block may represent a set of samples or transform coefficients consisting of M columns and N rows.
  • a sample may generally represent a pixel or a value of a pixel, may represent a pixel/pixel value of a luminance component, or a pixel/pixel value of a saturation component.
  • a sample may be used as a term corresponding to one picture (or image) as a pixel or pel.
  • the encoding apparatus 100 subtracts a prediction signal (predicted block, prediction sample array) output from the inter prediction unit 180 or the intra prediction unit 185 from the input video signal (original block, original sample array)
  • a signal residual signal, residual block, residual sample array
  • the generated residual signal is transmitted to the conversion unit 120.
  • a unit that subtracts the prediction signal (prediction block, prediction sample array) from the input image signal (original block, original sample array) in the encoding apparatus 100 may be referred to as a subtraction unit 115.
  • the prediction unit may perform prediction on a block to be processed (hereinafter, referred to as a current block) and generate a predicted block including prediction samples for the current block.
  • the prediction module may determine whether intra prediction or inter prediction is applied on a per CU basis.
  • the prediction unit may generate information about prediction, such as prediction mode information, as described later in the description of each prediction mode, and may transmit information about prediction to the entropy encoding unit 190.
  • Information about prediction is encoded by the entropy encoding unit 190 and may be output in the form of a bitstream.
  • the intra prediction unit 185 may predict the current block by referring to samples in the current picture.
  • the referenced samples may be located in the vicinity of the current block or may be located away from each other according to the prediction mode.
  • prediction modes may include a plurality of non-directional modes and a plurality of directional modes.
  • the non-directional mode may include, for example, a DC mode and a planar mode (Planar mode).
  • the directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes according to a detailed degree of the prediction direction. However, this is an example, and more or less directional prediction modes may be used depending on the setting.
  • the intra prediction unit 185 may determine a prediction mode applied to the current block by using the prediction mode applied to the neighboring block.
  • the inter prediction unit 180 may derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on the reference picture.
  • the inter prediction unit 180 may predict motion information in units of blocks, subblocks, or samples based on the correlation between motion information between neighboring blocks and the current block.
  • the motion information may include a motion vector and a reference picture index.
  • the motion information may further include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information.
  • the neighboring block may include a spatial neighboring block existing in the current picture and a temporal neighboring block existing in the reference picture.
  • the reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different.
  • the temporal neighboring block may be referred to as a collocated reference block or a colCU (colCU), and a reference picture including the temporal neighboring block may be referred to as a collocated picture (colPic).
  • the inter prediction unit 180 constructs a motion information candidate list based on motion information of neighboring blocks, and indicates which candidate is used to derive a motion vector and/or a reference picture index of the current block. Can generate information. Inter prediction may be performed based on various prediction modes. For example, when a skip mode and a merge mode are used, the inter prediction unit 180 may use motion information of a neighboring block as motion information of a current block.
  • a residual signal is not transmitted.
  • MVP motion vector prediction
  • MVD motion vector difference
  • the prediction unit may generate a prediction signal (prediction sample) based on various prediction methods described later. For example, the prediction unit may not only apply intra prediction or inter prediction to predict one block, but also apply intra prediction and inter prediction together (simultaneously). This may be referred to as CIIP (combined inter and intra prediction). Also, the prediction unit may perform intra block copy (IBC) to predict a block. IBC may be used for content (eg, game) video/video coding, such as, for example, screen content coding (SCC). Also, IBC may be referred to as CPR (current picture referencing). IBC basically performs prediction in the current picture, but can be performed similarly to inter prediction in that it derives a reference block in the current picture. That is, the IBC may use at least one of the inter prediction techniques described in this document.
  • IBC intra block copy
  • the prediction signal generated by the prediction unit may be used to generate a reconstructed signal or may be used to generate a residual signal.
  • the transform unit 120 may generate transform coefficients by applying a transform technique to the residual signal.
  • the transformation technique uses at least one of DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), KLT (Karhunen-Loeve Transform), GBT (Graph-Based Transform), or CNT (Conditionally Non-linear Transform).
  • DCT Discrete Cosine Transform
  • DST Discrete Sine Transform
  • KLT Kerhunen-Loeve Transform
  • GBT Graph-Based Transform
  • CNT Conditionally Non-linear Transform
  • GBT refers to transformation obtained from a graph representing relationship information between pixels.
  • CNT refers to a transformation obtained based on the prediction signal and generating a prediction signal using all previously reconstructed pixels.
  • the conversion process may be applied to a pixel block having the
  • the quantization unit 130 quantizes the transform coefficients and transmits the quantized transform coefficients to the entropy encoding unit 190.
  • the entropy encoding unit 190 may encode a quantized signal (information on quantized transform coefficients) and output it as a bitstream. Information about the quantized transform coefficients may be referred to as residual information.
  • the quantization unit 130 may rearrange the quantized transform coefficients in a block form into a one-dimensional vector form based on a coefficient scan order, and quantize the quantized transform coefficients based on the characteristics of the quantized transform coefficients in a one-dimensional vector form. It is also possible to generate information about transform coefficients.
  • the entropy encoding unit 190 may perform various encoding techniques such as exponential Golomb, context-adaptive variable length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC).
  • the entropy encoding unit 190 may encode information necessary for video/image restoration (eg, values of syntax elements) in addition to quantized transform coefficients together or separately.
  • the encoded information (eg, video/video information) may be transmitted or stored in a bitstream format in units of network abstraction layer (NAL) units.
  • the video/video information may further include information on various parameter sets, such as adaptation parameter set (APS), picture parameter set (PPS), sequence parameter set (SPS), or video parameter set (VPS).
  • Signaled/transmitted information and/or syntax elements described later in this document may be encoded through the above-described encoding procedure and included in the bitstream.
  • the bitstream may be transmitted over a network or may be stored in a digital storage medium.
  • the network may include a broadcasting network and/or a communication network
  • the digital storage medium may include a storage medium such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD.
  • a transmission unit (not shown) for transmitting and/or a storage unit (not shown) for storing may be configured as internal/external elements of the encoding apparatus 100, or the transmission unit It may be a component of the entropy encoding unit 190.
  • the quantized transform coefficients output from the quantization unit 130 may be used to generate a reconstructed signal.
  • a residual signal may be restored by applying inverse quantization and inverse transform through the inverse quantization unit 140 and the inverse transform unit 150 in the loop for the quantized transform coefficients.
  • the addition unit 155 adds the reconstructed residual signal to the prediction signal output from the inter prediction unit 180 or the intra prediction unit 185 to obtain a reconstructed signal (a reconstructed picture, a reconstructed block, and a reconstructed sample array). Can be generated.
  • the predicted block may be used as a reconstructed block.
  • the addition unit 155 may be referred to as a restoration unit or a restoration block generation unit.
  • the generated reconstructed signal may be used for intra prediction of the next processing target block in the current picture, and may be used for inter prediction of the next picture through filtering as described later.
  • the filtering unit 160 may improve subjective/objective image quality by applying filtering to the reconstructed signal.
  • the filtering unit 160 may apply various filtering methods to the reconstructed picture to generate a modified reconstructed picture, and may transmit the modified reconstructed picture to the DPB 175 of the memory 170.
  • Various filtering methods may include, for example, deblocking filtering, sample adaptive offset (SAO), adaptive loop filter (ALF), and bilateral filter.
  • the filtering unit 160 may generate filtering information and transmit the filtering information to the entropy encoding unit 190 as described later in the description of each filtering method.
  • the filtering information may be output in the form of a bitstream through entropy encoding in the entropy encoding unit 190.
  • the modified reconstructed picture transmitted to the DPB 175 may be used as a reference picture in the inter prediction unit 180.
  • the encoding apparatus 100 may avoid prediction mismatch between the encoding apparatus 100 and the decoding apparatus 200 by using the modified reconstructed picture, and may improve encoding efficiency.
  • the DPB 175 may store the modified reconstructed picture for use as a reference picture in the inter prediction unit 180.
  • the stored motion information may be transmitted to the inter prediction unit 180 for use as motion information of a spatial neighboring block or motion information of a temporal neighboring block.
  • the memory 170 may store reconstructed samples of reconstructed blocks in the current picture, and transfer information on the reconstructed samples to the intra prediction unit 185.
  • FIG. 3 shows an example of a schematic block diagram of a decoding apparatus for decoding an image signal according to an embodiment of the present specification.
  • the decoding device 200 of FIG. 3 may correspond to the decoding device 22 of FIG. 1.
  • the decoding apparatus 200 includes an entropy decoding module 210, a de-quantization module 220, an inverse transform module 230, and an adder. (addition module) 235, filtering module 240, memory 250, inter prediction module 260, and intra prediction module 265 may be included. have.
  • the inter prediction unit 260 and the intra prediction unit 265 may be collectively referred to as a prediction module. That is, the prediction unit may include an inter prediction unit 180 and an intra prediction unit 185.
  • the inverse quantization unit 220 and the inverse transform unit 230 may be collectively referred to as a residual processing module. That is, the residual processing unit may include an inverse quantization unit 220 and an inverse transform unit 230.
  • the entropy decoding unit 210, the inverse quantization unit 220, the inverse transform unit 230, the addition unit 235, the filtering unit 240, the inter prediction unit 260, and the intra prediction unit 265 are implemented. It may be configured by one hardware component (eg, a decoder or a processor) according to an example. Also, the memory 250 may include the DPB 255, and may be configured by one hardware component (eg, a memory or a digital storage medium) according to an embodiment.
  • the decoding apparatus 200 may reconstruct an image in response to a process in which the video/image information is processed by the encoding apparatus 100 of FIG. 2.
  • the decoding apparatus 200 may perform decoding using a processing unit applied by the encoding apparatus 100.
  • the processing unit may be, for example, a coding unit, and the coding unit may be divided from a coding tree unit or a maximum coding unit according to a quad tree structure and/or a binary tree structure.
  • the reconstructed image signal decoded and output through the decoding device 200 may be reproduced through the playback device.
  • the decoding apparatus 200 may receive a signal output from the encoding apparatus 100 of FIG. 2 in the form of a bitstream, and the received signal may be decoded through the entropy decoding unit 210.
  • the entropy decoding unit 210 may parse the bitstream to derive information (eg, video/video information) necessary for image restoration (or picture restoration).
  • the video/video information may further include information on various parameter sets, such as adaptation parameter set (APS), picture parameter set (PPS), sequence parameter set (SPS), or video parameter set (VPS).
  • APS adaptation parameter set
  • PPS picture parameter set
  • SPS sequence parameter set
  • VPS video parameter set
  • the decoding apparatus may decode a picture based on information on a parameter set.
  • Signaled/received information and/or syntax elements described later in this document may be decoded through a decoding procedure and obtained from a bitstream.
  • the entropy decoding unit 210 acquires information in the bitstream using a coding technique such as exponential Golomb coding, CAVLC, or CABAC, and a value of a syntax element required for image restoration, and a quantized value of a transform coefficient for a residual. Can be printed.
  • a bin corresponding to each syntax element is received in a bitstream, and information about the syntax element to be decoded and decoding information of a block to be decoded and a neighbor or a symbol/bin decoded in a previous step
  • the symbol corresponding to the value of each syntax element is determined by determining the context model using the information of, and performing arithmetic decoding of the bin by predicting the probability of occurrence of the bin according to the determined context model.
  • the CABAC entropy decoding method may update the context model using information of the decoded symbol/bin for the context model of the next symbol/bin after the context model is determined.
  • the entropy decoding unit 210 Among the information decoded by the entropy decoding unit 210, information on prediction is provided to the prediction unit (inter prediction unit 260 and intra prediction unit 265), and the register on which entropy decoding is performed by the entropy decoding unit 210 Dual values, that is, quantized transform coefficients and related parameter information may be input to the inverse quantization unit 220. In addition, information about filtering among information decoded by the entropy decoding unit 210 may be provided to the filtering unit 240. Meanwhile, a receiving unit (not shown) for receiving a signal output from the encoding device 100 may be further configured as an inner/outer element of the decoding device 200, or the receiving unit may be a component of the entropy decoding unit 210. May be.
  • the decoding apparatus 200 may be referred to as a video/video/picture decoding apparatus.
  • the decoding apparatus 200 may be divided into an information decoder (video/video/picture information decoder) and a sample decoder (video/video/picture sample decoder).
  • the information decoder may include an entropy decoding unit 210, and the sample decoder is an inverse quantization unit 220, an inverse transform unit 230, an addition unit 235, a filtering unit 240, a memory 250, and an inter prediction. It may include at least one of the unit 260 and the intra prediction unit 265.
  • the inverse quantization unit 220 may output transform coefficients through inverse quantization of the quantized transform coefficients.
  • the inverse quantization unit 220 may rearrange the quantized transform coefficients into a two-dimensional block shape. In this case, the reordering may be performed based on the coefficient scan order performed by the encoding apparatus 100.
  • the inverse quantization unit 220 may perform inverse quantization on quantized transform coefficients by using a quantization parameter (eg, quantization step size information) and obtain transform coefficients.
  • a quantization parameter eg, quantization step size information
  • the inverse transform unit 230 inversely transforms the transform coefficients to obtain a residual signal (residual block, residual sample array).
  • the prediction unit may perform prediction on the current block and generate a predicted block including prediction samples for the current block.
  • the prediction unit may determine whether intra prediction or inter prediction is applied to the current block based on information on prediction output from the entropy decoding unit 210, and may determine a specific intra/inter prediction mode.
  • the prediction unit may generate a prediction signal (prediction sample) based on various prediction methods described later. For example, the prediction unit may not only apply intra prediction or inter prediction to predict one block, but also apply intra prediction and inter prediction together (simultaneously). This may be referred to as CIIP (combined inter and intra prediction). Also, the prediction unit may perform intra block copy (IBC) to predict a block. IBC may be used for content (eg, game) video/video coding, such as, for example, screen content coding (SCC). Also, IBC may be referred to as CPR (current picture referencing). IBC basically performs prediction in the current picture, but can be performed similarly to inter prediction in that it derives a reference block in the current picture. That is, the IBC may use at least one of the inter prediction techniques described in this document.
  • IBC intra block copy
  • the intra prediction unit 265 may predict the current block by referring to samples in the current picture.
  • the referenced samples may be located near the current block or may be spaced apart according to the prediction mode.
  • prediction modes may include a plurality of non-directional modes and a plurality of directional modes.
  • the intra prediction unit 265 may determine a prediction mode applied to the current block by using the prediction mode applied to the neighboring block.
  • the inter prediction unit 260 may derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on the reference picture.
  • motion information may be predicted in units of blocks, subblocks, or samples based on correlation between motion information between neighboring blocks and current blocks.
  • the motion information may include a motion vector and a reference picture index.
  • the motion information may further include inter prediction direction (L0 prediction, L1 prediction, Bi prediction) information.
  • the neighboring block may include a spatial neighboring block existing in the current picture and a temporal neighboring block existing in the reference picture.
  • the inter prediction unit 260 may construct a motion information candidate list based on neighboring blocks, and derive a motion vector and/or a reference picture index of the current block based on the received candidate selection information.
  • Inter prediction may be performed based on various prediction modes, and information on prediction may include information indicating a mode of inter prediction for a current block.
  • the addition unit 235 is reconstructed by adding the obtained residual signal to the prediction signal (predicted block, prediction sample array) output from the prediction unit (including the inter prediction unit 260 and/or the intra prediction unit 265). Signals (restored pictures, reconstructed blocks, reconstructed sample arrays) can be generated. When there is no residual for a block to be processed, such as when the skip mode is applied, the predicted block may be used as a reconstructed block.
  • the addition unit 235 may be referred to as a restoration unit or a restoration block generation unit.
  • the generated reconstructed signal may be used for intra prediction of the next processing target block in the current picture, and may be used for inter prediction of the next picture through filtering as described later.
  • the filtering unit 240 may improve subjective/objective image quality by applying filtering to the reconstructed signal. For example, the filtering unit 240 may apply various filtering methods to the reconstructed picture to generate a modified reconstructed picture, and may transmit the modified reconstructed picture to the DPB 255 of the memory 250 .
  • Various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter.
  • the modified reconstructed picture delivered to the DPB 255 of the memory 250 may be used as a reference picture by the inter prediction unit 260.
  • the memory 250 may store motion information of a block from which motion information in a current picture is derived (or decoded) and/or motion information of blocks in a picture that have already been reconstructed.
  • the stored motion information may be transmitted to the inter prediction unit 260 to be used as motion information of a spatial neighboring block or motion information of a temporal neighboring block.
  • the memory 250 may store reconstructed samples of reconstructed blocks in the current picture, and may be transmitted to the intra prediction unit 265.
  • embodiments described in the filtering unit 160, the inter prediction unit 180, and the intra prediction unit 185 of the encoding apparatus 100 are respectively the filtering unit 240 and the inter prediction unit 260 of the decoding apparatus.
  • the intra prediction unit 265 may be applied to be the same or correspond to each other.
  • FIG. 4 shows an example of a content streaming system according to an embodiment of the present specification.
  • Content streaming systems to which the embodiments of the present specification are applied are largely an encoding server 410, a streaming server 420, a web server 430, and a media storage 440. ), a user equipment 450, and a multimedia input device 460.
  • the encoding server 410 generates a bitstream by compressing content input from a multimedia input device 460 such as a smartphone, a camera, or a camcorder into digital data, and transmits the generated bitstream to the streaming server 420.
  • a multimedia input device 460 such as a smartphone, a camera, or a camcorder
  • the encoding server 410 may be omitted.
  • the bitstream may be generated by an encoding method or a bitstream generation method to which an embodiment of the present specification is applied, and the streaming server 420 may temporarily store the bitstream while transmitting or receiving the bitstream.
  • the streaming server 420 transmits multimedia data to the user device 450 based on a user request through the web server 430, and the web server 430 serves as an intermediary that informs the user of what kind of service exists.
  • the web server 430 transmits information on the requested service to the streaming server 420, and the streaming server 420 transmits multimedia data to the user.
  • the content streaming system may include a separate control server, and in this case, the control server serves to control commands/responses between devices in the content streaming system.
  • the streaming server 420 may receive content from the media storage 440 and/or the encoding server 410. For example, when receiving content from the encoding server 410, the streaming server 420 may receive the content in real time. In this case, in order to provide a smooth streaming service, the streaming server 420 may store the bitstream for a predetermined time.
  • the user device 450 includes, for example, a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistants (PDA), a portable multimedia player (PMP), a navigation system, and a slate PC ( slate PC), tablet PC, ultrabook, wearable device, e.g., smartwatch, smart glass, head mounted display (HMD)), It can include digital TV, desktop computer, and digital signage.
  • PDA personal digital assistants
  • PMP portable multimedia player
  • HMD head mounted display
  • Each server in the content streaming system may be operated as a distributed server, and in this case, data received from each server may be distributedly processed.
  • FIG. 5 shows an example of a video signal processing apparatus according to an embodiment of the present specification.
  • the video signal processing apparatus of FIG. 5 may correspond to the encoding apparatus 100 of FIG. 1 or the decoding apparatus 200 of FIG. 2.
  • the video signal processing apparatus 500 for processing a video signal includes a memory 520 for storing a video signal, and a processor 510 for processing a video signal while being combined with the memory 520.
  • the processor 510 may be configured with at least one processing circuit for processing a video signal, and may process a video signal by executing instructions for encoding/decoding a video signal. That is, the processor 510 may encode original video data or decode an encoded video signal by executing encoding/decoding methods described below.
  • the processor 510 may be composed of one or more processors corresponding to each of the modules of FIG. 2 or 3.
  • the memory 520 may correspond to the memory 170 of FIG. 2 or the memory 250 of FIG. 3.
  • the video/image coding method according to the present specification may be performed based on a split structure described later.
  • Procedures such as prediction, residual processing (e.g., (inverse) transformation, (inverse) quantization), syntax element coding, and filtering, which will be described later, are CTU (coding tree unit) derived based on the load structure, CU (and/ Alternatively, it may be performed based on TU, PU).
  • the block division procedure may be performed by the video division unit 110 of the encoding apparatus 100 described above, and division-related information is (encoded) processed by the entropy encoding unit 190 and transferred to the decoding apparatus 200 in the form of a bitstream. Can be delivered.
  • the entropy decoding unit 210 of the decoding apparatus 200 derives the block division structure of the current block based on the division-related information obtained from the bitstream, and based on this, a series of procedures (e.g., prediction, registration) for decoding an image. Dual processing, block/picture restoration, and in-loop filtering) can be performed.
  • an image processing unit may have a hierarchical structure.
  • One picture may be divided into one or more tiles or tile groups.
  • One tile group may include one or more tiles.
  • One tile may contain more than one CTU.
  • the CTU can be divided into one or more CUs.
  • a tile is a rectangular region of CTUs within a particular tile column and a particular tile row in a picture.
  • the tile group may include an integer number of tiles according to a tile raster scan in a picture.
  • the tile group header may convey information/parameters applicable to the corresponding tile group.
  • the tile group may have one type of tile groups including an intra (I) tile group, a predictive (P) tile group, and a bi-predictive (B) tile group.
  • inter prediction is not used and only intra prediction can be used.
  • a coded original sample value may be signaled without prediction.
  • Intra prediction or inter prediction may be used for blocks in a P tile group, and when inter prediction is used, only uni prediction may be used.
  • intra prediction or inter prediction may be used for blocks in the B tile group, and when inter prediction is used, not only unidirectional prediction but also bi prediction may be used.
  • FIG. 6 illustrates an example of a picture division structure according to an embodiment of the present specification.
  • a picture having 216 (18 by 12) luminance CTUs is divided into 12 tiles and 3 tile groups.
  • the encoder determines the size of a tile/tile group and a maximum and minimum coding unit according to a characteristic (e.g., resolution) of a video image or in consideration of coding efficiency or parallel processing, and provides information about this or information for inducing it. It can be included in the bitstream.
  • a characteristic e.g., resolution
  • the decoder may obtain information indicating whether the tile/tile group of the current picture and the CTU in the tile are divided into a plurality of coding units. Coding efficiency can be increased if such information is not always acquired (decoded) by the decoder, but is acquired (decoded) only under certain conditions.
  • the tile group header may include information/parameters commonly applicable to the tile group.
  • APS APS syntax
  • PPS PPS syntax
  • SPS SPS syntax
  • VPS VPS syntax
  • the high-level syntax of the present specification may include at least one of APS syntax, PPS syntax, SPS syntax, and VPS syntax.
  • information on the division and configuration of a tile/tile group may be configured in an encoder through a higher level syntax and then transmitted to a decoder in the form of a bitstream.
  • FIG. 7A to 7D illustrate an example of a block division structure according to an embodiment of the present specification.
  • 7A is a QT (quadtree, QT)
  • FIG. 7b is a binary tree (BT)
  • FIG. 7c is a ternary tree (TT)
  • FIG. 7d shows an example of block division structures by an asymmetric tree (AT). do.
  • one block may be divided based on a QT division scheme.
  • one subblock divided by the QT division method may be further divided recursively according to the QT division method.
  • a leaf block that is no longer divided by the QT division method may be divided by at least one of BT, TT, or AT.
  • BT can have two types of division, such as horizontal BT (2NxN, 2NxN) and vertical BT (Nx2N, Nx2N).
  • TT may have two types of division, such as horizontal TT (2Nx1/2N, 2NxN, 2Nx1/2N) and vertical TT (1/2Nx2N, Nx2N, 1/2Nx2N).
  • AT is horizontal-up AT (2Nx1/2N, 2Nx3/2N), horizontal-down AT (2Nx3/2N, 2Nx1/2N), vertical-left AT ( It can have four types of division: 1/2Nx2N, 3/2Nx2N), and vertical-right AT (3/2Nx2N, 1/2Nx2N).
  • Each BT, TT, AT can be further divided recursively using BT, TT, AT.
  • Block A may be divided into four sub-blocks (A0, A1, A2, A3) by QT.
  • Sub-block A1 may be divided into four sub-blocks (B0, B1, B2, B3) by QT again.
  • Block B3 that is no longer divided by QT may be divided by vertical BT (C0, C1) or horizontal BT (D0, D1). Like block C0, each sub-block may be further divided recursively in the form of horizontal BT (E0, E1) or vertical BT (F0, F1).
  • Block B3 which is no longer divided by QT may be divided into vertical TT (C0, C1, C2) or horizontal TT (D0, D1, D2). Like block C1, each sub-block may be further divided recursively in the form of horizontal TT (E0, E1, E2) or vertical TT (F0, F1, F2).
  • Block B3 which is no longer divided by QT, can be divided into vertical ATs (C0, C1) or horizontal ATs (D0, D1). Like block C1, each sub-block can be further divided recursively in the form of a horizontal AT (E0, E1) or a vertical TT (F0, F1).
  • BT, TT, and AT division can be applied together in one block.
  • a sub-block divided by BT may be divided by TT or AT.
  • sub-blocks divided by TT may be divided by BT or AT.
  • Sub-blocks divided by AT may be divided by BT or TT.
  • each sub-block may be divided by vertical BT.
  • each sub-block may be divided by horizontal BT. In this case, the order of division is different, but the shape of the final division is the same.
  • the order of searching for the block may be variously defined. In general, search is performed from left to right and from top to bottom, and searching for a block means the order of determining whether to divide additional blocks of each divided sub-block, or if the block is no longer divided, each sub It may mean an encoding order of a block, or a search order when a subblock refers to information of another neighboring block.
  • VPDUs virtual pipeline data units
  • VPDUs may be defined as non-overlapping units within one picture.
  • successive VPDUs can be processed simultaneously by multiple pipeline stages.
  • the VPDU size is roughly proportional to the buffer size in most pipeline stages. Therefore, keeping the VDPU size small is important when considering the buffer size from a hardware perspective.
  • the VPDU size can be set equal to the maximum TB size.
  • the VPDU size may be 64x64 (64x64 luminance samples) size.
  • the VPDU size may be changed (increased or decreased) in consideration of the TT and/or BT partition described above.
  • an MxN block may represent a block including samples consisting of M columns and N rows.
  • FIG. 8 shows an example of a case in which the ternary tree (TT) and the binary tree (BT) are divided according to an embodiment of the present specification.
  • TT ternary tree
  • BT binary tree
  • at least one of the following restrictions may be applied as illustrated in FIG. 8.
  • -TT split is not allowed for a CU with either width or height, or both width and height equal to the width or height, or for a CU with both width and height equal to 128. 128).
  • vertical BT is not allowed
  • the encoding apparatus 100 may derive a residual block (residual samples) based on a block (prediction samples) predicted through intra/inter/IBC prediction, and the derived residual samples By applying transform and quantization to, quantized transform coefficients can be derived. Information on the quantized transform coefficients (residual information) may be included in the residual coding syntax, encoded, and then output in the form of a bitstream.
  • the decoding apparatus 200 may obtain information (residual information) on quantized transform coefficients from the bitstream, and decode the quantized transform coefficients to derive the quantized transform coefficients.
  • the decoding apparatus 200 may derive residual samples through inverse quantization/inverse transformation based on the quantized transform coefficients.
  • transform_skip_flag a transform skip flag
  • Transformation/inverse transformation may be performed based on transformation kernel(s).
  • a multiple transform selection (MTS) scheme may be applied.
  • some of a plurality of transform kernel sets may be selected and applied to the current block.
  • the transformation kernel can be referred to in various terms such as transformation matrix and transformation type.
  • the transform kernel set may represent a combination of a vertical transform kernel (vertical transform kernel) and a horizontal transform kernel (horizontal transform kernel).
  • MTS index information (or tu_mts_idx syntax element) may be generated/encoded by the encoding device 100 and signaled to the decoding device 200 to indicate one of the transform kernel sets.
  • a transform kernel set according to the value of MTS index information may be derived as shown in Table 1 below. Table 1 shows the horizontal conversion type (trTypeHor) and the vertical conversion type (trTypeVer) according to the MTS index (tu_mts_idx).
  • trTypeHor and trTypeVer are indexes indicating the transformation kernel, 0 is DCT-2 (Discrete Cosine Transform type 2), 1 is DST-7 (Discrete Sine Transform type 7), 2 may respectively correspond to DCT-8 (Discrete Cosine Transform type 8).
  • the transform kernel set may be determined based on, for example, cu_sbt_horizontal_flag and cu_sbt_pos_flag. If cu_sbt_horizontal_flag is 1, the current coding unit is horizontally divided into two transform units. If cu_sbt_horizontal_flag is 0, the current coding unit is vertically divided into two transform units. If cu_sbt_pos_flag is 1, tu_cbf_luma, tu_cbf_cb, and tu_cbf_cr of the first transform unit in the current coding unit do not exist in the bitstream.
  • tu_cbf_luma, tu_cbf_cb and tu_cbf_cr of the second transform unit in the current coding unit do not exist in the bitstream.
  • Table 2 below shows trTypeHor and trTypeVer according to cu_sbt_horizontal_flag and cu_sbt_pos_flag.
  • the transform kernel set may be determined based on the intra prediction mode for the current block as shown in Table 3 below.
  • trTypeHor may indicate a horizontal direction conversion kernel
  • trTypeVer may indicate a vertical direction conversion kernel
  • a trTypeHor/trTypeVer value of 0 may indicate DCT2
  • a trTypeHor/trTypeVer value of 1 may indicate DST7
  • a trTypeHor/trTypeVer value of 2 may indicate DCT8.
  • this is an example, and different values may be mapped to different DCT/DSTs by appointment.
  • Table 4 below exemplarily shows the basis functions for DCT2, DCT8, and DST7 described above.
  • Transformation/inverse transformation may be performed in units of CU or TU. That is, the transform/inverse transform may be applied to residual samples in a CU or residual samples in a TU.
  • the CU size and the TU size may be the same, or a plurality of TUs may exist in the CU region.
  • the CU size may generally indicate the luminance component (sample) CB size.
  • the TU size may generally indicate a luminance component (sample) TB size.
  • the color difference component (sample) CB or TB size is the luminance component (sample) CB or TB depending on the component ratio according to the color format (chroma format, e.g.
  • the TU size can be derived based on maxTbSize. For example, when the CU size is larger than maxTbSize, a plurality of TUs (TB) of maxTbSize may be derived from the CU, and transformation/inverse transformation may be performed in units of TU (TB).
  • maxTbSize may be considered to determine whether to apply various intra prediction types, such as an ISP.
  • the information on maxTbSize may be determined in advance, or may be generated and encoded in an encoding device and signaled to a decoding device.
  • FIG. 9 is a block diagram for entropy encoding according to an embodiment of the present specification.
  • the block diagram of FIG. 9 corresponds to an example of the entropy encoding unit 190 of FIG. 2.
  • some or all of the video/video information may be entropy-encoded by the entropy encoding unit 190, and as described in FIG. 3, some or all of the video/video information is an entropy decoding unit ( 210) can be entropy decoded.
  • the video/video information may be encoded/decoded in units of syntax elements.
  • that information is encoded/decoded may include encoding/decoding by the method described in this paragraph.
  • CABAC 9 shows a block diagram of CABAC for encoding one syntax element.
  • the encoding process of CABAC first converts the input signal into a binary value through binarization when the input signal is a syntax element rather than a binary value. If the input signal is already binary, it is bypassed without going through binarization.
  • each binary number 0 or 1 constituting the binary value is called a bin.
  • the binary string (empty string) after binarization is 110, each of 1, 1, and 0 is referred to as one bin.
  • the bin(s) for one syntax element may represent a value of a corresponding syntax element.
  • the binarized bins are input into a regular coding engine or a bypass coding engine.
  • the regular coding engine allocates a context model that reflects a probability value to the corresponding bin, and codes the corresponding bin based on the assigned context model.
  • the regular coding engine may update the probability model for the corresponding bin after performing coding for each bin. Bins coded in this way are called context-coded bins.
  • the bypass coding engine omits the procedure of estimating the probability for the input bin and the procedure of updating the probability model applied to the corresponding bin after coding. Instead of allocating context, a uniform probability distribution (eg, 50:50) is applied to code the input bins to improve coding speed. Bins coded in this way are called bypass bins.
  • the context model may be allocated and updated for each context-coded (normally coded) bin, and the context model may be indicated based on ctxIdx or ctxInc.
  • ctxIdx may be derived based on ctxInc.
  • a context index (ctxIdx) indicating a context model for each of the regularly coded bins may be derived as a sum of a context index increment (ctxInc) and a context index offset (ctxIdxOffset).
  • ctxInc may be derived differently for each bin.
  • ctxIdxOffset may be represented by the lowest value of ctxIdx.
  • ctxIdx may be referred to as an initial value (initValue) of ctxIdx.
  • ctxIdxOffset is a value generally used to distinguish context models for other syntax elements, and a context model for one syntax element may be identified/derived based on ctxInc.
  • Entropy decoding performs the same process as entropy encoding in reverse order.
  • 10A and 10B illustrate an entropy encoding method and related components according to an embodiment of the present specification.
  • the encoding apparatus 100 performs an entropy coding procedure for image/video information.
  • the image/video information may include partitioning related information, prediction related information (eg, inter/intra prediction classification information, intra prediction mode information, inter prediction mode information), residual information, and in-loop filtering related information, or It can contain various syntax elements for.
  • Entropy coding may be performed in units of syntax elements. Steps S1010 to S1020 of FIG. 10A may be performed by the entropy encoding unit 190 of the encoding apparatus 100 of FIG. 2 described above.
  • the encoding device 100 performs binarization on the target syntax element (S1010).
  • binarization may be based on various binarization methods such as a Truncated Rice binarization process and a fixed-length binarization process, and a binarization method for the target syntax element may be predefined.
  • the binarization procedure may be performed by the binarization unit 192 in the entropy encoding unit 190.
  • the encoding device 100 performs entropy encoding on the target syntax element (S1020).
  • the encoding apparatus 100 may encode an empty string of a target syntax element based on an entropy coding technique such as CABAC or CAVLC based on normal coding (context based) or bypass coding, and the output may be included in a bitstream.
  • the entropy encoding procedure may be performed by the entropy encoding processing unit 193 in the entropy encoding unit 190.
  • the bitstream can be delivered to a decoding device through a (digital) storage medium or a network.
  • 11A and 11B illustrate an entropy decoding method and related components according to an embodiment of the present specification.
  • the decoding apparatus 200 may decode encoded image/video information.
  • the image/video information may include partitioning related information, prediction related information (eg, inter/intra prediction classification information, intra prediction mode information, inter prediction mode information), residual information, and in-loop filtering related information, or It can contain various syntax elements related to it.
  • Entropy coding may be performed in units of syntax elements. Steps S1110 to S1120 may be performed by the entropy decoding unit 210 of the decoding apparatus 200 of FIG. 3 described above.
  • the decoding apparatus 200 performs binarization on the target syntax element (S1110). Binarization can be based on various binarization methods such as the Truncated Rice binarization process and the Fixed-length binarization process, and the binarization method for the target syntax element can be predefined.
  • the decoding apparatus may derive available bin strings (empty string candidates) for available values of a target syntax element through a binarization procedure.
  • the binarization procedure may be performed by the binarization unit 212 in the entropy decoding unit 210.
  • the decoding apparatus 200 performs entropy decoding on the target syntax element (S1220).
  • the decoding apparatus 200 sequentially decodes and parses each bin for a relative syntax element from the input bit(s) in the bitstream, and compares the derived bin string with the available bin strings for the corresponding syntax element. If the derived empty string is the same as one of the available empty strings, a value corresponding to the corresponding empty string is derived as a value of the corresponding syntax element. If not, the next bit in the bitstream is further parsed and the above-described procedure is performed again. Through this process, the information can be signaled using variable length bits without using a start bit or an end bit for specific information (specific syntax element) in the bitstream. Through this, relatively fewer bits can be allocated to a low value, and overall coding efficiency can be improved.
  • the decoding apparatus 200 may decode each bin in the bin string from the bitstream based on an entropy coding technique such as CABAC or CAVLC based on context or bypass.
  • the bitstream may include various information for video/video decoding.
  • the bitstream can be delivered to a decoding device through a (digital) storage medium or a network.
  • pictures constituting the video/video may be encoded/decoded according to a series of decoding orders.
  • a picture order corresponding to an output order of a decoded picture may be set differently from a decoding order, and based on this, not only forward prediction but also backward prediction may be performed during inter prediction.
  • step S1210 may be performed by the prediction units 180 and 185 of the encoding apparatus 100 described in FIG. 2, and step S1220 may be performed by the residual processing units 115, 120, and 130. , S1230 may be performed by the entropy encoding unit 190.
  • Step S1210 may include an inter/intra prediction procedure described in this document
  • step S1220 may include a residual processing procedure described in this document
  • step S1230 includes an information encoding procedure described in this document. can do.
  • the picture encoding procedure is not only a procedure of encoding information for picture restoration (eg, prediction information, residual information, partitioning information) schematically as described in FIG. 2 to output in a bitstream form,
  • a procedure for generating a reconstructed picture for the current picture and a procedure for applying in-loop filtering to the reconstructed picture (optional) may be included.
  • the encoding apparatus 100 may derive (modified) residual samples from the quantized transform coefficients through the inverse quantization unit 140 and the inverse transform unit 150, and predictive samples corresponding to the output of step S1210 and ( A reconstructed picture may be generated based on the modified) residual samples.
  • the reconstructed picture generated in this way may be the same as the reconstructed picture generated by the decoding apparatus 200 described above.
  • a modified reconstructed picture can be generated through an in-loop filtering procedure for the reconstructed picture, which can be stored in the memory 170 (DPB 175), and, as in the case of the decoding device 200, a subsequent picture It can be used as a reference picture in an inter prediction procedure upon encoding of. As described above, in some cases, some or all of the in-loop filtering procedure may be omitted.
  • (in-loop) filtering-related information may be encoded by the entropy encoding unit 190 and output in the form of a bitstream, and the decoding apparatus 200
  • the in-loop filtering procedure may be performed in the same manner as the encoding apparatus 100.
  • noise generated during image/video coding such as blocking artifacts and ringing artifacts may be reduced, and subjective/objective visual quality may be improved.
  • the encoding device 100 and the decoding device 200 can derive the same prediction result, so that the reliability of picture coding is This can be improved and the amount of data transmitted for picture coding can be reduced.
  • Step S1310 may be performed by the entropy decoding unit 210 of the decoding apparatus 200 of FIG. 3, step S1320 may be performed by the prediction units 260 and 265, and step S1330 may be performed by the residual processing unit ( 220, 230), step S1340 may be performed by the addition unit 235, step S1350 may be performed by the filtering unit 240.
  • Step S1310 may include the information decoding procedure described in this document
  • step S1320 may include the inter/intra prediction procedure described in this document
  • step S1330 includes the residual processing procedure described in this document.
  • step S1340 may include the block/picture restoration procedure described in this document
  • step S1350 may include the in-loop filtering procedure described in this document.
  • the picture decoding procedure is a procedure for obtaining image/video information (through decoding) from a bitstream (S1310), a picture restoration procedure (S1320 to S1340), and a reconstructed picture, as described in FIG. It may include an in-loop filtering procedure (S1350) for.
  • the picture restoration procedure is based on prediction samples and residual samples obtained through the process of inter/intra prediction (S1320) and residual processing (S1330, inverse quantization and inverse transformation of a quantized code or coefficient) described in this document. Can be done.
  • a modified reconstructed picture may be generated through an in-loop filtering procedure for a reconstructed picture generated through a picture restoration procedure, and the modified reconstructed picture may be output as a decoded picture, and the decoding apparatus 200 It is stored in the DPB 255 of and can be used as a reference picture in inter prediction train when decoding a picture later.
  • the in-loop filtering procedure may be omitted, and in this case, the reconstructed picture may be output as a decoded picture, stored in the DPB 255 of the decoding device 200, and referenced in the inter prediction train when decoding a subsequent picture. Can be used as a picture.
  • the in-loop filtering procedure S1350 may include a deblocking filtering procedure, a sample adaptive offset (SAO) procedure, an adaptive loop filter (ALF) procedure, and/or a bi-lateral filter procedure as described above. And some or all of them may be omitted.
  • one or some of the deblocking filtering procedure, the SAO procedure, the ALF procedure, and the bilateral filter procedure may be sequentially applied, or all may be sequentially applied.
  • the SAO procedure may be performed.
  • the ALF procedure may be performed. This may be similarly performed in the encoding device 100.
  • a reconstructed block may be generated based on intra prediction/inter prediction for each block, and a reconstructed picture including the reconstructed blocks may be generated.
  • the current picture/slice/tile group is an I picture/slice/tile group
  • blocks included in the current picture/slice/tile group may be reconstructed based only on intra prediction.
  • inter prediction may be applied to some blocks in the current picture/slice/tile group
  • intra prediction may be applied to the remaining some blocks.
  • the color component of a picture may include a luminance component and a chrominance component, and the methods and embodiments proposed in this document may be applied to the luminance component and the chrominance component unless explicitly limited in this document.
  • FIG. 14 illustrates an example of a hierarchical structure for a coded image according to an embodiment of the present specification.
  • the coded image is a video coding layer (VCL) that deals with the decoding process of the image and itself, a subsystem that transmits and stores encoded information, and a network abstraction (NAL) that exists between the VCL and the subsystem and is responsible for network adaptation. layer).
  • VCL video coding layer
  • NAL network abstraction
  • VCL data including video data (tile group data) compressed in the VCL is generated, or a parameter set including information such as PPS (picture parameter set), SPS (sequence parameter set), VPS (video parameter set), or An additionally required SEI (supplemental enhancement information) message may be generated in the process of decoding an image.
  • PPS picture parameter set
  • SPS sequence parameter set
  • VPS video parameter set
  • SEI Supplemental Enhancement information
  • NAL unit data may be added to a raw byte sequence payload (RBSP) generated in VCL to generate a NAL unit.
  • RBSP may refer to tile group data, parameter set, and SEI message generated in the VCL.
  • NAL unit type information specified according to RBSP data included in the corresponding NAL unit may be included.
  • the NAL unit may be divided into a VCL NAL unit and a Non-VCL NAL unit according to the RBSP generated from the VCL.
  • the VCL NAL unit may mean a NAL unit that includes information about an image (tile group data), and the Non-VCL NAL unit is an NAL that includes information (parameter set or SEI message) necessary for decoding an image. It can mean a unit.
  • VCL NAL unit and Non-VCL NAL unit may be transmitted through a network with header information added according to the data standard of the sub-system.
  • the NAL unit may be converted into a data format of a predetermined standard such as an H.266/VVC file format, a real-time transport protocol (RTP), and a transport stream (TS) and then transmitted through various networks.
  • a predetermined standard such as an H.266/VVC file format, a real-time transport protocol (RTP), and a transport stream (TS)
  • the NAL unit type may be specified according to the RBSP data structure included in the corresponding NAL unit, and information on the NAL unit type may be stored in the NAL unit header and signaled.
  • the NAL unit may be largely classified into a VCL NAL unit type and a Non-VCL NAL unit type according to whether or not information on an image (tile group data) is included.
  • the VCL NAL unit type may be classified according to the nature and type of a picture included in the VCL NAL unit, and the non-VCL NAL unit type may be classified according to the type of the parameter set.
  • NAL unit type specified according to the type of a parameter set included in the Non-VCL NAL unit type.
  • NAL unit A type for a NAL unit including APS
  • VPS Video Parameter Set
  • NAL unit a type for a NAL unit including SPS
  • NAL unit A type for a NAL unit including PPS
  • NAL unit types have syntax information for the NAL unit type, and the syntax information may be stored in the NAL unit header and signaled.
  • syntax information may be nal_unit_type, and NAL unit types may be specified by nal_unit_type values.
  • the tile group header may include information/parameters commonly applicable to the tile group.
  • APS APS syntax
  • PPS PPS syntax
  • SPS SPS syntax
  • VPS VPS syntax
  • the higher-level syntax may include at least one of APS syntax, PPS syntax, SPS syntax, and VPS syntax.
  • the image/video information encoded by the encoding device 100 by the decoding device 200 and signaled in the form of a bitstream includes intra-picture partitioning-related information, intra/inter prediction information, residual information, and in-loop filtering information.
  • information included in the APS, information included in the PPS, information included in the SPS, and/or the information included in the VPS may be included.
  • Embodiments of the present specification relate to an intra prediction method, and in the case of a decoder, the video/video decoding method based on intra prediction of FIG. 17 and the intra prediction unit 265 in the decoding apparatus 200 of FIG. 18 may be represented.
  • the encoder may be represented by the intra prediction-based video/spiritual encoding method of FIG. 15 and an intra prediction unit in the encoding apparatus 100 of FIG. 16.
  • the data encoded by FIGS. 15 and 16 may be stored in the form of a bitstream.
  • Intra prediction may mean prediction in which prediction samples for a current block are generated based on reference samples in a picture to which the current block belongs (hereinafter, referred to as a current picture).
  • a current picture When intra prediction is applied to the current block, surrounding reference samples to be used for intra prediction of the current block may be derived.
  • the neighboring reference samples of the current block are a sample adjacent to the left boundary of the current block of size nWxnH and a total of 2xnH samples adjacent to the bottom-left, a sample adjacent to the top boundary of the current block, and A total of 2xnW samples adjacent to the top-right side and one sample adjacent to the top-left side of the current block may be included.
  • the peripheral reference samples of the current block may include upper peripheral samples of a plurality of columns and left peripheral samples of a plurality of rows.
  • the neighboring reference samples of the current block are a total of nH samples adjacent to the right boundary of the current block of size nWxnH, a total of nW samples adjacent to the bottom boundary of the current block, and the lower right ( It may include one sample adjacent to the bottom-right).
  • the decoder can construct surrounding reference samples to be used for prediction by substituting the samples that are not available with the available samples. Further, surrounding reference samples to be used for prediction may be constructed through interpolation of available samples.
  • prediction samples may be derived based on an average or interpolation of neighboring reference samples of the current block, and (ii) prediction among neighboring reference samples of the current block
  • the prediction sample may be derived based on a reference sample existing in a specific (prediction) direction with respect to the sample.
  • it may be referred to as a non-directional mode or a non-angular mode
  • it may be referred to as a directional mode or an angular mode.
  • a prediction sample may be generated.
  • LIP linear interpolation intra prediction
  • chrominance prediction samples may be generated based on luminance samples using a linear model. This case may be referred to as an LM mode.
  • a temporary prediction sample of the current block is derived based on the filtered surrounding reference samples, and at least one reference sample derived according to the intra prediction mode among existing surrounding reference samples, that is, unfiltered surrounding reference samples, and A prediction sample of the current block may be derived through a weighted sum of the temporary prediction samples. This case may be referred to as a position dependent prediction combination (PDPC).
  • PDPC position dependent prediction combination
  • the encoding apparatus 100 selects a reference sample line with the highest prediction accuracy among a plurality of reference sample lines located around the current block, and derives a prediction sample from the corresponding line using a reference sample located in the prediction direction.
  • intra prediction encoding may be performed by indicating (signaling) the used reference sample line to the decoding apparatus 200.
  • This case may be referred to as multi-reference line (MRL) intra prediction or MRL-based intra prediction.
  • MRL multi-reference line
  • the encoder/decoder divides the current block into vertical or horizontal subpartitions and performs intra prediction based on the same intra prediction mode, but may derive and use surrounding reference samples in units of subpartitions.
  • the intra prediction mode for the current block is equally applied to the subpartitions, but the intra prediction performance can be improved in some cases by deriving and using the surrounding reference samples in units of the subpartition.
  • This prediction method may be referred to as an ISP (intra sub-partition) or ISP-based intra prediction.
  • the above-described intra prediction methods may be referred to as an intra prediction type to distinguish them from the existing intra prediction modes.
  • the intra prediction type may be referred to in various terms such as an intra prediction technique or an additional intra prediction mode.
  • the intra prediction type (or additional intra prediction mode) may include at least one of the aforementioned LIP PDPC, MRL, and ISP. Meanwhile, post-processing filtering may be performed on the derived prediction samples as necessary.
  • the intra prediction procedure may include determining an intra prediction mode/type, deriving a neighboring reference sample, and deriving an intra prediction mode/type based prediction sample.
  • a post-processing filtering step may be performed on the derived prediction samples as necessary.
  • FIG. 15 shows an example of a flowchart for intra prediction in a process of encoding a video signal according to an embodiment of the present specification
  • FIG. 16 shows an example of an intra prediction unit in an encoding apparatus according to an embodiment of the present specification.
  • Operation S1510 of FIG. 15 may be performed by the intra prediction unit 185 of the encoding apparatus 100, and operation S1520 may be performed by the residual processing unit of the encoding apparatus 100. Specifically, step S1520 may be performed by the subtraction unit 115 of the encoding apparatus 100.
  • the prediction information may be derived by the intra prediction unit 185 and encoded by the entropy encoding unit 190.
  • the residual information may be derived by the residual processing unit and encoded by the entropy encoding unit 190.
  • the residual information is information about residual samples.
  • the residual information may include information on quantized transform coefficients for residual samples.
  • residual samples may be derived as transform coefficients through the transform unit 120 of the encoding apparatus 100, and transform coefficients may be derived as quantized transform coefficients through the quantization unit 130.
  • Information about the quantized transform coefficients may be encoded by the entropy encoding unit 190 through a residual coding procedure.
  • the encoding apparatus 100 performs intra prediction on the current block (S1510).
  • the encoding apparatus 100 may derive an intra prediction mode/type for the current block, derive neighboring reference samples of the current block, and generate prediction samples in the current block based on the intra prediction mode/type and neighboring reference samples. do.
  • the procedure of determining the intra prediction mode/type, deriving neighboring reference samples, and generating prediction samples may be simultaneously performed, or one procedure may be performed before the other procedure.
  • the intra prediction unit 185 of the encoding apparatus 100 may include an intra prediction mode/type determiner 186, a reference sample derivation unit 187, and a prediction sample derivation unit 188.
  • the intra prediction mode/type determination unit 186 determines an intra prediction mode/type corresponding to the current block, the reference sample derivation unit 187 derives neighboring reference samples of the current block, and the prediction sample derivation unit 188 Predictive samples of the current block can be derived. Meanwhile, although not shown, when a prediction sample filtering procedure described later is performed, the intra prediction unit 185 may further include a prediction sample filter unit (not shown).
  • the encoding apparatus 100 may determine a mode/type applied to the current block from among a plurality of intra prediction modes/types. The encoding apparatus 100 may compare rate-distortion (RD) costs for intra prediction modes/types and determine an optimal intra prediction mode/type for the current block.
  • RD rate-distortion
  • the encoding apparatus 100 may perform a prediction sample filtering procedure.
  • Predictive sample filtering may be referred to as post filtering. Some or all of the prediction samples may be filtered by the prediction sample filtering procedure. In some cases, the prediction sample filtering procedure may be omitted.
  • the encoding apparatus 100 generates residual samples for the current block based on the (filtered) prediction samples (S1520).
  • the encoding apparatus 100 may compare prediction samples from original samples of a current block based on a phase and derive residual samples.
  • the encoding apparatus 100 may encode image information including information on intra prediction (prediction information) and residual information on residual samples (S1530).
  • the prediction information may include intra prediction mode information and intra prediction type information.
  • the encoding apparatus 100 may output the encoded image information in the form of a bitstream.
  • the output bitstream may be delivered to the decoding apparatus 200 through a storage medium or a network.
  • the residual information may include a residual coding syntax to be described later.
  • the encoding apparatus 100 may derive quantized transform coefficients by transforming/quantizing residual samples.
  • the residual information may include information on quantized transform coefficients.
  • the encoding apparatus 100 may generate a reconstructed picture (including reconstructed samples and a reconstructed block). To this end, the encoding apparatus 100 may perform inverse quantization/inverse transformation on the quantized transform coefficients again to derive (modified) residual samples. The reason for performing inverse quantization/inverse transformation after transforming/quantizing the residual samples in this way is to derive residual samples identical to the residual samples derived from the decoding apparatus 200 as described above.
  • the encoding apparatus 100 may generate a reconstructed block including reconstructed samples for the current block based on the prediction samples and (modified) residual samples. A reconstructed picture for the current picture may be generated based on the reconstructed block. As described above, the in-loop filtering procedure for the reconstructed picture may be further applied.
  • Steps S1710 to S1730 may be performed by the intra prediction unit 265 of the decoding apparatus 200, and the prediction information of step S1710 and the residual information of step S1740 are obtained from the bitstream by the entropy decoding unit 210 Can be.
  • the residual processing unit of the decoding apparatus 200 may derive residual samples for the current block based on the residual information.
  • the inverse quantization unit 220 of the residual processing unit derives the transformation coefficients by performing inverse quantization based on the quantized transformation coefficients derived based on the residual information, and the inverse transformation unit 230 of the residual processing unit performs transformation. Residual samples for the current block can be derived by performing inverse transformation on the coefficients. Operation S1750 may be performed by the addition unit 235 or the restoration unit of the decoding apparatus 200.
  • the decoding apparatus 200 may derive an intra prediction mode/type for the current block based on the received prediction information (intra prediction mode/type information) (S1710).
  • the decoding apparatus 200 may derive neighboring reference samples of the current block (S1720).
  • the decoding apparatus 200 generates prediction samples in the current block based on the intra prediction mode/type and surrounding reference samples (S1730).
  • the decoding apparatus 200 may perform a prediction sample filtering procedure. Predictive sample filtering may be referred to as post filtering. Some of the prediction samples or the government may be filtered by the prediction sample filtering procedure. In some cases, the prediction sample filtering procedure may be omitted.
  • the decoding apparatus 200 generates residual samples for the current block based on the received residual information.
  • the decoding apparatus 200 may generate a reconstructed sample for the current block based on the prediction samples and the residual samples, and derive a reconstructed block including the reconstructed samples (S1740).
  • a reconstructed picture for the current picture may be generated based on the reconstructed block.
  • the in-loop filtering procedure for the reconstructed picture may be further applied.
  • the intra prediction unit 265 of the decoding apparatus 200 may include an intra prediction mode/type determination unit 266, a reference sample derivation unit 267, and a prediction sample derivation unit 268, and the intra prediction mode
  • the /type determination unit 266 determines an intra prediction mode/type for the current block based on the intra prediction mode/type information generated and signaled by the intra prediction mode/type determination unit 186 of the encoding apparatus 100 .
  • the reference sample derivation unit 267 may derive neighboring reference samples of the current block, and the prediction sample derivation unit 267 may derive prediction samples of the current block.
  • the intra prediction unit 265 may further include a prediction sample filter unit (not shown).
  • Intra prediction mode information may include flag information (eg, intra_luma_mpm_flag) indicating whether, for example, most probable mode (MPM) is applied to a current block or a remaining mode is applied, MPM
  • the prediction mode information may further include index information (eg, intra_luma_mpm_idx) indicating one of intra prediction mode candidates (MPM candidates).
  • Intra prediction mode candidates may be composed of an MPM candidate list or an MPM list.
  • the intra prediction mode information may further include remaining mode information (eg, intra_luma_mpm_remainder) indicating one of the remaining intra prediction modes except for intra prediction mode candidates (MPM candidates). I can.
  • the decoding apparatus 200 may determine an intra prediction mode of the current block based on the intra prediction mode information.
  • the intra prediction type information may be implemented in various forms.
  • the intra prediction type information may include intra prediction type index information indicating one of the intra prediction types.
  • the intra prediction type information includes reference sample line information indicating whether MRL is applied to the current block and which reference sample line is used when MRL is applied (eg, intra_luma_ref_idx), whether ISP is applied to the current block.
  • ISP flag information or ISP mode flag indicating eg, intra_subpartitions_mode_flag
  • ISP type information indicating the split type of subpartitions when the ISP is applied
  • ISP direction flag eg, intra_subpartitions_split_flag
  • a flag indicating whether to apply PDPC It may include at least one of information or flag information indicating whether to apply linear interpolation prediction (LIP).
  • LIP linear interpolation prediction
  • Intra prediction mode information and/or intra prediction type information may be encoded/decoded through the coding method described in this document.
  • intra prediction mode information and/or intra prediction type information may be encoded/decoded through entropy coding (eg, CABAC, CAVLC) based on a truncated (rice) binary code. have.
  • entropy coding eg, CABAC, CAVLC
  • an intra prediction mode applied to the current block may be determined using an intra prediction mode of a neighboring block.
  • the decoding apparatus 200 receives one of the MPM candidates in the MPM list derived based on the intra prediction mode of the neighboring block (eg, left and/or upper neighboring block) of the current block and additional candidate modes. It may be selected based on the index, or one of the remaining intra prediction modes that are not included in the MPM candidates may be selected based on the remaining intra prediction mode information. For example, whether the intra prediction mode applied to the current block is among MPM candidates or is in the remaining mode may be indicated based on an MPM flag (eg, intra_luma_mpm_flag).
  • an MPM flag eg, intra_luma_mpm_flag
  • the value of the MPM flag of 1 indicates that the intra prediction mode for the current block is in the MPM candidates (MPM list), and the value of the MPM flag is 0 indicates that the intra prediction mode for the current block is in the MPM candidates (MPM list). May indicate that it does not exist.
  • the MPM index may be signaled in the form of a syntax element such as mpm_idx or intra_luma_mpm_idx, and the remaining intra prediction mode information may be signaled in the form of a syntax element such as rem_intra_luma_pred_mode or intra_luma_mpm_remainder.
  • the remaining intra prediction mode information may index the remaining intra prediction modes that are not included in MPM candidates among all intra prediction modes and indicate one of them.
  • the intra prediction mode may be an intra prediction mode for a luminance component (sample).
  • the intra prediction mode information may include at least one of an MPM flag (eg, intra_luma_mpm_flag), an MPM index (eg, mpm_idx or intra_luma_mpm_idx), or remaining intra prediction mode information (eg, rem_intra_luma_pred_mode or intra_luma_mpm_remainder).
  • the MPM list may be referred to in various terms such as an MPM candidate list and candModeList.
  • the intra prediction mode signaling procedure in the encoding apparatus 100 and the intra prediction mode determination procedure in the decoding apparatus 200 may be performed as shown in FIGS. 19 and 20.
  • FIG. 19 shows an example of a flowchart for encoding intra prediction information in a process of encoding a video signal according to an embodiment of the present specification.
  • the encoding apparatus 100 constructs an MPM list for a current block (S1910).
  • the MPM list may include candidate intra prediction modes (MPM candidates) that are likely to be applied to the current block.
  • the MPM list may include intra prediction modes of neighboring blocks, or may further include specific intra prediction modes according to a predetermined method. A detailed MPM list construction method will be described later.
  • the encoding apparatus 100 determines an intra prediction mode of the current block (S1920).
  • the encoding apparatus 100 may perform prediction based on various intra prediction modes, and may determine an optimal intra prediction mode based on rate-distortion optimization (RDO) based thereon.
  • RDO rate-distortion optimization
  • the encoding apparatus 100 may determine the optimal intra prediction mode by using only the MPM candidates configured in the MPM list, or by further using the remaining intra prediction modes as well as the MPM candidates configured in the MPM list. You can decide.
  • the intra prediction type of the current block is a specific type (eg, LIP, MRL, or ISP)
  • the encoding apparatus 100 considers only MPM candidates as intra prediction mode candidates for the current block, The prediction mode can be determined.
  • the intra prediction mode for the current block may be determined only among MPM candidates, and encoding/signaling of the MPM flag may not be performed.
  • the decoding apparatus 200 may infer that the MPM flag is 1 without separately decoding the MPM flag.
  • the encoding apparatus 100 when the intra prediction mode of the current block is one of MPM candidates in the MPM list, the encoding apparatus 100 generates an MPM index (mpm_idx) indicating one of the MPM candidates. If the intra prediction mode of the current block is not in the MPM list, the encoding apparatus 100 remines intra prediction mode information indicating the same mode as the intra prediction mode of the current block among the remaining intra prediction modes not included in the MPM list. Create
  • the encoding apparatus 100 may encode the intra prediction mode information and output it in the form of a bitstream (S1930).
  • the intra prediction mode information may include the above-described MPM flag, MPM index, and/or remaining intra prediction mode information. Since the MPM index and the remaining intra prediction mode information indicate an intra prediction mode for one block in an alternative relationship, they are not signaled at the same time. That is, the MPM flag value 1 and the MPM index are signaled together, or the MPM flag 0 and the remaining intra prediction mode information are signaled together. However, as described above, when a specific intra prediction type is applied to the current block, the MPM flag may not be signaled and only the MPM index may be signaled. That is, in this case, the intra prediction mode information may include only the MPM index.
  • the decoding device 200 may determine an intra prediction mode in response to intra prediction mode information determined and signaled by the encoding device 100.
  • the decoding apparatus 200 acquires intra prediction mode information from a bitstream (S2010).
  • the intra prediction mode information may include at least one of an MPM flag, an MPM index, and a remaining intra prediction mode index.
  • the decoding apparatus 200 constructs an MPM list (S2020).
  • the MPM list is configured in the same way as the MPM list configured in the encoding device 100. That is, the MPM list may include intra prediction modes of neighboring blocks, or may further include specific intra prediction modes according to a predetermined method. Specifically, the MPM list construction method will be described later.
  • step S2020 in FIG. 20 is shown to be performed after the step S2010, this is only an example, and the step S2020 may be performed prior to the step S2010 or may be performed simultaneously.
  • the decoding apparatus 200 determines an intra prediction mode of the current block based on the MPM list and intra prediction mode information (S2030). For example, when the value of the MPM flag is 1, the decoding apparatus 200 may derive a candidate indicated by the MPM index from among MPM candidates in the MPM list as the intra prediction mode of the current block. As another example, when the value of the MPM flag is 0, the decoding apparatus 200 derives an intra prediction mode indicated by the remaining intra prediction information from among the remaining intra prediction modes not included in the MPM list as the intra prediction mode of the current block. can do.
  • the decoding apparatus 200 selects a candidate indicated by the MPM index in the MPM list without checking the MPM flag. It can also be derived as the intra prediction mode of the current block.
  • FIG. 21 shows an example of a flowchart for determining an intra prediction mode in a process of decoding a video signal according to an embodiment of the present specification.
  • FIG. 22 shows an example of 33 directional intra prediction modes according to an embodiment of the present specification
  • FIG. 23 shows an example of 65 directional intra prediction modes according to an embodiment of the present specification.
  • the intra prediction mode may include, for example, two non-directional intra prediction modes and 33 directional intra prediction modes.
  • the non-directional intra prediction modes may include a planar intra prediction mode and a DC intra prediction mode, and the directional intra prediction modes may include 2 to 34 intra prediction modes.
  • the planar intra prediction mode may be referred to as a planner mode, and the DC intra prediction mode may be referred to as a DC mode.
  • the directional intra prediction modes may be extended from 33 to 65 as shown in FIG. 23.
  • the intra prediction mode may include two non-directional intra prediction modes and 65 directional intra prediction modes.
  • the extended directional intra prediction modes can be applied to blocks of all sizes, and can be applied to both a luminance component and a color difference component.
  • the intra prediction mode may include two non-directional intra prediction modes and 129 directional intra prediction modes.
  • Non-directional intra prediction modes may include a planar intra prediction mode and a DC intra prediction mode, and the directional intra prediction modes may include intra prediction modes 2 to 130.
  • the intra prediction mode may further include a cross-component linear model (CCLM) mode for color difference samples in addition to the aforementioned intra prediction modes.
  • CCLM cross-component linear model
  • the CCLM mode can be classified into LT_CCLM, L_CCLM, and T_CCLM depending on whether a left sample is considered, an upper sample is considered, or both are considered to derive an LM parameter, and can be applied only to a color difference component.
  • the intra prediction mode may be indexed, for example, as shown in Table 5 below.
  • the intra prediction type (or additional intra prediction mode) may include at least one of the aforementioned LIP, PDPC, MRL, and ISP.
  • the intra prediction type may be indicated based on intra prediction type information, and the intra prediction type information may be implemented in various forms.
  • the intra prediction type information may include intra prediction type index information indicating one of the intra prediction types.
  • the intra prediction type information includes reference sample line information (e.g., intra_lumma_ref_idx) indicating whether MRL is applied to the current block and in which case the reference sample line is used, and when the ISP is applied, the split type of subpartitions It may include at least one of indicating ISP type information (eg, intra_subpartitions_split_flag), flag information indicating whether PDPC is applied, or flag information indicating whether LIP is applied.
  • reference sample line information e.g., intra_lumma_ref_idx
  • the split type of subpartitions It may include at least one of indicating ISP type information (eg, intra_subpartitions_split_flag), flag information indicating whether PDPC is applied, or flag information indicating whether LIP is applied.
  • the encoder can use the intra prediction mode of the neighboring block to encode the intra prediction mode of the current block.
  • the encoder/decoder may construct an MPM list for the current block.
  • the MPM list may also be referred to as an MPM candidate list.
  • MPM may mean a mode used to improve coding efficiency in consideration of similarity between a current block and a neighboring block when coding an intra prediction mode.
  • a method of configuring an MPM list including three MPM candidates may be used in order to keep the complexity for generating the MPM list low.
  • the MPM list may include 3 MPM candidates.
  • a remanufacturing mode may be used.
  • the remaining mode includes 64 remaining candidates, and information on the remaining intra prediction mode indicating one of the 64 remaining candidates may be signaled.
  • the remaining intra prediction mode information may include a 6-bit syntax element (eg, rem_intra_luma_pred_mode).
  • MCM most probable mode
  • the encoder may check or derive the prediction mode of the neighboring block (S2410). For example, the encoder may determine the prediction mode of the current block based on the prediction mode of the left neighboring block and the prediction mode of the upper neighboring block, and at this time, the prediction mode of the corresponding neighboring block may be determined as MPM. Determining the MPM may be expressed as listing up MPM candidates (or MPM list).
  • the encoder checks whether the prediction mode of the left neighboring block and the prediction mode of the upper neighboring block are the same (S2420).
  • the initial MPM list may be formed by performing a pruning process for intra prediction modes of two adjacent blocks.
  • the first MPM may be set as the prediction mode of the left neighboring block, and the second MPM may be determined as the prediction mode of the upper neighboring block.
  • the MPM may be set to one of an intra planner mode, an intra DC mode, or an intra vertical mode (50th intra prediction mode) (S2430).
  • the two intra prediction modes may be set as MPM, and one of the default intra modes is added to the MPM list after pruning check by MPMs.
  • the default intra modes may include an intra planner mode, an intra DC mode, and/or an intra vertical mode (50th intra prediction mode).
  • the MPM list may be configured as follows.
  • A denotes an intra prediction mode of the left neighboring block
  • B denotes an intra prediction mode of an upper neighboring block
  • 0 denotes an intra planner mode
  • 1 denotes an intra DC mode
  • 50 denotes an intra vertical mode
  • candModeList denotes an MPM list.
  • the encoder may determine whether the prediction mode of the left neighboring block is less than 2 (S2440).
  • the first MPM may be set to an intra planner mode
  • the second MPM may be set to an intra DC mode
  • the third MPM is an intra vertical mode (50th intra prediction mode). It may be set to (S2450).
  • the first MPM may be set as the prediction mode of the left neighboring block
  • the second MPM may be set to (prediction mode of the left neighboring block-1)
  • the third The MPM may be set to (prediction mode of the left neighboring block + 1) (S2460).
  • the MPM list may be configured as described later.
  • candModeList [3] ⁇ 0, 1, 50 ⁇ .
  • candModeList[3] (A, 2 + ((A + 61)% 64 ), 2 + ((A-1)% 64 ) ⁇
  • A denotes an intra prediction mode of the left neighboring block
  • B denotes an intra prediction mode of an upper neighboring block
  • 0 denotes an intra planner mode
  • 1 denotes an intra DC mode
  • 50 denotes an intra vertical mode
  • candModeList denotes an MPM list.
  • an additional pruning process may be performed to remove duplicate modes so that only unique modes can be included.
  • a 6-bit fixed length code (6-bit FLC) may be used for entropy coding of 64 non-MPM modes excluding 3 MPMs. That is, an index representing 64 non-MPM modes may be entropy-coded with a 6-bit fixed length code.
  • the encoder may determine whether an optimal intra prediction mode to be applied to the current block belongs to the previously configured MPM candidate. If the intra prediction mode of the current block belongs to the MPM candidate, the encoder may encode the MPM flag and the MPM index.
  • the MPM flag may indicate whether the intra prediction mode of the current block is derived from a neighboring intra-prediction block (ie, whether the intra prediction mode of the current block belongs to the MPM list).
  • the MPM index may indicate which MPM mode is applied as an intra prediction mode of the current block among MPM candidates. On the other hand, if the intra prediction mode of the current block does not belong to the MPM candidate, the encoder may encode the intra prediction mode of the current block.
  • the encoder/decoder may configure an MPM list including 6 MPMs.
  • the following three types of modes can be considered.
  • two neighboring blocks that is, a left neighboring block A and an upper neighboring block B may be considered.
  • the following initialized default MPM list may be considered.
  • 6-MPMs are updated by performing a pruning process for two neighboring intra modes. If the two neighboring modes are the same and larger than the DC mode (1), 6-MPMs are added to the three default modes (A, planner, DC) and the neighboring mode, and a predefined offset value is added to the modular operation. It will contain three derived modes obtained by. Otherwise, if the two neighboring modes are different from each other, the two neighboring modes are allocated to the first two MPM modes and the remaining four MPM modes are derived from the default modes and neighboring modes. During the 6-MPM list creation process, pruning is used to remove identical modes so that unique modes can be included in the MPM list. For entropy coding of 61 non-MPM modes, a truncated binary code (TBC) is used.
  • TBC truncated binary code
  • neighboring reference samples to be used for intra prediction of the current block may be derived.
  • the neighboring reference samples of the current block are a sample adjacent to the left boundary of the current block of size nWxnH and a total of 2xnH samples adjacent to the bottom-left, a sample adjacent to the top boundary of the current block, and A total of 2xnW samples adjacent to the top-right side and one sample adjacent to the top-left side of the current block may be included.
  • the peripheral reference samples of the current block may include upper peripheral samples of a plurality of columns and left peripheral samples of a plurality of rows.
  • the decoder may construct surrounding reference samples to be used for prediction through interpolation of available samples.
  • the decoder may construct surrounding reference samples to be used for prediction through extrapolation of available samples. Substitute or padding pixels that have not yet been decoded or available with the latest referenceable sample, updating the referenceable sample with the latest sample, starting at the bottom left and until reaching the top right reference sample. can do.
  • MDL Multiple reference line
  • intra prediction only neighboring samples of the upper first line and the neighboring samples of the left first line of the current block are used as reference samples for intra prediction.
  • intra prediction may be performed by using as reference samples neighboring samples located in sample lines separated by 1 to 3 sample distances in the upper and/or left direction of the current block.
  • 25 shows examples of reference samples taking into account multi-reference line (MRL) intra prediction according to an embodiment of the present specification.
  • 25 shows directly neighboring (closest) reference samples at reference line 0 and extended reference samples at reference lines 1 to 3 used in MRL intra prediction.
  • the MRL index (eg, mrl_idx) indicates which line is used for intra prediction for the current block.
  • the MRL index may be signaled through the coding unit syntax as shown in Table 6 below.
  • the MRL index may be signaled in the form of a syntax element intra_luma_ref_idx.
  • intra_luma_ref_idx [x0][y0] represents an intra reference line index IntraLumaRefLineIdx[x0][y0] defined as shown in Table 7 below.
  • intra_luma_ref_idx may be referred to as a (intra) reference sample line index, a reference line index, or mrl_idx. Also, intra_luma_ref_idx may be referred to as intra_luma_ref_line_idx.
  • Table 7 shows the values of IntraLumaRefLineIdx[x0][y0] according to intra_luma_ref_idx[x0][y0].
  • the MRL may be disabled for a block of the first line (row) in the CTU. This is to prevent samples of extended reference lines outside the current CTU line from being used. Also, when the above-described additional reference line is used, the PDPC may not be available.
  • ISP Intra Sub-Partition
  • a block to be currently coded is regarded as one coding unit, and coding is performed without splitting.
  • the ISP method performs intra prediction coding by dividing a block to be currently coded in a horizontal direction or a vertical direction. In this case, a reconstructed block is generated by performing encoding/decoding in units of divided blocks, and the reconstructed block is used as a reference block of the next divided block.
  • blocks are divided as shown in Table 8 below based on the block size.
  • FIGS. 26A and 26B illustrate an example of block division according to an intra sub-partition (ISP) according to an embodiment of the present specification.
  • ISP intra sub-partition
  • the ISP method determines an optimal mode by generating an MPM according to each division method (horizontal division and vertical division) in order to reduce coding complexity and comparing an appropriate prediction mode among prediction modes in the generated MPM list from an RDO perspective.
  • the ISP may not be used. That is, ISP intra prediction can be applied only when the 0th reference line is used (that is, when intra_luma_ref_idx is 0). Also, when the ISP method is used, the PDPC cannot be used.
  • the ISP intra prediction method first encodes/decodes information indicating whether the ISP is applied (e.g., intra_subparitions_mode_flag) in block units, and if the ISP is applied to the current block, information about whether it is horizontal or vertical (intra_subpartitions_split_flag) is encoded. /Decoded. Table 9 below shows an example of the syntax structure reflecting the ISP.
  • the intra prediction mode for the current block is equally applied to each subpartition (subblocks), and the intra prediction performance can be improved by deriving neighboring reference samples in units of subpartitions. That is, when ISP intra prediction is applied, a residual sample processing procedure is performed in units of sub-partitions.
  • intra prediction samples are derived for each subpartition, and reconstructed samples are obtained by adding a residual signal (residual samples) for the corresponding subpartition to this.
  • the residual signal (residual samples) may be derived through an inverse quantization/inverse transform procedure based on residual information (quantized transform coefficient information or residual coding syntax) in the above-described bitstream.
  • prediction samples for the first subpartition are derived and residual samples are derived, and reconstructed samples for the first subpartition may be derived based on this.
  • some of the reconstructed samples in the first subpartition eg, left or upper peripheral reference samples of the second subpartition
  • peripheral reference samples for the second subpartition are peripheral reference samples for the second subpartition.
  • prediction samples for the second subpartition and residual samples may be derived, and reconstructed samples for the second subpartition may be derived based on this.
  • the sub-partition according to the application of the ISP may also be referred to as a sub-block.
  • an interpolation filter for interpolation may be derived through various methods.
  • the interpolation filter may be determined based on at least one of a size of a current block and an intra prediction mode applied to the current block.
  • the interpolation filter may include, for example, a Gaussian filter and a Cubic filter.
  • a filter for mitigating a block boundary may be applied in order to reduce an error between prediction samples of the current block and neighboring samples that have already been reconstructed. For example, whether to apply an intra boundary filter and a filter type may be determined according to the predicted mode and the size of the block.
  • CIIP can be applied to the current CU.
  • a CU is coded in merge mode
  • the CU contains at least 64 luminance samples (the product of the CU width and the CU height is greater than or equal to 64)
  • an additional flag indicates whether the CIIP mode is applied to the current CU. May be signaled to indicate.
  • the CIIP mode may also be referred to as a multi-hypothesis mode or an inter/intra multiple hypothesis mode.
  • Up to four intra prediction modes including DC, PLANAR, HORIZONTAL, and VERTICAL modes can be used to predict the luminance component in the CIIP mode. If the CU shape is very wide (for example, if the width is more than twice the height), the HORIZONTAL mode is not allowed. If the CU shape is very narrow (ie, the height is more than twice the width), the VERTICAL mode is not allowed. For these cases, three intra prediction modes are allowed.
  • the CIIP mode uses three most probable modes (MPMs) for intra prediction.
  • the CIIP MPM candidate list is formed as follows.
  • intraModeA and intraModeB The prediction modes of block A and block B are named intraModeA and intraModeB, respectively, and are derived as follows.
  • intraModeX is set to DC
  • intraModeX is DC or PLANAR
  • intraModeX is DC or PLANAR
  • intra prediction mode of block X is "vertical-like" directional mode (greater than 34).
  • intraModeX is set to VERTICAL
  • intraModeX is set to HORIZONTAL if the intra-prediction mode of block X is a "horizontal-like" directional mode (mode less than or equal to 34).
  • 3 MPMs are set in the order of ⁇ intraModeA, PLANAR, DC ⁇
  • the first two MPMs are set in the order of ⁇ intraModeA, intraModeB ⁇
  • the MPM flag is inferred as 1 without signaling. Otherwise, an MPM flag for indicating whether the CIIP intra prediction mode is one of the CIIP MPM candidate modes is signaled.
  • the MPM flag is 1, an MPM index indicating which of the MPM candidate modes is used in CIIP intra prediction is additionally signaled. Otherwise, if the MPM flag is 0, the intra prediction mode in the MPM candidate list is set to a "missing" mode. For example, if the PLANAR mode is not in the MPM candidate list, PLANAR becomes the missing mode, and the intra prediction mode is set to PLANAR. Since 4 possible intra prediction modes are allowed in CIIP, the MPM candidate list contains only 3 intra prediction candidates. For color difference components, the DM mode is always applied without additional signaling. That is, the same prediction mode as the luminance component is used for the color difference components. The intra prediction mode of the CU coded with CIIP will be stored and used for intra mode coding of the next neighboring CUs.
  • the inter prediction signal P inter in the CIIP mode is derived using the same inter prediction process applied to the general merge mode, and the intra prediction signal P intra is derived using the CIIP intra prediction according to the intra prediction process. Then, the intra and inter prediction signals are combined using a weighted average, where the weight value depends on the intra prediction mode and where the sample is located in the coding block as follows.
  • the same weight is applied to the intra prediction and inter prediction signals.
  • the weights are determined based on the intra prediction mode (horizontal mode or vertical mode in this case) and the sample position in the block.
  • the horizontal prediction mode will be described as an example (weights for the vertical mode are similar, but can be derived in an orthogonal direction).
  • Set the width of the block to W and the height of the block to H.
  • the coding block is initially divided into 4 co-regional parts, each dimension is (W/4)xH. Starting from the part closest to the intra prediction reference samples and ending the part farthest from the intra prediction samples, the weight wt for each of the four regions is set to 6, 5, 3, and 2.
  • the final CIIP prediction signal may be derived as in Equation 5 below.
  • Equation 1 P CIIP is a CIIP prediction sample value, P inter is an inter prediction sample value, P intra is an intra prediction sample value, and wt is a weight.
  • intra prediction When intra prediction is performed on the current block, prediction of a luminance component block (luminance block) of the current block and prediction of a chrominance component block (color difference block) may be performed.
  • the intra prediction mode may be set separately from the corresponding intra prediction mode in the luminance component (luminance block).
  • the intra prediction mode for the color difference component may be indicated based on intra chroma prediction mode information
  • the intra chroma prediction mode information may be indicated in the form of a syntax element intra_chroma_pred_mode
  • the intra chroma prediction mode information is a planar mode.
  • DC mode, vertical mode, horizontal mode, DM (Derived Mode) it may indicate one of the CCLM mode.
  • the planar mode may represent a 0th intra prediction mode, a DC mode 1st intra prediction mode, a vertical mode 26th intra prediction mode, and a horizontal mode 10th intra prediction mode.
  • DM may also be referred to as direct mode.
  • CCLM may be referred to as LM.
  • DM and CCLM are dependent intra prediction modes for predicting a color difference block using information on a luminance block.
  • the DM may represent a mode in which an intra prediction mode identical to an intra prediction mode for a luminance component is applied as an intra prediction mode for a color difference component.
  • the CCLM subsamples the reconstructed samples of the luminance block in the process of generating the predictive samples for the color difference block, and then applies the CCLM parameters ( ⁇ , ⁇ ) to the subsampled samples. Intra prediction mode used as prediction samples of may be indicated.
  • FIG. 27 is a diagram for describing multiple direct mode (DM) for intra prediction of a chroma block according to an embodiment of the present specification.
  • MDM Multiple direct mode
  • the DM mode which is an existing single mode, is extended to a plurality of modes. That is, when configuring the intra prediction mode of the color difference image, a plurality of DM modes may be selected as follows.
  • the CCLM mode can be applied to the current chroma block.
  • the CCLM mode is an intra prediction mode using correlation between a luminance block and a color difference block corresponding to the luminance block.
  • a linear model may be derived based on surrounding samples of the luminance block and the surrounding samples of the chrominance block, and prediction samples of the chrominance block may be derived based on the linear model and reconstructed samples of the luminance block. More specifically, when the CCLM mode is applied to the current chrominance block, parameters for the linear model are derived based on the surrounding samples used for intra prediction of the current chrominance block and the surrounding samples used for intra prediction of the current luminance block. Can be. For example, the linear model may be expressed based on Equation 2 below.
  • pred c (i,j) represents a predicted sample of the (i,j) locus of the current color difference block
  • rec L '(i,j) represents a reconstructed sample of the (i,j) coordinates of the current luminance block
  • rec L '(i,j) may represent a down-sampled reconstructed sample of the current luminance block.
  • CCLM cross-component linear model
  • the parameters ⁇ and ⁇ of the linear model may be derived based on surrounding samples used for intra prediction of the current luminance block and surrounding samples used for intra prediction of the current color difference block.
  • the parameters ⁇ and ⁇ may be derived based on Equations 3 and 4 below.
  • L(n) may represent upper peripheral samples and/or left peripheral samples of the current luminance block
  • C(n) may represent upper peripheral samples and/or left peripheral samples of the current color difference block
  • L(n) may represent down-sampled upper peripheral samples and/or left peripheral samples of the current luminance block
  • N may represent a value that is twice the smaller of the width and height of the current color difference block.
  • the 8 intra prediction modes may include 5 existing intra prediction modes and CCLM mode(s).
  • Table 10 below shows a mapping table for deriving intra-chroma prediction mode when CCLM is not available (when sps_cclm_enabled_flag is 0), and Table 11 shows intra prediction when CCLM is enabled (when sps_cclm_enabled_flag is 1) Represents a mapping table for mode derivation.
  • the intra-chroma prediction mode is an intra luma prediction mode and a signaled intra-chroma prediction mode for a luminance block (e.g., when DUAL_TREE is applied) covering a current block or a center-right lower sample of the chroma block It may be determined based on the value of (intra_chroma_pred_mode) information.
  • the indexes of IntraPredModeC[xCb][yCb] derived from Tables 10 and 11 may correspond to the indexes of the intra prediction modes of Table 5.
  • Embodiments of the present specification relate to intra prediction, and more particularly, to a method of determining an intra prediction mode candidate during ISP intra prediction, and a method of encoding/decoding information on an intra prediction mode. This can be performed in an encoder (encoding device 100) and a decoder (decoding device 200), and a specific method will be described later.
  • the present embodiment provides a method of intuitively determining a prediction mode candidate when determining an intra prediction mode candidate to be used for ISP intra prediction (Intra Sub-Partitions coding mode).
  • ISP intra prediction mode Intra Sub-Partitions coding mode.
  • up to 6 intra prediction mode candidates may be selected through an MPM list generation method and used for ISP intra prediction.
  • Conventional intra prediction uses 67 intra modes to perform intra prediction encoding/decoding, and multi-reference line intra prediction uses 65 intra modes excluding planar mode and DC mode. Predictive encoding/decoding is performed.
  • the ISP intra prediction performs intra prediction encoding/decoding using 66 intra modes excluding a DC mode. Since the three intra predictions (conventional normal intra prediction, multiple reference line intra prediction, and sub-partition intra prediction) all use different numbers of intra modes to perform intra prediction/decoding, MPM list generation for each prediction All are different.
  • the existing intra prediction constructs 6 MPM lists using all 67 intra modes. Since the multiple reference line intra prediction uses 65 intra modes excluding the planar mode and the DC mode, 6 MPM lists are constructed using 65 intra modes excluding the planar mode and the DC mode. Finally, since the sub-partition intra prediction uses 66 intra modes excluding the DC mode, 6 MPM lists are constructed using 66 intra modes excluding the DC mode. In this case, in the case of sub-partition intra prediction, the MPM list is constructed in different ways according to horizontal division and vertical division. In this way, six MPM lists are constructed using different methods for one intra prediction.
  • 29 illustrates a procedure for decoding intra prediction information according to an embodiment of the present specification.
  • the decoder determines to apply ISP intra prediction to the current block, it generates an MPM list for ISP intra prediction, and parses the MPM index. That is, since the decoder must always generate an MPM list when ISP intra prediction is applied, this leads to an increase in decoder complexity. In addition, since the MPM list for ISP intra prediction is generated differently according to the division mode (vertical or horizontal division) of the ISP intra prediction, the complexity of generating the MPM list may increase.
  • an embodiment of the present specification provides a method of configuring a fixed number of intra prediction mode candidates when constructing an intra prediction candidate for ISP intra prediction.
  • An embodiment of the present specification provides a method of using only two fixed intra prediction mode candidates for ISP intra prediction, and provides a binarization method and a multiple transform set (MTS) mapping method for this.
  • MTS multiple transform set
  • an embodiment of the present specification provides a method of using only three fixed intra prediction mode candidates for ISP intra prediction, and provides a binarization method and an MTS mapping method for this.
  • an embodiment of the present specification provides a method of using only four fixed intra prediction mode candidates for ISP intra prediction, and provides a binarization method and an MTS mapping method for this.
  • the intra prediction encoding/decoding structure can be simplified, encoding/decoding complexity can be reduced, and coding loss can be minimized.
  • the present embodiment provides a method of using only two fixed intra prediction mode candidates in ISP intra prediction for a current block. That is, the method provided in this embodiment does not require additional work such as generating an MPM list for separate ISP intra prediction.
  • ISP intra prediction may be performed only by signaling of a 1-bit flag.
  • a candidate mode for ISP intra prediction may be defined as follows.
  • ISP intra mode candidate 1 HORIZONTAL mode (when the block is divided in the horizontal direction), VERTICAL mode (when the block is divided in the vertical direction)
  • PLANAR mode or vertical/horizontal mode is used for ISP intra prediction.
  • the horizontal/vertical mode is automatically determined which mode is applied according to the sub-partition intra division mode of the current block.
  • the planner mode or the horizontal mode is selected as a candidate mode
  • a planar or vertical mode is selected as a candidate mode of the current block.
  • a selected one of the two ISP intra prediction mode candidates is transmitted from the encoder to the decoder through 1-bit flag signaling.
  • the decoder may determine the intra prediction mode of the current block through parsing the 1-bit flag as shown in Table 12 below.
  • the 1-bit flag used in this embodiment may be variously referred to as an ISP default (mode) flag, a planar (mode) flag, or an ISP planar (mode) flag. Hereinafter, for convenience of description, it is referred to as an ISP intra mode flag.
  • the 1-bit ISP intra mode flag indicating the ISP intra prediction mode may be encoded and decoded through context modeling as shown in Table 13 below.
  • a transform set according to the selected intra prediction mode may be set as shown in Tables 14 to 16.
  • DST-7 and DCT-2 represent transform kernels.
  • a vertical transform kernel and a horizontal transform kernel of the current block may be derived based on the ISP intra prediction mode.
  • the transform set may include a vertical transform kernel and a horizontal transform kernel.
  • the decoder may derive a residual sample for the current block by inverse transforming a residual signal (eg, a transform coefficient) for the current block based on the derived vertical transform kernel and the horizontal transform kernel.
  • the encoder may derive a residual signal (eg, transform coefficient) for the current block by transforming the residual sample for the current block based on the derived vertical transform kernel and the horizontal transform kernel.
  • FIG. 30 illustrates an example of a procedure for decoding intra prediction information using a 1-bit flag when an intra prediction mode is limited for a block to which an ISP is applied according to an embodiment of the present specification.
  • the decoder checks whether ISP prediction is applied to the current block.
  • the decoder may parse the 1-bit flag (ISP intra mode flag) described above.
  • 31 illustrates an example of a procedure for determining an intra prediction mode using a 1-bit flag when an intra prediction mode is limited for a block to which an ISP is applied according to an embodiment of the present specification.
  • the decoder may check whether the ISP intra mode is true (1), and when the ISP intra mode is 1, the planner mode may be determined as the intra prediction mode of the current block.
  • the decoder checks the ISP division direction of the current block by using an ISP direction flag indicating the division direction of the current block.
  • the decoder determines the horizontal mode as the intra prediction mode of the current block.
  • the decoder determines the vertical mode as the intra prediction mode of the current block.
  • the encoder performs coding within 6 MPM lists for ISP intra prediction, and then transmits the most efficient intra mode to the decoder.
  • a combination of an intra prediction mode (for full-RD) selected from 67 intra prediction modes through a rough mode decision and one or two intra modes selected from the MPM list for ISP intra prediction Thereafter, ISP intra prediction is again performed using only the intra mode existing in the ISP intra prediction mode.
  • ISP intra prediction configures different MPM lists according to the division mode (horizontal or vertical)
  • the MPM list according to the ISP division mode and the intra mode to perform prediction must be separately stored and managed, which uses memory in the encoder. And sorting for mode determination.
  • the encoder performs a very complex method for ISP intra prediction, which increases the complexity of the encoder structure.
  • the encoder may determine that ISP intra prediction is applied to the current block.
  • the encoder may determine an intra prediction mode for the current block among two intra prediction mode candidates, and among two fixed intra prediction mode candidates. 1 bit of information indicating an intra prediction mode for the current block (ie, an ISP intra mode flag) may be generated and encoded.
  • the present embodiment proposes a method of using only three fixed intra prediction mode candidates in ISP intra prediction for a current block. That is, the method proposed in this embodiment does not require an additional operation such as generating an MPM list for separate ISP intra prediction, and performs sub-partition intra prediction only through signaling of 1- or 2-bit information for the selected ISP intra prediction mode. Can be done.
  • the ISP intra prediction mode can be configured as follows.
  • ISP intra mode candidate 1 HORIZONTAL mode (when the block is divided in the horizontal direction), VERTICAL mode (when the block is divided in the vertical direction)
  • ISP intra mode candidate 2 Mode 25 (when the block is divided in the horizontal direction), Mode 43 (when the block is divided in the vertical direction)
  • Mode25 and Mode 43 denote mode numbers in the intra67 mode configuration as shown in FIG. 23.
  • Mode 25 is a mode indicating a direction between a horizontal direction and an upper left diagonal direction
  • Mode 43 is a mode indicating a direction between a vertical direction and an upper left diagonal direction.
  • the HORIZONTAL/VERTICAL mode and Mode 25/Mode 43 are automatically determined according to the ISP division direction of the current block. That is, when the ISP division direction of the current block is horizontal (when the current block is divided horizontally), the intra prediction mode candidate of the current block is PLANAR or HORIZONTAL mode or Mode 25, and the ISP division direction of the current block is vertical. In the case of (when the current block is divided in the vertical direction), PLANAR, VERTICAL mode, and Mode 43 are selected as intra prediction mode candidates of the current block.
  • the ISP intra prediction mode candidate may be defined as follows.
  • ISP intra mode candidate 1 HORIZONTAL mode
  • ISP intra mode candidate 2 VERTICAL mode
  • a PLANAR mode, a HORIZONTAL mode, or a VERTICAL mode may be used for ISP intra prediction, and each mode may be determined based on 2-bit ISP intra mode information.
  • three intra prediction modes with the highest frequency may be selected and used from 67 intra prediction modes during ISP intra prediction through encoding of various images.
  • a selected mode among the three sub-partition intra prediction mode candidates may be transmitted from the encoder to the decoder through signaling of 1 or 2 bits of information.
  • the decoder may determine a prediction mode (ISP intra prediction mode) of the current block through parsing 1 or 2 bit information as shown in Tables 17 and 18 below.
  • a symbol of 0 or 1 constituting a codeword such as '1', '01', or '00' may be referred to as a bin, which is also applied in the following description.
  • ISP intra mode information of 1 or 2 bits for deriving the prediction mode (ISP intra prediction mode) of the current block may be encoded and decoded through context modeling as shown in Table 19 below.
  • transform sets according to the ISP intra prediction mode may be set as shown in Tables 20 to 22. That is, as shown in Tables 20 to 22, a vertical transform kernel and a horizontal transform kernel for the current block may be derived based on the ISP intra prediction mode.
  • the decoder may derive a residual sample for the current block by inverse transforming a residual signal (eg, a transform coefficient) for the current block based on the derived vertical transform kernel and the horizontal transform kernel.
  • the encoder may derive a residual signal (eg, transform coefficient) for the current block by transforming the residual sample for the current block based on the derived vertical transform kernel and the horizontal transform kernel.
  • FIG. 32 shows an example of a procedure for decoding intra prediction information using 1 or 2 bits of information when an intra prediction mode is limited for a block to which an ISP is applied according to an embodiment of the present specification.
  • the decoder checks whether ISP prediction is applied to the current block.
  • the decoder may parse the above-described 1 or 2 bits of information (ISP intra mode information).
  • 33 illustrates an example of a procedure for determining an intra prediction mode using 1 or 2 bits of information when an intra prediction mode is limited for a block to which an ISP is applied according to an embodiment of the present specification.
  • 33 shows an example of a method for determining an intra prediction mode of a decoder when the ISP intra prediction mode is set as shown in Table 17.
  • the decoder first checks a first bin of ISP intra mode information, and if the first bin is'True' (1), determines a planner mode as an intra prediction mode of the current block. If the first bin of the ISP intra mode information is'false' (0), the decoder checks the second bin of the ISP intra mode information. If the second bin of the ISP intra mode information is'True' (1), the decoder checks the ISP division direction of the current block. When the ISP division direction is horizontal (when the current block is divided in the horizontal direction), the decoder determines the horizontal mode as the intra prediction mode of the current block.
  • the decoder determines the vertical mode as the intra prediction mode of the current block. If the second bin of ISP intra mode information is'False' (0), the decoder checks the ISP division direction of the current block. When the ISP division direction is horizontal (when the current block is divided in the horizontal direction), the decoder determines mode 43 as an intra prediction mode of the current block. When the ISP division direction is vertical (when the current block is divided in the vertical direction), the decoder determines mode 25 as the intra prediction mode of the current block. However, this is only an example, and other modes may be used.
  • FIG. 34 illustrates another example of a procedure for determining an intra prediction mode using 1 or 2 bits of information when an intra prediction mode is limited for a block to which an ISP is applied according to an embodiment of the present specification.
  • 34 shows an example of a method for determining an intra prediction mode of a decoder when the ISP intra prediction mode is set as shown in Table 18.
  • the decoder first checks a first bin of ISP intra mode information, and if the first bin is'True' (1), determines a planner mode as an intra prediction mode of the current block. If the first bin of the ISP intra mode information is'false' (0), the decoder checks the second bin of the ISP intra mode information. If the second bin of ISP intra mode information is'True' (1), the decoder determines the horizontal mode as the intra prediction mode of the current block. If the second bin of the ISP intra mode information is'false' (0), the decoder determines the vertical mode as the intra prediction mode of the current block.
  • the encoder performs coding within 6 MPM lists for ISP intra prediction, and then transmits the most efficient intra mode to the decoder.
  • a combination of an intra prediction mode (for full-RD) selected from 67 intra prediction modes through a rough mode decision and one or two intra modes selected from the MPM list for ISP intra prediction Thereafter, ISP intra prediction is again performed using only the intra mode existing in the ISP intra prediction mode.
  • ISP intra prediction configures different MPM lists according to the division mode (horizontal or vertical)
  • the MPM list according to the ISP division mode and the intra mode to perform prediction must be separately stored and managed, which uses memory in the encoder. And sorting for mode determination.
  • the encoder performs a very complex method for ISP intra prediction, which increases the complexity of the encoder structure.
  • the encoder may determine that ISP intra prediction is applied to the current block.
  • the encoder may determine an intra prediction mode for the current block among three intra prediction mode candidates, and among three fixed intra prediction mode candidates. 1 or 2 bits of information indicating the intra prediction mode for the current block (ie, ISP intra mode information) may be generated and encoded.
  • This embodiment proposes a method of using only four fixed intra prediction mode candidates in ISP intra prediction for a current block. That is, the method proposed in this embodiment does not require an additional operation such as generating an MPM list for separate ISP intra prediction, and only through signaling of 1-3 or 2-bit information for the selected ISP intra prediction mode. You can make predictions.
  • the ISP intra prediction mode can be configured as follows.
  • ISP intra mode candidate 1 HORIZONTAL mode (when the block is divided in the horizontal direction), VERTICAL mode (when the block is divided in the vertical direction)
  • ISP intra mode candidate 2 Mode 25 (when the block is divided in the horizontal direction), Mode 43 (when the block is divided in the vertical direction)
  • ISP intra mode candidate 3 Mode 10 (when the block is divided in the horizontal direction), Mode 60 (when the block is divided in the vertical direction)
  • Mode25, Mode 43, Mode10, and Mode60 denote mode numbers in the intra67 mode configuration as shown in FIG. 23.
  • Mode 25 is a mode indicating a direction between the horizontal direction and the upper left diagonal direction
  • Mode 43 is a mode indicating the direction between the vertical direction and the upper left diagonal direction
  • Mode 10 is the horizontal direction and the lower left diagonal direction.
  • Mode 25 is a mode indicating a direction between the horizontal direction and the upper left diagonal direction
  • Mode 43 is a mode indicating the direction between the vertical direction and the upper left diagonal direction
  • Mode 10 is the horizontal direction and the lower left diagonal direction.
  • It is a mode that indicates the direction between directions
  • Mode60 is a mode that indicates the direction between the vertical direction and the upper right diagonal direction.
  • the HORIZONTAL/VERTICAL mode, Mode 25/Mode 43, and Mode 10/Mode 60 are automatically determined according to the ISP division direction of the current block. That is, if the ISP division direction of the current block is horizontal (when the current block is divided horizontally), PLANAR, HORIZONTAL mode, Mode 25, or Mode 10 is selected as the intra prediction mode candidate of the current block, and the ISP of the current block When the division direction is vertical (when the current block is divided in the vertical direction), PLANAR, VERTICAL mode, Mode 43, or Mode 60 is selected as the intra prediction mode candidate of the current block.
  • the ISP intra prediction mode candidate may be defined as follows.
  • ISP intra mode candidate 1 HORIZONTAL mode
  • ISP intra mode candidate 2 VERTICAL mode
  • ISP intra mode candidate 3 Mode 66
  • a PLANAR mode a HORIZONTAL mode, a VERTICAL mode, or Mode 66 may be used for ISP intra prediction.
  • intra prediction modes with the highest frequency may be selected and used from 67 intra prediction modes.
  • a selected mode among the three sub-partition intra prediction mode candidates may be transmitted from the encoder to the decoder through signaling of 1-3 or 2-bit information.
  • the decoder may determine the prediction mode (ISP intra prediction mode) of the current block through parsing 1-3 or 2-bit information as shown in Tables 23 and 24 below.
  • ISP intra mode information of 1-3 or 2 bits for deriving a prediction mode (ISP intra prediction mode) of the current block may be encoded and decoded through context modeling as shown in Table 25 below.
  • a transform set according to the ISP intra prediction mode may be set as shown in Tables 26 to 28. That is, as illustrated in Tables 26 to 28, a vertical transform kernel and a horizontal transform kernel for the current block may be derived based on the ISP intra prediction mode.
  • the decoder may derive a residual sample for the current block by inverse transforming a residual signal (eg, a transform coefficient) for the current block based on the derived vertical transform kernel and the horizontal transform kernel.
  • the encoder may derive a residual signal (eg, transform coefficient) for the current block by transforming the residual sample for the current block based on the derived vertical transform kernel and the horizontal transform kernel.
  • DST-7 may be used as a transform kernel for horizontal and vertical transforms in the same manner for all intra prediction modes. Since DST-7, which has high compression efficiency for a block to which intra prediction has been applied, is applied to a block partitioned by an ISP, a process for determining a transform kernel may be omitted, and thus transform complexity and transform efficiency may be improved together.
  • 35 shows an example of a procedure for decoding intra prediction information using 1-3 or 2 bits of information when an intra prediction mode is limited for a block to which an ISP is applied according to an embodiment of the present specification.
  • the decoder checks whether ISP prediction is applied to the current block.
  • the decoder may parse 1-3 or 2 bits of information (ISP intra mode information) described above.
  • the encoder performs coding within 6 MPM lists for ISP intra prediction, and then transmits the most efficient intra mode to the decoder.
  • a combination of an intra prediction mode (for full-RD) selected from 67 intra prediction modes through a rough mode decision and one or two intra modes selected from the MPM list for ISP intra prediction Thereafter, ISP intra prediction is again performed using only the intra mode existing in the ISP intra prediction mode.
  • ISP intra prediction configures different MPM lists according to the division mode (horizontal or vertical)
  • the MPM list according to the ISP division mode and the intra mode to perform prediction must be separately stored and managed, which uses memory in the encoder. And sorting for mode determination.
  • the encoder performs a very complex method for ISP intra prediction, which increases the complexity of the encoder structure.
  • the encoder may determine that ISP intra prediction is applied to the current block.
  • the encoder may determine an intra prediction mode for the current block among four intra prediction mode candidates, and among four fixed intra prediction mode candidates.
  • 1-3 or 2 bits of information indicating an intra prediction mode for the current block may be generated and encoded.
  • Embodiment 1 provides a detailed syntax structure to which the method proposed in Embodiment 1 is applied. It is provided in the following syntax format, and each syntax element and related content will be apparent to those skilled in the art.
  • the decoder parses an ISP mode flag (intra_subpartitions_mode_flag) indicating whether or not the current block is divided by the ISP (ISP intra prediction is applied), and when the ISP intra prediction is applied, the current block division direction is determined.
  • the indicated ISP direction flag (intra_subpartitions_split_flag) is parsed.
  • ISP intra prediction If ISP intra prediction is not applied, MPM candidate list construction and mode determination are performed through parsing information such as MPM flags and MPM indexes. However, when ISP intra prediction is applied, the decoder parses the ISP intra mode flag (intra_subpartitions_intra_mode_flag). As described in Embodiment 1, the decoder determines the PLANAR mode and the horizontal/vertical mode as intra prediction mode candidates, and determines the intra prediction mode of the current block based on the ISP intra mode flag and the ISP direction flag.
  • ISP intra mode flag Intra_subpartitions_intra_mode_flag
  • intra_subpartitions_mode_flag[x0][y0] indicates whether the ISP is applied to the current block. If intra_subpartitions_mode_flag[x0][y0] is 1, the current intra coding unit is divided into NumIntraSubPartitions[x0][y0] rectangular transform block subpartitions. If intra_subpartitions_mode_flag[x0][y0] is 0, the current intra coding unit is not divided into NumIntraSubPartitions[x0][y0] rectangular transform block subpartitions.
  • intra_subpartitions_split_flag[x0][y0] indicates whether the intra-subpartition split type is horizontal or vertical.
  • intra_subpartitions_intra_mode_flag[x0][y0] represents an intra prediction mode for the current coding unit to which the intra subpartition is applied.
  • intra_subpartitions_mode_flag[x0][y0] is 1, the intra prediction mode is derived according to the following "Derivation process for luminance intra prediction mode (Embodiment 4)".
  • the input to this process is:
  • the luminance intra mode IntraPredModeY[xCb][yCb] is derived.
  • Table 30 below shows names related to values for the intra prediction mode IntraPredModeY[xCb][yCb].
  • the intra modes INTRA_LT_CCLM, INTRA_L_CCLM, and INTRA_T_CCLM are applied only to color difference components.
  • IntraPredModeY[xCb][yCb] is derived by the following procedures 1 and 2.
  • IntraPredModeY[xCb][yCb] is derived by the following procedures a and b.
  • IntraPredModeY[xCb][yCb] is set to INTRA_PLANAR.
  • IntraPredModeY[xCb][yCb] is derived by the following procedures i, ii.
  • IntraSubPartitionsSplitType is ISP_HOR_SPLIT (horizontal division)
  • IntraPredModeY[xCb][yCb] is set to INTRA_ANGULAR18 (horizontal mode).
  • IntraPredModeY[xCb][yCb] is set to INTRA_ANGULAR50 (vertical mode).
  • IntraPredModeY[xCb][yCb] is determined from the MPM candidate list or other modes excluding the MPM candidate.
  • the binarization process and input parameters of the syntax element reflecting the ISP prediction mode flag (intra_subpartitions_intra_mode_flag) according to the embodiment of the present specification are shown in Table 31, and the bin index (binIdx) of the syntax element may be set as shown in Table 32 below. .
  • the ISP prediction mode flag (intra_subpartitions_intra_mode_flag) may have a fixed length FL of 1 bit.
  • an ISP intra prediction mode is indicated through an existing MPM flag without defining an additional flag.
  • the MPM flag may serve as a 1-bit flag (ISP intra mode flag) for indicating the intra prediction mode of the current block described in the first embodiment. That is, according to the present embodiment, the MPM flag (intra_luma_mpm_idx) is signaled/parsed regardless of the ISP mode flag (intra_subpartitions_mode_flag) value, and when the value of the ISP mode flag is 1, the MPM flag (without the MPM index) is Intra prediction mode can be indicated. Meanwhile, only when the value of the ISP mode flag is 0, the MPM index (mpm_idx) or the MPM remaining index (mpm_remainder) may be additionally signaled/parsed.
  • FIG. 36 shows an example of a procedure for decoding intra prediction information using MPM (most probable mode) information when an intra prediction mode is limited for a block to which an ISP is applied according to an embodiment of the present specification.
  • MPM most probable mode
  • the decoder checks whether the MRL mode is applied, and when the MRL mode is applied, immediately determines an intra prediction mode of a current block through MPM candidate list construction and parsing MPM indexes. When the MRL mode is not applied, the decoder first parses the MPM flag regardless of whether or not ISP intra prediction is applied.
  • the decoder parses the ISP direction flag without constructing the MPM candidate list and parsing the MPM index/remaining index. That is, when the ISP mode flag is 1, the decoder determines the intra prediction mode of the current block based on the MPM flag and the ISP direction flag. For example, the decoder first checks the MPM flag, and when the MPM flag is 1, determines the PLANAR mode as the intra prediction mode of the current block. When the MPM flag is 0, the decoder determines the intra prediction mode according to the ISP division direction of the current block. When the ISP division direction is horizontal, the horizontal mode is used, and when the ISP division direction is vertical, the vertical mode is the intra prediction mode of the current block. Decide as When ISP intra prediction is not applied, the decoder may derive the intra prediction mode of the current block based on the MPM mode or the non-MPM mode.
  • the coding unit syntax structure according to the present embodiment may be expressed as Table 33 below.
  • the decoder parses the MPM flag (intra_luma_mpm_flag). If the ISP mode flag (intra_luma_mpm_flag) is 0 (if ISP intra prediction is not applied), the decoder parses the MPM index or the remaining index according to the MPM flag. If the ISP mode flag (intra_luma_mpm_flag) is 1 (when ISP intra prediction is applied), the decoder determines the intra prediction mode of the current block based on the MPM flag (intra_luma_mpm_flag) and the ISP direction flag (intra_subpartitions_split_flag).
  • intra_subpartitions_mode_flag[x0][y0] indicates whether the ISP is applied to the current block. If intra_subpartitions_mode_flag[x0][y0] is 1, the current intra coding unit is divided into NumIntraSubPartitions[x0][y0] rectangular transform block subpartitions. If intra_subpartitions_mode_flag[x0][y0] is 0, the current intra coding unit is not divided into NumIntraSubPartitions[x0][y0] rectangular transform block subpartitions.
  • intra_luma_mpm_flag[x0][y0] indicates an intra prediction mode for luminance samples.
  • the array indices x0 and y0 indicate a position (x0, y0) of an upper left luminance sample of a coding block with respect to an upper left luminance sample of a picture. If intra_luma_mpm_flag[x0][y0] is 1 or intra_subpartitions_mode_flag[x0][y0] is 1, the intra prediction mode is derived according to "Derivation process for luminance intra prediction mode (Example 5)".
  • the input to this process is:
  • the luminance intra mode IntraPredModeY[xCb][yCb] is derived.
  • names related to values for the intra prediction mode IntraPredModeY[xCb][yCb] are the same as in Table 30 described above.
  • IntraPredModeY[xCb][yCb] is derived by the following procedures 1 and 2.
  • IntraPredModeY[xCb][yCb] is derived by the following procedures a and b.
  • IntraPredModeY[xCb][yCb] is set to INTRA_PLANAR.
  • IntraPredModeY[ xCb ][ yCb] is derived as follows 1) and 2).
  • IntraSubPartitionsSplitType is ISP_HOR_SPLIT
  • IntraPredModeY[xCb][yCb] is set to INTRA_ANGULAR18 (horizontal mode).
  • IntraPredModeY[xCb][yCb] is set to INTRA_ANGULAR50 (vertical mode).
  • IntraPredModeY[xCb][yCb] is determined from the MPM candidate list or other modes excluding the MPM candidate.
  • the bin index (binIdx) of the MPM flag (intra_luma_mpm_flag) may be configured as shown in Table 34 below.
  • the MPM flag (intra_luma_mpm_flag) may be encoded/decoded based on the ISP mode flag (intra_subpartitions_mode_flag).
  • an intra mode parsing method that separately considers PLANAR and DC mode is applied to the method proposed in the first embodiment.
  • the method according to the present embodiment uses a directional mode flag (intra_luma_angular_mode_flag) indicating whether the current intra prediction mode is a directional mode without defining an additional flag. That is, according to the present embodiment, when the value of the ISP flag is 1, the directional mode flag (intra_luma_angular_mode_flag) may serve as a 1-bit flag for indicating the intra prediction mode of the current block described in the first embodiment.
  • the method according to the present embodiment can be expressed in, for example, the following standard document format, and the details indicated are obvious to those skilled in the art. Table 35 below is an example of a coding unit syntax structure according to the present embodiment.
  • the decoder parses the directional mode flag (intra_luma_angular_mode_flag). If the ISP mode flag (intra_luma_mpm_flag) is 0 (if ISP intra prediction is not applied), the decoder parses the MPM index or the remaining index according to the MPM flag. If the ISP mode flag (intra_luma_mpm_flag) is 1 (when ISP intra prediction is applied), the decoder parses the planar flag (intra_luma_planar_flag) indicating whether to use the PLANAR mode.
  • the decoder parses the directional mode flag (intra_luma_angular_mode_flag). If the directional mode flag (intra_luma_angular_mode_flag) is 1, the decoder parses the CIIP direction flag (ciip_intra_luma_vert_flag) indicating the CIIP intra prediction direction. intra_luma_planar_flag) is parsed.
  • intra_luma_angular_mode_flag[x0][y0] is 1, the intra prediction mode for luminance samples is a directional mode (angular mode). If intra_luma_angular_mode_flag[x0][y0] is 0, the intra prediction mode for luminance samples is a non-directional mode (non-angular mode).
  • intra_subpartitions_mode_flag[x0][y0] is 1, the intra prediction mode is derived according to the following "Derivation process for luminance intra prediction mode (Example 6)".
  • intra_luma_planar_flag[x0][y0] is 1, the intra prediction mode for luminance samples is set to the PLANAR mode. If intra_luma_planar_flag[x0][y0] is 0, the intra prediction mode for luminance samples is set to the DC mode. If intra_luma_planar_flag[x0][y0] does not exist, its value is deduced as 1.
  • intra_luma_mpm_flag[ x0 ][ y0] indicates the directional intra prediction mode for luminance samples. do.
  • Array indices x0 and y0 represent positions (x0, y0) of an upper left luminance sample of a coding block with respect to an upper left luminance sample of a picture.
  • intra_luma_angular_mode_flag[x0][y0] is 1 and intra_luma_mpm_flag[x0][y0] is 1, the intra prediction mode is derived according to the following "Derivation process for luminance intra prediction mode (Example 6)".
  • ciip_intra_luma_vert_flag[x0][y0] is 1
  • the intra prediction mode for luminance samples is set to INTRA_ANGULAR50 (vertical mode).
  • the intra prediction mode for luminance samples is set to INTRA_ANGULAR18 (horizontal mode). If intra_luma_vert_flag[x0][y0] does not exist, its value is deduced as 0.
  • intra_luma_angular_mode_flag[ x0 ][ y0 ] the syntax elements intra_luma_angular_mode_flag[ x0 ][ y0 ], ciip_intra_luma_vert_flag[ x0 ][ y0 ], and intra_luma_planar_flag[ x0 ][ y0] are merged and combined of intra-picture prediction.
  • Intra prediction mode for luminance samples used in inter-picture merge and intra-picture prediction is indicated.
  • the array indices x0 and y0 indicate a position (x0, y0) of an upper left luminance sample of a coding block with respect to an upper left luminance sample of a picture.
  • IntraPredModeY may be derived as shown in Table 37 below.
  • the decoder checks whether the directional mode flag (intra_angular_mode_flag) is 1. If the directional mode flag (intra_angular_mode_flag) is 1, the intra prediction mode is determined from the vertical mode (INTRA_ANGULAR50) and the horizontal mode (INTRA_ANGULAR18) according to the CIIP direction flag (ciip_intra_luma_vert_flag).
  • the vertical mode (INTRA_ANGULAR50) is selected, and when the CIIP direction flag (ciip_intra_luma_vert_flag) is 0, the horizontal mode (INTRA_ANGULAR18) is determined as the intra prediction mode of the current block.
  • an intra prediction mode is determined among the planar mode and the DC mode according to the planar flag (intra_luma_planar_flag).
  • the planar flag (intra_luma_planar_flag) is 1
  • the planar mode is determined as the intra prediction mode of the current block
  • the planar flag (intra_luma_planar_flag) is 0, the DC mode is determined as the intra prediction mode of the current block.
  • the input to this process is:
  • the luminance intra mode IntraPredModeY[xCb][yCb] is derived.
  • names related to values for the intra prediction mode IntraPredModeY[xCb][yCb] are the same as in Table 30 described above.
  • IntraPredModeY[xCb][yCb] is derived by procedures 1 and 2 below.
  • IntraPredModeY[xCb][yCb] is derived by the following procedures a and b.
  • IntraPredModeY[xCb][yCb] is set equal to INTRA_PLANAR.
  • IntraPredModeY[xCb][yCb] is derived by the following procedures i and ii.
  • IntraSubPartitionsSplitType is ISP_HOR_SPLIT (when the current block is divided horizontally)
  • IntraPredModeY[xCb][yCb] is set to INTRA_ANGULAR18 (horizontal mode).
  • IntraPredModeY[xCb][yCb] is set to INTRA_ANGULAR50.
  • IntraPredModeY[xCb][yCb] is determined from the MPM candidate list or other modes excluding the MPM candidate.
  • binarization process and input parameters of the syntax element according to the embodiment of the present specification are shown in Table 38, and the bin index (binIdx) of the corresponding syntax element may be set as shown in Table 39 below.
  • the directional mode flag (intra_luma_angular_mode_flag) is determined between 0 and 4 based on the ctxInc derivation process below when the CIIP flag is 0, and when the CIIP flag is 1, ctxInc is (cbWidth> 2 * cbHeight
  • (Condition-A:B) is outputted when A is satisfied with the condition and B is not satisfied. That is, if the width of the block is greater than twice the height or the height of the block is greater than twice the height, ctxInc is set to 10, otherwise ctxInc is set to 11.
  • the input of this process is the luminance position (x0, y0) representing the upper left luminance sample of the current luminance block with respect to the upper left sample of the current picture, the color component cIdx, the current coding quadtree depth cqDepth, the current coding in units of luminance samples.
  • the width and height of the block are cbWidth and cbHeight.
  • the process of inducing availability for the block is called, and the position (xCurr, yCurr) set to (x0, y0) and the neighboring position (xNbY, yNbY) set to (x0-1, y0) are entered, and the left neighbor AvailableL allocated as a variable indicating whether the block can be used is output.
  • the process of inducing usability for the block is called, and the position (xCurr, yCurr) set to (x0, y0) and the neighboring position (xNbY, yNbY) set to (x0-1, y0 + cbHeight-1) are input.
  • the availableLB allocated as a variable indicating whether the neighboring block can be used is output.
  • the process of inducing usability for the block is called, and the position (xCurr, yCurr) set to (x0, y0) and the neighboring position (xNbY, yNbY) set to (x0, y0-1) are input, and AvailableA allocated as a variable indicating whether or not it is available is displayed.
  • the process of inducing usability for the block is called, and the location (xCurr, yCurr) set to (x0, y0) and the neighboring location (xNbY, yNbY) set to (x0 + cbWidth-1, y0-1) are input, and the upper right
  • the availableAR allocated as a variable indicating whether the neighboring block can be used is output.
  • the same CTU check process for the block is called, and the location set as (x0, y0) (xCurr, yCurr) and the neighbor location set as (x0, y0-1) (xNbY, yNbY) are input, and the current block and the neighboring block
  • the sameCtuA assigned as a variable indicating whether it belongs to the same CTU is output.
  • the same CTU check process for the block is called, and the position (xCurr, yCurr) set to (x0, y0) and the neighboring position (xNbY, yNbY) set to (x0 + cbWidth-1, y0-1) are input.
  • the sameCtuAR allocated as a variable indicating whether a block and a neighboring block belong to the same CTU is output.
  • ctxInc is derived as shown in Equation 5 and Table 40 below using syntax elements condL, condLB, condA, and condAR for intra_luma_angular_mode_flag[x0][y0].
  • the encoded information (eg, encoded video/video information) derived by the encoding apparatus 100 based on the above-described embodiments of the present specification may be output in a bitstream form.
  • the encoded information may be transmitted or stored in a bitstream form in units of network abstraction layer (NAL) units.
  • NAL network abstraction layer
  • the bitstream may be transmitted over a network or may be stored in a non-transitory digital storage medium.
  • the bitstream is not directly transmitted from the encoding device 100 to the decoding device 200, but may be provided with a streaming/download service through an external server (eg, a content streaming server).
  • the network may include a broadcasting network and/or a communication network
  • the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD.
  • FIG. 37 shows an example of a video signal encoding procedure according to an embodiment of the present specification. The operations of FIG. 37 may be performed by the encoding device 100 or the video signal processing device 500.
  • the encoder divides the current block into a plurality of subblocks. For example, the encoder may calculate the RD cost for each of various prediction modes for the current block and determine to apply ISP intra prediction. When ISP intra prediction is applied, the encoder may divide the current block vertically or horizontally as shown in FIG. 26A or 26B in consideration of the size of the current block and the RD cost.
  • the encoder encodes ISP information including an ISP mode flag (intra_subpartitions_mode_flag) indicating whether to divide the current block and an ISP direction flag (intra_subpartitions_split_flag) indicating the direction of division of the current block.
  • ISP mode flag when ISP intra prediction is applied, the ISP mode flag may be coded as 1.
  • ISP direction flag when the current block is divided in the horizontal direction, the ISP direction flag (intra_subpartitions_split_flag) may be coded as 0, and when the current block is divided in the vertical direction, the ISP direction flag (intra_subpartitions_split_flag) may be coded as 1.
  • step S3730 the encoder determines candidate modes corresponding to some of all intra modes based on ISP information.
  • candidate modes may include a non-directional mode and at least one directional mode.
  • at least one directional mode may be determined among all directional modes based on the division direction of the current block.
  • the encoder may determine two intra prediction modes (PLANAR, vertical/horizontal) among all intra modes (eg, 67) as candidate modes.
  • the vertical/horizontal mode is determined by the ISP division direction.
  • a horizontal mode may be determined
  • a vertical mode may be determined as a directional candidate mode.
  • the encoder may determine three intra prediction modes (PLANAR, vertical/horizontal, 25/43) among all intra modes (eg, 67) as candidate modes.
  • the directional candidate mode is determined by the ISP division direction. If the ISP division direction is horizontal, the horizontal mode and the 25th mode are determined as the directional candidate mode. If the ISP division direction is vertical, the vertical mode and the 43rd mode are the directional candidate modes. Can be determined as
  • the encoder may determine four intra prediction modes (PLANAR, vertical/horizontal, 25/43, 10/60) among all intra modes (eg, 67) as candidate modes.
  • the directional candidate mode is determined by the ISP segmentation direction. If the ISP segmentation direction is horizontal, the horizontal mode, the 25th mode, and the 10th mode are determined as the directional candidate mode. If the ISP segmentation direction is vertical, the vertical mode and the 43rd mode , Mode 60 may be determined as a directional candidate mode.
  • step S3740 the encoder determines an intra prediction mode of the current block among candidate modes. For example, the encoder does not determine the intra prediction mode by calculating the RD cost for each of all intra modes, but calculates the RD cost for only 2 to 4 partial candidate modes to determine the intra prediction mode of the current block. You can decide.
  • the encoder encodes intra prediction information including an intra prediction mode of the current block.
  • an MPM candidate list is configured for the current block to which the ISP intra prediction mode is applied, and the MPM index (mpm_idx) or the remaining index (mpm_remainder) is not coded, and information of 1 to 3 bits (ISP intra mode flag/ Information) may be used to signal the intra prediction mode of the current block.
  • the encoder may indicate the intra prediction mode of the current block using a 1-bit ISP intra mode flag (intra_subpartitions_intra_mode_flag).
  • ISP intra mode flag (intra_subpartitions_intra_mode_flag)
  • the ISP intra mode flag (intra_subpartitions_intra_mode_flag)
  • the ISP intra mode flag (intra_subpartitions_intra_mode_flag)
  • the ISP intra mode flag May be coded as '0'.
  • the decoder may determine a prediction mode among the horizontal mode and the vertical mode based on the ISP direction flag (intra_subpartitions_split_flag) indicating the ISP division direction of the current block.
  • the encoder may indicate the intra prediction mode of the current block using 2-bit ISP intra mode information.
  • ISP intra mode information is coded as '1'
  • ISP intra mode information is '01'
  • ISP intra mode information is coded as '00'.
  • the decoder may determine the prediction mode based on ISP intra mode information indicating the ISP division direction of the current block and the ISP direction flag (intra_subpartitions_split_flag).
  • the encoder uses 1-3 or 2 bits of ISP intra mode information to determine the intra prediction mode of the current block. Can instruct.
  • ISP intra mode information is coded as '1' or '00'
  • the ISP intra mode information is If coded as '01' and the intra prediction mode of the current block is the 25th/43th mode, ISP intra mode information is coded as '001' or '10', and the intra prediction mode of the current block is 10th/60th
  • ISP intra mode information is coded as '000' or '11'.
  • the decoder may determine the prediction mode based on ISP intra mode information indicating the ISP split direction of the current block and the ISP direction flag (intra_subpartitions_split_flag).
  • the encoder derives the prediction samples of the current block based on the intra prediction mode, derives residual samples of the current block by subtracting the prediction samples from the original samples of the current block, and obtains each of the residual samples in the current block including the residual samples.
  • Horizontal transformation and vertical transformation may be applied to subblocks.
  • a horizontal transformation kernel for horizontal transformation and a vertical transformation kernel for vertical transformation may be determined based on an intra prediction mode.
  • the horizontal transform kernel and the vertical transform kernel may be determined based on the intra prediction mode as shown in Tables 14 to 16, 20 to 22, 26, and 27.
  • a horizontal transform kernel for horizontal transform and a vertical transform kernel for vertical transform may be determined as DST-7 for all intra prediction modes as shown in Table 28.
  • FIG. 38 illustrates an example of a video signal decoding procedure according to an embodiment of the present specification. The operations of FIG. 38 may be performed by the decoding apparatus 200 or the video signal processing apparatus 500.
  • the decoder acquires ISP information including an ISP mode flag (intra_subpartitions_mode_flag) indicating whether to divide the current block and an ISP direction flag (intra_subpartitions_split_flag) indicating the direction of division of the current block. For example, when the ISP mode flag is 1, the decoder may determine that the ISP intra prediction is applied to the current block (that the current block is divided by the ISP).
  • the current block is divided in the horizontal direction, and when the current block is divided in the vertical direction, if the ISP direction flag (intra_subpartitions_split_flag) is 1, the current block may be divided in the vertical direction.
  • the decoder may divide the current block into a plurality of subblocks based on ISP information.
  • ISP intra prediction When ISP intra prediction is applied, the intra prediction mode is equally applied to all subblocks of the current block, but a reference sample may be derived for each subblock.
  • the decoder determines an intra prediction mode of the current block from the intra prediction mode information.
  • the decoder may determine the intra prediction mode of the current block from among some candidate modes rather than all modes.
  • the candidate modes are composed of 2 to 4, and a final intra prediction mode may be determined based on 1 to 3 bits of information (ISP intra mode information) and an ISP direction flag among the candidate modes.
  • the ISP intra mode information can be parsed by the decoder when the current block is divided by the ISP (ISP intra prediction is applied) and the MRL is not applied.
  • the intra prediction mode when the ISP intra mode information corresponds to a first value (eg, 0), the intra prediction mode is determined as a PLANAR mode, and the ISP intra mode information corresponds to a second value, and the ISP direction If the flag corresponds to the horizontal direction, the intra prediction mode is determined as the horizontal mode, and if the ISP intra mode information corresponds to a second value (eg 1) and the ISP direction flag corresponds to the vertical direction, the intra prediction mode is determined as the vertical mode.
  • a first value eg, 0
  • the intra prediction mode information corresponds to a PLANAR mode
  • the ISP intra mode information corresponds to a second value
  • the ISP direction flag corresponds to the vertical direction
  • ISP intra mode information corresponds to a first value (eg, 0)
  • the intra prediction mode is determined as a PLANAR mode
  • ISP intra mode information is a second value (eg, 01).
  • the ISP direction flag corresponds to the horizontal direction
  • the intra prediction mode is determined as the horizontal mode
  • the ISP intra mode information corresponds to a second value (eg, 01) and the ISP direction flag corresponds to the vertical direction
  • the intra prediction mode is determined as the vertical mode
  • the ISP intra mode information corresponds to a third value (eg 00) and the ISP direction flag corresponds to the horizontal direction
  • the intra prediction mode is a horizontal mode and a top-left diagonal mode.
  • the intra prediction mode is the vertical mode and the upper left. It may be determined as a second directional mode (eg, Mode10) positioned between the diagonal modes.
  • the intra prediction mode is determined as a PLANAR mode
  • the ISP intra mode information is a second value ( Example: 01) and the ISP direction flag corresponds to the horizontal direction
  • the intra prediction mode is determined as the horizontal mode
  • the ISP intra mode information corresponds to the second value (eg 01)
  • the ISP direction flag corresponds to the vertical direction.
  • the intra prediction mode is determined as a vertical mode
  • the ISP intra mode information corresponds to a third value (e.g., 001 or 10) and the ISP direction flag corresponds to the horizontal direction
  • the intra prediction mode is set to the horizontal mode and the top-left left)
  • Intra prediction mode if the first directional mode (eg, Mode25) located between the diagonal modes is determined, and the ISP intra mode information corresponds to the third value (eg, 001 or 10) and the ISP direction flag corresponds to the vertical direction.
  • the second directional mode eg, Mode10 located between the vertical mode and the upper left diagonal mode
  • the ISP intra mode information corresponds to the fourth value (eg, 000 or 11)
  • the ISP direction flag corresponds to the horizontal direction.
  • the intra prediction mode is determined as a third directional mode (eg, Mode10) located between the horizontal mode and the bottom-left diagonal mode, and the ISP intra mode information corresponds to the fourth value (eg, 000 or 11). And if the ISP direction flag corresponds to the vertical direction, the intra prediction mode may be determined as a fourth directional mode (eg, Mode60) located between the vertical mode and the top-right diagonal mode.
  • a third directional mode eg, Mode10
  • the intra prediction mode information corresponds to the fourth value (eg, 000 or 11).
  • the intra prediction mode may be determined as a fourth directional mode (eg, Mode60) located between the vertical mode and the top-right diagonal mode.
  • step S3840 the decoder performs reconstruction through prediction and transformation for each of the subblocks based on the intra prediction mode.
  • a horizontal transform kernel and a vertical transform kernel for transforming a subblock divided from a current block may be set to DST-7 for all intra prediction modes.
  • the horizontal transform kernel and the vertical transform kernel may be determined based on the intra prediction mode as shown in Tables 14 to 16, 20 to 22, 26, and 27.
  • the embodiments described in the present invention may be implemented and performed on a processor, microprocessor, controller, or chip.
  • the functional units illustrated in each drawing may be implemented and executed on a computer, processor, microprocessor, controller, or chip.
  • the video signal processing apparatus 500 includes a memory 520 for storing a video signal, and a processor 510 coupled to the memory 520. can do.
  • the processor 510 For encoding of a video signal, the processor 510 divides the current block into a plurality of subblocks, an ISP mode flag indicating whether to divide the current block, and an ISP direction flag indicating a dividing direction of the current block. Encoding ISP information including, and determining candidate modes corresponding to some of all intra modes based on the ISP information, determining an intra prediction mode of the current block among the candidate modes, and It is set to encode intra prediction information including an intra prediction mode.
  • the candidate modes include a non-directional mode and at least one directional mode, and the at least one directional mode may be determined among all directional modes based on a division direction of the current block.
  • the non-directional mode includes a planar mode, and when the current block is divided in a horizontal direction, the at least one directional mode includes a horizontal mode, and the current block is divided in a vertical direction. If so, the at least one directional mode may include a vertical mode.
  • the processor 510 derives prediction samples of the current block based on the intra prediction mode, and derives residual samples of the current block by subtracting the prediction samples from the original samples of the current block. , It may be set to apply a horizontal transform and a vertical transform to each subblock in the current block including the residual samples.
  • the horizontal transformation kernel for the horizontal transformation and the vertical transformation kernel for the vertical transformation may be determined based on the intra prediction mode.
  • the horizontal transformation kernel for the horizontal transformation and the vertical transformation kernel for the vertical transformation may be determined as DST-7 (Discrete Sine Transform type 7) for all intra prediction modes.
  • the processor 510 obtains ISP information including an ISP (intra sub-partition) mode flag indicating whether to divide the current block and an ISP direction flag indicating the direction of division of the current block. And divide the current block into a plurality of subblocks based on the ISP information, determine an intra prediction mode of the current block from intra prediction information, and predict the subblocks based on the intra prediction mode And it is set to perform restoration through conversion.
  • ISP intra sub-partition
  • a horizontal transform kernel and a vertical transform kernel for transforming a subblock divided from the current block may be set to DST-7 (Discrete Sine Transform type 7) for all intra prediction modes.
  • the processor 510 may be configured to obtain ISP intra mode information based on the ISP mode flag, and determine the intra prediction mode based on the ISP intra mode information and the ISP direction flag. .
  • the ISP intra mode information may be parsed when the current block is divided by the ISP and a multi reference line (MRL) is not applied.
  • MTL multi reference line
  • the intra prediction mode is determined as a PLANAR mode, and if the ISP intra mode information corresponds to a second value and the ISP direction flag corresponds to a horizontal direction, The intra prediction mode is determined as a horizontal mode, and when the ISP intra mode information corresponds to a second value and the ISP direction flag corresponds to a vertical direction, the intra prediction mode may be determined as a vertical mode.
  • the intra prediction mode is determined as a PLANAR mode, and if the ISP intra mode information corresponds to a second value and the ISP direction flag corresponds to a horizontal direction, The intra prediction mode is determined as a horizontal mode, and when the ISP intra mode information corresponds to a second value and the ISP direction flag corresponds to a vertical direction, the intra prediction mode is determined as a vertical mode, and the ISP intra mode information is If it corresponds to a third value and the ISP direction flag corresponds to a horizontal direction, the intra prediction mode is determined as a first directional mode located between a horizontal mode and a top-left diagonal mode, and the ISP intra mode information is If it corresponds to a third value and the ISP direction flag corresponds to a vertical direction, the intra prediction mode may be determined as a second directional mode located between a vertical mode and an upper left diagonal mode.
  • the intra prediction mode is determined as a PLANAR mode, and if the ISP intra mode information corresponds to a second value and the ISP direction flag corresponds to a horizontal direction, The intra prediction mode is determined as a horizontal mode, and when the ISP intra mode information corresponds to a second value and the ISP direction flag corresponds to a vertical direction, the intra prediction mode is determined as a vertical mode, and the ISP intra mode information is If it corresponds to a third value and the ISP direction flag corresponds to a horizontal direction, the intra prediction mode is determined as a first directional mode located between a horizontal mode and a top-left diagonal mode, and the ISP intra mode information is If it corresponds to a third value and the ISP direction flag corresponds to the vertical direction, the intra prediction mode is determined as a second directional mode located between the vertical mode and the upper left diagonal mode, and the ISP intra mode information corresponds to the fourth value.
  • the intra prediction mode is determined as a third directional mode located between the horizontal mode and the bottom-left diagonal mode, and the ISP intra mode information corresponds to a fourth value.
  • the intra prediction mode may be determined as a fourth directional mode located between a vertical mode and a top-right diagonal mode.
  • the processing method to which the present invention is applied may be produced in the form of a program executed by a computer, and may be stored in a computer-readable recording medium.
  • Multimedia data having a data structure according to the present invention can also be stored in a computer-readable recording medium.
  • the computer-readable recording medium includes all kinds of storage devices and distributed storage devices in which computer-readable data is stored.
  • the computer-readable recording medium includes, for example, Blu-ray disk (BD), universal serial bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical It may include a data storage device.
  • the computer-readable recording medium includes media implemented in the form of a carrier wave (for example, transmission through the Internet).
  • the bitstream generated by the encoding method may be stored in a computer-readable recording medium or transmitted through a wired or wireless communication network.
  • an embodiment of the present invention may be implemented as a computer program product using a program code, and the program code may be executed in a computer according to the embodiment of the present invention.
  • the program code may be stored on a carrier readable by a computer.
  • a non-transitory computer-readable medium stores one or more instructions executed by one or more processors.
  • the one or more commands may include an ISP (intra sub-partition) mode flag indicating whether to divide the current block into a plurality of subblocks and indicate whether to divide the current block, and the current block.
  • ISP intra sub-partition
  • the candidate modes include a non-directional mode and at least one directional mode, and the at least one directional mode may be determined among all directional modes based on a division direction of the current block.
  • the non-directional mode includes a planar mode, and when the current block is divided in a horizontal direction, the at least one directional mode includes a horizontal mode, and the current block is divided in a vertical direction. If so, the at least one directional mode may include a vertical mode.
  • the one or more instructions derive the prediction samples of the current block based on the intra prediction mode, and subtract the prediction samples from the original samples of the current block, thereby subtracting the residual samples of the current block.
  • the video signal processing apparatus 500 (or the encoding apparatus 100) may be controlled to derive and apply a horizontal transform and a vertical transform to each of the subblocks in the current block including the residual samples.
  • the horizontal transformation kernel for the horizontal transformation and the vertical transformation kernel for the vertical transformation may be determined based on the intra prediction mode.
  • the horizontal transformation kernel for the horizontal transformation and the vertical transformation kernel for the vertical transformation may be determined as DST-7 (Discrete Sine Transform type 7) for all intra prediction modes.
  • the one or more commands are ISP information including an intra sub-partition (ISP) mode flag indicating whether to divide a current block and an ISP direction flag indicating a dividing direction of the current block for decoding a video signal. And, based on the ISP information, the current block is divided into a plurality of subblocks, an intra prediction mode of the current block is determined from the intra prediction information, and the subblocks are selected based on the intra prediction mode.
  • the video signal processing apparatus 500 (or the encoding apparatus 100) is controlled to perform restoration through prediction and transformation.
  • a horizontal transform kernel and a vertical transform kernel for transforming a subblock divided from the current block may be set to DST-7 (Discrete Sine Transform type 7) for all intra prediction modes.
  • the one or more commands are video signal processing to obtain ISP intra mode information based on the ISP mode flag, and determine the intra prediction mode based on the ISP intra mode information and the ISP direction flag
  • the device 500 (or the encoding device 100) can be controlled.
  • the ISP intra mode information may be parsed when the current block is divided by the ISP and a multi reference line (MRL) is not applied.
  • MTL multi reference line
  • the intra prediction mode is determined as a PLANAR mode, and if the ISP intra mode information corresponds to a second value and the ISP direction flag corresponds to a horizontal direction, The intra prediction mode is determined as a horizontal mode, and when the ISP intra mode information corresponds to a second value and the ISP direction flag corresponds to a vertical direction, the intra prediction mode may be determined as a vertical mode.
  • the intra prediction mode is determined as a PLANAR mode, and if the ISP intra mode information corresponds to a second value and the ISP direction flag corresponds to a horizontal direction, The intra prediction mode is determined as a horizontal mode, and when the ISP intra mode information corresponds to a second value and the ISP direction flag corresponds to a vertical direction, the intra prediction mode is determined as a vertical mode, and the ISP intra mode information is If it corresponds to a third value and the ISP direction flag corresponds to a horizontal direction, the intra prediction mode is determined as a first directional mode located between a horizontal mode and a top-left diagonal mode, and the ISP intra mode information is If it corresponds to a third value and the ISP direction flag corresponds to a vertical direction, the intra prediction mode may be determined as a second directional mode located between a vertical mode and an upper left diagonal mode.
  • the intra prediction mode is determined as a PLANAR mode, and if the ISP intra mode information corresponds to a second value and the ISP direction flag corresponds to a horizontal direction, The intra prediction mode is determined as a horizontal mode, and when the ISP intra mode information corresponds to a second value and the ISP direction flag corresponds to a vertical direction, the intra prediction mode is determined as a vertical mode, and the ISP intra mode information is If it corresponds to a third value and the ISP direction flag corresponds to a horizontal direction, the intra prediction mode is determined as a first directional mode located between a horizontal mode and a top-left diagonal mode, and the ISP intra mode information is If it corresponds to a third value and the ISP direction flag corresponds to the vertical direction, the intra prediction mode is determined as a second directional mode located between the vertical mode and the upper left diagonal mode, and the ISP intra mode information corresponds to the fourth value.
  • the intra prediction mode is determined as a third directional mode located between the horizontal mode and the bottom-left diagonal mode, and the ISP intra mode information corresponds to a fourth value.
  • the intra prediction mode may be determined as a fourth directional mode located between a vertical mode and a top-right diagonal mode.
  • the embodiments described in the present invention may be implemented and performed on a processor, microprocessor, controller, or chip.
  • the functional units illustrated in each drawing may be implemented and executed on a computer, processor, microprocessor, controller, or chip.
  • the decoder and encoder to which the present invention is applied include a multimedia broadcasting transmission/reception device, a mobile communication terminal, a home cinema video device, a digital cinema video device, a surveillance camera, a video chat device, a real-time communication device such as video communication, a mobile streaming device, Storage media, camcorders, video-on-demand (VoD) service providers, OTT video (Over the top video) devices, Internet streaming service providers, three-dimensional (3D) video devices, video telephony video devices, and medical video devices. And can be used to process video signals or data signals.
  • an OTT video (Over the top video) device may include a game console, a Blu-ray player, an Internet-connected TV, a home theater system, a smartphone, a tablet PC, and a digital video recorder (DVR).
  • an embodiment of the present invention may be implemented by various means, for example, hardware, firmware, software, or a combination thereof.
  • an embodiment of the present invention is one or more ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs ( field programmable gate arrays), processors, controllers, microcontrollers, microprocessors, etc.
  • ASICs application specific integrated circuits
  • DSPs digital signal processors
  • DSPDs digital signal processing devices
  • PLDs programmable logic devices
  • FPGAs field programmable gate arrays
  • processors controllers, microcontrollers, microprocessors, etc.
  • an embodiment of the present invention may be implemented in the form of a module, procedure, or function that performs the functions or operations described above.
  • the software code can be stored in a memory and driven by a processor.
  • the memory may be located inside or outside the processor, and may exchange data with the processor through various known means.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Discrete Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

Des modes de réalisation de la présente invention concernent un procédé et un appareil de traitement d'un signal vidéo. Un procédé de codage de signal vidéo selon un mode de réalisation de la présente invention comprend les étapes consistant à: diviser un bloc courant en une pluralité de sous-blocs; coder des informations de sous-partition intra (ISP) comprenant un drapeau de mode d'ISP indiquant si oui ou non le bloc courant est divisé et un indicateur de direction d'ISP indiquant une direction de division du bloc courant; déterminer, sur la base des informations d'ISP, des modes candidats correspondant à une partie de tous les modes intra; et coder des informations de prédiction intra comprenant un mode de prédiction intra du bloc courant. En limitant un mode de prédiction intra pour un bloc auquel une prédiction intra d'ISP a été appliquée, la quantité de données et le temps de traitement requis pour la signalisation du mode de prédiction intra peuvent être réduits.
PCT/KR2020/003179 2019-03-07 2020-03-06 Procédé et appareil de traitement de signal vidéo Ceased WO2020180151A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US201962815344P 2019-03-07 2019-03-07
US62/815,344 2019-03-07
US201962821989P 2019-03-21 2019-03-21
US62/821,989 2019-03-21

Publications (1)

Publication Number Publication Date
WO2020180151A1 true WO2020180151A1 (fr) 2020-09-10

Family

ID=72338028

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2020/003179 Ceased WO2020180151A1 (fr) 2019-03-07 2020-03-06 Procédé et appareil de traitement de signal vidéo

Country Status (1)

Country Link
WO (1) WO2020180151A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024131778A1 (fr) * 2022-12-20 2024-06-27 Mediatek Inc. Prédiction intra avec dérivation basée sur une région
WO2025011499A1 (fr) * 2023-07-07 2025-01-16 Mediatek Inc. Héritage de modèle de prédiction intra d'extrapolation

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20140049098A (ko) * 2011-09-12 2014-04-24 퀄컴 인코포레이티드 비디오 코딩에서 비정방형 변환 유닛들 및 예측 유닛들
KR20170058866A (ko) * 2015-11-19 2017-05-29 한국전자통신연구원 영상 부호화/복호화 방법 및 장치
KR20180044943A (ko) * 2015-09-23 2018-05-03 엘지전자 주식회사 영상 코딩 시스템에서 인트라 예측 방법 및 장치
KR20180131571A (ko) * 2016-04-29 2018-12-10 인텔렉추얼디스커버리 주식회사 영상 신호 부호화/복호화 방법 및 장치

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20140049098A (ko) * 2011-09-12 2014-04-24 퀄컴 인코포레이티드 비디오 코딩에서 비정방형 변환 유닛들 및 예측 유닛들
KR20180044943A (ko) * 2015-09-23 2018-05-03 엘지전자 주식회사 영상 코딩 시스템에서 인트라 예측 방법 및 장치
KR20170058866A (ko) * 2015-11-19 2017-05-29 한국전자통신연구원 영상 부호화/복호화 방법 및 장치
KR20180131571A (ko) * 2016-04-29 2018-12-10 인텔렉추얼디스커버리 주식회사 영상 신호 부호화/복호화 방법 및 장치

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
BROSS, BENJAMIN ET AL.: "Versatile Video Coding (Draft 4).", JOINT VIDEO EXPERTS TEAM (JVET) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/WG 11., no. JVET-M1001-v5, 27 February 2019 (2019-02-27), Marrakech MA., pages 1 - 274, XP030202598 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024131778A1 (fr) * 2022-12-20 2024-06-27 Mediatek Inc. Prédiction intra avec dérivation basée sur une région
WO2025011499A1 (fr) * 2023-07-07 2025-01-16 Mediatek Inc. Héritage de modèle de prédiction intra d'extrapolation

Similar Documents

Publication Publication Date Title
WO2021054796A1 (fr) Procédé de codage d'image basé sur une transformée, et dispositif associé
WO2021101317A1 (fr) Procédé et dispositif de codage/décodage d'image utilisant une transformation de couleur sans perte, et procédé de transmission de train de bits
WO2021006624A1 (fr) Filtre à boucle adaptatif appliquant un codage vidéo ou d'image
WO2020171660A1 (fr) Procédé et dispositif de traitement d'un signal vidéo pour prédiction intra
WO2021206445A1 (fr) Procédé de codage d'image basé sur une transformée et appareil associé
WO2020197155A1 (fr) Procédé et dispositif de décodage d'image et procédé et dispositif d'encodage d'image dans un système de codage d'image
WO2021096290A1 (fr) Procédé de codage d'image basé sur une transformée et dispositif associé
WO2021172914A1 (fr) Procédé de décodage d'image pour un codage résiduel et dispositif associé
WO2021060844A1 (fr) Procédé et dispositif de codage/décodage d'image utilisant un mode palette, et procédé de transmission de train de bits
WO2021006633A1 (fr) Codage vidéo ou d'image reposant sur un filtrage en boucle
WO2021040482A1 (fr) Dispositif et procédé de codage d'image à base de filtrage de boucle adaptatif
WO2020204418A1 (fr) Procédé de codage de vidéo ou d'image et dispositif associé
WO2020184952A1 (fr) Procédé et dispositif de traitement de signal vidéo permettant de traiter des informations de différence de vecteur de mouvement pour l'interprédiction dans un signal vidéo
WO2021006698A1 (fr) Procédé et dispositif de codage d'image dans un système de codage d'image
WO2021040406A1 (fr) Procédé de codage de vidéo ou d'image et dispositif associé
WO2021054798A1 (fr) Procédé et dispositif de codage d'image à base de transformée
WO2020171673A1 (fr) Procédé et appareil de traitement de signal vidéo pour prédiction intra
WO2021006697A1 (fr) Procédé de décodage d'image pour un codage résiduel et appareil associé
WO2021040410A1 (fr) Procédé de décodage vidéo pour codage résiduel et dispositif associé
WO2020197207A1 (fr) Codage d'image ou vidéo sur la base d'un filtrage comprenant un mappage
WO2020213866A1 (fr) Codage de vidéo ou d'image à base de paramètre de liste de mise à l'échelle
WO2021172916A1 (fr) Procédé de décodage d'image et appareil associé
WO2020251268A1 (fr) Procédé de décodage d'image destiné à un composant de chrominance et dispositif associé
WO2020184928A1 (fr) Codage vidéo ou d'image basé sur une cartographie de luminance et une mise à l'échelle chromatique
WO2020009400A1 (fr) Procédé et appareil permettant de traiter un signal vidéo à l'aide d'une prédiction intra

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20766515

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20766515

Country of ref document: EP

Kind code of ref document: A1