WO2020071846A1 - Procédé et appareil permettant de traiter un signal vidéo à l'aide d'une prédiction intra - Google Patents

Procédé et appareil permettant de traiter un signal vidéo à l'aide d'une prédiction intra

Info

Publication number
WO2020071846A1
WO2020071846A1 PCT/KR2019/013024 KR2019013024W WO2020071846A1 WO 2020071846 A1 WO2020071846 A1 WO 2020071846A1 KR 2019013024 W KR2019013024 W KR 2019013024W WO 2020071846 A1 WO2020071846 A1 WO 2020071846A1
Authority
WO
WIPO (PCT)
Prior art keywords
prediction
unit
mpm
current block
sample
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/KR2019/013024
Other languages
English (en)
Korean (ko)
Inventor
허진
유선미
이령
최장원
최정아
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
LG Electronics Inc
Original Assignee
LG Electronics Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by LG Electronics Inc filed Critical LG Electronics Inc
Publication of WO2020071846A1 publication Critical patent/WO2020071846A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/11—Selection of coding mode or of prediction mode among a plurality of spatial predictive coding modes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards

Definitions

  • the present specification relates to a method and apparatus for processing a video signal using intra prediction, and more particularly, to a method and apparatus for efficiently processing information related to prediction.
  • Compression encoding refers to a series of signal processing techniques for transmitting digitized information over a communication line or storing it in a form suitable for a storage medium.
  • Media such as video, image, and audio may be the subject of compression encoding.
  • a technique for performing compression encoding on an image is referred to as video image compression.
  • Next-generation video content will have the characteristics of high spatial resolution, high frame rate and high dimensionality of scene representation. In order to process such content, a huge increase in terms of memory storage, memory access rate and processing power will be produced.
  • the video codec standard after the high efficiency video coding (HEVC) standard requires an intra prediction technique capable of efficiently coding information related to prediction while improving coding efficiency.
  • embodiments of the present specification are to provide a video signal processing method and apparatus for efficiently processing information related to prediction.
  • a video signal processing method using intra prediction includes: checking a most probable mode (MPM) flag for a current block and, if the MPM flag is 1, a reference line for prediction of the current block Identifying an index and an MPM index, and generating a prediction sample for the current block using a prediction direction indicated by the MPM index from a reference line indicated by the reference line index.
  • MPM most probable mode
  • the video signal processing method further includes: if the MPM is not applied, checking re-maining intra mode information, and generating a prediction sample of the current block based on the re-maining intra mode information. can do.
  • checking the reference line index and the MPM index may include parsing the MPM index after parsing the reference line index or parsing the reference line index after parsing the MPM index. It can contain.
  • the reference line index and the MPM index may be parsed simultaneously as one syntax element.
  • the reference line index may indicate the location of a peripheral reference sample line within a predetermined sample distance to the left or upper side of the current block used for prediction of the current block.
  • the reference line index may indicate the location of a peripheral reference sample line within a predetermined sample distance to the left or upper side of the current block used for prediction of the current block. Also, the reference line index may indicate a reference sample line spaced by a distance of 1, 2, or 4 samples from the current block.
  • a video signal processing apparatus using intra prediction includes a memory that stores the video signal, and a processor coupled with the memory, wherein the processor is the most probable mode for the current block ) Flag, and if the MPM flag is 1, a reference line index and an MPM index for prediction of the current block are checked, and a prediction direction indicated by the MPM index from a reference line indicated by the reference line index Is set to generate a prediction sample for the current block.
  • the processor may be configured to check remodeling intra mode information and generate a prediction sample of the current block based on the remodeling intra mode information.
  • the processor may be configured to parse the MPM index after parsing the reference line index or parse the reference line index after parsing the MPM index.
  • information related to prediction can be efficiently processed by using an association between the most probable mode (MPM) and a multiple reference line (MRL).
  • FIG. 1 shows an example of a video coding system according to an embodiment of the present specification.
  • FIG. 3 is an embodiment of the present specification, and shows a schematic block diagram of a decoding apparatus in which decoding of a video signal is performed.
  • FIG 5 shows an example of a multi-type tree structure according to an embodiment of the present specification.
  • FIG. 6 illustrates an example of a signaling mechanism of split information of a quadtree with nested multi-type tree structure accompanying a multitype tree according to an embodiment of the present disclosure.
  • FIG. 7 illustrates an example of a method of dividing a CTU into multiple CUs based on a quadtree and nested multi-type tree structure according to an embodiment of the present specification.
  • FIG. 8 shows an example of a case in which ternary tree partitioning according to an embodiment of the present disclosure is limited.
  • FIGS. 12 and 13 illustrate an intra prediction-based video / video decoding method according to an embodiment of the present specification and an example of an intra prediction unit in a decoder according to an embodiment of the present invention.
  • FIG. 16 shows an example of a flowchart for constructing a list of most probable mode (MPM) candidates according to an embodiment of the present specification.
  • 17A and 17B show examples of reference samples for a wide-angle intra prediction mode according to an embodiment of the present specification.
  • 21A and 21B show an example of a method for generating bottom samples and right samples according to an embodiment of the present specification.
  • FIG. 22 shows an example of a method for generating bottom samples and right samples according to an embodiment of the present specification.
  • FIG. 23 shows an example of reference samples defined in a PDPC according to an embodiment of the present specification.
  • FIG 24 illustrates an example of reference lines for applying multiple reference line prediction according to an embodiment of the present specification.
  • 26 shows an example of the location of samples for deriving a linear model parameter according to an embodiment of the present specification.
  • FIG 27 illustrates an example of an intra prediction process using MRL (multi reference line) according to an embodiment of the present specification.
  • 29 shows an example of an encoding process for a block to which MRL intra prediction is applied according to an embodiment of the present specification.
  • FIG. 30 shows an example of a flowchart for performing intra prediction according to an embodiment of the present specification.
  • 32 is a diagram schematically showing an example of a service system including a digital device.
  • 33 is a block diagram illustrating a digital device according to an embodiment.
  • 34 is a configuration block diagram illustrating another embodiment of a digital device.
  • 35 is a configuration block diagram illustrating another embodiment of a digital device.
  • FIGS. 33 to 35 are block diagram illustrating a detailed configuration of the control unit of FIGS. 33 to 35 to illustrate one embodiment.
  • FIG. 37 is a diagram illustrating an example in which a screen of a digital device displays a main image and a sub image simultaneously according to an embodiment.
  • the term 'processing unit' in the present specification means a unit in which encoding / decoding processing processes such as prediction, transformation, and / or quantization are performed.
  • the processing unit may be interpreted to include a unit for a luminance component and a unit for a chroma component.
  • the processing unit may correspond to a block, a coding unit (CU), a prediction unit (PU), or a transform unit (TU).
  • the processing unit may be interpreted as a unit for a luminance component or a unit for a color difference component.
  • the processing unit may correspond to a coding tree block (CTB), a coding block (CB), a PU or a transform block (TB) for the luminance component.
  • the processing unit may correspond to CTB, CB, PU or TB for the color difference component.
  • the present invention is not limited thereto, and the processing unit may be interpreted to include a unit for a luminance component and a unit for a color difference component.
  • processing unit is not necessarily limited to square blocks, and may be configured in a polygonal shape having three or more vertices.
  • a pixel, a pixel, or a coefficient transformation coefficient or transformation coefficient that has undergone first-order transformation
  • a sample a pixel value, a pixel value, or a coefficient (a transform coefficient or a transform coefficient that has undergone first-order transformation) is used.
  • the video coding system can include a source device 10 and a receiving device 20.
  • the source device 10 may transmit the encoded video / video information or data to the receiving device 20 through a digital storage medium or a network in a file or streaming form.
  • the source device 10 may include a video source 11, an encoding device 12, and a transmitter 13.
  • the receiving device 20 may include a receiver 21, a decoding device 22 and a renderer 23.
  • the encoding device 10 may be referred to as a video / video encoding device, and the decoding device 20 may be referred to as a video / video decoding device.
  • the transmitter 13 may be included in the encoding device 12.
  • the receiver 21 may be included in the decoding device 22.
  • the renderer 23 may include a display unit, and the display unit may be configured as a separate device or an external component.
  • the video source may acquire a video / image through a capture, synthesis, or generation process of the video / image.
  • the video source may include a video / image capture device and / or a video / image generation device.
  • the video / image capture device may include, for example, one or more cameras, a video / image archive including previously captured video / images, and the like.
  • the video / image generating device may include, for example, a computer, a tablet and a smart phone, and the like (electronically) to generate the video / image.
  • a virtual video / image may be generated through a computer or the like, and in this case, the video / image capture process may be replaced by a process in which related data is generated.
  • the encoding device 12 may encode an input video / image.
  • the encoding apparatus 12 may perform a series of procedures such as prediction, transformation, and quantization for compression and coding efficiency.
  • the encoded data (encoded video / video information) may be output in the form of a bitstream.
  • the transmitting unit 13 may transmit the encoded video / video information or data output in the form of a bitstream to a receiving unit of a receiving device through a digital storage medium or a network in a file or streaming format.
  • Digital storage media include universal serial bus (USB), secure digital (SD), compact disk (CD), digital video disk (DVD), bluray, hard disk drive (HDD), and solid state drive (SSD). It may include various storage media.
  • the transmission unit 13 may include an element for generating a media file through a predetermined file format, and may include an element for transmission through a broadcast / communication network.
  • the receiver 21 may extract the bitstream and transmit it to the decoding device 22.
  • the decoding apparatus 22 may decode a video / image by performing a series of procedures such as inverse quantization, inverse transformation, and prediction corresponding to the operation of the encoding apparatus 12.
  • the renderer 23 may render the decoded video / image.
  • the rendered video / image may be displayed through the display unit.
  • FIG. 2 is an embodiment of the present specification, and shows a schematic block diagram of an encoding device in which encoding of a video / image signal is performed.
  • the encoding device 100 of FIG. 2 may correspond to the encoding device 12 of FIG. 1.
  • the encoding apparatus 100 includes an image segmentation unit 110, a subtraction unit 115, a conversion unit 120, a quantization unit 130, an inverse quantization unit 140, and an inverse conversion unit 150, Includes an adder 155, a filtering unit 160, a decoded picture buffer (DPB) 170, an inter prediction unit 180, an intra prediction unit 185, and an entropy encoding unit 190 Can be.
  • the inter prediction unit 180 and the intra prediction unit 185 may be collectively called a prediction unit. That is, the prediction unit may include an inter prediction unit 180 and an intra prediction unit 185.
  • the transform unit 120, the quantization unit 130, the inverse quantization unit 140, and the inverse transform unit 150 may be included in a residual processing unit.
  • the residual processing unit may further include a subtraction unit 115.
  • the inter prediction unit 180, the intra prediction unit 185 and the entropy encoding unit 190 may be configured by one hardware component (for example, an encoder or processor) according to an embodiment.
  • the decoded picture buffer 170 may be configured by one hardware component (eg, a memory or digital storage medium) according to an embodiment.
  • the image splitter 110 may divide the input image (or picture, frame) input to the encoding apparatus 100 into one or more processing units.
  • the processing unit may be referred to as a coding unit (CU).
  • the coding unit may be recursively divided according to a quad-tree binary-tree (QTBT) structure from a coding tree unit (CTU) or a largest coding unit (LCU).
  • QTBT quad-tree binary-tree
  • CTU coding tree unit
  • LCU largest coding unit
  • one coding unit may be divided into a plurality of coding units of a deeper depth based on a quad tree structure and / or a binary tree structure.
  • a quad tree structure may be applied first, and a binary tree structure may be applied later.
  • a binary tree structure may be applied first.
  • the coding procedure according to the present specification may be performed based on the final coding unit that is no longer split.
  • the maximum coding unit may be directly used as a final coding unit based on coding efficiency according to image characteristics, or the coding unit may be recursively divided into coding units having a lower depth than optimal if necessary.
  • the coding unit of the size of can be used as the final coding unit.
  • the coding procedure may include procedures such as prediction, transformation, and reconstruction, which will be described later.
  • the processing unit may further include a prediction unit (PU) or a transformation unit (TU).
  • the prediction unit and transform unit may be partitioned or partitioned from the above-described final coding unit, respectively.
  • the prediction unit may be a unit of sample prediction
  • the transformation unit may be a unit for deriving a transform coefficient and / or a unit for deriving a residual signal from the transform coefficient.
  • the unit may be used interchangeably with terms such as a block or area depending on the case.
  • the MxN block may represent samples of M columns and N rows or a set of transform coefficients.
  • the sample may generally represent a pixel or a pixel value, and may indicate only a pixel / pixel value of a luma component or only a pixel / pixel value of a saturation component.
  • the sample may be used as a term for one picture (or image) corresponding to a pixel or pel.
  • the encoding apparatus 100 subtracts a prediction signal (a predicted block, a prediction sample array) output from the inter prediction unit 180 or the intra prediction unit 185 from the input image signal (original block, original sample array)
  • a signal residual block, residual sample array
  • a unit that subtracts a prediction signal (a prediction block, a prediction sample array) from an input image signal (original block, original sample array) in the encoding apparatus 100 may be referred to as a subtraction unit 115.
  • the prediction unit may perform prediction on a block to be processed (hereinafter referred to as a current block), and generate a predicted block including prediction samples for the current block.
  • the prediction unit may determine whether intra prediction or inter prediction is applied in units of a current block or CU. As described later in the description of each prediction mode, the prediction unit may generate various information regarding prediction, such as prediction mode information, and transmit it to the entropy encoding unit 190.
  • the prediction information may be encoded by the entropy encoding unit 190 and output in the form of a bitstream.
  • the intra prediction unit 185 may predict the current block by referring to samples in the current picture.
  • the referenced samples may be located in the neighborhood of the current block or may be located apart depending on a prediction mode.
  • prediction modes may include a plurality of non-directional modes and a plurality of directional modes.
  • the non-directional mode may include, for example, a DC mode and a planar mode (planar mode).
  • the directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes depending on the degree of detail of the prediction direction. However, this is an example, and more or less directional prediction modes may be used depending on the setting.
  • the intra prediction unit 185 may determine a prediction mode applied to the current block using a prediction mode applied to neighboring blocks.
  • the inter prediction unit 180 may derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on the reference picture.
  • motion information may be predicted in units of blocks, subblocks, or samples based on the correlation of motion information between a neighboring block and a current block.
  • the motion information may include a motion vector and a reference picture index.
  • the motion information may further include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information.
  • the neighboring block may include a spatial neighboring block existing in the current picture and a temporal neighboring block present in the reference picture.
  • the reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different.
  • the temporal neighboring block may be referred to by a name such as a collocated reference block or a colCU, and a reference picture including a temporal neighboring block may also be called a collocated picture (colPic).
  • the inter prediction unit 180 constructs a motion information candidate list based on neighboring blocks, and generates information indicating which candidates are used to derive a motion vector and / or reference picture index of the current block. can do. Inter prediction may be performed based on various prediction modes. For example, in the case of the skip mode and the merge mode, the inter prediction unit 180 may use motion information of neighboring blocks as motion information of the current block.
  • the residual signal may not be transmitted.
  • a motion vector of a current block is obtained by using a motion vector of a neighboring block as a motion vector predictor and signaling a motion vector difference. I can order.
  • the prediction signal generated by the inter prediction unit 180 or the intra prediction unit 185 may be used to generate a reconstructed signal or may be used to generate a residual signal.
  • the transform unit 120 may generate transform coefficients by applying a transform technique to the residual signal.
  • the transformation technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), a graph-based transform (GBT), or a conditionally non-linear transform (CNT).
  • DCT discrete cosine transform
  • DST discrete sine transform
  • KLT Karhunen-Loeve transform
  • GBT graph-based transform
  • CNT conditionally non-linear transform
  • the transform process may be applied to pixel blocks having the same size of a square, or may be applied to blocks of variable sizes other than squares.
  • the quantization unit 130 quantizes the transform coefficients and transmits them to the entropy encoding unit 190, and the entropy encoding unit 190 encodes a quantized signal (information about quantized transform coefficients) and outputs it as a bitstream. have.
  • Information about the quantized transform coefficients may be referred to as residual information.
  • the quantization unit 130 may rearrange block-type quantized transform coefficients into a one-dimensional vector form based on a coefficient scan order, and the quantized transform based on the one-dimensional vector form quantized transform coefficients Information about coefficients may be generated.
  • the entropy encoding unit 190 may perform various encoding methods such as exponential Golomb (CAVLC), context-adaptive variable length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC).
  • the entropy encoding unit 190 may encode information necessary for video / image reconstruction (eg, values of syntax elements, etc.) together with the quantized transform coefficients together or separately.
  • the encoded information (eg, video / video information) may be transmitted or stored in the unit of a network abstraction layer (NAL) unit in the form of a bitstream.
  • NAL network abstraction layer
  • the bitstream can be transmitted over a network or stored on a digital storage medium.
  • the network may include a broadcasting network and / or a communication network
  • the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD.
  • the signal output from the entropy encoding unit 190 may be configured as an internal / external element of the encoding apparatus 100 by a transmitting unit (not shown) and / or a storing unit (not shown) for storing, or the transmitting unit It may be a component of the entropy encoding unit 190.
  • the quantized transform coefficients output from the quantization unit 130 may be used to generate a prediction signal.
  • the residual signal may be reconstructed by applying inverse quantization and inverse transform to the quantized transform coefficients through the inverse quantization unit 140 and the inverse transform unit 150 in the loop.
  • the adder 155 adds the reconstructed residual signal to the predicted signal output from the inter predictor 180 or the intra predictor 185, so that the reconstructed signal (restored picture, reconstructed block, reconstructed sample array) Can be created. If there is no residual for the block to be processed, such as when the skip mode is applied, the predicted block may be used as a reconstructed block.
  • the adding unit 155 may be called a restoration unit or a restoration block generation unit.
  • the generated reconstructed signal may be used for intra prediction of the next processing target block in the current picture, or may be used for inter prediction of the next picture through filtering as described below.
  • the filtering unit 160 may apply subjective filtering to the reconstructed signal to improve subjective / objective image quality.
  • the filtering unit 160 may generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and may transmit the modified reconstructed picture to the decoded picture buffer 170.
  • Various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, and the like.
  • the filtering unit 160 may generate various information regarding filtering as described later in the description of each filtering method and transmit it to the entropy encoding unit 190.
  • the filtering information may be encoded by the entropy encoding unit 190 and output in the form of a bitstream.
  • the modified reconstructed picture transmitted to the decoded picture buffer 170 may be used as a reference picture in the inter prediction unit 180.
  • inter prediction When the inter prediction is applied through the encoding apparatus 100, prediction mismatches in the encoding apparatus 100 and the decoding apparatus 200 may be avoided, and encoding efficiency may be improved.
  • the decoded picture buffer 170 may store the corrected reconstructed picture for use as a reference picture in the inter prediction unit 180.
  • FIG. 3 is an embodiment of the present specification, and shows a schematic block diagram of a decoding apparatus in which decoding of a video signal is performed.
  • the decoding device 200 of FIG. 3 may correspond to the decoding device 22 of FIG. 1.
  • the entropy decoding unit 210, the inverse quantization unit 220, the inverse transform unit 230, the addition unit 235, the filtering unit 240, the inter prediction unit 260, and the intra prediction unit 265 described above are exemplary embodiments. It may be configured by one hardware component (for example, a decoder or processor). Also, the decoded picture buffer 250 may be implemented by one hardware component (eg, a memory or digital storage medium) according to an embodiment.
  • the decoding apparatus 200 may restore an image in response to a process in which the video / image information is processed by the encoding apparatus 100 of FIG. 2.
  • the decoding apparatus 200 may perform decoding using a processing unit applied by the encoding apparatus 100.
  • the processing unit may be, for example, a coding unit, and the coding unit may be divided according to a quad tree structure and / or a binary tree structure from a coding tree unit or a largest coding unit. Then, the decoded video signal decoded and output through the decoding apparatus 200 may be reproduced through the reproduction apparatus.
  • the decoding apparatus 200 may receive the signal output from the encoding apparatus 100 of FIG. 2 in the form of a bitstream, and the received signal may be decoded through the entropy decoding unit 210.
  • the entropy decoding unit 210 may parse the bitstream to derive information (eg, video / image information) necessary for image reconstruction (or picture reconstruction).
  • the entropy decoding unit 210 decodes information in a bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and quantizes a value of a syntax element required for image reconstruction and a transform coefficient for residual.
  • the CABAC entropy decoding method receives bins corresponding to each syntax element in a bitstream, and decodes syntax information of the decoding target syntax elements and surrounding and decoding target blocks, or symbols / bins decoded in the previous step.
  • the context model is determined using the information of, and the probability of occurrence of the bin is predicted according to the determined context model to perform arithmetic decoding of the bin to generate a symbol corresponding to the value of each syntax element. have.
  • the CABAC entropy decoding method may update the context model using the decoded symbol / bin information for the next symbol / bin context model after determining the context model.
  • a prediction unit inter prediction unit 260 and intra prediction unit 265
  • the entropy decoding unit 210 performs entropy decoding.
  • the dual value that is, quantized transform coefficients and related parameter information may be input to the inverse quantization unit 220.
  • information related to filtering among information decoded by the entropy decoding unit 210 may be provided to the filtering unit 240.
  • a receiving unit (not shown) receiving a signal output from the encoding apparatus 100 may be further configured as an internal / external element of the decoding apparatus 200, or the receiving unit may be a component of the entropy decoding unit 210. It might be.
  • the inverse quantization unit 220 may inverse quantize the quantized transform coefficients to output transform coefficients.
  • the inverse quantization unit 220 may rearrange the quantized transform coefficients in a two-dimensional block form. In this case, reordering may be performed based on the coefficient scan order performed by the encoding apparatus 100.
  • the inverse quantization unit 220 may perform inverse quantization on the quantized transform coefficients by using a quantization parameter (for example, quantization step size information), and obtain transform coefficients.
  • a quantization parameter for example, quantization step size information
  • the inverse transform unit 230 inversely transforms the transform coefficients to obtain a residual signal (residual block, residual sample array).
  • the prediction unit may perform prediction on the current block and generate a predicted block including prediction samples for the current block.
  • the prediction unit may determine whether intra prediction or inter prediction is applied to the current block based on information about prediction output from the entropy decoding unit 210, and may determine a specific intra / inter prediction mode.
  • the inter prediction unit 260 may derive the predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on the reference picture.
  • motion information may be predicted in units of blocks, subblocks, or samples based on the correlation of motion information between a neighboring block and a current block.
  • the motion information may include a motion vector and a reference picture index.
  • the motion information may further include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information.
  • the neighboring block may include a spatial neighboring block existing in the current picture and a temporal neighboring block present in the reference picture.
  • the inter prediction unit 260 may construct a motion information candidate list based on neighboring blocks, and derive a motion vector and / or reference picture index of the current block based on the received candidate selection information.
  • Inter prediction may be performed based on various prediction modes, and information about prediction may include information indicating a mode of inter prediction for a current block.
  • the adding unit 235 adds the obtained residual signal to the prediction signal (predicted block, prediction sample array) output from the inter prediction unit 260 or the intra prediction unit 265, thereby restoring signals (restored pictures, reconstructed blocks). , A reconstructed sample array). If there is no residual for the block to be processed, such as when the skip mode is applied, the predicted block may be used as a reconstructed block.
  • the adding unit 235 may be called a restoration unit or a restoration block generation unit.
  • the generated reconstructed signal may be used for intra prediction of the next processing target block in the current picture, or may be used for inter prediction of the next picture through filtering as described below.
  • the filtering unit 240 may improve subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filtering unit 240 may generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and may transmit the modified reconstructed picture to the decoded picture buffer 250.
  • Various filtering methods may include, for example, deblocking filtering, sample adaptive offset (SAO), adaptive loop filter (ALF), bilateral filter, and the like.
  • the corrected reconstructed picture transmitted to the decoded picture buffer 250 may be used as a reference picture by the inter prediction unit 260.
  • the embodiments described in the filtering unit 160, the inter prediction unit 180, and the intra prediction unit 185 of the encoding device 100 are respectively the filtering unit 240 and the inter prediction unit 260 of the decoding device.
  • the intra prediction unit 265 may be applied to the same or corresponding.
  • FIG. 4 is an embodiment of the present specification, and is a structural diagram of a content streaming system.
  • the content streaming system to which the present specification is applied may largely include an encoding server 410, a streaming server 420, a web server 430, a media storage 440, a user device 450, and a multimedia input device 460. have.
  • the encoding server 410 serves to compress a content input from multimedia input devices such as a smartphone, camera, camcorder, etc. into digital data to generate a bitstream and transmit it to the streaming server 420.
  • multimedia input devices 460 such as a smartphone, camera, and camcorder directly generate a bitstream
  • the encoding server 410 may be omitted.
  • the bitstream may be generated by an encoding method or a bitstream generation method to which the present specification is applied, and the streaming server 420 may temporarily store the bitstream in the process of transmitting or receiving the bitstream.
  • the streaming server 420 transmits multimedia data to the user device 450 based on a user request through the web server 430, and the web server 430 serves as a medium that informs the user of the service.
  • the web server 430 delivers it to the streaming server 420, and the streaming server 420 transmits multimedia data to the user.
  • the content streaming system may include a separate control server, in which case the control server serves to control commands / responses between devices in the content streaming system.
  • Streaming server 420 may receive content from media storage 440 and / or encoding server 410. For example, when content is received from the encoding server 410, the content may be received in real time. In this case, in order to provide a smooth streaming service, the streaming server 420 may store the bitstream for a predetermined time.
  • Each server in the content streaming system can be operated as a distributed server, and in this case, data received from each server can be distributed.
  • FIG 5 shows an example of a multi-type tree structure according to an embodiment of the present specification.
  • FIG. 6 illustrates an example of a signaling mechanism of split information of a quadtree with nested multi-type tree structure accompanying a multitype tree according to an embodiment of the present disclosure.
  • a flag (eg, mtt_split_cu_binary_flag) for indicating whether the division type is binary division or ternary division may be signaled.
  • a flag eg, mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag
  • a multi-type tree splitting mode of a CU may be determined as shown in Table 1 below.
  • FIG. 7 illustrates an example of a method of dividing a CTU into multiple CUs based on a quadtree and nested multi-type tree structure according to an embodiment of the present specification.
  • the CU may correspond to a coding block (CB).
  • the CU may include a coding block of luma samples and two coding blocks of corresponding chroma samples.
  • the size of the CU can be set as much as the size of the CTU.
  • the luma sample unit of the CU may be set to 4x4. For example, in the case of a 4: 2: 0 color format (or chroma format), the maximum chroma CB size may be 64x64 and the minimum chroma CB size may be 2x2.
  • the following parameters may be defined as syntax elements of a slice parameter set (SPS).
  • SPS slice parameter set
  • -CTU size the root size of a quaternary tree
  • the CTU size may be set to 64x64 blocks of 128x128 luma samples and two corresponding chroma samples (4: 2: 0 chroma format).
  • MinQtSize is set to 16x16
  • MaxBtSize is set to 128x1208
  • MaxTtSize is set to 64x64
  • MinBtSize and MinTtSize can be set to 4x4
  • MaxMttDepth can be set to 4.
  • Quadtree leaf nodes may have a size of 16x16 (MinQtSize) to a size of 128x128 (CTU size). If the leaf QT node is 128x128, it may not be additionally divided into a binary tree / ternary tree. In this case, it is because MaxBtSize and MaxTtSize (64x64) are exceeded even if divided. In other cases, leaf QT nodes may be further divided into a multi-type tree structure. If the multi-type tree depth reaches MaxMttdepth (eg, 4), further division may not be considered. If the width of the multitype tree node is equal to MinBtSize and less than or equal to 2xMinTtSize, additional horizontal partitioning may not be considered any more. If the height of the multitype tree node is equal to MinBtSize and less than or equal to 2xMinTtSize, additional vertical splitting may not be considered any more.
  • MinMttdepth MaxMttdepth
  • FIG. 8 shows an example of a case in which ternary tree partitioning according to an embodiment of the present disclosure is limited.
  • TT splitting may be limited in certain cases. For example, if the width or height of the luma coding block is greater than a predetermined value (eg, 32 or 64), the TT segmentation may be limited as illustrated in FIG. 6.
  • a predetermined value eg, 32 or 64
  • the coding tree scheme can support that luma and chroma blocks have separate block tree structures.
  • luma CTBs and chroma CTBs in one CTU have the same coding tree structure. Can be limited.
  • luma and chroma blocks may have a separate block tree structure from each other.
  • the luma CTB may be divided into CUs based on a specific code tree structure, and the chroma CTB may be divided into chroma CUs based on another coding tree structure.
  • the CTU of an I-slice consists of a coding block of luma components and two coding blocks of chroma components
  • a CU of a P-slice or B-slice can be composed of blocks of three color components. do.
  • the quadtree coding tree structure accompanying the multi-type tree has been described through “dividing the tree structure of CTU”, the structure in which the CU is divided is not limited.
  • the BT structure or the TT structure may be interpreted as a concept included in a multiple partitioning tree (MPT) structure, and the CU may be interpreted as being split through a QT structure and an MPT structure.
  • MPT multiple partitioning tree
  • a syntax element for example, MPT_split_type
  • MPT_split_mode a syntax element including information on which direction is split between and horizontal.
  • the CU may be divided by QT structure, BT structure, or TT structure. That is, according to the QT structure, the CU of the lower depth is divided into 1/4 the size of the CU of the upper depth, or the CU of the lower depth is divided into 1/2 the size of the CU of the upper depth, or according to the TT structure Unlike a CU at a lower depth, which is divided into 1/4 or 1/2 the size of a CU at a higher depth, a CU at a lower depth may, in some cases, be 1/5, 1/3, or 3/8 of the CU at a higher depth. , 3/5, 2/3, or 5/8, and the CU is not limited thereto.
  • the tree node block may be restricted such that all samples of all coded CUs are located inside the picture boundaries.
  • the above-described quadtree coding block structure with a multi-type tree can provide a very flexible block partitioning structure.
  • the result of the coding block structure by different splitting patterns may potentially have a coding block structure by splitting types supported in a multitype tree. Accordingly, in order to limit the occurrence of redundant partition patterns, the amount of data for transmission of partition information can be reduced by using only partition types supported in a multi-type tree.
  • continuous binary splitting for one direction of two levels has the same coding block structure as binary splitting for the central portion after ternary splitting.
  • binary tree splitting for the central portion of the ternary tree splitting may be limited.
  • This limitation of segmentation can be applied to CUs of all pictures.
  • the signaling of corresponding syntax elements can be modified to reflect the restriction of such segmentation and reduce the number of bits signaled for segmentation. For example, as illustrated in FIG.
  • the syntax element (mtt_split_su_binary_flag) indicating whether the partition is a binary partition or a ternary partition may not be signaled
  • the decoding apparatus 200 may determine a syntax element (mtt_split_su_binary_flag) indicating whether the division is a binary division or a ternary division to 0.
  • a picture (slice) in which only intra prediction is performed an intra picture or an I-picture (I-slice), one motion vector and a reference index to predict each unit
  • the picture (slice) to be used may be referred to as a predictive picture or a P-picture (P-slice), a picture (slice) using two or more motion vectors and reference indices as a pair prediction picture or a B-picture (B-slice).
  • Inter prediction refers to a prediction method for deriving a sample value of a current block based on data elements (eg, sample values or motion vectors) of a picture other than the current picture. That is, it refers to a method of predicting a sample value of a current block by referring to reconstructed regions of another reconstructed picture other than the current picture.
  • data elements eg, sample values or motion vectors
  • intra prediction (or intra-prediction) will be described in more detail.
  • Intra prediction refers to a prediction method that derives a sample value of a current block from data elements (eg, sample values) of the same decoded picture (or slice). That is, it means a method of predicting the sample value of the current block by referring to the reconstructed regions in the current picture.
  • data elements eg, sample values
  • the intra prediction may represent prediction for generating a prediction sample for the current block based on a reference sample outside the current block in a picture (hereinafter, the current picture) to which the current block belongs.
  • the embodiments of the present specification describe the detailed technique of the prediction method described in FIGS. 2 and 3 above, and the embodiment of the present invention includes the intra prediction-based video / image decoding method of FIG. 12 and the decoding apparatus of FIG. It may correspond to the device of the intra prediction unit 265 in (200). Further, the embodiment of the present invention may correspond to the intra prediction-based video / video decoding method of FIG. 10 described later and the apparatus of the intra prediction unit 185 in the encoding apparatus 100 of FIG. 11.
  • the data encoded by FIGS. 10 and 11 is a memory included in the encoding device 100 or the decoding device 200 in the form of a bitstream, or a memory functionally combined with the encoding device 100 or the decoding device 200 Can be stored in.
  • peripheral reference samples to be used for intra prediction of the current block may be derived.
  • the neighboring reference samples of the current block have a total of 2 x nH samples adjacent to the left boundary and a sample adjacent to the left boundary of the current block of size nW x nH, a total of 2 x nW samples adjacent to the upper boundary and a sample adjacent to the upper boundary of the current block.
  • the number of samples, and one sample of the sum of the current block may be included.
  • the peripheral reference samples of the current block may include upper peripheral samples in a plurality of columns and left peripheral samples in a plurality of rows.
  • the surrounding reference samples of the current block may include samples located on the left or right vertical lines adjacent to the current block and on the upper or lower horizontal lines.
  • the decoding apparatus 200 may construct surrounding reference samples to be used for prediction by substituting samples that are not available with available samples.
  • surrounding reference samples to be used for prediction may be configured through interpolation of available samples. For example, samples located on the vertical line adjacent to the right of the current block and samples located on the horizontal line adjacent to the bottom of the current block are samples located on the left vertical line of the current block and samples located on the horizontal line above the current block. It can be replaced based on or configured through interpolation.
  • prediction samples may be derived based on the average or interpolation of the neighboring reference samples of the current block, and ii) specific to the prediction sample among the neighboring reference samples of the current block (
  • the prediction sample may be derived based on the reference sample present in the prediction) direction.
  • a prediction mode such as i) may be referred to as a non-directional prediction mode or a non-angle prediction mode, and a prediction mode such as ii) may be referred to as a directional prediction mode or an angular prediction mode.
  • a prediction sample through interpolation of a first neighboring sample located in the prediction direction of the intra prediction mode of the current block and a second neighboring sample located in the opposite direction of the prediction direction based on the prediction sample of the current block among the neighboring reference samples This can be generated.
  • a prediction technique based on linear interpolation between a prediction sample based on a prediction sample of a current block and reference samples located in a direction opposite to the prediction direction may be referred to as linear interpolation inter prediction (LIP).
  • the temporary prediction sample of the current block is derived based on the filtered surrounding reference samples, and at least one reference sample derived according to the intra prediction mode among the existing surrounding reference samples, that is, the filtered surrounding reference samples.
  • a prediction sample of the current block may be derived through a weighted sum of temporary prediction samples. Prediction through weighted sum of a plurality of samples may be referred to as position dependent intra prediction (PDPC).
  • PDPC position dependent intra prediction
  • the intra prediction procedure may include an intra prediction mode determination step, a peripheral reference sample derivation step, and an intra prediction mode-based prediction sample derivation step, and may include a post-processing filtering step for the predicted sample derived as needed. have.
  • FIGS. 10 and 11 illustrate an example of an intra prediction based video encoding method according to an embodiment of the present specification and an intra prediction unit in an encoding apparatus according to an embodiment of the present invention.
  • step S1010 may be performed by the intra prediction unit 185 of the encoding apparatus 100, and steps S1020 and S1030 may be performed by the residual processing unit.
  • step S1020 may be performed by the subtraction unit 115 of the encoding apparatus 100
  • step S1030 may include residual information derived by the residual processing unit and prediction information derived by the intra prediction unit 185. It can be performed by the entropy encoding unit 190 using.
  • the residual information is information about residual samples, and may include information about quantized transform coefficients for residual samples.
  • residual samples may be derived as transform coefficients through the transform unit 120 of the encoding apparatus 100, and the derived transform coefficients may be derived as quantized transform coefficients through the quantization unit 130.
  • Information about the quantized transform coefficients may be encoded in the entropy encoding unit 190 through a residual coding procedure.
  • the encoding apparatus 100 may perform intra prediction on the current block.
  • the encoding apparatus 100 determines an intra prediction mode for the current block, derives neighbor reference samples of the current block, and generates prediction samples in the current block based on the intra prediction mode and neighbor reference samples.
  • the procedure for determining the intra prediction mode, deriving the peripheral reference sample, and generating the prediction samples may be performed simultaneously or sequentially.
  • the intra prediction unit 185 of the encoding apparatus 100 may include a prediction mode determination unit 186, a reference sample derivation unit 187, and a prediction sample generation unit 188, and the prediction mode determination unit 186 determines an intra prediction mode for the current block, the reference sample derivation unit 187 derives a neighboring reference sample of the current block, and the prediction sample generator 188 derives a motion sample of the current block. have.
  • the intra prediction unit 185 may further include a prediction sample filter unit (not shown).
  • the encoding apparatus 100 may determine a prediction mode to be applied to a current block among a plurality of intra prediction modes.
  • the encoding apparatus 100 may compare a rate-distortion cost (RD cost) for intra prediction modes and determine an optimal intra prediction mode for the current block.
  • RD cost rate-distortion cost
  • the encoding apparatus 100 may perform prediction sample filtering. Predictive sample filtering may be referred to as post filtering. Filtering on some or all of the prediction samples may be performed by the prediction sample filtering procedure. In some cases, prediction sample filtering may be omitted.
  • the encoding apparatus 100 may generate a residual sample for the current block based on the (filtered) prediction sample. Thereafter, in step S1030, the encoder 100 may encode video data including prediction mode information including an intra prediction mode and information on residual samples.
  • the encoded video data can be output in the form of a bitstream.
  • the output bitstream may be delivered to the decoding device 200 through a storage medium or network.
  • the encoding apparatus 100 may generate a reconstructed picture including reconstructed samples and reconstructed blocks based on reference samples and residual samples. Derivation of the reconstructed picture by the encoding apparatus 100 is to derive the same prediction result as that performed by the decoding apparatus 200 in the encoding apparatus 100, thereby improving coding efficiency. Furthermore, subsequent procedures such as in-loop filtering may be performed on the reconstructed picture.
  • FIGS. 12 and 13 illustrate an intra prediction-based video / video decoding method according to an embodiment of the present specification and an example of an intra prediction unit in a decoder according to an embodiment of the present invention.
  • the decoding apparatus 200 may perform an operation corresponding to an operation performed by the encoding apparatus 100.
  • the decoding apparatus 200 may derive a prediction sample by performing prediction on the current block based on the received prediction information.
  • the decoding apparatus 200 may determine an intra prediction mode for the current block based on the prediction mode information obtained from the encoding apparatus 100.
  • the decoding apparatus 200 may derive a peripheral reference sample of the current block.
  • the decoding apparatus 200 may generate a prediction sample in the current block based on the intra prediction mode and surrounding reference samples.
  • the decoding apparatus 200 may perform a prediction sample filtering procedure, and the prediction sample filtering procedure may be referred to as post filtering. Some or all of the prediction samples may be filtered by the prediction sample filtering procedure. In some cases, the prediction sample filtering procedure may be omitted.
  • the decoding apparatus 200 may generate a residual sample based on residual information obtained from the encoding apparatus 100.
  • the decoding apparatus 200 may generate a reconstruction sample for the current block based on the (filtered) prediction samples and residual samples, and generate a reconstruction picture using the generated reconstruction samples.
  • the prediction mode information used for prediction may include a flag (eg, prev_intra_luma_pred_flag) to indicate whether the most probable mode (MPM) is applied to the current block or whether the remaining mode is applied.
  • the prediction mode information may further include an index (mpm_idx) indicating one of the intra prediction mode candidates (MPM candidates).
  • MPM candidates may be configured as an MPM candidate list or an MPM list.
  • the prediction mode information further includes remaining mode information (eg, rem_intra_luma_pred_mpde) indicating one of the remaining intra prediction modes except the intra prediction mode candidates (MPM candidates). You can.
  • the decoding apparatus 200 may determine the intra prediction mode of the current block based on the prediction information.
  • Prediction mode information may be encoded and decoded through a coding method described below.
  • prediction mode information may be encoded or decoded through entropy coding (eg, CABAC, CAVLC) based on truncated binary code.
  • the intra prediction mode may include two non-directional intra prediction modes and 33 directional intra prediction modes.
  • the non-directional intra prediction modes may include a planar intra prediction mode and a DC intra prediction mode, and the directional intra prediction mode may include intra prediction modes 2 to 34.
  • the planner intra prediction mode may be referred to as a planner mode, and the DC intra prediction mode may be referred to as a DC mode.
  • non-directional intra prediction modes include a planner mode and a DC mode
  • directional intra prediction modes may include intra prediction modes 2 to 66.
  • the extended directional intra prediction may be applied to blocks of all sizes, and may be applied to both luma and chroma components.
  • the intra prediction mode may include two non-directional intra prediction modes and 129 directional intra prediction modes.
  • the non-directional intra prediction modes may include a planner mode and a DC mode
  • the directional intra prediction modes may include intra prediction modes 2 to 130.
  • a current block to be coded and a neighboring block may have similar image characteristics. Therefore, there is a high probability that the current block and the neighboring blocks have the same or similar intra prediction mode. Accordingly, the encoding apparatus 100 may use the intra prediction mode of the neighboring blocks to encode the intra prediction mode of the current block.
  • the encoding apparatus 100 may construct an MPM list for the current block.
  • the MPM list may be referred to as an MPM candidate list.
  • MPM refers to a mode used to improve coding efficiency in consideration of the similarity between a current block and a neighboring block when coding an intra prediction mode.
  • a method of constructing an MPM list including three MPMs may be used to keep the complexity of MPM list generation low.
  • a re-maining mode may be used.
  • the re-maining mode includes 64 remaining candidates, and re-forming intra prediction mode information indicating one of the 64 remaining candidates may be signaled.
  • the re-maining intra prediction mode information may include a 6-bit syntax element (eg, a rem_intra_luma_pred_mode syntax element).
  • FIG. 16 shows an example of a flowchart for constructing a list of most probable mode (MPM) candidates according to an embodiment of the present specification.
  • the encoding apparatus 100 may check or derive the prediction mode of the neighboring block (S1610). For example, the encoding apparatus 100 may determine the prediction mode of the current block based on the prediction mode of the left neighboring block and the prediction mode of the upper neighboring block, and may determine the prediction mode of the corresponding neighboring block as MPM. Determining the MPM can be expressed as a list up of MPM candidates (or MPM lists).
  • the encoding apparatus 100 may check whether the prediction mode of the left neighboring block and the prediction mode of the upper neighboring block are the same (S1620).
  • the initial MPM list may be formed by a pruning process for intra prediction modes of two adjacent blocks.
  • the first MPM may be set as the left neighboring mode
  • the second MPM may be set as the predicting mode of the upper neighboring block
  • the third MPM may be an intra planner mode.
  • Intra DC mode, or an intra vertical mode (intra prediction mode No. 50) may be set (S1630).
  • the two intra prediction modes may be set to MPM, and one of the default intra modes may be added to the MPM list after a pruning check by MPMs. Can be added.
  • the default intra modes may include an intra planner mode, an intra DC mode, and / or an intra vertical mode (intra prediction mode # 50).
  • the MPM list may be configured by the method shown in Table 2 below.
  • A is the intra prediction mode of the left neighboring block
  • B is the intra prediction mode of the upper neighboring block
  • 0 is the intra planner mode
  • 1 is the intra DC mode
  • 50 is the intra vertical mode
  • candModeList represents the MPM list.
  • the encoding apparatus 100 determines whether the prediction mode of the upper neighboring block is less than 2 (S1640).
  • the first MPM may be set as the prediction mode of the left neighboring block
  • the second MPM may be set as (the prediction mode of the left neighboring block-1)
  • the third The MPM may be set to (the prediction mode + 1 of the left neighboring block) (S1660).
  • an additional pruning process may be performed to remove duplicate modes so that only unique modes can be included in the MPM list.
  • a 6-bit fixed length code can be used for entropy coding of 64 non-MPM modes except 3 MPM. That is, an index representing 64 non-MPM modes may be entropy coded with a 6-bit fixed length code (6-bit FLC).
  • the encoding apparatus 100 may determine whether the optimal intra prediction mode to be applied to the current block belongs to the previously configured MPM candidate.
  • the encoding apparatus 100 may encode the intra prediction mode of the current block.
  • 17A and 17B show examples of reference samples for a wide-angle intra prediction mode according to an embodiment of the present specification.
  • 2W + 1 length upper reference samples and 2H + 1 length left reference samples may be defined to support the wide-angle prediction direction.
  • intra-prediction modes that are replaced by wide-angle intra prediction modes may be different according to an aspect ratio of the current block.
  • Intra prediction modes, which are replaced by wide-angle intra prediction modes according to the aspect ratio may be defined as shown in Table 4 below.
  • peripheral reference samples used for intra prediction of the current block may be derived.
  • the neighboring reference samples of the current block are nWxnH-sized samples adjacent to the left boundary and a total of 2xnH samples adjacent to the bottom-left, samples adjacent to the upper boundary of the current block, and top-right ), And a total of 2xnW samples and one sample neighboring the top-left of the current block.
  • the peripheral reference samples of the current block may include multiple columns of upper peripheral samples and multiple rows of left peripheral samples.
  • the neighboring reference samples of the current block include nHxH total samples adjacent to the right boundary of the current block, nW samples adjacent to the lower boundary of the current block, and 1 sample neighboring the lower right side of the current block. It may include.
  • the decoding apparatus 200 may configure surrounding reference samples to be used for prediction through extrapolation of available samples. It can be configured by updating the referenceable sample to the latest sample, starting at the bottom left and reaching the top right reference sample, replacing the sample that has not yet been decoded or not available with the last available sample. have.
  • the prediction unit of the encoding device 100 / decoding device 200 may derive a reference sample according to the intra prediction mode of the current block among neighboring reference samples of the current block, and the derived reference A prediction sample of the current block may be generated based on the sample.
  • the coding apparatus may (i) derive a prediction sample based on an average or interpolation of neighboring reference samples of the current block, and (ii) peripheral reference samples of the current block
  • the prediction sample may be derived based on a reference sample existing in a specific direction with respect to the medium prediction sample.
  • it may be referred to as a non-directional mode or non-angle mode, and in the case of (ii), a directional mode or an angular mode.
  • a prediction sample may be generated through interpolation between a first neighboring sample and a second neighboring sample located in a direction opposite to the prediction direction of the intra prediction mode of the current block based on the prediction sample of the current block among the neighboring reference samples.
  • the prediction method using linear interpolation may be referred to as linear interpolation intra prediction (LIP).
  • the coding device derives a temporary prediction sample of the current block based on the filtered surrounding reference samples, and at least one derived according to the intra prediction mode among existing surrounding reference samples, that is, unfiltered surrounding reference samples.
  • Prediction samples of the current block may be derived by weighting a reference sample of and a temporary reference sample. This weighted polymerization-based intra prediction method may be referred to as position dependent intra prediction (PDPC).
  • PDPC position dependent intra prediction
  • the encoding apparatus 100 may derive a prediction sample by selecting a reference sample having the highest prediction accuracy among neighboring multiple reference samples of the current block, and perform intra prediction encoding by transmitting a reference sample used at this time.
  • an intra prediction method using multiple reference samples may be referred to as multi-reference line intra prediction (MRL).
  • LIP Linear interpolation intra prediction
  • a prediction sample may be generated through interpolation between the first surrounding sample and the second surrounding sample among the surrounding samples. That is, a prediction sample may be generated through interpolation between a second neighboring sample and a first neighboring sample located in a direction opposite to the prediction direction of the intra prediction mode of the current block based on the prediction sample of the current block among the neighboring samples.
  • FIG. 19 shows examples of neighboring samples for linear interpolation prediction according to an embodiment of the present disclosure
  • FIG. 20 shows an example of a method for performing linear interpolation prediction.
  • a right sample buffer and a lower sample buffer may be generated as described above.
  • the coding apparatus generates a bottom right sample (BR) using a peripheral reference sample to generate a right sample buffer and a bottom sample buffer.
  • FIG. 19 shows a method of generating a bottom right sample using a top right sample and a bottom left sample
  • FIG. 20 shows the most top right sample and the most left bottom (most) as much as twice the length of the current block to be coded. Shows how to create the bottom right sample using the bottom left) sample.
  • the lower right sample may be generated using another sample by Equation 1 or Equation 2 below.
  • the lower right sample may be generated through various methods in addition to the above Equation 1 or Equation 2.
  • the coding apparatus After the creation of the bottom right sample, the coding apparatus generates bottom samples (bottom buffer) and right samples (right buffer) using the bottom left sample and the top right sample.
  • 21A and 21B show an example of a method for generating bottom samples and right samples according to an embodiment of the present specification.
  • the lower samples may be generated through linear interpolation of the lower left sample and the lower right sample, and the lower right samples may be generated through linear interpolation of the upper right sample and lower right sample.
  • various weighting values may be used in a method of generating a bottom sample using a bottom left sample and a bottom right sample or a method of generating a right sample using a top right sample and a bottom right sample.
  • FIG. 22 shows an example of a method for generating bottom samples and right samples according to an embodiment of the present specification.
  • the coding apparatus may perform interpolation prediction by using the bottom samples and right samples generated after generation of the bottom samples and right samples.
  • a method of generating a prediction sample C using linear interpolation intra prediction may be as follows. In FIG. 20, an example of a case in which a vertical mode mode in which the prediction mode has positive directionality is applied will be described.
  • the coding apparatus may generate prediction values by applying methods 2) to 4) to all samples in a block to be encoded.
  • the linear interpolation intra prediction method can be applied to all directional prediction modes.
  • the PDPC derives filtered reference samples using a filter predefined for the PDPC, derives a temporary prediction sample of the current block based on the intra prediction mode of the current block and the filtered reference samples, and the existing reference sample It may mean an intra prediction method of deriving a prediction sample of the current block by weighting the (ie, at least one reference sample derived according to the intra prediction mode among the unfiltered reference samples) and the temporary prediction sample.
  • the predefined filter may be one of five 7-tap filters.
  • the predefined filter may be one of a 3-tap filter, a 5-tap filter, or a 7-tap filter.
  • the three-tap filter, the five-tap filter, and the seven-tap filter may represent filters having three filter coefficients, filters having five filter coefficients, and filters having seven filter coefficients, respectively.
  • the prediction result of the intra planner mode may be further corrected by PDPC.
  • the PDPC is an intra-planar mode, an intra DC mode, a horizontal intra prediction mode, a vertical intra prediction mode, an intra prediction mode in the lower left direction (ie, intra prediction mode 2), and a left without additional signaling. It may be applied to eight directional intra prediction modes adjacent to the lower direction intra prediction mode, eight directional intra prediction modes adjacent to the upper right direction and eight directional intra prediction modes adjacent to the upper right direction intra prediction mode.
  • a prediction sample of (x, y) coordinates predicted based on the intra prediction mode and a linear combination of reference samples may be derived by the following Equation (4).
  • intra DC mode when the PDPC is applied to the intra planar mode, intra DC mode, horizontal intra prediction mode and vertical intra prediction mode, for example, a DC mode boundary filter of high efficiency video coding (HEVC) or a vertical / horizontal mode edge filter Additional boundary filters may not be required.
  • HEVC high efficiency video coding
  • vertical / horizontal mode edge filter Additional boundary filters may not be required.
  • 23 shows an example of reference samples defined in a PDPC according to an embodiment of the present specification.
  • 23A to 23D reference samples (R x, -1 , R 1, y , and R -1, -1 ) defined in PDPC applied to various prediction modes are illustrated.
  • the weights of the PDPC can be derived based on prediction modes.
  • PDPC weights can be derived as shown in Table 5 below.
  • the coding apparatus According to the intra prediction combination according to the location, the coding apparatus generates a prediction sample using the reference sample according to the prediction mode, and then improves (updates) the prediction sample using the surrounding reference samples.
  • PDPC is based on the 65 directional intra prediction modes in FIG. 15, planner, DC, 2 (lower right direction mode), VDIA (upper left direction mode), Hor (horizontal direction mode), Ver (vertical direction mode), 2 mode around the mode (3 to 10 mode), VDIA mode of the peripheral mode (58 to 65 mode) can be limitedly applied.
  • PDPC is not applied to all prediction samples in a current block to be coded, and may be variably applied to some predictions in a current block in consideration of a block size.
  • FIG 24 illustrates an example of reference lines for applying multiple reference line prediction according to an embodiment of the present specification.
  • MRL extends conventional prediction to use neighbor samples with one or more (eg, 1 to 3) sample distances to the left and top of the current prediction block.
  • Conventional direct neighboring reference sample lines and extended reference lines are shown in FIG. 24.
  • mrl_idx indicates which line is used for intra prediction of the CU in connection with intra prediction modes (eg, directional or non-directional prediction modes).
  • an interpolation filter for interpolation may be derived through various methods.
  • a filter for relaxing the block boundary may be applied to reduce the error between the prediction samples of the current block and the neighboring already reconstructed samples. For example, whether the filter is applied and the filter type may be determined according to the predicted mode and the size of the block.
  • intra prediction When intra prediction is performed on the current block, prediction for the luma component block (luma block) of the current block and prediction for the chroma component block (chroma block) may be performed, in this case, for the chroma component (chroma block)
  • the intra prediction mode may be set separately from the intra prediction mode for the luma component (luma block).
  • DM and LM are dependent intra prediction modes for predicting chroma blocks using information of luma blocks.
  • the DM may indicate a mode in which the same intra prediction mode as the intra prediction mode for the luma component is applied as the intra prediction mode for the chroma component.
  • the LM subsamples the reconstructed samples of the luma block, and then applies the at least one LM parameter to the subsampled samples to predict the samples derived from the chroma block. It may indicate an intra prediction mode to be used.
  • FIG. 25 shows an example of a multiple direct mode according to an embodiment of the present specification.
  • MDM multiple direct mode
  • MDM uses an existing single mode, DM mode, by expanding to multiple modes. That is, when configuring the intra prediction mode of the chroma image, a plurality of DM modes may be selected as follows.
  • 26 shows an example of the location of samples for deriving a linear model parameter according to an embodiment of the present specification.
  • a cross-component linear model (CCLM) mode may be applied to the current chroma block.
  • the CCLM mode is an intra prediction mode using correlation between a luma block and a chroma block corresponding to the luma block, and a linear model may be derived based on the peripheral samples of the luma block and the peripheral samples of the chroma block, A mode in which prediction samples of the chroma block is derived may be represented based on the derived linear model and reconstructed samples of the luma block.
  • parameters for the linear model based on neighboring samples used for intra prediction of the current chroma block and neighboring samples used for intra prediction of the current chroma block can be derived.
  • the linear model may be expressed based on Equation 5 below.
  • predc (i, j) may represent a prediction sample of (i, j) coordinates of the current chroma block
  • recL '(i, j) represents a reconstruction sample of (i, j) coordinates of the current luma block. You can.
  • recL '(i, j) may indicate a down-sampled reconstructed sample of the current luma block.
  • parameter And parameters May be derived based on neighboring samples used for intra prediction of the current luma block and neighboring samples used for intra prediction of the current chroma block.
  • parameter And parameters Can be derived based on the following equations (6) and (7).
  • L (n) may represent upper peripheral samples and left peripheral samples of the current luma block
  • C (n) represents upper peripheral samples and left peripheral samples of the current chroma block. You can.
  • L (n) may represent down-sampled upper peripheral samples and left peripheral samples of the current luma block.
  • N may represent a value that is twice the smaller of the width and height of the current chroma block.
  • a total of six intra prediction modes may be allowed for the chroma intra mode coding.
  • the six intra prediction modes may include five existing intra prediction modes and one CCLM.
  • Table 6 below may indicate chroma intra prediction mode information and derivation processes.
  • FIG. 27 shows an example of a decoding process for a block to which MRL intra prediction is applied according to an embodiment of the present specification.
  • the process illustrated in FIG. 27 may be performed by the intra prediction unit 265 of the decoding device 200.
  • the decoding apparatus 200 may parse an index (MRL index) (mrl_idx) indicating the reference sample line first in the CU level (S2710). Thereafter, the decoding apparatus 200 parses the MPM flag (intra_luma_mpm_flag) indicating whether the MPM is used (S2720). If MPM is used for the current block (that is, intra_luma_mpm_flag is 1), the decoding apparatus 200 may parse the MPM index (intra_luma_mpm_idx) (S2730). If MPM is not used (that is, the value of intra_luma_mpm_flag is 0), the decoding apparatus 200 parses the intra mode information (S2740).
  • MRL index MRL index
  • mrl_idx the reference sample line first in the CU level
  • the intra mode information may indicate an intra prediction mode used for the current block among the remodeling intra prediction modes not included in the MPM list.
  • Intra-mode information may be called re-maining mode information, which may be indicated as intra_luma_mpm_remainder.
  • FIG. 28 illustrates another example of a decoding process for a block to which MRL intra prediction is applied according to an embodiment of the present specification.
  • the process illustrated in FIG. 28 may be performed by the intra prediction unit 265 of the decoding device 200.
  • the coding apparatus can limitly apply MRL to candidate modes generated in the MPM list without applying MRL to all intra prediction modes. That is, the MRL may be restricted to a candidate mode in the MPM list.
  • MRL intra prediction method since the decoding apparatus 200 checks reference sample line information (MRL index) (mrl_idx) at all CU levels, encoding waste may occur accordingly. Therefore, the embodiment of the present specification provides a method of checking the reference sample line (mrl_idx) information after checking the MPM flag (intra_luma_mpm_flag) based on the fact that the MRL is limitedly applied in the candidate mode of the MPM list.
  • the decoding apparatus 200 first parses the MPM flag (intra_luma_mpm_flag) (S2710). If MPM is applied (that is, the value of intra_luma_mpm_flag is 1), the decoding apparatus 200 parses the reference sample line information (mrl_idx) (S2820) and parses the MPM index information (intra_luma_mpm_idx) (S2830). Although the MRL index (mrl_idx) in FIG.
  • MRL index (mrl_idx) may be parsed later than the MPM index, and may be parsed at the same time as one syntactic element. It might be.
  • the decoding apparatus 200 may parse the intra-mode information (intra_luma_mpm_remainder) immediately without re-checking the reference sample line information (mrl_idx). (S2840). Through this method, bits unnecessarily allocated to reference sample line information can be reduced, signaling of intra prediction related information can be optimized, and overall intra coding efficiency can be improved.
  • the MPM flags (intra_luma_mpm_flag), MPM index information (intra_luma_mpm_idx), reference sample line information (MRL index) (mrl_idx), and (remodeling) intra mode information (intra_luma_mpm_remainder) according to an embodiment of the present specification are shown in the following Table 7 Likewise, it may be configured to be included in the CU syntax at the CU level. CU syntax may be encoded by the encoding device 100 and included in the bitstream.
  • the order of each syntax element may correspond to the signaling / parsing order.
  • the MRL index (mrl_idx) is shown to be configured to be parsed earlier than the MPM index (intra_luma_mpm_idx) in Table 7, as described above, this is an example, and the MRL index (mrl_idx) is parsed later than the MPM index (intra_luma_mpm_idx). It may be configured, or may be configured to be parsed simultaneously as one syntax element.
  • FIG. 29 shows an example of an encoding process for a block to which MRL intra prediction is applied according to an embodiment of the present specification.
  • the operations of FIG. 29 may be performed by the intra prediction unit 185 of the encoding device 100.
  • the encoding apparatus 100 may generate / configure the MPM flag (intra_luma_mpm_flag) and then generate / configure the reference sample line information (mrl_idx). Specifically, as illustrated in FIG. 29, the encoding apparatus 100 first generates / configures an MPM flag (intra_luma_mpm_flag) (S2910) and then determines whether MPM is applied (S2920). If MPM is applied (that is, the value of intra_luma_mpm_flag is 1), reference sample line information (mrl_idx) information is generated / configured (S2930) and MPM index information (intra_luma_mpm_idx) is generated / configured (S2940).
  • FIG. 30 shows an example of a flowchart for performing intra prediction according to an embodiment of the present specification. Each process of FIG. 30 may be performed by the intra prediction unit 185 of the encoding device 100 or the inter prediction unit 265 of the decoding device 200.
  • step S3020 when the MPM is applied, the coding device identifies a reference line index and an MPM index for prediction of the current block. For example, if the MPM flag is 1, the coding device may confirm that the MPM is applied, and check the reference line index (eg, mrl_idx or intra_luma_ref_idx) and MPM index (eg, intra_luma_mpm_idx).
  • the reference line index eg, mrl_idx or intra_luma_ref_idx
  • MPM index eg, intra_luma_mpm_idx
  • the reference line index is information indicating a line (row or column) in which the reference sample used for prediction of the current block is located, spaced apart by a predetermined sample distance as well as a line directly adjacent to the left or upper boundary of the current block. You can indicate the line. For example, if the reference line index is 0, a line spaced from the current block by 0 sample distance (directly adjacent) is indicated as a reference line, and if the reference line index is 1, a line spaced 1 sample distance from the current block is referenced. If the reference line index is 2 and the reference line index is 2, a line spaced by 3 sample distances from the current block may be indicated as the reference line.
  • the MPM index may be information indicating an intra prediction mode used for prediction of a current block among MPM lists configured for a current block when MPM is applied.
  • the coding device may check the MPM index (intra_luma_mpm_idx) after parsing the reference line index (mrl_idx) as shown in Table 7, but conversely, parse the reference line index (mrl_idx) after first checking the MPM index (intra_luma_mpm_idx). can do.
  • reference line index (mrl_idx) and the MPM index (intra_luma_mpm_idx) may be parsed simultaneously as one syntax element.
  • step S3030 the coding apparatus generates a prediction sample of the current block from the reference sample of the reference line indicated by the reference line index based on the prediction mode indicated by the MPM index.
  • the decoding device parses the remodeling intra mode information (intra_luma_mpm_remainder), and a prediction sample of the current block based on the remodeling intra mode information (intra_luma_mpm_remainder) You can create the remodeling intra mode information (intra_luma_mpm_remainder), and a prediction sample of the current block based on the remodeling intra mode information (intra_luma_mpm_remainder)
  • FIG. 31 shows an example of a block diagram of an apparatus for processing an image signal according to an embodiment of the present specification.
  • the video signal processing device of FIG. 36 may correspond to the encoding device 100 of FIG. 2 or the decoding device 200 of FIG. 3.
  • the image processing apparatus 3600 for processing an image signal includes a memory 3620 storing an image signal and a processor 3610 processing an image signal while being combined with the memory.
  • the processor 3610 may be configured with at least one processing circuit for processing an image signal, and may process an image signal by executing instructions for encoding or decoding the image signal. That is, the processor 3610 may encode the original image data or decode the encoded image signal by executing the above-described encoding or decoding methods.
  • the processor 3110 may parse the remodeling intra mode information and generate a prediction sample of the current block based on the remodeling intra mode information.
  • the MPM flag, the reference line index, the MPM index, and the remodeling intra-mode information may be included in coding unit syntax and transmitted from an encoder.
  • the processor 3110 may parse the MPM index after parsing the reference line index, or parse the reference line index after parsing the MPM index.
  • the reference line index and the MPM index may be parsed simultaneously as one syntax element.
  • the reference line index may indicate the location of the surrounding reference sample line within a predetermined sample distance to the left or upper side of the current block used for prediction of the current block. Also, the reference line index may indicate a reference sample line spaced by a distance of 1, 2, or 4 samples of the current block.
  • encoded information eg, encoded video / image information
  • the encoded information may be transmitted or stored in units of network abstraction layer (NAL) units in the form of a bitstream.
  • NAL network abstraction layer
  • the bitstream can be transmitted over a network, or it can be stored in a non-transitory digital storage medium.
  • the bitstream is not directly transmitted from the encoding device to the decoding device, but may be streamed / downloaded through an external server (ex. Content streaming server).
  • the network may include a broadcasting network and / or a communication network
  • the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD.
  • the embodiments described herein may be implemented and implemented on a processor, microprocessor, controller, or chip.
  • the functional units illustrated in each drawing may be implemented and implemented on a computer, processor, microprocessor, controller, or chip.
  • embodiments of the present specification may be implemented as a computer program product using program codes, and the program codes may be executed on a computer by the embodiments of the present specification.
  • the program code can be stored on a computer readable carrier.
  • the decoding device and the encoding device to which the present specification is applied may be included in a digital device.
  • digital device includes, for example, all digital devices capable of performing at least one of transmission, reception, processing, and output of data, content, and services.
  • the processing of the data, content, service, etc. by the digital device includes an operation of encoding and / or decoding data, content, service, and the like.
  • These digital devices are paired or connected (hereinafter referred to as 'pairing') with other digital devices, external servers, etc. through a wire / wireless network to transmit and receive data. Convert accordingly.
  • Digital devices include, for example, standing devices such as network TV, HBBTV (Hybrid Broadcast Broadband TV), smart TV (Smart TV), IPTV (internet protocol television), and PC (Personal Computer). , PDA (Personal Digital Assistant), smart phones (Smart Phone), tablet PCs (Tablet PC), notebooks, mobile devices (mobile devices or handheld devices).
  • a digital TV is illustrated in FIG. 33 and a mobile device in FIG. 34, which will be described later, for convenience.
  • wired / wireless network refers to a communication network that supports various communication standards or protocols for interconnection and / or data transmission and reception between digital devices or digital devices and external servers.
  • a wired / wireless network may include both a communication network to be supported in the current or future by a standard and a communication protocol therefor, for example, Universal Serial Bus (USB), Composite Video Banking Sync (CVBS), component, S-Video (Analog), communication standards or protocols for wired connections such as DVI (Digital Visual Interface), HDMI (High Definition Multimedia Interface), RGB, D-SUB, Bluetooth, Radio Frequency Identification (RFID), infrared communication (Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, Digital Living Network Alliance (DLNA), Wireless LAN (WLAN) (Wi-Fi), Wireless broadband (Wibro), World Interoperability for Microwave (Wimax) Access), HSDPA (High Speed Downlink Packet Access), LTE (Long Term Evolution
  • a digital device in the case of merely referring to a digital device in the present specification, it may mean a fixed device or a mobile device or include both depending on context.
  • the digital device is an intelligent device that supports, for example, a broadcast reception function, a computer function or support, and at least one external input, e-mail, web browsing through a wired / wireless network described above ( It can support web browsing, banking, games, and applications.
  • the digital device may include an interface for supporting at least one input or control means (hereinafter referred to as an input means) such as a handwritten input device, a touch screen, and a space remote control.
  • the digital device can use a standardized general-purpose operating system (OS). For example, a digital device can add, delete, modify, and update various applications on a general-purpose OS kernel. You can configure and provide a user-friendly environment.
  • OS general-purpose operating system
  • the external input described in this specification includes external input devices, that is, all input means or digital devices connected to the above-mentioned digital devices by wire / wireless and capable of transmitting / receiving related data through them.
  • the external input is, for example, a high-definition multimedia interface (HDMI), a game device such as a play station or an X-box, a smart phone, a tablet PC, a printer, a digital digital device such as a smart TV. Includes all devices.
  • HDMI high-definition multimedia interface
  • server described in this specification means a client (client), that is, includes all digital devices or systems that supply data to the above-described digital device, referred to as a processor (processor) Also.
  • a server include a web page or a portal server that provides web content, an advertising server that provides advertising data, a content server that provides content, and a social media server (SNS).
  • Social Network Service may include an SNS server providing a service, a service server provided by a manufacturer, or a manufacturing server.
  • channel (channel) refers to a path (path), means (means), etc. for transmitting and receiving data, for example, a broadcasting channel (broadcasting channel).
  • the broadcast channel is expressed in terms of a physical channel, a virtual channel, and a logical channel according to the activation of digital broadcasting.
  • a broadcast channel can be called a broadcast network.
  • a broadcast channel refers to a channel for providing or accessing broadcast content provided by a broadcasting station, and the broadcast content is mainly based on real-time broadcasting and is also called a live channel.
  • live channels include not only real-time broadcasting, but also non-real-time broadcasting in some cases. It may be understood as a term meaning the entire channel.
  • arbitrary channel is further defined in relation to a channel in addition to the above-described broadcast channel.
  • the arbitrary channel may be provided together with a service guide such as an electronic program guide (EPG) along with a broadcast channel, and a service guide, a GUI (Graphic User Interface) or an OSD screen (On-Screen Display) with only a random channel. screen).
  • EPG electronic program guide
  • GUI Graphic User Interface
  • OSD screen On-Screen Display
  • a random channel is a channel randomly allocated by a receiver, and a channel number that is not basically overlapped with a channel number for expressing the broadcast channel is allocated.
  • the receiver receives a broadcast signal that transmits broadcast content and signaling information therefor through the tuned channel.
  • the receiver parses channel information from the signaling information, and configures a channel browser, an EPG, and the like based on the parsed channel information and provides it to the user.
  • the receiver responds accordingly.
  • the broadcast channel is a content previously promised between the transmitting and receiving terminals
  • a random channel when a random channel is repeatedly allocated with the broadcast channel, it may cause confusion or confusion of the user, so it is preferable not to repeatedly allocate as described above.
  • the random channel number is not duplicated with the broadcast channel number as described above, there is still a possibility of confusion in the channel surfing process of the user, and it is required to allocate the random channel number in consideration of this.
  • any channel according to the present specification can also be implemented to be accessed as a broadcast channel in the same manner in response to a user's request to switch a channel through an input means in the same way as a conventional broadcast channel.
  • the random channel number is a type in which characters are written in parallel, such as random channel-1, random channel-2, and the like, for the convenience of discrimination or identification of the user's random channel access and broadcast channel number. It can be defined and marked as.
  • the display of an arbitrary channel number may be implemented by recognizing in a numeric form, such as the number of the broadcasting channel, in a form in which characters are written in parallel as in random channel-1.
  • an arbitrary channel number may be provided in a numeric form, such as a broadcast channel, and a channel number may be defined and displayed in various ways distinguishable from a broadcast channel, such as video channel-1, title-1, and video-1. have.
  • the digital device executes a web browser for a web service, and provides various types of web pages to the user.
  • the web page also includes a web page including a video (video content), in this specification, the video is processed separately or independently from the web page.
  • the separated video can be implemented to allocate an arbitrary channel number as described above, provide it through a service guide, or the like, and be output according to a channel switching request by a user in a service guide or broadcast channel viewing process.
  • predetermined content, images, audio, items, etc. are separately processed from the broadcast content, games, and the application itself, and for playback, processing, etc. Any channel number can be assigned and implemented as described above.
  • Service systems including digital devices include a content provider (CP) 3210, a service provider (SP) 3220, a network provider (NP) 3230, and a HNED (Home Network End User). ) (Customer) 3240.
  • the HNED 3240 is, for example, a client 3200, that is, a digital device.
  • the content provider 3210 produces and provides various content. As shown in FIG. 32 as such a content provider 3210, terrestrial broadcaster, cable SO (System Operator) or MSO (Multiple SO), satellite broadcaster (satellite broadcaster) , Various Internet broadcasters, and private content providers (Private CPs). Meanwhile, the content provider 3210 provides various applications in addition to broadcast content.
  • the service provider 3220 provides services to the client 300 in a uni-cast or multi-cast manner. Meanwhile, the service provider 3220 may transmit data to a plurality of pre-registered clients 3200 at a time, and for this, an Internet Group Management Protocol (IGMP) protocol may be used.
  • IGMP Internet Group Management Protocol
  • the above-described content provider 3210 and service provider 3220 may be identical or single entities. For example, by providing the content produced by the content provider 3210 as a service package and providing it to the HNED 3240, the function of the service provider 3220 may also be performed or vice versa.
  • the network provider 3230 provides a network for data exchange between the content provider 3210 or / and the service provider 3220 and the client 3200.
  • the client 3200 can establish a home network to transmit and receive data.
  • the content provider 3210 or / and the service provider 3220 in the service system may use conditional access or content protection means to protect transmitted content.
  • the client 300 may use a processing means such as a cable card (POD: Point of Deployment), a DCAS (Downloadable CAS), etc. in response to the limited reception or content protection.
  • a processing means such as a cable card (POD: Point of Deployment), a DCAS (Downloadable CAS), etc. in response to the limited reception or content protection.
  • the client 3200 may also use a bidirectional service through a network (or communication network). In this case, rather, the client 3200 may perform the function of a content provider, and the existing service provider 3220 may receive it and transmit it back to another client.
  • FIG. 33 is a block diagram illustrating a digital device according to an embodiment.
  • FIG. 33 may correspond to, for example, the client 3200 of FIG. 32, and refers to the digital device described above.
  • the digital device 3300 includes a network interface 3301, a TCP / IP manager 3302, a service delivery manager 3303, an SI decoder 3304, Demultiplexer (demux) 3305, audio decoder 3306, video decoder 3307, display A / V and OSD module 3308, service control manager control manager (3309), service discovery manager (service discovery manager) 3310, SI & metadata database (SI & Metadata DB) 3311, metadata manager 3312, service manager 3313, UI And a manager 3314 or the like.
  • SI & metadata database SI & Metadata DB
  • the network interface unit 3301 receives or transmits Internet protocol (IP) packets through a network. That is, the network interface unit 3301 receives services, content, and the like from the service provider 3220 through a network network.
  • IP Internet protocol
  • the TCP / IP manager 3302 is configured to transmit packets between IP packets received by the digital device 3300 and IP packets transmitted by the digital device 3300, that is, between a source and a destination. Get involved. Then, the TCP / IP manager 3302 classifies the received packet (s) to correspond to an appropriate protocol, and the service delivery manager 3305, the service discovery manager 3310, the service control manager 3309, and the metadata manager 3312 ), And the like.
  • the service delivery manager 3303 is responsible for controlling received service data. For example, the service delivery manager 3303 may use RTP / RTCP when controlling real-time streaming data.
  • the service delivery manager 3303 parses the received data packet according to RTP and transmits it to the demultiplexer 3305 or control of the service manager 3313 According to the SI & metadata database 3311. Then, the service delivery manager 3303 uses the RTCP to feed back the network reception information to a server providing a service.
  • the demultiplexer 3305 demultiplexes the received packets into audio, video, and system information (SI) data, and transmits them to the audio / video decoder 3306/3307 and the SI decoder 3304, respectively.
  • SI system information
  • the SI decoder 3304 decodes service information such as program specific information (PSI), program and system information protocol (PSIP), and digital video broadcasting-service information (DVB-SI).
  • PSI program specific information
  • PSIP program and system information protocol
  • DVB-SI digital video broadcasting-service information
  • the audio / video decoder 3306/3307 decodes each audio data and video data demultiplexed by the demultiplexing unit 405.
  • the audio data and video data thus decoded are provided to the user through the display unit 3308.
  • the application manager may include, for example, a UI manager 3314 and a service manager 3313.
  • the application manager may manage the overall state of the digital device 3300, provide a user interface, and manage other managers.
  • the UI manager 3314 provides a graphical user interface (GUI) for a user using an on-screen display (OSD) or the like, and receives key input from a user to perform device operation according to the input. For example, when the UI manager 3314 receives a key input for channel selection from the user, the UI manager 3314 transmits the key input signal to the service manager 3313.
  • GUI graphical user interface
  • OSD on-screen display
  • the service manager 3313 controls a manager associated with a service, such as a service delivery manager 3303, a service discovery manager 3310, a service control manager 3309, and a metadata manager 3312.
  • the service manager 3313 creates a channel map and selects a channel using the channel map according to a key input received from the user interface manager 3314. Then, the service manager 3313 receives the service information of the channel from the SI decoder 3304 and sets the audio / video PID (packet identifier) of the selected channel to the demultiplexer 3305.
  • the PID set in this way is used in the demultiplexing process described above. Accordingly, the demultiplexer 3305 filters the audio data, video data, and SI data using the PID.
  • the service discovery manager 3310 provides information necessary to select a service provider that provides a service. When a signal regarding channel selection is received from the service manager 3313, the service discovery manager 3310 finds a service using the information.
  • the service control manager 3309 is responsible for selecting and controlling services.
  • the service control manager 3309 uses IGMP or RTSP when a user selects a live broadcasting service such as a conventional broadcasting method, and selects a service such as video on demand (VOD).
  • the RTSP is used to select and control services.
  • the RTSP protocol may provide a trick mode for real-time streaming.
  • the service control manager 3309 may initialize and manage a session through the IMS gateway 3350 using an IP multimedia subsystem (IMS) and a session initiation protocol (SIP).
  • IMS IP multimedia subsystem
  • SIP session initiation protocol
  • the protocols are one embodiment, and other protocols may be used according to implementation examples.
  • the metadata manager 3312 manages metadata associated with a service and stores the metadata in the SI & metadata database 3311.
  • the SI & metadata database 3311 includes service information decoded by the SI decoder 3304, metadata managed by the metadata manager 3312, and information necessary to select a service provider provided by the service discovery manager 3310. To save.
  • the SI & metadata database 3311 can store set-up data and the like for the system.
  • the SI & metadata database 3311 may be implemented using non-volatile RAM (NVRAM), flash memory, or the like.
  • NVRAM non-volatile RAM
  • the IMS gateway 3350 is a gateway that collects functions necessary for accessing the IMS-based IPTV service.
  • FIG. 34 is a configuration block diagram illustrating another embodiment of a digital device.
  • FIG. 34 illustrates a block diagram of a mobile device as another embodiment of a digital device.
  • the wireless communication unit 3410 may include one or more modules that enable wireless communication between the mobile device 3400 and the wireless communication system or between a mobile device and a network in which the mobile device is located.
  • the wireless communication unit 3410 may include a broadcast reception module 3411, a mobile communication module 3412, a wireless Internet module 3413, a short-range communication module 3414, and a location information module 3415. .
  • the broadcast receiving module 3411 receives broadcast signals and / or broadcast related information from an external broadcast management server through a broadcast channel.
  • the broadcast channel may include a satellite channel and a terrestrial channel.
  • the broadcast management server may mean a server that generates and transmits broadcast signals and / or broadcast-related information or a server that receives previously generated broadcast signals and / or broadcast-related information and transmits them to a terminal.
  • the broadcast signal may include a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, and may also include a TV broadcast signal or a radio broadcast signal combined with a data broadcast signal.
  • the broadcast related information may mean information related to a broadcast channel, broadcast program, or broadcast service provider. Broadcast-related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 3412.
  • Broadcast-related information may exist in various forms, for example, an electronic program guide (EPG) or an electronic service guide (ESG).
  • EPG electronic program guide
  • ESG electronic service guide
  • the broadcast reception module 3411 includes, for example, ATSC, digital video broadcasting-terrestrial (DVB-T), satellite (DVB-S), media forward link only (MediaFLO), handheld (DVB-H), ISDB-T (Digital broadcasting signals can be received using digital broadcasting systems such as integrated services digital broadcast-terrestrial.
  • the broadcast receiving module 511 may be configured to be suitable for other broadcasting systems as well as the digital broadcasting system described above.
  • the broadcast signal and / or broadcast-related information received through the broadcast receiving module 3411 may be stored in the memory 3460.
  • the mobile communication module 3412 transmits and receives wireless signals to and from at least one of a base station, an external terminal, and a server on a mobile communication network.
  • the wireless signal may include various types of data according to transmission and reception of a voice signal, a video call signal, or a text / multimedia message.
  • the wireless Internet module 3413 may include a module for wireless Internet access, or may be built in or external to the mobile device 3400.
  • Wireless Internet technology may include wireless LAN (WLAN) (Wi-Fi), wireless broadband (Wibro), world interoperability for microwave access (Wimax), and high speed downlink packet access (HSDPA).
  • the A / V input unit 3420 is for inputting an audio or / and video signal, and may include a camera 3421 and a microphone 3342.
  • the camera 3421 processes image frames such as still images or moving images obtained by an image sensor in a video call mode or a shooting mode.
  • the processed image frame may be displayed on the display portion 3451.
  • the user input unit 3430 generates input data for the user to control the operation of the terminal.
  • the user input unit 3430 may be configured with a key pad, a dome switch, a touch pad (static pressure / power outage), a jog wheel, a jog switch, or the like.
  • the sensing unit 3440 displays the current state of the mobile device 3400, such as the open / closed state of the mobile device 3400, the location of the mobile device 3400, the presence or absence of user contact, the orientation of the mobile device, and acceleration / deceleration of the mobile device. It senses and generates a sensing signal for controlling the operation of the mobile device 3400. For example, when the mobile device 3400 is moved or tilted, the position or tilt of the mobile device may be sensed. In addition, whether power is supplied to the power supply unit 3490 or whether external devices are coupled to the interface unit 3470 may be sensed. Meanwhile, the sensing unit 3440 may include a proximity sensor 341 including near field communication (NFC).
  • NFC near field communication
  • the output unit 3450 is for generating output related to vision, hearing, or tactile sense, and may include a display unit 3451, an audio output module 3452, an alarm unit 3345, and a haptic module 3454. have.
  • the display unit 3451 displays (outputs) information processed by the mobile device 3400. For example, when the mobile device is in a call mode, a user interface (UI) or a graphic user interface (GUI) related to the call is displayed. When the mobile device 3400 is in a video call mode or a photographing mode, the photographed or / and received video or UI, GUI is displayed.
  • UI user interface
  • GUI graphic user interface
  • Two or more display units 3451 may be present depending on the implementation form of the mobile device 3400.
  • a plurality of display units may be spaced apart or integrally disposed on one surface, or may be disposed on different surfaces.
  • the touch sensor may be configured to convert a change in pressure applied to a specific portion of the display portion 3451 or capacitance generated in a specific portion of the display portion 3451 into an electrical input signal.
  • the touch sensor may be configured to detect not only the touched position and area, but also pressure at the time of touch.
  • the control unit 3480 can know which area of the display unit 3451 has been touched, and the like.
  • a proximity sensor 341 may be disposed in an inner area of the mobile device surrounded by the touch screen or near the touch screen.
  • the proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object present in the vicinity without mechanical contact using electromagnetic force or infrared rays.
  • Proximity sensors have a longer lifespan and higher utilization than contact sensors.
  • the proximity sensor examples include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive type proximity sensor, a magnetic type proximity sensor, and an infrared proximity sensor.
  • the touch screen When the touch screen is capacitive, it is configured to detect the proximity of the pointer due to a change in electric field according to the proximity of the pointer. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.
  • the act of causing the pointer to be recognized as being located on the touch screen without being touched by the pointer on the touch screen is referred to as a “proximity touch”, and on the touch screen
  • the act of actually touching the pointer is referred to as "contact touch.”
  • the location on the touch screen that is a proximity touch with the pointer means a location where the pointer is perpendicular to the touch screen when the pointer is touched close.
  • the alarm unit 3345 outputs a signal for notifying the occurrence of the event of the mobile device 3400. Examples of events generated in the mobile device include call signal reception, message reception, key signal input, and touch input.
  • the alarm unit 3345 may also output a signal for notifying the occurrence of an event in a form other than a video signal or an audio signal, for example, vibration.
  • the video signal or the audio signal may also be output through the display unit 3451 or the audio output module 3452 so that the display unit and the audio output modules 3451 and 3452 may be classified as part of the alarm unit 3345.
  • the haptic module 3454 may not only transmit the tactile effect through direct contact, but may also implement the user to feel the tactile effect through muscle sensations such as fingers or arms. Two or more haptic modules 3454 may be provided according to a configuration aspect of the mobile device 3400.
  • the memory 3460 may store a program for the operation of the control unit 3480, and may temporarily store input / output data (eg, a phone book, a message, a still image, a video, etc.).
  • the memory 3460 may store data related to various patterns of vibration and sound output when a touch is input on the touch screen.
  • the memory 3460 includes a flash memory type, a hard disk type, a multimedia card micro type, and a card type memory (for example, SD or XD memory, etc.), Random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, It may include a storage medium of at least one of a magnetic disk and an optical disk.
  • the mobile device 3400 may operate in connection with a web storage that performs a storage function of the memory 3460 on the Internet.
  • the identification module is a chip that stores various information for authenticating the usage rights of the mobile device 3400, a user identification module (UIM), a subscriber identification module (SIM), and a universal user authentication module ( universal subscriber identity module, USIM).
  • the device provided with the identification module (hereinafter referred to as an 'identification device') may be manufactured in a smart card format. Therefore, the identification device may be connected to the terminal 3400 through the port.
  • the interface unit 3470 When the mobile terminal 3400 is connected to an external cradle, the interface unit 3470 becomes a passage through which power from the cradle is supplied to the mobile terminal 3400, or various command signals input from the cradle by the user. It can be a passage to the mobile terminal. Various command signals or power input from the cradle may be operated as signals for recognizing that the mobile terminal is correctly mounted on the cradle.
  • the control unit 3480 typically controls the overall operation of the mobile device. For example, it performs related control and processing for voice calls, data communication, video calls, and the like.
  • the control unit 3480 may include a multimedia module 3481 for multimedia playback.
  • the multimedia module 3401 may be implemented in the control unit 3480, or may be implemented separately from the control unit 3480.
  • the controller 3480, in particular the multimedia module 3401 may include the above-described encoding device 100 and / or decoding device 200.
  • the control unit 3480 may perform a pattern recognition process capable of recognizing handwriting input or picture drawing input performed on a touch screen as characters and images, respectively.
  • the embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), It may be implemented using at least one of a processor, a controller, micro-controllers, microprocessors, and electrical units for performing other functions.
  • ASICs application specific integrated circuits
  • DSPs digital signal processors
  • DSPDs digital signal processing devices
  • PLDs programmable logic devices
  • FPGAs field programmable gate arrays
  • It may be implemented using at least one of a processor, a controller, micro-controllers, microprocessors, and electrical units for performing other functions.
  • the embodiments described herein may include a control unit ( 3480) can be implemented by itself.
  • embodiments such as procedures and functions described herein may be implemented as separate software modules.
  • Each of the software modules can perform one or more functions and operations described herein.
  • Software code can be implemented in a software application written in an appropriate programming language.
  • the software code is stored in the memory 3460 and can be executed by the control unit 3480.
  • 35 is a configuration block diagram illustrating another embodiment of a digital device.
  • the digital device 3500 include a broadcast receiving unit 3505, an external device interface unit 3535, a storage unit 3540, a user input interface unit 3550, a control unit 3570, a display unit 3580, audio It may include an output unit 3585, a power supply unit 3590 and a photographing unit (not shown).
  • the broadcast reception unit 3505 may include at least one tuner 3510, a demodulation unit 3520, and a network interface unit 3530. However, in some cases, the broadcast reception unit 3505 includes a tuner 3510 and a demodulation unit 3520, but may not include the network interface unit 3530, and vice versa.
  • a multiplexer is provided to multiplex the signal demodulated by the demodulator 3520 via the tuner 3510 and the signal received through the network interface unit 3530. It might be.
  • a demultiplexer may be provided to demultiplex the multiplexed signal or demultiplex the demodulated signal or the signal that has passed through the network interface unit 3530. have.
  • the tuner 3510 receives an RF broadcast signal by tuning a channel selected by a user or all pre-stored channels among radio frequency (RF) broadcast signals received through an antenna. In addition, the tuner 3510 converts the received RF broadcast signal into an intermediate frequency (IF) signal or a baseband signal.
  • IF intermediate frequency
  • the tuner 3510 can process both digital broadcast signals or analog broadcast signals.
  • the analog baseband video or audio signal (CVBS / SIF) output from the tuner 3510 may be directly input to the controller 3570.
  • the tuner 3510 may receive a single carrier RF broadcast signal according to an advanced television system committee (ATSC) scheme or a multiple carrier RF broadcast signal according to a digital video broadcasting (DVB) scheme.
  • ATSC advanced television system committee
  • DVD digital video broadcasting
  • the tuner 3510 may sequentially tune and receive RF broadcast signals of all broadcast channels stored through a channel storage function among RF broadcast signals received through an antenna and convert them into an intermediate frequency signal or a baseband signal. .
  • the demodulator 3520 receives and demodulates the digital IF signal DIF converted by the tuner 3510. For example, when the digital IF signal output from the tuner 3510 is an ATSC method, the demodulator 3520 performs 8-vestigal side band (8-VSB) demodulation, for example. Also, the demodulator 3520 may perform channel decoding. To this end, the demodulator 3520 includes a trellis decoder, a de-interleaver, a Reed-Solomon decoder, and the like, trellis decoding, deinterleaving, and Reed Soloman decoding can be performed.
  • 8-VSB 8-vestigal side band
  • the demodulator 3520 when the digital IF signal output from the tuner 3510 is a DVB method, the demodulator 3520 performs, for example, coded orthogonal frequency division modulation (COFDMA) demodulation. Further, the demodulator 3520 may perform channel decoding. To this end, the demodulator 3520 may include a convolution decoder, a deinterleaver, and a lead-soloman decoder, and perform convolutional decoding, deinterleaving, and read soloman decoding.
  • COFDMA coded orthogonal frequency division modulation
  • the demodulator 3520 may output a stream signal TS after demodulation and channel decoding.
  • the stream signal may be a signal in which a video signal, an audio signal or a data signal is multiplexed.
  • the stream signal may be an MPEG-2 transport stream (TS) in which an MPEG-2 standard video signal and a Dolby AC-3 standard audio signal are multiplexed.
  • the MPEG-2 TS may include a header of 4 bytes and a payload of 184 bytes.
  • the demodulator 3520 described above may be separately provided according to the ATSC method and the DVB method. That is, the digital device may separately include an ATSC demodulator and a DVB demodulator.
  • the stream signal output from the demodulator 3520 may be input to the controller 3570.
  • the control unit 3570 may control demultiplexing, video / audio signal processing, and the like, and control an image output through the display unit 3580 and an audio output unit through the audio output unit 3585.
  • the external device interface unit 3535 provides an environment in which various external devices are interfaced to the digital device 3500.
  • the external device interface unit 3535 may include an A / V input / output unit (not shown) or a wireless communication unit (not shown).
  • the external device interface unit 3535 includes digital versatile disk (DVD), blu-ray, game devices, cameras, camcorders, computers (laptops, tablets), smartphones, Bluetooth devices, and cloud It can be connected to external devices such as (cloud) by wire / wireless.
  • the external device interface unit 3535 transmits a video, audio, or data (including image) signal input from the outside through the connected external device to the control unit 3570 of the digital device.
  • the control unit 3570 may control the processed image, audio, or data signal to be output to a connected external device.
  • the external device interface unit 3535 may further include an A / V input / output unit (not shown) or a wireless communication unit (not shown).
  • a / V input / output unit USB terminal, CVBS (composite video banking sync) terminal, component terminal, S-video terminal (analog), DVI (DVI) so that video and audio signals from external devices can be input to the digital device (3500) digital visual interface (HDMI) terminal, a high definition multimedia interface (HDMI) terminal, an RGB terminal, a D-SUB terminal, and the like.
  • digital visual interface HDMI
  • HDMI high definition multimedia interface
  • RGB terminal a D-SUB terminal
  • the wireless communication unit may perform short-range wireless communication with other electronic devices.
  • the digital device 3500 includes, for example, Bluetooth, radio frequency identification (RFID), infrared data association (IrDA), ultra wideband (UWB), ZigBee, digital living network alliance (DLNA). Networks may be connected to other electronic devices according to a communication protocol.
  • RFID radio frequency identification
  • IrDA infrared data association
  • UWB ultra wideband
  • ZigBee wireless living network alliance
  • DLNA digital living network alliance
  • the external device interface unit 3535 may be connected to at least one of various set-top boxes and various terminals described above, and perform input / output operations with the set-top box.
  • the external device interface unit 3535 may receive an application or a list of applications in an adjacent external device and transmit it to the control unit 3570 or the storage unit 3540.
  • the network interface unit 3530 provides an interface for connecting the digital device 3500 with a wired / wireless network including an Internet network.
  • the network interface unit 3530 may include, for example, an Ethernet terminal or the like for connection with a wired network, and for example, a wireless LAN (WLAN) (Wi-) for connection with a wireless network.
  • WLAN wireless LAN
  • Wi- wireless LAN
  • Fi wireless broadband
  • Wimax world interoperability for microwave access
  • HSDPA high speed downlink packet access
  • the network interface unit 3530 may access a predetermined web page through a connected network or another network linked to the connected network. That is, it is possible to connect to a predetermined web page through a network and transmit or receive data with the corresponding server.
  • content or data provided by a content provider or a network operator may be received. That is, it is possible to receive content such as a movie, advertisement, game, VOD, broadcast signal, and related information provided by a content provider or a network provider through a network.
  • the network interface unit 3530 may select and receive a desired application from among applications that are open to the public through a network.
  • the storage unit 3540 may store a program for processing and controlling each signal in the control unit 3570, or may store a signal-processed image, audio, or data signal.
  • the storage unit 3540 may perform a function for temporarily storing an image, audio, or data signal input from the external device interface unit 3535 or the network interface unit 3530.
  • the storage unit 3540 may store information regarding a predetermined broadcast channel through a channel storage function.
  • the storage unit 3540 may store various platforms, which will be described later.
  • the storage unit 3540 may be, for example, a flash memory type, a hard disk type, a multimedia card micro type, or a card type memory (for example, SD or XD) Memory, etc.), RAM (RAM), and ROM (EEPROM, etc.).
  • the digital device 3500 may play a content file (video file, still image file, music file, document file, application file, etc.) stored in the storage unit 3540 and provide it to the user.
  • the storage unit 3540 is provided separately from the control unit 3570, the scope of the present specification is not limited thereto. That is, the storage unit 3540 may be included in the control unit 3570.
  • the user input interface unit 3550 transmits a signal input by the user to the control unit 3570 or a signal from the control unit 3570 to the user.
  • the user input interface unit 3550 controls power on / off, channel selection, screen setting, etc. from the remote control device 3600 according to various communication methods such as an RF communication method and an infrared (IR) communication method.
  • the signal may be received and processed, or may be processed to transmit the control signal of the control unit 3570 to the remote control device 3600.
  • the user input interface unit 3550 may transmit a control signal input from a local key (not shown), such as a power key, a channel key, a volume key, and a set value, to the control unit 3570.
  • a local key such as a power key, a channel key, a volume key, and a set value
  • the user input interface unit 3550 transmits a control signal input from a sensing unit (not shown) that senses a user's gesture to the control unit 3570 or senses a signal from the control unit 3570 (Not shown).
  • the sensing unit may include a touch sensor, a voice sensor, a position sensor, and a motion sensor.
  • the control unit 3570 de-multiplexes the stream input through the tuner 3510, the demodulator 3520, or the external device interface unit 3535 or processes the demultiplexed signals to generate a signal for video or audio output. And output.
  • the control unit 3570 may include the aforementioned encoding device and / or decoding device.
  • the image signal processed by the control unit 3570 may be input to the display unit 3580 and displayed as an image corresponding to the corresponding image signal. Also, the image signal processed by the control unit 3570 may be input to an external output device through the external device interface unit 3535.
  • the audio signal processed by the control unit 3570 may be audio output to the audio output unit 3585. Also, the audio signal processed by the control unit 3570 may be input to the external output device through the external device interface unit 3535.
  • control unit 3570 may include a demultiplexing unit, an image processing unit, and the like.
  • the control unit 3570 may control the overall operation of the digital device 3500.
  • the controller 3570 may control the tuner 3510 to tune an RF broadcast corresponding to a channel selected by a user or a pre-stored channel.
  • the control unit 3570 may control the digital device 3500 by a user command input through the user input interface unit 3550 or an internal program. In particular, it is possible to access a network so that a user can download a desired application or application list into the digital device 3500.
  • control unit 3570 controls the tuner 3510 such that a signal of a selected channel is input according to a predetermined channel selection command received through the user input interface unit 3550. And it processes the video, audio or data signal of the selected channel.
  • the control unit 3570 allows the channel information, etc. selected by the user to be output through the display unit 3580 or the audio output unit 3585 together with the processed image or audio signals.
  • control unit 3570 may be input from an external device input through the external device interface unit 3535, for example, a camera or camcorder, according to an external device image playback command received through the user input interface unit 3550.
  • the video signal or the audio signal can be output through the display unit 3580 or the audio output unit 3585.
  • control unit 3570 may control the display unit 3580 to display an image.
  • a broadcast image input through the tuner 3510, an external input image input through the external device interface unit 3535, an image input through the network interface unit, or an image stored in the storage unit 3540 It can be controlled to be displayed on the display unit 3580.
  • the image displayed on the display unit 3580 may be a still image or a video, and may be a 2D image or a 3D image.
  • control unit 3570 may control to play content.
  • the content at this time may be content stored in the digital device 3500, or received broadcast content, or external input content input from the outside.
  • the content may be at least one of a broadcast image, an external input image, an audio file, a still image, a connected web screen, and a document file.
  • the controller 3570 may control to display a list of applications or applications that can be downloaded from the digital device 3500 or from an external network.
  • the control unit 3570 may control to install and operate an application downloaded from an external network along with various user interfaces.
  • an image related to an application to be executed can be controlled to be displayed on the display unit 3580 by a user's selection.
  • a channel browsing processing unit for generating a thumbnail image corresponding to a channel signal or an external input signal.
  • the generated thumbnail image can be input to the control unit 3570 as it is or encoded. Also, the generated thumbnail image may be encoded in a stream form and input to the control unit 3570.
  • the controller 3570 may display a list of thumbnails having a plurality of thumbnail images on the display unit 3580 using the input thumbnail images. Meanwhile, the thumbnail images in the thumbnail list can be updated sequentially or simultaneously. Accordingly, the user can easily grasp the contents of a plurality of broadcast channels.
  • the display unit 3580 converts image signals, data signals, OSD signals, or image signals received from the external device interface unit 3535 processed by the control unit 3570 into R, G, and B signals, respectively. Generate a drive signal.
  • the display unit 3580 may be configured as a touch screen and used as an input device in addition to an output device.
  • the audio output unit 3585 receives a signal processed by the controller 3570, for example, a stereo signal, a 3.1 channel signal, or a 5.1 channel signal, and outputs the audio.
  • the audio output unit 3585 may be implemented as various types of speakers.
  • a photographing unit (not shown) for photographing a user may be further provided. Image information photographed by the photographing unit (not shown) may be input to the control unit 3570.
  • the control unit 3570 may detect a user's gesture by individually or in combination with an image captured by the photographing unit (not shown) or a signal detected from the sensing unit (not shown).
  • the power supply unit 3590 supplies the corresponding power throughout the digital device 3500.
  • the power supply unit 3590 may include a converter (not shown) that converts AC power into DC power.
  • a PWM-operable inverter (not shown) may be further provided to control luminance or dimming. have.
  • the remote control device 3600 transmits a user input to the user input interface unit 3550.
  • the remote control device 3600 Bluetooth (bluetooth), RF (radio frequency) communication, infrared (IR) communication, UWB (Ultra Wideband), ZigBee (ZigBee) method can be used.
  • the remote control device 3600 may receive an image, audio, or data signal output from the user input interface unit 3550, display it on the remote control device 3600, or output voice or vibration.
  • the above-described digital device 3500 may be a digital broadcast receiver capable of processing a fixed or mobile ATSC type or DVB type digital broadcast signal.
  • the digital device may omit some components or further include components not illustrated, as required.
  • the digital device does not have a tuner and a demodulator, and can also receive and play content through a network interface unit or an external device interface unit.
  • FIGS. 33 to 35 are block diagram illustrating a detailed configuration of the control unit of FIGS. 33 to 35 to illustrate one embodiment.
  • control unit examples include a voice processing unit and a data processing unit.
  • the demultiplexing unit 3610 demultiplexes the input stream.
  • the demultiplexer 3610 can demultiplex the input MPEG-2 TS video, audio, and data signals.
  • the stream signal input to the demultiplexer 3610 may be a stream signal output from a tuner or demodulator or an external device interface.
  • the video decoder 3625 decodes the demultiplexed video signal, and the scaler 3635 scales the resolution of the decoded video signal to be output from the display unit.
  • the video decoder 3625 can support various standards.
  • the video decoder 3625 performs the function of the MPEG-2 decoder when the video signal is encoded in the MPEG-2 standard, and the video signal is encoded in the digital multimedia broadcasting (DMB) method or the H.264 standard.
  • DMB digital multimedia broadcasting
  • H.264 the function of the H.264 decoder can be performed.
  • the video signal decoded by the video processing unit 3620 is input to the mixer 3650.
  • the OSD generation unit 3640 generates OSD data according to a user input or by itself. For example, the OSD generating unit 3640 generates data for displaying various data on a screen of the display unit 4180 in a graphic or text form based on a control signal of the user input interface unit.
  • the generated OSD data includes various data such as a user interface screen of a digital device, various menu screens, widgets, icons, and viewing rate information.
  • the OSD generator 3640 may generate data for displaying subtitles of broadcast images or broadcast information based on EPG.
  • the mixer 3650 mixes the OSD data generated by the OSD generating unit 3640 and the image signal processed by the image processing unit and provides it to the formatter 3660. Because the decoded video signal and OSD data are mixed, the OSD is displayed overlaid on the broadcast video or the external input video.
  • a frame rate converter (FRC) 3655 converts a frame rate of an input video.
  • the frame rate converter 3655 may convert the input 60 Hz image frame rate to have a frame rate of, for example, 120 Hz or 240 Hz, depending on the output frequency of the display unit.
  • various methods may exist in the method for converting the frame rate. For example, when the frame rate converter 3655 converts the frame rate from 60 Hz to 120 Hz, the same first frame is inserted between the first frame and the second frame, or the first frame and the second frame are predicted from the first frame. It can be converted by inserting 3 frames.
  • the frame rate converter 3655 converts the frame rate from 60 Hz to 240 Hz, three or more identical frames or predicted frames may be inserted and converted between existing frames. Meanwhile, when a separate frame conversion is not performed, the frame rate conversion unit 3655 may be bypassed.
  • the formatter 3660 changes the output of the input frame rate conversion unit 3655 to match the output format of the display unit.
  • the formatter 3660 may output R, G, and B data signals, and these R, G, and B data signals may be output as low voltage differential signaling (LVDS) or mini-LVDS Can be.
  • LVDS low voltage differential signaling
  • the formatter 3660 may configure and output a 3D format according to the output format of the display unit, thereby supporting 3D service through the display unit.
  • a voice processing unit (not shown) in the control unit may perform voice processing of a demultiplexed voice signal.
  • the voice processing unit (not shown) may support various audio formats. For example, even when an audio signal is encoded in formats such as MPEG-2, MPEG-4, AAC, HE-AAC, AC-3, BSAC, a decoder corresponding thereto may be provided and processed.
  • the voice processing unit (not shown) in the control unit may process a base, treble, volume control, and the like.
  • the data processing unit (not shown) in the control unit may perform data processing of the demultiplexed data signal.
  • the data processing unit can decode the demultiplexed data signal even when it is encoded.
  • the encoded data signal may be EPG information including broadcast information such as a start time and an end time of a broadcast program broadcast on each channel.
  • each component may be integrated, added, or omitted depending on the specification of the actual digital device. That is, if necessary, two or more components may be combined into one component, or one component may be subdivided into two or more components.
  • the function performed in each block is for describing an embodiment of the present specification, and the specific operation or device does not limit the scope of the present specification.
  • FIG. 37 is a diagram illustrating an example in which a screen of a digital device displays a main image and a sub image simultaneously according to an embodiment.
  • the digital device may simultaneously display the main image 3710 and the auxiliary image 3720 on the screen 3700.
  • the main image 3710 may be referred to as a first image, and the auxiliary image 3720 may be referred to as a second image.
  • the main image 3710 and the auxiliary image 3720 may include a video, a still image, an electronic program guide (EPG), a graphical user interface (GUI), an on-screen display (OSD), and the like.
  • the main image 3710 may mean an image that is relatively smaller in size than the screen 3700 of the electronic device while being simultaneously displayed on the screen 3700 of the electronic device and the auxiliary image 3720, as a picture in picture (PIP). Also referred to as.
  • PIP picture in picture
  • the main image 3710 is displayed as being displayed on the upper left of the screen 3700 of the digital device, but the location where the main image 3710 is displayed is not limited thereto, and the main image 3710 is a digital device. Can be displayed at any location within the screen 3700.
  • the main image 3710 and the auxiliary image 3720 may be directly or indirectly related to each other.
  • the main image 3710 may be a streaming video
  • the auxiliary image 3720 may be a GUI that sequentially displays thumbnails of videos including information similar to the streaming video.
  • the main image 3710 may be a broadcasted image
  • the auxiliary image 3720 may be an EPG.
  • the main image 3710 may be a broadcast image
  • the auxiliary image 3720 may be a GUI. Examples of the main image 3710 and the auxiliary image 3720 are not limited thereto.
  • the main image 3710 may be a broadcast image received through a broadcasting channel
  • the auxiliary image 3720 may be information related to a broadcast image received through a broadcast channel.
  • Information related to a broadcast video received through a broadcast channel may include, for example, EPG information including a comprehensive channel schedule, broadcast program detailed information, and broadcast program review information, but is not limited thereto.
  • the main image 3710 may be a broadcast image received through a broadcast channel
  • the auxiliary image 3720 may be an image generated based on information pre-stored in a digital device.
  • the image generated based on the information pre-stored in the digital device may include, for example, a basic user interface (UI) of the EPG, basic channel information, an image resolution manipulation UI, and a bedtime reservation UI. Does not work.
  • UI basic user interface
  • the main image 3710 is a broadcast image received through a broadcast channel
  • the auxiliary image 3720 may be information related to a broadcast image received through a network.
  • the information related to the broadcast image received through the network may be, for example, information obtained through a network-based search engine. More specifically, for example, information related to a character currently being displayed on the main image 3710 may be obtained through a network-based search engine.
  • information related to a broadcast image received through a network may be obtained by using, for example, an artificial intelligence (AI) system.
  • AI artificial intelligence
  • an estimated-location in map of a place currently being displayed on the main image 3710 can be obtained by using network-based deep-learning, and digital The device may receive information about the estimated location on the map of the place currently being displayed on the main image 3710 through the network.
  • the digital device may receive at least one of image information of the main image 3710 and image information of the auxiliary image 3720 from the outside.
  • the video information of the main video 3710 includes, for example, a broadcast signal received through a broadcasting channel, a source code information of the main video 3710, and a main received through a network. IP packet (internet protocol packet) information of the image 3710 may be included, but is not limited thereto.
  • the video information of the secondary video 3720 includes, for example, a broadcast signal received through a broadcast channel, source code information of the secondary video 3720, IP packet information of the secondary video 3720 received through a network, etc. It may include, but is not limited to.
  • the digital device may decode and use video information of the main video 3710 or video information of the secondary video 3720 received from the outside. However, in some cases, the digital device may store image information of the main image 3710 or image information of the auxiliary image 3720 internally.
  • the digital device may display the main image 3710 and the auxiliary image 3720 on the screen 3700 of the digital device based on the image information of the main image 3710 and information related to the auxiliary image 3720.
  • the decoding apparatus 200 of the digital device includes a main image decoding apparatus and an auxiliary image decoding apparatus, and the main image decoding apparatus and the auxiliary image decoding apparatus respectively include image information of the main image 3710 and auxiliary image 3720 ) Can decode video information.
  • the renderer includes a main image renderer (first renderer) and an auxiliary image renderer (second renderer), and the main image renderer displays the main image 3710 on the screen 3700 of the digital device based on the information decoded by the main image decoding device. ) May be displayed in the first area, and the auxiliary image renderer may display the auxiliary image 3720 in the second area of the screen 3700 of the digital device based on the decoded information in the auxiliary image decoding apparatus. .
  • the decoding apparatus 200 of the digital device may decode image information of the main image 3710 and image information of the auxiliary image 3720. Based on the information decoded by the decoding apparatus 200, the renderer may process the main image 3710 and the auxiliary image 3720 together to be simultaneously displayed on the screen 3700 of the digital device.
  • the image service processing method receiving image information, decoding a (main) image based on the image information, rendering or displaying the decoded image in a first area on the display, and second in the display And rendering or displaying an auxiliary image in the area.
  • the step of decoding the first image may follow the decoding procedure in the decoding apparatus 200 according to FIG. 3 described above.
  • decoding the first image may include deriving prediction samples for the current block based on inter or intra prediction, and residual samples for the current block based on the received residual information. And generating reconstruction samples based on predictive samples and / or residual samples.
  • decoding the first image may include performing an in-loop filtering procedure on the reconstructed picture including the reconstructed samples.
  • the auxiliary image may be an electronic program guide (EPG), an on-screen display (OSD), or a graphical user interface (GUI).
  • EPG electronic program guide
  • OSD on-screen display
  • GUI graphical user interface
  • the video information may be received through a broadcast network, and information on the auxiliary video may be received through the broadcast network.
  • the image information may be received through a communication network, and information regarding the auxiliary image may be received through the communication network.
  • the video information may be received through a broadcast network, and information regarding the auxiliary video may be received through a communication network.
  • the image information may be received through a broadcasting network or a communication network, and information regarding the auxiliary image may be stored in a storage medium in the digital device.
  • an embodiment of the present specification may be implemented in the form of a module, procedure, function, etc. that performs the functions or operations described above.
  • the software code can be stored in memory and driven by a processor.
  • the memory is located inside or outside the processor, and can exchange data with the processor by various means already known.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

Selon certains modes de réalisation, la présente invention concerne un procédé et un appareil permettant de traiter un signal vidéo à l'aide d'une prédiction intra. Le procédé pour traiter un signal vidéo selon un mode de réalisation de la présente invention comprend les étapes consistant : à vérifier si un mode le plus probable (MPM) est appliqué à un bloc courant, sur la base d'un drapeau MPM; lorsque le MPM est appliqué, à vérifier un indice de ligne de référence et un indice MPM pour la prédiction du bloc courant; et à générer un échantillon de prédiction du bloc courant à partir d'un échantillon de référence d'une ligne de référence indiquée par l'indice de ligne de référence, sur la base d'un mode de prédiction indiqué par l'indice MPM.
PCT/KR2019/013024 2018-10-06 2019-10-04 Procédé et appareil permettant de traiter un signal vidéo à l'aide d'une prédiction intra Ceased WO2020071846A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201862742354P 2018-10-06 2018-10-06
US62/742,354 2018-10-06

Publications (1)

Publication Number Publication Date
WO2020071846A1 true WO2020071846A1 (fr) 2020-04-09

Family

ID=70055105

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2019/013024 Ceased WO2020071846A1 (fr) 2018-10-06 2019-10-04 Procédé et appareil permettant de traiter un signal vidéo à l'aide d'une prédiction intra

Country Status (1)

Country Link
WO (1) WO2020071846A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111654696A (zh) * 2020-04-24 2020-09-11 北京大学 一种帧内的多参考行预测方法、装置、存储介质及终端
WO2024242888A1 (fr) * 2023-05-22 2024-11-28 Google Llc Angles de prédiction intra étendus

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20140098039A (ko) * 2014-07-16 2014-08-07 에스케이텔레콤 주식회사 화면 내 예측 모드를 효율적으로 처리하는 영상 부호화/복호화 방법 및 장치
US20150110197A1 (en) * 2011-08-29 2015-04-23 Ibex Pt Holdings Co., Ltd. Method for generating prediction block in amvp mode
KR20150141177A (ko) * 2011-09-28 2015-12-17 한국전자통신연구원 인트라 예측 모드 부호화/복호화 방법 및 장치

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150110197A1 (en) * 2011-08-29 2015-04-23 Ibex Pt Holdings Co., Ltd. Method for generating prediction block in amvp mode
KR20150141177A (ko) * 2011-09-28 2015-12-17 한국전자통신연구원 인트라 예측 모드 부호화/복호화 방법 및 장치
KR20140098039A (ko) * 2014-07-16 2014-08-07 에스케이텔레콤 주식회사 화면 내 예측 모드를 효율적으로 처리하는 영상 부호화/복호화 방법 및 장치

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
BROSS ET AL.: "CVersatile Video Coding (Draft 2=", JVET-K0175-V6. JOINT VIDEO EXPERTS TEAM (JVET) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/WG 11 11TH MEETING, 21 September 2018 (2018-09-21), XP030193577 *
JIANG, HUI-YU ET AL.: "CE3-related: Advanced MPM based on intra reference lines election scheme", JVET-K0175-V3. JOINT VIDEO EXPERTS TEAM (JVET) OF ITU-T SG 16 WP 3 AND ISO/EEC JTC 1/SC 29/WG 11 11TH MEETING, 14 July 2018 (2018-07-14), XP030199224 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111654696A (zh) * 2020-04-24 2020-09-11 北京大学 一种帧内的多参考行预测方法、装置、存储介质及终端
WO2024242888A1 (fr) * 2023-05-22 2024-11-28 Google Llc Angles de prédiction intra étendus

Similar Documents

Publication Publication Date Title
WO2020122640A1 (fr) Procédé et dispositif de traitement de signal vidéo sur la base d'une transformée de vecteurs de mouvements basés sur l'historique
WO2020141914A1 (fr) Procédé et appareil de traitement de signal vidéo sur la base d'une prédiction de vecteur de mouvement basé sur l'historique
WO2020117018A1 (fr) Procédé et dispositif de traitement de signaux vidéo sur la base d'une interprédiction
WO2020197083A1 (fr) Procédé et dispositif d'interprédiction basée sur le dmvr et le bdof
WO2020197084A1 (fr) Procédé et appareil d'inter-prédiction sur la base d'un dmvr
WO2020117016A1 (fr) Procédé et dispositif de traitement de signal vidéo sur la base d'une inter-prédiction
WO2015137783A1 (fr) Procédé et dispositif de configuration d'une liste de candidats de fusion pour le codage et le décodage de vidéo intercouche
WO2014051408A1 (fr) Procédé de compensation sao des erreurs de prévision inter-couches de codage et appareil correspondant
WO2020071871A1 (fr) Procédé et appareil de traitement de service d'image
WO2014107066A1 (fr) Procédé d'encodage de vidéo pouvant être mise à l'échelle et appareil utilisant un suréchantillonnage d'image en considération d'un déphasage et procédé et appareil de décodage de vidéo pouvant être mise à l'échelle
WO2015005753A1 (fr) Procédé et appareil de décodage inter-couche de signaux vidéo à l'aide d'un vecteur de disparité basé sur la profondeur, et procédé et appareil de codage inter-couche de signaux vidéo à l'aide d'un vecteur de disparité basé sur la profondeur
WO2020141913A1 (fr) Procédé et appareil permettant de traiter un signal vidéo sur la base d'une inter-prédiction
WO2020009449A1 (fr) Procédé et dispositif permettant de traiter un signal vidéo à l'aide d'une prédiction affine
WO2014109594A1 (fr) Procédé et dispositif pour coder une vidéo entre couches pour compenser une différence de luminance, procédé et dispositif pour décoder une vidéo
WO2020117013A1 (fr) Procédé et appareil de traitement de signal vidéo sur la base d'une inter-prédiction
WO2020130680A1 (fr) Procédé et dispositf de traitement de signal vidéo à l'aide d'une prédiction intra
WO2015194896A1 (fr) Procédé et dispositif d'encodage vidéo inter-couche pour compenser une différence de luminance, et procédé et dispositif de décodage vidéo
WO2015012622A1 (fr) Procédé pour déterminer un vecteur de mouvement et appareil associé
WO2014163453A1 (fr) Procédé et appareil d'encodage vidéo intercouche et procédé et appareil de décodage vidéo intercouche permettant de compenser une différence de luminance
WO2021040482A1 (fr) Dispositif et procédé de codage d'image à base de filtrage de boucle adaptatif
WO2015093920A1 (fr) Procédé de codage vidéo inter-couches utilisant une compensation de luminosité et dispositif associé, et procédé de décodage vidéo et dispositif associé
WO2020060312A1 (fr) Procédé et dispositif de traitement du signal d'image
WO2020141853A1 (fr) Procédé et appareil de traitement de signal vidéo sur la base d'une inter-prédiction
WO2020256485A1 (fr) Procédé de codage/décodage d'image et dispositif utilisant un procédé de limitation adaptative de taille de bloc de chrominance et de transmission de flux binaire
WO2020262963A1 (fr) Procédé et appareil de codage/décodage d'image utilisant maximum pour un bloc de chrominance, et procédé de transmission de flux binaire

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19869280

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19869280

Country of ref document: EP

Kind code of ref document: A1