WO2020001170A1 - 一种图像编码方法、解码方法、编码器、解码器及存储介质 - Google Patents

一种图像编码方法、解码方法、编码器、解码器及存储介质 Download PDF

Info

Publication number
WO2020001170A1
WO2020001170A1 PCT/CN2019/086046 CN2019086046W WO2020001170A1 WO 2020001170 A1 WO2020001170 A1 WO 2020001170A1 CN 2019086046 W CN2019086046 W CN 2019086046W WO 2020001170 A1 WO2020001170 A1 WO 2020001170A1
Authority
WO
WIPO (PCT)
Prior art keywords
reference sample
value
block
intra prediction
coding block
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/086046
Other languages
English (en)
French (fr)
Inventor
李明
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
ZTE Corp
Original Assignee
ZTE Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ZTE Corp filed Critical ZTE Corp
Priority to KR1020217002646A priority Critical patent/KR102513756B1/ko
Priority to US17/255,018 priority patent/US11343513B2/en
Priority to EP19824813.0A priority patent/EP3817383A4/en
Priority to JP2020572841A priority patent/JP7393366B2/ja
Publication of WO2020001170A1 publication Critical patent/WO2020001170A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/11—Selection of coding mode or of prediction mode among a plurality of spatial predictive coding modes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/13—Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/147—Data rate or code amount at the encoder output according to rate distortion criteria
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/174—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a slice, e.g. a line of blocks or a group of blocks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/567—Motion estimation based on rate distortion criteria
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques

Definitions

  • the present disclosure relates to the field of communication technologies, and in particular, to an image encoding method, a decoding method, an encoder, a decoder, and a storage medium.
  • Digital video and image compression coding technology uses the correlation between pixel samples in video and image to remove the redundancy existing in video and image.
  • the compression coding technology includes intra prediction (Intra Prediction) and inter prediction (Inter Prediction).
  • intra prediction uses the spatial domain correlation between adjacent pixel samples in an image to predict the pixels to be encoded by using adjacent encoded pixels, thereby reducing the redundancy.
  • Inter prediction not only uses the correlation between adjacent pixels in the spatial domain in a single frame image, but also uses the correlation between adjacent images in the time domain. For example, using motion estimation (ME) and motion compensation (Motion) Compensation (MC), using the encoded image as a reference to predict the current encoded image.
  • ME motion estimation
  • Motion Motion Compensation
  • the intra prediction method used by related video and image coding standards is to use the coded pixels (i.e., reference pixels) at the left and upper adjacent positions of the current coding block to construct the prediction of pixels in the current coding block. value.
  • the intra prediction mode indicates a method of constructing a prediction value of a pixel of a coded block using neighboring pixels of the coded block, for example, a direct current (DC) mode, an intra prediction direction, and the like.
  • DC direct current
  • the present disclosure desires to provide an image encoding method, a decoding method, an encoder, a decoder, and a storage medium, which can reduce intra prediction errors in video image encoding.
  • an embodiment of the present disclosure provides an image encoding method, including:
  • the first reference sample value, the intra prediction mode, and the prediction difference parameter are encoded, and the encoded bits are written into the code stream.
  • obtaining the prediction difference parameters by way of example includes:
  • determining the first reference sample value of the coding block according to the pixel sample values in the coding block includes:
  • the first reference sample of the coding block is determined according to the pixel samples in the coding block.
  • the matching block of the coding block is located in a region where the coding block is located and has the same size as the coding block.
  • a method for determining whether a reference sample exists includes at least one of the following:
  • the reference sample is judged to be absent
  • the reference sample is judged to be absent
  • the reference sample is judged to be non-existent
  • the reference sample is judged to be non-existent
  • the reference sample value is determined to be absent.
  • determining the first reference sample value of the coding block according to the pixel sample values in the coding block includes:
  • a soft decision method is used to determine the first reference sample.
  • using a soft decision method to determine the first reference sample value according to the pixel sample values in the coding block includes:
  • the first reference sample value is calculated using the constraint optimization method, and the constraint optimization calculation formula is:
  • D is the coding distortion of the coding block
  • k is the traversal range, including one or more pixels in the coding block
  • d (a, b) is the error value between a and b
  • x k is the code to be encoded in the coding block Pixel samples
  • p k is the predicted value of x k calculated from the intra prediction mode using the first reference sample
  • R is the coding bit overhead of the coding block
  • R C is the upper limit of the coding bit overhead of the coding block.
  • using a soft decision method to determine the first reference sample value according to the pixel sample values in the coding block includes:
  • the first reference sample is calculated using the unconstrained optimization method.
  • the unconstrained optimization calculation formula is:
  • r is the first reference sample value
  • m is the intra prediction mode of the coding block
  • D is the coding distortion of the coding block
  • R is the coding bit overhead of the coding block
  • ⁇ is the Lagrange factor
  • determining the first reference sample value of the coding block according to the pixel sample values in the coding block includes:
  • the first reference sample value is set as the difference between the average value of the original value of the coding block and the average value of the reference sample values that partially exist.
  • the first reference sample value includes at least two sample values and a use mode indication parameter, wherein the sample use mode parameter is used to indicate that at least two sample values included in the first reference sample value are set in the encoding block. Refer to how it is used in the sample process.
  • the first reference sample includes two samples, and the first reference sample of the coding block is determined according to the pixel samples in the coding block, including:
  • the values of the reference samples that do not exist in the left adjacent reference pixel position of the coding block and the values of the reference samples that do not exist in the upper adjacent reference pixel position are set.
  • determining the intra prediction mode of the coding block and calculating the intra prediction value of the coding block according to the first reference sample value include:
  • the intra-frame prediction mode of the coding block is determined by using a rate-distortion optimization method, and the intra-prediction value of the coding block is calculated by using the intra prediction mode.
  • determining the intra prediction mode of the coding block according to the first reference sample value, and calculating the intra prediction value of the coding block further includes:
  • the rate-distortion optimization method is used to determine the intra prediction mode of the coding block, and the intra prediction mode is calculated to obtain the intra prediction value of the coding block.
  • filtering the reference samples of the coding block to obtain the filtered reference samples includes:
  • the rate-distortion optimization method is used to determine the filtering control parameters, and the filtering control parameters are used to indicate whether to filter the reference samples;
  • the filtering control parameter instructs to filter the reference samples
  • the reference samples of the coding block are filtered to obtain the filtered reference samples.
  • filtering the reference samples of the coding block to obtain the filtered reference samples includes:
  • a rate control optimization method is used to determine a filtering control parameter, and the filtering control parameter is used to indicate whether to filter the reference samples;
  • the filtering control parameter instructs to filter the reference samples
  • the reference samples of the coding block are filtered to obtain the filtered reference samples.
  • filtering the reference samples of the coding block to obtain the filtered reference samples includes:
  • the reference sample values of the coding block are filtered to obtain the filtered reference sample values.
  • filtering the reference samples of the coding block to obtain the filtered reference samples includes:
  • a rate control optimization method is used to determine a filtering control parameter, and the filtering control parameter is used to indicate whether to filter the reference samples;
  • the filtering control parameter instructs to filter the reference samples
  • the reference samples of the coding block are filtered to obtain the filtered reference samples.
  • filtering the reference samples of the coding block to obtain the filtered reference samples includes:
  • the reference samples of the coding block are filtered to obtain the filtered reference samples.
  • filtering the reference samples of the coding block to obtain the filtered reference samples includes:
  • a rate control optimization method is used to determine a filtering control parameter, and the filtering control parameter is used to indicate whether to filter the reference samples;
  • the filtering control parameter instructs to filter the reference samples
  • the reference samples of the coding block are filtered to obtain the filtered reference samples.
  • filtering the reference samples of the coding block to obtain the filtered reference samples includes:
  • the reference samples of the coding block are filtered to obtain the filtered reference samples.
  • the reference samples of the coding block are filtered to obtain the filtered reference samples.
  • the method further includes:
  • encoding the first reference sample includes:
  • the coded bits are written into a data unit in a code stream, where the data unit in the code stream includes at least one of the following: one or more parameter sets, a slice header, and a block-level data unit.
  • encoding the first reference sample includes:
  • the first reference sample includes one or more samples
  • the one or more samples of the first reference sample are encoded to obtain the encoded bits of the one or more samples of the first reference sample, and the encoded bits are written into the data unit in the code stream.
  • encoding one or more samples of the first reference sample to obtain coded bits of the one or more samples of the first reference sample further includes:
  • the coding sample use mode parameter writes the coded bits of the sample use mode parameter into a data unit in the code stream.
  • the first reference sample includes sample control parameters
  • the sample control parameters include at least one of the following control parameters: sequence layer sample control parameters, image layer sample control parameters, and slice layer sample control parameters. Block-level sample control parameters.
  • an embodiment of the present disclosure provides an image decoding method, including:
  • determining the intra prediction reference samples of the decoded block according to the first reference samples includes:
  • the non-existent reference sample values of the decoded block are determined according to the first reference sample value to obtain the intra prediction reference sample value.
  • the matching block of the decoded block is located in an area where the decoded block is located and has the same size as the decoded block.
  • a method for determining whether a reference sample exists includes at least one of the following:
  • the reference sample is judged to be absent
  • the reference sample is judged to be absent
  • the reference sample is judged to be non-existent
  • the reference sample is judged to be absent.
  • obtaining the first reference sample value of the decoded block by parsing the bitstream includes:
  • obtaining the first reference sample value of the decoded block by parsing the bitstream includes:
  • the first reference sample includes sample control parameters
  • the sample control parameters include at least one of the following control parameters: sequence layer sample control parameters, image layer sample control parameters, and slice layer sample control parameters. Block-level sample control parameters.
  • determining the non-existing reference sample value of the decoded block according to the first reference sample value includes:
  • a non-existent reference sample of the decoded block is set using the first reference sample.
  • the first reference sample includes a sample control parameter
  • the non-existent reference sample of the decoded block is determined according to the first reference sample, including:
  • the sample control parameter indicates that a non-existing reference sample is constructed using the first reference sample
  • the non-existing reference sample of the decoded block is set using the first reference sample.
  • the first reference sample value includes a parameter usage mode parameter
  • the non-existent reference sample value of the decoded block is determined according to the first reference sample value, including:
  • the non-existing reference sample value of the decoded block is set correspondingly by using the sample values included in the first reference sample value according to the sample use mode parameter.
  • the first reference sample includes two samples, and according to the sample usage parameter, using the samples included in the first reference sample to set a non-existing reference sample corresponding to the decoding block, including:
  • two samples in the first reference sample are used to set a reference sample that does not exist in the left adjacent reference pixel position of the decoded block and a reference sample that does not exist in the upper adjacent reference pixel position.
  • the method further includes:
  • the intra prediction value of the decoded block is calculated according to the intra prediction mode.
  • filtering the intra prediction reference samples to obtain the filtered reference samples includes:
  • the filtering control parameter indicates that the intra prediction reference samples are filtered, the intra prediction reference samples are filtered to obtain the filtered reference samples.
  • filtering the intra prediction reference samples to obtain the filtered reference samples includes:
  • the code stream is analyzed to obtain filtering control parameters
  • the filtering control parameter instructs to filter the intra prediction reference samples
  • the intra prediction reference samples are filtered to obtain the filtered reference samples.
  • filtering the intra prediction reference samples to obtain the filtered reference samples includes:
  • the intra prediction reference samples are filtered to obtain the filtered reference samples.
  • filtering the intra prediction reference samples to obtain the filtered reference samples includes:
  • the intra prediction mode of the decoded block is a preset mode, analyze the code stream to obtain filter control parameters;
  • the filtering control parameter instructs to filter the intra prediction reference samples
  • the intra prediction reference samples are filtered to obtain the filtered reference samples.
  • filtering the intra prediction reference samples to obtain the filtered reference samples includes:
  • the intra prediction mode of the decoded block is a preset mode
  • filtering processing is performed on the intra prediction reference samples to obtain the filtered reference samples.
  • filtering the intra prediction reference samples to obtain the filtered reference samples includes:
  • the intra prediction mode of the decoded block is a preset mode, analyze the code stream to obtain a filter control parameter
  • the filtering control parameter instructs to filter the intra prediction reference samples
  • the intra prediction reference samples are filtered to obtain the filtered reference samples.
  • filtering the intra prediction reference samples to obtain the filtered reference samples includes:
  • the intra prediction reference sample is filtered to obtain the filtered reference sample.
  • the prediction difference parameter is used to calculate the prediction difference of the decoded block, including:
  • the prediction difference parameter is scaled and transformed to obtain the prediction difference of the decoded block.
  • the method further includes:
  • an embodiment of the present disclosure provides an image encoding device.
  • the image encoding device includes a first reference sample value determination unit, an intra prediction value determination unit, a prediction difference parameter obtaining unit, and an encoding unit, where:
  • a first reference sample value determining unit configured to determine a first reference sample value of a coding block according to pixel sample values in the coding block
  • An intra prediction value determination unit configured to determine an intra prediction mode of a coding block according to a first reference sample value determined by the first reference sample value determination unit, and calculate an intra prediction value of the coding block;
  • a prediction difference parameter obtaining unit configured to obtain a prediction difference parameter according to the intra prediction value calculated by the original value of the encoded block and the intra prediction value determination unit;
  • An encoding unit configured to encode the first reference sample determined by the first reference sample determination unit, the intra prediction mode determined by the intra prediction value determination unit, and the prediction difference parameter obtained by the prediction difference parameter obtaining unit, and encode the encoded bits Write the code stream.
  • the prediction difference parameter obtaining unit is exemplarily set as:
  • the first reference sample determination unit is exemplarily set as:
  • the first reference sample of the coding block is determined according to the pixel samples in the coding block.
  • the first reference sample determination unit is exemplarily set as:
  • a soft decision method is used to determine the first reference sample.
  • the first reference sample determination unit is exemplarily set as:
  • the first reference sample value is set as the difference between the average value of the original value of the coding block and the average value of the reference sample values that partially exist.
  • the intra prediction value determination unit is exemplarily set as:
  • the intra-frame prediction mode of the coding block is determined by using a rate-distortion optimization method, and the intra-prediction value of the coding block is calculated by using the intra prediction mode.
  • the intra prediction value determination unit is exemplarily set as:
  • the rate-distortion optimization method is used to determine the intra prediction mode of the coding block, and the intra prediction mode is calculated to obtain the intra prediction value of the coding block.
  • the coding unit is exemplarily set as:
  • the coded bits are written into a data unit in a code stream, where the data unit in the code stream includes at least one of the following: one or more parameter sets, a slice header, and a block-level data unit.
  • an embodiment of the present disclosure provides an image decoding device.
  • the image decoding device includes a parsing unit, an intra prediction reference sample determination unit, an intra prediction value determination unit, a prediction difference determination unit, and a restoration value obtaining unit. :
  • a parsing unit configured to parse a bitstream to obtain a first reference sample value, an intra prediction mode, and a prediction difference parameter of a decoded block
  • An intra prediction reference sample determination unit configured to determine an intra prediction reference sample of a decoded block according to the first reference sample parsed by the parsing unit;
  • An intra prediction value determination unit configured to construct an intra prediction value of a decoded block according to the intra prediction reference sample determined by the intra prediction reference sample determination unit and the intra prediction mode parsed by the parsing unit;
  • a prediction difference determining unit configured to calculate a prediction difference of a decoded block using a prediction difference parameter parsed by a parsing unit
  • the restoration value obtaining unit is configured to calculate a sum of the intra prediction value constructed by the intra prediction value determination unit and the prediction difference calculated by the prediction difference determination unit to obtain a restoration value of the decoded block.
  • the parsing unit is exemplarily set as:
  • the intra prediction value determination unit is exemplarily set as:
  • the intra prediction value of the decoded block is calculated according to the intra prediction mode.
  • the intra prediction reference sample determination unit is exemplarily set as:
  • the non-existent reference sample values of the decoded block are determined according to the first reference sample value to obtain the intra prediction reference sample value.
  • the prediction difference determination unit is exemplarily set as:
  • the prediction difference parameter is scaled and transformed to obtain the prediction difference of the decoded block.
  • the recovery value obtaining unit is exemplarily set as:
  • an encoder including: a first processor, a first storage medium, and a first communication bus, wherein the first processor and the first storage medium are connected through a first communication bus;
  • the first processor invokes an image encoding-related program stored in the first storage medium, and executes the following steps:
  • the first reference sample value, the intra prediction mode, and the prediction difference parameter are encoded, and the encoded bits are written into the code stream.
  • an embodiment of the present disclosure provides a decoder, including: a second processor, a second storage medium, and a second communication bus, wherein the second processor and the second storage medium are connected through a second communication bus;
  • the second processor invokes a program related to image decoding stored in the second storage medium, and executes the following steps:
  • an embodiment of the present disclosure provides a computer storage medium, which is applied to an encoder.
  • the computer-readable storage medium stores one or more image coding-related programs, and one or more image coding-related programs may be A plurality of first processors execute to implement the image coding method described above.
  • an embodiment of the present disclosure provides a computer storage medium, which is applied to a decoder.
  • the computer-readable storage medium stores one or more image decoding-related programs, and one or more image decoding-related programs may be A plurality of second processors execute to implement the image decoding method described above.
  • the embodiments of the present disclosure provide an image encoding method, a decoding method, an encoder, a decoder, and a storage medium.
  • the image encoding method includes: determining a first reference sample value of an encoding block according to pixel samples in the encoding block; determining an intra prediction mode of the encoding block according to the first reference sample value, and calculating an intra prediction of the encoding block. Value; obtain the prediction difference parameter according to the original value of the coded block and the intra prediction value; encode the first reference sample value, the intra prediction mode, and the prediction difference parameter, and write the encoded bit into the code stream.
  • the first reference sample value is calculated and determined according to the pixel sample value to be encoded of the coding block, and the first value will be further used to construct the intra prediction reference sample value of the coding block. Therefore, when the samples of the neighboring blocks of the coding block are unavailable or partially unavailable, the prediction difference between the intra prediction prediction value and the coding block is effectively reduced, and the coding efficiency of the intra prediction mode is improved. .
  • FIG. 1 is a schematic flowchart of an image encoding method according to an embodiment of the present disclosure
  • FIG. 2 is a schematic flowchart of an image decoding method according to an embodiment of the present disclosure
  • FIG. 3 is a schematic structural diagram of an encoding device according to an embodiment of the present disclosure.
  • FIG. 4 is a schematic structural diagram of a decoding device according to an embodiment of the present disclosure.
  • FIG. 5 is a schematic diagram of a system architecture of an encoder according to an embodiment of the present disclosure.
  • FIG. 6 is a schematic diagram of a system architecture of a decoder according to an embodiment of the present disclosure
  • FIG. 7 is a schematic structural diagram of an intra prediction unit of an encoder according to an embodiment of the present disclosure.
  • FIG. 8 is a schematic diagram of an intra prediction method according to an embodiment of the present disclosure.
  • FIG. 9 is a schematic diagram of another intra prediction method according to an embodiment of the present disclosure.
  • FIG. 10 is a schematic flowchart of a data processing process in which an entropy coding unit of an encoder encodes an output parameter of an intra prediction unit according to an embodiment of the present disclosure
  • FIG. 11 is a schematic structural diagram of an intra prediction unit of a decoder according to an embodiment of the present disclosure
  • FIG. 12 is a schematic flowchart of a data processing process in which a parsing unit of a decoder according to an embodiment of the present disclosure parses an intra prediction mode and a filtering parameter in a decoded block code stream;
  • FIG. 13 is a schematic structural diagram of an electronic device including an encoder according to an embodiment of the present disclosure.
  • FIG. 14 is a schematic structural diagram of an electronic device including a decoder according to an embodiment of the present disclosure
  • FIG. 15 is a schematic diagram of a system architecture of an electronic system according to an embodiment of the present disclosure.
  • the intra prediction method used by the video and image coding standards is to use the coded pixels at the left and upper adjacent positions of the current coding block to construct the predicted value of the pixels in the current coding block.
  • the intra prediction mode indicates a method of constructing a prediction value of a pixel of a coded block using neighboring pixels of the coded block, for example, a direct current (DC) mode, an intra prediction direction, and the like.
  • Related standards have designed multiple intra prediction modes. For example, the H.264 / AVC standard uses 9 intra prediction modes, including DC mode and 8 different intra prediction directions; the H.265 / HEVC standard uses flat mode, DC mode, and 33 different intra predictions.
  • Direction prediction mode is to use the coded pixels at the left and upper adjacent positions of the current coding block to construct the predicted value of the pixels in the current coding block.
  • the intra prediction mode indicates a method of constructing a prediction value of a pixel of a coded block using neighboring pixels of the coded block, for example, a direct
  • a reference pixel does not exist (e.g., the current coded block is on the image boundary) or cannot be used by the current coded block (e.g., the current block is on the boundary of a slice), the reference pixel is considered “not present”.
  • these reference pixels are marked as "unavailable”, and the intra prediction mode using these reference pixels is also unavailable; in H.265 In the / HEVC standard, "non-existent" reference pixels are filled according to a specified rule, and the filled reference pixels are marked as "available”.
  • the DC mode is an intra prediction mode that can always be used.
  • the intra prediction value of the DC mode is set to 1 ⁇ ((BitDepth–1).
  • " ⁇ " is an Arithmetic Left Shift bit operator defined in the standard
  • BitDepth represents the bit depth of the component.
  • Bit Depth refers to the number of bits used in the digital representation of a physical quantity. In this standard, for example, for the luminance component BitDepth is equal to BitDepth Y , BitDepth Y is the bit depth of the luminance component. The current bit depth commonly used in consumer electronics is 8 bits.
  • the DC mode prediction value is 128.
  • the DC mode uses the average of the "available” reference pixel samples as the intra prediction value.
  • the case of unencoded pixels and reference pixels "unavailable” mainly includes: pixels outside the image boundary, pixels outside the slice boundary, and when the constrained_intra_pred_flag value is equal to 1, adjacent macroblocks use frames. Inter coding.
  • the "existent" reference pixel is used to fill the "non-existent” reference pixel; when the reference pixels of the coding block do not exist, the values of the reference pixels are used to fill in 1 ⁇ (BitDepth–1).
  • left neighboring reference pixels can be used to fill neighboring "non-existent" pixels.
  • the reference pixel “absence” mainly includes: unencoded pixels, pixels outside the image boundary, pixels outside the tile boundary, pixels outside the tile boundary, and the value of constant_intra_pred_flag is equal to
  • the 1-phase block uses inter-frame coding.
  • the correlation method is equivalent to setting the prediction value of the intra prediction to "1 ⁇ (BitDepth–1)". If there is a large deviation between the pixel samples of the coded block and "1 ⁇ (BitDepth–1)", the prediction difference of the intra prediction is large, which brings a large coding bit overhead.
  • a video refers to an image sequence composed of images.
  • the bitstream refers to the bitstream generated by the video encoder encoding the video, and also refers to the transport stream containing the bitstream generated by the video encoder encoding the video and the bitstream generated by the video encoder after processing the system layer. / Or media files, decode the bitstream to obtain video.
  • System layer processing is an encapsulation operation performed on a video code stream, for example, encapsulating the video code stream as a data payload into a transport stream or encapsulating the video code stream as a payload into a media file.
  • System layer processing also includes packaging a transport stream or media file containing a video code stream as a data payload into a stream for transmission or a file for storage.
  • the data unit generated by the system layer processing is also called the system layer data unit, and the information added in the system layer data unit (such as the header information of the system layer data unit) during the system layer processing of the encapsulated data payload is called the system layer information.
  • the sub-stream refers to a part of the stream obtained by performing an extraction operation on the stream.
  • the sub-stream can be decoded to obtain a video image.
  • the video image can be an image with a lower resolution than the video image obtained by decoding the stream. It may also be an image with a lower frame rate than the video obtained by decoding the bitstream, and the video image may contain part of the video image obtained by decoding the bitstream.
  • an embodiment of the present disclosure provides an image encoding method, including:
  • Step 101 Determine a first reference sample value of the coding block according to the pixel sample values in the coding block.
  • step 101 exemplarily includes: determining whether a reference sample value of the coding block exists, and the reference sample value is a restored pixel sample value of one or more pixels adjacent to the coding block; If the sample value does not exist, the first reference sample value of the coding block is determined according to the pixel sample value in the coding block;
  • the reference samples in the matching block of the coding block it is determined whether the reference samples in the matching block of the coding block exist; if some or all of the reference samples in the matching block of the coding block do not exist, it is determined that some or all of the reference samples of the coding block do not exist, according to The pixel samples in the coding block determine the first reference samples of the coding block.
  • the matching block of the coding block is located in a region where the coding block is located and has the same size as the coding block.
  • a method for determining whether a reference sample exists includes at least one of the following:
  • the reference sample is judged to be absent
  • the reference sample is judged to be absent
  • the reference sample is judged to be non-existent
  • the reference sample is judged to be non-existent
  • the reference sample value is determined to be absent.
  • the soft reference method is used to determine the first reference sample value according to the pixel sample values in the coding block.
  • using a soft decision method to determine the first reference sample value of the coded block according to the pixel sample values in the coded block includes: using a constraint optimization method to calculate the first reference sample value, and the constraint optimization calculation formula is:
  • D is the coding distortion of the coding block
  • k is the traversal range, including one or more pixels in the coding block
  • d (a, b) is the error value between a and b
  • x k is the code to be encoded in the coding block Pixel samples
  • p k is the predicted value of x k calculated from the intra prediction mode using the first reference sample
  • R is the coding bit overhead of the coding block
  • R C is the upper limit of the coding bit overhead of the coding block.
  • a soft decision method to determine the first reference sample value of the encoding block according to the pixel sample values in the encoding block.
  • Examples include: using an unconstrained optimization method to calculate the first reference sample value, and the unconstrained optimization calculation formula is :
  • r is the first reference sample value
  • m is the intra prediction mode of the coding block
  • D is the coding distortion of the coding block
  • R is the coding bit overhead of the coding block
  • ⁇ is the Lagrange factor
  • step 101 exemplarily includes: if it is determined that all reference sample values of the coding block do not exist, then setting the first reference sample value as an average value of the original value of the coding block; if it is determined that some reference sample values of the coding block do not exist , The first reference sample value is set to a difference between an average value of the original value of the coding block and a mean value of the reference sample values that are partially present.
  • the first reference sample value includes at least two sample values and a use mode indication parameter, wherein the sample use mode parameter is used to indicate that at least two sample values included in the first reference sample value are set in the encoding block. How to use it in the reference sample process.
  • the first reference sample value includes two samples
  • determining the first reference sample value of the coding block according to the pixel samples in the coding block includes: setting the left adjacent reference pixel points of the coding block respectively according to the two samples. The value of the reference sample that does not exist in the position, and the value of the reference sample that does not exist in the position of the upper adjacent reference pixel point.
  • Step 102 Determine the intra prediction mode of the coding block according to the first reference sample value, and calculate the intra prediction value of the coding block.
  • step 102 includes: using a first reference sample value to set a non-existing reference sample value of the coding block; and using a rate-distortion optimization method to determine an intra prediction mode of the coding block according to the reference sample value of the coding block, using the frame The intra prediction mode calculates the intra prediction value of the coded block.
  • step 102 further includes: setting a non-existing reference sample value of the coding block using the first reference sample value; filtering the reference sample value of the coding block to obtain a filtered reference sample value; and according to the filtered processing
  • the reference sample values are determined using the rate-distortion optimization method to determine the intra prediction mode of the coding block, and the intra prediction value of the coding block is calculated by using the intra prediction mode.
  • Filtering the reference samples of the coding block to obtain the filtered reference samples includes the following methods:
  • Method 1 Use the rate-distortion optimization method to determine the filtering control parameter.
  • the filtering control parameter is used to indicate whether to filter the reference samples. If the filtering control parameter indicates to filter the reference samples, the reference samples of the coding block are filtered. Get the filtered reference samples.
  • Method 2 If the size of the coding block is equal to the preset value, use the rate-distortion optimization method to determine the filtering control parameters.
  • the filtering control parameters are used to indicate whether to filter the reference samples. When the filtering control parameters indicate to filter the reference samples, At this time, the reference samples of the coding block are filtered to obtain the filtered reference samples.
  • the preset value includes a value set first or one of a plurality of values set in advance.
  • Method 3 If the size of the encoding block is equal to a preset value, the reference samples of the encoding block are filtered to obtain the filtered reference samples.
  • Method 4 When the intra prediction mode of the coding block is a preset mode, use a rate-distortion optimization method to determine the filtering control parameters, and the filtering control parameters are used to indicate whether to filter the reference samples; when the filtering control parameters indicate to the reference samples, When the values are filtered, the reference samples of the coding block are filtered to obtain the filtered reference samples.
  • the preset mode includes a preset mode or one of a plurality of preset modes.
  • Method 5 When the intra prediction mode of the coding block is a preset mode, the reference samples of the coding block are filtered to obtain the filtered reference samples.
  • Method 6 When the size of the coding block is equal to a preset value and the intra prediction mode of the coding block is a preset mode, use a rate-distortion optimization method to determine a filtering control parameter, and the filtering control parameter is used to indicate whether the reference sample value is Perform filtering; when the filtering control parameter instructs to filter the reference samples, filter the reference samples of the coding block to obtain the filtered reference samples.
  • Method 7 When the size of the coding block is equal to a preset value and the intra prediction mode of the coding block is a preset mode, filter the reference samples of the coding block to obtain the filtered reference samples.
  • the reference samples of the coding block are filtered to obtain the filtered reference samples.
  • the method further includes: encoding the filtering control parameters of the coding block, and writing the coding bits into the code stream.
  • Step 103 Obtain a prediction difference parameter according to the original value of the encoded block and the intra prediction value.
  • the difference between the original value of the coding block and the intra prediction value is calculated to obtain the prediction difference value of the coding block; the prediction difference value is transformed and quantized to obtain the prediction difference parameter.
  • Step 104 encode the first reference sample value, the intra prediction mode, and the prediction difference parameter, and write the encoded bit into the code stream.
  • encoding the first reference sample includes: encoding the first reference sample by obtaining the encoded bit of the first reference sample; and writing the encoded bit into a data unit in the code stream.
  • the data unit in the code stream includes at least one of the following: one or more parameter sets, a slice header, and a block-level data unit.
  • encoding the first reference sample includes: the first reference sample includes one or more samples; encoding one or more samples of the first reference sample to obtain a first reference sample; A coded bit that refers to one or more samples of the sample value, and writes the coded bit to a data unit in the code stream.
  • encoding one or more samples of the first reference sample to obtain coded bits of the one or more samples of the first reference sample further includes: a parameter usage mode parameter, and The sample uses the coded bits of the mode parameter to write to the data unit in the code stream.
  • the first reference sample value includes sample value control parameters
  • the sample value control parameter includes at least one of the following control parameters: sequence layer sample value control parameters, image layer sample value control parameters, and slice layer sample values. Control parameters, block-level sample control parameters.
  • an embodiment of the present disclosure provides an image decoding method, including:
  • Step 201 Parse the bitstream to obtain the first reference sample value, the intra prediction mode, and the prediction difference parameter of the decoded block;
  • parsing the bitstream to obtain the first reference sample value of the decoded block includes:
  • parsing the bitstream to obtain the first reference sample value of the decoded block includes:
  • sample usage mode parameter is used to indicate the multiple sample values included in the first reference sample value in the process of setting the reference sample value of the decoded block Usage in.
  • the first reference sample includes sample control parameters, and the sample control parameters include at least one of the following control parameters: sequence layer sample control parameters, image layer sample control parameters, slice layer sample control parameters, block layer Sample control parameters.
  • Step 202 Determine an intra prediction reference sample value of the decoded block according to the first reference sample value.
  • step 202 includes: judging whether a reference sample value of the decoded block exists, and the reference sample value is a restored pixel sample value of one or more pixels adjacent to the decoded block; if part or all of the reference sample values of the decoded block If not, the non-existing reference sample value of the decoded block is determined according to the first reference sample value, and the intra prediction reference sample value is obtained.
  • the first reference sample determines a non-existing reference sample of the decoded block, and obtains an intra prediction reference sample.
  • the matching block of the decoded block is located in an area where the decoded block is located and has the same size as the decoded block.
  • a method for determining whether a reference sample exists includes at least one of the following:
  • the reference sample is judged to be absent
  • the reference sample is judged to be absent
  • the reference sample is judged to be non-existent
  • the reference sample is judged to be absent.
  • determining the non-existing reference sample value of the decoding block according to the first reference sample value includes: using the first reference sample value to set the non-existing reference sample value of the decoding block.
  • the first reference sample value includes a sample value control parameter
  • the non-existent reference sample value of the decoding block is determined according to the first reference sample value, including: if the sample value control parameter indicates that the first reference sample value is used to construct Non-existing reference samples, use the first reference samples to set non-existing reference samples for the decoded block.
  • the first reference sample value includes a parameter usage mode parameter
  • the non-existent reference sample value of the decoded block is determined according to the first reference sample value, including: if the first reference sample value includes at least two sample values , According to the sample use mode parameter, the non-existing reference sample values of the decoded block are set correspondingly by using the samples included in the first reference sample value.
  • the first reference sample value includes two sample values
  • the non-existing reference sample of the decoded block is set correspondingly by using the samples included in the first reference sample value according to the sample use mode parameter or using a preset rule. Values, including: using two samples in the first reference sample to set reference samples that do not exist in the left adjacent reference pixel position of the decoded block and that that do not exist in the upper adjacent reference pixel position according to a preset rule Reference sample.
  • the method further includes: filtering the intra prediction reference samples to obtain the filtered reference samples; using After filtering the reference samples, the intra prediction value of the decoded block is calculated according to the intra prediction mode.
  • filtering the intra prediction reference samples to obtain the filtered reference samples includes at least the following methods:
  • Method 1 Parse the bitstream to obtain filtering control parameters, where the filtering control parameters are used to indicate whether to filter the intra prediction reference samples; if the filtering control parameters indicate to filter the intra prediction reference samples, the intra prediction The prediction reference samples are filtered to obtain the filtered reference samples.
  • Method 2 If the size of the decoded block is equal to a preset value, analyze the code stream to obtain filtering control parameters; when the filtering control parameter instructs to filter the intra prediction reference samples, perform filtering processing on the intra prediction reference samples, Get the filtered reference samples.
  • Method 3 If the size of the decoded block is equal to a preset value, the intra prediction reference samples are filtered to obtain the filtered reference samples.
  • Method 4 If the intra prediction mode of the decoded block is a preset mode, analyze the code stream to obtain the filtering control parameters; when the filtering control parameters indicate that the intra prediction reference samples are filtered, the intra prediction reference samples are filtered Filtering is performed to obtain the filtered reference samples.
  • Method 5 If the intra prediction mode of the decoded block is a preset mode, perform filtering processing on the intra prediction reference samples to obtain the filtered reference samples.
  • Method 6 If the size of the decoded block is equal to the preset value, and the intra prediction mode of the decoded block is the preset mode, the code stream is parsed to obtain the filtering control parameters; when the filtering control parameter indicates that the intra prediction reference samples are to be performed During filtering, the intra prediction reference samples are filtered to obtain the filtered reference samples.
  • Method 7 If the size of the decoded block is equal to the preset value and the intra prediction mode of the decoded block is the preset mode, the intra prediction reference samples are filtered to obtain the filtered reference samples.
  • Step 203 Construct the intra prediction value of the decoded block according to the intra prediction reference samples and the intra prediction mode.
  • Step 204 Calculate the prediction difference of the decoded block using the prediction difference parameter.
  • step 204 includes: performing scaling and transformation processing on the prediction difference parameter to obtain the prediction difference of the decoded block.
  • Step 205 Calculate the sum of the intra prediction value and the prediction difference to obtain the recovered value of the decoded block.
  • the method further includes: performing a loop filtering process on the recovered value of the decoded block to obtain a decoded value.
  • an embodiment of the present disclosure provides an image encoding device including a first reference sample value determination unit 3001, an intra prediction value determination unit 3002, a prediction difference parameter obtaining unit 3003, and an encoding unit 3004, where:
  • the first reference sample value determination unit 3001 is configured to determine the first reference sample value of the coding block according to the pixel sample values in the coding block;
  • the intra prediction value determination unit 3002 is configured to determine an intra prediction mode of a coding block according to the first reference sample value determined by the first reference sample determination unit 3001, and calculate an intra prediction value of the coding block;
  • the prediction difference parameter obtaining unit 3003 is configured to obtain the prediction difference parameter according to the original value of the encoded block and the intra prediction value calculated by the intra prediction value determination unit 3002.
  • the encoding unit 3004 is configured to encode the first reference sample determined by the first reference sample determination unit 3001, the intra prediction mode determined by the intra prediction value determination unit 3002, and the prediction difference parameter obtained by the prediction difference parameter obtaining unit 3003. To write the encoded bits to the code stream.
  • the prediction difference parameter obtaining unit 3003 is exemplarily set as follows: calculating the difference between the original value of the coding block and the intra prediction value to obtain the prediction difference value of the coding block; performing transformation and quantization processing on the prediction difference value to obtain the prediction Bad parameter.
  • the first reference sample value determination unit 3001 is exemplarily set to: determine whether a reference sample value of the coding block exists, and the reference sample value is a restored pixel sample value of one or more pixels adjacent to the coding block; if If some or all of the reference samples of the coding block do not exist, determining the first reference samples of the coding block according to the pixel samples in the coding block;
  • the reference samples in the matching block of the coding block it is determined whether the reference samples in the matching block of the coding block exist; if some or all of the reference samples in the matching block of the coding block do not exist, it is determined that some or all of the reference samples of the coding block do not exist, according to The pixel samples in the coding block determine the first reference samples of the coding block.
  • the first reference sample value determination unit 3001 is exemplarily configured to determine the first reference sample value using a soft decision method according to the pixel sample values in the coding block.
  • the first reference sample value determination unit 3001 is exemplarily set as follows: if it is determined that all reference sample values of the coding block do not exist, the first reference sample value is set as an average value of the original values of the coding block; if the coding block is determined Part of the reference sample values does not exist, the first reference sample value is set to the difference between the average value of the original value of the encoding block and the average value of the reference sample values that partially exist.
  • the intra prediction value determination unit 3002 is exemplarily set as follows: using a first reference sample value to set a non-existent reference sample value of the coding block; and using a rate-distortion optimization method to determine the Intra-prediction mode.
  • the intra-prediction value of a coded block is calculated using the intra-prediction mode.
  • the intra prediction value determination unit 3002 is exemplarily set as follows: using the first reference sample value to set a non-existent reference sample value of the coding block; filtering the reference sample value of the coding block to obtain a filtered reference sample Value; according to the filtered reference samples, use the rate-distortion optimization method to determine the intra prediction mode of the coding block, and use the intra prediction mode to calculate the intra prediction value of the coding block.
  • the encoding unit 3004 is exemplarily set to: obtain the encoded bits of the first reference sample by encoding the first reference sample; and write the encoded bits into a data unit in a code stream, where the data in the code stream
  • the unit contains at least one of the following: one or more parameter sets, a slice header, and a block-level data unit.
  • an embodiment of the present disclosure provides an image decoding device, including an analysis unit 4001, an intra prediction reference sample determination unit 4002, an intra prediction value determination unit 4003, a prediction difference determination unit 4004, and a restoration value obtaining unit. 4005, of which:
  • the parsing unit 4001 is configured to parse the code stream to obtain the first reference sample value, the intra prediction mode, and the prediction difference parameter of the decoded block;
  • the intra prediction reference sample determination unit 4002 is configured to determine the intra prediction reference samples of the decoded block according to the first reference samples parsed by the parsing unit 4001;
  • the intra prediction value determination unit 4003 is configured to construct an intra prediction value of a decoded block according to the intra prediction reference sample determined by the intra prediction reference sample determination unit 4002 and the intra prediction mode analyzed by the parsing unit 4001.
  • the prediction difference determining unit 4004 is configured to calculate a prediction difference of a decoded block using a prediction difference parameter analyzed by the parsing unit 4001;
  • the restoration value obtaining unit 4005 is configured to calculate the sum of the intra prediction value constructed by the intra prediction value determination unit 4003 and the prediction difference calculated by the prediction difference determination unit 4004 to obtain a restoration value of the decoded block.
  • the parsing unit 4001 is exemplarily set as: parsing one or more data units in the bitstream to obtain a first reference sample value, wherein the data unit includes at least one of the following: one or more parameter sets, and a slice header , Block-level data unit.
  • the intra prediction value determination unit 4003 is exemplarily set as follows: filtering the intra prediction reference samples to obtain the filtered reference samples; using the filtered reference samples according to the intra prediction mode The intra prediction value of the decoded block is calculated.
  • the intra prediction reference sample value determination unit 4002 is exemplarily set to: determine whether a reference sample value of a decoded block exists, and the reference sample value is a restored pixel sample value of one or more pixels adjacent to the decoded block; If some or all of the reference sample values of the decoded block do not exist, the non-existent reference sample values of the decoded block are determined according to the first reference sample value to obtain the intra prediction reference sample value.
  • the first reference sample determines a non-existing reference sample of the decoded block, and obtains an intra prediction reference sample.
  • the prediction difference determining unit 4004 is exemplarily set to perform scaling and transformation processing on the prediction difference parameters to obtain the prediction difference of the decoded block.
  • the restored value obtaining unit 4005 is exemplarily set to: perform a loop filtering process on the restored value of the decoded block to obtain a decoded value.
  • an embodiment of the present disclosure provides a schematic structural diagram of an encoder.
  • the encoder includes a division unit 201, a prediction unit 202, a first adder 207, a transformation unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transformation unit 211, a second adder 212, a filtering unit 213, The decoded image buffer 214, and the entropy encoding unit 215.
  • the prediction unit 202 includes a block division unit 203, a motion estimation unit 204, a motion compensation unit 205, and an intra prediction unit 206.
  • the input of the encoder is video
  • the output of the encoder is a code stream generated by encoding the input video.
  • the encoding process of the encoder is to sequentially encode the images in the input video in the encoding order.
  • the encoding order is determined by parameters such as the prediction structure set in the encoder configuration file.
  • the encoding order of the images in the video (corresponding to the decoding order at the decoding end) and the playback order of the images may be the same or different.
  • the dividing unit 201 is configured to receive a video image, divide the image in the video according to a preset configuration, and output a maximum coding unit and related attribute information of the maximum coding unit.
  • an image may be divided into one or more slices, and each slice may include an integer maximum coding unit or a non-integer maximum coding unit.
  • the largest coding unit is a square image area.
  • the image may also be divided into one or more tiles, and each tile may contain an integer maximum coding unit or a non-integer maximum coding unit.
  • the dividing unit 201 may be configured to divide the image in a fixed manner, or may be configured to dynamically adjust the division manner of the image. For example, in order to adapt to the maximum transmission unit (MTU) of the network, a dynamic segmentation method is adopted to ensure that the number of encoded bits per segment does not exceed the limit of the MTU.
  • MTU maximum transmission unit
  • the prediction unit 202 is configured to divide the largest coding unit divided by the dividing unit 201 into one or more coding blocks, and determine a prediction value of the coding block.
  • the input of the prediction unit 202 includes the maximum coding unit output by the dividing unit 201 and related attribute information of the maximum coding unit.
  • the maximum coding unit related attribute information includes the position of the maximum coding unit in the image, the position in the slice, and the tile, and the like.
  • the manner of dividing the maximum coding unit includes quadtree division, binary tree division, and tritree division.
  • the prediction unit 202 may further divide the coded blocks obtained by the division into more coded blocks.
  • the encoding block may be further divided into one or more prediction blocks to determine a prediction value.
  • the prediction unit 202 determining the prediction value of the encoding block exemplarily includes: determining an inter prediction value of the encoding block according to the decoded image in the decoded image buffer 214; or according to the current encoding output by the second adder 212
  • the recovered part of the image (not processed by the filtering unit 213) determines the intra prediction value of the coded block.
  • the prediction unit 202 determines the prediction value of the coding block, and further includes: using a rate-distortion optimization (RDO) method to determine the prediction value of the coding block, and obtaining the inter prediction, intra, and intra prediction used by the prediction value. Predict related output parameters.
  • RDO rate-distortion optimization
  • the prediction unit 202 includes a block division unit 203, a motion estimation unit 204, a motion compensation unit 205, and an intra prediction unit 206.
  • the four sub-units in the prediction unit 202 are described separately below:
  • the block division unit 203 is configured to determine a block division manner of a coding block during a coding process of a maximum coding unit.
  • the division method may use one or more of quad-tree division, binary-tree division, and tri-tree division.
  • the block dividing unit 203 may divide the coding block into one or more coding blocks, and the divided coding block may be further divided into more coding blocks.
  • a coded block may also be divided into one or more prediction blocks.
  • the block division unit 203 determines the division manner of the coding block by using the RDO method.
  • the output parameters of the block dividing unit include parameters of a division manner of a coding block, and these parameters are used to indicate a division manner of the encoding block.
  • the motion prediction unit 204 is configured to use one or more decoded images in the decoded image buffer 214 as reference images, and construct one or more reference image lists according to the reference images to determine a matching block of the encoded block in the reference image.
  • Each reference image list includes one or more reference images.
  • the output of the motion prediction unit 204 is used to indicate parameters of the position of the matching block, and may include a reference image list indication, a reference image index (Reference index), a motion vector (Motion Vector, MV), and the like.
  • the reference image list indicates the reference image list where the reference image containing the matching block is located
  • the reference image index indicates that the reference image list contains the reference image of the matching block
  • the MV indicates that the coding block and the matching block are in the same
  • the motion compensation unit 205 is configured to construct a prediction value of a coding block according to a matching block, and calculate a difference between the coding block and the prediction value.
  • the output of the motion compensation unit 205 also includes parameters for constructing the prediction value of the coding block, such as a weighted value for the matching block, a filter type and a parameter for filtering the matching block, and the like.
  • the motion estimation unit 204 and the motion compensation unit 205 jointly use the RDO method to determine the matching block with the best rate-distortion performance and the output parameters of the two units for the coding block.
  • the motion estimation unit 204 and the motion compensation unit 205 may use the current coded image where the coded block is located as a reference image to obtain the intra prediction value of the coded block.
  • intra prediction refers to a prediction value obtained by using only data in an image where a coding block is located as a reference.
  • the motion estimation unit 204 and the motion compensation unit 205 use the part that has been partially recovered in the currently encoded image, that is, the part that has not been processed by the filter unit 213, and the input data can come from the output of the second adder 212
  • an image buffer is used to store the output data of the second adder 212.
  • this image buffer is a special image buffer in the decoded image buffer 214.
  • the intra prediction unit 206 is configured to determine an intra prediction value of a coding block.
  • the intra prediction unit 206 determines the intra prediction reference samples for the coding block based on the partially recovered part of the image where the coding block is located, and uses the intra prediction reference samples as the input value of the filter to calculate the Intra prediction.
  • the filter may be an interpolation filter or a low-pass filter (for example, a filter for calculating a DC value).
  • the intra prediction unit 206 uses a method of RDO to determine a method (ie, an intra prediction mode) and an intra prediction value for calculating an intra prediction value of a coded block.
  • the intra prediction unit 206 determines whether the pixel samples adjacent to the coded block have been partially recovered. When the intra prediction unit 206 determines that the neighboring samples of the encoded block all exist, the intra prediction unit 206 sets the neighboring samples or the filtered values of the neighboring samples as the intra prediction reference samples. In contrast, when the intra prediction unit 206 determines that all or part of the samples of the coding block do not exist, the intra prediction unit 206 determines the first reference sample and uses the first reference sample to determine the intra prediction reference sample of the coding block. .
  • the intra prediction unit 206 determines whether adjacent samples exist at the positions of the adjacent pixels according to the positions of the adjacent pixels corresponding to the reference samples used in the intra prediction mode. In general, the intra prediction unit 206 determines whether adjacent samples of the coding block exist in the following manner:
  • the coding block When the coding block is located at the boundary position of the image, there are no adjacent samples at the position of the reference pixels outside the boundary of the image. For example, when the coded block is located at the upper boundary of the image, adjacent samples on the coded block do not exist;
  • the method for determining the first reference sample value by the intra prediction unit 206 includes multiple methods. One is that the intra prediction unit 206 sets the value of the first reference sample value as a non-existent neighbor of the coding block. Samples (for example, when a coded block is on a slice boundary, the first reference sample is set using the neighboring sample of the coded block in a different slice). The other is that the intra prediction unit 206 determines the first reference sample value according to the pixel sample value to be encoded in the encoding block. The other is that the intra prediction unit 206 uses a soft decision method to calculate the first reference sample value in the RDO process.
  • the soft decision can be expressed using the following constraint optimization problem:
  • D represents the coding distortion of the coding block
  • k's traversal range is all pixels in the coding block
  • d (a, b) represents the error value between a and b
  • the error criterion that can be used includes the mean square error criterion Square Error (MSE), Absolute Error Sum (Sum of Absolute Value, SAD), or perceptual error criteria related to the Human Visual System (Human Visual System, HVS)
  • x k is the sample value of the pixel to be encoded in the encoding block
  • P k is the predicted value of x k .
  • the intra prediction unit 206 uses the intra prediction reference samples to calculate the value of p k according to the intra prediction mode;
  • R represents the coding bit overhead of the coding block, including The cost of the coding mode information of the coding block, the cost of the first reference sample, the coding cost of the prediction difference, etc.
  • the value of R can be estimated according to the coding block mode, prediction difference, quantization parameters, etc. (for example, Rate-Distortion , RD) model estimation), can also be obtained by actual encoding of the coding block;
  • R C is the upper limit value of the coding block coding bit overhead, the encoder can use the rate control (Rate Control, RC) unit of the coding block Bit points
  • the allocation (Bit Allocation) process is determined.
  • the intra prediction unit 206 solves the constraint optimization 1 to determine the optimal first reference sample and the intra prediction mode at the same time.
  • the intra prediction unit 206 may use a correlation algorithm to solve the constraint optimization1.
  • Constrained Optimization 1 can be transformed into an unconstrained optimization problem:
  • r is the first reference sample
  • m is the intra prediction mode of the coding block
  • D and R are the same as D and R in constraint optimization 1
  • ⁇ is the Lagrange factor.
  • a common method is that the encoder can Use the functional relationship between ⁇ and Quantization Parameter (QP) to determine the value of ⁇ .
  • QP Quantization Parameter
  • the encoder can determine the value of QP in the RC unit.
  • the intra prediction unit 206 solves the unconstrained optimization 1, and can determine the optimal first reference sample and the intra prediction mode at the same time.
  • the intra prediction unit 206 may use an existing algorithm to solve the unconstrained optimization1.
  • the intra-prediction unit 206 needs less computational complexity to solve Unconstrained Optimization 1.
  • the intra prediction unit 206 can use the approximate method to solve the unconstrained optimization 1 according to the position of the neighboring pixels of the coding block and the used intra prediction mode, thereby further Reduce computational complexity.
  • the intra prediction unit 206 may determine one or more first reference samples for the decoded block, and for a case where the first reference sample includes multiple samples, further determine the samples in the first reference sample.
  • the use mode indicating parameter is used to indicate the sample value in the first reference sample value used in the process of determining the intra prediction reference sample value at different positions. For example, a first reference sample with a different value is used for the left adjacent sample and the upper adjacent sample of the coded block.
  • the indication parameter is also included in the first reference sample value.
  • the intra prediction unit 206 may derive the indication parameter according to a preset rule.
  • the preset rule used by the intra prediction unit 206 is: in the order in which the samples in the first reference sample are written into the code stream, the first One sample is used to construct the left-neighboring intra-prediction reference samples of the coding block, and the second sample is used to construct the upper-neighboring intra-reference reference samples of the coding block.
  • the intra prediction unit 206 may further derive the indication parameter according to the position of the encoded block in the image and the intra prediction mode of the encoded block.
  • the preset rule used by the intra prediction unit 206 is: in the order in which the samples in the first reference sample are written into the code stream, the first One sample is used to construct the left-neighboring intra-prediction reference sample of the coding block, and the second sample is used to construct the upper-neighboring intra-frame reference sample of the coding block.
  • the left adjacent samples of the image do not exist in the image) and are not the upper boundary of the image (the upper adjacent samples of the coded block exist in the image)
  • the upper boundary of the coded block is the boundary of the tile or tile (the coded block) (The upper neighboring samples of the code cannot be used as reference samples for intra prediction).
  • the coding block uses the DC mode
  • the second sample is used as the left neighboring samples and the upper neighboring samples of the coding block in this mode.
  • the intra prediction unit 206 searches for a matching block of the coding block in the part where the coding block has been partially recovered, and uses the matching block as the intra prediction value of the coding block.
  • the output parameters of the intra prediction unit 206 also include information for indicating the position of the matching block, for example, the relative offset between the positions of the coding block and the matching block in the image in the same image coordinate system. It should be noted that the output parameters of the intra prediction unit 206 include a first reference sample value and a parameter used to indicate an intra prediction mode.
  • the first adder 207 is configured to calculate a prediction difference between an original value and a prediction value of the coding block.
  • the predicted difference is an input value of the transformation unit 208.
  • the transform unit 208 is configured to transform the prediction difference and output a transform coefficient obtained by transforming the prediction difference.
  • the transformation unit 208 may use one or more transformation methods to transform the prediction difference. From the perspective of signal processing, each transformation method can be represented by a transformation matrix.
  • the transform unit 208 may use a rectangular block of the same size and shape as the encoding block (here, a square is a special case of a rectangle) as the transform block of the prediction difference value, and may also divide the prediction difference value into a plurality of rectangular blocks (including a pixel height Case), and transform the rectangular blocks in turn.
  • the transform unit 208 may transform the prediction difference multiple times.
  • the prediction difference is divided into multiple rectangular blocks for transformation, and the prediction difference is transformed multiple times, the transformation unit 208 uses the RDO method to determine the transformation parameters for the prediction difference, and the transformation Parameters and are used to indicate how the transformation process is performed.
  • the transformation unit 208 uses the transformation parameters as output parameters.
  • the quantization unit 209 is configured to perform quantization processing on the transform coefficients obtained by transforming the prediction difference output by the transform unit 208, and output the quantized values of the transform coefficients.
  • quantizers that can be used by the quantization unit include a scalar quantizer and a vector quantizer.
  • the quantization unit 209 uses a scalar quantizer to quantize the transform coefficients, and the quantization parameters of the quantizer are determined by an encoder control unit.
  • the encoder control unit may use an existing bit rate control method to determine the quantization step size of the quantizer, and determine the QP according to the correspondence between the quantization step size and the QP in the encoder.
  • the relevant parameter of the quantization unit 209 is QP.
  • the inverse quantization unit 210 is configured to use the same QP as the quantization unit 209 to perform a scaling operation on the quantized value of the transform coefficient to obtain a restored value of the transform coefficient.
  • the inverse transform unit 211 is configured to process the restored value of the transform coefficient using an inverse transform with the transform used in the transform unit 208 to obtain a restored value of the prediction difference.
  • the second adder 212 is configured to calculate the restoration value of the encoding block according to the restoration value of the prediction difference and the prediction value of the encoding block output by the prediction unit 202, and store the restoration value of the encoding block to the image buffer.
  • the image buffer may be a piece of storage space allocated separately during the image encoding process, or may be an image buffer in the decoded image buffer 214.
  • the filtering unit 213 is configured to perform filtering processing on the data in the image buffer to obtain a decoded image of the image.
  • the filtering unit 213 may be a cascade of one or more filters.
  • the filtering unit 213 is composed of two filters: a deblocking filter and a sample value adaptive additive offset compensation filter (Sample Adaptive Offset (SAO)).
  • the filtering unit 213 may also include a neural network filter.
  • the operation of filtering the data in the image buffer by the filtering unit 213 may be performed at the image layer, that is, after the restoration values of all the coding blocks in the image are written to the image buffer, the image buffer is The data is filtered.
  • the filtering operation performed by the filtering unit 213 on the data in the image buffer may be performed at the block level, that is, when the recovered data of a coding block is no longer used as reference data for subsequent coding blocks, the coding block The recovered data is filtered.
  • the filtering unit 213 determines a filter parameter using an existing RDO method, and uses it as an output parameter of the filtering unit 213.
  • the filter parameters include indication information of the used filter, filter coefficients, and control parameters of the filter.
  • the decoded image buffer 214 is configured to store a decoded image output from the filtering unit 213.
  • the decoded image buffer 214 determines parameter instructions related to the management of the decoded image, and is set to control operations such as the storage duration and output of the decoded image in the decoded image buffer 214. In this embodiment, these parameter instructions may be used as output parameters of the decoded image buffer 214.
  • the entropy encoding unit 215 is configured to perform binarization and entropy encoding on the encoded data of the image, convert the parameters into a field consisting of one or more "0" and "1" bits in accordance with the standard, and according to the code stream in the standard Syntax Structure organizes fields into code streams.
  • the entropy coded data includes texture data and non-texture data of the image.
  • the texture data is mainly the quantized value of the transform coefficients of the coding block;
  • the non-texture data includes all data except the texture data, including the output parameters of each unit in the aforementioned encoder, as well as parameter sets, header information, auxiliary information and other parameters .
  • the entropy encoding unit 215 generates a code stream according to a code stream organization form of a video coding standard.
  • an embodiment of the present disclosure provides a schematic diagram of a system architecture of a decoder for decoding a code stream generated by the foregoing encoder.
  • the decoder includes a parsing unit 301, a prediction unit 302, an inverse quantization unit 305, an inverse transform unit 306, an adder 307, a filtering unit 308, and a decoded image buffer 309.
  • the prediction unit 302 includes a motion compensation unit 303 and an intra prediction unit 304.
  • the input of the decoder is a code stream
  • the output is a decoded video generated after decoding the input code stream.
  • the parsing unit 301 is configured to parse the input code stream and use the entropy decoding method and binarization method specified in the standard to convert one or more "0" and "1" bit strings corresponding to each field in the code stream into Value of the corresponding parameter.
  • the parsing unit 301 derives the values of other parameters according to the values of the parameters. For example, when the value of the flag bit in the code stream indicates that the decoded block is the first decoded block in the image, it will be used to indicate the location of the decoded block.
  • the address parameter of the first decoded block in the picture in the picture is set to 0.
  • the parsing unit 301 passes parameters for constructing the prediction value of the decoded block to the prediction unit 302.
  • the parameters used to construct the prediction value of the decoded block include the output parameters of the division unit 201 and the prediction unit 202 in the foregoing encoder.
  • the parsing unit 301 passes parameters for constructing the decoded block prediction difference recovery value to the inverse quantization unit 305 and the inverse transform unit 306.
  • the parameters for constructing the decoded block prediction difference recovery value include the output parameters of the transform unit 208 and the quantization unit 209 in the foregoing encoder, and the quantized values of the transform coefficients output by the quantization parameter 209 in the foregoing encoder.
  • the prediction unit 302 is configured to construct a prediction value of a decoded block based on a parameter for constructing a prediction value of the decoded block.
  • the input of the prediction unit 302 also includes a partially recovered portion (not processed by the filtering unit 308) of the current decoded image output by the adder 307, and a decoded image stored in the decoded image buffer 309.
  • the prediction unit 302 uses the same method as the motion prediction unit 204 in the foregoing encoder to construct one or more reference image lists, and each reference image list includes one or Multiple reference pictures.
  • the reference pictures come from the decoded picture buffer 309.
  • the motion compensation unit 303 determines one or more matching blocks of the decoded block in the reference image according to the reference image list instruction, the reference image index, and the motion vector passed by the parsing unit 301, and uses the same The method determines the inter prediction value.
  • the prediction unit 302 uses the inter prediction value output by the motion compensation unit 303 as the prediction value of the decoded block.
  • the motion compensation unit 303 may use the current decoded image where the decoded block is located as a reference image to obtain the intra prediction value of the decoded block.
  • the intra prediction refers to a prediction value obtained by using only data in an image in which a decoding block is located as a reference.
  • the motion compensation unit 303 uses the part that has been partially recovered in the current decoded image (not processed by the filter unit 308), and the input data can come from the output of the adder 307, for example, using an image buffer to store the addition
  • the output data of the decoder 307 may optionally be a special image buffer in the decoded image buffer 309.
  • the prediction unit 302 constructs a predicted value of the decoded block according to a parameter for constructing a predicted value of the decoded block.
  • the intra prediction unit 304 obtains a first reference sample value among the parameters for constructing a decoded block prediction value.
  • the intra prediction unit 304 sets the neighboring samples or the values of the neighboring samples that have undergone the filtering process as the intra prediction reference samples.
  • the intra prediction unit 304 uses the first reference sample to determine the intra prediction reference samples of the decoded block.
  • the intra prediction unit 304 uses the following conditions to determine whether adjacent samples of the decoded block exist:
  • the decoded block When the decoded block is located at the boundary position of the image, there are no adjacent samples at the position of the reference pixels outside the boundary of the image. For example, when the decoded block is located on the upper boundary of the image, adjacent samples on the decoded block do not exist;
  • Adjacent samples located in different slices than the decoded block do not exist
  • Adjacent samples located on tiles different from the decoding block do not exist
  • the method for setting the first reference sample value by the intra prediction unit 304 includes a variety of methods.
  • One method is that the intra prediction unit 304 decodes the pixel sample values that do not exist at the position of the intra prediction reference pixel of the decoded block. Set to the value of the first reference sample. If the first reference sample value contains only one sample value, when the neighboring sample values of the decoded block do not exist in whole or in part, the intra prediction unit 304 sets the neighboring sample values of the decoded block as the first reference sample value. value.
  • the intra prediction unit 304 may determine the intra prediction reference samples at different positions according to the use of the parameter indicating method of the reference samples in the first reference sample.
  • the samples in the first reference samples used in the value process for example, use the first reference samples with different values for the left adjacent samples and the upper adjacent samples of the decoded block.
  • the intra prediction unit 304 may derive an indication parameter according to a preset rule.
  • the preset rule used by the intra prediction unit 304 is: according to the order in which the samples in the first reference sample are in the code stream ( (Ie, the parsing order of the two samples), the first sample is used to construct the left-neighboring intra prediction reference sample of the decoded block, and the second sample is used to construct the upper-neighboring intra reference of the decoded block Sample value.
  • the intra prediction unit 304 may derive the indication parameter according to the position of the decoded block in the image and the intra prediction mode of the decoded block.
  • the preset rule used by the intra prediction unit 304 is: according to the sequence of the samples in the first reference sample in the code stream (that is, Analysis order of two samples), the first sample is used to construct the left adjacent intra-prediction reference sample of the decoded block, and the second sample is used to construct the upper adjacent intra-reference sample of the decoded block Value when the decoded block is located on the left edge of the image (the left adjacent samples of the decoded block do not exist in the image) and is not the upper boundary of the image (the upper adjacent samples of the decoded block are present in the image) The boundary is the boundary of a slice or tile (the upper neighboring samples of the decoded block cannot be used as reference samples for intra prediction).
  • the second sample is used as the left of the decoded block
  • the intra prediction unit 304 determines the intra prediction mode according to the parameters used to construct the prediction value of the decoded block, and uses the same method as the intra prediction unit 206 in the aforementioned encoder to calculate the intra of the decoded block using the intra prediction reference samples. Predictive value. Alternatively, when the parameter for constructing the prediction value of the decoded block indicates that the prediction value of the decoded block is constructed using the matching block, the intra prediction unit 304 obtains between the decoded block and the matching block from the parameter for constructing the decoded block prediction value. Position offset. A matching block is obtained in the current partial decoded image where the decoding block is located, and the matching block is set as the intra prediction value of the decoding block. The prediction unit 302 uses the intra-prediction value output by the intra-prediction unit 304 as the prediction value of the decoded block.
  • the inverse quantization unit 305 is configured to receive quantized values of QP and transform coefficients in a parameter for constructing a decoded block prediction difference recovery value.
  • the inverse quantization unit 305 performs a scaling operation on the quantized value of the transform coefficient using QP to obtain a restored value of the transform coefficient. Therefore, the inverse quantization unit in the decoder may also be referred to as a scaling unit.
  • the inverse quantization unit 305 outputs to the inverse transform unit 306 the restored values of the transform coefficients and the parameters used to construct the decoded block prediction difference restored values, among the transform parameters output by the transform unit 208 in the encoder.
  • the inverse transform unit 306 uses the same method as the inverse transform unit 211 in the aforementioned encoder to calculate and obtain the recovered value of the prediction difference of the decoded block.
  • a transform method used by the inverse transform unit 306 is specified, that is, a transform method used by a decoder to convert a restored value of a transform coefficient into a restored value of a prediction difference.
  • the adder 307 calculates the restored value of the decoded block according to the restored value of the predicted difference output by the inverse transform unit 306 and the predicted value of the decoded block output by the predictive unit 302, and stores the restored value of the decoded block in the image buffer.
  • the image buffer may be a piece of storage space allocated separately during the image decoding process, or may be an image buffer in the decoded image buffer 309.
  • the filtering unit 308 is configured to receive the filter parameters output by the analysis unit 301, and perform filtering processing on the data in the image buffer according to the filter parameters to obtain a decoded image of the image.
  • the filter parameter is an output parameter of the filtering unit 213 in the foregoing encoder, and includes instruction information of a used filter, a filter coefficient, and a control parameter of the filter.
  • the filtering unit 308 may be a cascade of one or more filters.
  • the filtering unit 308 is configured by cascading two filters: a deblocking filter and a sample value adaptive additive offset compensation filter.
  • the filtering unit 308 may also include a neural network filter.
  • the operation of filtering the data in the image buffer by the filtering unit 308 may be performed at the image layer, that is, after the restoration values of all decoded blocks in the image are written to the image buffer, the image buffer is The data is filtered.
  • the operation of filtering the data in the image buffer by the filtering unit 308 may be performed at the block layer, that is, when the recovered data of a decoding block is no longer used as reference data for subsequent decoding blocks, the decoding block The recovered data is filtered.
  • the decoded image buffer 309 is configured to store the decoded image output by the filtering unit 308.
  • the decoder uses the decoded image management-related parameter instruction (that is, the output parameter of the decoded image buffer 214 in the foregoing encoder) output by the parsing unit 301 to control the storage duration of the decoded image in the decoded image buffer 309 And output operations.
  • the decoded image management-related parameter instruction that is, the output parameter of the decoded image buffer 214 in the foregoing encoder
  • the intra prediction unit 206 includes an adjacent sample detection unit 401, an adjacent sample processing unit 402, an intra prediction mode selection unit 403, and an intra prediction calculation unit 404.
  • the data flow refers to the entry parameters and return parameters of the functions on the software implementation, the data passed on the bus on the hardware implementation, and the data shared between the storage units (including register shared data).
  • the adjacent sample detection unit 401 is configured to receive a part of the data that has been partially recovered in the currently encoded image, and generate adjacent sample information of the encoded block.
  • the neighboring sample information of the coding block includes the position of the neighboring pixel where the neighboring sample is located, whether the neighboring sample exists, and the value of the neighboring sample (if the neighboring sample exists).
  • part of the data that has been partially recovered in the currently encoded image includes position information of the code block in the image and position information of the code block in the image division (for example, fragmentation).
  • the input data includes a data stream 40 and the output data is a data stream 41.
  • the data stream 40 is the output data of the second adder 212 before the encoder encodes the encoding block, and is a part of the data that has been partially recovered in the currently encoded image (not processed by the filtering unit 213 in the encoder).
  • the data stream 40 also includes the position information of the coded block in the image and the position information of the coded block in the image division (such as fragmentation).
  • the source of these position information includes the block division unit 201 and the block division unit 203 in the foregoing encoder Output data.
  • the data stream 41 is the output data of the neighboring sample detection unit 401. It is the neighboring sample information of the coding block, including the position of the neighboring pixels where the neighboring samples are located, whether the neighboring samples exist, and the values of the neighboring samples. Value (if adjacent samples exist).
  • the adjacent sample processing unit 402 is configured to receive the adjacent sample information of the coded block, the intra prediction mode indication information, and the original value of the coded block, and generate an intra prediction reference sample.
  • the input of the adjacent sample processing unit 402 is a data stream 41, a data stream 43, and a data stream 44, and an output is a data stream 42.
  • the data stream 41 is the output of the adjacent sample detection unit 401.
  • the data stream 43 is provided by the intra prediction mode selection unit 403 and includes intra prediction mode indication information.
  • the data stream 44 is output data of the block dividing unit 201 in the foregoing encoder, and is an original value of the encoded block.
  • the output data stream 42 of the neighboring sample processing unit 402 is an intra prediction reference sample.
  • the intra prediction mode selection unit 403 is configured to receive the intra prediction reference samples and the original values of the coding blocks, and generate intra prediction mode indication information.
  • the input of the intra prediction mode selection unit 403 is a data stream 42, a data stream 44, and an output is a data stream 45.
  • the data stream 42 is output data of the adjacent sample processing unit 402.
  • the data stream 44 is the output data of the block dividing unit 201 in the aforementioned encoder, and is the original value of the encoded block.
  • the data stream 45 is intra prediction mode indication information (that is, a mode index of the intra prediction mode), which indicates an intra prediction mode used for encoding a coding block, and is set to calculate an intra prediction value of the coding block.
  • the data stream 45 also contains the intra prediction reference samples in the data stream 42.
  • the intra prediction calculation unit 404 is configured to calculate and obtain the intra prediction value of the coding block according to the intra prediction reference sample value and the intra prediction mode indication information.
  • the input of the intra prediction calculation unit 404 is a data stream 45, and the output is a data stream 46.
  • the data stream 45 is output data of the intra prediction mode selection unit 403.
  • the data stream 46 is the intra prediction calculation unit 404 using the intra prediction reference samples in the data stream 45 to calculate the intra prediction of the encoded block according to the intra prediction mode indicated by the intra prediction mode indication information in the data stream 45 value.
  • the intra prediction mode selection unit 403 traverses some or all candidate intra prediction modes in the process of selecting the optimal coding mode for the coding block, calculates the intra prediction value of the coding block in the candidate mode, and selects to minimize the coding.
  • the intra prediction mode of the weighted sum of the coding distortion and coding bits of the block is used as the optimal intra prediction mode of the coding block, and the indication information of the optimal intra prediction mode and the corresponding intra prediction value are output at the same time, that is, the intra prediction mode.
  • the data stream 45 output by the selecting unit 403 includes instruction information and an intra prediction value of the optimal intra prediction mode of the coding block.
  • the intra prediction unit 206 may calculate the intra prediction value of the coded block using the intra prediction method 1 and the intra prediction method 2.
  • Intra-prediction method Intra-prediction method 1, where "Curr" represents a coded block, and its adjacent decoded pixels are represented in gray.
  • FIG. 8 is a schematic diagram of an intra prediction method provided by an embodiment of the present disclosure. In FIG. 8, the upper adjacent and left adjacent decoded pixels of a coding block are exemplarily shown. In a variety of different encoding orders, decoded pixels may also exist in the right-neighbor or lower-neighbor of the encoding block.
  • the intra prediction method 1 includes one or more intra prediction modes, for example, a DC prediction mode, a planar prediction mode, a directional interpolation prediction mode, and the like.
  • the intra prediction mode selection unit 403 in the intra prediction unit 206 outputs a parameter indicating the used intra prediction mode.
  • FIG. 9 is a schematic diagram of another intra prediction method according to an embodiment of the present disclosure, in which “Curr” represents a coding block, and a gray area represents a part of an image where the coding block is partially restored before the coding block is encoded. Data, "Ref" indicates the matching block of the coding block.
  • the intra prediction mode selection unit 403 outputs parameters indicating the construction of a matching block, for example, the position parameter of the matching block in the image.
  • the intra prediction method 2 may also include one or more intra prediction modes, for example, a block matching mode, a string matching mode, and the like.
  • the intra prediction unit 206 uses the intra prediction method 1.
  • the output data stream 45 of the intra prediction mode selection unit 403 in the intra prediction unit 206 further includes instruction information of the intra prediction method, which instructs the use of the intra prediction method 1 It is also the intra prediction method 2 to calculate the intra measured value of the coding block.
  • the neighboring sample detection unit 401 determines the position of the encoded block in the image according to the position information of the encoded block in the data stream 40. For the intra prediction method 1, the adjacent sample detection unit 401 determines whether a pixel sample exists at a pixel position adjacent to a coding block in a part of data that has been partially recovered in the image of the data stream 40.
  • the determination method used by the adjacent sample detection unit 401 is as follows.
  • the neighboring sample detection unit 401 includes the neighboring sample information related to the intra prediction method 1 into the incoming data stream 41.
  • the coding block When the coding block is located at the boundary position of the image, there are no adjacent samples at the reference pixel point position outside the image boundary. For example, when the coded block is located at the upper boundary of the image, adjacent samples on the coded block do not exist.
  • Adjacent samples located in a different slice than the coded block do not exist.
  • Adjacent samples that are located on a different tile than the coded block do not exist.
  • the neighboring sample detection unit 401 determines a range in which all samples in the "Ref" block in FIG. 9 exist. When the "Ref" block is within the range, All samples in the "Ref” block exist; conversely, some or all of the samples in the "Ref” block do not exist.
  • the adjacent sample detection unit 401 determines the range in which all samples exist in the "Ref” block using the same determination method as that set as the intra prediction method 1 described above.
  • the neighboring sample detection unit 401 includes the neighboring sample information related to the intra prediction method 2 into the incoming data stream 41.
  • the neighboring sample processing unit 402 processes the data of the data stream 41.
  • the adjacent sample processing unit 402 may optionally perform filtering processing on the adjacent samples in the data stream 41.
  • a filtering processing method is that the neighboring sample processing unit 402 may use an existing RDO method to determine whether to filter neighboring samples, and generate flag information to indicate whether to filter neighboring samples.
  • one method is to determine whether to use filtering for a block of a specific size.
  • the neighboring sample processing unit 402 uses the existing RDO method to determine whether to perform neighboring sample values of the coding block. Filter and generate flag information to indicate whether to filter adjacent samples.
  • one method is to process the neighboring samples when the size of the coding block is equal to a preset value or one of a plurality of preset values The unit 402 filters neighboring samples of the coding block.
  • one method is to determine whether to use filtering for a specific intra prediction mode.
  • the intra prediction mode in the data stream 43 is a preset mode or one of a plurality of preset modes
  • the adjacent sample processing unit 402 uses the existing RDO method to determine whether the Adjacent samples are filtered, and flag information is generated to indicate whether to filter adjacent samples.
  • one method is that if the intra prediction mode in the data stream 43 is a preset mode or one of a plurality of preset modes, The neighboring sample processing unit 402 filters neighboring samples of the coded block.
  • one method is to determine whether to use filtering for a block of a specific size and a specific intra prediction mode.
  • the intra prediction mode in the data stream 43 is a preset mode or a plurality of preset values
  • the adjacent sample processing unit 402 determines whether to filter the adjacent samples of the coding block using the existing RDO method, and generates flag information indicating whether to filter the adjacent samples.
  • one method is that when the size of the coding block is equal to a preset value or one of a plurality of preset values, and the data stream 43 When the intra prediction mode of is a preset mode or one of a plurality of preset modes, the neighboring sample processing unit 402 filters neighboring samples of the coding block.
  • the neighboring sample processing unit 402 includes neighboring samples of the coded block into the data stream 42.
  • the adjacent sample processing unit 402 determines the first reference sample.
  • the neighboring sample processing unit 402 determines that the input data of the first reference sample is the neighboring sample (if the neighboring sample part exists) in the data stream 41 and the neighboring pixel position and data stream corresponding to the neighboring sample.
  • the adjacent sample processing unit 402 calculates the first reference sample using the method of the constraint optimization 1 or the unconstrained optimization 1 in the foregoing embodiment. It should be noted that, in the method of the constraint optimization 1 or the unconstrained optimization 2, the first reference sample value used may be one or more.
  • the adjacent sample processing unit 402 needs to record the usage parameters of different first reference samples, and the parameters are set to indicate that different first reference samples are used in setting the adjacent samples. Use.
  • the adjacent sample processing unit 402 sets the value of the non-existent adjacent sample to the value of the first reference sample, and sets the adjacent samples in the input data stream 41 to "present" or "available".
  • the adjacent sample processing unit 402 may perform the filtering processing on the adjacent samples using the aforementioned filtering method.
  • the adjacent sample processing unit 402 includes the data that has been filtered or not filtered into the data stream 42.
  • the adjacent sample processing unit 402 may determine the first reference sample using a simplified method.
  • One method is that if the data stream 41 indicates that no neighboring samples of the coding block exist, the neighboring sample processing unit 402 sets the first reference sample value as the average value of the original values of the coding blocks in the data stream 44.
  • One method is that when the intra prediction mode information in the data stream 43 indicates the DC mode, the adjacent sample processing unit 402 sets the value of the first pre-reference to the average value of the original value of the encoding block in the data stream 44 and The difference between the mean values of some adjacent samples (if any) in the data stream 41.
  • One method is that the neighboring sample processing unit 402 sets the value of the first pre-reference to the average value of the original value of the coding block in the data stream 44 and the mean value of some neighboring sample values (if any) existing in the data stream 41 The difference.
  • the intra prediction mode selection unit 403 determines an intra prediction mode for a coding block.
  • the intra prediction mode selection unit 403 may determine an intra prediction mode of a coding block using an existing RDO method.
  • the intra prediction mode selection unit 403 may determine the intra prediction mode and the first reference sample using a joint optimization method together with the neighboring sample processing unit 402. For example, the intra prediction mode selection unit 403 passes the candidate intra prediction mode to the neighboring sample processing unit 402 through the data stream 43; the neighboring sample processing unit 402 determines the first reference sample value in the candidate intra prediction mode.
  • the intra prediction mode selection unit 403 uses the Adjacent samples calculate the cost function value of the candidate intra prediction mode.
  • the neighboring sample processing unit 402 determines the first reference sample using the method of the constraint optimization 1 or the unconstrained optimization 2
  • the neighboring sample processing unit 402 also determines the candidate frames at the same time.
  • the cost function value of the prediction mode In this case, the neighboring sample processing unit 402 also passes the cost function value to the intra prediction mode selection unit 403 through the data stream 42.
  • the intra prediction mode selection unit 403 compares the cost function value of each candidate intra prediction mode, and selects the candidate intra prediction mode with the smallest cost function value as the intra prediction mode of the coding block.
  • the method in which the neighboring sample processing unit 402 determines the first reference sample is similar to the method set as the intra prediction method 1 described above. The difference is that when the intra prediction method 2 is allowed, the intra prediction mode selection unit 403 includes in the data stream 43 the position of the "Ref" block in FIG. 9 in the image.
  • the neighboring sample processing unit 402 may use a constraint
  • the method of optimization 1 or unconstrained optimization 2 determines the first reference sample value, and sets the sample value that does not exist in the "Ref" block as the first reference sample value.
  • the first reference sample value used may be one or multiple.
  • the adjacent sample processing unit 402 needs to record the usage parameters of different first reference samples, and the parameters are set to indicate that different first reference samples are used in setting the adjacent samples. Use.
  • the adjacent sample processing unit 402 may determine the first reference sample using a simplified method. For example, one method is that the neighboring sample processing unit 402 sets the value of the first pre-reference to the average value of the original value of the coding block in the data stream 44 and the neighboring sample values (if any ) The difference between the mean values. In particular, if the data stream 41 indicates that no neighboring samples of the coding block exist, the neighboring sample processing unit 402 sets the first reference sample value to the average of the original values of the coding blocks in the data stream 44. value.
  • the intra prediction mode selection unit 403 determines the position of the "Ref” block for the coding block.
  • the intra prediction mode selection unit 403 may use the existing RDO method to determine the position of the “Ref” block of the coding block.
  • the intra prediction mode selection unit 403 may determine the position of the “Ref” block and the first reference sample value using a joint optimization method together with the neighboring sample processing unit 402.
  • the intra prediction mode selection unit 403 passes the position of the candidate "Ref” block to the neighboring sample processing unit 402 through the data stream 43; the neighboring sample processing unit 402 determines the first position of the candidate "Ref” block Reference samples, use this first reference sample to set the samples in the "Ref” block, and pass the processed samples in the "Ref” block to the intra prediction mode selection unit 403 through the data stream 42; the intra prediction mode The selection unit 403 calculates a cost function value of the position of the candidate "Ref” block using the samples in the "Ref” block in the data stream 42.
  • the neighboring sample processing unit 402 determines the first reference sample using the method of the constraint optimization 1 or the unconstrained optimization 2, the neighboring sample processing unit 402 also determines the candidate “Ref The cost function value of the position of the block. In this case, the neighboring sample processing unit 402 also passes the cost function value to the intra prediction mode selection unit 403 through the data stream 42.
  • the intra prediction mode selection unit 403 compares the cost function value of the position of each candidate "Ref” block, selects the position of the candidate "Ref” block with the smallest cost function value as the position of the "Ref” block of the coding block, and passes the data stream
  • the 45 output is set as a parameter indicating the position of the "Ref” block, and the parameter is expressed as the position offset between the encoding block and the "Ref" block in the same image coordinate system.
  • the adjacent sample processing unit 402 may replace the non-existing adjacent samples or “Ref” blocks. Samples that do not exist in the default settings (such as "1 ⁇ (BitDepth–1)" in the background introduction).
  • the aforementioned joint optimization method of the neighboring sample processing unit 402 and the intra prediction mode selection unit 403 uses the default value as the aforementioned one candidate first reference sample. When the default value is used, the adjacent sample processing unit 402 adds a flag bit in the data stream 42 to indicate the use of the default value.
  • FIG. 10 is a schematic flowchart of a data processing process in which an entropy coding unit of an encoder encodes an output parameter of an intra prediction unit according to an embodiment of the present disclosure.
  • the input of the processing flow is an output parameter of the intra prediction unit 206, and the output of the processing flow is a code stream corresponding to the intra prediction mode and the first reference sample value.
  • Step 501 Code a first reference sample.
  • the first reference sample value may also include a sample value control parameter.
  • the sample value control parameter indicates whether the first reference sample value or a default value is used to determine whether the coding block is encoded using the intra prediction mode. No adjacent samples exist.
  • the sample control parameters include at least one of the following control parameters: sequence layer sample control parameters, image layer sample control parameters, slice layer sample control parameters, and block layer sample control parameters.
  • the sequence layer sample control parameter is set to indicate whether it is necessary to use a first reference sample for an image in the entire video to determine a non-existent adjacent sample in the process of encoding a coding block using an intra prediction mode.
  • the intra prediction unit 206 determines the first reference sample value of the coding block if some or all neighboring samples of the coding block do not exist; otherwise,
  • the intra prediction unit 206 sets a non-existent neighboring sample value as a default value.
  • the entropy encoding unit 215 encodes sequence layer sample control parameters in a parameter set whose effective range can be the entire video.
  • the entropy encoding unit 215 may encode the sample value of the first reference sample in the parameter set whose effective range may be the entire video. The value can be set to an encoding block using the intra prediction mode throughout the video.
  • the image layer sample control parameter is set to indicate whether it is necessary to use a first reference sample for an image to determine a non-existent adjacent sample in the process of encoding a coding block using an intra prediction mode.
  • the intra prediction unit 206 determines the first reference sample value of the coded block when some or all adjacent sample values of the coded block do not exist; otherwise, When the value of the image layer sample control parameter indicates "No", the intra prediction unit 206 sets a non-existent neighboring sample value as a default value.
  • the entropy encoding unit 215 encodes image layer sample control parameters in a parameter set whose effective range may be one image.
  • the entropy encoding unit 215 may encode the sample value of the first reference sample in the parameter set whose effective range may be one image.
  • the value can be set to a coding block using an intra prediction mode in an image.
  • the slice layer sample control parameter is set to indicate whether it is necessary to use a first reference sample for a slice to determine non-existent neighboring samples in the process of encoding a coding block using an intra prediction mode.
  • the intra prediction unit 206 determines the first reference sample value of the coding block if some or all neighboring samples of the coding block do not exist; otherwise, When the value of the slice-layer sample control parameter indicates "No", the intra prediction unit 206 sets a non-existent neighboring sample value as a default value.
  • the entropy encoding unit 215 encodes slice layer sample control parameters in the slice header. Alternatively, when the value of the slice-layer sample control parameter indicates "Yes", the entropy encoding unit 215 may encode the sample value of the first reference sample in the slice header, and the sample value may be set as Coding block using intra prediction mode in one slice.
  • the block layer sample control parameter is set to indicate whether it is necessary to use a first reference sample for a coding block to determine non-existent neighboring samples in the process of coding a coding block using an intra prediction mode.
  • the intra prediction unit 206 determines the first reference sample value of the coded block when some or all neighboring sample values of the coded block do not exist; otherwise, When the value of the block-layer sample control parameter indicates "No", the intra prediction unit 206 sets a non-existent neighboring sample value as a default value.
  • the entropy encoding unit 215 encodes a block layer sample control parameter in a data unit of an encoding block.
  • the entropy encoding unit 215 may encode the sample value of the first reference sample in the data unit of the encoding block, and the sample value It may be set as a coding unit using an intra prediction mode in a data unit of the coding block or a division of the coding block.
  • the encoder may set the block layer sample control parameter and the corresponding effective range of the first reference sample as one slice.
  • An implementation method is that when an encoder first uses an intra prediction mode in a slice and uses a first reference sample to construct neighboring samples of a coding block, the entropy coding unit 215 is in a data unit of the coding block. The coding block layer sample control parameter and the first reference sample. The encoder uses the same coding block layer sample control parameters and the first reference sample for subsequent coding blocks in the slice using the intra prediction mode. The entropy coding unit 215 no longer encodes the block layer sample control parameters, First reference sample.
  • the encoder may set the block layer sample control parameter and the corresponding effective range of the first reference sample as one slice.
  • An implementation method is that when an encoder first uses an intra prediction mode in a slice and uses a first reference sample to construct neighboring samples of a coding block, the entropy coding unit 215 is in a data unit of the coding block.
  • the coding block layer sample control parameters (denoted as "control parameter 1") and the first reference sample (denoted as "sample 1"). If the encoder uses the control parameter 1 and the sample value 1 in the coding process of the subsequent coding block using the intra prediction mode in the slice, the entropy coding unit 215 no longer encodes the block layer sample control parameters, the first Reference samples.
  • the entropy encoding unit 215 encodes" control parameter 2 "and” sample 2 "in the data unit of the coding block.
  • the entropy coding Unit 215 does not encode the block layer sample control parameter and the first reference sample for the coding block whose block size meets the "not used” condition, and sets the coding block whose block size meets the "usable” condition to "inevitable use” at the encoder. At this time, the entropy encoding unit 215 does not encode the block layer sample control parameters for encoding blocks whose block size meets the "inevitable use” condition, and only encodes the first reference sample.
  • the encoder sets the neighboring sample processing unit 402 in FIG. 7 to use the intra prediction mode of the coding block as a judgment condition for whether to use the first reference sample to construct the intra prediction neighboring samples of the coding block . Then the entropy encoding unit 215 does not encode the block layer sample control parameters and the first reference sample for the coded block whose intra prediction mode meets the "not used" condition. When the coding block is set to "inevitable use", the entropy encoding unit 215 does not encode the block layer sample control parameters for the coding block whose intra prediction mode meets the "inevitable use” condition, and only encodes the first reference sample.
  • the encoder can combine the two special cases described above.
  • the entropy encoding unit 215 performs in step 501 is to encode the block layer sample control parameters. If the encoder uses one or more of sequence layer sample control parameters, image layer sample control parameters, and slice layer sample control parameters, the entropy encoding unit 215 needs to encode the sequence layer in the process of generating the corresponding parameter set. Sample control parameters and image layer sample control parameters. The entropy encoding unit 215 needs to encode the slice layer sample control parameters in the process of encoding the slice header. You can also use the aforementioned optional method to set the block layer encoding to the slice layer. Controlled block-level sample control parameters.
  • the encoder can set the sequence layer, image layer, and slice layer sample control parameters according to the configuration file, and can also use the RDO method to dynamically determine the values of the above control parameters.
  • the configuration file records parameter settings that are set to initialize the encoder.
  • the range of the sequence layer is larger than the image layer
  • the range of the image layer is larger than the slice layer
  • the range of the slice layer is larger than the block layer.
  • the control mechanism of the control parameters is: when a control parameter with a large control range indicates "available”, a control parameter with a small control range is coded and set to indicate "availability" within this smaller control range.
  • the control range of the block-level filtering control parameters of the coding block is greater than the sub-blocks, that is, when the block-level sample control parameters of the coding block are When “not used” is indicated, the sub-block does not use the first reference sample value to determine the reference sample value for intra prediction, and the entropy encoding unit 215 does not need to encode the block-layer sample value control parameter of the sub-block.
  • the entropy encoding unit 215 may correspondingly encode or derive a first reference sample value for the control parameter.
  • the first reference sample value corresponding to a control parameter with a small effective control range covers the first reference sample value corresponding to a control parameter with a larger control range.
  • the entropy encoding unit 215 encodes a first reference sample for a coded block, then in the process of constructing adjacent samples of the intra prediction mode, the first reference sample of the block layer is used instead of Slice first reference sample.
  • Step 502 Encode an intra prediction mode.
  • the entropy encoding unit 215 encodes an intra prediction mode of an encoding block.
  • Case 1 The intra prediction unit 206 of the encoder uses only the intra prediction method 1 shown in FIG. 7.
  • the entropy encoding unit 215 first derives one or more intra prediction modes that are most likely to be used by the coded block according to the intra prediction modes of the adjacent coded block of the coded block.
  • the entropy coding unit 215 encodes a flag bit, and the value of the flag bit is set to indicate "intra frame used by the coding block"
  • the prediction mode is the mode most likely to be used within the intra prediction mode "; if the most likely used intra prediction mode contains more than one mode, the intra prediction mode used by the entropy encoding unit 215 to encode the encoding block is the most likely Index number in the intra prediction mode used.
  • the intra prediction mode used by the coded block is one of the most likely used intra prediction modes
  • the intra prediction mode used by the coded block is in the frame other than the most likely used intra prediction mode.
  • the intra prediction unit 206 of the encoder uses the intra prediction method 1 in FIG. 8 and the intra prediction method 2 in FIG. 9.
  • the coding is used to indicate identification information of whether the intra prediction mode of the coding block belongs to the intra prediction method 1 or the intra prediction method 2.
  • An alternative use method is that the entropy encoding unit 215 encodes a flag indicating the above information.
  • An alternative method is that the intra prediction flag of the coding block is only used to indicate that the coding block uses the intra prediction mode in intra prediction method 1.
  • the entropy coding The unit 215 encodes the intra prediction flag of the coded block as "No", and encodes the reference image index of the coded block to a value corresponding to "the reference image used by the coded block is the image where the coded block is located".
  • the entropy encoding unit 215 may further encode a field related to the intra prediction method 2 to indicate an intra prediction mode used by the coding block.
  • the intra prediction mode that may be included in the intra prediction method 2 has a block matching mode. , String matching mode, etc., the entropy encoding unit 215 continues to encode parameters used to determine the matching block and the matching string, and the parameters are used to determine the position of the matching block and the matching string in the image where the encoding block is located.
  • Step 503 The entropy encoding unit 215 writes the encoded bits generated in the encoding process into the output code stream.
  • FIG. 11 is a schematic structural diagram of an intra prediction unit of a decoder according to an embodiment of the present disclosure.
  • the intra prediction unit 304 includes a neighboring sample detection unit 601, a neighboring sample processing unit 602, and an intra prediction calculation unit 603.
  • the input data of the adjacent sample detection unit 601 is part of the data that has been partially recovered in the currently decoded image, and the intra prediction mode indication information of the decoded block; the output data is the adjacent sample information of the decoded block.
  • the input data of the adjacent sample detection unit 601 is a data stream 60, a data stream 64, and the output data is a data stream 61.
  • the data stream 60 is the output data of the adder 307 before the decoder decodes the decoded block, and is a part of the data that has been partially recovered in the currently decoded image (not processed by the filtering unit 308 in the decoder).
  • the data stream 60 also includes the position information of the decoded block in the image and the position information of the decoded block in the image division (for example, fragmentation).
  • the source of the position information is the output data of the parsing unit 301 in the decoder.
  • the data stream 64 is the intra prediction mode indication information of the decoded block output by the parsing unit 301, and is set to indicate the intra prediction mode used by the decoded block.
  • the data stream 61 is the output data of the neighboring sample detection unit 601, and is the neighboring sample information of the decoded block, including the position of the neighboring pixel where the neighboring sample is located, whether the neighboring sample exists, and the value of the neighboring sample. Value (if adjacent samples exist).
  • the data stream 61 also includes the decoded block size.
  • the input of the neighboring sample processing unit 602 is the neighboring sample information of the decoded block and the first reference sample, and the output is the intra prediction reference sample.
  • the input of the adjacent sample processing unit 602 is a data stream 61, a data stream 62, and a data stream 64, and an output is a data stream 63.
  • the data stream 61 is the output of the adjacent sample detection unit 601.
  • the data stream 62 is a first reference sample value output by the parsing unit 301, and includes a sample value control parameter and a value of the first reference sample value.
  • the decoder inputs the data stream 64 to the adjacent sample processing unit 602.
  • the data stream 64 is the intra prediction mode indication information of the decoded block output by the parsing unit 301, and is set to indicate the intra prediction mode used by the decoded block.
  • the output data stream 63 of the neighboring sample processing unit 602 is an intra prediction reference sample.
  • the input of the intra prediction calculation unit 603 is an intra prediction reference sample and the data stream 64, and the output is an intra prediction value of the decoded block.
  • the input of the intra prediction calculation unit 603 is the intra prediction mode indication information of the data stream 63 and the decoded block, and the output is the data stream 65.
  • the data stream 63 is output data of the adjacent sample processing unit 602.
  • the data stream 64 is the intra prediction mode indication information of the decoded block output by the parsing unit 301, and is set to indicate the intra prediction mode used by the decoded block.
  • the intra prediction calculation unit 603 calculates the intra prediction value of the decoded block according to the intra prediction mode indicated by the intra prediction mode indication information in the data stream 64, and uses the intra prediction reference samples in the data stream 63 as output data.
  • Stream 65 is the intra prediction mode indication information of the data stream 63 and the decoded block, and the output data stream 65.
  • the intra prediction unit 304 in the foregoing decoder may use the intra prediction method 1 shown in FIG. 8 and the intra prediction method 2 shown in FIG. 9 to calculate the intra prediction value of the decoded block. , Output data stream 65.
  • “Curr” represents a decoded block, and adjacent decoded pixels are represented by gray.
  • the upper and left adjacent decoded pixel points of the decoded block are exemplarily marked.
  • the intra prediction method 1 includes one or more intra prediction modes, for example, a DC prediction mode, a planar prediction mode, a directional interpolation prediction mode, and the like.
  • the data stream 64 input to the intra prediction calculation unit 603 of the decoder is an index of the intra prediction mode
  • the output data stream 65 is a decoded block obtained using the intra prediction mode indicated by the data stream 64. Intra prediction.
  • “Curr” represents a decoded block
  • a gray area represents a part of data in the image where the decoded block is located before decoding the decoded block.
  • “Ref” represents a decoded block. Matching blocks.
  • the data stream 64 input by the decoder to the intra prediction calculation unit 603 is position information set to indicate "Ref", for example, between “Curr” and “Ref” in the same image coordinate system
  • the position offset of the output data stream 65 is to obtain “Ref” by using the "Ref" position information indicated by the data stream 64, and use "Ref” to construct the intra prediction value of the decoded block, for example, set the intra prediction value of the decoded block. Is the value in "Ref” or the value after filtering the sample in "Ref".
  • the intra prediction method 2 may also include one or more intra prediction modes, for example, a block matching mode, a string matching mode, and the like.
  • the intra prediction unit 304 uses the intra prediction method 1.
  • the data stream 64 contains the instruction information of the intra prediction method, and determines whether to use the intra prediction method 1 or the intra prediction method 2 to calculate the decoding block in the process of decoding the decoded block. First predicted value.
  • the neighboring sample detection unit 601 determines the position of the decoded block in the image according to the position information of the decoded block in the data stream 60.
  • the adjacent sample detection unit 601 determines that part of the data that has been partially recovered in the image of the data stream 60 is adjacent to the decoded block Whether a pixel sample exists at the pixel position of.
  • the determination method used by the adjacent sample detection unit 601 is as follows.
  • the neighboring sample detection unit 601 includes the neighboring sample information related to the intra prediction method 1 into the incoming data stream 61.
  • the data stream 61 also includes information on the size of the decoded block.
  • the decoded block When the decoded block is located at the boundary position of the image, there are no adjacent samples at the position of the reference pixels outside the boundary of the image. For example, when the decoded block is located on the upper boundary of the image, adjacent samples on the decoded block do not exist;
  • Adjacent samples located in different slices than the decoded block do not exist
  • Adjacent samples located on tiles different from the decoding block do not exist
  • the adjacent sample detection unit 601 determines whether all samples in the "Ref” block in FIG. 9 exist.
  • the neighboring sample detection unit 601 determines the position of "Ref” in the image according to the position information of "Ref” in the data stream 64 and the position of the decoded block in the image.
  • the adjacent sample value detection unit 601 determines the range in which all the sample values in the "Ref” block exist using the same determination method as that used for the intra prediction method 1 described above.
  • the neighboring sample detection unit 601 includes the neighboring sample information related to the intra prediction method 2 into the incoming data stream 61.
  • the data stream 61 also includes information on the size of the decoded block.
  • the neighboring sample processing unit 602 processes the data of the data stream 61.
  • the intra prediction mode indication information in the data stream 64 instructs the decoding block to use the intra prediction method 1
  • the adjacent samples in the data stream 61 all exist, according to the indication information in the data stream 64, the adjacent samples
  • the processing unit 602 performs filtering processing on adjacent sample values in the data stream 61.
  • one method is that the neighboring sample processing unit 602 determines whether to filter the neighboring samples according to the flag information contained in the data stream 64.
  • one approach is to use filtering only for blocks of a specific size.
  • the adjacent sample processing unit 602 determines whether or not according to the flag information included in the data stream 64. Filter the neighboring samples of the decoded block.
  • One method is that if the size of the decoded block in the data stream 61 is equal to a preset value or one of a plurality of preset values, the adjacent sample processing unit 602 filters the adjacent samples of the decoded block.
  • one approach is to use filtering only for specific intra prediction modes.
  • the intra prediction mode in the data stream 64 is a preset mode or one of a plurality of preset modes
  • the adjacent sample processing unit 602 determines according to the flag information included in the data stream 64. Whether to filter neighboring samples of the decoded block.
  • one method is that, if the intra prediction mode in the data stream 64 is a preset mode or one of a plurality of preset modes, the adjacent sample processing unit 602 performs adjacent samples of the decoded block. Values are filtered.
  • one method is to use filtering on specific size blocks, specific intra prediction modes.
  • the size of the decoded block in the data stream 61 is equal to a preset value or one of a plurality of preset values, and the intra prediction mode in the data stream 64 is a preset mode or a preset
  • the adjacent sample processing unit 602 determines whether to filter the adjacent samples of the decoded block according to the flag information included in the data stream 64.
  • one method is if the size of the decoded block in the data stream 61 is equal to a preset value or one of a plurality of preset values, and the intra prediction mode in the data stream 64 is preset In the mode or one of a plurality of preset modes, the adjacent sample processing unit 602 filters the adjacent samples of the decoded block.
  • the neighboring sample processing unit 602 includes neighboring samples of the decoded block into the data stream 63.
  • the adjacent sample processing unit 602 Obtain a first reference sample from the data stream 62, including a sample control parameter and a value of the first reference sample.
  • the sample control parameter indicates that the first reference sample is used to construct a non-existent adjacent sample
  • the adjacent sample processing unit 602 sets the value of the non-existent adjacent sample to the value of the first reference sample. , Set the adjacent samples of the decoded block to "present" or "available".
  • the neighboring sample processing unit 602 selects non-existing neighboring samples according to the usage mode parameters of different first reference samples in the data stream 62.
  • the value is set to the value of the corresponding first reference sample value, and the adjacent sample values of the decoded block are all set to "exist” or "available”.
  • the adjacent sample processing unit 602 uses a default value (for example, “1 ⁇ (BitDepth–1)” in the introduction section of the background art ) Set neighboring samples that do not exist, and set neighboring samples of the decoded block to "exist” or "available”.
  • the neighboring sample processing unit 602 may use the aforementioned filtering method to perform filtering processing on the neighboring samples.
  • the neighboring sample processing unit 602 includes neighboring samples of the decoded block into the data stream 63.
  • the intra prediction mode indication information in the data stream 64 instructs the decoding block to use the intra prediction method 2
  • the sample control parameter indicates that the first reference sample is used to construct a sample that does not exist in "Ref”
  • the adjacent The sample value processing unit 602 sets the value of a sample value that does not exist in "Ref" to the value of the first reference sample value.
  • the adjacent sample processing unit 602 replaces the samples that do not exist in "Ref" according to the usage mode parameters of the different first reference samples in the data stream 62.
  • the value of is set to the value of the corresponding first reference sample value.
  • the adjacent sample value processing unit 602 uses a default value (for example, using the sample value of the pixel position of the image or the slice boundary). Image filling method) Set non-existent neighboring samples, and set neighboring samples of decoded blocks to "present” or "available”.
  • the neighboring sample processing unit 602 includes the samples in “Ref” into the data stream 63.
  • the intra prediction calculation unit 603 calculates the intra prediction value of the decoded block according to the intra prediction mode indicated by the intra prediction mode indication information in the data stream 64, and uses the intra prediction reference samples in the data stream 63 as output data.
  • Stream 65 The intra prediction calculation unit 603 calculates the intra prediction value of the decoded block according to the intra prediction mode indicated by the intra prediction mode indication information in the data stream 64, and uses the intra prediction reference samples in the data stream 63 as output data.
  • Stream 65 The intra prediction calculation unit 603 calculates the intra prediction value of the decoded block according to the intra prediction mode indicated by the intra prediction mode indication information in the data stream 64, and uses the intra prediction reference samples in the data stream 63 as output data.
  • FIG. 12 is a schematic flowchart of a data processing process in which a parsing unit of a decoder according to an embodiment of the present disclosure analyzes intra prediction modes and filtering parameters in a decoded block code stream.
  • the input of the processing flow is the code stream corresponding to the first reference sample and the intra prediction mode in the input code stream, and the output is the first reference sample and the intra prediction mode.
  • the aforementioned decoder sets the first reference sample as the input data stream 62 of the adjacent sample processing unit 602 in the intra prediction unit 304.
  • the aforementioned decoder uses the intra prediction mode as the data stream 64 used in the intra prediction unit 304.
  • Step 701 Parse the first reference sample value.
  • the first reference sample value may further include a sample value control parameter, which indicates whether the first reference sample value or a default value is used to determine whether the decoding block is decoded using the intra prediction mode. No adjacent samples exist.
  • the sample control parameters include at least one of the following control parameters: sequence layer sample control parameters, image layer sample control parameters, slice layer sample control parameters, and block layer sample control parameters.
  • the sequence layer sample control parameter is used to indicate whether it is necessary to use a first reference sample for an image in the entire video to determine non-existent adjacent samples in the process of decoding the decoded block using the intra prediction mode.
  • the intra prediction unit 304 sets the phase of the decoded block by using the first reference sample when some or all adjacent samples of the decoded block do not exist. Neighbor samples; conversely, when the value of the sequence layer sample control parameter indicates "No", the intra prediction unit 304 will use the default value to set non-existent neighboring samples.
  • the analysis unit 301 obtains sequence layer sample control parameters in a parameter set whose effective range can be the entire video.
  • the parsing unit 301 may obtain the sample value of the first reference sample in the parameter set whose effective range may be the entire video, and the sample value The value can be used for decoding blocks using the intra prediction mode throughout the video.
  • the image layer sample control parameter is used to indicate whether it is necessary to use a first reference sample for an image to determine a non-existent adjacent sample in the process of decoding a decoded block using an intra prediction mode.
  • the intra prediction unit 304 sets the phase of the decoded block using the first reference sample when some or all adjacent samples of the decoded block do not exist. Adjacent samples; conversely, when the value of the image layer sample control parameter indicates "No", the intra prediction unit 304 uses the default value to set the adjacent samples of the decoded block.
  • the analysis unit 301 obtains image layer sample control parameters in a parameter set whose effective range can be one image.
  • the parsing unit 301 may obtain the sample value of the first reference sample in the parameter set whose effective range may be one image, and the sample value The value can be used for an image decoding block using the intra prediction mode.
  • the slice layer sample control parameter is used to indicate whether it is necessary to use a first reference sample for a slice to determine adjacent samples that do not exist in the process of decoding the decoded block using the intra prediction mode.
  • the intra prediction unit 304 uses the first reference sample value of the decoded block to set the phase when some or all adjacent samples of the decoded block do not exist. Adjacent samples; conversely, when the value of the slice-layer sample control parameter indicates "No", the intra prediction unit 304 uses the default value to set the adjacent samples of the decoded block.
  • the analysis unit 301 obtains slice layer sample control parameters in the slice header. Alternatively, when the value of the slice-layer sample control parameter indicates "Yes", the parsing unit 301 may obtain the sample value of the first reference sample in the slice header, and the sample value may be used for one slice. Decoded block using intra prediction mode in the slice.
  • the block layer sample control parameter is used to indicate whether it is necessary to use a first reference sample for a decoded block to determine non-existent adjacent samples in the process of decoding the decoded block using the intra prediction mode.
  • the intra prediction unit 304 sets the phase of the decoded block using a first reference sample when some or all adjacent samples of the decoded block do not exist. Adjacent samples; conversely, when the value of the block-layer sample control parameter indicates "No", the intra prediction unit 304 uses the default value to set the adjacent samples of the decoded block.
  • the parsing unit 301 obtains a block layer sample control parameter in a data unit of a decoded block.
  • the parsing unit 301 may obtain the sample value of the first reference sample in the data unit of the decoded block, and the sample value is available A decoding block using an intra prediction mode or a division of the decoding block in a data unit of a decoding block.
  • the decoder may configure the block layer sample control parameter and the corresponding first reference sample effective range as one slice.
  • An implementation method is that, for the first use of the intra prediction mode in the segment and the use of the first reference samples to construct the neighboring samples of the decoded block, the parsing unit 301 obtains a block layer in the data unit of the decoded block. The sample control parameters and the first reference sample. The decoder uses the same decoding block layer sample control parameter and the first reference sample for subsequent decoding blocks in the slice that use the intra prediction mode.
  • the decoder may configure the block layer sample control parameter and the corresponding first reference sample effective range as one slice.
  • An implementation method is that, for the first use of the intra prediction mode in the segment and the use of the first reference samples to construct the neighboring samples of the decoded block, the parsing unit 301 obtains a block layer in the data unit of the decoded block.
  • a sample control parameter (denoted as "control parameter 1") and a first reference sample (denoted as "sample 1").
  • the parsing unit 301 sets the first reference sample value of the decoded block to the control parameter 1 and the sample value 1 as data
  • the stream 62 is passed to an adjacent sample processing unit 602. If there is a new first reference sample (denoted as "control parameter 2", "sample 2") in the subsequent decoded block data unit of the slice using the intra prediction mode, the parsing unit 301 obtains a new first reference sample The value is passed to the adjacent sample processing unit 602 as a data stream 62.
  • the process in which the decoder configures the block layer sample control parameters and the corresponding first reference sample effective range as one slice can be summarized as the parsing unit 301 always uses the first obtained from the decoded block included in the slice for the last time.
  • an embodiment of the present disclosure proposes one. If the decoder configures the neighboring sample processing unit 602 in FIG. 10 to use the decoded block size as a judgment condition for whether to use the first reference sample to construct the intra prediction neighboring samples of the decoded block, the parsing unit 301 may not parse Decode the code stream of the block and directly set the block layer sample control parameter of the decoded block whose block size meets the "not used" condition to "No", and configure the decoded block with the block size that meets the "usable” condition at the decoder to " When it is used inevitably, the parsing unit 301 may directly set the block layer sample control parameter of the decoded block whose block size meets the "inevitably used” condition without parsing the code stream of the decoded block, and the parsing unit 301 parses the decoded block. The code stream obtains the value of the first reference sample at the block layer.
  • the “configuration” here refers to the configuration of the decoding process performed by
  • the decoder configures the adjacent sample processing unit 602 in FIG. 7 to use the intra prediction mode of the decoded block as a judgment condition for whether to use the first reference sample to construct the intra predicted adjacent samples of the decoded block.
  • the parsing unit 301 does not parse the data unit of the decoded block whose intra prediction mode meets the "not used” condition, and sets the decoded block layer sample control parameter to "No".
  • the parsing unit 301 does not parse a decoded block data unit whose intra prediction mode meets the "inevitable use” condition, and The block layer sample control parameter is set to "Yes", and the parsing unit 301 parses the code stream of the decoded block to obtain the value of the first reference sample value of the block layer.
  • “Configuration” here refers to the configuration of the decoding process performed by the decoder according to the parameter set instruction information or preset rules.
  • the decoder can combine the two special cases described above.
  • the analysis unit 301 executes step 701 to complete decoding of the block layer sample control parameter and the value of the first reference sample. If the parsing unit 301 obtains one or more of a sequence layer sample control parameter, an image layer sample control parameter, and a slice layer sample control parameter by analyzing the code stream, it needs to be explained that the effective control from the control parameters In terms of scope, the range of the sequence layer is larger than the image layer, the range of the image layer is larger than the slice layer, and the range of the slice layer is larger than the block layer.
  • control mechanism of control parameters is: when a control parameter with a large control range indicates "available”, parsing a control parameter with a small control range to obtain information for indicating "availability" within this smaller control range .
  • the control range of the block-layer filtering control parameter of the decoding block is larger than the sub-block, that is, when the block-layer filtering control parameter of the decoding block indicates When "not used”, the sub-block does not use adaptive filtering, and the parsing unit 301 no longer needs to parse the data unit of the sub-block to obtain the block-layer filtering control parameters of the sub-block.
  • the analysis unit 301 may analyze or derive the value of the first reference sample value corresponding to the control parameter.
  • the value of the first reference sample value corresponding to the control parameter with a small effective control range covers the value of the first reference sample value corresponding to the control parameter with a larger control range.
  • the parsing unit 301 obtains the block-layer first reference samples of the decoded block
  • the decoder uses the block-layer first reference samples for determining the neighboring samples of the intra prediction mode for the decoded block. Instead of the first reference sample of the slice layer.
  • Step 702 The parsing unit 301 parses the code stream to obtain an intra prediction mode of the decoded block.
  • Case 1 The decoder uses only the intra prediction method 1 in FIG. 8 of the intra prediction unit 304.
  • the parsing unit 301 first derives one or more intra-prediction modes most likely to be used by the decoded block according to the intra-prediction modes of the decoded block adjacent to the decoded block.
  • the parsing unit 301 parses a flag bit, and the value of the flag bit indicates whether "the intra prediction mode used by the decoded block is the mode in the most likely intra prediction mode". If “Yes”, if the intra prediction mode most likely to be used includes only one intra prediction mode, the parsing unit 301 sets the intra prediction mode to the intra prediction mode of the decoded block; if the intra prediction mode is most likely to be used The prediction mode only includes multiple intra prediction modes. The parsing unit 301 parses the index number, and sets the intra prediction mode corresponding to the index number in the most likely used intra prediction mode to the intra prediction mode of the decoded block.
  • the parsing unit 301 parses the index number and sets the intra prediction mode used by the decoded block to a value other than the intra prediction mode among the most likely used intra prediction modes.
  • Case 2 The decoder uses the intra prediction method 1 and the intra prediction method 2 in FIG. 8 of the intra prediction unit 304.
  • the analysis unit 301 analyzes identification information used to indicate whether the intra prediction mode of the decoded block belongs to the intra prediction method 1 or the intra prediction method 2.
  • An optional use method is that the parsing unit 301 parses a flag bit indicating the above information.
  • An optional method is: when the parsing unit 301 parses the decoded block's intra prediction flag as "No" and the reference image index of the decoded block indicates "the reference image used by the decoded block is the image where the decoded block is located", the decoded block
  • the intra prediction mode is set to intra prediction method 2.
  • the parsing unit 301 may further analyze fields related to the intra prediction method 2 in the bitstream to determine the intra prediction mode used by the decoded block.
  • the intra prediction modes that can be included in the intra prediction method 2 include a block matching mode. , String matching mode, etc.
  • the parsing unit 301 continues to parse the code stream to obtain a determined matching block and parameters related to the matching string for determining the position of the matching block and the matching string in the image where the decoded block is located. For example, the parsing unit 301 parses the bitstream to obtain the position offset between the decoded block and the “Ref” block in FIG. 8 in the same image coordinate system.
  • Step 703 The parsing unit 301 ends the process of parsing the intra prediction mode and filtering parameters of the decoded block.
  • the aforementioned decoder sets the first reference sample as the input data stream 62 of the adjacent sample processing unit 602 in the intra prediction unit 304.
  • the aforementioned decoder uses the intra prediction mode as the data stream 64 used in the intra prediction unit 304.
  • an embodiment of the present disclosure proposes an encoding device including an encoder.
  • the encoding device includes an acquisition unit 801, an encoder 802, and a transmission unit 803. among them:
  • the acquisition unit 801 is configured to acquire a video or an image.
  • the acquisition unit 801 may include at least one camera for capturing natural video or natural images; optionally, the acquisition unit 801 may also be configured with a camera for acquiring depth video or depth images; alternatively, the acquisition unit may also Configure an infrared camera; optionally, the acquisition unit can also be equipped with a remote sensing camera.
  • the acquisition unit 801 may be a device or a device containing a video or an image generated by radiation transmission or scanning.
  • the input unit 801 may perform pre-processing on the input video or image, for example, auto focus, auto white balance, auto exposure, backlight compensation, noise reduction, sharpening, stitching, increase or decrease image resolution, increase or Reduce video frame rate, virtual view composition, etc.
  • the acquisition unit 801 may also receive video or images output by other devices or units.
  • the acquisition unit 801 may be a constituent unit of a transcoder, and the transcoder inputs a part of the decoded image to the acquisition unit 801.
  • the acquisition unit 801 receives a video or an image transmitted from another device through a data connection.
  • the collecting unit 801 may also collect other media information, such as audio.
  • the collecting unit 801 may also receive artificially generated information, such as text, subtitles, computer-generated pictures or videos.
  • the encoder 802 is configured to receive the video or image output by the acquisition unit 801, encode the video or image, and output a video or image bitstream.
  • the storage or sending unit 803 is configured to receive the video or image code stream output by the encoder 802, and perform system layer processing on the video or image code stream.
  • the storing or sending unit 803 performs system layer processing on the video or image bitstream, including: packaging according to standards such as a transmission protocol and a media file format.
  • the storage or sending unit 803 stores the obtained transport stream or media file processed by the system layer into the memory of the electronic device or sends it through a wired or wireless network.
  • the input of the storage or sending unit 803 may also include an audio code stream, text, subtitles, pictures, and the like.
  • the storage or sending unit 803 packages these input and code streams output by the encoder 802 into a transport stream or a media file according to standards such as a media file format and a transmission protocol.
  • the electronic device in this embodiment may be a device capable of generating or processing a video or image stream in a video communication application, for example, a mobile phone, a computer, a media server, a portable mobile terminal, a digital video camera, a digital camera, a television broadcasting system device, content distribution Network equipment, surveillance cameras, conference television system equipment, etc.
  • the decoding device includes a receiving unit 901, a decoder 902, and a presentation unit 903. among them:
  • the receiving unit 901 is configured to receive a video or image bitstream.
  • the receiving unit 901 receives a video or image code stream from a wired or wireless network, or reads the memory of an electronic device to obtain a video or image code stream, or receives a video or image code stream transmitted from another device through a data connection.
  • the input of the receiving unit 901 may also be a transport stream or a media file containing a video or image code stream.
  • the receiving unit 901 extracts a video or image code stream from the received transport stream or media file according to standards such as a transmission protocol and a media file format.
  • the output of the receiving unit 901 may also include an audio code stream, text, subtitles, pictures, and the like.
  • the receiving unit 901 passes these outputs to a corresponding processing unit in the electronic device.
  • the receiving unit 901 outputs an audio code stream to an audio decoder included in the electronic device.
  • the decoder 902 is configured to decode the received video or image bitstream output by the receiving unit 901, and output the decoded recovered video or image.
  • the rendering unit 903 is configured to receive the decoded recovered video or image output by the decoder 902 and perform a rendering operation.
  • the presentation unit 903 may be a component part of an electronic device, such as a display screen; it may also be an independent device connected to the electronic device through a data connection, such as a projector, a display, and the like.
  • the rendering unit 903 may perform post-processing on the decoded recovered video or image, such as automatic focus adjustment, automatic white balance, automatic exposure adjustment, backlight compensation, noise reduction, sharpening, stitching, enhancing or reducing image resolution, Increase or decrease video frame rate, virtual view composition, etc.
  • the input of the presentation unit 903 may also include media data output from other units of the electronic device, such as audio, text, subtitles, pictures, and so on.
  • the input of the presentation unit 903 also includes artificially generated data, such as marked data such as the underline of the local lecturer on key content in the distance education application.
  • the presentation unit 903 superimposes the input media data and displays it to the viewer.
  • the electronic device in this embodiment may be a device capable of decoding or processing a video or image stream in a video communication application, for example, a mobile phone, a computer, a set-top box, a television, a player, a media server, a portable mobile terminal, a digital video camera, or a digital camera , Television broadcasting system equipment, content distribution network equipment, conference television system equipment, etc.
  • a mobile phone for example, a mobile phone, a computer, a set-top box, a television, a player, a media server, a portable mobile terminal, a digital video camera, or a digital camera , Television broadcasting system equipment, content distribution network equipment, conference television system equipment, etc.
  • an embodiment of the present disclosure proposes an electronic system including an encoding device and a decoding device.
  • the source device 1001 includes an encoding device shown in FIG. 13.
  • the storage or transmission network 1002 may include a memory of a device or an electronic system, and an external storage that performs data read and write operations through a data connection; and may also include a data transmission network composed of a wired network and a wireless network.
  • the storage or transmission network 1002 provides a storage or data transmission network for the storage or transmission unit 803 in the source device 1001.
  • the sink device 1003 includes a decoding device shown in FIG. 14.
  • the receiving unit 901 in the sink device 1003 receives the video or image code stream provided by the storage or transmission network 1002, the transport stream containing the video or image code stream, or the media file containing the video or image code stream.
  • An embodiment of the present disclosure provides an encoder including: a first processor, a first storage medium, and a first communication bus, wherein the first processor and the first storage medium are connected through a first communication bus;
  • the first processor invokes an image encoding-related program stored in the first storage medium, and executes the following steps:
  • the first reference sample value, the intra prediction mode, and the prediction difference parameter are encoded, and the encoded bits are written into the code stream.
  • An embodiment of the present disclosure provides a decoder including: a second processor, a second storage medium, and a second communication bus, wherein the second processor and the second storage medium are connected through a second communication bus;
  • the second processor invokes a program related to image decoding stored in the second storage medium, and executes the following steps:
  • an embodiment of the present disclosure provides a computer storage medium, which is applied to an encoder.
  • the computer-readable storage medium stores one or more image coding related programs, and one or more image coding related programs may be Or multiple first processors execute to implement any one of the image coding methods described above.
  • an embodiment of the present disclosure provides a computer storage medium, which is applied to a decoder.
  • the computer-readable storage medium stores one or more image decoding-related programs, and one or more image decoding-related programs can be Or multiple second processors execute to implement any one of the image decoding methods described above.
  • the electronic system in this embodiment may be a system or device capable of generating, storing, or transmitting and decoding a video or image stream in a video communication application, for example, a mobile phone, a computer, an IPTV system, an OTT system, an Internet multimedia system, or a digital television broadcasting system. , Monitoring systems, portable mobile terminals, digital video cameras, digital cameras, conference television system equipment, etc.
  • the embodiments of the present disclosure may be provided as a method or a computer program product. Therefore, the present disclosure may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, magnetic disk storage, optical storage, and the like) containing computer-usable program code therein.
  • a computer-usable storage media including, but not limited to, magnetic disk storage, optical storage, and the like
  • These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to work in a specific manner such that the instructions stored in the computer-readable memory produce a manufactured article including an instruction device, the instructions
  • the device implements the functions specified in one or more flowcharts and / or one or more blocks of the block diagram.
  • the first reference sample value is calculated and determined according to the pixel sample value to be encoded of the coding block, and the first value will be further used to construct the intra prediction reference sample value of the coding block. Therefore, when the samples of the neighboring blocks of the coding block are unavailable or partially unavailable, the prediction difference between the intra prediction prediction value and the coding block is effectively reduced, and the coding efficiency of the intra prediction mode is improved. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

本公开实施例提供了一种图像编码方法，包括：根据编码块中的像素样值，确定编码块的第一参考样值；根据第一参考样值，确定编码块的帧内预测模式，并计算编码块的帧内预测值；根据编码块的原始值与帧内预测值，获得预测差参数；对第一参考样值、帧内预测模式和预测差参数进行编码，将编码比特写入码流。另外，本公开还提供一种图像解码方法、编码器、解码器以及计算机存储介质。

Description

一种图像编码方法、解码方法、编码器、解码器及存储介质 技术领域
本公开涉及通讯技术领域，尤其涉及一种图像编码方法、解码方法、编码器、解码器及存储介质。
背景技术
数字视频和图像的压缩编码技术利用视频和图像中像素点样值之间的相关性来去除视频和图像中存在的冗余。其中，压缩编码技术包括帧内预测(Intra Prediction)和帧间预测(Inter Prediction)。具体的，帧内预测是利用图像中相邻像素点样值之间的空间域相关性，通过使用相邻已编码像素点预测待编码像素点，从而降低了空间域相邻像素点之间的冗余。帧间预测不但利用单帧图像中空间域相邻像素点之间相关性，还利用了时域相邻图像之间的相关性，例如，使用运动估计(Motion Estimation，ME)和运动补偿(Motion Compensation，MC)，使用已编码图像作为参考对当前编码图像进行预测。
目前，相关的视频和图像编码标准使用的帧内预测方法是使用当前编码块的左相邻和上相邻位置上已编码像素点(即参考像素点)来构造当前编码块中像素点的预测值。帧内预测模式指示了使用编码块相邻像素点构造编码块像素点预测值的方法，例如，直流(DC)模式、帧内预测方向等。
然而，相关的视频和图像编码方法在帧内预测参考像素均不可用，以及帧内参考像素部分可用的情况下，存在较大的帧内预测误差，且编码比特开销较大。
发明内容
为解决上述技术问题，本公开期望提供一种图像编码方法、解码方法、编码器、解码器及存储介质，能够减少视频图像编码中的帧内预测误差。
本公开的技术方案是这样实现的：
第一方面，本公开实施例提供了一种图像编码方法，包括：
根据编码块中的像素样值，确定编码块的第一参考样值；
根据第一参考样值，确定编码块的帧内预测模式，并计算编码块的帧内预测值；
根据编码块的原始值与帧内预测值，获得预测差参数；
对第一参考样值、帧内预测模式和预测差参数进行编码，将编码比特写入码流。
可选的，根据编码块的原始值与帧内预测值，获得预测差参数示例性包括：
计算编码块的原始值与帧内预测值的差值，得到编码块的预测差值；
对预测差值进行变换和量化处理，得到预测差参数。
可选的，根据编码块中的像素样值确定编码块的第一参考样值，包括：
判断编码块的参考样值是否存在，参考样值为编码块相邻的一个或多个像素点的已恢 复像素样值；
若编码块的部分或全部参考样值不存在，则根据编码块中的像素样值确定编码块的第一参考样值；
或者，判断编码块的匹配块中的参考样值是否均存在；
若编码块的匹配块中的部分或全部参考样值不存在，则判断为编码块的部分或全部参考样值不存在，根据编码块中的像素样值确定编码块的第一参考样值。
可选的，编码块的匹配块位于编码块所在图像中并且与编码块大小相同的区域。
可选的，判断参考样值是否存在的方法，包括以下至少一种：
若参考样值对应的参考像素点位置位于编码块所在图像中尚未编码的图像区域，则将参考样值判断为不存在；
若参考样值对应的参考像素点位置位于编码块所在图像的边界之外，则将参考样值判断为不存在；
在编码块所在图像中，若参考样值对应的参考像素点位置位于编码块所在分片之外，则将参考样值判断为不存在；
在编码块所在图像中，若参考样值对应的参考像素点位置位于编码块所在瓦片之外，则将参考样值判断为不存在；
若使用帧内预测受限模式，在编码块所在图像中，若参考样值对应的参考像素点位置位于非帧内预测模式的编码块中，则将参考样值判断为不存在。
可选的，根据编码块中的像素样值确定编码块的第一参考样值，包括：
根据编码块中的像素样值，使用软判决方法确定第一参考样值。
可选的，根据编码块中的像素样值，使用软判决方法确定第一参考样值示例性包括：
使用约束最优化方法计算第一参考样值，约束最优化计算公式为：
min D＝∑ kd(x k,p k)s.t.R≤R C
其中，D是编码块的编码失真；k是遍历范围，包括编码块中的一个或多个像素；d(a,b)是a和b之间的误差值；x k是编码块中待编码像素样值；p k是使用第一参考样值根据帧内预测模式计算得到的x k的预测值；R是编码块的编码比特开销；R C是编码块编码比特开销的上限值。
可选的，根据编码块中的像素样值，使用软判决方法确定第一参考样值示例性包括：
使用无约束最优化方法计算第一参考样值，无约束最优化计算公式为：
(r,m) opt＝arg min(D+λ·R)
其中，r是第一参考样值；m是编码块的帧内预测模式，D是编码块的编码失真，R是编码块的编码比特开销，λ是拉格朗日因子。
可选的，根据编码块中的像素样值确定编码块的第一参考样值示例性包括：
若判断编码块的全部参考样值不存在，则将第一参考样值设置为编码块原始值的平均值；
若判断编码块的部分参考样值不存在，则将第一参考样值设置为编码块原始值的平均值与部分存在的参考样值均值的差值。
可选的，第一参考样值包括至少两个样值，以及使用方式指示参数，其中，样值使用方式参数用于指示第一参考样值中包含的至少两个样值在设置编码块的参考样值过程中的使用方式。
可选的，第一参考样值包括两个样值，根据编码块中的像素样值确定编码块的第一参考样值，包括：
根据两个样值分别设置编码块左相邻参考像素点位置中不存在的参考样值的取值，和上相邻参考像素点位置中不存在的参考样值的取值。
可选的，根据第一参考样值，确定编码块的帧内预测模式，计算编码块的帧内预测值，包括：
使用第一参考样值设置编码块的不存在的参考样值；
根据编码块的参考样值，使用率失真优化方法确定编码块的帧内预测模式，使用帧内预测模式计算得到编码块的帧内预测值。
可选的，根据第一参考样值，确定编码块的帧内预测模式，计算编码块的帧内预测值，还包括：
使用第一参考样值设置编码块的不存在的参考样值；
对编码块的参考样值进行滤波，得到经滤波处理的参考样值；
根据经滤波处理的参考样值，使用率失真优化方法确定编码块的帧内预测模式，使用帧内预测模式计算得到编码块的帧内预测值。
可选的，对编码块的参考样值进行滤波，得到经滤波处理的参考样值，包括：
使用率失真优化方法确定滤波控制参数，滤波控制参数用于指示是否对参考样值进行滤波；
若滤波控制参数指示对参考样值进行滤波，则对编码块的参考样值进行滤波，得到经滤波处理的参考样值。
可选的，对编码块的参考样值进行滤波，得到经滤波处理的参考样值，包括：
若编码块的大小等于预设值时，则使用率失真优化方法确定滤波控制参数，滤波控制参数用于指示是否对参考样值进行滤波；
当滤波控制参数指示对参考样值进行滤波时，对编码块的参考样值进行滤波，得到经滤波处理的参考样值。
可选的，对编码块的参考样值进行滤波，得到经滤波处理的参考样值，包括：
若编码块的大小等于预设值时，对编码块的参考样值进行滤波，得到经滤波处理的参考样值。
可选的，对编码块的参考样值进行滤波，得到经滤波处理的参考样值，包括：
当编码块的帧内预测模式是预设的模式时，使用率失真优化方法确定滤波控制参数， 滤波控制参数用于指示是否对参考样值进行滤波；
当滤波控制参数指示对参考样值进行滤波时，对编码块的参考样值进行滤波，得到经滤波处理的参考样值。
可选的，对编码块的参考样值进行滤波，得到经滤波处理的参考样值，包括：
当编码块的帧内预测模式是预设的模式时，对编码块的参考样值进行滤波，得到经滤波处理的参考样值。
可选的，对编码块的参考样值进行滤波，得到经滤波处理的参考样值，包括：
当编码块的大小等于预设值时，并且编码块的帧内预测模式是预设的模式时，使用率失真优化方法确定滤波控制参数，滤波控制参数用于指示是否对参考样值进行滤波；
当滤波控制参数指示对参考样值进行滤波时，对编码块的参考样值进行滤波，得到经滤波处理的参考样值。
可选的，对编码块的参考样值进行滤波，得到经滤波处理的参考样值，包括：
当编码块的大小等于预设值时、并且编码块的帧内预测模式是预设的模式时，对编码块参考样值进行滤波，得到经滤波处理的参考样值。
可选的，对编码块的参考样值进行滤波，得到经滤波处理的参考样值，方法还包括：
对编码块的滤波控制参数进行编码，将编码比特写入码流。
可选的，对第一参考样值进行编码，包括：
通过对第一参考样值进行编码获得第一参考样值的编码比特；
将编码比特写入码流中的数据单元，其中，码流中的数据单元包含以下至少之一：一个或多个参数集，分片头，块层数据单元。
可选的，对第一参考样值进行编码，包括：
第一参考样值包括一个或多个样值；
对第一参考样值的一个或多个样值进行编码，获得第一参考样值的一个或多个样值的编码比特，将编码比特写入码流中的数据单元。
可选的，对第一参考样值的一个或多个样值进行编码，获得第一参考样值的一个或多个样值的编码比特，还包括：
编码样值使用方式参数，将样值使用方式参数的编码比特写入码流中的数据单元。
可选的，第一参考样值包括样值控制参数，样值控制参数包括以下至少之一的控制参数：序列层样值控制参数，图像层样值控制参数，分片层样值控制参数，块层样值控制参数。
第二方面，本公开实施例提供了一种图像解码方法，包括：
解析码流，获得解码块的第一参考样值、帧内预测模式和预测差参数；
根据第一参考样值确定解码块的帧内预测参考样值；
根据帧内预测参考样值以及帧内预测模式构造解码块的帧内预测值；
使用预测差参数计算解码块的预测差；
计算帧内预测值和预测差的和值，获得解码块的恢复值。
可选的，根据第一参考样值确定解码块的帧内预测参考样值，包括：
判断解码块的参考样值是否存在，参考样值为解码块相邻的一个或多个像素点的已恢复像素样值；
若解码块的部分或全部参考样值不存在，则根据第一参考样值确定解码块的不存在的参考样值，获得帧内预测参考样值。
或者，判断解码块的匹配块中的参考样值是否均存在；
若解码块的匹配块中的部分或全部参考样值不存在，则判断为解码块的部分或全部参考样值不存在，根据第一参考样值确定解码块的不存在的参考样值，获得帧内预测参考样值。
可选的，解码块的匹配块位于解码块所在图像中并且与解码块大小相同的区域。
可选的，判断参考样值是否存在的方法，包括以下至少一种：
若参考样值对应的参考像素点位置位于解码块所在图像中尚未解码的图像区域，则将参考样值判断为不存在；
若参考样值对应的参考像素点位置位于解码块所在图像的边界之外，则将参考样值判断为不存在；
在解码块所在图像中，若参考样值对应的参考像素点位置位于解码块所在分片之外，则将参考样值判断为不存在；
在解码块所在图像中，若参考样值对应的参考像素点位置位于解码块所在瓦片之外，则将参考样值判断为不存在；
若使用帧内预测受限模式，在解码块所在图像中，若参考样值对应的参考像素点位置位于非帧内预测模式的解码块中，则将参考样值判断为不存在。
可选的，解析码流获得解码块的第一参考样值，包括：
解析码流中的一个或多个数据单元，获得第一参考样值，其中，数据单元包括以下至少之一：一个或多个参数集，分片头，块层数据单元。
可选的，解析码流获得解码块的第一参考样值，包括：
解析码流中的一个或多个数据单元，获得样值使用方式参数，其中，样值使用方式参数用于指示第一参考样值中包含的样值在设置解码块的参考样值过程中的使用方式。
可选的，第一参考样值包括样值控制参数，样值控制参数包括以下至少之一的控制参数：序列层样值控制参数，图像层样值控制参数，分片层样值控制参数，块层样值控制参数。
可选的，根据第一参考样值确定解码块的不存在的参考样值，包括：
使用第一参考样值设置解码块的不存在的参考样值。
可选的，第一参考样值中包含样值控制参数，根据第一参考样值确定解码块的不存在的参考样值，包括：
若样值控制参数指示使用第一参考样值构造不存在的参考样值，则使用第一参考样值设置解码块的不存在的参考样值。
可选的，第一参考样值中包含样值使用方式参数，根据第一参考样值确定解码块的不存在的参考样值，包括：
若第一参考样值包含至少两个样值，则根据样值使用方式参数，使用第一参考样值中包含的样值对应设置解码块的不存在的参考样值。
可选的，第一参考样值包括两个样值，根据样值使用方式参数，使用第一参考样值中包含的样值对应设置解码块的不存在的参考样值，包括：
按照预设规则使用第一参考样值中的两个样值分别设置解码块左相邻参考像素点位置中不存在的参考样值，和上相邻参考像素点位置中不存在的参考样值。
可选的，根据帧内预测参考样值以及帧内预测模式构造解码块的帧内预测值，方法还包括：
对帧内预测参考样值进行滤波处理，得到经过滤波处理的参考样值；
使用经滤波处理的参考样值，按帧内预测模式计算得到解码块的帧内预测值。
可选的，对帧内预测参考样值进行滤波处理得到经过滤波处理的参考样值，包括：
解析码流，获得滤波控制参数，其中，滤波控制参数用于指示是否对帧内预测参考样值进行滤波；
若滤波控制参数指示对帧内预测参考样值进行滤波，则对帧内预测参考样值进行滤波处理，得到经过滤波处理的参考样值。
可选的，对帧内预测参考样值进行滤波处理得到经过滤波处理的参考样值，包括：
若解码块的大小等于预设值，则解析码流，获得滤波控制参数；
当滤波控制参数指示对帧内预测参考样值进行滤波时，对帧内预测参考样值进行滤波处理，得到经过滤波处理的参考样值。
可选的，对帧内预测参考样值进行滤波处理得到经过滤波处理的参考样值，包括：
若解码块的大小等于预设值时，对帧内预测参考样值进行滤波处理，得到经过滤波处理的参考样值。
可选的，对帧内预测参考样值进行滤波处理得到经过滤波处理的参考样值，包括：
若解码块的帧内预测模式是预设的模式，则解析码流，获得滤波控制参数；
当滤波控制参数指示对帧内预测参考样值进行滤波时，对帧内预测参考样值进行滤波处理，得到经过滤波处理的参考样值。
可选的，对帧内预测参考样值进行滤波处理得到经过滤波处理的参考样值，包括：
若解码块的帧内预测模式是预设的模式，则对帧内预测参考样值进行滤波处理，得到经过滤波处理的参考样值。
可选的，对帧内预测参考样值进行滤波处理得到经过滤波处理的参考样值，包括：
若解码块的大小等于预设值，并且解码块的帧内预测模式是预设的模式，则解析码流， 获得滤波控制参数；
当滤波控制参数指示对帧内预测参考样值进行滤波时，对帧内预测参考样值进行滤波处理，得到经过滤波处理的参考样值。
可选的，对帧内预测参考样值进行滤波处理得到经过滤波处理的参考样值，包括：
若解码块的大小等于预设值，并且解码块的帧内预测模式是预设的模式时，对帧内预测参考样值进行滤处理，得到经过滤波处理的参考样值。
可选的，使用预测差参数计算解码块的预测差，包括：
对预测差参数进行伸缩和变换处理，得到解码块的预测差。
可选的，在将帧内预测值和预测差的和值设置为解码块的恢复值之后，方法还包括：
对解码块的恢复值进行环路滤波处理得到解码值。
第三方面，本公开实施例提供了一种图像编码装置，图像编码装置包括第一参考样值确定单元、帧内预测值确定单元、预测差参数获得单元和编码单元，其中：
第一参考样值确定单元，设置为根据编码块中的像素样值，确定编码块的第一参考样值；
帧内预测值确定单元，设置为根据第一参考样值确定单元确定的第一参考样值，确定编码块的帧内预测模式，并计算编码块的帧内预测值；
预测差参数获得单元，设置为根据编码块的原始值与帧内预测值确定单元计算的帧内预测值，获得预测差参数；
编码单元，设置为对第一参考样值确定单元确定的第一参考样值、帧内预测值确定单元确定的帧内预测模式和预测差参数获得单元获得的预测差参数进行编码，将编码比特写入码流。
可选的，预测差参数获得单元示例性设置为：
计算编码块的原始值与帧内预测值的差值，得到编码块的预测差值；
对预测差值进行变换和量化处理，得到预测差参数。
可选的，第一参考样值确定单元示例性设置为：
判断编码块的参考样值是否存在，参考样值为编码块相邻的一个或多个像素点的已恢复像素样值；
若编码块的部分或全部参考样值不存在，则根据编码块中的像素样值确定编码块的第一参考样值；
或者，判断编码块的匹配块中的参考样值是否均存在；
若编码块的匹配块中的部分或全部参考样值不存在，则判断为编码块的部分或全部参考样值不存在，根据编码块中的像素样值确定编码块的第一参考样值。
可选的，第一参考样值确定单元示例性设置为：
根据编码块中的像素样值，使用软判决方法确定第一参考样值。
可选的，第一参考样值确定单元示例性设置为：
若判断编码块的全部参考样值不存在，则将第一参考样值设置为编码块原始值的平均值；
若判断编码块的部分参考样值不存在，则将第一参考样值设置为编码块原始值的平均值与部分存在的参考样值均值的差值。
可选的，帧内预测值确定单元示例性设置为：
使用第一参考样值设置编码块的不存在的参考样值；
根据编码块的参考样值，使用率失真优化方法确定编码块的帧内预测模式，使用帧内预测模式计算得到编码块的帧内预测值。
可选的，帧内预测值确定单元示例性设置为：
使用第一参考样值设置编码块的不存在的参考样值；
对编码块的参考样值进行滤波，得到经滤波处理的参考样值；
根据经滤波处理的参考样值，使用率失真优化方法确定编码块的帧内预测模式，使用帧内预测模式计算得到编码块的帧内预测值。
可选的，编码单元示例性设置为：
通过对第一参考样值进行编码获得第一参考样值的编码比特；
将编码比特写入码流中的数据单元，其中，码流中的数据单元包含以下至少之一：一个或多个参数集，分片头，块层数据单元。
第四方面，本公开实施例提供了一种图像解码装置，图像解码装置包括解析单元、帧内预测参考样值确定单元、帧内预测值确定单元、预测差确定单元和恢复值获得单元，其中：
解析单元，设置为解析码流获得解码块的第一参考样值、帧内预测模式和预测差参数；
帧内预测参考样值确定单元，设置为根据解析单元解析的第一参考样值确定解码块的帧内预测参考样值；
帧内预测值确定单元，设置为根据帧内预测参考样值确定单元确定的帧内预测参考样值以及解析单元解析的帧内预测模式构造解码块的帧内预测值；
预测差确定单元，设置为使用解析单元解析的预测差参数计算解码块的预测差；
恢复值获得单元，设置为计算帧内预测值确定单元构造的帧内预测值，和预测差确定单元计算的预测差的和值，获得解码块的恢复值。
可选的，解析单元示例性设置为：
解析码流中的一个或多个数据单元，获得第一参考样值，其中，数据单元包括以下至少之一：一个或多个参数集，分片头，块层数据单元。
可选的，帧内预测值确定单元示例性设置为：
对帧内预测参考样值进行滤波处理，得到经过滤波处理的参考样值；
使用经滤波处理的参考样值，按帧内预测模式计算得到解码块的帧内预测值。
可选的，帧内预测参考样值确定单元示例性设置为：
判断解码块的参考样值是否存在，参考样值为解码块相邻的一个或多个像素点的已恢复像素样值；
若解码块的部分或全部参考样值不存在，则根据第一参考样值确定解码块的不存在的参考样值，获得帧内预测参考样值。
或者，判断解码块的匹配块中的参考样值是否均存在；
若解码块的匹配块中的部分或全部参考样值不存在，则判断为解码块的部分或全部参考样值不存在，根据第一参考样值确定解码块的不存在的参考样值，获得帧内预测参考样值。
可选的，预测差确定单元示例性设置为：
对预测差参数进行伸缩和变换处理，得到解码块的预测差。
可选的，恢复值获得单元示例性设置为：
对解码块的恢复值进行环路滤波处理得到解码值。
第五方面，本公开实施例提供了一种编码器，包括：第一处理器、第一存储介质、第一通信总线，其中，第一处理器和第一存储介质通过第一通信总线连接；
第一处理器，调用第一存储介质存储的图像编码相关程序，并执行如下步骤：
根据编码块中的像素样值，确定编码块的第一参考样值；
根据第一参考样值，确定编码块的帧内预测模式，并计算编码块的帧内预测值；
根据编码块的原始值与帧内预测值，获得预测差参数；
对第一参考样值、帧内预测模式和预测差参数进行编码，将编码比特写入码流。
第六方面，本公开实施例提供了一种解码器，包括：第二处理器、第二存储介质、第二通信总线，其中，第二处理器和第二存储介质通过第二通信总线连接；
第二处理器，调用第二存储介质存储的图像解码相关程序，并执行如下步骤：
解析码流，获得解码块的第一参考样值、帧内预测模式和预测差参数；
根据第一参考样值确定解码块的帧内预测参考样值；
根据帧内预测参考样值以及帧内预测模式构造解码块的帧内预测值；
使用预测差参数计算解码块的预测差；
计算帧内预测值和预测差的和值，获得解码块的恢复值。
第七方面，本公开实施例提供了一种计算机存储介质，应用于编码器中，计算机可读存储介质存储有一个或多个图像编码相关程序，一个或者多个图像编码相关程序可被一个或者多个第一处理器执行，以实现上述图像编码方法。
第八方面，本公开实施例提供了一种计算机存储介质，应用于解码器中，计算机可读存储介质存储有一个或多个图像解码相关程序，一个或者多个图像解码相关程序可被一个或者多个第二处理器执行，以实现上述图像解码方法。
综上所述，本公开实施例提供了一种图像编码方法、解码方法、编码器、解码器及存储介质。其中，图像编码方法包括：根据编码块中的像素样值，确定编码块的第一参考样 值；根据第一参考样值，确定编码块的帧内预测模式，并计算编码块的帧内预测值；根据编码块的原始值与帧内预测值，获得预测差参数；对第一参考样值、帧内预测模式和预测差参数进行编码，将编码比特写入码流。采用上述技术方案，由于第一参考样值是根据编码块的待编码像素样值计算确定的，并且第一样值将进一步用于构造编码块的帧内预测参考样值。因此，在编码块相邻块的样值均不可用或者部分不可用的情况下，有效降低了帧内预测的预测值与编码块之间的预测差值，提高了帧内预测模式的编码效率。
附图说明
图1为本公开实施例提供的一种图像编码方法的流程示意图；
图2为本公开实施例提供的一种图像解码方法的流程示意图；
图3为本公开实施例提供的一种编码装置的结构示意图；
图4为本公开实施例提供的一种解码装置的结构示意图；
图5为本公开实施例提供的一种编码器的系统架构示意图；
图6为本公开实施例提供的一种解码器的系统架构示意图；
图7为本公开实施例提供的一种编码器的帧内预测单元的结构示意图；
图8为本公开实施例提供的一种帧内预测方法的示意图；
图9为本公开实施例提供的另一种帧内预测方法的示意图；
图10为本公开实施例提供的一种编码器的熵编码单元对帧内预测单元输出参数进行编码的数据处理的流程示意图；
图11为本公开实施例提供的一种解码器的帧内预测单元的结构示意图；
图12为本公开实施例提供的一种解码器的解析单元对解码块码流中的帧内预测模式和滤波参数进行解析的数据处理的流程示意图；
图13为本公开实施例提供的包含编码器的电子设备的结构示意图；
图14为本公开实施例提供的包含解码器的电子设备的结构示意图；
图15为本公开实施例提供的电子系统的系统架构示意图。
具体实施方式
相关技术中，视频和图像编码标准使用的帧内预测方法是使用当前编码块的左相邻和上相邻位置上已编码像素点来构造当前编码块中像素点的预测值。帧内预测模式指示了使用编码块相邻像素点构造编码块像素点预测值的方法，例如，直流(DC)模式、帧内预测方向等。相关标准设计了多种帧内预测模式。例如，H.264/AVC标准使用9种帧内预测模式，包括DC模式和8个不同帧内预测方向的预测模式；H.265/HEVC标准使用平面模式、DC模式和33个不同帧内预测方向的预测模式。
在相关的帧内预测方法中，首先需要参考像素点是否“可用”。当参考像素点不存在(例如当前编码块位于图像边界)或者不能被当前编码块使用(例如当前块位于分片的边 界)时，参考像素点被认为“不存在”。当参考像素点“不存在”时，在H.264/AVC标准中，这些参考像素点被标记为“不可用”，使用这些参考像素点的帧内预测模式也是不可用的；在H.265/HEVC标准中，按照指定的规则对“不存在”的参考像素点进行填充，将填充后的参考像素点标记为“可用”。
在H.264/AVC标准中，DC模式是始终可以使用的帧内预测模式。当编码块的参考像素点都“不可用”时，DC模式的帧内预测值设置为1<<(BitDepth–1)。其中，“<<”是标准中定义的算术左移位(Arithmetic Left Shift)比特位运算符，BitDepth表示分量的比特深度。比特深度(Bit Depth)指的是物理量的数字化表示所使用的比特位数。在该标准中，例如对于亮度分量BitDepth等于BitDepth Y，BitDepth Y是亮度分量的比特深度。目前消费电子产品上常用的比特深度是8比特，这种情况下，当参考像素都“不可用”时，DC模式预测值是128。当部分参考像素“可用”时，DC模式使用“可用”参考像素样值的均值作为帧内预测值。该标准中，未编码的像素，参考像素“不可用”的情况主要包括：图像边界外的像素点，分片边界外的像素点，constrained_intra_pred_flag取值等于1时相邻宏块(Macroblock)使用帧间(Inter)编码。
在H.265/HEVC标准中，当编码块的参考像素点位置上有“不存在”的参考像素点时，使用“存在”的参考像素点对“不存在”的参考像素点进行填充；当编码块的参考像素点都不存在时，使用参考像素点的取值均填充为1<<(BitDepth–1)。在该标准的帧内预测参考像素填充方法中，左相邻参考像素点可被用来填充上相邻“不存在”的像素点。该标准中，参考像素“不存在”的情况主要包括：未编码的像素，图像边界外的像素点，分片边界外的像素点，瓦片(Tile)边界外的像素点，constrained_intra_pred_flag取值等于1时相块使用帧间编码。
然而，相关的视频和图像编码标准存在以下缺陷：
(1)在帧内预测参考像素均不可用的情况下，相关方法等价于将帧内预测的预测值设置为“1<<(BitDepth–1)”。如果编码块的像素样值与“1<<(BitDepth–1)”存在较大偏差的情况下，帧内预测的预测差较大，带来大的编码比特开销。
(2)在帧内参考像素部分可用的情况下，限制可用的帧内预测模式、仅使用可用参考像素样值计算预测值(例如DC预测值)、使用可用参考像素样值进行填充的方法，由于这些方法的执行过程都根据参考像素样值而没有涉及编码块中像素样值，会带来较大的帧内预测误差。
下面将结合本公开实施例中的附图，对本公开实施例中的技术方案进行清楚、完整地描述。
下述实施例中，视频指的是由图像组成的图像序列。码流指的是视频编码器对视频进行编码产生的码流，也指对视频编码器编码视频产生的码流经过系统层处理后得到的包含视频编码器编码视频产生的码流的传输流和/或媒体文件，对码流进行解码可以获得视频。系统层处理是对视频码流进行的封装操作，例如，将视频码流作为数据载荷封装成传输流 或将视频码流作为载荷封装成媒体文件。系统层处理也包括将包含视频码流的传输流或媒体文件作为数据载荷封装成用于传输的流或者用于存储的文件。系统层处理生成的数据单元也称为系统层数据单元，在系统层处理封装数据载荷过程中在系统层数据单元中添加的信息(例如系统层数据单元的头信息等)称为系统层信息。子码流指的是从码流中进行抽取操作得到的部分码流，对子码流进行解码可以获得视频图像，该视频图像可以是比解码码流获得的视频图像的分辨率低的图像，也可以是比解码码流获得视频的帧率低的图像，该视频图像包含的可以是解码码流获得的视频图像中的部分内容。
实施例一
参见图1，本公开的实施例提出了一种图像编码方法，包括：
步骤101：根据编码块中的像素样值，确定编码块的第一参考样值；
可选的，步骤101示例性包括：判断编码块的参考样值是否存在，参考样值为编码块相邻的一个或多个像素点的已恢复像素样值；若编码块的部分或全部参考样值不存在，则根据编码块中的像素样值确定编码块的第一参考样值；
或者，判断编码块的匹配块中的参考样值是否均存在；若编码块的匹配块中的部分或全部参考样值不存在，则判断为编码块的部分或全部参考样值不存在，根据编码块中的像素样值确定编码块的第一参考样值。
其中，编码块的匹配块位于编码块所在图像中并且与编码块大小相同的区域。
示例性的，判断参考样值是否存在的方法，包括以下至少一种：
若参考样值对应的参考像素点位置位于编码块所在图像中尚未编码的图像区域，则将参考样值判断为不存在；
若参考样值对应的参考像素点位置位于编码块所在图像的边界之外，则将参考样值判断为不存在；
在编码块所在图像中，若参考样值对应的参考像素点位置位于编码块所在分片之外，则将参考样值判断为不存在；
在编码块所在图像中，若参考样值对应的参考像素点位置位于编码块所在瓦片之外，则将参考样值判断为不存在；
若使用帧内预测受限模式，在编码块所在图像中，若参考样值对应的参考像素点位置位于非帧内预测模式的编码块中，则将参考样值判断为不存在。
可选的，根据编码块中的像素样值，使用软判决方法确定第一参考样值。
示例性的，根据编码块中的像素样值，使用软判决方法确定编码块的第一参考样值示例性包括：使用约束最优化方法计算第一参考样值，约束最优化计算公式为：
min D＝∑ kd(x k,p k)s.t.R≤R C…………………………………………(1)
其中，D是编码块的编码失真；k是遍历范围，包括编码块中的一个或多个像素；d(a,b)是a和b之间的误差值；x k是编码块中待编码像素样值；p k是使用第一参考样值根据帧内预测模式计算得到的x k的预测值；R是编码块的编码比特开销；R C是编码块编码比特 开销的上限值。
示例性的，根据编码块中的像素样值，使用软判决方法确定编码块的第一参考样值示例性包括：使用无约束最优化方法计算第一参考样值，无约束最优化计算公式为：
(r,m) opt＝arg min(D+λ·R)…………………………………………(2)
其中，r是第一参考样值；m是编码块的帧内预测模式，D是编码块的编码失真，R是编码块的编码比特开销，λ是拉格朗日因子。
可选的，步骤101示例性包括：若判断编码块的全部参考样值不存在，则将第一参考样值设置为编码块原始值的平均值；若判断编码块的部分参考样值不存在，则将第一参考样值设置为编码块原始值的平均值与部分存在的参考样值均值的差值。
示例性而言，第一参考样值包括至少两个样值，以及使用方式指示参数，其中，样值使用方式参数用于指示第一参考样值中包含的至少两个样值在设置编码块的参考样值过程中的使用方式。
示例性的，第一参考样值包括两个样值，根据编码块中的像素样值确定编码块的第一参考样值，包括：根据两个样值分别设置编码块左相邻参考像素点位置中不存在的参考样值的取值，和上相邻参考像素点位置中不存在的参考样值的取值。
步骤102：根据第一参考样值，确定编码块的帧内预测模式，并计算编码块的帧内预测值；
可选的，步骤102，包括：使用第一参考样值设置编码块的不存在的参考样值；根据编码块的参考样值，使用率失真优化方法确定编码块的帧内预测模式，使用帧内预测模式计算得到编码块的帧内预测值。
可选的，步骤102，还包括：使用第一参考样值设置编码块的不存在的参考样值；对编码块的参考样值进行滤波，得到经滤波处理的参考样值；根据经滤波处理的参考样值，使用率失真优化方法确定编码块的帧内预测模式，使用帧内预测模式计算得到编码块的帧内预测值。
其中，对编码块的参考样值进行滤波，得到经滤波处理的参考样值，包括以下方法：
方法一：使用率失真优化方法确定滤波控制参数，滤波控制参数用于指示是否对参考样值进行滤波；若滤波控制参数指示对参考样值进行滤波，则对编码块的参考样值进行滤波，得到经滤波处理的参考样值。
方法二：若编码块的大小等于预设值时，则使用率失真优化方法确定滤波控制参数，滤波控制参数用于指示是否对参考样值进行滤波；当滤波控制参数指示对参考样值进行滤波时，对编码块的参考样值进行滤波，得到经滤波处理的参考样值。
其中，预设值包括先设定的一个值或者预先设定的多个值之一。
方法三：若编码块的大小等于预设值时，对编码块的参考样值进行滤波，得到经滤波处理的参考样值。
方法四：当编码块的帧内预测模式是预设的模式时，使用率失真优化方法确定滤波控 制参数，滤波控制参数用于指示是否对参考样值进行滤波；当滤波控制参数指示对参考样值进行滤波时，对编码块的参考样值进行滤波，得到经滤波处理的参考样值。
其中，预设的模式包括预先设定的一个模式或者预先设定的多个模式之一。
方法五：当编码块的帧内预测模式是预设的模式时，对编码块的参考样值进行滤波，得到经滤波处理的参考样值。
方法六：当编码块的大小等于预设值时，并且编码块的帧内预测模式是预设的模式时，使用率失真优化方法确定滤波控制参数，滤波控制参数用于指示是否对参考样值进行滤波；当滤波控制参数指示对参考样值进行滤波时，对编码块的参考样值进行滤波，得到经滤波处理的参考样值。
方法七：当编码块的大小等于预设值并且编码块的帧内预测模式是预设的模式时，对编码块参考样值进行滤波，得到经滤波处理的参考样值。
在一实施例中，对编码块的参考样值进行滤波，得到经滤波处理的参考样值，方法还包括：对编码块的滤波控制参数进行编码，将编码比特写入码流。
步骤103：根据编码块的原始值与帧内预测值，获得预测差参数；
示例性而言，计算编码块的原始值与帧内预测值的差值，得到编码块的预测差值；对预测差值进行变换和量化处理，得到预测差参数。
步骤104：对第一参考样值、帧内预测模式和预测差参数进行编码，将编码比特写入码流。
可选的，步骤104中，对第一参考样值进行编码，包括：通过对第一参考样值进行编码获得第一参考样值的编码比特；将编码比特写入码流中的数据单元，其中，码流中的数据单元包含以下至少之一：一个或多个参数集，分片头，块层数据单元。
可选的，步骤104中，对第一参考样值进行编码，包括：第一参考样值包括一个或多个样值；对第一参考样值的一个或多个样值进行编码，获得第一参考样值的一个或多个样值的编码比特，将编码比特写入码流中的数据单元。
在一实施例中，对第一参考样值的一个或多个样值进行编码，获得第一参考样值的一个或多个样值的编码比特，还包括：编码样值使用方式参数，将样值使用方式参数的编码比特写入码流中的数据单元。
在本公开实施例中，第一参考样值包括样值控制参数，样值控制参数包括以下至少之一的控制参数：序列层样值控制参数，图像层样值控制参数，分片层样值控制参数，块层样值控制参数。
实施例二
参见图2，本公开的实施例提出了一种图像解码方法，包括：
步骤201：解析码流，获得解码块的第一参考样值、帧内预测模式和预测差参数；
可选的，步骤201中，解析码流获得解码块的第一参考样值，包括：
解析码流中的一个或多个数据单元，获得第一参考样值，其中，数据单元包括以下至少之一：一个或多个参数集，分片头，块层数据单元。
在一实施例中，解析码流获得解码块的第一参考样值，包括：
解析码流中的一个或多个数据单元，获得样值使用方式参数，其中，样值使用方式参数用于指示第一参考样值中包含的多个样值在设置解码块的参考样值过程中的使用方式。
其中，第一参考样值包括样值控制参数，样值控制参数包括以下至少之一的控制参数：序列层样值控制参数、图像层样值控制参数、分片层样值控制参数、块层样值控制参数。
步骤202：根据第一参考样值确定解码块的帧内预测参考样值；
可选的，步骤202包括：判断解码块的参考样值是否存在，参考样值为解码块相邻的一个或多个像素点的已恢复像素样值；若解码块的部分或全部参考样值不存在，则根据第一参考样值确定解码块的不存在的参考样值，获得帧内预测参考样值。
或者，判断解码块的匹配块中的参考样值是否均存在；若解码块的匹配块中的部分或全部参考样值不存在，则判断为解码块的部分或全部参考样值不存在，根据第一参考样值确定解码块的不存在的参考样值，获得帧内预测参考样值。
需要说明的是，解码块的匹配块位于解码块所在图像中并且与解码块大小相同的区域。
可以理解的是，在根据第一参考样值确定解码块的不存在的参考样值之后，解码块的参考样值均存在。
实际应用中，判断参考样值是否存在的方法，包括以下至少一种：
若参考样值对应的参考像素点位置位于解码块所在图像中尚未解码的图像区域，则将参考样值判断为不存在；
若参考样值对应的参考像素点位置位于解码块所在图像的边界之外，则将参考样值判断为不存在；
在解码块所在图像中，若参考样值对应的参考像素点位置位于解码块所在分片之外，则将参考样值判断为不存在；
在解码块所在图像中，若参考样值对应的参考像素点位置位于解码块所在瓦片之外，则将参考样值判断为不存在；
若使用帧内预测受限模式，在解码块所在图像中，若参考样值对应的参考像素点位置位于非帧内预测模式的解码块中，则将参考样值判断为不存在。
在一实施例中，根据第一参考样值确定解码块的不存在的参考样值，包括：使用第一参考样值设置解码块的不存在的参考样值。
在一实施例中，第一参考样值中包含样值控制参数，根据第一参考样值确定解码块的不存在的参考样值，包括：若样值控制参数指示使用第一参考样值构造不存在的参考样值，则使用第一参考样值设置解码块的不存在的参考样值。
在一实施例中，第一参考样值中包含样值使用方式参数，根据第一参考样值确定解码 块的不存在的参考样值，包括：若第一参考样值包含至少两个样值，则根据样值使用方式参数，使用第一参考样值中包含的样值对应设置解码块的不存在的参考样值。
示例性的，第一参考样值包括两个样值，根据样值使用方式参数或使用预先设定的规则，使用第一参考样值中包含的样值对应设置解码块的不存在的参考样值，包括：按照预设规则使用第一参考样值中的两个样值分别设置解码块左相邻参考像素点位置中不存在的参考样值，和上相邻参考像素点位置中不存在的参考样值。
可选的，根据帧内预测参考样值以及帧内预测模式构造解码块的帧内预测值，方法还包括：对帧内预测参考样值进行滤波处理，得到经过滤波处理的参考样值；使用经滤波处理的参考样值，按帧内预测模式计算得到解码块的帧内预测值。
示例性的，对帧内预测参考样值进行滤波处理得到经过滤波处理的参考样值，包括至少以下方法：
方法一：解析码流，获得滤波控制参数，其中，滤波控制参数用于指示是否对帧内预测参考样值进行滤波；若滤波控制参数指示对帧内预测参考样值进行滤波，则对帧内预测参考样值进行滤波处理，得到经过滤波处理的参考样值。
方法二：若解码块的大小等于预设值，则解析码流，获得滤波控制参数；当滤波控制参数指示对帧内预测参考样值进行滤波时，对帧内预测参考样值进行滤波处理，得到经过滤波处理的参考样值。
方法三：若解码块的大小等于预设值时，对帧内预测参考样值进行滤波处理，得到经过滤波处理的参考样值。
方法四：若解码块的帧内预测模式是预设的模式，则解析码流，获得滤波控制参数；当滤波控制参数指示对帧内预测参考样值进行滤波时，对帧内预测参考样值进行滤波处理，得到经过滤波处理的参考样值。
方法五：若解码块的帧内预测模式是预设的模式，则对帧内预测参考样值进行滤波处理，得到经过滤波处理的参考样值。
方法六：若解码块的大小等于预设值，并且解码块的帧内预测模式是预设的模式，则解析码流，获得滤波控制参数；当滤波控制参数指示对帧内预测参考样值进行滤波时，对帧内预测参考样值进行滤波处理，得到经过滤波处理的参考样值。
方法七：若解码块的大小等于预设值，并且解码块的帧内预测模式是预设的模式时，对帧内预测参考样值进行滤处理，得到经过滤波处理的参考样值。
步骤203：根据帧内预测参考样值以及帧内预测模式构造解码块的帧内预测值；
步骤204：使用预测差参数计算解码块的预测差；
可选的，步骤204，包括：对预测差参数进行伸缩和变换处理，得到解码块的预测差。
步骤205：计算帧内预测值和预测差的和值，获得解码块的恢复值。
在实际应用中，在获得解码块的恢复值之后，方法还包括：对解码块的恢复值进行环路滤波处理得到解码值。
实施例三
参见图3，本公开的实施例提出了一种图像编码装置，包括第一参考样值确定单元3001、帧内预测值确定单元3002、预测差参数获得单元3003和编码单元3004，其中：
第一参考样值确定单元3001，设置为根据编码块中的像素样值，确定编码块的第一参考样值；
帧内预测值确定单元3002，设置为根据第一参考样值确定单元3001确定的第一参考样值，确定编码块的帧内预测模式，并计算编码块的帧内预测值；
预测差参数获得单元3003，设置为根据编码块的原始值与帧内预测值确定单元3002计算的帧内预测值，获得预测差参数；
编码单元3004，设置为对第一参考样值确定单元3001确定的第一参考样值、帧内预测值确定单元3002确定的帧内预测模式和预测差参数获得单元3003获得的预测差参数进行编码，将编码比特写入码流。
可选的，预测差参数获得单元3003示例性设置为：计算编码块的原始值与帧内预测值的差值，得到编码块的预测差值；对预测差值进行变换和量化处理，得到预测差参数。
可选的，第一参考样值确定单元3001示例性设置为：判断编码块的参考样值是否存在，参考样值为编码块相邻的一个或多个像素点的已恢复像素样值；若编码块的部分或全部参考样值不存在，则根据编码块中的像素样值确定编码块的第一参考样值；
或者，判断编码块的匹配块中的参考样值是否均存在；若编码块的匹配块中的部分或全部参考样值不存在，则判断为编码块的部分或全部参考样值不存在，根据编码块中的像素样值确定编码块的第一参考样值。
可选的，第一参考样值确定单元3001示例性设置为：根据编码块中的像素样值，使用软判决方法确定第一参考样值。
可选的，第一参考样值确定单元3001示例性设置为：若判断编码块的全部参考样值不存在，则将第一参考样值设置为编码块原始值的平均值；若判断编码块的部分参考样值不存在，则将第一参考样值设置为编码块原始值的平均值与部分存在的参考样值均值的差值。
可选的，帧内预测值确定单元3002示例性设置为：使用第一参考样值设置编码块的不存在的参考样值；根据编码块的参考样值，使用率失真优化方法确定编码块的帧内预测模式，使用帧内预测模式计算得到编码块的帧内预测值。
可选的，帧内预测值确定单元3002示例性设置为：使用第一参考样值设置编码块的不存在的参考样值；对编码块的参考样值进行滤波，得到经滤波处理的参考样值；根据经滤波处理的参考样值，使用率失真优化方法确定编码块的帧内预测模式，使用帧内预测模式计算得到编码块的帧内预测值。
可选的，编码单元3004示例性设置为：通过对第一参考样值进行编码获得第一参考 样值的编码比特；将编码比特写入码流中的数据单元，其中，码流中的数据单元包含以下至少之一：一个或多个参数集，分片头，块层数据单元。
实施例四
参见图4，本公开的实施例提出了一种图像解码装置，包括解析单元4001、帧内预测参考样值确定单元4002、帧内预测值确定单元4003、预测差确定单元4004和恢复值获得单元4005，其中：
解析单元4001，设置为解析码流获得解码块的第一参考样值、帧内预测模式和预测差参数；
帧内预测参考样值确定单元4002，设置为根据解析单元4001解析的第一参考样值确定解码块的帧内预测参考样值；
帧内预测值确定单元4003，设置为根据帧内预测参考样值确定单元4002确定的帧内预测参考样值以及解析单元4001解析的帧内预测模式构造解码块的帧内预测值；
预测差确定单元4004，设置为使用解析单元4001解析的预测差参数计算解码块的预测差；
恢复值获得单元4005，设置为计算帧内预测值确定单元4003构造的帧内预测值，和预测差确定单元4004计算的预测差的和值，获得解码块的恢复值。
可选的，解析单元4001示例性设置为：解析码流中的一个或多个数据单元，获得第一参考样值，其中，数据单元包括以下至少之一：一个或多个参数集，分片头，块层数据单元。
可选的，帧内预测值确定单元4003示例性设置为：对帧内预测参考样值进行滤波处理，得到经过滤波处理的参考样值；使用经滤波处理的参考样值，按帧内预测模式计算得到解码块的帧内预测值。
可选的，帧内预测参考样值确定单元4002示例性设置为：判断解码块的参考样值是否存在，参考样值为解码块相邻的一个或多个像素点的已恢复像素样值；若解码块的部分或全部参考样值不存在，则根据第一参考样值确定解码块的不存在的参考样值，获得帧内预测参考样值。
或者，判断解码块的匹配块中的参考样值是否均存在；若解码块的匹配块中的部分或全部参考样值不存在，则判断为解码块的部分或全部参考样值不存在，根据第一参考样值确定解码块的不存在的参考样值，获得帧内预测参考样值。
可选的，预测差确定单元4004示例性设置为：对预测差参数进行伸缩和变换处理，得到解码块的预测差。
可选的，恢复值获得单元4005示例性设置为：对解码块的恢复值进行环路滤波处理得到解码值。
实施例五
参见图5，本公开的实施例提出了一种编码器的结构示意图。示例性而言，编码器包括划分单元201、预测单元202、第一加法器207、变换单元208、量化单元209、反量化单元210、反变换单元211、第二加法器212、滤波单元213、解码图像缓冲区214，以及熵编码单元215。其中，预测单元202包括块划分单元203、运动估计单元204、运动补偿单元205和帧内预测单元206。
需要说明的是，编码器的输入是视频，编码器的输出是对输入视频进行编码后产生的码流。由于视频是由图像组成的图像序列，因此编码器的编码过程是对输入视频中的图像按照编码顺序依次进行编码。其中，编码顺序由编码器配置文件中设置的预测结构等参数来决定。视频中图像的编码顺序(对应于解码端的解码顺序)与图像的播放顺序可以是相同的，也可以是不同的。
划分单元201，设置为接收视频图像，并按照预设配置对视频中的图像进行划分，输出最大编码单元以及该最大编码单元相关属性信息。
可以理解的是，图像可以划分成一个或多个分片，每个分片中可以包含整数个最大编码单元，也可以包含非整数个最大编码单元。其中，最大编码单元是一个正方形的图像区域。当然，图像还可以划分成一个或多个瓦片，每个瓦片中可以包含整数个最大编码单元，也可以包含非整数个最大编码单元。
需要说明的是，划分单元201可以被配置为按照固定的方式对图像进行划分，也可以被配置为动态调整图像的划分方式。例如，为了适配网络的最大传输单元(Maximum Transmission Unit，MTU)，采用动态分片划分的方法，保证每个分片的编码比特数不超过MTU的限制。
预测单元202，设置为将划分单元201划分的最大编码单元再划分为一个或多个编码块，并确定该编码块的预测值。
可以理解的是，预测单元202的输入包括划分单元201输出的最大编码单元以及该最大编码单元相关属性信息。其中，最大编码单元相关属性信息包括最大编码单元在图像中的位置、在分片、瓦片中的位置等。实际应用中，对最大编码单元划分的方式包括四叉树划分、二叉树划分、三叉树划分等。
需要补充的是，预测单元202还可以对划分得到的编码块进一步进行划分成更多的编码块。还可以进一步将编码块划分成一个或多个预测块来确定预测值。
可选的，预测单元202确定编码块的预测值示例性包括：根据解码图像缓冲区214中的已解码图像，确定编码块的帧间预测值；或者，根据第二加法器212输出的当前编码图像中已恢复部分(未经过滤波单元213处理)，确定编码块的帧内预测值。
示例性的，预测单元202确定编码块的预测值，还包括：使用率失真优化(Rate-Distortion Optimization，RDO)方法确定编码块的预测值，并得到该预测值使用的帧间预测、帧内预测相关的输出参数。
示例性而言，预测单元202包括块划分单元203、运动估计单元204、运动补偿单元205和帧内预测单元206。下面对预测单元202中的四个子单元分别进行说明：
块划分单元203，设置为在最大编码单元在编码过程中，确定编码块的块划分方式。
实际应用中，划分方式可以采用四叉树划分、二叉树划分、三叉树划分中的一种或多种。
需要补充的是，块划分单元203可以将编码块划分成一个或多个编码块，划分得到的编码块可以进一步进行划分成更多的编码块。还可以将编码块划分成一个或多个预测块。
示例性的，块划分单元203使用RDO方法确定编码块的划分方式。
其中，块划分单元输出参数包括编码块的划分方式参数，这些参数用于指示编码块的划分方式。
运动预测单元204，设置为将解码图像缓冲区214中的一个或多个已解码图像作为参考图像，并根据参考图像构造一个或多个参考图像列表，确定编码块在参考图像中的匹配块。
其中，每个参考图像列表中包含一个或多个参考图像。
实际中，运动预测单元204的输出用于指示匹配块位置的参数，可以包括参考图像列表指示、参考图像索引(Reference Index)、运动矢量(Motion Vector，MV)等。其中，参考图像列表指示用于指示包含匹配块的参考图像所在的参考图像列表，参考图像索引用于指示参考图像列表中包含匹配块的参考图像，MV用于指示编码块与匹配块在同一个图像像素点坐标系下相互之间的相对位置偏移量。
运动补偿单元205，设置为根据匹配块构造编码块的预测值，并且计算编码块与预测值之间的差值。
实际中，运动补偿单元205的输出还包括构造编码块预测值的参数，例如对匹配块的加权值、对匹配块进行滤波处理的滤波器类型和参数等。
示例性的，运动估计单元204和运动补偿单元205联合使用RDO方法，为编码块确定率失真性能最优的匹配块以及两个单元的输出参数。
可选择地，运动估计单元204和运动补偿单元205可以使用编码块所在的当前编码图像作为参考图像，获得编码块的帧内预测值。
需要说明的是，帧内预测指的是仅使用编码块所在图像中的数据作为参考得到的预测值。在这种情况下，运动估计单元204和运动补偿单元205使用的是当前编码图像中已经部分恢复的部分，即未经过滤波单元213处理的部分，输入数据可以来自于第二加法器212的输出，例如使用一个图像缓冲区存储第二加法器212的输出数据。可选择地，这个图像缓冲区是解码图像缓冲区214中的一个特殊的图像缓冲区。
帧内预测单元206，设置为确定编码块的帧内预测值。
实际应用中，帧内预测单元206根据编码块所在图像中已经部分恢复的部分，为编码块确定帧内预测参考样值，将帧内预测参考样值作为滤波器的输入值来计算编码块的帧内 预测值。这里，滤波器可以是插值滤波器，也可以是低通滤波器(例如用于计算DC值的滤波器)。可选择的，帧内预测单元206使用RDO的方法确定用于计算编码块帧内预测值的方法(即帧内预测模式)以及帧内预测值。
帧内预测单元206确定编码块相邻已经部分恢复像素样值是否存在。当帧内预测单元206判断编码块的相邻样值均存在时，帧内预测单元206将相邻样值或相邻样值经过滤波处理的值设置为帧内预测参考样值。反之，当帧内预测单元206判断编码块的全部或部分样值不存在时，帧内预测单元206确定第一参考样值，并使用第一参考样值确定编码块的帧内预测参考样值。
其中，帧内预测单元206根据帧内预测模式使用的参考样值对应的相邻像素点的位置，判断相邻像素点位置上的相邻样值是否存在。通常情况下，帧内预测单元206通过以下方式判断编码块的相邻样值是否存在：
1、编码块的参考像素点位置上的像素尚未编码时，对应相邻样值不存在；
2、编码块位于图像的边界位置时，图像边界外的参考像素点位置上不存在相邻样值。例如，编码块位于图像的上边界时，编码块上相邻样值不存在；
3、位于不同于编码块所在分片的相邻样值不存在；
4、位于不同于编码块所在瓦片的相邻样值不存在。
5、在使用帧内预测受限的情况下，位于非帧内预测编码的编码块中的相邻样值不存在。
需要说明的是，帧内预测单元206确定第一参考样值的方法包括多种，一种是，帧内预测单元206将第一参考样值的取值设置为编码块的不存在的相邻样值(例如，当编码块位于分片边界时，使用编码块的位于不同分片中的相邻样值设置第一参考样值)。另一种是，帧内预测单元206根据编码块中待编码的像素样值确定第一参考样值。再一种是，帧内预测单元206使用软判决(Soft Decision)的方法，在RDO的过程中计算第一参考样值。
其中，软判决可以使用如下的约束最优化问题进行表述：
约束最优化1：
min D＝∑ kd(x k,p k)s.t.R≤R C…………………………………………(1)
其中，D表示编码块的编码失真；k的遍历范围是编码块中的全部像素；d(a,b)表示a和b之间的误差值，可采用的误差准则包括均方误差准则(Mean Square Error，MSE)、绝对误差和(Sum of Absolute Value，SAD)、也可以是与人眼视觉系统(Human Visual System，HVS)相关的感知误差准则；x k是编码块中待编码像素样值；p k是x k的预测值，在帧内预测过程中，帧内预测单元206使用帧内预测参考样值根据帧内预测模式计算p k取值；R表示编码块的编码比特开销，包括编码块的编码模式信息的开销、第一参考样值的开销、预测差编码开销等，R的值可以根据编码块的模式、预测差、量化参数等估计得到(例如使用率失真(Rate-Distortion，RD)模型估计得到)，也可以对编码块进行实际 编码得到；R C是编码块编码比特开销的上限值，编码器可以使用码率控制(Rate Control，RC)单元中对编码块的比特分配(Bit Allocation)过程确定。
需要说明的是，约束最优化1中，根据帧内预测计算过程，p k的值根据帧内预测参考样值和帧内预测模式确定，而帧内预测参考样值由第一参考样值确定。因此，帧内预测单元206对约束最优化1进行求解，可以同时确定最优的第一参考样值和帧内预测模式。帧内预测单元206可使用相关算法求解约束最优化1。
可以理解的是，为降低计算复杂度，使用Lagrange因子法，约束最优化1可以转换为无约束最优化问题：
无约束最优化1：
(r,m) opt＝arg min(D+λ·R)…………………………………………(2)
其中，r是第一参考样值；m是编码块的帧内预测模式；D和R与约束最优化1中的D和R相同；λ是Lagrange因子，一种常用的方法是，编码器可使用λ与量化参数(Quantization Parameter，QP)之间的函数关系确定λ的取值。编码器在RC单元中可以确定QP的取值。帧内预测单元206对无约束最优化1进行求解，可以同时确定最优的第一参考样值和帧内预测模式。帧内预测单元206可使用现有算法求解无约束最优化1。
通常情况下，与求解约束最优化1相比，帧内预测单元206求解无约束最优化1所需的计算复杂度较低。尽管如此，在无约束最优化1的基础上，帧内预测单元206可以根据编码块的相邻像素的位置和所使用的帧内预测模式，使用近似的方法求解无约束最优化1，从而进一步降低计算复杂度。
示例性而言，帧内预测单元206可以为解码块确定一个或多个第一参考样值，对于第一参考样值中包含多个样值的情况，进一步确定第一参考样值中样值的使用方式指示参数，用于指示在确定不同位置的帧内预测参考样值过程中使用的第一参考样值中的样值。例如，对编码块左相邻样值和上相邻样值使用取值不同的第一参考样值。这里，指示参数也包含在第一参考样值中。
可选择地，帧内预测单元206可以根据预先设定的规则推导指示参数。例如，当第一参考样值中包含两个参考样值时，帧内预测单元206使用的预先设定的规则是：按照将第一参考样值中的样值写入码流的顺序，第一个样值用于构造编码块的左相邻帧内预测参考样值，第二个样值用于构造编码块的上相邻帧内参考样值。
可选择地，帧内预测单元206还可以根据编码块在图像中的位置、编码块的帧内预测模式推导指示参数。
例如，当第一参考样值中包含两个参考样值时，帧内预测单元206使用的预先设定的规则是：按照将第一参考样值中的样值写入码流的顺序，第一个样值用于构造编码块的左相邻帧内预测参考样值，第二个样值用于构造编码块的上相邻帧内参考样值，当编码块位于图像左边界(编码块的左相邻样值在图像中不存在)且不是图像上边界(编码块的上相邻样值在图像中存在)时，如果编码块的上边界是分片或瓦片的边界(编码块的上相邻样 值不能作为帧内预测参考样值)，在编码块使用DC模式时，使用第二个样值作为该模式下编码块的左相邻样值和上相邻样值。
可选择地，帧内预测单元206在编码块所在图像中已经部分恢复的部分中搜索编码块的匹配块，将匹配块作为编码块的帧内预测值。
这种情况下，帧内预测单元206的输出参数还包括用于指示匹配块位置的信息，例如，在同一个图像坐标系下，编码块与匹配块在图像中位置的相对偏移量。需要说明的是，帧内预测单元206的输出参数包括第一参考样值、用于指示帧内预测模式的参数。
第一加法器207，设置为计算编码块的原始值与预测值之间的预测差值。其中，预测差值是变换单元208的输入值。
变换单元208，设置为对预测差值进行变换，输出预测差值进行变换后得到的变换系数。
实际应用中，变换单元208可以使用一种或多种变换方法对预测差值进行变换。从信号处理的角度，每一种变换方法都可以用一个变换矩阵来表示。变换单元208可以使用与编码块大小和形状相同的矩形块(这里，正方形是矩形的特例)作为预测差值的变换块，也可以将预测差值划分成多个矩形块(包括高度为一个像素的情况)并分别对矩形块依次进行变换处理。
可选择地，变换单元208可以对预测差值进行多次变换。在可以使用多种变换方法、对预测差值划分成多个矩形块进行变换、对预测差进行多次变换的情况下，变换单元208使用RDO的方法确定用于预测差值的变换参数，变换参数与用于指示变换过程的执行方式。变换单元208将变换参数作为输出参数。
量化单元209，设置为对变换单元208输出的是预测差值进行变换后得到的变换系数进行量化处理，输出变换系数的量化值。
通常，量化单元可以使用的量化器包括标量量化器和矢量量化器。在视频编码器中，量化单元209使用标量量化器对变换系数进行量化，量化器的量化参数由编码器控制(Code Control)单元确定。例如，编码器控制单元可以使用已有的码率控制方法，确定量化器的量化步长，并根据编码器中量化步长与QP的对应关系确定QP。量化单元209的相关参数是QP。
反量化单元210，设置为使用与量化单元209相同的QP，对变换系数的量化值进行缩放(Scaling)操作，得到变换系数的恢复值。
反变换单元211，设置为使用与变换单元208中使用的变换的逆变换对变换系数的恢复值进行处理，得到预测差的恢复值。
第二加法器212，设置为根据预测差的恢复值和预测单元202输出的编码块的预测值，计算编码块的恢复值，并将编码块的恢复值存储至图像缓冲区。
其中，图像缓冲区可以是对图像编码过程中单独分配的一段存储空间，也可以是解码图像缓冲区214中的一个图像缓冲区。
滤波单元213，设置为对图像缓冲区中的数据进行滤波处理，得到图像的解码图像。
实际中，滤波单元213可以是一种或多种滤波器级联构成。例如，在H.265/HEVC标准中，滤波单元213由去方块滤波和采样值自适应加性偏移量补偿滤波器(Sample Adaptive Offset，SAO)两个滤波器级联构成。滤波单元213也可以包括神经网络滤波器。
可选择地，滤波单元213对图像缓冲区中的数据进行滤波的操作可以在图像层进行操作，即等待图像中的全部编码块的恢复值均写入了图像缓冲区后，对图像缓冲区中的数据进行滤波处理。
可选择地，滤波单元213对图像缓冲区中的数据进行滤波的操作可以在块层进行操作，即当某个编码块的恢复数据不再用做后续编码块的参考数据时，对该编码块的恢复数据进行滤波处理。
滤波单元213使用现有RDO方法确定滤波器参数，并将其作为滤波单元213的输出参数。滤波器参数包括所使用的滤波器的指示信息、滤波器系数、滤波器的控制参数。
解码图像缓冲区214，设置为存储滤波单元213输出的解码图像。
解码图像缓冲区214确定解码图像管理相关的参数指令，设置为控制解码图像在解码图像缓冲区214中的存储时长和输出等操作。在本实施例中，这些参数指令可作为解码图像缓冲区214的输出参数。
熵编码单元215，设置为对图像的编码数据进行二值化和熵编码，将参数转换为符合标准的由一个或多个“0”、“1”比特组成的字段，并根据标准中码流语法结构(Syntax Structure)将字段组织成码流。
其中，熵编码数据包括图像的纹理数据和非纹理数据。其中，纹理数据主要是编码块的变换系数的量化值；非纹理数据包括除纹理数据外的其他所有数据，包括前述编码器中各单元的输出参数、以及参数集、头信息、辅助信息等参数。熵编码单元215根据视频编码标准的码流组织形式生成码流。
实施例六
参见图6，本公开的实施例提出了一种解码器的系统架构示意图，用于解码前述编码器所产生的码流。解码器包括解析单元301、预测单元302、反量化单元305、反变换单元306、加法器307、滤波单元308，以及解码图像缓冲区309。其中，预测单元302包括运动补偿单元303和帧内预测单元304。
需要说明的是，解码器的输入是码流，输出是对输入码流进行解码后产生的解码视频。
解析单元301，设置为对输入码流进行解析，使用标准中规定的熵解码方法和二值化方法，将码流中各字段对应的一个或多个“0”、“1”比特串转换成对应参数的取值。解析单元301根据参数的取值，推导得到其他参数的取值，例如，当码流中的标志位取值指示解码块是图像中的第一个解码块时，将用于指示解码块所在分片中第一个解码块在图像中的地址参数设置为0。
在一实施例中的，解析单元301将用于构造解码块预测值的参数传递给预测单元302。这里，用于构造解码块预测值的参数包括前述编码器中划分单元201、预测单元202的输出参数。
在一实施例中的，解析单元301将用于构造解码块预测差恢复值的参数传递给反量化单元305、反变换单元306。这里，用于构造解码块预测差恢复值的参数包括前述编码器中变换单元208、量化单元209的输出参数，以及前述编码器中量化参数209输出的变换系数的量化值。
预测单元302，设置为根据用于构造解码块预测值的参数构造解码块的预测值。
需要说明的是，预测单元302的输入还包括加法器307输出的当前解码图像中已经部分恢复的部分(未经过滤波单元308处理)、解码图像缓冲区309中存储的已解码图像。
示例性的，当参数指示解码块使用帧间预测时，预测单元302使用与前述编码器中运动预测单元204相同的方法，构造一个或多个参考图像列表，每个参考图像列表中包含一个或多个参考图像，参考图像来自于解码图像缓冲区309。运动补偿单元303根据解析单元301所传递的参考图像列表指示、参考图像索引、运动矢量，在参考图像中确定解码块的一个或多个匹配块，使用与前述编码器中运动补偿单元205相同的方法确定帧间预测值。预测单元302将运动补偿单元303输出的帧间预测值作为解码块的预测值。
可选择地，运动补偿单元303可以使用解码块所在的当前解码图像作为参考图像，获得解码块的帧内预测值。
这里，帧内预测指的是仅使用解码块所在图像中的数据作为参考得到的预测值。在这种情况下，运动补偿单元303使用的是当前解码图像中已经部分恢复的部分(未经过滤波单元308处理)，输入数据可以来自于加法器307的输出，例如使用一个图像缓冲区存储加法器307的输出数据，可选择地，这个图像缓冲区是解码图像缓冲区309中的一个特殊的图像缓冲区。
示例性的，当参数指示解码块使用帧内预测时，预测单元302根据用于构造解码块预测值的参数构造解码块的预测值。帧内预测单元304获得用于构造解码块预测值的参数中的第一参考样值。当帧内预测单元304判断解码块的相邻样值均存在时，帧内预测单元304将相邻样值或相邻样值经过滤波处理的值设置为帧内预测参考样值。反之，当帧内预测单元304判断解码块的全部或部分样值不存在时，帧内预测单元304使用第一参考样值确定解码块的帧内预测参考样值。
其中，帧内预测单元304使用以下条件判断解码块的相邻样值是否存在：
解码块的参考像素点位置上的像素尚未解码时，对应相邻样值不存在；
解码块位于图像的边界位置时，图像边界外的参考像素点位置上不存在相邻样值。例如，解码块位于图像的上边界时，解码块上相邻样值不存在；
位于不同于解码块所在分片的相邻样值不存在；
位于不同于解码块所在瓦片的相邻样值不存在；
在使用帧内预测受限的情况下，位于非帧内预测解码的解码块中的相邻样值不存在。
示例性而言，帧内预测单元304设置第一参考样值的方法包括多种，一种方法是，帧内预测单元304将解码块的帧内预测参考像素点位置上不存在的像素样值设置为第一参考样值的取值。如果第一参考样值中仅包含一个样值，当解码块的相邻样值全部或部分不存在时，帧内预测单元304将解码块的相邻样值设置为第一参考样值的取值。
可选择地，如果第一参考样值中包含多个样值时，帧内预测单元304可以根据第一参考样值中的参考样值的使用方式指示参数，确定不同位置的帧内预测参考样值过程中使用的第一参考样值中的样值，例如，对解码块左相邻样值和上相邻样值使用取值不同的第一参考样值。
可选择地，帧内预测单元304可以根据预先设定的规则推导指示参数。例如，当第一参考样值中包含两个参考样值时，帧内预测单元304使用的预先设定的规则是：按照将第一参考样值中的样值在码流中的先后顺序(即两个样值的解析顺序)，将第一个样值用于构造解码块的左相邻帧内预测参考样值，将第二个样值用于构造解码块的上相邻帧内参考样值。
可选择地，帧内预测单元304可以根据解码块在图像中的位置、解码块的帧内预测模式推导指示参数。例如，当第一参考样值中包含两个参考样值时，帧内预测单元304使用的预先设定的规则是：按照第一参考样值中的样值在码流中的先后顺序(即两个样值的解析顺序)，将第一个样值用于构造解码块的左相邻帧内预测参考样值，将第二个样值用于构造解码块的上相邻帧内参考样值，当解码块位于图像左边界(解码块的左相邻样值在图像中不存在)且不是图像上边界(解码块的上相邻样值在图像中存在)时，如果解码块的上边界是分片或瓦片的边界(解码块的上相邻样值不能作为帧内预测参考样值)，在解码块使用DC模式时，将第二个样值作为该模式下解码块的左相邻样值和上相邻样值。
帧内预测单元304根据用于构造解码块预测值的参数，确定帧内预测模式，使用与前述编码器中帧内预测单元206相同的方法，使用帧内预测参考样值计算解码块的帧内预测值。可选择地，当用于构造解码块预测值的参数指示使用匹配块构造解码块的预测值时，帧内预测单元304从用于构造解码块预测值的参数中获得解码块与匹配块之间位置偏移量，在解码块所在的当前部分解码图像中获得匹配块，将匹配块设置为解码块的帧内预测值。预测单元302将帧内预测单元304输出的帧内预测值作为解码块的预测值。
反量化单元305，设置为接收用于构造解码块预测差恢复值的参数中QP和变换系数的量化值。反量化单元305使用QP，对变换系数的量化值进行缩放操作，得到变换系数的恢复值。因此，解码器中的反量化单元也可以称为缩放(Scaling)单元。
反量化单元305，向反变换单元306输出变换系数的恢复值、用于构造解码块预测差恢复值的参数中前述编码器中变换单元208所输出的变换参数。
反变换单元306，使用与前述编码器中反变换单元211相同的方法计算得到解码块的预测差的恢复值。
这里需要说明的是，这里的“反变换”是相对于编码器中的“变换”来说的。在视频编码标准中，规定的是反变换单元306所使用的变换方法，即解码器将变换系数的恢复值转换为预测差的恢复值所使用的变换方法。
加法器307，根据反变换单元306输出的预测差的恢复值和预测单元302输出的解码块的预测值，计算解码块的恢复值，并将解码块的恢复值存储至图像缓冲区。
其中，图像缓冲区可以是对图像解码过程中单独分配的一段存储空间，也可以是解码图像缓冲区309中的一个图像缓冲区。
滤波单元308，设置为接收解析单元301输出的滤波器参数，并根据滤波器参数对图像缓冲区中的数据进行滤波处理，得到图像的解码图像。
其中，滤波器参数是前述编码器中滤波单元213的输出参数，包括所使用的滤波器的指示信息、滤波器系数、滤波器的控制参数。
实际中，滤波单元308可以是一种或多种滤波器级联构成。例如，在H.265/HEVC标准中，滤波单元308由去方块滤波和采样值自适应加性偏移量补偿滤波器两个滤波器级联构成。滤波单元308也可以包括神经网络滤波器。可选择地，滤波单元308对图像缓冲区中的数据进行滤波的操作可以在图像层进行操作，即等待图像中的全部解码块的恢复值均写入了图像缓冲区后，对图像缓冲区中的数据进行滤波处理。可选择地，滤波单元308对图像缓冲区中的数据进行滤波的操作可以在块层进行操作，即当某个解码块的恢复数据不再用做后续解码块的参考数据时，对该解码块的恢复数据进行滤波处理。
解码图像缓冲区309，设置为存储滤波单元308输出的解码图像。
在一实施例中，解码器使用解析单元301输出的解码图像管理相关的参数指令(即前述编码器中解码图像缓冲区214的输出参数)，控制解码图像在解码图像缓冲区309中的存储时长和输出等操作。
实施例七
参见图7，本公开的实施例提出了一种编码器的帧内预测单元的结构示意图。帧内预测单元206包含相邻样值检测单元401、相邻样值处理单元402、帧内预测模式选择单元403和帧内预测计算单元404。
需要说明的是，数据流指的是软件实现上函数的入口参数和返回参数，硬件实现上总线上传递的数据、存储单元之间共享的数据(包括寄存器共享数据)等。
相邻样值检测单元401，设置为接收当前编码图像中已经部分恢复的部分数据，并生成编码块的相邻样值信息。
可选的，编码块的相邻样值信息包括相邻样值所在相邻像素点的位置、相邻样值是否存在、相邻样值的取值(若相邻样值存在)。
可选的，当前编码图像中已经部分恢复的部分数据包括码块在图像中的位置信息、以及编码块在图像划分(例如分片)中的位置信息。
示例性而言，输入数据包括数据流40，输出数据是数据流41。数据流40是前述编码器在编码编码块之前，第二加法器212输出数据，是当前编码图像中已经部分恢复的部分数据(未经过前述编码器中滤波单元213处理)。数据流40中也包括编码块在图像中的位置信息、以及编码块在图像划分(例如分片)中的位置信息，这些位置信息的来源包括前述编码器中块划分单元201、块划分单元203的输出数据。数据流41是相邻样值检测单元401的输出数据，是编码块的相邻样值信息，包括相邻样值所在相邻像素点的位置、相邻样值是否存在、相邻样值的取值(若相邻样值存在)。
相邻样值处理单元402，设置为接收编码块的相邻样值信息、帧内预测模式指示信息以及编码块的原始值，并生成帧内预测参考样值。
示例性而言，相邻样值处理单元402的输入是数据流41、数据流43和数据流44，输出是数据流42。数据流41是相邻样值检测单元401的输出。数据流43是帧内预测模式选择单元403提供的，包含的是帧内预测模式指示信息。数据流44是前述编码器中块划分单元201输出数据，是编码块的原始值。相邻样值处理单元402的输出数据流42是帧内预测参考样值。
帧内预测模式选择单元403，设置为接收帧内预测参考样值和编码块的原始值，生成帧内预测模式指示信息。
示例性而言，帧内预测模式选择单元403的输入是数据流42、数据流44，输出是数据流45。数据流42是相邻样值处理单元402的输出数据。数据流44是前述编码器中块划分单元201的输出数据，是编码块的原始值。数据流45是帧内预测模式指示信息(即帧内预测模式的模式索引)，指示对编码块进行编码使用的帧内预测模式，设置为计算编码块的帧内预测值。数据流45也包含数据流42中的帧内预测参考样值。
帧内预测计算单元404，设置为根据帧内预测参考样值和帧内预测模式指示信息，计算得到编码块的帧内预测值。
帧内预测计算单元404的输入是数据流45，输出是数据流46。数据流45是帧内预测模式选择单元403的输出数据。数据流46是帧内预测计算单元404使用数据流45中的帧内预测参考样值，按照数据流45中帧内预测模式指示信息所指示的帧内预测模式，计算得到编码块的帧内预测值。
可选择地，在编码器中可以不存在帧内预测计算单元404。帧内预测模式选择单元403在为编码块选择最优编码模式的过程中，对部分或全部候选帧内预测模式进行遍历，分别计算候选模式下编码块的帧内预测值，选择可最小化编码块的编码失真与编码比特加权和的帧内预测模式作为编码块的最优帧内预测模式，同时输出该最优帧内预测模式的指示信息和对应的帧内预测值，即帧内预测模式选择单元403输出的数据流45中，包含编码块的最优帧内预测模式的指示信息和帧内预测值。
帧内预测单元206可使用帧内预测方法1和帧内预测方法2计算编码块的帧内预测值。帧内预测方法帧内预测方法1，其中，“Curr”表示的是编码块，其相邻已解码像素 点使用灰色表示。图8为本公开实施例提供的一种帧内预测方法的示意图，图8中，示例性地标示出编码块的上相邻和左相邻已解码像素点。在多种不同的编码顺序下，编码块的右相邻或下相邻也可能存在已解码像素点。帧内预测方法1包含一种或多种帧内预测模式，例如，直流预测模式、平面预测模式、方向性插值预测模式等。在使用帧内预测方法1时，帧内预测单元206中的帧内预测模式选择单元403输出的是指示所使用的帧内预测模式的参数。
图9为本公开实施例提供的另一种帧内预测方法的示意图，其中，“Curr”表示的是编码块，灰色区域表示对编码块进行编码前，编码块所在图像中已经部分恢复的部分数据，“Ref”表示编码块的匹配块。在使用帧内预测方法2时，帧内预测模式选择单元403输出的是指示构造匹配块的参数，例如，匹配块在图像中的位置参数。帧内预测方法2也可以包含一种或多种帧内预测模式，例如，块匹配模式、串匹配模式等。
通常情况下，帧内预测单元206使用帧内预测方法1。在允许使用帧内预测方法2的编码器中，帧内预测单元206中的帧内预测模式选择单元403的输出数据流45中还包含帧内预测方法的指示信息，指示使用帧内预测方法1还是帧内预测方法2计算编码块的帧内测值。
相邻样值检测单元401根据数据流40中编码块的位置信息，确定编码块在图像中的位置。对于帧内预测方法1，相邻样值检测单元401判断在数据流40的图像中已经部分恢复的部分数据中与编码块相邻的像素点位置的像素样值是否存在。相邻样值检测单元401使用的判断方法如下。相邻样值检测单元401将帧内预测方法1相关的相邻样值信息包含进入数据流41。
编码块的参考像素点位置上的像素尚未编码时，对应相邻样值不存在。
编码块位于图像的边界位置时，图像边界外的参考像素点位置上不存在相邻样值。例如，编码块位于图像的上边界时，编码块上相邻样值不存在。
位于不同于编码块所在分片的相邻样值不存在。
位于不同于编码块所在瓦片的相邻样值不存在。
在使用帧内预测受限的情况下，位于非帧内预测编码的编码块中的相邻样值不存在。
可选择地，当编码器允许使用帧内预测方法2时，相邻样值检测单元401判断图9中“Ref”块中所有样值均存在的范围，当“Ref”块位于范围内时，“Ref”块中所有样值均存在；反之，“Ref”块中的部分或全部样值均不存在。相邻样值检测单元401使用与上述设置为帧内预测方法1相同的判断方法，确定“Ref”块中所有样值均存在的范围。相邻样值检测单元401将帧内预测方法2相关的相邻样值信息包含进入数据流41。
相邻样值处理单元402对数据流41的数据进行处理。对于帧内预测方法1相关数据，当数据流41中的相邻样值均存在时，相邻样值处理单元402可选择地对数据流41中的相邻样值进行滤波处理。
可选择地，一种滤波处理方法是，相邻样值处理单元402可以使用已有RDO方法确 定是否对相邻样值进行滤波，并产生标志信息指示是否对相邻样值进行滤波。
可选择地，为降低相邻样值处理单元402的计算复杂度，一种方法是对特定大小的块判断是否使用滤波。示例性而言，当编码块的大小等于预先设定的值或者预先设定的多个值之一时，相邻样值处理单元402使用已有RDO方法确定是否对编码块的相邻样值进行滤波，并产生标志信息指示是否对相邻样值进行滤波。可选择地，为降低相邻样值处理单元402的计算复杂度，一种方法是，当编码块的大小等于预先设定的值或者预先设定的多个值之一时，相邻样值处理单元402对编码块的相邻样值进行滤波。
可选择地，为降低相邻样值处理单元402的计算复杂度，一种方法是对特定帧内预测模式判断是否使用滤波。示例性而言，如果数据流43中的帧内预测模式是预先设定的模式或预先设定的多个模式之一时，相邻样值处理单元402使用已有RDO方法确定是否对编码块的相邻样值进行滤波，并产生标志信息指示是否对相邻样值进行滤波。可选择地，为降低相邻样值处理单元402的计算复杂度，一种方法是，如果数据流43中的帧内预测模式是预先设定的模式或预先设定的多个模式之一时，相邻样值处理单元402对编码块的相邻样值进行滤波。
可选择地，为降低相邻样值处理单元402的计算复杂度，一种方法是对特定大小的块、特定帧内预测模式判断是否使用滤波。示例性而言，当编码块的大小等于预先设定的值或者预先设定的多个值之一、并且数据流43中的帧内预测模式是预先设定的模式或预先设定的多个模式之一时，相邻样值处理单元402使用已有RDO方法确定是否对编码块的相邻样值进行滤波，并产生标志信息指示是否对相邻样值进行滤波。可选择地，为降低相邻样值处理单元402的计算复杂度，一种方法是，当编码块的大小等于预先设定的值或者预先设定的多个值之一、并且数据流43中的帧内预测模式是预先设定的模式或预先设定的多个模式之一时，相邻样值处理单元402对编码块的相邻样值进行滤波。相邻样值处理单元402将编码块的相邻样值包含进数据流42。
对于帧内预测方法1相关数据，当数据流41中的相邻样值中部分存在或全部都不存在时，相邻样值处理单元402确定第一参考样值。相邻样值处理单元402确定第一参考样值的输入数据是数据流41中的相邻样值(若相邻样值部分存在)和相邻样值对应的相邻像素点位置、数据流43中的候选帧内预测模式和数据流44中的编码块原始值。相邻样值处理单元402使用前述实施例中约束最优化1或无约束最优化1的方法，计算第一参考样值。需要说明的是，约束最优化1或无约束最优化2的方法中，使用的第一参考样值可以是一个也可以是多个。在第一参考样值是多个的情况下，相邻样值处理单元402需要记录不同第一参考样值的使用方式参数，参数设置为指示不同第一参考样值在设置相邻样值中的使用方式。相邻样值处理单元402将不存在的相邻样值的取值设置为第一参考样值的取值，将输入数据流41中相邻样值均设置为“存在”或“可用”。可选择地，相邻样值处理单元402可使用前述滤波方法，对相邻样值进行滤波处理。相邻样值处理单元402将经过滤波处理或不经过滤波处理的数据包含进数据流42。
为降低计算复杂度，相邻样值处理单元402可使用简化的方法确定第一参考样值。一种方法是，若数据流41指示编码块的相邻样值均不存在，相邻样值处理单元402将第一参考样值设置为数据流44中编码块原始值的平均值。一种方法是，当数据流43中的帧内预测模式信息指示DC模式时，相邻样值处理单元402将第一预参考的取值设置为数据流44中编码块原始值的平均值与数据流41中部分存在的相邻样值(若有)均值的差值。一种方法是，相邻样值处理单元402将第一预参考的取值设置为数据流44中编码块原始值的平均值与数据流41中部分存在的相邻样值(若有)均值的差值。
帧内预测模式选择单元403为编码块确定帧内预测模式。帧内预测模式选择单元403可使用已有RDO方法确定编码块的帧内预测模式。在实现方法上，帧内预测模式选择单元403可以与相邻样值处理单元402一起使用联合优化的方法确定帧内预测模式和第一参考样值。例如，帧内预测模式选择单元403将候选帧内预测模式通过数据流43传递给相邻样值处理单元402；相邻样值处理单元402确定该候选帧内预测模式下的第一参考样值，使用该第一参考样值设置相邻样值，并将处理后的相邻样值通过数据流42传递给帧内预测模式选择单元403；帧内预测模式选择单元403使用数据流42中的相邻样值计算候选帧内预测模式的代价函数值。在上述联合优化过程中，如果相邻样值处理单元402使用约束最优化1或者无约束最优化2的方法确定第一参考样值，则相邻样值处理单元402也同时确定了候选帧内预测模式的代价函数值，在这种情况下，相邻样值处理单元402也将代价函数值通过数据流42传递给帧内预测模式选择单元403。帧内预测模式选择单元403比较各候选帧内预测模式的代价函数值，选择代价函数值最小的候选帧内预测模式，作为编码块的帧内预测模式。
对于帧内预测方法2相关数据，相邻样值处理单元402确定第一参考样值的方法与上述设置为帧内预测方法1的方法相似。不同之处在于，当允许使用帧内预测方法2时，帧内预测模式选择单元403包含在数据流43中的是图9中“Ref”块在图像中的位置。当相邻样值处理单元402判断“Ref”块中样值不是全部位于数据流41中指示的“Ref”块中所有样值均存在的范围内时，相邻样值处理单元402可使用约束最优化1或无约束最优化2的方法确定第一参考样值，将“Ref”块中不存在的样值设置为第一参考样值。与前述设置为帧内预测方法1的方法类似，约束最优化1或无约束最优化2的方法中，使用的第一参考样值可以是一个也可以是多个。在第一参考样值是多个的情况下，相邻样值处理单元402需要记录不同第一参考样值的使用方式参数，参数设置为指示不同第一参考样值在设置相邻样值中的使用方式。
与前述设置为帧内预测方法1的方法类似，对于帧内预测方法2相关数据，为降低计算复杂度，相邻样值处理单元402可使用简化的方法确定第一参考样值。例如，一种方法是，相邻样值处理单元402将第一预参考的取值设置为数据流44中编码块原始值的平均值与数据流41中部分存在的相邻样值(若有)均值的差值，特殊地，若数据流41指示编码块的相邻样值均不存在，相邻样值处理单元402将第一参考样值设置为数据流44中编 码块原始值的平均值。
与前述设置为帧内预测方法1的方法类似，对于帧内预测方法2相关数据，帧内预测模式选择单元403为编码块确定“Ref”块的位置。帧内预测模式选择单元403可使用已有RDO方法确定编码块的“Ref”块的位置。在实现方法上，帧内预测模式选择单元403可以与相邻样值处理单元402一起使用联合优化的方法确定“Ref”块的位置和第一参考样值。例如，帧内预测模式选择单元403将候选“Ref”块的位置通过数据流43传递给相邻样值处理单元402；相邻样值处理单元402确定该候选“Ref”块的位置的第一参考样值，使用该第一参考样值设置“Ref”块中样值，并将处理后的“Ref”块中样值通过数据流42传递给帧内预测模式选择单元403；帧内预测模式选择单元403使用数据流42中的“Ref”块中样值计算候选“Ref”块的位置的代价函数值。在上述联合优化过程中，如果相邻样值处理单元402使用约束最优化1或者无约束最优化2的方法确定第一参考样值，则相邻样值处理单元402也同时确定了候选“Ref”块的位置的代价函数值，在这种情况下，相邻样值处理单元402也将代价函数值通过数据流42传递给帧内预测模式选择单元403。帧内预测模式选择单元403比较各候选“Ref”块的位置的代价函数值，选择代价函数值最小的候选“Ref”块的位置，作为编码块的“Ref”块的位置，并通过数据流45输出设置为指示“Ref”块的位置的参数，参数表示为在同一图像坐标系下编码块与“Ref”块之间的位置偏移量。
需要说明的是，可选择地，当数据流41中的相邻样值中部分存在或全部都不存在时，相邻样值处理单元402可以将不存在的相邻样值或“Ref”块中不存在的样值设置为默认值(例如背景技术介绍部分的“1<<(BitDepth–1)”)。前述相邻样值处理单元402和帧内预测模式选择单元403的联合优化方法将默认值作为前述一个候选第一参考样值。当使用默认值时，相邻样值处理单元402在数据流42中增加一个设置为指示使用默认值的标志位。
实施例八
图10为本公开实施例提供的一种编码器的熵编码单元对帧内预测单元输出参数进行编码的数据处理的流程示意图。处理流程的输入是帧内预测单元206输出参数，处理流程的输出是帧内预测模式和第一参考样值对应的码流。
步骤501，编码第一参考样值。
除样值外，可选择地，第一参考样值还可以包含样值控制参数，样值控制参数指示使用第一参考样值还是默认值确定使用帧内预测模式对编码块进行编码过程中的不存在的相邻样值。样值控制参数包括以下至少之一的控制参数：序列层样值控制参数，图像层样值控制参数，分片层样值控制参数，块层样值控制参数。
序列层样值控制参数设置为指示是否需要对整个视频中的图像使用第一参考样值确定使用帧内预测模式对编码块进行编码过程中的不存在的相邻样值。当序列层样值控制参 数的取值指示为“是”时，帧内预测单元206在编码块的部分或全部相邻样值不存在的情况下，确定编码块的第一参考样值；反之，当序列层样值控制参数的取值指示为“否”时，帧内预测单元206将不存在的相邻样值设置为默认值。熵编码单元215在有效范围可以是整个视频的参数集中编码序列层样值控制参数。可选择地，当序列层样值控制参数的取值指示为“是”时，熵编码单元215可以在有效范围可以是整个视频的参数集中编码第一参考样值的样值取值，该样值取值可设置为整个视频中使用帧内预测模式的编码块。
图像层样值控制参数设置为指示是否需要对一个图像使用第一参考样值确定使用帧内预测模式对编码块进行编码过程中的不存在的相邻样值。当图像层样值控制参数的取值指示为“是”时，帧内预测单元206在编码块的部分或全部相邻样值不存在的情况下，确定编码块的第一参考样值；反之，当图像层样值控制参数的取值指示为“否”时，帧内预测单元206将不存在的相邻样值设置为默认值。熵编码单元215在有效范围可以是一个图像的参数集中编码图像层样值控制参数。可选择地，当图像层样值控制参数的取值指示为“是”时，熵编码单元215可以在有效范围可以是一个图像的参数集中编码第一参考样值的样值取值，该样值取值可设置为一个图像中使用帧内预测模式的编码块。
分片层样值控制参数设置为指示是否需要对一个分片使用第一参考样值确定使用帧内预测模式对编码块进行编码过程中的不存在的相邻样值。当分片层样值控制参数的取值指示为“是”时，帧内预测单元206在编码块的部分或全部相邻样值不存在的情况下，确定编码块的第一参考样值；反之，当分片层样值控制参数的取值指示为“否”时，帧内预测单元206将不存在的相邻样值设置为默认值。熵编码单元215在分片头中编码分片层样值控制参数。可选择地，当分片层样值控制参数的取值指示为“是”时，熵编码单元215可以在分片头中编码第一参考样值的样值取值，该样值取值可设置为一个分片中使用帧内预测模式的编码块。
块层样值控制参数设置为指示是否需要对一个编码块使用第一参考样值确定使用帧内预测模式对编码块进行编码过程中的不存在的相邻样值。当块层样值控制参数的取值指示为“是”时，帧内预测单元206在编码块的部分或全部相邻样值不存在的情况下，确定编码块的第一参考样值；反之，当块层样值控制参数的取值指示为“否”时，帧内预测单元206将不存在的相邻样值设置为默认值。熵编码单元215在编码块的数据单元中编码块层样值控制参数。可选择地，当块层样值控制参数的取值指示为“是”时，熵编码单元215可以在编码块的数据单元中编码第一参考样值的样值取值，该样值取值可设置为一个编码块的数据单元中使用帧内预测模式的编码块或该编码块的划分。
可选择地，编码器可以将块层样值控制参数以及对应的第一参考样值有效范围设置为一个分片。一种实施方法是，编码器在分片中第一个使用帧内预测模式、并且使用第一参考样值构造编码块的相邻样值时，熵编码单元215在该编码块的数据单元中编码块层样值控制参数以及第一参考样值。编码器对分片中后续使用帧内预测模式的编码块使用相同的编码块层样值控制参数、第一参考样值，熵编码单元215不再为这些编码块编码块层样值 控制参数、第一参考样值。
可选择地，编码器可以将块层样值控制参数以及对应的第一参考样值有效范围设置为一个分片。一种实施方法是，编码器在分片中第一个使用帧内预测模式、并且使用第一参考样值构造编码块的相邻样值时，熵编码单元215在该编码块的数据单元中编码块层样值控制参数(记为“控制参数1”)以及第一参考样值(记为“样值1”)。如果编码器在对分片中后续使用帧内预测模式的编码块编码过程中使用了控制参数1和样值1，熵编码单元215不再为该编码块编码块层样值控制参数、第一参考样值。如果编码器在对分片中后续某个使用帧内预测模式的编码块编码过程中使用了与控制参数1或样值1不同的块层样值控制参数(记为“控制参数2”、“样值2”)，熵编码单元215在该编码块的数据单元中编码“控制参数2”和“样值2”。
特殊地，如果编码器将图7中相邻样值处理单元402设置为以编码块大小来作为是否使用第一参考样值构造编码块的帧内预测相邻样值的判断条件，则熵编码单元215不为块大小符合“不使用”条件的编码块编码块层样值控制参数和第一参考样值，在编码器将块大小符合“可使用”条件的编码块设置为“必然使用”时，熵编码单元215不为块大小符合“必然使用”条件的编码块编码块层样值控制参数，只编码第一参考样值。
特殊地，如果编码器将图7中相邻样值处理单元402设置为以编码块的帧内预测模式来作为是否使用第一参考样值构造编码块的帧内预测相邻样值的判断条件，则熵编码单元215不为帧内预测模式符合“不使用”条件的编码块编码块层样值控制参数和第一参考样值，在编码器将帧内预测模式符合“可使用”条件的编码块设置为“必然使用”时，熵编码单元215不为帧内预测模式符合“必然使用”条件的编码块编码块层样值控制参数，只编码第一参考样值。
特殊地，编码器可以组合使用上述两种特殊情况。
需要说明的是，熵编码单元215执行步骤501完成的是对块层样值控制参数的编码。如果编码器使用了序列层样值控制参数、图像层样值控制参数、分片层样值控制参数中的一种或多种，熵编码单元215需要在生成对应参数集的过程中编码序列层样值控制参数、图像层样值控制参数，熵编码单元215需要在编码分片头的过程中编码分片层样值控制参数，也可以使用前述的可选择方法在块层编码设置为分片层控制的块层样值控制参数。编码器可以根据配置文件来设置序列层、图像层、分片层样值控制参数，也可以使用RDO的方法动态确定上述控制参数的取值。其中，配置文件中记载的是设置为对编码器进行初始化过程中的参数设置。
需要说明的是，从控制参数的有效控制范围来说，序列层的范围大于图像层，图像层的范围大于分片层，分片层的范围大于块层。通常情况下，控制参数的控制机制是：控制范围大的控制参数指示“可以使用”时，编码控制范围小的控制参数，设置为在这个较小的控制范围内指示“是否可以使用”。特别地，对于块层滤波控制参数，如果一个编码块划分成了多个子块，那么这个编码块的块层滤波控制参数的控制范围大于子块，即当这个 编码块的块层样值控制参数指示“不使用”的时候，子块不使用第一参考样值确定帧内预测的参考样值，熵编码单元215不需要编码子块的块层样值控制参数。
需要说明的是，在控制参数指示“可以使用”时，熵编码单元215可以为该控制参数对应编码或推导第一参考样值。通常情况下，有效控制范围小的控制参数对应的第一参考样值覆盖控制范围更大的控制参数对应的第一参考样值。例如，如果熵编码单元215为编码块编码了第一参考样值，那么该编码块在构造帧内预测模式的相邻样值过程中，使用的是块层第一参考样值，而不是分片层第一参考样值。
步骤502，编码帧内预测模式。熵编码单元215对编码块的帧内预测模式进行编码。
情况1：编码器的帧内预测单元206仅使用图7中所示帧内预测方法1。
熵编码单元215首先根据编码块相邻已编码块的帧内预测模式，推导编码块可能使用的一种或多种最可能使用的帧内预测模式。
如果编码块使用的帧内预测模式是最可能使用的帧内预测模式中的一种帧内预测模式，熵编码单元215编码标志位，标志位的取值设置为指示“编码块使用的帧内预测模式是最可能使用的帧内预测模式中的模式”；如果最可能使用的帧内预测模式中包含的模式多于一种，熵编码单元215编码编码块使用的帧内预测模式在最可能使用的帧内预测模式中的索引序号。
如果编码块使用的帧内预测模式是最可能使用的帧内预测模式中的一种帧内预测模式，编码编码块使用的帧内预测模式在除最可能使用的帧内预测模式中的帧内预测模式外的其他帧内预测模式中的索引序号。
情况2：编码器的帧内预测单元206使用图8中帧内预测方法1和图9中帧内预测方法2。
编码用于指示编码块的帧内预测模式属于帧内预测方法1还是帧内预测方法2的标识信息。一种可选择的使用方法是，熵编码单元215编码一个标志位指示上述信息。一种可选择使用的方法是，编码块的帧内预测标记仅用于指示编码块使用的是帧内预测方法1中的帧内预测模式；当编码块使用帧内预测方法2时，熵编码单元215将编码块的帧内预测标记编码为“否”，将编码块的参考图像索引编码为指示“编码块使用的参考图像是编码块所在图像”对应的取值。可选择地，熵编码单元215可进一步编码与帧内预测方法2相关的字段，用于指示编码块使用的帧内预测模式，帧内预测方法2中可包括的帧内预测模式有块匹配模式、串匹配模式等，熵编码单元215继续编码用于确定匹配块、匹配串相关的参数，参数用于确定匹配块、匹配串在编码块所在图像中的位置。
步骤503，熵编码单元215将该编码过程中生成编码比特写入输出码流。
实施例九
图11为本公开实施例提供的一种解码器的帧内预测单元的结构示意图。帧内预测单元304包含相邻样值检测单元601、相邻样值处理单元602和帧内预测计算单元603。
相邻样值检测单元601的输入数据是当前解码图像中已经部分恢复的部分数据、解码块的帧内预测模式指示信息；输出数据是解码块的相邻样值信息。
示例性而言，相邻样值检测单元601的输入数据是数据流60、数据流64，输出数据是数据流61。数据流60是前述解码器在解码解码块之前，加法器307的输出数据，是当前解码图像中已经部分恢复的部分数据(未经过前述解码器中滤波单元308处理)。数据流60中也包括解码块在图像中的位置信息、以及解码块在图像划分(例如分片)中的位置信息，这些位置信息的来源是前述解码器中解析单元301的输出数据。数据流64是解析单元301输出的解码块的帧内预测模式指示信息，设置为指示解码块使用的帧内预测模式。数据流61是相邻样值检测单元601的输出数据，是解码块的相邻样值信息，包括相邻样值所在相邻像素点的位置、相邻样值是否存在、相邻样值的取值(若相邻样值存在)。数据流61中也包括解码块大小。
相邻样值处理单元602的输入是解码块的相邻样值信息、第一参考样值，输出是帧内预测参考样值。
示例性而言，相邻样值处理单元602的输入是数据流61、数据流62和数据流64，输出是数据流63。数据流61是相邻样值检测单元601的输出。数据流62是解析单元301输出的第一参考样值，包括样值控制参数和第一参考样值的取值。可选择地，解码器将数据流64输入给相邻样值处理单元602。数据流64是解析单元301输出的解码块的帧内预测模式指示信息，设置为指示解码块使用的帧内预测模式。相邻样值处理单元602的输出数据流63是帧内预测参考样值。
帧内预测计算单元603的输入是帧内预测参考样值和数据流64，输出是解码块的帧内预测值。
示例性而言，帧内预测计算单元603的输入是数据流63和解码块的帧内预测模式指示信息，输出是数据流65。数据流63是相邻样值处理单元602的输出数据。数据流64是解析单元301输出的解码块的帧内预测模式指示信息，设置为指示解码块使用的帧内预测模式。帧内预测计算单元603按照数据流64中帧内预测模式指示信息所指示的帧内预测模式，使用数据流63中帧内预测参考样值，计算得到解码块的帧内预测值，作为输出数据流65。
与前述编码器中帧内预测单元206相似，前述解码器中帧内预测单元304可使用图8所示帧内预测方法1、图9所示帧内预测方法2计算解码块的帧内预测值，输出数据流65。图8所示的帧内预测方法1中，“Curr”表示的是解码块，其相邻已解码像素点使用灰色表示。图8中，示例性地标示出解码块的上相邻和左相邻已解码像素点。在多种不同的解码顺序下，解码块的右相邻或下相邻也可能存在已解码像素点。帧内预测方法1包含一种或多种帧内预测模式，例如，直流预测模式、平面预测模式、方向性插值预测模式等。在使用帧内预测方法1时，解码器输入帧内预测计算单元603的数据流64是帧内预测模式的索引，输出数据流65是使用数据流64指示的帧内预测模式获得的解码块的帧内预测值。
图9所示的帧内预测方法2中，“Curr”表示的是解码块，灰色区域表示对解码块进行解码前，解码块所在图像中已经部分恢复的部分数据，“Ref”表示解码块的匹配块。在使用帧内预测方法2时，解码器输入帧内预测计算单元603的数据流64是设置为指示“Ref”的位置信息，例如在同一个图像坐标系下“Curr”与“Ref”之间的位置偏移量，输出数据流65是使用数据流64指示的“Ref”位置信息获得“Ref”，使用“Ref”构造解码块的帧内预测值，例如将解码块的帧内预测值设置为“Ref”中的样值或者对“Ref”中样值进行滤波处理后的值。帧内预测方法2也可以包含一种或多种帧内预测模式，例如，块匹配模式、串匹配模式等。
通常情况下，帧内预测单元304使用帧内预测方法1。在允许使用帧内预测方法2的解码器中，数据流64中包含帧内预测方法的指示信息，确定在解码解码块的过程中使用帧内预测方法1还是帧内预测方法2计算解码块的第一预测值。
相邻样值检测单元601根据数据流60中解码块的位置信息，确定解码块在图像中的位置。当数据流64中的帧内预测模式指示信息指示解码块使用帧内预测方法1时，相邻样值检测单元601判断在数据流60的图像中已经部分恢复的部分数据中与解码块相邻的像素点位置的像素样值是否存在。相邻样值检测单元601使用的判断方法如下。相邻样值检测单元601将帧内预测方法1相关的相邻样值信息包含进入数据流61。数据流61中还包括解码块大小的信息。
解码块的参考像素点位置上的像素尚未解码时，对应相邻样值不存在；
解码块位于图像的边界位置时，图像边界外的参考像素点位置上不存在相邻样值。例如，解码块位于图像的上边界时，解码块上相邻样值不存在；
位于不同于解码块所在分片的相邻样值不存在；
位于不同于解码块所在瓦片的相邻样值不存在；
在使用帧内预测受限的情况下，位于非帧内预测解码的解码块中的相邻样值不存在。
当数据流64中的帧内预测模式指示信息指示解码块使用帧内预测方法2时，相邻样值检测单元601判断图9中“Ref”块中所有样值是否均存在。相邻样值检测单元601根据数据流64中“Ref”的位置信息和解码块在图像中的位置，确定“Ref”在图像中的位置。相邻样值检测单元601使用与上述用于帧内预测方法1相同的判断方法，确定“Ref”块中所有样值是否均存在的范围。相邻样值检测单元601将帧内预测方法2相关的相邻样值信息包含进入数据流61。数据流61中还包括解码块大小的信息。
相邻样值处理单元602对数据流61的数据进行处理。当数据流64中的帧内预测模式指示信息指示解码块使用帧内预测方法1时，当数据流61中的相邻样值均存在时，根据数据流64中的指示信息，相邻样值处理单元602对数据流61中的相邻样值进行滤波处理。
可选择地，一种方法是，相邻样值处理单元602根据数据流64中包含的标志信息，确定是否对相邻样值进行滤波。
可选择地，一种方法是仅对特定大小的块使用滤波。示例性而言，如果数据流61中 解码块的大小等于预先设定的值或者预先设定的多个值之一时，相邻样值处理单元602根据数据流64中包含的标志信息，确定是否对解码块的相邻样值进行滤波。一种方法是，如果数据流61中解码块的大小等于预先设定的值或者预先设定的多个值之一时，相邻样值处理单元602对解码块的相邻样值进行滤波。
可选择地，一种方法是仅对特定帧内预测模式使用滤波。示例性而言，如果数据流64中的帧内预测模式是预先设定的模式或预先设定的多个模式之一，相邻样值处理单元602根据数据流64中包含的标志信息，确定是否对解码块的相邻样值进行滤波。可选择地，一种方法是，如果数据流64中的帧内预测模式是预先设定的模式或预先设定的多个模式之一，相邻样值处理单元602对解码块的相邻样值进行滤波。
可选择地，一种方法是对特定大小的块、特定帧内预测模式使用滤波。示例性而言，如果数据流61中解码块的大小等于预先设定的值或者预先设定的多个值之一、并且数据流64中的帧内预测模式是预先设定的模式或预先设定的多个模式之一时，相邻样值处理单元602根据数据流64中包含的标志信息，确定是否对解码块的相邻样值进行滤波。可选择地，一种方法是，如果数据流61中解码块的大小等于预先设定的值或者预先设定的多个值之一、并且数据流64中的帧内预测模式是预先设定的模式或预先设定的多个模式之一时，相邻样值处理单元602对解码块的相邻样值进行滤波。相邻样值处理单元602将解码块的相邻样值包含进数据流63。
数据流64中的帧内预测模式指示信息指示解码块使用帧内预测方法1的情况下，当数据流61中的相邻样值中部分存在或全部都不存在时，相邻样值处理单元602从数据流62中获得第一参考样值，包括样值控制参数和第一参考样值的取值。当样值控制参数指示使用第一参考样值构造不存在的相邻样值时，相邻样值处理单元602将不存在的相邻样值的取值设置为第一参考样值的取值，将解码块的相邻样值均设置为“存在”或“可用”。
可选择地，当第一参考样值是多个的情况下，相邻样值处理单元602根据数据流62中不同第一参考样值的使用方式参数，将不存在的相邻样值的取值设置为对应的第一参考样值的取值，将解码块的相邻样值均设置为“存在”或“可用”。当样值控制参数指示不使用第一参考样值构造不存在的相邻样值时，相邻样值处理单元602使用默认值(例如背景技术介绍部分的“1<<(BitDepth–1)”)设置不存在的相邻样值，将解码块的相邻样值均设置为“存在”或“可用”。
可选择地，相邻样值处理单元602可使用前述滤波方法，对相邻样值进行滤波处理。相邻样值处理单元602将解码块的相邻样值包含进数据流63。
数据流64中的帧内预测模式指示信息指示解码块使用帧内预测方法2的情况下，当样值控制参数指示使用第一参考样值构造“Ref”中不存在的样值时，相邻样值处理单元602将“Ref”中不存在的样值的取值设置为第一参考样值的取值。
可选择地，当第一参考样值是多个的情况下，相邻样值处理单元602根据数据流62中不同第一参考样值的使用方式参数，将“Ref”中不存在的样值的取值设置为对应的第 一参考样值的取值。当样值控制参数指示不使用第一参考样值构造“Ref”中不存在的样值时，相邻样值处理单元602使用默认值(例如使用图像或分片边界像素点位置的样值进行图像填充的方法)设置不存在的相邻样值，将解码块的相邻样值均设置为“存在”或“可用”。相邻样值处理单元602将“Ref”中的样值包含进数据流63。
帧内预测计算单元603按照数据流64中帧内预测模式指示信息所指示的帧内预测模式，使用数据流63中帧内预测参考样值，计算得到解码块的帧内预测值，作为输出数据流65。
实施例十
图12为本公开实施例提供的一种解码器的解析单元对解码块码流中的帧内预测模式和滤波参数进行解析的数据处理的流程示意图。处理流程的输入是输入码流中第一参考样值和帧内预测模式对应的码流，输出是第一参考样值和帧内预测模式。前述解码器将第一参考样值设置为帧内预测单元304中相邻样值处理单元602的输入数据流62。前述解码器将帧内预测模式是作为帧内预测单元304中使用的数据流64。
步骤701，解析第一参考样值。
除样值外，可选择地，第一参考样值还可以包含样值控制参数，样值控制参数指示使用第一参考样值还是默认值确定使用帧内预测模式对解码块进行解码过程中的不存在的相邻样值。样值控制参数包括以下至少之一的控制参数：序列层样值控制参数，图像层样值控制参数，分片层样值控制参数，块层样值控制参数。
序列层样值控制参数用于指示是否需要对整个视频中的图像使用第一参考样值确定使用帧内预测模式对解码块进行解码过程中的不存在的相邻样值。当序列层样值控制参数的取值指示为“是”时，帧内预测单元304在解码块的部分或全部相邻样值不存在的情况下，使用第一参考样值设置解码块的相邻样值；反之，当序列层样值控制参数的取值指示为“否”时，帧内预测单元304将使用默认值设置不存在的相邻样值。解析单元301在有效范围可以是整个视频的参数集中获得序列层样值控制参数。可选择地，当序列层样值控制参数的取值指示为“是”时，解析单元301可以在有效范围可以是整个视频的参数集中获得第一参考样值的样值取值，该样值取值可用于整个视频中使用帧内预测模式的解码块。
图像层样值控制参数用于指示是否需要对一个图像使用第一参考样值确定使用帧内预测模式对解码块进行解码过程中的不存在的相邻样值。当图像层样值控制参数的取值指示为“是”时，帧内预测单元304在解码块的部分或全部相邻样值不存在的情况下，使用第一参考样值设置解码块的相邻样值；反之，当图像层样值控制参数的取值指示为“否”时，帧内预测单元304使用默认值设置解码块的相邻样值。解析单元301在有效范围可以是一个图像的参数集中获得图像层样值控制参数。可选择地，当图像层样值控制参数的取值指示为“是”时，解析单元301可以在有效范围可以是一个图像的参数集中获得第一参 考样值的样值取值，该样值取值可用于一个图像中使用帧内预测模式的解码块。
分片层样值控制参数用于指示是否需要对一个分片使用第一参考样值确定使用帧内预测模式对解码块进行解码过程中不存在的相邻样值。当分片层样值控制参数的取值指示为“是”时，帧内预测单元304在解码块的部分或全部相邻样值不存在的情况下，使用解码块的第一参考样值设置相邻样值；反之，当分片层样值控制参数的取值指示为“否”时，帧内预测单元304使用默认值设置解码块的相邻样值。解析单元301在分片头中获得分片层样值控制参数。可选择地，当分片层样值控制参数的取值指示为“是”时，解析单元301可以在分片头中获得第一参考样值的样值取值，该样值取值可用于一个分片中使用帧内预测模式的解码块。
块层样值控制参数用于指示是否需要对一个解码块使用第一参考样值确定使用帧内预测模式对解码块进行解码过程中的不存在的相邻样值。当块层样值控制参数的取值指示为“是”时，帧内预测单元304在解码块的部分或全部相邻样值不存在的情况下，使用第一参考样值设置解码块的相邻样值；反之，当块层样值控制参数的取值指示为“否”时，帧内预测单元304使用默认值设置解码块的相邻样值。解析单元301在解码块的数据单元中获得块层样值控制参数。可选择地，当块层样值控制参数的取值指示为“是”时，解析单元301可以在解码块的数据单元中获得第一参考样值的样值取值，该样值取值可用于一个解码块的数据单元中使用帧内预测模式的解码块或该解码块的划分。
可选择地，解码器可以将块层样值控制参数以及对应的第一参考样值有效范围配置为一个分片。一种实施方法是，对于分片中第一个使用帧内预测模式、并且使用第一参考样值构造解码块的相邻样值时，解析单元301在该解码块的数据单元中获得块层样值控制参数以及第一参考样值。解码器对分片中后续使用帧内预测模式的解码块使用相同的解码块层样值控制参数、第一参考样值。
可选择地，解码器可以将块层样值控制参数以及对应的第一参考样值有效范围配置为一个分片。一种实施方法是，对于分片中第一个使用帧内预测模式、并且使用第一参考样值构造解码块的相邻样值时，解析单元301在该解码块的数据单元中获得块层样值控制参数(记为“控制参数1”)以及第一参考样值(记为“样值1”)。如果分片中后续使用帧内预测模式的解码块数据单元中没有新的第一参考样值，解析单元301将解码块的第一参考样值设置为控制参数1和样值1，并作为数据流62传递给相邻样值处理单元602。如果分片中后续使用帧内预测模式的解码块数据单元中存在新的第一参考样值(记为“控制参数2”、“样值2”)，解析单元301获得新的第一参考样值，并作为数据流62传递给相邻样值处理单元602。
上述解码器将块层样值控制参数以及对应的第一参考样值有效范围配置为一个分片的处理过程可以归纳为解析单元301始终使用从分片包含的解码块中最后一次获得的第一参考样值设置数据流62。
特殊地，参见图10，本公开的实施例提出了一种。如果解码器将图10中相邻样值处 理单元602配置为以解码块大小来作为是否使用第一参考样值构造解码块的帧内预测相邻样值的判断条件，则解析单元301可不解析解码块的码流而直接将块大小符合“不使用”条件的解码块的块层样值控制参数设置为“否”，在解码器将块大小符合“可使用”条件的解码块配置为“必然使用”时，解析单元301可不需要解析解码块的码流而直接将块大小符合“必然使用”条件的解码块的块层样值控制参数设置为“是”，解析单元301解析解码块的码流获得块层第一参考样值的取值。这里的“配置”指的是解码器根据参数集指示信息或预设规则对解码过程进行的配置。
特殊地，如果解码器将图7中相邻样值处理单元602配置为以解码块的帧内预测模式来作为是否使用第一参考样值构造解码块的帧内预测相邻样值的判断条件，则解析单元301不解析帧内预测模式符合“不使用”条件的解码块的数据单元，将解码块层样值控制参数设置为“否”。如果解码器将帧内预测模式符合“可使用”条件的解码块配置为“必然使用”时，解析单元301不解析帧内预测模式符合“必然使用”条件的解码块数据单元，将解码块的块层样值控制参数设置为“是”，解析单元301解析解码块的码流获得块层第一参考样值的取值。这里的“配置”，指的是解码器根据参数集指示信息或预设规则对解码过程进行的配置。
特殊地，解码器可以组合使用上述两种特殊情况。
需要说明的是，解析单元301执行步骤701完成的是对块层样值控制参数和第一参考样值的取值的解码。如果解析单元301通过解析码流获得了序列层样值控制参数、图像层样值控制参数、分片层样值控制参数中的一种或多种，需要说明的是，从控制参数的有效控制范围来说，序列层的范围大于图像层，图像层的范围大于分片层，分片层的范围大于块层。通常情况下，控制参数的控制机制是：控制范围大的控制参数指示“可以使用”时，解析控制范围小的控制参数获得用于在这个较小的控制范围内指示“是否可以使用”的信息。特别地，对于块层滤波控制参数，如果一个解码块划分成了多个子块，那么这个解码块的块层滤波控制参数的控制范围大于子块，即当这个解码块的块层滤波控制参数指示“不使用”的时候，子块不使用自适应滤波，解析单元301不再需要解析子块的数据单元以获得子块的块层滤波控制参数。
需要说明的是，在控制参数指示“可以使用”时，解析单元301可以为解析或推导该控制参数对应的第一参考样值的取值。通常情况下，有效控制范围小的控制参数对应的第一参考样值的取值覆盖控制范围更大的控制参数对应的第一参考样值的取值。例如，如果解析单元301获得了解码块的块层第一参考样值，那么解码器为该解码块在确定帧内预测模式的相邻样值过程中，使用的是块层第一参考样值，而不是分片层第一参考样值。
步骤702，解析单元301解析码流，获得解码块的帧内预测模式。
情况1：解码器仅使用帧内预测单元304的图8中帧内预测方法1。
解析单元301首先根据解码块相邻已解码块的帧内预测模式，推导解码块可能使用的一种或多种最可能使用的帧内预测模式。
解析单元301解析标志位，标志位的取值指示“解码块使用的帧内预测模式是否是最可能使用的帧内预测模式中的模式”。若“是”，如果最可能使用的帧内预测模式中只包含一种帧内预测模式，解析单元301将该帧内预测模式设置为解码块的帧内预测模式；如果最可能使用的帧内预测模式中只包含多种帧内预测模式，解析单元301解析索引序号，将最可能使用的帧内预测模式中对应于该索引序号的帧内预测模式设置为解码块的帧内预测模式。反之，若标志位的取值指示为“否”，解析单元301解析索引序号，将解码块使用的帧内预测模式设置为在除最可能使用的帧内预测模式中的帧内预测模式外的其他帧内预测模式中该索引序号对应的帧内预测模式。
情况2：解码器使用帧内预测单元304的图8中帧内预测方法1和帧内预测方法2。
解析单元301解析用于指示解码块的帧内预测模式属于帧内预测方法1还是帧内预测方法2的标识信息。一种可选择的使用方法是，解析单元301解析一个指示上述信息的标志位。一种可选择使用的方法是，解析单元301解析解码块的帧内预测标记为“否”且解码块的参考图像索引指示“解码块使用的参考图像是解码块所在图像”时，将解码块的帧内预测模式设置为帧内预测方法2。可选择地，解析单元301可进一步解析码流中与帧内预测方法2相关的字段，确定解码块使用的帧内预测模式，帧内预测方法2中可包括的帧内预测模式有块匹配模式、串匹配模式等，解析单元301继续解析码流，获得确定匹配块、匹配串相关的参数用于确定匹配块、匹配串在解码块所在图像中的位置。例如，解析单元301解析码流获得同一个图像坐标系下解码块与图8中“Ref”块之间的位置偏移量。
步骤703，解析单元301解析解码块的帧内预测模式及滤波参数的过程结束。
前述解码器将第一参考样值设置为帧内预测单元304中相邻样值处理单元602的输入数据流62。前述解码器将帧内预测模式是作为帧内预测单元304中使用的数据流64。
实施例十一
参见图13，本公开的实施例提出了一种包含编码器的编码设备。编码设备包括采集单元801、编码器802和发送单元803。其中：
采集单元801，设置为采集视频或图像。
实际中，采集单元801可以是包含至少一个摄像头用于采集自然视频或自然图像；可选择地，采集单元801还可以配置用于采集深度视频或深度图像的摄像头；可选择地，采集单元还可以配置红外摄像头；可选择地，采集单元还可以配置遥感摄像头。采集单元801可以是包含通过放射线透射或扫描产生视频或图像的装置或设备。
可选择地，采集单元801中可以对输入视频或图像进行前处理，例如，自动聚焦、自动白平衡、自动曝光、背光补偿、降噪、锐化、拼接、提升或降低图像分辨率、提升或降低视频帧率、虚拟视图合成等。
采集单元801也可以接收其它设备或单元输出的视频或图像，例如，采集单元801可以是转码器中的一个组成单元，转码器将部分解码的图像输入采集单元801。例如，采 集单元801接收通过数据连接从其他设备传递过来的视频或图像。
需要说明的是，除视频或图像外，采集单元801也可以采集其他媒体信息，例如音频。采集单元801也可以接收人工产生的信息，例如文字、字幕、计算机产生的图片或视频等。
编码器802，设置为接收采集单元801输出的视频或图像，并对视频或图像进行编码，输出视频或图像码流。
存储或发送单元803，设置为接收编码器802输出的视频或图像码流，对视频或图像码流进行系统层处理。
示例性而言，存储或发送单元803对视频或图像码流进行系统层处理包括：按照传输协议、媒体文件格式等标准进行打包。存储或发送单元803将系统层处理后的得到的传输流或媒体文件存储至电子设备的存储器中或者通过有线、无线网络进行发送。
需要说明的是，除编码器802输出的视频或图像码流外，存储或发送单元803的输入还可以包括音频码流、文字、字幕、图片等。存储或发送单元803按照媒体文件格式、传输协议等标准将这些输入与编码器802输出的码流打包在传输流或媒体文件中。
本实施例的电子设备可以是视频通信应用中能够生成或处理视频或图像码流的设备，例如，手机、计算机、媒体服务器、便携式移动终端、数字摄像机、数字照相机、电视广播系统设备、内容分发网络设备、监控摄像头、会议电视系统设备等。
实施例十二
参见图14，本公开的实施例提出了一种包含解码器的结构示意图。解码设备包括接收单元901、解码器902和呈现单元903。其中：
接收单元901，设置为接收视频或图像码流。
示例性而言，接收单元901从有线、无线网络接收视频或图像码流或者读取电子设备存储器获得视频或图像码流或者接收通过数据连接从其他设备传递过来的视频或图像码流。
接收单元901的输入还可以是包含视频或图像码流的传输流或媒体文件。接收单元901根据传输协议、媒体文件格式等标准从接收到的传输流或媒体文件中提取视频或图像码流。
需要说明的是，除视频或图像码流外，接收单元901的输出还可以包括音频码流、文字、字幕、图片等。接收单元901将这些输出传递给电子设备中对应的处理单元。例如，接收单元901将音频码流输出给电子设备中包含的音频解码器。
解码器902，设置为对接收的接收单元901输出的视频或图像码流进行解码，输出解码恢复的视频或图像。
呈现单元903，设置为接收解码器902输出的解码恢复的视频或图像进行呈现操作。
需要说明的是，呈现单元903可以是电子设备的一个组成部分，例如显示屏；也可以是通过数据连接与电子设备连接的独立设备，例如投影机、显示器等。可选择地，呈现单 元903可以对解码恢复的视频或图像进行后处理，例如自动调焦、自动白平衡、自动曝光调整、背光补偿、降噪、锐化、拼接、提升或降低图像分辨率、提升或降低视频帧率、虚拟视图合成等。
需要说明的是，除解码恢复的视频或图像外，呈现单元903的输入还可以包括来自电子设备其他单元输出的媒体数据，例如音频、文字、字幕、图片，等。呈现单元903的输入还包括人工产生的数据，例如远程教育应用中本地授课者对重点内容的划线等标注数据。呈现单元903将输入的媒体数据进行叠合后显示给观看者。
本实施例的电子设备可以是视频通信应用中能够解码或处理视频或图像码流的设备，例如，手机、计算机、机顶盒、电视机、播放器、媒体服务器、便携式移动终端、数字摄像机、数字照相机、电视广播系统设备、内容分发网络设备、会议电视系统设备等。
实施例十三
参见图15，本公开的实施例提出了一种包含编码设备和解码设备的电子系统。
信源设备1001包括图13所示的编码设备。
存储或传输网络1002，可以包括设备或电子系统的存储器、通过数据连接进行数据读写操作的外部存储器；也可以包括有线网络、无线网络组成的数据传输网络。存储或传输网络1002为信源设备1001中存储或发送单元803提供了存储器或数据传输网络。
信宿设备1003包括图14所示的解码设备。信宿设备1003中接收单元901接收存储或传输网络1002提供视频或图像码流、包含视频或图像码流的传输流或者包含视频或图像码流的媒体文件。
实施例十四
本公开一个实施例提出了一种编码器，包括：第一处理器、第一存储介质、第一通信总线，其中，第一处理器和第一存储介质通过第一通信总线连接；
第一处理器，调用第一存储介质存储的图像编码相关程序，并执行如下步骤：
根据编码块中的像素样值，确定编码块的第一参考样值；
根据第一参考样值，确定编码块的帧内预测模式，并计算编码块的帧内预测值；
根据编码块的原始值与帧内预测值，获得预测差参数；
对第一参考样值、帧内预测模式和预测差参数进行编码，将编码比特写入码流。
实施例十五
本公开一个实施例提出了一种解码器，包括：第二处理器、第二存储介质、第二通信总线，其中，第二处理器和第二存储介质通过第二通信总线连接；
第二处理器，调用第二存储介质存储的图像解码相关程序，并执行如下步骤：
解析码流，获得解码块的第一参考样值、帧内预测模式和预测差参数；
根据第一参考样值确定解码块的帧内预测参考样值；
根据帧内预测参考样值以及帧内预测模式构造解码块的帧内预测值；
使用预测差参数计算解码块的预测差；
计算帧内预测值和预测差的和值，获得解码块的恢复值。
另一方面，本公开一个实施例提出了一种计算机存储介质，应用于编码器中，计算机可读存储介质存储有一个或多个图像编码相关程序，一个或者多个图像编码相关程序可被一个或者多个第一处理器执行，以实现如上述任一一种图像编码的方法。
再一方面，本公开一个实施例提出了一种计算机存储介质，应用于解码器中，计算机可读存储介质存储有一个或多个图像解码相关程序，一个或者多个图像解码相关程序可被一个或者多个第二处理器执行，以实现如上述任一一种图像解码方法。
本实施例的电子系统可以是视频通信应用中能够生成、存储或传输和解码视频或图像码流的系统或设备，例如，手机、计算机、IPTV系统、OTT系统、互联网多媒体系统、数字电视广播系统、监控系统、便携式移动终端、数字摄像机、数字照相机、会议电视系统设备等。
本公开还可有其他多种实施例，在不背离本公开精神及其实质的情况下，熟悉本领域的技术人员当可根据本公开作出各种相应的改变和变形，但这些相应的改变和变形都属于本公开所附的权利要求的保护范围。
本领域内的技术人员应明白，本公开的实施例可提供为方法、或计算机程序产品。因此，本公开可采用硬件实施例、软件实施例、或结合软件和硬件方面的实施例的形式。而且，本公开可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器和光学存储器等)上实施的计算机程序产品的形式。
本公开是参照根据本公开实施例的方法、设备(系统)和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
工业实用性
本公开实施例的技术方案中，由于第一参考样值是根据编码块的待编码像素样值计算确定的，并且第一样值将进一步用于构造编码块的帧内预测参考样值。因此，在编码块相 邻块的样值均不可用或者部分不可用的情况下，有效降低了帧内预测的预测值与编码块之间的预测差值，提高了帧内预测模式的编码效率。

Claims (64)

  1. 一种图像编码方法，其中，包括：
    根据编码块中的像素样值，确定所述编码块的第一参考样值；
    根据所述第一参考样值，确定所述编码块的帧内预测模式，并计算所述编码块的帧内预测值；
    根据所述编码块的原始值与所述帧内预测值，获得预测差参数；
    对所述第一参考样值、所述帧内预测模式和所述预测差参数进行编码，将编码比特写入码流。
  2. 根据权利要求1所述的方法，其中，所述根据编码块的原始值与帧内预测值，获得预测差参数包括：
    计算所述编码块的原始值与所述帧内预测值的差值，得到所述编码块的预测差值；
    对所述预测差值进行变换和量化处理，得到所述预测差参数。
  3. 根据权利要求1所述的方法，其中，所述根据编码块中的像素样值确定编码块的第一参考样值，包括：
    判断所述编码块的参考样值是否存在，所述参考样值为所述编码块相邻的一个或多个像素点的已恢复像素样值；
    若所述编码块的部分或全部参考样值不存在，则根据所述编码块中的像素样值确定所述编码块的第一参考样值；
    或者，判断所述编码块的匹配块中的参考样值是否均存在；
    若所述编码块的匹配块中的部分或全部参考样值不存在，则判断为所述编码块的部分或全部参考样值不存在，根据所述编码块中的像素样值确定所述编码块的第一参考样值。
  4. 根据权利要求3所述的方法，其中，所述编码块的匹配块位于所述编码块所在图像中并且与所述编码块大小相同的区域。
  5. 根据权利要求3所述的方法，其中，所述判断所述参考样值是否存在的方法，包括以下至少一种：
    若所述参考样值对应的参考像素点位置位于所述编码块所在图像中尚未编码的图像区域，则将所述参考样值判断为不存在；
    若所述参考样值对应的参考像素点位置位于所述编码块所在图像的边界之外，则将所述参考样值判断为不存在；
    在所述编码块所在图像中，若所述参考样值对应的参考像素点位置位于所述编码块所在分片之外，则将所述参考样值判断为不存在；
    在所述编码块所在图像中，若所述参考样值对应的参考像素点位置位于所述编码块所在瓦片之外，则将所述参考样值判断为不存在；
    若使用帧内预测受限模式，在所述编码块所在图像中，若所述参考样值对应的参考像素点位置位于非帧内预测模式的编码块中，则将所述参考样值判断为不存在。
  6. 根据权利要求3所述的方法，其中，所述根据编码块中的像素样值确定所述编码块的第一参考样值，包括：
    根据所述编码块中的像素样值，使用软判决方法确定所述第一参考样值。
  7. 根据权利要求6所述的方法，其中，所述根据编码块中的像素样值，使用软判决方法确定所述第一参考样值包括：
    使用约束最优化方法计算所述第一参考样值，所述约束最优化计算公式为：
    min D＝∑ kd(x k,p k)s.t.R≤R C
    其中，D是所述编码块的编码失真；k是遍历范围，包括所述编码块中的一个或多个像素；d(a,b)是a和b之间的误差值；x k是所述编码块中待编码像素样值；p k是使用第一参考样值根据帧内预测模式计算得到的x k的预测值；R是所述编码块的编码比特开销；R C是所述编码块编码比特开销的上限值。
  8. 根据权利要求6所述的方法，其中，所述根据编码块中的像素样值，使用软判决方法确定所述第一参考样值包括：
    使用无约束最优化方法计算所述第一参考样值，所述无约束最优化计算公式为：
    (r,m) opt＝arg min(D+λ·R)
    其中，r是所述第一参考样值；m是所述编码块的帧内预测模式，D是所述编码块的编码失真，R是所述编码块的编码比特开销，λ是拉格朗日因子。
  9. 根据权利要求3所述的方法，其中，所述根据所述编码块中的像素样值确定所述编码块的第一参考样值包括：
    若判断所述编码块的全部参考样值不存在，则将所述第一参考样值设置为所述编码块原始值的平均值；
    若判断所述编码块的部分参考样值不存在，则将所述第一参考样值设置为所述编码块原始值的平均值与所述部分存在的参考样值均值的差值。
  10. 根据权利要求3所述的方法，其中，所述第一参考样值包括至少两个样值，以及使用方式指示参数，其中，所述样值使用方式参数用于指示所述第一参考样值中包含的至少两个样值在设置所述编码块的参考样值过程中的使用方式。
  11. 根据权利要求10所述的方法，其中，所述第一参考样值包括两个样值，所述根据编码块中的像素样值确定所述编码块的第一参考样值，包括：
    根据所述两个样值分别设置所述编码块左相邻参考像素点位置中不存在的参考样值的取值，和上相邻参考像素点位置中不存在的参考样值的取值。
  12. 根据权利要求1所述的方法，其中，根据所述第一参考样值，确定所述编码块的帧内预测模式，计算所述编码块的帧内预测值，包括：
    使用所述第一参考样值设置所述编码块的不存在的参考样值；
    根据所述编码块的参考样值，使用率失真优化方法确定所述编码块的帧内预测模式，使用所述帧内预测模式计算得到所述编码块的帧内预测值。
  13. 根据权利要求1所述的方法，其中，根据所述第一参考样值，确定所述编码块的帧内预测模式，计算所述编码块的帧内预测值，还包括：
    使用所述第一参考样值设置所述编码块的不存在的参考样值；
    对所述编码块的参考样值进行滤波，得到经滤波处理的参考样值；
    根据所述经滤波处理的参考样值，使用率失真优化方法确定所述编码块的帧内预测模式，使用所述帧内预测模式计算得到所述编码块的帧内预测值。
  14. 根据权利要求13所述的方法，其中，所述对所述编码块的参考样值进行滤波，得到经滤波处理的参考样值，包括：
    使用率失真优化方法确定滤波控制参数，所述滤波控制参数用于指示是否对所述参考样值进行滤波；
    若所述滤波控制参数指示对所述参考样值进行滤波，则对所述编码块的参考样值进行滤波，得到所述经滤波处理的参考样值。
  15. 根据权利要求13所述的方法，其中，所述对所述编码块的参考样值进行滤波，得到所述经滤波处理的参考样值，包括：
    若所述编码块的大小等于预设值时，则使用率失真优化方法确定滤波控制参数，所述滤波控制参数用于指示是否对所述参考样值进行滤波；
    当所述滤波控制参数指示对所述参考样值进行滤波时，对所述编码块的参考样值进行滤波，得到所述经滤波处理的参考样值。
  16. 根据权利要求13所述的方法，其中，所述对所述编码块的参考样值进行滤波，得到所述经滤波处理的参考样值，包括：
    若所述编码块的大小等于预设值时，对所述编码块的参考样值进行滤波，得到所述经滤波处理的参考样值。
  17. 根据权利要求13所述的方法，其中，对所述编码块的参考样值进行滤波，得到所述经滤波处理的参考样值，包括：
    当所述编码块的帧内预测模式是预设的模式时，使用率失真优化方法确定滤波控制参数，所述滤波控制参数用于指示是否对所述参考样值进行滤波；
    当所述滤波控制参数指示对所述参考样值进行滤波时，对所述编码块的参考样值进行滤波，得到所述经滤波处理的参考样值。
  18. 根据权利要求13所述的方法，其中，对所述编码块的参考样值进行滤波，得到所述经滤波处理的参考样值，包括：
    当所述编码块的帧内预测模式是预设的模式时，对所述编码块的参考样值进行滤波，得到所述经滤波处理的参考样值。
  19. 根据权利要求13所述的方法，其中，对所述编码块的参考样值进行滤波，得到所述经滤波处理的参考样值，包括：
    当所述编码块的大小等于预设值时，并且所述编码块的帧内预测模式是预设的模式 时，使用率失真优化方法确定滤波控制参数，所述滤波控制参数用于指示是否对所述参考样值进行滤波；
    当所述滤波控制参数指示对所述参考样值进行滤波时，对所述编码块的参考样值进行滤波，得到所述经滤波处理的参考样值。
  20. 根据权利要求13所述的方法，其中，对所述编码块的参考样值进行滤波，得到所述经滤波处理的参考样值，包括：
    当所述编码块的大小等于预设值时、并且所述编码块的帧内预测模式是预设的模式时，对所述编码块参考样值进行滤波，得到所述经滤波处理的参考样值。
  21. 根据权利要求14、15、17或19所述的方法，其中，对所述编码块的参考样值进行滤波，得到所述经滤波处理的参考样值，方法还包括：
    对所述编码块的滤波控制参数进行编码，将编码比特写入所述码流。
  22. 根据权利要求1所述的方法，其中，所述对所述第一参考样值进行编码，包括：
    通过对所述第一参考样值进行编码获得所述第一参考样值的编码比特；
    将所述编码比特写入所述码流中的数据单元，其中，所述码流中的数据单元包含以下至少之一：一个或多个参数集，分片头，块层数据单元。
  23. 根据权利要求1所述的方法，其中，所述对所述第一参考样值进行编码，包括：
    所述第一参考样值包括一个或多个样值；
    对所述第一参考样值的一个或多个样值进行编码，获得所述第一参考样值的一个或多个样值的编码比特，将所述编码比特写入所述码流中的数据单元。
  24. 根据权利要求23所述的方法，其中，所述对所述第一参考样值的一个或多个样值进行编码，获得所述第一参考样值的一个或多个样值的编码比特，还包括：
    编码样值使用方式参数，将所述样值使用方式参数的编码比特写入所述码流中的数据单元。
  25. 根据权利要求1所述的方法，其中，所述第一参考样值包括样值控制参数，所述样值控制参数包括以下至少之一的控制参数：序列层样值控制参数，图像层样值控制参数，分片层样值控制参数，块层样值控制参数。
  26. 一种图像解码方法，其中，包括：
    解析码流，获得解码块的第一参考样值、帧内预测模式和预测差参数；
    根据所述第一参考样值确定所述解码块的帧内预测参考样值；
    根据所述帧内预测参考样值以及所述帧内预测模式构造所述解码块的帧内预测值；
    使用预测差参数计算所述解码块的预测差；
    计算所述帧内预测值和所述预测差的和值，获得所述解码块的恢复值。
  27. 根据权利要求26所述的方法，其中，所述根据所述第一参考样值确定所述解码块的帧内预测参考样值，包括：
    判断所述解码块的参考样值是否存在，所述参考样值为所述解码块相邻的一个或多个 像素点的已恢复像素样值；
    若所述解码块的部分或全部参考样值不存在，则根据所述第一参考样值确定所述解码块的不存在的参考样值，获得帧内预测参考样值。
    或者，判断所述解码块的匹配块中的参考样值是否均存在；
    若所述解码块的匹配块中的部分或全部参考样值不存在，则判断为所述解码块的部分或全部参考样值不存在，根据所述第一参考样值确定所述解码块的不存在的参考样值，获得帧内预测参考样值。
  28. 根据权利要求27所述的方法，其中，所述解码块的匹配块位于所述解码块所在图像中并且与所述解码块大小相同的区域。
  29. 根据权利要求27所述的方法，其中，所述判断所述参考样值是否存在的方法，包括以下至少一种：
    若所述参考样值对应的参考像素点位置位于所述解码块所在图像中尚未解码的图像区域，则将所述参考样值判断为不存在；
    若参考样值对应的参考像素点位置位于所述解码块所在图像的边界之外，则将所述参考样值判断为不存在；
    在所述解码块所在图像中，若所述参考样值对应的参考像素点位置位于所述解码块所在分片之外，则将所述参考样值判断为不存在；
    在所述解码块所在图像中，若所述参考样值对应的参考像素点位置位于所述解码块所在瓦片之外，则将所述参考样值判断为不存在；
    若使用帧内预测受限模式，在所述解码块所在图像中，若所述参考样值对应的参考像素点位置位于非帧内预测模式的解码块中，则将所述参考样值判断为不存在。
  30. 根据权利要求26所述的方法，其中，所述解析码流获得解码块的第一参考样值，包括：
    解析所述码流中的一个或多个数据单元，获得所述第一参考样值，其中，所述数据单元包括以下至少之一：一个或多个参数集，分片头，块层数据单元。
  31. 根据权利要求26所述的方法，其中，所述解析码流获得解码块的第一参考样值，包括：
    解析所述码流中的一个或多个数据单元，获得样值使用方式参数，其中，所述样值使用方式参数用于指示所述第一参考样值中包含的样值在设置所述解码块的参考样值过程中的使用方式。
  32. 根据权利要求26所述的方法，其中，所述第一参考样值包括样值控制参数，所述样值控制参数包括以下至少之一的控制参数：序列层样值控制参数，图像层样值控制参数，分片层样值控制参数，块层样值控制参数。
  33. 根据权利要求27的方法，其中，所述根据第一参考样值确定所述解码块的不存在的参考样值，包括：
    使用所述第一参考样值设置所述解码块的不存在的参考样值。
  34. 根据权利要求27所述的方法，其中，所述第一参考样值中包含样值控制参数，所述根据第一参考样值确定所述解码块的不存在的参考样值，包括：
    若所述样值控制参数指示使用所述第一参考样值构造所述不存在的参考样值，则使用所述第一参考样值设置所述解码块的不存在的参考样值。
  35. 根据权利要求27所述的方法，其中，所述第一参考样值中包含样值使用方式参数，所述根据第一参考样值确定所述解码块的不存在的参考样值，包括：
    若所述第一参考样值包含至少两个样值，则根据所述样值使用方式参数，使用所述第一参考样值中包含的样值对应设置所述解码块的不存在的参考样值。
  36. 根据权利要求35所述的方法，其中，所述第一参考样值包括两个样值，所述根据所述样值使用方式参数，使用所述第一参考样值中包含的样值对应设置所述解码块的不存在的参考样值，包括：
    按照预设规则使用所述第一参考样值中的两个样值分别设置所述解码块左相邻参考像素点位置中不存在的参考样值，和上相邻参考像素点位置中不存在的参考样值。
  37. 根据权利要求26所述的方法，其中，根据所述帧内预测参考样值以及所述帧内预测模式构造所述解码块的帧内预测值，方法还包括：
    对所述帧内预测参考样值进行滤波处理，得到经过滤波处理的参考样值；
    使用所述经滤波处理的参考样值，按所述帧内预测模式计算得到所述解码块的帧内预测值。
  38. 根据权利要求37所述的方法，其中，对所述帧内预测参考样值进行滤波处理得到经过滤波处理的参考样值，包括：
    解析所述码流，获得滤波控制参数，其中，所述滤波控制参数用于指示是否对所述帧内预测参考样值进行滤波；
    若所述滤波控制参数指示对所述帧内预测参考样值进行滤波，则对所述帧内预测参考样值进行滤波处理，得到所述经过滤波处理的参考样值。
  39. 根据权利要求37所述的方法，其中，对所述帧内预测参考样值进行滤波处理得到经过滤波处理的参考样值，包括：
    若所述解码块的大小等于预设值，则解析所述码流，获得滤波控制参数；
    当所述滤波控制参数指示对所述帧内预测参考样值进行滤波时，对所述帧内预测参考样值进行滤波处理，得到所述经过滤波处理的参考样值。
  40. 根据权利要求37所述的方法，其中，对所述帧内预测参考样值进行滤波处理得到经过滤波处理的参考样值，包括：
    若所述解码块的大小等于预设值时，对所述帧内预测参考样值进行滤波处理，得到所述经过滤波处理的参考样值。
  41. 根据权利要求37所述的方法，其中，对所述帧内预测参考样值进行滤波处理得 到经过滤波处理的参考样值，包括：
    若所述解码块的帧内预测模式是预设的模式，则解析所述码流，获得滤波控制参数；
    当所述滤波控制参数指示对所述帧内预测参考样值进行滤波时，对所述帧内预测参考样值进行滤波处理，得到所述经过滤波处理的参考样值。
  42. 根据权利要求37所述的方法，其中，对所述帧内预测参考样值进行滤波处理得到经过滤波处理的参考样值，包括：
    若所述解码块的帧内预测模式是预设的模式，则对所述帧内预测参考样值进行滤波处理，得到所述经过滤波处理的参考样值。
  43. 根据权利要求37所述的方法，其中，对所述帧内预测参考样值进行滤波处理得到经过滤波处理的参考样值，包括：
    若所述解码块的大小等于预设值，并且所述解码块的帧内预测模式是预设的模式，则解析所述码流，获得滤波控制参数；
    当所述滤波控制参数指示对所述帧内预测参考样值进行滤波时，对所述帧内预测参考样值进行滤波处理，得到所述经过滤波处理的参考样值。
  44. 根据权利要求37所述的方法，其中，对所述帧内预测参考样值进行滤波处理得到经过滤波处理的参考样值，包括：
    若所述解码块的大小等于预设值，并且所述解码块的帧内预测模式是预设的模式时，对所述帧内预测参考样值进行滤处理，得到所述经过滤波处理的参考样值。
  45. 根据权利要求26所述的方法，其中，所述使用预测差参数计算所述解码块的预测差，包括：
    对所述预测差参数进行伸缩和变换处理，得到所述解码块的预测差。
  46. 根据权利要求26所述的方法，其中，在所述将所述帧内预测值和所述预测差的和值设置为所述解码块的恢复值之后，方法还包括：
    对所述解码块的恢复值进行环路滤波处理得到解码值。
  47. 一种图像编码装置，其中，所述图像编码装置包括第一参考样值确定单元、帧内预测值确定单元、预测差参数获得单元和编码单元，其中：
    所述第一参考样值确定单元，设置为根据编码块中的像素样值，确定所述编码块的第一参考样值；
    所述帧内预测值确定单元，设置为根据所述第一参考样值确定单元确定的所述第一参考样值，确定所述编码块的帧内预测模式，并计算所述编码块的帧内预测值；
    所述预测差参数获得单元，设置为根据所述编码块的原始值与所述帧内预测值确定单元计算的所述帧内预测值，获得预测差参数；
    所述编码单元，设置为对所述第一参考样值确定单元确定的所述第一参考样值、所述帧内预测值确定单元确定的所述帧内预测模式和所述预测差参数获得单元获得的所述预测差参数进行编码，将编码比特写入码流。
  48. 根据权利要求47所述的装置，其中，所述预测差参数获得单元设置为：
    计算所述编码块的原始值与所述帧内预测值的差值，得到所述编码块的预测差值；
    对所述预测差值进行变换和量化处理，得到所述预测差参数。
  49. 根据权利要求47所述的装置，其中，所述第一参考样值确定单元设置为：
    判断所述编码块的参考样值是否存在，所述参考样值为所述编码块相邻的一个或多个像素点的已恢复像素样值；
    若所述编码块的部分或全部参考样值不存在，则根据所述编码块中的像素样值确定所述编码块的第一参考样值；
    或者，判断所述编码块的匹配块中的参考样值是否均存在；
    若所述编码块的匹配块中的部分或全部参考样值不存在，则判断为所述编码块的部分或全部参考样值不存在，根据所述编码块中的像素样值确定所述编码块的第一参考样值。
  50. 根据权利要求49所述的装置，其中，所述第一参考样值确定单元设置为：
    根据所述编码块中的像素样值，使用软判决方法确定所述第一参考样值。
  51. 根据权利要求49所述的装置，其中，所述第一参考样值确定单元设置为：
    若判断所述编码块的全部参考样值不存在，则将所述第一参考样值设置为所述编码块原始值的平均值；
    若判断所述编码块的部分参考样值不存在，则将所述第一参考样值设置为所述编码块原始值的平均值与所述部分存在的参考样值均值的差值。
  52. 根据权利要求47所述的装置，其中，所述帧内预测值确定单元设置为：
    使用所述第一参考样值设置所述编码块的不存在的参考样值；
    根据所述编码块的参考样值，使用率失真优化方法确定所述编码块的帧内预测模式，使用所述帧内预测模式计算得到所述编码块的帧内预测值。
  53. 根据权利要求47所述的装置，其中，所述帧内预测值确定单元设置为：
    使用所述第一参考样值设置所述编码块的不存在的参考样值；
    对所述编码块的参考样值进行滤波，得到经滤波处理的参考样值；
    根据所述经滤波处理的参考样值，使用率失真优化方法确定所述编码块的帧内预测模式，使用所述帧内预测模式计算得到所述编码块的帧内预测值。
  54. 根据权利要求47所述的装置，其中，所述编码单元设置为：
    通过对所述第一参考样值进行编码获得所述第一参考样值的编码比特；
    将所述编码比特写入所述码流中的数据单元，其中，所述码流中的数据单元包含以下至少之一：一个或多个参数集，分片头，块层数据单元。
  55. 一种图像解码装置，其中，所述图像解码装置包括解析单元、帧内预测参考样值确定单元、帧内预测值确定单元、预测差确定单元和恢复值获得单元，其中：
    所述解析单元，设置为解析码流获得解码块的第一参考样值、帧内预测模式和预测差参数；
    所述帧内预测参考样值确定单元，设置为根据所述解析单元解析的所述第一参考样值确定所述解码块的帧内预测参考样值；
    所述帧内预测值确定单元，设置为根据所述帧内预测参考样值确定单元确定的所述帧内预测参考样值以及所述解析单元解析的所述帧内预测模式构造所述解码块的帧内预测值；
    所述预测差确定单元，设置为使用所述解析单元解析的所述预测差参数计算所述解码块的预测差；
    所述恢复值获得单元，设置为计算所述帧内预测值确定单元构造的所述帧内预测值，和所述预测差确定单元计算的所述预测差的和值，获得所述解码块的恢复值。
  56. 根据权利要求55所述的装置，其中，所述解析单元设置为：
    解析所述码流中的一个或多个数据单元，获得所述第一参考样值，其中，所述数据单元包括以下至少之一：一个或多个参数集，分片头，块层数据单元。
  57. 根据权利要求55所述的装置，其中，所述帧内预测值确定单元设置为：
    对所述帧内预测参考样值进行滤波处理，得到经过滤波处理的参考样值；
    使用所述经滤波处理的参考样值，按所述帧内预测模式计算得到所述解码块的帧内预测值。
  58. 根据权利要求55所述的装置，其中，所述帧内预测参考样值确定单元设置为：
    判断所述解码块的参考样值是否存在，所述参考样值为所述解码块相邻的一个或多个像素点的已恢复像素样值；
    若所述解码块的部分或全部参考样值不存在，则根据所述第一参考样值确定所述解码块的不存在的参考样值，获得帧内预测参考样值。
    或者，判断所述解码块的匹配块中的参考样值是否均存在；
    若所述解码块的匹配块中的部分或全部参考样值不存在，则判断为所述解码块的部分或全部参考样值不存在，根据所述第一参考样值确定所述解码块的不存在的参考样值，获得帧内预测参考样值。
  59. 根据权利要求55所述的装置，其中，所述预测差确定单元设置为：
    对所述预测差参数进行伸缩和变换处理，得到所述解码块的预测差。
  60. 根据权利要求55所述的装置，其中，所述恢复值获得单元设置为：
    对所述解码块的恢复值进行环路滤波处理得到解码值。
  61. 一种编码器，其中，包括：第一处理器、第一存储介质、第一通信总线，其中，所述第一处理器和所述第一存储介质通过所述第一通信总线连接；
    第一处理器，调用第一存储介质存储的图像编码相关程序，并执行如下步骤：
    根据编码块中的像素样值，确定所述编码块的第一参考样值；
    根据所述第一参考样值，确定所述编码块的帧内预测模式，并计算所述编码块的帧内预测值；
    根据所述编码块的原始值与所述帧内预测值，获得预测差参数；
    对所述第一参考样值、所述帧内预测模式和所述预测差参数进行编码，将编码比特写入码流。
  62. 一种解码器，其中，包括：第二处理器、第二存储介质、第二通信总线，其中，所述第二处理器和所述第二存储介质通过所述第二通信总线连接；
    第二处理器，调用第二存储介质存储的图像解码相关程序，并执行如下步骤：
    解析码流，获得解码块的第一参考样值、帧内预测模式和预测差参数；
    根据所述第一参考样值确定所述解码块的帧内预测参考样值；
    根据所述帧内预测参考样值以及所述帧内预测模式构造所述解码块的帧内预测值；
    使用预测差参数计算所述解码块的预测差；
    计算所述帧内预测值和所述预测差的和值，获得所述解码块的恢复值。
  63. 一种计算机存储介质，其中，应用于编码器中，计算机可读存储介质存储有一个或多个图像编码相关程序，一个或者多个图像编码相关程序可被一个或者多个第一处理器执行，以实现如权利要求1至25所述的方法。
  64. 一种计算机存储介质，其中，应用于解码器中，计算机可读存储介质存储有一个或多个图像解码相关程序，一个或者多个图像解码相关程序可被一个或者多个第二处理器执行，以实现如权利要求26至46所述的方法。
PCT/CN2019/086046 2018-06-26 2019-05-08 一种图像编码方法、解码方法、编码器、解码器及存储介质 Ceased WO2020001170A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
KR1020217002646A KR102513756B1 (ko) 2018-06-26 2019-05-08 이미지 인코딩 방법, 디코딩 방법, 인코더, 디코더 및 저장매체
US17/255,018 US11343513B2 (en) 2018-06-26 2019-05-08 Image encoding method and decoding method, encoder, decoder, and storage medium
EP19824813.0A EP3817383A4 (en) 2018-06-26 2019-05-08 Image encoding method and decoding method, encoder, decoder, and storage medium
JP2020572841A JP7393366B2 (ja) 2018-06-26 2019-05-08 画像の符号化方法、復号化方法、エンコーダおよびデコーダ

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810671512.9 2018-06-26
CN201810671512.9A CN110650337B (zh) 2018-06-26 2018-06-26 一种图像编码方法、解码方法、编码器、解码器及存储介质

Publications (1)

Publication Number Publication Date
WO2020001170A1 true WO2020001170A1 (zh) 2020-01-02

Family

ID=68986043

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/086046 Ceased WO2020001170A1 (zh) 2018-06-26 2019-05-08 一种图像编码方法、解码方法、编码器、解码器及存储介质

Country Status (6)

Country Link
US (1) US11343513B2 (zh)
EP (1) EP3817383A4 (zh)
JP (1) JP7393366B2 (zh)
KR (1) KR102513756B1 (zh)
CN (1) CN110650337B (zh)
WO (1) WO2020001170A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2021132302A (ja) * 2020-02-19 2021-09-09 日本放送協会 画像符号化装置、画像復号装置及びこれらのプログラム
JP7514107B2 (ja) 2020-04-24 2024-07-10 日本放送協会 画像符号化装置、画像復号装置及びこれらのプログラム

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109889827B (zh) * 2019-04-11 2021-01-29 腾讯科技（深圳）有限公司 帧内预测编码方法、装置、电子设备及计算机存储介质
CN115086679B (zh) * 2022-06-08 2025-07-04 北京达佳互联信息技术有限公司 帧内预测方法、装置、电子设备及存储介质
CN116578273B (zh) * 2023-02-28 2024-02-09 中国长江三峡集团有限公司 一种编码结构设计方法、装置、存储介质及电子设备
WO2024215074A1 (ko) * 2023-04-10 2024-10-17 주식회사 케이티 영상 부호화/복호화 방법 및 비트스트림을 저장하는 기록 매체
WO2026019117A1 (ko) * 2024-07-16 2026-01-22 엘지전자 주식회사 영상 인코딩/디코딩 방법 및 장치, 그리고 비트스트림을 저장한 기록 매체

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060239349A1 (en) * 2005-04-22 2006-10-26 Renesas Technology Corp. Image coding unit and image coding method
CN103067699A (zh) * 2011-10-20 2013-04-24 中兴通讯股份有限公司 一种帧内预测模式编码器、解码器及其方法和电子设备
CN107105241A (zh) * 2010-07-31 2017-08-29 M&K控股株式会社 图像编码装置
CN108141585A (zh) * 2015-10-22 2018-06-08 Lg 电子株式会社 视频编码系统中的帧内预测方法和装置

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4722125B2 (ja) * 2005-04-01 2011-07-13 パナソニック株式会社 画像復号化装置及び画像復号化方法
CN101483780B (zh) * 2008-01-07 2011-08-24 华为技术有限公司 一种帧内dc预测的方法及装置
KR101591825B1 (ko) * 2008-03-27 2016-02-18 엘지전자 주식회사 비디오 신호의 인코딩 또는 디코딩 방법 및 장치
KR20110113561A (ko) 2010-04-09 2011-10-17 한국전자통신연구원 적응적인 필터를 이용한 인트라 예측 부호화/복호화 방법 및 그 장치
KR20140071507A (ko) * 2010-08-17 2014-06-11 엠앤케이홀딩스 주식회사 예측 블록 생성 장치
WO2012134046A2 (ko) * 2011-04-01 2012-10-04 주식회사 아이벡스피티홀딩스 동영상의 부호화 방법
CN107197250B (zh) * 2011-10-28 2019-09-03 三星电子株式会社 用于视频的帧内预测的方法和设备
US20160241882A1 (en) * 2013-10-11 2016-08-18 Sony Corporation Image processing apparatus and image processing method
EP3152906B1 (en) * 2014-07-07 2021-06-16 HFI Innovation Inc. Method of intra block copy search and compensation range
US10455249B2 (en) 2015-03-20 2019-10-22 Qualcomm Incorporated Downsampling process for linear model prediction mode
US10200713B2 (en) * 2015-05-11 2019-02-05 Qualcomm Incorporated Search region determination for inter coding within a particular picture of video data
US10194170B2 (en) * 2015-11-20 2019-01-29 Mediatek Inc. Method and apparatus for video coding using filter coefficients determined based on pixel projection phase
EP3361726A4 (en) * 2015-11-24 2018-08-15 Samsung Electronics Co., Ltd. Method and device for video decoding and method and device for video encoding
CN116962721A (zh) * 2016-05-04 2023-10-27 微软技术许可有限责任公司 利用样本值的非相邻参考线进行帧内图片预测的方法
WO2018026166A1 (ko) * 2016-08-01 2018-02-08 한국전자통신연구원 영상 부호화/복호화 방법, 장치 및 비트스트림을 저장한 기록 매체

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060239349A1 (en) * 2005-04-22 2006-10-26 Renesas Technology Corp. Image coding unit and image coding method
CN107105241A (zh) * 2010-07-31 2017-08-29 M&K控股株式会社 图像编码装置
CN103067699A (zh) * 2011-10-20 2013-04-24 中兴通讯股份有限公司 一种帧内预测模式编码器、解码器及其方法和电子设备
CN108141585A (zh) * 2015-10-22 2018-06-08 Lg 电子株式会社 视频编码系统中的帧内预测方法和装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3817383A4 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2021132302A (ja) * 2020-02-19 2021-09-09 日本放送協会 画像符号化装置、画像復号装置及びこれらのプログラム
JP7541439B2 (ja) 2020-02-19 2024-08-28 日本放送協会 画像符号化装置、画像復号装置及びこれらのプログラム
JP7514107B2 (ja) 2020-04-24 2024-07-10 日本放送協会 画像符号化装置、画像復号装置及びこれらのプログラム

Also Published As

Publication number Publication date
KR20210028212A (ko) 2021-03-11
EP3817383A1 (en) 2021-05-05
JP2021529473A (ja) 2021-10-28
CN110650337A (zh) 2020-01-03
CN110650337B (zh) 2022-04-01
EP3817383A4 (en) 2022-06-29
US20210274193A1 (en) 2021-09-02
US11343513B2 (en) 2022-05-24
JP7393366B2 (ja) 2023-12-06
KR102513756B1 (ko) 2023-03-24

Similar Documents

Publication Publication Date Title
US11343513B2 (en) Image encoding method and decoding method, encoder, decoder, and storage medium
CN110099278B (zh) 选择运动向量精度
KR102518627B1 (ko) 이미지 인코딩 방법, 디코딩방법, 인코더 및 디코더
JP7250917B2 (ja) 補間フィルタを使用するイントラ予測のための方法および装置
TW202005399A (zh) 基於區塊之自適應迴路濾波器(alf)之設計及發信令
CN113228686B (zh) 用于视频译码中去块效应滤波器的装置和方法
CN106170092A (zh) 用于无损编码的快速编码方法
CN113497937A (zh) 图像编码方法、图像解码方法及相关装置
CN113785573A (zh) 编码器、解码器和使用自适应环路滤波器的对应方法
CN112040229A (zh) 视频解码方法及视频解码器
CN111355951A (zh) 视频解码方法、装置及解码设备
CN111385572A (zh) 预测模式确定方法、装置及编码设备和解码设备
CN114913249B (zh) 编码、解码方法和相关设备
CN113330743A (zh) 编码器、解码器及去块效应滤波器自适应的对应方法
CN113455005A (zh) 用于帧内子分区译码工具所产生的子分区边界的去块效应滤波器
CN114679583A (zh) 视频编码器、视频解码器及对应方法
CN118474373A (zh) 编解码方法和装置
CN119999202A (zh) 用于基于外推滤波器的帧内预测模式的变换选择的系统和方法
CN113228632A (zh) 用于局部亮度补偿的编码器、解码器、以及对应方法
CN118235395A (zh) 应用于离线固定滤波输出的自适应环路滤波
WO2024093312A1 (zh) 图像编解码方法和装置
CN112055211A (zh) 视频编码器及qp设置方法
CN118646901B (zh) 编解码方法和装置
KR20250174382A (ko) 비선형 기하학적 분할 모드에 기반한 비디오 부호화 및 복호화를 위한 방법 및 장치

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19824813

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020572841

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 20217002646

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2019824813

Country of ref document: EP

Effective date: 20210126