WO2025217860A1 - Procédé de codage et procédé de décodage pour vidéo ou image, décodeur, codeur et support de stockage lisible - Google Patents

Procédé de codage et procédé de décodage pour vidéo ou image, décodeur, codeur et support de stockage lisible

Info

Publication number
WO2025217860A1
WO2025217860A1 PCT/CN2024/088460 CN2024088460W WO2025217860A1 WO 2025217860 A1 WO2025217860 A1 WO 2025217860A1 CN 2024088460 W CN2024088460 W CN 2024088460W WO 2025217860 A1 WO2025217860 A1 WO 2025217860A1
Authority
WO
WIPO (PCT)
Prior art keywords
reconstructed block
matching
block
reconstructed
vectors
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2024/088460
Other languages
English (en)
Chinese (zh)
Inventor
霍俊彦
马彦卓
杨付正
邱能富
李明
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Oppo Mobile Telecommunications Corp Ltd
Original Assignee
Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Oppo Mobile Telecommunications Corp Ltd filed Critical Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority to PCT/CN2024/088460 priority Critical patent/WO2025217860A1/fr
Publication of WO2025217860A1 publication Critical patent/WO2025217860A1/fr
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/11—Selection of coding mode or of prediction mode among a plurality of spatial predictive coding modes

Definitions

  • the embodiments of the present application relate to video coding and decoding technology, and relate to but are not limited to a video or image coding and decoding method, a decoder, an encoder and a readable storage medium.
  • Intra Template Matching Prediction uses a codec block template to search for a matching template with the lowest cost within a predefined search range within the current image, using a preset cost function. The best matching reconstructed block corresponding to the matching template is then used as the prediction block for the current codec block.
  • the codec block template is typically selected from a reconstructed area adjacent to the current codec block.
  • the number of BVs stored corresponding to the reconstructed blocks is often insufficient, which reduces the accuracy of intra-frame prediction and further reduces the encoding and decoding performance.
  • the present application provides a video or image encoding and decoding method, a decoder, an encoder and a readable storage medium, which can improve the accuracy of intra-frame prediction and thus improve encoding and decoding performance.
  • the present application provides a video or image decoding method, the method comprising:
  • a plurality of candidate reconstructed block vectors corresponding to the current block are cached.
  • the present application provides a video or image decoding method, the method comprising:
  • a prediction block corresponding to the current block is determined.
  • the present application provides a video or image decoding method, the method comprising:
  • At least one reconstructed block vector at at least one spatially adjacent and/or non-adjacent position corresponding to the second decoding block is determined; the at least one spatially adjacent and/or non-adjacent position corresponding to the second decoding block includes The location of the first decoding block;
  • the second decoded block is reconstructed based on the second prediction block, and a second reconstructed block corresponding to the second decoded block is determined.
  • the present application further provides a video or image encoding method, the method comprising:
  • a plurality of candidate reconstructed block vectors corresponding to the current block are cached.
  • the present application further provides a video or image encoding method, the method comprising:
  • a prediction block corresponding to the current block is determined.
  • the present application further provides a video or image encoding method, the method comprising:
  • the at least one spatially adjacent and/or non-adjacent position corresponding to the second coding block includes a position where the first coding block is located;
  • the second coding block is reconstructed based on the second prediction block, and a second reconstructed block corresponding to the second coding block is determined.
  • the present application provides an encoder, comprising a first determination part, a first prediction part, and a first cache part, wherein:
  • a first determining part configured to determine a current block and a prediction mode corresponding to the current block
  • a first prediction section configured to determine, according to the prediction mode, one or more first matching reconstructed block vectors corresponding to the current block; determine one or more second matching reconstructed block vectors based on the one or more first matching reconstructed block vectors; and predict the current block based on the prediction mode and the one or more second matching reconstructed block vectors to determine a prediction block corresponding to the current block;
  • the first determining portion is further configured to determine a plurality of candidate reconstructed block vectors corresponding to the current block based on the prediction mode and the one or more second matching reconstructed block vectors;
  • the first cache part is configured to cache a plurality of candidate reconstructed block vectors corresponding to the current block.
  • the present application provides an encoder, comprising a second determination part and a second prediction part, wherein:
  • the second determining part is configured to determine the current block and the prediction mode corresponding to the current block
  • the second prediction part is configured to determine, according to the prediction mode, at least one reconstructed block vector at at least one spatially adjacent and/or non-adjacent position corresponding to the current block; and determine, based on a reconstructed block vector in the at least one reconstructed block vector, multiple candidate reconstructed block vectors cached in the encoding information corresponding to the reconstructed block vector; and determine, based on the multiple candidate reconstructed block vectors corresponding to the reconstructed block vector, a prediction block corresponding to the current block.
  • the present application provides an encoder, a third determination part, a third prediction part, a first reconstruction part, and a second cache part, wherein:
  • the third determining part is configured to determine the first coding block and the prediction mode corresponding to the first coding block
  • the third prediction section is configured to determine one or more first matching reconstructed block vectors corresponding to the first coding block according to the prediction mode; determine one or more second matching reconstructed block vectors based on the one or more first matching reconstructed block vectors; and predict the first coding block based on the prediction mode and the one or more second matching reconstructed block vectors to determine a first prediction block corresponding to the first coding block;
  • the first reconstruction part is configured to reconstruct the first coding block based on the first prediction block and determine a first reconstructed block corresponding to the first coding block;
  • the third determining part is further configured to determine a plurality of candidate reconstructed block vectors corresponding to the first coding block based on the prediction mode and the one or more second matching reconstructed block vectors;
  • the second cache part is configured to cache the multiple candidate reconstructed block vectors corresponding to the first coding block in the first coding information corresponding to the first coding block;
  • the third prediction section is further configured to determine a prediction mode corresponding to the second coding block and the second coding block; determine, based on the prediction mode corresponding to the second coding block, at least one reconstructed block vector at at least one spatially adjacent and/or non-adjacent position corresponding to the second coding block; the at least one spatially adjacent and/or non-adjacent position corresponding to the second coding block includes a position where the first coding block is located; based on a reconstructed block vector in the at least one reconstructed block vector, determine multiple candidate reconstructed block vectors cached in the encoding information corresponding to the reconstructed block vector; and determine a prediction block corresponding to the second coding block based on the multiple candidate reconstructed block vectors corresponding to the reconstructed block vector;
  • the first reconstruction part is further configured to reconstruct the second coding block based on the second prediction block and determine a second reconstructed block corresponding to the second coding block.
  • the present application provides an encoder, comprising a first memory and a first processor, wherein:
  • the first memory is used to store a computer program that can be run on the first processor
  • the first processor is configured to execute the above-mentioned video or image encoding method provided in this application when running the computer program.
  • the present application provides a decoder, comprising a first parsing part, a fourth prediction part, a fourth determination part, and a third cache part, wherein:
  • the first parsing part is configured to parse the code stream and determine the prediction mode corresponding to the current block
  • the fourth prediction section is configured to determine one or more first matching reconstructed block vectors corresponding to the current block according to the prediction mode; determine one or more second matching reconstructed block vectors based on the one or more first matching reconstructed block vectors; and predict the current block based on the prediction mode and the one or more second matching reconstructed block vectors to determine a predicted block corresponding to the current block;
  • the fourth determining part is configured to determine a plurality of candidate reconstructed block vectors corresponding to the current block based on the prediction mode and the one or more second matching reconstructed block vectors;
  • the third cache portion is configured to cache multiple candidate reconstructed block vectors corresponding to the current block.
  • the present application provides a decoder, comprising a second parsing part and a fifth prediction part, wherein:
  • the second parsing part is configured to parse the code stream and determine the prediction mode corresponding to the current block
  • the fifth prediction part is configured to determine, according to the prediction mode, at least one reconstructed block vector at at least one spatially adjacent and/or non-adjacent position corresponding to the current block; determine, based on a reconstructed block vector in the at least one reconstructed block vector, multiple candidate reconstructed block vectors cached in the decoding information corresponding to the reconstructed block vector; and determine, based on the multiple candidate reconstructed block vectors corresponding to the reconstructed block vector, a prediction block corresponding to the current block.
  • the present application provides a decoder, comprising a third parsing part, a sixth prediction part, a fifth determination part, a second reconstruction part, and a fourth cache part, wherein:
  • the third parsing part is configured to parse the code stream and determine the prediction mode corresponding to the first decoding block
  • the sixth prediction part is configured to determine one or more first matching reconstructed block vectors corresponding to the first decoded block according to the prediction mode; determine one or more second matching reconstructed block vectors based on the one or more first matching reconstructed block vectors, and predict the first decoded block based on the prediction mode and the one or more second matching reconstructed block vectors to determine the first matching reconstructed block vectors.
  • a first prediction block corresponding to a decoding block
  • the second reconstruction part is configured to reconstruct the first decoding block based on the first prediction block and determine a first reconstructed block corresponding to the first decoding block;
  • the fifth determining part is configured to determine a plurality of candidate reconstructed block vectors corresponding to the first decoded block based on the prediction mode and the one or more second matching reconstructed block vectors;
  • the fourth cache part is configured to cache the multiple candidate reconstructed block vectors corresponding to the first decoding block in the first decoding information corresponding to the first decoding block;
  • the third parsing part is further configured to parse the code stream to determine the prediction mode corresponding to the second decoding block;
  • the sixth prediction part is further configured to determine, based on the prediction mode corresponding to the second decoding block, at least one reconstructed block vector at at least one spatially adjacent and/or non-adjacent position corresponding to the second decoding block; the at least one spatially adjacent and/or non-adjacent position corresponding to the second decoding block includes the position where the first decoding block is located; based on the reconstructed block vector in the at least one reconstructed block vector, determine multiple candidate reconstructed block vectors cached in the decoding information corresponding to the reconstructed block vector; based on the multiple candidate reconstructed block vectors corresponding to the reconstructed block vector, determine the prediction block corresponding to the second decoding block.
  • the second reconstruction part is further configured to reconstruct the second decoding block based on the second prediction block and determine a second reconstructed block corresponding to the second decoding block.
  • the present application provides an encoder, comprising a second memory and a second processor, wherein:
  • the second memory is used to store a computer program that can be run on the second processor
  • the second processor is configured to execute the above-mentioned video or image decoding method provided in this application when running the computer program.
  • an embodiment of the present application provides a bitstream, wherein the bitstream is generated by bit encoding based on information to be encoded; wherein the information to be encoded includes at least the encoding bits of a current block and a prediction mode of the current block;
  • the coding bits of the current block are obtained by reconstructing and encoding the current block based on a prediction block corresponding to the current block; the prediction block is used to cache multiple candidate reconstructed block vectors; the multiple candidate reconstructed block vectors are determined in the following manner:
  • the plurality of candidate reconstructed block vectors are determined based on the prediction mode and the one or more second matching reconstructed block vectors.
  • An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon.
  • the computer program is executed by a first processor, the video or image decoding method provided by the embodiment of the present application is implemented; alternatively, when the computer program is executed by a second processor, the video or image encoding method provided by the embodiment of the present application is implemented.
  • the embodiments of the present application provide a video or image encoding and decoding method, a decoder, an encoder, and a readable storage medium.
  • the decoder determines a prediction mode corresponding to a current block by parsing a bitstream; determines one or more first matching reconstructed block vectors corresponding to the current block based on the prediction mode; determines one or more second matching reconstructed block vectors based on the one or more first matching reconstructed block vectors, and predicts the current block based on the prediction mode and the one or more second matching reconstructed block vectors to determine the prediction block corresponding to the current block; determines multiple candidate reconstructed block vectors corresponding to the current block based on the prediction mode and the one or more second matching reconstructed block vectors; and caches multiple candidate reconstructed block vectors corresponding to the current block.
  • the encoder determines a prediction mode corresponding to the current block; determines one or more first matching reconstructed block vectors corresponding to the current block based on the prediction mode; determines one or more second matching reconstructed block vectors based on the one or more first matching reconstructed block vectors, and predicts the current block based on the prediction mode and the one or more second matching reconstructed block vectors to determine the prediction block corresponding to the current block; determines multiple candidate reconstructed block vectors corresponding to the current block based on the prediction mode and the one or more second matching reconstructed block vectors; and caches multiple candidate reconstructed block vectors corresponding to the current block.
  • multiple candidate reconstructed block vectors corresponding to the current block can be derived for caching, thereby increasing the number of BVs stored in each block.
  • the intra-frame encoding and decoding prediction mode that predicts the current block based on the BV of the reconstructed block, such as the intra-frame template matching prediction (IntraTMP) mode or the intra-frame block copy (Intra Block Copy, IBC) mode
  • the accuracy of the intra-frame prediction can be improved, thereby improving the encoding and decoding performance.
  • FIG1 is a schematic diagram of a template of a current block provided in an embodiment of the present application.
  • FIG2 is a block diagram of an encoder according to an embodiment of the present application.
  • FIG3 is a schematic block diagram of a decoder according to an embodiment of the present application.
  • FIG4 is a schematic diagram of a network architecture of a coding and decoding system provided in an embodiment of the present application.
  • FIG5 is a schematic diagram of an optional flow chart of a video or image decoding method provided in an embodiment of the present application.
  • FIG6( a ) is a schematic diagram of a template type 1 of the IntraTMP provided in an embodiment of the present application;
  • FIG6( b ) is a second schematic diagram of the template type of IntraTMP provided in an embodiment of the present application.
  • FIG6( c ) is a schematic diagram of the third template type of the IntraTMP provided in an embodiment of the present application.
  • FIG6( d ) is a fourth schematic diagram of the template type of IntraTMP provided in an embodiment of the present application.
  • FIG6( e ) is a fourth schematic diagram of the template type of IntraTMP provided in an embodiment of the present application.
  • FIG6( f ) is a fourth schematic diagram of the template type of IntraTMP provided in an embodiment of the present application.
  • FIG7 is a schematic diagram showing the meaning of initialization parameters corresponding to the current block and its template provided in an embodiment of the present application;
  • FIG8 is a schematic diagram of an optional template search area provided in an embodiment of the present application.
  • FIG9 is a schematic diagram of an optional spatially adjacent position and a spatially non-adjacent position of a current block provided by an embodiment of the present application;
  • FIG10 is a first schematic diagram of an IBC search range provided in an embodiment of the present application.
  • FIG11 is a second schematic diagram of an IBC search range provided in an embodiment of the present application.
  • FIG12 is a first schematic diagram of a detailed search range provided in an embodiment of the present application.
  • FIG13 is a second schematic diagram of a detailed search range provided in an embodiment of the present application.
  • FIG14( a ) is a third schematic diagram of a detailed search range provided in an embodiment of the present application.
  • FIG14( b ) is a fourth schematic diagram of a detailed search range provided in an embodiment of the present application.
  • FIG15 is a schematic diagram of an optional sub-pixel search range provided in an embodiment of the present application.
  • FIG16 is a schematic diagram of the shape of a filter in an intra-frame template matching prediction filtering mode provided by an embodiment of the present application.
  • FIG17 is a schematic diagram of predicted pixels in an intra-frame template matching prediction filtering mode provided by an embodiment of the present application.
  • FIG18 is a schematic diagram of an optional flow chart of a video or image decoding method provided in an embodiment of the present application.
  • FIG19 is a schematic diagram of an optional flow chart of a video or image decoding method provided in an embodiment of the present application.
  • FIG20 is a schematic diagram of an optional flow chart of a video or image encoding method provided in an embodiment of the present application.
  • FIG21 is a schematic diagram of an optional flow chart of a video or image encoding method provided in an embodiment of the present application.
  • FIG22 is a schematic diagram of an optional flow chart of a video or image encoding method provided in an embodiment of the present application.
  • FIG23 is a structural diagram of a decoder provided in an embodiment of the present application.
  • FIG24 is a second structural diagram of a decoder provided in an embodiment of the present application.
  • FIG25 is a third structural diagram of a decoder provided in an embodiment of the present application.
  • FIG26 is a fourth structural diagram of a decoder provided in an embodiment of the present application.
  • FIG27 is a structural diagram 1 of an encoder provided in an embodiment of the present application.
  • FIG28 is a second structural diagram of an encoder provided in an embodiment of the present application.
  • FIG29 is a third structural diagram of an encoder provided in an embodiment of the present application.
  • Figure 30 is a structural schematic diagram of an encoder provided in an embodiment of the present application.
  • ECM is based on the VTM-10.0 reference software and integrates various new tools to further explore codec performance.
  • the intra-frame template matching prediction weighted fusion mode IntraTMP Fusion, the first BV corresponding to the Fusion idx is stored. When used, the idx of this BV is obtained, which is the BV whose weighted number needs to be added later.
  • Intra template matching prediction linear filter model (IntraTMP FLM)
  • Digital video compression technology primarily compresses massive amounts of digital video data for easier transmission and storage.
  • Digital video compression standards can save significant amounts of video data, there is still a need for better digital video compression technologies to reduce bandwidth and traffic pressures associated with digital video transmission.
  • Video compression includes multiple modules such as intra-frame prediction (spatial domain) and inter-frame prediction (temporal domain) for reducing or removing inherent redundancy in the video, quantization and inverse quantization of residual information, loop filtering and entropy coding for improving subjective and objective reconstruction quality.
  • Most mainstream video compression standards describe block-based compression technology.
  • a video clip, a frame or a series of pictures will be divided into basic units based on CTU, which are further divided into blocks based on CU.
  • Intra-frame blocks are predicted by using the pixels around the block as reference, while inter-frame blocks refer to the information of neighboring blocks in space and reference information in other frames. Compared with the prediction signal, the residual information is transmitted through the CTU.
  • Video devices can achieve more efficient video encoding and decoding and transmission and storage by implementing these technologies.
  • Intra-frame template matching prediction is a special intra-frame prediction mode. Both the encoder and decoder use the template of the coding block (T) to search for the best matching template (T_BEST) with the minimum cost within a predefined search range in the current image according to a preset cost function.
  • the offset of the best matching template relative to the template of the current block (current coding block or decoding block) is the best block vector (BEST Block Vector, BV_BEST).
  • the reconstructed block (Ref Block) corresponding to the matching template is then used as the prediction block for the current block (Cur Block).
  • the template is usually selected from the reconstructed area adjacent to the current block. For example, the template of the current block can be shown in Figure 1.
  • IntraTMP is the position of the current coding block (xTbCmp, yTbCmp), the width of the current coding block nTbW, and the height of the current coding block nTbH.
  • the output of IntraTMP is the predicted value of the current block predSamples[x][y], where x ranges from 0 to nTbW-1 and y ranges from 0 to nTbH-1.
  • the prediction process of IntraTMP technology includes: determining the current template type, obtaining the reconstructed pixels of the current template, determining the block vector within the predefined search range, generating the predicted block and storing the BV. After the above process, the predicted block of the current block can be obtained.
  • IntraTMP can be used to predict the luminance component or the chrominance component.
  • IntraTMP Fusion mode only the first BV (the BV with the highest Fusion idx) among all BVs participating in the weighted fusion prediction process in IntraTMP Fusion mode is stored.
  • obtaining the Fusion idx of this BV retrieves the other BVs that will participate in the weighted fusion after this BV.
  • IntraTMP FLM mode the optimal BV (the one with the lowest template matching cost) is determined from the searched BVs and stored.
  • IntraTMP SubPel mode the optimal BV is also stored.
  • the list length can include up to 19 or more BVs. Since only one BV is stored for each prediction block, the number of BVs often insufficient when constructing the IntraTMP_Merge list. It can be seen that the number of BVs stored in the current IntraTMP technology is insufficient, which affects the accuracy of subsequent prediction of the encoding and decoding blocks using the BVs stored corresponding to each reconstructed block, thereby reducing the accuracy of intra-frame prediction and the encoding and decoding performance.
  • the embodiments of the present application provide a video or image encoding and decoding method, a decoder, an encoder, and a readable storage medium, which can improve the performance of video encoding and decoding.
  • the embodiments of the present application will be described in detail below with reference to the accompanying drawings.
  • the encoder 50 may include a transform and quantization unit 501, an intra-frame estimation unit 502, an intra-frame prediction unit 503, Inter-frame prediction unit 504, motion estimation unit 505, inverse transform and inverse quantization unit 506, filter control analysis unit 507, filtering unit 508, encoding unit 509 and decoded image cache unit 510, etc., wherein the filtering unit 508 can implement deblocking filtering and sample adaptive offset (Sample Adaptive Offset, SAO) filtering, and the encoding unit 509 can implement header information encoding and context-based adaptive binary arithmetic coding (Context-based Adaptive Binary Arithmetic Coding, CABAC).
  • SAO sample Adaptive Offset
  • the coding tree unit (Coding Tree A video coding block can be obtained by dividing the video coding block into a plurality of frames (CTUs) by a plurality of frames. Then, the residual pixel information obtained after intra-frame or inter-frame prediction is transformed by a transform and quantization unit 501, including transforming the residual information from the pixel domain to the transform domain and quantizing the obtained transform coefficients to further reduce the bit rate.
  • the intra-frame estimation unit 502 and the intra-frame prediction unit 503 are used to perform intra-frame prediction on the video coding block. Specifically, the intra-frame estimation unit 502 and the intra-frame prediction unit 503 are used to determine the intra-frame prediction mode to be used to encode the video coding block.
  • the inter-frame prediction unit 504 and the motion estimation unit 505 are used to perform inter-frame prediction coding of the received video coding block relative to one or more blocks in one or more reference frames to provide temporal prediction information.
  • the motion estimation performed by the motion estimation unit 505 is a process of generating a motion vector, which can estimate the motion of the video coding block.
  • the inter-frame prediction unit 504 then performs motion compensation based on the motion vector determined by the motion estimation unit 505, so that the frame
  • the inter-prediction unit 504 may also be referred to as a motion compensation unit.
  • the intra-prediction unit 503 is further configured to provide the selected intra-prediction data to the encoding unit 509, and the motion estimation unit 505 also sends the calculated motion vector data to the encoding unit 509.
  • the inverse transform and inverse quantization unit 506 is configured to reconstruct the video coding block by reconstructing a residual block in the pixel domain. This reconstructed residual block is subjected to a filter control analysis unit 507 and a filter unit 508 to remove blocking artifacts. The reconstructed residual block is then added to a prediction block in a frame in the decoded image buffer unit 510 to generate a reconstructed video coding block.
  • the encoding unit 509 is configured to encode various coding parameters and quantized transform coefficients.
  • context content may be based on neighboring coding blocks and may be used to encode information indicating the determined intra-prediction mode, thereby outputting a bitstream of the video signal.
  • the decoded image buffer unit 510 is configured to store the reconstructed video coding block for prediction reference. As the video image encoding proceeds, new reconstructed video encoding blocks are continuously generated, and these reconstructed video encoding blocks are stored in the decoded image buffer unit 510 .
  • the decoder (specifically, a “video decoder”) 60 includes a decoding unit 601, an inverse transform and inverse quantization unit 602, an intra-frame prediction unit 603, an inter-frame prediction unit 604, a filtering unit 605, and a decoded image buffer unit 606.
  • the decoding unit 601 can implement header information decoding and CABAC decoding
  • the filtering unit 605 can implement deblocking filtering and SAO filtering.
  • a code stream of the video signal is output; the code stream is input to the decoder 60 and first passes through the decoding unit 601 to obtain the decoded transform coefficients; the transform coefficients are processed by the inverse transform and inverse quantization unit 602 to generate a residual block in the pixel domain; the intra-frame prediction unit 603 can be used to generate prediction data for the current video decoding block based on the determined intra-frame prediction mode and the data of the previously decoded block from the current frame or picture; the inter-frame prediction unit 604 is to determine the prediction information for the video decoding block by parsing the motion vector and other associated syntax elements, and use The prediction information is used to generate a prediction block for the video decoding block being decoded; a decoded video block is formed by summing the residual block from the inverse transform and inverse quantization unit 602 with the corresponding prediction block generated by the intra-frame prediction unit 603 or the inter-frame prediction unit 604; the decoded video signal
  • FIG4 is a schematic diagram of a network architecture of a coding and decoding system provided by an embodiment of the present application.
  • the network architecture includes one or more electronic devices 13 to 1N and a communication network 01, wherein the electronic devices 13 to 1N can perform video interaction through the communication network 01.
  • the electronic device can be various types of devices with video coding and decoding functions.
  • the electronic device can include a smart phone, a tablet computer, a personal computer, a personal digital assistant, a navigator, a digital phone, a video phone, a television, a sensing device, a server, etc., which is not specifically limited in the embodiment of the present application.
  • the decoder or encoder described in the embodiment of the present application can be the above-mentioned electronic device.
  • the method of the embodiment of the present application is mainly applied to the intra-frame prediction unit 503 shown in Figure 2 and the intra-frame prediction unit 603 shown in Figure 3.
  • the embodiment of the present application can be applied to both the encoder and the decoder, and can even be applied to both the encoder and the decoder at the same time, but the embodiment of the present application does not specifically limit this.
  • the "current block” specifically refers to the coding block currently to be intra-frame predicted; when applied to the intra-frame prediction unit 504 part, the "current block” specifically refers to the decoding block currently to be intra-frame predicted.
  • FIG5 a flow chart of a video or image decoding method provided by an embodiment of the present application is shown.
  • the method may include:
  • S101 parse the code stream to determine the prediction mode corresponding to the current block.
  • the decoder receives a bitstream sent by the encoder and obtains a prediction mode corresponding to a current block in the bitstream by parsing the bitstream.
  • the current block represents the current block to be decoded.
  • the current block may include a coding unit (Coding Unit, CU), a transform unit (Transform Unit, TU), a coding block (Coding Block, CB) or a coding tree unit CTU, etc., which is not specifically limited in the embodiments of the present application.
  • a prediction mode is used to characterize an intra-frame prediction technology corresponding to a current block.
  • the prediction mode includes: making a prediction based on a reconstructed block corresponding to a candidate reconstructed block vector to determine a prediction mode of a prediction block corresponding to the current block. That is, the video or image decoding method of an embodiment of the present application can be widely used in a prediction mode that determines a reconstructed block based on BV and uses the reconstructed block to predict the current block.
  • the prediction mode may include an IntraTMP prediction mode or an IBC prediction mode. In some embodiments, it can be determined by parsing the second syntax element information in the code stream that is used to characterize different prediction modes with different values.
  • the prediction mode may include: determining a prediction mode (such as an IntraTMP mode) of a prediction block corresponding to the current block based on a reconstructed block vector determined by template matching. Determining a prediction mode of a prediction block corresponding to the current block based on a reconstructed block vector determined by template matching may include any of the following:
  • the prediction mode of the prediction block of the current block is determined by weighted fusion of the matching reconstructed blocks obtained based on template matching (such as IntraTMP Fusion mode), the prediction mode of the prediction block of the current block is determined by filtering the matching reconstructed blocks obtained based on template matching (such as IntraTMP FLM mode), and the prediction mode of the prediction block of the current block is determined by pixel-by-pixel prediction of the matching reconstructed blocks obtained based on template matching (such as IntraTMP SubPel mode).
  • template matching such as IntraTMP Fusion mode
  • the prediction mode of the prediction block of the current block is determined by filtering the matching reconstructed blocks obtained based on template matching
  • IntraTMP FLM mode such as IntraTMP FLM mode
  • the prediction mode of the prediction block of the current block is determined by pixel-by-pixel prediction of the matching reconstructed blocks obtained based on template matching (such as IntraTMP SubPel mode).
  • the IBC prediction mode may include the IBC fusion (IBC merge) mode and the IBC AMVP mode, etc.
  • the decoder can determine whether to use the IntraTMP prediction mode or the IBC prediction mode for the current block by parsing the syntax element information in the bitstream. It can also determine whether to use the IBC merge or the IBC AMVP mode under the IBC prediction mode for the current block by parsing the syntax element information in the bitstream; or determine to use the IntraTMP Fusion mode, the IntraTMP FLM mode, or the IntraTMP SubPel mode under the IntraTMP prediction mode for the current block.
  • S102 Determine one or more first matching reconstructed block vectors corresponding to the current block according to the prediction mode.
  • a matching block corresponding to the current block is determined in a reconstructed block of the current image based on a prediction mode to obtain one or more first matching reconstructed block vectors.
  • the one or more first matching reconstructed block vectors correspond to one or more first matching reconstructed blocks, and the one or more first matching reconstructed blocks are included in the current image corresponding to the current block.
  • one or more first matching reconstructed block vectors may be determined by searching the reconstructed blocks of the current image.
  • the prediction mode for determining the prediction block corresponding to the current block based on the reconstructed block vector determined by template matching may correspond to an IntraTMP prediction mode.
  • determining the prediction mode (such as the IntraTMP mode) of the prediction block corresponding to the current block based on the reconstructed block vector determined by template matching may include any one of the following:
  • the prediction mode of the prediction block of the current block is determined by weighted fusion of the matching reconstructed blocks obtained based on template matching (such as IntraTMP Fusion mode), the prediction mode of the prediction block of the current block is determined by filtering the matching reconstructed blocks obtained based on template matching (such as IntraTMP FLM mode), and the prediction mode of the prediction block of the current block is determined by pixel-by-pixel prediction of the matching reconstructed blocks obtained based on template matching (such as IntraTMP SubPel mode).
  • template matching such as IntraTMP Fusion mode
  • the prediction mode of the prediction block of the current block is determined by filtering the matching reconstructed blocks obtained based on template matching
  • IntraTMP FLM mode such as IntraTMP FLM mode
  • the prediction mode of the prediction block of the current block is determined by pixel-by-pixel prediction of the matching reconstructed blocks obtained based on template matching (such as IntraTMP SubPel mode).
  • the one or more first matching reconstructed block vectors may include: one or more reconstructed block vectors determined by the search process of the IntraTMP prediction mode.
  • each of the one or more first matching reconstructed block vectors corresponds to a TmpIdx value.
  • a template type corresponding to the current block is determined; based on the template type, template samples corresponding to the current block are determined; a search area corresponding to the current block is determined, and based on the template samples corresponding to the current block, a search is performed in the search area to determine one or more first matching reconstructed block vectors.
  • the template samples may include pixels in the template.
  • the IntraTMP technology uses the adjacent reconstructed pixels of the current block as a template to search for a matching template within a predefined search area.
  • the adjacent reconstructed pixels can be the upper reference pixel, upper left reference pixel, upper right reference pixel, left reference pixel, and lower left reference pixel of the current block. Therefore, the template type of the current block can be determined based on whether the adjacent reconstructed pixels of the current block are available.
  • the template type can be represented by refTemplateType, and the template type can include the following cases:
  • refTemplateType When only the left and upper left reference pixels are available, the value of refTemplateType is 4, and the template shape is shown in Figure 6(d);
  • the template of the current block can be composed of reconstructed pixels from one or more regions of the current block: the top, top right, left, bottom left, or top left.
  • the template size is pre-set. For example, when obtaining the left template, the template width templateW_size can be set to 4, and when obtaining the top template, the template height templateH_size can be set to 4.
  • the value of refTemplateType can be used to determine which portion of the reconstructed pixels to obtain as template samples.
  • refTemplateType when the value of refTemplateType is 1, the left, upper-left and upper reconstructed pixels of the current block are obtained as template samples; when the value of refTemplateType is 2, only the left 4 columns of reconstructed pixels of the current block are obtained as template samples; when the value of refTemplateType is 3, only the upper 4 rows of reconstructed pixels of the current block are obtained as template samples.
  • the following initialization process may be included:
  • TemplateW_size and templateH_size can be fixed constants, or they can be dynamically adjusted according to the size of the current block. TemplateW_size and templateH_size can be equal or unequal.
  • the meaning of the above initialization parameters can be shown in Figure 7.
  • the cost threshold between templates can be:
  • diffThreshold indicates that the distortion threshold of each pixel in the template area is 256.
  • offsetLCBY yTbCmp-ctbRsY
  • offsetLCBX xTbCmp-ctbRsX
  • iTemplateSizeH templateH_size
  • iTemplateSizeW templateW_size
  • iTemplateSizeH and iTemplateSizeW are used to update the search range.
  • iBvShift which is the precision of the block vector BV.
  • the BV precision can be integer pixel precision, in which case iBvShift is 0; the BV precision can also be sub-pixel precision, for example, when iBvShift is 1, it indicates 1/2 pixel precision, and when iBvShift is 2, it indicates 1/4 pixel precision.
  • the template search range can be set to a fixed size or dynamically adjusted based on the current block size, as follows:
  • searchRangeWidth TMP_SEARCH_RANGE_MULT_FACTOR*nTbW;
  • searchRangeHeight TMP_SEARCH_RANGE_MULT_FACTOR*nTbH;
  • searchRangeWidth and searchRangeHeight represent the width and height of the search range respectively
  • searchRangeHeight represent the width and height of the search range respectively
  • the value of TMP_SEARCH_RANGE_MULT_FACTOR is a preset value, for example, it is set to a fixed value of 5.
  • the search area of the current block may include a surrounding rectangular search area and an extended search area, as follows:
  • the surrounding rectangular search area includes a full reconstruction area and a pending reconstruction area.
  • the full reconstruction area includes the area where all sampling points in the determined area are reconstructed.
  • the full reconstruction area may include the four areas R1 to R4 in Figure 8.
  • the pending reconstruction area includes the area where it is uncertain whether all sampling points in the area are reconstructed.
  • the pending reconstruction area may include the two areas R5 to R6 in Figure 8.
  • the extended search area is defined as the area pointed to by the BVs corresponding to the prediction units (PUs) at spatially adjacent and non-adjacent locations. This area can be considered as R7 (not shown in Figure 8). Since the search points in R7 are not necessarily adjacent to each other, the search of the R7 area is performed point by point according to the list.
  • search points in the search area all search points in the specified search area can be traversed, or different schemes can be used to limit the search range to a local search range to balance the computational complexity and coding efficiency.
  • the search points can include searching, for example, for the undetermined reconstruction area (corresponding to the two areas R5 to R6 in Figure 8) or for the full reconstruction area (for example, corresponding to the four areas R1 to R4 in Figure 8), the width of the search range can be limited to the original 1/wIndex, and the height can be limited to the original 1/hIndex, where wIndex and hIndex are both any positive integers greater than or equal to 1.
  • the limited search range takes the area closer to the unit to be encoded.
  • the width of the R5 search area can be reduced to 1/2 of the original width, and the width can be reduced to 1/2 of the original width.
  • the final search range can take the upper right 1/4 area of the R5 area.
  • the search point in the search area corresponds to the block vector in the search area, for example, the search point is the upper left corner coordinate point corresponding to the block vector.
  • bvXMins and bvXMaxs represent the minimum offset and maximum offset of the block vector in the search area in the horizontal direction equivalent to the current block, respectively;
  • bvYMins and bvYMaxs represent the minimum offset and maximum offset of the block vector in the vertical direction equivalent to the current block, respectively
  • regionId represents the identifier of the search area where the block vector is located.
  • the horizontal and vertical offset ranges of the search points in the search area relative to the current block can be expressed as bvXMins regionId , bvXMaxs regionId , bvYMins regionId , and bvYMaxs regionId .
  • bvXMins regionId iHorMin regionId -xTbCmp;
  • bvXMaxs regionId iHorMax regionId -xTbCmp;
  • bvYMins regionId iVerMin regionId -yTbCmp
  • bvYMaxs regionId iVerMax regionId -yTbCmp;
  • iHorMin regionId represents the minimum offset and maximum offset of a search area in the horizontal direction.
  • pX lies between bvXMins and bvXMaxs
  • pY lies between bvYMins and bvYMaxs.
  • a matching reconstructed block for the current block can be found in the reconstruction region.
  • the adjacent reconstructed pixels of the matching reconstructed block are the matching template. This allows the template matching cost between the template corresponding to the current block and the matching template to be calculated, denoted as pDiff.
  • the availability of the search point (iPosHor, iPoxVer) in each search area is judged: if available, a matching reconstructed block of the current block can be found in the reconstructed area, and the adjacent reconstructed pixels of the matching reconstructed block are the matching template.
  • the matching cost between the adjacent template of the current block and the adjacent template of the reconstructed block can be calculated, also recorded as pDiff. If unavailable, the template matching cost calculation is not performed.
  • Availability judgment includes but is not limited to one or more of the following conditions being met simultaneously:
  • Each sampling point in the template does not exceed the valid coordinate range limited by the image sampling point boundary
  • Each sampling point in the template corresponding to the reconstruction block does not exceed the valid coordinate range limited by the image sampling point boundary;
  • Each sampling point in the template and each sampling point in the corresponding reconstructed block does not exceed the specified range of the search window
  • Each sampling point in the reconstructed block corresponding to the template is not in the current coding area
  • a search strategy of first performing a coarse search and then a fine search may be adopted, or only a fine search or only a coarse search may be performed.
  • the coarse search may include: determining the best coarse matching template in the search area with a first step size (eg, 3), or determining the best coarse matching template in the search area using a downsampled template (eg, a downsampling factor of 3).
  • the fine search may include: determining the best fine matching template in the search area with a second step size (eg, 1), or determining the best fine matching template near the best coarse matching template after performing the coarse search.
  • a second step size eg, 1
  • first search with a first step length it is possible to first search with a first step length, and then perform a second step length search based on the result of the first step length search to determine one or more first matching reconstructed block vectors; or, it is also possible to perform only the first step length search.
  • only the search of the second step length may be performed, and the search results of the second step length may be determined as one or more first matching reconstructed block vectors.
  • a coarse search is performed with a step size greater than 1.
  • a coarse search is performed with a step size of 2.
  • the best matching cost obtained by template matching is recorded as pDiff_BEST, and its corresponding block vector BV is recorded as the best block vector BV_BEST(pX_BEST, pY_BEST).
  • the search strategy is to perform only a search of the second step length, that is, only a fine search
  • a fine search is performed with a step length of 1, and the optimal matching cost obtained by recording the template matching is pDiff_BEST, and its corresponding block vector BV is recorded as the best block vector BV_BEST (pX_BEST, pY_BEST).
  • the following describes the process of determining one or more first matching reconstructed block vectors by searching the reconstructed block of the current image, taking the example of first searching the first step length and then searching the second step length.
  • first searching the first step length and then searching the second step length.
  • second step length search For details on performing only the first step length search or only the second step length search, please refer to the description of the corresponding step length search process in the following procedures.
  • searching in a search area based on template samples corresponding to the current block to determine one or more first matching reconstructed block vectors includes:
  • a search is performed in the search area with a first step length to determine a first step length block vector list; the first step length block vector list includes one or more matching reconstructed block vectors searched based on the first step length; based on the matching reconstructed block vectors in the first step length block vector list, a search is performed in the search area with a second step length to determine one or more first matching reconstructed block vectors; the second step length is smaller than the first step length.
  • a search is performed in the search area using the first step length for each block vector within the search area.
  • the template matching cost between the template sample corresponding to the current block and the template sample of each block vector searched is calculated.
  • one or more matching reconstructed block vectors are determined as the first step length block vector list.
  • the first step length block vector list may include matching reconstructed block vectors found during the entire coarse search process.
  • the one or more first matching reconstructed block vectors include all matching reconstructed block vectors found during the coarse and fine search processes.
  • searching in the search area with the first step length to determine the first step length block vector list includes:
  • a search is performed in the search area with the first step length to determine a first step length candidate reconstruction block; a template sample corresponding to the first step length candidate reconstruction block is determined; a template matching cost is calculated based on the template sample corresponding to the first step length candidate reconstruction block and the template sample corresponding to the current block to determine the template matching cost corresponding to the first step length candidate reconstruction block; a first step length block vector list is determined based on the template matching cost corresponding to one or more first step length candidate reconstruction blocks.
  • the first step candidate reconstruction block is equivalent to the reconstructed block searched each time the first step is searched.
  • the template type of the first step candidate reconstruction block is determined
  • the template corresponding to the first step candidate reconstruction block is determined based on the template type
  • the template sample corresponding to the first step candidate reconstruction block is determined based on the pixels in the template corresponding to the first step candidate reconstruction block.
  • the process of determining the template sample corresponding to the first step candidate reconstruction block is similar to the process of determining the template sample corresponding to the current block, and will not be repeated here.
  • a first step block vector list can be determined based on one or more first step candidate reconstruction blocks selected with a small template matching cost.
  • the template matching cost includes at least one of the sum of absolute differences (SAD) and the sum of absolute transformed differences (SATD).
  • the template matching cost may also include mean-square error (MSE), sum of squared differences (SSD), mean absolute deviation (MAD), mean square differences (MSD), normalized correlation coefficient (NCC), etc.
  • MSE mean-square error
  • SSD sum of squared differences
  • MAD mean absolute deviation
  • MSD mean square differences
  • NCC normalized correlation coefficient
  • the absolute error and SAD cost functions may be:
  • the first step length block is determined based on the template matching cost corresponding to one or more first step length candidate reconstruction blocks.
  • a list of vectors including:
  • the spatially adjacent and/or non-adjacent at least one location includes:
  • At least one first-step candidate reconstructed block vector corresponding to at least one first-step candidate reconstructed block with a low template matching cost is selected as a first-step initial block vector list.
  • a reconstructed block vector at each position i.e., each spatially adjacent or non-adjacent position
  • a prediction block corresponding to each reconstructed block vector is determined.
  • the prediction mode of the prediction block corresponding to each reconstructed block vector it is determined whether one or more candidate reconstructed block vectors are cached or stored in the decoding information corresponding to the prediction block. If it is determined that one or more candidate reconstructed block vectors are cached or stored, the one or more candidate reconstructed block vectors are obtained from the decoding information corresponding to the prediction block, and the first-step initial block vector list is updated, for example, by adding the one or more candidate reconstructed block vectors to the first-step initial block vector list.
  • the first-step initial block vector list is determined by performing the same process for each of the at least one position, using the one or more candidate reconstructed block vectors corresponding to each position to update the first-step initial block vector list.
  • a coarse search (a search with the first step length) is performed first, followed by a fine search (a search with the second step length).
  • the search process may include: constructing a coarse search list (equivalent to a block vector list with the first step length) within the search area, and determining a fine search list near BV in the coarse search list with the second step length.
  • a coarse search is performed with a step length of 3, while pX is between bvXMinsregionId and bvXMaxsregionId, and pY is between bvYMinsregionId and bvYMaxsregionId.
  • BV1_BEST corresponds to at least one first step candidate reconstructed block vector corresponding to at least one first step candidate reconstructed block with a low template matching cost.
  • multiple BVs stored at multiple predefined search locations are used to construct a candidate BV list for the current block, referred to as the IntraTMP_Merge list.
  • This list is constructed by adding BVs at spatially adjacent locations and adding BVs at spatially non-adjacent locations.
  • five spatially adjacent positions of the current block are used: left (xTbCmp-1, yTbCmp+nTbH-1), upper left (xTbCmp-1, yTbCmp-1), upper (xTbCmp+nTbW-1, yTbCmp-1), upper right (xTbCmp+nTbW, yTbCmp-1), and lower left (xTbCmp-1, yTbCmp+nTbH), as shown in Figure 9.
  • the IBC search range can be represented as shown in Figure 10. When the current CTU size is 256*256, the search range is modified to the area shown in Figure 11.
  • the PUs corresponding to these positions are checked to see if they use the IntraTMP/IBC technology. If so, the availability of the BV information stored therein is checked (availability includes but is not limited to: whether each position after the current CU plus the BV of this PU is within the IBC search range).
  • the IBC search range can be represented as shown in Figure 10. When the current CTU size is 256*256, its search range is modified to the area shown in Figure 11.
  • BVs i.e., multiple candidate reconstructed block vectors
  • embodiments of the present application can derive multiple BVs (i.e., multiple candidate reconstructed block vectors) from the decoded information corresponding to the reconstructed blocks at each spatially adjacent and non-adjacent position, and use these multiple candidate reconstructed block vectors to construct the IntraTMP-Merge list. This will be explained in subsequent embodiments.
  • the rough search list (the first step of the long initial block vector list) is updated to obtain the final rough search list (the first step of the long block vector list).
  • the specific operation of the update is: access each item in this list in turn, calculate the template matching cost corresponding to each BV, compare this cost with the template matching cost of the initial rough search list, and if it is less than the maximum template matching cost in the rough search list, then replace this BV with the worse BV in the rough search list.
  • the specific replacement operation is, for example: insert this BV into the rough search list in the order of cost size, and delete the item with the highest cost in the rough search list. And set the search area bestRegionId where it is located to 6.
  • the rough search list can also be updated by using other list merging-related search orders or merging strategies. The specific selection is made according to the actual situation, and the embodiments of this application are not limited.
  • M 1.
  • the one or more fine search reference points here constitute the first step's long block vector list.
  • the refined search range can be a fixed size or related to the search area.
  • the boundaries of the detailed search range must also be checked against the IBC search range shown in Figure 10 or Figure 11. If all four locations are within the IBC search range, the detailed search range remains unchanged. If any location is not within the IBC search range, the point at the corresponding x or y coordinate -1 is checked to see if it is within the IBC search range. This continues until the location reaches the detailed search starting point. The range at this point is the final detailed search range.
  • the ranges described below are all final detailed search ranges after the IBC range check.
  • both TmpRefineRangeHor and TmpRefineRangeVer can be set to 1. That is, for region IDs 0 to 5, the refinement range is a 3 ⁇ 3 pixel area with the refinement start point at [0,0] and vertical and horizontal coordinates offset within the range [-1,1], respectively. A full point-by-point search is performed within this range, as shown in Figure 12.
  • both TmpRefineRangeHor and TmpRefineRangeVer can be set to 5. That is, for a region ID of 6, the refinement range is an 11 ⁇ 11 pixel area with the refinement start point at [0,0] and vertical and horizontal coordinates offset within the range [-5,5], respectively. A full point-by-point search is performed within this range, as shown in Figure 13.
  • the fine search range is a 5 ⁇ 5 pixel area with the fine search starting point at [0,0] as the coordinate, and the vertical and horizontal coordinates are offset within the range of [-2,2], as shown in Figure 14(a).
  • the fine search range is a 5-pixel area with the fine search starting point at [0,0] as the coordinate, and the vertical and horizontal coordinates are offset within the range of [-5,5], as shown in Figure 14(b). A full point-by-point search is performed within this range.
  • I 0 (x, y) is the pixel value in the current template/current prediction block
  • I 1 (x, y) is the pixel value in the reference template/reference block
  • a and b are linear model parameters. a and b are calculated using the current block template and the reference block template (e.g., using the least squares method). The pixel values in the reference block are then substituted into the values to calculate the pixel values in the current prediction block.
  • the new search range is obtained based on the optimal matching block position obtained by the rough search:
  • iHorMaxrefine min(picWidth-nTbW,BestPosX+TmpRefineRangeHor)
  • iHorMinrefine max(iTemplateSizeW,BestPosX-TmpRefineRangeHor)
  • the adjusted block vectors BVbvXMins, bvXMaxs, bvYMins, bvYMaxs can be calculated by iVerMinrefine, iVerMaxrefine, iHorMinrefine, iHorMaxrefine:
  • the fine search is performed within the block vector range where pX falls between bvXMins refine and bvXMaxs refine , and pY falls between bvYMins refine and bvYMax refine .
  • one or more block vectors obtained from the coarse and fine search processes can be used as one or more first matching reconstructed block vectors.
  • a sub-pixel position search can be performed based on the one or more block vectors obtained from the coarse and fine search processes, and the one or more first matching reconstructed block vectors can be determined based on the results of the sub-pixel search.
  • the sub-pixel search includes: comparing the searched integer pixel position costs one by one within a certain fine search list to determine the minimum cost position; determining the initial sub-pixel position direction, and determining the sub-pixel position accuracy and final direction.
  • the sub-pixel search starts at [0,0] and is performed in eight directions with vertical and horizontal coordinates offset by -1/2 and 1/2, respectively.
  • the eight candidate positions are compared one by one, and the first four directions with the lowest cost are determined as the initial directions for the sub-pixel search.
  • the sub-pixel search starts at [0,0] and is performed in the first four directions with vertical and horizontal coordinates offset by the range [-3/4, 3/4], respectively.
  • a search is performed between multiple 1/4 pixel precision positions, and the costs of the 32 candidate positions are compared one by one to determine the position with the lowest cost, which is the sub-pixel position index tmpIsSubPel and the final direction index tmpSubIdx.
  • tmpIsSubPel has four values, corresponding to the integer pixel position (0), 1/2 pixel position (1), 1/4 pixel position (2 or 3), and 3/4 pixel position (2 or 3); the corresponding sequence numbers of the 1/4 and 3/4 positions are related to the interpolation position.
  • tmpSubIdx has eight values, corresponding to eight directions: left (0), right (1), up (2), down (3), upper left (4), upper right (5), lower left (6), and lower right (7), as shown in Figure 15.
  • the sub-pixel position (Dx, Dy) value is calculated based on tmpIsSubPel and tmpSubIdx to obtain the specific sub-pixel coordinate position, and then the optimal matching coordinate is updated.
  • Each of these items is a coordinate pair (pX_BEST, pY_BEST).
  • pX_BEST and pY_BEST are the horizontal and vertical offsets of the best matching template relative to the current block template, respectively, and are also the horizontal and vertical offsets of the best matching reconstructed block relative to the current block.
  • the decoder can determine whether to perform a sub-pixel search based on corresponding syntax elements in the bitstream.
  • the one or more first matched reconstructed block vectors may include both integer-pixel BVs obtained by the coarse search and the fine search, and sub-pixel BVs obtained by the sub-pixel search.
  • a template matching cost is calculated to calculate the sub-pixel precision position.
  • the sub-pixel positions include: up, down, left, right, upper left, upper right, lower left, and lower right; and the sub-pixel precision includes: 1/2, 1/4, and 3/4.
  • the decoder determines whether to perform a pixel-by-pixel search (sub-pixel search) by parsing the syntax elements in the bitstream.
  • a pixel-by-pixel search sub-pixel search
  • current technologies use sparse sub-pixel positions, which cannot cover all positions with 1/4 precision. This reduces the accuracy of intra-frame prediction and, in turn, encoding and decoding performance.
  • an embodiment of the present application further provides a method for decoding a video or image, comprising:
  • the prediction mode representation determines the prediction block of the current block by performing pixel-by-pixel prediction on the matching reconstructed block obtained based on template matching (such as the IntraTMP SubPel mode)
  • the bitstream is parsed to determine the value of the first syntax element information
  • the bitstream is parsed to determine the second syntax element information
  • the second syntax element information is obtained by quadratic curve fitting; and based on the second syntax element information, a second matching reconstructed block vector is determined.
  • the first syntax element information is used to indicate whether to perform pixel-wise search.
  • the data structure of the syntax element information in the code stream may be as follows:
  • the first syntax element information is equivalent to isTmpSubPel.
  • the first value of the first syntax element information isTmpSubPel can be 1 or true. It can also take other values to distinguish it from the second value (such as 0 or false), which is not limited in this embodiment of the application.
  • the second syntax element information is equivalent to tmpSubPelIdx.
  • the one or more first matched reconstructed block vectors include: matched reconstructed block vectors at integer pixel positions determined based on searches of the first step size and the second step size; that is, based on the BVs at integer pixel positions obtained by the coarse search and the fine search, the process of determining the second matched reconstructed block vector based on the second syntax element information may include:
  • a second matching reconstructed block vector is determined from the sub-pixel matching reconstructed block vector list.
  • At least one preset position may include eight positions: up, down, left, right, upper left, upper right, lower left, and lower right, and at least one pixel precision may include three pixel precisions: 1/2, 1/4, and 3/4.
  • the method further comprises: performing a search with a first sub-pixel precision at each preset position of at least one preset position based on each first matching reconstructed block vector, and determining a set of first sub-pixel matching reconstructed block vectors corresponding to each first matching reconstructed block vector;
  • a sub-pixel matching reconstructed block vector list is determined based on a third sub-pixel matching reconstructed block vector set corresponding to each first matching reconstructed block vector in the one or more first matching reconstructed block vectors.
  • the first pixel precision may include 1/2 pixel precision
  • at least one second pixel precision may include at least one of 1/4 pixel precision, 3/4 pixel precision, 1/8 pixel precision, 1/16 pixel precision, etc., which is not limited in the embodiments of the present application.
  • updating the first sub-pixel matching and reconstructed block vector set according to the second sub-pixel matching and reconstructed block vector, and determining the second sub-pixel matching and reconstructed block vector set corresponding to each first matching and reconstructed block vector includes:
  • the first sub-pixel matching and reconstruction block vector with the largest template matching cost in the first sub-pixel matching and reconstruction block vector set is replaced to determine the second sub-pixel matching and reconstruction block vector set;
  • a target preset first position corresponding to a maximum template matching cost is determined, and the first subpixel matched reconstructed block vector corresponding to the target preset first position in the first subpixel matched reconstructed block vector set is replaced with the second subpixel matched reconstructed block vector to determine a second subpixel matched reconstructed block vector set;
  • the second sub-pixel matching and reconstructed block vector is added to the first sub-pixel matching and reconstructed block vector set to determine a second sub-pixel matching and reconstructed block vector set.
  • the least likely sub-pixel direction/position can be replaced by a sub-pixel position selection method derived from quadratic curve fitting based on the template matching cost or the numerical distribution characteristics of the aCOST array.
  • this method can be combined with current equal-length encoding methods or unequal-length encoding methods.
  • updating the second sub-pixel matching and reconstruction block vector can include: determining the position corresponding to the maximum template matching cost according to the template matching cost at each position in the aCOST array, using the second sub-pixel matching and reconstruction block vector to replace the first sub-pixel matching and reconstruction block vector corresponding to the position in the first sub-pixel matching and reconstruction block vector set, and determining the second sub-pixel matching and reconstruction block vector set.
  • the method further comprises:
  • the target first matching and reconstruction block vector corresponding to the target first matching and reconstruction block vector
  • the corresponding target first sub-pixel matching and reconstruction block vector set is updated to determine the target second sub-pixel matching and reconstruction block vector set corresponding to the target first matching and reconstruction block vector;
  • a subpixel matching reconstructed block vector list is determined based on a set of second subpixel matching reconstructed block vectors corresponding to non-target first matching reconstructed block vectors in one or more first matching reconstructed block vectors and a set of target second subpixel matching reconstructed block vectors.
  • performing quadratic curve fitting at a sub-pixel position based on each first matching reconstructed block vector to determine a second sub-pixel matching reconstructed block vector corresponding to each first matching reconstructed block vector includes:
  • first position information by calculating a minimum template matching cost for each integer pixel position in a horizontal direction using a quadratic curve method based on the multiple template matching costs, and determining second position information by calculating a minimum template matching cost for each integer pixel position in a vertical direction using a quadratic curve method;
  • the first position information and the second position information are combined to determine a second sub-pixel matching and reconstructed block vector corresponding to each first matching and reconstructed block vector.
  • the plurality of preset first positions include: a pixel position above the integer pixel position, a pixel position below the integer pixel position, a pixel position to the left of the integer pixel position, and a pixel position to the right of the integer pixel position.
  • combining the first position information and the second position information to determine the second sub-pixel matching reconstructed block vector corresponding to each first matching reconstructed block vector includes:
  • a search is performed within a preset range with a preset step size, and a second matching reconstructed block vector is determined based on a template matching cost of the searched reconstructed block vector.
  • aCOST[1][1] represents the current integer pixel position
  • aCOST[0][1] represents the pixel position to the left of the current integer pixel position
  • aCOST[2][1] represents the pixel position to the right of the current integer pixel position
  • aCOST[1][0] represents the pixel position above the current integer pixel position
  • aCOST[1][2] represents the pixel position below the current integer pixel position.
  • combining the first position information and the second position information to determine the second sub-pixel matching reconstructed block vector corresponding to each first matching reconstructed block vector includes:
  • a search is performed within a preset range with a preset step size, and a second matching reconstructed block vector is determined based on a template matching cost of the searched reconstructed block vector.
  • the calculated sub-pixel position can be used directly or after performing a small surrounding range search based on it.
  • determining the second matching reconstructed block vector from the sub-pixel matching reconstructed block vector list based on the second syntax element information includes:
  • determining a second matching reconstructed block vector based on the second syntax element information includes:
  • Parsing the bitstream to determine a value of the third syntax element information determining an initial second matched reconstructed block vector based on the second syntax element information when the value of the third syntax element information represents a sub-pixel position quadratic curve fitting performed on the pixel position indicated by the second syntax element information; performing sub-pixel position quadratic curve fitting on the initial second matched reconstructed block vector to determine a sub-pixel reconstructed block vector corresponding to the initial second matched reconstructed block vector as the second matched reconstructed block vector.
  • the third syntax element information can be used to indicate whether to perform quadratic curve fitting on the pixel position indicated by the second syntax element information, and the second matching reconstructed block vector is determined based on the pixel reconstructed block vector obtained by the quadratic curve fitting.
  • determining the second matching reconstructed block vector from the sub-pixel matching reconstructed block vector list based on the second syntax element information includes:
  • the template matching costs of the sub-pixel positions and the integer pixel positions and the sub-pixel positions corresponding to the integer pixel positions (for example, the accuracy can be one or more of 1/2, 1/4, 3/4, and the direction can be up, down, left, right, upper left, lower left, upper right, lower right) calculated by the quadratic curve method can be sorted, and the first Y BVs (for example, Y can be 1, 2, 3, 4, etc.) with the smallest template matching costs after sorting are taken as candidate BVs.
  • the method further comprises:
  • a flag can be used to indicate that the position is obtained using this method instead of the sub-pixel position of the original method.
  • this method can be combined with unequal length coding.
  • one or more second matching reconstructed blocks required for use in the prediction process of the current block are determined.
  • the decoder may parse the bitstream and determine the prediction mode corresponding to the current block based on syntax element information indicating the prediction mode in the bitstream. For example, by parsing the syntax element information indicating the prediction mode in the bitstream, it may be determined whether the prediction mode corresponding to the current block is the IntraTMP Fusion mode, the IntraTMP FLM mode, or the IntraTMP SubPel mode.
  • one or more second matching reconstructed block vectors are selected from the list by using a template matching cost.
  • the one or more second matching reconstructed block vectors may include: multiple matching reconstructed block vectors for weighted fusion corresponding to the IntraTMP Fusion mode, or matching reconstructed block vectors with the lowest template matching cost corresponding to the IntraTMP FLM mode and the IntraTMP SubPel mode. In this way, for different prediction modes, the current block can be predicted based on the one or more second matching reconstructed block vectors to determine the predicted block corresponding to the current block.
  • the first N items are selected as multiple second matching reconstruction blocks based on one or more first matching reconstruction block vectors, and the multiple second matching reconstruction blocks are weightedly fused to determine the prediction block corresponding to the current block.
  • the first matching reconstruction block vector with the minimum template matching cost is determined based on one or more first matching reconstruction block vectors as a second matching reconstruction block vector, and multiple points are taken around the second matching reconstruction block vector, and the corresponding prediction values of the multiple points are weightedly fused and filtered to obtain the prediction value.
  • the first matching reconstructed block vector with the minimum template matching cost is determined based on one or more first matching reconstructed block vectors as a second matching reconstructed block vector, and the second matching reconstructed block vector is subjected to pixel-wise sub-pixel precision to obtain multiple sub-pixel block vectors, and filtering is performed based on the multiple sub-pixel block vectors to obtain the prediction block corresponding to the current block.
  • N candidate reconstructed blocks are obtained using the BVs. These N candidate reconstructed blocks are then weightedly fused to produce a prediction for the current block. Specifically, the final prediction is generated through the following steps: obtaining N candidate reconstructed blocks, determining weights for weighted fusion, and then generating a prediction through weighted fusion.
  • This is achieved using a simple translational copy.
  • the weight can be a predefined value or a value adaptively calculated using cost value, pixel value, etc. For example, the weight is derived by minimizing the MSE of the reconstructed value of the candidate template refTn and the pixel value of the template to be predicted refpredTn. Fusion weights.
  • the MSE minimization process takes as input the autocorrelation matrix of the first P matching reference samples refT and the cross-correlation vector of the first P matching reference samples refT and the adjacent template samples curT of the current block, and outputs the weight of each matching reference item corresponding to the reconstructed block.
  • the prediction block is calculated based on each candidate reconstructed block and its corresponding weighted fusion weight. Specifically, the value of each candidate reconstructed block and its corresponding weight are multiplied and accumulated to obtain the prediction block corresponding to the current block (i.e., weighted prediction).
  • the calculation formula is as follows:
  • each predicted value predSamples x,y is the prediction block output by the IntraTMP Fusion mode.
  • IntraTMP FLM mode a linear filter model is established using the best matching template found in the previous search step and the current block template. This mainly involves the following two steps: determining the reconstruction area for calculating the filter coefficients, and calculating the filter coefficients.
  • the specific prediction value calculation process is as follows:
  • C0 through C4 are the tap coefficients of the filter, respectively.
  • the point corresponding to tap coefficient C0 is the reconstructed pixel ref[i][j] at the corresponding position in the best matching block for the current predicted pixel Ypred [i][j].
  • the remaining points, C1 through C4 are reconstructed pixels adjacent to the current spatial position in the best matching reconstructed block.
  • the points in Figure 17 are the resulting predicted pixels Ypred [i][j].
  • the specific prediction value calculation process is as follows:
  • IntraTMP SubPel mode for the optimal BV, traverse 1/4, 1/2, 3/4 and eight directions of up, down, left, right, upper left, upper right, lower left, and lower right, sort them according to the template matching cost, and use the interpolation filter to calculate the prediction value of the reference block corresponding to the BV with the smallest template matching cost to obtain the prediction block corresponding to the current block.
  • the current block is reconstructed based on the prediction block corresponding to the current block to determine a reconstructed block corresponding to the current block. It will be appreciated that the reconstructed block corresponding to the current block may be used in the intra-frame prediction process for the next block to be encoded. Thus, according to the above process, blocks in the current image are reconstructed to obtain a reconstructed block in the current image.
  • S104 Determine multiple candidate reconstructed block vectors corresponding to the current block based on the prediction mode and one or more second matching reconstructed block vectors.
  • the embodiment of the present application can determine multiple candidate reconstructed block vectors corresponding to the current block based on the prediction mode and one or more second matching reconstructed block vectors, and store the multiple candidate reconstructed block vectors in the decoding information corresponding to the current block, thereby greatly increasing the number of BVs in the candidate list when constructing the candidate list for intra-frame prediction for each block in the current image, thereby improving prediction accuracy.
  • the prediction mode characterization determines the prediction block of the current block by weighted fusion of the matching reconstructed blocks obtained based on template matching (such as the IntraTMP Fusion mode)
  • multiple candidate reconstructed block vectors corresponding to the current block are determined based on multiple second matching reconstructed block vectors.
  • weighted fusion is performed on the matching reconstructed blocks obtained based on template matching to determine the prediction mode of the prediction block of the current block, and multiple second matching reconstructed block vectors can be determined as multiple candidate reconstructed block vectors corresponding to the current block.
  • all second matching reconstructed block vectors may be determined as a plurality of candidate reconstructed block vectors for storage.
  • the first template matching costs corresponding to the second matching reconstruction block vectors may be used to reconstruct the first template matching costs.
  • a plurality of candidate reconstructed block vectors corresponding to the current block are determined from the plurality of second matching reconstructed block vectors.
  • top X first template matching costs with the smallest template matching costs may be determined; X is an integer greater than 1; and X second matching reconstructed block vectors corresponding to the top X first template matching costs are determined as multiple candidate reconstructed block vectors corresponding to the current block.
  • X second matching reconstructed block vectors having template matching costs less than a preset first cost threshold may be determined as the multiple candidate reconstructed block vectors corresponding to the current block.
  • first template matching costs from small to large to determine a second template matching cost among the multiple first template matching costs, the increase value between which and the previous template matching cost is greater than a first preset growth threshold; and determine the second matching reconstructed block vector corresponding to the first template matching cost before the second template matching cost as the multiple candidate reconstructed block vectors corresponding to the current block.
  • the first template matching cost includes at least one of a sum of absolute differences (SAD) and a sum of absolute transformation differences (SATD).
  • SAD sum of absolute differences
  • SATD sum of absolute transformation differences
  • multiple second matching reconstruction block vectors can be sorted from small to large according to the matching cost (such as SAD or SATD, etc.), and the first X second matching reconstruction block vectors (for example, X can be 2) are determined as multiple candidate reconstruction block vectors for storage.
  • the second matching reconstruction block vectors before TmpIdx can be sorted, a matching cost (such as SAD) can be calculated, and several second matching reconstruction block vectors whose matching costs are less than a certain threshold can be selected.
  • several second matching reconstruction block vectors before the matching cost increases significantly can be taken as multiple candidate reconstruction block vectors.
  • the second smallest matching cost is greater than a first preset growth threshold, for example, greater than twice the smallest matching cost
  • a first preset growth threshold for example, greater than twice the smallest matching cost
  • the second matching reconstruction block vector with the smallest matching cost is determined as a candidate reconstruction block vector
  • the third smallest matching cost is greater than twice the second smallest matching cost
  • the first two second matching reconstruction block vectors with the smallest costs are determined as candidate reconstruction block vectors.
  • weighted fusion is performed on the matching reconstructed blocks obtained based on template matching to determine the prediction mode of the prediction block of the current block.
  • the first X second matching reconstructed block vectors with the highest weights among the multiple second matching reconstructed block vectors may be determined as the multiple candidate reconstructed block vectors corresponding to the current block; wherein the weight represents the weight corresponding to each second matching reconstructed block when weighted fusion prediction is performed on the multiple second matching reconstructed blocks corresponding to the multiple second matching reconstructed block vectors.
  • the weights are calculated by using Wiener filtering, and the calculated weights are sorted from large to small, and the first X second matching reconstruction block vectors (for example, X can be 2) are determined as multiple candidate reconstruction block vectors for storage.
  • At least one initial candidate reconstructed block vector is determined by at least one of the following methods:
  • the second matched reconstructed block vector represents a matched reconstructed block vector having the minimum template matching cost with the current block among the one or more first matched reconstructed block vectors;
  • a plurality of candidate reconstructed block vectors corresponding to the current block are determined.
  • At least one BV can be constructed by offsetting in at least one direction based on the optimal BV used for prediction in IntraTMP FLM mode (the second matched reconstructed block vector, i.e., the single BV of integer pixels used for prediction); the at least one BV obtained by offsetting in at least one direction and/or the second matched reconstructed block vector are used as at least one initial candidate reconstructed block vector.
  • At least one first matched reconstructed block vector with a lower template matching cost, other than the second matched reconstructed block vector is determined from the one or more first matched reconstructed block vectors, and the at least one first matched reconstructed block vector and/or the second matched reconstructed block vector selected based on the template matching cost is used as at least one initial candidate reconstructed block vector.
  • At least one initial candidate reconstruction block vector can be used as multiple candidate reconstruction block vectors stored in the IntraTMP FLM mode; or multiple candidate reconstruction block vectors can be determined from at least one initial candidate reconstruction block vector by further screening the at least one initial candidate reconstruction block vector, for example, sorting according to filter coefficients or template matching costs.
  • the at least one direction includes at least one of up, down, left, right, upper left, upper right, lower left, and lower right.
  • the second matching reconstructed block vector may be the BV corresponding to TmpIdx determined by RDO during the prediction process.
  • the second matching reconstructed block vector is offset in at least one direction to determine at least one initial candidate reconstructed block Vectors, including:
  • determining multiple candidate reconstructed block vectors corresponding to the current block based on at least one initial candidate reconstructed block vector includes:
  • At least one initial candidate reconstructed block vector is determined as a plurality of candidate reconstructed block vectors corresponding to the current block.
  • determining multiple candidate reconstructed block vectors corresponding to the current block based on at least one initial candidate reconstructed block vector includes:
  • a plurality of candidate reconstructed block vectors corresponding to the current block are determined from at least one initial candidate reconstructed block vector.
  • determining, based on at least one third template matching cost, a plurality of candidate reconstructed block vectors corresponding to the current block from at least one initial candidate reconstructed block vector includes:
  • the at least one third template matching cost determine the first X template matching costs with the smallest template matching cost, where X is an integer greater than 1; and determine the X reconstructed block vectors corresponding to the first X template matching costs as multiple candidate reconstructed block vectors corresponding to the current block. It should be understood that the X reconstructed block vectors include the second matching reconstructed block vector and/or one or more initial candidate reconstructed block vectors of the at least one initial candidate reconstructed block vector.
  • determining, based on at least one third template matching cost, a plurality of candidate reconstructed block vectors corresponding to the current block from at least one initial candidate reconstructed block vector includes:
  • X reconstructed block vectors having template matching costs less than a preset second cost threshold are determined as multiple candidate reconstructed block vectors corresponding to the current block; X is an integer greater than 1. It is understood that the X reconstructed block vectors include the second matching reconstructed block vector and/or one or more initial candidate reconstructed block vectors of the at least one initial candidate reconstructed block vector.
  • determining, based on at least one third template matching cost, a plurality of candidate reconstructed block vectors corresponding to the current block from at least one initial candidate reconstructed block vector includes:
  • a fourth template matching cost having an increase value greater than a second preset growth threshold relative to the previous template matching cost is determined; and a reconstructed block vector corresponding to the third template matching cost preceding the fourth template matching cost is determined as one of the multiple candidate reconstructed block vectors corresponding to the current block.
  • the reconstructed block vector corresponding to the third template matching cost preceding the fourth template matching cost includes the second matching reconstructed block vector and/or one or more initial candidate reconstructed block vectors of the at least one initial candidate reconstructed block vector.
  • the second matching reconstructed block vector and at least one initial candidate reconstructed block vector constructed after its offset in various directions can be sorted from small to large by template matching cost (such as SAD, SATD, etc.), and the first X reconstructed block vectors (for example, X can be 2) are determined as multiple candidate reconstructed block vectors.
  • the filter coefficients calculated by the IntraTMP FLM mode are also sorted from large to small, and the positions corresponding to the filter coefficients are the center position (0, 0), top (0, -1), bottom (0, 1), left (-1, 0), and right (1, 0).
  • the first X (for example, X can be 2) BVs with larger filter coefficients are selected and stored as multiple candidate reconstructed block vectors after offsetting the BVs to corresponding positions. For example: if the first two positions with larger filter coefficients are in the middle and on the top, then two BVs (bvX+0, bvY+0) and (bvX+0, bvY+(-1)) are stored.
  • the above-mentioned process of determining multiple candidate reconstructed block vectors corresponding to the current block in at least one initial candidate reconstructed block vector based on at least one third template matching cost is similar to the above-mentioned process of determining multiple candidate reconstructed block vectors corresponding to the current block in multiple second matching reconstructed block vectors based on multiple first template matching costs corresponding to multiple second matching reconstructed block vectors when the prediction mode representation is based on performing weighted fusion of the matching reconstructed block obtained by template matching to determine the prediction block of the current block, and will not be repeated here.
  • At least one initial candidate reconstructed block vector is determined by at least one of the following methods:
  • the second matched reconstructed block vector represents a first matched reconstructed block vector having the minimum template matching cost with the current block among the one or more first matched reconstructed block vectors;
  • a plurality of candidate reconstructed block vectors corresponding to the current block are determined.
  • the second matching reconstructed block vector may be the BV corresponding to TmpIdx determined by RDO during the prediction process.
  • a plurality of candidate reconstructed block vectors for storage may be determined from a plurality of sub-pixel block vectors obtained through sub-pixel traversal and the second matching reconstructed block vector by way of sub-pixel traversal; and/or at least one first matching reconstructed block vector other than the second matching reconstructed block vector used for prediction may be determined from one or more first matching reconstructed block vectors determined during the search process, and a plurality of candidate reconstructed block vectors for storage may be determined from the at least one first matching reconstructed block vector and the second matching reconstructed block vector.
  • Whether to perform sub-pixel traversal may be determined by parsing syntax element information in the bitstream that indicates whether to perform sub-pixel traversal, and/or by determining whether the storage accuracy reaches sub-pixel accuracy.
  • the process of performing sub-pixel accuracy and direction traversal based on the second matched reconstructed block vector to determine multiple sub-pixel block vectors may include:
  • the plurality of first sub-pixel block vectors and the plurality of second sub-pixel block vectors are determined as a plurality of sub-pixel block vectors; or the plurality of second sub-pixel block vectors are determined as a plurality of sub-pixel block vectors.
  • multiple sub-pixel block vectors can be determined through a sub-pixel search process.
  • the sub-pixel search process is completed in the following two steps:
  • the first step is to search with a step size of 1/2 pixel in a range less than one pixel offset, centered on the integer pixel position.
  • the second step is to search with a step size of 1/4 pixel in a range less than one pixel offset, centered on the 1/2 precision or integer pixel precision position corresponding to the minimum value, based on the minimum value found in the first step.
  • all BVs of the sub-pixel search process that is, the BVs of all positions involved in the above two steps (equivalent to multiple first sub-pixel block vectors and multiple second sub-pixel block vectors) can be used as multiple sub-pixel block vectors; or only the BVs involved in the 1/4 pixel search process, that is, only the BVs of the positions involved in the second step (equivalent to multiple second sub-pixel block vectors) can be used as multiple sub-pixel block vectors.
  • determining multiple candidate reconstructed block vectors corresponding to the current block based on at least one initial candidate reconstructed block vector includes:
  • At least one initial candidate reconstructed block vector is determined as a plurality of candidate reconstructed block vectors corresponding to the current block.
  • all of the at least one initial candidate reconstructed block vector may be determined as a plurality of candidate reconstructed block vectors.
  • determining multiple candidate reconstructed block vectors corresponding to the current block based on the at least one initial candidate reconstructed block vector may further include:
  • At least one third template matching cost between at least one initial candidate reconstructed block vector and the current block is determined; and multiple candidate reconstructed block vectors corresponding to the current block are determined from the at least one initial candidate reconstructed block vector based on the at least one third template matching cost.
  • determining, based on at least one third template matching cost, a plurality of candidate reconstructed block vectors corresponding to the current block from at least one initial candidate reconstructed block vector includes:
  • first X template matching costs with the smallest template matching cost are determined, where X is an integer greater than 1; and X reconstructed block vectors corresponding to the first X template matching costs are determined as multiple candidate reconstructed block vectors corresponding to the current block.
  • determining, based on at least one third template matching cost, a plurality of candidate reconstructed block vectors corresponding to the current block from at least one initial candidate reconstructed block vector includes:
  • X reconstructed block vectors having template matching costs less than a preset second cost threshold are determined as multiple candidate reconstructed block vectors corresponding to the current block; X is an integer greater than 1.
  • determining, based on at least one third template matching cost, a plurality of candidate reconstructed block vectors corresponding to the current block from at least one initial candidate reconstructed block vector includes:
  • a fourth template matching cost is greater than a second preset growth threshold; and reconstructed block vectors corresponding to the third template matching cost before the fourth template matching cost are determined as multiple candidate reconstructed block vectors corresponding to the current block.
  • the prediction mode characterizes the prediction of the current block by performing pixel-by-pixel prediction on the matching reconstructed block obtained based on template matching, such as the IntraTMP SubPel mode
  • the template matching cost of each sub-pixel and the corresponding direction can be calculated, and the template matching costs (such as SAD, SATD, etc.) of these sub-pixel BVs (which may include all BVs in the sub-pixel search process, or only include BVs involved in the 1/4 pixel search process) are sorted from small to large, and the first X sub-pixel BVs (for example, X can be 2) are stored as multiple candidate reconstructed block vectors.
  • template matching costs such as SAD, SATD, etc.
  • the prediction mode representation is based on the matching reconstructed block obtained by template matching and filtering to determine the prediction block of the current block
  • the prediction mode representation is based on the matching reconstructed block obtained by template matching and pixel-by-pixel prediction to determine the prediction block of the current block
  • the above-mentioned process of determining multiple candidate reconstructed block vectors corresponding to the current block in at least one initial candidate reconstructed block vector based on at least one third template matching cost is similar, and will not be repeated here.
  • the above template matching costs include at least one of the sum of absolute differences (SAD) and the sum of absolute transformation differences (SATD).
  • the value of X can be determined based on the prediction mode. For example, different restrictions are imposed on the number of BVs added by each IntraTMP mode. For example, for a PU encoded in IntraTMP Fusion mode, X can take the value of 3 or 4, corresponding to allowing the storage of 3 to 4 BVs; for a PU encoded in IntraTMP FLM mode, X can take the value of 3 or 4, corresponding to allowing the storage of 2 to 3 BVs; for a PU encoded in IntraTMP SubPel mode, X can take the value of 3 or 4, corresponding to allowing the storage of 2 to 3 BVs.
  • the decoder may further determine the value of the first syntax element information by parsing the bitstream; when the value of the first syntax element information is the first value, determine a second matching reconstructed block vector from one or more first matching reconstructed block vectors based on the template matching cost, and perform pixel-wise precision and direction traversal based on the second matching reconstructed block vector to determine multiple pixel-wise block vectors; the second matching reconstructed block vector represents the first matching reconstructed block vector with the smallest template matching cost with the current block among the one or more first matching reconstructed block vectors; based on the template matching cost with the current block, sort the one or more first matching reconstructed block vectors and the multiple pixel-wise block vectors to determine the top M block vectors with the smallest template matching cost as multiple candidate reconstructed block vectors; M is a positive integer greater than 1; the template matching cost includes: at least one of the sum of absolute errors (SAD) and the sum of absolute transform differences (SATD).
  • SAD sum of absolute errors
  • SATD
  • the value of the first syntax element information being a first value indicates that the one or more first matching reconstructed block vectors and the multiple sub-pixel block vectors are sorted based on the template matching cost with the current block, and the top M block vectors with the lowest template matching cost are determined as multiple candidate reconstructed block vectors.
  • the first value may include any value in the form of a number or character, and the specific value is selected based on actual circumstances and is not limited in this embodiment of the application.
  • the value of the first syntax element information is a second value, no sorting is performed, and the second value is different from the first value.
  • all BVs determined in the search process, as well as the optimal BV sub-pixel in the BV determined in the search process and the sub-pixel BV obtained by direction traversal may be sorted by template matching cost to determine multiple candidate reconstructed block vectors for storage.
  • the multiple candidate reconstructed block vectors corresponding to the current block are cached in the decoding information corresponding to the current block.
  • multiple candidate reconstructed block vectors can be cached in a preset data structure of the decoding information, such as a motion vector (MV) data structure.
  • MV motion vector
  • the embodiments of the present application effectively utilize the information of spatially adjacent and non-adjacent reconstructed blocks, and increase the search area range of IntraTMP without significantly increasing the coding complexity, increase the BV candidate positions derived based on the BV information of the reconstructed blocks, improve the prediction accuracy, and thus improve the encoding and decoding performance.
  • the embodiment of the present application further provides a video or image decoding method, as shown in FIG18 , including:
  • S201 parse the code stream to determine the prediction mode corresponding to the current block.
  • the prediction mode is used to characterize the intra-frame prediction technology corresponding to the current block.
  • the prediction mode includes: predicting the reconstructed block corresponding to the candidate reconstructed block vector to determine the prediction mode of the prediction block corresponding to the current block. That is to say, the video or image decoding method of the embodiment of the present application can be widely used in a prediction mode that determines the reconstructed block based on BV and predicts the current block using the reconstructed block.
  • the prediction mode may include an IntraTMP prediction mode or an IBC prediction mode. In some embodiments, it can be determined by parsing the second syntax element information in the code stream that is used to characterize different prediction modes with different values.
  • the prediction mode may include: determining a prediction mode (such as an IntraTMP mode) of a prediction block corresponding to the current block based on a reconstructed block vector determined by template matching. Determining a prediction mode of a prediction block corresponding to the current block based on a reconstructed block vector determined by template matching may include any of the following:
  • the prediction mode of the prediction block of the current block is determined by weighted fusion of the matching reconstructed blocks obtained based on template matching (such as IntraTMP Fusion mode), the prediction mode of the prediction block of the current block is determined by filtering the matching reconstructed blocks obtained based on template matching (such as IntraTMP FLM mode), and the prediction mode of the prediction block of the current block is determined by pixel-by-pixel prediction of the matching reconstructed blocks obtained based on template matching (such as IntraTMP SubPel mode).
  • template matching such as IntraTMP Fusion mode
  • the prediction mode of the prediction block of the current block is determined by filtering the matching reconstructed blocks obtained based on template matching
  • IntraTMP FLM mode such as IntraTMP FLM mode
  • the prediction mode of the prediction block of the current block is determined by pixel-by-pixel prediction of the matching reconstructed blocks obtained based on template matching (such as IntraTMP SubPel mode).
  • the IBC prediction mode may include the IBC fusion (IBC merge) mode and the IBC AMVP mode, etc.
  • the decoder can determine whether to use the IntraTMP prediction mode or the IBC prediction mode for the current block by parsing the syntax element information in the bitstream. It can also determine whether to use the IBC merge or the IBC AMVP mode under the IBC prediction mode for the current block by parsing the syntax element information in the bitstream; or determine to use the IntraTMP Fusion mode, the IntraTMP FLM mode, or the IntraTMP SubPel mode under the IntraTMP prediction mode for the current block.
  • S202 Determine, according to the prediction mode, at least one reconstructed block vector at at least one spatially adjacent and/or non-adjacent position corresponding to the current block.
  • the prediction mode when the prediction mode characterizes determining a prediction block corresponding to a current block based on a reconstructed block vector determined based on template matching (e.g., IntraTMP Fusion mode), at least one reconstructed block vector at at least one spatially adjacent and/or non-adjacent position corresponding to the current block is determined.
  • the at least one spatially adjacent and/or non-adjacent position includes:
  • At least one reconstructed block vector at at least one position is determined based on the reconstructed block vector corresponding to each position in at least one spatially adjacent and/or non-adjacent position (ie, each spatially adjacent position or spatially non-adjacent position).
  • S203 Based on a reconstructed block vector in the at least one reconstructed block vector, determine a plurality of candidate reconstructed block vectors cached in the decoding information corresponding to the reconstructed block vector.
  • the reconstructed block vector is obtained through a prediction and reconstruction process, so each reconstructed block vector corresponds to a prediction block.
  • the decoding information corresponding to the prediction block caches or stores the prediction mode used to predict the prediction block.
  • a prediction block corresponding to each reconstructed block vector is determined, and a determination is made as to whether the prediction block is obtained based on a prediction mode used to predict the candidate reconstructed block vector. If it is determined that the prediction block is obtained based on the prediction mode used to predict the candidate reconstructed block vector, multiple cached candidate reconstructed block vectors are read from a preset data structure in the coding information corresponding to the prediction block.
  • the preset data structure may include an MV data structure preset in the decoding information.
  • the prediction mode based on the candidate reconstructed block vector widely includes the prediction mode of determining the reconstructed block based on BV and using the reconstructed block to predict the current block.
  • it can be checked whether the prediction block is predicted using the IntraTMP prediction mode or the IBC prediction mode. If so, it means that BV information is stored in the decoding information corresponding to the prediction block.
  • only one BV information is cached or stored in the decoding information corresponding to the prediction block, while in the embodiment of the present application, multiple candidate reconstructed block vectors are cached or stored in the decoding information corresponding to the prediction block, which provides more options for constructing a candidate BV list for intra-frame prediction.
  • the multiple candidate reconstructed block vectors stored in the decoding information corresponding to a prediction block in the embodiment of the present application are obtained through the aforementioned S101-S105 embodiments and will not be repeated here.
  • each reconstructed block vector in at least one reconstructed block vector multiple candidate reconstructed block vectors corresponding to the reconstructed block vector are determined, and the at least one reconstructed block vector is processed in the same manner to obtain multiple candidate reconstructed block vectors corresponding to each reconstructed block vector.
  • multiple candidate reconstructed block vectors corresponding to each reconstructed block vector in at least one reconstructed block vector can be used to construct a reference list for intra-frame prediction, which is used to predict the current block and determine the prediction block corresponding to the current block.
  • a first step length search process may be used to determine a first step length initial block vector list corresponding to the current block.
  • the process of determining the first step length initial block vector list corresponding to the current block may include determining a template type corresponding to the current block; determining template samples corresponding to the current block based on the template type; determining a search area corresponding to the current block; and searching the search area with the first step length based on the template samples corresponding to the current block to determine the first step length initial block vector list corresponding to the current block.
  • searching in the search area with the first step length to determine the first step length initial block vector list corresponding to the current block includes: searching in the search area with the first step length to determine the first step length candidate reconstructed block; calculating the template matching cost based on the template samples corresponding to the first step length candidate reconstructed block and the template samples corresponding to the current block to determine the template matching cost corresponding to the first step length candidate reconstructed block; determining the first step length initial block vector list corresponding to the current block based on the template matching cost corresponding to one or more first step length candidate reconstructed blocks; wherein the template matching cost includes: at least one of the sum of absolute difference (SAD) and the sum of absolute transformation difference (SATD).
  • SAD sum of absolute difference
  • SATD sum of absolute transformation difference
  • the process of determining the prediction block corresponding to the current block may include:
  • a candidate reconstructed block vector list is determined based on multiple candidate reconstructed block vectors corresponding to each reconstructed block vector in at least one reconstructed block vector; a first step long initial block vector list is updated using the candidate reconstructed block vector list to determine a first step long block vector list; the first step long block vector list includes one or more matching reconstructed block vectors; and a prediction block corresponding to the current block is determined based on the one or more matching reconstructed block vectors.
  • the multiple candidate reconstructed block vectors corresponding to each reconstructed block vector in the at least one reconstructed block vector can be used in the IntraTMP_Merge list construction process during the search process of the IntraTMP prediction mode: after adding at least one reconstructed block vector corresponding to at least one spatially adjacent and spatially non-adjacent position as the IntraTMP Merge list, decoding information of the prediction block of each reconstructed block vector in the at least one reconstructed block vector is continuously obtained, and multiple candidate reconstructed block vectors are read therefrom to update the IntraTMP_Merge list; the first step long initial block vector list is then updated using the updated IntraTMP_Merge list to determine the first step long block vector list (the coarse search list obtained by the final coarse search process).
  • a fine search and/or sub-pixel search is performed based on the first step long block vector list to determine one or more second matching reconstructed block vectors, and a current block is predicted based on the one or more second matching reconstructed block vectors to determine the prediction block corresponding to the current block.
  • the specific search process and prediction process can refer to the same process in the aforementioned embodiment and will not be repeated here.
  • the following describes a process of determining a candidate reconstructed block vector list based on multiple candidate reconstructed block vectors corresponding to each reconstructed block vector in at least one reconstructed block vector.
  • determining a candidate reconstructed block vector list based on a plurality of candidate reconstructed block vectors corresponding to each of the at least one reconstructed block vector comprises:
  • each reconstructed block vector in the at least one reconstructed block vector is determined in sequence; and when each reconstructed block vector is available, a candidate reconstructed block vector list is determined based on a plurality of candidate reconstructed block vectors corresponding to each reconstructed block vector.
  • At least one reconstructed block vector corresponds to at least one spatially adjacent and spatially non-adjacent position.
  • the availability of each reconstructed block vector in the at least one reconstructed block vector may be determined sequentially in positional order (e.g., from spatially adjacent to spatially non-adjacent).
  • a candidate reconstructed block vector list is determined based on the plurality of candidate reconstructed block vectors corresponding to the reconstructed block vector.
  • the process of determining the availability of each reconstructed block vector may include but is not limited to:
  • the reconstructed block vectors at spatially adjacent positions determine whether the positions after adding the reconstructed block vectors at the spatially adjacent positions to the current block (e.g., current CU) are within the IBC search range: upper left (xTbCmp+BVx, yTbCmp), lower left (xTbCmp+BVx, yTbCmp+nTbH+BVy), upper right (xTbCmp+nTbW+BVx, yTbCmp+BVy), and lower right (xTbCmp+nTbW+BVx, yTbCmp+nTbH+BVy).
  • the IBC search range can be represented as shown in Figure 10. When the size of the coding tree unit (current CTU) corresponding to the current block is 256*256, its search range is modified to the area shown in Figure 11.
  • the similarity between the available BV and the BV already in the IntraTMP-Merge list can be compared to determine whether the available BV is too consistent with the BV already in the IntraTMP-Merge list; if the similarity is less than a preset similarity threshold, the available BV is added to the list.
  • the IBC search range can be represented as shown in Figure 10.
  • the size of the coding tree unit (current CTU) corresponding to the current block is 256*256, its search range is modified to the area shown in Figure 11.
  • the similarity between the available BV and the BV already in the IntraTMP-Merge list can be compared to determine whether the available BV is too consistent with the BV already in the IntraTMP-Merge list; if the similarity is less than the preset similarity threshold, the available BV is added to the list. Until the list length meets the requirements or all positions are checked, the construction of the IntraTMP-Merge list (that is, the candidate reconstructed block vector list) is completed.
  • determining a candidate reconstructed block vector list based on multiple candidate reconstructed block vectors corresponding to each reconstructed block vector includes:
  • the first initial candidate reconstructed block vector list is updated until the list length of the first initial candidate reconstructed block vector list reaches a preset length threshold, or until at least one reconstructed block vector is processed, and the first initial candidate reconstructed block vector list is determined as the candidate reconstructed block vector list.
  • each reconstructed block vector list is updated until the list length of the first initial candidate reconstructed block vector list reaches a preset length threshold, or until at least one reconstructed block vector is processed, and the first initial candidate reconstructed block vector list is determined as the candidate reconstructed block vector list.
  • the availability of each reconstructed block vector is sequentially determined according to a preset position order.
  • the first candidate reconstructed block vector among the multiple candidate reconstructed block vectors corresponding to the reconstructed block vector is added to a list, thereby determining a first initial candidate reconstructed block vector list. If the length of the first initial candidate reconstructed block vector list is insufficient at this point, at least one non-first candidate reconstructed block vector corresponding to each available reconstructed block vector in the at least one reconstructed block vector is sequentially determined.
  • the non-first candidate reconstructed block vector among the at least one non-first candidate reconstructed block vector is added to the first initial candidate reconstructed block vector list.
  • Each reconstructed block vector is processed sequentially in this manner until the length of the first initial candidate reconstructed block vector list reaches a preset length threshold, or at least one reconstructed block vector is completely processed.
  • the first initial candidate reconstructed block vector list is then determined as the candidate reconstructed block vector list.
  • availability checks may be performed again if the first candidate reconstructed block vectors of all the available reconstructed block vectors in the at least one reconstructed block vector have been added, but the list length does not meet the requirement, so that the candidate reconstructed block vectors can be added to the list to meet the length requirement.
  • the process of detecting the availability of the candidate reconstructed block vector is the same as the process of detecting the availability of the reconstructed block vector, which will not be repeated here.
  • determining a candidate reconstructed block vector list based on multiple candidate reconstructed block vectors corresponding to each reconstructed block vector includes:
  • the available non-first candidate reconstructed block vector is added to the first initial candidate reconstructed block vector list.
  • determining a candidate reconstructed block vector list based on multiple candidate reconstructed block vectors corresponding to each reconstructed block vector may further include:
  • determining or updating a second initial candidate reconstructed block vector list based on available candidate reconstructed block vectors among a plurality of candidate reconstructed block vectors corresponding to each reconstructed block vector includes:
  • the available candidate reconstructed block vector is added to the second initial candidate reconstructed block vector list.
  • the more than one BV items stored therein will also be used to construct the IntraTMP_Merge list (equivalent to the candidate reconstruction block vector list). More than one BV can be detected after the first BV of the corresponding block; or it can be detected after the first BV of all blocks has been checked and the list length does not meet the requirements.
  • the availability of each BV is checked. Whether the available BV information is too consistent with the existing items in the IntraTMP-Merge list (the same or similar, similarity is determined by the threshold). If it is too consistent, it will not be added to the list. Otherwise, it will be added to the list until the list length meets the requirements.
  • the method further comprises:
  • the candidate reconstructed block vector corresponding to the fifth template matching cost is used to replace the candidate reconstructed block vector corresponding to the maximum template matching cost in the candidate reconstructed block vector list, until each candidate reconstructed block vector corresponding to each position in at least one spatially adjacent and/or non-adjacent position is traversed.
  • the IntraTMP_Merge list can continue to replace existing items in the IntraTMP_Merge list with subsequent BVs based on their matching costs. That is, if the matching cost of a subsequent BV is lower than the matching cost of the item with the highest matching cost in the list, the subsequent BV is added to the list, replacing the original item with the highest matching cost. This process continues until all candidate items have been tested.
  • the method further comprises:
  • the available candidate reconstructed block vectors include: each available candidate reconstructed block vector in the at least one available candidate reconstructed block vector, or include: a non-first available candidate reconstructed block vector in the at least one non-first candidate reconstructed block vector.
  • each BV can be added to the IntraTMP_Merge list in an ordered manner based on its matching cost, thus forming an ordered IntraTMP_Merge list.
  • the list length reaches a preset target, subsequent candidate BVs are added in an ordered manner based on their matching cost, and the last item in the list is deleted. This process continues until all candidate items have been tested.
  • determining a prediction block corresponding to the current block based on one or more matching reconstructed block vectors includes:
  • One or more second matching reconstructed block vectors are determined based on the one or more first matching reconstructed block vectors, and a current block is predicted based on the prediction mode and the one or more second matching reconstructed block vectors to determine a prediction block corresponding to the current block.
  • IBC modes such as IBC merge and IBC AMVP modes
  • more than one BV corresponding to the cache of each reconstructed block vector can also be used to create candidate lists in these scenarios.
  • multiple candidate reconstructed block vectors cached at corresponding positions of each reconstructed block vector may be used to construct a reference BV list in the DIMD mode and/or TIMD mode, and then the prediction block corresponding to the current block may be obtained through prediction in the DIMD mode and/or TIMD mode, as follows:
  • the prediction mode represents a decoding-side derived prediction mode based on a non-angle mode for the current block (e.g., a DIMD mode)
  • multiple first candidate reconstructed blocks are determined based on multiple candidate reconstructed block vectors corresponding to the reconstructed block vector; where the multiple first candidate reconstructed blocks represent a candidate BV list used for DIMD mode prediction.
  • an intra-mode derivation result for the non-angle mode corresponding to the current block is determined, thereby determining a prediction block corresponding to the current block.
  • the prediction mode represents a template-based intra prediction mode derivation (e.g., TIMD mode) for the current block
  • multiple second candidate reconstructed blocks are determined based on multiple candidate reconstructed block vectors corresponding to the reconstructed block vector; here, the multiple second candidate reconstructed blocks represent a candidate BV list used for TIMD mode prediction.
  • a template-based intra mode derivation result corresponding to the current block is determined, thereby determining a prediction block corresponding to the current block.
  • the candidate reconstructed block vector list of the present application can also be applied to other prediction modes, such as the candidate selection process of DIMD, that is, when DIMD selects multiple candidate blocks for weighted fusion, its non-angle mode candidate block can be derived from the block pointed to by BV, and the BV can be derived from the multiple stored BVs described in the present invention.
  • the candidate selection process of TIMD that is, when TIMD selects multiple candidate blocks for weighted fusion, its candidate block can be derived from the block pointed to by BV, and the BV can be derived from the multiple stored BVs described in the present invention.
  • the decoder can obtain multiple candidate reconstructed block vectors based on the reconstructed block vector at a single position to establish the reference BV list required for intra-frame prediction, thereby improving the accuracy of intra-frame prediction and further improving decoding performance.
  • the method further comprises:
  • the prediction mode represents that a prediction block of the current block is determined by performing pixel-wise prediction on a matching reconstructed block obtained based on template matching, determining a value of the first syntax element information
  • the value of the first syntax element information is a first value
  • the one or more first matching reconstructed block vectors and the multiple pixel block vectors are sorted, and top M block vectors with the smallest template matching cost are determined as the multiple candidate reconstructed block vectors; M is a positive integer greater than 1.
  • the method further comprises:
  • the prediction mode indicates that a prediction block of the current block is determined by performing pixel-wise prediction based on a matching reconstructed block obtained by template matching, determining a value of first syntax element information; the first syntax element information is used to indicate whether to perform pixel-wise search;
  • the value of the first syntax element information is a first value
  • parsing the bitstream to determine second syntax element information the second syntax element information is obtained by quadratic curve fitting
  • the second matching reconstructed block vector is determined based on the second syntax element information.
  • the one or more first matched reconstructed block vectors include: matched reconstructed block vectors at integer pixel positions determined based on a search of the first step size and the second step size; and determining the second matched reconstructed block vector based on the second syntax element information includes:
  • the second matching reconstructed block vector is determined from the sub-pixel matching reconstructed block vector list based on the second syntax element information.
  • the method further comprises:
  • a sub-pixel matching reconstructed block vector list is determined based on a third sub-pixel matching reconstructed block vector set corresponding to each first matching reconstructed block vector in the one or more first matching reconstructed block vectors.
  • updating the first sub-pixel matching and reconstructed block vector set according to the second sub-pixel matching and reconstructed block vector to determine the second sub-pixel matching and reconstructed block vector set corresponding to each first matching and reconstructed block vector includes:
  • the first sub-pixel matching and reconstruction block vector with the largest template matching cost in the first sub-pixel matching and reconstruction block vector set is replaced to determine the second sub-pixel matching and reconstruction block vector set;
  • the second sub-pixel matching and reconstructed block vector is added to the first sub-pixel matching and reconstructed block vector set to determine the second sub-pixel matching and reconstructed block vector set.
  • the method further comprises:
  • a subpixel matching reconstructed block vector list is determined based on a set of second subpixel matching reconstructed block vectors corresponding to non-target first matching reconstructed block vectors in the one or more first matching reconstructed block vectors and the target second subpixel matching reconstructed block vector set.
  • performing quadratic curve fitting at a sub-pixel position based on each first matching reconstructed block vector to determine a second sub-pixel matching reconstructed block vector corresponding to each first matching reconstructed block vector includes:
  • first position information by calculating a minimum template matching cost for each integer pixel position in a horizontal direction using a quadratic curve method based on the multiple template matching costs, and determining second position information by calculating a minimum template matching cost for each integer pixel position in a vertical direction using a quadratic curve method;
  • the first position information and the second position information are combined to determine a second sub-pixel matching and reconstruction block vector corresponding to each first matching and reconstruction block vector.
  • the plurality of preset first positions include: a pixel position above the integer pixel position, a pixel position above the integer pixel position, One pixel position below the pixel position, one pixel position to the left of the integer pixel position, and one pixel position to the right of the integer pixel position.
  • combining the first position information and the second position information to determine the second sub-pixel matching reconstructed block vector corresponding to each first matching reconstructed block vector includes:
  • a search is performed within a preset range with a preset step size, and the second matching reconstructed block vector is determined based on the template matching cost of the searched reconstructed block vector.
  • determining the second matching reconstructed block vector from the sub-pixel matching reconstructed block vector list based on the second syntax element information includes:
  • Y is an integer greater than or equal to 1;
  • the second matching reconstructed block vector is determined from the first Y sub-pixel matching reconstructed block vectors.
  • the method further comprises:
  • the first syntax element information is encoded, and the obtained encoded bits are written into a bitstream.
  • the method further comprises:
  • the second syntax element information is encoded, and the obtained encoded bits are written into a bitstream.
  • the method further comprises:
  • the third syntax element information is used to instruct the decoder whether to perform a quadratic curve fitting of a pixel position indicated by the second syntax element information to determine a second matched reconstructed block vector;
  • the third syntax element information is encoded, and the obtained encoded bits are written into a bitstream.
  • the embodiment of the present application further provides a video or image decoding method, as shown in FIG19 , comprising:
  • S301 parse a bitstream to determine a prediction mode corresponding to a first decoding block.
  • S302 Determine one or more first matching reconstructed block vectors corresponding to the first decoded block according to the prediction mode.
  • S303 Determine one or more second matching reconstructed block vectors based on the one or more first matching reconstructed block vectors, and predict the first decoded block based on the prediction mode and the one or more second matching reconstructed block vectors to determine a first prediction block corresponding to the first decoded block.
  • S304 Reconstruct the first decoded block based on the first prediction block, and determine a first reconstructed block corresponding to the first decoded block.
  • S305 Determine multiple candidate reconstructed block vectors corresponding to the first decoded block based on the prediction mode and one or more second matching reconstructed block vectors.
  • S306 Cache the multiple candidate reconstructed block vectors corresponding to the first decoding block in the first decoding information corresponding to the first decoding block.
  • S309 Determine, based on a reconstructed block vector in the at least one reconstructed block vector, a plurality of candidate reconstructed block vectors cached in the decoding information corresponding to the reconstructed block vector.
  • S310 Determine a prediction block corresponding to the second decoding block based on multiple candidate reconstructed block vectors corresponding to the reconstructed block vector.
  • the first decoding block is equivalent to the "current block” in the S101-S105 process embodiment
  • the second decoding block is equivalent to the "current block” in the S201-S204 process embodiment
  • the process of each step in S301-S311 is consistent with the same process description in the aforementioned embodiment and will not be repeated here.
  • the embodiment of the present application further provides a video or image encoding method, which is applied to an encoder, as shown in FIG20 , including:
  • S401 Determine a current block and a prediction mode corresponding to the current block.
  • S402 Determine one or more first matching reconstructed block vectors corresponding to the current block according to the prediction mode.
  • S403 Determine one or more second matching reconstructed block vectors based on the one or more first matching reconstructed block vectors, and predict the current block based on the prediction mode and the one or more second matching reconstructed block vectors to determine a prediction block corresponding to the current block.
  • S404 Determine multiple candidate reconstructed block vectors corresponding to the current block based on the prediction mode and the one or more second matching reconstructed block vectors.
  • multiple candidate reconstructed block vectors corresponding to the current block are cached in the encoding information corresponding to the prediction block of the current block.
  • the method further comprises: reconstructing and encoding the current block based on the prediction block, and writing the obtained coded bits into a bitstream;
  • the prediction mode is encoded, and the obtained encoding bits are written into a bitstream.
  • the method further comprises:
  • the value of the first syntax element information is a first value
  • the second matching reconstructed block vector represents a first matching reconstructed block vector having a minimum template matching cost with the current block among the one or more first matching reconstructed block vectors
  • the one or more first matching reconstructed block vectors and the multiple pixel-wise block vectors are sorted based on a template matching cost with the current block, and top M block vectors with the smallest template matching cost are determined as the multiple candidate reconstructed block vectors; M is a positive integer greater than 1; and the template matching cost includes at least one of a sum of absolute errors and a sum of absolute transform errors.
  • the method further comprises:
  • the first syntax element information is encoded, and the resulting encoded bits are written into the bitstream.
  • the processing on the encoder side is consistent with the description of the same process on the decoder side and is not repeated here.
  • an embodiment of the present application further provides a video or image encoding method, which is applied to an encoder, as shown in FIG21 , including:
  • S501 Determine a current block and a prediction mode corresponding to the current block.
  • S502 Determine, according to the prediction mode, at least one reconstructed block vector at at least one spatially adjacent and/or non-adjacent position corresponding to the current block.
  • S503 Based on a reconstructed block vector in the at least one reconstructed block vector, determine a plurality of candidate reconstructed block vectors cached in the coding information corresponding to the reconstructed block vector.
  • S504 Determine a prediction block corresponding to the current block based on multiple candidate reconstructed block vectors corresponding to the reconstructed block vector.
  • an embodiment of the present application further provides a video or image encoding method, which is applied to an encoder, as shown in FIG22 , including:
  • S602 Determine one or more first matching reconstructed block vectors corresponding to the first coding block according to the prediction mode
  • S603 Determine one or more second matching reconstructed block vectors based on the one or more first matching reconstructed block vectors, and predict the first coding block based on the prediction mode and the one or more second matching reconstructed block vectors to determine a first prediction block corresponding to the first coding block.
  • S604 Reconstruct the first coding block based on the first prediction block to determine a first reconstructed block corresponding to the first coding block;
  • S607 Determine the second coding block and the prediction mode corresponding to the second coding block
  • S609 Determine, based on a reconstructed block vector in the at least one reconstructed block vector, a plurality of candidate reconstructed block vectors cached in the coding information corresponding to the reconstructed block vector;
  • S611 Reconstruct a second coding block based on the second prediction block, and determine a second reconstructed block corresponding to the second coding block.
  • the embodiment of the present application provides a decoder 1, comprising: a first parsing part 10, a fourth prediction part 11, a fourth determination part 12, and a third cache part 13, wherein:
  • the first parsing part 10 is configured to parse the code stream and determine the prediction mode corresponding to the current block;
  • the fourth prediction section 11 is configured to determine, according to the prediction mode, one or more first matching reconstructed block vectors corresponding to the current block; determine one or more second matching reconstructed block vectors based on the one or more first matching reconstructed block vectors; and predict the current block based on the prediction mode and the one or more second matching reconstructed block vectors to determine a predicted block corresponding to the current block;
  • the fourth determining part 12 is configured to determine a plurality of candidate reconstructed block vectors corresponding to the current block based on the prediction mode and the one or more second matching reconstructed block vectors;
  • the third buffer part 13 is configured to buffer a plurality of candidate reconstructed block vectors corresponding to the current block.
  • the fourth prediction part 11 is further configured to determine multiple candidate reconstructed block vectors corresponding to the current block based on multiple second matching reconstructed block vectors when the prediction mode representation determines the prediction block of the current block by weighted fusion of the matching reconstructed blocks obtained based on template matching.
  • the fourth prediction part 11 is further configured to determine at least one initial candidate reconstructed block vector by at least one of the following methods when the prediction mode characterization is based on filtering a matching reconstructed block obtained by template matching to determine a prediction block for the current block:
  • the second matched reconstructed block vector represents a matched reconstructed block vector having the minimum template matching cost with the current block among the one or more first matched reconstructed block vectors;
  • a plurality of candidate reconstructed block vectors corresponding to the current block are determined.
  • the fourth prediction part 11 is further configured to determine at least one initial candidate reconstructed block vector by at least one of the following methods when the prediction mode characterizes that a prediction block of the current block is determined by performing pixel-wise prediction on a matching reconstructed block obtained based on template matching:
  • the second matched reconstructed block vector represents a first matched reconstructed block vector having the smallest template matching cost with the current block among the one or more first matched reconstructed block vectors;
  • a plurality of candidate reconstructed block vectors corresponding to the current block are determined.
  • the fourth prediction part 11 is further configured to determine the multiple second matching reconstructed block vectors as multiple candidate reconstructed block vectors corresponding to the current block.
  • the fourth prediction part 11 is further configured to determine multiple candidate reconstructed block vectors corresponding to the current block from the multiple second matching reconstructed block vectors based on multiple first template matching costs corresponding to the multiple second matching reconstructed block vectors.
  • the fourth prediction part 11 is further configured to determine the first X first template matching costs with the smallest template matching costs among the multiple first template matching costs; X is an integer greater than 1;
  • the X second matching reconstructed block vectors corresponding to the first X first template matching costs are determined as multiple candidate reconstructed block vectors corresponding to the current block.
  • the fourth prediction part 11 is further configured to determine, among the multiple first template matching costs, X second matching reconstructed block vectors whose template matching costs are less than a preset first cost threshold as multiple candidate reconstructed block vectors corresponding to the current block; X is an integer greater than 1.
  • the fourth prediction part 11 is further configured to determine, by sorting the multiple first template matching costs from small to large, a second template matching cost among the multiple first template matching costs, the increase value of which compared with the previous template matching cost is greater than a first preset growth threshold;
  • the second matching reconstructed block vector corresponding to the first template matching cost before the second template matching cost is determined as the multiple candidate reconstructed block vectors corresponding to the current block.
  • the fourth prediction part 11 is further configured to determine, among the plurality of second matching reconstructed block vectors, first X second matching reconstructed block vectors with high weights as the plurality of candidate reconstructed block vectors corresponding to the current block; X is an integer greater than 1;
  • the weight represents the weight corresponding to the second matching reconstructed block when weighted fusion prediction is performed on multiple second matching reconstructed blocks corresponding to the multiple second matching reconstructed block vectors.
  • the at least one direction includes at least one of up, down, left, right, upper left, upper right, lower left, and lower right.
  • the fourth prediction part 11 is further configured to determine the first X filter coefficients with the largest filter coefficients among the multiple filter coefficients determined in the process of performing filter prediction on the matched reconstructed block obtained based on template matching; X is an integer greater than 1;
  • the second matching reconstructed block vector is shifted in at least one direction according to pixel positions corresponding to the first X filter coefficients to determine the at least one initial candidate reconstructed block vector.
  • the fourth prediction part 11 is further configured to search up, down, left, right, upper left, upper right, lower left, and lower right based on the second matched reconstructed block vector with a step size of 1/2 pixel to determine a plurality of first sub-pixel block vectors;
  • the plurality of second sub-pixel block vectors are determined as the plurality of sub-pixel block vectors.
  • the fourth prediction part 11 is further configured to determine the at least one initial candidate reconstructed block vector as a plurality of candidate reconstructed block vectors corresponding to the current block.
  • the fourth prediction part 11 is further configured to determine at least one third template matching cost between the at least one initial candidate reconstructed block vector and the current block;
  • a plurality of candidate reconstructed block vectors corresponding to the current block are determined from the at least one initial candidate reconstructed block vector.
  • the fourth prediction part 11 is further configured to determine the first X template matching costs with the smallest template matching cost among the at least one third template matching cost; X is an integer greater than 1;
  • the X reconstructed block vectors corresponding to the first X template matching costs are determined as multiple candidate reconstructed block vectors corresponding to the current block.
  • the fourth prediction part 11 is further configured to determine, among the at least one third template matching cost, X reconstructed block vectors whose template matching cost is less than a preset second cost threshold as multiple candidate reconstructed block vectors corresponding to the current block; X is an integer greater than 1.
  • the fourth prediction part 11 is further configured to determine the value of X according to the prediction mode.
  • the fourth prediction part 11 is further configured to determine a fourth template matching cost having an increase value greater than a second preset growth threshold compared to the previous template matching cost by sorting the at least one third template matching cost from small to large;
  • the reconstructed block vector corresponding to the third template matching cost before the fourth template matching cost is determined as a plurality of candidate reconstructed block vectors corresponding to the current block.
  • the template matching cost includes at least one of a sum of absolute differences (SAD) and a sum of absolute transform differences (SATD).
  • SAD sum of absolute differences
  • SATD sum of absolute transform differences
  • the third cache portion 13 is further configured to cache multiple candidate reconstructed block vectors corresponding to the current block in the decoding information corresponding to the current block.
  • the decoder 1 further includes a third reconstruction part, which is configured to reconstruct the current block based on the prediction block corresponding to the current block, and determine the reconstructed block corresponding to the current block.
  • the fourth determining part 12 is further configured to determine the template type corresponding to the current block when the prediction mode characterization determines the prediction block corresponding to the current block based on the reconstructed block vector determined based on template matching;
  • a search area corresponding to the current block is determined, and based on the template samples corresponding to the current block, a search is performed in the search area to determine the one or more first matching reconstructed block vectors.
  • the one or more first matching reconstructed block vectors correspond to one or more first matching reconstructed blocks; One or more first matching reconstructed blocks are included in the current image corresponding to the current block.
  • the fourth determining portion 12 is further configured to search the search area with a first step length based on the template sample corresponding to the current block to determine a first step length block vector list; the first step length block vector list includes one or more matching reconstructed block vectors searched based on the first step length;
  • the search area is searched with a second step length to determine the one or more first matching reconstructed block vectors; the second step length is smaller than the first step length.
  • the fourth determining part 12 is further configured to search the search area with the first step length to determine the first step length candidate reconstructed block;
  • the first step block vector list is determined based on template matching costs corresponding to one or more of the first step candidate reconstructed blocks; wherein the template matching cost includes at least one of a sum of absolute differences (SAD) and a sum of absolute transform differences (SATD).
  • SAD sum of absolute differences
  • SATD sum of absolute transform differences
  • the fourth determining part 12 is further configured to determine a first step length initial block vector list based on template matching costs corresponding to one or more of the first step length candidate reconstructed blocks;
  • the first step length initial block vector list is updated based on the one or more candidate reconstructed block vectors corresponding to each position to determine the first step length block vector list.
  • the fourth determining part 12 is further configured to parse the bitstream to determine the value of the first syntax element information
  • the value of the first syntax element information is a first value, determining a second matched reconstructed block vector from the one or more first matched reconstructed block vectors based on a template matching cost, and performing sub-pixel precision and direction traversal based on the second matched reconstructed block vector to determine a plurality of sub-pixel block vectors;
  • the second matched reconstructed block vector represents a first matched reconstructed block vector having a minimum template matching cost with the current block among the one or more first matched reconstructed block vectors;
  • the one or more first matching reconstructed block vectors and the multiple pixel block vectors are sorted, and the top M block vectors with the smallest template matching cost are determined as the multiple candidate reconstructed block vectors; M is a positive integer greater than 1; the template matching cost includes: at least one of the sum of absolute differences (SAD) and the sum of absolute transform differences (SATD).
  • SAD sum of absolute differences
  • SATD sum of absolute transform differences
  • the fourth determining portion 12 is further configured to, when the prediction mode indicates that a prediction block of the current block is determined by performing pixel-wise prediction based on a matching reconstructed block obtained by template matching, parse the bitstream to determine a value of first syntax element information; the first syntax element information is used to indicate whether to perform a pixel-wise search;
  • the value of the first syntax element information is a first value
  • parsing the bitstream to determine second syntax element information the second syntax element information is obtained by quadratic curve fitting
  • the second matching reconstructed block vector is determined based on the second syntax element information.
  • the one or more first matched reconstructed block vectors include matched reconstructed block vectors at integer pixel positions determined based on a search of the first step size and the second step size; the fourth determining portion 12 is further configured to perform a traversal search of at least one sub-pixel precision at each of at least one preset position based on each of the one or more first matched reconstructed block vectors to determine a set of first sub-pixel matched reconstructed block vectors corresponding to each first matched reconstructed block vector;
  • the second matching reconstructed block vector is determined from the sub-pixel matching reconstructed block vector list based on the second syntax element information.
  • each first matching reconstruction block vector Based on each first matching reconstruction block vector, a quadratic curve fitting of the pixel position is performed to determine each first matching reconstruction block vector.
  • the second sub-pixel corresponding to the building block vector matches the reconstructed block vector;
  • a sub-pixel matching reconstructed block vector list is determined based on a third sub-pixel matching reconstructed block vector set corresponding to each first matching reconstructed block vector in the one or more first matching reconstructed block vectors.
  • the fourth determining part 12 is further configured to use the second sub-pixel matching and reconstruction block vector to replace the first sub-pixel matching and reconstruction block vector with the largest template matching cost in the first sub-pixel matching and reconstruction block vector set, to determine the second sub-pixel matching and reconstruction block vector set;
  • the second sub-pixel matching and reconstructed block vector is added to the first sub-pixel matching and reconstructed block vector set to determine the second sub-pixel matching and reconstructed block vector set.
  • the fourth determining portion 12 is further configured to perform quadratic curve fitting at a sub-pixel position based on each first matching reconstructed block vector of the one or more first matching reconstructed block vectors to determine a second sub-pixel matching reconstructed block vector corresponding to each first matching reconstructed block vector;
  • a target first matching reconstructed block vector is determined from the first matching reconstructed block vector list, and quadratic curve fitting is performed at a sub-pixel position based on the target first matching reconstructed block vector to determine a second sub-pixel matching reconstructed block vector corresponding to the target first matching reconstructed block vector.
  • the fourth determination part 12 is further configured to determine, based on the integer pixel position of each first matching reconstructed block vector, a plurality of template matching costs corresponding to the reconstructed block vectors at a plurality of preset first positions, with the integer pixel position as the center; determine the first position information based on the plurality of template matching costs by calculating the minimum template matching cost of each integer pixel position in the horizontal direction using the quadratic curve method, and determine the second position information by calculating the minimum template matching cost of each integer pixel position in the vertical direction using the quadratic curve method; and determine the second sub-pixel matching reconstructed block vector corresponding to each first matching reconstructed block vector by combining the first position information and the second position information.
  • the plurality of preset first positions include: a pixel position above the integer pixel position, a pixel position below the integer pixel position, a pixel position to the left of the integer pixel position, and a pixel position to the right of the integer pixel position.
  • the fourth determining part 12 is further configured to determine a matching reconstructed block vector for the second sub-pixel based on the reconstructed block vector corresponding to the first position information and the second position information;
  • a search is performed within a preset range with a preset step size, and the second matching reconstructed block vector is determined based on the template matching cost of the searched reconstructed block vector.
  • the fourth determination part 12 is further configured to determine the sub-pixel matching reconstructed block vector list based on the template matching cost, using the second sub-pixel matching reconstructed block vector in the second sub-pixel matching reconstructed block vector list to replace the corresponding first sub-pixel matching reconstructed block vector in the first sub-pixel matching reconstructed block vector list.
  • the fourth determining part 12 is further configured to determine the first Y sub-pixel matching reconstructed block vectors with the smallest template matching cost from the sub-pixel matching reconstructed block vector list; Y is an integer greater than or equal to 1;
  • the second matching reconstructed block vector is determined from the first Y sub-pixel matching reconstructed block vectors.
  • the fourth determining part 12 is further configured to parse the code stream to determine a value of the third syntax element information; when the value of the third syntax element information represents a list of sub-pixel matching reconstructed block vectors determined by performing quadratic curve fitting at sub-pixel positions, perform quadratic curve fitting at sub-pixel positions based on a first matching reconstructed block vector among the one or more first matching reconstructed block vectors to determine a second sub-pixel matching reconstructed block vector corresponding to the first matching reconstructed block vector.
  • the embodiment of the present application provides a decoder 2, including: a second parsing part 20 and a fifth prediction part 21, wherein:
  • the second parsing part 20 is configured to parse the code stream and determine the prediction mode corresponding to the current block;
  • the fifth prediction part 21 is configured to determine at least one reconstructed block vector at at least one position spatially adjacent and/or non-adjacent to the current block according to the prediction mode; determine multiple candidate reconstructed block vectors cached in the decoding information corresponding to the reconstructed block vector based on the reconstructed block vector in the at least one reconstructed block vector; and determine multiple candidate reconstructed block vectors cached in the decoding information corresponding to the reconstructed block vector based on the multiple candidate reconstructed block vectors.
  • a block vector is reconstructed to determine a predicted block corresponding to the current block.
  • the prediction mode includes: performing prediction based on a reconstructed block corresponding to a candidate reconstructed block vector to determine a prediction mode of a prediction block corresponding to the current block.
  • the at least one spatially adjacent and/or non-adjacent position includes:
  • the fifth prediction part 21 is further configured to determine, for each reconstructed block vector in the at least one reconstructed block vector, a prediction block corresponding to the each reconstructed block vector, and determine whether the prediction block is obtained by a prediction mode based on the candidate reconstructed block vector;
  • the plurality of cached candidate reconstructed block vectors are read from a preset data structure of decoding information corresponding to the prediction block.
  • the fifth prediction part 21 is further configured to determine a first step length initial block vector list corresponding to the current block when the prediction mode characterizes the determination of the prediction block corresponding to the current block based on the reconstructed block vector determined based on template matching;
  • the fifth prediction part 21 is further configured to determine a candidate reconstructed block vector list based on a plurality of candidate reconstructed block vectors corresponding to each reconstructed block vector in the at least one reconstructed block vector;
  • the first step long initial block vector list is updated to determine a first step long block vector list; the first step long block vector list includes one or more matching reconstructed block vectors;
  • a prediction block corresponding to the current block is determined based on the one or more matching reconstructed block vectors.
  • the fifth prediction part 21 is further configured to determine a template type corresponding to the current block
  • a search area corresponding to the current block is determined, and based on the template sample corresponding to the current block, a search is performed in the search area with a first step length to determine a first step length initial block vector list corresponding to the current block.
  • the fifth prediction part 21 is further configured to search the search area with the first step length to determine the first step length candidate reconstruction block; perform template matching cost calculation based on the template samples corresponding to the first step length candidate reconstruction block and the template samples corresponding to the current block to determine the template matching cost corresponding to the first step length candidate reconstruction block; determine the first step length initial block vector list corresponding to the current block based on the template matching costs corresponding to one or more of the first step length candidate reconstruction blocks; wherein the template matching cost includes: at least one of: the sum of absolute errors (SAD) and the sum of absolute transform errors (SATD).
  • SAD sum of absolute errors
  • SATD sum of absolute transform errors
  • the fifth prediction part 21 is further configured to sequentially determine the availability of each reconstructed block vector in the at least one reconstructed block vector;
  • the candidate reconstructed block vector list is determined based on a plurality of candidate reconstructed block vectors corresponding to each reconstructed block vector.
  • the fifth prediction part 21 is further configured to determine the availability of the first candidate reconstructed block vector corresponding to each reconstructed block vector;
  • the first initial candidate reconstructed block vector list is updated according to available non-first candidate reconstructed block vectors in the at least one non-first candidate reconstructed block vector corresponding to each reconstructed block vector, until a list length of the first initial candidate reconstructed block vector list reaches the preset length threshold, or until processing of the at least one reconstructed block vector is completed, and the first initial candidate reconstructed block vector list is determined as the candidate reconstructed block vector list.
  • the fifth prediction part 21 is further configured to determine the availability of each candidate reconstructed block vector among the multiple candidate reconstructed block vectors corresponding to each reconstructed block vector;
  • a second initial candidate reconstructed block vector list is determined or updated based on available candidate reconstructed block vectors among the multiple candidate reconstructed block vectors corresponding to each reconstructed block vector, until a list length of the second initial candidate reconstructed block vector list reaches the preset length threshold, or until processing of the at least one reconstructed block vector is completed, and the second initial candidate reconstructed block vector list is determined as the candidate reconstructed block vector list.
  • the fifth prediction part 21 is further configured to, when the similarity between the available non-first candidate reconstructed block vector and the candidate reconstructed block vector in the first initial candidate reconstructed block vector list is less than a preset similarity threshold, Adding the available non-first candidate reconstructed block vector to the first initial candidate reconstructed block vector list;
  • the determining or updating a second initial candidate reconstructed block vector list based on available candidate reconstructed block vectors among the multiple candidate reconstructed block vectors corresponding to each reconstructed block vector includes:
  • the available candidate reconstructed block vector is added to the second initial candidate reconstructed block vector list.
  • the fifth prediction part 21 is further configured to determine a maximum template matching cost between a candidate reconstructed block vector in the candidate reconstructed block vector list and the current block when the list length of the candidate reconstructed block vector list reaches a preset length threshold;
  • the candidate reconstructed block vector corresponding to the fifth template matching cost is used to replace the candidate reconstructed block vector corresponding to the maximum template matching cost in the candidate reconstructed block vector list, until each candidate reconstructed block vector corresponding to each position in the first position is traversed.
  • the fifth prediction part 21 is further configured to determine a sixth template matching cost between the available candidate reconstructed block vector and the current block when adding the available candidate reconstructed block vector to the first initial candidate reconstructed block vector list or the second initial candidate reconstructed block vector list;
  • the available candidate reconstructed block vector includes: each available candidate reconstructed block vector in the at least one available candidate reconstructed block vector, or includes: a non-first available candidate reconstructed block vector in the at least one non-first candidate reconstructed block vector.
  • the fifth prediction section 21 is further configured to determine the one or more matching reconstructed block vectors included in the first step length block vector list as one or more first matching reconstructed block vectors; or to determine the one or more first matching reconstructed block vectors by searching based on the one or more matching reconstructed block vectors with a second step length, wherein the second step length is smaller than the first step length;
  • One or more second matching reconstructed block vectors are determined based on the one or more first matching reconstructed block vectors, and the current block is predicted based on the prediction mode and the one or more second matching reconstructed block vectors to determine a prediction block corresponding to the current block.
  • the fifth prediction part 21 is further configured to determine a plurality of first candidate reconstructed blocks based on a plurality of candidate reconstructed block vectors corresponding to the reconstructed block vector, when the prediction mode represents a decoding-side derived prediction mode for the current block based on a non-angular mode;
  • a prediction block corresponding to the current block is determined.
  • the fifth prediction part 21 is further configured to determine a plurality of second candidate reconstructed blocks based on a plurality of candidate reconstructed block vectors corresponding to the reconstructed block vector when the prediction mode indicates that a template-based intra prediction mode is derived for the current block;
  • a prediction block corresponding to the current block is determined.
  • the embodiment of the present application provides a decoder 3, comprising: a third parsing part 30, a sixth prediction part 31, a fifth determination part 32, a second reconstruction part 33, and a fourth cache part 34, wherein:
  • the third parsing part 30 is configured to parse the code stream and determine the prediction mode corresponding to the first decoding block
  • the sixth prediction section 31 is configured to determine, according to the prediction mode, one or more first matching reconstructed block vectors corresponding to the first decoded block; determine one or more second matching reconstructed block vectors based on the one or more first matching reconstructed block vectors; and predict the first decoded block based on the prediction mode and the one or more second matching reconstructed block vectors to determine a first prediction block corresponding to the first decoded block;
  • the second reconstruction part 33 is configured to reconstruct the first decoded block based on the first prediction block and determine a first reconstructed block corresponding to the first decoded block;
  • the fifth determining part 32 is configured to determine a plurality of candidate reconstructed block vectors corresponding to the first decoded block based on the prediction mode and the one or more second matching reconstructed block vectors;
  • the fourth cache part 34 is configured to cache the multiple candidate reconstructed block vectors corresponding to the first decoding block in the first decoding information corresponding to the first decoding block;
  • the third parsing part 30 is further configured to parse the code stream and determine the prediction mode corresponding to the second decoding block;
  • the sixth prediction part 31 is further configured to determine, based on the prediction mode corresponding to the second decoding block, at least one reconstructed block vector at at least one spatially adjacent and/or non-adjacent position corresponding to the second decoding block; the at least one spatially adjacent and/or non-adjacent position corresponding to the second decoding block includes the position where the first decoding block is located; based on the reconstructed block vector in the at least one reconstructed block vector, determine multiple candidate reconstructed block vectors cached in the decoding information corresponding to the reconstructed block vector; based on the multiple candidate reconstructed block vectors corresponding to the reconstructed block vector, determine the prediction block corresponding to the second decoding block.
  • the second reconstruction part 33 is further configured to reconstruct the second decoding block based on the second prediction block and determine a second reconstructed block corresponding to the second decoding block.
  • the embodiment of the present application further provides a decoder, including:
  • the first memory 14 stores a computer program that can be run on the first processor 15.
  • the first processor 15 executes the program, the video or image decoding method provided in the embodiment of the present application is implemented.
  • the first processor 15 can be implemented by software, hardware, firmware or a combination thereof, and can use circuits, single or multiple application specific integrated circuits (ASICs), single or multiple general integrated circuits, single or multiple microprocessors, single or multiple programmable logic devices, or a combination of the aforementioned circuits or devices, or other suitable circuits or devices, so that the first processor 15 can execute the corresponding steps of the video or image decoding method provided in the embodiment of the present application.
  • ASICs application specific integrated circuits
  • microprocessors single or multiple programmable logic devices
  • a combination of the aforementioned circuits or devices or other suitable circuits or devices
  • the embodiment of the present application provides an encoder 4, as shown in FIG27 , including: a first determination part 40, a first prediction part 41, and a first cache part 42, wherein:
  • a first determining portion 40 configured to determine a current block and a prediction mode corresponding to the current block
  • the first prediction section 41 is configured to determine one or more first matching reconstructed block vectors corresponding to the current block according to the prediction mode; determine one or more second matching reconstructed block vectors based on the one or more first matching reconstructed block vectors; and predict the current block based on the prediction mode and the one or more second matching reconstructed block vectors to determine a prediction block corresponding to the current block;
  • the first determining portion 40 is further configured to determine a plurality of candidate reconstructed block vectors corresponding to the current block based on the prediction mode and the one or more second matching reconstructed block vectors;
  • the first buffer part 42 is configured to buffer a plurality of candidate reconstructed block vectors corresponding to the current block.
  • the embodiment of the present application provides an encoder 5, as shown in FIG28 , including: a second determination part 50 and a second prediction part 51 , wherein:
  • the second determining part 50 is configured to determine the current block and the prediction mode corresponding to the current block
  • the second prediction part 51 is configured to determine, according to the prediction mode, at least one reconstructed block vector at at least one spatially adjacent and/or non-adjacent position corresponding to the current block; and based on a reconstructed block vector in the at least one reconstructed block vector, determine multiple candidate reconstructed block vectors cached in the encoding information corresponding to the reconstructed block vector; and determine the prediction block corresponding to the current block based on the multiple candidate reconstructed block vectors corresponding to the reconstructed block vector.
  • the embodiment of the present application provides an encoder 6, as shown in FIG29 , comprising: a third determination part 60, a third prediction part 61, a first reconstruction part 62, and a second buffer part 63, wherein:
  • the third determining part 60 is configured to determine the first coding block and the prediction mode corresponding to the first coding block;
  • the third prediction section 61 is configured to determine one or more first matching reconstructed block vectors corresponding to the first coding block according to the prediction mode; determine one or more second matching reconstructed block vectors based on the one or more first matching reconstructed block vectors; and predict the first coding block based on the prediction mode and the one or more second matching reconstructed block vectors to determine a first prediction block corresponding to the first coding block;
  • the first reconstruction part 62 is configured to reconstruct the first coding block based on the first prediction block and determine a first reconstructed block corresponding to the first coding block;
  • the third determining portion 60 is further configured to determine a plurality of candidate reconstructed block vectors corresponding to the first coding block based on the prediction mode and the one or more second matching reconstructed block vectors;
  • the second cache part 63 is configured to cache the multiple candidate reconstructed block vectors corresponding to the first coding block in the first coding information corresponding to the first coding block;
  • the third prediction part 61 is further configured to determine a prediction mode corresponding to the second coding block and the second coding block; determine at least one reconstructed block vector at at least one position corresponding to the second coding block in space adjacent and/or non-adjacent according to the prediction mode corresponding to the second coding block; at least one position corresponding to the second coding block in space adjacent and/or non-adjacent includes the first coding block.
  • the first reconstruction part 62 is further configured to reconstruct the second coding block based on the second prediction block and determine a second reconstructed block corresponding to the second coding block.
  • an encoder including:
  • the second memory 25 stores a computer program that can be run on the second processor 26.
  • the second processor 26 executes the program, the video or image encoding method provided in the embodiment of the present application is implemented.
  • An embodiment of the present application provides a code stream, which is generated by bit encoding based on information to be encoded; wherein the information to be encoded includes at least coded bits of a current block and a prediction mode of the current block; the coded bits of the current block are obtained by reconstructing and encoding the current block based on a prediction block corresponding to the current block; multiple candidate reconstructed block vectors are determined by the following process: determining the current block and the prediction mode corresponding to the current block; determining one or more first matching reconstructed block vectors corresponding to the current block based on the prediction mode; determining one or more second matching reconstructed block vectors based on the one or more first matching reconstructed block vectors, and predicting the current block based on the prediction mode and the one or more second matching reconstructed block vectors to determine the prediction block corresponding to the current block; and determining and caching multiple candidate reconstructed block vectors corresponding to the current block based on the prediction mode and the one or more second matching reconstructed block vectors
  • An embodiment of the present application provides a readable storage medium, which is a computer-readable storage medium on which a computer program is stored.
  • the computer program is executed by a first processor, the video or image decoding method provided by the embodiment of the present application is implemented; or, when the computer program is executed by a second processor, the video or image encoding method provided by the embodiment of the present application is implemented.
  • each unit may exist physically separately, or two or more units may be integrated into a single unit.
  • the aforementioned integrated units may be implemented in the form of hardware or in the form of software functional modules.
  • the integrated unit is implemented in the form of a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium.
  • the technical solution of this embodiment is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product.
  • the computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in this embodiment.
  • the aforementioned computer-readable storage media include: ferromagnetic random access memory (FRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic surface storage, optical disc, or compact disc read-only memory (CD-ROM), and other media that can store program codes, which are not limited in the embodiments of the present disclosure.
  • FRAM ferromagnetic random access memory
  • ROM read-only memory
  • PROM programmable read-only memory
  • EPROM erasable programmable read-only memory
  • EEPROM electrically erasable programmable read-only memory
  • flash memory magnetic surface storage
  • optical disc optical disc
  • CD-ROM compact disc read-only memory
  • the embodiments of the present application provide a video or image encoding and decoding method, a decoder, an encoder and a readable storage medium.
  • the decoder determines the prediction mode corresponding to the current block by parsing the code stream; determines one or more first matching reconstruction block vectors corresponding to the current block according to the prediction mode; determines one or more second matching reconstruction block vectors based on the one or more first matching reconstruction block vectors, and predicts the current block based on the prediction mode and the one or more second matching reconstruction block vectors to determine the prediction block corresponding to the current block; determines multiple candidate reconstruction block vectors corresponding to the current block based on the prediction mode and the one or more second matching reconstruction block vectors; and caches multiple candidate reconstruction block vectors corresponding to the current block.
  • the encoder determines the current block and the prediction mode corresponding to the current block; determines one or more first matching reconstruction block vectors corresponding to the current block according to the prediction mode; determines one or more second matching reconstruction block vectors based on the one or more first matching reconstruction block vectors, and predicts the current block based on the prediction mode and the one or more second matching reconstruction block vectors to determine the current block.
  • the method comprises the following steps: determining a prediction block corresponding to a block; determining a plurality of candidate reconstructed block vectors corresponding to the current block based on the prediction mode and one or more second matching reconstructed block vectors; and caching the plurality of candidate reconstructed block vectors corresponding to the current block.
  • a plurality of candidate reconstructed block vectors corresponding to the current block can be derived and cached, thereby increasing the number of BVs stored in each block.
  • intra-frame codec prediction modes such as the intra-frame template matching prediction (IntraTMP) mode or the intra-frame block copy (Intra Block Copy, IBC) mode
  • the accuracy of intra-frame prediction can be improved, thereby improving codec performance.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

Des modes de réalisation de la présente demande concernent un procédé de décodage pour une vidéo ou une image, un décodeur, un codeur et un support de stockage lisible par ordinateur, qui peuvent améliorer la précision de prédiction intra, ce qui permet d'améliorer les performances de codage et de décodage. Le procédé de décodage pour une vidéo ou une image consiste en : l'analyse d'un flux de code, et la détermination d'un mode de prédiction correspondant à un bloc courant ; sur la base du mode de prédiction, la détermination d'un ou de plusieurs premiers vecteurs de blocs reconstruits correspondants correspondant au bloc courant ; sur la base du ou des premiers vecteurs de blocs reconstruits correspondants, la détermination d'un ou de plusieurs seconds vecteurs de blocs reconstruits correspondants, la prédiction du bloc courant sur la base du mode de prédiction et du ou des seconds vecteurs de blocs reconstruits correspondants, et la détermination d'un bloc de prédiction correspondant au bloc courant ; sur la base du mode de prédiction et du ou des seconds vecteurs de blocs reconstruits correspondants, la détermination d'une pluralité de vecteurs de blocs reconstruits candidats correspondant au bloc courant ; et la mise en cache de la pluralité de vecteurs de blocs reconstruits candidats correspondant au bloc courant.
PCT/CN2024/088460 2024-04-17 2024-04-17 Procédé de codage et procédé de décodage pour vidéo ou image, décodeur, codeur et support de stockage lisible Pending WO2025217860A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2024/088460 WO2025217860A1 (fr) 2024-04-17 2024-04-17 Procédé de codage et procédé de décodage pour vidéo ou image, décodeur, codeur et support de stockage lisible

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2024/088460 WO2025217860A1 (fr) 2024-04-17 2024-04-17 Procédé de codage et procédé de décodage pour vidéo ou image, décodeur, codeur et support de stockage lisible

Publications (1)

Publication Number Publication Date
WO2025217860A1 true WO2025217860A1 (fr) 2025-10-23

Family

ID=97402754

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/088460 Pending WO2025217860A1 (fr) 2024-04-17 2024-04-17 Procédé de codage et procédé de décodage pour vidéo ou image, décodeur, codeur et support de stockage lisible

Country Status (1)

Country Link
WO (1) WO2025217860A1 (fr)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022104498A1 (fr) * 2020-11-17 2022-05-27 Oppo广东移动通信有限公司 Procédé de prédiction intra-trame, codeur, décodeur et support de stockage informatique
CN114666581A (zh) * 2018-03-07 2022-06-24 腾讯美国有限责任公司 视频编解码的方法和装置
CN114827594A (zh) * 2018-10-05 2022-07-29 腾讯美国有限责任公司 视频编码方法、装置、计算机设备及存储介质

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114666581A (zh) * 2018-03-07 2022-06-24 腾讯美国有限责任公司 视频编解码的方法和装置
CN114827594A (zh) * 2018-10-05 2022-07-29 腾讯美国有限责任公司 视频编码方法、装置、计算机设备及存储介质
WO2022104498A1 (fr) * 2020-11-17 2022-05-27 Oppo广东移动通信有限公司 Procédé de prédiction intra-trame, codeur, décodeur et support de stockage informatique

Similar Documents

Publication Publication Date Title
JP7525178B2 (ja) 適応的動きベクトル解像度を用いるビデオ信号処理方法及び装置
KR101403343B1 (ko) 부화소 움직임 추정을 이용한 인터 예측 부호화, 복호화방법 및 장치
JP6242984B2 (ja) 動画符号化装置
JP7233218B2 (ja) ビデオコーディングのためにブロックの複数のクラスのためのフィルタをマージすること
CN102150429B (zh) 用于使用构建的参考帧进行视频编码的系统及方法
TW201933866A (zh) 改良的解碼器側之運動向量推導
US20160073107A1 (en) Method and apparatus for video encoding/decoding using intra prediction
JP2018520581A (ja) スライスレベルのイントラブロックコピーおよび他のビデオコーディングの改善
TW201639368A (zh) 於視訊寫碼中針對子區塊推導運動資訊
US12439029B2 (en) Video encoding and decoding using deep learning based inter prediction
WO2023274360A1 (fr) Utilisation d'une unité de prédiction récursive dans un codage vidéo
CN118614060A (zh) 利用自适应混合的几何分区模式的方法和设备
CN117242479A (zh) 用于视频处理的方法、装置和介质
CN114390289B (zh) 参考像素候选列表构建方法、装置、设备及存储介质
US12192445B2 (en) Inter prediction method based on variable coefficient deep learning
CN114079782B (zh) 视频图像重建方法、装置、计算机设备及存储介质
US20150264367A1 (en) Systems and methods for low complexity encoding and background detection
CN120548704A (zh) 编解码方法、码流、编码器、解码器以及存储介质
US20230269399A1 (en) Video encoding and decoding using deep learning based in-loop filter
WO2023274302A1 (fr) Unité de prédiction récursive dans un codage vidéo
WO2025217860A1 (fr) Procédé de codage et procédé de décodage pour vidéo ou image, décodeur, codeur et support de stockage lisible
US12363279B2 (en) Method for predicting quantization parameter used in a video encoding/decoding apparatus
CN120937354A (zh) 视频编解码方法、解码器、编码器及计算机可读存储介质
US12537962B2 (en) Video encoding/decoding method and apparatus
US12452425B2 (en) Method and apparatus for video coding using spiral scan order

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24935412

Country of ref document: EP

Kind code of ref document: A1