WO2026021082A1 - Procédé et appareil de codage vidéo - Google Patents

Procédé et appareil de codage vidéo

Info

Publication number
WO2026021082A1
WO2026021082A1 PCT/CN2025/102686 CN2025102686W WO2026021082A1 WO 2026021082 A1 WO2026021082 A1 WO 2026021082A1 CN 2025102686 W CN2025102686 W CN 2025102686W WO 2026021082 A1 WO2026021082 A1 WO 2026021082A1
Authority
WO
WIPO (PCT)
Prior art keywords
encoded
videos
video
target
sample
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2025/102686
Other languages
English (en)
Chinese (zh)
Inventor
邓德鑫
吴家豪
李凯
陈颖
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Taobao China Software Co Ltd
Original Assignee
Taobao China Software Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Taobao China Software Co Ltd filed Critical Taobao China Software Co Ltd
Publication of WO2026021082A1 publication Critical patent/WO2026021082A1/fr
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234309—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 or from Quicktime to Realvideo
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
    • H04N21/2662—Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440218—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462—Content or additional data management e.g. creating a master electronic programme guide from data received from the Internet and a Head-end or controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4621—Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen

Definitions

  • This disclosure relates to the field of computer technology, and specifically to a video encoding method and apparatus.
  • video encoding typically involves encoding the video multiple times based on different encoding parameter values, determining the target encoding parameter values, and then performing formal video encoding. This approach reduces bandwidth costs while ensuring video quality.
  • This disclosure provides a video encoding method to improve video encoding efficiency while ensuring video encoding quality.
  • This disclosure also provides a video encoding apparatus, an electronic device, and a computer storage medium.
  • This disclosure provides a video encoding method, comprising: performing feature extraction processing on multiple videos to be encoded to obtain video features of the videos to be encoded; determining target encoding parameters for the videos to be encoded based on the total bitrate corresponding to the multiple videos to be encoded and the video features of the videos to be encoded; and encoding the videos to be encoded according to the target encoding parameters; wherein, when the encoding parameters of the videos to be encoded are their corresponding target encoding parameters, the bitrate of the encoded video after encoding processing is the target bitrate, the quality evaluation index value of the encoded video is the target quality evaluation index value, the sum of the target bitrates corresponding to the encoded videos is less than the total bitrate, and the sum of the target quality evaluation index values corresponding to the encoded videos is greater than the standard value of the video quality evaluation index required by the video demand side for the multiple encoded videos.
  • determining the target encoding parameters of the videos to be encoded based on the total bitrate corresponding to the plurality of videos to be encoded and the video features of the videos to be encoded in the plurality of videos to be encoded includes: determining the video content type to which the videos to be encoded in the plurality of videos to be encoded belong based on the video features of the videos to be encoded in the plurality of videos to be encoded; classifying the videos to be encoded in the plurality of videos to be encoded according to the video content type to which the videos to be encoded belong to, to obtain at least one group of videos to be encoded, wherein the video content type of the videos to be encoded in the group of videos to be encoded belongs to the same video content type; determining the target encoding parameters of the group of videos to be encoded based on the total bitrate corresponding to the plurality of videos to be encoded and the video content type of the group of videos to be encoded, and using these as the target encoding parameters of the videos to be encoded
  • determining the target encoding parameters of the video to be encoded based on the total bitrate corresponding to the plurality of videos to be encoded and the video features of the videos to be encoded includes: determining an encoding parameter prediction model corresponding to the plurality of videos to be encoded based on the total bitrate corresponding to the plurality of videos to be encoded; inputting the video features of the videos to be encoded into the encoding parameter prediction model respectively to obtain the target encoding parameters for the videos to be encoded output by the encoding parameter prediction model; wherein, the encoding parameter prediction model is used to determine the target encoding parameters of the videos to be encoded according to the video features of the videos to be encoded, and the target encoding parameters satisfy the following conditions: after encoding the videos to be encoded using the target encoding parameters, the target bitrate and target quality evaluation index value of the encoded video are obtained; the sum of the target bitrates of the encoded videos corresponding to the plurality of videos to be encoded is less than the total
  • the coding parameter prediction model is trained as follows: The total bitrate of multiple sample videos, and the sample video features and coding parameters of the sample videos within the multiple sample videos are obtained; the total bitrate and sample video features of the multiple sample videos are input into an initial coding parameter prediction model to obtain the initial coding parameters output by the initial coding parameter prediction model for the sample videos; a loss value is obtained between the initial coding parameters and the sample coding parameters of the sample videos; the loss values corresponding to the sample videos in the multiple sample videos are summed to obtain the total loss value for the multiple sample videos; the parameters of the initial coding parameter prediction model are adjusted based on the total loss value to obtain the trained coding parameter prediction model.
  • the sample encoding parameters of the sample video are obtained as follows: a candidate rate-distortion point group is obtained from multiple sample videos under multiple candidate encoding parameter conditions; a pre-determined rate-distortion point is selected from the candidate rate-distortion point group corresponding to the sample video, wherein the pre-determined rate-distortion point includes at least the following attribute parameters: pre-determined encoding parameters, pre-determined bitrate, and pre-determined quality assessment index value; after encoding the sample video using the pre-determined encoding parameters, the bitrate of the encoded video is the pre-determined bitrate, and the quality assessment index value of the encoded video is the pre-determined quality assessment index value; if the above attribute parameters of the pre-determined rate-distortion points corresponding to the sample videos in the multiple sample videos meet the following conditions, then the pre-determined encoding parameters in the pre-determined rate-distortion points of the sample video are used as the sample encoding parameters of the sample video: the sum of the pre-determined bit
  • obtaining a candidate rate-distortion (RDD) point group for a sample video under multiple candidate coding parameter conditions includes: encoding the sample video using candidate coding parameters to obtain the bitrate and quality evaluation index values of the encoded video after encoding under the candidate coding parameter conditions; obtaining the bitrate and quality evaluation index values corresponding to the multiple candidate coding parameters after encoding the sample video respectively; using the candidate coding parameter and the bitrate and quality evaluation index value corresponding to the candidate coding parameter as a candidate RDD point of the encoded video; and using the candidate RDD points corresponding to the multiple candidate coding parameters as a candidate RDD point group for the sample video under multiple candidate coding parameter conditions.
  • RDD rate-distortion
  • it also includes: obtaining the standard values of video quality evaluation indicators required by the video demand side for multiple encoded videos; the step of determining the target encoding parameters of the video to be encoded based on the total bitrate corresponding to the multiple videos to be encoded and the video features of the videos to be encoded among the multiple videos to be encoded includes: determining the target encoding parameters of the video to be encoded based on the total bitrate corresponding to the multiple videos to be encoded, the video features of the videos to be encoded among the multiple videos to be encoded, and the standard values of the video quality evaluation indicators.
  • determining the target encoding parameters of the video to be encoded based on the total bitrate corresponding to the plurality of videos to be encoded, the video features of the videos to be encoded among the plurality of videos to be encoded, and the standard value of the video quality evaluation index includes: predicting the required prediction bitrate of the video to be encoded based on the video features of the video to be encoded; determining the required prediction encoding parameters of the video to be encoded based on the required prediction bitrate of the video to be encoded; obtaining the prediction bitrate and prediction quality evaluation index value of the video to be encoded after encoding the video to be encoded using the prediction encoding parameters; if the sum of the prediction bitrates corresponding to the multiple encoded videos corresponding to the plurality of videos to be encoded is less than the total bitrate, and the sum of the prediction quality evaluation index values corresponding to the multiple encoded videos is greater than the standard value of the video quality evaluation index, then the prediction encoding parameters are used as the target encoding parameters of the
  • the predictive coding parameters include multiple candidate coding parameters; the method further includes: for the videos to be encoded among the multiple videos to be encoded, encoding the videos to be encoded using the multiple candidate coding parameters to obtain multiple predicted bitrates and multiple predicted quality evaluation index values; if the predicted bitrates and predicted quality evaluation index values corresponding to the multiple candidate coding parameters are all under the following conditions, then a target coding parameter is selected from the multiple candidate coding parameters: the sum of the predicted bitrates corresponding to the encoded videos is less than the total bitrate, and the predicted quality evaluation index values corresponding to the encoded videos are less than the standard value of the video quality evaluation index.
  • selecting a target encoding parameter from the plurality of candidate encoding parameters includes: encoding the video to be encoded using the plurality of candidate encoding parameters respectively to obtain a plurality of predicted quality assessment index values of the encoded video to be encoded; comparing the plurality of predicted quality assessment index values of the encoded video to be encoded to obtain a target predicted quality assessment index value, wherein the target predicted quality assessment index value is greater than other predicted quality assessment index values among the plurality of predicted quality assessment index values except for the target predicted quality assessment index value; and using the candidate encoding parameter corresponding to the target predicted quality assessment index value as the target encoding parameter of the video to be encoded.
  • This disclosure also provides an electronic device, including: a processor; and a memory for storing a computer program. After the electronic device is powered on and runs the computer program through the processor, it performs the above-described method.
  • This disclosure also provides a storage medium storing a computer program, which is executed by a processor to perform the above-described method.
  • This disclosure provides a computer program product comprising: a computer program that, when executed by a processor of an electronic device, causes the processor to perform the above-described method.
  • the video encoding method provided in this disclosure extracts video features from multiple videos to be encoded, and then determines target encoding parameters for the videos to be encoded based on the total bitrate and video features of the multiple videos to be encoded. Then, the target encoding parameters are used to encode the videos to be encoded, resulting in an encoded video.
  • the target encoding parameters for the videos to be encoded can be determined using the total bitrate and video features of the multiple videos to be encoded, eliminating the need to pre-encode the videos to obtain the encoding features of the pre-processed videos. Therefore, the efficiency of determining the target encoding parameters for the videos to be encoded is improved.
  • the method provided in this disclosure can determine the target encoding parameters for the videos to be encoded under the following conditions: the bitrate of the encoded video obtained after encoding the videos to be encoded using the target encoding parameters is the target bitrate; the quality evaluation index value of the encoded video is the target quality evaluation index value; the sum of the target bitrates of the multiple encoded videos corresponding to the multiple videos to be encoded is less than the total bitrate; and the sum of the target quality evaluation index values of the encoded videos is greater than the quality evaluation index standard value required by the video demand side for the encoded video. Therefore, the video encoding method provided in this disclosure improves video encoding efficiency while ensuring video encoding quality.
  • Figure 1 is a flowchart of a video encoding method provided in the first embodiment of this disclosure.
  • Figure 2 is a schematic diagram of a pixel pair of Pd(i,j) in the gray-level co-occurrence matrix provided in the prior art.
  • Figure 3 is a schematic diagram of two filtering operators in spatial information.
  • Figure 4 is a schematic diagram of the process by which the coding parameter prediction model provided in the embodiments of this disclosure determines the target coding parameters of multiple videos to be encoded and encodes the videos to be encoded according to the target coding parameters.
  • Figure 5 is a schematic diagram of a video encoding device provided in the second embodiment of this disclosure.
  • Figure 6 is a schematic diagram of an electronic device provided in the third embodiment of this disclosure.
  • This disclosure provides a video encoding method and apparatus. These will be described in detail in the following embodiments.
  • bandwidth refers to the maximum transmission rate of the video, which is the amount of data that can be transmitted per unit of time.
  • Video coding The process of converting video signals into digital codes to efficiently utilize bandwidth and storage space for storage and transmission. It involves breaking down the video signal into spatially and temporally coded units and applying compression algorithms to reduce the amount of data.
  • Encoding efficiency The data compression ratio that a video encoding algorithm can achieve at the same image quality. High encoding efficiency means that it can encode at a higher bitrate, thereby reducing the need for storage space and transmission bandwidth.
  • Support Vector Regression is a machine learning algorithm used to establish a regression relationship between input variables and a continuous target variable.
  • SVR can be used to build a bitrate control model to dynamically adjust coding parameters based on video content and quality requirements, thereby achieving higher coding efficiency and higher video quality.
  • An I-frame (Intra-coded frame), also known as a keyframe or intra-frame, is an independent frame in a video sequence.
  • An I-frame is decoded without relying on information from other frames; it contains complete image information.
  • an I-frame is typically inserted at regular intervals (e.g., per second).
  • An I-frame can be decoded into a complete image, making it very useful for video editing and random access.
  • P-frame Predictive-coded frame
  • P-frames are encoded using information from decoded preceding reference frames (usually previous I-frames and P-frames).
  • P-frames only contain the difference information between the current frame and the reference frame, not complete image information. This difference information can be represented using motion compensation techniques to reduce data volume and improve coding efficiency.
  • Quantization Parameter Used to control the quantization process in video coding. Quantization is the process of converting a continuous video signal into a discrete numerical representation. By controlling the QP value, the quantization precision can be adjusted, thereby affecting coding efficiency and video quality.
  • Constant Rate Factor A bitrate control mode used to adjust the encoding bitrate while maintaining constant video quality. By adjusting the CRF value, different encoding efficiencies and file sizes can be achieved while maintaining relatively consistent video quality.
  • Rate-distortion curve Describes the change in compression quality at different bitrates. During data compression, to reduce data size, some image or audio quality is usually sacrificed. The rate-distortion curve describes the degree of distortion between the compressed data and the original data at different compression ratios (bitrates). A typical rate-distortion curve plots bitrate (compression ratio) on the x-axis and distortion (quality loss) on the y-axis. Generally, as the compression ratio increases, distortion also increases, meaning the quality of the compressed data decreases. This is because higher compression ratios lead to more data loss or information loss.
  • the grouped knapsack problem is a variation of the traditional knapsack problem, solved using a dynamic programming-based optimization algorithm.
  • the traditional knapsack problem there is a set of items and a knapsack. Each item has a weight and a value. The goal is to select items to put into the knapsack such that the total weight does not exceed the knapsack's capacity, while maximizing the total value.
  • the items are divided into several groups, and only one item from each group can be placed into the knapsack. The goal is to select one item from each group such that the total weight does not exceed the knapsack's capacity, while maximizing the total value.
  • Figure 1 is a flowchart of a video encoding method provided in the first embodiment of this disclosure.
  • the video encoding method shown in Figure 1 includes:
  • Step S101 Perform feature extraction processing on multiple videos to be encoded to obtain the video features of the videos to be encoded among the multiple videos to be encoded;
  • Step S102 Based on the total bitrate corresponding to the plurality of videos to be encoded and the video features of the videos to be encoded among the plurality of videos to be encoded, determine the target encoding parameters of the videos to be encoded;
  • Step S103 Encode the video to be encoded according to the target encoding parameters of the video to be encoded; wherein, when the encoding parameters of the video to be encoded are the corresponding target encoding parameters among the plurality of videos to be encoded, the bitrate of the encoded video after encoding is the target bitrate, the quality evaluation index value of the encoded video is the target quality evaluation index value, the sum of the target bitrates corresponding to the encoded videos is less than the total bitrate, and the sum of the target quality evaluation index values corresponding to the encoded videos is greater than the video quality evaluation index standard value required by the video demand side for the plurality of encoded videos.
  • the method provided in this disclosure can be applied to the process of a video publisher uploading a batch of videos to a video platform.
  • the client sends this batch of videos to the server.
  • the server determines the target encoding parameters for each video in the batch based on the method provided in this disclosure, encodes each video based on the target encoding parameters, and then transmits it to the video platform. It can also be applied to a video platform selecting a batch of videos for a target user. Before displaying this batch of videos to the target user, the video platform's client provides this batch of videos to the server.
  • the server determines the target encoding parameters for each video in the batch based on the method provided in this disclosure, encodes each video based on the target encoding parameters, and then transmits it to the client for display to the target user.
  • the method provided in this disclosure determines the target encoding parameters of multiple videos to be encoded based on the total bitrate of the videos and their video features. It eliminates the need for pre-encoding the videos to determine encoding feature data and instead obtains the target encoding parameters through multiple encoding operations, thus improving the efficiency of determining the target encoding parameters for multiple videos.
  • the target encoding parameters obtained by the video encoding method provided in this disclosure satisfy the following conditions: the sum of the bitrates of the encoded videos corresponding to the multiple videos to be encoded is less than the total bitrate of the multiple videos to be encoded, and the sum of the quality evaluation index values of the encoded videos is greater than the standard value of the quality evaluation index required by the video demander for the multiple encoded videos.
  • the video encoding method provided in this disclosure improves the efficiency of determining the target encoding parameters of multiple videos to be encoded while ensuring that the sum of the bitrates of the encoded videos is less than the total bitrate and ensuring the video quality of the encoded videos; that is, it improves encoding efficiency while ensuring video encoding quality.
  • the process involves determining the video features of the videos to be encoded.
  • feature extraction is performed on multiple videos to obtain their respective video features. Specifically, video features are first extracted for each video, and then the target encoding parameters are determined.
  • the multiple videos to be encoded can be related, for example, multiple videos that are all about food descriptions. Alternatively, the multiple videos to be encoded can be multiple segments of a single complete video, each segment containing different video features.
  • the multiple videos to be encoded may include at least two videos, and further, each video to be encoded may include multiple video segments.
  • Multiple videos to be encoded can also be unrelated to each other. For example, there could be three videos: the first video has a static frame rate greater than a preset threshold; the second video has a dynamic frame rate greater than a preset threshold; and the third video has a content richness within its frames greater than a preset threshold. Therefore, these three types of videos have different video characteristics, and consequently, the target encoding parameters required to determine the videos to be encoded in subsequent steps are also different.
  • feature regions are extracted from both the spatial and temporal domains of the video to be encoded, obtaining the spatial features and derived features of the spatial features, as well as the temporal features and derived features of the temporal features.
  • the spatial features of the video to be encoded include at least one of the following features: Gray-Level Co-occurrence Matrix (GLCM), used to characterize the texture features of the image; Spatial Information (SI), used to characterize the amount of spatial detail in the image; and Colorfulness (CF), used to characterize the saturation or intensity of colors in the image.
  • GLCM Gray-Level Co-occurrence Matrix
  • SI Spatial Information
  • CF Colorfulness
  • the gray-level co-occurrence matrix (GLCM), which is used to describe image texture features. Specifically, the image is converted into an image with a gray level of L, and then a gray-level co-occurrence matrix is generated based on the relationship between the gray values of the pixels. The secondary statistics obtained from the gray-level co-occurrence matrix are used to describe the image texture features.
  • GLCM gray-level co-occurrence matrix
  • the gray-level co-occurrence matrix has a size of L x L.
  • P d (i, j) is obtained from Equation 1.
  • P d (i, j) P d (i, j
  • d, ⁇ ) (i, j 0, 1, 2,..., L-1)
  • i and j represent the gray values of the pixels
  • d represents the distance between two points
  • ⁇ represents the angle between two points, which is usually 0°, 45°, 90°, or 135°.
  • Figure 2 is a schematic diagram of a pixel pair P d (i, j) in the gray-level co-occurrence matrix provided in the prior art.
  • each element is normalized to obtain a normalized gray-level co-occurrence matrix.
  • the specific method for normalizing each element is as follows: [The text abruptly ends here, likely due to an incomplete sentence or missing information.] The average value of each element Pd (i,j) is obtained by dividing the sum of all elements in the gray-level co-occurrence matrix. Then, the quadratic statistics of the normalized gray-level co-occurrence matrix are extracted, as follows:
  • the quadratic statistics of the gray-level co-occurrence matrix include at least one of the following:
  • the entropy of the gray-level co-occurrence matrix (GLCM ent , where ent is an abbreviation for entropy) is used to characterize the complexity or non-uniformity of image texture features. A larger entropy indicates more complex texture information in the image; conversely, a smaller entropy indicates simpler texture information.
  • the entropy GLCM ent is calculated using formula 3:
  • the correlation (GLCM cor) of the gray-level co-occurrence matrix (where cor is an abbreviation for correlation) is used to characterize the similarity of the elements of the gray-level co-occurrence matrix in the column or row direction.
  • the correlation value is greater; conversely, when the elements in the matrix are not uniformly equal, the correlation value is smaller.
  • the correlation (GLCM cor) of the gray-level co-occurrence matrix is calculated using Formula 4:
  • the contrast ratio GLCM con (where con is an abbreviation for Contrast) of the gray-level co-occurrence matrix is used to measure how the element values of the matrix are distributed and the degree of local variation in the image, characterizing the image's sharpness and the depth of texture grooves. A higher contrast ratio indicates deeper texture grooves, greater contrast, and a sharper effect; conversely, a lower contrast ratio indicates shallower texture grooves and a blurrier effect.
  • the contrast ratio GLCM con is calculated using Formula 5:
  • GLCM hom gray-level co-occurrence matrix
  • the energy GLCM ene (where ene is an abbreviation for Energy) of the gray-level co-occurrence matrix is used to characterize the uniformity of the gray-level distribution and the coarseness of the texture in the image.
  • the energy GLCM ene of the gray-level co-occurrence matrix is calculated using Formula 7:
  • the entropy of the gray-level co-occurrence matrix GLCM ent includes the average entropy of the gray-level co-occurrence matrix (F9, GLCM ent _mean, mean means average): the average value between values used to characterize the complexity or non-uniformity of image texture features; the standard deviation of the entropy of the gray-level co-occurrence matrix (F10, GLCM ent _std, std is an abbreviation for Standard Deviation): the standard deviation between values used to characterize the complexity or non-uniformity of image texture features.
  • F9, GLCM ent _mean, mean means average the average value between values used to characterize the complexity or non-uniformity of image texture features
  • the standard deviation of the entropy of the gray-level co-occurrence matrix F10, GLCM ent _std, std is an abbreviation for Standard Deviation
  • the correlation of the gray-level co-occurrence matrix GLCM cor includes the correlation mean of the gray-level co-occurrence matrix (F1, GLCM cor _mean): the average value between the values that characterize the similarity of the elements of the gray-level co-occurrence matrix in the column or row direction; and the correlation standard deviation of the gray-level co-occurrence matrix (F2, GLCM cor _std): the standard deviation value between the values that characterize the similarity of the elements of the gray-level co-occurrence matrix in the column or row direction.
  • F1, GLCM cor _mean the average value between the values that characterize the similarity of the elements of the gray-level co-occurrence matrix in the column or row direction
  • the correlation standard deviation of the gray-level co-occurrence matrix F2, GLCM cor _std
  • the contrast of the gray-level co-occurrence matrix GLCM con includes the average contrast of the gray-level co-occurrence matrix (F3, GLCM con _mean): the average value between the values used to characterize the sharpness of the image and the depth of the grooves in the texture; and the standard deviation of the contrast of the gray-level co-occurrence matrix (F4, GLCM con _std): the standard deviation between the values used to characterize the sharpness of the image and the depth of the grooves in the texture.
  • the homogeneity of the gray-level co-occurrence matrix GLCM hom includes the mean of the homogeneity of the gray-level co-occurrence matrix (F7, GLCM hom _mean): the average value between the values used to characterize the degree of local variation of the image texture; and the standard deviation of the homogeneity of the gray-level co-occurrence matrix (F8, GLCM hom _std): the standard deviation between the values used to characterize the degree of local variation of the image texture.
  • the energy of the gray-level co-occurrence matrix GLCM ene includes the average energy of the gray-level co-occurrence matrix (F5, GLCM ene _mean): the average value between the values used to characterize the uniformity of the gray-level distribution of the image and the coarseness of the texture, and the standard deviation of the energy of the gray-level co-occurrence matrix (F6, GLCM ene _std): the standard deviation between the values used to characterize the uniformity of the gray-level distribution of the image and the coarseness of the texture.
  • F5 average energy of the gray-level co-occurrence matrix
  • F6 GLCM ene _std the standard deviation between the values used to characterize the uniformity of the gray-level distribution of the image and the coarseness of the texture.
  • Spatial Information which represents the amount of spatial detail in a frame of an image.
  • Spatial domain refers to the spatial distribution of an image or signal, corresponding to the temporal domain, which refers to the changes in the signal over time. The more complex the spatial scene, the higher its corresponding SI value.
  • Equation 9 Equation 9:
  • FIG. 3 is a schematic diagram of the two filtering operators in the spatial domain information.
  • 301 represents G1 and 302 represents G2 .
  • a large SI value indicates that the image contains a lot of edge information.
  • the derived features of spatial information SI include the following representations:
  • SI Spatial information
  • F26, SI_mean Spatial information
  • SI refers to the gray value of each pixel in an image or the intensity value of a signal at a spatial location. These values carry basic information about the image content or signal characteristics, such as brightness, color distribution, and texture features.
  • the average value of spatial information (SI_mean) is calculated by taking the arithmetic mean of the gray values of all pixels in the image to obtain a single value that represents the average brightness or signal strength level of the entire image.
  • Standard deviation of spatial information Used to characterize the degree of variation or dispersion of the gray values of all pixels in an image relative to their average value. That is, it is an index that characterizes the uniformity or variability of the data distribution of all pixels in an image, in order to determine the spatial heterogeneity of the data of all pixels or the sharpness and noise level of the image.
  • the average value of the corresponding average values at different times can be calculated again to determine the similarity of the image at different time points.
  • the standard deviation between the average values of spatial information (F29, mean_SI_std) is used to characterize the average values of spatial information in different regions of the image. After calculating the average values of spatial information in different regions of the image, the standard deviation of the average values of multiple different regions of the image is calculated to determine the fluctuation or deviation of spatial information in different regions.
  • the standard deviation of the average value at different times can be calculated again to determine the fluctuation or deviation of the spatial information of the image at different time points.
  • ⁇ rgyb represents the average color factor and ⁇ rgyb represents the variance color factor.
  • R(i,j), G(i,j), and B(i,j) represent the values of the (i,j)th pixel in an image across the R, G, and B channels of the RGB color space.
  • ⁇ rg represents the mean of rg(i,j)
  • ⁇ yb represents the mean of yb(i,j)
  • ⁇ rg represents the standard deviation of rg(i,j)
  • ⁇ yb represents the standard deviation of yb(i,j).
  • derived features of chroma CF include the following representations:
  • Average color value (F22, CF_mean) The average color value (CF_mean) is obtained by summing the color values of all pixels in the image and then dividing by the total number of pixels, so as to characterize the overall average color value level of the image.
  • the temporal features of the video to be encoded include at least one of the following features: temporal information (TI), used to characterize the temporal variation of the video sequence; normalized cross correlation (NCC), used to characterize the correlation between two adjacent video frames; and temporal coherence (TC), used to characterize the correlation between the temporal signals of two adjacent video frames.
  • temporal information used to characterize the temporal variation of the video sequence
  • NCC normalized cross correlation
  • TC temporal coherence
  • TI time-domain information
  • NCC Normalized Cross-Correlation
  • f ⁇ sub>n ⁇ /sub>(i,j) represents the grayscale value of the nth video frame at coordinates (i,j); H represents the average of all grayscale values in the nth video frame; H and W represent the height and width of the video frame, respectively.
  • the self-power spectrum and cross-power spectrum are calculated using the formula set provided in Formula 14:
  • Average of temporal coherence values (F11, mean_TC_mean): This represents the overall average level of a set of temporal coherence average data. It is calculated as follows:
  • Step 1 calculate the mean_TC of a set of time-domain coherence data:
  • Multiple signal pairs are obtained, for example, 3 signal pairs, each containing 10 signal pairs.
  • the average value of the time-domain coherence data for each signal pair is obtained based on the above method.
  • 3 average values of time-domain coherence data can be obtained for these three signal pairs.
  • These 3 average values of time-domain coherence data are taken as a set of time-domain coherence average values.
  • This set of time-domain coherence average values is divided by the number of groups (3) to obtain the average value among the time-domain coherence average values (mean_TC_mean).
  • the mean_TC_mean reflects the overall level of time-domain coherence data for these 3 signal pairs.
  • Standard deviation (F12, mean_TC_std) between the means of time-domain coherence A statistic used to characterize the dispersion of a set of time-domain coherence mean data, reflecting the volatility or range of variation of the time-domain coherence mean data. It is calculated as follows:
  • Step 1 calculate the mean_TC of a set of time-domain coherence data:
  • Temporal coherence measures the correlation between two signals.
  • a set consists of 10 signal pairs, each corresponding to a temporal coherence data point.
  • the overall temporal coherence level of this set of temporal coherence data is characterized by calculating the mean_TC of these 10 temporal coherence data points.
  • Step 2 calculate the standard deviation (mean_TC_mean) between the means of multiple sets of time-domain coherence data:
  • Multiple signal pairs are obtained, for example, 3 signal pairs, each containing 10 signal pairs.
  • the average value of the time-domain coherence data of each signal pair is obtained based on the above method.
  • 3 average values of time-domain coherence data can be obtained from these three signal pairs.
  • These 3 average values of time-domain coherence data are taken as a set of time-domain coherence average values. Divide this set of time-domain coherence average values by the number of groups (3) to obtain the average value (mean_TC_mean) among the time-domain coherence average values.
  • the standard deviation of the time-domain coherence data averages is calculated. The specific calculation is as follows: For each time-domain coherence data average, the square of the difference between it and the mean (mean_TC_mean) is calculated, obtaining the squares of the differences for each of the three time-domain coherence data averages. The squares of these three differences are then summed and divided by the number of time-domain coherence data averages (3) to obtain the variance. Finally, the square root of the variance is taken to obtain the standard deviation (mean_TC_std) among the time-domain coherence data averages.
  • std_TC_mean Average of the standard deviations of time-domain coherence (F13, std_TC_mean): This represents the average of the standard deviations of multiple sets of time-domain coherence data. Each set of time-domain coherence data has its own standard deviation, which reflects the dispersion or volatility of each dataset. std_TC_mean is the average of these standard deviations to obtain an indicator representing the overall volatility. It is calculated as follows:
  • Step 1 calculate the standard deviation (std_TC) among a set of time-domain coherence data:
  • Temporal coherence measures the correlation between two signals.
  • a set contains 10 signal pairs, each corresponding to a set of time-domain coherence data.
  • TC measures the correlation between each time-domain coherence data point and the average coherence data point of its corresponding pair.
  • the standard deviation of this set of time-domain coherence data is calculated based on the degree of dispersion (i.e., the volatility or range of variation among these time-domain coherence data).
  • the squares of the differences between the data points are used to obtain the squares of the differences for each of the 10 time-domain coherence data points.
  • the sum of these squares is then divided by the number of time-domain coherence data points (10) to obtain the variance.
  • the square root of the variance is taken to obtain the standard deviation (std_TC) of this set of time-domain coherence data.
  • Step 2 calculate the average of the standard deviations among multiple sets of time-domain coherence data, std_TC_mean:
  • each set of time-domain coherence data contains 10 time-domain coherence data, and each set of time-domain coherence data has a corresponding standard deviation (std_TC), the average of these 3 corresponding standard deviations can be obtained as the average of the standard deviations among the 3 sets of time-domain coherence data (std_TC_mean).
  • Standard deviation between standard deviations of time-domain coherence F14, std_TC_std: This represents the standard deviation between the standard deviations of multiple sets of time-domain coherence data.
  • Each set of time-domain coherence data has its own standard deviation (std_TC), which reflects the dispersion or fluctuation of each dataset.
  • std_TC_std is obtained by taking the standard deviation of multiple sets of time-domain coherence data again to obtain the dispersion or fluctuation between the standard deviations of multiple sets of time-domain coherence data. It is calculated as follows:
  • Step 1 calculate the standard deviation (std_TC) among a set of time-domain coherence data:
  • Temporal coherence measures the correlation between two signals.
  • a set contains 10 signal pairs, each corresponding to a temporal coherence data point.
  • TC measures the correlation between each temporal coherence data point and the average of its corresponding coherence data points.
  • the standard deviation of this set of time-domain coherence data is calculated based on the degree of dispersion (i.e., the volatility or range of variation among these time-domain coherence data).
  • the squares of the differences between the data points are used to obtain the squares of the differences for each of the 10 time-domain coherence data points.
  • the sum of these squares is then divided by the number of time-domain coherence data points (10) to obtain the variance.
  • the square root of the variance is taken to obtain the standard deviation (std_TC) of this set of time-domain coherence data.
  • Skewness of the mean of time-domain coherence (F15, mean_TC_skew, where skew is an abbreviation for skewness): This refers to the statistic obtained after performing skewness analysis on the mean of a set of time-domain coherence data. It is used to characterize the direction and degree of skewness of the mean of this set of time-domain coherence data.
  • the skewness value is 0; if the mean distribution of this set of time-domain coherence data is skewed to the right, the skewness value is 0; if the mean distribution of this set of time-domain coherence data is skewed to the left, the skewness value is negative.
  • the mean_TC_skew is used to determine whether the time-series coherence data is concentrated on one side of the mean, or whether there are outliers, etc.
  • the skewness (mean_TC_skew) of the mean temporal coherence is calculated through the following steps:
  • Step 1 Calculate the mean (mean_TC). Calculate the mean of a set of time-domain coherence data (TC).
  • Step 2 Calculate the skewness. Calculate the skewness of the mean obtained in Step 1.
  • the formula for calculating skewness is usually based on the third and fourth moments of the sample data, as shown in Formula 15:
  • n represents the number of sample data
  • TC i represents the temporal coherence value of each sample data
  • mean_TC represents the average of the temporal coherence data of n sample data
  • s is the standard deviation of the temporal coherence data of n sample data.
  • Skewness of the standard deviation of time-domain coherence (F16, std_TC_skew): This is a statistic obtained after performing skewness analysis on the standard deviations of a set of time-domain coherence data. It is used to characterize the direction and degree of skewness of the standard deviations of this set of time-domain coherence data.
  • std_TC_skew helps determine whether the standard deviations of time-series coherence data are concentrated on one side, or whether there are outliers.
  • skewness (std_TC_skew) of the standard deviation of temporal coherence is calculated through the following steps:
  • Step 1 Calculate the standard deviation (std_TC). Calculate the standard deviation for a set of time-domain coherence data (TC) to measure the dispersion of the data.
  • Step 2 Calculate the skewness. Calculate the skewness based on the standard deviation obtained in Step 1.
  • the formula for calculating skewness is usually based on the third and fourth moments of the sample data. The specific formula is shown in Formula 16:
  • x is each value of std_TC
  • ⁇ is the average value of std_TC
  • n is the number of std_TC values
  • std_TC is the standard deviation of the time-domain coherence data.
  • mean_TC_kur This is used to characterize the sharpness or flatness of the distribution of the mean of temporal coherence data. If the value of mean_TC_kur is greater than 3 (the kurtosis of the normal distribution is 3), it means that the distribution of mean_TC is sharper than the normal distribution, that is, the data is more concentrated around the mean and the tail is shorter; if the value of mean_TC_kur is less than 3, it means that the distribution of mean_TC is flatter than the normal distribution, that is, the data is more widely distributed and the tail is longer.
  • Entropy of the mean temporal coherence (F18, mean_TC_entropy): Used to measure the uncertainty or information content of a series of temporal coherence data. If mean_TC_entropy is low, it indicates that the coherence distribution of the signal is relatively concentrated, that is, the coherence level of the signal is similar at most time points or frequency bands, which indicates that the signal behavior is relatively regular or predictable. If mean_TC_entropy is high, it indicates that the coherence of the signal varies greatly with time or frequency, with high complexity or irregularity, and the signal behavior differs significantly in different time periods or frequency bands, making it difficult to predict.
  • Entropy of the standard deviation of temporal coherence (F19, std_TC_entropy): This is used to characterize the uncertainty and complexity of the distribution of temporal coherence variations. If std_TC_entropy is low, it indicates that the temporal coherence variation is relatively small and relatively regular, and the information uncertainty contained in its standard deviation is low, meaning that the fluctuation pattern of temporal coherence is relatively simple and predictable. If std_TC_entropy is high, it indicates that the standard deviation of temporal coherence itself has high uncertainty or complexity, meaning that the variation pattern of temporal coherence over time is very diverse and difficult to predict accurately, reflecting that the behavior of the signal varies significantly at different time points or under different conditions.
  • the target encoding parameters for encoding the video to be encoded are determined based on these features, thereby improving the efficiency and accuracy of acquiring the target encoding parameters.
  • the target encoding parameters of the videos to be encoded are determined based on these video features.
  • the target encoding parameters are determined based on the total bitrate corresponding to the multiple videos to be encoded and the video features of the videos to be encoded among the multiple videos to be encoded.
  • the total bitrate corresponding to the multiple videos to be encoded specifically refers to the total bandwidth condition of the multiple videos to be encoded during video transmission, which refers to the data transmission rate that the network can support, that is, the amount of data that can be transmitted through the network per unit time.
  • the encoding parameters of each video need to be adjusted.
  • the final target encoding parameters for each video should satisfy the following conditions: the sum of the bitrates of the encoded videos is less than the total bitrate, and the sum of the quality assessment index values of the encoded videos is greater than the standard video quality assessment index values required by the video requester for the multiple videos.
  • the encoding parameters may include at least one of the following parameters: bit rate, resolution, frame rate, quantization parameters, or constant quality rate.
  • Bitrate directly determines the bitrate of the encoded video, and is a key factor affecting the size and quality of the encoded video file. Bitrate refers to the amount of information transmitted per unit of time, and the unit of bitrate is usually bits per second or its multiples, such as kilobits per second or megabits per second. Setting the target bitrate of the video to be encoded in the video encoding software specifies the rate at which the encoder compresses the video, thereby ensuring that the encoded video reaches the target file size or meets the target network transmission requirements.
  • Resolution refers to the pixel size of a video, such as 1080p, 720p, etc. Reducing the resolution can significantly reduce the bit rate required for video encoding, but it will also reduce the detail of the video frames.
  • Frame rate is the number of frames displayed per second in a video.
  • a higher frame rate provides smoother dynamic effects, but it increases the bitrate requirement.
  • Appropriately reducing the frame rate can reduce the bitrate requirement, thereby reducing the bandwidth requirements of the encoded video.
  • the quantization parameter or constant quality rate is used in the encoder to control the degree of compression. The larger the value, the greater the compression, the lower the bitrate of the encoded video, and the lower the quality evaluation index value of the encoded video.
  • the encoding parameters are particularly important for determining the bitrate and quality evaluation index values of the encoded video. Therefore, in order to ensure that the target encoding parameters of the videos to be encoded among multiple videos meet the above conditions, the target encoding parameters can be determined in at least the following ways in this embodiment, which are described below.
  • the first method for determining target encoding parameters is as follows: Based on the video characteristics of the videos to be encoded among the plurality of videos to be encoded, determine the video content type to which the videos to be encoded belong; based on the video content type to which the videos to be encoded belong, classify the videos to be encoded among the plurality of videos to be encoded to obtain at least one group of videos to be encoded, wherein the video content type of the videos to be encoded in the group of videos to be encoded belongs to the same video content type; based on the total bitrate corresponding to the plurality of videos to be encoded, and in combination with the video content type of the group of videos to be encoded, determine the target encoding parameters of the group of videos to be encoded, and use them as the target encoding parameters of the videos to be encoded in the group of videos to be encoded.
  • the video content type is determined.
  • the first type of video content is real-time transmission content, such as a video stream of a speaker in a conference or a live video stream.
  • the second type of video content is static transmission content, such as a pre-recorded course video or a recorded product introduction video.
  • the first type of video content has higher video quality requirements; therefore, the bitrate allocated to the first type of video content is higher than that allocated to the second type.
  • the third type of video content to be encoded is static display content, such as a video stream composed of multiple images with similar content; it belongs to static display images.
  • the fourth type of video content to be encoded is dynamic content, such as a video stream containing various different actions; it belongs to dynamic images.
  • the fourth type of video content has higher video quality requirements; therefore, the bitrate allocated to the fourth type of video content is higher than that allocated to the third type.
  • the fifth type of video content to be encoded is a type rich in image detail, where the proportion of detailed information contained in each of the multiple frames of the video to be encoded within that video frame is greater than a preset proportion threshold.
  • the sixth type of video content to be encoded is a type lacking in image detail, where the proportion of detailed information contained in each of the multiple frames of the video to be encoded within that video frame is less than or equal to a preset proportion threshold.
  • the fifth type of video content has higher video quality requirements; therefore, the bitrate allocated to the fifth type of video content is higher than that allocated to the sixth type.
  • Multiple videos to be encoded can belong to the same video content type or different video content types. For the latter, the videos are categorized according to their respective video content types. Based on the different quality requirements of each content type, the required bitrate for that content type is determined, thereby determining the target encoding parameters for that content type.
  • the target encoding parameters for these 5 videos need to be allocated with a trade-off. If the first two videos belong to the first type of video content, and the last three belong to the second type, then the bitrate allocated to the videos belonging to the first type is higher than that allocated to the videos belonging to the second type. To reduce the computational complexity of the target encoding parameters, we can assign one target encoding parameter to the two videos belonging to the first type and another target encoding parameter to the last three videos belonging to the second type.
  • the step of encoding the video to be encoded according to the target encoding parameters of the video to be encoded includes: performing joint encoding processing on the videos to be encoded contained in the video group to be encoded according to the target encoding parameters of the video group to be encoded.
  • Videos belonging to the same content type have the same target encoding parameters. Therefore, multiple videos belonging to the same content type can be jointly encoded, reducing the computational load in determining the target encoding parameters and thus improving encoding efficiency.
  • the second method for determining target encoding parameters is as follows: Based on the total bitrate corresponding to the plurality of videos to be encoded, an encoding parameter prediction model is determined for the plurality of videos to be encoded; the video features of the videos to be encoded are respectively input into the encoding parameter prediction model to obtain the target encoding parameters output by the encoding parameter prediction model for the videos to be encoded; wherein, the encoding parameter prediction model is used to determine the target encoding parameters of the videos to be encoded according to the video features of the videos to be encoded, and the target encoding parameters satisfy the following conditions: after encoding the videos to be encoded using the target encoding parameters, the target bitrate and target quality evaluation index value of the encoded videos are obtained; the sum of the target bitrates corresponding to the encoded videos of the plurality of videos to be encoded is less than the total bitrate, and the sum of the target quality evaluation index values corresponding to the encoded videos is greater than the standard value of the video quality evaluation index required by the video
  • embodiments of this disclosure can determine the target coding parameters of the videos to be encoded using a pre-trained coding parameter prediction model.
  • One coding parameter prediction model corresponds to one total bitrate.
  • a prediction model for the coding parameters of each video is determined. Then, the prediction model determines the target coding parameters for each video based on its video features. In this process, the prediction model determines the target coding parameters based on the video features, rather than pre-coding the parameters and then obtaining coding feature data from the pre-coding results before determining the target coding parameters. This reduces the computational load in the target coding parameter determination process and improves the efficiency of target coding parameter determination.
  • the target encoding parameters obtained by the encoding parameter prediction model can satisfy the following conditions: after the video to be encoded is encoded using the target encoding parameters, the target bitrate and target quality evaluation index value of the encoded video are obtained. Also, the sum of the target bitrates of multiple encoded videos is less than the total bitrate, and the sum of the target quality evaluation index values is less than the standard value of the video quality evaluation index required by the video demander for multiple encoded videos. Therefore, the process of determining the target encoding parameters through the pre-trained encoding parameter prediction model improves the determination rate of the target encoding parameters while ensuring that the video quality of the encoded video meets the requirements of the video demander. In other words, it improves encoding efficiency while ensuring the video quality of the encoded video.
  • the coded parameter prediction model can be trained in the following way:
  • the multiple sample videos can be historical videos obtained during the historical processing. Target historical videos are selected from these historical videos as sample videos.
  • the sample encoding parameters of the sample videos must satisfy the following conditions: the bitrate of the sample videos after encoding with the sample encoding parameters is less than the total bitrate of the multiple sample videos, and the quality evaluation index value of the encoded samples is greater than the standard value of the quality evaluation index required by the video demand side after encoding the multiple sample videos.
  • the initial encoding parameter prediction model is trained using the total bitrate of the multiple sample videos and the sample video features of the multiple sample videos. The training is complete when the sum of the loss values between the initial encoding parameters and the sample encoding parameters obtained by the initial encoding parameter prediction model meets the preset conditions.
  • the encoding parameter prediction model used in this embodiment can be a target model obtained after training a Support Vector Regression (SVR) model.
  • SVR Support Vector Regression
  • Figure 4 is a schematic diagram of the process by which the encoding parameter prediction model provided in this embodiment determines the target encoding parameters of multiple videos to be encoded and encodes each video according to the target encoding parameters.
  • the multiple videos to be encoded include three videos: a first video A, a second video B, and a third video C. After determining the total bitrate of the multiple videos to be encoded, the corresponding encoding parameter prediction model under this total bitrate condition is obtained.
  • the encoding parameter prediction model determines the target encoding parameters Ab for video A based on the video feature data of video A, the target encoding parameters Bb for video B based on the video feature data of video B, and the target encoding parameters Cb for video C based on the video feature data of video C. Based on this, video A is encoded using the target encoding parameters Ab to obtain the target bitrate and target video evaluation index value of the encoded video A. Video B is encoded using the target encoding parameters Bb to obtain the target bitrate and target quality evaluation index value of the encoded video B. Video C is encoded using target encoding parameters Cb to obtain the target bitrate and target quality evaluation index value of the encoded video C.
  • the sum of the target bitrates of encoded video A, encoded video B, and encoded video C is less than the total bitrate of the multiple videos to be encoded, and the sum of the target quality evaluation index values of encoded video A, encoded video B, and encoded video C is greater than the standard video quality evaluation index value required by the video requester for the multiple encoded videos.
  • machine learning models such as Gaussian regression, extreme tree, random forest or multilayer perceptron, without limitation.
  • the sample encoding parameters of the sample video are obtained in the following way:
  • a candidate rate-distortion (RDD) point group is obtained from multiple sample videos under multiple candidate coding parameter conditions.
  • a pre-determined RDD point is selected from the candidate RDD point group corresponding to the sample video.
  • the pre-determined RDD point includes at least the following attribute parameters: pre-determined coding parameters, pre-determined bitrate, and pre-determined quality assessment index value. After encoding the sample video using the pre-determined coding parameters, the bitrate of the encoded video is the pre-determined bitrate, and the quality assessment index value of the encoded video is the pre-determined quality assessment index value.
  • the pre-determined coding parameters in the pre-determined RDD points of the sample video are used as the sample coding parameters of the sample video: the sum of the pre-determined bitrates corresponding to the encoded videos in the multiple sample videos is less than the total bitrate, and the sum of the pre-determined quality assessment index values corresponding to all encoded videos is greater than the standard video quality assessment index value required by the video demand side for multiple encoded videos.
  • the process of determining the sample encoding parameters for the sample video can be carried out in the manner described above.
  • a candidate rate-distortion (RDD) point group is obtained from multiple sample videos under multiple candidate coding parameter conditions.
  • the sample videos are encoded using the candidate coding parameters to obtain the bitrate and quality evaluation index values of the encoded video under the candidate coding parameter conditions.
  • the bitrate and quality evaluation index values corresponding to the multiple candidate coding parameters after encoding the sample videos are obtained respectively.
  • the candidate coding parameter and the bitrate and quality evaluation index value corresponding to the candidate coding parameter are used as a candidate RDD point of the encoded video.
  • the candidate RDD points corresponding to the multiple candidate coding parameters are used as a candidate RDD point group obtained from the sample videos under multiple candidate coding parameter conditions.
  • Multiple sample videos contain multiple sample videos.
  • multiple candidate coding parameters are used for encoding processing to obtain the candidate bitrate and candidate quality evaluation index values of the encoded video.
  • a candidate rate-distortion point includes at least the following parameters: candidate coding parameters, candidate bitrate, and candidate quality evaluation index value. Encoding the sample video separately using multiple candidate coding parameters yields the candidate bitrate and candidate quality evaluation index values corresponding to each candidate coding parameter, which are then considered as multiple candidate RDBMS points. These multiple candidate RDBMS points corresponding to a single sample video are grouped as the candidate RDBMS point set for that sample video.
  • the pre-determined RDD point for each sample video is determined. Then, the pre-determined RDD point is judged as follows: if the judgment result satisfies the following conditions, the pre-determined RDD point of that sample video is determined as the sample RDD point of the sample video, and the pre-determined coding parameters contained in the pre-determined RDD point of the sample video are the sample coding parameters of the sample video:
  • the sum of the predetermined bitrates for the encoded videos in the multiple sample videos is less than the total bitrate of the multiple sample videos, and the sum of the predetermined quality assessment index values for the encoded videos is greater than the standard value of the video quality assessment index required by the video demand side for the encoded videos.
  • the initial coding parameter prediction model is trained using the sample coding parameters as labels and the sample video features of the sample coded videos, thereby obtaining the corresponding coding parameter prediction models under different total bitrate conditions.
  • the above describes how to determine the target coding parameters of a video to be encoded using a coding parameter prediction model.
  • the target encoding parameters obtained by the first and second methods described above satisfy the following conditions: the sum of the target bitrates of the encoded videos is less than the total bitrate, and the sum of the target quality assessment index values is greater than the standard value of the video quality assessment index required by the video requester for the multiple encoded videos.
  • the standard value of the video quality assessment index required by the video requester for the multiple encoded videos varies depending on the specific video type or the specific requirements of the video requester.
  • the target encoding parameters of the video to be encoded can also be determined in a third way according to the embodiments of this disclosure:
  • the determination of the target encoding parameters of the video to be encoded based on the total bitrate of the multiple videos to be encoded and the video features of the videos to be encoded can be achieved in the following way: the target encoding parameters of the video to be encoded are determined based on the total bitrate of the multiple videos to be encoded, the video features of the videos to be encoded, and the standard values of the video quality evaluation indicators.
  • the total bitrate of multiple videos to be encoded limits the sum of bitrates usable during the encoding process of all videos.
  • the sum of the bitrates used during the encoding process of each video cannot exceed the total bitrate. Based on this, to ensure that the quality evaluation index values of all encoded videos meet preset requirements, a corresponding target encoding parameter needs to be assigned to each video.
  • the target encoding parameters for each video can be determined based on the video characteristics of the videos and the video quality evaluation index standards required by the video client.
  • the bitrate requirement for the target encoding parameters of the videos to be encoded among multiple videos to be encoded is simply to meet the bitrate requirement for obtaining SD video picture quality.
  • the second preset video quality assessment standard threshold indicates that the video quality is of ultra-high definition (UHD) quality.
  • UHD ultra-high definition
  • SD standard definition
  • the bitrate in the target encoding parameters of the video to be encoded directly determines the bitrate of the encoded video. In this case, the bitrate requirement for the target encoding parameters of the multiple videos to be encoded is simply to meet the bitrate requirement for obtaining UHD video quality.
  • this disclosure specifically determines the target encoding parameters of the video to be encoded in the following way:
  • determining the target encoding parameters for the videos to be encoded based on the total bitrate corresponding to the plurality of videos to be encoded, the video features of the videos to be encoded among the plurality of videos to be encoded, and the standard value of the video quality evaluation index includes: predicting the required prediction bitrate for the videos to be encoded based on the video features of the videos to be encoded; determining the required prediction encoding parameters for the videos to be encoded based on the required prediction bitrate of the videos to be encoded; obtaining the prediction bitrate and prediction quality evaluation index value of the video after encoding the videos to be encoded using the prediction encoding parameters; if the sum of the prediction bitrates corresponding to the multiple encoded videos corresponding to the plurality of videos to be encoded is less than the total bitrate, and the sum of the prediction quality evaluation index values corresponding to the multiple encoded videos is greater than the standard value of the video quality evaluation index, then the prediction encoding parameters are used as the target encoding parameters for the videos to be encoded
  • Multiple videos to be encoded have different video features, resulting in different bitrates required during encoding. Videos with more complex video features require a higher bitrate than those with less complex features. For example, consider videos where the proportion of independent video frames exceeds a preset threshold. For instance, a first video might consist of 100 frames, including 70 independent frames. Each independent frame needs to display its entire content, while the predicted frame's content is determined based on the difference between its preceding independent frame and the current predicted frame. A second video might also consist of 100 frames, including 30 independent frames and the remainder being predicted frames. In this case, the first video has a higher video feature complexity than the second video. Consequently, the first video requires a higher bitrate for encoding than the second video.
  • the predicted bitrates obtained from each of the multiple videos to be encoded are summed, and the sum of these sums is compared with the total bitrate.
  • the predicted quality assessment index values obtained from each video to be encoded are then summed and compared with the standard value of the video quality assessment index.
  • the sum of the predicted bitrates of the multiple encoded videos corresponding to the multiple videos to be encoded is less than the total bitrate, and the sum of the predicted quality evaluation index values of the multiple encoded videos is greater than the standard value of the video quality evaluation index, then it indicates that the predicted coding parameters for the videos to be encoded can meet the requirements of the video demand side for the quality of the encoded videos, and the predicted coding parameters are used as the target coding parameters for the videos to be encoded.
  • the above-mentioned determination of predictive coding parameters for each video to be encoded may include multiple candidate coding parameters.
  • the multiple candidate coding parameters are used to encode the video to be encoded respectively to obtain multiple predicted bitrates and multiple predicted quality evaluation index values. If the predicted bitrates and predicted quality evaluation index values corresponding to the multiple candidate coding parameters are all under the following conditions, then the target coding parameter is selected from the multiple candidate coding parameters: the sum of the predicted bitrates corresponding to the encoded videos is less than the total bitrate, and the predicted quality evaluation index values corresponding to the encoded videos are less than the standard value of the video quality evaluation index.
  • multiple videos to be encoded include two videos to be encoded.
  • Each video to be encoded has two candidate encoding parameters. After encoding the videos to be encoded using these two candidate encoding parameters, two corresponding predicted bitrates and two predicted quality evaluation index values are obtained.
  • the predicted bitrates of the encoded videos corresponding to all the videos to be encoded are summed to obtain the total predicted bitrate value, which is obtained by summing the predicted bitrates of each encoded video with the predicted bitrates of the other encoded videos.
  • the predicted quality evaluation index values of the encoded videos corresponding to all the videos to be encoded are summed to obtain the total predicted quality evaluation index value, which is obtained by summing the predicted quality evaluation index values of each encoded video with the predicted quality evaluation index values of the other encoded videos.
  • the total bitrate for multiple videos to be encoded is 5 Mbps (megabits per second).
  • the dataset contains two videos to be encoded: a first video A and a second video B.
  • the first video A contains two candidate coding parameters, Aa1 and Aa2.
  • Aa1 corresponds to a prediction bitrate of Aa11 and a prediction quality evaluation metric of Aa12.
  • Aa2 corresponds to a prediction bitrate of Aa21 and a prediction quality evaluation metric of Aa22.
  • the second video to be encoded, B contains two candidate coding parameters, Bb1 and Bb2.
  • the prediction bitrate corresponding to Bb1 is Bb11, and the corresponding prediction quality evaluation metric is Bb12.
  • the prediction bitrate corresponding to Bb2 is Bb21, and the corresponding prediction quality evaluation metric is Bb22.
  • the sum of the predicted bitrates corresponding to all the videos to be encoded includes the following types: the sum of Aa11 and Bb11, the sum of Aa11 and Bb21, the sum of Aa21 and Bb11, and the sum of Aa21 and Bb21.
  • the sum of the predicted quality assessment index values corresponding to all videos to be encoded includes the following types: the sum of Aa12 and Bb12, the sum of Aa12 and Bb22, the sum of Aa22 and Bb12, and the sum of Aa22 and Bb22.
  • the sum of any one of the predicted bitrates is less than the total bitrate of 5 Mbps, but the sum of any one of the predicted quality assessment metric values is less than the standard value of the video quality assessment metric corresponding to the video demand side.
  • the standard value of the video quality assessment metric corresponding to the video demand side can be greater than a second preset video quality assessment metric standard threshold, where the second preset video quality assessment metric standard threshold indicates that the video picture quality belongs to ultra-high definition quality.
  • the bitrate corresponding to ultra-high definition quality is greater than the bitrate corresponding to standard definition quality.
  • the target coding parameter can be selected from multiple candidate coding parameters corresponding to each video to be encoded, as follows:
  • the video to be encoded is encoded using the multiple candidate encoding parameters to obtain multiple predicted quality assessment index values after encoding. These multiple predicted quality assessment index values are then compared to obtain a target predicted quality assessment index value, which is greater than all other predicted quality assessment index values except the target value. The candidate encoding parameters corresponding to the target predicted quality assessment index value are then used as the target encoding parameters for the video to be encoded.
  • the target coding parameter for the first video to be encoded is candidate coding parameter Aa1
  • the target coding parameter for the second video to be encoded is candidate coding parameter Bb1.
  • the above is the third way to determine the target encoding parameters.
  • the target bitrate and target quality evaluation index values of the encoded video must meet the following conditions: the sum of the target bitrates is less than the total bitrate, and the sum of the target quality evaluation index values is greater than the standard value of the video quality evaluation index required by the video demander for the encoded video.
  • the target encoding parameters of the determined video to be encoded can satisfy the following conditions: the sum of the bitrates of the encoded videos is less than the total bitrate, and the sum of the quality evaluation index values of the encoded videos is greater than the sum of the subsequent quality evaluation index values of the encoded videos under other candidate encoding parameter conditions.
  • the bitrate of the encoded video is the target bitrate
  • the quality evaluation index of the encoded video is the target quality evaluation index value.
  • the bitrate of the encoded video is the candidate bitrate
  • the quality evaluation index of the encoded video is the candidate quality evaluation index value.
  • the sum of the target bitrates of the encoded videos is less than the total bitrate, and the sum of the candidate quality evaluation index values of the encoded videos is greater than the sum of the candidate quality evaluation index values of the other candidate coding parameters.
  • the sum of the target bitrates of encoded video A, encoded video B, and encoded video C is less than the total bitrate of the multiple videos to be encoded. Furthermore, the sum of the target quality evaluation index values of encoded video A, encoded video B, and encoded video C is greater than the sum of the candidate quality evaluation index values of the encoded videos under other candidate encoding parameters (the total value obtained by adding the candidate quality evaluation index values corresponding to the above three encoded videos respectively).
  • the second embodiment of this disclosure provides a video encoding apparatus.
  • a video encoding apparatus For related parts, please refer to the description of the corresponding method embodiment. Please refer to Figure 5, which is a schematic diagram of a video encoding apparatus provided in the second embodiment of this disclosure.
  • the video encoding apparatus shown in the figure includes: a video feature acquisition unit 501, used to perform feature extraction processing on multiple videos to be encoded to obtain video features of the videos to be encoded; a target encoding parameter determination unit 502, used to determine the target encoding parameters of the videos to be encoded based on the total bitrate corresponding to the multiple videos to be encoded and the video features of the videos to be encoded; and an encoding processing unit 503, used to encode the videos to be encoded according to the target encoding parameters; wherein, when the encoding parameters of the videos to be encoded are the corresponding target encoding parameters, the bitrate of the encoded video after encoding processing is the target bitrate, the quality evaluation index value of the encoded video is the target quality evaluation index value, the sum of the target bitrates corresponding to the encoded videos is less than the total bitrate, and the sum of the target quality evaluation index values corresponding to the encoded videos is greater than the standard value of the video quality evaluation index required by the video demand side for the multiple encode
  • the third embodiment of this disclosure provides an electronic device.
  • the electronic device shown in the figure includes: a memory and a processor; the memory is used to store a computer program, and after the computer program is run by the processor, it executes the method provided in the embodiment of this disclosure.
  • the fifth embodiment of this disclosure provides a computer program product, the computer program product comprising: a computer program that, when executed by a processor of an electronic device, enables the processor to at least implement the methods provided in the foregoing embodiments.
  • a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
  • Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and/or non-volatile memory, such as read-only memory (ROM) or flash RAM.
  • RAM random access memory
  • ROM read-only memory
  • flash RAM flash random access memory
  • Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include non-transitory computer-readable media, such as modulated data signals and carrier waves.
  • PRAM phase-change memory
  • SRAM static random access memory
  • DRAM dynamic random access memory
  • RAM random access memory
  • ROM read-only memory
  • this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
  • a computer-usable storage media including, but not limited to, disk storage, CD-ROM, optical storage, etc.
  • the embodiments disclosed herein may involve the use of user data.
  • user-specific personal data may be used within the scope permitted by applicable laws and regulations of the country in which it is used (e.g., with the user's explicit consent and effective notification to the user, etc.).
  • user information including but not limited to user device information, user personal information, etc.
  • data including but not limited to data used for analysis, stored data, displayed data, etc.
  • the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

Des modes de réalisation de la présente divulgation concernent un procédé et un appareil de codage vidéo. Le procédé consiste à : lors de l'extraction de caractéristiques vidéo de vidéos à coder parmi une pluralité de vidéos à coder, sur la base d'un débit binaire total correspondant à la pluralité de vidéos à coder et des caractéristiques vidéo des vidéos à coder, déterminer des paramètres de codage cibles des vidéos à coder ; puis utiliser les paramètres de codage cibles pour effectuer un traitement de codage sur les vidéos à coder, de façon à obtenir des vidéos codées. Dans le procédé, les paramètres de codage cibles des vidéos à coder peuvent être déterminés directement au moyen du débit binaire total correspondant à la pluralité de vidéos à coder et des caractéristiques vidéo des vidéos à coder, sans qu'il soit nécessaire d'effectuer un traitement de précodage sur les vidéos à coder pour obtenir des caractéristiques de codage de vidéos ayant subi un prétraitement. Par conséquent, la qualité de codage vidéo est assurée tout en améliorant l'efficacité de détermination des paramètres de codage cibles des vidéos à coder.
PCT/CN2025/102686 2024-07-26 2025-06-23 Procédé et appareil de codage vidéo Pending WO2026021082A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202411023034.2 2024-07-26
CN202411023034.2A CN119182935A (zh) 2024-07-26 2024-07-26 一种视频编码方法及装置

Publications (1)

Publication Number Publication Date
WO2026021082A1 true WO2026021082A1 (fr) 2026-01-29

Family

ID=93903070

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2025/102686 Pending WO2026021082A1 (fr) 2024-07-26 2025-06-23 Procédé et appareil de codage vidéo

Country Status (2)

Country Link
CN (1) CN119182935A (fr)
WO (1) WO2026021082A1 (fr)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119182935A (zh) * 2024-07-26 2024-12-24 淘宝（中国）软件有限公司 一种视频编码方法及装置
CN119850684B (zh) * 2025-03-21 2025-08-12 深圳市德斯戈智能科技有限公司 一种机器视觉集成系统中的目标跟踪方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190253763A1 (en) * 2018-02-15 2019-08-15 Cisco Technology, Inc. Constant Quality Video Encoding with Encoding Parameter Fine-Tuning
CN113938682A (zh) * 2020-06-29 2022-01-14 北京金山云网络技术有限公司 视频编码方法、装置和电子设备
CN117478886A (zh) * 2023-10-31 2024-01-30 小红书科技有限公司 多媒体数据编码方法、装置、电子设备及存储介质
CN118158414A (zh) * 2024-04-11 2024-06-07 书行科技(北京)有限公司 视频编码方法、装置、电子设备及计算机存储介质
CN119182935A (zh) * 2024-07-26 2024-12-24 淘宝（中国）软件有限公司 一种视频编码方法及装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190253763A1 (en) * 2018-02-15 2019-08-15 Cisco Technology, Inc. Constant Quality Video Encoding with Encoding Parameter Fine-Tuning
CN113938682A (zh) * 2020-06-29 2022-01-14 北京金山云网络技术有限公司 视频编码方法、装置和电子设备
CN117478886A (zh) * 2023-10-31 2024-01-30 小红书科技有限公司 多媒体数据编码方法、装置、电子设备及存储介质
CN118158414A (zh) * 2024-04-11 2024-06-07 书行科技(北京)有限公司 视频编码方法、装置、电子设备及计算机存储介质
CN119182935A (zh) * 2024-07-26 2024-12-24 淘宝（中国）软件有限公司 一种视频编码方法及装置

Also Published As

Publication number Publication date
CN119182935A (zh) 2024-12-24

Similar Documents

Publication Publication Date Title
Wang et al. VideoSet: A large-scale compressed video quality dataset based on JND measurement
US11049286B2 (en) Deep neural network color space optimization
US9282330B1 (en) Method and apparatus for data compression using content-based features
US10887365B2 (en) System and methods for bit rate control
US9525869B2 (en) Encoding an image
CN112913237A (zh) 使用深度神经网络的人工智能编码和人工智能解码方法和设备
CN111182303A (zh) 共享屏幕的编码方法、装置、计算机可读介质及电子设备
US11259029B2 (en) Method, device, apparatus for predicting video coding complexity and storage medium
KR20190117651A (ko) 이미지 프로세싱 및 비디오 압축 방법
CN110225343B (zh) 一种视频编码的码率控制方法及装置
WO2021129007A1 (fr) Procédé et dispositif de détermination de débit binaire vidéo, appareil informatique, et support de stockage
CN119182935A (zh) 一种视频编码方法及装置
CN101056408B (zh) 用于确定视频编码中跳跃模式的阈值的方法和装置
Li et al. A real-time blind quality-of-experience assessment metric for http adaptive streaming
WO2026011968A1 (fr) Procédé et appareil de codage vidéo, dispositif, support de stockage lisible
Feng et al. Content-adaptive rate control method for user-generated content videos
Amirpour et al. DeepVCA: Deep video complexity analyzer
Yang et al. A content-based rate control algorithm for screen content video coding
Amirpour et al. VQM4HAS: A real-time quality metric for HEVC videos in HTTP Adaptive Streaming
Benjak et al. Deferred demosaicking: efficient first-person view drone video encoding
CN116347088A (zh) 确定编码模式的方法、装置、电子设备及存储介质
Akramullah Video quality metrics
US10848772B2 (en) Histogram-based edge/text detection
CN109951707A (zh) 一种目标运动矢量选择方法、装置、电子设备及介质
CN110446048B (zh) 帧内预测场景中预测模式的选取方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 25843466

Country of ref document: EP

Kind code of ref document: A1