CN103200419B - High-speed recognizing method of change degree of video content - Google Patents

Abstract

The present invention provides a method for identifying the change degree of video content at a high speed. The method includes: establishing a buffer zone for the video to be evaluated according to the application scene, the application scene includes an offline scene and an online scene, and initializing local variables; Create the buffer header to read the next frame, and judge the type of the frame, if it is a key frame, then execute the next step, otherwise execute this step; calculate the number of bytes of the frame, and add it to the total number of bytes; from Create a buffer header for the video to be evaluated and read the next frame, continue to judge the type of the frame, if it is a predicted frame, execute the previous step, if not, execute the next step; pass the number of bytes and the total bytes of the frame Calculate the degree of change in video content. The method of the invention can evaluate the content characteristics of the video at a high speed with relatively low computational complexity and space overhead, and can meet the requirement of simple and fast classification of the video content.

Description

A high-speed method for identifying the degree of change in video content

technical field

The invention belongs to the technical field of multimedia communication, and relates to a method for identifying changes in video content, in particular to a method for identifying the degree of change in video content at high speed.

Background technique

With the development of the Internet, audio and video media streams gradually occupy the main traffic of the network. However, the Internet is a best-effort transmission network. During the transmission of streaming media, bandwidth, packet loss, jitter, and delay occur from time to time, which will adversely affect the video quality. . Previous research on QoS (Quality of Service) was aimed at improving network performance and overall resource utilization, but current ISPs and ICPs pay more attention to the actual experience of contracted customers, and QoS cannot meet this need. Therefore, QoE (Quality of Experience) is introduced to describe the viewing quality of multimedia users.

At present, many research organizations such as ITU and VQEG have proposed their own video quality evaluation models, and the more famous ones are G.1070, E-Model, and Evalvid. These models mainly consider the impact of QoS parameters on QoE, but ignore the impact of video content on QoE. Existing experiments show that different video contents have significantly different effects on QoE. Videos with drastic content changes (such as football games, etc.) are much more affected by packet loss and jitter than videos with gentle content changes (such as news broadcasts, etc.), as shown in Figure 1. This difference leads to a relatively low accuracy of the QoE evaluation model.

Existing video content recognition technology based on pattern recognition can analyze video image information and obtain content information. However, this method has problems such as slow recognition speed, high resource overhead, and requires a large amount of data training in the early stage, and the information it recognizes is far more than the QoE evaluation model needs, and there are resource redundancy and waste. Therefore, it is difficult to introduce it in the real-time monitoring of QoE. In addition, there are related research at home and abroad to estimate the content information of the video by analyzing the pixel information of each image in the video, or analyzing the information of the DCT coefficient in the predicted frame motion vector after encoding.

Contents of the invention

In view of the shortcomings of the prior art described above, the purpose of the present invention is to provide a method for high-speed recognition of the degree of change in video content, which is used to solve the problems of slow recognition speed, large resource overhead, and training requiring a large amount of data in the prior art. And the identified information is much more than what the QoE evaluation model needs, and there are problems of resource redundancy and waste.

In order to achieve the above object and other related objects, the present invention provides a method for identifying the change degree of video content at high speed. The methods include:

S1, set the number of key frame bytes and the total number of bytes in the video frame to 0, set up a buffer for the video to be evaluated according to the application scenario, the application scenario includes an offline scenario and an online scenario, and initialize local variables; when the application scenario For online evaluation, a network buffer is established for the video to be evaluated. The elements in the network buffer are the queues of data packets sent sequentially from the server to the client, and the network buffer established for the video to be evaluated points to the network data of the media stream server Packet queue; when the application scenario is offline evaluation, a file buffer is established for the video to be evaluated, and the elements in the file buffer are continuous frames of the video to be evaluated;

S2. Read the next frame from the buffer head of the video to be evaluated. When the application scenario is offline evaluation, the elements in the file buffer area are frames, and the first frame is directly read from the buffer area, and the data of the first frame is stored. into the temporary frame data cache, and perform the next step; when the application scenario is online evaluation, the elements in the network buffer area established for the video to be evaluated are data packets, and all data packets that store the first frame need to be read first, and then Assemble the data packets into a complete frame, and proceed to the next step; for the media stream of the RTP protocol, in order to ensure the number of data packets to be read, clear the temporary frame data buffer data, from the video to be evaluated Read out the first data packet in the built-up buffer, check whether there is a flag in the RTP protocol field in the data packet, and take out the data in the RTP data field in the data packet and store it in the temporary frame data buffer data. If there is a flag bit in the RTP protocol field in the data packet, it means that the data packet is the last data packet of the frame carried, and that a frame has been assembled, and the next step can be performed; if the RTP protocol field in the data packet is checked If there is no flag bit, it means that the data packet is a middle data packet of the carried frame;

S3, judging whether the frame read out from the buffer head of the video to be evaluated is a key frame, if so, then perform the next step; if not, then return to step S2, and re-execute step S2;

S4, calculating the number of bytes occupied by the temporary frame data buffer, and accumulating to the total number of bytes;

S5, read out the next frame from the head of the buffer for the video to be evaluated and store it in the temporary frame data buffer, this step is consistent with step S2;

S6, judging whether the frame read from the buffer header for the video to be evaluated is a predicted frame, if so, return to step S4; if not, perform the next step;

S7. Calculate a video content change degree measurement value.

Preferably, when the application scenario is online evaluation, it means that the video to be evaluated already exists; when the application scenario is offline evaluation, it means that the video to be evaluated is generated in real time; in the offline application scenario, the frame in the file buffer is established for the video to be evaluated The storage order of the video to be evaluated must be consistent with the storage order of the video to be evaluated; in the online application scenario, the storage order of the frames in the network buffer area established for the video to be evaluated must be consistent with the sequence of encoding of the video to be evaluated.

Preferably, the step S2 also includes: reading the next frame from the head of the buffer for the video to be evaluated. In different application scenarios, the execution methods are different; The element is a video frame, so the frame is directly read from the file buffer and returned; in the online application scenario, the network buffer area established for the video to be evaluated stores network data packets, and all stored data packets need to be read. out, assembled into a complete video frame and returned.

Preferably, in the step S4, if the temporary frame data buffer is a key frame, then the number of key frame bytes is the number of bytes occupied by the temporary frame data buffer.

Preferably, the video content change degree metric value MDVC is a decimal with an interval of [0,1), and MDVC indicates the severity of video content change.

As mentioned above, the method for identifying the degree of change of video content at high speed according to the present invention has the following beneficial effects:

1. The present invention has the characteristics of low computational complexity, small space overhead, and fast convergence speed, which can well solve the problem of video content classification;

2. The present invention can quickly analyze the content characteristics of the video;

3. The present invention can be used to identify videos encoded by MPEG4, H.264 (but not limited to these two encodings), and has high accuracy.

Description of drawings

FIG. 1 is a flow chart of the method for identifying the degree of change of video content at high speed according to the present invention.

Fig. 2(a) shows a schematic diagram of the difference of the QoE model for the football game video.

Fig. 2(b) shows a schematic diagram of the difference of QoE models for news broadcast video.

FIG. 3 is a schematic diagram showing the relationship between video content and MDVC under different encoding parameters in the method for identifying the degree of change of video content at high speed according to the present invention.

FIG. 4 is a schematic diagram showing different MOS curves of videos with different contents affected by the packet loss rate in the method for quickly identifying the change degree of video content of the present invention.

Detailed ways

Embodiments of the present invention are described below through specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation modes, and various modifications or changes can be made to the details in this specification based on different viewpoints and applications without departing from the spirit of the present invention.

Please refer to attached picture. It should be noted that the diagrams provided in this embodiment are only schematically illustrating the basic idea of the present invention, and only the components related to the present invention are shown in the diagrams rather than the number, shape and shape of the components in actual implementation. Dimensional drawing, the type, quantity and proportion of each component can be changed arbitrarily during actual implementation, and the component layout type may also be more complicated.

The present invention will be described in detail below in conjunction with the embodiments and the accompanying drawings.

Current mainstream coding, such as H.264, MPEG4, etc., divides video frames into two types: key frames (I frames) and predictive frames (P frames, B frames). Wherein, the key frame is used to store complete image information, and the predicted frame is used to store a large amount of motion vectors and a small amount of image information. Under the same encoding conditions, the encoding conditions include encoding parameters such as resolution, code rate, GOP (GROUPOF PICTURE, referred to as group of pictures) style, the larger the information contained in the motion vector, the faster the frame changes before and after the video, so the video motion The degree is more severe. By counting the number of predicted frame bytes, the motion degree of the video can be estimated, and the method for identifying the change degree of the video content at high speed according to the present invention has a fast recognition speed and a small amount of temporary data, and can be applied to real-time monitoring.

This embodiment provides a method for high-speed identification of the degree of change in video content. The method is shown in FIG. 1 and specifically includes:

S1, the initialization step, the key frame bytes (KeySize) and the total bytes (TotalSize) in the video frame are set to 0, and a buffer (pFrameBuffer) is established for the video to be evaluated according to the application scenario Task (offline or online), The application scenario includes an offline scenario and an online scenario, and local variables are initialized; when the application scenario Task is an online evaluation, a network buffer is established for the video to be evaluated, and the elements in the network buffer are the queues of packets sent sequentially from the server to the client , and point the network buffer (pFrameBuffer) for the video to be evaluated to the network data packet queue at the media streaming server; when the application scenario (Task) is offline evaluation, a file buffer is established for the video to be evaluated, and the file buffer The elements in are the continuous frames of the video to be tested; when the application scenario Task is online evaluation, it means that the video to be evaluated already exists, for example, VoD and other video-on-demand services; when the application scenario Task is offline evaluation, it means that the video to be evaluated already exists. Review videos are generated in real time, e.g. video conferencing, live broadcast services. The buffer (pFrameBuffer) established for the video to be evaluated can be a part of frames in a video stream, or all frames of the entire video. In the offline application scenario, the storage order of the frames in the file buffer for the video to be evaluated must be consistent with the storage order of the video to be evaluated; in the online application scenario, the storage order of the frames in the network buffer for the video to be evaluated must be consistent with The order of encoding of the video to be evaluated is consistent.

S2. Read the next frame from the head of the buffer (pFrameBuffer) for the video to be evaluated. When the application scenario Task is offline evaluation, the elements in the file buffer (pFrameBuffer) are frames, and the header is directly read from the buffer. frame, store the first frame data in the temporary frame data cache, and execute the next step; when the application scenario Task is online evaluation, the element in the network buffer (pFrameBuffer) established for the video to be evaluated is a data packet, which needs to be Read and store all data packets of the first frame, then assemble the data packets into a complete frame, and proceed to the next step; The number of read data packets is to clear the temporary frame data buffer data, read the first data packet from the buffer (pFrameBuffer) established for the video to be evaluated, and check whether there is a Mark flag in the RTP protocol field in the data packet. And the data of the RTP data field in the data packet is taken out and stored in the temporary frame data buffer data, if there is a flag bit in the RTP protocol field in the data packet, it means that the data packet is the last data of the frame carried package, and indicate that a frame has been assembled, and the next step can be performed; if there is no flag bit in the RTP protocol field in the data packet, it means that the data packet is the middle data packet of the carried frame; it should be noted Yes, reading the next frame from the header of the buffer (pFrameBuffer) for the video to be evaluated is performed in different ways in different application scenarios; in offline application scenarios, the elements stored in the file buffer for the video to be evaluated are video frames , so read the frame directly from the file buffer and return it; in the online application scenario, the network cache area established for the video to be evaluated stores network data packets, and all stored data packets need to be read out and assembled into a Complete video frames are returned.

S3, judging whether the frame read from the buffer (pFrameBuffer) header for the video to be evaluated is a key frame, if so, then perform the next step; if not, then return to step S2, and re-execute step S2;

S4. Calculate the number of bytes (Size) occupied by the temporary frame data buffer, and add it to the total number of bytes (TotalSize). If the temporary frame data buffer is a key frame, then the key frame byte number (KeySize) is the occupied byte number (Size) of the temporary frame data buffer;

S5, read out the next frame from the buffer (pFrameBuffer) header for the video to be evaluated and store it in the temporary frame data buffer, this step is consistent with step S2; when the application scene Task is offline evaluation, the file buffer (pFrameBuffer ) is a frame, read the first frame directly from the buffer, store the first frame data in the temporary frame data buffer, and execute the next step; when the application scenario Task is online evaluation, establish The elements in the network cache area (pFrameBuffer) are data packets. It is necessary to read all the data packets storing the first frame first, then assemble the data packets into a complete frame, and proceed to the next step; for the RTP protocol (Real -time Transport Protocol (referred to as real-time transport protocol) media stream, in order to ensure the number of data packets that need to be read, the temporary frame data buffer data is cleared, and the queue is read from the buffer (pFrameBuffer) established for the video to be evaluated For the first data packet, check whether there is a Mark flag in the RTP protocol field in the data packet, and take out the data in the RTP data field in the data packet and store it in the temporary frame data buffer data. If it is found that the RTP protocol field in the data packet exists Flag bit, then it means that the data packet is the last data packet of the frame carried, and a frame has been assembled, and the next step can be performed; if there is no flag bit in the RTP protocol field in the data packet, then it means The data packet is a middle data packet of the carried frame. It should be noted that the execution method of reading the next frame from the header of the buffer (pFrameBuffer) for the video to be evaluated is different in different application scenarios; in offline application scenarios, the elements stored in the file buffer for the video to be evaluated are established It is a video frame, so the frame is directly read from the file buffer and returned; in the online application scenario, the network buffer area established for the video to be evaluated stores network data packets, and all stored data packets need to be read out. Assembled into a complete video frame and returned.

S6, judging whether the frame read from the buffer (pFrameBuffer) header for the video to be evaluated is a predicted frame, if so, then return to step S4; if not, then perform the next step;

S7, calculating the video content change degree metric value MDVC, namely The video content change degree metric value MDVC is a decimal with an interval of [0,1), and MDVC indicates the severity of video content change. Qualitatively, the larger the MDVC, the more active the video content features.

In practical applications, the method for identifying the degree of change in video content at high speed described in this embodiment can solve the problem of video content estimation quickly and calibratedly, and provide quantitative data for the video QoE evaluation model, so that the model can realize content self-adaptation, as shown in the figure Figure 2 shows the difference of QoE models under different video contents. Figure 2(a) shows the QoE model of football game video, and Figure 2(b) shows the QoE model of news broadcast video. The present invention can prove its feasibility with following experiment. In the experiment, 12 video clips with different contents were divided into two groups, one group was Fast-Moving Video, and the other group was Slow-Moving Video. All videos are encoded according to different encoding parameters, as shown in Table 1, and their respective MDVC values are calculated.

Table 1: Test parameter values

parameter value frame rate fps_v 24,25,30,48,60 code rate bitrate_v 386k, 512k, 1024k, 2400k, 5000k resolution res_v 320x240, 352x288, 640x480, 960x720 GOP size λ 12,30,60,120 The frequency of B frames between P frames β 0,1,2,3,4 encoding type H.264, MPEG4

Figure 3 shows the relationship between video content and MDVC under different encoding parameters. In the figure, '○' and '◇' represent the MDVC values of H.264 and MPEG4 encoded slow motion video, and '+' and '*' represent the H. 264 and MPEG4 encoded MDVC values for high-speed mobile video. It can be seen from the results in the figure that the MDVC value of high-speed moving video is higher than that of slow-moving video under all encoding conditions, which proves the correctness of the method.

Using the calculation of MDVC, the content-related parameters can be provided for the video QoE evaluation model. Figure 4 shows different MOS curves of videos with different contents affected by the packet loss rate. It can be seen that the larger the MDVC (high-speed motion video), the faster the MOS value drops, indicating that the video QoE damage is more severe. Introducing the video content measurement method in the QoE evaluation model can realize content self-adaptation for QoE evaluation, thereby improving measurement accuracy.

The method for identifying the change degree of video content at high speed in the present invention can evaluate the content characteristics of video at high speed with low computational complexity and space overhead, and can meet the need for simple and fast classification of video content. The present invention can be used for video-on-demand services such as VoD, and can also be used for real-time streaming media services such as video conferences.

The feature of the present invention is to use less space and time overhead to quickly analyze the content characteristics of the video. Compared with the video content analysis method based on pattern recognition, the algorithm has the characteristics of low computational complexity, small space overhead, and fast convergence speed. It can solve the problem of video content classification very well. Practice shows that this algorithm can be used to identify videos encoded by MPEG4, H.264 (but not limited to these two encodings), and has high accuracy.

The present invention can quickly evaluate the change degree of video content under O(1) computational complexity, and is applicable to two common video playback scenarios, online and offline, and provides content for the user quality of experience (QoE) evaluation model combining video and video and audio Measure parameters to improve the evaluation accuracy of the model.

To sum up, the present invention effectively overcomes various shortcomings in the prior art and has high industrial application value.

The above-mentioned embodiments only illustrate the principles and effects of the present invention, but are not intended to limit the present invention. Anyone skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by those skilled in the art without departing from the spirit and technical ideas disclosed in the present invention should still be covered by the claims of the present invention.

Claims (4)

1. A method for high-speed recognition of the degree of change of video content, characterized in that the method comprises: S1, set the number of key frame bytes and the total number of bytes in the video frame to 0, set up a buffer for the video to be evaluated according to the application scenario, the application scenario includes an offline scenario and an online scenario, and initialize local variables; when the application scenario For online evaluation, a network buffer is established for the video to be evaluated. The elements in the network buffer are the queues of data packets sent sequentially from the server to the client, and the network buffer established for the video to be evaluated points to the network data of the media stream server Packet queue; when the application scenario is offline evaluation, a file buffer is established for the video to be evaluated, and the elements in the file buffer are continuous frames of the video to be evaluated; S2. Read the next frame from the buffer head of the video to be evaluated. When the application scenario is offline evaluation, the elements in the file buffer area are frames, and the first frame is directly read from the buffer area, and the data of the first frame is stored. into the temporary frame data cache, and perform the next step; when the application scenario is online evaluation, the elements in the network buffer area established for the video to be evaluated are data packets, and all data packets that store the first frame need to be read first, and then Assemble the data packets into a complete frame, and proceed to the next step; for the media stream of the RTP protocol, in order to ensure the number of data packets to be read, clear the temporary frame data buffer data, from the video to be evaluated Read out the first data packet in the built-up buffer, check whether there is a flag in the RTP protocol field in the data packet, and take out the data in the RTP data field in the data packet and store it in the temporary frame data buffer data. If there is a flag bit in the RTP protocol field in the data packet, it means that the data packet is the last data packet of the frame carried, and that a frame has been assembled, and the next step can be performed; if the RTP protocol field in the data packet is checked If there is no flag bit, it means that the data packet is a middle data packet of the carried frame; S3, judging whether the frame read out from the buffer head of the video to be evaluated is a key frame, if so, then perform the next step; if not, then return to step S2, and re-execute step S2; S4, calculating the number of bytes occupied by the temporary frame data buffer, and accumulating to the total number of bytes; S5, read out the next frame from the head of the buffer for the video to be evaluated and store it in the temporary frame data buffer, this step is consistent with step S2; S6, judging whether the frame read from the buffer header for the video to be evaluated is a predicted frame, if so, return to step S4; if not, perform the next step; S7. Calculate a video content change degree measurement value. 2. The method for identifying the degree of change of video content at high speed according to claim 1, characterized in that: when the application scenario is online evaluation, it means that the video to be evaluated already exists; when the application scenario is offline evaluation, it means that the video to be evaluated is Real-time generation; in the offline application scenario, the storage order of the frames in the file buffer established for the video to be evaluated must be consistent with the storage order of the video to be evaluated; in the online application scenario, the storage of frames in the network buffer area established for the video to be evaluated The sequence must be the same as the coded sequence of the video to be evaluated. 3. The method for identifying the degree of change of video content at a high speed according to claim 1, characterized in that: in the step S4, if the temporary frame data buffer is a key frame, then the key frame byte count is the temporary The number of bytes occupied by the frame data buffer. 4. the method for the high-speed identification video content change degree according to claim 1 is characterized in that: described video content change degree metric value MDVC is an interval and is the decimal number of [0,1), and MDVC represents the sharpness of video content change degree.