TW201141237A - Frame rate up conversion system and method - Google Patents

Abstract

The invention is directed to a frame rate up conversion (FRUC) system and method. A motion estimation (ME) unit is configured to generate at least one motion vector (MV) according to a frame input. A triple-line buffer based motion compensation (MC) unit is configured to generate an interpolated frame according to the MV, a reference frame and a current frame, thereby generating a frame output with a frame rate higher than a frame rate of the frame input.

Description

201141237 VI. Description of the Invention: [Technical Field] [0001] The present invention relates to an improvement in frame rate, and more particularly to a spatial interpolation (spat ia 1 i nterpo 1 at i on) processing for an inner frame With flat. Sliding. [Prior Art] [0002] The frame rate up conversion (FRUC) is often applied to the display of digital images (for example, digital television), which can generate one or more images between two adjacent original frames. An illustration box to increase the rate at which the frame is displayed, for example from 60 Hz to 12 〇 Hz or even 24 〇 Hz. The inner frame is usually generated by the interpolation technique of motion compensation (m〇ti〇n c〇mpensati〇n, MC). The first figure shows a block-based action estimation/compensation technique that produces an inner frame based on the previous frame A and the current frame. First, it is estimated that the macroblock (MB) of the current frame is opposite to the action of the corresponding macroblock of the previous frame. Next, based on the motion estimation to get the inner illustration 裎. ❹ ] ] 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于 于There is usually a boundary effect along the boundary between adjacent blocks (side (1). To overcome the problem of broken regions' traditional systems or methods use line-buffers to store the pixels of the current block and Pre-block and down-block certain pixels. For example, for an 8x8 block-based system or method, use ten line buffers to store the 8 lines of the current block'. - The first line of the line and the next block. Since the memory is 099114570 Form No. A0101 Page 3 / Total 22 Page 0992025738-0 201141237 It takes a lot of (four) to take the pixels of the ten line buffers, so the traditional system Or the method does not display the image. In addition, the ten line buffers n will increase the area of the transfer area. [0004] [0005] [0006] [0007] In view of the traditional line and method, there is no material rhyme (4) Problems and boundary effects, so ' Μ A novel system and method for reading, and the U's square has no broken (four) domain and boundary effect inner frame. [Invention] In view of the above, the present invention is to provide a lifting frame. A system and method for fr_ rate up conversion (FRUC) uses less buffer resources than conventional to repair and smooth the resulting inset frame. According to an embodiment of the invention, the elevated frame rate system includes motion estimates ( Motion estimation, ME) unit and three-line buffer-based motion compensation material. The action material Yang input input generates at least one motion vector (m〇ti〇n German input, MV). The motion compensation unit is based on the motion vector and the reference map. The frame and the i-frame are used to generate an inner frame, whereby a frame is rotated, and the frame rate is higher than the frame rate of the frame input. [Embodiment] FIG. 2A shows an embodiment of the present invention. A system block diagram of a frame rate up conversion (FRUC). A second block diagram shows a flow chart of a method for increasing the frame rate according to an embodiment of the present invention. Motion estimation unit (motion estimation) 21 and motion compensation (MC) unit 22 099114570 Form number A0101 Page 4 of 22 page 0992025738-0 201141237 〇 [0008]. In step 31 'action estimation unit 21 A continuous frame round with a raw frame rate (eg, hidden z) is received for generating an action vector ((4) _ Μ, or an action vector map (Vector map, MV map). In step 32, the motion compensation unit 22 (especially a motion compensation unit based on a triple llne buffer) is based on an action vector/action vector map, a reference frame (eg, a front-frame or a back_frame), and The current frame is used to generate an inset frame, thereby producing a continuous frame output with a frame rate increase (eg, swollen z). In the embodiment of the invention, block-based motion compensation is employed. The third A diagram shows a detailed block diagram of the action compensation unit 22 of the second A diagram in the embodiment of the present invention. Fig. 3B is a detailed flow chart showing the steps (step 32) of the inset frame in the second B diagram of the embodiment of the present invention. In the present embodiment, the motion compensation unit 22 includes a temp〇ral interpolation unit 221, a spatial interpolation unit 222, and a smoothing unit 223. In step 321, the time interpolation unit 221 generates a time frame according to the motion vector/action vector map, the reference frame and the current frame' (where the reference frame and the current frame are available from the motion estimation unit 21 or the figure). The frame storage memory is obtained. In view of the fact that there is often a cracked (or split) region in the illustration frame during the time period caused by the block-based motion compensation, in step 322, the spatial interpolation unit 222 is used for the time illustration. The box performs spatial interpolation, _ = area. For details on the repair of the rupture zone, there will be a boundary effect between the blocks based on the block-based compensation. Therefore, in step 323, a smoothing unit is used to reduce the boundary effect. Details on block boundary smoothing: Post 099114570 Form No. A0101 Page 5 of 22 0992025738-0 201141237 This manual will be explained in detail. 4 is a detailed block diagram of the spatial interpolation unit 222 of the third A diagram in the embodiment of the present invention. Figure 4B shows a detailed flow chart of the step of spatial interpolation of the third B diagram to repair the rupture zone (step 322) in the embodiment of the present invention. In the present embodiment, the spatial interpolation unit 222 includes a memory 2221, a three-line buffer 2222, and a spatial interpolation processor 2223. The memory 222 1 provides lines of some pixel blocks. The three-line buffer 2222 includes three line buffers for storing the current line to be processed, the last line of the previous block, and the first line of the next block (step 3222). Then, spatial interpolation is performed on the current line using the spatial interpolation processor 2223 according to the first line of the previous (upper end adjacent) block and the first line of the next (lower end adjacent) block (step 3223). . Since this embodiment uses only a three-wire buffer for spatial interpolation (and smoothing), this example can significantly reduce hardware resources and speed up interpolation (and smoothing) compared to conventional systems and methods.

[0010] FIG. 5A shows an example in which the last line of the previous block N-1 is stored in the buffer 1, and the current line of the current block N is stored in the buffer 2 and the next block N+1. The first line is stored in the buffer 3, and the block N-1, the block N and the block N+1 are contiguous blocks in the vertical direction of the image. For the same block N, the following line covers the contents of the buffer 2 each time. As another example shown in Fig. 5B, after processing the last line of the block N, the first line of the block N + 1 becomes the current line. Since the current line has been previously stored in the buffer 3, it is not necessary to retrieve it from the memory 2221 again. Furthermore, the last line remaining in the block N of buffer 2 will be the last line of the previous block N. At the same time, the block is retrieved by the memory 2221. 099114570 Form No. A0101 Page 6 of 22 0992025738-0 201141237 The first line of N + 2 is stored in the buffer j. For the same block N+i, the following line covers the contents of the buffer 3 (not the buffer 2 shown in Figure 5A). Thereby, the examples shown in the fifth A diagram and the fifth B diagram can be repeatedly executed in all the blocks. [0011] The sixth figure shows an example of performing spatial interpolation (step 3222) by spatially interpolating the processor 2 2 2 3 . In one embodiment, spatial interpolation is performed on the pixels pc of the current line based on the pixel pi of the last line of the previous block and the pixel port 2 of the first line of the lower_block. For example, the value of the pixel pc can be calculated by the following equation: pc = [pl * nl + p2 * n2] / (nl + n2), where nl and n2 are respectively the weighting of the pixel "and the weighting." [0012] In an embodiment, spatial interpolation is performed on the pixels 目前 of the current line according to the following four pixels: image correction of the last _ line of the pre-block, the first (four) pixel P2 of the next block, and the adjacent area of the left side The image of the block is, for example, the pixel p 4 adjacent to the right block. [0013] Next, the smoothing process is performed on the empty_leak frame by the flattening unit 223 (step 323). In this embodiment, Low-pass 〇W〇P-Pass filtering (LPF) is used to smooth the block boundary, thereby reducing the boundary effect. The seventh is to perform the smoothing (four) sub. In this real pass, according to the current line of the pixel be itself and the former - the pixel b of the last __ line of the block is used to smooth the pixel bc of the current line. For example, the value of the smoothed pixel be' can be calculated by the following formula: be' = [bl * nl + bc * n2] / ( nHn2), where nl and n2 are the weighting of pixels 1)1 and 1) respectively; 099114570 Form No. A0101 Page 7 of 22 0992025738-0 201141237 [0014] The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention; all other equivalent changes or modifications which are not departing from the spirit of the invention should be included in the following [0015] The first figure shows an example of generating an inner frame according to the previous frame and the current frame. The second A shows the frame rate of the embodiment of the present invention. System Block Diagrams FIG. 2B is a flow chart showing a method for increasing the frame rate according to an embodiment of the present invention. FIG. 3A is a detailed block diagram showing a motion compensation (MC) unit of the second A picture in the embodiment of the present invention. FIG. 3B is a detailed block diagram showing the steps of generating the inset frame in the second B diagram of the embodiment of the present invention. FIG. 4A is a detailed block diagram showing the spatial interpolation unit of the third A diagram in the embodiment of the present invention. Figure B shows a detailed flow chart of the steps of spatial interpolation of the third B diagram to repair the rupture zone in the embodiment of the present invention. The fifth A diagram and the fifth B diagram illustrate the last line, the current line and the lower part of the previous block. The first block The lines are stored in the three-line buffer. The sixth figure shows an example of performing spatial interpolation by spatially interpolating the processor. The seventh figure shows an example of performing smoothing. [Key Symbol Description] 099114570 Form No. A0101 Page 8 / Total 22 pages 0992025738-0 201141237 [0016] Ο 21 motion estimation unit 22 motion compensation unit 221 time interpolation unit 222 space interpolation unit 223 smoothing unit 2221 memory 2222 three-wire buffer 2223 space interpolation processor 31-32 Steps 321-323 Step 3222-3223 Step pi ' p2 ' p3 ' p4 ' pc ' bl ' be Pixel ο pTC Special 099114570 Form No. A0101 Page 9 / Total 22 Page 0992025738-0

Claims (1)

201141237 VII. Patent application scope: 1. A system for increasing the frame rate, comprising: an action estimating unit, which inputs according to a continuous frame to generate at least one motion vector; and a motion compensation unit based on a three-wire buffer, according to The motion vector, a reference frame and a current frame are used to generate an inner frame, whereby a frame output is generated whose frame rate is 13⁄4 at the frame rate of the frame input. 2. The system for increasing the frame rate according to claim 1, wherein the action compensation unit comprises: a time interpolation unit, according to the motion vector, the reference frame and the current frame to generate a time An illustration box; and a spatial interpolation unit, performing spatial interpolation on the illustrated frame during the time to generate a space frame. 3. The system for increasing the frame rate as described in claim 2, further comprising a smoothing unit for performing smoothing processing on the spatial interpolation frame along a periphery of the adjacent block. 4. The system for increasing the frame rate as described in claim 2, wherein the spatial interpolation unit comprises: a memory for providing a plurality of lines of the pixel block; and a three-line buffer comprising three a line buffer for storing a current line, a last line of one of the previous blocks and a first line of the next block, and a spatial interpolation processor, according to the stored previous area The last line of the block and the first line of the next block are used to perform spatial interpolation on the current line. 099114570 Form No. A0101 Page 10 of 22 0992025738-0 201141237 The system for increasing the frame rate as described in item 4 of the patent application, wherein the line buffer storing the current line is subjected to the next current line. Covering a system for increasing the frame rate as described in claim 4, during the first period, the three line buffers include: a first buffer for storing the previous block Ν-1 a last line; a second buffer for storing the current line of the current block; and a third buffer for storing the first line of the next Ν + 1; Ο wherein the block Ν -1, the block Ν and the block Ν + 1 is a continuous block of images in the vertical direction. The system for increasing the frame rate according to claim 6 of the patent application, during the second period, the first buffer is used to store a first line of a block Ν + 2, and the second buffer is used to store the area The last line of the block, and the third buffer is used to store the current line of the block Ν + 1, wherein the block Ν, the block Ν+1 and the block Ν + 2 are in the vertical direction of the image a contiguous block 〇Ο a system for increasing the frame rate as described in claim 4, according to a pixel of the last line of the previous block and a pixel of the first line of the next block, Spatial interpolation is performed on one of the pixels of the current line. A method for increasing a frame rate, comprising: performing motion estimation (ME) to generate at least one motion vector (MV) according to a continuous frame input; and ίο. according to the motion vector, a reference frame, and a current frame The three-line buffer-based motion compensation is performed to generate an inner frame, whereby a frame output is generated whose frame rate is higher than the frame rate input by the frame. The method for increasing the frame rate as described in claim 9 wherein the 099114570 form number A0101 page 11/22 page 0992025738-0 201141237 the motion compensation step comprises: according to the motion vector, the reference frame and the current a frame, the execution time is inserted to generate a frame for a period of time; and spatial interpolation is performed on the frame of the time to generate a space in the frame 0 11 . The frame rate is increased as described in claim 10 The method further includes smoothing the space interpolating the circular frame along one of the adjacent blocks. 12. The method of increasing a frame rate as described in claim 1, wherein the spatial interpolation step comprises: providing a plurality of lines of pixel blocks; and placing a current line, a last line of a previous block And storing the first line of the next block into the three line buffer; and performing the current line according to the stored last line of the previous block and the first line of the next block Spatial interpolation. 13 • The method for increasing the frame rate as described in claim 12 of the patent application further includes a step in which the current line covers the contents of the line buffer storing the current line. 14. The method of increasing the frame rate according to claim 12, wherein during the first period, the line buffer storing step comprises: storing the last line of the previous block N-1 to a first a buffer; storing the current line of the current block N to a second buffer; and storing the first line of the next block to a third buffer; wherein the block N-1, the area Block N and the block Ν + ι are contiguous blocks of an image in the vertical direction. 15. The method for increasing the frame rate according to claim 14 of the patent application, during the second period, the line buffer storing step comprises: 099114570 Form No. A0101 Page 12 / Total 22 Page 0992025738-0 201141237 The first line of the block N + 2 is stored to the first buffer; the last line of the block N is stored to the second buffer; and the current line of the block N+1 is stored to the first a triple buffer; wherein the block N, the block N + 1 and the block N + 2 are consecutive blocks of the image in the vertical direction. The method for increasing the frame rate according to claim 12, wherein the spatial interpolation step comprises: according to one of the last line of the previous block and the first line of the next block One pixel, performing spatial interpolation on one pixel of the current line. ❹ 〇 099114570 Form No. A0101 Page 13 of 22 0992025738-0