JPH09214976A - Still image data generation device and still image data generation method - Google Patents

Abstract

(57) [Abstract] [PROBLEMS] To generate high-quality still image data from interlaced moving image data. SOLUTION: Bottom-bottom residuals (sum of absolute difference values) Eb, top-t defined by blocks constituting a target frame and corresponding predetermined blocks of frames before and after the block.
op residual Et, and bottom-top motion vector B are calculated,
If any one of the residuals exceeds the predetermined value, it is regarded as a moving image block. If the residuals of both are less than the predetermined value, the length of the motion vector exceeds the predetermined value. Further, when the residual Ebt after motion compensation is less than a predetermined ratio of the uncompensated residual E0, it is further set as a moving image block, and when neither case is applicable, it is set as a still image block.
The moving image block uses one field data to generate the still image data in the block, and the still image block uses both field data to generate the still image data in the block.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a still image data generating apparatus and a still image data generating method, and more particularly, to generating a still image data from interlaced moving image data, a block of a predetermined frame and a frame before and after the frame. And the sum of absolute differences defined by the corresponding block of
The present invention relates to a still image data generation device and a still image data generation method that accurately determine a moving image block or a still image block based on a motion vector and generate high quality still image data.

[0002]

2. Description of the Related Art Conventionally, still image data is generated from interlaced moving image data.

When still image data is generated from interlaced moving image data, if data of a predetermined frame of moving image data is used as it is to generate still image data, scanning of two fields constituting one frame is performed. Since there is a time lag (1/60 seconds in the case of the NTSC system) in the generated still image, the moving image portion moving between them has a blur.

Therefore, the pixels constituting one frame are divided into blocks each having a predetermined number of pixels, and a moving image block or a still image block is determined in units of the blocks. The determination of a moving image block and a still image block is performed by comparing corresponding blocks between two fields forming one frame, and if the difference is a predetermined value or more, it is determined as a moving image block and a predetermined value is determined. If it is less than this, it is determined to be a still image block. Then, in the moving image block, still image data in the block is generated by using only one field data of the two field data constituting one frame, and one frame is constituted in the still image block. Both field data are used to generate still image data in the block.

By doing so, it is possible to generate still image data in which the blurring caused by the motion of the moving image is suppressed from the moving image data of the interlace.

[0006]

However, in the conventional method for determining a moving image block and a still image block,
Especially when the spatial variation of the still image block is large,
For example, in the case of a still image block with many high frequency components in which the pixel values of adjacent pixels in the block change abruptly, a difference of a predetermined value or more occurs between the two field data corresponding to the block, and thus the still image block There was a problem that image blocks were easily mistaken for moving image blocks.

As described above, the still image of the still image block is generated by using the two field data, and the still image of the moving image block is generated by using the one field data. There is a problem that the resolution becomes discontinuous at the boundary with the block, and as a result, the boundary between the still image block and the moving image block becomes an unnatural image.

Further, in the case of realizing an image processing apparatus for performing the above-mentioned processing, a circuit for determining a still image block and a moving image block is required, which complicates the configuration, enlarges the apparatus, and reduces the cost. There was a problem that was difficult to realize in.

The present invention has been made in view of such circumstances, and more accurately determines a moving image block and a still image block by using interlaced moving image data of a predetermined frame and frames before and after the predetermined frame. The present invention realizes a still image data generation device that generates high quality still image data.

[0010]

According to another aspect of the present invention, there is provided a still image data generating apparatus for accumulating data of a predetermined frame and data of at least one field of frames before and after the moving image data. A storage unit, a calculation unit that calculates, for each block of the stored predetermined frame, a residual and a motion vector defined by the block and a corresponding block of a predetermined field of a frame before and after the storage unit, and a calculated residual. A determination unit that determines whether a block forming a predetermined frame is a moving image block or a still image block using at least one of a difference and a motion vector, and a generation unit that generates still image data based on the determination. It is characterized by including.

According to a still image data generation method of a fourth aspect of the present invention, data of a predetermined frame in the moving image data is stored.
Data of at least one field of the preceding and succeeding frames is accumulated, and for each accumulated block of the predetermined frame, the residual and motion specified by the block and the corresponding block of the predetermined field of the preceding and following frames A vector is calculated, and at least one of the calculated residual and motion vector is used to determine whether the block forming a predetermined frame is a moving image block or a still image block, and still image data based on the determination. Is generated.

A still image data generation device according to a fifth aspect of the present invention includes data of a predetermined frame in the moving image data,
A storage unit that stores data of at least one field of a frame before and after the frame, and a residual defined by a block and a corresponding block of a predetermined field of a frame before and after the frame for each block of the stored predetermined frame. A calculating means for calculating a difference and a motion vector, and a judging means for judging a block forming a predetermined frame as either a moving image block or a still image block by using at least one of the calculated residual and motion vector. And, corresponding to the determination, a generation unit that generates interpolation image data for gradually shifting from a still image to a moving image or from a moving image to a still image near the boundary between the moving image block and the still image block. Characterize.

A still image data generating method according to a sixth aspect of the present invention is: a still image data generating method;
Data of at least one field of the preceding and succeeding frames is accumulated, and for each accumulated block of the predetermined frame, the residual and motion specified by the block and the corresponding block of the predetermined field of the preceding and following frames A vector is calculated, and using at least one of the calculated residual and motion vector, the block that makes up the predetermined frame is determined to be either a moving image block or a still image block. It is characterized in that, in the vicinity of the boundary between the block and the still image block, interpolated image data for gradually shifting from the still image to the moving image or from the moving image to the still image is generated.

A still image data generating device according to a seventh aspect of the present invention includes data of a predetermined frame in the moving image data,
A storage unit that stores data of at least one field of a frame before and after the frame, and a residual defined by a block and a corresponding block of a predetermined field of a frame before and after the frame for each block of the stored predetermined frame. A calculating means for calculating a difference and a motion vector, and a judging means for judging a block forming a predetermined frame as either a moving image block or a still image block by using at least one of the calculated residual and motion vector. In response to the determination, when the block is the block at the bottom of the frame, the pixel on the line immediately above the bottom line in the block becomes the line further below that at the bottom line in the block. It is characterized in that it is provided with a generation means for generating interpolated image data corresponding to the pixel of the bottom line in the block. .

According to a still image data generating method of the present invention, in a moving image data, data of a predetermined frame,
Data of at least one field of the preceding and succeeding frames is accumulated, and for each accumulated block of the predetermined frame, the residual and motion specified by the block and the corresponding block of the predetermined field of the preceding and following frames A vector is calculated, and at least one of the calculated residual and motion vector is used to determine the block that constitutes a predetermined frame as either a moving image block or a still image block. Is the bottom block of the frame, it is assumed that the pixels in the line immediately above the bottom line in the block are located one line below the bottom line in the block. It is characterized in that interpolation image data corresponding to the pixels on the lower line is generated.

In the still image data generating device according to the first aspect, the accumulating means accumulates data of a predetermined frame and data of at least one field of frames before and after the moving image data, The calculating means calculates, for each block of the accumulated predetermined frame, the residual and the motion vector defined by the block and the corresponding block of the predetermined field of the frame before and after the accumulated frame, and the determining means calculates. Using at least one of the residual and the motion vector, the block forming the predetermined frame is determined to be either a moving image block or a still image block,
The generation means generates still image data based on the determination. For example, when the determination unit determines that the block is a moving image block, the generation unit causes the generation unit to generate the corresponding interpolated image data, and when the determination unit determines that the block is a still image block, it outputs both field data. It is possible to generate the interpolated image data corresponding to the base.

In the still image data generating method according to the fourth aspect, of the moving image data, the data of a predetermined frame and the data of at least one field of the frames before and after it are accumulated, and the accumulated predetermined data is accumulated. For each block of the frame of, the residual and motion vector specified by the block and the corresponding block of the predetermined field of the preceding and following frames are calculated, and at least one of the calculated residual and motion vector is used. , A block forming a predetermined frame is determined to be either a moving image block or a still image block, and still image data is generated based on the determination. For example, when a block is determined to be a moving image block, corresponding interpolation image data is generated based on one field data, and when it is determined to be a still image block, corresponding interpolation image data is determined based on both field data. Is generated.

In the still image data generating device according to the fifth aspect, the accumulating means accumulates data of a predetermined frame and data of at least one field of frames before and after the moving image data, The calculating means calculates, for each block of the accumulated predetermined frame, the residual and the motion vector defined by the block and the corresponding block of the predetermined field of the frame before and after the accumulated frame, and the determining means calculates. Using at least one of the residual and the motion vector, the block forming the predetermined frame is determined to be either a moving image block or a still image block,
Corresponding to the determination, the generation means generates the interpolated image data for transitioning from the still image to the moving image or from the moving image to the still image stepwise in the vicinity of the boundary between the moving image block and the still image block. For example, as the moving image block approaches the still image block, the interpolated image data generated stepwise from the interpolated image data generated using one field data to the interpolated image data generated using both field data is changed. It can be generated by the generation means.

In the still image data generating method according to the sixth aspect, of the moving image data, the data of a predetermined frame and the data of at least one field of the frames before and after the data are accumulated and accumulated. For each block of the frame of, the residual and motion vector specified by the block and the corresponding block of the predetermined field of the preceding and following frames are calculated, and at least one of the calculated residual and motion vector is used. , Blocks that make up a predetermined frame are judged to be either moving picture blocks or still picture blocks, and in response to the judgment, in the vicinity of the boundary between the moving picture blocks and the still picture blocks, from still pictures to moving pictures Alternatively, interpolated image data for moving from a moving image to a still image is generated. For example, as the moving image block approaches the still image block, the interpolated image data generated stepwise from the interpolated image data generated using one field data to the interpolated image data generated using both field data is changed. Is generated.

In the still image data generating device according to the seventh aspect, the accumulating means accumulates data of a predetermined frame and data of at least one field of frames before and after the moving image data, The calculating means calculates, for each block of the accumulated predetermined frame, the residual and the motion vector defined by the block and the corresponding block of the predetermined field of the frame before and after the accumulated frame, and the determining means calculates. Using at least one of the residual and the motion vector, the block forming the predetermined frame is determined to be either a moving image block or a still image block,
In response to the determination, the generation means, when the block is the block at the bottom of the frame, the pixel in the line one above the bottom line in the block is further one below the bottom line in the block. The interpolated image data corresponding to the pixel on the bottom line in the block is generated. For example, when generating the interpolated image data of the bottom line of the bottom block of the frame, the generation means,
The corresponding interpolation image data can be generated by the stylized generation process similar to the generation process of the interpolation image data corresponding to another line.

In the still image data generating method according to the eighth aspect, of the moving image data, data of a predetermined frame and data of at least one field of frames before and after the same are accumulated, and the accumulated predetermined data is accumulated. For each block of the frame of, the residual and motion vector specified by the block and the corresponding block of the predetermined field of the preceding and following frames are calculated, and at least one of the calculated residual and motion vector is used. , The block that constitutes a predetermined frame is determined to be either a moving image block or a still image block, and if the block is the block at the bottom of the frame according to the determination, 1 of the bottom line in the block It is assumed that the pixels on the uppermost line are located one line below the lowermost line in the block. Interpolated image data is generated corresponding to the pixels of the lower line. For example, when the interpolated image data of the bottom line of the bottom block of the frame is generated,
The corresponding interpolation image data is generated by the stylized generation process similar to the generation process of the interpolation image data corresponding to another line.

[0022]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention will be described below. In order to clarify the correspondence between each means described in the claims and the following embodiments, parentheses after each means are described. Within
The characteristics of the present invention will be described as follows by adding a corresponding embodiment (but one example). However, of course,
It does not mean that each means is limited to those described.

A still image data generating apparatus according to claim 1 is a still image data generating apparatus for generating predetermined still image data from interlaced moving image data, wherein the moving image data includes a predetermined frame of data. , Storage means for storing data of at least one field of frames before and after that (for example, the frame memory 1 of FIG. 1).
1) and a calculation means for calculating a residual and a motion vector defined for each block of a predetermined frame stored in the storage means and a block and a corresponding block of a predetermined field of a frame before and after the same (for example, Using the motion detection circuit 17) in FIG. 1 and at least one of the residual and the motion vector calculated by the calculation means, the block forming a predetermined frame is determined to be either a moving image block or a still image block. A determination unit (for example, the moving image / still image determination device 20 in FIG. 1) and a generation unit (for example, the interpolated image generation device 1 in FIG. 1) that generates still image data based on the determination.
3) and are provided.

A still image data generating device according to a fifth aspect is a still image data generating device for generating predetermined still image data from interlaced moving image data, wherein the moving image data includes a predetermined frame of data. , Storage means for storing data of at least one field of frames before and after that (for example, the frame memory 1 of FIG.
1) and a calculation means for calculating a residual and a motion vector defined for each block of a predetermined frame stored in the storage means and a block and a corresponding block of a predetermined field of a frame before and after the same (for example, Using the motion detection circuit 17) in FIG. 1 and at least one of the residual and the motion vector calculated by the calculation means, the block forming a predetermined frame is determined to be either a moving image block or a still image block. Corresponding to the determination means (for example, the moving image / still image determination device 20 in FIG. 1) and the determination, in the vicinity of the boundary between the moving image block and the still image block, a stepwise transition from still image to moving image or moving image to still image is performed. It is characterized by including a generation unit (for example, the boundary determination / coefficient switching circuit 32 in FIG. 2) that generates the interpolated image data.

A still image data generating device according to a seventh aspect is a still image data generating device for generating predetermined still image data from interlaced moving image data, wherein data of a predetermined frame among the moving image data is stored. , Storage means for storing data of at least one field of frames before and after that (for example, the frame memory 1 of FIG. 1).
1) and a calculation means for calculating a residual and a motion vector defined for each block of a predetermined frame stored in the storage means and a block and a corresponding block of a predetermined field of a frame before and after the same (for example, Using the motion detection circuit 17) in FIG. 1 and at least one of the residual and the motion vector calculated by the calculation means, the block forming a predetermined frame is determined to be either a moving image block or a still image block. Corresponding to the determination means (for example, the moving image / still image determination device 20 in FIG. 1), when the block is the bottom block of the frame, the pixel on the line immediately above the bottom line in the block is As it is located one line below the bottom line in the block,
It is characterized by comprising a generating means (for example, the filter arithmetic circuit 31 of FIG. 2) for generating the interpolated image data corresponding to the pixel on the bottom line in the block.

FIG. 1 is a block diagram showing the configuration of an embodiment of an image compression recording system to which the still image data generating apparatus of the present invention is applied.

The image input device 1 is composed of, for example, a broadcast video tape recorder and a device for converting its output signal into an analog-digital (A-D) signal, and outputs a digitized image signal to the pre-processing device 2. It is designed to do.

The pre-processing device 2 performs resolution conversion for the image signal input from the image input device 1 so as to set the sampling number and the number of frames per second to a predetermined value, and pixel number conversion for limiting the pixels to be encoded. The image is compressed and output to the image compression device 3.

The image compression device 3 compresses and encodes the interlaced moving image data input from the pre-processing device 2 into predetermined compressed moving image data and compressed still image data, and then a video CD (Compact Disc), DVD (Digital Video Disc)
Is output to a recording device 4 which records a digital video signal on a recording medium such as.

The control circuit 23 of the image compression device 3 generates compressed still image data from the moving image data input from the preprocessing device 2 (when a still image generation command is input from a device not shown). , The switch 24 is switched to the terminal b side. Further, when generating compressed moving image data, the control circuit 23 switches the switch 24 to the terminal b side when detecting compressed moving image data (I picture data) for which motion compensation is not performed, and performs compression for motion compensation. When moving image data (P picture or B picture data) is detected, it is switched to the terminal a side.

The frame memory 11 accumulates the moving image data input from the pre-processing device 2 to the image compression device 3 in units of frames, in the case of this embodiment, three frames, and appropriately stores them.
The signal is output to the motion detection circuit 17 and the residual detection circuit 18.

The subtraction circuit 12 receives data (predicted image data (data supplied from the terminal a of the switch 24)) or zero data (from the moving image data input from the preprocessor 2 through the switch 24). The data supplied from the terminal b of the switch 24)) is subtracted and output to the interpolation image generation device 13 as prediction error data.

When generating the compressed still image data, the interpolation image generation device 13 interpolates image data corresponding to the determination result of the moving image block or still image block determined for each block supplied from the moving image still image determination device 20. Is generated and output to a DCT (Discrete Cosine Transform) circuit 14. In addition, the interpolation image generation device 13
When the compressed moving image data is generated (when the compressed still image data is not generated), the prediction error data input from the subtraction circuit 12 is directly (without being processed) DC
It is adapted to output to the T circuit 14.

The DCT circuit 14 is used by the interpolation image generation device 13
Two-dimensional DCT (discrete cosine transform) is applied to the input image data to remove redundancy, concentrate energy in a part of frequency components, and then output the quantization circuit 15. .

The quantizing circuit 15 quantizes the DCT coefficient output from the DCT circuit 14 and outputs the quantized DCT coefficient to the variable length coding circuit 16 and the inverse quantization circuit 21.

The variable length coding circuit 16 is a quantization circuit 15.
On the basis of the appearance probability of the supplied quantized representative value or quantized representative vector, code allocation is performed so that the average code length becomes shorter, and the code is output to the recording device 4.

Inverse quantization circuit 21 and inverse DCT circuit 22
Respectively perform the inverse transform processing of the quantization circuit 15 and the DCT circuit 14, and the quantized image data input from the quantization circuit 15 is inversely quantized by the inverse quantization circuit 21, Further, after being subjected to inverse discrete cosine transform in the inverse DCT circuit 22, it is inputted to the adding circuit 25.

The motion detection circuit 17 includes a frame memory 11
A motion vector and a residual after motion compensation (a sum of absolute differences) are calculated based on predetermined frame data input from, and output to the motion compensation circuit 19 and the moving image / still image determination device 20. There is.

The motion compensation circuit 19 includes a frame memory 26.
The image data of the frame read immediately before is subjected to motion compensation based on the motion vector input from the motion detection circuit 17, predictive image data is generated, and the subtraction circuit is provided via the terminal a of the switch 24. It outputs to 12 and outputs to the addition circuit 25.

The addition circuit 25 adds the prediction error data input from the inverse DCT circuit 22 to the prediction image data input from the motion compensation circuit 19 and stores it in the frame memory 26.

The residual detection circuit 18 is provided in the frame memory 11
Based on the frame data input further, a residual is obtained for each block constituting a predetermined frame and output to the moving image / still image determination device 20.

The moving picture / still picture judging device 20 is based on the motion vector and the residual information after motion compensation supplied from the motion detecting circuit 17 and the residual information supplied from the residual detecting circuit 18 to be processed. The moving image or the still image is determined for each block constituting the frame, and the determination result is output to the interpolation image generation device 13.

FIG. 2 is a block diagram showing the configuration of an embodiment of the interpolation image generating apparatus 13.

The boundary determination / coefficient switching circuit 32 determines the interpolation coefficient used when generating the interpolated image data based on the determination information supplied from the moving image / still image determination device 20,
The data is output to the filter calculation circuit 31.

The filter calculation circuit 31 generates the corresponding still image data by generating the interpolation image data from the image data input from the subtraction circuit 12 based on this interpolation coefficient, and outputs it to the DCT circuit 14. Has been done.

Next, the processing operation of the image compression recording system of FIG. 1 will be described with reference to the flowchart of FIG.

In step S1 of FIG. 3, the image input device 1
Outputs the image signal reproduced from a video tape or the like to the pre-processing device 2 after digitizing it. The pre-processing device 2 converts this image signal into moving image data having a predetermined number of samplings, frames and pixels, and outputs the moving image data to the image compression device 3. Thus, for example,
Luminance data having 704 pixels and 480 pixels in the vertical direction and color difference data having 352 pixels in the horizontal direction and 240 pixels in the vertical direction are input to the image compression device 3.

Now, it is assumed that the I picture data is input to the image compression device 3.

In the subsequent step S2, the control circuit 23 determines whether or not compressed still image data is generated from this moving image data. When a still image generation command is not input from the device (not shown) (when it is instructed to generate compressed moving image data), a NO determination is made in this step, and the process branches to step S3. In step S3, the control circuit 23 determines whether or not the image data currently input from the preprocessing device 2 is image data for which motion compensation is performed (P picture or B picture data). In this case, since the I picture data has been input, a NO determination is made, and in step S4, the control circuit 23 connects the switch 24 to the terminal b.

In the subsequent step S5, the subtraction circuit 12 subtracts the signal input from the switch 24 from the moving image data input from the preprocessing device 2 and outputs the subtracted signal to the interpolation image generating device 13. In this case, since the switch 24 is connected to the terminal b whose other terminal is grounded, the signal input from the switch 24 is zero, and the moving image data input from the preprocessing device 2 is substantially the same. Output to the interpolation image generation device 13 as it is. At this time, the moving image data input from the pre-processing device 2 is stored in the frame memory 11 on a frame-by-frame basis.

In the case of generating predetermined compressed still image data from moving image data, the interpolation image generating device 13 is based on the determination result supplied from the moving image / still image determining device 20.
Interpolation image data for generating predetermined still image data is generated from the moving image signal input from the subtraction circuit 12, and is output to the DCT circuit 14. In this case, since it is not a process of generating compressed still image data from moving image data, the moving image data input from the subtraction circuit 12 is directly output to the DCT circuit 14.

The moving image signal is subjected to discrete cosine transform in the DCT circuit 14 to concentrate energy in a part of frequency components, and then in the quantizing circuit 15 to a value corresponding to a predetermined quantizing level. Each pixel value or a value resulting from the combination thereof is rounded and output to the variable length coding circuit 16.

The variable length encoding circuit 16 encodes the quantized image data so that the average code length becomes shorter, and outputs it to the recording device 4.

In the following step S6, the recording device 4 records the input compressed image data on the recording medium.

In this way, the moving image data generated by the image input apparatus 1 is compression-encoded and recorded on the recording medium.

The moving image data input from the preprocessor 2 is image data (P picture or B
Picture data), YE is determined in step S3.
S is determined, and in the subsequent step S7, the control circuit 2
3 connects the switch 24 to the terminal a.

In step S8, the frame memory 11
The predetermined frame data and the frame data before and after it are output to the motion detection circuit 17 and the residual detection circuit 18. The motion detection circuit 17 uses a predetermined frame and the frame data before it for a block (16 * 16 pixels for luminance data and 8 * 8 pixels for color difference data) forming a predetermined frame.
The motion vector is calculated and output to the motion compensation circuit 19 and the moving image / still image determination device 20.

Then, in step S9, the motion compensation circuit 19
On the basis of this motion vector, the motion compensation is performed on the data of the previous frame supplied from the frame memory 26 to generate predicted image data, which is output to the subtraction circuit 12 via the switch 24. .

In the subsequent step S10, the subtraction circuit 12
Subtracts the predicted image data input from the motion compensation circuit 19 from the moving image data input from the preprocessing device 2 and outputs the subtracted image data to the interpolation image generation device 13. Then, in this case, since it is not the process of generating still image data, the interpolation image generation device 13 outputs the image data input from the subtraction circuit 12 to the DCT circuit 14 as it is.

Thereafter, the moving image data is discrete cosine transformed by the DCT circuit 14, quantized by the quantizing circuit 15, and then code-assigned by the variable-length coding circuit 16 for recording. It is output to the device 4. At this time, the image data quantized by the quantization circuit 15 is also input to the dequantization circuit 21, dequantized, and the inverse DCT circuit 22.
Then, after being subjected to inverse discrete cosine transformation (after being transformed into prediction error data), it is outputted to the addition circuit 25. The addition circuit 25 adds the prediction image data input from the motion compensation circuit 19 to this prediction error data, and adds the prediction image data to the frame memory 26.
To memorize.

In step S6, a recording process is executed to record the image data on the recording medium. In this way, the compressed moving image data is recorded on the recording medium.

The above processing is the same as the conventional processing.

Now, it is assumed that an instruction is issued to generate compressed still image data from the moving image data input from the preprocessing device 2. In this case, YES is determined in step S2, and the control circuit 23 connects the switch 24 to the terminal b in step S11.

Then, in step S12, the frame memory 11 outputs the target frame data and the frame data before and after it to the motion detection circuit 17 and the residual detection circuit 18. The motion detection circuit 17 calculates a motion vector defined by the block of the target frame and the corresponding block of the subsequent frame (or the previous frame), and the motion compensation circuit 19 and the moving image / still image determination device 20. Output to. Further, the motion detection circuit 17 calculates the residual after motion compensation and outputs it to the moving image / still image determination device 20. For each block of the field of the target frame, the residual detection circuit 18 calculates the residual between the block and the block of the corresponding field of the preceding and following frames, and outputs it to the moving image / still image determination device 20. At this time, the equation of the residual E calculated is shown in Equation 1.

[0065]

[Equation 1]

In Expression 1, n is the number of pixels in the block, d_o (i) is the pixel value of the pixel of a predetermined block i, and d_r (i) is the pixel value of the corresponding block i. It is a pixel value. That is, in the case of this embodiment, the residual is the sum of absolute values of the differences of the pixels.

Here, the motion vector calculated in step S12 and the residual will be described with reference to the diagram shown in FIG.

FIG. 4 shows three frames necessary for calculating the motion vector and residual used when generating the compressed still image data corresponding to the target frame.

The target frame and the frames before and after the target frame are each composed of two fields, and are hereinafter referred to as a top field and a bottom field, respectively.

Bottom-bottom residual Eb is the target frame
It is the residual calculated between the block in the bottom field and the corresponding block in the bottom field (corresponding field) of the previous frame.

The top-top residual Et is a residual calculated between the block of the top field of the target frame and the corresponding block of the top field (corresponding field) of the subsequent frame.

Bottom-top The motion vector B is the block of the bottom field of the target frame and the top of the subsequent frame.
It is a motion vector calculated with the corresponding block after the movement of the field (field not corresponding).

The uncompensated residual E0 is the bottom of the target frame.
It is the residual calculated in the block of the m field and the corresponding block of the top field of the subsequent frame.

The residual Ebt after motion compensation is
It is the residual calculated between the block in which the block in the bottom field is motion-compensated and the corresponding block in the top field in the subsequent frame.

As the bottom-top motion vector B, the uncompensated residual E0, and the motion-compensated residual Ebt, those defined by the target frame and the previous frame may be used.

At the subsequent step S13, the calculation result of these blocks is used to determine the moving image or still image of the block.

Details of the subroutine processing of step S13 of FIG. 3 will be described with reference to the flowchart of FIG.

In step S21 of FIG. 5, the moving image / still image determination device 20 determines that the top-top residual Et is the reference value Th for each block.
Determine if it exceeds 1. Top-top of that block
When it is determined that the residual Et exceeds the reference value Th1, the process branches to step S22, and the block is determined to be a moving image block.

In step S21, the moving image / still image determination device 20 determines that the top-top residual Et of the block is the reference value Th.
When it is determined that it is 1 or less, further step S23.
Then, the bottom-bottom residual Eb of the block is the reference value Th1.
Judgment is made. Video still image determination device 2
0 is the block bottom-bottom residual Eb is the reference value Th
If it is determined that the number exceeds 1, the process branches to step S22, and the block is determined to be a moving image block.

In step S23, the bottom of the block
-If it is determined that the bottom residual Eb is less than or equal to the reference value Th1, the moving image / still image determination device 20 determines in step S24 whether the length (size) L of the motion vector B exceeds the reference value Th2. To judge. When it is determined that the length L of the motion vector B is equal to or less than the reference value Th2, the process branches to step S26, and the block is determined to be a still image.

When it is determined in step S24 that the length L of the motion vector B exceeds the reference value Th2, the residual Ebt after motion compensation is equal to the uncompensated residual E0 in step S25. If it is determined whether or not it exceeds a certain rate, and it is determined that the residual Ebt after motion compensation is equal to or more than a certain rate k of the uncompensated residual E0, step S26.
If it is determined that the block is a still image block, and the residual Ebt after motion compensation is smaller than the constant rate k of the uncompensated residual E0, the step S2 is performed.
It branches to 2, and the block is determined to be a moving image block.

Step S2 shown in the flowchart of FIG.
1 and step S23, the residual (top-top residual Et defined between the corresponding fields of the adjacent frames is defined).
And the bottom-bottom residual Eb) are used for the determination, so that an accurate determination result can be obtained as compared with the determination using the residual defined between two fields of the same frame.

Further, by using the determination based on the motion vector shown in step S24, an input image in which an object of a predetermined size moves quickly is erroneously recognized as a still image. Happens to change, but it happens that top-top residual Et and bottom-bottom residual
It is possible to prevent a moving image block whose Eb is a predetermined value or less from being mistaken as a still image block.

Further, by using the determination shown in step S25, when a block similar to the target block happens to exist in the subsequent top field, the still image block is erroneously recognized as the moving image block corresponding to the motion vector. Can be suppressed.

In this way, it is possible to accurately determine whether the target block is a moving image block or a still image block, using the residual and motion vector between predetermined blocks. Note that top-top residual Et and bottom-bo
If at least one of the ttom residuals Eb exceeds the reference value Th1, it may be determined that the target block is a moving image block, and otherwise, it may be determined that it is a still image block.

Subsequently, in step S14 of FIG. 3, the interpolation image generation device 13 determines from the moving image data input from the subtraction circuit 12 based on the determination result of the moving image still image supplied from the moving image still image determination device 20. Generate still image data.

Here, details of the processing in this step will be described with reference to FIG.
This will be described with reference to FIG.

FIG. 6 shows a state in which a moving image block and a still image block are adjacent to each other.

Data for two fields is shown in each of the moving image block and the still image block shown in FIG. Each field contains the pixels of the lines labeled top and bottom.

I line j of still image block and moving image block
When the pixel value of the top field of the column is t (i, j) and the pixel value of the bottom field of the i line and j column is b (i, j), the filter calculation circuit 31 leaves the pixel value of the top field as it is. However, the pixel value t (i,
j), b (i, j) and pixel value t (i + 1, j), i line j
Interpolated image data of the pixel value bn (i, j) of the bottom field of the column is generated according to the equation (1). bn (i, j) = k1 * t (i, j) + k2 * b (i, j) + k1 * t (i + 1, j) (1)

Here, the interpolation coefficient k1 and the interpolation coefficient k2 are set by the boundary judgment / coefficient switching circuit 32 based on the judgment result supplied from the moving picture / still picture judgment device 20, and the filter calculation circuit 3
It is an interpolation coefficient output to 1, and has different values for a still image block and a moving image block. Further, in the vicinity of the boundary between the still image block and the moving image block, the boundary determination / coefficient switching circuit 3 is arranged so that the pixel values do not become discontinuous values.
2 changes the interpolation coefficient k1 and the interpolation coefficient K2 stepwise and outputs them to the filter calculation circuit 31. FIG. 7 shows an example of specific values of the interpolation coefficient K1 and the interpolation coefficient K2 in the still image block.

FIG. 7 shows the interpolation coefficient K1 and the interpolation coefficient K2 used to obtain the pixel values of four pixels in the still image block near the boundary between the still image block and the moving image block.
Examples of values for are shown.

For example, in the pixel value of the luminance signal, the values of the corresponding interpolation coefficient K1 and interpolation coefficient K2 when calculating the pixel value of the fourth (or later) pixel from the boundary are:
The values are 0 and 1, respectively, and by substituting them into equation (1), bn (i, j) = b (i, j). This means that the pixel value of the bottom field of the still image block is used as it is for the generated interpolation image data (pixel value of the bottom field). Similarly, for the interpolated image data of the pixels of the still image block sufficiently (4 pixels or more) away from the boundary, the pixel values of the pixels of the still image block are used as they are. That is, as the still image data corresponding to the still image block, the two field data of the still image block are used as they are.

Also, the third, second, and 1 from the boundary
When the pixel value of the second pixel is calculated from the interpolation coefficient K1 and the interpolation coefficient K2 shown in FIG. 7, they are as follows. (1/8) * t (i, j) + (6/8) * b (i, j) + (1/8) * t (i + 1, j), (2/8) * t (i , j) + (4/8) * b (i, j) + (2/8) * t (i + 1, j), (3/8) * t (i, j) + (2/8) * b (i, j) + (3/8) * t (i + 1, j)

As the moving image block is approached, the weight of the pixel value of the top field (one field) increases, and the bottom value
It can be seen that the weight of the pixel value in the om field decreases. That is, in the vicinity of the boundary between the still image block and the moving image block in the still image block, the closer to the boundary,
Still image data in which one field (top field) data has a larger weight is generated.

On the other hand, in the moving image block, the boundary determination / coefficient switching circuit 32 sets the interpolation coefficient K1 to 1/2 and the interpolation coefficient K2.
Is set to 0 and supplied to the filter calculation circuit 31. Therefore, the filter calculation circuit 31 uses the pixel values of only the top field (one field) to generate the pixel value of the bottom field. In this case, bn (i, j) is (1/2) * t (i,
j) + (1/2) * t (i + 1, j), which is the average value of the pixel values in the top field on the i-th line and the top field on the i + 1-th line. There is. That is, the filter calculation circuit 31 generates more natural interpolated image data (bottom field data) using only the top field data.

In the moving image block, the boundary determination / coefficient switching circuit 32 does not change the values of the interpolation coefficients K1 and K2 in the vicinity of the boundary with the still image block and in the central portion. This prevents a moving image that is less likely to be clear than a still image from becoming more unclear.

As described above, in the moving image block, the interpolated image data is generated based on one of the field data and in the still image block, based on the two field data, the boundary between the still image block and the moving image block is generated. In the vicinity, the coefficient changes stepwise, so that more natural still image data is generated.

The same applies to the color difference signal. Since the number of pixels forming one frame in the color difference signal is half that of the luminance signal, the interpolation coefficient having the same value as the interpolation coefficient corresponding to the pixel of the luminance signal is used.

Further, in the equation (1), when bn (i, j) is the pixel value of the bottom line on the screen, the filter calculation circuit 31 virtually generates the pixel value of the line immediately below it. t (i + 1,
j), and its value is the pixel value of the top field of the i line
It is calculated as the same as t (i, j). That is, in this case, bn (i, j) is given by the equation (2). bn (i, j) = 2 * k1 * t (i, j) + k2 * b (i, j) (2)

In this way, the interpolation image generating device 13
Generates natural interpolation image data based on the determination result supplied from the moving image / still image determination device 20, so that higher quality still image data can be generated.

Returning to the explanation of FIG. 3, the following step S15
Then, the interpolation image generation device 13 uses the still image data D
Output to the CT circuit 14. The still image data is discrete cosine transformed by the DCT circuit 14, quantized by the quantization circuit 15, coded by the variable length coding circuit 16, and then output to the recording device 4. Then, in the subsequent step S6,
The recording device 4 records the compressed still image data on a recording medium.

As described above, by accurately determining the blocks of the interlaced moving image data as the moving image blocks and the still image blocks, the still image data with less blurring and the minimum reduction in resolution is generated. can do.

In addition, since the interpolation coefficient is changed stepwise in the vicinity of the boundary between the moving image block and the still image block in consideration of the discontinuity of resolution, more natural still image data can be generated. .

Furthermore, since the still image data generating apparatus of the present invention basically uses the circuit which constitutes the conventional image compressing apparatus as it is, the design is easy and the cost is low and the configuration is simple. Can be achieved with.

Although the sum of the absolute differences is used as the residual in the above embodiment, it may be the sum of squared differences. Further, although the data of the moving image block is generated from the image data of the top field, it may be generated from the image data of the bottom field. Furthermore, the data of the still image block is generated from the data of the top field as it is and the data of the bottom field calculated, but from the data of the bottom field as it is and the data calculated from the top field. It can also be generated. Further, the moving image block / still image block determination method and the corresponding interpolation method can flexibly support various pixel numbers and resolutions.

[0107]

As described above, according to the still image data generating device of the first aspect and the still image data generating method of the fourth aspect, the block of a predetermined frame and the predetermined frames of the frames before and after the block. Since the residual or motion vector specified by the corresponding block of the field is calculated and the calculated residual or motion vector is used to determine the moving image block or the still image block, the moving image block and the still image block are determined. The misjudgment determination of the image block can be suppressed.

According to the still image data generating apparatus and the still image data generating method of the sixth aspect, in accordance with the judgment, the stepwise process is performed near the boundary between the moving image block and the still image block. Since the interpolation image data for transitioning from the still image to the moving image or from the moving image to the still image is generated, more natural still image data can be generated at the boundary.

According to the still image data generating apparatus and the still image data generating method of the eighth aspect, when the block is the lowermost block of the frame, 1 of the lowermost line in the block is used. Since it is assumed that the pixel on the uppermost line is located on the line immediately below the lowermost line in the block, the interpolation image data corresponding to the pixel on the lowermost line in the block is generated.
High-quality still image data can be generated by a standard process.

[Brief description of drawings]

FIG. 1 is a block diagram showing the configuration of an embodiment of an image compression recording system using a still image data generation device of the present invention.

FIG. 2 is a block diagram showing a configuration of an example of an interpolation image generating device 13.

FIG. 3 is a flowchart illustrating a processing operation of the image compression recording system in FIG.

FIG. 4 is a diagram showing a relationship between blocks in a frame used when calculating a residual and a motion vector.

5 is a flowchart illustrating details of a subroutine process shown in step S13 of FIG.

FIG. 6 is a diagram showing pixels when a still image block and a moving image block are adjacent to each other.

FIG. 7 is a diagram showing a relationship between a distance from a boundary in a still image block and a value of an interpolation coefficient.

[Explanation of symbols]

1 image input device, 2 pre-processing device, 3 image compression device, 4 recording device, 11 frame memory, 1
2 subtraction circuit, 13 interpolation image generation device, 14 D
CT circuit, 15 quantization circuit, 16 variable length coding circuit, 17 motion detection circuit, 18 residual detection circuit,
19 motion compensation circuit, 20 video still image determination device,
21 inverse quantization circuit, 22 inverse DCT circuit, 23
Control circuit, 24 switches, 25 addition circuit,
26 frame memory, 31 filter arithmetic circuit,
32 Boundary judgment / coefficient switching circuit

Claims (8)

[Claims] 1. A still image data generation device for generating predetermined still image data from interlaced moving image data, comprising: a predetermined frame of data of the moving image data;
A storage unit that stores data of at least one field of a frame before and after the frame, and a block corresponding to the block and a predetermined field of a frame before and after the block for each block of the predetermined frame stored in the storage unit. Calculating means for calculating a residual and a motion vector defined by, and using at least one of the residual and the motion vector calculated by the calculating means, a block constituting the predetermined frame, a moving image block, Alternatively, a still image data generation device comprising: a determination unit that determines one of still image blocks, and a generation unit that generates the still image data based on the determination. 2. The remaining amount defined by a block in at least one of the two fields forming the predetermined frame and a block in a corresponding field of preceding and succeeding frames. A difference is calculated, and the determination unit determines that the corresponding block is a moving image block when at least one of the residuals is larger than a predetermined reference value, and determines a still image block when the difference is smaller than the reference value. The still image data generation device according to claim 1. 3. The calculation means includes two fields defined by a block in both fields of two fields constituting the predetermined frame and a block in a corresponding field of preceding and succeeding frames. Residuals,
Calculating a motion vector defined by a block in one field of the predetermined frame and a corresponding block of a non-corresponding field of the preceding or following frame, and the determining means determines the corresponding block as the motion If the length of the vector is equal to or greater than a predetermined value and the residual after motion compensation is smaller than a predetermined ratio of the residual that has not been motion compensated, it is determined to be a moving image block. The still image data generation device according to claim 1, wherein the still image data generation device determines that the block is a block. 4. A still image data generating method for generating predetermined still image data from interlaced moving image data, comprising: predetermined frame data of the moving image data;
Data of at least one field of the preceding and following frames is accumulated, and for each accumulated block of the predetermined frame, the residual defined by the block and the corresponding block of the predetermined field of the preceding and following frames And a motion vector are calculated, and by using at least one of the calculated residual and motion vector, the block forming the predetermined frame is determined to be either a moving image block or a still image block. A still image data generation method, wherein the still image data is generated based on the still image data. 5. A still image data generation device for generating predetermined still image data from interlaced moving image data, wherein a predetermined frame of data is included in the moving image data.
A storage unit that stores data of at least one field of a frame before and after the frame, and a block corresponding to the block and a predetermined field of a frame before and after the block for each block of the predetermined frame stored in the storage unit. Calculating means for calculating a residual and a motion vector defined by, and using at least one of the residual and the motion vector calculated by the calculating means, a block constituting the predetermined frame, a moving image block, Or a determination means for determining whether the image is a still image block, and in accordance with the determination, in the vicinity of the boundary between the moving image block and the still image block, a stepwise transition from a still image to a moving image or from a moving image to a still image A still image data generation device, comprising: 6. A still image data generation method for generating predetermined still image data from interlaced moving image data, comprising: predetermined frame data of the moving image data;
Data of at least one field of the preceding and following frames is accumulated, and for each accumulated block of the predetermined frame, the residual defined by the block and the corresponding block of the predetermined field of the preceding and following frames And a motion vector are calculated, and by using at least one of the calculated residual and motion vector, the block forming the predetermined frame is determined to be either a moving image block or a still image block. Correspondingly, in the vicinity of the boundary between the moving image block and the still image block, the still image data generation for generating the interpolated image data for gradually shifting from the still image to the moving image or from the moving image to the still image is generated. Method. 7. A still image data generation device for generating predetermined still image data from interlaced moving image data, wherein a predetermined frame data of the moving image data,
A storage unit that stores data of at least one field of a frame before and after the frame, and a block corresponding to the block and a predetermined field of a frame before and after the block for each block of the predetermined frame stored in the storage unit. Calculating means for calculating a residual and a motion vector defined by, and using at least one of the residual and the motion vector calculated by the calculating means, a block constituting the predetermined frame, a moving image block, Or a determination unit that determines one of the still image blocks, and, in response to the determination, when the block is the bottom block of the frame, the pixel on the line immediately above the bottom line in the block is It is assumed that the lowermost line in the block is positioned one line below the lowermost line. Still image data generating apparatus characterized by comprising generating means for generating an interpolated image data corresponding to the element. 8. A still image data generation method for generating predetermined still image data from interlaced moving image data, comprising: predetermined frame data of the moving image data;
Data of at least one field of the preceding and following frames is accumulated, and for each accumulated block of the predetermined frame, the residual defined by the block and the corresponding block of the predetermined field of the preceding and following frames And a motion vector are calculated, and by using at least one of the calculated residual and motion vector, the block forming the predetermined frame is determined to be either a moving image block or a still image block. Correspondingly, when the block is the lowermost block of the frame, the pixel on the line one above the bottom line in the block is located on the line one further below the bottom line in the block. Still image data characterized by generating interpolated image data corresponding to the pixels of the bottom line in the block Forming method.