JPH04286488A - Moving picture signal encoder - Google Patents

Abstract

PURPOSE:To reduce the redundancy in a moving picture coder at a low bit rate. CONSTITUTION:A valid block map is obtained by a frame difference signal and segmentation and elimination (6) of isolated invalid block are applied to the obtained valid block map. In the case of extracting a talker area, a threshold level for a segmentation circuit 5 is selected in response to the rate of a valid block number of a preceding pattern to a valid block number of a current pattern and only the extracted talker area is coded. Since a background is fixed and a talker mainly moves in the television telephone set or the like, useless coding information quantity caused due to noise from the background or the like is eliminated by segmenting the talker part and encoding the part, then the encoding efficiency is improved.

Description

[Detailed description of the invention]

0001

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture signal encoding apparatus using band compression technology.

0002

BACKGROUND OF THE INVENTION As a coding method for moving image signals using conventional band compression technology, for example, "I
"SDN-compatible color video video telephone device" is known. This encoding method extracts facial regions and creates a map. Then, the image encoding unit performs inter-frame intra-frame adaptive prediction, and at this time, if it is a face area, encoding is performed up to the final stage, and if it is other areas, encoding is stopped at the previous stage. This reduces the amount of code.

0003

However, in the above-described encoding method, the background portion other than the face is coarsely encoded, so that useless information is generated due to noise in the background portion. Furthermore, if there is a change from the background to the face between consecutive screens, the coding changes from coarse to fine, so a large amount of prediction error signal will occur here as well, resulting in unnecessary information being encoded. turn into. As a result, encoding efficiency decreases.

0004

[Means for Solving the Problems] The moving picture signal encoding device of the present invention divides a screen into blocks each consisting of a plurality of pixels in encoding a moving picture signal using correlation between screens.
Detecting the difference between screens for each block, when the difference value is greater than or equal to a predetermined first threshold, it is determined as a valid block, and when the difference value is less than a predetermined first threshold, it is determined as an invalid block, Validity determination means for creating a first effective block map for each frame; first weighting means for applying a first weight to the first effective block map; a second weighting unit that performs second weighting, a first effective block map that has been subjected to the first weighting, and a fourth effective block map that has performed the second weighting; an addition means for obtaining a weighted second effective block map; and an addition means for obtaining a weighted second effective block map; ratio determining means for determining whether the number of effective blocks of the effective block map is equal to or greater than a predetermined second threshold; If the output of the ratio determiner indicates that it is less than the second threshold, select a fourth threshold as the threshold in the segmentation. , said second
Refer to the blocks near each target block in the effective block map of When the value is less than the threshold, the block is regarded as an invalid block, and the segmentation means obtains a third valid block map.In the isolated invalid block in the third valid block map, neighboring blocks are referred to and the value of the neighboring block is determined. is greater than or equal to a predetermined fifth threshold, the invalid block is replaced with a valid block, and when the value of the neighboring block is less than the fifth threshold, the invalid block is left as an invalid block, and the fourth block is replaced with a valid block. an isolated invalid block removing means for obtaining an effective block map; a delay means for delaying an input signal to align the time with a fourth effective block map; and an area defined as a valid block in the fourth effective block map, and encoding means that performs encoding using either the inter-screen correlation or the intra-screen correlation, or both.

[0005]

[Operation] In videophones, etc., the background part is fixed and the speaker mainly moves, so if the speaker's part is cut out and encoded, the amount of coded information will be wasted due to noise from the background etc. can be removed and coding efficiency can be increased.

In the present invention, the coding efficiency is improved by cutting out the speaker's part and encoding only the speaker's part.

[0007] Next, a method for extracting speakers will be explained in detail with reference to the drawings. Time t0, t1, t2 in Figure 1
Assume that after the speaker moves as shown in FIG. 2, the speaker stands still at time t3 and only the mouth part moves. And time t
When the difference between the screens at time t2 and time t2 is calculated, the shaded area in FIG. 2 is obtained, and the difference between the screens at time t2 and time t3 is shown by the shaded area in FIG. 7A. It is assumed that the isolated hatched portion in the background portion is a differential signal generated by background noise. Next, the screen is divided into blocks each consisting of a plurality of pixels (n pixels in the horizontal direction x n pixels in the vertical direction), and when the sum of absolute values of the difference signals in each block is equal to or greater than a predetermined first threshold, the A block is defined as a valid block, and when the sum of absolute values of the difference signals is less than a first threshold value, the block is defined as an invalid block. The first effective block map between times t1 and t2 obtained by the above processing is shown in FIG. 3B, and the effective block map between times t2 and t3 is shown in FIG.
Shown in B. The black portions in FIGS. 3B and 7B are valid blocks. It is assumed that FIG. 3A is the fourth effective block map obtained between the screens at time t0 and time t1. Then, the first weighting is applied to the effective block map of the current screen, that is, the first effective block map, and the second weighting is applied to the fourth effective block map, which is the effective block map of the previous screen. . An example of weighting is shown below.

For example, the effective block of the current frame is “2”.
”, and the invalid blocks of the current frame are set to “0”. The valid blocks of the previous frame are set to “1”, and the invalid blocks are set to “0”. The valid block map of the previous frame weighted in this way and the effective block map of the current frame are added and combined to obtain a second block map.The second effective block map at time t2 obtained by weighting as described above is as shown in FIG. 4A.

Next, segmentation is performed on the additively combined effective block map shown in FIG. 4A. At this time, when the denominator is the value of the number of valid blocks in the previous frame, that is, the number of valid blocks in the fourth valid block map, and the number of valid blocks in the first valid block map, which is the valid blocks of the current frame, is the numerator. When the ratio is greater than or equal to a predetermined second threshold, for example, an example of segmentation in which the number of effective blocks in the current frame is the same as the number of effective blocks in the previous frame or is 1/2 or more is shown in FIG. This will be explained with reference to FIG. For example, if k in Fig. 5 is the target block for segmentation,
Blocks a, b, c, d, e, f near block k,
Refer to the values of g and h. That is, the value of the second effective block map in FIG. 4A is referred to. Neighboring blocks a, b
, c, d, e, f, g, h, and the values of block k are
When the value is equal to or greater than a predetermined third threshold, the target block k is regarded as a valid block, and neighboring blocks a, b, c,
When the values of d, e, f, g, h, and block k are less than a predetermined third threshold, target block k is determined to be an invalid block.

A third effective block map newly obtained by segmentation is shown in FIG. 4B. An isolated invalid block may occur in the third valid block map. Therefore, if only the valid block area in the third valid block map is encoded, the isolated invalid block part within the valid block area will not be encoded, and encoding distortion will occur in that part. This may result in a very unsightly encoded image. Therefore, isolated invalid blocks are removed. As a method for removing isolated invalid blocks, processing similar to segmentation is performed on invalid blocks. That is, the blocks near the invalid block are referred to, and when the neighboring block is equal to or greater than a predetermined fifth threshold, the target invalid block is replaced with a value indicating a valid block. Through the above processing, the area that was an isolated invalid block in FIG. 4B is removed,
Obtain a fourth valid block map. FIG. 6 shows a fourth valid block map from which isolated invalid blocks have been removed.

Next, as at time t3, the ratio of the number of valid blocks in the previous frame to the number of valid blocks in the current frame is less than the second threshold, and the number of valid blocks in the current frame is lower than the number of valid blocks in the previous frame. An example of segmentation when the number of blocks is considerably smaller than the number of blocks, for example, less than 1/2, will be described with reference to FIGS. 6, 7, 8, and 9. When the fourth effective block map obtained at time t2 in FIG. 6 and the first effective block map at time t3 in FIG. 7B are combined by applying the first and second weighting, the result is shown in FIG. 8A. A second effective block map shown in FIG. When segmentation is performed on this second effective block map based on the third threshold value, a third effective block map shown by diagonal lines in FIG. 8B is obtained. Then, isolated invalid blocks are removed from the third valid block map based on the fifth threshold value to obtain a fourth valid block map. At this time, since there is no isolated invalid block in the third valid block map obtained at time t3, the fourth valid block map becomes the same as the third valid block map. Then, encoding is performed only within the effective block area within this fourth effective block map. However, as shown in FIG. 8B, the effective block area of this fourth effective block map is missing the chest part of the speaker and the upper right part of the head. It is conceivable that an unencoded area may occur in the speaker's head or the speaker's head, and a discontinuous portion may occur in the speaker area of the encoded image, making the encoded image unsightly. Therefore, when the ratio of the number of effective blocks in the previous frame to the number of effective blocks in the current frame is less than the second threshold and the number of effective blocks in the current frame is small, as at time t3, the threshold in segmentation is By switching, loss of the speaker area is prevented. For example, in the weighted second effective block map of FIG. 8A, when performing segmentation, the threshold value for segmentation is
If it is not, by setting the threshold sufficiently low so that k, which is the target block for segmentation, is an effective block, a third effective block map as shown in FIG. 9 can be obtained, and the missing speaker region can be can be prevented. The threshold value at this time is defined as a fourth threshold value. As described above, when the ratio of the number of effective blocks in the previous frame to the number of effective blocks in the current frame is greater than or equal to the second threshold, segmentation is performed by selecting the third threshold, and the number of effective blocks in the previous frame is , if the ratio of the number of effective blocks in the current frame is less than the second threshold and the number of effective blocks in the current frame is much smaller than the number of effective blocks in the previous frame, select the fourth threshold and perform the segmentation. conduct. By encoding the effective block area in FIG. 6 or FIG. 9, that is, the speaker area, using either the inter-screen correlation or the intra-screen correlation, or both, it is possible to eliminate noise caused by background noise. Useless information can be easily deleted and encoding efficiency can be improved.

[0012] For each of the above threshold values and weighting values, optimum values statistically investigated in advance are used. Further, the arrangement of reference blocks in segmentation and isolated invalid block removal may be other than the above arrangement and number of blocks.

[0013]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described in detail with reference to the drawings. FIG. 10 shows an embodiment of the present invention. The input moving image signal is sent to the validity/invalidity determiner 1 via a line 10.
and delay 9. The validity/invalidity determiner 1 stores the signal of the previous screen, obtains a frame difference signal from the moving image signal newly inputted via the line 10, and converts this frame difference signal into a matrix of n pixels in the horizontal direction x vertical direction. It is divided into blocks each consisting of a plurality of n pixels, and for each block, the sum of absolute values of frame difference values within the block is determined. If the sum of absolute values of the calculated frame differences is greater than or equal to a predetermined first threshold, the block is considered a valid block, and if the sum of absolute values of frame differences is less than the first threshold, the block is invalidated. A first effective block map is obtained as a block. The first valid block map obtained by the validity determiner 1 is provided to a weighting circuit 2 and a ratio determiner 8. The weighting circuit 2 calculates, for the first valid block map given from the validity determiner 1,
A predetermined first weighting is performed. Weighting circuit 2
can be configured by ROM (read only memory). For example, if the output of valid/invalid judge 1 indicates that it is a valid block, set the output to “2”.
Logic that outputs "0" when the block is an invalid block may be written in the ROM in advance. As another method, the logic may be assembled using a logic gate circuit or the like. The first effective block map weighted by the weighting circuit 2 is provided to the adder 4. The adder 4 adds the first effective block map given from the weighting circuit 2 and the fourth effective block map given from the weighting circuit 3 to obtain a weighted second effective block map. get. The second effective block map obtained by the adder 4 is provided to the segmentation circuit 5. Next, an example of the ratio determiner 8 is shown in FIG. The ratio determiner 8 includes counters 81, 82 and RO
Consists of M83. Counter 81 of ratio determiner 8
counts the number of valid blocks in the first valid block map supplied from the validity determiner 1 via the line 120. The counter 82 of the ratio determiner 8 counts the number of valid blocks in the fourth valid block map supplied via the line 680 from the isolated invalid block removal circuit 6. The effective block numbers output from the counters 81 and 82 are supplied to the ROM 83. The ROM 83 determines whether the ratio of the number of valid blocks in the first valid block map to the number of valid blocks in the fourth valid block map calculated by the counters 81 and 82 is greater than or equal to a predetermined second threshold. Make a determination as to whether or not. In the ROM 83, a second threshold value is stored in an address area where the ratio of the number of valid blocks in the first valid block map to the number of valid blocks in the fourth valid block map is greater than or equal to a predetermined second threshold value. A signal indicating the above is written in advance, and the ratio of the number of valid blocks in the first valid block map to the number of valid blocks in the fourth valid block map is less than a predetermined second threshold. A signal indicating that the value is less than the second threshold is written in advance in the address area of . A signal indicating whether or not the output of the ROM 83 is equal to or higher than the second threshold is sent to the line 85 as the output of the ratio determiner 8.
The signal is supplied to the segmentation circuit 5 via the segmentation circuit 5. The segmentation circuit 5 receives the second
Segmentation processing is performed on all blocks in the effective block map. For example, if the judgment signal given from the ratio judger 8 indicates that it is greater than or equal to the second threshold, the third threshold is selected as the threshold for segmentation, and the judgment signal is equal to or higher than the second threshold. If it indicates that it is less than the threshold, the fourth threshold is selected as the threshold for segmentation, and segmentation is performed. Segmentation is performed by dividing the target block into k as shown in Figure 5.
Then, k and a, b, c, d, e, f near k
, g, h, and if these values are greater than or equal to the threshold selected by the judgment signal given from the comparison/judgment unit 8, that block k is regarded as a valid block, and the reference block and block k are If the value is less than the selected threshold, the block k is designated as an invalid block and a third valid block map is obtained. An example of the segmentation circuit 5 is shown in FIG. Segmentation circuit 5
are line memories 50, 51, delays 52, 53, 54
, 55, 56, 57, and ROM58. The second valid block map of the output of adder 4 is transferred via line 45 to line memory 50, delay 52.
and is supplied to the ROM 58. The line memory 50 is
The valid block map supplied via line 45 is delayed by one block line and supplied to line memory 51, delay 54 and ROM 58. Line memory 51 delays the signal supplied from line memory 50 by one block line and supplies it to delay 56 and ROM 58 . delay 5
2, 53, 54, 55, 56, and 57 delay the supplied signal by one clock and supply it to the ROM 58 and the next stage delay. The output signal of delay 54 is block k to be segmented in FIG. The ROM 58 is configured such that the signal supplied from the comparison/determination unit 8 via the line 85 is
, a judgment value based on the third threshold is written in the address area, and the signal supplied from the comparison/judgment unit 8 via the line 85 becomes the third threshold. If it indicates that it is less than the second threshold, a determination value based on the fourth threshold is written in that address area. For example, a value indicating a valid block is written in an address area where the signal supplied from the line memory and delay is greater than or equal to a predetermined third threshold, and a value indicating a valid block is written in an address area where the signal is less than the third threshold. Write a value indicating an invalid block. Furthermore, a value indicating a valid block is written in an address area that is equal to or greater than the fourth threshold, and a value indicating an invalid block is written in an address area that is less than the fourth threshold. As described above, by using the line memory, delay, and ROM, it is possible to refer to the values of the block k in which segmentation is to be performed and neighboring blocks.

Next, the isolated invalid block removal circuit 6 will be explained with reference to FIG. Isolated invalid block removal circuit 6
Similarly to the segmentation circuit 5, line memories 60, 61, delays 62, 63, 64, 65, 66, 67
and ROM 68, and removes isolated invalid blocks by referring to the values of blocks near the invalid block. The signal supplied from the delay 64 to the ROM 68 is the target block k for isolated invalid block removal. In other words, if the signal supplied from the delay 64 to the ROM 68 is an invalid block, and the neighboring block is greater than or equal to the predetermined fifth threshold, the ROM 68 outputs a value indicating a valid block, and the signal supplied from the delay 64 is If the detected signal is an invalid block and the neighboring blocks are less than the fifth threshold, the ROM 68 outputs a value indicating the invalid block. Further, when the signal supplied from the delay 64 is a valid block, the ROM 68 outputs a value indicating the valid block. By writing the above logic in the ROM 68 in advance, isolated invalid blocks are removed and valid blocks are connected. The fourth valid block map at the output of the isolated invalid block removal circuit 6 is supplied via line 680 to the weighting circuit 3 and the encoder 7. The weighting circuit 3 performs second weighting on the fourth valid block map supplied from the isolated invalid block removal circuit 6. The weighting circuit 3, like the weighting circuit 2, can be constructed from a ROM. At this time, values for performing the second weighting are written in the ROM in advance. The fourth effective block map obtained by weighting the output of the weighting circuit 3 is output to the adder 4.
supplied to Next, the delay 9 compensates for the delay time from when the input video signal is supplied to when the fourth effective block map is given to the encoder 7, and outputs the fourth effective block map. and time alignment of input signals. The time compensated input video signal at the output of delay 9 is supplied to encoder 7 via line 970. encoder 7
The configuration is shown in FIG. A moving picture signal supplied from delay 9 via line 970 is supplied to motion vector detector 71 and subtracter 72 inside encoder 7 . The fourth valid block map output from the isolated invalid block removal circuit 6 is:
It is supplied via a line 680 to a quantizer 74 inside the encoder 7 as a signal indicating the region of encoding execution. The motion vector detector 71 stores the signal of the previous screen and newly detects the signal on the line 9.
Inter-screen motion with the signal input via 70 is detected, and a motion vector indicating the amount and direction of motion is supplied to frame memory 77 and variable length encoder 78 . Subtractor 72 subtracts the input signal supplied via line 970 and the motion compensated prediction signal supplied from frame memory 77 to obtain a motion compensated prediction error. The motion compensated prediction error signal obtained by the subtracter 72 is supplied to an orthogonal transformer 73. The orthogonal transformer 73 is the subtracter 7
Orthogonal transformation is performed on the motion compensated prediction error signal supplied from 2 to convert the spatial domain prediction error signal into a frequency domain prediction error signal. The frequency domain prediction error signal output from the orthogonal transformer 73 is supplied to a quantizer 74 . The quantizer 74 quantizes the prediction error signal supplied from the orthogonal transformer 73 for blocks for which the fourth valid block map supplied via the line 680 indicates that the block is a valid block, A fourth signal supplied via line 680
Blocks whose valid block map indicates that they are invalid blocks are stopped from being encoded by setting the output of the quantizer to zero. The output signal of the quantizer 74 is supplied to an inverse orthogonal transformer 75 and a variable length encoder 78. The inverse orthogonal transformer 75 performs inverse orthogonal transform on the prediction error signal supplied from the quantizer 74 and returns it to a spatial domain prediction error signal. The output signal of the inverse orthogonal transformer 75 is supplied to an adder 76. The adder 76 receives the spatial domain prediction error signal supplied from the inverse orthogonal transformer 75 and the frame memory 7
7 and the motion compensated prediction signal supplied from 7 to obtain a locally decoded signal. The locally decoded signal output from adder 76 is supplied to frame memory 77 . frame memory 7
7 converts the locally decoded signal supplied from the adder 76 according to the motion vector supplied from the motion vector detector 71.
A motion compensated prediction signal is obtained by varying the amount of delay. The motion compensated prediction signal output from frame memory 77 is supplied to subtracter 72 and adder 76 . Next, the variable length encoder 78 converts the quantized frequency-domain prediction error signal supplied from the quantizer 74 and the motion vector supplied from the motion vector detector 71 into an efficient code such as a Huffman code. The code is used to perform variable length encoding to reduce redundancy. The variable length code with reduced redundancy matches the encoding speed and the transmission path speed, and becomes the output of the variable length encoder 78.
output to the transmission line. As explained in detail above, according to the fourth valid block map, encoding is performed only on the portion indicated as the valid block area, that is, the speaker area, and encoding is stopped on the background portion indicated as an invalid block. .

Regarding the method of stopping encoding, the subtracter 72
You can also replace the output of with zero. For each of the above threshold values and weighting values, optimal values statistically investigated in advance are used.

[0016]

Effects of the Invention As explained in detail above, the video signal encoding method of the present invention encodes only within the speaker region obtained by segmentation, thereby eliminating unnecessary noise caused by background noise. Information can be deleted and encoding efficiency can be increased.

[Brief explanation of the drawing]

[Figure 1] ~

FIG. 9 is a diagram illustrating a screen cropping method according to the present invention.

[
FIG. 10: Block diagram according to an embodiment of the present invention

[Figure 11]
~

FIG. 14 is a block diagram showing each part of the embodiment of the present invention in more detail.

[Description of symbols] 1 Validity/invalidity determination unit 2, 3 Weighting unit 4, 76 Adder 5 Segmentation unit 6 Isolated invalid block removal unit 7 Encoding unit 8 Ratio determination unit 9, 52, 53, 54, 55, 56, 57, 62, 63
, 64, 65, 66, 67 delay 50, 51, 6
0, 61 Line memory 58, 68, 83
ROM 71 Motion vector detector 72 Subtractor 73 Orthogonal transformer 74 Quantizer 75 Inverse orthogonal transformer 76 Adder 77 Frame memory 78 Variable length encoder 81, 82 Counter

Claims (1)

[Claims] Claim 1: In encoding a moving image signal using correlation between screens, a screen is divided into blocks each consisting of a plurality of pixels, a difference between screens is detected for each block, and the difference value is determined in advance. valid/invalid determining means for determining a valid block when the difference value is greater than or equal to a first threshold value, and determining the block as an invalid block when the difference value is less than a predetermined first threshold value, and creating a first valid block map for each frame; a first weighting unit that applies a first weight to one effective block map; a second weighting unit that applies a second weight to a fourth effective block map; an addition means for obtaining a weighted second effective block map by adding and combining the first effective block map subjected to the weighting and the fourth effective block map subjected to the second weighting; , when performing segmentation on the second effective block map, the number of effective blocks of the first effective block map relative to the number of effective blocks of the fourth effective block map is greater than or equal to a predetermined second threshold. If the output of the ratio determining means is equal to or higher than the second threshold, the third threshold is selected as the threshold for segmentation, If the output of the ratio determiner indicates that it is less than the second threshold, select a fourth threshold as the threshold in the segmentation;
Referring to blocks near each target block in the second effective block map, when the values of the neighboring blocks and the target block are equal to or greater than the selected threshold,
Segmentation means for determining the block as a valid block and determining the block as an invalid block when the value is less than the selected threshold value to obtain a third valid block map;
For an isolated invalid block in the third valid block map, neighboring blocks are referred to, and when the value of the neighboring block is equal to or greater than a predetermined fifth threshold, the invalid block is replaced with a valid block, and the neighboring block is replaced with a valid block. When the value of the block is less than the fifth threshold, isolated invalid block removing means leaves the invalid block as an invalid block and obtains a fourth invalid block map, and delays the input signal to obtain a fourth valid block map. and a delay unit that performs time alignment with the fourth effective block map, and encodes the area determined as a valid block in the fourth valid block map using either inter-screen correlation or intra-screen correlation, or both. A moving image signal encoding device comprising: encoding means.