Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T3/00—Geometric image transformations in the plane of the image
  • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
  • G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation

Definitions

• the present invention relates to tap filter processing operable to determine interpolated pixel values of pixels in a digital picture, especially a pixel interpolating method that is used in encoding/decoding processing for motion pictures by inter-picture prediction encoding.
• Inter-picture prediction encoding using correlation between pictures is employed in high-efficient, high-compressed encoding of a motion picture.
• the inter-picture prediction encoding is also employed in encoding standards such as MPEG-2, MPEG-4 (Part 2), and H.264
• encoding standards such as MPEG-2, MPEG-4 (Part 2), and H.264
• a present picture is divided into rectangular pixel blocks, each of which consists of a plurality of pixels, and the plurality of pixels are processed per block.
• the inter-picture prediction encoding generates a motion vector and a predicted block: the motion vector shows motion between a block on the present picture and a block on a reference picture that is a reference target; and the predicted block is a block that has undergone motion compensation based on the block on the reference picture.
• the inter-picture prediction encoding then performs encoding for the difference between the pixel values of pixels of the predicted block and those of the block on the present picture.
• each component of the motion vector is expressed in a subdivided unit that is smaller than a full-pel, such as a half-pel and a quarter-pel.
• generation of a predicted picture from a reference picture requires the interpolating processing of pixels, and the calculation method thereof is defined by each of the encoding standards.
• the 6-tap filtering includes tap coefficients ⁇ 1, -5, 20, 20, -5, 1 ⁇ .
• a pixel value for an interpolated pixel at (k+l/2)th location is calculated as follows. First, according to Equation 1, an intermediate pixel value "b" of the interpolated pixel is calculated. Then, according to Equation 2, normalization and saturation manipulation are performed for the intermediate pixel value "b", and a pixel value "c" is calculated as the final pixel value of the interpolated pixel.
• Fig. 8 shows location relationship of the pixels of the reference picture, that is, the pixels such as full-pels, interpolated half-pels, and interpolated quarter-pels (Document 1: Figure 8-4).
• pixels “A” to “U”, each surrounded with a circle are full-pels.
• Pixels “b' ⁇ “h”, “j”, “m”, “s”, and “aa” to "hh”, each surrounded with a square are half-pels.
• the remaining pixels “a”, “c”, “d”, “e”, “f ', “g”, “i”, "k”, “n”, “p”, “q”, and “r”, each surrounded with a square, are quarter-pels.
• the pixel values of the pixels "b” and “s” can be calculated by performing the above-mentioned 6-tap filtering to the pixel values of the pixels aligned along a horizontal line on which each of the pixels “b” and “s” is located.
• the pixel values of the pixels “h” and “m” can be calculated by performing the 6-tap filtering to the pixel values of the pixels aligned along a vertical line on which each of the pixels “h” and “m” is located.
• the pixel value of the pixel “j” can be calculated by performing 6-tap filtering in the vertical direction to the pixel values of six pixels "aa”, "bb", “b”, “s", “gg”, and "hh", which are calculated by the 6-tap filtering in the horizontal direction.
• the pixel value of the pixel "j" can be calculated by reversing the order of the filter direction and performing the 6-tap filter in the horizontal direction to the pixel values of the six pixels "cc", “dd”, “h”, “m”, “ee”, and “ff ', which are calculated by the 6-tap filtering in the vertical direction.
• the pixel value of a quarter-pel can be calculated by the average of the two pixel values of the adjoining half-pels or full-pels.
• the pixel value of a pixel "a” is calculated by the average of the pixel value of a full-pel "G” and the pixel value of a half-pel "b”
• the pixel value of a pixel "e” is calculated by the average of the pixel value of the half-pel "b” and the pixel value of a half-pel "h”
• the pixel value of a pixel "f ' is calculated by the average of the pixel value of the half-pel "b” and the pixel value of a half-pel "j".
• Fig. 9 is a flow chart showing an example of the conventional pixel interpolating method of calculating a pixel value of an interpolated pixel by the above-mentioned 6-tap filter. In the following, the conventional pixel interpolating method shown in Fig. 9 is explained.
• variable "k” is an index for reading the pixel value of a target reference pixel from an array "a” storing the pixel values of pixels which constitute a reference picture.
• the variable "k” indicates the full-pel "J”.
• the variable "m” is an index that indicates a location of the interpolated pixel in the predicted block.
• a variable "i” is an index indicating the reference point to be used in reading the pixel value of the reference pixel. More specifically, the variable "i” indicates a full-pel located on the upper left of an interpolated pixel to be calculated first. In the example shown in Fig. 8, the variable "i” indicates the pixel "G".
• Step S2 the pixel values of five reference pixels beginning from the edge of the reference area are read. This processing is a preparatory step for calculating the pixel value of the first interpolated pixel.
• Step S3 the pixel value of the reference pixel located at the variable "k" is read.
• an intermediate pixel value "b" of the interpolated pixel is calculated by Equation 1.
• Step S5 normalization and saturation manipulation of the intermediate pixel value "b" are performed to calculate the pixel value "c" of the interpolated pixel.
• the normalization means to divide the intermediate pixel value "b", which is calculated by Equation 1, by a value "32".
• the function Clip (x) is a function suppressing (clipping) a variable "x" within a range of 0 to 255, when the variable "x" exceeds the range of 0 to 255.
• Step S6 the pixel value "c" of the interpolated pixel calculated at Step S5 is outputted.
• Step S 7 for the next loop, the variable "m” and the variable “k” are incremented by the value "1", respectively.
• Step S8 whether or not the number of the generated interpolated pixels has reached the predetermined number "N" is judged.
• the judgment result is "No” (it has not reached the predetermined number "N")
• the control is returned to Step S3, and Step S3 to Step S8 are repeated.
• the judgment result is "Yes” (it has reached the predetermined number "N")
• the control is advanced to Step S9, and a series of the pixel interpolating processing ends.
• Fig. 10 shows a block diagram of a conventional pixel interpolating device using a 6-tap filter.
• Fig. 10 illustrates the 6-tap filter which is constructed so as to be used for the above-mentioned interpolating processing, based on the structure of a tap filter disclosed by Document 2 (Published Japanese patent application No. H07-15734).
• the conventional pixel interpolating device comprises registers (symbolized as "R” in the figure) 1 to 6, multiplying units (symbolized as “M”) 7 to 10, adding units (symbolized as "+”) 11 and 12, and a clipper 13.
• the registers 1 to 6 constitute a shift register with input of a pixel value "a [k]" of a reference pixel.
• the operation of the conventional pixel interpolating device shown in Fig. 10 is explained in the following.
• the shift register of the registers 1 to 6 by performing shift operation in synchronization with a shift control signal (not shown in the figure) and transferring, step by step, the inputted pixel value "a[k]" of the reference pixel, the pixel values of six consecutive reference pixels are stored one by one in the registers.
• the multiplying units 7 to 10 multiply the respective pixel values of the reference pixels stored in the registers 2 to 5 and the respectively predetermined coefficients (a coefficient "-5" for the multiplying units 7 and 10, and a coefficient "20" for the multiplying units 8 and 9) shown in the figure, thereby obtaining the respective products.
• the adding unit 11 adds the value stored in the register 1, the value stored in the register 6, and the product values stored in the multiplying units 7 to 10, thereby calculating the total sum.
• the adding unit 12 adds a constant "16" to the total sum calculated by the adding unit 11.
• the clipper 13 performs saturation manipulation, and outputs the pixel value "c" of the interpolated pixel.
• the multiplying units 7 to 10 can be constructed by units functioning bit-shift, addition, and sign change.
• the multiplying units 7 to 10 thus constructed are smaller in circuit scale than the regular multiplier.
• Document 3 discloses an art to use a look-up table which stores the multiplication result, thereby reducing the operation load.
• this method has a problem of the circuit scale enlarged as the result of installing the look-up table therein.
• the conventional pixel interpolating method and the conventional pixel interpolating device can not fully deal with the following problem.
• An object of the present invention is to provide a pixel interpolating method and a pixel interpolating device, operable to improve the speed of pixel interpolating processing and to decrease the power consumption therein.
• a first aspect of the present invention provides a pixel interpolating method using tap filtering, comprising: adding one of pixel values of an adjoining pixel pair composed of two adjoining reference pixels to another of the pixel values thereof, thereby obtaining a pixel value sum of the adjoining pixel pair; subtracting one of the pixel values of the adjoining pixel pair from another of the pixel values thereof, thereby obtaining a pixel value difference of the adjoining pixel pair; judging whether an absolute value of the pixel value difference is greater than a predetermined value, thereby generating a judgment result; and performing a first multiplication to multiply a first coefficient to the pixel value sum, thereby obtaining a first product.
• the pixel interpolating method further comprises: performing a second multiplication to multiply a plurality of second coefficients to the pixel value difference, thereby obtaining a plurality of second products; and performing a first addition/subtraction of the first product and the plurality of second products to each of a plurality of registers, thereby accumulating interpolated pixel values of a plurality of contiguous pixels to be interpolated, each of the plurality of registers being arranged in correspondence to each of the plurality of contiguous pixels to be interpolated.
• the pixel interpolating method further comprises: performing a second addition/subtraction of the first product to corresponding one of the plurality of registers, thereby accumulating interpolated pixel values of the plurality of contiguous pixels to be interpolated.
• the pixel interpolating method provided herein is operable to omit the processing of the second multiplication step pertaining to the pixel value difference of these adjoining pixels and the processing of addition or subtraction of the second product to the register. Therefore, the pixel interpolating processing can be performed at higher speed as much as a time otherwise needed for the omitted steps of processing.
• a second aspect of the present invention provides the pixel interpolating method as defined in the first aspect, wherein the adding, the subtracting, the judging, the performing the first multiplication, the performing the second multiplication, the performing the first addition/subtraction, and the performing the second addition/subtraction are practiced to a plurality of adjoining pixel pairs in addition to the adjoining pixel pair, thereby obtaining each of the interpolated pixel values of the plurality of contiguous pixels to be interpolated.
• the pixel values of the plurality of the interpolated pixels within a block can be obtained by a series of processing.
• a third aspect of the present invention provides the pixel interpolating method as defined in the first aspect, wherein the predetermined value is equal to or smaller than a value defined by a lowest bit in binary representation of each pixel value of the pixels composing the adjoining pixel pair.
• the pixel interpolating method provided herein is operable to omit the processing of the second multiplication step pertaining to the pixel value difference of these adjoining pixels and the processing of addition or subtraction of the second product to the register.
• a fourth aspect of the present invention provides the pixel interpolating method as defined in the first aspect, wherein the first coefficient and the plurality of second coefficients are coefficients composed of exponentiations of 2, wherein the performing the first multiplication bit-shifts the pixel value sum by a value of exponent of the first coefficient, and wherein the performing the second multiplication bit-shifts the pixel value difference by a value of exponent of a corresponding one of the plurality of second coefficients.
• the multiplying operation can be performed at higher speed than the regular multiplier.
• MPEG-4 AVC can be performed.
• a sixth aspect of the present invention provides the pixel interpolating method as defined in the fifth aspect, wherein the tap filtering comprises: a first step to input the pixel value a[k]; a second step to calculate a difference and sum of the pixel value a[k] and a pixel value of an adjacent pixel a[k-l] stored in a first register of the plurality of resisters, thereby storing the difference to a second register of the plurality of resisters, and storing the sum to a third register of the plurality of resisters after left-shifting by 4 bits; a third step to judge whether an absolute value of the difference is greater than a predetermined value, thereby generating a judgment result; a fourth step to subtract a value stored in the second register from a value stored in a fourth register of the plurality of resisters; a fifth step to left-shift a value stored in the second register by 2 bits, thereby adding the left-shifted value to a fifth register of the plurality of resisters, and subtracting the left-
• MPEG-4 AVC can be performed efficiently and easily, by tap filtering with tap coefficients given by ⁇ 1, -5, 20, 20, -5, 1 ⁇ .
• An eighth aspect of the present invention provides a pixel interpolating device comprising: an adding unit operable to add one of pixel values of an adjoining pixel pair composed of two adjoining reference pixels to another of the pixel values thereof, thereby obtaining a pixel value sum of the adjoining pixel pair; a subtracting unit operable to subtract one of the pixel values of the adjoining pixel pair from another of the pixel values thereof, thereby obtaining a pixel value difference of the adjoining pixel pair; a judging unit operable to judge whether an absolute value of the pixel value difference is equal to or smaller than a predetermined value; a plurality of registers; a first multiplying unit operable to multiply a predetermined coefficient to the pixel value sum; a second multiplying unit operable to multiply a predetermined coefficient to the pixel value difference; and a plurality of adding/subtracting units which are operable to add to or subtract from each of the plurality of registers the pixel value difference calculated by the subtracting unit or the multiplication results calculated by the first
• the judgment unit determines that the absolute value of the pixel value difference is equal to or smaller than the predetermined value
• the judgment unit controls the second multiplying unit and the plurality of adding/subtracting units to skip processing related to the adjoining pixel pair for which the absolute value of the pixel value difference is judged to be equal to or smaller than the predetermined value.
• a pixel interpolating device provided herein is operable to omit the multiplication processing of the second multiplying unit pertaining to the pixel value difference of these adjoining pixels, and also operable to omit the pertaining processing of addition or subtraction of a plurality of adding/subtracting units. Therefore, when the pixel interpolating device according to the present structure is used, speedy generation of predicted pictures in motion compensation can be realized, owing to the reduction of the J- £ ⁇
• Fig. 1 is a flow chart of a pixel interpolating method in Embodiment 1 of the present invention
• Fig. 2 is a descriptive illustration showing a pixel interpolating method in Embodiment 1 of the present invention
• Fig. 3 is a block diagram of a pixel interpolating device in Embodiment 2 of the present invention
• Fig. 4 is a block diagram of a pixel interpolating device in Embodiment 3 of the present invention.
• Fig. 5 is a layout illustration showing two packed pixel values in Embodiment 3 of the present invention.
• Fig. 6 is a block diagram of a motion picture encoding device in Embodiment 4 of the present invention.
• Fig. 7 is a block diagram of a motion picture decoding device in Embodiment 5 of the present invention
• Fig. 8 illustrates location relationship of full-pels, interpolated half-pels, and interpolated quarter-pels
• Fig. 9 is a flow chart showing an example of a conventional pixel interpolating method operable to determine a pixel value of an interpolated pixel by a 6-tap filter.
• Fig. 10 is a block diagram of a conventional pixel interpolating device by a 6-tap filter.
• Embodiment 1 is explained referring to the accompanying drawings.
• Fig. 1 is a flow chart of a pixel interpolating method in Embodiment 1 of the present invention.
• the flow chart shown in Fig. 1 is an example of the pixel interpolating method of the present embodiment.
• Fig. 2 is a descriptive illustration showing the pixel interpolating method in Embodiment 1 of the present invention.
• the pixel value of an interpolated pixel is calculated by filtering pixel values of pixels of the reference picture with a 6-tap filter having tap coefficients ⁇ 1, -5, 20, -5, 1 ⁇ .
• a[k] is the pixel value of a (k)th full-pel located at the (k)th position among pixels aligning on a horizontal or vertical line of the reference picture
• a pixel value "c” is a pixel value of a half-pel to be interpolated, which is located at the (k+1/2) position in between the (k)th full-pel and the (k+l)th full-pel.
• the pixel value "c” can be calculated by obtaining an intermediate pixel value "b” by Equation 1 and putting the value "b” into Equation 2 for normalization and saturation manipulation.
• Equation 3 the right hand term of Equation 1 can be replaced by a form of summation of terms of (a[k-2]-a [k-1]), (a[k-l]-a [k]), (a[k+l]-a [k+2]), and (a[k+2]-a [k+3]), which represent differences of the pixel values of respectively adjoining reference pixels, and (a [k] +a [k+1]), which represents a sum of the pixel values of another pair of adjoining reference pixels. (Equation 3)
• Equation 3 (a[k-2]-a[k-l])-4*(a[k-l]-a[k])+16*(a[k]+a[k+l])+4*(a[k+l]-a*[k+2]) -(a [k+2] -a [k+3]) Furthermore, as the result of replacement by the form of Equation 3, the coefficients of the multiplication to the pixel value differences of the adjoining pixels become coefficients "1", "-4", "4", and "-1" respectively, and the coefficient of the multiplication to the pixel value sum of the adjoining pixels becomes a coefficient "16".
• Equation 3 is rewritten as Equation 4.
• Equation 2 Equation 5
• the pixel value "c" of the interpolated pixel can be calculated by inputting into Equation 5 the intermediate pixel value "b", which is calculated by Equation 4.
• Equation 1 and Equation 2 the terms of multiplication and division by coefficients can be respectively performed by a single bit-shift operation, thereby it is possible to make the operation lighter.
• processing of the interpolating pixels is performed based on the form of Equation 4, wherein the operation terms consist of the subtraction and addition of the pixel values of the adjoining pixels.
• the numerical values denote coefficients of the right hand terms of Equation 3
• the rows indicate sets of adjoining pixels to be referred to
• the columns indicate intermediate pixel values of the interpolated pixels to be calculated.
• Calculation of the pixel value of an interpolated pixel is performed along a column to which the pixel value of the interpolated pixel to be calculated belongs.
• the coefficient "16" in the table calculation is performed for the product of the coefficient and the pixel value sum of the adjoining pixels belonging to the same row as the coefficient.
• the flow possesses loop structure that is composed of a series of processing from Step S 13 to Step S24.
• One loop of the processing corresponds to the processing for one row of the table shown in Fig. 2.
• Step SlO shown in Fig. 1 pixel interpolating processing starts.
• Step S 12 processing prior to the loop processing is performed as prologue processing.
• Step S 13 the pixel value a[k] of the pixel of the location "k” is read out.
• the pixel value a[k-l] of the pixel of at the location "k-1" adjoining the pixel at the location "k” is already read out in the previous loop processing, and the value is stored.
• the present processing is the first loop processing
• the pixel value a[k-l] is already read out in the prologue processing of Step S12, and the value is stored.
• Step S 14 a sum of the pixel value a[k-l] and the pixel value a[k] is left-shifted by 4 bits (equivalent to a value 16 times as large as the sum), then added to the intermediate pixel value b[m+2].
• the processing corresponds to the processing related to the coefficient "16" of the row that is surrounded by the dashed lines in the table shown in Fig. 2.
• Step S15 the difference (a[k-l]-a [k]) between the pixel value a[k-l] and the pixel value a[k] is calculated, and is used to replace the intermediate pixel value b[m+4].
• the processing corresponds to the processing related to the coefficient "1" of the row that is surrounded by the dashed lines in the table shown in Fig. 2.
• Step S16 whether the absolute value (
• Step S 16 When the judgment result of Step S 16 is "Yes” (the absolute value is equal to or less than the specific value), Step S 17 to Step S20 are skipped, and the control moves to Step S21.
• the judgment result is "No” (the absolute value exceeds the specific value)
• the control moves to Step S 17. ,
• Step S17 the difference (a[k-l]-a [k]) of the pixel values of the adjoining pixels (that is, b [m+4]) is subtracted from the intermediate pixel value b [mj. Thereby, the intermediate pixel value b[m] of the interpolated pixel is determined.
• the processing corresponds to the processing related to the coefficient "-1" of the row which is surrounded by the dashed lines in the table shown in Fig. 2.
• Step S 18 the difference of the pixel values of the adjoining pixels is left-shifted by 2 bits (equivalent to a value four times as large as the difference), and stored into the register "r".
• Step S 19 the value of the register "r" is added to the intermediate pixel value b[m+l].
• the processing corresponds to the processing related to the coefficient "4" of the row that is surrounded by the dashed lines in the table shown in Fig. 2.
• Step S20 the value of the register "r" is subtracted from the intermediate ⁇ g
• the processing corresponds to the processing related to the coefficient "-4" of the row which is surrounded by the dashed lines in the table shown in
• Step S 13 to Step S20 By the processing from Step S 13 to Step S20 mentioned above, the processing belonging to the row that is surrounded by the dashed lines in the table shown in Fig. 2 is performed.
• Step S21 normalization processing and saturation manipulation of the intermediate pixel value b [m] are performed to obtain the pixel value c [m].
• Step S22 the pixel value c [m] is outputted.
• Step S23 the variable "m” and the variable “k” are respectively increased by the value "1" for the next loop.
• Step S24 whether the series of the loop processing is completed or not is judged.
• the judgment result is "No” (the loop processing is not completed)
• the control returns to Step S13, and the loop processing from Step S13 to Step S24 is repeated.
• the judgment result is, "Yes" (the loop processing is completed)
• the control moves to Step S25.
• Step S25 the unfinished processing is performed as the epilogue processing. Then, the series of the pixel interpolating processing is completed at Step S26.
• Step S17 to Step S20 of Fig. 1 is skipped, in order not to perform the operation of the term related to the difference.
• the operation amount necessary for the pixel interpolating processing is reduced. Therefore, it is possible to improve the speeding of the pixel interpolating processing. Furthermore, the reduction in the operation amount is expected to lead to the reduction of the power consumption needed for the operation.
• the predetermined value "b ⁇ ” is set, for example, to a value equivalent to the least significant bit of a binary code, by which the pixel value of the pixel to be processed is represented.
• the predetermined value "b ⁇ ” is set to a value close to zero, or, a value of a noise level.
• the pixel interpolating method of the present embodiment When the pixel interpolating method of the present embodiment is applied to the motion picture encoding, it is expected that the effectiveness of the above-mentioned skipped processing is more clearly exhibited. This is due to the features of the motion picture encoding, in which high frequency components of a picture are removed in a low bit rate, thereby producing many pairs of the adjoining pixels that possess zero difference of the pixel values.
• the following sets of processing are performed in one loop processing: read-out of the pixel value of one pixel, calculation of the difference and sum of the pixel values of the one pixel and one adjoining pixel, and operation relating to the difference and sum.
• the pixel interpolating method of the present embodiment is very efficient and useful in realizing speedy processing of pixel interpolation, because of avoiding the unnecessary overlapped calculation, when compared with the conventional art that must calculate Equation 3 or Equation 4 every time when the pixel values of the interpolated pixels are calculated.
• Equation 3 the coefficient of each term of Equation 3 is the exponentiation of 2, thereby each multiplication can be instead performed by the bit-shift processing, as shown in Equation 4. It should be also noted that this fact contributes for the pixel interpolating method of the present embodiment to improve the speeding of the processing.
• Fig. 3 is a block diagram of a pixel interpolating device in Embodiment 2 of the present invention.
• the pixel interpolating device of the present embodiment practically performs the pixel interpolating method of the present invention described in Embodiment 1 of the present invention.
• the pixel interpolating device of the present embodiment comprises registers 11 to 16, adders 21 to 27, bit shifters 31 to 33, selectors 41 to 43, a comparator 51, a clipper 61, and constant units 71 and 72.
• the register is written as a symbol "R”, the adder as a symbol "+”, the bit shifter as a symbol "BS”, the selector as a symbol "SEL”, and the comparator as a symbol "C”.
• the registers 11 to 16 are controlled by a control signal (not shown in the figure) in inputting data.
• the pixel value a[k] of the pixel at the location "k” is inputted into the register 11 in synchronization with the control signal.
• the pixel values of the reference pixels are inputted into the register 11, one pixel value per cycle of the control signal, in the order of the array of the reference pixels.
• the pixel values a [k-5], a [k-4], a [k-3], a [k-2]... are inputted in this order. Therefore, the pixel values of the adjoining pixels are respectively stored in the register 11 and the register 12.
• the adder 21 calculates the difference of the pixel values of the adjoining pixels, and the adder 22 calculates the sum of the pixel values of the adjoining pixels. (The adder 21 functions as a subtractor. It is the same for the adder 23 and the adder 26.)
• the following explains a case in which the pixel value a[k+3] is stored in the register 11 in a certain cycle of the control signal. At this cycle, the pixel values a [k+3] and a [k+2] are stored in the register 11 and the register 12, respectively.
• the adder 21 outputs the difference (a[k+2]-a [k+3]) of the pixel values of the adjoining pixels.
• the adder 22 outputs the sum (a [k+2] +a [k+3]) of the pixel values of the adjoining pixels.
• the comparator 51 compares the absolute value (
• the signal outputted by the comparator 51 indicates that the absolute value of the difference exceeds the constant "b ⁇ "
• each of the selectors 41, 42, and 43 selects each of the outputs of the adder 23, 25, and 26.
• the bit shifter 31 bit-shifts the difference of the pixel values outputted by the adder 21 by 2 bits to the left (that is to multiply 4 times), and outputs the result.
• the bit shifter 32 bit-shifts the sum of the pixel values outputted by the adder 22 by 4 bits to the left (that is to multiply 16 times), and outputs the result.
• the difference (a[k+l]-a [k+2]) of the pixel values of the adjoining pixels one cycle before is stored in the register 13.
• the value of Equation 6 is determined.
• the value of Equation 6 is outputted by the selector 41. (Equation 6)
• Equation 8 By adding the value stored in the register 14 and the output of the bit shifter 32 by the adder 24, the value of Equation 8 is determined. (Equation 8)
• Equation 10 By adding the value stored in the register 15 and the output of the bit shifter 31 by the adder 25, the value of Equation 10 is determined. The value of Equation 10 is outputted by the selector 42.
• Equation 12 By subtracting the difference of the pixel values outputted by the adder 21 from the value stored in the register 16 by the adder 26, the value of Equation 12 is determined.
• the determined value of Equation 12 is equal to the right hand term of
• Equation 3 The determined value is then outputted from the selector 43 as the intermediate pixel value b[m] of the interpolated pixel.
• the adder 27 adds the constant "16" set in the constant unit 72 to the intermediate pixel value outputted from the selector 43.
• the bit shifter 33 bit-shifts the result to the right by 5 bits (that is equivalent to dividing by 32).
• the clipper 61 performs the saturation manipulation to, the bit-shifted result. After all, the pixel value c[m] of the interpolated pixel (the value of Equation 5) is obtained.
• Equation 3 by converting 6-tap filtering for the interpolation of the pixel value into the form of Equation 3, the terms of Equation 3 are changed to possess the coefficients "4" and "16" that are the exponentiation of 2. Thereby, it becomes possible to perform the two pieces of multiplication operation only by a 2-bit left shifter and a 4-bit left shifter, respectively. Furthermore, in normalization of the intermediate pixel value of the interpolated pixel, the denominator of the division is the constant "32" that is the exponentiation of 2; therefore, it is possible to perform the division operation by a 5-bit right shifter. Since the multiplication operation and the division operation in the pixel interpolating device of the present embodiment can be performed by the bit shifters as mentioned above, it is possible to reduce the circuit scale of the pixel interpolating device.
• Fig. 4 is a block diagram of a pixel interpolating device in Embodiment 3 of the present invention.
• the pixel interpolating device of the present embodiment is operable to process at once the picture data having the 16-bit structure, in which two pixel values having the 8-bit structure are packed.
• the pixel interpolating device of the present embodiment is structured by arranging in parallel two pieces of the pixel interpolating device of Embodiment 2 of the present invention shown in Fig. 3 and omitting the common parts thereof.
• the pixel interpolating device of the present embodiment includes an upper part that determines the pixel values of the odd-numbered interpolated pixels, a lower part that determines the pixel values of the even-numbered interpolated pixels, and the other common parts.
• the upper part comprises registers 11, 14a, and 16a, adders 21a to 27a, bit shifters 31a to 33a, selectors 41a to 43a, a comparator 51a, a clipper 6Ia 3 and a constant unit 72a.
• the lower part comprises registers 13b and 15b, adders 21b to 27b, bit shifters 31b to 33b, selectors 41b to 43b, a comparator 51b, a clipper 61b and a constant unit 72b.
• the common part comprises a constant unit 71.
• the register is written as a symbol "R”
• the adder is written as a symbol "+”
• the bit shifter is written as a symbol "BS”
• the selector is written as a symbol "SE”
• the comparator is written as a symbol "C”.
• Fig. 5 is a layout illustration showing two packed pixel values in Embodiment 3 of the present invention.
• the image data having the 16-bit structure illustrated in Fig. 5 contains a pixel value "a_even” of the even-numbered pixel in the upper 8 bits thereof, and a pixel value "a_odd” of the odd-numbered pixel in the lower 8 bits thereof.
• the arrangement of the two packed pixel values may differ from that of Fig. 5.
• the operation of the pixel interpolating device of the present embodiment is outlined in the following, with reference to Fig. 4.
• the pixel value a[2k], which is the upper pixel data of the 16-bit pixel data shown in Fig. 5, is inputted into an input terminal 91a.
• the pixel value a[2k+l], which is the lower pixel data of the 16-bit pixel data, is inputted into an input terminal 91b.
• the pixel value c[2m+l] of the odd-numbered interpolated pixel is outputted to an output terminal 99a.
• the pixel value c[2m] of the even-numbered interpolated pixel is outputted to an output terminal 99b.
• one piece of the 16-bit picture data, in which two pixel values are packed is inputted every cycle of the control signal.
• the odd-numbered pixel values are inputted into the input terminal 91a, in the order of the pixel values a[2k-3], a[2k-l], a[2k+l], a[2k+3], ..., for every cycle.
• the even-numbered picture values are inputted into the input terminal 91b, in the order of the pixel values a[2k-2], a[2k], a[2k+2], a[2k+4], ..., for every cycle.
• the adder 21a calculates the difference (a[2k+l]-a [2k+2]) of the pixel values of the adjoining pixels.
• the adder 22a calculates the sum (a [2k+l] +a ⁇
• the adder 21b calculates the difference (a[2k+2]-a [2k+3]) of the pixel values of the adjoining pixels.
• the adder 22b calculates the sum (a [2k+2] +a [2k+3]) of the pixel values of the adjoining pixels.
• bit shifters, the adders, the registers, and the clippers which are located in latter stages than the adders 21a, 21b, 22a, and 22b, perform the almost same operation as the corresponding units of the pixel interpolating device of Embodiment 2 of the present invention. Therefore, the further description of the operation thereof is omitted.
• the selector 43 a outputs the value of Equation 13 as the intermediate pixel value b[2m+l] of the odd-numbered interpolated pixel.
• the selector 43b outputs the value of Equation 14 as the intermediate pixel value b[2m] of the even-numbered interpolated pixel.
• Equation 5 The normalization processing and the saturation manipulation according to Equation 5 are performed for the intermediate pixel value of Equation 13 outputted from the selector 43 a and the intermediate pixel value of Equation 14 outputted from the selector 43b at the respective latter stages.
• the pixel value c[2m+l] of the odd-numbered interpolated pixel is outputted to the output terminal 99a.
• the pixel value c[2m] of the even-numbered interpolated pixel is outputted to the output terminal 99b.
• the selector 41b selects the output of the adder 23b
• the selector 42b selects the output of the adder 25b
• the selector 43 a selects the output of the adder 26a.
• the selector 41a selects the output of the adder 23 a
• the selector 42a selects the output of the adder 25a
• the selector 43 b selects the output of the adder 26b.
• the selector 41b selects the output of the register 13b
• the selector 42b selects the output of the register 15b
• the selector 43a selects the output of the register 16a.
• the bit shifter 31a, the adder 23b, the adder 25b, and the adder 26a do not perform the operation.
• the selector 41a selects the output of the adder 21a
• the selector 42a selects the output of the adder 24a
• the selector 43b selects the output of the selector 42b.
• the bit shifter 31b, the adder 23 a, the adder 25 a, and the adder 26b do not perform the operation.
• the operating units related to the operation of the difference do not perform the operation; therefore, the power consumption that is otherwise necessary for the operation by these operators can be reduced.
• the pixel interpolating device of the present embodiment can enjoy equally the features that the pixel interpolating device of Embodiment 2 of the present invention enjoys.
• the pixel interpolating device of the present embodiment is extended from the current twofold parallel structure to a fourfold parallel structure, it is easy to realize an extended pixel interpolating device which is operable to input pixel data having a 32-bit structure in which four pieces of 8-bit structure pixel values are packed, and to perform the pixel interpolating processing of four interpolated pixels at once.
• the explanation is given along the generation of the interpolated pixel by the 6-tap filter in conformity to the H.264 encoding standard.
• the present invention is not limited to the generation of the interpolated pixel according to the H.264 encoding standard, but can be applied to the generation of interpolated pixels and the motion compensation by other kinds of tap filter processing. In these applications, the above-mentioned effects are also equally enjoyed.
• Embodiment 4 is not limited to the generation of the interpolated pixel according to the H.264 encoding standard, but can be applied to the generation of interpolated pixels and the motion compensation by other kinds of tap filter processing. In these applications, the above-mentioned effects are also equally enjoyed.
• FIG. 6 is a block diagram of a motion picture encoding device in Embodiment 4 of the present invention.
• a motion picture encoding device 100 of the present embodiment shown in Fig. 6 comprises a subtractor 110, an orthogonal transforming/quantizing unit 120, a variable-length encoding unit 130, an inverse quantizing/inverse orthogonal transforming unit 140, an adder 150, an in-loop filter 160, a frame buffer 170, and a motion compensator 180.
• the motion compensator 180 comprises a predicted picture generator 181 and a pixel interpolating unit 182.
• Fig. 6 illustrates a part of the processing blocks of the motion picture encoding device 100 of the present embodiment;
• Fig. 6 illustrates only the processing blocks related to the inter-picture prediction encoding, but excluding the processing blocks related to the intra-picture prediction encoding.
• the pixel interpolating unit 182 which the motion picture encoding device 100 of the present embodiment comprises, is the pixel interpolating device of Embodiment 2 or the pixel interpolating device of Embodiment 3 of the present invention.
• the original picture inputted from an input terminal 101 is processed by the subtractor 110 to calculate the difference from the predicted picture outputted by the motion compensator 180.
• the calculated difference or the remainder is outputted to the orthogonal transforming/quantizing unit 120.
• the orthogonal transforming/quantizing unit 120 performs orthogonal transformation (for example, discrete cosine transformation) to the remainder outputted by the subtractor 110, and quantizes the transformed coefficients obtained, thereby outputting the quantized, transformed coefficients.
• the variable-length encoding unit 130 performs variable length encoding to the quantized, transformed coefficients outputted by the orthogonal transforming/quantizing unit 120, thereby outputting the encoded result as encoded picture data to an output terminal 109.
• the inverse quantizing/inverse orthogonal transforming unit 140 performs inverse quantization/inverse orthogonal transformation to the quantized, transformed coefficients, thereby obtaining a restored remainder.
• the adder 150 adds the restored remainder to the predicted picture outputted by the motion compensator
• the reconstructed picture is removed block noise by the in-loop filter 160, and stored in the frame buffer 170 as a decoded picture.
• the motion compensator 180 reads out the decoded picture as a reference picture from the frame buffer 170, performs the operation in the pixel interpolating unit
• the predicted picture generator 182 based on the information of the motion vector (not shown in the figure) to calculate the interpolated pixel, and generates, in the predicted picture generator 181, the predicted picture for which the motion compensation is made.
• the processing in the motion compensator 180 among the processing from performing the variable length encoding to the inputted original picture up to outputting the encoded picture data, the processing in the motion compensator 180, especially, the pixel interpolating processing in the pixel interpolating unit 182, occupies a large percentage of the entire processing. Therefore, use of the pixel interpolating device of Embodiment 2 or Embodiment 3 of the present invention for the pixel interpolating unit 182 makes it possible to realize high-speed pixel interpolating processing, and hence the motion compensator 180 can perform the motion compensation processing at high speed. As a result, it is possible to provide the motion picture encoding device 100, which is operable to process at high speed. Furthermore, the motion picture encoding device 100 thus provided possesses the features that the power consumption is low, since the pixel interpolating unit 182 is constituted so as to avoid unnecessary operation.
• the motion picture encoding device 100 of the present embodiment can efficiently encode the inputted original picture to variable length coded picture data in conformity to the H.264 standard.
• the motion picture encoding device 100 of the present embodiment is not limited to the variable length encoding in conformity to the H.264 encoding standard. By using other kinds of tap filter processing, the motion picture encoding device 100 can be extendedly applied to the variable length encoding according to the other encoding standards.
• Fig. 7 is a block diagram of a motion picture decoding device in Embodiment 5 of the present invention.
• the same symbols are given to elements each having the same function as elements of Fig. 6 in order to omit explanation.
• a motion picture decoding device 200 of the present embodiment shown in Fig. 7 comprises a variable length decoding unit 210, an inverse quantizing/inverse orthogonal transforming Unit 140, an adder 150, an in-loop filter 160, a frame buffer 170, and a motion compensator 180.
• the motion compensator 180 comprises a predicted picture generator 181 and a pixel interpolating unit 182.
• Fig. 7 illustrates a part of the processing blocks of the motion picture decoding device 200 of the present embodiment; Fig. 7 illustrates only the processing blocks related to the inter-picture prediction decoding, but excluding the processing blocks related to the intra-picture prediction decoding.
• the pixel interpolating unit 182, which the motion picture decoding device 200 of the present embodiment comprises, is the pixel interpolating device of Embodiment 2 or the pixel interpolating device of Embodiment 3 of the present invention.
• variable length encoded picture data fed into the input terminal 201 is decoded into quantized, transformed coefficients by the variable length decoding unit 210, and outputted to the inverse quantizing/inverse orthogonal transforming unit 140.
• the inverse quantizing/inverse orthogonal transforming unit 140 performs the inverse quantization/inverse orthogonal, transformation to the quantized, transformed coefficients decoded by the variable length decoding unit 210, thereby obtaining a remainder.
• the adder 150 adds the remainder and the predicted picture outputted by the motion compensator 180, thereby generating a reconstructed picture.
• the reconstructed picture is removed block noise by the in-loop filter 160, and temporarily stored in the frame buffer 170 as a decoded picture.
• the decoded picture is then outputted to an output terminal 209.
• the motion compensator 180 reads out the decoded picture as a reference picture from the frame buffer 170, performs the operation in the pixel interpolating unit 182 based on the information of the motion vector (not shown in the figure) to calculate the interpolated pixel, and generates, in the predicted picture generator 181, the predicted picture for which the motion compensation is made.
• the processing in the motion compensator 180 among the processing from decoding the variable length encoded picture data inputted and outputting the decoded picture, the processing in the motion compensator 180, especially, the pixel interpolating processing in the pixel interpolating unit 182, occupies a large percentage of the entire processing.
• the pixel interpolating device of Embodiment 2 or Embodiment 3 of the present invention makes it possible to realize high-speed pixel interpolating processing, and hence the motion compensator 180 can perform the motion compensation processing at high speed.
• the motion picture decoding device 200 which is operable to process at high speed.
• the motion picture decoding device 200 thus provided possesses the features that the power consumption is low, since the pixel interpolating unit 182 is constituted so as to avoid unnecessary operation.
• the motion picture decoding device 200 of the present embodiment can efficiently decode the variable length encoded picture data in conformity to the H.264 standard
• the motion picture decoding device 200 of the present embodiment is not limited to the variable length decoding in conformity to the H.264 encoding standard. By using other kinds of tap filter processing, the motion picture decoding device 200 can be extendedly applied to the variable length decoding of other picture data encoded in conformity to the other encoding standards.
• the present invention it is possible to provide the pixel interpolating method and device in which reduction in the power consumption and speeding in the processing are both realized.
• the pixel interpolating device thus provided can be applied to purposes such as motion compensation in prediction encoding of a motion picture, or pixel generation in size-enlarging or size-reducing of a picture.
• the pixel interpolating method according to the present invention can be applied to a motion picture processing device, such as a camera-built-in cellular phone, which requires high-speed processing of motion pictures, and the related fields.

Landscapes

• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Image Processing (AREA)
• Compression Or Coding Systems Of Tv Signals (AREA)
• Television Systems (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=36695052

Family Applications (1)

Country Status (4)

Families Citing this family (3)

Family Cites Families (5)

• 2006
  • 2006-02-17 EP EP20060714502 patent/EP1851718A2/de not_active Withdrawn
  • 2006-02-17 WO PCT/JP2006/303363 patent/WO2006090811A2/en not_active Ceased
  • 2006-02-17 CN CN2006800060241A patent/CN101128846B/zh not_active Expired - Fee Related
  • 2006-02-17 JP JP2007535161A patent/JP4847460B2/ja not_active Expired - Fee Related

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events