JPH11150663A - Interpolation device and recording medium recording interpolation processing program - Google Patents

Abstract

(57) Abstract: An object of the present invention is to provide an interpolation device for interpolating an image in which lattice points that output a predetermined color and empty lattice points that do not output the color are arranged in a checkered pattern. With respect to a recording medium on which such an image interpolation processing program is recorded, it is necessary to estimate the spatial directionality of a vacancy based on the pixel output of an adjacent lattice point, and to accurately obtain the interpolation amount of the vacancy. Aim. SOLUTION: In an interpolation device, a horizontal slope HD indicating a difference between pixel outputs of two grid points horizontally adjacent to an empty grid point, and two grid points vertically adjacent to the empty grid point. Comparing the horizontal slope HD and the vertical slope VD with the slope calculating means for calculating the vertical slope VD indicating the difference between the pixel outputs of
For an arbitrary value Th, “HD> VD + Th”, “VD> H
D + Th ”or“ | HD−VD | ≦ Th ”is satisfied, and an interpolation amount determining unit that determines an interpolation amount of a vacant point is configured.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an interpolation device for interpolating an image in which lattice points for outputting a predetermined color and empty lattice points for not outputting the color are arranged in a checkered pattern, and to such an interpolation apparatus. The present invention relates to a recording medium on which an image interpolation processing program is recorded.

[0002]

2. Description of the Related Art Some electronic cameras use a single image sensor to generate an image signal of a color image. As shown in FIG. 12A, for example, such an image sensor includes three color filters of RGB (red, green, and blue) arranged in a Bayer array. Where green (G)
Focusing on, as shown in FIG. 12 (2), the image sensor is configured by arranging lattice points that are pixels that output green and empty lattice points that are pixels that do not output green in a checkered pattern. It can be said that there is.

Further, in a color filter having a configuration in which two image pickup devices composed of only green (G) as shown in FIG. 13A are shifted by pixels, a green color is output as shown in FIG. It can be said that virtual vacant lattice points that do not exist and lattice points that output green color are arranged in a checkered pattern. Further, in the two-plate type color filter, one image sensor is composed of only G, and the other image sensor has “pixels that output red” and “pixels that output blue” arranged in a checkered pattern. Some are configured. That is, when attention is paid to red, pixels that output blue are vacancy points, and when attention is paid to blue, pixels that output red are vacancy points.

Conventionally, a method of interpolating a color image corresponding to an image signal generated by such an image pickup device includes a method of calculating an average value of pixel outputs of lattice points adjacent to a vacant lattice point by calculating a pixel output of the vacant lattice point. (Interpolation amount). Also, US Pat. No. 5,373,322 (Class 34)
8) describes the spatial direction of vacancies by color gradient (chromi
There is disclosed a technique for estimating the amount of interpolation based on pixel outputs of grid points adjacent to each other in a direction of strong similarity by estimating the amount by interpolation.

FIG. 14 is a diagram for explaining the conventional interpolation disclosed in US Pat. No. 5,373,322. In the drawing, G11, G13,..., G66, etc. indicate green (G) pixel outputs, and R12, R14,.
Reference numeral 56 denotes red (R) pixel output, and B21, B2
Reference numerals 3,..., B67, etc. indicate blue (B) pixel outputs.

In the above-mentioned US patent, when calculating the green interpolation amount G34 of the pixel outputting R34, information HDiff relating to the color gradient in the horizontal direction is obtained by HDiff = | ((R32 + R36) / 2) -R34 | Information on color gradient in direction HDiff
VDiff = | ((R14 + R54) / 2) -R34 |

The method for calculating G34 is H
The cases are classified according to the magnitude relationship between Diff and VDiff.
That is, when HDiff <VDiff (when the similarity in the horizontal direction is strong), G34 is obtained by G34 = (G33 + G35) / 2, and when VDiff <HDiff (when the similarity in the vertical direction is strong), G34 is determined. G34 = (G24 + G44) / 2. If HDiff = VDiff, G34 is determined by G34 = (G24 + G44 + G33 + G35) / 4.

[0008]

That is, in such prior art, information on the color inclination of green is estimated based on the output of the red pixel, even though the processing is for obtaining the amount of interpolation of green. . Therefore, in an image in which the relationship between the green color gradient and the red color gradient is not constant, the “similarity of the green pixel output” does not match the “similarity of the red pixel output”, and the space of the empty grid point There was a risk that the general direction was incorrectly estimated.

Therefore, when a direction having a strong similarity is erroneously estimated as described above, an appropriate interpolation amount cannot be obtained, and in the worst case, the image may be broken. Therefore, according to the present invention, the spatial direction of a vacancy is estimated based on the pixel output of an adjacent lattice point, and the interpolation amount of the vacancy is accurately obtained. It is intended to provide a device.

According to the present invention, the spatial directionality of a vacancy is estimated based on the pixel output of an adjacent lattice point, and the interpolation amount of the vacancy is accurately determined. It is an object of the present invention to provide a recording medium in which an interpolation processing program that is frequently obtained is recorded.

[0011]

According to a first aspect of the present invention, there is provided an interpolation apparatus for interpolating an image in which lattice points for outputting a predetermined color and empty lattice points for not outputting the color are arranged in a checkered pattern. In the interpolating apparatus for calculating the interpolation amount that becomes the pixel output of the vacant grid point, the positions of the grid point and the vacant grid point are defined as “the horizontal axis X whose right direction is positive” and “the downward direction is positive. When the pixel output of the lattice point is represented by G (X, Y), the coordinates of the empty lattice point are represented by (k,
p), the horizontal slope HD indicating the difference between the pixel outputs of the two grid points horizontally adjacent to the vacant grid point, and the horizontal slope HD of the two grid points vertically adjacent to the vacant grid point. The vertical inclination VD indicating the difference between the pixel outputs is represented by HD = | G (k−1, p) −G (k + 1, p) | VD = | G (k, p−1) −G (k, p + 1) | Assuming that the inclination calculating means to calculate and the interpolation amount of the vacant point is G '(k, p), the horizontal inclination HD and the vertical inclination VD calculated by the inclination calculating means are compared with each other to obtain an arbitrary value.
For Th, when “HD> VD + Th” is satisfied, G ′ (k, p) = ｛G (k, p−1) + G (k, p
+1)｝ / 2, and when “VD> HD + Th” holds, G ′ (k, p) = {G (k−1, p) + G (k +
1, p)｝ / 2, and when “| HD−VD | ≦ Th” is satisfied, G ′ (k, p) = {G (k−1, p) + G (k +
(P) + G (k, p−1) + G (k, p + 1)｝ / 4.

According to a second aspect of the present invention, in the interpolation apparatus of the first aspect, the inclination calculating means sets the value of Th to a predetermined constant value. According to a third aspect of the present invention, in the interpolation apparatus according to the first aspect, the slope calculating means determines the value of Th in association with the pixel output of a grid point adjacent to the empty grid point. I do.

According to a fourth aspect of the present invention, in the interpolation apparatus according to the first aspect, the inclination calculating means includes a maximum value of a pixel output of a grid point adjacent to the empty grid point, and a grid value adjacent to the empty grid point. Th based on the average value of the pixel outputs of the points, the average value of the pixel outputs of the grid points horizontally adjacent to the empty grid point, or the average of the pixel outputs of the grid points vertically adjacent to the empty grid point.
Is determined.

According to a fifth aspect of the present invention, there is provided an interpolation apparatus for interpolating an image in which lattice points for outputting a predetermined color and empty lattice points for not outputting the color are arranged in a checkered pattern. In the interpolating apparatus for obtaining an interpolation amount that becomes a pixel output of a horizontal grid point, a horizontal slope indicating a difference between pixel outputs of two grid points horizontally adjacent to the empty grid point, and a vertical slope adjacent to the empty grid point in the vertical direction Inclination calculating means for calculating a vertical inclination indicating a difference between pixel outputs of two grid points, a horizontal average indicating an average of pixel outputs of two grid points horizontally adjacent to an empty grid point,
An average calculating means for calculating a vertical average indicating an average of pixel outputs of two grid points vertically adjacent to the empty grid point, and a horizontal inclination and a vertical inclination calculated by the inclination calculating means are predetermined. The first determination means for determining whether or not these inclinations can be considered to be the same by comparing based on the conditions obtained, and the horizontal average and the vertical average calculated by the average calculation means are determined in advance. The second determination means for determining whether or not these averages can be considered to be the same by comparing based on the conditions set forth above, and the horizontal inclination and the vertical inclination do not match by the first determination means. If it is determined, the first interpolation amount determining means for determining the interpolation amount of the empty grid point based on the pixel output of two grid points adjacent to the empty grid point in the direction of less inclination, Determination means and second determination means If it is determined that the horizontal inclination and the vertical inclination match and the horizontal average and the vertical average match, the value corresponding to the average of the pixel outputs of the four grid points adjacent to the empty grid point is determined to be empty. The horizontal slope and the vertical slope are matched by the second interpolation amount determining means as the interpolation amount of the child point, the first determination means and the second determination means, and the horizontal average and the vertical average do not match. Is determined, the interpolation amount of the empty grid point is determined based on the similarity between the pixel outputs of the grid points adjacent to the empty grid point and the pixel outputs of the grid points arranged around these grid points. And a third interpolation amount determining means.

According to a sixth aspect of the present invention, there is provided an interpolation apparatus for interpolating an image in which lattice points for outputting a predetermined color and empty lattice points for not outputting the color are arranged in a checkered pattern. In the interpolating apparatus for obtaining an interpolation amount that becomes a pixel output of a horizontal grid point, a horizontal slope indicating a difference between pixel outputs of two grid points horizontally adjacent to the empty grid point, and a vertical slope adjacent to the empty grid point in the vertical direction Inclination calculating means for calculating a vertical inclination indicating a difference between pixel outputs of two grid points, a horizontal average indicating an average of pixel outputs of two grid points horizontally adjacent to an empty grid point,
An average calculating means for calculating a vertical average indicating an average of pixel outputs of two grid points vertically adjacent to the empty grid point, and a horizontal average and a vertical average calculated by the average calculating means are predetermined. By comparing based on the conditions that have been determined, the determination means to determine whether or not these averages can be considered to match, if the determination means determines that the horizontal average and the vertical average do not match, The first interpolation amount calculation means for performing weighting corresponding to the horizontal inclination and the vertical inclination calculated by the inclination calculation means to calculate the interpolation amount of the vacancy point, and the horizontal average and the vertical average match by the determination means. And a second interpolation amount calculating means for setting a value corresponding to an average of pixel outputs of four grid points adjacent to the vacancy point as an interpolation amount of the vacancy point when it is determined that I do.

That is, in the prior art, when the horizontal average and the vertical average match, if the interpolation amount of the vacancy is calculated by weighting corresponding to the horizontal inclination and the vertical inclination, the smoothness of the image may be impaired. However, in the interpolation device according to the sixth aspect of the present invention, the second interpolation amount calculating means may calculate a value corresponding to the average of the pixel outputs of four grid points adjacent to the empty grid point. it can.

According to a tenth aspect of the present invention, there is provided a recording medium on which an interpolation processing program is recorded, wherein a lattice point for outputting a predetermined color and a vacant lattice point for not outputting the color are arranged in a checkerboard pattern. In a computer, and records an interpolation processing program for calculating an interpolation amount that becomes a pixel output of the empty grid point, wherein the positions of the grid points and the empty grid points are defined as “the right side is defined as a positive direction. When the pixel output of the lattice point is represented by G (X, Y), the coordinates of the vacant lattice point are represented by ((X, Y)). k, p), a horizontal slope H indicating a difference between pixel outputs of two grid points horizontally adjacent to the vacant grid point.
HD = | G (k−1, p) −G (k + 1, p) |, where D and the vertical slope VD indicating the difference between the pixel outputs of two grid points vertically adjacent to the vacant grid point. VD = | G (k, p−1) −G (k, p + 1) |, and the inclination calculation procedure is assuming that the interpolation amount of the vacancy is G ′ (k, p). The horizontal inclination HD and the vertical inclination VD are compared, and an arbitrary value is obtained.
For Th, when “HD> VD + Th” is satisfied, G ′ (k, p) = ｛G (k, p−1) + G (k, p
+1)｝ / 2, and when “VD> HD + Th” holds, G ′ (k, p) = {G (k−1, p) + G (k +
1, p)｝ / 2, and when “| HD−VD | ≦ Th” is satisfied, G ′ (k, p) = {G (k−1, p) + G (k +
(P) + G (k, p−1) + G (k, p + 1)｝ / 4. An interpolation processing program for causing a computer to execute an interpolation amount determination procedure is recorded.

According to another aspect of the present invention, there is provided a recording medium on which an interpolation processing program is recorded, wherein a grid point for outputting a predetermined color and an empty grid point for not outputting the color are arranged in a checkerboard pattern. Is executed by a computer, and an interpolation processing program for obtaining an interpolation amount that becomes a pixel output of the empty lattice point is recorded on the recording medium, wherein the pixel output of two lattice points horizontally adjacent to the empty lattice point is stored. A horizontal slope indicating the difference between the horizontal grid points and a vertical tilt indicating the difference between the pixel outputs of two grid points adjacent in the vertical direction to the empty grid point. Average calculation for calculating a horizontal average indicating an average of pixel outputs of two grid points adjacent in the direction and a vertical average indicating an average of pixel outputs of two grid points adjacent in the vertical direction to the empty grid point. Procedure and the slope calculation procedure. By comparing the horizontal slope and the vertical slope based on predetermined conditions, a first determination procedure for determining whether or not these slopes can be considered to be coincident, and an average calculation procedure are performed. By comparing the horizontal average and the vertical average based on a predetermined condition, a second determination procedure for determining whether or not these averages can be considered to match, and a first determination procedure If it is determined that the horizontal inclination does not match the vertical inclination,
A first interpolation amount determining step of determining an interpolation amount of the empty lattice point based on pixel outputs of two lattice points adjacent to the empty lattice point in a direction with a small inclination, a first determination step, and a If it is determined that the horizontal slope and the vertical slope match by the second determination procedure, and that the horizontal average and the vertical average match,
A second interpolation amount determination procedure, and a first determination procedure and a second determination procedure, in which a value corresponding to an average of pixel outputs of four grid points adjacent to the empty grid point is used as an interpolation quantity of the empty grid point. When it is determined that the horizontal tilt and the vertical tilt match and the horizontal average and the vertical average do not match, the pixel outputs of the grid points adjacent to the empty grid points and the pixel outputs around these grid points are determined. Based on the similarity with the pixel output of the grid point, based on the similarity, the third interpolation amount determination procedure for determining the interpolation amount of the empty grid point, and an interpolation processing program for causing a computer to execute, characterized by recording I do.

According to a ninth aspect of the present invention, there is provided a recording medium on which an interpolation processing program is recorded, wherein a lattice point for outputting a predetermined color and a vacant lattice point for not outputting the color are arranged in a checkerboard pattern. Is executed by a computer, and an interpolation processing program for obtaining an interpolation amount that becomes a pixel output of the empty lattice point is recorded on the recording medium, wherein the pixel output of two lattice points horizontally adjacent to the empty lattice point is stored. A horizontal slope indicating the difference between the horizontal grid points and a vertical tilt indicating the difference between the pixel outputs of two grid points adjacent in the vertical direction to the empty grid point. Average calculation for calculating a horizontal average indicating an average of pixel outputs of two grid points adjacent in the direction and a vertical average indicating an average of pixel outputs of two grid points adjacent in the vertical direction to the empty grid point. Procedure and the average calculation procedure By comparing the horizontal average and the vertical average based on a predetermined condition, a determination procedure for determining whether or not these averages can be considered to be the same, and the horizontal average and the vertical average are determined by the determination procedure. If it is determined that does not match, a first interpolation amount calculation procedure of calculating the interpolation amount of the vacant point by performing weighting corresponding to the horizontal inclination and the vertical inclination calculated by the inclination calculation procedure, the determination If it is determined by the procedure that the horizontal average and the vertical average match, a value corresponding to the average of the pixel outputs of the four grid points adjacent to the vacancy is used as the interpolation amount of the vacancy. An interpolation processing program for causing a computer to execute the second interpolation amount calculation procedure is recorded.

That is, in the prior art, when the horizontal average and the vertical average match, if the interpolation amount of the vacancy is calculated by weighting corresponding to the horizontal inclination and the vertical inclination, the smoothness of the image may be impaired. However, according to the interpolation processing program recorded on the recording medium according to the ninth aspect of the present invention, the second interpolation amount calculation procedure corresponds to the average of the pixel outputs of four lattice points adjacent to the empty lattice point. Can be calculated.

[0021]

Embodiments of the present invention will be described below in detail with reference to the drawings. In addition, about the embodiment corresponding to the invention of Claim 1 thru | or Claim 6,
The description will be made using an electronic still camera to which the interpolation device according to the invention is applied.

(First Embodiment) FIG. 1 is a functional block diagram of the first embodiment. This embodiment corresponds to an embodiment corresponding to the first and second aspects of the present invention. In the present embodiment and each of the embodiments described below, pixels that output green are grid points, and pixels that output red or blue are vacant points. That is, the interpolation processing in this embodiment and each embodiment described below
This corresponds to a process of obtaining an interpolation amount corresponding to a green pixel output of these empty lattice points.

In FIG. 1, the output of the control unit 40 is a photographing optical system 41 and an OLPF (optical low-pass filter: Optic).
cal Low Pass Filter) 42, image sensor 43, A / D converter 44, image buffer memory 45, and signal processor 46
Connected to. The signal processing unit 46 includes an interpolation processing unit 47, and the interpolation processing unit 47 includes a slope calculation unit 48 and an interpolation amount calculation unit 49.

The optical image acquired by the photographing optical system 41 is
Filtered by the LPF 42 and the image sensor 43
Given to. Note that the optical image acquired by the photographing optical system 41 may be directly provided to the image sensor 43 without providing the OLPF 42. The output of the image sensor 43 is connected to an A / D converter 44, the output of the A / D converter 44 is connected to an image buffer memory 45, and the output of the image buffer memory 45 is connected to a signal processor 46.

In this embodiment, as shown in FIG. 12A, an image sensor having a structure in which three color filters of RGB are arranged in a Bayer array is applied as the image sensor 43.
The image buffer memory 45 stores three colors corresponding to these three colors.
It consists of two areas. Here, the image buffer memory 4
The area corresponding to green in 5 is an area for storing green image data (corresponding to a pixel output of a grid point) generated by the A / D converter 44, and a green image obtained by interpolation processing described later. And an area for storing data (corresponding to the amount of interpolation of a vacancy).

As for the correspondence between the first and second aspects of the present invention and the present embodiment, the inclination calculating means corresponds to the inclination calculating section 48, and the interpolation amount determining means corresponds to the interpolation amount calculating section 4.
9 corresponds. FIG. 2 is an operation flowchart of the first embodiment. FIG. 3 is an operation flowchart of the interpolation processing of the first embodiment.

Here, in order to simplify the following description,
The position of each pixel of the image sensor 43 is represented by a coordinate system composed of a "horizontal axis X having a positive right direction" and a "vertical axis Y having a downward positive direction". The pixel output of the grid point is G (X, Y)
, And the interpolation amount of the vacancy is represented by G ′ (X, Y).

FIG. 4 is a diagram showing the pixel output of such a grid point and the interpolation amount of an empty grid point. In the figure, X_m
ax indicates the maximum value of the X component, and Y_max indicates the maximum value of the Y component. Hereinafter, the operation of the first embodiment will be described with reference to FIGS. When the main power supply is turned on, the imaging optical system 41 and the OLPF
An optical image provided via 42 is formed on a photoelectric conversion surface (not shown), and a signal charge is generated by the photoelectric conversion. Further, the image sensor 43 scans the signal charges thus generated to generate an image signal, and the A / D converter 44
Give to.

The A / D conversion section 44 performs A / D conversion on the image signal given in this way to generate image data (S1 in FIG. 2). In this embodiment and each embodiment described later, the image data corresponds to a pixel output corresponding to each pixel of the image sensor 43. The image data generated in this manner is divided into three colors of RGB and stored in the areas in the image buffer memory 45 corresponding to each color (FIG. 2S).
2).

Such processing is repeatedly performed based on the timing control by the control unit 40. Further, as shown in FIG. 4, the green image data (pixel output of grid points / interpolation amount of vacant grid points) is stored in association with the coordinates shown in the above-described coordinate system. The controller 40 determines whether or not a release button (not shown) provided on the housing of the electronic still camera has been pressed (FIG. 2S).
3).

The control unit 40 makes such a determination,
When it recognizes that the release button has not been pressed,
By the above-described timing control, generation (FIG. 2S1) and storage (FIG. 2S2) of image data are repeatedly instructed. When the control unit 40 recognizes that the release button has been pressed, the interpolation processing unit 4 in the signal processing unit 46 recognizes that the release button has been pressed.
7 is instructed to execute an interpolation process as described later (S4 in FIG. 2).

That is, in the present embodiment, the image data stored in the image buffer memory 45 at the time when the release button is pressed is subjected to the interpolation processing. Note that, in order to display an image, the interpolation process may be performed at any time regardless of the time point when the release button is pressed. Further, after the release, the processing after S1 may be repeatedly performed.

The image data thus interpolated is stored in the image buffer memory 45 (FIG. 2S
5). Hereinafter, the operation of the interpolation processing will be described. Interpolation processing unit 4
7 selects an empty lattice point to be interpolated when the control unit 40 instructs execution of the interpolation processing (FIG. 3S
1).

For example, in FIG. 4, arbitrary coordinates (k,
When “k + p = odd number” holds for p), a vacancy is associated with the coordinates. Therefore, in the present embodiment, the coordinates associated with each pixel are (1,
1) to (X_max, Y_max) are sequentially searched, and coordinates at which the sum of the X component and the Y component is an odd number are defined as vacancy points. When the interpolation processing unit 47 selects the vacancy to be interpolated as described above, the inclination calculation unit 48 and the interpolation amount calculation unit 4
9 is started.

The inclination calculator 48 calculates the coordinates (k,
For p), the horizontal slope HD indicating the difference between the pixel outputs of two horizontally adjacent grid points and the vertical slope VD indicating the difference between the pixel outputs of two vertically adjacent grid points are defined as HD = | G (K-1, p) -G (k + 1, p) | VD = | G (k, p-1) -G (k, p + 1) | (FIG. 3S2).

The interpolation amount calculator 49 compares the horizontal slope HD with the vertical slope VD, and determines whether or not the difference between the horizontal slope HD and the vertical slope VD is equal to or less than a predetermined value (S3 in FIG. 3). When the interpolation amount calculating unit 49 recognizes that the difference between the horizontal inclination HD and the vertical inclination VD is equal to or smaller than a predetermined value,
The average value of the pixel outputs of the four grid points adjacent to the empty grid point is set as the interpolation amount (S4 in FIG. 3).

For example, the interpolation amount calculator 49 calculates an arbitrary value Th
When | HD−VD | ≦ Th holds, the interpolation amount G ′ (k, p) of the vacancy is calculated as G ′ (k, p) = ｛G (k−1, p) + G (k +
(1, p) + G (k, p-1) + G (k, p + 1)} / 4.

When the interpolation amount calculating unit 49 recognizes that the difference between the horizontal inclination HD and the vertical inclination VD exceeds a predetermined value,
The average value of the pixel outputs of two grid points adjacent to the vacant grid point in a direction with a small inclination is set as an interpolation amount (S5 in FIG. 3).
For example, the interpolation amount calculation unit 49 calculates
When “HD> VD + Th” is satisfied, it is considered that the vertical inclination is small, and the interpolation amount G ′ (k, p) of the vacancy is calculated as G ′ (k, p) = ｛G (k, p−1) + G (k , P
+1)｝ / 2, and when “VD> HD + Th” holds,
Assuming that the lateral inclination is small, the interpolation amount G '(k,
p) is given by G ′ (k, p) = ｛G (k−1, p) + G (k +
1, p)｝ / 2.

When the interpolation amount of the vacancy is determined in this way, the interpolation processing unit 47 determines whether or not the interpolation processing of all the vacancy is completed (S6 in FIG. 3). For example, such a determination is made in the process of searching for the coordinates associated with each pixel as described above, and for any coordinates (k, p), “k> X_max-3” and “p> Y_max-3”. ”Is determined.

The interpolation processing unit 47 repeats the above-described processing (the processing after S1 in FIG. 3) until it recognizes that the interpolation processing for all vacancies has been completed. That is, in the present embodiment, since the spatial directionality of the vacancy is estimated based on the horizontal inclination and the vertical inclination, the similarity of the green pixel output of the vacancy can be accurately determined.

Therefore, according to the present embodiment, it is possible to reliably improve the reliability of the amount of interpolation of a vacancy. (Second Embodiment) FIG. 5 is a functional block diagram of the second embodiment. This embodiment corresponds to an embodiment corresponding to the invention described in claim 5.

In the figure, components having the same functions as those of the functional block diagram of the first embodiment shown in FIG. 1 are denoted by the same reference numerals, and the description of the configuration is omitted. The difference between the configuration of the second embodiment and the configuration of the first embodiment is that the interpolation processing unit 52 shown in FIG.
Is provided, and the interpolation processing unit 52 is composed of an inclination calculation unit 48, an average calculation unit 53, and an interpolation amount calculation unit 54.

Further, regarding the correspondence between the invention of claim 5 and the present embodiment, the inclination calculating means corresponds to the inclination calculating section 48, the average calculating means corresponds to the average calculating section 53, and the first judging means. Corresponds to the function of comparing the horizontal inclination and the vertical inclination of the interpolation amount calculation unit 54, the second determination means corresponds to the function of comparing the horizontal average and the vertical average of the interpolation amount calculation unit 54,
The first interpolation amount determining unit, the second interpolation amount determining unit, and the third interpolation amount determining unit correspond to a function of the interpolation amount calculating unit 54 for calculating the interpolation amount.

FIG. 6 is an operation flowchart of the interpolation processing of the second embodiment. Hereinafter, the operation of the second embodiment will be described with reference to FIGS. The feature of the present embodiment lies in the operation of the interpolation process, and the other operations are described in the following.
The description is omitted because it is the same as the first embodiment.

When recognizing that the release button has been pressed in the same manner as in the first embodiment, the control unit 40 instructs the interpolation processing unit 52 in the signal processing unit 46 to execute interpolation processing. When instructed in this way, the interpolation processing unit 52 selects a vacancy to be subjected to the interpolation processing (S1 in FIG. 6), as in the first embodiment, and calculates the inclination calculation unit 48, the average calculation unit 53, The interpolation amount calculation unit 54 is started.

In the same manner as in the first embodiment, the inclination calculating section 48 calculates the horizontal inclination HD and the vertical inclination VD for the coordinates (k, p) of the vacancy points by the following expression: HD = │G (k−1, p) -G (k + 1, p) | VD = | G (k, p-1) -G (k, p + 1) | (S2 in FIG. 6).

The average calculation unit 53 calculates the coordinates (k,
Regarding p), a horizontal average HM indicating an average of pixel outputs of two grid points adjacent in the horizontal direction and a horizontal average HM indicating two
HM = (G (k−1, p) + G (k + 1, p)) /
2 VM = (G (k, p-1) + G (k, p + 1)) /
2 (S3 in FIG. 6).

The interpolation amount calculation unit 54 compares the horizontal inclination HD with the vertical inclination VD, and determines whether or not the difference between the horizontal inclination HD and the vertical inclination VD is equal to or smaller than a predetermined value (S4 in FIG. 6). When the interpolation amount calculation unit 54 recognizes that the difference between the horizontal inclination HD and the vertical inclination VD exceeds a predetermined value (NO side in FIG. 6S4) by such a determination, the interpolation amount calculation unit 54 becomes empty as in the first embodiment. The average value of the pixel outputs of the two grid points adjacent to the child point in the direction with a small inclination is set as the interpolation amount (S5 in FIG. 6).

On the other hand, when recognizing that the difference between the horizontal inclination HD and the vertical inclination VD is equal to or smaller than a predetermined value (YES side in S4 in FIG. 6), the interpolation amount calculation unit 54 compares the horizontal average HM with the vertical average VM. Then, it is determined whether or not the difference between the horizontal average HM and the vertical average VM is equal to or less than a predetermined value (S6 in FIG. 6). Interpolation amount calculation unit 54
Is such a determination (corresponding to S4 and S6 in FIG. 6).
When it is recognized that the difference between the horizontal inclination HD and the vertical inclination VD is equal to or smaller than a predetermined value and the difference between the horizontal average HM and the vertical average VM is equal to or smaller than a predetermined value (YES in S6 in FIG. 6),
The average value of the pixel outputs of the four grid points adjacent to the empty grid point is set as the interpolation amount (S7 in FIG. 6).

For example, the interpolation amount calculating section 54 calculates an arbitrary value T
For h and γ, “| HD-VD | ≦ Th and | HM-V
M | ≦ γ ”holds, the vacancy interpolation amount G ′
(K, p) is represented by G ′ (k, p) = ｛G (k−1, p) + G (k +
(1, p) + G (k, p-1) + G (k, p + 1)} / 4.

Here, the range G of the pixel output at the grid point is set to “0”.
≦ G ≦ 255 ”, Th and γ are about 20 to 50. Further, the interpolation amount calculation unit 54 determines the lateral inclination H by the above-described determination (corresponding to S4 and S6 in FIG. 6).
When it is recognized that the difference between D and the vertical inclination VD is equal to or less than a predetermined value, and that the difference between the horizontal average HM and the vertical average VM exceeds a predetermined value (NO side in FIG. 6S6), it is adjacent to a vacancy. The correlation between the pixel output of a grid point and the pixel output of each grid point arranged at a position where these grid points are shifted in four directions of up, down, left, and right, and a correlation indicating the strength of the correlation for each direction The amount is calculated (S8 in FIG. 6).

For example, the interpolation amount calculator 54 calculates the correlation amounts C-up, C-down, C-left corresponding to the four directions of up, down, left, and right.
And C-right are expressed as follows: C-up = | G (k, p-1) -G (k, p-1-
n) | + | G (k, p + 1) -G (k, p + 1-n) |
+ | G (k-1, p) -G (k-1, pn) | + | G
(K + 1, p) -G (k + 1, pn) | C-down = | G (k, p-1) -G (k, p-1 +
n) | + | G (k, p + 1) -G (k, p + 1 + n) |
+ | G (k-1, p) -G (k-1, p + n) | + | G
(K + 1, p) -G (k + 1, p + n) | C-left = | G (k, p-1) -G (kn, p-
1) | + | G (k, p + 1) -G (kn, p + 1) |
+ | G (k−1, p) −G (k−1−n, p) | + | G
(K + 1, p) -G (k + 1-n, p) | C-right = | G (k, p-1) -G (k + n, p-
1) | + | G (k, p + 1) -G (k + n, p + 1) |
+ | G (k-1, p) -G (k-1 + n, p) | + | G
(K + 1, p) -G (k + 1 + n, p) |.

Here, assuming that “n = 2”, the upward correlation amount C-up is represented by pixel outputs of four grid points indicated by “「 ”and“ □ ”as shown in FIG. It corresponds to the sum of the absolute values of the differences from the pixel outputs of the four grid points. The interpolation amount calculation unit 54 calculates the correlation amounts C-up, C
By selecting the smallest value among -down, C-left and C-right while allowing overlap, the direction having the strongest correlation is identified (S9 in FIG. 6).

Further, the interpolation amount calculating section 54 outputs the pixel outputs (G (k, p−1), G (k,
p + 1), G (k-1, p), (k + 1, p))
The pixel output of the lattice point in the direction with the strongest correlation is set as the interpolation amount of the empty lattice point (S10 in FIG. 6). For example, if the upward correlation is strong, G ′ (k, p) is G (k, p−1), and if the downward correlation is strong, G ′ (k, p) is G (k, p +
1), if the correlation in the left direction is strong, G ′ (k, p) is
G (k−1, p), and when the correlation in the right direction is strong, G ′
Let (k, p) be G (k + 1, p).

When there are a plurality of directions having a strong correlation, the average value of the pixel outputs of the lattice points in these directions may be used as the interpolation amount of the empty lattice point. When the interpolation amount of the vacancy is determined in this manner, the interpolation processing unit 52 determines whether or not the interpolation processing of all the vacancies is completed, as in the first embodiment (see FIG. 6S11), the above-described processing (the processing from S1 onward in FIG. 6) is repeatedly performed until it is recognized that the interpolation processing of all the vacancies has been completed.

That is, in this embodiment, the difference between the horizontal inclination HD and the vertical inclination VD is equal to or less than a predetermined value, and the horizontal average HM
When it is recognized that the difference between the vertical average VM and the vertical average VM exceeds a predetermined value, the correlation amounts in the four directions of the lattice points adjacent to the empty lattice point are calculated, and the pixel output of the lattice point in the direction with the strongest correlation is empty. It can be the interpolation amount of the child point. Therefore, according to the present embodiment, it is possible to reliably improve the reliability of the interpolation amount of the vacancy points as compared with the first embodiment.

In the present embodiment, the arbitrary value Th used when comparing the horizontal tilt HD and the vertical tilt VD is a constant value corresponding to the range of the pixel output at the lattice point.
For example, by applying the interpolation device according to claims 3 and 4, a monotonic non-decreasing function h having a value G determined based on a pixel output of a grid point adjacent to a vacancy as a variable, For the determined constants a and b, Th may be calculated by Th = a + b × h (G).

Here, the variable G of the function h may be the maximum value or the average value of the pixel outputs of the grid points adjacent to the empty grid point. The average value of the pixel outputs of the points or the average value of the pixel outputs of two grid points adjacent in the vertical direction may be used. (Third Embodiment) FIG. 8 is an operation flowchart of an interpolation process according to a third embodiment.

This embodiment corresponds to an embodiment corresponding to the sixth aspect of the present invention. The feature of the present embodiment lies in the method of the interpolation processing, and the hardware configuration is the same as the functional block diagram shown in FIG. Further, the operation other than the interpolation processing is the same as that of the second embodiment, and thus the description is omitted.

Here, regarding the correspondence between the invention described in claim 6 and the present embodiment, the inclination calculating means corresponds to the inclination calculating section 48, the average calculating means corresponds to the average calculating section 53, and the determining means Corresponds to a function of comparing the horizontal average and the vertical average of the interpolation amount calculation unit 54, and the first interpolation amount calculation unit and the second interpolation amount calculation unit calculate the interpolation amount of the interpolation amount calculation unit 54. Corresponds to the function.

The operation of the third embodiment will be described below with reference to FIGS. When the control unit 40 recognizes that the release button has been pressed in the same manner as the second embodiment, the control unit 40 instructs the interpolation processing unit 52 in the signal processing unit 46 to
Instructs execution of interpolation processing. When instructed in this way, the interpolation processing unit 52 selects a vacancy point to be subjected to the interpolation processing (S1 in FIG. 8), as in the second embodiment, and calculates the inclination calculation unit 48, the average calculation unit 53, The interpolation amount calculation unit 54 is started.

As in the second embodiment, the inclination calculating section 48 calculates the horizontal inclination HD and the vertical inclination VD for the coordinates (k, p) of the vacant lattice point as follows: HD = │G (k−1, p) -G (k + 1, p) | VD = | G (k, p-1) -G (k, p + 1) | (FIG. 8S2).

As in the second embodiment, the average calculation unit 53 calculates the horizontal average HM and the vertical average VM for the coordinates (k, p) of the vacancy points as HM = (G (k−1, p) + G (K + 1, p)) /
2 VM = (G (k, p-1) + G (k, p + 1)) /
2 (S3 in FIG. 8).

The interpolation amount calculator 54 compares the horizontal average HM with the vertical average VM, and determines whether or not the difference between the horizontal average HM and the vertical average VM is equal to or smaller than a predetermined value (S4 in FIG. 8). When the interpolation amount calculation unit 54 recognizes that the difference between the horizontal average HM and the vertical average VM is equal to or smaller than a predetermined value (YES side in S4 of FIG. 8), the interpolation amount calculation unit 54 determines the four adjacent four vacancy points. The average value of the pixel outputs at the grid points is set as the interpolation amount (S5 in FIG. 8).

When the interpolation amount calculation unit 54 recognizes that the difference between the horizontal average HM and the vertical average VM exceeds a predetermined value (NO in FIG. 8S4), the horizontal inclination HD and the horizontal inclination HD A weight corresponding to the vertical inclination VD is weighted to calculate an interpolation amount of a vacancy (S6 in FIG. 8). For example, the interpolation amount calculation unit 54 sets “| HM−VM
|> Γ ”, the interpolation amount G ′ (k, p) is given by G ′ (k, p) = [{G (k−1, p) + G (k−1) for the positive monotone non-decreasing function f. , P)}
/ (2 × f (HD)) + {G (k, p−1) + G (k, p + 1)}
/ (2 × f (VD)] / (1 / f (HD) + 1 / f (VD))

In this embodiment, f (X) = X + c is used as an example of the function f for a predetermined constant c. Note that f (X) ≠ 0. Here, when the range G of the pixel output at the lattice point is set to “0 ≦ G ≦ 255”, γ is set to about 20 to 50.

When the interpolation amount of the vacancy is determined in this way, the interpolation processing unit 52 performs the same operation as in the second embodiment.
It is determined whether or not the interpolation processing for all vacancies has been completed (S7 in FIG. 8), and the above-described processing (the processing from S1 onward in FIG. 8) is performed until it is recognized that the interpolation processing for all vacancies has been completed. Repeat.

That is, in the present embodiment, when the difference between the horizontal average HM and the vertical average VM exceeds a predetermined value (when the horizontal average HM and the vertical average VM are not considered to match), the horizontal average The interpolation amount of the vacant point is calculated based on the weight coefficient corresponding to the HM and the vertical average VM. Therefore,
According to the present embodiment, even when the values of the horizontal average HM and the vertical average VM are very similar, the interpolation amount of the vacancy can be calculated with high accuracy.

In the present embodiment, the weighting coefficient is calculated by the equation indicated by the function f every time the interpolation amount of the vacancy is calculated. For example, the weighting coefficient is provided in the interpolation amount calculation units 49 and 54. A weight coefficient may be stored in the memory in advance in association with the horizontal inclination and the vertical inclination, and may be read each time the interpolation amount is calculated. Further, in the present embodiment and the above-described second embodiment, when it is recognized that the difference between the horizontal average HM and the vertical average VM is equal to or smaller than a predetermined value, 4
Although the average value of the pixel outputs of two grid points is used as the interpolation amount,
In such a case, the value of the horizontal average or the vertical average may be used as the interpolation amount.

Furthermore, in each of the above-described embodiments, the interpolation processing for green was performed. However, the present invention is not limited to such interpolation processing. For example, “pixels for outputting red” and “pixels for outputting blue” Is applied to an image signal generated by an image sensor arranged in a checkered pattern, so that red and blue interpolation processing can be performed.

By the way, the present invention can be applied to any image pickup device having a configuration in which lattice points and vacant lattice points are arranged in a checkered pattern, irrespective of a single plate, multiple plates, and a color filter. As shown in FIG. 12A, the present invention can also be applied to an imaging device that does not look like a checkered pattern. For example, in FIG. 12A, assuming that the center portion of the four green (G) pixel outputs is a vacancy point, FIG.
By rotating the imaging element indicated by 45 degrees by 45 degrees, it can be said that grid points are arranged in four directions, up, down, left, and right of the vacant grid point. Therefore, the interpolation amount of the vacancy can be calculated in the same manner as in the above-described embodiments.

(Fourth Embodiment) In each of the above-described embodiments, the interpolation processing section 47 is provided in the electronic still camera.
Although the interpolation process is performed by providing 52, such an interpolation process can also be performed by an image processing device or a computer. FIG. 9 is a functional block diagram of the fourth embodiment.

This embodiment corresponds to performing an interpolation process by a personal computer using the "recording medium on which the interpolation process program is recorded" according to the seventh to ninth aspects. In FIG. 9, components having the same functions as those in the functional block diagrams shown in FIGS. 1 and 5 are denoted by the same reference numerals, and description of the configuration is omitted.

The difference between the electronic still camera 60 shown in FIG. 9 and the embodiment shown in FIGS. 1 and 5 is that a signal processing section 61 is provided instead of the signal processing section 46 and a memory card driver 50 is provided. Is provided, and the output of the image buffer memory 45 is connected to the signal processing unit 61, and the output of the signal processing unit 61 is connected to the memory card driver 50.

In the figure, a personal computer 62
Has a CPU 63, a memory card driver 64, and an image memory 65. The CPU 63 is connected to the memory card driver 64 and the image memory 65, and the memory card driver 64 and the image memory 65 are mutually connected. Note that an interpolation processing program recorded on the CD-ROM 66 is installed in the CPU 63 in advance.

Here, the “recording medium on which the interpolation processing program is recorded” according to any one of claims 7 to 9 is a CD-ROM.
Corresponding to the ROM 66, a tilt calculation procedure, an interpolation amount determination procedure,
Average calculation procedure, first determination procedure, second determination procedure, first interpolation amount determination procedure, second interpolation amount determination procedure, third interpolation amount determination procedure, interpolation amount calculation procedure, determination procedure, first The interpolation amount calculation procedure and the second interpolation amount calculation procedure correspond to the interpolation processing performed by the CPU 63.

FIG. 10 is an operation flowchart of the fourth embodiment. Hereinafter, the operation of the fourth embodiment will be described with reference to FIGS. In the electronic still camera 60, in the image sensor 43, as in the first embodiment,
A signal charge is generated. Further, the image pickup device 43 scans the signal charges thus generated to generate an image signal,
The signal is supplied to the A / D converter 44.

The A / D converter 44 performs A / D conversion on the image signal given in this way to generate image data (S1 in FIG. 10). The image data generated in this way is temporarily stored in the image buffer memory 45 (FIG. 10S
2). Such processing is repeatedly performed based on the timing control by the control unit 40.

The control unit 40 determines whether or not a release button (not shown) has been pressed (S3 in FIG. 10). When recognizing that the release button has been pressed, the control unit 40 sends a signal to the signal processing unit 61 and the memory card driver 50. Then, an instruction is given to record the image data stored in the image buffer memory 45 on the memory card 51 (S4 in FIG. 10). The signal processing unit 61 adjusts the image data stored in the image buffer memory 45 for a problem (for example, a dark level) unique to the image sensor, and performs processing such as γ correction (here, No interpolation processing is performed.), And is given to the memory card driver 50. Note that the γ correction may be performed in the personal computer 62 without being performed in the signal processing unit 61.

The memory card driver 50 records the image data on the memory card 51 in the form of an image file, and also stores an “interpolation request flag” (indicating that the image data requires interpolation processing) provided in the header of the image file. Is set. When the memory card 51 on which the image data is recorded is loaded into the personal computer 62, the CPU 63
4 to read out the header of the image file (FIG. 10)
S5).

When reading the header of the image file in this manner, the CPU 63 determines whether or not the "interpolation request flag" is set (S6 in FIG. 10). CPU63
Only when it is recognized that the “interpolation request flag” is set by such a determination, the image data is read out via the memory card driver 64 and the image memory 65 is read out.
(S7 in FIG. 10).

The CPU 63 performs an interpolation process on the image data stored in the image memory 65 (FIG. 10S).
8). Note that the interpolation processing according to the present embodiment is the same as the interpolation processing according to each of the above-described embodiments, and a description thereof will not be repeated. C
When the interpolation processing is completed, the PU 63 records the interpolated image data again on the memory card 51 via the memory card driver 64 (S9 in FIG. 10). The image data subjected to the interpolation processing is transferred to the memory card 51.
May be selected by an operator using an input device (for example, a keyboard or the like) of the personal computer 42.

That is, in the present embodiment, the same interpolation processing as in the above-described embodiments is performed by the personal computer 62.
Can be performed reliably. Therefore, according to the present embodiment, since the spatial directionality of a vacant lattice point can be reliably estimated based on the pixel output of an adjacent lattice point, the accuracy of the interpolation processing can be reliably improved. .

In this embodiment, the personal computer 62 obtains the image data to be subjected to the interpolation processing via the memory card 51. However, any method can be used to obtain such image data. You may. For example, when the present invention is applied to a personal computer having communication means, it is possible to perform interpolation processing on image data provided via the communication means.

[0085]

As described above, claims 1 and 7 are as described above.
According to the invention described in (1), since the spatial directionality of a vacancy point can be estimated based on the pixel outputs of adjacent lattice points, the reliability of the interpolation amount of the vacancy point can be reliably improved. In addition, since the spatial directionality of a vacancy can be easily estimated by performing a simple calculation, the interpolation amount of the vacancy can be quickly and reliably obtained.

According to the second aspect of the present invention, the spatial orientation of the vacancy is easily performed by performing a simple operation while standardizing the estimation of the spatial orientation of the vacancy. Can be estimated. According to the third aspect of the present invention, it is possible to reliably estimate the spatial directionality of a vacancy while applying the characteristics to the characteristics of an image to be interpolated.

According to the fourth aspect of the present invention, it is possible to reliably estimate the spatial directionality of a vacancy while applying the characteristics of the image to be interpolated. According to the fifth and eighth aspects of the present invention, even when the spatial direction of a vacancy cannot be estimated only by the horizontal inclination or the vertical inclination, it is adjacent to the horizontal average, the vertical average, and the vacancy. Based on the grid points arranged around the grid points, it is possible to reliably estimate the spatial directionality of the empty grid points. Therefore, the reliability of the interpolation amount of the vacancy can be reliably improved.

According to the sixth and ninth aspects of the present invention, the interpolation amount of the vacancy can be calculated with high accuracy without impairing the smoothness of the image.

[Brief description of the drawings]

FIG. 1 is a functional block diagram of a first embodiment.

FIG. 2 is an operation flowchart of the first embodiment.

FIG. 3 is an operation flowchart of an interpolation process according to the first embodiment.

FIG. 4 is a diagram showing a pixel output of a grid point and an interpolation amount of an empty grid point.

FIG. 5 is a functional block diagram of a second embodiment.

FIG. 6 is an operation flowchart of an interpolation process according to the second embodiment.

FIG. 7 is a diagram illustrating a method of calculating a correlation amount.

FIG. 8 is an operation flowchart of an interpolation process according to the third embodiment.

FIG. 9 is a functional block diagram of a fourth embodiment.

FIG. 10 is an operation flowchart of the fourth embodiment.

FIG. 11 is a diagram illustrating a configuration of an image sensor.

FIG. 12 is a diagram illustrating a configuration of an image sensor.

FIG. 13 is a diagram illustrating a conventional interpolation process.

[Explanation of symbols]

 Reference Signs List 40 control unit 41 photographing optical system 42 OLPF 43 image sensor 44 A / D conversion unit 45 image buffer memory 46, 61 signal processing unit 47, 52 interpolation processing unit 48 inclination calculation unit 49, 54 interpolation amount calculation unit 50, 64 memory card Driver 51 Memory card 53 Average calculation unit 60 Electronic still camera 62 Personal computer 63 CPU 65 Image memory 66 CD-ROM

Claims (9)

[Claims] An interpolation amount for interpolating an image in which lattice points outputting a predetermined color and empty lattice points not outputting the color are arranged in a checkered pattern, and outputting pixel data of the empty lattice point In the interpolating apparatus, the positions of the lattice points and the empty lattice points are represented by a coordinate system composed of “a horizontal axis X having a positive right direction” and a vertical axis Y having a positive downward direction. The pixel output of the grid point is represented by G (X, Y)
When the coordinates of the vacancy are (k, p),
The horizontal slope HD indicating the difference between the pixel outputs of two grid points horizontally adjacent to the empty grid point, and the difference between the pixel outputs of two grid points vertically adjacent to the empty grid point are shown. Inclination calculating means for calculating the vertical inclination VD indicated by HD = | G (k-1, p) -G (k + 1, p) | VD = | G (k, p-1) -G (k, p + 1) | When the interpolation amount of the vacant point is G ′ (k, p), the horizontal inclination HD and the vertical inclination V calculated by the inclination calculating means are calculated.
D and for any value Th, “HD> VD + T
h ”holds, then G ′ (k, p) = ｛G (k, p−1) + G (k, p
+1)｝ / 2, and when “VD> HD + Th” holds, G ′ (k, p) = {G (k−1, p) + G (k +
1, p)｝ / 2, and when “| HD−VD | ≦ Th” is satisfied, G ′ (k, p) = {G (k−1, p) + G (k +
(P) + G (k, p−1) + G (k, p + 1)｝ / 4. 2. The interpolation device according to claim 1, wherein the slope calculation means sets the value of Th to a predetermined constant value. 3. The interpolation device according to claim 1, wherein the inclination calculating unit determines the value of Th in association with a pixel output of a grid point adjacent to the empty grid point. apparatus. 4. The interpolation device according to claim 1, wherein the slope calculating means calculates a maximum value of a pixel output of a grid point adjacent to the vacant grid point, and a maximum value of a pixel output of a grid point adjacent to the vacant grid point. Determining the value of Th based on an average value, an average value of pixel outputs of grid points horizontally adjacent to the empty grid point or an average value of pixel outputs of grid points vertically adjacent to the empty grid point; An interpolation device characterized by the above-mentioned. 5. An interpolation amount for interpolating an image in which lattice points for outputting a predetermined color and vacant lattice points not outputting the color are arranged in a checkered pattern, and the pixel output of the vacant lattice point is performed. In the interpolating apparatus, a horizontal slope indicating a difference between pixel outputs of two grid points horizontally adjacent to the empty grid point, and a horizontal slope of two grid points vertically adjacent to the empty grid point, Inclination calculating means for calculating a vertical inclination indicating a difference between pixel outputs; a horizontal average indicating an average of pixel outputs of two grid points horizontally adjacent to the vacant grid point; Means for calculating a vertical average indicating the average of the pixel outputs of two grid points vertically adjacent to each other, and determining the horizontal inclination and the vertical inclination calculated by the inclination calculating means based on predetermined conditions. These slopes match by comparing By comparing the horizontal average and the vertical average calculated by the average calculation unit based on predetermined conditions, the first determination unit that determines whether or not the average can be regarded as being equal to each other, The second determining means for determining whether or not it can be considered that, if it is determined that the horizontal inclination and the vertical inclination do not match by the first determination means, the inclination of the vacancy with respect to the vacant point First interpolation amount determining means for determining an interpolation amount of the empty grid point based on pixel outputs of two grid points adjacent to each other in a smaller direction; and a lateral inclination by the first determining means and the second determining means. When it is determined that the horizontal average and the vertical average match and the horizontal average and the vertical average match, a value corresponding to the average of the pixel outputs of four grid points adjacent to the empty grid point is calculated as the empty grid. A second interpolation amount determining means for interpolating the point, When it is determined that the horizontal inclination and the vertical inclination match by the first determination unit and the second determination unit, and that the horizontal average and the vertical average do not match, a grid adjacent to the empty grid point A third interpolation amount determining means for determining the interpolation amount of the empty grid point based on the similarity between the pixel output of the point and the pixel outputs of the grid points arranged around these grid points. An interpolation device characterized by the above-mentioned. 6. An interpolation amount of an image in which lattice points for outputting a predetermined color and empty lattice points for not outputting the color are arranged in a checkered pattern, and the interpolation amount becomes a pixel output of the empty lattice point. In the interpolating apparatus, a horizontal slope indicating a difference between pixel outputs of two grid points horizontally adjacent to the empty grid point, and a horizontal slope of two grid points vertically adjacent to the empty grid point, Inclination calculating means for calculating a vertical inclination indicating a difference between pixel outputs; a horizontal average indicating an average of pixel outputs of two grid points horizontally adjacent to the vacant grid point; Average calculating means for calculating a vertical average indicating an average of pixel outputs of two grid points vertically adjacent to each other, and calculating a horizontal average and a vertical average calculated by the average calculating means based on predetermined conditions. Comparisons show that these averages are consistent. Determining means for determining whether or not the horizontal average and vertical average do not coincide with each other. If the horizontal average does not match the vertical average, the determination means determines whether the horizontal average and the vertical average correspond to the horizontal inclination and the vertical inclination calculated by the inclination calculation means. First interpolation amount calculating means for performing weighting to calculate the interpolation amount of the vacancy, and when the determination means determines that the horizontal average and the vertical average are the same, adjacent to the vacancy A second interpolation amount calculating means for setting a value corresponding to an average of pixel outputs of four lattice points to be an interpolation amount of the empty lattice point. 7. A computer that interpolates an image in which grid points that output a predetermined color and empty grid points that do not output the color are arranged in a checkerboard pattern, and outputs the pixel output of the empty grid point. A recording medium on which an interpolation processing program for obtaining an interpolation amount is recorded, wherein the positions of the lattice points and the vacant lattice points are “a horizontal axis X having a positive right direction” and a vertical axis Y having a downward positive direction. And the pixel output of the grid point is represented by G (X, Y).
When the coordinates of the vacancy are (k, p),
The horizontal slope HD indicating the difference between the pixel outputs of two grid points horizontally adjacent to the empty grid point, and the difference between the pixel outputs of two grid points vertically adjacent to the empty grid point are shown. And the vertical inclination VD shown by HD = | G (k-1, p) -G (k + 1, p) | VD = | G (k, p-1) -G (k, p + 1) | When the interpolation amount of the vacant point is G ′ (k, p), the horizontal inclination HD and the vertical inclination V calculated by the inclination calculation procedure are calculated.
D and for any value Th, “HD> VD + T
h ”holds, then G ′ (k, p) = ｛G (k, p−1) + G (k, p
+1)｝ / 2, and when “VD> HD + Th” holds, G ′ (k, p) = {G (k−1, p) + G (k +
1, p)｝ / 2, and when “| HD−VD | ≦ Th” is satisfied, G ′ (k, p) = {G (k−1, p) + G (k +
(P) + G (k, p−1) + G (k, p + 1)｝ / 4. A recording medium for recording an interpolation processing program for causing a computer to execute the procedure. 8. A computer that performs interpolation of an image in which lattice points that output a predetermined color and empty lattice points that do not output the color are arranged in a checkerboard pattern. A recording medium on which an interpolation processing program for obtaining an interpolation amount is recorded, wherein a horizontal slope indicating a difference between pixel outputs of two grid points horizontally adjacent to the vacant grid point, Calculating a vertical slope indicating a difference between pixel outputs of two grid points vertically adjacent to each other, and calculating an average of pixel outputs of two grid points horizontally adjacent to the empty grid point. A horizontal average, and an average calculation procedure of calculating a vertical average indicating an average of pixel outputs of two grid points vertically adjacent to the vacant grid point; and a horizontal slope calculated by the slope calculation procedure. The vertical inclination is determined based on predetermined conditions. By comparing, the first determination procedure to determine whether or not these slopes can be considered to match, and the horizontal average and the vertical average calculated by the average calculation procedure based on a predetermined condition By comparing them, a second determination procedure for determining whether or not these averages can be considered to be consistent, and it is determined that the horizontal slope and the vertical slope do not match by the first determination procedure In this case, a first interpolation amount determining step of determining an interpolation amount of the empty lattice point based on pixel outputs of two lattice points adjacent to the empty lattice point in a direction with a small inclination, When it is determined that the horizontal inclination and the vertical inclination coincide with each other by the determination procedure and the second determination procedure, and that the horizontal average and the vertical average coincide with each other, the four lattice points adjacent to the empty lattice point are determined. The value corresponding to the average of the pixel output is The second interpolation amount determination procedure to be the interpolation amount of the grid points, the horizontal inclination and the vertical inclination match by the first determination procedure and the second determination procedure, and the horizontal average and the vertical average match. If it is determined that the vacancy is not present, the interpolation of the vacancy is performed based on the similarity between the pixel outputs of the lattice points adjacent to the vacancy and the pixel outputs of the lattice points arranged around these lattice points. A recording medium for recording an interpolation processing program for causing a computer to execute a third interpolation amount determination procedure for determining an amount. 9. A computer that executes interpolation of an image in which grid points for outputting a predetermined color and empty grid points that do not output the color are arranged in a checkered pattern, and outputs a pixel output of the empty grid point. A recording medium on which an interpolation processing program for obtaining an interpolation amount is recorded, wherein a horizontal slope indicating a difference between pixel outputs of two grid points horizontally adjacent to the vacant grid point, Calculating a vertical slope indicating a difference between pixel outputs of two grid points vertically adjacent to each other, and calculating an average of pixel outputs of two grid points horizontally adjacent to the empty grid point. An average calculation procedure of calculating a horizontal average and a vertical average indicating an average of pixel outputs of two grid points vertically adjacent to the empty grid point; and a horizontal average calculated by the average calculation procedure. Vertical average based on predetermined conditions By comparing, it is determined whether or not these averages can be considered to match, and, if it is determined by the determination procedure that the horizontal average and the vertical average do not match, the inclination calculation procedure A first interpolation amount calculating step of performing weighting corresponding to the horizontal inclination and the vertical inclination calculated by the above to calculate the interpolation amount of the vacant point, and the horizontal average and the vertical average match by the determination procedure. Is determined, the second interpolation amount calculation procedure in which a value corresponding to the average of the pixel outputs of the four grid points adjacent to the vacancy point is used as the interpolation amount of the vacancy point is executed by the computer. Recording medium on which the interpolation processing program is recorded.